I have been trying to run a flume agent on my windows system to get twitter data. I am following this blog
https://acadgild.com/blog/streaming-twitter-data-using-flume
But, whenever i try to run the flume agent I get the follwing error -
flume-ng agent -n TwitterAgent -f ./conf/flume.conf
E:\apache-flume-1.7.0-bin\apache-flume-1.7.0-bin\bin>powershell.exe -NoProfile -InputFormat none -ExecutionPolicy unrestricted -File E:\apache-flume-1.7.0-bin\apache-flume-1.7.0-bin\bin\flume-ng.ps1 agent -n TwitterAgent -f ./conf/flume.conf
Resolve-Path : Cannot find path 'E:\apache-flume-1.7.0-bin\apache-flume-1.7.0-bin\bin\conf\flume.conf' because it does not exist.
At E:\apache-flume-1.7.0-bin\apache-flume-1.7.0-bin\bin\flume-ng.ps1:217 char:24 + $confFile = '"' + (Resolve-Path $confFile).Path + '"' + ~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (E:\apache-flume...conf\flume.conf:String) [Resolve-Path], ItemNotFoundException
+ FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.ResolvePathCommand
I have tried debugging it on my own, but have failed to do so. I tried to look for this particular error but could not find it anywhere.
Does E:\apache-flume-1.7.0-bin\apache-flume-1.7.0-bin\bin\conf\flume.conf exist at that location? Are you sure it's in \bin\conf\flume.conf and not \conf\flume.conf? In which case use:
flume-ng agent -n TwitterAgent -f ./../conf/flume.conf
Or (as is more common), run from the E:\apache-flume-1.7.0-bin\apache-flume-1.7.0-bin folder with the command:
bin/flume-ng agent -n TwitterAgent -f ./conf/flume.conf
Related
Its and windows machine and trying to install Sql Server(unattended) using powershell
Write-Host "`nStarting the install of SQL Server" -ForegroundColor Green
[string]$userName = 'domain\username'
[string]$userPassword = 'xyz'
[securestring]$secStringPassword = ConvertTo-SecureString $userPassword -AsPlainText -Force
[pscredential]$credObject = New-Object System.Management.Automation.PSCredential ($userName, $secStringPassword)
Start-Process -Filepath "$driveLetter\setup.exe" -ArgumentList "/ConfigurationFile=`"$temp_file_location`"" -Credential $credObject -NoNewWindow -Wait -RedirectStandardOutput $standardOutputFile -RedirectStandardError $errorOutputFile
ERROR:
PS C:\Windows\system32> C:\Users\manish\Desktop\custommain.ps1
domain
eu-west-1
fetching the password from parameter stores
Cleaning the outputfile error
finding and replacing the domains and secrets
writting the content of ini into temp file C:\xyz-softwares\ms-sql-server\scripts\batch-files\temp_ConfigurationFile.ini
Mounting SQL Server Image
Getting Disk drive of the mounted image
Starting the install of SQL Server
Start-Process : This command cannot be run due to the error: The user name or password is incorrect.
At C:\xyz-softwares\ms-sql-server\scripts\powershell\ms-sqlserver-installation.ps1:65 char:2
+ Start-Process -Filepath "$driveLetter\setup.exe" -ArgumentList "/ ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (:) [Start-Process], InvalidOperationException
+ FullyQualifiedErrorId : InvalidOperationException,Microsoft.PowerShell.Commands.StartProcessCommand
SQL Installation Failed Installation with exit code
SQL Installation Failed Installation with exit code
At C:\xyz-softwares\ms-sql-server\scripts\powershell\ms-sqlserver-installation.ps1:72 char:9
+ throw $error_message
+ ~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : OperationStopped: (SQL Installatio... with exit code:String) [], RuntimeException
+ FullyQualifiedErrorId : SQL Installation Failed Installation with exit code
PS C:\Windows\system32>
I have tried with powershell with domain credentials.
Although i have exe file and windows machine is connected with domain
Tried to start process command with domain credetials.
when i type command at windows terminal http-server following error occured
error
http-server
http-server : File C:\Users\RKanth\AppData\Roaming\npm\http-server.ps1 cannot be loaded because running scripts is disabled on this system. For more information, see about_Execution_Policies at https:/go.microsoft.com/fwlink/?LinkID=135170.
At line:1 char:1
+ http-server
+ ~~~~~~~~~~~
+ CategoryInfo : SecurityError: (:) [], PSSecurityException
+ FullyQualifiedErrorId : UnauthorizedAccess
Open PowerShell by Run as Administrator
Run the following command:
Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope LocalMachine
Confirm with ‘Y’
Open PowerShell by Running as Administrator using the windows search bar.
Run the following command: Get-ExecutionPolicy
Then you can See: Restricted
Again run the following command: Set-ExecutionPolicy Unrestrict
Press y
Now you can Run again the following command and check: Get-ExecutionPolicy
Then you can See: Unrestricted
Done!!!
After using gsutil for more than 1 year I suddenly have this error:
.....
At destination listing 8350000...
At destination listing 8360000...
CommandException: Caught non-retryable exception - aborting rsync
.....
I tried to locate the files with this sync problem but I am not able to do so. Is there a "skip error" option of is there a way I can have gsutil more verbose?
My command line is like this:
gsutil -V -m rsync -d -r -U -P -C -e -x -x 'Download/*' /opt/ gs://mybucket1/kraanloos/
I have created a script to split the problem. This gives me more info for a solution
!#/bin/bash
array=(
3ware
AirTime
Amsterdam
BigBag
Download
guide
home
Install
Holding
Multimedia
newsite
Overig
Trak-r
)
for i in "${array[#]}"
do
echo Processing : $i
PROCESS="/usr/bin/gsutil -m rsync -d -r -U -P -C -e -x 'Backup/*' /opt/$i/ gs://mybucket1/kraanloos/$i/"
echo $PROCESS
$PROCESS
echo ""
echo ""
done
I've been struggling with the same problem the last few days. One way to make it super verbose is to put the -D flag before the rsync argument, as in:
gsutil -D rsync ...
By doing that, I found that my problem is due to having # characters in filenames, as in this question.
In my case, it was because of a broken link to a directory.
As blambert said, use the -D option to see exactly what file causes the problem.
I had struggled with this problem as well. I figured it out now.
you need to re-authenticate your Google Cloud SDK Shell and set a target project again.
It seems like rsync will not show the correct error message.
try cp instead, it will guide you to authentic and set the correct primary project
gsutil cp OBJECT_LOCATION gs://DESTINATION_BUCKET_NAME/
after that, your gsutil rsync should run fine.
im trying to use psexec.exe in powershell, but appear connect but do not nothing. Supposedly i call psexec in his route in my computer, with the parameters i usually use in a normal CMD.
C:\c\PsExec.exe \\Hostname.domain.CORP -u domain\admx097168 -p Password cmd -c “c:\temp\hola.vbs”
This is the response on the console.
PS C:\Users\x097168> C:\Consola\testmasiveSCCM.ps1
C:\c\PsExec.exe :
En C:\Consola\testmasiveSCCM.ps1: 2 Carácter: 1
+ C:\c\PsExec.exe \\Hostname.domain.CORP -u domain\admx097 ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:String) [], RemoteException
+ FullyQualifiedErrorId : NativeCommandError
PsExec v1.98 - Execute processes remotely
Copyright (C) 2001-2010 Mark Russinovich
Sysinternals - www.sysinternals.com
Microsoft Windows [Versi¢n 10.0.10240]
(c) 2015 Microsoft Corporation. Todos los derechos reservados.
appear still working, but do nothing, will be necessary stopped the scripting to continue.
Anyone can help me? first of all, thank you very much for reading my query.
one last thing, i can't use invoke-command, cause is forbidden in my domain.
at finally i can do it with the next secuence.
C:\c\PsExec.exe \xxx.xx.xx.xxx -u prbes\admx000000 -p Pasw0rd -i -d cscript.exe c:\temp\test.vbs
I have a simple script where i need to connect to SFTP and put a file,below is the script.But I am getting a error message showing "No such file or directory mgtest#100.45.5.27)".Can somebody help me on this.
#!/bin/sh
# sample automatic ftp script to dump a file
USER="mgtest"
PASSWORD="cald123"
HOST="100.45.5.27"
sftp -b $USER#$HOST << EOF
$PASSWORD
#cd test_path
put test_file.txt
quit
EOF
Execution Screen and error
$ sh -x sftp_test.sh
+ sh -x sftp_test.sh
+ USER=mgtest
+ PASSWORD=testpass
+ HOST=100.45.5.27
+ sftp -b mgtest#100.45.5.27
+ 0<<
testpass
#cd test_path
#put test_file.txt
quit
No such file or directory (mgtest#100.45.5.27).
The parameter -b expects a batchfile, not username and password. See the docs for sftp