OpenRefine changing the port and host when executable is run directly - openrefine

The refine.ini allow setting the port and host without the need to re-building, but it says the following:
# NOTE: This file is not read if you run the Refine executable directly
# It is only read of you use the refine shell script or refine.bat
from my limited observation i noticed that when the executable is run directly the value for port and host are always the ones set as default in Refine.java.
is there a way to change the port and host when running the executable directly without the need to re-build ?

The way OpenRefine is started varies across platforms and so the mechanisms for changing the host, port, and other options, also vary.
Linux
On Linux you can set the host and port (and other options) with options when you run refine. -p controls the port and -i the host. For example:
./refine -p 3334 -i localhost
Will run OpenRefine on http://localhost:3334
Use ./refine -h to list all options that are supported
OS X
On OS X the options are set in the Info.plist file (in path_to_openrefine/OpenRefine.app/Contents)
You'll need to find the 'array' element that follows the line:
<key>JVMOptions</key>
Typically this looks something like:
<key>JVMOptions</key>
<array>
<string>-Xms256M</string>
<string>-Xmx1024M</string>
<string>-Drefine.version=2.6-beta.1</string>
<string>-Drefine.webapp=$APP_ROOT/Contents/Resource/webapp</string>
</array>
Add in values for -Drefine.host and -Drefine.port - so you have something like:
<key>JVMOptions</key>
<array>
<string>-Xms256M</string>
<string>-Xmx1024M</string>
<string>-Drefine.version=2.6-beta.1</string>
<string>-Drefine.webapp=$APP_ROOT/Contents/Resource/webapp</string>
<string>-Drefine.host=localhost</string>
<string>-Drefine.port=3334</string>
</array>
There is another way of editing the Info.plist file if you'd prefer to do this via
a GUI. See https://github.com/OpenRefine/OpenRefine/wiki/FAQ:-Allocate-More-Memory section on Macs which works through a similar method for extending the memory allocation.
Windows
On Windows you can set options on the command line if you run via refine.bat, and as you note also via refine.ini
If you are running refine.exe on Windows I'm not aware you can use command line options, but you can set run time options in openrefine.l4j.ini (or refine.l4j.ini for older versions of refine).
Add to this file lines:
-Drefine.port=3334
-Drefine.host=localhost
and save, and when you run openrefine.exe (or google-refine.exe for older versions of refine) it will use these values for the port/host

Related

How do I port forward in dropbear ssh?

I have been using openSSH for a little bit and just learned the basics of port forwarding in OpenSSH. I own some equipment that has dropbear installed on it but it seems the options are different. The equipment has an internal webpage operating on port 443 and I would like to forward that to another PC securely.
Port-forwarding requires the ssh client and ssh server to interoperate, and of course for the feature to be present and allowed in both. At build time Dropbear has 4 distinct settings for this
#define DROPBEAR_CLI_LOCALTCPFWD 1
#define DROPBEAR_CLI_REMOTETCPFWD 1
#define DROPBEAR_SVR_LOCALTCPFWD 1
#define DROPBEAR_SVR_REMOTETCPFWD 1
These are all set by default in the official (current dropbear-2022.82) source. AFAICT every public release since 2003 has had some form of TCP forwarding support (but not necessarily enabled when it was built).
Usefully, these options control both the feature itself, and whether the feature is documented in the -h help output—if the relevant options are omitted from the help output then they were omitted from the build.
With Dropbear server you should be able to run dropbear -h or (sshd -h if it has been renamed), the presence of the -j and/or -k options indicate DROPBEAR_SVR_LOCALTCPFWD and DROPBEAR_SVR_REMOTETCPFWD respectively were set at build time.
With Dropbear client you should run dbclient -h (or ssh -h), the presence of the -L and/or -R indicate DROPBEAR_CLI_LOCALTCPFWD and DROPBEAR_CLI_REMOTETCPFWD respectively were set at build time.
(If the binaries were renamed you can confirm their identity with the -V option.)
Finally, for Dropbear server it must be started without the -j or -k options to allow it to observe client requests for local and remote forwarding respectively.
If all of the above is as expected (specifically the capabilities and run-time options of the target system Dropbear ssh server) you should then be able to do something like one of:
ssh -L10443:127.0.0.1:443 dropbearhost
ssh -L10443:x.x.x.x:443 dropbearhost
where localhost:10443 (e.g. https://localhost:10443/ on the initiating system will forward to 127.0.0.1:443 on dropbearhost (or x.x.x.x:443 with an alternate IP if the web server is bound to a specific address). If SNI is enforced, then adding the virtualhost name to the hosts file on the initiating system should fix that (and might also be required if the web content uses redirects).
If you happen to be building Dropbear, you change those build options by hand after running configure:
grep TCPFWD default_options.h >> localoptions.h
and amend, setting them to 0 or 1 as required. (Note though in older versions these defines were named, set and used slightly differently.)

VScode remote connection error: The process tried to write to a nonexistent pipe

I use vscode with remote-ssh to connect my server, after configuring, I want to connect my host, but it failed, the dialog box display:"could not establish connection to XX, The process tried to write to a nonexistent pipe."
output:
[16:45:20.916] Log Level: 3
[16:45:20.936] remote-ssh#0.49.0
[16:45:20.936] win32 x64
[16:45:20.944] SSH Resolver called for "ssh-remote+aliyun", attempt 1
[16:45:20.945] SSH Resolver called for host: aliyun
[16:45:20.945] Setting up SSH remote "aliyun"
[16:45:21.012] Using commit id "c47d83b293181d9be64f27ff093689e8e7aed054" and quality "stable" for server
[16:45:21.014] Install and start server if needed
[16:45:21.019] Checking ssh with "ssh -V"
[16:45:21.144] > OpenSSH_for_Windows_7.7p1, LibreSSL 2.6.5
[16:45:21.214] Running script with connection command: ssh -T -D 5023 aliyun bash
[16:45:21.221] Terminal shell path: C:\WINDOWS\System32\cmd.exe
[16:45:21.504] >
>
>
> ]0;C:\WINDOWS\System32\cmd.exe
[16:45:21.505] Got some output, clearing connection timeout
[16:45:21.577] >
>
>
>
[16:45:21.592] > Bad owner or permissions on C:\\Users\\DY/.ssh/config
>
[16:45:21.689] > The process tried to write to a nonexistent pipe.
>
[16:45:22.091] "install" terminal command done
[16:45:22.092] Install terminal quit with output: The process tried to write to a nonexistent pipe.
[16:45:22.093] Received install output: The process tried to write to a nonexistent pipe.
[16:45:22.096] Resolver error: The process tried to write to a nonexistent pipe
[16:45:22.107] ------
Add the absolute file path to a custom SSH config file(C:\Users\{USERNAME}\.ssh\config), and my problem is solved.
If you format/re-install Server OS, but use same IP as before,
you may encounter fingerprint mismatched.
You may need to delete old fingerprint in this file:
C:\Users\xxx.ssh\known_host
and old IP in the file:
C:\Users\xxx.ssh\config
Then try to add host again.
What worked for me:
delete ssh config folder both in C:\Program Data\ssh and C:\<user>\.ssh
In VS Code, press F1, choose Remote-SSH: Connect to Host...
Do NOT enter anything in the prompt, but instead choose + Add New SSH Host..
Enter the full ssh command, including the key (in case of Windows,
you may want to enclose the path with double quote mark) ssh -i "C:\path\to\key" user#host. (you need to make sure the key has a limited permission. Remove all inherited permissions, and only give a full control to the owner.)
You will be asked to choose a folder in which a new config file will be created. Choose any of the two options.
There will a prompt notifying that the new config file
has been created. Click connect
At least three things may be happening:
Option 1
The location of your config file is not the absolute location, meaning you are probably using the location of the folder where the config file is.
If that is the case, access your User Settings in VSCode. Scroll to the Extensions>Remote - SSH. And add config at the end of the absolute file path of your custom SSH config file. In Windows, it can be
C:\Users\user\.ssh\config
See image below
Option 2
Authentication problems.
If that is the case, one of the things that may solve is generating new SSH keys.
In Windows, for that, I recommend using MobaXterm.
In MobaXterm, open a new terminal and write
ssh-keygen -b 4096 -t rsa
Then, in the config file, make sure that the IdentityFile points to the location of the key. MobaXterm's home directory, usually, is C:\Users\user\Documents\MobaXterm. If it makes it easy, one can copy/move the keys to C:\Users\user\.ssh and then just add, in the config file, IdentityFile ~/.ssh/KEY_rsa (where KEY_rsa is the name of the [public] key).
Note that if you use PuTTY to generate the keys, on the server OpenSSH authorized_keys file, one doesn't want the public key that one saves, but the one that appears on top (see image bellow):
Option 3
Your config file may be wrong.
The config file tends to look as follows. Double check if the fields have the information needed for the connection to be established.
Host Test # This is the name we want to give the host
User user # This is the username
Hostname blabla.com # This is the hostname
PreferredAuthentications publickey
IdentityFile ~/.ssh/KEY_rsa # This is the location of the key
IdentitiesOnly yes
Port 50 # This varies
What worked for me was to delete all of the contents of folder: C:\Users\MYNAME.ssh That meant to delete both the config file and known-hosts. The config was probably the most important one to delete.
The solution in my case was editing the json settings file for VSC as shown here: https://code.visualstudio.com/docs/remote/troubleshooting#_troubleshooting-hanging-or-failing-connections
In VSC go to File, Preferences, Setting and click on the upper right hand icon (Open Settings, JSON). Add these two lines to settings.json and retry connecting:
"remote.SSH.showLoginTerminal": true,
"remote.SSH.useLocalServer": false
In my case I had another setup:
Git bash in Windows was configured and I am using the ssh.exe provided by this tool
In the "Remote SSH" extension in VSCode, I specified the full path of this ssh.exe
I am using multiple servers (with ProxyJump)
The error message is the same as the OP but in the logs it was written that the ssh config file was not found, where all the folder names was concatenated (because it did not recognize the windows path separator)
Problem: the ssh.exe is using a different path convention thant VSCode. ssh.exe is using the "/c/Users/..." pattern and VSCode is using the "C:\Users..." pattern.
Solution:
Make sure the SSH config is at a standard place (C:\Users\LOGIN\.ssh\config)
Remove the absolute path of the config file in the "Remote SSH" settings in VSCode
VSCode will still be able to access the settings using the standard path, and the ssh.exe configuration will still look at the same standard path so the connexion is working.
Note:
I have the error only when connecting with multiple ssh servers (using ProxyJump). When connecting only to the first server, the solution of #pszrux and this one are both working for me.
This is probably something everyone has tried before looking here, but it worked for me. The server I was trying to ssh into was not responding, leading to the nonexistant pipe error. I rebooted the server and everything worked fine.
OS: windows 10
In my case there were permission issues. Repeatedly changing inheritance in windows did not solve the issue. Finally this worked
change the folder in which the config file is stored.
From C:/users/usr/.ssh/config to D:/config
and changed the config path in vscode remote ssh settings.
This worked for me.
This seems to be a problem with varied causes and corresponding remedies. In my case the problem had to do with the version of ssh I was using. In my Windows path there were two places were an instance of ssh.exe resided:
C:\Program Files (x86)\OpenSSH\bin
C:\WINDOWS\System32\OpenSSH\
After using both paths to set the "Remote.SSH: Path" parameter (which is in "Remote.SSH: Settings" [see here]), i.e. first C:\Program Files (x86)\OpenSSH\bin\ssh.exe and then C:\WINDOWS\System32\OpenSSH\ssh.exe, the problem still persisted.
Then I looked at this and tried the git-provided ssh.exe, which I already had on my system (otherwise, just install git, it's good stuff anyway :) )
Setting the SSH path parameter with that version did the trick for me, i.e. setting path to:
C:\Program Files\Git\usr\bin\ssh.exe
In my case, I did what dalilander said, but instead of deleting the entire '.ssh' folder, I just needed to delete the file 'known_hosts' and then it worked. So the servers I had saved were not deleted.
The path of that folder is C:\Users\yourUsername\.ssh
For Windows:
Adding the escape character before the private key file name & using quotes around the path solved my issue.
//config file
Host 12.12.12.12
HostName 12.12.12.12
IdentityFile "C:\Users\USERNAME/\PRIVATEKEY" <----Note /\
User username
Trying to add the full path in "IdentityFile" made the trick
" IdentityFile C:\Users\xxx.ssh\xxx"
The solution below may be the last resort but it perfectly solved the issue for me in a Windows 10 local machine. I simply delete the known_hosts file under the directory C:\Users\[your-username]\.ssh, relaunch VS Code and reconnect the remote server through Remote Explorer. Everything works normally afterward.
This seems to be a general error when the ssh connection fails for one of a multitude of reasons.
Adding what my issue was, and what helped, because I don't see it in the other answers in here: I had re-installed the box I was connecting to, and with it, reset the key it was using to authenticate. The ssh process tried to connect and failed with the usual "someone might be MITMing you this very moment, the identification changed" error, visible in the VSCode terminal. Solution was to go to my authorized_keys file and remove the offending key.
Obviously only know that if you know for sure why the identification changed, and that it's harmless. Don't actually get MITMed.
I had this problem once.
All you need to do is,
Go to /Users/XXX/.ssh
if you are on the windows, use command : "del /f known_hosts" to delete the known_hosts on the command prompt.
3.Then go to C:\Users\XXX.ssh\config on the vs code( config file )
4.Delete the host and the user if the host that you are trying to connect to is already there.
5.Then try to connect to the new host as usual.This will work.
The problem here could the mismatch of the finger prints once you reinstall the OS o n your host machine.
So to solve this problem by deleting the host that was saved.
once the config file on the vs code is edited it should look like..below picture is to show how the config file should look(after deleting the host saved)
If you're using WSL and might think that you should update ~/.ssh/config, that might not be the case.
Copy the content from ~/.ssh/config
Append it to C:\User\xxx\.ssh\config windows file
Make sure the public/private key is on C:\User\xxx\.ssh\ and is listed in config
Reconnect
Had an existing(working) configuration and had the same error when I added a new one. What worked for me is instead of just adding a new host configuration, I also commented out the first working config. Didn't know what happened but it worked.
In my case this was an offending key in my known_hosts in Windows (vscode on windows, remote developing via ssh on linux).
The error that comes back in vscode is not explaining in any way.
In my case, the path to key file was wrong.
For me, (windows) the permissions on the .pem file were the Critical issue. I had Administrator group only on the pem file and it was not working. I had to explicitly add the Admin user as well (even though admin is of course in administrator group).
In my case, I had no internet connection.
I was connecting to the server via VPN but the remote configuration was incorrect and I couldn't access the server. (DNS related issues) The connection indicator was showing no errors, so I didn't think of that at first.
Oops :)
I really didn't want to delete my C:Users\valo\.ssh\config, so I played a little with the various entries. It turned out that for me the option IdentitiesOnly yes was the problem. I also disabled security inheritance on all key files in the .ssh folder and left only myself, with Full Rights. Here is what my C:Users\valo\.ssh\config looks like now:
CanonicalizeHostname yes
Host aws.r3
HostName 3.31.45.216
ForwardAgent yes
User ubuntu
IdentityFile C:Users\valo\.ssh\u1-client-20210203-090555.pem
# IdentitiesOnly yes # VSCode Remote doesn't like this flag...?
Host github
HostName github.com
User git
IdentityFile ~/.ssh/id_val_ed25519
Host github.vm
HostName github.com
User git
IdentityFile ~/.ssh/id_valo_ed25519
Host *
ForwardAgent yes
AddKeysToAgent yes
LogLevel FATAL
ChallengeResponseAuthentication no
Now I can connect to aws.r3 with VSCode Remote.
A possible solution:
First run cat $HOME/.ssh/id_rsa.pub on your computer. That will get you a key. Save this key somewhere.
Then open this file by running vim $HOME/.ssh/authorized_keys on the computer that you're are ssh'ing to. Then copy the key in a new line of this file and close it by typing :wq.
You are all set.
In my case it was because the name I gave the host in config was myuser#myhostname. So if your config file looks like this:
Host myuser#myhostname
HostName 12.64.88.234
User myuser
Port 22
Try changing it to this:
Host myuser
HostName 12.64.88.234
User myuser
Port 22
Mine was solved by adding ".pem" extension while specifying the private key.
Here's the sample config text for reference.
Host ec2-3-234-8-176.compute-1.amazonaws.com
HostName ec2-3-234-8-176.compute-1.amazonaws.com
IdentityFile ~/.ssh/privatekey.pem
User username
There can be several reasons that have nothing to do with the accepted answer. For me:
Ubuntu 14.04.1 LTS didn't seem to work
Issues with EC2 auth, see pem file config and pem file permissions
And for yet another seeming cause/solution:
Adding the config path explicitly to settings only caused an EISDIR error.
Removing the listing from known_hosts made it need to confirm the fingerprint, but it didn't provide a way to do that. I could see it trying in the terminal output.
The solution a coworker recommended was to delete the vscode-server files from the server. That was the key step in my case. But...
Connecting to the server using another client, I attempted to rm -rf ~/.vscode-server. I could not delete many of the files because "device or resource busy".
That eventually required doing the following:
fuser ~/.vscode-server/[one of the child files ...]. But, you can probably skip this, because mainly I needed to know what to search for. Plus, fuser and lsof were finicky about returning results -- they often just sit waiting for something, I don't know what.
ps -e | grep node since node is the running process using vscode-server files.
For each PID in the list of node processes from step 2, run ps -o user= -p PID, substituting PID with each process PID in turn. This creates a formatted list of the process's associated user.
This is to determine which node processes you own, so you're not even trying to kill anybody else's.
Starting with the lowest node PID I own, run kill -9 PID. You won't need to run this for all PIDs, because killing a lower PID killed a child PID after a few seconds. So keep checking which node processes still exist after killing one: ps -e | grep node
Finally, once all mine are killed, I can rm -rf ~/.vscode-server
Then, I was able to connect via ssh in VS Code again. And, since I left the fingerprint removed from known_hosts, it even asked to confirm the connection to the server (up top, in the command prompt).
Also, for reference, I left the remote-ssh: settings config file entry, mentioned in other solutions, blank.
For reference or further explanation of certain steps above (I don't intend to elaborate more than I did):
rm: cannot remove ‘.vscode-server/bin/xxxxxx/.nfs000000000xxxxxxxxxxxx’: Device or resource busy
How find out which process is using a file in Linux?
https://unix.stackexchange.com/questions/284934/return-owner-of-process-given-pid/284938
In my case it worked when I added the port in my expression, eg ssh user#host-or-ip -p 22. With '22' the default port number, but you can check which port the ssh system is listening on with the sudo service ssh status command.
WSL Related
In my case, the issue was that my keys were set up on Ubuntu on WSL and VS Code was looking for them in Windows. I copied the keys over from the WSL side to Windows and voila! Worked like a charm!
Steps that I took.
Navigated to the /home//.ssh folder in WSL and then entered explorer.exe .
From there, I copied the id_rsa and id_rsa.pub files and pasted them in the windows side, under C:\Users<username>.ssh
Then I tried connecting again from VS Code and it worked perfectly.

How can I play a wav sound on the server side using cgi?

How can I run a command from a (bash) CGI script to play a wav sound on the server side?
You can execute your command line audio player as described by nak, but this may not work due to the permissions of the user running Apache. By default Apache is run as www-data:www-data (or apache:apache or www:www on some distros). As a quick fix/test you can set Apache to run as a user that has permissions to access the audio device on the machine by modifying your /etc/apache2/apache2.conf (or /etc/httpd/httpd.conf") file to have:
User USER_THAT_CAN_PLAY_AUDIO
Group USER_THAT_CAN_PLAY_AUDIO
Warning: this is not secure and is not intended to be a permanent solution!
This is how I would do it
#!/bin/sh
echo Content-type: text/plain
echo ""
echo "Server is playing sine.wav!"
aplay -q sine.wav
I stumbled over this old question looking how to solve the same problem: to have my personal Apache webserver warning me when someone makes a specific request (in my case a call for chat without the need to have any IM running).
The solution below is what I use on Slackware 14.1: according to your distro YMMV.
launch visudo
add the line TheUserRunningApache ALL=(ALL) NOPASSWD: /usr/bin/play (TheUserRunningApache is the user name used by your Apache)
In the PHP page you want to play a sound add this line: system ("sudo /usr/bin/play SOUND.WAV");
If you don't want to give access to Apache to the /usr/bin folder, even if limited just to play, you can copy the sox executable (the program used to run /usr/bin/play) elsewhere, but you'll have to modify the last two instructions above accordingly.

Can I forward env variables over ssh?

I work with several different servers, and it would be useful to be able to set some environment variables such that they are active on all of them when I SSH in. The problem is, the contents of some of the variables contain sensitive information (hashed passwords), and so I don't want to leave it lying around in a .bashrc file -- I'd like to keep it only in memory.
I know that you can use SSH to forward the DISPLAY variable (via ForwardX11) or an SSH Agent process (via ForwardAgent), so I'm wondering if there's a way to automatically forward the contents of arbitrary environment variables across SSH connections. Ideally, something I could set in a .ssh/config file so that it would run automatically when I need it to. Any ideas?
You can, but it requires changing the server configuration.
Read the entries for AcceptEnv in sshd_config(5) and SendEnv in ssh_config(5).
update:
You can also pass them on the command line:
ssh foo#host "FOO=foo BAR=bar doz"
Regarding security, note than anybody with access to the remote machine will be able to see the environment variables passed to any running process.
If you want to keep that information secret it is better to pass it through stdin:
cat secret_info | ssh foo#host remote_program
You can't do it automatically (except for $DISPLAY which you can forward with -X along with your Xauth info so remote programs can actually connect to your display) but you can use a script with a "here document":
ssh ... <<EOF
export FOO="$FOO" BAR="$BAR" PATH="\$HOME/bin:\$PATH"
runRemoteCommand
EOF
The unescaped variables will be expanded locally and the result transmitted to the remote side. So the PATH will be set with the remote value of $HOME.
THIS IS A SECURITY RISK Don't transmit sensitive information like passwords this way because anyone can see environment variables of every process on the same computer.
Something like:
ssh user#host bash -c "set -e; $(env); . thescript.sh"
...might work (untested)
Bit of a hack but if you cannot change the server config for some reason it might work.

Automatically cd to a given remote path when connecting via ssh

I have a bunch of remote servers that I regularly connect to via ssh; which I've setup in my ~/.ssh/config file. I was wondering if it was possible to specify a remote path to cd to when I connect to some of these servers?
For example, I may have something like this in my config file:
Host testbox
HostName 192.123.456.789
User root
And when I ssh in to testbox, I'd like to also cd to /var/www/apps/myapp.
I've had a look around but cannot see an option that would do that via the .ssh/config file.
Cheers,
Diego
You can do this with a tool I've open sourced that allows you to SSH and CD – aptly named sshcd. For the example you've given, you'd simply use:
sshcd root#testbox:/var/www/apps/myapp
Hope this helps!
There's an option in the authorized_keys file.
Do a man on sshd, look under the heading "AUTHORIZED_KEYS FILE FORMAT". You can add various options to each authorized key - one is command="command". As the manpage says, "Specifies that the command is executed whenever this key is used for authentication."