While trying to solve another issue with connection problems to our servers, I thought to solve the problem by setting the MaxConnections and MaxStartups to my sshd.conf
When restarting ssh everything seemed fine, but this morning I found out that our Jenkins server didn't connect to any of the dev servers. So I tried logging into the system, finding out that I cannot log in to any of our dev servers anymore.
Looks like I made a F##$up in the sshd.conf and created a lockout for all the dev servers.
When trying to login I get an "port 22: Connection refused" error.
Is there any other way to get into the systems without having to connect every disk to another server to adjust the sshd.conf??
There are several options available for recovery in this situation:
Use the interactive serial console. This requires advanced preparation.
Add a startup script to fix the file, and then reboot to trigger the script.
Shutdown the instance, attach the disk to a recovery instance, use the recovery instance to mount the disk and fix the file, then detatch the disk and recreate the instance using the fixed disk. (you can also do this from a snapshot for added safety).
Related
I have a google compute engine VM, running ubuntu, and utilising Laravel Forge.
I seem to get blocked by the VM after accessing SSH a few times (2-4), even if I'm logging in correctly. Restarting the VM unblocks me.
I first noticed the issue as I was having trouble logging into SSH, after a few attempts it would become unreachable. My website hosted on it also wouldn't resolve. After restarting the vm, I could try log into ssh again and my website works. This happened a couple time before I figured out how to correctly log in with SSH.
Next, trying to log in to the database with HeidiSQL, which uses plink, I log in fine. But it seems to keep reconnecting via SSH every time I do something, and after 2-4 of these reconnects, I get the same problem with the VM being unreachable by SSH and my website hosted on it being down.
Using SQLyog, which seems to maintain the one SSH connection, rather than constantly reconnecting like HeidiSQL, I have no problems.
When my website is down, I use those "down for everyone or just me" websites to see if it is down, and apparently it's just down for me, so I must be getting blocked.
So I guess my questions are:
1. Is this normal?
2. Can I unblock myself without restarting the VM?
3. Can I make blocking occur in a less strict way?
4. Why does HeidiSQL keep reconnecting via SSH rather than maintaining the one connection like SQLyog seems to?
You have encountered sshguard, which is enabled by default on the GCE Ubuntu images (at least on the 14.10 image, where I encountered it myself). There is a whitelist file at /etc/sshguard/whitelist.
The sshguard default configuration on my VM has a "dangerousness" threshold of 40. Most "attacks" that sshguard detects incur dangerousness of 10, so getting blocked after 4 reconnects sounds about right.
The attack signatures are listed here: http://www.sshguard.net/docs/reference/attack-signatures/
I would bet that you are connecting from an IP that has an invalid reverse DNS configuration (I was). Four connects like that and the default config blocks you for 20 minutes.
On my Laptop's installation of Weblogic 11g, Changing network (WIFI to Wired etc.) while the server is running seems to break Datasource connections.
I have to reboot (which takes forever) to get my apps running on Weblogic to work again.
I tried to go over the Data Source connection settings in the Console to disable "this" monitoring. But couldn't find a way to turn it off.
Wondering if there's a way to turn this freaky behavior off.
Try the following two steps in your connection pool settings:
Select "Remove Infected Connections Enabled"
Reduce "Test Frequency"
I was careless when I made some changes to my C# web service which accesses a postgresql database server at my work, and I forgot to close some connections I made. After testing the service many times over the past few days, I realized too late that I forgot to close my connections and now I can't access the server.
I've googled other solutions but they all involve me changing the max connection size in the postgres.conf file. The problem is that I don't have such a file on my own computer. I'm assuming the conf file is on the original machine that hosts the database server?
Is there any solution to this so that I can access the server again, and kill the old connections, without going to the original host machine of the database?
The only other options are:
to connect as the database superuser, which has superuser_reserved_connections other users cannot use; or
if you can, ssh in to the DB server and kill -TERM individual PostgreSQL worker backends. (on windows, that'd be rdp'ing in and using End Process on the backend postgres.exe processes, but it's much harder to pick a safe one to terminate on Windows, you'd want to use Process Explorer to examine the process heirachy and arguments so you can pick a regular backend).
In future you can prevent this by setting a connection limit on the user your appserver connects as; see ALTER USER.
I'm running automated testing procedures that emulates keystrokes and mouseclicks 24/7.
Although it runs fine locally, on an RDP session it stops running once minimized or disconnected. Apparently, the GUI doesn't exist if you can't physically see it on the screen.
There is a registry work-around for keeping the GUI active for minimizing the window, but I know of no way to keep it alive after disconnect.
Ideally, I would have this run on the server Windows console session which would not care about being disconnected but in a hosted environment (I tried Amazon and Go Daddy) there is no way to access the console session.
Does anyone know how I can get around this? Basically any solution that allows me to run my application on a VPS. I need the reliability of a host but the flexibility to run it as if I was sitting right in front.
Yes, you can.
There are two types of sessions in Windows: The "console" session which is always active, and there can only be a max of one of, and "terminal" sessions, a la RDP. Using "rdpwrap" on Github, you can have an unlimited number of terminal sessions.
RDP sessions will become "deactivated" when there is not a connection to them. Programs will still run, but anything that depends on GUI interaction will break badly.
Luckily, we may "convert" a terminal session into a console session instead of disconnecting from Remote Desktop normally by running the following command from inside the terminal session:
for /f "skip=1 tokens=3" %%s in ('query user %USERNAME%') do (tscon.exe %%s /dest:console)
This will disconnect you from the session, but it will still run with full graphical context. This answers your question. You can reconnect to it and it will become a terminal session again, and you can do this infinitely. And, of course, autohotkey works perfectly.
But, what if you need more than one persistent, graphics-enabled session?
To get an unlimited amount of graphics-persistent sessions, you can run Remote Desktop and start terminal sessions from within the "main" session described above. Normally Remote Desktop prevents this "loopback" behavior, but if you specify "127.0.0.2" for the destination, you will be able to start a terminal session with any number of the users on the remote machine.
The graphics-persistentness will only be present on terminal servers if they are not minimized, unless you create and set RemoteDesktop_SuppressWhenMinimized to 2 at the following registry location:
HKEY_LOCAL_MACHINE\Software\Microsoft\Terminal Server Client
With this you can get an unlimited number of completely independent graphics-persistent remote sessions from a single machine.
This could be a workaround, altough I have not tried it myself and it involves having another machine
Let's assume that at the moment you are creating a session to myserver.com
Local Client ----> myserver.com
Instead of doing that, you could try having a separate server (let's call it myslave.com) and use that to establish a session
Local Client ----> myslave.com ----> myserver.com
Then if you disconnect the Local Client ---> myslave.com session the GUI of the session between myslave.com ----> myserver.com should remain active.
It will work only if you are connected to the console session of myslave.com.
I found a similar way. I had same problem, i downloaded rdp wraper which allows you configure multiple session rpd server and one tool which is included (rdpchecker.exe) allows you connect to localhost so you can connect to your server from your server and you dont need that middle client.
This could be a workaround, altough I have not tried it myself and it involves having >another machine
Let's assume that at the moment you are creating a session to myserver.com
Local Client ----> myserver.com
Instead of doing that, you could try having a separate server (let's call it myslave.com) and use that to establish a session
Local Client ----> myslave.com ----> myserver.com
Then if you disconnect the Local Client ---> myslave.com session the GUI of the session
between myslave.com ----> myserver.com should remain active
If you are using a windows server you don't even need another machine.
1) Connect to the server with the remote desktop connection (#con1).
2) Create a new alias for your server system like "127.0.0.2" in Windows\System32\drivers\etc\hosts .
3) Now establish a new remote desktop connection from your windows server (in #con1) to itself (#con2).
4) Finally start your GUI needing application e.g. UI-Path in #con2 and then close #con1.
I ran into the same problem and noticed that using VNC (TightVNC) to take over the remote machine seems to solve the issue. I guess VNC uses the console screen. Once activated and logged-in it stays logged-in, also after a VNC disconnect. Make sure that the screen never turns off in the power options.
Take note that keeping the console logged-in on a VPS is in general not recommended.
As part of our databuild run a 3rd party program (3D Studio Max) to export a number of assets. Unfortunately if a user is not currently logged in, or the machine is locked, then Max does not run correctly.
This can be solved for freshly booted machines by using a method such as TweakUI for automatic login. However when a user connects via Remote Desktop (to initiate a non-scheduled build, change a setting, whatever) then after the session ends the machine is left in a locked state with Max unable to run.
I'm looking for a way to configure windows (via fair means or foul) so either it does not lock when the remote session ends, or it "unlocks" itself a short while after. I'm aware of a method under XP where you can run a batchfile on the machine which kicks the remote user off, but this does not appear to work on Windows Server.
There is a separate terminal service connection available called the 'console' connection.
You can connect to this space using mstsc /console /v:servername. Use mstsc /? for full command line options.
This allows you to connect, open up the terminal services manager and boot the bad sessions.
Logging in over RDP shouldn't affect whether the console locks. If you don't log out of RDP (just closing the client keeps your session pending), then your session will be locked. You can solve that with idle timeouts in Terminal Services Manager.
If your console is locking, that's a seperate policy in Local Computer Settings or some such. If you have a domain, set it with a GPO. If you need the exact name of the policy, let me know and I'll dig it up for you.
I assume by unlock you want to make sure that disconnected sessions are logged off. To do this
Administrative Tools | Terminal Services Configuration
Right-Click RDP-TCP on the Connections folder and choose Properties
Go to the Sessions tab and select the Override user settings check box
Configure the End a Disconnected session to your needed timeout value
more reading at http://technet.microsoft.com/en-us/library/cc758177.aspx
You might want to look at using the "shadow" utility. This allows you to essentially proxy into an existing remote desktop session. You could log into the console of the machine with the account you need, then users could open non-console remote desktop sessions to the machine (or to another machine) then use shadow to connect to the same console session. The users will have to be in the administrators group on the machine.
Although, this might be as simple as telling people not to use the console session when logging into the machine using remote desktop.
Possible Solution from here.
To disable the Lock Computer button,
open Regedit and browse to
HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Policies\
System and
HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\CurrentVersion\Policies\
System and create a new REG_DWORD
value in each called
DisableLockWorkstation. Setting this
value to 0 will allow the Lock
Computer button to be used, while 1
will disable it.
There may be a problem if you are running these tasks as Administrator and others are logging in via Remote Desktop as Administrator. The task should be run from its own account.
With the most recent terminal services client you can connect to the console using the /ADMIN switch.
So "Computer:" will be something like:
myworkstation.mydomain.local /ADMIN
-Ed