I'm using "redis-rs" for rust and I tested sending a few thousand requests locally
and it works really well at the few first, except that at some point it stops accepting requests
and starts showing me this error when I send a command to redis:
"Only one usage of each socket address (protocol/network address/port) is normally permitted"
I am opening a client and a connection on every request to the http server that takes them in,
that's probably a bad idea in the first place, but shouldn't the connections stop existing and close after the function that opened it ends?
Is there any better solution, like some kind of global connection?
thanks
Well if it is an http server, the crate you are using likely is doing multithreading to handle requests. It is possible that one thread got caught in the process of closing the connection just as another thread began processing the next request.
Or in your case, maybe the remote database has not finished closing the previous request by the time the next connection is created. Either way, it's easier to think of it as a race condition between threads.
Since you don't know what thread will request a connection next, it may be better to store the connection as a global resource. Since I assume a mutex lock is faster than opening and closing a socket, I used lazy_static to create a single thread safe connection.
use lazy_static::lazy_static;
use parking_lot::Mutex;
use std::sync::Arc;
lazy_static! {
pub static ref LOCAL_DB: Arc<Mutex<Connection>> = {
let connection = Connection::open("local.sqlite").expect("Unable to open local DB");
connection.execute_batch(CREATE_TABLE).unwrap();
Arc::new(Mutex::new(connection))
};
}
// I can then just use it anywhere in functions without any complications.
let lock = LOCAL_DB.lock();
lock.execute_batch("begin").unwrap();
// etc.
Related
I have multi-threaded (linux) server that registers async_writes and async_reads on the same native file descriptor through a socket object. I noticed under very heavy load when the server was dropping connections, on a very rare occasion a client would receive a garbled first message.
Tracking it down, the async_read detects an error on the socket and closes the socket. This closes the native file descriptor. If that file descriptor is reused before the original async_write has a chance to fire, it will find its native file descriptor valid and proceed to send its message (which is really a message from a previous session).
The only way I could see to fix this was to make the the async_read and async_write callbacks know if there were other callbacks registered and only close the socket if it were the last one.
Has anyone seen this issue?
Haven't seen it but it sounds plausible. Although I am surprised to see a new native file descriptor getting the exact same number than a recently closed descriptor.
You might want to put the socket in a shared_ptr and query shared_ptr::is_unique in both async_read and async_write. That'd be the easiest way to let the other callback know if both callbacks are registered. If is_unique is true you can be sure that no one else is still using this socket and can close it.
So if the connection gets dropped, async_read can check is_unique. If it is true, close the socket. And let go of the shared_ptr in either case.
Then, when async_write also fires it will find is_unique true and can close the socket, unless async_read has already closed it.
The only drawback is of course: async_write has to fire also (perhaps with an error code) in order to close the socket.
Oh I've seen exactly this in production code. (Much fun: we would be talking a proprietary protocol on a TCP socket to mysql server). The problem is when some thread "handles" (mis-handles) errors by closing sockets using the native handle (fd). Don't. Use shutdown (perhaps with cancel) instead and let the destructor take care of close. Of course, the real problem is the non-owning copies of the handle (fd) that are the cause of the resource race.
Critical Note:
Tracking it down, the async_read detects an error on the socket and closes the socket. This closes the native file descriptor
That's patently UNTRUE for Asio itself. Perhaps you have (third-party) code in the completion handlers doing that, but as I mention above, you cannot afford to do that.
I have an application that needs to stream data up to a server over a particular port. When the application starts it performs a number of checks, for network connectivity, a reachable host, among others. I'd like it to also check the port to ensure it's able to receive data.
I have this code:
Private Function CheckPort(ByVal Host As String) As Boolean
Dim IPs() As IPAddress = Dns.GetHostAddresses(Host)
Dim s As New Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp)
s.Connect(IPs(0), <port # removed>)
If s.Poll(-1, SelectMode.SelectWrite) Then
Return True
End If
Return False
End Function
The code works well, but I'm concerned that by performing the check I might be inadvertently preventing subsequent messages from the application from reaching the port. Will performing this check interfere with the actual data I want to send to the host? Should I close the socket with s.close() before returning the function?
I mainly agree with CodeCaster's response.
If you say the server is buggy, it is also probable that a few minutes after you check the port connection, the connection is broken or even closed by the server.
If you still want to do it as a means to reduce risk of making the user write some message that later cannot be sent, this would be a good approach to follow. An alternative is that you can save the data as draft locally or somewhere else for sending it later when the server is available.
Now, going to your code:
You are opening a connection inside function CheckPort so when the function finishes you will lose any reference to s, the open socket. In C, you would have a leak of resources; in Java, a garbage collector will take care of the Socket and close it. I don't know how it is in VB but I'd close that socket before leaving the function. In any case it's a healthy practice.
Additionally, when you have data ready to send you won't be able to do it on the same connection if you lose the reference (s) to the open socket or you close the socket (unless VB has any trick that I don't know). You will have to open a new connection to send data. Don't worry, you will be able to do it even if you made the check before. The server will see it as just a different connection, maybe from a different source port (or could be the same if you closed the connection before, that depends on the OS).
Checking a port before connecting to it is as useful as verifying file existence before opening it: not at all.
The file can be deleted in between checking and opening, and your socket can be closed for a variety of reasons.
Just connect and write to it, it'll throw an appropriate exception when it won't work.
Do you know if it's possible actually to disconnect a rtmpconnection and how ?
There is no "disconnect" method in the official doc, and also in the rtmpconnection.lzx . So if you know a way out to disconnect the rtmp connection,please let me know. Thanks in advance.
The <rtmpconnection> class in OpenLaszlo uses the ActionScript 3 NetConnection class to connect to the server. The NetConnection class has a method close(), here is the documentation for that:
Closes the connection that was opened locally or to the server and
dispatches a netStatus event with a code property of
NetConnection.Connect.Closed.
This method disconnects all NetStream objects running over the
connection. Any queued data that has not been sent is discarded. (To
terminate local or server streams without closing the connection, use
NetStream.close().) If you close the connection and then want to
create a new one, you must create a new NetConnection object and call
the connect() method again.
The close() method also disconnects all remote shared objects running
over this connection. However, you don't need to recreate the shared
object to reconnect. Instead, you can just call SharedObject.connect()
to reestablish the connection to the shared object. Also, any data in
the shared object that was queued when you issued
NetConnection.close() is sent after you reestablish a connection to
the shared object.
With Flash Media Server, the best development practice is to call
close() when the client no longer needs the connection to the server.
Calling close() is the fastest way to clean up unused connections. You
can configure the server to close idle connections automatically as a
back-up measure.
In the LZX source code for the <rtmpconnection> I can see that NetConnection.close() is only called in case of a connection failure:
<!--- Handle connection failure, attempt to reconnect using altsrc
#keywords private -->
<method name="_handleConnectionFailure" args="msg"><![CDATA[
this._nc.close();
if (this.debug) {
if ($debug) Debug.warn("error connecting to", this._connecturl, ":", msg);
}
....
I don't know why there is no close method defined on the <rtmpconnection> class, but you could add that code for your yourself, by extending the <rtmpconnection> and adding a close method. Just make sure you handle the state variables correctly.
As I haven't used Red5 in a long time, I cannot tell you if Red5 automatically closes NetConnections which are idle.
I'm building a program that has a very basic premise.
For X amount of Objects
Open Conection
Perform Actions
Close Connection
Open Next
Each of these connections is made on a socks5 proxy and after about the 200th connection I get "The operation has timeout" errors. I have tested all the proxies and they work just fine and the really wierd thing is if I shut down the program and restart it again the problems go away. So I'm left to believe that when I'm closing my connection that its really not closing the connection and the computer is being overloaded. How cna i force all socks connections to close that are associated with a class?
socket.Shutdown(SocketShutdown.Both);
//socket.Close();
socket.Disconnect(true);
socket = null;
In reponse to a tip to use netstat I checked it out. I noticed connections where lingering but finally would go away. However, the problem still remains, after about the 100th connection, 5 second pause between connections. I get timeout errors. If I close the proram and restart it they go away. So for some reason I feel that the connections are leaving behind something. Netstat dosent show it. I've even tried adding the instances of the client to a list and each time one is finish remove it from the list and then set it to null. Is there a way to kill a port? Maybe that would work, if I killed the port the connection was being made on? Is it possible this is a Windows OS issue? Something thats used to prevent viruses? I'm making roughly a connection a minute and mainint that connection for about 1 minute before moving on to the next with atleast 20 concurent if not more connections at the same time. What dosent make sense to me is that shuting down the program seem sto clean up whatever resources I'm not cleaning up in my code. I'm using an class I found on the internet that allows socks5 proxies to be used with the socket class. So i'm really at a loss, any advice or direction to head woudl be great? It dosent have to be pretty. I'm have tempted to wite to a text file where I was in my connection list and shutdown the program and then have anohter program restart it to pick up where it left off at this point.
Sounds like your connections aren't really closed. Without seeing the code, it's hard to troubleshoot this; can you boil it down to a program that loops through an open-close sequence?
If the connection doesn't close as expected, you can probably see what state it is in with netstat. Do you have 200 established connections, or are they in some sort of closing state?
Sockets implement IDisposable. Only calling Dispose or Close will cause the socket to give give up the unmanaged resources in a deterministic manner. This is causing you to run out of the resources that the socket uses (probably a network handle of some sort), even though you may not any managed object useing them.
So you should probably just do
socket.Shutdown(SocketShutdown.Both);
socket.Close();
To be clear setting the socket to Null does not do this because setting the socket to null only causes the sockets to be placed on the freachable queue, to have its finalizer called when it gets around to processing the freachable queue.
You may want to review this article which gives a good model on how Unmanaged resources are dealt with in .NET
Update
I checked and Sockets do indeed contain a handle to a WSASocket. So unless you call close or dispose you'll have to wait until the Finalizers run (or exiting the appplication) for you to get them back.
I have a requirement where I need to send continuous updates to my clients. Client is browser in this case. We have some data which updates every sec, so once client connects to our server, we maintain a persistent connection and keep pushing data to the client.
I am looking for suggestions of this implementation at the server end. Basically what I need is this:
1. client connects to server. I maintain the socket and metadata about the socket. metadata contains what updates need to be send to this client
2. server process now waits for new client connections
3. One other process will have the list of all the sockets opened and will go through each of them and send the updates if required.
Can we do something like this in Apache module:
1. Apache process gets the new connection. It maintains the state for the connection. It keeps the state in some global memory and returns back to root process to signify that it is done so that it can accept the new connection
2. the Apache process though has returned the status to root process but it is also executing in parallel where it going through its global store and sending updates to the client, if any.
So can a Apache process do these things:
1. Have more than one connection associated with it
2. Asynchronously waiting for new connection and at the same time processing the previous connections?
This is a complicated and ineffecient model of updating. Your server will try to update clients that have closed down. And the server has to maintain all that client data and meta data (last update time, etc).
Usually, for continuous updates ajax is used in a polling model. The client has a javascript timer that when it fires, hits a service that provides updated data. The client continues to get updates at regular intervals without having to write an apache module.
Would this model work for your scenario?
More reasons to opt for poll instead of push
Periodic_Refresh
With a little patch to resume a SUSPENDED mpm_event connection, I've got an asynchronous Apache module working. With this you can do the improved polling:
javascript connects to Apache and asks for an update;
if there's no updates, then instead of answering immediately the module uses SUSPENDED;
some time later, after an update or a timeout happens, callback fires somewhere;
callback gives an update (or a "no updates" message) to the client and resumes the connection;
client goes to step 1, repeating the poll which with Keep-Alive will use the same connection.
That way the number of roundtrips between the client and the server can be decreased and the client receives the update immediately. (This is known as Comet's Reverse Ajax, AFAIK).