We have a clojure code that runs on Databricks, and fetches some large amount of data from Azure SQL Database.
Recently, we are getting frequent connection timeout errors.
I am new to Clojure, so I don't understand why this error occurs. Sometimes the code runs perfectly while sometimes it fails.
We have tried different connection parameters like "connectionretry" and "logintimeout" but it didn't work.
com.microsoft.sqlserver.jdbc.SQLServerException: Connection timed out (Read failed)
at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:234)
at com.microsoft.sqlserver.jdbc.SimpleInputStream.getBytes(SimpleInputStream.java:352)
at com.microsoft.sqlserver.jdbc.DDC.convertStreamToObject(DDC.java:796)
at com.microsoft.sqlserver.jdbc.ServerDTVImpl.getValue(dtv.java:3777)
at com.microsoft.sqlserver.jdbc.DTV.getValue(dtv.java:247)
at com.microsoft.sqlserver.jdbc.Column.getValue(Column.java:190)
at com.microsoft.sqlserver.jdbc.SQLServerResultSet.getValue(SQLServerResultSet.java:2054)
at com.microsoft.sqlserver.jdbc.SQLServerResultSet.getValue(SQLServerResultSet.java:2040)
at com.microsoft.sqlserver.jdbc.SQLServerResultSet.getObject(SQLServerResultSet.java:2372)
at clojure.java.jdbc$dft_read_columns$fn__226.invoke(jdbc.clj:457)
at clojure.core$mapv$fn__6953.invoke(core.clj:6627)
at clojure.lang.LongRange.reduce(LongRange.java:233)
at clojure.core$reduce.invokeStatic(core.clj:6544)
at clojure.core$mapv.invokeStatic(core.clj:6618)
at clojure.core$mapv.invoke(core.clj:6618)
at clojure.java.jdbc$dft_read_columns.invokeStatic(jdbc.clj:457)
at clojure.java.jdbc$dft_read_columns.invoke(jdbc.clj:453)
at clojure.java.jdbc$result_set_seq$row_values__233.invoke(jdbc.clj:483)
at clojure.java.jdbc$result_set_seq$thisfn__235.invoke(jdbc.clj:493)
at clojure.java.jdbc$result_set_seq$thisfn__235$fn__236.invoke(jdbc.clj:493)
at clojure.lang.LazySeq.sval(LazySeq.java:40)
at clojure.lang.LazySeq.seq(LazySeq.java:49)
at clojure.lang.RT.seq(RT.java:521)
at clojure.core$seq__4357.invokeStatic(core.clj:137)
at clojure.core$map$fn__4785.invoke(core.clj:2637)
at clojure.lang.LazySeq.sval(LazySeq.java:40)
at clojure.lang.LazySeq.seq(LazySeq.java:49)
at clojure.lang.Cons.next(Cons.java:39)
at clojure.lang.RT.next(RT.java:688)
at clojure.core$next__4341.invokeStatic(core.clj:64)
at clojure.core$dorun.invokeStatic(core.clj:3033)
at clojure.core$doall.invokeStatic(core.clj:3039)
at clojure.core$doall.invoke(core.clj:3039)
at clojure.java.jdbc$query$fn__340$fn__341.invoke(jdbc.clj:1007)
at clojure.java.jdbc$query$fn__340.invoke(jdbc.clj:1006)
The issue isn't related to Azure SQL Database JDBC driver but the way you are establishing the connection and running queries.
Make sure you are closing the connection properly after every commit. Additionally, you have a sufficient sleep time between connections.
Also, make sure that the large data you are fetching must have required bandwidth and properly queried.
You can refer these official documentation for troubleshooting:
Troubleshooting connectivity issues and other errors with Azure SQL Database and Azure SQL Managed Instance
Resolving connectivity errors to SQL Server
Related
I'm running SQL Server 2016 (13.0.5850.14) with a linked server connection to Google BigQuery using the Simba BigQuery driver v2.4.1.1009 (64 bit). I have multiple queries that I'm using to bring data back to our premise operational data store. The solution works fine for small record sets but there seems to be a point where larger record sets kill the SQL Service.
One of my queries returns a record set within a few seconds when limited to 40,000 records but kills the service when returning the full record set of approx 47,000 records.
SQL traces don't show any errors, neither does the event log. The SQL log does say:
Message
SQL Server is terminating because of fatal exception 40000015. This error may be caused by an unhandled Win32 or C++ exception, or by an access violation encountered during exception handling. Check the SQL error log for any related stack dumps or messages. This exception forces SQL Server to shutdown. To recover from this error, restart the server (unless SQLAgent is configured to auto restart).
I've already extended the timeout on the connections but that doesn't seem to have made any difference and I suspect its more some sort of memory issue with the driver but I really don't know what more I can do? Appreciate any suggestions
I've been using Anylogic's feature to collect data to a database provided by the program.
However I have trouble extracting the data again. I've tried using the export to excel function, but the program is getting stuck (probably because of the large amount of data).
Now I'm looking at connecting directly to the database. Anylogic provides me with a connection string: jdbc:hsqldb:hsql://localhost:9001/nau_sterilcentral;file:C:\Users\nbn\Models\NAU sterilcentral\database\db
I only use the first part (until the file:C:\...) since DBeaver doesn't ask for more. My connection string i DBeaver looks like this: jdbc:hsqldb:hsql://localhost:9001/nau_sterilcentral
I expect the db to be running since i can query it from Anylogic, but I'm not sure. When I try to connect i keep getting an error:
connection exception: connection failure: java.io.EOFException
org.hsqldb.HsqlException: connection exception: connection failure: java.io.EOFException
I don't know AnyLogic, but:
HSQLDB needs to be started in server mode to allow a connection from a different process. If AnyLogic starts HSQLDB in embedded mode, you can't access the database as long as AnyLogic is running.
Also the JDBC URL needs to be either a server URL using localhost:9001 or a "local" URL containing a file name, you can not mix both things (and the syntax for the file based URL was wrong as well)
Assuming AnyLogic starts HSQLDB in embedded mode, you have to stop AnyLogic, then you can connect using a file based URL. The syntax for that would be:
jdbc:hsqldb:C:\Users\nbn\Models\NAU sterilcentral\database\db
How can I capture queries that fail on the SQL Server due to connection drops/failures?
Profiler doesnt seem to capture if there is a connectivity loss.
Without catching the exception in the code this is not the easiest thing to do.
Have you checked the error logs in SQL? DA has a related answer for doing this: https://dba.stackexchange.com/questions/10364/sql-server-error-log-monitoring
You can check that your profiler is set up correctly:
http://support.microsoft.com/kb/224587
There are a number of sql monitoring tools out there, just google 'SQL server monitoring'.
I am trying to dump the whole data from the developer db server to my local machine using SQL Developer, but whenever I try to export, the connection gets halted before I finish dumping/exporting.
Is there a way to adjust that db connection timeout?
There is no connection time limit imposed by SQL Developer. If your connection is getting lost, that implies either that there is something in the network (a firewall for example) that limits the length of a connection or that there is something configured in the database (a profile, Resource Manager, etc.) that is causing the connection to be terminated. Since you haven't told us what error you get, it is impossible to guess which of these options is the most likely source of your problem.
Of course, it would probably be more effective to use the proper tools (the DataPump version of the export and import utilities) for this sort of thing.
I have been getting these intermittent ColdFusion Database connection reset errors and was wondering if anyone had experience with this and had a particular solution that worked?
Here is the error:
Error Executing Database Query.[Macromedia][SQLServer JDBC Driver]A problem occurred when attempting to contact the server (Server returned: Connection reset). Please ensure that the server parameters passed to the driver are correct and that the server is running. Also ensure that the maximum number of connections have not been exceeded for this server.
This doesn't happen with any particular query, the code breaks in different queries every time, returning a SQLState error 08s01. These query's logic are fine, no logic errors etc. I checked the network logs and there were no database server connection refusals at the time of the error. Once the first error occurs, it keeps happening for no more than a minute or so at random times of the day, every few days. I've googled this thing and so far anyone that has had this issue was only on CF6 or 7, which the fixes coldFusion put out are only for CF6 or 7.
Server configuration wise:
The ColdFusion server is version 8
The database server is SQL Server
2005 Standard
The database connections allowed
setting is set to unlimited on both
SQL Server and ColdFusion
Any help would be greatly appreciated, Thanks!
I'd personally try to update JDBC driver ( http://sourceforge.net/projects/jtds/ ) and add Validation Query if not already existing. I used once tool which was constantly checking TCP connections between servers, detecting even milisecond gap, check this tool http://www.colasoft.com/ping_tool/ .
Have you already resolved the issue?
I'm having the same problem on CF8 VPS, and after doing some research online I found this post:
http://www.mail-archive.com/cf-talk#houseoffusion.com/msg302264.html
It says it's related to some features on the network card and something on the SP2 for Win2k3 that messes it up. Don't know if that applies to you but just in case.