ADODB strange behavior - sql

Recently I had a very strange problem.
The application is written in classic ASP, but I guess it is the same case for every connection that uses ADO/OLEDB.
Those are connection parameters.
conn=Server.CreateObject("ADODB.Connection");
conn.Provider="Microsoft.Jet.OLEDB.4.0";
conn.Open("D:/db/testingDb.mdb");
In short this code:
conn.Open("myconnection");
bigQuery = "...";
rs = conn.execute(bigQuery);
while (!rs.eof) {
...
smallQuery = "..."
rssmall = conn.execute(smallQuery);
...
rssmall.close();
...
rs.movenext();
}
rs.close();
conn.close();
Doesn't work if bigQuery returns more than certain number of rows (in my case ~20).
But if I use one more connection for inner loop like stealthyninja suggested:
conn.Open("myconnection");
conn2.Open("myconnection")
bigQuery = "...";
rs = conn.execute(bigQuery);
while (!rs.eof) {
...
smallQuery = "..."
rssmall = conn2.execute(smallQuery);
...
rssmall.close();
...
rs.movenext();
}
rs.close();
conn2.close();
conn.close();
Problem vanishes.
I am using Access database and IIS7 if that matters.
Does anyone has a logical explanation for this?

Michael Todd's comment has it. ADODB doesn't support MARS (Multiple Active Result Sets) which is what you are trying to do. The reason it seems to work with only 20 records is because that's how much it is initially transmitting to the client-side.
Standard solutions to this are
Retrieve the whole outer rowset into a holding structure or cache first, then process it and execute the inner queries, or
Use two different connections, as you have demonstrated.

Related

Query across two SQLite databases in Delphi TFDQuery [duplicate]

I have an application that uses a SQLite database and everything works the way it should. I'm now in the process of adding new functionalities that require a second SQLite database, but I'm having a hard time figuring out how to join tables from the different databases.
If someone can help me out with this one, I'd really appreciate it!
Edit: See this question for an example case you can adapt to your language when you attach databases as mentioned in the accepted answer.
If ATTACH is activated in your build of Sqlite (it should be in most builds), you can attach another database file to the current connection using the ATTACH keyword. The limit on the number of db's that can be attached is a compile time setting(SQLITE_MAX_ATTACHED), currently defaults to 10, but this too may vary by the build you have. The global limit is 125.
attach 'database1.db' as db1;
attach 'database2.db' as db2;
You can see all connected databases with keyword
.databases
Then you should be able to do the following.
select
*
from
db1.SomeTable a
inner join
db2.SomeTable b on b.SomeColumn = a.SomeColumn;
Note that "[t]he database names main and temp are reserved for the primary database and database to hold temporary tables and other temporary data objects. Both of these database names exist for every database connection and should not be used for attachment".
Here is a C# example to complete this Question
/// <summary>
/// attachSQL = attach 'C:\\WOI\\Daily SQL\\Attak.sqlite' as db1 */
/// path = "Path of the sqlite database file
/// sqlQuery = #"Select A.SNo,A.MsgDate,A.ErrName,B.SNo as BSNo,B.Err as ErrAtB from Table1 as A
/// inner join db1.Labamba as B on
/// A.ErrName = B.Err";
/// </summary>
/// <param name="attachSQL"></param>
/// <param name="sqlQuery"></param>
public static DataTable GetDataTableFrom2DBFiles(string attachSQL, string sqlQuery)
{
try
{
string conArtistName = "data source=" + path + ";";
using (SQLiteConnection singleConnectionFor2DBFiles = new SQLiteConnection(conArtistName))
{
singleConnectionFor2DBFiles.Open();
using (SQLiteCommand AttachCommand = new SQLiteCommand(attachSQL, singleConnectionFor2DBFiles))
{
AttachCommand.ExecuteNonQuery();
using (SQLiteCommand SelectQueryCommand = new SQLiteCommand(sqlQuery, singleConnectionFor2DBFiles))
{
using (DataTable dt = new DataTable())
{
using (SQLiteDataAdapter adapter = new SQLiteDataAdapter(SelectQueryCommand))
{
adapter.AcceptChangesDuringFill = true;
adapter.Fill(dt);
return dt;
}
}
}
}
}
}
catch (Exception ex)
{
MessageBox.Show("Use Process Exception method An error occurred");
return null;
}
}
Well, I don't have much experience with SQLite you have to access both databases in a single query.
You can have something like :
select name from DB1.table1 as a join DB2.table2 as b where a.age = b.age;
In databases like SQLServer you can access other databases in this hierarchical fashion, this should also work for SQLite.
I think you can initiate an instance of sqlite with more than 1 databases !

SQL Azure - Transient "ExecuteReader requires an open connection" exception

I'm using SQL Azure in a Windows Azure app running as a cloud service. Most of the time my database actions works completely fine (that is, after handling all sorts of timeouts and what not), however i'm running into a problem that seems
using (var connection = new SqlConnection(m_connectionString))
{
m_ConnectionRetryPolicy.ExecuteAction(() => connection.Open());
using (var command = connection.CreateCommand())
{
command.CommandText = "SELECT * FROM X WHERE Y = Z";
var reader = m_CommandRetryPolicy.ExecuteAction(() => command.ExecuteReader());
return LoadData(reader).FirstOrDefault();
}
}
The line that fails is the Command.ExecuteReader with an:
ExecuteReader requires an open and available Connection. The connection's current state is closed
Things that i have already considered
I'm not "reusing" an old connection or saving a connection is a member variable
There should be no concurrency issues - the repository class that these methods belong to is created each time it is needed
Have anyone else experienced this? I could of course just add this to the list of exception which would yield a retry, but I'm not very comfortable with that as
I had a bunch of these errors a few days ago (West Europe) on my production deployment, but they went away by themselves. At the same time I was seeing timeouts, throttling and other errors from SQL Azure. I assume that there was a temporary problem with the platform (or at least the server that I am running on).
You probably aren't doing anything wrong in your code, but are suffering from degraded performance on SQL Azure. Try and handle the errors, perform retries, exponential back-off, queues (to reduce concurrency), splitting your load across databases — that sort of thing.
write every thing within try and catch,finally block.
as follows:
try
{
con.open();
m_ConnectionRetryPolicy.ExecuteAction(() => connection.Open());
using (var command = connection.CreateCommand())
{
command.CommandText = "SELECT * FROM X WHERE Y = Z";
var reader = m_CommandRetryPolicy.ExecuteAction(() => command.ExecuteReader());
return LoadData(reader).FirstOrDefault();
}
con.close();
}
catch(exception ex)
{
}
finally
{
con.close();
}
Remember to close connection in finally block as well.
There is an Enterprise Library that MS has produced specifically for SQL Azure, here are some examples from their patterns and Practice.
It's similar to what you are doing, however it does more on the reliability (and these examples show how to get a reliable connection)
http://msdn.microsoft.com/en-us/library/hh680899(v=pandp.50).aspx
Are you sure it's the reader that's failing and not the opening of the connection? I'm encountering an exception when I wrap the connection.Open() in the m_ConnectionRetryPolicy.ExecuteAction().
However it works just fine for me if I skip the ExecuteAction wrapper and open the connection using connection.OpenWithRetry(m_ConnectionRetryPolicy).
And I'm also using command.ExecuteReaderWithRetry(m_ConnectionRetryPolicy) which is working for me.
I have no idea though why it's not working when wrapped in ExecuteAction though.
I believe this means that Azure has closed the connection behind the scenes, without telling the connection pooler. This is by design. So, the connection pooler gives you what it thinks is an available, open connection, but when you try to use it, it finds out it's not open after all.
This seems very clunky to me, but it's the way Azure is at the moment.

When to close the result set (Basic ODBC question)

I am working on some small project for the local firm and the following code runs fine on my machine, but it produces errors on their server. Currently I don't have access to that server, and this is not a field that I know a lot about, so I have to ask you guys.
The page is written in the classic ASP (javascript for scripting).
The logic goes like this:
conn.Open("myconnection");
bigQuery = "...";
rs = conn.execute(bigQuery);
while (!rs.eof) {
...
smallQuery = "..."
rssmall = conn.execute(smallQuery);
...
rssmall.close();
...
rs.movenext();
}
rs.close();
conn.close();
As I said this runs fine on my machine, but it returns some error (the worst thing is that I don't even know what error) on company's server if bigQuery returns more than ~20 rows. Is there something wrong with my code (this really isn't my field, but I guess it is ok to gather data in the loop like this?), or is there something wrong with their IIS server.
Thanks.
edit:
More info:
It 's Access database. Everything is pretty standard:
conn=Server.CreateObject("ADODB.Connection");
conn.Provider="Microsoft.Jet.OLEDB.4.0";
conn.Open("D:/db/testingDb.mdb");
Queries are bit long, so I wont post them. They are totally ordinary selects so they aren't the issue.
I had a legacy Classic ASP application which I inherited that was very similar (big queries with other queries running within the loop retrieving the first query's results) that ran fine until forty or more records were returned. The way I "solved" the problem was to instantiate another connection object to run the other query. So using your pseudo code, try --
conn.Open("myconnection");
conn2.Open("myconnection")
bigQuery = "...";
rs = conn.execute(bigQuery);
while (!rs.eof) {
...
smallQuery = "..."
rssmall = conn2.execute(smallQuery);
...
rssmall.close();
...
rs.movenext();
}
rs.close();
conn2.close();
conn.close();
What server are they actually running?
Most newer versions of Windows Server don't actually come with the Jet 4.0 driver for 64 bit at all so you can't use an access database with that driver if your app runs as a 64 bit app. You can try running as 32 bit which might solve the problem.
There is an alternative driver packaged as an office component which may be an option.
Try writing a simple test page that literally opens and closes the database connection like so:
<%
Dim conn
Set conn = Server.CreateObject("ADODB.Connection")
conn.Provider = "Microsoft.Jet.OLEDB.4.0"
conn.Open("D:/db/testingDb.mdb")
Response.Write("Database Opened OK")
conn.Close()
%>
Run this on the production server and if you see Database Opened OK then you'll know it's definitely the query rather than the database causing the issue.
If you get an error trying to open the database then you need to changed to using the newer driver or try the app in 32 bit mode
In the case that it is the query causing the issue then it may be that you'll need to use the various additional arguments to the Open() method to try using a different cursor (forward only if you only need to iterate over the results once) which will change how ADODB retrieves the data and hopefully mediate any performance bottleneck related to the query.
Edit
If you want to try debugging the code a bit more add the following at the start of the file. This causes the ASP script processor to continue even if it hits an error
On Error Resume Next
Then at intervals throughout the file where you expect an error might have happened do
If Err <> 0 Then
Response.Write(Err.Number & " - " & Err.Description)
End If
See this article from ASP 101 for the basics of error handling in ASP and VBScript

NHibernate and Raw ADO.NET usage

Background: I am using Nhibernate in an ASP.NET MVC application with an open-session-in-view pattern and I need to use raw ADO.NET to execute some performance-critical database operations.
I'm somewhat confused about how I should be getting my connection instance as I've seen two different methods in numerous blog posts.
Do I want to use:
var connection = Session.Connection;
Or:
var connection = ((ISessionFactoryImplementor)sessionFactory).ConnectionProvider.GetConnection();
I can't seem to find a conclusive answer anywhere and I'm hoping that someone with some extensive NHibernate experience can chime in here.
If you already have a session, use the connection from it.
That will also allow you to share the transaction (if one is open) by enlisting your commands on it.
i'm using something in the lines of (also uses the underlying already-open transaction)
SqlCommand command = new SqlCommand(updateString, (SqlConnection)NHibernateSession.Connection);
command.Parameters.AddRange(parameters.ToArray());
try
{
ITransaction tx = NHibernateSession.Transaction;
tx.Enlist(command);
command.ExecuteNonQuery();
}
catch (SqlException)
{
NHibernateSessionManager.Instance.RollbackTransaction();
throw;
}

How to have ADO.NET consume persisted XML ADO recordset for updates

I am working on a .NET 2.0 conversion of a multi-layer client-server application. For the time-being, we're converting the server-side to .NET but leaving client apps in COM (VB6). My current work is focused on getting data to/from the detached COM based clients.
The current version under development is using ADO(COM) recordsets persisted as XML and sent to the clients. Clients then load up the XML in to a recordset object for read/write access to data. Data is sent back to the server in the same format.
Upon receipt of a persisted XML recordset for update, we do some rather kludgy parsing to produce UPDATE statements which get pushed through ADO.NET to update the source database accordingly.
In (very rough) pseudo-code:
adodb.recordset oRS = load(ADOXML)
for each rec in oRS
{
string sSQL = "Update table set value = " + rec.ValueField + " where key = " + rec.KeyField
executeNonQuery(sSQL)
}
*DISCLAIMER: The actual logic is nowhere near this crude, but the end-result is the same.
I'm wondering if anyone has a better solution to get the updated data back in to the database...?
Thanks,
Dan
Have you considered using a TableAdapter to perform an update on the data once you have set all rows to modified?
If you have a preconfigured SQLTableAdapter which has update functions built in then by reading in the table from XML into the dataset you can then call the update function from the table adapter to update the central DB with every row in the table.
var tableadapter = new MyTableAdapter();
var dataset = new MyDataSet();
dataset.ReadXml("c:\myxmldata.xml");
foreach(var row in dataset.tables[0].rows)
{
row.SetModified();
}
tableadapter.Update(dataset);