I would like to be able to tag an SQL query somehow, so I can relate the query execution to the web request that triggered the query. I already have a unique request id, that I tag my logs and other monitoring with, so I can easily do a complete trace across the weblogs and new relic for example.
But when I look at a report of long running SQL queries for example, I cannot trace that back to the request that triggered the SQL Query. I would really like to be able to tag the query with my request id somehow.
I can't find anything online. When I search I just find blogs about storing tags and tag clouds in SQL. Not really what I need.
Hope the question makes sense.
This is a very interesting post...
I hope, adding an extra nullable parameter to your stored procedure(s) will ensure that the profiler will catch the unique id passed during a call (in the trace) whether you use that parameter inside the procedure or not (i.e. to do something meaningful...like inserting into an audit table with unique id, procedure name, timestamp etc).
But I think that will make life difficult as you now have to update all your procedures.
If you already have logging turned on (web server) and it captures the same unique id in its request (log file) along with a timestamp then you probably can code a small utility app that reads the log file and find matching entries in the traced table by the timestamp alone.
The only thing that might go wrong is if your web server and database server have differeing times (you need to offset your calculation accordingly).
I don't know if this will help but it is certainly a very interesting project and I am hoping somebody have experienced this thing and came up with a nice solution.
Will be closely watching this post if such a solution exists....
All the best...
If I understand correctly, you want to follow up the query execution in Activity Monitor. But have you considered using a DMV or SQL PROFILER ?
In my opinion, your best bet would be to wrap it in a stored proc. This way you will be able to FILTER your trace only for this object. Here's an example of a simple select and the same select wrapped in stored proc named sproc1 :
As you can see in this image, you can start a SQL PROFILER trace and filter it on the ObjectName. You can then add other column like CPU, StartTime, ...
If you can't use a stored proc, then I would suggest to insert a comment before the exec like this:
/* ID1234 */
select * from table1
Then use SQL PROFILER the same way but you now filter on the TextData using your ID
Here the result :
Earlier we were using Sybase as the back end. Now we are migrating to SQL server where as front end remains the same i.e. PowerBuilder.
Issue :
I have a DataWindow code which takes two retrieval arguments adt_from_date and adt_to_date. Both Date format. This works fine for PB-Sybase combination, but throws an error 37000 for SQL server.
Here is our code. If I hard-code the dates. i.e. for e.g. if I replace :adt_from_date by '20141001'. The code works fine.
SELECT A.MEMBER_NO AS 'MEMBER_NO',
ROUND(SUM(TRANSACTION_CHARGES),2) as 'AMOUNT',
SUBSTRING(DATENAME(MM, :adt_from_date ),1,3) +'-'+substring(convert(varchar,datepart(YY, :adt_from_date )),3,2) as 'REASON'
FROM TRAN_SERVICE_TAX_DRV A,
MEMBER_MASTER B
WHERE A.MEMBER_NO = B.MEMBER_NO
AND A.TRADE_DATE BETWEEN :adt_from_date AND :adt_to_date
GROUP BY A.MEMBER_NO
Please suggest something on this.
I'd suggest you look at the error text being loaded into the transaction instead of just the number; it will probably be much more helpful. If we're talking about SQLState 37000, that doesn't narrow it down much.
If that doesn't shed some light on it, I'd look at tracing either on the PowerBuilder side, or on the database side. Starting on the PB side probably makes most sense. If you're just running this DataWindow from the IDE, the trace is just a checkbox on the connection properties. If in the app, replace your "DBMS='xxx'" with "DBMS='tra xxx'" (there are other options described in the Connecting to your Database manuals).
Good luck.
We are getting the following error using Oracle:
[Oracle JDBC Driver]Application failover does not support non-single-SELECT statement
The error occurs when we try to make a delete or insert over a large number of rows (tens of millions of rows).
I know that the script works, because it was working for almost an year before these error messages start to pop.
We know that no one change any database configuration, so we figure out that the problem must be on the volume of processed data (row number is growing as time goes by...).
But we never see that kind of error before! What does it means? It seems that a failover engine tries to recover from an error, but when oracle is 'taken over' by this engine, it enter in a more restricted state, where some kinds of queries does not work (like Windows Safe Mode...)
Well, if this is what is happening, how can I get the real error message? The one that trigger the failover mechanism?
BTW, below is one of the deletes that triggers the error:
delete from odf_ca_rnv_av_snapshot_week
(we tried this one just to test the simplest delete we could think of... a truncate won't help us with the real deal :) )
check this link
the error seems to come not from Oracle or JDBC, but from "progress". It means that it can only recover from SELECT statements and not from DML.
You'll have to figure out why the failover occurs in the first place.
I am having this big database on one MSSQL server that contains data indexed by a web crawler.
Every day I want to update SOLR SearchEngine Index using DataImportHandler which is situated in another server and another network.
Solr DataImportHandler uses query to get data from SQL. For example this query
SELECT * FROM DB.Table WHERE DateModified > Config.LastUpdateDate
The ImportHandler does 8 selects of this types. Each select will get arround 1000 rows from database.
To connect to SQL SERVER i am using com.microsoft.sqlserver.jdbc.SQLServerDriver
The parameters I can add for connection are:
responseBuffering="adaptive/all"
batchSize="integer"
So my question is:
What can go wrong while doing this queries every day ? ( except network errors )
I want to know how is SQL Server working in this context ?
Further more I have to take a decicion regarding the way I will implement this importing and how to handle errors, but first I need to know what errors can arise.
Thanks!
Later edit
My problem is that I don't know how can this SQL Queries fail. When i am calling this importer every day it does 10 queries to the database. If 5th query fails I have to options:
rollback the entire transaction and do it again, or commit the data I got from the first 4 queries and redo somehow the queries 5 to 10. But if this queries always fails, because of some other problems, I need to think another way to import this data.
Can this sql queries over internet fail because of timeout operations or something like this?
The only problem i identified after working with this type of import is:
Network problem - If the network connection fails: in this case SOLR is rolling back any changes and the commit doesn't take place. In my program I identify this as an error and don't log the changes in the database.
Thanks #GuidEmpty for providing his comment and clarifying out this for me.
There could be issues with permissions (not sure if you control these).
Might be a good idea to catch exceptions you can think of and include a catch all (Exception exp).
Then take the overall one as a worst case and roll-back (where you can) and log the exception to include later on.
You don't say what types you are selecting either, keep in mind text/blob can take a lot more space and could cause issues internally if you buffer any data etc.
Though just a quick re-read and you don't need to roll-back if you are only selecting.
I think you would be better having a think about what you are hoping to achieve and whether knowing all possible problems will help?
HTH
We got a legacy vb.net applicaction that was working for years
But all of a sudden it stops working yesterday and gives sql server timeout
Most part of application gives time out error , one part for example is below code :
command2 = New SqlCommand("select * from Acc order by AccDate,AccNo,AccSeq", SBSConnection2)
reader2 = command2.ExecuteReader()
If reader2.HasRows() Then
While reader2.Read()
If IndiAccNo <> reader2("AccNo") Then
CAccNo = CAccNo + 1
CAccSeq = 10001
IndiAccNo = reader2("AccNo")
Else
CAccSeq = CAccSeq + 1
End If
command3 = New SqlCommand("update Acc Set AccNo=#NewAccNo,AccSeq=#NewAccSeq where AccNo=#AccNo and AccSeq=#AccSeq", SBSConnection3)
command3.Parameters.Add("#AccNo", SqlDbType.Int).Value = reader2("AccNo")
command3.Parameters.Add("#AccSeq", SqlDbType.Int).Value = reader2("AccSeq")
command3.Parameters.Add("#NewAccNo", SqlDbType.Int).Value = CAccNo
command3.Parameters.Add("#NewAccSeq", SqlDbType.Int).Value = CAccSeq
command3.ExecuteNonQuery()
End While
End If
It was working and now gives time out in command3.ExecuteNonQuery()
Any ideas ?
~~~~~~~~~~~
Some information :
There isnt anything that has been changed on network and the app uses local database
The main issue is that even in development environment it donest work anymore
I'll state the obvious - something changed. It could be an upgrade that isn't having the desired effect - it could be a network component going south - it could be a flakey disk - it could be many things - but something in the access path has changed. What other problem indications are you seeing, including problems not directly related to this application? Where is the database stored (local disk, network storage box, written by angels on the head of a pin, other)? Has your system administrator "helped" or "improved" things somehow? The code has not worn out - something else has happened.
Is it possible that this query has been getting slower over time and is now just exceeded the default timeout?
How many records would be in the acc table and are there indexes on AccNo and AccSeq?
Also what version of SQL are you using?
How long since you updated statistics and rebuilt indexes?
How much has your data grown? Queries that work fine for small datasets can be bad for large ones.
Are you getting locking issues? [AMJ] Have you checked activity monitor to see if there are locks when the timeout occurs?
Have you run profiler to grab the query that is timing out and then run it directly onthe server? Is it faster then? Could also be network issues in moving the information from the database server to the application. That would at least tell you if it s SQl Server issue or a network issue.
And like Bob Jarvis said, what has recently changed on the server? Has something changed in the database structure itself? Has someone added a trigger?
I would suggest that there is a lock on one of the records that you are trying to update, or there are transactions that haven't been completed.
I know this is not part of your question, but after seeing your sample code i have to make this comment: is there any chance you could change your method of executing sql on your database? It is bad on so many levels.
Perhaps should you set the CommandTimeout property to a higher delay?
Doing so will allow your command to wait a little longer for the underlying database to respond. As I see it, perhaps are you not letting time enough for your database engine to perform all what is required before creating another command to perform your update.
Know that the SqlDataReader continues to "SELECT" while feeding the in-memory objects. Then, while reading, you require your code to update some other table, which your DBE just can't handle, by the time your SqlCommand requires, than times out.
any chances of a "quotes" as part of the strings you are passing to queries?
any chances of date dependent queries where a special condition is not working anymore?
Have you tested the obvious?
Have you run the "update Acc Set AccNo=#NewAccNo,AccSeq=#NewAccSeq where AccNo=#AccNo and AccSeq=#AccSeq" query directly on your SQL Server Management Studio? (Please replace the variables with some hard coded values)
Have you run the same test on another colleague's PC?
Can we make sure that the SQLConnection is working fine. It could be the case that SQL login criteria is changed and connection is getting a timeout. It will be probably more helpful if you post the error message here.
You can rewrite the update as a single query. This will run much faster than the original query.
UPDATE subquery
SET AccNo = NewAccNo, AccSeq = NewAccSeq
FROM
(SELECT AccNo, AccSeq,
DENSE_RANK() OVER (PARTITION BY AccNo ORDER BY AccNo) NewAccNo,
ROW_NUMBER() OVER (PARTITION BY AccNo ORDER BY AccDate, AccSeq)
+ 10000 NewAccSeq
FROM Acc) subquery
After HLGEM's suggestions, I would check the data and make sure it is okay. In cases like this, 95% of the time it is the data.
Make sure disk is defragged. Yes, I know, but it does make a difference. Not the built-in defragger. One that defrags and optimizes like PerfectDisk.
This may be a bit of a long shot, but if your entire application has stopped working, have you run out of space for the transaction log in your database? Either it's been specified to an absolute size, and that has been reached, or your disk is just full.
May be your tables include more information, and defined SqlConnection.ConnectionTimeout property value in config file with little value. And this value isn't necessary to execute your queries.
you can trying optimize your queries, and also rebuilt indexes.