I'm trying to log hibernate activity (only dml operations) to an sql script file.
My goal is to have a way to reconstruct the database from a given starting point to the current state by executing the generated script.
I can get the sql queries from log4j logs but they have more information than the raw sql queries and i would need to parse them and extract only the helpful statements.
So i'm looking for a programatic way, maybe by listening the persist/merge/delete operations and accessing the hibernate-generated sql statements.
I don't like to reinvent the wheel so, if anybody know a way for doing this i would appreciate it very much.
Thanks in advance
Generally the best way to do this is to just turn on logging on your SQL server. All the major RDBMSes support logging all the SQL statements that they run. This has the added advantage of catching things that happened outside of Hibernate.
You could also try to use NHProf which will intercept/record hibernate traffic to the database and dump it into an XML file. You might have to parse the file by hand, but all the information will be there.
You could also hook at the JDBC level directly and record the JDBC statements that are performed.
P6Spy is a great tool to inspect what's going on. It can log the queries, though I don't know if you can replay them as is.
I'm sure there are other such tool (or at worse you could try subclass the DataSource, Connection and PreparedStatement implementation of your choice to do that yourself).
Related
When I troubleshoot a large .NET app which uses only stored procedures, I capture the sql which includes the SP name from SQL Server Profiler and then it's easy to do a global search for the SP in the source files and find the exact line which produced the SQL.
When using Entity Framework, this is not possible due to the dynamic creation of SQL statements. However there are times when I capture some problematic sql statements from production and want to know where in the code they were generated from.
I know one can have EF generate logs and tracing on demand. This probably would be taxing for a busy server and produces too much logs. I read some stuff about using mini profiler but not sure if it fits my needs as I don't have access to the production server. I do however have access to attach SQL Server Profiler to the database server.
My idea is to find a way to have EF attach/inject a unique code to the generated SQL but it doesn't affect the outcome of the SQL. I can then use it to cross reference it to the line of code which injected it into the SQL. The unique code is static which means a unique static code is used for every EF linq statement. Maybe sent as a dummy sql or a comment along with the sql statement.
I know this will add some extra traffic but in my case, it will add extra flexibility and cut a lot of troubleshooting time.
Any ideas of how to do this or any alternatives?
One very simple approach would be to execute something via ExecuteStoreCommand(): Refresh data from stored procedure. I'm not sure if you can "execute" just a comment, but at the very least you should be able to do something like:
ExecuteStoreCommand("DECLARE #MyTag VARCHAR(100) = 'some_unique_id';");
This is very simple, but you would have to find the association in two steps:
Get the SessionID (i.e. SPID) from poorly performing query in SQL Server Profiler
Search the Profiler entries for the prior SQL statement for that same SPID
Another option that might be a little more complicated but would remove that additional step when it comes to making that association is to "intercept" the commands before they get executed and inject a comment with your unique id. Please see the following S.O. Answer for details. You shouldn't need the full extent of what they did, but even if you do, it seems like all of the code (or all the relevant stuff) is there:
Adding a query hint when calling Table-Valued Function
By the way, this situation is a point in favor of using Stored Procedures instead of an ORM. And, what do you expect to be able to do in terms of performance tuning once you do find the offending app code? (another point in favor of using Stored Procedures instead of an ORM ;-).
I am investigating a legacy app that uses an Oracle 8i database in a test environment, specifically trying to find out what tables are accessed for read, insert, update or delete when the user performs an app function.
What is the best/easiest way to do this? Can I simply get a list of all sql statements sent to the database? Can I see when stored procedures are called?
Having little experience with Oracle but getting help from a DBA, I'm thinking I should either use a trace or look at the redo log with LogMiner, but how?
Thanks!
What you could do is to harvest the sql's from v$sql. If the SQL's are properly written - using bind variables - you should be able to catch most of the statements in a table for this. I currently have no running v8 at hand but this should be possible.
In order to get most of them, you probably need to repeat the harvesting during the various workloads that run on the database.
My boss and I have been trying to see what sort of auditing plan we could try for our stored procedures. Currently there're two external applications taking information from our database through stored procedures and we're interested in auditing when they're being executed, and what values are passed as parameters. So far what I've done is simply create a table for the stored procedures one of the apps is using, and as they use the same input parameters, have one column per parameter. Obviously this isn't the best choice, but we wanted to get quick info to see if they were running batch processes and when they were running them. I've tried SQL Server Audit, but it doesn't catch the parameters unless you're executing a SP in a query.
SQL Server Profiler will do this for you; its included for free. Setup a trace and let it run.
You can also apply quite a bit of filtering to the trace, so you don't need to track everything; you can also direct the output to a file, or sql table for later analysis. This is probably your best bet for a time limited audit.
I think I've used the SQL Server Profiler (http://msdn.microsoft.com/en-us/library/ms181091.aspx) in the past to audit SQL execution. It's not something you would run all the time, but you can get a snapshot of what's running and how it's being executed.
I haven't tried using them, but you might look at event notifications and see if they will work for you.
From BOL
Event notifications can be used to do the following:
Log and review changes or activity occurring on the database.
I'm facing a new challenge here.
I can't seem to find precedence for replication from MySQL, running on a Linux box to MS SQL Server.
Has anybody done this before?
Most importantly all changes made to the MySQL database should be replicated on the MS database realtime or close. MS database are not likely to be updated in any other way, so a bidirectional facility is not required.
I thought one way is to read the changes out of the binary log.
Has anyone parsed one before?
Thanks for your help guys.
Triggers in MySQL could be used to catch changes and call a UDF, which could then execute ODBC queries to MSSQL. Likely terrible for performance, though.
If immediate replication isn't required:
Write triggers in MySQL that capture insert, update, and delete statements in a log table.
Poll the log table from MSSQL using ODBC and execute them, then delete those log entries.
Of course, T-SQL and MySQL's variant of SQL isn't exactly the same, but it should be close for trivial CUD operations.
Check to see if DBSync will help you do what you want
I had similar task, but I had to replicate from MSSQL 2008 to Mysql in real time.
I tried this application http://enterprise.replicator.daffodilsw.com/ and it worked but it didn't look reliable. But you can check I may be wrong.
Finally I decided to use interface OLE DB and postgress instead instead of Mysql. It works properly.
Is there any way to confirm that a particular breach of security was done through SQL injection?
There is no easy way here, but if you have the enabled the SQL server you use to log every single sql statement, here is what I would do.
Normally, when I SQL inject somewhere, i use one of these as my always true statement for passing throgh the Where clause, after ending the former string.
1=1
0=0
both being used as :
blahblahblah' or 1=1 --
You would not use this clauses in everyday code. So if you spot one of these in your history, well, it is a high candidate. Test the sql history to find :
(space)(number)(optional spaces)(equal)(optional spaces)(same number)(space)
Keep in mind that is heuristical, and will not always work, but could be the only way to give a hint after it had happened . Also, if you are in doubt about SQL injection, you should check the code for string concatenation and use of parameters.
after the attack has already happened? no. there isn't.
you'll have to check all your sql serevr access point for potential risk.
tere are some tools you can use. Check here under SQL Injection tools section.
SQL injection can happen any time you pass a query back to the database.
SQL Injection
Use mod_security to log POST requests and install an Intrusion Detection System to log/stop suspicious activity from now on. Logging every SQL request is an overhead if you are just looking for the breach points.
There are open source alternatives for IDS these days. I use PHPIDS for all my PHP applications.
Only one reliable way is probably analysing the SQL log files. Those should be done by a DBA who can spot things quickly as the size of logs would be huge.
It is better to prevent those.
There are some tools for that but the best one is the brain of the developer.
Stick with one simple rule - always use parameters when generating SQL query.
Just do the code review and if you find string cocatenations - that is first and highly possible place for SQL Injection.
You can log all http requests and check the requested pages for GET/POST sql injection tryouts.