Good morning,
I just received a new assignment and I am struggling with finding an appropriate solution. I have searched through the SO Forums, and through Google, but have not found a workable solution. Below is my scenario:
We are working out of Microsoft Access to connect to an SQL Server Database via an ODBC Connection.
I wasgiven an incredibly large pass-through SQL query, larger than is able to be processed in MS Access. In this pass-through query, there is a subquery in a WITH...AS method.
I am hoping to be able to split this one, singularly large, SQL pass through query into two: Query One (the subquery), and Query Two (which references the results of the subquery)
I know that by using general Access queries, I can write a Macro like follows...
Sub myQuery()
' Edited from http://www.dbforums.com/showthread.php?1667831-Run-multiple-queries-in-sequence-on-click
' On Error GoTo ErrHandler
' Run the first query
MsgBox "Starting first query"
DoCmd.OpenQuery "first_Query"
DoEvents
' Run the second query
MsgBox "Done. Now starting second query"
DoCmd.OpenQuery "second_Query"
DoEvents
MsgBox "Done!"
End Sub
However, these need to be pass-through queries. I believe that the enormously large SQL String is created via a number of user inputs. Regardless, I don't have the ability to change the pass-through SQL that I was given.
Is there anyway I can write a macro that calls the first pass-through query, and then calls the second pass-through query that REFERENCES the result of the first?
Here is an example with what I am working with...
WITH queryOne AS
(
SELECT fooID
FROM tblFoo
WHERE foodate > ...
)
SELECT foo, fooone, footwo, foothree
FROM tblOtherFoo
WHERE fooID = OtherFooID
However, the query is 50000+ characters, exceeding that ~37k limit.
Please feel free to ask any questions. I am stumped by this and would appreciate any feedback or alternative resources.
Thank you!
It not clear what you mean by something that references the first or previous? Why break up something you been given that supposed works just fine?
So just place that existing t-sql you been given into a stored procedure. In T-SQL you can easy have some SQL operate on some previous SQL, but why break up such a HUGE massive monster slew of code and introduce bugs? It will take you YEARS AND YEARS to break up a KNOWN working huge T-SQL that been built and developed for you (something that long likely took a few years and a team of developers to create).
A conservative estimate would be such a routine cost $50,000, or even $100,000 to develop.
No question that the working T-SQL you been given might reference previous data, or even do selects into #Temp tables that additional T-SQL can work on.
If you ALREADY have a working PT code and query given to you?
Simply take that T-SQL query, and simply paste it into a stored procedure. You will do this in SQL Server and NOT even touch or bother with Access.
So don't create some macro in Access that calls multiple separate queries, but place all of the T-SQL in a stored procedure, and simply call that huge mess one time from Access.
It possible that the T-SQL you been given is incorrect, but assuming that the T-SQL is correct, then simply place all that long mess into a working stored procedure. You do not place this SQL in MS Access and you don’t need to have that mess inside of Access.
So get that T-SQL working in SQL Server – don’t bother with MS Access until this long query mess is working in SQL Server. ONLY THEN do you fire up Access.
So you THEN create a simple PT query in Access that calls the huge long T-SQL mess you been given. But that “mess” is to be placed in SQL Server – not in Access.
So create a PT query in Access that calls your “supposed” working T-SQL you been given. The SQL you save in the Access PT query will be this
Exec my_StupidLongSQLProc
Save the above as a PT query. Then in VBA code go:
Currentdb.QueryDefs("MyPTQuery").Execute
If you need to pass some values from Access, then go:
With CurrentDb.QueryDefs("MyPTQuery")
.sql = "exec My_StupidLongSQLproc " & p1 & "," p2
.Execute
End with
In above we pass two VBA values from Access to the big mess of sql you have – the stored procedure in above is just an example that access two parameters passed from Access VBA. If the T-SQL you been given does not require values from Access, then the first single .execute will do the job.
And if you REALLY did get such a long routine that is correct T-SQL, then it likely already has parameters in the working T-SQL (and again you don’t want to mess with or change such a huge long working T-SQL that you been given).
So you only need one line of code in Access, and your existing long T-SQL you been given if written correctly can be placed in a stored procedure (assuming that you actually been given a correctly working PT query).
So if you REALLY did get a huge massive working T-SQL statement, then simply place that KNOWN AND WORKING T-SQL in SQL Server as a stored procedure and call it with one line of code as per above.
So trying to split this up from Access will only server to cause world-wide poverty and ANY tiny miss step or breaking up of that huge long routine will cause world-wide poverty and starving children as you try and “fix” this great working T-SQL that you been given. As noted, something that long to create would take a teams of developers HUGE resources. If you touch or break up one line of code and mess it up, then you need that team of developers to spend several months trying to fix what you broke.
So the INSTANT you start breaking up such a huge long mess is the instant you lost this battle and will waste several years of your life trying to fix this crazy long T-SQL that you been given that is already claimed to be proper working code.
Related
I have T-SQL queries stored on a hard drive: I:\queries\query1.sql and I:\queries\query2.sql.
I usually work in a way that I execute a query from a drive, and then I copy results into Excel, and then I work on it.
My problem here is that query1.sql is already long, and now I would like to extend it by getting a result of query2.sql, and join it with a result of query1.sql.
What I could do is appending a code from query2.sql to query1.sql. But then the query is getting really long and hard to maintain.
I would like to do something like this:
SELECT * FROM ("Result of I:\queries\query1.sql") q1
LEFT JOIN ("Result of I:\queries\query2.sql") q2 ON q1.ID=q2.ID
Is there any way to write a query or stored procedure, which will be again stored on a drive to do this?
Basically, you need to ask your DBA for a database when you are able to store things in the database. This can be on the same system where the data is stored. Or, it could be on a linked system. Gosh, you could run SQL Server locally and store the information and data there.
Then, the queries that you are storing in files should be views in the database. You can then run the queries and store and combine the results locally.
You are essentially recreating database functionality using text files and data files -- going through a lot of effort when SQL Server already supports this functionality.
To expand on Gordon's comment (+1), why are you running scripts off of a drive? Most DBA's I've known would treaten bodily harm over this as executing code that they can't control / troubleshoot / see source code control on brings a whole host of security and supportability issues.
Far better to store this code in a Stored Procedure, which will have a saved query execution plan, can be tracked using various DMV's, and have permissions assigned to it, then your outside Excel doc can just set a connection and execute the SP.
I'm not sure if a stored procedure is the correct path here so first, I'll explain what I need to accomplish.
I need to query one table with a variable as such:
SELECT *
FROM db.partList
WHERE column1 = 'order_no';
Then, I need to query another table as such:
SELECT [serial_no]
FROM db.resultsList;
Finally I need to iterate through the results of the above, and return the first [serial_no] from db.partList that is not in the list produced.
The original programmer was doing this in a way that was blowing up the customer's network unnecessarily. There shouldn't be any reason this can't be done locally. Now I'm here to clean it up.
Thanks in advance.
So my questions are, would this be correct use of a stored procedure? If so, could someone perhaps give me some sample code to start working with? I don't often have to dive that deep into SQL Server.
This is SQL Server 2012.
I've got some example code of how I would do it in other languages if needed. I'm just not familiar enough with stored procedures to do this quickly.
I have an excel file that will select roughly 1100 rows with 5 columns of data. Most columns are 5 digits long and are integers. I am using a macro to connect to a SQL server database and insert these rows into one maybe two tables. This is all its doing and then it closes the connection. So the user opens an excel file that has the rows, clicks a button and it executes the macro.
My question is, should the query be written in Excel since its simple and merely inserts the data into a few tables. Or is it more efficient calling a stored procedure and passing all of the values in the stored procedure and have it allocate where the values go in the different tables. When I mean efficient, i mean which is the quickest? I know this will probably take a few seconds to complete. I just feel going to a stored procedure is an extra point along the path that the data has to get to before it reaches the tables. Am I wrong? Any thoughts?
There are some advantages to using stored procedures in SQL Server. One is that SQL Server precompiles and saves the query execution plan, which increases performance. With your current method, SQL Server will generally need to generate the execution plan each time. Stored procedures can also reduce client/server network traffic.
So, even though it may seem like an extra point along the path, it actually can be faster.
In addition to #mark d.'s answer, another reason for using a stored procedure is security.
Your comment says that a customer is entering the data into Excel, so if you are putting direct SQL into your spreadsheet, then there is a risk that someone will open your spreadsheet and find out information about your database. But if you use a stored procedure then there is far less that can be learned.
Either way, make sure that you aren't hardcoding any connection string/account credentials into the spreadsheet.
One of my managers created a Access database and is working on some data analysis - what if scenarios. Based on different conditions, he produces a report in Access.
He asked me to do some data manipulation, so I imported the database into SQL and wrote a routine with a cursor that'll do what he wants. I then export the results back into Access. Before I get any heat for using a cursor, this was supposed to be a one time only deal, so that was the fastest way for me to get it done.
As you'd expect, now he wants me to run it all the time and asked me to convert my routine to Access so he can just run it. Before you tell me to just use SQL, he's very set on Access and is often traveling and off line.
So, my question is: is there a "easy" way to convert a T-SQL query with a cursor into Access? It's been a long time since I worked with Access, but I suspect it'd have to be re-written in VBA. I'm thinking that maybe another solution would be to call the query from Access and run it in SQL, but I don't know if that can be done or if it'd work on my case because of him being off line (maybe install SQL express in his laptop?)
Any suggestions would be appreciated.
Thanks,
Alex
This is how I got around it:
1.Downloaded and install SQL server express in the user's machine.
2.uploaded Access database structure and data to the local SQL.
3.created the stored procedure that I wanted to run in the local SQL server.
4.back in Access, deleted all the tables and recreated them as linked tables to SQL
5.Create a form in Access with a big button that executes the stored procedure
`Private Sub Command0_Click()
Dim qdf As DAO.QueryDef
Set qdf = CurrentDb.CreateQueryDef("")
qdf.Connect = CurrentDb.TableDefs("ANY TABLE").Connect
qdf.SQL = "EXEC dbo.[stored procedure name]"
qdf.ReturnsRecords = False
qdf.Execute
Set qdf = Nothing`
The stored procedure truncates one re-populates one of the tables. So after executing it I can open up the table in Access and see the changes. My manager can continue to use Access and SQL server is used in the back-end. Happy Ending! :)
I've an SSIS package that runs a stored proc for exporting to an excel file. Everything worked like a champ until I needed to a do a bit of rewriting on the stored proc. The proc now takes about 1 minute to run and the exported columns are different, so my problems are the following;
1) SSIS complains when I hit the preview button "No column information returned by command"
2) It times out after about 30 seconds.
What I've done.
Tried to clean up/optimize the query. That helped a bit, but it still is doing some major calculations and it runs just fine in SSMS.
Changed the timeout values to 90 seconds. Didn't seem to help. Maybe someone here can?
Thanks,
Found this little tidbit which helped immensely.
No Column Names
Basically all you need to do is add the following to your SQL query text in SSIS.
SET FMTONLY OFF
SET NOCOUNT ON
Only problem now is it runs slow as molasses :-(
EDIT: It's running just too damn slow.
Changed from using #tempTable to tempTable. Adding in appropriate drop statements. argh...
Although it appears you may have answered part of your own question, you are probably getting the "No column information returned by command" error because the table doesn't exist at the time it tries to validate the metadata. Creating the tables as non-temporary tables resolves this issue.
If you insist on using temporary tables, you can create the temporary tables in the step preceeding the data flow. You would need to create it as a ## table and turn off connection sharing for the connection for this to work, but it is an alternative to creating permanent tables.
A shot in the dark based on something obscure I hit years ago: When you modified the procedure, did you add a call to a second procedure? This might mess up SSIS's ability to determine the returned data set.
As for (2), does the procedure take 30+ or 90+ seconds to run in SSMS? If not, do you know that the query is actually getting into SQL from SSIS? Might be worth firing up SQL Profiler to see what's actually being sent to SQL Server. [Which was the way I found out my obscure factoid.]