I am trying to fasten up a SQL Server report regarding the IBM OS/400 operating system for my sales department.
A colleague of mine (which left the company) did this report and used a ton of sub selects.
The report usually takes about 30 min to process and often just fails to be displayed. I already tried to cut out some tables/rows in hopes of fastening up the process without success (all is needed by the sales department).
It works over all relevant data (orders, customers, articles, our order at the manufacturer, the manufacturer and so on). Any ideas?
I can't index it, due to the OS/400 system; guess it would be a new programming task for our contractor which leads to costs.
Can I use some clever joins? or somehow reduce the amount of subselects?
Are you using 4 part names in your query? That's probably your problem...
From SQL server...
-- Pull all rows from the table(s) back to MS SQL server and do the where locally on the MS SQL server
select * from LINKEDSVR.MYIBMI.MYLIB.MYTBL where locnbr = '00335';
-- Sends the statement to IBM i server for processing, only results are returned..
select * from openquery(LINKEDSVR, 'select * from MYTBL where locnbr = ''00335''');
Try running the subselects first, sending the output of each to its own table.
Update statistics on the tables. Then run the rest of the SQL, replacing what were originally subselects with the tables created in the first step.
Treat multiple layers of nesting the same way: each layer is its own insert into another table.
I've found that query optimizers have a hard time with complex SQL. Breaking-out the subqueries into separate steps often resolves this.
Between runs my preference is to leave the data intact as a reference in case debugging is needed, then truncate the tables as the first step of a run.
Responding to eraser's comments
Assuming your original query takes this general form:
select [columns] from
(-- subquery
select [columns] from TableA
) as Subquery
from TableB
where mainquery_where_clause
Re-write:
-- Create a table to handle results for your subquery:
Create Table A ;
-- Update the data distribution statistics:
update stats (TableA) ;
-- Now run the subquery:
insert into SubQTable select [columns] from TableA
-- Now run the re-written main query:
Select [columns]
from TableA, TableB
where TableA.joincol = TableB.joincol
and mainquery_where_clause ;
I noticed some syntax issues with the SQL you posted. Looks like something got left out. But the principle of my answer remains the same. Please note that applying my suggestion may not help, as there are potentially many variables to your scenario; you mentioned subqueries, so I chose to address that.
Halfer's suggestion is a great one: edit your original question, adding the SQL code, and putting it in the "{}" supplied by the text editing tool.
I strongly suggest that you obtain the SQL execution plan and post the results.
I investigated a problem when running a certain set of searches (from a .NET 3.5 application) against a Full Text Search DB on a SQL Server 2008 R2. Using profiler I extracted the long running query (120 seconds until Command Timeout was reached) and ran it in my SQL Server Management Studio. Duration was "0 Seconds" and depending on which one I tried 0 to 6 rows were returned.
The query looks like follows:
exec sp_executesql
N'SELECT TOP 1000 [DBNAME].[dbo].[FTSTABLE].[ID] AS [Id], [DBNAME].[dbo].[FTSTABLE].[Title], [DBNAME].[dbo].[FTSTABLE].[FirstName], [ABOUT 20 OTHERS]
FROM [DBNAME].[dbo].[FTSTABLE]
WHERE ( (
( Contains(([DBNAME].[dbo].[FTSTABLE].[Title], [DBNAME].[dbo].[FTSTABLE].[FirstName], [ABOUT 10 OTHERS]), #FieldsList1))
AND ( Contains(([DBNAME].[dbo].[FTSTABLE].[Title], [DBNAME].[dbo].[FTSTABLE].[FirstName], [ABOUT 10 OTHERS]), #FieldsList2))
AND ( Contains(([DBNAME].[dbo].[FTSTABLE].[Title], [DBNAME].[dbo].[FTSTABLE].[FirstName], [ABOUT 10 OTHERS]), #FieldsList3))
))'
,N'#FieldsList1 nvarchar(10),#FieldsList2 nvarchar(10),#FieldsList3 nvarchar(16)'
,#FieldsList1=N'"SomeString1*"'
,#FieldsList2=N'"SomeString2*"'
,#FieldsList3=N'"SomeString3*"'
The query looks a little weird as it is generated from an OR Mapper, but right now I don't want to optimize the query, as in SSMS it runs in less than one second, which shows it is not really the query making trouble.
I wrote a small testprogram:
SqlConnection conn = new SqlConnection("EXACTSAMECONNECTIONSTRING_USING_SAME_USER_ETC")
conn.Open();
SqlCommand command = conn.CreateCommand()
command.CommandText = "EXACTLY SAME STRING, LITERALLY, AS ABOVE IN SSMS- exec sp_executessql.....";
command.CommandTimeout = 120;
var reader = command.ExecuteReader();
while(reader.NextResult())
{
Console.WriteLine(reader[0]);
}
I got from my local PC also a SQLException after 120 seconds when command timeout was exeeded.
The SQL Server was at no moment under load heavier than a few single percent. There were no blocks at that table at any time during my tests.
I solved it after some time: I reduced the TOP 1000 to TOP 200 and suddenly the query from .NET code executed also in less than a second.
The questions I have:
Why in general is there such a huge difference between SSMS and simplest SQLCommand .NET code?
Why did reducing to TOP 200 have any effect, especially considering there were max 6 rows in the result.
This is tied to how query plans are built. When you run it in SSMS, you probably replace the variables manually, so it's not the same.
You can read a full explanation here : http://www.sommarskog.se/query-plan-mysteries.html
edit : maybe start with the paragraph "The Default Settings" and look at the results with manual enabling or disabling of ARITHABORT. This is the most common cause.
So the preliminary answer (not yet fully verified due to its complexity) can be derived from Keorl's answer, or mostly from the link provided therein.
To describe the different symptoms, I'll explain what happens:
The SQL Server cached the query against the fulltext indexed table, which includes the execution plan of the query. This means, if the first query to run (which puts the plan into the cache) is a very rare query with an absurd execution plan, this plan is cached and used for all subsequent queries, ruining performance for most runs.
One thing I could reproduce in the end: rerunning the FT indexer/gatherer solved the problem (this time). Also here the explanation is simple: an index update throws away precompiled/cached queries. Thus a better query than the previously cached one could run as the first and store a much better overall plan in the cache.
Answer to Q1: Why in general is there such a huge difference between SSMS and simplest SQLCommand .NET code?
So why didn't this happen with SSMS? Also this can be extracted from Keorl's answer: SSMS circumvents this in setting ARITHABORT option, which results in its own newly compiled query which is then cached. Thus the different observations for the same query just using SSMS and Code.
Answer to Q2: Why did reducing to TOP 200 have any effect, especially considering there were max 6 rows in the result?
For Dynamic SQL as used in example above, cache is stored depending on hashes of the complete query. As the query is different for TOP 200 and TOP 1000 two different compiles would cached. Parameters are not part of the hash though, so queries with just changing parameters would still result in same cache entry being used.
Concluding this: Thanks Keorl for providing the means to find an answer.
On some Microsoft Access queries, I get the following message: Operation must use an updatable query. (Error 3073). I work around it by using temporary tables, but I'm wondering if there's a better way. All the tables involved have a primary key. Here's the code:
UPDATE CLOG SET CLOG.NEXTDUE = (
SELECT H1.paidthru
FROM CTRHIST as H1
WHERE H1.ACCT = clog.ACCT AND
H1.SEQNO = (
SELECT MAX(SEQNO)
FROM CTRHIST
WHERE CTRHIST.ACCT = Clog.ACCT AND
CTRHIST.AMTPAID > 0 AND
CTRHIST.DATEPAID < CLOG.UPDATED_ON
)
)
WHERE CLOG.NEXTDUE IS NULL;
Since Jet 4, all queries that have a join to a SQL statement that summarizes data will be non-updatable. You aren't using a JOIN, but the WHERE clause is exactly equivalent to a join, and thus, the Jet query optimizer treats it the same way it treats a join.
I'm afraid you're out of luck without a temp table, though maybe somebody with greater Jet SQL knowledge than I can come up with a workaround.
BTW, it might have been updatable in Jet 3.5 (Access 97), as a whole lot of queries were updatable then that became non-updatable when upgraded to Jet 4.
--
I had a similar problem where the following queries wouldn't work;
update tbl_Lot_Valuation_Details as LVD
set LVD.LGAName = (select LGA.LGA_NAME from tbl_Prop_LGA as LGA where LGA.LGA_CODE = LVD.LGCode)
where LVD.LGAName is null;
update tbl_LOT_VALUATION_DETAILS inner join tbl_prop_LGA on tbl_LOT_VALUATION_DETAILS.LGCode = tbl_prop_LGA.LGA_CODE
set tbl_LOT_VALUATION_DETAILS.LGAName = [tbl_Prop_LGA].[LGA_NAME]
where tbl_LOT_VALUATION_DETAILS.LGAName is null;
However using DLookup resolved the problem;
update tbl_Lot_Valuation_Details as LVD
set LVD.LGAName = dlookup("LGA_NAME", "tbl_Prop_LGA", "LGA_CODE="+LVD.LGCode)
where LVD.LGAName is null;
This solution was originally proposed at https://stackoverflow.com/questions/537161/sql-update-woes-in-ms-access-operation-must-use-an-updateable-query
The problem defintely relates to the use of (in this case) the max() function. Any aggregation function used during a join (e.g. to retrieve the max or min or avg value from a joined table) will cause the error. And the same applies to using subqueries instead of joins (as in the original code).
This is incredibly annoying (and unjustified!) as it is a reasonably common thing to want to do. I've also had to use temp tables to get around it (pull the aggregated value into a temp table with an insert statement, then join to this table with your update, then drop the temp table).
Glenn
There is no error in the code. But the error is Thrown because of the following reason.
- Please check weather you have given Read-write permission to MS-Access database file.
- The Database file where it is stored (say in Folder1) is read-only..?
suppose you are stored the database (MS-Access file) in read only folder, while running your application the connection is not force-fully opened. Hence change the file permission / its containing folder permission like in C:\Program files all most all c drive files been set read-only so changing this permission solves this Problem.
I know my answer is 7 years late, but here's my suggestion anyway:
When Access complains about an UPDATE query that includes a JOIN, just save the query, with RecordsetType property set to Dynaset (Inconsistent Updates).
This will sometimes allow the UPDATE to work.
Thirteen years later I face the same issue. DISTINCTROW did not solve my problem, but dlookup did.
I need to update a table from an aggregate query. As far as I understand, MS Access always assumes that de junction between the to-update table and the aggregate query is one-to-many., even though unique records are assured in the query.
The use of dlookup is:
DLOOKUP(Field, SetOfRecords, Criteria)
Field: a string that identifies the field from which the data is retrieved.
SetOfRecords: a string that identifies the set o record from which the field is retrieved, being a table name or a (saved) query name (that doesn’t require parameters).
Criteria: A string used to restrict the range of data on which the DLookup function is performed, equivalent to the WHERE clause in an SQL expression, without the word WHERE.
Remark
If more than one field meets criteria, the DLookup function returns the first occurrence. You should specify criteria that will ensure that the field value returned by the DLookup function is unique.
The query that worked for me is:
UPDATE tblTarifaDeCorretagem
SET tblTarifaDeCorretagem.ValorCorretagem =
[tblTarifaDeCorretagem].[TarifaParteFixa]+
DLookUp(
"[ParteVariável]",
"cstParteVariavelPorOrdem",
"[IdTarifaDeCorretagem] = " & [tblTarifaDeCorretagem].[IdTarifaDeCorretagem]
);
I would try building the UPDATE query in Access. I had an UPDATE query I wrote myself like
UPDATE TABLE1
SET Field1 =
(SELECT Table2.Field2
FROM Table2
WHERE Table2.UniqueIDColumn = Table1.UniqueIDColumn)
The query gave me that error you're seeing. This worked on my SQL Server though, but just like earlier answers noted, Access UPDATE syntax isn't standard syntax. However, when I rebuilt it using Access's query wizard (it used the JOIN syntax) it worked fine. Normally I'd just make the UPDATE query a passthrough to use the non-JET syntax, but one of the tables I was joining with was a local Access table.
This occurs when there is not a UNIQUE MS-ACCESS key for the table(s) being updated. (Regardless of the SQL schema).
When creating MS-Access Links to SQL tables, you are asked to specify the index (key) at link time. If this is done incorrectly, or not at all, the query against the linked table is not updatable
When linking SQL tables into Access MAKE SURE that when Access prompts you for the index (key) you use exactly what SQL uses to avoid problem(s), although specifying any unique key is all Access needs to update the table.
If you were not the person who originally linked the table, delete the linked table from MS-ACCESS (the link only gets deleted) and re-link it specifying the key properly and all will work correctly.
(A little late to the party...)
The three ways I've gotten around this problem in the past are:
Reference a text box on an open form
DSum
DLookup
I had the same issue.
My solution is to first create a table from the non updatable query and then do the update from table to table and it works.
Mine failed with a simple INSERT statement. Fixed by starting the application with 'Run as Administrator' access.
MS Access - joining tables in an update query... how to make it updatable
Open the query in design view
Click once on the link b/w tables/view
In the “properties” window, change the value for “unique records” to “yes”
Save the query as an update query and run it.
You can always write the code in VBA that updates similarly. I had this problem too, and my workaround was making a select query, with all the joins, that had all the data I was looking for to be able to update, making that a recordset and running the update query repeatedly as an update query of only the updating table, only searching the criteria you're looking for
Dim updatingItems As Recordset
Dim clientName As String
Dim tableID As String
Set updatingItems = CurrentDb.OpenRecordset("*insert SELECT SQL here*");", dbOpenDynaset)
Do Until updatingItems .EOF
clientName = updatingItems .Fields("strName")
tableID = updatingItems .Fields("ID")
DoCmd.RunSQL "UPDATE *ONLY TABLE TO UPDATE* SET *TABLE*.strClientName= '" & clientName & "' WHERE (((*TABLE*.ID)=" & tableID & "))"
updatingItems.MoveNext
Loop
I'm only doing this to about 60 records a day, doing it to a few thousand could take much longer, as the query is running from start to finish multiple times, instead of just selecting an overall group and making changes. You might need ' ' around the quotes for tableID, as it's a string, but I'm pretty sure this is what worked for me.
I kept getting the same error until I made the connecting field a unique index in both connecting tables. Only then did the query become updatable.
Philip Stilianos
In essence, while your SQL looks perfectly reasonable, Jet has never supported the SQL standard syntax for UPDATE. Instead, it uses its own proprietary syntax (different again from SQL Server's proprietary UPDATE syntax) which is very limited. Often, the only workarounds "Operation must use an updatable query" are very painful. Seriously consider switching to a more capable SQL product.
For some more details about your specific problems and some possible workarounds, see Update Query Based on Totals Query Fails.
I kept getting the same error, but all SQLs execute in Access very well.
and when I amended the permission of AccessFile.
the problem fixed!!
I give 'Network Service' account full control permission, this account if for IIS
When I got this error, it may have been because of my UPDATE syntax being wrong, but after I fixed the update query I got the same error again...so I went to the ODBC Data Source Administrator and found that my connection was read-only. After I made the connection read-write and re-connected it worked just fine.
Today in my MS-Access 2003 with an ODBC tabla pointing to a SQL Server 2000 with sa password gave me the same error.
I defined a Primary Key on the table in the SQL Server database, and the issue was gone.
There is another scenario here that would apply. A file that was checked out of Visual Source Safe, for anyone still using it, that was not given "Writeablity", either in the View option or Check Out, will also recieve this error message.
Solution is to re-acquire the file from Source Safe and apply the Writeability setting.
To further answer what DRUA referred to in his/her answer...
I develop my databases in Access 2007. My users are using access 2007 runtime. They have read permissions to a database_Front (front end) folder, and read/write permissions to the database_Back folder.
In rolling out a new database, the user did not follow the full instructions of copying the front end to their computer, and instead created a shortcut. Running the Front-end through the shortcut will create a condition where the query is not updateable because of the file write restrictions.
Copying the front end to their documents folder solves the problem.
Yes, it complicates things when the users have to get an updated version of the front-end, but at least the query works without having to resort to temp tables and such.
check your DB (Database permission) and give full permission
Go to DB folder-> right click properties->security->edit-> give full control
& Start menu ->run->type "uac" make it down (if it high)
The answer given above by iDevlop worked for me. Note that I wasn't able to find the RecordsetType property in my update query. However, I was able to find that property by changing my query to a select query, setting that property as iDevlop noted and then changing my query to an update query. This worked, no need for a temp table.
I'd have liked for this to just be a comment to what iDevlop posted so that it flowed from his solution, but I don't have a high enough score.
I solved this by adding "DISTINCTROW"
so here this would be
UPDATE DISTINCTROW CLOG SET CLOG.NEXTDUE
My project is in Visual Foxpro and I use MS SQL server 2008. When I fire sql queries in batch, some of the queries don't execute. However, no error is thrown. I haven't used BEGIN TRAN and ROLLBACK yet. What should be done ??
that all depends... You don't have any sample of your queries posted to give us an indication of possible failure. However, one thing I've had good response with from VFP to SQL is to build into a string (I prefer using TEXT/ENDTEXT for readabilty), then send that entire value to SQL. If there are any "parameter" based values that are from VFP locally, you can use "?" to indicate it will come from a variable to SQL. Then you can batch all in a single vs multiple individual queries...
vfpField = 28
vfpString = 'Smith'
text to lcSqlCmd noshow
select
YT.blah,
YT.blah2
into
#tempSqlResult
from
yourTable YT
where
YT.SomeKey = ?vfpField
select
ost.Xblah,
t.blah,
t.blah2
from
OtherSQLTable ost
join #tempSqlResult t
on ost.Xblah = t.blahKey;
drop table #tempSqlResult;
endtext
nHandle = sqlconnect( "your connection string" )
nAns = sqlexec( nHandle, lcSqlCmd, "LocalVFPCursorName" )
No I don't have error trapping in here, just to show principle and readability. I know the sample query could have easily been done via a join, but if you are working with some pre-aggregations and want to put them into temp work areas like Localized VFP cursors from a query to be used as your next step, this would work via #tempSqlResult as "#" indicates temporary table on SQL for whatever the current connection handle is.
If you want to return MULTIPLE RESULT SETs from a single SQL call, you can do that too, just add another query that doesn't have an "into #tmpSQLblah" context. Then, all instances of those result cursors will be brought back down to VFP based on the "LocalVFPCursorName" prefix. If you are returning 3 result sets, then VFP will have 3 cursors open called
LocalVFPCursorName
LocalVFPCursorName1
LocalVFPCursorName2
and will be based on the sequence of the queries in the SqlExec() call. But if you can provide more on what you ARE trying to do and their samples, we can offer more specific help too.