Wondering if anyone may be able to help me with some SQL here. I'm tasked with retrieving some data from a legacy DB system - It's an IBM Informix DB running v7.23C1. It may well be that what I'm trying to do here is pretty simple, but for the life of me I can't figure it out.
I'm used to MS SQL Server, rather than any other DB system and this one seems quite old: http://publib.boulder.ibm.com/epubs/pdf/3731.pdf (?)
Basically, I just want to run a query that includes nesting, but I can't seem to figure out how to do this. So for example, I have a query that looks like this:
SELECT cmprod.cmp_product,
(stock.stk_stkqty - stock.stk_allstk) stk_bal,
stock.stk_ospurch,
stock.stk_backord,
'Current Sales Period',
'Current Period -1',
'Current Period -2',
cmprod.cmp_curcost,
stock.stk_credate,
stock.stk_lastpurch,
stock.stk_binref
FROM informix.stock stock,
informix.cmprod cmprod
WHERE stock.stk_product = cmprod.cmp_product
AND (cmp_category = 'VOLV'
OR cmp_category = 'VOLD'
OR cmp_category = 'VOLA')
AND stk_loc = 'ENG';
Now, basically where I have values like 'Current Period -1' I want to include a nested field which will run a query to get the sales within a given date range. I'm sure I can put those together separately, but can't seem to get the compiler to be happy with my code when executed altogether.
Probably something like (NB, this specific query is for another column, but you get the idea):
SELECT s.stmov_product, s.stmov_trandate, s.stmov_qty
FROM informix.stmove s
WHERE s.stmov_product = '1066823'
AND s.stmov_qty > 0
AND s.stmov_trandate IN (
SELECT MAX(r.stmov_trandate)
FROM informix.stmove r
WHERE r.stmov_product = '1066823'
AND r.stmov_qty > 0)
What makes things a little worse is I don't have access to the server that this DB is running on. At the moment I have a custom C# app that connects via an ODBC driver and executes the raw SQL, parsing the results back into a .CSV.
Any and all help appreciated!
Under all circumstances, Informix 7.23 is so geriatric that it is unkind to be still running it. It is not clear whether this is an OnLine (Informix Dynamic Server, IDS) or SE (Standard Engine) database. However, 7.23 was the version prior to the Y2K-certified 7.24 releases, so it is 15 years old or thereabouts, maybe a little older.
The syntaxes supported by Informix servers back in the days of 7.23 were less comprehensive than they are in current versions. Consequently, you'll need to be careful. You should have the manuals for the server — someone, somewhere in your company should. If not, you'll need to try finding it in the graveyard manuals section of the IBM Informix web pages (start at http://www.informix.com/ for simplicity of URL; however, archaic manuals take some finding, but you should be able to get there from http://pic.dhe.ibm.com/infocenter/ifxhelp/v0/index.jsp choosing 'Servers' in the LHS).
If you are trying to write:
SELECT ...
(SELECT ... ) AS 'Current - 1',
(SELECT ... ) AS 'Current - 2',
...
FROM ...
then you need to study the server SQL Syntax for 7.23 to know whether it is allowed. AFAICR, OnLine (Informix Dynamic Server) would allow it and SE probably would not, but that is far from definitive. I simply don't remember what the limitations were in that ancient a version.
Judging from the 7.2 Informix Guide to SQL: Syntax manual (dated April 1996 — 17 years old), you cannot put a (SELECT ...) in the select-list in this version of Informix.
You may have to create a temporary table holding the results you want (along with appropriate key information), and then select from the temporary table in the main query.
This sort of thing is one of the problems with not updating your server for so long.
Sorry to be blunt, but can you at least have mercy on us by shortening the syntax?.. table.columns can be presented AS aliases. Meanwhile, if you cant upgrade to a newer version of Informix, you will have to rely on one or more SELECT INTO temp table queries in order to achieve your objective, which BTW, would make your coding more portable across different versions. You also have to evaluate whether using TEMP tables implies unacceptable processing times.
Related
I have this report and need to add totals for each person (the red circle)
existing report
new report
I cannot change the existing report so I export data from MS SQL to MS Access and create a new report there. I got it working for one employee but have trouble with a query which would for multiple employees.
This query extract data use as input:
SELECT [TIME].[RCD_NUM], [TIME].[EMP_ID], [TIME].[PPERIOD], [TIME].[PRUN], [TIME].[TDATE], [TIME].[PC], [TIME].[RATE], [TIME].[HOURS], [TIME].[AMOUNT], [TIME].[JOB_ID], [TIME].[UPDATED], [TIME].[UPDATED_BY], [TIME].[LOG_DATE], [TIME].[ORIGINAL_REC_NUM]
FROM [TIME]
WHERE ((([TIME].[EMP_ID])=376) And (([TIME].[TDATE])<=#12/31/2006# And ([TIME].[TDATE])>=#1/1/2006#) And (([TIME].[PC])<599));
this query populates the report:
SELECT *
FROM TIME1
WHERE RCD_NUM = (SELECT Max(RCD_NUM) FROM [TIME1] UQ WHERE UQ.PPERIOD = [TIME1].PPERIOD AND UQ.PC = [TIME1].PC);
the problem is if I remove EMP_ID from the first query like this
SELECT [TIME].[RCD_NUM], [TIME].[EMP_ID], [TIME].[PPERIOD], [TIME].[PRUN], [TIME].[TDATE], [TIME].[PC], [TIME].[RATE], [TIME].[HOURS], [TIME].[AMOUNT], [TIME].[JOB_ID], [TIME].[UPDATED], [TIME].[UPDATED_BY], [TIME].[LOG_DATE], [TIME].[ORIGINAL_REC_NUM]
FROM [TIME]
WHERE ((([TIME].[TDATE])<=#12/31/2006# And ([TIME].[TDATE])>=#1/1/2006#) And (([TIME].[PC])<599));
then the second query doesn't work and ms access freezes when running this query.
any help/idea please?
Caveat: I won't pretend to know the precise cause of the problem, but I have had to repeatedly refactor queries in Access to get them working even though the original SQL statements are completely valid in regards to syntax and logic. Sometimes I've had to convolute a sequence of queries just to avoid bugs in Access. Access is often rather dumb and will simply (re)execute queries and subqueries exactly as given without optimization. At other times Access will attempt to combine queries by performing some internal optimizations, but sometimes those introduce frustrating bugs. Something as simple as a name change or column reordering can be the difference between a functioning query and one that crashes or freezes Access.
First consider:
Can you leave the data on SQL Server and link to the results in Access (rather than export/importing it into Access)? Even if you need or prefer to use Access for creating the actual report, you could use all the power of SQL Server for querying the data--it is likely less buggy and more efficient.
Common best practice is to create SQL Server stored procedures that return just what data you need in Access. A pass-through query is created in Access to retrieve the data, but all data operations are performed on the server.
Perhaps this is just a performance issue where limiting the set by [EMP_ID] selects a small subset, but the full table is large enough to "freeze" Access.
How long have you let Access remain frozen before killing the process? Be patient... like many, many minutes (or hours). Start it in the morning and check after lunch. :) It might eventually return a result set. This does not imply it is tolerable or that there is no other solution, but it can be useful to know if it eventually returns data or not.
How many possible records are there?
Are the imported data properly indexed? Add indexes to all key fields and those which are used in WHERE clauses.
Is the database located on a network share or is it local? Try copying the database to a local drive.
Other hints:
Try the BETWEEN operator for dates in the WHERE clause.
Try refactoring the "second" query by performing a join in the FROM clause rather than the WHERE clause. In doing this, you may also want to save the subquery as a named query (just as [TIME1] is saved). Whether or not a query is saved or embedded in another statement CAN change the behavior of Access (see caveat) even though the results should be identical.
Here's a version with the embedded aggregate query. Notice how all column references are qualified with the source. Some of the original query's columns do not have a source alias prefixing the column name. Remember the caveat... such picky details can affect Access behavior.:
SELECT TIME1.*
FROM TIME1 INNER JOIN
(SELECT UQ.PPERIOD, UQ.PC, Max(UQ.RCD_NUM) As Max_RCD_NUM
FROM [TIME1] UQ
GROUP BY UQ.PPERIOD, UQ.PC) As TIMEAGG
ON (TIME1.PPERIOD = TIMEAGG.PPERIOD) And (TIME1.PC = TIMEAGG.PC)
AND (TIME1.RCD_NUM = TIMEAGG.Max_RCD_NUM)
I've just come across some SQL syntax that I thought was invalid, but actually works fine (in SQL Server at least).
Given this table:
create table SomeTable (FirstColumn int, SecondColumn int)
The following insert statement executes with no error:
insert SomeTable(Any.Text.Here.FirstColumn, It.Does.Not.Matter.What.SecondColumn)
values (1,2);
The insert statement completes without error, and checking select * from SomeTable shows that it did indeed execute properly. See fiddle: http://sqlfiddle.com/#!6/18de0/2
SQL Server seems to just ignore anything except the last part of the column name given in the insert list.
Actual question:
Can this be relied upon as documented behaviour?
Any explanation about why this is so would also be appreciated.
It's unlikely to be part of the SQL standard, given its dubious utility (though I haven't checked specifically (a)).
What's most likely happening is that it's throwing away the non-final part of the column specification because it's superfluous. You have explicitly stated what table you're inserting into, with the insert into SomeTable part of the command, and that's the table that will be used.
What you appear to have done here is to find a way to execute SQL commands that are less readable but have no real advantage. In that vein, it appears similar to the C code:
int nine = 9;
int eight = 8;
xyzzy = xyzzy + nine - eight;
which could perhaps be better written as xyzzy++; :-)
I wouldn't rely on it at all, possibly because it's not standard but mostly because it makes maintenance harder rather than easier, and because I know DBAs all over the world would track me down and beat me to death with IBM DB2 manuals, their choice of weapon due to the voluminous size and skull-crushing abilities :-)
(a) I have checked non-specifically, at least for ISO 9075-2:2003 which dictates the SQL03 language.
Section 14.8 of that standard covers the insert statement and it appears that the following clause may be relevant:
Each column-name in the insert-column-list shall identify an updatable column of T.
Without spending a huge amount of time (that document is 1,332 pages long and would take several days to digest properly), I suspect you could argue that the column could be identified just using the final part of the column name (by removing all the owner/user/schema specifications from it).
Especially since it appears only one target table is possible (updatable views crossing table boundaries notwithstanding):
<insertion target> ::= <table name>
Fair warning: I haven't checked later iterations of the standard so things may have changed. But I'd consider that unlikely since there seems to be no real use case for having this feature.
This was reported as a bug on Connect and despite initially encouraging comments the current status of the item is closed as "won't fix".
The Order by clause used to behave in a similar fashion but that one was fixed in SQL Server 2005.
I get errors when I try to run the script on SQL Server 2012 as well as SQL Server 2014 and SQL Server 2008 R2. So you can certainly not rely on the behavior you see with sqlfiddle.
Even if this were to work, I would never rely on undocumented behavior in production code. Microsoft will include notice of breaking changes with documented features but not undocumented ones. So if this were an actual T-SQL parsing bug that was later fixed, it would break the mal-formed code.
I have joined a new job where I am required to use FileMaker (and gradually transition systems to other databases). I have been a DB Admin of a MS SQL Server database for ~2 years, and I am very well versed in PL/SQL and T-SQL. I am trying to pan my SQL knowledge to FMP using the ExecuteSQL functionaloty, and I'm kinda running into a lot of small pains :)
I have 2 tables: Movies and Genres. The relevant columns are:
Movies(MovieId, MovieName, GenreId, Rating)
Genres(GenreId, GenreName)
I'm trying to find the movie with the highest rating in each genre. The SQL query for this would be:
SELECT M.MovieName
FROM Movies M INNER JOIN Genres G ON M.GenreId=G.GenreId
WHERE M.Rating=
(
SELECT MAX(Rating) FROM Movies WHERE GenreId = M.GenreId
)
I translated this as best as I could to an ExecuteSQL query:
ExecuteSQL ("
SELECT M::MovieName FROM Movies M INNER JOIN Genres G ON M::GenreId=G::GenreId
WHERE M::Rating =
(SELECT MAX(M2::Rating) FROM Movies M2 WHERE M2::GenreId = M::GenreId)
"; "" ; "")
I set the field type to Text and also ensured values are not stored. But all I see are '?' marks.
What am I doing incorrectly here? I'm sorry if it's something really stupid, but I'm new to FMP and any suggestions would be appreciated.
Thank you!
--
Ram
UPDATE: Solution and the thought process it took to get there:
Thanks to everyone that helped me solve the problem. You guys made me realize that traditional SQL thought process does not exactly pan to FMP, and when I probed around, what I realized is that to best use SQL knowledge in FMP, I should be considering each column independently and not think of the entire result set when I write a query. This would mean that for my current functionality, the JOIN is no longer necessary. The JOIN was to bring in the GenreName, which is a different column that FMP automatically maps. I just needed to remove the JOIN, and it works perfectly.
TL;DR: The thought process context should be the current column, not the entire expected result set.
Once again, thank you #MissJack, #Chuck (how did you even get that username?), #pft221 and #michael.hor257k
I've found that FileMaker is very particular in its formatting of queries using the ExecuteSQL function. In many cases, standard SQL syntax will work fine, but in some cases you have to make some slight (but important) tweaks.
I can see two things here that might be causing the problem...
ExecuteSQL ("
SELECT M::MovieName FROM Movies M INNER JOIN Genres G ON
M::GenreId=G::GenreId
WHERE M::Rating =
(SELECT MAX(M2::Rating) FROM Movies M2 WHERE M2::GenreId = M::GenreId)
"; "" ; "")
You can't use the standard FMP table::field format inside the query.
Within the quotes inside the ExecuteSQL function, you should follow the SQL format of table.column. So M::MovieName should be M.MovieName.
I don't see an AS anywhere in your code.
In order to create an alias, you must state it explicitly. For example, in your FROM, it should be Movies AS M.
I think if you fix those two things, it should probably work. However, I've had some trouble with JOINs myself, as my primary experience is with FMP, and I'm only just now becoming more familiar with SQL syntax.
Because it's incredibly hard to debug SQL in FMP, the best advice I can give you here is to start small. Begin with a very basic query, and once you're sure that's working, gradually add more complicated elements one at a time until you encounter the dreaded ?.
There's a number of great posts on FileMaker Hacks all about ExecuteSQL:
Since you're already familiar with SQL, I'd start with this one: The Missing FM 12 ExecuteSQL Reference. There's a link to a PDF of the entire article if you scroll down to the bottom of the post.
I was going to recommend a few more specific articles (like the series on Robust Coding, or Dynamic Parameters), but since I'm new here and I can't include more than 2 links, just go to FileMaker Hacks and search for "ExecuteSQL". You'll find a number of useful posts.
NB If you're using FMP Advanced, the Data Viewer is a great tool for testing SQL. But beware: complex queries on large databases can sometimes send it into fits and freeze the program.
The first thing to keep in mind when working with FileMaker and ExecuteSQL() is the difference between tables and table occurrences. This is a concept that's somewhat unique to FileMaker. Succinctly, tables store the data, but table occurrences define the context of that data. Table occurrences are what you're seeing in FileMaker's relationship graph, and the ExecuteSQL() function needs to reference the table occurrences in its query.
I agree with MissJack regarding the need to start small in building the SQL statement and use the Data Viewer in FileMaker Pro Advanced, but there's one more recommendation I can offer, which is to use SeedCode's SQL Explorer. It does require the adding of table occurrences and fields to duplicate the naming in your existing solution, but this is pretty easy to do and the file they offer includes a wizard for building the SQL query.
I have this query:
SELECT
COUNT(*) AS 'RedactedCount'
,s.Redacted1
,s.[Redacted2]
,s.[Redacted3] AS 'Redacted3'
FROM RedactedTable1 s
LEFT OUTER JOIN RedactedTable2 g ON s.Redacted5= g.Redacted5
WHERE g.Redacted6= 31013 AND s.DateTime >= '2013-03-02 00:00:00'
GROUP BY s.Redacted1,s.Redacted2, s.Redacted3
Which have a very odd behavior. This query takes a whole 1min30secs to complete. Should I change de date to 2013-04-02 00:00:00 (today as I'm writing this post), it is near instant, which is the expected behavior.
But if I change the date to 2013-02-02 (a 2 months time span instead of 1), the query takes only 20 secs.
Does Anyone has encountered this problem ? I am completely stunned on the result. It will also be an important SQL request of a web application that I am working on.
Microsoft SQL Server Management Studio 11.0.3128.0
Microsoft Data Access Components (MDAC) 6.1.7601.17514
Microsoft MSXML 3.0 6.0
Microsoft Internet Explorer 9.0.8112.16421
Microsoft .NET Framework 4.0.30319.296
Système d'exploitation 6.1.7601
Note: The database is poorly designed, and contains absolutely no indexes. Yes, this is bad. Unfortunately, this is a commercial software and I have no rights to make changes on the database model. However, I do not think that the problem I have is caused by this.
P.S.: Sorry if my query is heavily redacted as I am on a strict NDA. I tried to made it as readable as possible.
Thanks !
First of all, it is pointless to put a predicate condition on a table on the outer side of an outer join. As soon as you do this, all rows in the final result set that do not have a match in that table are eliminated, effectively making the overall query behave as though it was an inner join.
The condition on RedactedTable2.Redacted6 should be part of the join conditions if you want the join to include rows where there is no matching row in table RedactedTable2.
SELECT COUNT(*) AS 'RedactedCount',
s.Redacted1, s.[Redacted2],
s.[Redacted3] AS 'Redacted3'
FROM RedactedTable1 s
LEFT JOIN RedactedTable2 g
ON g.Redacted5= s.Redacted5
And g.Redacted6= 31013
WHERE s.DateTime >= '2013-03-02 00:00:00'
GROUP BY s.Redacted1,s.Redacted2, s.Redacted3
As to why the difference in performance, my suspicion is that your issue is caused by something in the data in the tables that is causing the query processor to use a different execution plan in one case than it was in the other. This can easily happen. If the optimizer "guesses" that it would need to examine more than a certain percentage of data rows using one query execution plan, based on database statistics about the distribution of data values in the tables), then it will switch to a different plan.
Run both queries with the ShowPlan option turned on, and look and see what the differences are.
In addition to what Charles suggested you might ask the database administrator (assuming you have one) to run UPDATE STATISTICS on at least RedactedTable1 and RedactedTable2. UPDATE STATISTICS requires you to have ALTER permissions on the table/view so I doubt you have permissions to run it. But you can probably ask for it to be done. Problems like what you are describing are frequently caused by out of date statistics.
Aaron Bertrand got the answer in a comment.
This problem was caused by parameter sniffing done by MSSQL.
Declaring and using dummy variables prevents MSSQL to falsely optimize the query using past optimizations.
The following link helped me learn about parameter sniffing http://blogs.technet.com/b/mdegre/archive/2012/03/19/what-is-parameter-sniffing.aspx
Situation:
I'm doing some penetration testing for a friend of mine and have total clearance to go postal on a demo environment. Reason for this is because I saw a XSS-hole in his online ASP-application (error page with error as param allowing html).
He has a Access DB and because of his lack of input-validation I came upon another hole: he allows sql injection in a where-clause.
I tried some stuff from:
http://www.krazl.com/blog/?p=3
But this gave limited result:
MSysRelationships is open, but his Objects table is shielded.
' UNION SELECT 1,1,1,1,1,1,1,1,1,1 FROM MSysRelationships WHERE '1' = '1 <-- worked so I know the parent table has at least 9 columns. I don't know how I can exploit the relation table to get tablenames ( I can't find any structures explanation so I don't know on what to select.
Tried brute-forceing some tablenames, but to no avail.
I do not want to trash his DB, but I do want to point out the serious flaw with some backing.
Anyone has Ideas?
Usually there are two ways to proceed from here. You could try to guess table names by the type of data which is stored in them which often works ("users" usually stores the user data ...). The other method would be to generate speaking error messages in the application to see if you can fetch table or column names from there.