I have this monster query written in T-SQL that pulls together and crunches data from multiple tables. I can export the result to CSV or Excel easily enough, but would like to send it right into SPSS. The ODBC drivers in SPSS only recognize Tables and Views in my SQL database. Any ideas how to get the results of my query into SPSS?
Options:
Export to Excel then import to SPSS... formatting things like dates become unwieldy
Save query as a table in my database... but then I would have to make a new table every time I run the query, yes?
As recommended below, simply run my SQL statement in the GET DATA statement of my SPSS syntax, but I am struggling with that...
UPDATE: In an attempt to use SPSS to run my SQL query I edited this code and get this error indicating that SPSS doesn't like my declaration of nvarchar (currently investigating how to handle this using alternative method). I have tested my connection between SPSS and SQL and the connection is good:
SQLExecDirect failed :[Microsoft][ODBC SQL Server Driver][SQL Server]Incorrect syntax near 'N'.
Here is my query simplified to pull just one field from one table:
GET DATA
/TYPE=ODBC
/CONNECT='DSN=temp_Hisp;Description=tempHisp;UID=;Trusted_Connection=Yes;APP=IBM SPSS '+
'Products: Statistics Common;WSID=ARCH5-50;DATABASE=temp_HispTreat'
/SQL='With CTE_BASENG As (Select StudyID, Visit, Question, CAST(Response As Int) As RESPONSE from temp_HispTreat.dbo.BAS AS PVTable outer apply (values (N'BAS1',BAS1), +'
'(N'BAS24',BAS24)) P(Question, Response)) select SubVis.IRB#, SubVis.StudyID, SubVis.Clin_num, Subvis.Visit, BASENG.BAS_ENGTOT From (Select Distinct IRB#, StudyID, +'
'Clin_Num, Visit_ID As Visit from temp_HispTreat.dbo.Subjects, temp_HispTreat.dbo.StudyStructure where subjects.IRB# = 5516 and StudyStructure.IRB = 5516) As SubVis left join (Select StudyID, +'
'Visit, SUM (Scoring.dbo.GetValue9(response)) As BAS_ENGTOT from CTE_BASENG group by StudyID, Visit) AS BASENG On SubVis.Studyid = BASENG.StudyID And SubVis.Visit = BASENG.Visit'
/ASSUMEDSTRWIDTH=255.
CACHE.
EXECUTE.
Thanks all: Solved. There is quite a bit of tweaking necessary to get SPSS to run SQL query, but this is the best way to export SQL data into SPSS. In my case (values (N'BAS1',BAS1) had to be changed to (values ("BAS1",BAS1) but all of my commands, e.g. outer apply, union, etc, ran like champs! Appreciate the help.
You can use GET DATA procedure to import data from SQL directly in SPSS. See the SQL subcommand. You can use your complicated query here. For example:
GET DATA
/TYPE = ODBC
/CONNECT = "DSN = DSNname"
/SQL = "SELECT * FROM empl_data "
"WHERE ((bdate>=#1/1/1960# and edate<=#12/31/1960#) or bdate is null)".
It is clear why (values (N'BAS1',BAS1) caused the error. Because you are using single quotes for the argument of the SQL subcommand \SQL = ' '. And the first single quote in (values (N'BAS1',BAS1) defines the end of the argument. Switching to double quotes solves it.
I tried to rearrange your code. I can not test it, but I believe it should work:
GET DATA
/TYPE = ODBC
/CONNECT = "DSN=temp_Hisp;DATABASE=temp_HispTreat"
/SQL = "With CTE_BASENG As (Select StudyID, Visit, Question, "
"CAST(Response As Int) As RESPONSE "
"from temp_HispTreat.dbo.BAS AS PVTable "
"outer apply (values (N'BAS1',BAS1), (N'BAS24',BAS24)) "
"P(Question, Response)) "
"select SubVis.IRB#, SubVis.StudyID, SubVis.Clin_num, Subvis.Visit, "
"BASENG.BAS_ENGTOT "
"From (Select Distinct IRB#, StudyID, Clin_Num, Visit_ID As Visit "
"from temp_HispTreat.dbo.Subjects, temp_HispTreat.dbo.StudyStructure "
"where subjects.IRB# = 5516 and StudyStructure.IRB = 5516) As SubVis "
"left join (Select StudyID, Visit, "
"SUM(Scoring.dbo.GetValue9(response)) As BAS_ENGTOT "
"from CTE_BASENG group by StudyID, Visit) AS BASENG On "
"SubVis.Studyid = BASENG.StudyID And SubVis.Visit = BASENG.Visit".
The SQL is processed by the ODBC driver, so the capabilities of that driver will determine what sort of SQL can be issued. The capabilities may be database specific. Someetimes there are multiple drivers available for a particular database, some from the IBM SPSS Data Access Pack and some from a db vendor directly, so you may want to investigate what is available for your particular database.
Related
I have several tables with the same data structure (they're filled with a bunch of stuff, in separate .accdb files to account for the 2GB limit) and need to retrieve info from one of them based on a field in a form.
Upon researching I came up with the following, but it won't seem to work.
SELECT MyNumber, MyName, MyPage, MyDrawing
FROM Switch([Forms]![View_Info]![Contract] = "Contract1", "tblContract1", [Forms]![View_Info]![Contract] = "Contract2", "tblContract2")
WHERE (MyNumber = [Forms]![View_Info]![MyNumber])
Syntax error in FROM clause.
In this example I only used 4 fields and 2 tables but in fact there are around 9 tables and 20 fields in each that I wish to retrieve.
Can someone shed some light on this? I have a really hard time with SQL, so I apologize if this is quite basic.
Thanks in advance, Rafael.
You cannot return the table name from a function in the SQL FROM clause. If your table is determined dynamically, then you must build the SQL command string dynamically.
Dim tableName As String, sql As String
tableName = Switch(...)
sql = "SELECT ... FROM [" & tableName & "] WHERE ..."
As #forpas explains in his answer, you can use a UNION query, but this will always query all the tables. Since the filter is not based on a table column, the filtering will occur on the client side, i.e. in your application.
Try this UNION:
SELECT MyNumber, MyName, MyPage, MyDrawing
FROM tblContract1
WHERE (MyNumber = [Forms]![View_Info]![MyNumber]) AND [Forms]![View_Info]![Contract] = "Contract1"
UNION
SELECT MyNumber, MyName, MyPage, MyDrawing
FROM tblContract2
WHERE (MyNumber = [Forms]![View_Info]![MyNumber]) AND [Forms]![View_Info]![Contract] = "Contract2"
Each query of the UNION contains in the WHERE clause the condition:
[Forms]![View_Info]![Contract] = "Contract?"
I'm trying to send a email with multiple data from an SQL statement but I can't get the result of the SQL statement to be saved in the variable.
str = "SELECT Test.*,TestTopicScore.[Easy Question Score],TestTopicScore.[Medium Question Score],TestTopicScore.[Hard Question Score] from Test INNER JOIN TestTopicScore on Test.TestID = TestTopicScore.TestID ORDER BY Test.Score DESC"
mycommand = New OleDbCommand(str, myconnection)
myReader = mycommand.ExecuteReader
myReader.Read()
emailcontents = myReader("emailcontents")
The connection with the access database is fine and the data is read but i just cant get it to save in a variable.
The error I get:
An unhandled exception of type 'System.IndexOutOfRangeException' occurred in System.Data.dll
Additional information: emailcontents
Let's look at your query first, I've formatted it a bit to make it easier to see what's going on:
SELECT Test.*
,TestTopicScore.[Easy Question Score]
,TestTopicScore.[Medium Question Score]
,TestTopicScore.[Hard Question Score]
FROM Test
INNER JOIN TestTopicScore
ON Test.TestID = TestTopicScore.TestID
ORDER BY Test.Score DESC
What that is doing is selecting several different fields, not one value. Your code emailcontents = myReader("emailcontents") attempts to read one field which is named "emailcontents". Unfortunately, there is no such named field.
It is worse than that: you don't even know how many fields Test.* is going to return.
So, start by enumerating every field which you want from Test.
Then, you appear to want just one string as the result, so you will need to concatenate the fields as strings instead of returning them separately.
In Access, the operator to join strings together is &, and it should convert numeric values into strings for you, thus:
SELECT Test.SomeField & Chr(13) & Chr(10)
& Test.SomeOtherField
& Test.AnotherField
& TestTopicScore.[Easy Question Score]
& TestTopicScore.[Medium Question Score]
& TestTopicScore.[Hard Question Score]
AS [emailcontents]
FROM Test
INNER JOIN TestTopicScore
ON Test.TestID = TestTopicScore.TestID
ORDER BY Test.Score DESC
Notice how I put & Chr(13) & Chr(10) between Test.SomeField and Test.SomeOtherField: that puts in a new line between the two. You might want to do it in between all the fields.
Also, I gave a name to the concatenated data with AS.
Now we're getting into really messy stuff by having to put the formatting of the output into the database. You don't want to do that. Let me emphasise this so it stands out: A better way is to read all the fields individually in your VB.NET code and format them into a string there.
ETA: Not only all that, but it looks like you might return more than one set of results (from seeing the ORDER BY), in which case you will have to read all the results in a loop.
I have this query:
Select
'<ALL>' as name,
'<ALL>' as pid,
'<ALL>' as type
union all
Select distinct
instructor.name as name,
instructor.Pid as pid,
instructor_type as type
From sisinfo.dbo.SISCRSI instructor
inner join section_info as section
on section.sctn_id_code = instructor.sctn_id_code
Where section.sctn_term_code in (#Terms)
and section.subj_code in (#Subject)
order by name
When I run it in SSMS, it completes in pretty close to zero time. It also runs fast in the SSRS query designer when I feed it some values. But when I preview the report, the query takes at least two minutes to run. I'm not sure what is going on here, I've never had this kind of a problem with SSRS before.
As discussed in the comments, let's get rid of the parameters to see if your query is getting affected by parameter sniffing.
To do this we build the SQL statement from scratch. Most things in SSRS can be expressions including the SQL query so you can build it as a string. With the parameters, we'll convert them to a comma delimited list using JOIN.
So for your SQL statement, go into the Dataset Properties dialogue (not the query editor) and press the fx expression editor button to edit the query expression as text and make it be an expression as per below:
="Select '<ALL>' as name, '<ALL>' as pid, '<ALL>' as type "
&"union all "
&"Select distinct instructor.name as name, instructor.Pid as pid, instructor_type as type "
&"From sisinfo.dbo.SISCRSI instructor "
&"inner join section_info as section on section.sctn_id_code = instructor.sctn_id_code "
&"Where section.sctn_term_code in (" & Join(Parameters!Terms.Value, ",") & ") "
&"and section.subj_code in (" & Join(Parameters!Subject.Value, ",") & ") "
&"order by name "
What we have done here is turn the SQL expression into a string where we supply the multivalue parameters as strings rather than parameters, thereby eliminating parameter sniffing as a cause of a slow running query.
If your query is slow after this, you know it isn't parameter sniffing that is the problem but at least you will have dismissed it as a cause.
There's a couple of options I tried when I came across the same thing.
First was to change the parameters for variables in the stored procedure, and then use those new variables in your query.
Declare #TermsNew DataType = #Terms
Declare #SubjectNew DataType = #Subject
Select '<ALL>' as name, '<ALL>' as pid, '<ALL>' as type
union all
Select distinct instructor.name as name, instructor.Pid as pid, instructor_type as type
From sisinfo.dbo.SISCRSI instructor
inner join section_info as section on section.sctn_id_code = instructor.sctn_id_code
Where section.sctn_term_code in (#TermsNew) and section.subj_code in (#SubjectNew)
order by name
Another option that has already been suggested here is to add
Option(Recompile);
at the end of your query to recompile the execution plan.
The thing that worked for me though having tried the above... If you're pulling big result sets make sure that in the tablix properties that it is not set to "Keep results on one page if possible". I turned that off and my report went from 70 seconds to 7.
Actually, it turns out that this query was not the one bogging down, it was another one that was running at the same time (I think SSRS can run multiple queries at once). I thought it was the posted one as it was populating a field. Thanks for all the suggestions, I will probably wind up using them in the future.
This sounds like parameter sniffing to me - I see this all the time when the SELECT is at all complex. I would add the OPTION (RECOMPILE) hint to the end of your SQL.
I am trying to update a field in a table using an SQL update query where there is a like statement referencing a value in another table. They syntax unfortunately is not working. Below is my code. In short, I am trying to put a '1' in the field 'Query07ParolaChiave' in the table 'tblSearchEngine01' when the value located in table 'tblsearchengine07' is present in the field 'tblMasterListOfEventsNotes' located in the table 'tblSearchEngine01'. I think my code is almost complete but there is a syntax issue which i cant find.
st_sql = "UPDATE tblSearchEngine01, tblSearchEngine07 SET tblSearchEngine01.Query07ParolaChiaveSelect = '1' WHERE ((([tblSearchEngine01].[tblMasterListOfEventsNotes]) Like " * " & [tblsearchengine07].[ParolaChiave] & " * "))"
Application.DoCmd.RunSQL (st_sql)
I suggest you 2 solutions :
This one is using EXISTS functions, and will check for each row in tblSearchEngine01 if there is a matching value in tblsearchengine07
UPDATE
tblSearchEngine01
SET
tblSearchEngine01.Query07ParolaChiaveSelect = '1'
WHERE
EXISTS (SELECT 1
FROM tblsearchengine07
WHERE [tblSearchEngine01].[tblMasterListOfEventsNotes] Like '*' & [tblsearchengine07].[ParolaChiave] & '*')
This one is more performant because it uses JOIN
UPDATE
tblSearchEngine01
INNER JOIN tblsearchengine07
ON [tblSearchEngine01].[tblMasterListOfEventsNotes] Like '*' & [tblsearchengine07].[ParolaChiave] & '*'
SET
tblSearchEngine01.Query07ParolaChiaveSelect = '1'
I read something like in ADO/VBA, you have to use % instead of * as the wildcard.
You can have more information on wildcard and LIKE comparator here
UPDATE
Why the '1' after select in your first solution?
EXISTS (SELECT 1 ... is better for performance because it return only the number 1 instead of fields, anyway EXISTS just stop the excecution after 1 element found.
'Performant' means more consuming in regards to space and memory?
JOIN is more performant in term of time of execution, RDBMS are far better at joining tables than using subquery, in some rare case, it's more interesting to use the 1st solution.
Also, any initial thoughts as to why my original solution (coming straight from an Access Query which works) does not function?
I cannot really know but perhaps it's because of " * ", because you are saying SPACE + * + SPACE + VALUE + SPACE + * + SPACE. For ex : 'John' LIKE ' John '
May be with "*" instead of " * " could solve it...
I have no other track, I'm not Access sql developper, I usually play around Sql server/Oracle/mySql, hope it helped. ;)
Try to change your like this way:
... Like '*" & tblsearchengine07.parolachiave & "*'))"
The like statement go into the WHERE clause.
If you do want to use LIKE without you care about caps letters, then you can use it like this:
LIKE COLUMN_NAME = '%WhatYouLike%'
My suggestion is:
Use a table variable (#Table) with a unique/primary key coming from the table to be updated.
SELECT all the data to be updated (you can add the like statement here) and then INSERT that in the created table variable.
Construct the UPDATE statement with an INNER JOIN to the table variable matching with the unique/primary key.
I know this may take a lot of steps but believe me these are more efficient than using a black list approach.
I've had a look around but cannot find the issue with this SQL Statement:
strSQL = "SELECT Directory.DisplayName, Department.DisplayName, Call.CallDate, Call.Extension, Call.Duration, Call.CallType, Call.SubType FROM (((Department INNER JOIN Directory ON Department.DepartmentID = Directory.DepartmentID) INNER JOIN Extension ON (Department.DepartmentID = Extension.DepartmentID) AND (Directory.ExtensionID = Extension.ExtensionID)) INNER JOIN Site ON Extension.SiteCode = Site.SiteCode) INNER JOIN Call ON Directory.DirectoryID = Call.DirectoryID WHERE (Call.CallDate)>=27/11/2012"
Regardless of what I change the WHERE it always returns every single value in the database (atleast I assume it does since excel completely hangs when I attempt this) this SQL statement works perfectly fine in Access (if dates have # # around them). Any idea how to fix this, currently trying to create a SQL statement that allows user input on different dates, but have to get over the this random hurdle first.
EDIT: The date field in the SQL Database is a DD/MM/YY HH:MM:SS format, and this query is done in VBA - EXCEL 2010.
Also to avoid confusion have removed TOP 10 from the statement, that was to stop excel from retrieving every single row in the database.
Current Reference I have activated is: MicrosoftX Data Objects 2.8 Library
Database is a MSSQL, using the connection string:
Provider=SQLOLEDB;Server=#######;Database=#######;User ID=########;Password=########;
WHERE (Call.CallDate) >= #27/11/2012#
Surround the date variable with #.
EDIT: Please make date string unambiguous, such as 27-Nov-2012
strSQL = "SELECT ........ WHERE myDate >= #" & Format(dateVar, "dd-mmm-yyyy") & "# "
If you are using ado, you should look at Paramaters instead of using dynamic query.
EDIT2: Thanks to #ElectricLlama for pointing out that it is SQL Server, not MS-Access
strSQL = "SELECT ........ WHERE myDate >= '" & Format(dateVar, "mm/dd/yyyy") & "' "
Please verify that the field Call.CallDate is of datatype DATETIME or DATE
If you are indeed running this against SQL Server, try this syntax for starters:
SELECT Directory.DisplayName, Department.DisplayName, Call.CallDate,
Call.Extension, Call.Duration, Call.CallType, Call.SubType
FROM (((Department INNER JOIN Directory
ON Department.DepartmentID = Directory.DepartmentID)
INNER JOIN Extension ON (Department.DepartmentID = Extension.DepartmentID)
AND (Directory.ExtensionID = Extension.ExtensionID))
INNER JOIN Site ON Extension.SiteCode = Site.SiteCode)
INNER JOIN Call ON Directory.DirectoryID = Call.DirectoryID
WHERE (Call.CallDate)>= '2012-11-27'
The date format you see is simply whatever format your client tool decides to show it in. Dates are not stored in any format, they are effectively stored as a duration since x.
By default SQL Uses the format YYYY-MM-DD if you want to use a date literal.
But you are much better off defining a parameter of type date in your code and keeping your date a data type 'date' for as long as possible. This may include only allowing them to enter the date using a calendar control to stop ambiguities.