A similar question was asked.
The goal is to create the function described here:
def DB_Query(d1, d2):
conn = pyodbc.connect('DSN=DATASOURCE')
tbl = "SELECT TableA.Field_1 \
FROM TableA \
WHERE TableA.Date>=? AND TableA.Date<=?"
df_tbl= pd.read_sql_query(tbl, conn, params = (d1,d2))
conn.close
return df_tbl
This worked on database with SQL Server driver, but it won't work on Microsoft ODBC for Oracle driver.
When I give, for e.g., d1 = '2020-02-20' and d2 = '2020-02-25', I get error
('HY004', '[HY004] [Microsoft][ODBC Driver Manager] SQL data type out of range (0) (SQLBindParameter)')
I understand in Oracle, you need DATE 'YYYY-MM-DD' to express a date, which is different from SQL server where you can just use 'YYYY-MM-DD'.
I've tried add DATE in front of ? but doesn't work. Any ideas?
Found solution here. Just define d1 as date(2020,02,20).
How to parameterize datestamp in pyODBC query?
Related
The VB for application version is 7.1; Teradata version 16.20.53.27.
The SQL statement is very long, the length of it is 59972.I run it into Teradata directly, it works well.
I use the following to execute this SQL in VBA:
set connection = CreateOjbect("ADODB.connection")
connection.open "DSN=xxx UID=XXX PWD=XXX"
Set ObjRecordSet=CreateObject("ADO.RecordSet")
ObjRecordSet.open StrQuery connection
ObjRecordSet.close
connection.close
the error message is
Run-time error '-2147217833 (80040e57)
[Teradata] [ODBC Teradata Driver]Teradata Database
A character string failed conversion to a numeric value
I check the long SQL statement, could not find the convert function for numeric value, only converting string into varchar(80). Thanks!
After separating the long SQL query, I find the root cause:
select *
from table1
where column1 = '29991231'
while data type of column1 is date. It can be executed in Teradata SQL assist while it fails in VBA.
I change it to be:
select *
from table1
where column1 = to_date('31-12-2999', 'DD-MM-YYYY')
it has no error when VBA call this SQL query.
Anyway, thanks for your attention!
I am running a sql script within R. There is a date filter within the script and right now it is hardcoded in. Using the extract I run some analysis, etc. My final goal is to turn this script into Shiny.
I want to be able to make the date filter a prompt using readlines. Does anyone know if I can stick the date in the SQL script using that readline output?
For example:
Readline asks Start date?
Input as 2020-10-01 and gets set as X
The sql code reads:
SELECT * from database
WHERE DATE= 'X'
Thank you!
Using Parameterised Query
As pointed out in the comments, using paste is unsafe as it leaves the system vulnerable to exploits such as sql injection attacks. This can be mitigated by using parametrised queries, this is a helpful document for reference.
# Get input
r <- readline("Input Date: ")
# Using parameterised Query
sql <- "SELECT * FROM database WHERE DATE = ?"
# Send the paramterised query
query <- DBI::dbSendQuery(conn_string, sql)
# Bind the parameter
DBI::dbBind(query, list(r))
# Fetch Result
DBI::dbFetch(query)
You can also use the sqlInterpolate if having trouble with parameterised query,
query <-
sqlInterpolate(conn_string,
"SELECT * FROM database WHERE DATE = ?date",
date = r
)
# Get Results
dbGetQuery(conn_string, query)
Unsafe way
You can use paste0() to process the date string using single quotes correctly.
# Get input
r <- readline("Input Date: ")
# Paste input with query string noting the use of single quotes '
query <- paste0("SELECT * FROM database WHERE DATE = '", r, "'")
# Make query
dbGetQuery(conn, query)
Trying to solve issue with wrong display of national characters (Polish) in results of query to MS SQL database.
The script is pretty standard
First the definnition connection object
library(DBI)
db.conn <- DBI::dbConnect(odbc::odbc(),
Driver = "SQL Server Native Client 11.0",
Server = "10.0.0.100",
Port = 1433,
Database = "DB",
UID = "user",
PWD = rstudioapi::askForPassword("Database password"),
encoding = "latin1"
)
then SQL statement
db_sql = "
select
*
from test
where active = 'ACTIVE'
order by name_id"
Then execution of SQL
db_query <- dbSendQuery(db.conn, db_sql)
db_data <- dbFetch(db_query)
or
db_data <- dbGetQuery(db.conn, db_sql)
It does not matter whether in connection object definition I use "latin1", "windows-1250" or "utf-8" parameter for encoding parameter the results are always the same
Strings with U+009C or similar
It also does not matter what codepage I select in RStudio Global options.
Problem solved.
First it is necessary to change locale to Polish
Sys.setlocale(category = "LC_ALL", locale = "Polish")
Then set proper encoding in
DBI::dbConnect(odbc::odbc()
...
encoding = "windows-1250"
and voilla, working.
I have this monster query written in T-SQL that pulls together and crunches data from multiple tables. I can export the result to CSV or Excel easily enough, but would like to send it right into SPSS. The ODBC drivers in SPSS only recognize Tables and Views in my SQL database. Any ideas how to get the results of my query into SPSS?
Options:
Export to Excel then import to SPSS... formatting things like dates become unwieldy
Save query as a table in my database... but then I would have to make a new table every time I run the query, yes?
As recommended below, simply run my SQL statement in the GET DATA statement of my SPSS syntax, but I am struggling with that...
UPDATE: In an attempt to use SPSS to run my SQL query I edited this code and get this error indicating that SPSS doesn't like my declaration of nvarchar (currently investigating how to handle this using alternative method). I have tested my connection between SPSS and SQL and the connection is good:
SQLExecDirect failed :[Microsoft][ODBC SQL Server Driver][SQL Server]Incorrect syntax near 'N'.
Here is my query simplified to pull just one field from one table:
GET DATA
/TYPE=ODBC
/CONNECT='DSN=temp_Hisp;Description=tempHisp;UID=;Trusted_Connection=Yes;APP=IBM SPSS '+
'Products: Statistics Common;WSID=ARCH5-50;DATABASE=temp_HispTreat'
/SQL='With CTE_BASENG As (Select StudyID, Visit, Question, CAST(Response As Int) As RESPONSE from temp_HispTreat.dbo.BAS AS PVTable outer apply (values (N'BAS1',BAS1), +'
'(N'BAS24',BAS24)) P(Question, Response)) select SubVis.IRB#, SubVis.StudyID, SubVis.Clin_num, Subvis.Visit, BASENG.BAS_ENGTOT From (Select Distinct IRB#, StudyID, +'
'Clin_Num, Visit_ID As Visit from temp_HispTreat.dbo.Subjects, temp_HispTreat.dbo.StudyStructure where subjects.IRB# = 5516 and StudyStructure.IRB = 5516) As SubVis left join (Select StudyID, +'
'Visit, SUM (Scoring.dbo.GetValue9(response)) As BAS_ENGTOT from CTE_BASENG group by StudyID, Visit) AS BASENG On SubVis.Studyid = BASENG.StudyID And SubVis.Visit = BASENG.Visit'
/ASSUMEDSTRWIDTH=255.
CACHE.
EXECUTE.
Thanks all: Solved. There is quite a bit of tweaking necessary to get SPSS to run SQL query, but this is the best way to export SQL data into SPSS. In my case (values (N'BAS1',BAS1) had to be changed to (values ("BAS1",BAS1) but all of my commands, e.g. outer apply, union, etc, ran like champs! Appreciate the help.
You can use GET DATA procedure to import data from SQL directly in SPSS. See the SQL subcommand. You can use your complicated query here. For example:
GET DATA
/TYPE = ODBC
/CONNECT = "DSN = DSNname"
/SQL = "SELECT * FROM empl_data "
"WHERE ((bdate>=#1/1/1960# and edate<=#12/31/1960#) or bdate is null)".
It is clear why (values (N'BAS1',BAS1) caused the error. Because you are using single quotes for the argument of the SQL subcommand \SQL = ' '. And the first single quote in (values (N'BAS1',BAS1) defines the end of the argument. Switching to double quotes solves it.
I tried to rearrange your code. I can not test it, but I believe it should work:
GET DATA
/TYPE = ODBC
/CONNECT = "DSN=temp_Hisp;DATABASE=temp_HispTreat"
/SQL = "With CTE_BASENG As (Select StudyID, Visit, Question, "
"CAST(Response As Int) As RESPONSE "
"from temp_HispTreat.dbo.BAS AS PVTable "
"outer apply (values (N'BAS1',BAS1), (N'BAS24',BAS24)) "
"P(Question, Response)) "
"select SubVis.IRB#, SubVis.StudyID, SubVis.Clin_num, Subvis.Visit, "
"BASENG.BAS_ENGTOT "
"From (Select Distinct IRB#, StudyID, Clin_Num, Visit_ID As Visit "
"from temp_HispTreat.dbo.Subjects, temp_HispTreat.dbo.StudyStructure "
"where subjects.IRB# = 5516 and StudyStructure.IRB = 5516) As SubVis "
"left join (Select StudyID, Visit, "
"SUM(Scoring.dbo.GetValue9(response)) As BAS_ENGTOT "
"from CTE_BASENG group by StudyID, Visit) AS BASENG On "
"SubVis.Studyid = BASENG.StudyID And SubVis.Visit = BASENG.Visit".
The SQL is processed by the ODBC driver, so the capabilities of that driver will determine what sort of SQL can be issued. The capabilities may be database specific. Someetimes there are multiple drivers available for a particular database, some from the IBM SPSS Data Access Pack and some from a db vendor directly, so you may want to investigate what is available for your particular database.
I've had a look around but cannot find the issue with this SQL Statement:
strSQL = "SELECT Directory.DisplayName, Department.DisplayName, Call.CallDate, Call.Extension, Call.Duration, Call.CallType, Call.SubType FROM (((Department INNER JOIN Directory ON Department.DepartmentID = Directory.DepartmentID) INNER JOIN Extension ON (Department.DepartmentID = Extension.DepartmentID) AND (Directory.ExtensionID = Extension.ExtensionID)) INNER JOIN Site ON Extension.SiteCode = Site.SiteCode) INNER JOIN Call ON Directory.DirectoryID = Call.DirectoryID WHERE (Call.CallDate)>=27/11/2012"
Regardless of what I change the WHERE it always returns every single value in the database (atleast I assume it does since excel completely hangs when I attempt this) this SQL statement works perfectly fine in Access (if dates have # # around them). Any idea how to fix this, currently trying to create a SQL statement that allows user input on different dates, but have to get over the this random hurdle first.
EDIT: The date field in the SQL Database is a DD/MM/YY HH:MM:SS format, and this query is done in VBA - EXCEL 2010.
Also to avoid confusion have removed TOP 10 from the statement, that was to stop excel from retrieving every single row in the database.
Current Reference I have activated is: MicrosoftX Data Objects 2.8 Library
Database is a MSSQL, using the connection string:
Provider=SQLOLEDB;Server=#######;Database=#######;User ID=########;Password=########;
WHERE (Call.CallDate) >= #27/11/2012#
Surround the date variable with #.
EDIT: Please make date string unambiguous, such as 27-Nov-2012
strSQL = "SELECT ........ WHERE myDate >= #" & Format(dateVar, "dd-mmm-yyyy") & "# "
If you are using ado, you should look at Paramaters instead of using dynamic query.
EDIT2: Thanks to #ElectricLlama for pointing out that it is SQL Server, not MS-Access
strSQL = "SELECT ........ WHERE myDate >= '" & Format(dateVar, "mm/dd/yyyy") & "' "
Please verify that the field Call.CallDate is of datatype DATETIME or DATE
If you are indeed running this against SQL Server, try this syntax for starters:
SELECT Directory.DisplayName, Department.DisplayName, Call.CallDate,
Call.Extension, Call.Duration, Call.CallType, Call.SubType
FROM (((Department INNER JOIN Directory
ON Department.DepartmentID = Directory.DepartmentID)
INNER JOIN Extension ON (Department.DepartmentID = Extension.DepartmentID)
AND (Directory.ExtensionID = Extension.ExtensionID))
INNER JOIN Site ON Extension.SiteCode = Site.SiteCode)
INNER JOIN Call ON Directory.DirectoryID = Call.DirectoryID
WHERE (Call.CallDate)>= '2012-11-27'
The date format you see is simply whatever format your client tool decides to show it in. Dates are not stored in any format, they are effectively stored as a duration since x.
By default SQL Uses the format YYYY-MM-DD if you want to use a date literal.
But you are much better off defining a parameter of type date in your code and keeping your date a data type 'date' for as long as possible. This may include only allowing them to enter the date using a calendar control to stop ambiguities.