I am inserting some data from python to sql server. In the dataframe, the data looks ok. However after inserting into sql table, the number converts to some exponential expression. I tried to convert it into a number but the output is not correct. Below is the screenshot of the data in the table:
The first row number should be: 9013513. However, when I convert the exponential number to real number using the code below, it comes out as 9013510. Below is my code:
declare #stproduct nvarchar(255)
set #stproduct = '9.01351e+006'
SELECT CONVERT(numeric(16,0), CAST(#stproduct AS FLOAT))
Related
I am not so familiar with using convert in SQL so thats why i am stuck with the following situation:
The column has a nvarchar(max) and i would like to convert it into decimal (18,2). But there are some rows that consist a "full" amount (see red box). For all values with the full amount i would like to have it as 1222,00
When creating a SQL view I got this error:
Error converting data type nvarchar to numeric.
How can i still convert this column into decimal?
Many thanks
From your pictures it seems to me like you are using SQL-Server. If so your problem is having , as decimal point instead of . It has nothing to do with having whole and decimal numbers in your data.
So you should replace it before converting data.
SELECT CONVERT(decimal(18,2), REPLACE(ColumnName,',','.')) FROM TableName
DB<>fiddle
I want to load data from the last n days from a data source. To do this, I have a project parameter "number_of_days". I use the parameter in an OleDB data source with a SQL Command, with a clause
WHERE StartDate >= CAST(GETDATE() -? as date)
This parameter is mapped to a project parameter, an Int32. But, if I want to load the last 10 days, it is only giving me the last 8 days.
Version info:
SQL Server Data Tools 15.1.61710.120
Server is SQL Server 2017 standard edition.
I set up a test package, with as little data as possible. There is this data source:
Parameter:
Parameter mapping:
The T-SQL expression (wrong result):
CAST(GETDATE() -? as date)
The SSIS expression for date_calc (correct result):
(DT_DBTIMESTAMP) (DT_DBDATE) DATEADD("DD", - #[$Project::number_of_days] , GETDATE())
I would think that the T-SQL expression and the SSIS expression give the same result (today minus 10 days) but that is not the case when I run the package and store the results in a table. See column date_diff, which gives 8 days instead of 10:
If I replace the parameter by the actual value, I do get the correct result.
A data viewer also shows the incorrect date. When I deploy the package, I get the same result as from the debugger.
Is this a bug, or am I missing something here?
I think the main problem is how OLEDB source detect the parameter data type, i didn't find an official documentation that mentioned that, but you can do a small experiment to see this:
Try to write the following Query in the SQL Command in the OLEDB Source:
SELECT ? as Column1
And then try to parse the query, you will get the following error:
The parameter type for '#P1' cannot be uniquely deduced; two possibilities are 'sql_variant' and 'xml'.
Which means that the query parser try to figure out what is the data type of these parameter, it is not related to the variable data type that you have mapped to it.
Then try to write the following query:
SELECT CAST(? AS INT) AS Column1
And then try to parse the query, you will get:
The SQL Statement was successfully parsed.
Now, let's apply these experiment to your query:
Try SELECT CAST(GETDATE() - ? AS DATE) as Column1 and you will get a wrong value, then try SELECT CAST(GETDATE() - CAST(? AS INT) AS DATE) AS Column1 and you will get a correct value.
Update 1 - Info from official documentation
From the following OLEDB Source - Documentation:
The parameters are mapped to variables that provide the parameter values at run time. The variables are typically user-defined variables, although you can also use the system variables that Integration Services provides. If you use user-defined variables, make sure that you set the data type to a type that is compatible with the data type of the column that the mapped parameter references.
Which implies that the parameter datatype is not related to the variable data type.
Update 2 - Experiments using SQL Profiler
As experiments, i created an SSIS package that export data from OLEDB Source to Recordset Destination. The Data source is the result of the following query:
SELECT *
FROM dbo.DatabaseLog
WHERE PostTime < CAST(GETDATE() - ? as date)
And The Parameter ? is mapped to a Variable of type Int32 and has the Value 10
Before executing the package, i started and SQL Profiler Trace on the SQL Server Instance, after executing the package the following queries are recorded into the trace:
exec [sys].sp_describe_undeclared_parameters N'SELECT *
FROM dbo.DatabaseLog
WHERE PostTime < CAST(GETDATE() -#P1 as date)'
declare #p1 int
set #p1=1
exec sp_prepare #p1 output,N'#P1 datetime',N'SELECT *
FROM dbo.DatabaseLog
WHERE PostTime < CAST(GETDATE() -#P1 as date)',1
select #p1
exec sp_execute 1,'1900-01-09 00:00:00'
exec sp_unprepare 1
The first command exec [sys].sp_describe_undeclared_parameters is to describe the parameter type, if we run it separately it returns the following information:
It shows that the parameter data type is considered as datetime.
The other commands shows some weird statement:
First, the value of #P1 is set to 1
The final query is executed with the following value 1900-01-09 00:00:00
Discussion
In SQL Server database engine the base datetime value is 1900-01-01 00:00:00 which can be retrieved by executing the folloing query:
declare #dt datetime
set #dt = 0
Select #dt
On the other hand, in SSIS:
A date structure that consists of year, month, day, hour, minute, seconds, and fractional seconds. The fractional seconds have a fixed scale of 7 digits.
The DT_DATE data type is implemented using an 8-byte floating-point number. Days are represented by whole number increments, starting with 30 December 1899, and midnight as time zero. Hour values are expressed as the absolute value of the fractional part of the number. However, a floating point value cannot represent all real values; therefore, there are limits on the range of dates that can be presented in DT_DATE.
On the other hand, DT_DBTIMESTAMP is represented by a structure that internally has individual fields for year, month, day, hours, minutes, seconds, and milliseconds. This data type has larger limits on ranges of the dates it can present.
Based on that, i think that there is a difference between the datetime base value between SSIS date data type (1899-12-30) and the SQL Server datetime (1900-01-01), which leads to a difference in two days when performing an implicit conversion to evaluate the parameter value.
References
Integration Services Data Types
Parsing Data
Data type conversion (Database Engine)
I have the task to create a query that will provide the average date/time, which is supposed to be a decimal number as the date/time is stored as an integer timestamp.The column in the original table consists of the date (YYYY-DD-MM) and time (HH:MM:SS). I have tried a couple of functions but they didn't work.
Tried:
SELECT avg(tweets.created) AS AverageDate
FROM Tweets;
But it says: "data type mismatch in criteria expression"
Cheers.
I am having some number in DB table column, i have written stored procedure to format that number. The requirement is i need to display the number with comma and decimal points. I have tried below, but only comma is coming.. any solutions,
PARSENAME(CONVERT(VARCHAR,CAST(TotalAmount/1000 as MONEY),1),2)
Actual result: 1,134
Expected result: 1,134.0
I want to do this in SQL server itself.
You can accomplish currency formatting as:
SELECT CONVERT(varchar, CAST(5555 AS money), 1)
but i would suggest doing such formatting on the front end, not on the data side
I have to run column checks for data consistency and the only thing that is throwing off my code is checking for character lengths for dates between certain parameters.
SEL
sum(case when ( A.date is null or (character_length(A.date) >8)) then 1 else 0 end ) as Date
from
table A
;
The date format of the column is YYYY-MM-DD, and the type is DA. When I run the script in SQL Assistant, I get an error 3580 "Illegal use of CHARACTERS, MCHARACTERS, or OCTET_LENGTH functions."
Preliminary research suggests that SQL Assistant has issues with the character_length function, but I don't know how to adjust the code to make it run.
with chareter length are you trying to get the memory used? Becuase if so that is constant for a date field. If you are trying to get the length of the string representation i think LENGTH(A.date) will suffice. Unfortanatly since teradata will pad zeros on conversions to string, I think this might always return 10.
UPDATE :
Okay so if you want a date in a special 'form' when you output it you need to select it properly. In teradata as with most DBs Date are not store in strings, but rather as ints, counting days from a given 'epoch' date for the database (for example the epoch might be 01/01/0000). Each date type in teradata has a format parameter, which places in the record header instructions on how to format the output on select. By default a date format is set to this DATE FROMAT 'MM/DD/YYYY' I believe. You can change that by casting.
Try SELECT cast(cast(A.date as DATE FORMAT 'MM-DD-YYYY') as CHAR(10)) FROM A. and see what happens. There should be no need to validate the form of the dates past a small sample to see if the format is correct. The second cast forces the database to perform the conversion and use the format header specified. Other wise what you might see is the database will pass the date in a date form to SQL Assitant and sql assitant will perform the conversion on the application level, using the format specified in its own setting rather then the one set in the database.