Create table in SAS using DB2 timestamp - sql

We've recently gotten the accelerator (IDAA) working on our DB2, which I mainly access using SAS.
This requires us, due to network issues, to create tables first, before inserting rows.
My problem is creating a table with the correct timestamp format, I can create the table using a select statement, but this is very slow, but here I can see the format in SAS is DATETIME30.6
But if I try something like:
RSUBMIT prod_acc;
Proc delete data=user.table1; run; %PUT &sqlxrc &sqlxmsg;
proc sql inobs=MAX stimer feedback noerrorstop;
connect to db2(ssid=server);
create table user.table1
(
date datetime30.6
,reference char(16)
,transact char(20)
,alias char(60)
,amount decimal(15,2)
,currency char(3)
);
%PUT &sqlxrc &sqlxmsg;
quit;
run;
Which gives the following in the log
(
15 date datetime30.6
-----------
1 22
200
WARNING 1-322: Assuming the symbol DATE was misspelled as datetime30.
ERROR 22-322: Syntax error, expecting one of the following: a quoted string,
an integer constant, ), ',', CHECK, DISTINCT, FORMAT, INFORMAT, LABEL, LEN,
LENGTH, NOT, PRIMARY, REFERENCES, TRANSCODE, UNIQUE, ^, ~.
ERROR 200-322: The symbol is not recognized and will be ignored.
And if I look in DB2, the column has the type timestmp which SAS don't recognize as a type.
(
31 date timestmp
--------
22
76
ERROR 22-322: Syntax error, expecting one of the following: CHAR, CHARACTER, DATE, DEC,
DECIMAL, DOUBLE, FLOAT, INT, INTEGER, NUM, NUMERIC, REAL, SMALLINT, VARCHAR.
ERROR 76-322: Syntax error, statement will be ignored.
Tried googling and found a lot of different versions of answers, but nothing I can see is relevant to this, the closest was something about manually creating the format, but I can't figure out how to do that.
Any ideas?

It is probably more natural in SAS to define a table's structure using a DATA step rather than PROC SQL.
data userdb.table1;
stop;
length date 8 reference $16 transact $20 alias $60 amount 8 currency $3 ;
format date datetime30.6 amount 15.2 ;
run;
If your libref is pointing to a database then you should be able to use DBTYPE= dataset option to tell SAS what data types to use for your fields in the external database. At least it works for Teradata. These dataset options should work inside PROC SQL also.
proc delete data=userdb.table1; run;
data userdb.table1
(dbtype=
( date='timestamp'
reference='varchar(16)'
transact='varchar(20)'
alias='varchar(60)'
amount='decimal(15,2)'
currency='char(3)'
)
);
stop;
length date 8 reference $16 transact $20 alias $60 amount 8 currency $3 ;
format date datetime30.6 amount 15.2 ;
run;

Can't you just:
create table user.table1
(
"date" TIMESTAMP(6)
,reference char(16)
,transact char(20)
,alias char(60)
,amount decimal(15,2)
,currency char(3)
);
? Remember, in DB2, date is a reserved word, and then it's always safe to put that into double quotes. Alternatively, use a non-reserved word for the column name, like dt or so.

These two lines are incongruous:
connect to db2(ssid=server);
create table user.table1
The first creates a connection for a pass-through query, while the latter creates the table using the libname engine. In this case your first statement is irrelevant as it's not used; you should remove it (unless you use it later and just left it in by mistake in your example).
Since you used the libname syntax, you must follow SAS syntax rather than DB2. There is no specification for datetime type in the create table statement, specifically under the column-definition documentation page. Instead you have this list to choose from:
CHARACTER | VARCHAR <(width)> indicates a character column with a column width of width. The default column width is eight characters.
INTEGER | SMALLINT indicates an integer column.
DECIMAL | NUMERIC | FLOAT <(width<, ndec>)> indicates a floating-point column with a column width of width and ndec decimal
places.
REAL | DOUBLE PRECISION indicates a floating-point column.
DATE indicates a date column.
The way I find best to specify datetime (meaning, most likely to work as you expect) is not to use date but numeric, and then use the format argument to define it as datetime.
proc sql;
create table table1
( date num format=datetime30.6
,reference char(16)
,transact char(20)
,alias char(60)
,amount decimal(15,2)
,currency char(3)
);
quit;
However, I would suggest your best choice is to use passthrough to create the table, so you can use DB2 syntax - since you're creating a table there, not in SAS itself.

Related

SQL - How to change data type float to nvarchar and remove scientific notation

How do I change the data type float to nvarchar in order to remove the scientific notation and still keep precision? Consider the following:
CREATE TABLE ConversionDataType (ColumnData FLOAT);
INSERT INTO ConversionDataType VALUES (25566685456126);
INSERT INTO ConversionDataType VALUES (12345545546845);
INSERT INTO ConversionDataType VALUES (12345545545257);
When I do a simple read I get the following data, as expected:
select * from ConversionDataType
ColumnData
------------------------------------
25566685456126
12345545546845
12345545545257
Now when I try update the data type to an nvarchar, it gets stored in scientific notation which is something I don't want:
update ConversionDataType
set ColumnData = CAST(ColumnData AS NVARCHAR)
The result set is as follows:
25566700000000
12345500000000
12345500000000
It replaces some digits and adds zeros after the 6th index. How can I go about this? I had a look at the Convert function but that is only for converting date time data types.
Being valid what others said in comment, if you just want to convert float to varchar without scientific notation, you need to convert to numeric. You can try this:
SELECT CAST(CAST(CAST(25566685456126291 AS FLOAT) AS NUMERIC) AS NVARCHAR)
Output:
C1
------------------------------
25566685456126292
Whereas
SELECT CAST(CAST(25566685456126291 AS FLOAT) AS NVARCHAR) AS C1
gives:
C1
------------------------------
2.55667e+016
If you need to change datatype, I think you should add a new column, update it and (if you want) delete the old column and rename the new column at the end.
CREATE TABLE TEST1 (C1 FLOAT)
INSERT INTO TEST1 VALUES (25566685456126291);
ALTER TABLE TEST1 ADD C2 VARCHAR(18)
UPDATE TEST1 SET C2=CAST(CAST(C1 AS NUMERIC) AS VARCHAR)
SELECT * FROM TEST1
Output:
C1 C2
---------------------- ------------------
2.55666854561263E+16 25566685456126292
FLOAT was a very bad decision as this is not a precise data type. If you wanted to store the phone numbers as numbers, you'd have to go for DECIMAL instead.
But you'll have to use NVARCHAR instead. And this is the only reasonable design, as phone numbers can have leading zeros or start with a plus sign. So the first thing is to introduce an NVARCHAR column:
ALTER TABLE ConversionDataType ADD ColumnDataNew NVARCHAR(30);
The function to convert a number into a string in SQL Server is FORMAT. It lets you state the format you want to use for the conversion, which is integer in your case (a simple '0'):
update ConversionDataType set ColumnDataNew = format(ColumnData, '0');
At last remove the old column and then rename the new one with the same name. SQL Server lacks an ALTER TABLE syntax to rename a column, so we must call sp_RENAME instead (at least this is what I have read on the Internet; here is a link to the docs: https://msdn.microsoft.com/de-de/library/ms188351.aspx).
ALTER TABLE ConversionDataType DROP COLUMN ColumnData;
EXEC sp_RENAME 'ConversionDataType.ColumnDataNew', 'ColumnData', 'COLUMN';
Here you can see the results: http://rextester.com/GLLB27702
SELECT CONVERT(NVARCHAR(250), StudentID) FROM TableA
StudentID is your Float Column of database
or Simply use
SELECT CONVERT(NVARCHAR(250), yourFloatVariable)

Can't convert postgresql table column from type varchar to int

I have a database table of that I have used to store the data returned from a web spider. I have a column that contains ticket prices for different events all in the varchar type (as the scrapy spider has to scrape the data in unicode). I'm trying to return the min price of the column and since the min() function only works for data of type INT, I tried to convert the column to integers using a solution from this SO post:
ALTER TABLE vs_tickets ALTER COLUMN ticketprice TYPE integer USING (ticketprice::integer);
but I got the error: ERROR: invalid input syntax for integer:
I also tried: change_column :vs_tickets, :ticketprice, 'integer USING CAST(ticketprice AS integer)' but that didn't work either.
What is the proper way to convert the column to type INT?
Edit:
You have decimal places in the string, so a simple cast is not going to work. You can do a double conversion:
cast(cast(ticketprice as decimal(10, 2)) as int)
or:
(ticketprice::decimal(10, 2))::int
(The parens are not strictly necessary.)
EDIT:
Or, as Erwin points out, just use numeric:
(ticketprice::numeric)::int
Postgres is much smarter about numeric than most other databases . . . after all, it supports numbers that are egregiously large ;)
The final query is:
ALTER TABLE vs_tickets
ALTER COLUMN ticketprice TYPE integer USING (ticketprice::numeric::integer);
I'm going to bet on your column have wrong characters.
Also you may want use float or numeric because you will lose decimals if convert to integers.
You need create a function to check if a text is numeric like this isnumeric-with-postgresql
Then check each row like this
select ticketprice
from vs_tickets
where ISNUMERIC(ticketprice) = false;
As your comment you also should try
SELECT ticketprice::float
You will be best off adding an INT column, moving your data with a cast and then removing the old varchar column.
ALTER TABLE vs_tickets ADD COLUMN ticketprice_int TYPE int;
GO
update vs_tickets SET ticketprice_int = cast(ticketprice as int);
// if you fail to cast the varchar to int you can use Gordon's method
// update vs_tickets SET ticketprice_int = cast(cast(ticketprice as decimal(10, 2)) as int);
GO
ALTER TABLE vs_tickets DROP COLUMN ticketprice;
GO
ALTER TABLE vs_tickets RENAME COLUMN ticketprice_int to ticketprice;
GO
With this at minimum you will be able to tell if and where a cast/convert fails and be able to check and recheck at each step before you can't turn back.

SQL Server Management Studio ZEROFILL

I am trying to add a auto_increment primary key with ZEROFILL with a max size of six.
So it outputs:
000001
000002 etc...
However I am really struggling to achieve this and cant find the answer. How do I set up this column type in SQL Server Management Studio?
Thanks
You cannot do this with an integer field in SQL Server (nor would I recommend it with a Varchar).
Let SQL Server store the field as an Identity, and then (assuming this is for display), format the data when you select it like such:
SELECT RIGHT('000000' + CONVERT(VARCHAR(6),ID), 6) FROM Table
What you're trying to achieve is not possible. Display formatting is done in a presentation layer and not in the database. You need to separate a value from the presentation of a value. The two values 1 and 000001 are the same.
If you want to return something formatted, then you would have to return the value as a string. Just cast it to a string, add a number of zeroes at beginning and then keep the leftmost n characters.
SELECT FORMAT(2, N'000000')
--OR
SELECT FORMAT(2, CAST(REPLICATE(0, 6) AS NVARCHAR(6)))
https://learn.microsoft.com/en-us/sql/t-sql/functions/format-transact-sql?view=sql-server-ver15
By creating a function
CREATE FUNCTION MyZeroFill (
#N BIGINT,
#D SMALLINT
) RETURNS VARCHAR(50) AS
BEGIN
RETURN RIGHT('0000000000000000000000000000000000000000000000'+CAST
(#N AS VARCHAR),#D)
END

mysql syntax error for timestamp

I have this piece of SQL that is being fed to Mysql.
CREATE TABLE pn_history(
member INT,
action INT,
with INT,
timestamp DATETIME,
details VARCHAR(256)
)
But is comes back as an error about the syntax.
#1064 - You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'with INT,
timestamp DATETIME,
details VARCHAR(256)
)' at line 4
Why is this failing?
Both 'with' and 'timestamp' are reserved words in MySQL. So to get this to work, you'd need to escape each one:
CREATE TABLE pn_history(
member INT,
action INT,
`with` INT,
`timestamp` DATETIME,
details VARCHAR(256)
)
Really though, you need to consider changing the names of your columns identifiers.
Read more about MySQL Reserved Words.
EDIT: Actually, TIMESTAMP is not a reserved word. The documentation says:
MySQL allows some keywords to be used
as unquoted identifiers because many
people previously used them. Examples
are those in the following list:
ACTION
BIT
DATE
ENUM
NO
TEXT
`TIME
TIMESTAMP
So I guess that means peer pressure took TIMESTAMP off the reserved word list. Hah!
The problem is the name of the with column. Change the name into something like withValue.
CREATE TABLE pn_history(
member INT,
action INT,
withValue INT,
timestamp DATETIME,
details VARCHAR(256)
)
timestamp is a keyword (it is a data type in mysql) which may be causing you problems.
I would suggest using a different name, but if it must be named timestamp, try using backticks to quote it.

How to create timestamp column with default value 'now'?

How to create a table with a timestamp column that defaults to DATETIME('now')?
Like this:
CREATE TABLE test (
id INTEGER PRIMARY KEY AUTOINCREMENT,
t TIMESTAMP DEFAULT DATETIME('now')
);
This gives an error.
As of version 3.1.0 you can use CURRENT_TIMESTAMP with the DEFAULT clause:
If the default value of a column is CURRENT_TIME, CURRENT_DATE or CURRENT_TIMESTAMP, then the value used in the new row is a text representation of the current UTC date and/or time. For CURRENT_TIME, the format of the value is "HH:MM:SS". For CURRENT_DATE, "YYYY-MM-DD". The format for CURRENT_TIMESTAMP is "YYYY-MM-DD HH:MM:SS".
CREATE TABLE test (
id INTEGER PRIMARY KEY AUTOINCREMENT,
t TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);
according to dr. hipp in a recent list post:
CREATE TABLE whatever(
....
timestamp DATE DEFAULT (datetime('now','localtime')),
...
);
It's just a syntax error, you need parentheses: (DATETIME('now'))
The documentation for the DEFAULT clause says:
If the default value of a column is an expression in parentheses, then the expression is evaluated once for each row inserted and the results used in the new row.
If you look at the syntax diagram you'll also notice the parentheses around 'expr'.
This is a full example based on the other answers and comments to the question. In the example the timestamp (created_at-column) is saved as unix epoch UTC timezone and converted to local timezone only when necessary.
Using unix epoch saves storage space - 4 bytes integer vs. 24 bytes string when stored as ISO8601 string, see datatypes. If 4 bytes is not enough that can be increased to 6 or 8 bytes.
Saving timestamp on UTC timezone makes it convenient to show a reasonable value on multiple timezones.
SQLite version is 3.8.6 that ships with Ubuntu LTS 14.04.
$ sqlite3 so.db
SQLite version 3.8.6 2014-08-15 11:46:33
Enter ".help" for usage hints.
sqlite> .headers on
create table if not exists example (
id integer primary key autoincrement
,data text not null unique
,created_at integer(4) not null default (strftime('%s','now'))
);
insert into example(data) values
('foo')
,('bar')
;
select
id
,data
,created_at as epoch
,datetime(created_at, 'unixepoch') as utc
,datetime(created_at, 'unixepoch', 'localtime') as localtime
from example
order by id
;
id|data|epoch |utc |localtime
1 |foo |1412097842|2014-09-30 17:24:02|2014-09-30 20:24:02
2 |bar |1412097842|2014-09-30 17:24:02|2014-09-30 20:24:02
Localtime is correct as I'm located at UTC+2 DST at the moment of the query.
It may be better to use REAL type, to save storage space.
Quote from 1.2 section of Datatypes In SQLite Version 3
SQLite does not have a storage class set aside for storing dates
and/or times. Instead, the built-in Date And Time Functions of SQLite
are capable of storing dates and times as TEXT, REAL, or INTEGER
values
CREATE TABLE test (
id INTEGER PRIMARY KEY AUTOINCREMENT,
t REAL DEFAULT (datetime('now', 'localtime'))
);
see column-constraint .
And insert a row without providing any value.
INSERT INTO "test" DEFAULT VALUES;
It is syntax error because you did not write parenthesis
if you write
Select datetime('now')
then it will give you utc time but if you this write it query then you must add parenthesis before this
so (datetime('now')) for UTC Time.
for local time same
Select datetime('now','localtime')
for query
(datetime('now','localtime'))
If you want millisecond precision, try this:
CREATE TABLE my_table (
timestamp DATETIME DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ', 'now'))
);
This will save the timestamp as text, though.
This alternative example stores the local time as Integer to save the 20 bytes. The work is done in the field default, Update-trigger, and View.
strftime must use '%s' (single-quotes) because "%s" (double-quotes) threw a 'Not Constant' error on me.
Create Table Demo (
idDemo Integer Not Null Primary Key AutoIncrement
,DemoValue Text Not Null Unique
,DatTimIns Integer(4) Not Null Default (strftime('%s', DateTime('Now', 'localtime'))) -- get Now/UTC, convert to local, convert to string/Unix Time, store as Integer(4)
,DatTimUpd Integer(4) Null
);
Create Trigger trgDemoUpd After Update On Demo Begin
Update Demo Set
DatTimUpd = strftime('%s', DateTime('Now', 'localtime')) -- same as DatTimIns
Where idDemo = new.idDemo;
End;
Create View If Not Exists vewDemo As Select -- convert Unix-Times to DateTimes so not every single query needs to do so
idDemo
,DemoValue
,DateTime(DatTimIns, 'unixepoch') As DatTimIns -- convert Integer(4) (treating it as Unix-Time)
,DateTime(DatTimUpd, 'unixepoch') As DatTimUpd -- to YYYY-MM-DD HH:MM:SS
From Demo;
Insert Into Demo (DemoValue) Values ('One'); -- activate the field Default
-- WAIT a few seconds --
Insert Into Demo (DemoValue) Values ('Two'); -- same thing but with
Insert Into Demo (DemoValue) Values ('Thr'); -- later time values
Update Demo Set DemoValue = DemoValue || ' Upd' Where idDemo = 1; -- activate the Update-trigger
Select * From Demo; -- display raw audit values
idDemo DemoValue DatTimIns DatTimUpd
------ --------- ---------- ----------
1 One Upd 1560024902 1560024944
2 Two 1560024944
3 Thr 1560024944
Select * From vewDemo; -- display automatic audit values
idDemo DemoValue DatTimIns DatTimUpd
------ --------- ------------------- -------------------
1 One Upd 2019-06-08 20:15:02 2019-06-08 20:15:44
2 Two 2019-06-08 20:15:44
3 Thr 2019-06-08 20:15:44