Goal:
Retrieve data in a notepad with ';' as a separator between column.
The data in the notepad should be:
2001-11-11 00:00:000;1
2001-11-11 00:00:000;2
2001-11-11 00:00:000;0
Problem:
How should I transfer data from table into notepad with this data
(datetime) (int)
date Number
--------------------------------
2001-11-11 00:00:000 1
2001-11-11 00:00:000 2
2001-11-11 00:00:000 0
2001-11-11 00:00:000 4
Go into tools-options Open the Query Results Tree, SQL Server, Results to Text. In there you will see output format and you should be able to choose custom delimiter which you can then set to a semi colon.
If you now change your output to the text (ctrl+T) or results to file (ctrl + shift + F) you should get the output you desire.
If your database is MySQL for example, you just dump the database using:
mysqldump --fields-terminated-by=str databasename
where str is ";".
select "results to file" on the menu and run:
select CAST(date as varchar(20))+';'+CAST(number as varchar(20))
from yourTable
Related
I have data in txt format with several files, each file has 3 columns and few files have 4 columns, how do create the table which will only read 1st 3 columns?
i have been using BQ UI for creating tables, this is causing error when reading data with 4 columns,
EX : file1.txt
1234|sample|test
1254|sample2|test2
12324|sample3|test3
File2.txt
1234|var1|test
1254|var2|test2
12324|var3|test3
file3.txt
1234|var1|test|123242
1254|var2|test2|1321412
12324|var3|test3|1312123
Schema i have now is
field1 Int
field2 String
field3 String
filed4 Int
i have these file in gcs bucket and the external table is pointed to this location,
Error while reading table:
testing-dataset-project.testing_tables.test_data, error message:
CSV table references column position 3, but line starting at
position:0 contains only 3 columns.
You can use the option Allow jagged rows as described here.
I have a character column which has dates (dd/mm/yyyy) in character format.
While applying filter (where clause), I need that these characters are recognized as dates in the where statement, without actually making any change to the existing column or without creating a new column.
How can I make this happen.
Any help would be deeply appreciated.
Thank you.
In proc sql, you can come close with like:
select (case when datecol like '__/__/____'
then . . .
else . . .
end)
This is only an approximation. _ is a wildcard that matches any character, not just numbers. On the other hand, this is standard SQL, so it will work in any database.
The SAS INPUT function with a ? informat modifier will convert a string (source value) to a result and not show an error if the source value is not conformant to the informat.
INPUT can be used in a WHERE statement or clause. The input can also be part of a BETWEEN statement.
* some of these free form values are not valid date representations;
data have;
length freeform_date_string $10;
do x = 0 to 1e4-1;
freeform_date_string =
substr(put(x,z4.),1,2) || '/' ||
substr(put(x,z4.),3,2) || '/' ||
'2018'
;
output;
end;
run;
* where statement;
data want;
set have;
where input(freeform_date_string,? ddmmyy10.);
run;
* where clause;
proc sql;
create table want2 as
select * from have
where
input(freeform_date_string,? ddmmyy10.) is not null
;
* where clause with input used with between operator operands;
proc sql;
create table want3 as
select * from have
where
input(freeform_date_string,? ddmmyy10.)
between
'15-JAN-2018'D
and
'15-MAR-2018'D
;
quit;
It is not great idea to store date as character value, it can lead to lot of data accuracy related issues and you may not even know that you have data issues for a long time. say someone enters wrong character date and you may not even know. it is always good to maintain date as date value rather than as character value
In your code Filter dates using like becomes little complex for dates. You can try below code which will work for you by using input statement in where clause
data have;
input id datecolumn $10.;
datalines;
1 20/10/2018
1 25/10/2018
2 30/10/2018
2 01/11/2018
;
proc sql;
create table want as
select * from have
where input(datecolumn, ddmmyy10.) between '20Oct2018'd and '30Oct2018'd ;
using like as shown below for above same code
proc sql;
create table want as
select * from have
/*include all dates which start with 2 */
where datecolumn like '2%' and datecolumn like '%10/2018'
or datecolumn = '30/10/2018';
Edit1:
looks like you have data quality issue and sample dataset is shown below. try this. Once again i want to say approach of storing dates as character values is not good and can lead to lot of issues in future.
data have;
input id datecolumn $10.;
datalines;
1 20/10/2018
1 25/10/2018
2 30/10/2018
2 01/11/2018
3 01/99/2018
;
proc sql;
create table want(drop=newdate) as
select *, case when input(datecolumn, ddmmyy10.) ne .
then input(datecolumn, ddmmyy10.)
else . end as newdate from have
where calculated newdate between '20Oct2018'd and '30Oct2018'd
;
or you can put your case statement without making and dropping new column as shown below.
proc sql;
create table want as
select * from have
where
case when input(datecolumn, ddmmyy10.) ne .
then input(datecolumn, ddmmyy10.) between '20Oct2018'd and '30Oct2018'd
end;
I have a set of input data in FoxPro. One of the fields, the grp field, is a concatenated string, the
individual portions of which are delimited by the pipe symbol, "|". Here are some examples of the values it can take:
grp:
ddd|1999|O|%
bce|%
aaa|2009|GON|Fixed|big|MAE|1
bbb|PAL|N|Fixed|MAE|1
aaa|SMK|O|Fixed|MAE|1|1
ddd|ERT|O|%
eef|%|N|%
afd|2000|O|%
afd|200907|O|%
swq|%|O|%
%
I would like to write a query that will separate the data above into separate fields and output them to another sql table, where the deciding factor for the separation is the pipe symbol. Taking the first two rows as an example, the output should read
Record 1:
Field1 = ddd Field2 = 1999 Field3 = O Field4 = %
Record 2:
Field1 = bce Field2 = % Field3 holds no value Field4 holds no value
It will not be known in advance what the greatest number of pipe symbols in the data will be. In the example above, it is 6, in records 3 and 5.
Is it actually possible to do this?
You can create a cursor and append the data into it using 'append from' (another way would be to use 2 alines, one for rows other for columns data). For example using your data as a text variable:
Local lcSample, lcTemp, lnFields, ix
TEXT to m.lcSample noshow
ddd|1999|O|%
bce|%
aaa|2009|GON|Fixed|big|MAE|1
bbb|PAL|N|Fixed|MAE|1
aaa|SMK|O|Fixed|MAE|1|1
ddd|ERT|O|%
eef|%|N|%
afd|2000|O|%
afd|200907|O|%
swq|%|O|%
%
ENDTEXT
lnFields = 0
Local Array laText[1]
For ix=1 To Alines(laText, m.lcSample)
m.lnFields = Max(m.lnFields, Occurs('|', m.laText[m.ix]))
Endfor
#Define MAXCHARS 20 && max field width expected
Local Array laField[m.lnFields,4]
For ix = 1 To m.lnFields
m.laField[m.ix,1] = 'F' + Ltrim(Str(m.ix))
m.laField[m.ix,2] = 'C'
m.laField[m.ix,3] = MAXCHARS
m.laField[m.ix,4] = 0
Endfor
lcTemp = Forcepath(Sys(2015)+'.txt', Sys(2023))
Strtofile(m.lcSample, m.lcTemp)
Create Cursor myData From Array laField
Append From (m.lcTemp) Delimited With "" With Character "|"
Erase (m.lcTemp)
Browse
However, in real world, this doesn't sound to be very realistic. You should know something about the data ahead.
And also, you could use FoxyClasses' import facilities to get the data. It lets you to choose the delimiters, map the columns etc. but requires some developer intervening for the final processing of the data.
The ALINES() function makes parsing easy. You could apply it to each line in turn. #Cetin has already showed you how to find out how many fields you need.
I had to do something very similar with some client data. They provided a field that was a space separated list of numbers that needed to be pulled out into a single column of numbers to match to an offer. Initially I dumped it to a text file, and imported it back into a new table. Something like the following:
create table groups ;
(group1 c(5), group2 c(5), group3 c(5), group4 c(5), group5 c(5))
select grp from infile to file grps.tmp noconsole plain
select groups
append from grps.tmp delimited with "" with character "|"
I am trying to find last Saturday date in HIVE in YYYY-MM-DD format using:
SET DATE_DM2=date_sub(from_unixtime(unix_timestamp(),'yyyy-MM-dd'),cast(((from_unixtime(unix_timestamp(), 'u') % 7)+1) as int));
But this is giving error
Change this line:
c = "hive -e 'use Data; SELECT * FROM table1 WHERE partitiondate='${DATE_D}';'"
into:
c = "hive -e \"use Data; SELECT * FROM table1 WHERE partitiondate='{DATE_D}';\"".format(DATE_D=DATE_D)
The call to format is what #Mai was mentioning. As for printing c, print c will show you the value of c at runtime, so you'll know if the value is as you expect.
P.S. calling commands.getoutput will not only fetch the rows but all of the standard output of calling hive in command line, and store that in a single string - meaning you'll probably need to do some parsing if you need to work with those rows. Or better yet, check out HiveClient.
I have a field in sql that contains a 1 or 0 at the end. What I am trying to do is if the field has a 1 at the end but no corresponding 0 I would like to add that record.
3 different examples
Field value
Data1 ( I would like to add another record containing Data0)
Data0 ( I would like to add another record containing Data1)
Data0 and Data1 both exists in table ( Do nothing )
insert into test(col, another_column_1,...,another_column_n)
select substr(col,1, length(col)-1) || --this is the base value
max(mod(substr(col,length(col),length(col))+1,2)) --this is the termination (0 or 1)
as col ,
max(another_column_1),
...
max(another_column_n)
from test
where substr(col,length(col),length(col)) between '0' and '1'
group by substr(col,1, length(col)-1)
having count(*)=1;
you can see test here
Updated for Oracle
If I understood your question correctly, you can use the LIKE operator.
You can do something like:
(field like '%1' or field like '%0') and not field like '%0%1'
But generally, SQL is not suitable for text processing. This LIKE thing is pretty much the most advanced text processing feature in SQL.