I have written an SQL script having the below query. The query works fine.
update partner set is_seller_buyer=1 where id in (select id from partner
where names in
(
'A','B','C','D','E',... -- Around 100 names.
));
But now instead of writing around 100 names in a query itself, I want to fetch all the names from the CSV file. I read about SQL*Loader on the Internet, but I did not get much on an update query.
My CSV file only contain names.
I have tried
load data
infile 'c:\data\mydata.csv'
into table partner set is_wholesaler_reseller=1
where id in (select id from partner
where names in
(
'A','B','C','D','E',... -- Around 100 names.
));
fields terminated by "," optionally enclosed by '"'
( names, sal, deptno )
How can I achieve this?
SQL*Loader does not perform updates, only inserts. So, you should insert your names into a separate table, say names, and run your update from that:
update partner set is_seller_buyer=1 where id in (select id from partner
where names in
(
select names from names
));
Your loader script can be changed to:
load data
infile 'c:\data\mydata.csv'
into table names
fields terminated by "," optionally enclosed by '"'
( names, sal, deptno )
An alternate to this is to use External Tables which allows Oracle to treat a flat file like it is a table. An example to get you started can be found here.
Related
I'm very new to AWS & Athena. I'm using Athena to query a data file (CSV) from S3 using glue crawlers to create catalog and then querying that info. I have the catalog table created by glue, containing fName, sName, mName info. I'm trying to search a regexp pattern from all the rows and columns with a single query.
I have created a second table containing the column names of the primary table, i.e. fName, sName, mName.
I would like to loop through the second table rows -> using each value in my regexp_like function to search for any names starting with 'B'
e.g.
where regexp_like(fname,'^B')
where regexp_like(sname,'^B')
where regexp_like(mname,'^B')
and display all of them.
Is this possible? I have not been able to get the first query working even when hardcoding the search criteria
e.g.
select * from primary_table
where regexp_like((Select column from secondary_table where column_name='fname'),'^B')
above SQL -> Select column from secondary_table where column_name='fname' resolves to fname as string, not fname column in primary table.
Storing column names and table names as data is generally not recommended. In order to use the information, you need to use dynamic SQL -- that is, construct the query as a string and execute that.
You can get something similar using a lot of logic. But you have to check for each column explicitly:
select p.*
from primary p join
secondary s
on s.column_name = 'fname' and regexp_like(p.fname, '^B') or
s.column_name = 'sname' and regexp_like(p.sname, '^B') or
s.column_name = 'mname' and regexp_like(p.mname, '^B') ;
In Hue/Hive,
Describe mytablename;
gives the list of columns, their types and comments. Is there any way to query in Hive, treating result from describe as a table ?
For example I want to count the number of numeric/character/specific type columns, filter column names, total number of columns (currently requires scrolling down per 100 each, which is a hassle with 1000+ columns), etc
Queries such as
select count(*) from (Describe mytablename);
select count(*) from (select * from describe mytablename);
are of course invalid
Any ideas ?
You can create a sql file --> hive.sql containing "describe dbname.tablename"
hive -f hive.sql > /path/file.txt
create table dbname.desc
(
name String,
type String,
desc String
)
ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t'
then, load data from path '/path/file.txt' into table dbname.desc.
I've a data in file like
[street#226 S Wabash Ave,city#Chicago,state#IL]
[street#227 B H Tower,city#San Diego,state#CA]
I've created table using this code:
create external table if not exists test (
address map<string,string>
)
row format delimited fields terminated by '\;'
COLLECTION ITEMS TERMINATED BY ','
map keys terminated by '#'
;
but when I'm reading the file, I'm getting it parsed like this:
{"[street#939 W El Camino":null,"city#Chicago":null,"state#IL]":null}
{"[street#415 N Mary Ave":null,"city#Chicago":null,"state#IL]":null}
What should I do ?
Solved. I first loaded the data as a string in the table and then created another table using
select str_to_map(regexp_replace(address,'\\[|\\]',''),',','#') as address from test;
I'm using a stored procedure to bulk insert a large csv file into a table, all in one field that is set to varchar(8000). I've had to do it this way as some of the data is enclosed in quotation marks and some are not. In SQL Server 2008 to be usable as a data file for bulk import, a CSV file must comply with the following restrictions:
Data fields never contain the field terminator.
Either none or all of the values in a data field are enclosed in quotation marks ("").
My data is thus:
Field1
"data", "data2", "data3", "data4", 123, 567, 354, 5,64,4565,54
Which is now in a temptable with SQL Server. How do I now clean the data and insert into a table to look like the below: (I already have this new table setup with the correct headings)
Field1
data
Field2
data 2
Field 3
data 3
And so on.
Ultimately it all needs to be performed in a stored procedure as it needs to be in reporting services. I've been looking at the functions, but how do I make it work when some of the fields do not have double quotes? Is the comma enough? Also is the XML function the best?
I take the following approach to importing data:
Insert the table into a staging table where all the columns are character strings.
Copy the table from there into the appropriate table with the column types and structure that I want.
In your case, the second part of the code would be:
insert into FinalTable(col1, . . . )
select (case when left(col1, 1) = '"' then replace(col1, '"', '')
else col1
end),
(case when isnumeric(col2) = 1 then cast(col2 as float)
end),
. . .
from StagingTable
There are, no doubt, solutions in SSIS or using a format file. I prefer a staging table approach because I find it easier to debug data issues from the database using the staging table.
When I add element to column (varchar) I get extra space. For example if I have a table Student with name varchar(10) and I do:
INSERT INTO Student (id,name) VALUES (1,'student')
SELECT name FROM Student WHERE id=1
I get student[space][space][space].
How can I fix without changing the type to text?
Most databases output results from a query in a fixed tabular format. To see where a string really begins and ends, you need to concatenate another character. Here are three ways, depending on the database:
select '"'+name+'"'
select '"'||name||'"'
select concat('"', name, '"')
You'll probably find that the spaces are an artifact of the query tool.