I'm trying to upload a csv file into a table in a Oracle database. I've used the following command line:
CREATE TABLE tmp_x (codice varchar(100), descrizione varchar(300), tipo varchar(100), macroarea varchar(100), includere varchar(100));
LOAD DATA INFILE '/home/-/Documenti/Tabelle_per_App/OK/REPARTI_modifiche.csv' into table tmp_x field terminated by ','
LINES TERMINATED BY "\n" (codice, descrizione, tipo, #vmacroarea, includere) SET macroarea = NULLIF(#vmacroarea,'')
no error appears, but the table remain empty.
I need to do it with a command line query, without creating external tables.
Thanks a lot.
CODICE
DESCRIZIONE
TIPO
MACROAREA
INCLUDERE
C19
COLLABORATORI COVID19
I
NO
CCV
COVID COLLABORATORI VACCINATI
E
NO
CET
CONTROLLI ENDOSCOPIA TORACICA
I
NO
CGF
CHIRURGIA GEN. FAMILIARI
I
NO
CHL
CHIRURGIA PLASTICA - FAMILIARI
I
NO
CKM
CHECK UP S MARCO
I
NO
CPR
POSTRICOVERO CARDIOLOGIA
I
NO
CSF
CHIRURG.SENOLOGICA - FAMILIARI
I
NO
CTF
CHIRURGIA TORACICA - FAMILIARI
I
NO
CVF
CHIRURG.VASCOLARE - FAMILIARI
I
NO
D0800
Cardiologia
I
MEDICA
SI
D0900
Chirurgia Generale
I
CHIRURGICA
SI
D090003
Chirurgia Plastica
I
CHIRURGICA
SI
D090010
Chirurgia Senologica
I
CHIRURGICA
SI
D130000
Chirurgia Toracica
I
CHIRURGICA
SI
D1400
Chirurgia Vascolare
I
CHIRURGICA
SI
Perhaps you wish to create an external table. If such be the case, then please read https://oracle-base.com/articles/9i/external-tables-9i for instance.
Or, you may create a table in Oracle (not external) using the plain create table command, like the one you gave, after which you load data from the .csv file using sqlldr. For sqlldr, read: https://docs.oracle.com/en/database/oracle/oracle-database/19/sutil/oracle-sql-loader.html#GUID-8D037494-07FA-4226-B507-E1B2ED10C144
Related
I'm trying to export a bunch of DB2 tables to CSV, with column names. I don't see any straight forward way to do this. I followed this to get the data I want. But I have to execute that over hundreds of tables. Is there a way to dynamically get all the columns and tables given N schema names?
I also tried this which exports all tables to csv in a schema but this doesn't give me column names. So if someone could show me show to change this script to get column names in the CSVs my work is done.
The server is running: Red Hat Linux Server.
Using files
The following db2 command generates the export script:
export to exp.sql of del modified by nochardel
select
x'0a'||'export to file_header of del modified by nochardel VALUES '''||columns||''''
||x'0a'||'export to file_data of del messages messages.msg select '||columns||' from '||tabname_full
||x'0a'||'! cat file_header file_data > '||tabname_full||'.csv'
from
(
select rtrim(c.tabschema)||'.'||c.tabname as tabname_full, listagg(c.colname, ', ') as columns
from syscat.tables t
join syscat.columns c on c.tabschema=t.tabschema and c.tabname=t.tabname
where t.tabschema='SYSIBM' and t.type='T'
group by c.tabschema, c.tabname
--fetch first 10 row only
)
;
It's better to place the command above to some file like gen_exp.sql and run it to produce the export script:
db2 -tf gen_exp.sql
The export script exp.sql consists of 3 commands for each table:
* db2 export command to get a comma separated list of columns
* db2 export command to get table data
* concatenation command to collect both outputs above to a single file
You run this script as follows:
db2 -vf exp.sql -z exp.sql.log
Using pipe
gen_exp_sh.sql:
export to exp.sh of del modified by nochardel
select
x'0a'||'echo "'||columns||'" > '||filename
||x'0a'||'db2 "export to pipe_data of del messages messages.msg select '||columns||' from '||tabname_full||'" >/dev/null 2>&1 </dev/null &'
||x'0a'||'cat pipe_data >> '||filename
from
(
select
rtrim(c.tabschema)||'.'||c.tabname as tabname_full
, rtrim(c.tabschema)||'.'||c.tabname||'.csv' as filename
, listagg(c.colname, ', ') as columns
from syscat.tables t
join syscat.columns c on c.tabschema=t.tabschema and c.tabname=t.tabname
where t.tabschema='SYSIBM' and t.type='T'
group by c.tabschema, c.tabname
--fetch first 10 row only
)
;
Run it as follows:
db2 -tf gen_exp_sh.sql
The export shell script exp.sh consists of 3 commands for each table:
* echo command to write a comma separated list of columns to a file
* db2 export command to get table data to a pipe (started in a background)
* simple cat command to read from the pipe and add data to the same file with the columns list
Usage:
You must create the pipe first and source (dot space script notation - it's important) the export script afterwards:
mkfifo pipe_data
db2 connect to mydb ...
. ./exp.sh
rm -f pipe_data
Try to use this great tool: https://www.sql-workbench.eu/. It's universal and you may transfer data between any type of database motors.
I need to store Spanish text in SQL server.
I am using varchar data type for string fields and collation is 'SQL_Latin1_General_CP1_CI_AS'.
Will it support for Spanish text?
Yes u could store in this way :
CREATE TABLE #TM(NAME NVARCHAR(MAX));
INSERT INTO #TM
VALUES(N'El hardware inalámbrico no autorizado se puede introducir fácilmente');
Explanation :
N actually stands for National language character set Which means that you are passing an NCHAR, NVARCHAR or NTEXT value
from Microsoft https://msdn.microsoft.com/en-IN/library/ms186939.aspx
Result :
NAME
El hardware inalámbrico no autorizado se puede introducir fácilmente
I have gz files in a folder. I need only 3 columns from these files, but each line has over 100 of them. At the moment I create a view this way.
drop table MAK_CHARGE_RCR;
create external table MAK_CHARGE_RCR
(LINE string)
STORED as SEQUENCEFILE
LOCATION '/apps/hive/warehouse/mydb.db/file_rcr';
drop view VW_MAK_CHARGE_RCR;
create view VW_MAK_CHARGE_RCR as
Select LINE[57] as CREATE_DATE, LINE[64] as SUBS_KEY, LINE[63] as RC_TERM_NAME
from
(Select split(LINE, '\\|') as LINE
from MAK_CHARGE_RCR) a;
The view has the fields I need. Now I have to do the same, but without CTAS and I am not sure how to go about it. What can I do?
I was told the table must look like this
create external table MAK_CHARGE_RCR
(CREATE_DATE string, SUBS_KEY string, RC_TERM_NAME etc)
I could split the line like this
ROW FORMAT DELIMITED
FIELDS TERMINATED BY '\\|'
but I'll need to list every column. I have another group of files with over 1000 columns. All of them I'll need to list. This just seems a bit excessive, so I wondered if it is possible to do
create external table arstel.MAK_CHARGE_RCR
(split(LINE, '\\|')[57] string,
split(LINE, '\\|')[64] string
etc)
This doesn't work obviously, but maybe there are work arounds?
RegexSerDe
For educational purposes
P.s.
I intend to create an enhanced version of the CSV SerDe that excepts an additional parameter with the positions of the requested columns.
Demo
bash
echo {a..c}{1..100} | xargs -n 100 | tr ' ' '|' | \
hdfs dfs -put - /user/hive/warehouse/mytable/data.txt
hive
create external table mytable
(
col58 string
,col64 string
,col65 string
)
row format serde 'org.apache.hadoop.hive.serde2.RegexSerDe'
with serdeproperties ("input.regex" = "^(?:([^|]*)\\|){58}(?:([^|]*)\\|){6}([^|]*)\\|.*$")
stored as textfile
location '/user/hive/warehouse/mytable'
;
select * from mytable
;
+---------------+---------------+---------------+
| mytable.col58 | mytable.col64 | mytable.col65 |
+---------------+---------------+---------------+
| a58 | a64 | a65 |
| b58 | b64 | b65 |
| c58 | c64 | c65 |
+---------------+---------------+---------------+
Hy, I know many questions has been asked on it already but this is somewhat different.
I have csv file containing millions of records. I tried the following commands to copy from csv to my table i.e.
copy "client_data" from '/home/mike/Desktop/client_data.txt' with delimiter ',' CSV;
BUT the problem arrises as the data in the csv is inconsistent state i.e.
following lines would like charm
12/12/12 20:17:35,304000000,"123","1"
12/12/12 20:17:36,311000000,"123","2"
12/12/12 20:17:36,814000000,"123","2"
12/12/12 20:17:36,814000000,"123","2"
12/12/12 20:17:37,317000000,"123",".1"
12/12/12 20:17:38,863000000,"123","TS"
12/12/12 20:17:39,835000000,"123","2"
12/12/12 20:17:40,337000000,"123","1"
but hundreds of rows are some what like
12/12/12 20:20:03,790000000,"123","1
{'""}__{""'} /""'\
( $AMZA./)#FRIDI
{__}""'{__} /) (\. ,,DON,,"
12/12/12 20:20:30,501000000,"123","INAM NIKALTA NHE HE KITNE SAWALO K JAWB DAY
/G\A\,':/\,':/S\K,':\"
12/12/12 20:22:55,928000000,"123","PAKISTAN KI BUNYAAD
2=QUAID-E-AZAM"
12/12/12 20:22:56,431000000,"123","QUIED E AZAM
MOHAMMAD ALI JINNAH
[KFK FEROZ]"
which are un parseable due to line breaks, commas, invalid characters, etc.
is there any way to parse these and load the data in postgres table in efficient way?
below is the table structure
create table "client_data" (
date_stamp text,
points bigint,
msisdn character varying(13),
data text
)
with (OIDS = false);
alter table "client_data" owner to postgres;
Did it using MySql import option as MySql had more sophisticated and relative easy import approach for importing data.
I have a problem using LOAD DATA INFILE commond .
I created a table using the command below;
temp.executeUpdate("CREATE TABLE Patient (patientID INT AUTO_INCREMENT, name VARCHAR(100),address VARCHAR(150), phone VARCHAR(15), birthdate DATE, PRIMARY KEY (patientID))");
and trying to read from a file using this command;
temp = connect.createStatement();
temp.executeUpdate("LOAD DATA LOCAL INFILE 'patient.txt' INTO TABLE Patient {name,address,phone,birthdate} FIELDS ENCLOSED BY '\"' ");
temp.executeUpdate(" UPDATE Patient SET name=NULL WHERE name= '-' ");
temp.executeUpdate( " UPDATE Patient SET address = NULL WHERE address = '-' ");
temp.executeUpdate(" UPDATE Patient SET phone = NULL WHERE phone = '-' ");
temp.executeUpdate(" UPDATE Patient SET birthdate = NULL WHERE birthdate = '-'");
and my sample text file is this;
"omer" "trabzon" "3253008" 1990-06-10
"ali" "ankara" "2234887" 1999-11-12
However it can't read the first fields and skip to the second ones.So,
second fields are replaced to the first fields.
could you help to get the first fields into the right places?
thanks
add a "\N" before "omer" and before "ali" (at the beginning of each row) for the autoincrement column. I would enclose also the date with quotes
I have just found out that problem was the sequence of the load data file line. the right sequence is
temp.executeUpdate("LOAD DATA LOCAL INFILE 'patient.txt' INTO TABLE Patient FIELDS ENCLOSED BY '\"' {name,address,phone,birthdate}")