I have a problem using LOAD DATA INFILE commond .
I created a table using the command below;
temp.executeUpdate("CREATE TABLE Patient (patientID INT AUTO_INCREMENT, name VARCHAR(100),address VARCHAR(150), phone VARCHAR(15), birthdate DATE, PRIMARY KEY (patientID))");
and trying to read from a file using this command;
temp = connect.createStatement();
temp.executeUpdate("LOAD DATA LOCAL INFILE 'patient.txt' INTO TABLE Patient {name,address,phone,birthdate} FIELDS ENCLOSED BY '\"' ");
temp.executeUpdate(" UPDATE Patient SET name=NULL WHERE name= '-' ");
temp.executeUpdate( " UPDATE Patient SET address = NULL WHERE address = '-' ");
temp.executeUpdate(" UPDATE Patient SET phone = NULL WHERE phone = '-' ");
temp.executeUpdate(" UPDATE Patient SET birthdate = NULL WHERE birthdate = '-'");
and my sample text file is this;
"omer" "trabzon" "3253008" 1990-06-10
"ali" "ankara" "2234887" 1999-11-12
However it can't read the first fields and skip to the second ones.So,
second fields are replaced to the first fields.
could you help to get the first fields into the right places?
thanks
add a "\N" before "omer" and before "ali" (at the beginning of each row) for the autoincrement column. I would enclose also the date with quotes
I have just found out that problem was the sequence of the load data file line. the right sequence is
temp.executeUpdate("LOAD DATA LOCAL INFILE 'patient.txt' INTO TABLE Patient FIELDS ENCLOSED BY '\"' {name,address,phone,birthdate}")
Related
I had exported a bunch of tables (>30) as CSV files from MySQL database using phpMyAdmin. These CSV file contains NULL values like:
"id","sourceType","name","website","location"
"1","non-commercial","John Doe",NULL,"California"
I imported many such csv to a PostgreSQL database with TablePlus. However, the NULL values in the columns are actually appearing as text rather than null.
When my application fetches the data from these columns it actually retrieves the text 'NULL' rather than a null value.
Also SQL command with IS NULL does not retrieve these rows probably because they are identified as text rather than null values.
Is there a SQL command I can do to convert all text NULL values in all the tables to actual NULL values? This would be the easiest way to avoid re-importing all the tables.
PostgreSQL's COPY command has the NULL 'some_string' option that allows to specify any string as NULL value: https://www.postgresql.org/docs/current/sql-copy.html
This would of course require re-importing all your tables.
Example with your data:
The CSV:
"id","sourceType","name","website","location"
"1","non-commercial","John Doe",NULL,"California"
"2","non-commercial","John Doe",NULL,"California"
The table:
CREATE TABLE import_with_null (id integer, source_type varchar(50), name varchar(50), website varchar(50), location varchar(50));
The COPY statement:
COPY import_with_null (id, source_type, name, website, location) from '/tmp/import_with_NULL.csv' WITH (FORMAT CSV, NULL 'NULL', HEADER);
Test of the correct import of NULL strings as SQL NULL:
SELECT * FROM import_with_null WHERE website IS NULL;
id | source_type | name | website | location
----+----------------+----------+---------+------------
1 | non-commercial | John Doe | | California
2 | non-commercial | John Doe | | California
(2 rows)
The important part that transforms NULL strings into SQL NULL values is NULL 'NULL' and could be any other value NULL 'whatever string'.
UPDATE For whoever comes here looking for a solution
See answers for two potential solutions
One of the solutions provides a SQL COPY method which must be performed before the import itself. The solution is provided by Michal T and marked as accepted answer is the better way to prevent this from happening in the first place.
My solution below uses a script in my application (Built in Laravel/PHP) which can be done after the import is already done.
Note- See the comments in the code and you could potentially figure out a similar solution in other languages/frameworks.
Thanks to #BjarniRagnarsson suggestion in the comments above, I came up with a short PHP Laravel script to perform update queries on all columns (which are of type 'string' or 'text') to replace the 'NULL' text with NULL values.
public function convertNULLStringToNULL()
{
$tables = DB::connection()->getDoctrineSchemaManager()->listTableNames(); //Get list of all tables
$results = []; // an array to store the output results
foreach ($tables as $table) { // Loop through each table
$columnNames = DB::getSchemaBuilder()->getColumnListing($table); //Get list of all columns
$columnResults = []; // array to store the results per column
foreach ($columnNames as $column) { Loop through each column
$columnType = DB::getSchemaBuilder()->getColumnType($table, $column); // Get the column type
if (
$columnType == 'string' || //check if column type is string or text
$columnType == 'text'
) {
$query = "update " . $table . " set \"" . $column . "\"=NULL where \"" . $column . "\"='NULL'"; //Build the update query as mentioned in comments above
$r = DB::update($query); //perform the update query
array_push($columnResults, [
$column => $r
]); //Push the column Results
}
}
array_push($results, [
$table => $columnResults
]); // push the table results
}
dd($results); //Output the results
}
Note I was using Laravel 8 for this.
With the below code I can retrieve the content of the internal table t_t005e, however when put into the field wa_upload-region, only the first column of the data is retrieved, however I want to retrieve the third column data.
TYPES: BEGIN OF ty_upload,
" ...
region TYPE regio,
" ...
END OF ty_upload.
DATA: wa_upload TYPE ty_upload,
t_t005e TYPE STANDARD TABLE OF t005e.
READ TABLE t_t005e
INTO wa_upload-region
WITH KEY land1 = 'GB'
regio = 'YK'
counc = ''.
As a result, I have created a work area wa_t005e, with the same type as the lines of t_t005e.
I want to first read the internal table t_t005e into the work area wa_t005e, then to the field wa_upload-region.
Following is my work in progress:
DATA: wa_t005e TYPE t005e.
LOOP AT t_t005e INTO wa_t005e.
ASSIGN COMPONENT wa_t005e-regio OF STRUCTURE
wa_t005e TO <wa_upload-region>.
ENDLOOP.
How to get the data of wa_t005e-regio into the field wa_upload-region?
There is no way of reading the value of only one column from a table directly into one field of a structure, at least in systems pre-7.40. If you do have a 7.40 system, you can use a "table expression" like this:
TRY.
wa_upload-region = t_t005e[ land1 = 'GB' regio = 'YK' counc = '' ]-regio.
CATCH cx_sy_itab_line_not_found.
ENDTRY.
In older system, you will have to read the whole table line into a structure, then you can just take the field from it, like this:
READ TABLE t_t005e INTO wa_t005e WITH KEY land1 = 'GB' regio = 'YK' counc = ''.
wa_upload-region = wa_t005e-regio.
If you want to use ASSIGN and the like, you can do that too. First you would read the table line into a structure again (in this case a field symbol to stay in theme). Then assign the needed component/field of the structure to a single-value field symbol.
DATA: t_upload TYPE STANDARD TABLE OF ty_upload,
t_t005e TYPE STANDARD TABLE OF t005e.
FIELD-SYMBOLS: <fs_upload> TYPE ty_upload,
<fs_t005e> TYPE t005e,
<region> TYPE regio. " or type any
SELECT *
FROM t005e
INTO CORRESPONDING FIELDS OF TABLE t_t005e.
READ TABLE t_t005e ASSIGNING <fs_t005e> WITH KEY land1 = 'GB' regio = 'YK' counc = ''.
ASSIGN COMPONENT 'REGIO' OF STRUCTURE <fs_t005e> TO <region>. " <---
*Other option: number of column
*ASSIGN COMPONENT 3 OF STRUCTURE <fs_t005e> TO <region>.
APPEND INITIAL LINE TO t_upload ASSIGNING <fs_upload>.
<fs_upload>-region = <region>.
WRITE <fs_upload>-region.
But is reading only one entry from the table really what you want to do? You didn't specify all keys of t005e in the READ statement. It would only select the first line that fits.
I have a query string which is
var projectid = sqlConn.Query<int>("insert into project (name,customer_id, service_id,user_id) values (#name,#customerid,#serviceid,#userid);SELECT SCOPE_IDENTITY(); ", new { name = Name, customerid = Customer, serviceid = Service, userid = userId }).Single();
where name is a nvarchar field. Now I know I need to use 'N' character and I have tried the followings
'N#name'
N'#name'
but it doesn't seem to work. Can anyone help me with this please?
You don't need to use N prefix with variables.
It is needed only with explicitly typed strings (string constants) like insert into ... values(N'some text', ...)
If your name variable contains unicode string - then no special actions are required.
I am new to apache hive and was running queries on sample data which is saved in a csv file as below:
0195153448;"Classical Mythology";"Mark P. O. Morford";"2002";"Oxford University Press";"//images.amazon.com/images/P/0195153448.01.THUMBZZZ.jpg";"http://images.amazon.com/images/P/0195153448.01.MZZZZZZZ.jpg";"images.amazon.com/images/P/0195153448.01.LZZZZZZZ.jpg"
and the table which i created is of form
hive> describe book;
OK
isbn bigint
title string
author string
year string
publ string
img1 string
img2 string
img3 string
Time taken: 0.085 seconds, Fetched: 8 row(s)
and the script which I used to create the table is:
create table book(isbn int,title string,author string, year string,publ string,img1 string,img2 string,img3 string) row format delimited fields terminated by '\;' lines terminated by '\n' location 'path';
When I try to retrieve the data from the table by using the following query:
select *from book limit 1;
I get the following result:
NULL "Classical Mythology" "Mark P. O. Morford" "2002" "Oxford University Press" "http://images.amazon.com/images/P/0195153448.01.THUMBZZZ.jpg" "images.amazon.com/images/P/0195153448.01.MZZZZZZZ.jpg" "images.amazon.com/images/P/0195153448.01.LZZZZZZZ.jpg"
Even though I specify the first column type as int or bigint the data into the table is getting loaded as NULL.
I tried searching on the internet and could figure out that I have to specify the row delimiter. I used that too but no change in the data from the table.
Is there anything that I am making a mistake... Please help.
I have Ticket table that has some columns like this :
ID : int
Body : nvarchar
Type : int
I have many rows where the Body column has value like this :
IPAddress = sometext, ComputerName = sometext , GetID = sometext, CustomerName=sometext-sometext , PharmacyCode = 13162900
I want update all rows' Type column where the Body column has at least five of the following keys:
IPAddress, ComputerName, GetID, CustomerName, PharmacyCode
You could do it with a simple update statement like that
UPDATE Ticket
SET Type = 4
WHERE Body LIKE '%IPAddress%'
and Body LIKE '%ComputerName%'
and Body LIKE '%GetID%'
and Body LIKE '%CustomerName%'
and Body LIKE '%PharmacyCode%'
if you know the 'keys' are always in the same order you could concatenate the LIKE conditions like so
UPDATE Ticket
SET Type = 4
WHERE Body LIKE '%IPAddress%ComputerName%GetID%CustomerName%PharmacyCode%'
If you have the possibility to change the data model it would be much better to explode this key & value column into an own table and link it back to this table as it is done in a proper relational model.
If you could calculate number of key value pair by number of = present in your string you could use this query
Update tblname set col=val where len(colname) - len(replace(colname,'=','')>5
The where part actually gives number of equal signs present in your string.