I want to upload a csv file into the database via my php script.
The database contains this numbers
part Description
1 test
2 pc
3 monitor
When i upload the csv the code must look if the part already exist in the database, if not he must add him.
for example:
my cvs:
1 test
2 pc
4 monitor
5 keyboard
if i upload this script he must only add the keyboard.
The database contains 8000 products, how can i compare this efficient and fast?
Upload in temp table the csv file the try an insert select
Insert into table as (select * from temptable minus select * from table)
Related
I am working on preparing an SSIS job where I am importing a .CVSV file to OLE DB destination (sql database). We are going to get these files on daily basis. The .CSV file contains records of doctors. Each row represents a doctor. Below image shows how I am able to do this successfully. No problems upto this point.
Here's what I need help with:
If the doctor is no longer active we are going to get the same .CSV file without the record of him/her. How do I check to see if the record is not in .CSV file but it exists in SQL database? I need to update that doctor row in SQL database and update the IsActive field for that row to false.
Naturally, this is psuedo-code.
SELECT DoctorID
FROM DrTable
where NOT EXISTS (select DoctorID from CSVTable where CSVTable.DoctorID=DrTable))
You could do the update in the same statement using:
UPDATE DrTable
Set IsActive = 0
WHERE Doctorid IN ( SELECT DoctorID
FROM DrTable
where NOT EXISTS (select DoctorID from CSVTable where CSVTable.DoctorID=DrTable)))
Apart from Eric's answer, you could also use the 'merge' function provided by SSIS by default.
You would have to make sure that your that your flat file source and the database are both of the same data type with respect to all the columns and they are both sorted over the key that you are using to join them.
I need to copy the contents of a table from one database to another database with an identical table which is currently empty.
I plan to dump the table data from the old table and then simply import it to the empty table in the new database. However, I ran into some behavior I don't understand using pg_dump.
I try to dump the table data to a file with the command:
pg_dump -a -t '"my_table"' my_database > /tmp/my_table.sql
This works, but I only get 8 records and there are over 1000 records in the table if I view the table like so:
SELECT * FROM my_table;
So, I tried to use the COPY command to generate a .csv file and I see similar behavior:
COPY my_table TO '/tmp/my_table.csv' WITH CSV HEADER;
I get the same 8 records as pg_dump. But, with:
COPY (SELECT * FROM my_table) TO '/tmp/my_table.csv' WITH CSV HEADER;
I get all 1266 records.
I would assume these commands should all return the same data, but obviously, I'm wrong. What is the difference?
Is it possible that my_table is part of an inheritance hierarchy? I ask because http://www.postgresql.org/docs/9.0/interactive/sql-copy.html#AEN58984 has this:
COPY only deals with the specific table named; it does not copy data to or from child tables. Thus for example COPY table TO shows the same data as SELECT * FROM ONLY table. But COPY (SELECT * FROM table) TO ... can be used to dump all of the data in an inheritance hierarchy.
You should be able to check by running:
SELECT * FROM ONLY my_table;
If that returns just the 8 records then we're on the right track, and we just need to find the child tables (for which How to find child tables that inherit from another table in PSQL will be helpful).
If not then I'm not sure - I wondered if maybe Rules or Triggers were getting involved, but I can't see how at the moment. Still, maybe it gives someone else an idea...?
I am new to SQL and SSIS work. I'd like to dynamically export SQL datatable to a CSV file everysec using SSIS. one of column name RECIPE STATUS, value is 1 OR 2 (int) (1 is new Recipe, 2 is old recipe which is existed), Another column name is RECIPE NAME, value is AAABBB (varchar) some more columns with values In Database table, only one row data avaialble, it will be changed ev sec. So we are trying to export it to csv file to log different/unique RECIPENAMES and data for analysis.
Table Schema is
SELECT TOP 5 [RowID]
,[RowInsertTime]
,[TransId]
,[RecipeStatus]
,[RecipeName]
,[RecipeTagName]
,[Value]
,[ReadStatus]
,[sData1]
,[sData2]
,[sData3]
,[nData1]
,[nData2]
,[nData3]
FROM [MES].[dbo].[MESWrite_RecipeData]
While exporting, based on RECIPE STATUS value if 1, then Insert/create a new row in CSV and export. (Its a simple insert TSQL statement, insert into csvfile (values from databsetable)
if value 2 means RECIPENAME value is AAABBB existed in CSV already. so find and update other columns values. so in this case don't create a new row in csv. (i can say update all othercolumns with values from database table to csv where RECIPENAME="AAABBB") like Update Query in T-SQL
. finally if we have to send this SSIS package to a customer, it can be executable file. or how to make secured? Please help me ..
I have 12 columns with +/- 2000 rows in a sqlite DB.
Now I want to add a 13th column with the same amount of rows.
If I import the text from a cvs file it will add this after the existing rows (now I have a 4000 row table)
How can I avoid adding it underneath these rows?
Do I need to create a script to run trough each row of the table and add the text from the cvs file for each row?
If you have the code that imported the original data, and if the data has not changed in the meantime, you could just drop the table and reimport it.
Otherwise, you indeed have to create a script that looks up the corresponding record in the table and updates it.
You could also import the new data into a temporary table, and then copy the values over with a command like this:
UPDATE MyTable
SET NewColumn = (SELECT NewColumn
FROM TempTable
WHERE ID = MyTable.ID)
I ended up using Razor SQL great program.
http://www.razorsql.com/
I want to import data in the form of csv file into a table.[using Oracle SQL developer].I have such hundred files and each has about 50 columns.
From the wiki of SQL*Loader (http://www.orafaq.com/wiki/SQL*Loader_FAQ)
load data
infile 'c:\data\mydata.csv'
into table emp
fields terminated by "," optionally enclosed by '"'
( empno, empname, sal, deptno ) //these are the columns headers
What i don't want to do is list down all the column headers.I just want all the enteries in the csv file to be assigned to members in the tables in the order in which they appear.
Moreover after all think i want to automate it for all the 100 files.
You should write down the columns (and their type optionally) so as to assign the values of your csv file to each column. You should do this because the order of the columns in the table in your Oracle Database is not known in the script.
After you write the columns in the order they appear in your csv files, you can automate this script for all of your files by typing:
infile *.csv
You can try oracle csv loader. It automatically creates the table and the controlfile based on the csv content and loads the csv into an oracle table using sql loader.
An alternative to sqlldr that does what you are looking for is the LOAD command in SQLcl. It simply matches header row in the csv to the table and loads it. However this is not as performant nor as much control as sqlldr.
LOAD [schema.]table_name[#db_link] file_name
Here's the full help for it.
sql klrice/klrice
...
KLRICE#xe>help load
LOAD
-----
Loads a comma separated value (csv) file into a table.
The first row of the file must be a header row. The columns in the header row must match the columns defined on the table.
The columns must be delimited by a comma and may optionally be enclosed in double quotes.
Lines can be terminated with standard line terminators for windows, unix or mac.
File must be encoded UTF8.
The load is processed with 50 rows per batch.
If AUTOCOMMIT is set in SQLCL, a commit is done every 10 batches.
The load is terminated if more than 50 errors are found.
LOAD [schema.]table_name[#db_link] file_name
KLRICE#xe>
Example from a git repo I have at https://github.com/krisrice/maxmind-oracledb
SQL> drop table geo_lite_asn;
Table GEO_LITE_ASN dropped.
SQL> create table geo_lite_asn (
2 "network" varchar2(32),
3 "autonomous_system_number" number,
4 "autonomous_system_organization" varchar2(200))
5 /
Table GEO_LITE_ASN created.
SQL> load geo_lite_asn GeoLite2-ASN-CSV_20180130/GeoLite2-ASN-Blocks-IPv4.csv
--Number of rows processed: 397,040
--Number of rows in error: 0
0 - SUCCESS: Load processed without errors
SQL>