My source is a CSV file which has column names and its related data in rows. How to do manual mapping rows to columns in SQL. Below is the file structure.
CSV file
EmpNo
ColumnNames
ColumnValues
1
EmpName
'John'
1
EmpDOB
'08/30/1985'
1
EmpDesignation
'DBA'
Table : EmployeeDetails
EmpNo
EmpName
EmpDOB
EmpDesignation
1
John
08/30/1985
DBA
You can use openrowset to get data from csv file. Here an example that could help you.
INSERT INTO myTable (EmpNo, EmpName, EmpDesignation)
SELECT EmpNo, ColumnValues, ColumnNames
FROM OPENROWSET('BULK', 'C:\path\to\myFile.csv',
FORMATFILE = 'C:\path\to\myFormatFile.fmt') AS rows;
you have to improve columns declared in insert into table and select to matching csv columns with table columns.
Related
We have a pl sql procedure in which we get xml data with multiple records, first gets stored in temp table, by verifying or after doing some checks on the received xml in a loop we are inserting into another table which is the final table, some times few records are not getting pushed to final table for some reason,
now in the same plsql code at the end of loop, we need to recheck is all the rows from the temp table say table A got inserted into the other table say B.
Ex:
Employee Records with columns in table A
<Employee>
Empid, empname, empdesg, empaddr,empemail, empMobNo
</Employee>
Employee Records with columns in table B
<Employee>
EmpNo, empFirstName, EmpLevel, empAddress, empContactDetails, empPhNum
</Employee>
Now we have got 30 employee records from source to plsql which got stored in Table A, but for some reasons only 27 got inserted into Table B, need to compare for missing records in Table B and re insert them into Table B from table A based on empid/empNo.
Please suggest how to do this.
Thi is the idea:
INSERT INTO B
SELECT * FROM A WHERE empid IN
(
SELECT empid FROM A
MINUS
SELECT empNo FROM B
);
I'm using SQL Server and I have a column called "Full Name" that I pulled from a separate table. I have a column for First Name & Last name (among many other columns).
Here is the code that I found and used and it worked:
SELECT Full_Name,
LEFT(Full_Name,CHARINDEX(' ',Full_Name )-1) AS First_Name,
RIGHT(Full_Name, LEN(Full_Name)- CHARINDEX(' ', Full_Name)) AS Last_Name
FROM table_name
That extracted the name, but upon viewing the entire table using this command:
SELECT * FROM table_name
I don't see it? Is there a method that I can use to insert the data into the First_name and Last_name columns without adding any additional Rows
Thank you !
Perhaps creating a view wil be sufficient:
CREATE VIEW table_name_v
SELECT Full_Name,
LEFT(Full_Name,CHARINDEX(' ',Full_Name )-1) AS First_Name,
RIGHT(Full_Name, LEN(Full_Name)- CHARINDEX(' ', Full_Name)) AS Last_Name
FROM table_name
Then instead of using the table name in subsequent queries you use the view instead
SELECT * FROM table_name_v
The problem you face otherwise is that if you add first and last name columns to this table that you now end-up with 3 columns to maintain, or that you may need to replace the data entry screen(s) that use full_name to start using first and last name columns instead.
If you really do want to proceed by adding the columns, you could try using "computed columns" which would avoid the necessity to change data entry screens etc.
CREATE TABLE mytable(
full_name VARCHAR(15) NOT NULL
);
INSERT INTO mytable(full_name) VALUES ('fred flintstone');
select * from mytable
full_name
fred flintstone
alter table mytable
add
first_name as LEFT(Full_Name,CHARINDEX(' ',Full_Name )-1)
, Last_Name as RIGHT(Full_Name, LEN(Full_Name)- CHARINDEX(' ', Full_Name))
select * from mytable
full_name
first_name
Last_Name
fred flintstone
fred
flintstone
db<>fiddle for computed columns here
If you realy do want to store these 2 separate columns (rather than computing them) then you need to add the columns to your table and run an update statement to populate them as follows:
alter table mytable
add
first_name varchar(100)
, last_name varchar(100)
update mytable
set
first_name = LEFT(Full_Name,CHARINDEX(' ',Full_Name )-1)
, Last_Name = RIGHT(Full_Name, LEN(Full_Name)- CHARINDEX(' ', Full_Name))
db<>fiddle for adding columns and updating here
When I tried to insert into a Partiotioned table I am getting the bellow error:
SemanticException [Error 10044]: Line 1:23 Cannot insert into target table because column number/types are different : Table insclause-0 has 6 columns, and the 3 columns are partitioned and we not required any filters we have to dump/store from non partitioned table to partitioned table.
My table:
Source:
id name salary dep
1 sai 1000 sales
2 syam 2000 hr
3 sundar 3000 bank
Target:
id name salary dep
1 sai 1000 sales
2 syam 2000 hr
3 sundar 3000 bank
partition (name string, dep string)
Please let me how to copy from source to target
tried below way.
insert into target_partitioned_table partition(name,dep) select id from source_table;
You should list all columns in the select, partition columns should be the last and in the same order. Order of columns matters.
Check the table DDL. If it is partitioned by Name and Dep, then partition columns should be the last: id, salary, name, dep. If columns ordered like in your question, it does not look like the table is partitioned by (Name, Dep), or the order of columns is wrong in the file or in your data example. Insert columns in the same order which DESCRIBE command returns.
Query should contain all columns in exactly the same order.
For static partition load you do not need partition columns in the select, values are static in the partition spec:
insert into table target_partitioned_table partition(name='Some Name',dep='Sales')
select id, salary from source_table;
For dynamic partition load (partitions are taken from the dataset and should be in the select in the same order):
set hive.exec.dynamic.partition=true;
set hive.exec.dynamic.partition.mode=nonstrict;
insert into table target_partitioned_table partition(name,dep)
select id, salary, name, dep from source_table;
The below is a simplified version of the problem I am facing
Let's say I have an employee and a department table in Hive. My goal is to load the data from these 2 tables into a 3rd one below. However, the 3rd table has a few dummy columns set to null and will not be filled by data from either of the employee or department tables. Is it possible to still load the employee and department data and just set the other fields to null?
Employee table(id,first_name,last_name,age,department_id,salary)
1,John,Smith,23,1,40000
2,Bob,Wilson,25,1,45000
3,Fred,Krug,37,2,75000
4,Jeremy,Fisher,41,3,110000
Department table(id,name)
1,Sales
2,IT
3,Marketing
End result(dummy_column0,employeeID,first_name,last_name,age,salary,department_name,dummy_column1)
null,1,John,Smith,23,40000,Sales,null
null,2,Bob,Wilson,25,45000,Sales,null
null,3,Fred,Krug,37,75000,IT,null
null,4,Jeremy,Fisher,41,110000,Marketing,null
Question is given the schema of the end result, how do I load the rest of the non-null data into the 3rd table? Any help would be much appreciated! The end results table already exists at this point so I cannot just recreate it from scratch
Yes. Hive doesn't care of the column names. Its just position of the columns that matter the most. you just have to structure your query in a way so dummy columns have nulls.
insert overwrite table tablename
select null, employeeID, first_name,last_name, age, salary, dept.deptName, null
from employee e join dept d on e.dept_id = d.dept_id;
I want to transfer data of table A,column emp_name to Table B with column name EMP_NAME.
I want that EMP_ID column of Table B to be equal to row_id.How this can be done in sql
or ssis..?
what have you tried?
You will need two connections, one for each DB and one data flow component which will have a OleDBSource and an OleDBDestination component inside.
On the OleDBSource you can select your connection and write your query and then you drag the green arrow to the OleDBDestination. Double click the OleDBDestination select destination connection and table and click on mapping.
Should be it
SQL:
SELECT ROW_NUMBER() OVER(ORDER BY emp_name DESC) AS 'emp_id on table B', emp_name
FROM table
The SQL is:
INSERT INTO
TableB (EMP_ID, EMP_NAME)
SELECT
A.row_id
A.emp_name
FROM
TableA AS A
but for this to work EMP_ID on TableB must not be defined as an IDENTITY column (assuming you are in SQL Server).