I currently import a file dynamically using SSIS into a SQL table, part of the process imports the column headers from the source file into the first row of a table (table 1 below).
I then want to compare the headers from table 1 with table 2 which has static columns, and most importantly highlight any column headers from table 1 that do not exist in table 2.
So for example Table 1 looks like this:
Column 1
Column 2
CustomerID
CustomerName
And table 2 has static column headers like the following:
CustomerID
CustomerName
1
Joe
2
Daniel
So basically when I load a file and a new column header is added (in this example lets say CustomerLocation is added) and loaded into table 1, I want a SQL query to compare the 2 tables and highlight that the column CustomerLocation is missing/does not exist in table 2 currently.
I was thinking along the lines of using the sys tables to compare.
You certainly could use sys.columns to return your static columns from Table2 and compare them to the dynamic columns in Table1 and use UNPIVOT on a select of your first row.
I have found that it was far easier to wrap this all in a T-SQL block and insert to two lists into temp tables before comparing due to data type conflicts (probably be solved by using CAST)
BEGIN
DECLARE #table1 TABLE (colname VARCHAR(MAX))
DECLARE #table2 TABLE (colname VARCHAR(MAX))
INSERT INTO #table1 SELECT COLNAME FROM (SELECT a, b, c FROM TABLE1 WHERE...first row condition) a UNPIVOT (COLNAME FOR COLS IN ([a],[b],[c])) a
INSERT INTO #table2 SELECT CAST (name AS NVARCHAR(100)) name FROM sys.columns WHERE object_id = OBJECT_ID('TABLE2')
SELECT a.colname cols1, b.colname cols2
FROM #table2 a
FULL OUTER JOIN #table1 b ON (a.colname = b.colname)
END
You can easily change the final select to return what you want
Related
I am working on a project whos purpose is to rename all table names and column names in the sql database from one language to another. I have gather all the local table names into a table ltbl_TableNames and want to add all the columns of these tables into a table called ltbl_TableColumns.
I also want every table column to have a link to their table name. For example, the table 'Sales' have a column named 'Sum'. The table 'Sales' has the ID '10000'. I want to add that ID in a column named 'TableName_ID' for linking purposes. Is this possible to do without a lot of hassle?
Disclaimer: I am thinking about the renaming process. I only want to gather the column names with link to their parent table name.
Thanks in advance for answers.
You may use sys.tables for getting list of table name in your database.
Similarly for column name you may use information_schema.columns as this will give records with table name.
From the above 2 records you can easily make you required result.
;WITH CTABLE AS
( SELECT * FROM SYS.TABLES WHERE TYPE_DESC = 'USER_TABLE' )
, COLUMNNAME AS
( SELECT * FROM INFORMATION_SCHEMA.COLUMNS )
SELECT * INTO NEWTABLE FROM
(SELECT CTABLE.NAME AS TABLENAME , COLUMNNAME.COLUMN_NAME, COLUMNNAME.COLUMN_NAME + '_' + CAST(CTABLE.OBJECT_ID AS VARCHAR(15)) AS NEW_COLUMNNAME
FROM CTABLE INNER JOIN COLUMNNAME ON CTABLE.NAME = COLUMNNAME.TABLE_NAME ) AS D
You may try this for your result.
I am using Postgresql db. I have data in two tables. Table A has 10 records and Table B 5 records.
I would like to copy Table A data to Table B but only copy the new entries (5 records) and ignore the duplicates/already existing data
I would like to copy data from Table A to Table B where Table B will have 10 records (5 old records + 5 new records from Table A)
Can you please help me as to how can this be done?
Assuming id is your primary key, and table structures are identical(both table has common columns as number of columns and data type respectively), use not exists :
insert into TableB
select *
from TableA a
where not exists ( select 0 from TableB b where b.id = a.id )
If you are looking to copy rows unique to A that are not in B then you can use INSERT...SELECT. The SELECT statement should use the union operator EXCEPT:
INSERT INTO B (column)
SELECT column FROM A
EXCEPT
SELECT column FROM B;
EXCEPT (https://www.postgresql.org/docs/current/queries-union.html) compares the two result sets and will return the distinct rows present in result A but not in B, then supply these values to INSERT. For this to work both the columns and respective datatypes must match in both SELECT queries and your INSERT.
INSERT INTO Table_A
SELECT *
FROM Table_B
ON CONFLICT DO NOTHING
Here, the conflict will be taken based on your primary key.
I have a Redshift table with the following column
How can I extract the value starting by cat_ from this column please (there is only one for each row and at different position in the array)?
I want to get those results:
cat_incident
cat_feature_missing
cat_duplicated_request
Thanks!
There is no easy way to extract multiple values from within one column in SQL (or at least not in the SQL used by Redshift).
You could write a User-Defined Function (UDF) that returns a string containing those values, separated by newlines. Whether this is acceptable depends on what you wish to do with the output (eg JOIN against it).
Another option is to pre-process the data before it is loaded into Redshift, to put this information in a separate one-to-many table, with each value in its own row. It would then be trivial to return this information.
You can do this using tally table (table with numbers). Check this link on information how to create this table: http://www.sqlservercentral.com/articles/T-SQL/62867/
Here is example how you would use it. In real life you should replace temporary #tally table with a permanent one.
--create sample table with data
create table #a (tags varchar(500));
insert into #a
select 'blah,cat_incident,mcr_close_ticket'
union
select 'blah-blah,cat_feature_missing,cat_duplicated_request';
--create tally table
create table #tally(n int);
insert into #tally
select 1
union select 2
union select 3
union select 4
union select 5
;
--get tags
select * from
(
select TRIM(SPLIT_PART(a.tags, ',', t.n)) AS single_tag
from #tally t
inner join #a a ON t.n <= REGEXP_COUNT(a.tags, ',') + 1 and n<1000
)
where single_tag like 'cat%'
;
Thanks!
In the end I managed to do it with the following query:
SELECT SUBSTRING(SUBSTRING(tags, charindex('cat_', tags), len(tags)), 0, charindex(',', SUBSTRING(tags, charindex('cat_', tags), len(tags)))) tags
FROM table
Due to a certain requirement, I need to create two temp tables in Stored Procedure, after processing some data into them, I need to combine the two temp tables to show as one result set and generate into excel. So I'm thinking to use UNION when I want to show the final result set.
The issue is, the first temp table (Table A) is fixed to 20 columns, and the second temp table has 50 columns (Table B). My plan is, before processing data for Table A, I want to add 30 nullable columns and insert data for first 20 columns, and the rest is all NULL
After I process the data for Table B, I use UNION to combine Table A & B so that they will show as one result set.
What I can think of right now is to hard code some columns that are destined to have null values when I declare the temp table:
Declare #tmpTableA table (
....
ProcessDate datetime,
Mode int,
Col21 varchar(10)
Col22 varchar(10)
....
Col50 varchar(50)
)
When I insert data into Table A, I have to manually add null from Col21 onwards
Insert into(.... Col21, Col22, Col23....)
Values (.... NULL, NULL, NULL....)
After I complete processing data for Table A & B, I use UNION to merge Table A and B
Select *....Col49,Col50 From Table A
Union
Select *....CompleteDate,ContactPerson From Table B
Instead of hard-coding Col21 to Col50 into Table A, is there any elegant way to achieve that like using while loop to dynamically add N number of columns into Table A?
EDIT:
According to latest requirement, Table B has not only 50 columns but 100 columns! I really need a way to dynamically loop those columns rather than hard-coding for over 80 columns
I think you can just do
select * into #tableA from #tableB where 1=2
with this both tables will have same columns
You don't need to add columns to table A, just add 30 NULLs to select from Table A.
Select *,NULL,...,NULL,NULL From Table A
Union
Select * From Table B
You could add aliases to make the result a bit cleaner
Select *,...,NULL CompleteDate, NULL ContactPerson From Table A
Union
Select * From Table B
I currently have only one column in a table. Each row contains the exact same amount of characters. I need a way to be able to split this one column into multiple columns. The table has over 1.1 million rows, so efficiency in code is pretty important. All of the rows are set up with the same structure, meaning the first 5 characters are always dedicated for firstname, the next 6 are always for lastname for example. I know how to take the left most characters using:
insert into table2
select left(column1,2), substring(column1, 3, len(column1) ) from table1
The problem with the above, is that 1) it creates a new table instead of modifying the existing. And 2) Once I have the first set of characters split into a column I don't know how to grab the next set.
Below is an example of how I would like it to work:
Current
Column 1
abcdefghijklmnop
qrstuvwxyz123456
What I would like:
Column 1 ----------- Column 2 --------- Column 3
abcdefg--------------hij----------------klmnop
qrstuvw--------------xyz----------------123456
picture below:
I can usually fix this through how I important the data in SQL. However the txt file that I use to import the data has no way to dilimate the columns correctly so it has to all be brought in as one column.
Here is the exact scenario that you have mentioned here.
DROP TABLE dbo.table1
DROP TABLE dbo.table2
CREATE TABLE dbo.table1
(column1 VARCHAR(100))
CREATE TABLE dbo.table2
(column1 VARCHAR(100),
column2 VARCHAR(100),
column3 VARCHAR(100)
)
INSERT INTO dbo.table1
VALUES ('abcdefghijklmnop'),('qrstuvwxyz123456')
INSERT INTO dbo.table2
SELECT LEFT(column1,7) AS column1,SUBSTRING(column1,8,3) column2,RIGHT(column1,6) column3
FROM dbo.table1
SELECT *
FROM table1
SELECT *
FROM table2
this should help
select 'dgfsrsdgsdghshdsdfhzhs' column1 into #table_name
ALTER TABLE #table_name
ADD firstname varchar(5) ,
lastname varchar(6);
update #table_name
set firstname = SUBSTRING (column1,1,5),
lastname = SUBSTRING (column1,6,6)
select * from #table_name
assuming that the modification has to be done in the same table and SUBSTRING startposition and length are taken according the output that is being expected below code will do the needful
UPDATE table1
SET column1=SUBSTRING(c1,1,8),
column2=SUBSTRING(c1,8,3),
column3=SUBSTRING(c1,11,7)