I have a table with names of other tables. How can I create a query that gets the table from the values?
For example, if my table has the values:
tables
------
users
users2
users3
I want to create Dynamic SQL function that knows how to take the values (i.e. the table names) from the reference table and then select * from that named value.
If you just want a one time script you can run the query:
select 'select * from ' || table_name || ';' cmd
from user_tables;
And then run the output
Related
I have a table that has a column called "table_name" of type string and a column called "keys" of type array. I need to write an SQL procedure/queries/something that generates tables called "table_name" and each element in the "keys" column is column name. Any thoughts on how to begin this?
For example,
table_name
keys
table_1
['A','B','C']
table_2
['D','E']
would generate table_1:
A
B
C
and table_2:
D
E
Thanks!
You'll need two steps.
Dynamically generate the CREATE TABLE statement form this data
Execute that statement.
Step 1 will look something like:
SELECT 'CREATE TABLE '|| table_name || '(' || array_to_string(keys, ' INT, ') || ");" as create_table_statement
FROM yourtable;
Each row outputted should contain a single column with your create table statement. Note that I made an assumption about your data types as those aren't present in your table or array.
Step 2 is up to you. Copy/paste into a client and execute or do this in a proc or outside scripting language.
Let's say that I have table with A LOT of columns. I have one column with primary key that has autoincrement set to 1. I want to insert a new row and in this new row I have following requirements:
The row must have generated ID
All non-specified columns have to be copied from row with id='9999'
I have to be able to set some values of columns by hand (for example columns name and age
I have tried:
Insert Into demo_table
Select * From demo_table Where id='9999';
However, I get this error:
An explicit value for the identity column in table 'demo_table' can only be specified when a column list is used and IDENTITY_INSERT is ON.
What do I need:
I want to duplicate a row -> let the id be set by database (I have PK and autoincrement configured) -> set some columns by hand -> have other column's values duplicated, without specifying column names (as I have a lot of columns and their names could change in future.)
Form of solution:
I would prefer if I was able to achive this using only one query. If necessary, I have stored procedures available.
My question:
Is this even possible? How could I achive such query/procedure?
There is a way to build sql query by table schema:
USE <databaseName>
DECLARE
#SourceTableName NVARCHAR(255) = <TableName>,
#SqlQuery NVARCHAR(MAX)
IF OBJECT_ID('tempdb.dbo.#IdentityCols', 'U') IS NOT NULL
DROP TABLE #IdentityCols;
CREATE TABLE #IdentityCols(
ColumnName NVARCHAR(255)
)
INSERT INTO #IdentityCols
SELECT
--TABLE_NAME,
COLUMN_NAME
FROM
INFORMATION_SCHEMA.COLUMNS
WHERE
COLUMNPROPERTY(object_id(TABLE_SCHEMA+'.'+TABLE_NAME), COLUMN_NAME, 'IsIdentity') = 1 AND TABLE_NAME = #SourceTableName
UNION
SELECT
--o.name,
c.name
FROM
sys.objects o inner join
sys.columns c on o.object_id = c.object_id
WHERE
c.is_identity = 1 AND o.name = #SourceTableName
--STRING_AGG in SQL SERVER 2017 and greater. As aproach for early versions is cursor or loop
SELECT #SqlQuery = 'SELECT ' + STRING_AGG(COLUMN_NAME, ',') + ' FROM ' + #SourceTableName
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME=#SourceTableName AND COLUMN_NAME NOT IN (Select ColumnName FROM #IdentityCols)
exec sp_executesql #SqlQuery
For more information you can see this questions:
How can I show the table structure in SQL Server query?
How do you determine what SQL Tables have an identity column programmatically
How to concatenate text from multiple rows into a single text string in SQL Server
SQL Server Loop through Table Rows without Cursor
SQL Server loop - how do I loop through a set of records
For anyone interested, how I've solved this problem:
After I've read your comments (thanks btw) and some threads online, I've realized why I cannot do what I asked. However, I've come seen solution to similar problem somewhere, where they wanted to select * except one specific column, they solved it like this:
copied the entire table
selected from there.
I've come up with similar solution to my problem
-- check data, remove after testing
select * from demo_table order by ID desc;
-- create table with one column I want to duplicate
select * into Temp_demo_table from demo_table where ID=9999;
-- drop id, so it does not get included in the inter-table insert
alter table Temp_demo_table
drop column ID;
-- update columns that I need to modify, doesn't have to have WHERE clause, becuase there's only one row there
update Temp_demo_table set MyCustomProperty='abc', name=NULL, age=NULL
-- insert the desired and edited row
insert into demo_table
select * from Temp_demo_table;
-- drop the temp table
drop table Temp_demo_table;
-- check data, remove after testing
select * from demo_table order by ID desc;
I realize how inefficient this is, however the function (on my api) executing this command will not be called so often (max 100 times per day). I believe that this query could be optimized, however I do not have sufficient knowledge to do it at this moment (100% going to put it in my TODO :D).
Edit-1:
Just found out that you can write queries in oracle db like this:
(select * from demo_table) - (select name, age from demo_table)
I currentlly don't know if I can apply this to sql server, however as soon as I have an access to mssql, I'll try it out and keep this answear updated!
I have a CSV file in blob storage with the following format:
**Column,DataType**
Acc_ID, int
firstname, nvarchar(500)
lastname, nvarchar(500)
I am trying to read this file in data factory and loop through the column names and check the destination table if these columns already exits, if not I want to create the missing columns in the SQL table.
I know that we can use the following SQL query to create columns that do not exist.
IF NOT EXISTS (
SELECT
*
FROM
INFORMATION_SCHEMA.COLUMNS
WHERE
TABLE_NAME = 'contact_info' AND COLUMN_NAME = 'acc_id')
BEGIN
ALTER TABLE contact_info
ADD acc_id int NULL
END;
But I am not sure if we can read the CSV file and pass the column names from the CSV file to the above SQL query in a data factory pipeline. Any suggestions for this please?
You can create a column if not exist using the Pre-copy script in the Copy data activity.
• Table columns before executing the pipeline.
SELECT TABLE_NAME, COLUMN_NAME, DATA_TYPE FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME = 'contact_info'
• Source file:
ADF pipeline:
Using the lookup activity, get the list columns and datatypes by connecting the source dataset to the source file.
Output of lookup activity:
Connect the lookup output to the ForEach activity to loop all the values from the lookup.
#activity('Lookup1').output.value
Add Copy data activity inside ForEach activity and connect the source to the SQL table. Select query instead of a table in Use query properties. Write a query that does not result in any result as we are using this copy activity only to add a column to the table if not exist.
select * from dbo.contact_info where 1= 2
In the Copy data activity sink, connect the sink dataset to the SQL table, and in the Pre-copy script write your query to add a new column. Here use the current ForEach loop items (column, datatype) values instead of hardcoding the values as below.
#{concat('IF NOT EXISTS ( SELECT * FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME = ','''','contact_info','''',' AND COLUMN_NAME = ','''',item().Column,'''',') ALTER TABLE contact_info ADD ',item().Column,' ', item().DataType,' NULL')}
When the pipeline is executed, the FoEach loop executes till it completes all the values in the lookup output and creates a new column in the table if not exist.
Columns in the table after the pipeline is executed:
I was wondering how can I drop table with concat by selecting value from other table.
This is what I am trying to figure out:
DROP TABLE SELECT 'table' || (select value from IncrementTable)
So basically table name is table6 for example.
Goal is: eg.. DROP TABLE table6
You can't do this directly. Table and column names have to be known when the statement is byte-compiled; they can't be generated at runtime. You have to figure out the table name and generate the appropriate statement string in the program using the database, and execute it.
I want to create a table in Hive using a select statement which takes a subset of a data from another table. I used the following query to do so :
create table sample_db.out_table as
select * from sample_db.in_table where country = 'Canada';
When I looked into the HDFS location of this table, there are no field separators.
But I need to create a table with filtered data from another table along with a field separator. For example I am trying to do something like :
create table sample_db.out_table as
select * from sample_db.in_table where country = 'Canada'
ROW FORMAT SERDE
FIELDS TERMINATED BY '|';
This is not working though. I know the alternate way is to create a table structure with field names and the "FIELDS TERMINATED BY '|'" command and then load the data.
But is there any other way to combine the two into a single query that enables me to create a table with filtered data from another table and also with a field separator ?
Put row format delimited .. in front of AS select
do it like this
Change the query to yours
hive> CREATE TABLE ttt row format delimited fields terminated by '|' AS select *,count(1) from t1 group by id ,name ;
Query ID = root_20180702153737_37802c0e-525a-4b00-b8ec-9fac4a6d895b
here is the result
[root#hadoop1 ~]# hadoop fs -cat /user/hive/warehouse/ttt/**
2|\N|1
3|\N|1
4|\N|1
As you can see in the documentation, when using the CTAS (Create Table As Select) statement, the ROW FORMAT statement (in fact, all the settings related to the new table) goes before the SELECT statement.