I am trying to take the results of a select query in SQL and place them in another table in a different database. The table structure is identical. The select query is as follows;
USE Warwick
Go
Select tblOperations.Link, Project.*
From tblOperations
Inner Join Warwick.dbo.Project
On tblOperations.Link= Warwick.dbo.Project.[Project ID]
Where tblOperations.Job# = Warwick.dbo.Project.[Job Number] and
tblOperations.[Status] = 'Active' or tblOperations.[Status] = 'Pending'
The join lets me select just the jobs that are considered active. I need to copy the results into the table WCI_DB.dbo.Project, which already exists. I would lke to append and not overwrite if the record exists.
Any help would be appreciated.
Thanks.
You should tag your question with the database, which seems to be SQL Server. The SQL syntax is insert:
insert into WCI_DB.dbo.Project
<your select here>;
Normally, you want to list columns after the table name:
insert into WCI_DB.dbo.Project(list of columns>
<your select here>;
However, if this is a one-time exercise and you know the columns are the same, then it is small sin to omit them once.
To create a new table, using select into, which is documented here.
select . . .
into WCI_DB.dbo.Project
. . .
Related
I'm using BigQuery. I have a query-scheduler to generate a table (RESULT TABLE) that depends on another table (SOURCE TABLE). The case is, this source table doesn't always have data, there's a possibility that this source table is empty.
I want to Schedule the Query to make the RESULT TABLE only if there's data in SOURCE TABLE.
The example would be:
IF COUNT(1) FROM data.source_table > 0 THEN RUN:
SELECT *
FROM data.source_table
LEFT JOIN data.other_source_table
ELSE [Don't Run]
Thanks in Advance
The syntax is
IF condition THEN [sql_statement_list]
[ELSEIF condition THEN sql_statement_list]
[ELSEIF condition THEN sql_statement_list]...
[ELSE sql_statement_list]
END IF;
So for your case it's
IF COUNT(1) FROM data.source_table > 0
THEN
SELECT *
FROM data.source_table
LEFT JOIN data.other_source_table;
END IF;
For more details, you can read https://cloud.google.com/bigquery/docs/reference/standard-sql/scripting#if
At the moment you can't set a destination table when using BigQuery Scripting. It means that solutions based on IF statement will not work for your case.
Besides that, it seems that when you set a destination table, BigQuery creates the table before your query's execution, which means that independently of the results, the table will be created.
The query below is only SQL. In other words, it doesn't contains scripting. If you use it to create a scheduled query and set a destination table, you will see that even when the sub query is not run an empty table will be created.
SELECT
*
FROM
UNNEST(
(SELECT
(
CASE (SELECT COUNT(1) FROM data.source_table) > 0
WHEN TRUE
THEN (
SELECT ARRAY(
SELECT AS STRUCT *
FROM data.source_table
LEFT JOIN data.other_source_table)
)
END
)
)
)
As a workaround, you could keep your existing scheduled query and create another scheduled query just like below to run some minutes after the first one:
IF (SELECT count(1) FROM `dataset.destination_table`) = 0
THEN DROP TABLE `dataset.destination_table`;
END IF
To summarize, your solution would be:
Run a scheduled query that will create a destination table,
A few minutes later, run a scheduled query that will check if the created table is empty. If so, the table will be deleted.
I hope it helps
I'd like to create 3-4 separate temporary tables within a single BigQuery query (all of the tables are based on different data sources) and then join them in various ways later on in the query.
I'm trying to do this by using multiple WITH statements, but it appears that you're only able to use one WITH statement in a query if you're not nesting them. Each time I've tried, I get an error saying that a 'SELECT' statement is expected.
Am I missing something? I'd prefer to do this all in one query if at all possible.
I don't know what you mean by "temporary tables", but I suspect you mean common table expressions (CTEs).
Of course you can have a query with multiple CTEs. You just need the right syntax:
with t1 as (
select . . .
),
t2 as (
select . . .
),
t3 as (
select . . .
)
select *
from t1 cross join t2 cross join t3;
bq mk --table --expiration [INTEGER] --description "[DESCRIPTION]"
--label [KEY:VALUE, KEY:VALUE] [PROJECT_ID]:[DATASET].[TABLE]
Expiration date will make your table temporary. You can create one table at the time with "bq mk" but you can use this in a script.
You can use the DDL, but here also you can only create one table at the time.
{CREATE TABLE | CREATE TABLE IF NOT EXISTS | CREATE OR REPLACE TABLE}
table_name [( column_name column_schema[, ...] )]
[PARTITION BY partition_expression] [OPTIONS(options_clause[, ...])]
[AS query_statement]
If by "temporary table" you meant "subqueries" this is the syntax you have to use:
WITH
subQ1 AS (
SELECT * FROM Roster
WHERE SchoolID = 52),
subQ2 AS (
SELECT SchoolID FROM subQ1)
SELECT DISTINCT * FROM subQ2;
You should be able to use common table expressions without issue. However, if you have large queries with a significant amount of common table expressions/subqueries you may run into resource issues in BQ specifically regarding the ability for BQ to create an execution plan. Temporary tables have helped me in these scenarios, but there are likely better practices.
CREATE TEMP TABLE T1 AS (
WITH
X as (query),
Y as (query),
Z as (query)
SELECT * FROM
(combination_of_above_queries)
);
CREATE TEMP TABLE T2 AS (
WITH
A as (query),
B as (query),
C as (query)
SELECT * FROM
(combination_of_above_queries)
);
CREATE TABLE new_table AS (
SELECT *
FROM (combination of T1 and T2)
)
Apologies, I just saw happened to come across this question and wanted to share what helped me... hope it is helpful for you.
https://cloud.google.com/bigquery/docs/reference/standard-sql/data-definition-language#temporary_tables
I am fairly new to SQL, and I am hoping someone can help me with a problem I'm having. I haven't been able to find any answers helping me figure out this exact problem.
I have two tables in two SQL Server databases on two different servers that I want to compare using the column ItemID. I want to find records from Table1 that have an ItemID that does not exist in Table2 and insert those into a table variable. I have the following code:
--Create table variable to hold query results
DECLARE #ItemIDTable TABLE
(
[itemid][NVARCHAR](20) NULL
);
--Query data and insert results into table variable
INSERT INTO #ItemIDTable
([itemid])
SELECT a.[itemid]
FROM database1.dbo.table1 a
WHERE NOT EXISTS (SELECT 1
FROM [Database2].[dbo].[table2]
WHERE a.itemid = [Database2].[dbo].[table2].[itemid])
ORDER BY itemid
This works on a test server where the two databases are on the same server, but not in real life where they are on different servers. I tried the following using OPENQUERY, but I know I haven't got it quite right.
--Create table variable to hold query results
DECLARE #ItemIDTable TABLE
(
[ItemID][nvarchar](20) NULL
);
--Query data and insert results into table variable
INSERT INTO #ItemIDTable
([ItemID])
SELECT a.[ItemID]
FROM Database1.dbo.Table1 a
WHERE NOT EXISTS (SELECT 1
FROM OPENQUERY([Server2], SELECT * FROM [Database2].[dbo].[Table2]')
WHERE a.ItemID = [Database2].[dbo].[Table2].[ItemID])
ORDER BY ItemID
I'm pretty sure I need to do something in the WHERE clause, where I have the two databases on two servers, I'm just not quite sure how to structure it. Could anyone help?
You can't create an OPENQUERY that is correlated to an outer query. You could populate a temp table with the results of an OPENQUERY and do your WHERE NOT EXISTS against the temp table, or you might want to look into Synonyms.
Openquery works like this:
select *
from openquery
(LINKED_SERVER_NAME,
'select query goes here'
)
Note that the sql portion is single quoted. That means you might have to quote the quotes if necessary. For example:
select *
from openquery
(LINKED_SERVER_NAME,
'
select SomeTextField
from SomeTable
where SomeDateField = ''20141014''
'
)
I'm doing a data conversion between systems and have prepared a select statement that identifies the necessary rows to pull from table1 and joins to table2 to display a pair of supporting columns. This select statement also places blank columns into the result in order to format the result for the upload to the destination system.
Beyond this query, I will also need to update some column values which I'd like to do in a separate statement operation in a new table. Therefore, I'm interested in running the above select statement as a subquery inside a SELECT INTO that will essentially plop the results into a staging table.
SELECT
dbo_tblPatCountryApplication.AppId, '',
dbo_tblPatCountryApplication.InvId,
'Add', dbo_tblpatinvention.disclosurestatus, ...
FROM
dbo_tblPatInvention
INNER JOIN
dbo_tblPatCountryApplication ON dbo_tblPatInvention.InvId = dbo_tblPatCountryApplication.InvId
ORDER BY
dbo_tblpatcountryapplication.invid;
I'd like to execute the above statement so that the results are dumped into a new table. Can anyone please advise how to embed the statement into a subquery that will play nicely with a SELECT INTO?
You can simply add an INTO clause to your existing query to create a new table filled with the results of the query:
SELECT ...
INTO MyNewStagingTable -- Creates a new table with the results of this query
FROM MyOtherTable
JOIN ...
However, you will have to make sure each column has a name, as in:
SELECT dbo_tblPatCountryApplication.AppId, -- Cool, already has a name
'' AS Column2, -- Given a name to create that new table with select...into
...
INTO MyNewStagingTable
FROM dbo_tblPatInvention INNER JOIN ...
Also, you might like to use aliases for your tables, too, to make code a little more readable;
SELECT a.AppId,
'' AS Column2,
...
INTO MyNewStagingTable
FROM dbo_tblPatInvention AS i
INNER JOIN dbo_tblPatCountryApplication AS a ON i.InvId = a.InvId
ORDER BY a.InvId
One last note is that it looks odd to have named your tables dbo_tblXXX as dbo is normally the schema name and is separated from the table name with dot notation, e.g. dbo.tblXXX. I'm assuming that you already have a fully working select query before adding the into clause. Some also consider using Hungarian notation in your database (tblName) to be a type of anti-pattern to avoid.
If the staging table doesn't exist and you want to create it on insert then try the following:
SELECT dbo_tblPatCountryApplication.AppId,'', dbo_tblPatCountryApplication.InvId,
'Add', dbo_tblpatinvention.disclosurestatus .......
INTO StagingTable
FROM dbo_tblPatInvention
INNER JOIN dbo_tblPatCountryApplication
ON dbo_tblPatInvention.InvId = dbo_tblPatCountryApplication.InvId;
If you want to insert them in a specific order then use try using a sub-query in the from clause:
SELECT *
INTO StagingTable
FROM
(
SELECT dbo_tblPatCountryApplication.AppId, '', dbo_tblPatCountryApplication.InvId,
'Add', dbo_tblpatinvention.disclosurestatus .......
FROM dbo_tblPatInvention
INNER JOIN dbo_tblPatCountryApplication ON
dbo_tblPatInvention.InvId = dbo_tblPatCountryApplication.InvId
order by dbo_tblpatcountryapplication.invid
) a;
Try
INSERT INTO stagingtable (AppId, ...)
SELECT ... --your select goes here
I am trying to update one table from another, im able to update fine as long as the customer record exists, but there are some entries that dont.
To solve this i've tried running the following insert
SELECT *
INTO SalBudgetCust
FROM SalBudgetCust_temp
WHERE NOT EXISTS (
SELECT Customer
FROM SalBudgetCust
WHERE Customer = SalBudgetCust_temp.Customer
)
but im prompted with
There is already an object named 'SalBudgetCust' in the database.
Im stuck at this point... could anyone offer a little guideance?
SELECT INTO implicitly creates the table you name. You should instead use INSERT INTO ... SELECT * FROM ..., so that the existing table is used.
It should be INSERT INTO instead of SELECT * INTO ... like
INSERT INTO SalBudgetCust SELECT * FROM SalBudgetCust_temp
WHERE NOT EXISTS
(
SELECT Customer FROM SalBudgetCust WHERE Customer = SalBudgetCust_temp.Customer
)
The general syntax to insert data of one table into another is :
INSERT INTO new_table
SELECT * FROM old_table
WHERE some_condition;
Where, new_table is the table where you want to insert data, old_table is table from where you are fetching data and some_condition is the expression / condition based upon which you want to fetch data from old table.
You may use other clauses like order by, group by, and even sub queries after where clause.
May refer this SQL INSERT INTO and it's subsequent pages.