How to handle U-SQL error if the Table does not exist - azure-data-lake

How to handle U-SQL error (E_CSC_USER_DDLENTITYOFNEITHERTYPEEXISTS) if the table not exists.
How to use IF object_id('table1') Is not null in U-SQL?
T-SQL example:
if object_id('table1') Is not null
Select empName, empStoreNum, empSales, 'East' As SalesDistrict
FROM store1
UNION ALL
if object_id('table2') is not null
Select empName, empStoreNum, empSales, 'East' As SalesDistrict
FROM store2
UNION ALL
if object_id('table3') is not null
Select empName, empStoreNum, empSales, 'East' As SalesDistrict
FROM store3

U-SQL currently offers FILE.EXISTS and PARTITION.EXISTS but not TABLE.EXISTS.
I added it to our backlog. Please file a feature request (or upvote an existing one if present) at http://aka.ms/adlfeedback.
Current workarounds would be to code gen the script based on the table name using the usql.tables catalog view and then execute the code generated script, or keep the tables existing, just have them empty if they are not "existing".

Related

SQL: Insert a new unique row based off previous row

I'm working with Deltek Vision ERP software trying to create a custom solution to provide a list of deliverables for projects. I've created a custom grid in the Project area of Vision that has a table in SQL structured like this:
SELECT TOP (1000) [WBS1]
,[WBS2]
,[WBS3]
,[Seq]
,[CreateUser]
,[CreateDate]
,[ModUser]
,[ModDate]
,[CustDeliverables]
,[CustDueDate]
,[CustCompletionDate]
FROM [Vision_Prod].[dbo].[Projects_Deliverables]
The table has three user entered fields, which are the last three columns listed above.
What I'm trying to accomplish is have deliverables set at WBS2 level also roll up to WBS1, so basically what needs to happen is that any time a record is created with a value in WBS2 the record is duplicated but the duplicate has no value in WBS2.
I've setup a workflow in Vision so that when someone enters a deliverable into the grid on a phase it kicks off a stored procedure to accomplish this. The problem is the Seq field. This is a unique identifier the system is assigning when a record is created. When my stored procedure fires I'm getting an error that the sequence has to be included in the record.
This is the stored procedure I'm using:
INSERT INTO [Vision_Prod].[dbo].[Projects_Deliverables] (WBS1, WBS2, WBS3, Seq, CreateUser, ModUser, ModDate, CreateDate, CustDueDate, CustCompletionDate)
SELECT WBS1, '', '', (NEXT VALUE FOR Seq), CreateUser, ModUser, ModDate, CreateDate, CustDueDate, CustCompletionDate
FROM Projects_Deliverables
WHERE Projects_Deliverables.WBS2 IS NOT Null and Projects_Deliverables.WBS1 = #WBS1
If anyone can help me figure out how to get the system to assign a new sequence when a record is created in this way that would be most appreciated.
You could create a dummy table using VALUES to create the an extra row and then fiter out when not needed. Something (but not tested) like this:
SELECT TOP (1000)
[WBS1]
,CASE c WHEN 2 THEN NULL ELSE [WBS2] END AS [WBS2]
,[WBS3]
,[Seq]
,[CreateUser]
,[CreateDate]
,[ModUser]
,[ModDate]
,[CustDeliverables]
,[CustDueDate]
,[CustCompletionDate]
FROM [Vision_Prod].[dbo].[Projects_Deliverables]
CROSS JOIN (VALUES (1), (2)) t(c)
WHERE (WBS2 IS NULL AND c < 2)
OR WBS2 IS NOT NULL
This was the solution:
REPLACE(CAST(NEWID() AS VARCHAR(36)), '-', '')

SQL Insert calculated average value from another table

I tried to insert a calculated average from a table to another table writing the sql like below but it didnt work. Can someone please help me out ? how can I write it like a stored procedure in Oracle to cater for many states i.e. CA, IL, GA, WI.... ?
INSERT INTO Employee(averageSalary, averageTax)
(SELECT AVG(Salary), AVG(Tax)
FROM HrDeptEmployee
WHERE State = 'NY')
Leave off the parentheses around the select, this is not a subselect.
EDIT:
For the second question in the comment (error: null value of ID column of target table):
Add the id to insert to your select list like this (assuming you want to use id 1):
INSERT INTO Employee(ID, averageSalary, averageTax)
SELECT 1, AVG(Salary), AVG(Tax)
FROM HrDeptEmployee
WHERE State = 'NY'

Data to be inserted by SSIS

I have a table known as Customer(DATABASE AAA) containing 3 fields
Cust_id Cust_name Cust_salary
1 A 2000
2 B 3000
3 C NULL
I want to put data of these 3 columns in Employee(DATABASE BBB) which has the same structure as of Customer.
I want to transfer records of only those customer in which Cust_salary part is not null.
This work is to be done in SSIS only. MY values for Cust_id is auto generated & before putting values to Employee_id,the Employee table should be deleted.The auto generated identity should be preserved.
You could create a SQL Execute Task in SSIS and run the following:
INSERT INTO Employee
(EmployeeId, EmployeeName, EmployeeSalary)
SELECT Cust_id, Cust_name, Cust_salary
FROM Customer
WHERE Cust_salary IS NOT NULL
Darren Davies answer seems correct, but if for some obscure reason you have an EmployeeID is also an identity column and needs to match Cust_ID, and assuming any entries already in the Employee table correspond with the correct customer you can use an Execute SQL Task in SSIS with a connection open to Database BBB to run the following:
SET IDENTITY_INSERT Employee ON
INSERT INTO Employee (EmployeeID, EmployeeName, EmployeeSalary)
SELECT Cust_ID, Cust_Name, Cust_Salary
FROM AAA..Customer
WHERE Cust_Salary IS NOT NULL
AND NOT EXISTS
( SELECT 1
FROM Employee
WHERE EmployeeID = Cust_ID
)
SET IDENTITY_INSERT Employee OFF
This will maintain the integrity of the Identity fields in each table, and only insert new Customers to the Employee table.
what have you tried?
You will need two connections, one for each DB and one data flow component which will have a OleDBSource and an OleDBDestination component inside.
On the OleDBSource you can select your connection and write your query and then you drag the green arrow to the OleDBDestination. Double click the OleDBDestination select destination connection and table and click on mapping.
Should be it

How can I create a view that splits data from one record in the source table into up to three records in the view?

I am attempting to create a view in SQL Server 2000 that splits data into individual records from a source table. Source table looks like this:
ShipID, AirlineID, Airport1, Airport2, AirlineID2, Airport3, Airport4, AirlineID3, Airport5, Airport6
My view needs to look like this:
ShipID, Airline, Origin, Destination
So my view needs to split each source record into up to three records in the view
Anyone know how best to do this?
This should set you to your way. Mind added column Transfer. It is here to allow you to sort your flights by timeline.
select ShipID, 1 Transfer, AirlineID, Airport1 Origin, Airport2 Destination
from mySourceTable
where Airport1 is not null
union all
select ShipID, 2, AirlineID2, Airport3, Airport4
from mySourceTable
where Airport3 is not null
union all
select ShipID, 3, AirlineID3, Airport5, Airport6
from mySourceTable
where Airport5 is not null
order by 1, 2
in general you do a few queries with a union between them, e.g., if airport 2 was always destination
SELECT shipid, airlineid, airport1, 'ORIGIN' FROM flight
UNION
SELECT shipid, airlineid, airport2, 'DESTINATION' FROM flight
That would get two records for each row in a table named flight. However, you have to give more information on what is in the various columns and what rows you want out, in particular, if they're filled with blank space or NULL if not used. And say what product you are using (MySql, Sql Server, Oracle, Access, Postgres, sticks and napkins, etc.) Try reading some other posts tagged "SQL" to see what people include by way of data.

Insert blank row between groups of rows and sorted by ID in sql

I have a table which has the following columns and values
ID TYPE NAME
1 MAJOR RAM
2 MAJOR SHYAM
3 MAJOR BHOLE
4 MAJOR NATHA
5 MINOR JOHN
6 MINOR SMITH
My requirement is to right a stored procedure (or SQL query) which would return the same resultset except that there will be blank line after the TYPE changes from one type to another type (major, minor).
MAJOR RAM
MAJOR SHYAM
MAJOR BHOLE
MAJOR NATHA
MINOR JOHN
MINOR SMITH
While i use this query for adding blank line but it is not sorted by basis of ID
select TYPE, NAME from (
select
TYPE as P1,
1 as P2,
ID,
TYPE,
NAME
from EMP
union all
select distinct
TYPE,
2,
'',
'',
N''
from EMP
) Report
order by P1, P2
go
How i sort data by ID
Thanks in advance
Yes, yes, don't do this, but here's the query to do it, assuming SQL Server 2008 R2. Other versions/rdbms you can achieve same functionality by writing two separate queries unioned together.
Query
; WITH DEMO (id, [type], [name]) AS
(
SELECT 1,'MAJOR','RAM'
UNION ALL SELECT 2,'MAJOR','SHYAM'
UNION ALL SELECT 3,'MAJOR','BHOLE'
UNION ALL SELECT 4,'MAJOR','NATHA'
UNION ALL SELECT 5,'MINOR','JOHN'
UNION ALL SELECT 6,'MINOR','SMITH'
)
, GROUPED AS
(
SELECT
D.[type]
, D.[name]
, ROW_NUMBER() OVER (ORDER BY D.[type] ASC, D.[name] DESC) AS order_key
FROM
DEMO D
GROUP BY
--grouping sets introduced with SQL Server 2008 R2
-- http://msdn.microsoft.com/en-us/library/bb510427.aspx
GROUPING SETS
(
[type]
, ([type], [name])
)
)
SELECT
CASE WHEN G.[name] IS NULL THEN NULL ELSE G.[type] END AS [type]
, G.[name]
FROM
GROUPED G
ORDER BY
G.order_key
Results
If you don't like the nulls, use coalsece to make empty strings
type name
MAJOR SHYAM
MAJOR RAM
MAJOR NATHA
MAJOR BHOLE
NULL NULL
MINOR SMITH
MINOR JOHN
NULL NULL
I agree with billinkc.
In a sequential mind, like mine, it can occur different.
The approach is to use a cursor and insert the records into a temp table.
This table can have a column, INT type, lets say it is called "POSITION" which increments with every insert.
Check for ID changes, and add the empty row everytime it does.
Finally make the SELECT order by "POSITION".
My context was:
An interface that dinamically adjust to what the user needs, one of the screens shows a payment table, grouped by provider with the approach early mentioned.
I decided to manage this from database and skip maintainance for the screen at client side because every provider has different payment terms.
Hope this helps, and lets keep an open mind, avoid saying "don't do this" or "this is not what SQL was designed for"