I'm trying to join two tables, hlo2aLookup and ODP_StudySite in my load script. Here's the load script:
hlo2aNewCol:
LOAD [~SiteKey]
RESIDENT hlo2aLookup;
left Join
LOAD [~SiteKey],[~SiteKey] as [Key2],
if(exists([~SiteKey],[~SiteKey]),'y','n') as [Key2Exits],
if(isnull([~SiteKey]),'y','n') as [Key2isNull]
Resident ODP_StudySite;
The matchkey is ~Sitekey, and where there is no match on table, i want it to display 'No match', instead of a null value. I have tested EXISTS and ISNULL (as seen in example below, in Key2Exists and Key2IsNull) - However it isn't working. This is what i'm seeing in the table right now:
Any ideas?
Best wishes
Dave
Dont think you can "flag" the records before/during the tables are joined.
You can achieve this in few steps:
join the tables into a temp table (in the script below hlo2aNewCol_Temp)
use the result from the temp table and write your logic here. This table will be the "final" one
drop the temp table
hlo2aNewCol_Temp:
LOAD
[~SiteKey]
RESIDENT hlo2aLookup;
left Join
LOAD
[~SiteKey],
[~SiteKey] as [Key2]
Resident ODP_StudySite;
hlo2aNewCol:
Load
[~SiteKey],
Key2, // this is probably not needed. just for reference.
if( isNull(Key2), 'No Match', Key2 ) as KeyStatus
Resident hlo2aNewCol_Temp;
Drop Table hlo2aNewCol_Temp;
Related
I have an MSSQL database Table (named here: TABLE) with four columns (ID, lookup, date, value) and I want to check for a large amount of data whether they are in the database, using python. The data I want to add are here called: to_be_added with columns index, lookup, date, value.
To check whether the data already exist I use the following sql. It returns the index from the to_be_added data which are not yet in the database. I first check which lookup are in the database and then only perform the join on the subset (here called existing).
SELECT to_be_added."index",existing."ID" FROM
(
(
select * from dbo.TABLE
where "lookup" in (1,2,3,4,5,6,7,...)
) existing
right join
(
select * from
( Values
(1, 1, 1/1/2000, 0.123),(2, 2, 1/2/2000, 0.456),(...,...,...)
)t1(index,lookup,date,value)
)to_be_added
on existing.lookup = to_be_added.lookup
and existing.date = to_be_added.date
)
WHERE existing."ID" IS NULL
I do it batchwise as otherwhise the sql command is getting too large to commit and execution time is too long. As I have millions of lines to compare I am looking for a more efficent command as it becomes quite time consuming.
Any help appreciated
I would do the following:
Load the data from Excel into a table in your DB e.g. table = to_be_added
Run a query like this:
SELECT a.index
FROM to_be_added a
LEFT OUTER JOIN existing e ON
a.lookup = e.lookup
and a.date = e.date
WHERE e.lookup IS NULL;
Ensure that table "existing" has an index on lookup+date
I have a group of enitities which need to have another record associated with them from another table.
When I try to output an Id for the table to be matched on it doesn't work because you can only output from inserted, updated etc.
DECLARE #SignatureGlobalIdsTbl table (ID int,
CompanyBankAccountId int);
INSERT INTO GlobalIds (TypeId)
-- I Cannot output cba.Id into the table since its not from inserted
OUTPUT Inserted.Id,
cba.Id
INTO #SignatureGlobalIdsTbl (ID,
CompanyBankAccountId)
SELECT (#DocumentsGlobalTypeKey)
FROM CompanyBankAccounts cba
INNER JOIN Companies c ON c.CompanyId = cba.CompanyId
WHERE SignatureDocumentId IS NULL
AND (SignatureFile IS NOT NULL
AND SignatureFile != '');
INSERT INTO Documents (DocumentPath,
DocumentType,
DocumentIsExternal,
OwnerGlobalId,
OwnerGlobalTypeID,
DocumentName,
Extension,
GlobalId)
SELECT SignatureFile,
#SignatureDocumentTypeKey,
1,
CompanyGlobalId,
#OwnerGlobalTypeKey,
[dbo].[fnGetFileNameWithoutExtension](SignatureFile),
[dbo].[fnGetFileExtension](SignatureFile),
documentGlobalId
FROM (SELECT c.GlobalId AS CompanyGlobalId,
cba.*,
s.ID AS documentGlobalId
FROM CompanyBankAccounts cba
INNER JOIN Companies c ON c.CompanyId = cba.CompanyId
CROSS JOIN #SignatureGlobalIdsTbl s) info
WHERE SignatureDocumentId IS NULL
AND (SignatureFile IS NOT NULL
AND SignatureFile != '');
I Tried to use cross join to prevent cartesian production but that did not work. I also tried to output the rownumber over some value but I could not get that to be stored in the table either.
If I have two seperate queries which return the same amount of records, how can I pair the records together without creating cartesian production?
'When I try to output an Id for the table ... it doesn't work.'
This seems to be because one of the columns you want to OUTPUT is not actually part of the insert. It's an annoying problem and I wish SQL Server would allow us to do it.
Someone may have a much better answer for this than I do, but the way I usually approach this is
Create a temporary table/etc of the data I want to insert, with a column for ID (starts blank)
Do an insert of the correct amount of rows, and get the IDs out into another temporary table,
Assign the IDs as appropriate within the original temporary table
Go back and update the inserted rows with any additional data needed (though that's probably not needed here given you're just inserting a constant)
What this does is to flag/get the IDs ready for you to use, then you allocate them to your data as needed, then fill in the table with the data. It's relatively simple although it does do 2 table hits rather than 1.
Also consider doing it all within a transaction to keep the data consistent (though also probably not needed here).
How can I pair the records together?
A cross join unfortunately multiplies the rows (number of rows on left times the number of rows on the right). It is useful in some instances, but possibly not here.
I suggest when you do your inserts above, you get an identifier (e.g., companyID) in your temp table and join on that.
If you don't have a matching record and just want to assign them in order, you can use an answer similar to my answer in another recent question How to update multiple rows in a temp table with multiple values from another table using only one ID common between them?
Further notes
I suggest avoiding table variables (e.g., DECLARE #yourtable TABLE) and use temporary tables (CREATE TABLE #yourtable) instead - for performance reasons. If it's only a small amount of rows it's OK, but it gets worse as it gets larger as SQL Server assumed that table variables only have 1 row
In your bottom statement, why is there the SELECT statement in the FROM clause? Couldn't you just get rid of that select statement and have the FROM clause list the tables you want?
I figured out a way to have access to the output, by using a merge statement.
DECLARE #LogoGlobalIdsTbl TABLE (ID INT, companyBankAccountID INT)
MERGE GlobalIds
USING
(
SELECT (cba.CompanyBankAccountId)
FROM CompanyBankAccounts cba
INNER JOIN Companies c on c.CompanyId = cba.CompanyId
WHERE cba.LogoDocumentId IS NULL AND (cba.LogoFile IS NOT NUll AND cba.LogoFile != '')
) src ON (1=0)
WHEN NOT MATCHED
THEN INSERT ( TypeId )
VALUES (#DocumentsGlobalTypeKey)
OUTPUT [INSERTED].[Id], src.CompanyBankAccountId
INTO #LogoGlobalIdsTbl;
I have a table that contains several repair categories, and items that are associated with each repair category. I am trying to insert all the standard items from a specific repair category that don't already exist into a Details table.
TblEstimateDetails is a join table for an Estimate Table and StandardItem Table. And TblCategoryItems is a join table for the Repair Categories and their respective Standard Items.
For example in the attached image, Left side are all the Standard Items in a Repair Category, and Right side are all the Standard Items that are already in the EstimateDetails table.
Standard Items All vs Already Included
I need to be able to insert the 6 missing GUIDS from the left, and into the table on the right, and only for a specific estimate GID.
This is being used in an Access VBA script, which I will translate into the appropriate code once I get the sql syntax correct.
Thank you!
INSERT INTO TblEstimateDetails(estimate_GID, standard_item_GID)
SELECT
'55DEEE29-7B79-4830-909C-E59E831F4297' AS estimate_GID
, standard_item_GID
FROM TblCategoryItems
WHERE repair_category_GID = '32A8AE6D-A512-4868-8E1A-EF0357AB100E'
AND NOT EXISTS
(SELECT standard_item_GID
FROM TblEstimateDetails
WHERE estimate_GID = '55DEEE29-7B79-4830-909C-E59E831F4297');
Some things to try: 1) simplify to a select query to see if it selects the right records, 2) use a NOT IN statement instead of NOT EXISTS. There's no reason NOT EXISTS shouldn't work, I'd just try a different way if it isn't working.
SELECT '55DEEE29-7B79-4830-909C-E59E831F4297' AS estimate_GID,
standard_item_GID
FROM TblCategoryItems
WHERE repair_category_GID = '32A8AE6D-A512-4868-8E1A-EF0357AB100E'
AND standard_item_GID NOT IN
(SELECT standard_item_GID FROM TblEstimateDetails
WHERE estimate_GID = '55DEEE29-7B79-4830-909C-E59E831F4297');
Got it figured out. Access needs the subquery to be correlated to main query to work. So I set the WHERE clause in the subquery to equal the matching column in the main query. And I had to join the Estimates table so that it picked only the items in a specific estimate.
SELECT
'06A2E0A9-9AE5-4073-A856-1CCE6D9C48BB' AS estimate_GID
, CI.standard_item_GID
FROM TblCategoryItems CI
INNER JOIN TblEstimates E ON CI.repair_category_GID=E.repair_category_GID
WHERE E.repair_category_GID = '15238097-305E-4456-B86F-3787C9B8219B'
AND NOT EXISTS
(SELECT ED.standard_item_GID
FROM TblEstimateDetails ED
WHERE E.estimate_GID=ED.estimate_GID
);
Hi i got a little problem right now i'm doing Stored Proc in sql server. I wanna get some result from multiple table and put it into a temporary table. So I thought i could use a "Select into" statement wich work fine until i decided to add multiple select into the temporary table. For example: Here's my code:
while #Compteur <= #wrhCodeEnd
select lp.lotQty ,lp.lotID into ZeTable from TB_lotLot ltlt
inner join TB_lotPal lp on ltlt.lotID=lp.lotID
inner join TB_palSuper ps ON lp.spID=ps.spID
inner join TB_expExpeditionLot eel ON eel.lotID=lp.spID
where ps.wrhID <> #Compteur and
ltlt.wrhID = #Compteur and
lp.lotID not in(select ZeTable.lotID from ZeTable)
the thing is I don't know if I can make multiple select into on the same temporary table and I also wanna check with a where clause the information in the table is not already there.
Thx in Advance
You can create temporary table and can use insert statement to add records with required columns and can drop it after use.
While I have looked at different examples on the internet I am having trouble selecting data from multiple tables. I have 2 tables.
1st has 'logUID', 'groupUID' field.
the other
2nd has 'logUID' , 'User ID' , 'LogEntry' , 'LogTypeUID'
What I am trying to accomplish is to get the log entries where the LogUID match each other and return the groupUID field , so I know which group the log was against. When I have tried select's or joins I seem to be getting one logUID field repeating.
SELECT TOP 10000 [log].[dbo].[Logs].[LogUID], [log].[dbo].[LogGroups].[LogUID], [Weblog].[dbo].[LogGroups].[GroupUID]
FROM [log].[dbo].[LogGroups], [log].[dbo].[Logs] INNER JOIN [log].[dbo].[LogGroups] as LG
ON [log].[dbo].[Logs].[LogUID] = LG.LogUID ;
Any help.
Remove extra FROM [log].[dbo].[LogGroups],
You have the [LogGroups] table in there twice...
FROM table1, table2 means table1 CROSS JOIN table2
So, just change your code to the following...
SELECT TOP 10000
[Logs].[LogUID],
[LogGroups].[LogUID],
[LogGroups].[GroupUID]
FROM
[log].[dbo].[Logs]
INNER JOIN
[log].[dbo].[LogGroups]
ON [Logs].[LogUID] = [LogGroups].LogUID ;
See first of all you need to create a relationship between the tables you have created. After creating a relationship you can either create a key relationship using the two columns. So that whatever data goes in the logUID field of first table, the same data goes in the logUID field of second table. So this would help you to fetch the data from both the tables.
Hope this would help.