I have 2 sql tables
first table called processed
relatedorders (type in nvarchar)
132,112,144,155
116,113
11,44,15,16
and second table called orders and looks like
orderid (type int)
132
112
155
116
I want to select orderid from orders table where the orderid exists in relatedorders in processed table
I tried somthing like
SELECT *
FROM orders
WHERE EXISTS(SELECT relatedorders
from processed
where orders.orderid like %relatedorders %)
but it doesn't seem to work as it should, what am I doing wrong?
The proper way to model this is with a one-to-many relationship through a join table. Add another table, perhaps called ProcessedRelatedOrders containing the id of an order and the id of a related order, one row per relationship. You will likely want. Use both columns as the primary key. The index on the primary key should allow fast look up for a join with the processed order.
Related
I have a table where the most important information is that it is auto-incremented, the rest of the fields in the database are not relevant. Before inserting the data into the table, I created a "helper" table to store the newly created IDs in this table.
I have a second table like this - also the most important information is that the ID is auto-incremented, and the other data is not relevant to this example. In this case, I have also created an auxiliary table that stores the newly created ID values from this table.
Now I would like to take the values from auxiliary table 1 and 2 and insert them into a third table that will take the smallest ID from auxiliary table 1 and the smallest ID from auxiliary table 2 and insert them as a record into this third table, for example:
Record ID of third table | Smallest ID from first table | Smallest ID from third table.
I have no idea how to build the query constructs in my case - could someone give me some advice, or ready-made (different) code to follow?
My code:
DECLARE #inserted1 TABLE (contact_id udt_id)
INSERT INTO t_usr_contact (contact_firstname, contact_lastname)
OUTPUT INSERTED.contact_id INTO #inserted1(contact_id)
SELECT
'Firma',
'Temporary_value'
FROM t_sup_supplier AS sup
WHERE sup.sup_id IN (175,176) AND sup.grp_id IS null
DECLARE #inserted2 TABLE (grp_id udt_id)
INSERT INTO t_usr_group (grp_label_en)
OUTPUT INSERTED.grp_id INTO #inserted2(grp_id)
SELECT
'Supplier contact'
FROM t_sup_supplier AS sup2
WHERE sup2.sup_id IN (175,176) AND sup2.grp_id IS null
INSERT INTO t_usr_contact_group (grp_id, contact_id)
I would like to go the easiest way, which is as below, but it doesnt work :/.
VALUES (#inserted2.grp_id, #inserted2.contact_id)
As for the data example, after the insert in the first table I will get the following records and in the auxiliary table number 1 I will get the following records:
**Table t_usr_contact:**
175 - Firma - Temporary_value
176 - Firma - Temporary_value
**Table #inserted1:**
175
176
**Table t_usr_group:**
201 - Supplier_contact
202 - Supplier_contact
**Table #inserted2:**
201
202
**Table t_usr_contact_group:**
201 - 175
202 - 176
I've got no idea what you're ultimately trying to do, but if you want two tables each with N rows to become one table made from the columns of the two input tables, like you've got in your example (where your table of 175,176 and your table of 201,202 shall become a table of 175|201,176|202) then you need to join them. To join them you need a key. You haven't got a key so you'll have to fake one:
INSERT INTO thirdtable
SELECT contact_id,grp_id
FROM
(SELECT *, ROW_NUMBER() OVER(ORDER BY contact_id) as FakeKey FROM #inserted1) x
INNER JOIN
(SELECT *, ROW_NUMBER() OVER(ORDER BY grp_id) as FakeKey FROM #inserted2) x
ON x.FakeKey = y.FakeKey
This, of course, joins the data in a very arbitrary fashion based on the order of the assigned IDs. If you want some specific order, like contact 175 exists first and has to get group 202, then you can make the query that inserts the group (eg 202) based on the input 175 output the 175 and the 202 together into a (temp) common table then split it into the detail and middleman tables after
I didn't find a working solution for creating a "lookup column" in a Firebird database.
Here is an example:
Table1: Orders
[OrderID] [CustomerID] [CustomerName]
Table2: Customers
[ID] [Name]
When I run SELECT * FROM ORDERS I want to get OrderID, CustomerID and CustomerName....but CustomerName should automatically be computed by looking for the "CustomerID" in the "ID" column of "Customer" Table, returning the content of the "Name" column.
Firebird has calculated fields (generated always as/computed by), and these allow selecting from other tables (contrary to an earlier version of this answer, which stated that Firebird doesn't support this).
However, I suggest you use a view instead, as I think it performs better (haven't verified this, so I suggest you test this if performance is important).
Use a view
The common way would be to define a base table and an accompanying view that gathers the necessary data at query time. Instead of using the base table, people would query from the view.
create view order_with_customer
as
select orders.id, orders.customer_id, customer.name
from orders
inner join customer on customer.id = orders.customer_id;
Or you could just skip the view and use above join in your own queries.
Alternative: calculated fields
I label this as an alternative and not the main solution, as I think using a view would be the preferable solution.
To use calculated fields, you can use the following syntax (note the double parentheses around the query):
create table orders (
id integer generated by default as identity primary key,
customer_id integer not null references customer(id),
customer_name generated always as ((select name from customer where id = customer_id))
)
Updates to the customer table will be automatically reflected in the orders table.
As far as I'm aware, the performance of this option is less than when using a join (as used in the view example), but you might want to test that for yourself.
FB3+ with function
With Firebird 3, you can also create calculated fields using a trigger, this makes the expression itself shorter.
To do this, create a function that selects from the customer table:
create function lookup_customer_name(customer_id integer)
returns varchar(50)
as
begin
return (select name from customer where id = :customer_id);
end
And then create the table as:
create table orders (
id integer generated by default as identity primary key,
customer_id integer not null references customer(id),
customer_name generated always as (lookup_customer_name(customer_id))
);
Updates to the customer table will be automatically reflected in the orders table. This solution can be relatively slow when selecting a lot of records, as the function will be executed for each row individually, which is a lot less efficient than performing a join.
Alternative: use a trigger
However if you want to update the table at insert (or update) time with information from another table, you could use a trigger.
I'll be using Firebird 3 for my answer, but it should translate - with some minor differences - to earlier versions as well.
So assuming a table customer:
create table customer (
id integer generated by default as identity primary key,
name varchar(50) not null
);
with sample data:
insert into customer(name) values ('name1');
insert into customer(name) values ('name2');
And a table orders:
create table orders (
id integer generated by default as identity primary key,
customer_id integer not null references customer(id),
customer_name varchar(50) not null
)
You then define a trigger:
create trigger orders_bi_bu
active before insert or update
on orders
as
begin
new.customer_name = (select name from customer where id = new.customer_id);
end
Now when we use:
insert into orders(customer_id) values (1);
the result is:
id customer_id customer_name
1 1 name1
Update:
update orders set customer_id = 2 where id = 1;
Result:
id customer_id customer_name
1 2 name2
The downside of a trigger is that updating the name in the customer table will not automatically be reflected in the orders table. You would need to keep track of these dependencies yourself, and create an after update trigger on customer that updates the dependent records, which can lead to update/lock conflicts.
No need here a complex lookup field.
No need to add a persistant Field [CustomerName] on Table1.
As Gordon said, a simple Join is enough :
Select T1.OrderID, T2.ID, T2.Name
From Customers T2
Join Orders T1 On T1.IDOrder = T2.ID
That said, if you want to use lookup Fields (as we do it on a Dataset) with SQL you can use some thing like :
Select T1.OrderID, T2.ID,
( Select T3.YourLookupField From T3 where (T3.ID = T2.ID) )
From Customers T2 Join Orders T1 On T1.IDOrder = T2.ID
Regards.
I have 2 tables SVC_ServiceTicket and SVC_CustomersVehicle
The table ServiceTicket has a column customerID which is a foreign key to CustomersVehicle.So in ServiceTicket column customerID can have duplicate values.
When I do
select sst.ServiceTicketID,sst.CustomerID
from ServiceTicket sst,CustomersVehicle scv
where sst.CustomerID=scv.CV_ID
then it gives me duplicate customerID.So my requirement is if there are duplicate values of customerID then I want the latest customerID and as well serviceticket of that corresponding(latest customerID)
For example in the below screenshot there are customerID 13 is repeating so in this case I want latest customerID as well as serviceticket so the values I want is 8008 and 13
Please tell me how to do
Use aggregate function MAX. Also I would recommend to use a JOIN.
SELECT MAX(sst.ServiceTicketID) AS ServiceTicketID,sst.CustomerID
FROM ServiceTicket sst JOIN
CustomersVehicle scv ON sst.CustomerVehicleID=scv.CV_ID
GROUP BY sst.CustomerID
Here want to delete rows with a duplicated column's value (Product) which will be then used as a primary key.
The column is of type nvarchar and we don't want to have 2 rows for one product.
The database is a large one with about thousands rows we need to remove.
During the query for all the duplicates, we want to keep the first item and remove the second one as the duplicate.
There is no primary key yet, and we want to make it after this activity of removing duplicates.
Then the Product columm could be our primary key.
The database is SQL Server CE.
I tried several methods, and mostly getting error similar to :
There was an error parsing the query. [ Token line number = 2,Token line offset = 1,Token in error = FROM ]
A method which I tried :
DELETE FROM TblProducts
FROM TblProducts w
INNER JOIN (
SELECT Product
FROM TblProducts
GROUP BY Product
HAVING COUNT(*) > 1
)Dup ON w.Product = Dup.Product
The preferred way trying to learn and adjust my code with something similar
(It's not correct yet):
SELECT Product, COUNT(*) TotalCount
FROM TblProducts
GROUP BY Product
HAVING COUNT(*) > 1
ORDER BY COUNT(*) DESC
--
;WITH cte -- These 3 lines are the lines I have more doubt on them
AS (SELECT ROW_NUMBER() OVER (PARTITION BY Product
ORDER BY ( SELECT 0)) RN
FROM Word)
DELETE FROM cte
WHERE RN > 1
If you have two DIFFERENT records with the same Product column, then you can SELECT the unwanted records with some criterion, e.g.
CREATE TABLE victims AS
SELECT MAX(entryDate) AS date, Product, COUNT(*) AS dups FROM ProductsTable WHERE ...
GROUP BY Product HAVING dups > 1;
Then you can do a DELETE JOIN between ProductTable and Victims.
Or also you can select Product only, and then do a DELETE for some other JOIN condition, for example having an invalid CustomerId, or EntryDate NULL, or anything else. This works if you know that there is one and only one valid copy of Product, and all the others are recognizable by the invalid data.
Suppose you instead have IDENTICAL records (or you have both identical and non-identical, or you may have several dupes for some product and you don't know which). You run exactly the same query. Then, you run a SELECT query on ProductsTable and SELECT DISTINCT all products matching the product codes to be deduped, grouping by Product, and choosing a suitable aggregate function for all fields (if identical, any aggregate should do. Otherwise I usually try for MAX or MIN). This will "save" exactly one row for each product.
At that point you run the DELETE JOIN and kill all the duplicated products. Then, simply reimport the saved and deduped subset into the main table.
Of course, between the DELETE JOIN and the INSERT SELECT, you will have the DB in a unstable state, with all products with at least one duplicate simply disappeared.
Another way which should work in MySQL:
-- Create an empty table
CREATE TABLE deduped AS SELECT * FROM ProductsTable WHERE false;
CREATE UNIQUE INDEX deduped_ndx ON deduped(Product);
-- DROP duplicate rows, Joe the Butcher's way
INSERT IGNORE INTO deduped SELECT * FROM ProductsTable;
ALTER TABLE ProductsTable RENAME TO ProductsBackup;
ALTER TABLE deduped RENAME TO ProductsTable;
-- TODO: Copy all indexes from ProductsTable on deduped.
NOTE: the way above DOES NOT WORK if you want to distinguish "good records" and "invalid duplicates". It only works if you have redundant DUPLICATE records, or if you do not care which row you keep and which you throw away!
EDIT:
You say that "duplicates" have invalid fields. In that case you can modify the above with a sorting trick:
SELECT * FROM ProductsTable ORDER BY Product, FieldWhichShouldNotBeNULL IS NULL;
Then if you have only one row for product, all well and good, it will get selected. If you have more, the one for which (FieldWhichShouldNeverBeNull IS NULL) is FALSE (i.e. the one where the FieldWhichShouldNeverBeNull is actually not null as it should) will be selected first, and inserted. All others will bounce, silently due to the IGNORE clause, against the uniqueness of Product. Not a really pretty way to do it (and check I didn't mix true with false in my clause!), but it ought to work.
EDIT
actually more of a new answer
This is a simple table to illustrate the problem
CREATE TABLE ProductTable ( Product varchar(10), Description varchar(10) );
INSERT INTO ProductTable VALUES ( 'CBPD10', 'C-Beam Prj' );
INSERT INTO ProductTable VALUES ( 'CBPD11', 'C Proj Mk2' );
INSERT INTO ProductTable VALUES ( 'CBPD12', 'C Proj Mk3' );
There is no index yet, and no primary key. We could still declare Product to be primary key.
But something bad happens. Two new records get in, and both have NULL description.
Yet, the second one is a valid product since we knew nothing of CBPD14 before now, and therefore we do NOT want to lose this record completely. We do want to get rid of the spurious CBPD10 though.
INSERT INTO ProductTable VALUES ( 'CBPD10', NULL );
INSERT INTO ProductTable VALUES ( 'CBPD14', NULL );
A rude DELETE FROM ProductTable WHERE Description IS NULL is out of the question, it would kill CBPD14 which isn't a duplicate.
So we do it like this. First get the list of duplicates:
SELECT Product, COUNT(*) AS Dups FROM ProductTable GROUP BY Product HAVING Dups > 1;
We assume that: "There is at least one good record for every set of bad records".
We check this assumption by positing the opposite and querying for it. If all is copacetic we expect this query to return nothing.
SELECT Dups.Product FROM ProductTable
RIGHT JOIN ( SELECT Product, COUNT(*) AS Dups FROM ProductTable GROUP BY Product HAVING Dups > 1 ) AS Dups
ON (ProductTable.Product = Dups.Product
AND ProductTable.Description IS NOT NULL)
WHERE ProductTable.Description IS NULL;
To further verify, I insert two records that represent this mode of failure; now I do expect the query above to return the new code.
INSERT INTO ProductTable VALUES ( "AC5", NULL ), ( "AC5", NULL );
Now the "check" query indeed returns,
AC5
So, the generation of Dups looks good.
I proceed now to delete all duplicate records that are not valid. If there are duplicate, valid records, they will stay duplicate unless some condition may be found, distinguishing among them one "good" record and declaring all others "invalid" (maybe repeating the procedure with a different field than Description).
But ay, there's a rub. Currently, you cannot delete from a table and select from the same table in a subquery ( http://dev.mysql.com/doc/refman/5.0/en/delete.html ). So a little workaround is needed:
CREATE TEMPORARY TABLE Dups AS
SELECT Product, COUNT(*) AS Duplicates
FROM ProductTable GROUP BY Product HAVING Duplicates > 1;
DELETE ProductTable FROM ProductTable JOIN Dups USING (Product)
WHERE Description IS NULL;
Now this will delete all invalid records, provided that they appear in the Dups table.
Therefore our CBPD14 record will be left untouched, because it does not appear there. The "good" record for CBPD10 will be left untouched because it's not true that its Description is NULL. All the others - poof.
Let me state again that if a record has no valid records and yet it is a duplicate, then all copies of that record will be killed - there will be no survivors.
To avoid this can may first SELECT (using the query above, the check "which should return nothing") the rows representing this mode of failure into another TEMPORARY TABLE, then INSERT them back into the main table after the deletion (using transactions might be in order).
Create a new table by scripting the old one out and renaming it. Also script all objects (indexes etc..) from the old table to the new. Insert the keepers into the new table. If you're database is in bulk-logged or simple recovery model, this operation will be minimally logged. Drop the old table and then rename the new one to the old name.
The advantage of this over a delete will be that the insert can be minimally logged. Deletes do double work because not only does the data get deleted, but the delete has to be written to the transaction log. For big tables, minimally logged inserts will be much faster than deletes.
If it's not that big and you have some downtime, and you have Sql Server Management studio, you can put an identity field on the table using the GUI. Now you have the situation like your CTE, except the rows themselves are truly distinct. So now you can do the following
SELECT MIN(table_a.MyTempIDField)
FROM
table_a lhs
join table_1 rhs
on lhs.field1 = rhs.field1
and lhs.field2 = rhs.field2 [etc]
WHERE
table_a.MyTempIDField <> table_b.MyTempIDField
GROUP BY
lhs.field1, rhs.field2 etc
This gives you all the 'good' duplicates. Now you can wrap this query with a DELETE FROM query.
DELETE FROM lhs
FROM table_a lhs
join table_b rhs
on lhs.field1 = rhs.field1
and lhs.field2 = rhs.field2 [etc]
WHERE
lhs.MyTempIDField <> rhs.MyTempIDField
and lhs.MyTempIDField not in (
SELECT MIN(lhs.MyTempIDField)
FROM
table_a lhs
join table_a rhs
on lhs.field1 = rhs.field1
and lhs.field2 = rhs.field2 [etc]
WHERE
lhs.MyTempIDField <> rhs.MyTempIDField
GROUP BY
lhs.field1, lhs.field2 etc
)
Try this:
DELETE FROM TblProducts
WHERE Product IN
(
SELECT Product
FROM TblProducts
GROUP BY Product
HAVING COUNT(*) > 1)
This suffers from the defect that it deletes ALL the records with a duplicated Product. What you probably want to do is delete all but one of each group of records with a given Product. It might be worthwhile to copy all the duplicates to a separate table first, and then somehow remove duplicates from that table, then apply the above, and then copy remaining products back to the original table.
I've been searching and I've yet to find an example using merge for populating related tables.
The Northwind DB Order & OrderDetail tables could be used. (In our scenario, our tables are 3 levels deep.)
For simplicity let's say we have the following tables.
Orders
OrderID PK
OrderNumber
OrderDetails
OrderID - PK
OrderLineItemNumber PK - FK to Orders.OrderID field
OrderDetailDetails
OrderID - PK - FK to OrderDetails.OrderID
OrderLineItemNumber - PK - FK to OrderDetails.OrderLineItemNumber
OrderSequenceNumber - PK
Also, in this scenario, records get written to staging tables that are identical the tables above. The merge would need to merge records from the 3 staging tables to the 3 matching production tables.
The production Order table's OrderId will not share the staging Order tables OrderId value.
So if the merge conditions are met, then there must be an insert into the Order table to generate OrderId (set to identity) because OrderId is needed for the OrderDetail & OrderDetailDetails rows to be created.
Right now I've written a service in C# that does all this but it's not that performant.
MERGE was discovered so we're looking into it to see if it can be used in a situation such as this. Any tips or pointers would be greatly appreciated.
Thanks.
Edit: I am now using Output store values into a Temporary table called #MergeOutput.
Declare #MergeOutput Table
(
ActionType varchar(10),
InsertedOrderId int,
StagingOrderID int,
DeletedOrderId int
);
However, I need to do a Merge on all 3 tables. (Order, OrderDetail & OrderDetailDetails)
Also, these tables have more fields than just the Id's.
So I've started creating the 2nd Merge for the OrderDetail table.
MERGE OrderDetail AS OD
USING(
SELECT OrderID,
OrderLineItemNumber,
ProductId
FROM OrderDetail AS OD
where OrderId IN (Select StagingOrderID from #MergeOutput where ActionType = 'INSERT'
) AS src(OrderID,
OrderLineItemNumber,
ProductId
)
ON (OD.OrderId = src.Order AND OD.OrderLineItemNumber = src.OrderLineItemNumber)
WHEN NOT MATCHED By Target THEN
INSERT INTO <-- (This doesn't work no matter what I've tried so far.)
Select (Select Distinct InsertedOrderID from #MergeOutput where StagingOrderId = OrderID), src.OrderLineItemNumber, src.ProductId
;
I see the following errors with the code above.
"Incorrect syntax near the keyword 'into'
I need the functionality of the Merge to move records on all 3 tables
Looks like I've finally got this to working. I had to change the Insert statement to as follows.
Insert(OrderId, OrderLineItemId, ProductID)
Values((Select Distinct InsertedOrderID from #MergeOutput where StagingOrderId = OrderID), src.OrderLineItemNumber, src.ProductId)
I had tried this Insert statement earlier on. I just figured out I had to wrap the selection parens ().
Thanks for everyone's help. I'm hoping I can carry this over to the merge for the 3rd table.
This is a nasty problem that keeps coming up. You need to extract the inserted identity values. In SQL Server you can do this using the OUTPUT clause (http://msdn.microsoft.com/en-us/library/ms177564.aspx) with the INSERT "virtual table". This allows you to get all inserted IDs out in one statement.
You can then push the IDs into a temp table and use them to insert the detail records with the appropriate master IDs like this:
INSERT INTO Detail
SELECT * from Staging_Detail
JOIN #MasterIDs on Staging_Detail.MasterID = #MasterIDs.MasterID