SQL - find missing records in each date - sql

I have two tables:
A product table that has productID of all products (productName, productID)
A transaction table of records with columns: date (for last 3 months) and productID. All productID should have at least one record for each date.
However, there are missing productID records for some datss, and I would like to find the missing productID and the respective dates they are missing from.
Right now, I'm only able to find missing records for one particular date, which is:
Select productName, productID
from product a inner join transaction b
On a.productid=b.productid
Where a.productid not in
(
Select distinct productid
From transaction
Where date='2019-03-01'
)
I would like to have a table that consolidates all dates and their respective missing productid records. How can I proceed from here?
create table product
(ProductID INT,
ProductName Char (30)
);
insert into product values
(1, 'apple'),
(2, 'orange'),
(3, 'pear');
create table transaction1
(transDate Date,
productID INT,
revenue FLOAT);
insert into transaction1 values
('2019-03-01', 1, 2.5),
('2019-03-01', 2, 4.0),
('2019-03-01', 2, 8.0),
('2019-03-01', 3, 6.0),
('2019-03-02', 1, 7.0),
('2019-03-02', 3, 14.0),
('2019-03-03', 1, 1.5) ;
What i have will be similar to this, where productID=2 is missing from 03-02. I am working for dates from beginning of the year till yesterday, which are all recorded in the transaction table.
#SimonPrice

The most tricky part of your problem is to generate the list of all dates for your expected date which is 90 days in your case. Here I have created all dates using a loop. One additional column 'Common' added with value 1 to Product and Temp table to join with the product table for all date and all product.
Note: Just change the value of #HowManyDay as per days you wants to check.
DECLARE #HowManyDay INT
SET #HowManyDay = 8
DECLARE #LoopCount INT
SET #LoopCount = 0
DECLARE #TempTable TABLE(Common INT,All_Date DATE)
WHILE #LoopCount <= #HowManyDay-1
BEGIN
INSERT INTO #TempTable (Common,All_Date)
VALUES (1,DATEADD(DD,-#LoopCount,GETDATE()))
SET #LoopCount = #LoopCount + 1
END
SELECT B.All_Date,C.productID,C.productName,T.date
FROM #TempTable B
INNER JOIN
(
SELECT 1 [Common],*
FROM product
)C ON B.Common = C.Common
LEFT JOIN transaction_details T ON C.productID = T.productID AND B.All_Date = T.date
WHERE T.date IS NULL
ORDER BY 3 ASC,1 DESC

Related

Insert into 2 different data to 2 table at the same time using first data's id

I have 2 tables: Order and product_order. Every order has some product in it and that's because I store products another table.
Table Order:
Id name
Table PRODUCT_ORDER:
id product_id order_id
Before I start to insert, I don't know what the Order Id is. I want to insert the data into both tables at once and I need the order id to do that.
Both id's are auto incremented. I'm using SQL Server. I can insert first order and then find the id of the order and than execute the second insert, but I want to do these both to execute at once.
The output clause is your friend here.
DECLARE #Orders TABLE (OrderID INT IDENTITY, OrderDateUTC DATETIME, CustomerID INT)
DECLARE #OrderItems TABLE (OrderItemID INT IDENTITY, OrderID INT, ProductID INT, Quantity INT, Priority TINYINT)
We'll use these table variables as demo tables with IDs to insert into. You're liking going to be passing the set of items for an order in together, but for the purpose of a demo we'll ad hoc them as a VALUES list.
DECLARE #Output TABLE (OrderID INT)
INSERT INTO #Orders (OrderDateUTC, CustomerID)
OUTPUT INSERTED.OrderID INTO #Output
VALUES (GETUTCDATE(), 1)
We inserted the Order into the Orders table, and used the OUTPUT clause to cause the inserted (and generated by the engine) into the table variable #Output. We can now use this table however we'd like:
INSERT INTO #OrderItems (OrderID, ProductID, Quantity, Priority)
SELECT OrderID, ProductID, Quantity, Priority
FROM (VALUES (5,1,1),(2,1,2),(3,1,3)) AS x(ProductID, Quantity, Priority)
CROSS APPLY #Output
We cross applied it to our items list, and inseted it as if it was any other row.
DELETE FROM #Output
INSERT INTO #Orders (OrderDateUTC, CustomerID)
OUTPUT INSERTED.OrderID INTO #Output
VALUES (GETUTCDATE(), 1)
INSERT INTO #OrderItems (OrderID, ProductID, Quantity, Priority)
SELECT OrderID, ProductID, Quantity, Priority
FROM (VALUES (1,1,1)) AS x(ProductID, Quantity, Priority)
CROSS APPLY #Output
Just to demo a little farther here's another insert. (You likely wouldn't need the DELETE normally, but we're still using the same variable here)
Now when we select that data we can see the two separate orders, with their IDs and the products that belong to them:
SELECT *
FROM #Orders o
INNER JOIN #OrderItems oi
ON o.OrderID = oi.OrderID
OrderID OrderDateUTC CustomerID OrderItemID OrderID ProductID Quantity Priority
------------------------------------------------------------------------------------------------
1 2022-12-08 23:23:21.923 1 1 1 5 1 1
1 2022-12-08 23:23:21.923 1 2 1 2 1 2
1 2022-12-08 23:23:21.923 1 3 1 3 1 3
2 2022-12-08 23:23:21.927 1 4 2 1 1 1
Dale is correct. You cannot insert into multiple tables at once, but if you use a stored procedure to handle your inserts, you can capture the ID and use it in the next insert.
-- table definitions
create table [order]([id] int identity, [name] nvarchar(100))
go
create table [product_order]([id] int identity, [product_id] nvarchar(100), [order_id] int)
go
-- stored procedure to handle inserts
create procedure InsertProductWithOrder(
#OrderName nvarchar(100),
#ProductID nvarchar(100))
as
begin
declare #orderID int
insert into [order] ([name]) values(#OrderName)
select #orderID = ##identity
insert into [product_order]([product_id], [order_id]) values(#ProductID, #orderID)
end
go
-- insert records using the stored procedure
exec InsertProductWithOrder 'Order ONE', 'AAAAA'
exec InsertProductWithOrder 'Order TWO', 'BBBBB'
-- verify the results
select * from [order]
select * from [product_order]

SQL Query - Run query multiple times but with a different variable date

I have a lengthy query written in SQL that uses CTEs and multiple variables to produce a report of about 1500 customer records with many columns based on a particular date, #ToDate. Some of the tables are ordered CTEs so I only get the latest value based on the #ToDate.
I've omitted specifics but the structure is as follows:
Declare #ToDate date .....
Declare #Category varchar ....;
with cte1 as (select * from table1 where table1.start_date <= #ToDate and (table1.end_date > #ToDate or table1.end_date is null))
,cte2 as (select * from table2 where table2.start_date <= #ToDate and (table2.end_date > #ToDate or table2.end_date is null))
select * from cte1
left join cte2 on cte2.id = cte1.id
where .....
which gives me the following results
|RunDate |CustomerID|DOB |Category|Col5 |Col6 |
|----------|----------|----------|--------|------|------|
|2021-08-30|11111 |2000-01-01|Cat1 | | |
|2021-08-30|22222 |2000-02-02|Cat2 | | |
I'd like to run the same script multiple times but with a different date. So run with #ToDate = '2021-08-30' which gives me one set of results and then every past Monday n number of times which would give me results like this...
|RunDate |CustomerID|DOB |Category|Col5 |Col6
|----------|----------|----------|--------|------|------|
|2021-08-30|11111 |2000-01-01|Cat1 | | |
|2021-08-30|22222 |2000-02-02|Cat2 | | |
|2021-08-23|11111 |2000-01-01|Cat1 | | |
|2021-08-23|22222 |2000-02-02|Cat2 | | |
|2021-08-23|33333 |2000-03-03|Cat9 | | |
I do have a calendar table available so I can easily identify the past n Mondays (or other day I like).
The only variable to change is the #ToDate as this is the Run Date, or As At Date if you will. Essentially I want to run it multiple times for the past few Mondays so I can get what the results were like at 30-08, 23-08, 16-08 etc...
I've never used loops and research suggests I should maybe avoid them or use them as a last resort. I'm not sure on the best approach and if I do use loops, how I wrap it around my query.
Thanks in advance
The question really needs a bit more elaboration but I have give a guess at what you are trying to do with this example.
I have create a Customers and Orders table and then display the results for the date range
I don't think you need to loop with cursors and such as you can get the loop effect by just using the #DateRanges and join on that. it being a CTE or not.
Please let me know if this is not what you meant and I will remove the answer
-- Setup a temp table to hold the dates I want to look for
IF EXISTS (SELECT * FROM tempdb.dbo.sysobjects O WHERE O.xtype in ('U') AND O.id = object_id(N'tempdb..#DateRanges'))
BEGIN
PRINT 'Removing temp table #DateRanges'
DROP TABLE #DateRanges;
END
CREATE TABLE #DateRanges (
[Date] DATE
)
-- Add some dates
INSERT INTO #DateRanges ([Date])
VALUES ('2021-08-30'),
('2021-08-23'),
('2021-08-16')
-- Setup some customers
IF EXISTS (SELECT * FROM tempdb.dbo.sysobjects O WHERE O.xtype in ('U') AND O.id = object_id(N'tempdb..#Customers'))
BEGIN
PRINT 'Removing temp table #Customers'
DROP TABLE #Customers;
END
CREATE TABLE #Customers (
CustomerId BIGINT IDENTITY(1,1) NOT NULL,
[Name] NVARCHAR(50),
DOB DATE NOT NULL,
CONSTRAINT PK_CustomerId PRIMARY KEY (CustomerId)
)
INSERT INTO #Customers ([Name], DOB)
VALUES('Bob', '1989-01-01'),
('Robert', '1994-01-01'),
('Andrew', '1992-01-01');
-- Setup some orders
IF EXISTS (SELECT * FROM tempdb.dbo.sysobjects O WHERE O.xtype in ('U') AND O.id = object_id(N'tempdb..#Order'))
BEGIN
PRINT 'Removing temp table #Order'
DROP TABLE #Order;
END
CREATE TABLE #Order (
OrderId BIGINT IDENTITY(1,1) NOT NULL,
CustomerId BIGINT NOT NULL,
CreatedDate DATE NOT NULL,
Category NVARCHAR(50) NOT NULL,
CONSTRAINT PK_OrderId PRIMARY KEY (OrderId)
)
INSERT INTO #Order(CustomerId, CreatedDate, Category)
VALUES
(1, '2021-08-30', 'Cat1'),
(1, '2021-08-23', 'Cat2'),
(2, '2021-08-30', 'Cat1'),
(2, '2021-08-23', 'Cat2'),
(2, '2021-08-16', 'Cat3'),
(3, '2021-08-30', 'Cat1'),
(3, '2021-08-16', 'Cat2')
-- Using the #DateRanged temp table we can the use this to ge the data we need so no need for a loop
SELECT *
FROM #DateRanges AS DR
LEFT JOIN #Order AS O ON O.
CreatedDate <= DR.[Date] AND O.CreatedDate >= DATEADD(D, -6, DR.[Date])

How to add up prices delimited by id's of products

I have a dbo.products where all the products are stored. These products have a unique ID, Name, category and Price.
In a customer database (dbo.customers) the selected orders are pipe separated in one column. For example "12|61|42|48|56|57". Is it possible to get the total price based on what the customer selects? So if 12 is the ID of a price 10 product and 61 is a price 8 product the outcome should be 18 when the product column is "12|61"?
I've tried to refer to the id of the dbo.prices but I can only select one price at a time.
select Price
from [dbo].[products]
where id in (12|61|42|48|56|57)
So what I expect is the total value of all the selected product, please help.
Made third table as orders store customer selected products in that table ,
after that manage foreign key,primary key structure which will identify customer and products data.
then will get what you want.
use query
sum() for calculating total price etc.
So let's assume that you get your data from some external system, and you are forced to accept that orders will be pipe-delimited garbage. Let's also assume you don't have the capacity to split this raw data out into a normalised format, storing it in a proper, relational manner. So in this world you are stuck with a bad data model, and all you can do is query it.
First thing you will need is a way to split out the orders into a usable format. You could use the built in SPLIT function, but if you don't have access to this because you're on an older version of SQL Server then you could write your own. Here's an example:
CREATE FUNCTION [dbo].[split] (
#string VARCHAR(MAX),
#delimiter CHAR(1))
RETURNS TABLE
AS
RETURN
(
WITH Pieces(id, [start], [stop]) AS (
SELECT 1, 1, CONVERT(INT, CHARINDEX(#delimiter, #string))
UNION ALL
SELECT ID + 1, [stop] + 1, CONVERT(INT, CHARINDEX(#delimiter, #string, [stop] + 1)) FROM Pieces WHERE [stop] > 0
)
SELECT
id,
SUBSTRING(#string, [start], CASE WHEN [stop] > 0 THEN [stop] - [start] ELSE LEN(#String) END) AS item
FROM
Pieces
);
Once you have that in place you can do things like this:
SELECT * FROM dbo.split('12|61|42|48|56|57', '|');
Results:
id item
1 12
2 61
3 42
4 48
5 56
6 57
Now you can finally construct a query to bring all these parts together. Here's an example, using table variables, as I had to guess what your schema looks like:
DECLARE #customer TABLE (id INT, customer_name VARCHAR(20), [order] VARCHAR(50));
DECLARE #price TABLE (id INT, price INT);
INSERT INTO #customer SELECT 1, 'Fred', '1|2|3';
INSERT INTO #customer SELECT 2, 'Bill', '2|4';
INSERT INTO #customer SELECT 3, 'Mary', '3';
INSERT INTO #price SELECT 1, 5;
INSERT INTO #price SELECT 2, 3;
INSERT INTO #price SELECT 3, 2;
INSERT INTO #price SELECT 4, 1;
SELECT c.id, c.customer_name, COUNT(s.item) AS items_ordered, SUM(p.price) AS total_price FROM #customer c CROSS APPLY dbo.split(c.[order], '|') s INNER JOIN #price p ON p.id = s.item GROUP BY c.id, c.customer_name ORDER BY 1;
Results:
id customer_name items_ordered total_price
1 Fred 3 10
2 Bill 2 4
3 Mary 1 2

UPDATE source table, AFTER Grouping?

I have a table (source) with payments for a person - called 'Item' in the example below.
This table will have payments for each person, added to it over a period.
I then generate invoices, which basically takes all the payments for a particular person, and sums them up into a single row.
This must be stored in an invoice table, for auditing reasons.
I do this in the example below.
What I am missing, though, as I am not sure how to do it, is that each payment, once assigned to the Invoice table, needs to had the Invoice ID that it was assigned to, stored in the Items table.
So, see the example below:
CREATE TABLE Items
(
ID INT NOT NULL IDENTITY(1,1),
PersonID INT NOT NULL,
PaymentValue DECIMAL(16,2) NOT NULL,
AssignedToInvoiceID INT NULL
)
CREATE TABLE Invoice
(
ID INT NOT NULL IDENTITY(1,1),
PersonID INT NOT NULL,
Value DECIMAL(16,2)
)
INSERT INTO Items (PersonID, PaymentValue) VALUES (1, 100)
INSERT INTO Items (PersonID, PaymentValue) VALUES (2, 132)
INSERT INTO Items (PersonID, PaymentValue) VALUES (2, 65)
INSERT INTO Items (PersonID, PaymentValue) VALUES (1, 25)
INSERT INTO Items (PersonID, PaymentValue) VALUES (3, 69)
SELECT * FROM Items
INSERT INTO Invoice (PersonID, Value)
SELECT PersonID, SUM(PaymentValue) FROM Items
WHERE AssignedToInvoiceID IS NULL
GROUP BY PersonID
SELECT * FROM Invoice
DROP TABLE Items
DROP TABLE Invoice
What I need to do is then update the Items table, to say that the first row has been assigned to Invoice.ID 1, row two was assigned to Invoice ID 2. Row 3, was assigned to Invoice ID 2 as well.
Note, there are many other columns in the table. This is a basic example.
Simply, I need to record which invoice, each source row was assigned to.
The key thing here to ensure payments are correctly linked to invoices is to ensure that:
A: No updates are made to Items between reading the unassigned items and updating AssignedToInvoiceID.
B: No new invoices are created with the Items being process before updating AssignedToInvoiceID.
As you are updating two tables it will have to be a two step process. To ensure A it will need to be run in a transaction with a least REPEATABLE READ isolation. To ensure B requires a transaction with SERIALIZABLE isolation. See SET TRANSACTION ISOLATION LEVEL
It can be done like this:
BEGIN TRAN
SET TRANSACTION ISOLATION LEVEL SERIALIZABLE
DECLARE #newInvoices TABLE (PersonID INT, InvoiceID INT)
INSERT INTO Invoice (PersonID, Value)
OUTPUT inserted.ID, inserted.PersonID INTO #newInvoices(InvoiceID, PersonID)
SELECT PersonID, SUM(PaymentValue) FROM Items
WHERE AssignedToInvoiceID IS NULL
GROUP BY PersonID
UPDATE Items
SET AssignedToInvoiceID = InvoiceID
FROM Items
INNER JOIN #newInvoices newInvoice ON newInvoice.PersonID = Items.PersonID
WHERE AssignedToInvoiceID IS NULL
COMMIT
An alternative if you are using SQL Server 2012 or later is to use the SEQUNCE object, this will allow the Items to be assigned new invoice IDs before the Invoices are created, reducing the locking required.
It works like this:
-- Run once with your table setup.
CREATE SEQUENCE InvoiceIDs AS INT START WITH 1 INCREMENT BY 1
CREATE TABLE Items
(
ID INT NOT NULL IDENTITY(1,1),
PersonID INT NOT NULL,
PaymentValue DECIMAL(16,2) NOT NULL,
AssignedToInvoiceID INT NULL
)
CREATE TABLE Invoice
(
-- No longer a IDENTITY column
ID INT NOT NULL,
PersonID INT NOT NULL,
Value DECIMAL(16,2)
)
BEGIN TRAN
DECLARE #newInvoiceLines TABLE (PersonID INT, InvoiceID INT, PaymentValue DECIMAL(16,2))
-- Reading and updating AssignedToInvoiceID happens in one query so is thread safe.
UPDATE Items
SET AssignedToInvoiceID = newInvoices.InvoiceID
OUTPUT inserted.PersonID, inserted.AssignedToInvoiceID, inserted.PaymentValue
INTO #newInvoiceLines(PersonID, InvoiceID, PaymentValue)
FROM Items
INNER JOIN (
SELECT PersonID, NEXT VALUE FOR InvoiceIDs AS InvoiceID
FROM Items
GROUP BY PersonID
) AS newInvoices ON newInvoices.PersonID = Items.PersonID
WHERE Items.AssignedToInvoiceID IS NULL
INSERT INTO Invoice (ID, PersonID, Value)
SELECT InvoiceID, PersonID, SUM(PaymentValue) FROM #newInvoiceLines
GROUP BY PersonID, InvoiceID
COMMIT
You will still want to use a transaction to ensure the Invoice gets created.
Based on what I understand, you could run an update from join after you have inserted records into Invoices table, like so:
update items
set assignedtoinvoiceid = v.id
from
items m
inner join invoice v on m.personid = v.personid
Demo
Each time you do an invoice "run", select the most recent invoice for each person something like,
update items
set AssignedToInvoiceID = inv.id
from
(select personid, max(id) id
from invoice
group by personid) inv
where items.personid = inv.personid
and AssignedToInvoiceID is null
this assumes that AssignedToInvoiceID is null when it isn't populated, if it gets defaulted to an empty string or something then you would need to change the where condition.
1) Get MAX(ID) from Invoice table before inserting new rows from Items table. Store this value into a variable: #MaxInvoiceID
2) After inserting records into Invoice table, update AssignedToInvoiceID in Items table with Invoice.ID>#MaxInvoiceID
Refer below code:
CREATE TABLE #Items
(
ID INT NOT NULL IDENTITY(1,1),
PersonID INT NOT NULL,
PaymentValue DECIMAL(16,2) NOT NULL,
AssignedToInvoiceID INT NULL
)
CREATE TABLE #Invoice
(
ID INT NOT NULL IDENTITY(1,1),
PersonID INT NOT NULL,
Value DECIMAL(16,2)
)
DECLARE #MaxInvoiceID INT;
SELECT #MaxInvoiceID=ISNULL(MAX(ID),0) FROM #Invoice
SELECT #MaxInvoiceID
INSERT INTO #Items (PersonID, PaymentValue) VALUES (1, 100)
INSERT INTO #Items (PersonID, PaymentValue) VALUES (2, 132)
INSERT INTO #Items (PersonID, PaymentValue) VALUES (2, 65)
INSERT INTO #Items (PersonID, PaymentValue) VALUES (1, 25)
INSERT INTO #Items (PersonID, PaymentValue) VALUES (3, 69)
SELECT * FROM #Items
INSERT INTO #Invoice (PersonID, Value)
SELECT PersonID, SUM(PaymentValue)
FROM #Items
WHERE AssignedToInvoiceID IS NULL
GROUP BY PersonID
SELECT * FROM #Invoice
UPDATE Itm
SET Itm.AssignedToInvoiceID=Inv.ID
FROM #Items Itm
JOIN #Invoice Inv ON Itm.PersonID=Inv.PersonID AND Itm.AssignedToInvoiceID IS NULL AND Inv.ID>#MaxInvoiceID
SELECT * FROM #Items
DROP TABLE #Items
DROP TABLE #Invoice

Returning a set of the most recent rows from a table

I'm trying to retrieve the latest set of rows from a source table containing a foreign key, a date and other fields present. A sample set of data could be:
create table #tmp (primaryId int, foreignKeyId int, startDate datetime,
otherfield varchar(50))
insert into #tmp values (1, 1, '1 jan 2010', 'test 1')
insert into #tmp values (2, 1, '1 jan 2011', 'test 2')
insert into #tmp values (3, 2, '1 jan 2013', 'test 3')
insert into #tmp values (4, 2, '1 jan 2012', 'test 4')
The form of data that I'm hoping to retrieve is:
foreignKeyId maxStartDate otherfield
------------ ----------------------- -------------------------------------------
1 2011-01-01 00:00:00.000 test 2
2 2013-01-01 00:00:00.000 test 3
That is, just one row per foreignKeyId showing the latest start date and associated other fields - the primaryId is irrelevant.
I've managed to come up with:
select t.foreignKeyId, t.startDate, t.otherField from #tmp t
inner join (
select foreignKeyId, max(startDate) as maxStartDate
from #tmp
group by foreignKeyId
) s
on t.foreignKeyId = s.foreignKeyId and s.maxStartDate = t.startDate
but (a) this uses inner queries, which I suspect may lead to performance issues, and (b) it gives repeated rows if two rows in the original table have the same foreignKeyId and startDate.
Is there a query that will return just the first match for each foreign key and start date?
Depending on your sql server version, try the following:
select *
from (
select *, rnum = ROW_NUMBER() over (
partition by #tmp.foreignKeyId
order by #tmp.startDate desc)
from #tmp
) t
where t.rnum = 1
If you wanted to fix your attempt as opposed to re-engineering it then
select t.foreignKeyId, t.startDate, t.otherField from #tmp t
inner join (
select foreignKeyId, max(startDate) as maxStartDate, max(PrimaryId) as Latest
from #tmp
group by foreignKeyId
) s
on t.primaryId = s.latest
would have done the job, assuming PrimaryID increases over time.
Qualms about inner query would have been laid to rest as well assuming some indexes.