Sqlite select query take too time? - sql

I was developed a pos software with sqlite database.It was fined in early but now it is too slow with many data storage.Some of select view query take at least 3 minutes for only read in c# data reader.It is so bad.It may be wrong in database table structure.The worst performance views is following:
Firstly i have Stocks Table,Storage Storage Table,Purchase Receive Items Table and Sale Voucher Items.
Stock table has 3000 rows.Stock Storage table has 4600 rows. Purchase Receive Items table has 4500 rows and sale voucher items table has 130,00 rows.I was created view for show sum of purchase item and sum of sale items like that=>
select
Sto.stockcode,
Sto.StockName,
IfNull((select sum(ss.OpeningQty) from StockStorages ss
where ss.stockid=Sto.stockid and ss.status='A'),0) [OpeningQty],
IfNull((select sum(pur.qty) from purchasereceiveitems pur
where pur.stockid=Sto.stockid and pur.status='A'),0) [Purqty],
IfNull((select sum(svi.qty) from salevoucheritems svi
where svi.stockid=Sto.stockid and svi.status='A'),0) [Salqty],
IfNull((select sum(sti.qty) from StockTransactionItems sti
where sti.stockid=Sto.stockid and sti.status='A'),0) [Transactionqty],
IfNull((select sum(ss.CurrentQty) from StockStorages ss
where ss.stockid=Sto.stockid and ss.status='A'),0) [CurrentQty]
from stocks Sto
where Sto.status='A'
group by Sto.StockCode
How can i improve my structure or my code.Please advice me.

Turn your correlated subqueries (O(n^2) performance) into left joins like this:
select
s.stockcode,
s.stockname,
ifnull(a.opening_qty, 0) opening_qty,
ifnull(b.Purqty, 0) Purqty,
ifnull(c.salqty, 0) salqty,
ifnull(d.Transactionqty, 0) Transactionqty,
ifnull(e.CurrentQty, 0) CurrentQty
from StockCode s
left join (select
stockid,
sum(OpeningQty) opening_qty
from StockStorages
where status='A'
group by stockid) a on s.stockid = a.stockid
left join (select
stockid,
sum(qty) Purqty
from purchasereceiveitems
where status='A'
group by stockid) b on s.stockid = b.stockid
(select
stockid,
sum(qty) salqty
from salevoucheritems
where status='A'
group by stockid) c on s.stockid = c.stockid
(select
stockid,
sum(qty) Transactionqty
from StockTransactionItems
where status='A'
group by stockid) d on s.stockid = d.stockid
(select
stockid,
sum(CurrentQty) CurrentQty
from StockStorages
where status='A'
group by stockid) e on s.stockid = e.stockid
where s.status = 'A';

This query can be sped up by indexing the columns that are used for lookups, so that the subqueries no longer need to search the entire table for each single value:
CREATE INDEX StockStorages_ss ON StockStorages (stockid, status);
CREATE INDEX purchasereceiveitems_ss ON purchasereceiveitems (stockid, status);
CREATE INDEX salevoucheritems_ss ON salevoucheritems (stockid, status);
CREATE INDEX StockTransactionItems_ss ON StockTransactionItems(stockid, status);
CREATE INDEX StockStorages_ss ON StockStorages (stockid, status);

Related

Sql Joining 3 Tables Query for Loan Total Result

I have to get the total of Customer Loan from 3 tables the two tables are given loan to sum and the other subtract the paid amount and the tables have Customer ID in common. so far I can get the result only if the Customer ID exist in all tables but if it doesn't exist in one table I won't get Customer in my result. or I get NULL customer IDs when I anchor to the customer.
SELECT
AS1.C_ID AS [Customer ID],
ISNULL(AS1.OldCustomerLoan, 0) AS [Old Loan],
ISNULL(AS2.NewGivenLoan, 0) AS [New Given Loan],
ISNULL(AS3.LoanPaid, 0) AS [PaidLoanAmount],
(ISNULL(AS1.OldCustomerLoan, 0) +
ISNULL(AS2.NewGivenLoan, 0) -
ISNULL(AS3.LoanPaid,0) ) AS Total
FROM
Customer C
LEFT OUTER JOIN
(SELECT
MOC.C_ID,
SUM(MOC.Quantity) AS OldCustomerLoan
FROM
Money_On_Customer MOC (NOLOCK)
GROUP BY
MOC.C_ID) AS1
ON c.C_Id = AS1.C_Id
LEFT OUTER JOIN
(SELECT
NGL.C_ID
,SUM(NGL.G_Take_Loan) AS NewGivenLoan
FROM
Given_Loan NGL
GROUP BY
NGL.C_ID) AS2
ON c.C_Id = AS2.C_Id
LEFT OUTER JOIN
(SELECT
GLP.C_ID, SUM(GLP.G_P_Loan) AS LoanPaid
FROM
Given_Loan_Paid GLP
GROUP BY
GLP.C_ID ) AS3
ON c.C_Id = AS3.C_Id
Here Is a picture of my two results:
When I get NULL Customer IDs
When I don't get All the Customers
You need to use c.c_id for the first column
In order to only get records where they exist in at least one of the tables you can add this to you query, just put you current query in place of the dots, and add the leftid col
Select *
From
(Select c.c_id custonerid,
Coalesce(Coalesce(as1.c_id,as2.c_id),as3.c_id) leftid,......
From ....
) ilv
Where leftid is not null
You might be able to just add
Where coalesce(coalesce(as1.c_id,as2.c_id),as3.c_id) is not null
To then end of your query
#Ab Bennett's answer is right,
because you should use your main(master) table's primary key column
if Customer ID is not available in as1 (Money_On_Customer) it will show null.
Hope you understand my explanation.
UPDATE:
and use following for getting customer id
COALESCE(c.C_Id, AS1.C_Id, AS2.C_Id)
you will get first not null Customer ID
Here is My answer with the help of #Ab Bennett
Select *
From
(
SELECT
C.C_Name AS [Customer ID],
ISNULL(AS1.OldCustomerLoan, 0) AS [Old Loan],
ISNULL(AS2.NewGivenLoan, 0) AS [New Given Loan],
ISNULL(AS3.LoanPaid, 0) AS [PaidLoanAmount],
(ISNULL(AS1.OldCustomerLoan, 0) +
ISNULL(AS2.NewGivenLoan, 0) -
ISNULL(AS3.LoanPaid,0) ) AS Total
FROM
Customer C
LEFT OUTER JOIN
(SELECT
MOC.C_ID,
SUM(MOC.Quantity) AS OldCustomerLoan
FROM
Money_On_Customer MOC (NOLOCK)
GROUP BY
MOC.C_ID) AS1
ON c.C_Id = AS1.C_Id
LEFT OUTER JOIN
(SELECT
NGL.C_ID
,SUM(NGL.G_Take_Loan) AS NewGivenLoan
FROM
Given_Loan NGL
GROUP BY
NGL.C_ID) AS2
ON c.C_Id = AS2.C_Id
LEFT OUTER JOIN
(SELECT
GLP.C_ID, SUM(GLP.G_P_Loan) AS LoanPaid
FROM
Given_Loan_Paid GLP
GROUP BY
GLP.C_ID ) AS3
ON c.C_Id = AS3.C_Id) ilv
Where not ([Old Loan] = 0 and [New Given Loan]=0 and PaidLoanAmount =0 )

selecting distinct values from master table and repeated values from joined tables

I'm trying to select distinct values from master table and repeated values from child tables.
I have 4 tables:
invoice
invoiceLine
product
businessPartner
My query:
select
c_invoice.c_invoice_ID,
c_bpartner.name as "Business Partner",
M_Product.name as "Product",
c_invoiceline.priceentered as "amount"
from adempiere.c_invoice
left join adempiere.c_invoiceline on c_invoice.c_invoice_ID=c_invoiceline.c_invoice_ID
left join adempiere.M_Product on c_invoiceline.M_Product_ID =M_Product.M_Product_ID
left join adempiere.C_BPartner on c_invoice.c_bpartner_ID=c_bpartner.c_bpartner_id
where c_invoice.sh_booking_ID=1000019 and c_invoice.c_doctypetarget_id=1000005
My query result:
INVOICEID BUSINESS Partner PRODUCT AMT
1000005; "Tehmoor"; "Charge 1"; 1200
1000005; "Tehmoor"; "Standard"; 1500
1000006; "Rafay"; "Charge 1"; 1200
1000006; "Rafay"; "Standard"; 1100
and expected result
INVOICEID BUSINESS Partner PRODUCT AMT
1000005; "Tehmoor"; "Charge 1"; 1200
; NULL; "Standard"; 1500
1000006; "Rafay"; "Charge 1"; 1200
; NULL; "Standard"; 1100
You can try something like this:
select
CASE WHEN row_number() OVER (PARTITION BY mast.id) = 1 THEN
mast.title
ELSE NULL END as title,
joined.measure
from mast
left join joined on (mast_id = mast.id)
I've created a fiddle for it, so you can examine my example schema.
I think it's better to handle this kind of requirement in a host language, because in SQL it's a bit tricky.
I have reproduced your schema in my local environment. Below is the query using which you can achieve your desired result.
SELECT
CASE WHEN (Rank() Over(ORDER BY i.c_invoice_id ASC)) = (Row_Number() Over (ORDER BY i.c_invoice_id ASC)) THEN pt.b_name ELSE NULL END AS "Business Partner",
CASE WHEN (Rank() Over(ORDER BY i.c_invoice_id ASC)) = (Row_Number() Over (ORDER BY i.c_invoice_id ASC)) THEN i.c_invoice_id ELSE NULL END AS Invoice_Id,
pr.b_name,
il.price
FROM invoice i
LEFT JOIN c_invoice_line il ON il.c_invoice_id = il.c_invoice_id
LEFT JOIN c_product pr ON il.product_line_id = pr.b_prod_id
LEFT JOIN c_bpartner pt ON pt.b_partner_id = Trim(il.c_prod_id);
If required, change your table and column names accordingly.

SQL - Only show results for one column in first instance of a duplicated record

I am finding it difficult to explain exactly what I am trying to achieve so I think it best to show a visual representation.
Example of how my query results currently look
Example of how I want the results to look
The report I am running shows a list of every product within orders. Each product has its own cost assigned to it. Another column is a delivery charge, but this is a charge assigned to the order; not individual products. I want to be able to show the delivery charge against the first product in each order ONLY.
I have attempted, for far too long, to try and find an answer to this query but have had no luck. I don't know if it is even possible so assistance of any sort, or even just being pointed in the right direction, would be of great help.
Thanks
EDIT.
If it helps here is my query:
SELECT dbo.Orders.EncryptedOrderId,
dbo.OrderProduct.ProductID,
dbo.OrderProduct.QuantityPerRecipient,
dbo.OrderProduct.NumRecipients,
dbo.OrderProduct.TotalQuantity,
dbo.DocType.Name AS [Product Type],
dbo.ProductGroup_Culture.Name AS [Product Group],
RIGHT(CatalogNo, CHARINDEX('_', REVERSE('_' + CatalogNo)) -1) AS [HamptonsType],
FORMAT(dbo.Orders.DateOrderCreated, 'dd/MM/yyyy HH:mm:ss') AS 'DateOrderCreated',
CAST(REPLACE(dbo.Orders.ClearingResult, 'utf-8', 'utf-16') AS XML ).value('(/UserData//CostCenter/node())[1]', 'nvarchar(max)') AS [Cost Center],
dbo.Users.FirstName,
dbo.Users.LastName,
dbo.Users.CompanyName AS [Branch Name],
dbo.Users.Department AS Subsidiary,
dbo.Users.Custom1,
dbo.Users.Custom2,
dbo.Users.Custom3,
dbo.Users.Custom4,
dbo.Users.Custom5,
dbo.OrderProduct.TotalPrice,
dbo.Orders.ShippingCharges,
dbo.OrderProduct.OrderProductID,
dbo.FileSubmissionDocument.OriginalFileType,
COALESCE (dbo.FileSubmissionDocument.Title, dbo.Product_Culture.Name) AS [Product Name],
OPDV.FriendlyValue AS 'BCard Recipient'
FROM dbo.DocType
INNER JOIN dbo.Doc
ON dbo.DocType.DocTypeID = dbo.Doc.DocTypeID
INNER JOIN dbo.OrderProduct
ON dbo.Doc.ProductID = dbo.OrderProduct.ProductID
LEFT JOIN dbo.Product
ON dbo.Product.ProductID = dbo.Doc.ProductID
LEFT JOIN dbo.ProductGroupMembership
ON dbo.ProductGroupMembership.ProductID = dbo.Doc.ProductID
LEFT JOIN dbo.ProductGroup_Culture
ON dbo.ProductGroup_Culture.ProductGroupID = dbo.ProductGroupMembership.ProductGroupID
INNER JOIN dbo.Orders
ON dbo.OrderProduct.OrderID = dbo.Orders.OrderID
INNER JOIN dbo.Users
ON dbo.Orders.UserID = dbo.Users.UserID
INNER JOIN dbo.Product_Culture
ON dbo.OrderProduct.ProductID = dbo.Product_Culture.ProductID
INNER JOIN dbo.Store_Culture
ON dbo.Store_Culture.StoreID = dbo.Users.AssignedToStoreID FULL OUTER
JOIN dbo.FileSubmissionDocument
ON dbo.OrderProduct.OrderProductID = dbo.FileSubmissionDocument.SubOrderProductID - 1
LEFT JOIN (SELECT OP.OrderProductID,
OP.DialID,
OP.FriendlyValue
FROM OrderProductDialValue OP
LEFT JOIN Dial DI ON DI.DialID = OP.DialID
LEFT JOIN OrderProduct OT ON OT.OrderProductID = OP.OrderProductID
LEFT JOIN Product PR ON PR.ProductID = OT.ProductID
WHERE PR.ExternalID = 'BCName'
AND DI.UProduceDialName = 'Name') OPDV ON OPDV.OrderProductID = dbo.OrderProduct.OrderProductID
WHERE ('#CUSTOMERNAME' is null
OR '#CUSTOMERNAME' = ''
OR dbo.Store_Culture.Name LIKE '%' + '#CUSTOMERNAME' + '%')
AND dbo.OrderProduct.IsDraft = 0
AND dbo.Orders.IsCart=0
AND dbo.Orders.IsSaveForLater=0
AND (('#DATE' <= dbo.Orders.DateOrderCreated)
OR ('#DATE' IS NULL)
OR ('#DATE'=''))
AND ((DATEADD(day, 1, '#DATE') >= dbo.Orders.DateOrderCreated)
OR ('#DATE' IS NULL)
OR ('#DATE'=''))
AND dbo.Users.LastName NOT LIKE '%TEST%'
ORDER BY dbo.Orders.OrderID DESC, dbo.OrderProduct.OrderProductID DESC
The query runs through a reporting system on an online portal so the values that show as #CUSTOMERNAME or #DATE are variables based on values given at the time when the report is run.
this may help you
select orderid,
productid,
productvalue,
case ROW_NUMBER() over (partition by orderid order by orderid)
when 1 then deliverycharge
else null end as 'deliverycharge'
from ........
I assume your query looks like
select orderID
, productID
, productValue
, DeliveryCharge
from test_t
order by orderID
, productValue desc
;
and that you want delivery charges listed for the most expensive product of each order.
If supported by your rdbms, you can use the analytic RANK function
select orderID
, productID
, productValue
, DeliveryCharge
, CASE RANK() OVER ( PARTITION BY orderID ORDER BY productValue DESC ) WHEN 1 THEN DeliveryCharge ELSE NULL END r
from test_t
order by orderID
, productValue desc
;
If your rdbms does not support RANK, you can emulate it using a left join with a suitably aggregated copy of your table:
select t.orderID
, t.productID
, t.productValue
, rt.mdc DeliveryCharge
from test_t t
left join (
select orderID
, max(productValue) mp
, max(DeliveryCharge) mdc
from test_t
group by orderID
) rt
on (
rt.orderID = t.orderID
AND rt.mp = t.productValue
)
order by orderID
, productValue desc
;
The tacit assumption is that the delivery charge is by order (which seems reasonable as you wouldn't want to selectively drop it otherwise, right ?).
Moreover, both solutions will produce multiple rows containing the delivery charge per order if that order contains multipleproducts with the same productValue.
Tested on oracle 12c1;

SQL Multiple JOIN Query Not successful

Background
I have three tables: Stock, PurchaseEntry, and SalesEntry.
These tables have the fields ProductName, MRP, LandinPrice and Qty in common.
I need to join these tables in order to get SUM(Qty) in PurchaseEntry and SalesEntry with the same ProductName, MRP, and LandinPrice as Stock.
Problem
I tried using the following query, but the result was not as I desired.
SUM(Qty) of PurchaseEntry Gives some random results, whereas SUM(Qty) of SalesEntry is correct.
But when I use JOIN on PurchaseEntry alone individually, I get the correct values.
Question
How I can get the proper results, and what type of JOINs should be used?
SQL Code
SELECT Stock.ProductName,
Stock.MRP,
Stock.LandinPrice,
SUM(PurchaseEntry.TotalQty)
AS PurchaseQty,
SUM(SalesEntry.Qty)
AS SalesQty
FROM Stock
JOIN PurchaseEntry
ON Stock.ProductName = PurchaseEntry.ProductName
AND Stock.MRP = PurchaseEntry.MRP
AND Stock.LandinPrice = PurchaseEntry.LandinPrice
JOIN SalesEntry
ON Stock.ProductName = SalesEntry.ProductName
AND Stock.MRP = SalesEntry.MRPPrice
AND Stock.LandinPrice = SalesEntry.LandingAmt
GROUP BY Stock.ProductName,
Stock.MRP,
Stock.LandinPrice
Solved:
I tried changing the Query myself. And I got the proper Output that i required. Below is that Query.
SELECT Stock.ProductName,
Stock.MRP,Stock.LandinPrice,
(SELECT SUM(PurchaseEntry.TotalQty)
FROM PurchaseEntry
WHERE Stock.ProductName=PurchaseEntry.ProductName
AND Stock.MRP=PurchaseEntry.MRP
AND Stock.LandinPrice=PurchaseEntry.LandinPrice) AS PurchaseQty,
(SELECT SUM(SalesEntry.Qty)
FROM SalesEntry
WHERE SalesEntry.ProductName=Stock.ProductName
AND SalesEntry.MRPPrice=Stock.MRP
AND SalesEntry.LandingAmt= Stock.LandinPrice ) AS SalesQty
FROM Stock
GROUP BY Stock.ProductName,Stock.MRP,Stock.LandinPrice
You might try something like
SELECT Stock.ProductName,Stock.MRP,Stock.LandinPrice,
(SELECT SUM(PurchaseEntry.TotalQty)
FROM PurchaseEntry
WHERE PurchaseEntry.ProductName=Stock.ProductName
AND PurchaseEntry.MRP=Stock.MRP
AND PurchaseEntry.LandinPrice=Stock.LandinPrice) AS PurchaseQty
(SELECT SUM(SalesEntry.Qty)
FROM SalesEntry
WHERE SalesEntry.ProductName=Stock.ProductName
AND SalesEntry.MRPPrice=Stock.MRP
AND SalesEntry.LandingAmt=Stock.LandinPrice) AS SalesQty
FROM Stock
Your existing query would have PurchaseQty multiplied by the number of sales and SalesQty multiplied by the number of purchases. Of course I'm assuming that the combination of stock ProductName, MRP, and LandinPrice are unique.
SELECT
Stock.ProductName,
Stock.MRP,
Stock.LandinPrice,
SUM(PurchaseEntry.TotalQty) AS PurchaseQty,
Sales.SalesQty
FROM
Stock
INNER JOIN PurchaseEntry ON Stock.ProductName = PurchaseEntry.ProductName AND Stock.MRP = PurchaseEntry.MRP
AND Stock.LandinPrice = PurchaseEntry.LandinPrice
INNER JOIN
(
SELECT SUM(SE.Qty) AS 'SalesQty', SE.ProductName, SE.LandingAmt, SE.MRPPrice
FROM SalesEntry SE INNER JOIN
Stock S ON S.LandinPrice = SE.LandingAmt AND S.MRP = SE.MRPPrice AND S.ProductName = SE.ProductName
GROUP BY SE.ProductName, SE.LandingAmt, SE.MRPPrice
) Sales ON Sales.ProductName = Stock.ProductName AND Sales.LandingAmt = Stock.LandinPrice
AND Sales.MRPPrice = Stock.MRP
GROUP BY
Stock.ProductName,
Stock.MRP,
Stock.LandinPrice

Convert heavy asp code to run in sql

A bit of asp code which in context does; timeline the product's sales performance based on the start year which product was online to the number of years active, for example a product published on 2000 would have the peak sales for couple of years but by 2004 there would be none...but code this is for all products in the system and grouping by year...(if you questions I can answer them or I think looking at code would give a fair idea on what it's doing !!)
Firstly it runs this query to get the data ( which I want to change ):
SELECT Products.ProductID, Products.AnticipatedSalesPattern, Convert(Char(10),Invoices.Date,103) As [date], Orders.Cost
FROM (Orders INNER JOIN Products ON Orders.ProductID = Products.ProductID)
INNER JOIN Invoices ON Orders.Invoice = Invoices.InvoiceNum
WHERE (Products.IsResource=1 AND Orders.Returned<>1 AND Orders.Cost<>0)
ORDER BY Products.ProductID, Invoices.Date;
Here is how it process the data(in asp), I know it's not effective just made it sample modeling data...
while not dbrecords.eof
RecordsCount = RecordsCount + 1
if dbrecords("ProductID") <> LastPID then
PIDsCount = PIDsCount + 1
LastPID = dbrecords("ProductID")
FirstSaleDate = dbrecords("Date")
if month(FirstSaleDate) < 9 then
FirstSchoolYear = Year(FirstSaleDate) - 1
else
FirstSchoolYear = Year(FirstSaleDate)
end if
end if
YearsSinceFirstSale = int(DateDiff("d",FirstSaleDate,dbrecords("Date"))/365)
MyArray(FirstSchoolYear-2000,YearsSinceFirstSale) = MyArray(FirstSchoolYear-2000,YearsSinceFirstSale) + dbrecords("Cost")
MyArrayTotals(FirstSchoolYear-2000) = MyArrayTotals(FirstSchoolYear-2000) + dbrecords("Cost")
TotalSales = TotalSales + dbrecords("Cost")
dbrecords.movenext
wend
Now what I like is to remove the whole process of putting the data into an array and the query to return the yearly data
What i'm curently stuck on is to write sql to pass the years since first sale and keeping track of each products (how to implement that in sql), I beleieve parameters are accepted via CTE statements...
Also if you think that this can be achieved some other way much effieciently that would be great too
Any help would be greatly appreciated...
So far I got to;
This doesn't provide with the identical data produced by the asp...and also let me know if there is a better way to do this...
DROP VIEW inline_view;
GO
CREATE VIEW inline_view AS
SELECT p2.ProductID, p2.AnticipatedSalesPattern, Invoices.Date, Orders.Cost, case when MONTH(date) < 9 then YEAR(date)-1 else YEAR(date) end as year,
(select top 1 case when MONTH(i.date) < 9 then YEAR(i.date)-1 else YEAR(i.date) end from Invoices i inner join Orders o on i.InvoiceNum=o.Invoice
inner join Products p on o.ProductID = p.ProductID where p2.ProductID = p.productID order by i.Date asc) as startsale
FROM (Orders INNER JOIN Products p2 ON Orders.ProductID = p2.ProductID)
INNER JOIN Invoices ON Orders.Invoice = Invoices.InvoiceNum
WHERE (p2.IsResource=1 AND Orders.Returned<>1 AND Orders.Cost<>0)
;
GO
SELECT * FROM (
select SUM(cost) sum, datediff(y, startsale, year) as year, startsale from inline_view
group by year, startsale
) as data
PIVOT
(
sum(sum)
--years after product is online
FOR year IN ([1], [2], [3],[4],[5],[6],[7],[8],[9], [10],[11], [12])
) as pvt;