SQL Server list too long - sql

I am trying to run this SQL statement:
select *
from table
where 1 = 1
and date >= '1/1/2020'
and id = any (**'list of 1300 items'**)
order by date asc;
The issue is that my list in the second and statement is actually over 1000 expressions, so it does not let me run this statement. Does anyone know a better way to do this? Basically, I have and ID that I want to locate in our database and the easiest way I know to do this is with a list but clearly mine is too long. I'm new to this so any insight would be greatly appreciated!

There are multiple ways to do that. For example, if your SQL server version is not old, you could use openJSON(). ie:
DECLARE #ids VARCHAR(MAX) = '[1,3,34,434,43]' -- your 1300+ ids
select *
from yourtable
where [date] >= '20200101'
and id IN (SELECT [value] FROM OPENJSON(#ids))
order by [date] asc;

You can think of string_split, if your SQL Server version is 2016 or later. Thanks to #Cetin Bazos for the basic script.
DECLARE #ids NVARCHAR(MAX) = '1,3,34,434,43' -- your 1300+ ids
select *
from yourtable
where [date] >= '20200101'
and id IN (SELECT [value] FROM STRING_SPLIT(#ids,','))
order by [date] asc;

Related

How can I improve the native query for a table with 7 millions rows?

I have the below view(table) in my database(SQL SERVER).
I want to retrieve 2 things from this table.
The object which has the latest booking date for each Product number.
It will return the objects = {0001, 2, 2019-06-06 10:39:58} and {0003, 2, 2019-06-07 12:39:58}.
If all the step number has no booking date for a Product number, it wil return the object with Step number = 1. It will return the object = {0002, 1, NULL}.
The view has 7.000.000 rows. I must do it by using native query.
The first query that retrieves the product with the latest booking date:
SELECT DISTINCT *
FROM TABLE t
WHERE t.BOOKING_DATE = (SELECT max(tbl.BOOKING_DATE) FROM TABLE tbl WHERE t.PRODUCT_NUMBER = tbl.PRODUCT_NUMBER)
The second query that retrieves the product with booking date NULL and Step number = 1;
SELECT DISTINCT *
FROM TABLE t
WHERE (SELECT max(tbl.BOOKING_DATE) FROM TABLE tbl WHERE t.PRODUCT_NUMBER = tbl.PRODUCT_NUMBER) IS NULL AND t.STEP_NUMBER = 1
I tried using a single query, but it takes too long.
For now I use 2 query for getting this information but for the future I need to improve this. Do you have an alternative? I also can not use stored procedure, function inside SQL SERVER. I must do it with native query from Java.
Try this,
Declare #p table(pumber int,step int,bookdate datetime)
insert into #p values
(1,1,'2019-01-01'),(1,2,'2019-01-02'),(1,3,'2019-01-03')
,(2,1,null),(2,2,null),(2,3,null)
,(3,1,null),(3,2,null),(3,3,'2019-01-03')
;With CTE as
(
select pumber,max(bookdate)bookdate
from #p p1
where bookdate is not null
group by pumber
)
select p.* from #p p
where exists(select 1 from CTE c
where p.pumber=c.pumber and p.bookdate=c.bookdate)
union all
select p1.* from #p p1
where p1.bookdate is null and step=1
and not exists(select 1 from CTE c
where p1.pumber=c.pumber)
If performance is main concern then 1 or 2 query do not matter,finally performance matter.
Create NonClustered index ix_Product on Product (ProductNumber,BookingDate,Stepnumber)
Go
If more than 90% of data are where BookingDate is not null or where BookingDate is null then you can create Filtered Index on it.
Create NonClustered index ix_Product on Product (ProductNumber,BookingDate,Stepnumber)
where BookingDate is not null
Go
Try row_number() with a proper ordering. Null values are treated as the lowest possible values by sql-server ORDER BY.
SELECT TOP(1) WITH TIES *
FROM myTable t
ORDER BY row_number() over(partition by PRODUCT_NUMBER order by BOOKING_DATE DESC, STEP_NUMBER);
Pay attention to sql-server adviced indexes to get good performance.
Possibly the most efficient method is a correlated subquery:
select t.*
from t
where t.step_number = (select top (1) t2.step_number
from t t2
where t2.product_number = t.product_number and
order by t2.booking_date desc, t2.step_number
);
In particular, this can take advantage of an index on (product_number, booking_date desc, step_number).

QSqlQuery Not Executing When Using Table Variable

I am using Python 3 and PyQt 5 to execute a query that includes a table variable. My other queries work, but this specific one is causing me issues. The query is:
declare #Temp table
(
SKU varchar(25),
Lines int
)
insert into #Temp
SELECT convert(varchar, [Sku]) as [SKU]
,count([Order Number]) as [Lines]
from [Database].dbo.[Table Name]
GROUP BY [Sku]
ORDER BY [LINES]
Select RANK() OVER(Order by Lines Desc) / (Select Convert(float,Count(*)) from #Temp) as [CUM PERCENT SKU]
,sum(Lines) Over(Order by Lines Desc) / convert(float,sum(Lines) Over()) as [CUM PERCENT LINES]
from #Temp
Order By [Lines] DESC
I replaced the Database name and the Table name, but the rest of the text is the exact query.
Executing the above in SQL Server 2012 works perfectly. Is there a limitation to QSql that I don't know about?
Thank you in advance!
After a bit more research, it turns out that you need to use prepare() when you are trying to create a table (apparently even a temporary one).
I'm posting the answer in case anyone else has the same issue and doesn't feel like going through the C++ StackOverflow answers.
query = QSqlQuery()
query.prepare(*query text here*)
query.exec_()

Data changes in a CTE after SELECT query

I have a query like this :
USE [MyDataBase]
GO
DECLARE #day DATE = '2017-05-18'
DECLARE #camp VARCHAR(500) = '9015';
WITH ODCALLS AS (SELECT * FROM [dbo].[ODCalls_2017_05]
WHERE CONVERT(DATE, CallLocalTime) = #day AND LastCampaign = #camp AND CallType = 1
)
SELECT COUNT(*) FROM ODCALLS -- this returns 2998
SELECT DATEPART(HOUR, CallLocalTime) AS dHOUR, COUNT(*) AS [Calls Received]
FROM ODCALLS
GROUP BY DATEPART(HOUR, CallLocalTime)
ORDER BY dHOUR -- this returns 24 rows as there are 24 hours in a day
SELECT COUNT(*) FROM ODCALLS -- this returns 2907
The instance is SQL SERVER EXPRESS 2014 and the data in the OdCalls_2017_05 table doesn't change, here' a preview of the results :
I have no idea why this happens, does any one have any explanation ?
Thanks
I'm surprised that works at all. Common Table Expressions do not act like temporary tables. The query is used as input into the next query (like a View can), and optimization may be performed across both the subsequent query and the CTE, so a result set from the CTE may never exist on its own.
Subsequent calls to the CTE shouldn't work, as that violates the definition of a CTE. See https://blogs.sentryone.com/aaronbertrand/backtobasics-ctes/ for more.

Fetch data from table using SQL

I have a table named "Orders" with 1-1000 rows and 3 columns (S.no, Order and Status). I need to fetch Order from 50-1000 which has its Status as "Cancelled". How can i do this in SQL Server?
Logic operator:
SELECT Order
FROM Orders
WHERE Status = 'Cancelled'
AND (S.no > 50 AND S.no < 1000)
BETWEEN:
SELECT Order
FROM Orders
WHERE Status = 'Cancelled'
AND (S.no BETWEEN 50 and 1000)
select *
from orders
where no between 50 and 1000
and status = 'Cancelled'
Assuming you meant to say that the column was named "no". S.no would not be a valid column name.
You can try something like this:
SELECT *
FROM Orders
WHERE (S.no BETWEEN 50 AND 1000) AND (Status = 'Cancelled')
Hope this helps
If you're using SQL Server, you don't have access to Limit and Offset (unless that's changed in the last year or so, in which case please someone correct me).
There's a really nice generalizable solution discussed here: Equivalent of LIMIT and OFFSET for SQL Server?
I'd definitely take a look at that. If indeed your s_no values range from 1-1000, then the solution above by Notulysses should work just fine. But if you don't have so_no between 1-1000 (or in some other easy to filter way) then check out the solution linked to above. If you can what Notulysses recommended, go for it. If you need a generalizable solution, the one above is very good. I've also copied it below, for reference
;WITH Results_CTE AS
(
SELECT
Col1, Col2, ...,
ROW_NUMBER() OVER (ORDER BY SortCol1, SortCol2, ...) AS RowNum
FROM Table
WHERE <whatever>
)
SELECT *
FROM Results_CTE
WHERE RowNum >= #Offset
AND RowNum < #Offset + #Limit

How to write this sql query

I have a SQL Server table with the following structure
cod_turn (PrimaryKey)
taken (bit)
time (datetime)
and several other fields which are irrelevant to the problem. I cant alter the table structure because the app was made by someone else.
given a numeric variable parameter, which we will assume to be "3" for this example, and a time, I need to create a query which looking from that time on, it looks the first 3 consecutive records which are not marked as "taken". I cant figure out how to make the query in pure sql, if possible.
PS: I accepted the answer because it was correct, but I made a bad description of the problem. I will open another question later. Feeling stupid after seeing the size of the answers =)
SELECT TOP 3 * FROM table WHERE taken = 0 AND time>=#Time ORDER BY time
Where #Time is whatever time you pass in.
Assuming current versions of SQL Server and assuming you've named you "numeric variable parameter" as #top int. Note:the parenthesis around #top are required when using a parameter-ized TOP
SELECT TOP (#top)
cod_turn,
taken ,
time
FROM yourtable
WHERE Taken = 0 AND time>=#Time
ORDER BY time DESC
You can also do
with cte as
(
SELECT
ROW_NUMBER() over (order by time desc) rn
cod_turn,
taken ,
time
FROM yourtable
WHERE Taken = 0 AND time>=#Time
)
SELECT
cod_turn,
taken ,
time
FROM CTE
WHERE rn <= #top
ORDER BY time DESC
SELECT TOP 3
*
FROM
table
WHERE
time >= #inserted_time
AND taken = 0
ORDER BY
cod_turn ASC
select MT.*
from
(
select cod_turn, ROW_NUMBER() OVER (ORDER BY cod_turn) [RowNumber] -- or by time
from myTable
where taken = 0
and time >= #myTime
) T
inner join myTable MT on MT.cod_turn = T.cod_turn
where T.RowNumber < #myNumber
select top 3 * from theTable where taken = 0 and time > theTime orderby time