I want to do a paging system using php
i'am using access for my data base and it doesn't allow me to do a limit in my selected request.
SELECT * FROM table LIMIT 10,20
I want to add a column in my request so that i can have temporary IDs with successive integer that will allow me to do between condition
|id_temp | id | name |
| 1 | 56 | exmp |
| 2 | 180 | exmp |
| 3 | 193 | exmp |
| 4 | 194 | exmp |
| 5 | 363 | exmp |
| 6 | 500 | exmp |
I tried to use TOP as an alternative solution but it only allows me to select the top 10 and not the others.
Then I took the ID of my elements in order to use between but the problem is that i can not have the same amount everytime in my result. Moreover i have a really limited server and a big request so I can't use not in.
Access does not support OFFSET, so what you can do is a trick like this:
select top 10 *
from tablename
where id > (select max(id) from (select top 30 id from tablename order by id ))
order by id
it will return the rows from 31st to 40th ordered by id.
Note: never use TOP or LIMIT in sql without ORDER BY because the result is not guaranteed to be what you expect.
If you want that id_temp column, you can get it like this:
SELECT
((select count(*) from tablename where id < t.id)+1) AS id_temp,
t.*
FROM tablename AS t
ORDER BY t.id;
Related
I have table
CREATE TABLE ard_signals
(id, val, str, date_val, dt);
This table records the values of all ID's signals. Unique ID's around 950. At the moment, the table contains about 50 million rows with different values of these signals. Each ID can have only a numeric values, string values or date values.
I get the last value of each ID, which, by condition, is less than input date:
select ID,
max(val) keep (dense_rank last order by dt desc) as val,
max(str) keep (dense_rank last order by dt desc) as str,
max(date_val) keep (dense_rank lastt order by dt desc) as date_val,
max(dt)
where dt <= to_date(any_date)
group by id;
I have index on ID. At the moment, the request takes about 30 seconds. Help, please, what ways of optimization it is possible to make for the given request?
EXPLAIN PLAN: with dt index
Example Data(This kind of rows are about 950-1000):
ID
VAL
STR
DATE_VAL
DT
920
0
20.07.2022 9:59:11
490
yes
20.07.2022 9:40:01
565
233
20.07.2022 9:32:03
32
1
20.07.2022 9:50:01
TL;DR You need your application to maintain a table of distinct id values.
So, you want the last record for each group (distinct id value) in your table, without doing a full table scan. It seems like it should be easy to tell Oracle: iterate through the distinct values for id and then do an index lookup to get the last dt value for each id and then give me that row. But looks are deceiving here -- it's not easy at all, if it is even possible.
Think about what an index on (id) (or (id, dt)) has. It has a bunch of leaf blocks and a structure to find the highest value of id in each block. But Oracle can only use the index to get all the distinct id values by reading every leaf block in the index. So, we might be find a way to trade our TABLE FULL SCAN for an INDEX_FFS for a marginal benefit, but it's not really what we're hoping for.
What about partitioning? Can't we create ARD_SIGNALS with PARTITION BY LIST (id) AUTOMATIC and use that? Now the data dictionary is guaranteed to have a separate partition for each distinct id value.
But again - think about what Oracle knows (from DBA_TAB_PARTITIONS) -- it knows what the highest partition key value is in each partition. It is true: for a list partitioned table, that highest value is guaranteed to be the only distinct value in the partition. But I think using that guarantee requires special optimizations that Oracle's CBO does not seem to make (yet).
So, unfortunately, you are going to need to modify your application to keep a parent table for ARDS_SIGNALS that has a (single) row for each distinct id.
Even then, it's kind of difficult to get what we want. Because, again, want Oracle to iterate through the distinct id values, then use the index to find the one with the highest dt for that id .. and then stop. So, we're looking for an execution plan that makes use of the INDEX RANGE SCAN (MIN/MAX) operation.
I find that tricky with joins, but not so hard with scalar subqueries. So, assuming we named our parent table ARD_IDS, we can start with this:
SELECT /*+ NO_UNNEST(#ssq) */ i.id,
( SELECT /*+ QB_NAME(ssq) */ max(dt)
FROM ard_signals s
WHERE s.id = i.id
AND s.dt <= to_date(trunc(SYSDATE) + 2+ 10/86400) -- replace with your date variable
) max_dt
FROM ard_ids i;
---------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Starts | E-Rows | A-Rows | A-Time | Buffers |
---------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | | 1000 |00:00:00.01 | 22 |
| 1 | SORT AGGREGATE | | 1000 | 1 | 1000 |00:00:00.02 | 3021 |
| 2 | FIRST ROW | | 1000 | 1 | 1000 |00:00:00.01 | 3021 |
|* 3 | INDEX RANGE SCAN (MIN/MAX)| ARD_SIGNALS_N1 | 1000 | 1 | 1000 |00:00:00.01 | 3021 |
| 4 | TABLE ACCESS FULL | ARD_IDS | 1 | 1000 | 1000 |00:00:00.01 | 22 |
---------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
3 - access("S"."ID"=:B1 AND "S"."DT"<=TO_DATE(TO_CHAR(TRUNC(SYSDATE#!)+2+.00011574074074074074
0740740740740740740741)))
Note the use of hints to keep Oracle from merging the scalar subquery into the rest of the query and losing our desired access path.
Next, it is a matter of using those (id, max(dt)) combinations to look up the rows from the table to get the other column values. I came up with this; improvements may be possible (especially if (id, dt) is not as selective as I am assuming it is):
with k AS (
select /*+ NO_UNNEST(#ssq) */ i.id, ( SELECT /*+ QB_NAME(ssq) */ max(dt) FROM ard_signals s WHERE s.id = i.id AND s.dt <= to_date(trunc(SYSDATE) + 2+ 10/86400) ) max_dt
from ard_ids i
)
SELECT k.id,
max(val) keep ( dense_rank first order by dt desc, s.rowid ) val,
max(str) keep ( dense_rank first order by dt desc, s.rowid ) str,
max(date_val) keep ( dense_rank first order by dt desc, s.rowid ) date_val,
max(dt) keep ( dense_rank first order by dt desc, s.rowid ) dt
FROM k
INNER JOIN ard_signals s ON s.id = k.id AND s.dt = k.max_dt
GROUP BY k.id;
--------------------------------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Starts | E-Rows | A-Rows | A-Time | Buffers | OMem | 1Mem | Used-Mem |
--------------------------------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | | 1000 |00:00:00.04 | 7009 | | | |
| 1 | SORT GROUP BY | | 1 | 1000 | 1000 |00:00:00.04 | 7009 | 302K| 302K| 268K (0)|
| 2 | NESTED LOOPS | | 1 | 1005 | 1000 |00:00:00.04 | 7009 | | | |
| 3 | NESTED LOOPS | | 1 | 1005 | 1000 |00:00:00.03 | 6009 | | | |
| 4 | TABLE ACCESS FULL | ARD_IDS | 1 | 1000 | 1000 |00:00:00.01 | 3 | | | |
|* 5 | INDEX RANGE SCAN | ARD_SIGNALS_N1 | 1000 | 1 | 1000 |00:00:00.02 | 6006 | | | |
| 6 | SORT AGGREGATE | | 1000 | 1 | 1000 |00:00:00.02 | 3002 | | | |
| 7 | FIRST ROW | | 1000 | 1 | 1000 |00:00:00.01 | 3002 | | | |
|* 8 | INDEX RANGE SCAN (MIN/MAX) | ARD_SIGNALS_N1 | 1000 | 1 | 1000 |00:00:00.01 | 3002 | | | |
| 9 | TABLE ACCESS BY GLOBAL INDEX ROWID| ARD_SIGNALS | 1000 | 1 | 1000 |00:00:00.01 | 1000 | | | |
--------------------------------------------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
5 - access("S"."ID"="I"."ID" AND "S"."DT"=)
8 - access("S"."ID"=:B1 AND "S"."DT"<=TO_DATE(TO_CHAR(TRUNC(SYSDATE#!)+2+.000115740740740740740740740740740740740741)))
... 7000 gets and 4/100ths of a second.
You want to see that latest entry per ID at a given time.
If you would usually be interested in times at the beginning of the recordings, an index on the time could help to limit the rows to have to work with. But I consider this highly unlikely.
It is much more likely that you are interested in the situation at more recent times. This means for instance that when looking for the latest entries until today, for one ID the latest entry may be found yesterday, while for another ID the latest entry my be from two years ago.
In my opinion, Oracle already chooses the best approach to deal with this: read the whole table sequentially.
If you have several CPUs at hand, parallel execution might speed up things:
select /*+ parallel(4) */ ...
If you really need this to be much faster, then you may want to consider an additional table with one row per ID, which gets a copy of the lastest date and value by a trigger. I.e. you'd introduce redundancy for the gain of speed, as it is done in data warehouses.
I have created a Visual Studio application that tracks the amount of time someone spends on some another desktop application (a productivity app, if you may). This application has a backend of an SQL server (MSSQL specifically, considering the dialect). In a page of this application, I am attempting to select and display some already input data through an SQL query. I want the query to carry out the following:
The accessed desktop apps that have the same name are grouped together (1) and the amount of time that is spent on these is added (2). Then, these are ordered in an descending order (3), and the top 5 records are selected (4).
E.g.
CREATE TABLE ApplicationInfo {
AppName varchar(50)
Duration int
}
INSERT INTO ApplicationInfo (AppName, Duration)
Values
(Word, 1), (Spotify, 2)
(Spotify, 1), (Word, 2),
(Discord, 2), (Access, 4),
(Notepad, 3)
As a proper table, that is:
| AppName | Duration |
|---------|----------|
| Word | 1 |
| Spotify | 2 |
| Spotify | 1 |
| Word | 2 |
| Discord | 2 |
| Access | 4 |
| Notepad | 3 |
(I have tried using table markdown, but the table format does not render, for some reason)
After steps (1) and (2),
| AppName | Duration |
|---------|----------|
| Word | 3 |
| Spotify | 3 |
| Discord | 2 |
| Access | 4 |
| Notepad | 3 |
After steps (3) and (4),
| AppName | Duration |
|---------|----------|
| Access | 4 |
| Notepad | 3 |
| Spotify | 3 |
| Word | 3 |
| Discord | 2 |
I have tried this SQL statement with test data, but it does not return what I want it to return:
SELECT TOP 5 AppName, DurationHrs, SUM(DurationHrs) OVER (PARTITION BY AppName) FROM ApplicationInfo ORDER BY SUM(DurationHrs) DESC;
I believe the LIMIT keyword would not be suitable for this, due to the MSSQL dialect (as opposed to MySQL). Could someone suggest an SQL statement that would be appropriate for this scenario (following the MSSQL dialect)?
SELECT TOP 5 AppName, SUM(DurationHrs)
FROM ApplicationInfo
GROUP BY AppName
ORDER BY SUM(DurationHrs) DESC
You're close. Ordering by SUM(DurationHrs) without the OVER (PARTITION BY AppName) requires you to do some grouping for the aggregate. If you just want to order by that third column, you could copy the whole expression or give it an alias and use that, or use the order by numeric shortcut:
SELECT TOP 5 AppName, DurationHrs, SUM(DurationHrs) OVER (PARTITION BY AppName) FROM ApplicationInfo ORDER BY SUM(DurationHrs) OVER (PARTITION BY AppName) DESC;
SELECT TOP 5 AppName, DurationHrs, SUM(DurationHrs) OVER (PARTITION BY AppName) total_durationHrs FROM ApplicationInfo ORDER BY total_durationHrs DESC;
SELECT TOP 5 AppName, DurationHrs, SUM(DurationHrs) OVER (PARTITION BY AppName) FROM ApplicationInfo ORDER BY 3 DESC;
If you want to group the data by AppName then you would do:
SELECT TOP 5 AppName, SUM(DurationHrs)
FROM ApplicationInfo
GROUP BY AppName
ORDER BY SUM(DurationHrs) DESC;
I have a table with the following layout, to store orders for users, and to remember which orders are being processed right now:
Sequence | User | Order | InProcess
---------+------+-------+----------
1 | 1 | 1 |
2 | 1 | 2 |
3 | 2 | 1 |
4 | 3 | 1 |
5 | 1 | 3 |
6 | 4 | 1 |
7 | 2 | 2 |
E.g., line 4 | 3 | 1 | means that the 4th order ever is for user 3, and it's his/her 1st order. Now I want to select the order which to process next. This has to be done according to the following criterias:
Older orders (with lower sequence numbers) are processed first.
Only one order is processed per user at once.
Once an order is selected as being processed it gets marked as InProcess.
Once an order is completed, it is deleted from this list.
So, after some time this may look like this:
Sequence | User | Order | InProcess
---------+------+-------+----------
1 | 1 | 1 | X
2 | 1 | 2 |
3 | 2 | 1 | X
4 | 3 | 1 | X
5 | 1 | 3 |
6 | 4 | 1 |
7 | 2 | 2 |
When now being asked for the next order to process, the answer would be the line with sequence number 6, since orders for users 1, 2 and 3 are already being processed, so no additional order for them may be processed. The question is: How do I get efficiently to this row?
Basically what I need is the SQL equivalent of
Of all orders, select the first order which is not in process, and whose user is not having an order already being processed.
The question is just how to tell this with SQL? BTW: I'm looking for a standard SQL solution, not DBMS-specific ways to go. However, if for whatever reason limiting the question to a specific DBMS, these are the ones I have to support (in this order):
PostgreSQL
MariaDB
MySQL
SQL Server
MongoDB
Any ideas?
I think captures your logic:
select t.*
from (select t.*, max(in_process) over (partition by user_id) as any_in_process
from t
) t
where any_in_process is null
order by sequence
fetch first 1 row only;
Fetching one row is database specific, but the rest is pretty generic.
You can get the next order to be processed by using the ROW_NUMBER() window function, as in:
select *
from (
select
*,
row_number() over(order by "order", "sequence") as as rn
from t
where "user" not in (
select "user" from t where inprocess = 'X'
)
) x
where rn = 1
Available in PostgreSQL, MariaDB 10.2, MySQL 8.0, SQL Server 2012.
Background:
There are two defined Databases - X and Y. They look exactly the same but contain different data and most importantly, the ID in database X goes from 0 to 1000 and the ID values in database Y go from 10000 to 10500. This cannot be changed!
The table layout (columns) in the databases (it's the same in both) looks like this:
+--------+-------+--------------------------------------+
| id | hash | more stuff ... |
+--------+-------+--------------------------------------+
ID is unique (primary key) and the hash column contains a value calculated from the other columns. The hash can be the same between different entries.
The problem
The table from the databases needs to be merged so that there is only one entry with a specific hash, no duplicates. But the tricky part is that if there is a duplicate where entry A from database X has the same hash value as entry B from database Y, the query should always take the one from database Y (or X if that is more preferred, but the SQL query should be able to choose which on that is prioritized).
Example:
This is an example of how two tables with entries could look like and how the result should be. In this example we prioritize table Y.
Database X table:
+--------+-------+--------------------------------------+
| id | hash | more stuff ... |
+--------+-------+--------------------------------------+
| 1 | 100 | ... |
| 2 | 101 | ... |
| 3 | 102 | ... |
+--------+---------+------------------------------------+
Database Y table:
+--------+-------+--------------------------------------+
| id | hash | more stuff ... |
+--------+-------+--------------------------------------+
| 10000 | 200 | ... |
| 10001 | 201 | ... |
| 10002 | 100 | ... |
+--------+---------+------------------------------------+
Note that the hash value of 100 exists in both tables! Y is prioritized and therefore entry with id = 10002 should exist in the merge, NOT id = 1. The query should therefore result in the following merge:
+--------+-------+--------------------------------------+
| id | hash | more stuff ... |
+--------+-------+--------------------------------------+
| 2 | 101 | ... |
| 3 | 102 | ... |
| 10000 | 200 | ... |
| 10001 | 201 | ... |
| 10002 | 100 | ... |
+--------+---------+------------------------------------+
Solution Ideas:
I came up with the following query during testing (not optimized just purely to get the correct result):
attach database 'X.db' as d1;
attach database 'Y.db' as d2;
SELECT * FROM
(
SELECT * FROM
(
SELECT *
FROM d1.table AS c1
UNION
SELECT *
FROM d2.table AS c2
) ORDER BY entryId DESC
) GROUP BY hashNum
This will actually return the result I want but it's not reliable. The result is correct because the GROUP BY implementation seems to choose the entries ordered last for the result. If entry with ID 10002 has hash value 100 and entry with ID 1 has 100 the GROUP BY syntax will either give me a result where ID 10002 or 1 exists depending on the ORDER BY statement (DESC gives entry with ID 1 and ASC gives entry with ID 10002).
This however relies on the implementation of GROUP BY. The SQL statement doesn't use any syntax to actually say from which database the result preferably should come from.
Is there a better way to make this query? SQLite doesn't support RIGHT and FULL OUTER JOIN).
You want all records from table Y, and those records from table X that do not have a hash in Y:
SELECT * FROM X WHERE hash NOT IN (SELECT hash FROM Y)
UNION ALL
SELECT * FROM Y
I have a table called documents that has the fields id, title, version and content.
Now I want to get all ids (or rows) for every title with the highest version.
Suppose I have the following data:
+----+-------+---------+---------+
| id | title | version | content |
+----+-------+---------+---------+
| 2 | foo | 1 | abc1 |
| 3 | foo | 2 | abc2 |
| 4 | foo | 3 | abc3 |
| 5 | bar | 1 | abcd1 |
| 6 | bar | 2 | abcd2 |
| 7 | baz | 1 | abcde1 |
+----+-------+---------+---------+
I want to receive either the ids 4,6,7 or the whole rows for these entries.
Performance is not an issue as there will be only a few hundred entries.
To retrieve the entire rows, you need to GROUP BY version with a MAX() aggregate, and join that as a subquery against the whole table to pull in the remaining columns. The JOIN condition will need to be against the combination of title, version since together they uniquely identify a title's greatest version.
The following should work independently of the RDBMS you are using:
SELECT
documents.id,
documents.title,
documents.version,
documents.content
FROM
documents
JOIN (
/* subquery pulls greatest version per title */
SELECT
title,
MAX(version) AS version
FROM documents
GROUP BY title
/* title, version pair is joined back against the main table */
) maxversions ON (documents.title = maxversions.title AND documents.version = maxversions.version)
This is a simple group by:
select max(version)
from documents
group by title
You can join back to documents, if you like, to get the full document information.
How about:
SELECT * FROM dataz
WHERE version IN (
SELECT TOP 1 version
FROM dataz
ORDER BY version DESC
)
This will get the IDs you want:
Select max(id) as id from documents
group by title
having count(1) = max(version)
order by id