This question already has answers here:
How can I SELECT rows with MAX(Column value), PARTITION by another column in MYSQL?
(22 answers)
Closed 11 months ago.
I am trying to get data from my database.
Query upon sub-query upon another sub-query - and as the intermediate result I get looks like this:
item
quantity
pen
34
pencil
42
notebook
42
eraser
12
I need to build another query upon this result set to get the rows where item_quantity has it's maximal value (42 in the example above). The rows with pencils and notebooks. However, I have found out that the task is a bit trickier than I expected it to be.
SELECT * FROM sub_query_result HAVING quantity = MAX(quantity)
always returns an empty result set
SELECT * FROM sub_query_result HAVING quantity = 42
is pointless since I need to know the exact max quantity in advance
SELECT * FROM sub_query_result WHERE quantity = MAX(quantity)
simply works not ("Invalid use of group function")
I can see solutions that work but that I do not like -- due to extra actions I need to take on my back-end code that executes this sql request, or due to their inefficiency:
I can create a temporary table, get max. quantity from it and place to a variable. Then I can use this variable inside the query to that temporary table and get the data I need.
I can do
SELECT * FROM query_result HAVING quantity = (SELECT MAX(quantity) FROM
<Query upon sub-query upon another sub-query that shall return query_result>)
but that way I request the very same data twice! which in general is not a good approach.
So... Anything I missed? Any simple and elegant solutions that can solve my problem?
Order by quantity descending to get the max values first. Only select the first row, i.e. the one having the max values. ANSI SQL version:
SELECT * FROM query_result
ORDER BY quantity DESC
FETCH FIRST 1 ROW WITH TIES
WITH TIES means you can get several rows, if there are several rows having the same maximum quantity.
Have a query that uses double select (with select max) to fetch the row with the latest 'calculation_time' column among multiple rows which can have the same 'patient_set_id'. If there are multiple rows with the same 'patient_set_id', only the row with the latest 'calculation_time' should be retrieved. Calculation time is a date.
So far I've tried this but I'm not really sure if there is any better way for this, maybe using ORDER BY. But I'm very new to sql and need to know which one would be the fastest and more appropriate?
SELECT median from diagnostic_risk_stats WHERE
calculation_time=(SELECT MAX(calculation_time) FROM diagnostic_risk_stats WHERE
patient_set_id = UNHEX(REPLACE('5a9dbfca-74d6-471a-af27-31beb4b53bb2', "-","")));
You can use not exists as follows:
SELECT median
from diagnostic_risk_stats t
WHERE not exists
(Select 1 from diagnostic_risk_stats tt
Where t.patient_set_id = tt.patient_set_id
And tt.calculation_time > t.calculation_time)
And t.patient_set_id = UNHEX(REPLACE('5a9dbfca-74d6-471a-af27-31beb4b53bb2', "-",""));
Updating this post for better clarity.
The following issue is due to the CROSS-APPLY (I also hard a CROSS-JOIN), but I cannot find an easy way using an inner join or partition by to
make each record unique - no duplicates and also
2) make sure that each unique record is in order...174..175, 176, 177 etc. I expect about 3200 records here, not 174000 ish which I am getting.
Can someone provide their take as to what I can do to ensure I get the records in order? Thanks community! See code and data.
After executing the below query - I get the following results. As you see in the below data, if I order by Contract I get duplicates - which I am not supposed to do with Id's generated for each cross apply. The cross apply is causing these duplicates of the same record with different ids generated - the way it should work is once the record is created with the id - that should be the only instance of that record.
I have also highlighted in bold the parts of my code that get me the row id's following the latest max instance of the id from the previous selected row - but causes me duplication.
So for setting the expectation - the image below of a sub-select tells me what already exists in the table that I am referencing logic against - tells me what DOES NOT exist
173 records - see image
The next image tells me that my entire query is selecting what DOES NOT EXIST and generating the records starting at line 174 - that is very good but, I was alarmed at how many records I was getting - there are only 3875 distinct contracts so I should only be getting 3875 records - not 174ish as you see.
Way too many records - see image
So I re-ordered the query by Contract_TaskOrder - and I am seeing a ton of duplicates and re-assingnment of the duplicates a new id. See image below. The cross-apply is causing records for each unique instance of a contract in the [Stage].[BUD_PLN_Spendpln_Dataload_Staging_E_Grouped] table and creating how many times it is defined via a cartesian product - this is very very bad. I am trying to find a way with an inner join to do this - but to no avail.
As you see - duplicates names of contracts - crap...see this
Here is my full query - the bold parts need to be improved to allow for not only the duplicates to be removed but also the id to start from 174 and go in order for each unique NAME - tied to Contract-TaskOrder. If I just did a partition by to include the top 1 or max record of each combo I would have records with very widely sparsed ACLORDER ids based off the sequence.
SELECT * FROM
(
SELECT DISTINCT
'READWRITE' AS [ACCESSMODE],
3 AS [ACCESSMODEORDER],
**coalesce(const.maxs, 0) + row_number() over (
order by (select NULL)) AS [ACLORDER]**,
0 AS [ACLSFK],
'MEMBER' AS [FLAG],
4 AS [FLAGORDER],
'N' AS [ISUSER],
5 AS [ISUSERORDER],
('SPS-Contract-' + Contract_TaskOrder) AS [NAME],
0 AS [NAMEORDER],
Contract_TaskOrder AS [OBJECTNAME],
1 AS [OBJECTNAMEORDER],
'SL_DIMENSION' AS [OBJECTTYPE],
2 AS [OBJECTTYPEORDER],
NULL AS [PARENT],
NULL AS [PARENTORDER]
FROM
**[Stage].[BUD_PLN_Spendpln_Dataload_Staging_E_Grouped] cross apply (
select max(ACLORDER) as maxs
from [BFS_ODI_WORK].[Stage].[BUD_PLN_SECURITY_MEMBERS] ) const
)**
M
WHERE
1 = 1
AND NOT EXISTS
(
SELECT
1
FROM
[BFS_ODI_WORK].[Stage].[BUD_PLN_SECURITY_MEMBERS] k
WHERE
M.[NAME] = k.[NAME]
AND M.[FLAG] = k.[FLAG]
AND M.[OBJECTNAME] = k.[OBJECTNAME]
AND M.[OBJECTTYPE] = k.[OBJECTTYPE]
)
order by
NAME
SELECT DISTINCT
MAX(pb.revision) OVER (PARTITION BY pb.batch) AS REVNO,
pb.lot, pb.batch
FROM
db1 ps
INNER JOIN
db2 pb ON ps.Title = pb.Lot
WHERE
pb.proposalsubmitted IS NOT NULL
AND PS.Contract LIKE 'CLS'
AND BATCH <> 6
GROUP BY
pb.lot, pb.batch, pb.revision
ORDER BY
pb.lot
Above is my code snippet - I'm trying to retrieve the Max Revision number per batch. There can be multiple batches per LOT number. An added problem is that this database was setup in SharePoint at one time and the batch column is actually a VARCHAR and not Integer. I'm not sure if that is what the issue is because every time I do a pull I end up with:
The actual batch numbers and corresponding revisions are here:
Thanks guys - it maybe a fundamental issue on my part or a misunderstanding of the over() partition by
I am attempting to pull ALOT of data from a fox pro database, work with it and insert it into a mysql db. It is too much to do all at once so want to do it in batches of say 10 000 records. What is the equivalent to LIMIT 5, 10 in Fox Pro SQL, would like a select statement like
select name, address from people limit 5, 10;
ie only get 10 results back, starting at the 5th. Have looked around online and they only make mention of top which is obviously not of much use.
Take a look at the RecNo() function.
FoxPro does not have direct support for a LIMIT clause. It does have "TOP nn" but that only provides the "top-most records" within a given percentage, and even that has a limitation of 32k records returned (maximum).
You might be better off dumping the data as a CSV, or if that isn't practical (due to size issues), writing a small FoxPro script that auto-generates a series of BEGIN-INSERT(x10000)-COMMIT statements that dump to a series of text files. Of course, you would need a FoxPro development environment for this, so this may not apply to your situation...
Visual FoxPro does not support LIMIT directly.
I used the following query to get over the limitation:
SELECT TOP 100 * from PEOPLE WHERE RECNO() > 1000 ORDER BY ID;
where 100 is the limit and 1000 is the offset.
It is very easy to get around LIMIT clause using TOP clause ; if you want to extract from record _start to record _finish from a file named _test, you can do :
[VFP]
** assuming _start <= _finish, if not you get a top clause error
*
_finish = MIN(RECCOUNT('_test'),_finish)
*
SELECT * FROM (SELECT TOP (_finish - _start + 1) * FROM (SELECT TOP _finish *, RECNO() AS _tempo FROM _test ORDER BY _tempo) xx ORDER BY _tempo DESC) yy ORDER BY _tempo
**
[/VFP]
I had to convert a Foxpro database to Mysql a few years ago. What I did to solve this was add an auto-incrementing id column to the Foxpro table and use that as the row reference.
So then you could do something like.
select name, address from people where id >= 5 and id <= 10;
The Foxpro sql documentation does not show anything similar to limit.
Here, adapt this to your tables. Took me like 2 mins, i do this waaaay too often.
N1 - group by whatever, and make sure you got a max(id), you can use recno() to make one, sorted correctly
N2 - Joins N1 where the ID = Max Id of N1, display the field you want from N2
Then if you want to join to other tables, put that all in brackets and give it an alias and include it in a join.
Select N1.reference, N1.OrderNoteCount, N2.notes_desc LastNote
FROM
(select reference, count(reference) OrderNoteCount, Max(notes_key) MaxNoteId
from custnote
where reference != ''
Group by reference
) N1
JOIN
(
select reference, count(reference) OrderNoteCount, notes_key, notes_desc
from custnote
where reference != ''
Group by reference, notes_key, notes_desc
) N2 ON N1.MaxNoteId = N2.notes_key
To expand on Eyvind's answer I would create a program to uses the RecNo() function to pull records within a given range, say 10,000 records.
You could then programmatically cycle through the large table in chucks of 10,000 records at a time and preform your data load into you MySQL database.
By using the RecNO() function you can be certain not to insert rows more than once, and be able to restart at a know point in the data load process. That by it's self can be very handy in the event you need to stop and restart the load process.
Depending on the number of the returned rows and if you are using .NET Framework you can offset/limit the gotten DataTable on the following way:
dataTable = dataTable.AsEnumerable().Skip(offset).Take(limit).CopyToDataTable();
Remember to add the Assembly System.Data.DataSetExtensions.