I have to write a query wherein i need to allocate a ID (unique key) for a particular record which is not being used / is not being generated / does not exist in database.
In short, I need to generate an id for a particular record and show it on print screen.
E. g.:
ID Name
1 abc
2 def
5 ghi
So, the thing is that it should return ID=3 as the next immediate which is not being generated yet, and after this generation of the id, I will store this data back to database table.
It's not an HW: I am doing a project, and I have a requirement where I need to write this query, so I need some help to achieve this.
So please guide me how to make this query, or how to achieve this.
Thanks.
I am not able to add comments,, so thats why i am writing my comments here..
I am using MySQL as the database..
My steps would be like this:-
1) Retrieve the id from the database table which is not being used..
2) As their are no. of users (website based project), so i want no concurrency to happen,, so if one ID is generated to one user, then it should lock the database, until the same user recieves the id and store the record for that id.. After that, the other user can retrieve the ID whichever is not existing.. (Major requirement)..
How can i achive all these things in MySQL,, Also i suppose Quassnoi's answer will be worth,, but its not working in MySQL.. so plz explain the bit about the query as it is new to me.. and will this query work in MySQL..
I named your table unused.
SELECT id
FROM (
SELECT 1 AS id
) q1
WHERE NOT EXISTS
(
SELECT 1
FROM unused
WHERE id = 1
)
UNION ALL
SELECT *
FROM (
SELECT id + 1
FROM unused t
WHERE NOT EXISTS
(
SELECT 1
FROM unused ti
WHERE ti.id = t.id + 1
)
ORDER BY
id
LIMIT 1
) q2
ORDER BY
id
LIMIT 1
This query consists of two parts.
The first part:
SELECT *
FROM (
SELECT 1 AS id
) q
WHERE NOT EXISTS
(
SELECT 1
FROM unused
WHERE id = 1
)
selects a 1 is there is no entry in the table with this id.
The second part:
SELECT *
FROM (
SELECT id + 1
FROM unused t
WHERE NOT EXISTS
(
SELECT 1
FROM unused ti
WHERE ti.id = t.id + 1
)
ORDER BY
id
LIMIT 1
) q2
selects a first id in the table for which there is no next id.
The resulting query selects the least of these two values.
Depends on what you mean by "next id" and how it's generated.
If you're using a sequence or identity in the database to generate the id, it's possible that the "next id" is not 3 or 4 but 6 in the case you've presented. You have no way of knowing whether or not there were values with id of 3 or 4 that were subsequently deleted. Sequences and identities don't necessarily try to reclaim gaps; once they're gone you don't reuse them.
So the right thing to do is to create a sequence or identity column in your database that's automatically incremented when you do an INSERT, then SELECT the generated value.
The correct way is to use an identity column for the primary key. Don't try to look at the rows already inserted, and pick an unused value. The Id column should hold a number large enough that your application will never run out of valid new (higher) values.
In your description , if you are skipping values that you are trying to use later, then you are probably giving some meaning to the values. Please reconsider. You probably should only use this field as a look up (a reference) value from another table.
Let the database engine assign the next higher value for your ID. If you have more than one process running concurrently, you will need to use LAST_INSERT_ID() function to determine the ID that the database generated for your row. You can use LAST_INSERT_ID() function within the same transaction before you commit.
Second best (but not good!) is to use the max value of the index field plus one. You would have to do a table lock to manage the concurrency issues.
/*
This is a query script I wrote to illustrate my method, and it was created to solve a Real World problem where we have multiple machines at multiple stores creating transfer transactions in their own databases,
that are then synced to other databases on the store (this happens often, so getting the Nth free entry for the Nth machine should work) where the transferid is the PK and then those are synced daily to a MainFrame where the maximum size of the key (which is the TransactionID and StoreID) is limited.
*/
--- table variable declarations
/* list of used transaction ids (this is just for testing, it will be the view or table you are reading the transaction ids from when implemented)*/
DECLARE #SampleTransferIDSourceTable TABLE(TransferID INT)
/* Here we insert the used transaction numbers*/
DECLARE #WorkTable TABLE (WorkTableID INT IDENTITY (1,1), TransferID INT)
/*this is the same table as above with an extra column to help us identify the blocks of unused row numbers (modifying a table variable is not a good idea)*/
DECLARE #WorkTable2 TABLE (WorkTableID INT , TransferID INT, diff int)
--- Machine ID declared
DECLARE #MachineID INT
-- MachineID set
SET #MachineID = 5
-- put in some rows with different sized blocks of missing rows.
-- comment out the inserts after two to the bottom to see how it handles no gaps or make
-- the #MachineID very large to do the same.
-- comment out early rows to test how it handles starting gaps.
INSERT #SampleTransferIDSourceTable ( TransferID ) VALUES ( 1 )
INSERT #SampleTransferIDSourceTable ( TransferID ) VALUES ( 2 )
INSERT #SampleTransferIDSourceTable ( TransferID ) VALUES ( 4 )
INSERT #SampleTransferIDSourceTable ( TransferID ) VALUES ( 5 )
INSERT #SampleTransferIDSourceTable ( TransferID ) VALUES ( 6 )
INSERT #SampleTransferIDSourceTable ( TransferID ) VALUES ( 9 )
INSERT #SampleTransferIDSourceTable ( TransferID ) VALUES ( 10 )
INSERT #SampleTransferIDSourceTable ( TransferID ) VALUES ( 20 )
INSERT #SampleTransferIDSourceTable ( TransferID ) VALUES ( 21 )
INSERT #SampleTransferIDSourceTable ( TransferID ) VALUES ( 24 )
INSERT #SampleTransferIDSourceTable ( TransferID ) VALUES ( 25 )
INSERT #SampleTransferIDSourceTable ( TransferID ) VALUES ( 30 )
INSERT #SampleTransferIDSourceTable ( TransferID ) VALUES ( 31 )
INSERT #SampleTransferIDSourceTable ( TransferID ) VALUES ( 33 )
INSERT #SampleTransferIDSourceTable ( TransferID ) VALUES ( 39 )
INSERT #SampleTransferIDSourceTable ( TransferID ) VALUES ( 40 )
INSERT #SampleTransferIDSourceTable ( TransferID ) VALUES ( 50 )
-- copy the transaction ids into a table with an identiy item.
-- When implemented add where clause before the order by to limit to the local StoreID
-- Zero row added so that it will find gaps before the lowest used row.
INSERT #WorkTable (TransferID)
SELECT 0
INSERT #WorkTable (TransferID)
SELECT TransferID FROM #SampleTransferIDSourceTable ORDER BY TransferID
-- copy that table to the new table with the diff column
INSERT #WorkTable2
SELECT WorkTableID,TransferID,TransferID - WorkTableID
FROM #WorkTable
--- gives us the (MachineID)th unused ID or the (MachineID)th id beyond the highest id used.
IF EXISTS (
SELECT Top 1
GapStart.TransferID + #MachineID - (GapStart.diff + 1)
FROM #WorkTable2 GapStart
INNER JOIN #WorkTable2 GapEnd
ON GapStart.WorkTableID = GapEnd.WorkTableID - 1
AND GapStart.diff < GapEnd.diff
AND gapEnd.diff >= (#MachineID - 1)
ORDER BY GapStart.TransferID
)
SELECT Top 1
GapStart.TransferID + #MachineID - (GapStart.diff + 1)
FROM #WorkTable2 GapStart
INNER JOIN #WorkTable2 GapEnd
ON GapStart.WorkTableID = GapEnd.WorkTableID - 1
AND GapStart.diff < GapEnd.diff
AND gapEnd.diff >= (#MachineID - 1)
ORDER BY GapStart.TransferID
ELSE
SELECT MAX(TransferID) + #MachineID FROM #SampleTransferIDSourceTable
Should work under MySql.
SELECT TOP 100
T1.ID + 1 AS FREE_ID
FROM TABLE1 T1
LEFT JOIN TABLE2 T2 ON T2.ID = T1.ID + 1
WHERE T2.ID IS NULL
are you allowed to have a utility table? if so i would create a table like so:
CREATE TABLE number_helper (
n INT NOT NULL
,PRIMARY KEY(n)
);
Fill it with all positive 32 bit integers (assuming the id you need to generate is a positive 32 bit integer)
Then you can select like so:
SELECT MIN(h.n) as nextID
FROM my_table t
LEFT JOIN number_helper h ON h.n = t.ID
WHERE t.ID IS NULL
Haven't actually tested this but it should work.
Related
I have a table that looks like:
ID|CREATED |VALUE
1 |1649122158|200
1 |1649122158|200
1 |1649122158|200
That I'd like to look like:
ID|CREATED |VALUE
1 |1649122158|200
And I run the following query:
DELETE FROM MY_TABLE T USING (SELECT ID,CREATED,ROW_NUMBER() OVER (PARTITION BY ID ORDER BY CREATED DESC) AS RANK_IN_KEY FROM MY_TABLE T) X WHERE X.RANK_IN_KEY <> 1 AND T.ID = X.ID AND T.CREATED = X.CREATED
But it removes everything from MY_TABLE and not just other rows with the same value. This is more than just selecting distinct records, I'd like to enforce a unique constraint to get the latest value of ID and keep just one record for it, even if there were duplicates.
So
ID|CREATED |VALUE
1 |1649122158|200
1 |1649122159|300
2 |1649122158|200
2 |1649122158|200
3 |1649122170|500
3 |1649122160|200
Would become (using the same final unique constraint statement):
ID|CREATED |VALUE
1 |1649122159|300
2 |1649122158|200
3 |1649122170|500
How can I improve my logic to properly handle these unique constraint modifications?
Check out this post: https://community.snowflake.com/s/question/0D50Z00008EJgemSAD/how-to-delete-duplicate-records-
If all columns make up a unique records, the recommended solution is the insert all the records into a new table with SELECT DISTINCT * and do a swap. You could also do a INSERT OVERWRITE INTO the same table.
Something like INSERT OVERWRITE INTO tableA SELECT DISTINCT * FROM tableA;
The following setup should leave rows with id of 1 and 3. And not delete all rows as you say.
Schema
create table t (
id int,
created int ,
value int
);
insert into t values(1, 1649122158, 200);
insert into t values(1 ,1649122159, 300);
insert into t values(2 ,1649122158, 200);
insert into t values(2 ,1649122158, 200);
insert into t values(3 ,1649122170, 500);
insert into t values(3 ,1649122160, 200);
Delete statement
with x as (
SELECT
id, created,
row_number() over(partition by id) as r
FROM t
)
delete from t
using x
where x.id = t.id and x.r <> 1 and x.created = t.created
;
Output
select * from t;
1 1649122158 200
3 1649122170 500
The logic is such, that the table in the using clause is joined with the operated on table. Following the join logic, it just matches by some key. In your case, you have key as {id,created}. This key is duplicated for rows with id of 2. So the whole group is deleted.
I'm no savvy in database schemas. But as a thought, you may add a row with a rank to existing table. And after that you can proceed with deletion. This way you do not need to create other table and insert values to that. Be warned that data may become fragmented(physically, on disks). So you will need to run some kind of tune up later.
Update
You may find this almost one-liner interesting:
SO answer
I will duplicate code here, as it is so small and well written.
WITH
u AS (SELECT DISTINCT * FROM your_table),
x AS (DELETE FROM your_table)
INSERT INTO your_table SELECT * FROM u;
I have a table and i would like to replicate/clone records within the same table. However i would like to do that with a condition. And the condition is i have a column called recordcount with numeric values. For example Row 1 can take on a value of recordcount say 7, then i would like my row 1 to be replicated 7 times. Row 2 could take on a value say 9 then i would like row 2 to be replicated 9 times.
Any help is appreciated. Thank you
What you can do (and I'm pretty sure it's not a best practice),
Is to hold a table with just numbers, which has rowcount that correspond to the numeric value.
Join that with your table, and project your table only.
Example:
create table nums(x int);
insert into nums select 1;
insert into nums select 2;
insert into nums select 2;
insert into nums select 3;
insert into nums select 3;
insert into nums select 3;
create table t (txt varchar(10) , recordcount int);
insert into t select 'A',1;
insert into t select 'B',2;
insert into t select 'C',3;
select t.*
from t
inner join nums
on t.recordcount = nums.x
order by 1
;
Will project:
"A",1
"B",2
"B",2
"C",3
"C",3
"C",3
I have a recursive CTE which gives me a listing of a set of parent child keys as follows, lets say its in a temp table called [#relationtree]:
Parent | Child
--------------
1 | 3
3 | 5
5 | 6
5 | 9
I want to create a copy of these relationships into a table with, lets say, the following stucture:
CREATE TABLE [dbo].[Relations]
(
[Id] int identity(1,1)
[ParentId] int
)
How can I insert the above records but recursively obtain the previously inserted identity value to be able to insert that value as the ParentId column for each copy of a child I insert?
I would expect to have at the end of this in [dbo].[Relations] (given our current seed value is, say 50)
Id | ParentId
-------------
... other rows present before this query ...
50 | NULL
51 | 50
52 | 51
53 | 51
I'm not sure that scope_identity can work in this situation, or that creating a new temp table with a list of new IDs and inserting identity columns manually is the correct approach?
I could write a cursor / loop to do this, but there must be a nice way of doing some recursive select magic!
Since you're trying to put the tree into a segment of the table it looks like you're going to need to use SET IDENTITY_INSERT ON for the table anyway. You're going to need to make sure that there is room for the new tree. In this case, I'll assume that 49 is the current maximum id in your table so that we don't need to be concerned with overrunning a tree that's later in the table.
You'll need to be able to map the IDs from the old tree to the new tree. Unless there's some rule around the ids, the exact mapping should be irrelevant as long as it's accurate, so in that case, I'd just do something like this:
SET IDENTITY_INSERT dbo.Relations ON
;WITH CTE_MappedIDs AS
(
SELECT
old_id,
ROW_NUMBER() OVER(ORDER BY old_id) + 49 AS new_id
FROM
(
SELECT DISTINCT parent AS old_id FROM #relationtree
UNION
SELECT DISTINCT child AS old_id FROM #relationtree
) SQ
)
INSERT INTO dbo.Relations (Id, ParentId)
SELECT
CID.new_id,
PID.new_id
FROM
#relationtree RT
INNER JOIN CTE_MappedIDs PID ON PID.old_id = RT.parent
INNER JOIN CTE_MappedIDs CID ON CID.old_id = RT.parent
-- We need to also add the root node
UNION ALL
SELECT
NID.new_id,
NULL
FROM
#relationtree RT2
INNER JOIN CTE_MappedIDs NID ON NID.old_id = RT2.parent
WHERE
RT2.parent NOT IN (SELECT DISTINCT child FROM #relationtree)
SET IDENTITY_INSERT dbo.Relations OFF
I haven't tested that, but if it doesn't work as expected then hopefully it will point you in the right direction.
I know you already have a working answer, but I think you can accomplish the same thing a little more simply (not that there is anything at all wrong with Tom H's answer) using the LAG function to inspect the previous row, assuming you have SQL Server 2012 or later.
Setup:
CREATE TABLE #relationtree (
Parent INT,
Child INT
)
CREATE TABLE #relations (
Id INT IDENTITY(1,1),
ParentId INT
)
INSERT INTO #relationtree (Parent, Child) VALUES(1,3), (3,5), (5,6), (5,9)
INSERT INTO #relations (ParentId) values(1), (3), (5)
Solution:
DECLARE #offset INT = IDENT_CURRENT('#relations')
;WITH relationtreeids AS (
SELECT *,
ROW_NUMBER() OVER(ORDER BY Parent, Child) - 2 AS UnmodifiedParentId -- Simulate an identity field
FROM #relationtree
)
INSERT INTO #relations
-- The LAG window function allows you to inspect the previous row
SELECT CASE WHEN LAG(Parent) OVER(ORDER BY Parent) IS NULL
THEN NULL
WHEN LAG(Parent) OVER(ORDER BY Parent) = Parent
THEN UnmodifiedParentId + #offset ELSE UnmodifiedParentId + #offset + 1
END AS ParentId
FROM relationtreeids
Output:
Id ParentId
1 1
2 3
3 5
4 NULL
5 4
6 5
7 5
Imagine having a table as the one below:
create table test (
id int auto_increment,
some int,
columns int
)
And then this table get used alot. Rows are inserted and rows are deleted and over time there might be gaps in the number that once was auto incremented. As an example, if I at some point make the following query:
select top 10 id from test
I might get something like
3
4
6
7
9
10
13
14
18
19
How do I design a query that returns the missing values 1,2,5,8 etc?
The easiest way is to get ranges of missing values:
select (id + 1) as firstmissing, (nextid - 1) as lastmissing
from (select t.id, lead(id) over (order by id) as nextid
from test t
) t
where nextid is not null and nextid <> id + 1;
Note this uses the lead() function, which is available in SQL Server 2012+. You can do something similar with apply or a subquery in earlier versions. Here is an example:
select (id + 1) as firstmissing, (nextid - 1) as lastmissing
from (select t.id, tt.id as nextid
from test t cross apply
(select top 1 id
from test t2
where t2.id > t.id
order by id
) tt
) t
where nextid is not null and nextid <> id + 1;
Simple way is by using cte..
;WITH cte
AS (SELECT 1 id
UNION ALL
SELECT id + 1 id from cte
WHERE id < (SELECT Max(id)
FROM tablename))
SELECT *
FROM cte
WHERE id NOT IN(SELECT id
FROM tablename)
Note: this will start from 1. If you want start from the min value of your table just replace
"SELECT 1 id" to "SELECT Min(id) id FROM tablename"
Why does it matter? I'm not trying to be snarky, but this question is usually asked in the context of "I want to fill in the gaps" or "I want to compress my id values to be contiguous". In either case, the answer is "don't do it". In your example, there was at some point a row with id = 5. If you're going to do either of the above, you'll be assigning a different, unrelated set of business data that id. If there's anything that references the id external to your database, now you've just invented a problem that you didn't have before. The id should be treated as immutable and arbitrary for all intents and purposes. If you really require it to be gapless, don't use identity and never do a hard delete (i.e. if you need to deactivate a row, you need a column which says whether it's active or not).
I need to test my mail server. How can I make a Select statement
that selects say ID=5469 a thousand times.
If I get your meaning then a very simple way is to cross join on a derived query on a table with more than 1000 rows in it and put a top 1000 on that. This would duplicate your results 1000 times.
EDIT: As an example (This is MSSQL, I don't know if Access is much different)
SELECT
MyTable.*
FROM
MyTable
CROSS JOIN
(
SELECT TOP 1000
*
FROM
sysobjects
) [BigTable]
WHERE
MyTable.ID = 1234
You can use the UNION ALL statement.
Try something like:
SELECT * FROM tablename WHERE ID = 5469
UNION ALL
SELECT * FROM tablename WHERE ID = 5469
You'd have to repeat the SELECT statement a bunch of times but you could write a bit of VB code in Access to create a dynamic SQL statement and then execute it. Not pretty but it should work.
Create a helper table for this purpose:
JUST_NUMBER(NUM INT primary key)
Insert (with the help of some (VB) script) numbers from 1 to N. Then execute this unjoined query:
SELECT MYTABLE.*
FROM MYTABLE,
JUST_NUMBER
WHERE MYTABLE.ID = 5469
AND JUST_NUMBER.NUM <= 1000
Here's a way of using a recursive common table expression to generate some empty rows, then to cross join them back onto your desired row:
declare #myData table (val int) ;
insert #myData values (666),(888),(777) --some dummy data
;with cte as
(
select 100 as a
union all
select a-1 from cte where a>0
--generate 100 rows, the max recursion depth
)
,someRows as
(
select top 1000 0 a from cte,cte x1,cte x2
--xjoin the hundred rows a few times
--to generate 1030301 rows, then select top n rows
)
select m.* from #myData m,someRows where m.val=666
substitute #myData for your real table, and alter the final predicate to suit.
easy way...
This exists only one row into the DB
sku = 52 , description = Skullcandy Inkd Green ,price = 50,00
Try to relate another table in which has no constraint key to the main table
Original Query
SELECT Prod_SKU , Prod_Descr , Prod_Price FROM dbo.TB_Prod WHERE Prod_SKU = N'52'
The Functional Query ...adding a not related table called 'dbo.TB_Labels'
SELECT TOP ('times') Prod_SKU , Prod_Descr , Prod_Price FROM dbo.TB_Prod,dbo.TB_Labels WHERE Prod_SKU = N'52'
In postgres there is a nice function called generate_series. So in postgreSQL it is as simple as:
select information from test_table, generate_series(1, 1000) where id = 5469
In this way, the query is executed 1000 times.
Example for postgreSQL:
CREATE EXTENSION IF NOT EXISTS "uuid-ossp"; --To be able to use function uuid_generate_v4()
--Create a test table
create table test_table (
id serial not null,
uid UUID NOT NULL,
CONSTRAINT uid_pk PRIMARY KEY(id));
-- Insert 10000 rows
insert into test_table (uid)
select uuid_generate_v4() from generate_series(1, 10000);
-- Read the data from id=5469 one thousand times
select id, uid, uuid_generate_v4() from test_table, generate_series(1, 1000) where id = 5469;
As you can see in the result below, the data from uid is read 1000 times as confirmed by the generation of a new uuid at every new row.
id |uid |uuid_generate_v4
----------------------------------------------------------------------------------------
5469|"10791df5-ab72-43b6-b0a5-6b128518e5ee"|"5630cd0d-ee47-4d92-9ee3-b373ec04756f"
5469|"10791df5-ab72-43b6-b0a5-6b128518e5ee"|"ed44b9cb-c57f-4a5b-ac9a-55bd57459c02"
5469|"10791df5-ab72-43b6-b0a5-6b128518e5ee"|"3428b3e3-3bb2-4e41-b2ca-baa3243024d9"
5469|"10791df5-ab72-43b6-b0a5-6b128518e5ee"|"7c8faf33-b30c-4bfa-96c8-1313a4f6ce7c"
5469|"10791df5-ab72-43b6-b0a5-6b128518e5ee"|"b589fd8a-fec2-4971-95e1-283a31443d73"
5469|"10791df5-ab72-43b6-b0a5-6b128518e5ee"|"8b9ab121-caa4-4015-83f5-0c2911a58640"
5469|"10791df5-ab72-43b6-b0a5-6b128518e5ee"|"7ef63128-b17c-4188-8056-c99035e16c11"
5469|"10791df5-ab72-43b6-b0a5-6b128518e5ee"|"5bdc7425-e14c-4c85-a25e-d99b27ae8b9f"
5469|"10791df5-ab72-43b6-b0a5-6b128518e5ee"|"9bbd260b-8b83-4fa5-9104-6fc3495f68f3"
5469|"10791df5-ab72-43b6-b0a5-6b128518e5ee"|"c1f759e1-c673-41ef-b009-51fed587353c"
5469|"10791df5-ab72-43b6-b0a5-6b128518e5ee"|"4a70bf2b-ddf5-4c42-9789-5e48e2aec441"
Of course other DBs won't necessarily have the same function but it could be done:
See here.
If your are doing this in sql Server
declare #cnt int
set #cnt = 0
while #cnt < 1000
begin
select '12345'
set #cnt = #cnt + 1
end
select '12345' can be any expression
Repeat rows based on column value of TestTable. First run the Create table and insert statement, then run the following query for the desired result.
This may be another solution:
CREATE TABLE TestTable
(
ID INT IDENTITY(1,1),
Col1 varchar(10),
Repeats INT
)
INSERT INTO TESTTABLE
VALUES ('A',2), ('B',4),('C',1),('D',0)
WITH x AS
(
SELECT TOP (SELECT MAX(Repeats)+1 FROM TestTable) rn = ROW_NUMBER()
OVER (ORDER BY [object_id])
FROM sys.all_columns
ORDER BY [object_id]
)
SELECT * FROM x
CROSS JOIN TestTable AS d
WHERE x.rn <= d.Repeats
ORDER BY Col1;
This trick helped me in my requirement.
here, PRODUCTDETAILS is my Datatable
and orderid is my column.
declare #Req_Rows int = 12
;WITH cte AS
(
SELECT 1 AS Number
UNION ALL
SELECT Number + 1 FROM cte WHERE Number < #Req_Rows
)
SELECT PRODUCTDETAILS.*
FROM cte, PRODUCTDETAILS
WHERE PRODUCTDETAILS.orderid = 3
create table #tmp1 (id int, fld varchar(max))
insert into #tmp1 (id, fld)
values (1,'hello!'),(2,'world'),(3,'nice day!')
select * from #tmp1
go
select * from #tmp1 where id=3
go 1000
drop table #tmp1
in sql server try:
print 'wow'
go 5
output:
Beginning execution loop
wow
wow
wow
wow
wow
Batch execution completed 5 times.
The easy way is to create a table with 1000 rows. Let's call it BigTable. Then you would query for the data you want and join it with the big table, like this:
SELECT MyTable.*
FROM MyTable, BigTable
WHERE MyTable.ID = 5469