This question already has answers here:
How to get rid of gaps in rowid numbering after deleting rows?
(4 answers)
Closed 5 months ago.
I've created 2 rows in an table in SQL (sqlite3 on cmd) and then deleted 1 of them.
CREATE TABLE sample1( name TEXT, id INTEGER PRIMARY KEY AUTOINCREMENT);
INSERT INTO sample1 VALUES ('ROY',1);
INSERT INTO sample1(name) VALUES ('RAJ');
DELETE FROM sample1 WHERE id = 2;
Later when I inserted another row, its id was given 3 by the system instead of 2.
INSERT INTO sample1 VALUES ('AMIE',NULL);
SELECT * FROM sample1;
picture of table
How do I correct it so the next values are given right id's automatically? Or how do I clear the sql database cache to solve it?
The simplest fix to resolve the problem you describe, is to omit AUTOINCREMENT.
The result of your test would then be as you wish.
However, the rowid (which the id column is an alias of, if INTEGER PRIMARY KEY is specified, with or without AUTOINCREMENT), will still be generated and probably be 1 higher than the highest existing id (alias of rowid).
There is a subtle difference between using and not using AUTOINCREMENT.
without AUTOINCREMENT then the generated value of the rowid and therefore it's alias will be the highest existing rowid for the table plus 1 (not absolutely guaranteed though).
with AUTOINCREMENT the generated value will be 1 plus the higher of:-
the highest existing rowid, or
the highest used rowid
the highest, in some circumstances, may have only existed briefly
In your example as 2 had been used then 2 + 1 = 3 even though 2 had been deleted.
Using AUTOINCREMENT is inefficient as to know what the last used value was requires a system table, sqlite_sequence and it being accessed to store the latest id and also to retrieve the id.
The SQLite AUTOINCREMENT documentation, says this:-
The AUTOINCREMENT keyword imposes extra CPU, memory, disk space, and disk I/O overhead and should be avoided if not strictly needed. It is usually not needed.
There are other differences, such as with AUTOINCREMENT if the id 9223372036854775807 has been reached, then another insert will result in an SQLITE_FULL error. Whilst without AUTOINCREMENT then an unused id (there would be one as current day storage devices could not hold that number of rows).
The intention of id's (rowid's) is to uniquely identify a row and to be able to access such a row efficiently if accessing it by the id. The intention is not for it to be used as a sequence/order. Using it as a sequence/order number will probably invariably result in unanticipated sequences or inefficient overheads trying to maintain such a sequence/order.
You should always consider that rows are unordered unless specifically ordered by a clause that orders the output, such as an ORDER BY clause.
However, if you take your example a little further, omitting AUTOINCREMENT, will still probably result in the order/sequence issues as if, for example, the row with an id of 1 were deleted instead of 2 then you would end up with id's of 2 and 3.
Perhaps consider the following which shows a) how the limited issue you have posed, is solved without AUTOINCREMENT, and b) that it is not the solution if it is not the highest id that is deleted:-
DROP TABLE IF EXISTS sample1;
CREATE TABLE IF NOT EXISTS sample1( name TEXT, id INTEGER PRIMARY KEY);
INSERT INTO sample1 VALUES ('ROY',1);
INSERT INTO sample1(name) VALUES ('RAJ');
DELETE FROM sample1 WHERE id = 2;
INSERT INTO sample1 VALUES ('AMIE',NULL);
/* Result 1 */
SELECT * FROM sample1;
/* BUT if a lower than the highest id is deleted */
DELETE FROM sample1 WHERE id=1;
INSERT INTO sample1 VALUES ('EMMA',NULL);
/* Result 2 */
SELECT * FROM sample1;
Result 1 (your exact issue resolved)
Result 2 (if not the highest id deleted)
I'm trying to figure out how to insert data from Table1 into Table2, then use the newly-created ID from Table2 to update the corresponding row in Table1.
I'm using Postgres 12.4 for what it's worth
Example:
I've got two tables, e.g. users and metadata
The users tables has the following columns
| id | info | metadata_id |
The metadata table has the following columns
| id | data |
I want to migrate all of my info values from the users table into the data column of the metadata table, and update my users.metadata_id (currently blank) with the corresponding metadata.id values, essentially backfilling foreign keys.
Is there any way to accomplish this gracefully? I've got a working query which locks both tables and creates a temporary sequence to insert into the metadata.id and users.metadata_id but this seems brittle and I would need to start the sequence after the highest-existing ID in the metadata table, which isn't ideal.
I've also tried to use a data-modifying CTE with a RETURNING clause to update the users table, but couldn't get that to work.
You can't use returning here, since you need to keep track of the association of users and metadata while inserting.
I think it is simpler to first pre-generate the metadata serial of each user in a CTE, using nextval(). You can then use that information to insert into metadata and update the users table:
with
candidates as (
select u.*, nextval(pg_get_serial_sequence('metadata', 'id')) new_metadata_id
from users u
),
inserted as (
insert into metadata (id, data) overriding system value
select new_metadata_id, info from candidates
)
update users u
set metadata_id = c.new_metadata_id
from candidates c
where c.id = u.id
We need the overriding system value clause in the insert statement so Postgres allows us to write to a serial column.
Demo on DB Fiddle
For example, I have a table:
User ID(int) | Card ID(int) | Deck(int)
1841 | 14 | 1
1841 | 14 | 1
it is defined that the int values in deck column would always take on 1 or 2 as a value(1 indicating that it is in the deck). and card ID is not unique for a user(this indicate that a user have 2 card 14) , as shown in the example above. what if i want to remove one card 14 in the deck and the other would remain. what is the proper sql command, i tried UPDATE but it
you can define limit at the end of update query
update [table name] set Deck=2 where User_ID=1841 and Card_id=14 limit 1;
Basically you're missing a way of referencing any single particular row. Depending how critical to the application is need for such reference, it is almost always bad idea to allow such situation. There are many solutions for this, for example
1) Every row usually contains unique OID or ROWID field , which is not displayed with "SELECT * FROM TABLE", but can be used if requested implicitly. Depending on what database engine you use, e.g. with PostgreSQL try
SELECT OID, * FROM TABLE WHERE OID = 'somevalue'
this is usually used if you don't want to enforce UNIQUE on the table, but rather deal with possible mistaken input later if it will unfortunately appear.
2) You can add ID column, for example autoincremental ( refer to DB manual ), and then update it to contain unique IDs
ALTER TABLE table_name ADD column_name column-definition;
3) You can use self incrementing "running total", eg. with MySQL it looks more/less like this:
SET #runtot:=0;
SELECT *, (#runtot := #runtot + 1) AS rt FROM table WHERE rt='somevalue'
(this will do calculation every time so probably will be inefficient )
4) You can use LIMIT as explained in previous answer
5) You can JOIN some another table with unique IDs and possibly update resulting relation, or combine some query to create and use static VIEW
6) You can use SELECT with some dynamically allocated value, for example RAND() or NOW() or similar. It probably won't create unique identifiers across whole table, depending what function you'll use and how you will use it
7) combine two or more above solutions altogether
..and probably many other solutions. Anyway usually there's some "Id" column used with some UNIQUE constraint.
Example:
INDEX | SOME_DATA
1 | some_data1
3 | some_data3
4 | some_data4
5 | some_data5
I want to do an INSERT to add ---> 2 | some_data2.
Is there an SQL command or some way of doing this?
[edit]
OK. Maybe if i explain every thing what i want to accomplish you'll see why i'm trying to do what i want to do. I creating a Web admin page in php that simply displays data from a database table.now these displayed rows are "draggable". example,you can drag row #2 to say, row #4. now on the "mouseup" event, i want to save the new order in the database.
1| data1 -------------------------> 1| data1
2| data2 -------After Drag-----> 3| data3
3| data3 -------------------------> 4| data4
4| data4 -------------------------> 2| data2
See my problem? i cant merely just update the value in the INDEX column for the one value that was dragged. I'll have to update all the values that were dragged over. 3 has to become a 2, 4 a 3 and 2 has to become the new 4. Thats too many sql updates (expecially if the table is larger). So i want to remove row #2 and insert it in its correct location.
Just insert the data; tables are not fundamentally ordered. The only ordering of table results occurs when you define it yourself in your select statement.
Edit: if what you want is to have a separate ordering, you would be well served by having a separate "order" column. I'd recommend making it of type float, so you can insert entries anywhere between other entries without requiring any updating. For example, if you have entry "A" with "order" column value "1", entry "B" with "order" column value "2", and you want to insert entry "C" so it shows up between entry "A" and entry "B", just insert it with the "order" column value set to 1.5 (1 + 2 / 2.0). That way, you can perform your select with an "order by" on your "order" column, and things should turn out the way you want. Floats are a reasonably good solution to this issue, since there's enough space to support a lot of "in-between" adds.
You should just be able to add it like regular:
INSERT INTO some_table (index, some_data) VALUES (2, 'some_data2')
After inserting, if you actually want the rows to be ordered by index, run this:
ALTER TABLE some_table ORDER BY index;
In general the order is really unimportant though. You should be using indexes for columns you do finds on, and ORDER BY when getting result sets. You should not rely on the insert order of rows in the table for either optimizing or sorting.
Yes, you can specify the primary key of the row when you insert.
insert into yourtablename (index, some_data) values (2, "some_data2");
It's a bad idea however to worry about having contiguous primary keys. Their job should be to uniquely identify the row and nothing more.
Okay, I'm answering again because your update shows you are asking a completely different question than everyone thought you were.
The method that you described -- update the index of the moved row and every one in the list after it -- is exactly the right method here.
Why do you say that "that's too many sql updates"? As long as you have a MySQL index applied to the index column, those updates will be lightening fast, even for very large lists (they will be on the order of milliseconds).
I think you are either pre-optimizing, or you don't have an index defined. To add an index (if you don't already have one), run this command:
CREATE INDEX row_index_index ON `some_table` (`index`);
You also might want to reconsider renaming your "index" column to something like "position" to avoid confusion about nomenclature with an actual MySQL index.
Is it possible to reorder rows in SQL database?
For example; how can I swap the order of 2nd row and 3rd row's values?
The order of the row is important to me since i need to display the value according to the order.
Thanks for all the answers. But 'Order by' won't work for me.
For example, I put a list of bookmarks in database.
I want to display based on the result I get from query. (not in alphabet order). Just when they are inserted.
But user may re-arrange the position of the bookmark (in any way he/she wants). So I can't use 'order by'.
An example is how the bookmark display in the bookmark in firefox. User can switch position easily. How can I mention that in DB?
Thank you.
It sounds like you need another column like "ListOrder". So your table might look like:
BookMark ListOrder
======== =========
d 1
g 2
b 3
f 4
a 5
Then you can "order by" ListOrder.
Select * from MyTable Order By ListOrder
If the user can only move a bookmark one place at a time, you can use integers as the ListOrder, and swap them. For example, if the user wants to move "f" up one row:
Update MyTable
Set ListOrder=ListOrder+1
Where ListOrder=(Select ListOrder-1 From MyTable where BookMark='f')
Update MyTable
Set ListOrder=ListOrder-1
Where BookMark='f'
If the user can move a bookmark up or down many rows at once, then you need to reorder a segment. For example, if the user wants to move "f" to the top of the list, you need to:
if (increment) {
update MyTable
Set ListOrder=ListOrder-1
where ListOrder<=1 -- The New position
and ListOrder >(Select ListOrder from MyTable where BookMark='f')
} else {
update MyTable
Set ListOrder=ListOrder+1
where ListOrder>=1 -- The New position
and ListOrder <(Select ListOrder from MyTable where BookMark='f')
}
update MyTable
Set ListOrder=1 -- The New Position
Where Bookmark='f'
As others have mentioned, it's not a good idea to depend on the physical order of the database table. Relational tables are conceptually more like unordered sets than ordered lists. Assuming a certain physical order may lead to unpredictable results.
Sounds like what you need is a separate column that stores the user's preferred sort order. But you'll still need to do something in your query to display the results in that order.
It is possible to specify the physical order of records in a database by creating a clustered index, but that is not something you'd want to do on an arbitrary user-specified basis. And it may still lead to unexpected results.
Use ORDER BY in your SELECT query. For example, to order by a user's last name, use:
SELECT * FROM User ORDER BY LastName
The order of the rows on the actual database should not matter.
You should use the ORDER BY clause in your queries to order them as you need.
Databases can store the data in any way they want. Using the "order by" clause is the only way to guarantee an ordering of the data. In your bookmark example, you could have an integer field that indicates the ordering, and then update that field as a user moves things around. Then ORDER BY that column to get things in the right order.
A little late to the party, but anyone still looking for an answer to this problem, you need to use the Stern-Brocot technique.
Here's an article explaining the theory behind it
For each item you need to store a numerator and denominator. Then you can also add a computed column which is the division of both. Each time you move an item inbetween 2 others, the item's numerator becomes the sum of both neighboring numerators, and the item's denominator becomes the sum of both neighboring denominators.
These numbers won't skyrocket as fast as with the "averaging" method, where you lose all accuracy after 17 swaps.
I also created a demo where the method is implemented.
I have a solution for this that I have used a few times. I keep an extra field "sort_order" in the table, and update this when reordering. I've used this in cases when I have some sort of containers with items, and the order of the items should be editable inside the container. When reordering, I only update the sort_order for the items in the current container, which means not to many (usually in practice only a few) rows have to be updated.
In short, I do the following:
add a sort_order field to the items table
when inserting a new row, I set sort_order=id
when reordering (needs id of item to move, and id of item to insert after):
select id, sort_order from items where container = ID order by sort_order
split the id and sort_order from rows in two arrays
remove the id of the item to move from the id-list
insert the id of the item to move after the id of the item to insert after
merge the list of ids and the list of sort_order into a two dimensional array, as [[id, sort_order], [id2, sort_order], ...]
run update item set sort_order=SORT_ORDER where id=ID (executemany) with merged list
(If moving item to another container, after updating "container foreign key" move first or last depending on app.)
(If the update involves a large number of items, I do not think this solution is a good approach.)
I have made an example using python and mysql on http://wannapy.blogspot.com/2010/11/reorder-rows-in-sql-database.html (copy and try it) along with some extra explanations.
I guess a simple order by would be what you're looking for?
select my_column from my_table order by my_order_column;
As others have stated use an order by.
Never depend on the order data exists in a physical table, always base it of the data you are working with, be it one or more key fields.
First, let me agree with everyone here that the order in the table shouldn't matter. Use a separate [SortOrder] column that you update and include an Order By clause.
That said, SQL Server databases do allow for a single "clustered index" on a table that will actually force the position in the underlying table storage. Primarily useful if you have a big dataset and always query by something specific.
Add a position column to your table and store as a simple integer.
If you need to support multiple users or lists, your best bet is to create a bookmarks table, an users table and a table to link them.
bookmarks: id,url
users: id,name
users_bookmarks: user_id, bookmark_id, position, date_created
Assuming date_created is populated when inserting rows you can get secondary list ordering based on date.
select bookmark_id from users_bookmarks where user_id = 1 order by position, date_created;
At times like this, I am reminded of a quote from the Matrix: "Do not try and order the database. That's impossible. Instead, only realize the truth... there is no order. Then you will see that it the table that orders itself, it is you who orders the table."
When working with MySQL through a GUI, there is always a decision to make. If you run something like SELECT * FROM users, MySql will always make a decision to order this by some field. Normally, this will be the primary key.
+----------------
| id | name |
-----------------
| 1 | Brian |
| 2 | Carl |
| 3 | Albert |
-----------------
When you add an ORDER BY command to the query, it will make the decision to order by some other field.
For Example Select * From users ORDER BY name would yield:
+----------------
| id | name |
-----------------
| 3 | Albert |
| 1 | Brian |
| 2 | Carl |
-----------------
So to your question, you appear to want to change the default order by which your table displays this information. In order to do that, check to see what your Primary Key field
is. For most practical purposes, having a unique identifying natural number tends to do the trick. MySQL has an AUTO_INCREMENT function for this. When creating the table, it would look something like field_name int NOT NULL AUTO_INCREMENT.
All of this is to say: if you would like to change "the row order", you would need to update this value. However, since the identifier is something that other tables would use to reference your field, this seems a little bit reckless.
If you for example went: UPDATE table Set id = 1 where id = 2;, this would initially fail, since the id fields would end up being both an identical value and fail the PrimaryKey check (which insists on both uniqueness and having a value set). You could Juggle this by running three update statements in a row:
UPDATE users Set id = 100000000 where id = 1;
UPDATE users Set id = 1 where id = 2;
UPDATE users Set id = 2 where id = 100000000;
This would result in the rows for this table looking like:
+----------------
| id | name |
-----------------
| 1 | Carl |
| 2 | Brian |
| 3 | Albert |
----------------+
Which technically would work to reorder this table, but this is in a bubble. MySQL being a relational database means that any table which was depending on that data to be consistent will now be pointed to the wrong data. For example, I have a table which stores birthdays, referencing the initial user table. It's structure might look like this:
+----------------------------+
| id | user_id | birthdate |
+----------------------------+
| 1 | 1 | 1993-01-01 |
| 1 | 2 | 1980-02-03 |
| 1 | 3 | 1955-01-01 |
+----------------------------+
By switching the ID's on the user table, you MUST update the user_id value on the birthdays table. Of course MySQL comes prepared for this: enter "Foreign Key Constraints". As long as you configured all of your foreign key constraints to Cascade Updates, you wouldn't need to manually change the reference to every value you changed.
These queries would all be a lot of manual work and potentially weaken your data's integrity. If you have fields you would like to rank and reorder regularly, the answer posed by Mike Lewis on this question with the "table order" would be a more sensible answer (and if that is the case, then his is the best solution and just disregard this answer).
In response to your post here, the answer you may be looking for is:
To order chronologically, add a DateAdded or similar column with a datetime or smalldatetime datatype.
On all methods that insert into the database, make sure you insert CURRENT_TIMESTAMP in the DateAdded column.
On methods that query the database, add ORDER BY DateAdded at the end of the query string.
NEVER rely on the physical position in the database system. It may work MOST of the time but definitely not ALL of the time.
The question lacks any detail that would let anyone give you correct answer. Clearly you could read the records into memory and then update them. But this is bad on so many different levels.
The issue is like this. Depending on the schema that is actually implemented there is logic to the way that the records are physically written to disk. Sometimes they are written in order of insert and other times they are inserted with space between blocks (see extents).
So changing the physical order is not likely without swapping column data; and this has a deep effect on the various indices. You are left having to change the logical order.
As I read your update... I'm left to understand that you may have multiple users and each user is to have bookmarks that they want ordered. Looks like you need a second table that acts as an intersection between the user and the bookmark. Then all you need is an inner join and an order by.
But there is not enough information to offer a complete solution.
Here is a stored procedure script to increment or decrement (one at a time) in MySQL.
Note, MySQL doesn't allow you to select in the same query you're updating so the above answers don't work.
I have also set it to return an error if there is no item above / below if you're incrementing / decrementing, respectively.
DELIMITER $$
CREATE PROCEDURE `spReorderSequenceItems` (
IN _SequenceItemId INT,
IN _SequenceId INT,
IN IncrementUp TINYINT,
OUT Error VARCHAR(255)
)
BEGIN
DECLARE CurrentPosition INT;
SELECT Position INTO CurrentPosition
FROM tblSequenceItems
WHERE SequenceItemId = _SequenceItemId;
IF IncrementUp = 1 THEN
IF (
SELECT Position
FROM tblSequenceItems
WHERE Position = CurrentPosition + 1 AND SequenceId = _SequenceId
) THEN
UPDATE tblSequenceItems
SET Position = Position - 1
WHERE Position = CurrentPosition + 1 AND SequenceId = _SequenceId;
UPDATE tblSequenceItems
SET Position = Position + 1
WHERE SequenceItemId = _SequenceItemId;
ELSE
SELECT 'No Item Above' AS _Error INTO Error;
END IF;
ELSE
IF (
SELECT Position
FROM tblSequenceItems
WHERE Position = CurrentPosition - 1 AND SequenceId = _SequenceId
) THEN
UPDATE tblSequenceItems
SET Position = Position + 1
WHERE Position = CurrentPosition - 1 AND SequenceId = _SequenceId;
UPDATE tblSequenceItems
SET Position = Position - 1
WHERE SequenceItemId = _SequenceItemId;
ELSE
SELECT 'No Item Below' AS _Error INTO Error;
END IF;
END IF;
END
$$
DELIMITER ;
Call it with
CALL spReorderSequenceItems(1, 1, 1, #Error);
SELECT #Error;