I have a SQLite table sorted by column ID. But I need to sort it by another numerical field called RunTime.
CREATE TABLE Pass_2 AS
SELECT RunTime, PosLevel, PosX, PosY, Speed, ID
FROM Pass_1
The table Pass_2 looks good, but I need to renumber the ID column from 1 .. n without resorting the records.
It is a principle of SQL databases that the underlying tables have no natural or guaranteed order to their records. You must specify the order in which you want to see the records when SELECTing from a table using an ORDER BY clause.
You can obtain the records you want using SELECT * FROM your_table ORDER BY RunTime, and that is the correct and reliable way to do this in any SQL database.
If you want to attempt to get the records in Pass_2 to "be" in RunTime order, you can add the ORDER BY clause to the SELECT you use to create the table but remember: you are not guaranteed to get the records back in the order in which they were added to the table.
When might you get the records back in a different order? This is most likely to happen when your query can be answered using columns in a covering index -- in that case the records are more likely to be returned in index order than any "natural" order (but again, no guarantees with an ORDER BY clause).
If you want a new ID column starting at 1, then use the ROW_NUMBER() function. Instead of ID in your query use this ROW_NUMBER() OVER(ORDER BY Runtime) AS ID.... This will replace the old ID column with a freshly calculated column
Related
I'm having an issue with Sequences when inserting data into a Postgres table through SQL Alchemy.
All of the data is inserted fine, the id BIGSERIAL PRIMARY KEY column has all unique values which is great.
However when I query the first 10/20 rows etc. of the table, the id values are not ascending in numeric order. There are gaps in the sequence, fine, that's to be expected, I mean rows will go through values randomly not ascending like:
id
15
22
16
833
30
etc...
I've gone through plenty of SO and Postgres forum posts around this and have only found people talking about having huge serial gaps in their sequences, not about incorrect ascending order when being created
Screenshots of examples:
The table itself has being created through standard DDL statement like so:
CREATE TABLE IF NOT EXISTS schema.table_name (
id BIGSERIAL NOT NULL,
col1 text NOT NULL,
col2 JSONB[] NOT NULL,
etc....
PRIMARY KEY (id)
);
However when I query the first 10/20 rows etc. of the table
Your query has no order by clause, so you are not selecting the first rows of the table, just an undefined set of rows.
Use order by - you will find out that sequence number are indeed assigned in ascending order (potentially with gaps):
select id from ht_data order by id limit 30
In order to actually check the ordering of the sequence, you would actually need another column that stores the timestamp when each row was created. You could then do:
select id from ht_data order by ts limit 30
In general, there is no defined "order" within a SQL table. If you want to view your data in a certain order, you need an ORDER BY clause:
SELECT *
FROM table_name
ORDER BY id;
As for gaps in the sequence, the contract of an auto increment column generally only guarantees that each newly generated id value with be unique and, most of the time (but not necessarily always), will be increasing.
How could you possibly know if the values are "out of order"? SQL tables represent unordered sets. The only indication of ordering in your table is the serial value.
The query that you are running has no ORDER BY. The results are not guaranteed to be in any particular ordering. Period. That is a very simply fact about SQL. That you want the results of a SELECT to be ordered by the primary key or by insertion order is nice, but not how databases work.
The only way you could determine if something were out of order would be if you had a column that separate specified the insert order -- you could have a creation timestamp for instance.
All you have discovered is that SQL lives up to its promise of not guaranteeing ordering unless the query specifically asks for it.
We have avro partitioned table in hive. When we query table, partition column is displaying at the end. Is there any way to display partition column at first?
Eg: select * from tablea
Output:
Col1 col2 partition_column
Expected output:
Partition_column col1 col2
Partition column is not stored in files, so, avro or not avro, it does not matter in this context. Partition column corresponds partition sub-folder within table folder and stored in the metadata.
Historically partition column is the last one. dynamic partitioning using Insertoverwrite table partition (partition_column) SELECT * from ...` is rather common scenario. Hive will know partition is the last column.
The dynamic partition columns must be specified last among the columns
in the SELECT statement and in the same order in which they appear in
the PARTITION() clause.
You can change the order of columns displayed when running SELECT * only by creating a view in which you list all columns in the required order, OR select columns explicitly in your select.
Also according to the Codd's theory, column and row order is immaterial, you always must specify columns order desired explicitly in the select and rows order using ORDER BY, instead of relying on columns order and row order in the table or view. But in Hive the partitioning column is the last one in the table.
Consider also this: You may even not know, what you selecting from: table or view. And you may be not notified that upstream system decided to change the table or view eventually. View or table can change the order of columns. Consider view the same as a table when doing selects. It is just abstraction level. Use explicit column list to make your program working reliably always and do not have strong dependency on column order in the underlying table/view, which is immaterial.
I have one table CSBCA1_5_FPCIC_2012_EES207201222743, having two columns employee_id and employee_name
I have used following query
SELECT ROW_NUMBER() OVER (ORDER BY EMPLOYEE_ID) AS ID, EMPLOYEE_ID,EMPLOYEE_NAME
FROM CSBCA1_5_FPCIC_2012_EES207201222743
But, it returns the rows in ascending order of employee_id, but I need the rows in order they were inserted into the table.
SQL Server does not track the order of inserted rows, so there is no reliable way to get that information given your current table structure. Even if employee_id is an IDENTITY column, it is not 100% foolproof to rely on that for order of insertion (since you can fill gaps and even create duplicate ID values using SET IDENTITY_INSERT ON). If employee_id is an IDENTITY column and you are sure that rows aren't manually inserted out of order, you should be able to use this variation of your query to select the data in sequence, newest first:
SELECT
ROW_NUMBER() OVER (ORDER BY EMPLOYEE_ID DESC) AS ID,
EMPLOYEE_ID,
EMPLOYEE_NAME
FROM dbo.CSBCA1_5_FPCIC_2012_EES207201222743
ORDER BY ID;
You can make a change to your table to track this information for new rows, but you won't be able to derive it for your existing data (they will all me marked as inserted at the time you make this change).
ALTER TABLE dbo.CSBCA1_5_FPCIC_2012_EES207201222743
-- wow, who named this?
ADD CreatedDate DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP;
Note that this may break existing code that just does INSERT INTO dbo.whatever SELECT/VALUES() - e.g. you may have to revisit your code and define a proper, explicit column list.
There is a pseudocolumn called %%physloc%% that shows the physical address of the row.
See Equivalent of Oracle's RowID in SQL Server
SQL does not do that. The order of the tuples in the table are not ordered by insertion date. A lot of people include a column that stores that date of insertion in order to get around this issue.
I am trying to insert values from a table into another existing table and have just the values I am inserting be sorted in descending order based on a specific column while leaving the existing records at the top of the table. How do I do that? I have tried to use an Order By statement but whether I use the column name of the table I'm pulling from or the destination table's column name I get an error. Also this is being run in VBA using DoCmd.RunSQL.
Here is my existing query:
INSERT INTO AllMetersAvgRSSI
(longitude,latitude,AvgRSSI)
Select
Prem.longitude, Prem.latitude,
DataByColl.[Avg RSSI]
From [Prem]
Left
Join DataByColl ON (Prem.meter_miu_id
= DataByColl.[MIU ID])
Order BY [AvgRSSI] desc
Final Result
I continued to fiddle with this and discovered than you can use an order by just like I have shown above to do exactly as I was trying to do. The problem I was apparently having was caused by the names of the column I wanted sorted being changed only from Avg RSSI to AvgRSSI. When I changed the destination table to have the same field name as the source table it orders the incoming information while leaving the existing information alone. I also did a test where I changed the name of the destination table to AverageRSSI and it worked the same way. So in the end it was the names of the fields being differed only by a space that was causing the problem. The final Query is:
INSERT INTO AllMetersAvgRSSI
(longitude,latitude,[Avg RSSI])
Select
Prem.longitude, Prem.latitude,
DataByColl.[Avg RSSI]
From [Prem]
Left
Join DataByColl ON (Prem.meter_miu_id
= DataByColl.[MIU ID])
Order BY [Avg RSSI] desc
Ordering in an INSERT makes no sense from a database standpoint. How the database puts the rows into a table depends on the underlying physical structure of the table, not the order in which they are inserted.
Maybe your application relies on an auto incrementing column being in a certain order which would then be dependent on the order of insertion, but if that's the case then I would say that you've made a mistake in your database design as there shouldn't be business logic designed around an auto incrementing column.
Remove the ORDER BY from your INSERT statement and if you need to retrieve rows in a particular order later then use an ORDER BY there.
Create a temp table, add the first result set in the desired order. Insert your new values into the table, query the table to return your new results with an order by into your temp table, select your temp table the results will be in the order you added them unless you do another order by.
Don't forget to drop your temp table after displaying the results.
consider a tale is as follows,
EmployeeId | Name | Phone_Number
Now, i insert 10 records... When i query them back, select * from myTable they are not selected in the order i inserted. I can obviously keep an autoincrement index and ORDER BY index. But i dont want to alter the table. How can i do this without altering the table?
Any ordering of result must be done using ORDER BY, if you don't use it the result will be returned in an undetermined order.
Unfortunately there is no way to do this.
Without an ORDER BY clause, there is no guaranteed order for the data to be returned in.
You would need to order by a column that indicates the inserted order, such as an IDENTITY field or a "Creation Date" field.
Isn't "EmployeeId" an auto-increment field? If it is, you can order by it to get data in order in which you inserted it.
There is no standard way to do this without adding an additional date, autoincrement index or some other counter to your table. Depending on your database there are some hacks you could do with SQL triggers to track this info in a separate table, but I suspect you don't want to do that (not all databases support them and they are not generally portable).