I have a table in informix (Version 11.50.UC4) called NextRecordID with just one column called id and it will have one row. What I want to do is copy this value into another table. But don't want my query to fail if this table does not exist. Something like
if table NextRecordID exists
then insert into sometable values ('NextRecordID', (select id from NextRecordID))
else insert into sometable values ('NextRecordID', 1)
I ended up using the below SQL query. Its not ANSI SQL but works the informix server I am using.
insert into sometable values ('NextRecordID',
select case (select 1 from systables where tabname='nextrecordid')
when 1 then (select nextid from nextrecordid)
else (select 1 from systables where tabname='systables') end
from systables where tabname='systables');
What is happening here is within insert query I get the value to be inserted by using select query. Now that select query is interesting. It uses case statement of Informix. I have written a select query to check if the table nextrecordid exists in systables and return 1 if it exists. If this query returns 1, I query the table nextrecordid for the value or else I wrote a query to return the default value 1. This work for me.
You should be able to do this by checking the systables table.
Thank you for including server version information - it makes answering your question easier.
You've not indicated which language(s) you are using.
Normally, though, you design a program to expect a certain schema (certain tables to be present), and then fail - preferably under control - if those tables are not present. Also, it is not clear whether you would get into problems because of repeated execution of the second INSERT statement. Nor is it clear when the NextRecordID table is updated - presumably, once the value has been used, it must be updated.
You should look at SERIAL (BIGSERIAL) and see whether that is appropriate for you.
You should also look at whether a SEQUENCE would be appropriate to use here - it certainly looks rather like it might be applicable.
As Adam Hughes points out, if you want to check whether the NextRecordID table is present in the database, you would look in the systables table. Be aware, though, that your search will need to be against an all lower-case name (nextrecordid).
Also, MODE ANSI databases complicate life - you have to worry about the table's owner (because there could be multiple tables called nextrecordid in a MODE ANSI database). Most likely, you don't have to worry about that - any more than you are likely to have to worry about delimited identifiers for table "someone"."NextRecordID" (which is a different table from someone.NextRecordID).
Related
Does PostgreSQL support computed / calculated columns, like MS SQL Server? I can't find anything in the docs, but as this feature is included in many other DBMSs I thought I might be missing something.
Eg: http://msdn.microsoft.com/en-us/library/ms191250.aspx
Postgres 12 or newer
STORED generated columns are introduced with Postgres 12 - as defined in the SQL standard and implemented by some RDBMS including DB2, MySQL, and Oracle. Or the similar "computed columns" of SQL Server.
Trivial example:
CREATE TABLE tbl (
int1 int
, int2 int
, product bigint GENERATED ALWAYS AS (int1 * int2) STORED
);
fiddle
VIRTUAL generated columns may come with one of the next iterations. (Not in Postgres 15, yet).
Related:
Attribute notation for function call gives error
Postgres 11 or older
Up to Postgres 11 "generated columns" are not supported.
You can emulate VIRTUAL generated columns with a function using attribute notation (tbl.col) that looks and works much like a virtual generated column. That's a bit of a syntax oddity which exists in Postgres for historic reasons and happens to fit the case. This related answer has code examples:
Store common query as column?
The expression (looking like a column) is not included in a SELECT * FROM tbl, though. You always have to list it explicitly.
Can also be supported with a matching expression index - provided the function is IMMUTABLE. Like:
CREATE FUNCTION col(tbl) ... AS ... -- your computed expression here
CREATE INDEX ON tbl(col(tbl));
Alternatives
Alternatively, you can implement similar functionality with a VIEW, optionally coupled with expression indexes. Then SELECT * can include the generated column.
"Persisted" (STORED) computed columns can be implemented with triggers in a functionally equivalent way.
Materialized views are a related concept, implemented since Postgres 9.3.
In earlier versions one can manage MVs manually.
YES you can!! The solution should be easy, safe, and performant...
I'm new to postgresql, but it seems you can create computed columns by using an expression index, paired with a view (the view is optional, but makes makes life a bit easier).
Suppose my computation is md5(some_string_field), then I create the index as:
CREATE INDEX some_string_field_md5_index ON some_table(MD5(some_string_field));
Now, any queries that act on MD5(some_string_field) will use the index rather than computing it from scratch. For example:
SELECT MAX(some_field) FROM some_table GROUP BY MD5(some_string_field);
You can check this with explain.
However at this point you are relying on users of the table knowing exactly how to construct the column. To make life easier, you can create a VIEW onto an augmented version of the original table, adding in the computed value as a new column:
CREATE VIEW some_table_augmented AS
SELECT *, MD5(some_string_field) as some_string_field_md5 from some_table;
Now any queries using some_table_augmented will be able to use some_string_field_md5 without worrying about how it works..they just get good performance. The view doesn't copy any data from the original table, so it is good memory-wise as well as performance-wise. Note however that you can't update/insert into a view, only into the source table, but if you really want, I believe you can redirect inserts and updates to the source table using rules (I could be wrong on that last point as I've never tried it myself).
Edit: it seems if the query involves competing indices, the planner engine may sometimes not use the expression-index at all. The choice seems to be data dependant.
One way to do this is with a trigger!
CREATE TABLE computed(
one SERIAL,
two INT NOT NULL
);
CREATE OR REPLACE FUNCTION computed_two_trg()
RETURNS trigger
LANGUAGE plpgsql
SECURITY DEFINER
AS $BODY$
BEGIN
NEW.two = NEW.one * 2;
RETURN NEW;
END
$BODY$;
CREATE TRIGGER computed_500
BEFORE INSERT OR UPDATE
ON computed
FOR EACH ROW
EXECUTE PROCEDURE computed_two_trg();
The trigger is fired before the row is updated or inserted. It changes the field that we want to compute of NEW record and then it returns that record.
PostgreSQL 12 supports generated columns:
PostgreSQL 12 Beta 1 Released!
Generated Columns
PostgreSQL 12 allows the creation of generated columns that compute their values with an expression using the contents of other columns. This feature provides stored generated columns, which are computed on inserts and updates and are saved on disk. Virtual generated columns, which are computed only when a column is read as part of a query, are not implemented yet.
Generated Columns
A generated column is a special column that is always computed from other columns. Thus, it is for columns what a view is for tables.
CREATE TABLE people (
...,
height_cm numeric,
height_in numeric GENERATED ALWAYS AS (height_cm * 2.54) STORED
);
db<>fiddle demo
Well, not sure if this is what You mean but Posgres normally support "dummy" ETL syntax.
I created one empty column in table and then needed to fill it by calculated records depending on values in row.
UPDATE table01
SET column03 = column01*column02; /*e.g. for multiplication of 2 values*/
It is so dummy I suspect it is not what You are looking for.
Obviously it is not dynamic, you run it once. But no obstacle to get it into trigger.
Example on creating an empty virtual column
,(SELECT *
From (values (''))
A("virtual_col"))
Example on creating two virtual columns with values
SELECT *
From (values (45,'Completed')
, (1,'In Progress')
, (1,'Waiting')
, (1,'Loading')
) A("Count","Status")
order by "Count" desc
I have a code that works and use the term calculated, I'm not on postgresSQL pure tho we run on PADB
here is how it's used
create table some_table as
select category,
txn_type,
indiv_id,
accum_trip_flag,
max(first_true_origin) as true_origin,
max(first_true_dest ) as true_destination,
max(id) as id,
count(id) as tkts_cnt,
(case when calculated tkts_cnt=1 then 1 else 0 end) as one_way
from some_rando_table
group by 1,2,3,4 ;
A lightweight solution with Check constraint:
CREATE TABLE example (
discriminator INTEGER DEFAULT 0 NOT NULL CHECK (discriminator = 0)
);
I do not fully understand the "USE" statement in Transact-SQL and how it affects the scope of temp tables. I have a user-defined table type in one database but not another, and I've found I need to "USE" that database in order to define a table of that type. Earlier in the query, I define a temporary table. After the "USE" statement, SSMS does not recognize the temp table as a valid object name, however I can still query from it without error.
The skeleton of my SQL query is as follows:
USE MYDATABASE1
[... a bunch of code I did not write...]
SELECT * INTO #TEMP_TABLE FROM #SOME_EARLIER_TEMP_TABLE
USE MYDATABASE2
DECLARE #MYTABLE MyUserDefinedTableType -- this table type only exists in MYDATABASE2
INSERT INTO #MYTABLE(Col1, Col2)
SELECT Col1, Col2 FROM (SELECT * FROM MYDATABASE2.dbo.SOME_TABLE_VALUED_FUNCTION(param1, param2)) T
SELECT A.*, B.Col2
FROM #TEMP_TABLE A
CROSS APPLY DATABASE2.dbo.SOME_OTHER_TABLE_VALUED_FUNCTION(#MYTABLE, A.SomeColumn) B
In the last SELECT statement, SSMS has red squiggly lines under "A.*" and "#TEMP_TABLE", however there is no error running the query.
So my question is: am I doing something "wrong" even though my query still works? Assuming the initial "USE MYDATABASE1" is necessary, what is the correct way to switch databases while still having #TEMP_TABLE available as a valid object name? (Note that moving the definition of #TEMP_TABLE to after "USE MYDATABASE2" would just shift the problem to #SOME_EARLIER_TEMP_TABLE.)
In SQL USE basically tells the query which database is the "default" database.
Temp tables can play tricks on intellisense - unless they're explicitly defined using the CREATE TABLE #MyTempTable route, intellisense doesn't really know what to do with them a lot of the time. Don't worry though - temp tables are scoped to the query.
Although I do feel it's worth pointing out: while UDTs are database specific, you can create an assembly to use across databases
I feel like this should be simple but I'm relatively unskilled in SQL and I can't seem to figure it out. I'm used to wrangling data in python (pandas) or Spark (usually pyspark) and this would be a one-liner in either of those. Specifically, I'm using Snowflake SQL, but I think this is probably relevant to a lot of flavors of SQL.
Essentially I just want to trim the first character off of a specific column. More generally, what I'm trying to do is replace a column with a substring of the same column. I would even settle for creating a new column that's a substring of an existing column. I can't figure out how to do any of these things.
On obvious solution would be to create a temporary table with something like
CREATE TEMPORARY TABLE tmp_sub AS
SELECT id_col, substr(id_col, 2, 10) AS id_col_sub FROM table1
and then join it back and write a new table
CREATE TABLE table2 AS
SELECT
b.id_col_sub as id_col,
a.some_col1, a.some_col2, ...
FROM table1 a
JOIN tmp_sub b
ON a.id_col = b.id_col
My tables have roughly a billion rows though and this feels extremely inefficient. Maybe I'm wrong? Maybe this is just the right way to do it? I guess I could replace the CREATE TABLE table2 AS... to INSERT OVERWRITE INTO table1 ... and at least that wouldn't store an extra copy of the whole thing.
Any thoughts and ideas are most welcome. I come at this humbly from the perspective of someone who is baffled by a language that so many people seem to have mastery over.
I'm not sure the exact syntax/functions in Snowflake but generally speaking there's a few different ways of achieving this.
I guess the general approach that would work universally is using the SUBSTRING function that's available in any database.
Assuming you have a table called Table1 with the following data:
+-------+-----------------------------------------+
Code | Desc
+-------+-----------------------------------------+
0001 | 1First Character Will be Removed
0002 | xCharacter to be Removed
+-------+-----------------------------------------+
The SQL code to remove the first character would be:
select SUBSTRING(Desc,2,len(desc)) from Table1
Please note that the "SUBSTRING" function may vary according to different databases. In Oracle for example the function is "SUBSTR". You just need to find the Snowflake correspondent.
Another approach that would work at least in SQLServer and MySQL would be using the "RIGHT" function
select RIGHT(Desc,len(Desc) - 1) from Table1
Based on your question I assume you actually want to update the actual data within the table. In that case you can use the same function above in an update statement.
update Table1 set Desc = SUBSTRING(Desc,2,len(desc))
You didn't try this?
UPDATE tableX
SET columnY = substr(columnY, 2, 10 ) ;
-Paul-
There is no need to specify the length, as is evidenced from the following simple test harness:
SELECT $1
,SUBSTR($1, 2)
,RIGHT($1, -2)
FROM VALUES
('abcde')
,('bcd')
,('cdef')
,('defghi')
,('e')
,('fg')
,('')
;
Both expressions here - SUBSTR(<col>, 2) and RIGHT(<col>, -2) - effectively remove the first character of the <col> column value.
As for the strategy of using UPDATE versus INSERT OVERWRITE, I do not believe that there will be any difference in performance or outcome, so I might opt for the UPDATE since it is simpler. So, in conclusion, I would use:
UPDATE tableX
SET columnY = SUBSTR(columnY, 2)
;
I need to select a column only if it exists in table, else it can be set to null.
Sample table below, lets say the marks col is not necessary be there, so need to be checked if it exists
Table1:
name marks
joe 10
john 11
mary 13
Query:
select
name,
marks if it exists else null as marks1 -- pseudo code
from
table1
What should go in line to select marks ?
SQL Doesn't permit that. Your result set has two options:
Static inclusion
All from table or subquery through column-expansion with * and tbl.*
Perhaps this will suit your needs, SELECT * FROM table1; You'll always get that column, if it exists.
try this
IF COL_LENGTH('your_table_name','column_name_you_want_to_select') IS NULL BEGIN
--This means columns does not exist or permission is denied
END
else
--Do whatever you want
It is possible to achieve this in PostgreSQL using JSON. Consider the following SQL query:
SELECT c.relname, c.relkind, c.relispartition
FROM pg_class c
WHERE c.relkind IN ('r','p') AND
c.relnamespace=(SELECT oid FROM pg_namespace WHERE nspname='public')
In PostgreSQL 10+, that will show you the names of all the tables in public schema, including whether they are partitioned and if so whether the table is the partitioned table or one of the partitions of it. However, if you try to run the same query on PostgreSQL 9.6 or earlier, it will fail since the relispartition column does not exist on the pg_class table prior to PostgreSQL 10.
An obvious solution would be to dynamically generate the SQL based on a condition, or have two different versions of the SQL. However, suppose you don't want to do that, you want to have a single query which works on both versions – in other words, you want to conditionally select the relispartition column if it exists.
The core SQL language does not have any facility to conditionally select a column, but it is achievable in PostgreSQL using the row_to_json function, as follows:
SELECT c.relname, c.relkind,
(row_to_json(c)->>'relispartition')::boolean AS relispartition
FROM pg_class c
WHERE c.relkind IN ('r','p') AND
c.relnamespace=(SELECT oid FROM pg_namespace WHERE nspname='public')
If you try running that, you will find on PostgreSQL 10+ the relispartition column is returned as true/false, whereas in pre-10 versions it is NULL. You could make it return false instead of NULL in pre-10 versions by doing COALESCE((row_to_json(c)->>'relispartition')::boolean,false).
What this is doing, is row_to_json(c) turns all the data of the row into JSON. Next, ->>'relispartition' selects the value of the relispartition JSON object key as text, which will be the same as the value of the relispartition column; if there is no such key in the JSON, the result of that will be NULL. Then, ::boolean converts the string value true/false back into a PostgreSQL boolean value. (If your column is of some other type, use the appropriate cast for the type of your column.)
(Obviously this approach will not work in Postgres versions which are too old to have the necessary JSON support – I have tested it works in Postgres 9.4; while I haven't tested it in Postgres 9.3, it probably works there. However, I would not expect it to work in 9.2 or earlier – the ->> operator was added in 9.3, and the JSON type and row_to_json function was added in 9.2. However, I expect few people will need to support those old unsupported versions–9.3 was released in 2013, and 9.2 supported ended in 2017.)
Try this:
IF EXISTS( SELECT 1
FROM information_schema.columns
WHERE table_name='your_table' and column_name='your_column') THEN
SELECT your_column as 'some_column'
ELSE
SELECT NULL as 'some_column'
END IF
Replying to an old question yet again but here's my hacky solution to this problem since I don't know how to write SQL functions... yet! %I formats the string as an identifier, and if there is no such table the return value is NULL and the alias is used!
SELECT (SELECT format('%I', 'my_column')
AS my_column_alias
FROM information_schema.columns
WHERE table_name='my_table'
AND column_name='my_column')
FROM source_table
Hope this helps everybody out there =)
I've read many of the gory details of write consistency and I understand how it works in the simple cases. What I'm not clear on is what this means for nested sub-queries.
Here's a concrete example:
A table with PK id, and other columns state, temp and date.
UPDATE table SET state = DECODE(state, 'rainy', 'snowy', 'sunny', 'frosty') WHERE id IN (
SELECT id FROM (
SELECT id,state,temp from table WHERE date > 50
) WHERE (state='rainy' OR state='sunny') AND temp < 0
)
The real thing was more convoluted (in the innermost query), but this captures the essence.
If we assume the state column is not nullable, can this update ever fail due to concurrent modification (i.e., the DECODE function doesn't find a match, a value of 'rainy' or 'sunny', and so tries to insert null into a non-nullable column)?
Oracle supports "statement level read and write consistency" (as all other serious DBMS)
This means that the statement as a whole will not see any changes to the database that occurred after the statement started.
As your UPDATE is one single statement there shouldn't be a case where the decode returns null.
Btw: the statement can be simplified, you don't need the outer SELECT in the sub-query:
UPDATE table SET state = DECODE(state, 'rainy', 'snowy', 'sunny', 'frosty')
WHERE id IN (
SELECT id
FROM table
WHERE date > 50
AND (state='rainy' OR state='sunny')
AND temp < 0
)
I don't see any reason to be concerned. The subquery explicitly retrieves only IDs of rows with state 'rainy' or 'sunny' and that's what outer DECODE is going to get. Thole thing is one statement, and is going to be executed within transaction boundaries.
Answering my own question: turns out there is a bug in Oracle which can cause this query to fail. Details confirmed by Tom Kyte, in the discussion starting here.