I need to select a column only if it exists in table, else it can be set to null.
Sample table below, lets say the marks col is not necessary be there, so need to be checked if it exists
Table1:
name marks
joe 10
john 11
mary 13
Query:
select
name,
marks if it exists else null as marks1 -- pseudo code
from
table1
What should go in line to select marks ?
SQL Doesn't permit that. Your result set has two options:
Static inclusion
All from table or subquery through column-expansion with * and tbl.*
Perhaps this will suit your needs, SELECT * FROM table1; You'll always get that column, if it exists.
try this
IF COL_LENGTH('your_table_name','column_name_you_want_to_select') IS NULL BEGIN
--This means columns does not exist or permission is denied
END
else
--Do whatever you want
It is possible to achieve this in PostgreSQL using JSON. Consider the following SQL query:
SELECT c.relname, c.relkind, c.relispartition
FROM pg_class c
WHERE c.relkind IN ('r','p') AND
c.relnamespace=(SELECT oid FROM pg_namespace WHERE nspname='public')
In PostgreSQL 10+, that will show you the names of all the tables in public schema, including whether they are partitioned and if so whether the table is the partitioned table or one of the partitions of it. However, if you try to run the same query on PostgreSQL 9.6 or earlier, it will fail since the relispartition column does not exist on the pg_class table prior to PostgreSQL 10.
An obvious solution would be to dynamically generate the SQL based on a condition, or have two different versions of the SQL. However, suppose you don't want to do that, you want to have a single query which works on both versions – in other words, you want to conditionally select the relispartition column if it exists.
The core SQL language does not have any facility to conditionally select a column, but it is achievable in PostgreSQL using the row_to_json function, as follows:
SELECT c.relname, c.relkind,
(row_to_json(c)->>'relispartition')::boolean AS relispartition
FROM pg_class c
WHERE c.relkind IN ('r','p') AND
c.relnamespace=(SELECT oid FROM pg_namespace WHERE nspname='public')
If you try running that, you will find on PostgreSQL 10+ the relispartition column is returned as true/false, whereas in pre-10 versions it is NULL. You could make it return false instead of NULL in pre-10 versions by doing COALESCE((row_to_json(c)->>'relispartition')::boolean,false).
What this is doing, is row_to_json(c) turns all the data of the row into JSON. Next, ->>'relispartition' selects the value of the relispartition JSON object key as text, which will be the same as the value of the relispartition column; if there is no such key in the JSON, the result of that will be NULL. Then, ::boolean converts the string value true/false back into a PostgreSQL boolean value. (If your column is of some other type, use the appropriate cast for the type of your column.)
(Obviously this approach will not work in Postgres versions which are too old to have the necessary JSON support – I have tested it works in Postgres 9.4; while I haven't tested it in Postgres 9.3, it probably works there. However, I would not expect it to work in 9.2 or earlier – the ->> operator was added in 9.3, and the JSON type and row_to_json function was added in 9.2. However, I expect few people will need to support those old unsupported versions–9.3 was released in 2013, and 9.2 supported ended in 2017.)
Try this:
IF EXISTS( SELECT 1
FROM information_schema.columns
WHERE table_name='your_table' and column_name='your_column') THEN
SELECT your_column as 'some_column'
ELSE
SELECT NULL as 'some_column'
END IF
Replying to an old question yet again but here's my hacky solution to this problem since I don't know how to write SQL functions... yet! %I formats the string as an identifier, and if there is no such table the return value is NULL and the alias is used!
SELECT (SELECT format('%I', 'my_column')
AS my_column_alias
FROM information_schema.columns
WHERE table_name='my_table'
AND column_name='my_column')
FROM source_table
Hope this helps everybody out there =)
Related
Does PostgreSQL support computed / calculated columns, like MS SQL Server? I can't find anything in the docs, but as this feature is included in many other DBMSs I thought I might be missing something.
Eg: http://msdn.microsoft.com/en-us/library/ms191250.aspx
Postgres 12 or newer
STORED generated columns are introduced with Postgres 12 - as defined in the SQL standard and implemented by some RDBMS including DB2, MySQL, and Oracle. Or the similar "computed columns" of SQL Server.
Trivial example:
CREATE TABLE tbl (
int1 int
, int2 int
, product bigint GENERATED ALWAYS AS (int1 * int2) STORED
);
fiddle
VIRTUAL generated columns may come with one of the next iterations. (Not in Postgres 15, yet).
Related:
Attribute notation for function call gives error
Postgres 11 or older
Up to Postgres 11 "generated columns" are not supported.
You can emulate VIRTUAL generated columns with a function using attribute notation (tbl.col) that looks and works much like a virtual generated column. That's a bit of a syntax oddity which exists in Postgres for historic reasons and happens to fit the case. This related answer has code examples:
Store common query as column?
The expression (looking like a column) is not included in a SELECT * FROM tbl, though. You always have to list it explicitly.
Can also be supported with a matching expression index - provided the function is IMMUTABLE. Like:
CREATE FUNCTION col(tbl) ... AS ... -- your computed expression here
CREATE INDEX ON tbl(col(tbl));
Alternatives
Alternatively, you can implement similar functionality with a VIEW, optionally coupled with expression indexes. Then SELECT * can include the generated column.
"Persisted" (STORED) computed columns can be implemented with triggers in a functionally equivalent way.
Materialized views are a related concept, implemented since Postgres 9.3.
In earlier versions one can manage MVs manually.
YES you can!! The solution should be easy, safe, and performant...
I'm new to postgresql, but it seems you can create computed columns by using an expression index, paired with a view (the view is optional, but makes makes life a bit easier).
Suppose my computation is md5(some_string_field), then I create the index as:
CREATE INDEX some_string_field_md5_index ON some_table(MD5(some_string_field));
Now, any queries that act on MD5(some_string_field) will use the index rather than computing it from scratch. For example:
SELECT MAX(some_field) FROM some_table GROUP BY MD5(some_string_field);
You can check this with explain.
However at this point you are relying on users of the table knowing exactly how to construct the column. To make life easier, you can create a VIEW onto an augmented version of the original table, adding in the computed value as a new column:
CREATE VIEW some_table_augmented AS
SELECT *, MD5(some_string_field) as some_string_field_md5 from some_table;
Now any queries using some_table_augmented will be able to use some_string_field_md5 without worrying about how it works..they just get good performance. The view doesn't copy any data from the original table, so it is good memory-wise as well as performance-wise. Note however that you can't update/insert into a view, only into the source table, but if you really want, I believe you can redirect inserts and updates to the source table using rules (I could be wrong on that last point as I've never tried it myself).
Edit: it seems if the query involves competing indices, the planner engine may sometimes not use the expression-index at all. The choice seems to be data dependant.
One way to do this is with a trigger!
CREATE TABLE computed(
one SERIAL,
two INT NOT NULL
);
CREATE OR REPLACE FUNCTION computed_two_trg()
RETURNS trigger
LANGUAGE plpgsql
SECURITY DEFINER
AS $BODY$
BEGIN
NEW.two = NEW.one * 2;
RETURN NEW;
END
$BODY$;
CREATE TRIGGER computed_500
BEFORE INSERT OR UPDATE
ON computed
FOR EACH ROW
EXECUTE PROCEDURE computed_two_trg();
The trigger is fired before the row is updated or inserted. It changes the field that we want to compute of NEW record and then it returns that record.
PostgreSQL 12 supports generated columns:
PostgreSQL 12 Beta 1 Released!
Generated Columns
PostgreSQL 12 allows the creation of generated columns that compute their values with an expression using the contents of other columns. This feature provides stored generated columns, which are computed on inserts and updates and are saved on disk. Virtual generated columns, which are computed only when a column is read as part of a query, are not implemented yet.
Generated Columns
A generated column is a special column that is always computed from other columns. Thus, it is for columns what a view is for tables.
CREATE TABLE people (
...,
height_cm numeric,
height_in numeric GENERATED ALWAYS AS (height_cm * 2.54) STORED
);
db<>fiddle demo
Well, not sure if this is what You mean but Posgres normally support "dummy" ETL syntax.
I created one empty column in table and then needed to fill it by calculated records depending on values in row.
UPDATE table01
SET column03 = column01*column02; /*e.g. for multiplication of 2 values*/
It is so dummy I suspect it is not what You are looking for.
Obviously it is not dynamic, you run it once. But no obstacle to get it into trigger.
Example on creating an empty virtual column
,(SELECT *
From (values (''))
A("virtual_col"))
Example on creating two virtual columns with values
SELECT *
From (values (45,'Completed')
, (1,'In Progress')
, (1,'Waiting')
, (1,'Loading')
) A("Count","Status")
order by "Count" desc
I have a code that works and use the term calculated, I'm not on postgresSQL pure tho we run on PADB
here is how it's used
create table some_table as
select category,
txn_type,
indiv_id,
accum_trip_flag,
max(first_true_origin) as true_origin,
max(first_true_dest ) as true_destination,
max(id) as id,
count(id) as tkts_cnt,
(case when calculated tkts_cnt=1 then 1 else 0 end) as one_way
from some_rando_table
group by 1,2,3,4 ;
A lightweight solution with Check constraint:
CREATE TABLE example (
discriminator INTEGER DEFAULT 0 NOT NULL CHECK (discriminator = 0)
);
I have been trying to find an optimal solution to select unique values from each column. My problem is I don't know column names in advance since different table has different number of columns. So first, I have to find column names and I could use below query to do it:
select column_name from information_schema.columns
where table_name='m0301010000_ds' and column_name like 'c%'
Sample output for column names:
c1, c2a, c2b, c2c, c2d, c2e, c2f, c2g, c2h, c2i, c2j, c2k, ...
Then I would use returned column names to get unique/distinct value in each column and not just distinct row.
I know a simplest and lousy way is to write select distict column_name from table where column_name = 'something' for every single column (around 20-50 times) and its very time consuming too. Since I can't use more than one distinct per column_name, I am stuck with this old school solution.
I am sure there would be a faster and elegant way to achieve this, and I just couldn't figure how. I will really appreciate any help on this.
You can't just return rows, since distinct values don't go together any more.
You could return arrays, which can be had simpler than you may have expected:
SELECT array_agg(DISTINCT c1) AS c1_arr
,array_agg(DISTINCT c2a) AS c2a_arr
,array_agg(DISTINCT c2b) AS c2ba_arr
, ...
FROM m0301010000_ds;
This returns distinct values per column. One array (possibly big) for each column. All connections between values in columns (what used to be in the same row) are lost in the output.
Build SQL automatically
CREATE OR REPLACE FUNCTION f_build_sql_for_dist_vals(_tbl regclass)
RETURNS text AS
$func$
SELECT 'SELECT ' || string_agg(format('array_agg(DISTINCT %1$I) AS %1$I_arr'
, attname)
, E'\n ,' ORDER BY attnum)
|| E'\nFROM ' || _tbl
FROM pg_attribute
WHERE attrelid = _tbl -- valid, visible table name
AND attnum >= 1 -- exclude tableoid & friends
AND NOT attisdropped -- exclude dropped columns
$func$ LANGUAGE sql;
Call:
SELECT f_build_sql_for_dist_vals('public.m0301010000_ds');
Returns an SQL string as displayed above.
I use the system catalog pg_attribute instead of the information schema. And the object identifier type regclass for the table name. More explanation in this related answer:
PLpgSQL function to find columns with only NULL values in a given table
If you need this in "real time", you won't be able to archive it using a SQL that needs to do a full table scan to archive it.
I would advise you to create a separated table containing the distinct values for each column (initialized with SQL from #Erwin Brandstetter ;) and maintain it using a trigger on the original table.
Your new table will have one column per field. # of row will be equals to the max number of distinct values for one field.
For on insert: for each field to maintain check if that value is already there or not. If not, add it.
For on update: for each field to maintain that has old value != from new value, check if the new value is already there or not. If not, add it. Regarding the old value, check if any other row has that value, and if not, remove it from the list (set field to null).
For delete : for each field to maintain, check if any other row has that value, and if not, remove it from the list (set value to null).
This way the load mainly moved to the trigger, and the SQL on the value list table will super fast.
P.S.: Make sure to pass all you SQL from trigger to explain plan to make sure they use best index and execution plan as possible. For update/deletion, just check if old value exists (limit 1).
I am wondering if there is a way to write the SQL so that it would return me a result as usual but now, on the first row that would return also the attribute names.
To explain what I mean:
say you have a table "test" which has 2 attributes "id" and "name":
id name
1 nik
2 tst
query:
SELECT * FROM test;
produces:
1 nik
2 tst
but what I want it to return is this:
id name
1 nik
2 tst
Is this possible?
edit: I am using PostreSQL
You cannot return the names and the actual column values in a single result unless you give up on the real datatypes (which is probably not what you want).
Your example mixes character data and numeric data in the id column and Postgres will (rightfully) refuse to return such a result set.
Edit:
I tested the "union" solution given e.g. by JNK and it fails (as expected) on Postgres, Oracle and SQL Server precisely because of the non-matching datatypes. MySQL follows it's usual habits of not throwing errors and simply converts everything to characters.
Extremely generic answer since you don't provide an RDBMS:
SELECT id, name FROM(
SELECT 'id' as 'id', 'name' as 'name', 1 as 'Rank'
UNION ALL
SELECT *, 2 as 'Rank' FROM test) as X
ORDER BY [RANK]
EDIT
Thanks to Martin for pointing out the need for the ORDER BY
Assuming you are on SQL Server, you can get the column names of a specific table by using this query:
select column_name 'Column Name', data_type 'Data Type'
from information_schema.columns
where table_name = 'putYourTableNameHere'
Then, you'll have to UNION your things together.
I agree with OMG Ponies above, the way to get this meta-data is usually from the interface you use.
for example the Perl DBI module has a method fetchrow_hashref where the columns of the returned row are returned as an associative array (hash) where the colnames are the keys.
print $ref->{'name'}; # would print nik or tst
Update:
I had forgotten to add that some of these interface layers have a method that return s the col names and you could use that instead of adding the names into your result set.
The DBI method you'd use would be $sth->{NAMES}->[0] would return the first column name.
Depending on your tools / technique, but here are a couple:
If you're using SSMS (Sql-Server), and want to copy/paste your results with the column headers:
Query Window -->
R-Click
Results -->
Grid or Text -->
Check-mark the 'Include column headers in the result set' option
If you're using Sql-Server, you can query meta-tables (sys.columns, etc.)
If you're using an ASP.NET databound control, you usually have access to methods or properties (sqldatareader.getname(i), etc.)
Anyway -- just depends on the layer you're trying to get the names from -- if these above don't help, then edit / re-tag your question so we can focus on whatever tool you're wanting to use to do this.
EDIT for PostgresSQL
If you're using PostgresSQL, you can query meta-tables (information_schema.columns, etc.)
Postgresql allows adding comments to objects such as tables. For example I've added a comment to table "mytable" by using this SQL command:
COMMENT ON TABLE mytable IS 'This is my table.';
My question is:
If I want to use a SQL-command to get all tables along with their respective comment - how would I do this? What would be the appropriate query for this?
Thanks in advance!
Cheers!
All comments are stored in pg_description
To get the comments on a table, you need to join it to pg_class
As an alternative you can also use the function obj_description() to retrieve this information:
SELECT obj_description(oid)
FROM pg_class
WHERE relkind = 'r'
Edit
In psql you can simply use the \d+ command to show all tables including their comments. Or use the \dd command to show all comments in the system
The main problem with "show comments" is to remember the name of specific fucntions, catalog names, etc. to retrieve the comment... Or its pages on the Guide. At this answer we solve in 2 ways: by a summary of the ordinary way (the pg-way) to show comments; and by offering shortcut functions, to reduce the "remember problem".
The pg-way
The simplest, on psql, is to use \dt+ to show table comments and \d+ to show column comments. Some for function comments?
To get on SQL, and for people that remember all parameters, the pg-way is to use the obj_description() function (Guide) in conjunction with adequate reg-type:
Function: select obj_description('mySchema.myFunction'::regproc, 'pg_proc')
Table or View: ("... and most everything else that has columns or is otherwise similar to a table",guide) select obj_description('mySchema.myClass'::regclass, 'pg_class')
other generic: select obj_description('mySchema.myObject'::regName, pg_regName), where regName is 1 in 10 of datatype-oid references Guide, and pg_regName is the same replacing prefix reg by prefix pg_.
other specific: similar select obj_description('schema.myObject'::regName, catalog_name), where catalog_name is to be more specific about a (1 in 95) key-word at catalogs Guide. It can reduce some "namespace pollution". For example pg_proc for functions, pg_aggregate for aggregate functions.
to get comment for a shared database object, analog but using the function shobj_description() (same page Guide).
Column: select col_description('mySchema.myObject'::regClass, column_number), where column_number is the column's ordinal position (at the CREATE TABLE). No column-name... See col_description(table,column_name) complement bellow.
IMPORTANT: the use of same reg-type and _catalog_name_ (e. g. ::regclass and pg_class) seems redundant and sometimes obj_description('obj'::regObj) works fine, with only reg-type! ...But, as the Guide say:
it is deprecated since there is no guarantee that OIDs are unique across different system catalogs; therefore, the wrong comment might be returned.
Shortcut functions to get comments
if you are finding it difficult to remember all the type-casts and parameters, the best is to adopt a new and simplest function to retrieve comments.
CREATE FUNCTION rel_description(
p_relname text, p_schemaname text DEFAULT NULL
) RETURNS text AS $f$
SELECT obj_description((CASE
WHEN strpos($1, '.')>0 THEN $1
WHEN $2 IS NULL THEN 'public.'||$1
ELSE $2||'.'||$1
END)::regclass, 'pg_class');
$f$ LANGUAGE SQL;
-- EXAMPLES OF USE:
-- SELECT rel_description('mytable');
-- SELECT rel_description('public.mytable');
-- SELECT rel_description('otherschema.mytable');
-- SELECT rel_description('mytable', 'otherschema');
-- PS: rel_description('public.mytable', 'otherschema') is a syntax error,
-- but not generates exception: returns the same as ('public.mytable')
We need also something less ugly to show column comments. There are no kind of pg_get_serial_sequence() function to get ordinal position of a column from its name. The native col_description('mySchema.myObject'::regClass, column_number) needs a complement:
CREATE FUNCTION col_description(
p_relname text, -- table name or schema.table
p_colname text, -- table's column name
p_database text DEFAULT NULL -- NULL for current
) RETURNS text AS $f$
WITH r AS (
SELECT CASE WHEN array_length(x,1)=1 THEN array['public',x[1]] ELSE x END
FROM regexp_split_to_array(p_relname,'\.') t(x)
)
SELECT col_description(p_relname::regClass, ordinal_position)
FROM r, information_schema.columns i
WHERE i.table_catalog = CASE
WHEN $3 IS NULL THEN current_database() ELSE $3
END and i.table_schema = r.x[1]
and i.table_name = r.x[2]
and i.column_name = p_colname
$f$ LANGUAGE SQL;
-- SELECT col_description('tableName','colName');
-- SELECT col_description('schemaName.tableName','colName','databaseName);
NOTES:
As recommended by this answer: "If you want to know which queries does psql run when you do \dt+ or \d+ customers, just launche it with psql -E".
It is possible to express multiline comment, using any multiline string (with E\n or $$...$$)... But you can't apply trim() or use another dynamic aspect. Must use dynamic SQL on COMMENT clause for it.
No comments to see? PostgreSQL programmers not use COMMENT clause because it is ugly to use: there are no syntax to add comment on CREATE TABLE or on CREATE FUNCTION; and there are no good IDE to automatize it.
The modern http://postgREST.org/ interface show comments on the Web!
You can use pg_catalog.obj_description function and information_schema.tables schema view:
SELECT t.table_name, pg_catalog.obj_description(pgc.oid, 'pg_class')
FROM information_schema.tables t
INNER JOIN pg_catalog.pg_class pgc
ON t.table_name = pgc.relname
WHERE t.table_type='BASE TABLE'
AND t.table_schema='public';
FUNCTIONS-INFO-COMMENT-TABLE
INFORMATION_SCHEMA Support in MySQL, PostgreSQL
If you still have tables with mixed case names you can use the following to get a complete list of all tables in a database with their comment, as well as catalog, schema, table type, etc. (tested in PostGIS - pgAdmin 4):
select *,
obj_description((table_schema||'.'||quote_ident(table_name))::regclass)
from information_schema.tables where table_schema <> 'pg_catalog'
I have a table in informix (Version 11.50.UC4) called NextRecordID with just one column called id and it will have one row. What I want to do is copy this value into another table. But don't want my query to fail if this table does not exist. Something like
if table NextRecordID exists
then insert into sometable values ('NextRecordID', (select id from NextRecordID))
else insert into sometable values ('NextRecordID', 1)
I ended up using the below SQL query. Its not ANSI SQL but works the informix server I am using.
insert into sometable values ('NextRecordID',
select case (select 1 from systables where tabname='nextrecordid')
when 1 then (select nextid from nextrecordid)
else (select 1 from systables where tabname='systables') end
from systables where tabname='systables');
What is happening here is within insert query I get the value to be inserted by using select query. Now that select query is interesting. It uses case statement of Informix. I have written a select query to check if the table nextrecordid exists in systables and return 1 if it exists. If this query returns 1, I query the table nextrecordid for the value or else I wrote a query to return the default value 1. This work for me.
You should be able to do this by checking the systables table.
Thank you for including server version information - it makes answering your question easier.
You've not indicated which language(s) you are using.
Normally, though, you design a program to expect a certain schema (certain tables to be present), and then fail - preferably under control - if those tables are not present. Also, it is not clear whether you would get into problems because of repeated execution of the second INSERT statement. Nor is it clear when the NextRecordID table is updated - presumably, once the value has been used, it must be updated.
You should look at SERIAL (BIGSERIAL) and see whether that is appropriate for you.
You should also look at whether a SEQUENCE would be appropriate to use here - it certainly looks rather like it might be applicable.
As Adam Hughes points out, if you want to check whether the NextRecordID table is present in the database, you would look in the systables table. Be aware, though, that your search will need to be against an all lower-case name (nextrecordid).
Also, MODE ANSI databases complicate life - you have to worry about the table's owner (because there could be multiple tables called nextrecordid in a MODE ANSI database). Most likely, you don't have to worry about that - any more than you are likely to have to worry about delimited identifiers for table "someone"."NextRecordID" (which is a different table from someone.NextRecordID).