Im trying to now only display the user, schema & tables but also show the database the schema & tables belong to. Any suggestions on what I should add to this query?
SELECT
u.usename as user, t.schemaname as schema, t.tablename as table,
has_table_privilege(u.usename,t.schemaname||'.'||t.tablename,'select') AS "Select",
has_table_privilege(u.usename,t.schemaname||'.'||t.tablename,'insert') AS "Insert",
has_table_privilege(u.usename,t.schemaname||'.'||t.tablename,'update') AS "Update",
has_table_privilege(u.usename,t.schemaname||'.'||t.tablename,'delete') AS "Delete",
has_table_privilege(u.usename,t.schemaname||'.'||t.tablename,'references') AS "Reference"
FROM pg_user u
CROSS JOIN pg_tables t
WHERE t.schemaname != 'information_schema' and t.schemaname != 'pg_internal' and t.schemaname != 'pg_catalog' and t.tablename not like '% %'
ORDER BY u.usename, t.schemaname, t.tablename;
I suggest looking into information_schema.*. These are sql standards and information_schema.tables should have what you want.
You can replace the use of pg_tables as such:
select
u.usename as user,
t.table_catalog as database,
t.table_schema as schema,
t.table_name as table,
has_table_privilege(u.usename, t.table_schema || '.' || t.table_name, 'select') as "Select",
has_table_privilege(u.usename, t.table_schema || '.' || t.table_name, 'insert') as "Insert",
has_table_privilege(u.usename, t.table_schema || '.' || t.table_name, 'update') as "Update",
has_table_privilege(u.usename, t.table_schema || '.' || t.table_name, 'delete') as "Delete",
has_table_privilege(u.usename, t.table_schema || '.' || t.table_name, 'references') as "Reference"
from
pg_user u
cross join information_schema.tables t
where
t.table_schema != 'information_schema' and t.table_schema != 'pg_internal' and t.table_schema != 'pg_catalog'
and t.table_schema not like '% %'
order by
u.usename, t.table_schema, t.table_name;
Related
I have a large Snowflake database with 70+ tables and 3000+ fields. Is there a query I can use across the entire database to find all columns with all NULLs? I have a command I can use to find all the columns
select * from prod_db.information_schema.columns
Is there a way to modify that command to identify which columns are all NULLs? If there is not a way to do it across the entire database. Is there a way to do it across a table? I do not want to type:
select column_name from prod_db.information_schema.table_name
3000+ times. Thanks!
This uses a SQL generator to generate a SQL statement that will locate columns matching two criteria:
The column is in a table with one or more rows
The column has all nulls.
To be highly efficient, rather than checking the each table entirely, it uses a UNION ALL block that looks for a single non-null row in each table. It uses TOP 1 to find a not null row. That way as soon as it finds a not null row, it returns that row and stops scanning that table so it can move to another table scan.
This means that the large UNION ALL section will list tables where it finds a not null row, which is the opposite of what we want. To use this information, a CTE wrapped around the UNION ALL will do an anti-join against the column view in the information schema.
with COLS as
(
select 'select top 1 ''' || C.TABLE_CATALOG || ''' as TABLE_CATALOG, ''' || C.TABLE_SCHEMA ||
''' as TABLE_SCHEMA, ''' || C.TABLE_NAME || ''' as TABLE_NAME, ''' || C.COLUMN_NAME ||
''' as COLUMN_NAME from "' ||
C.TABLE_CATALOG || '"."' || C.TABLE_SCHEMA || '"."' || C.TABLE_NAME || '"' ||
' where "' || C.COLUMN_NAME || '" is not null'
as NULL_CHECK
from INFORMATION_SCHEMA.COLUMNS C
left join INFORMATION_SCHEMA.TABLES T on
C.TABLE_CATALOG = T.TABLE_CATALOG and
C.TABLE_SCHEMA = T.TABLE_SCHEMA and
C.TABLE_NAME = T.TABLE_NAME
where C.IS_NULLABLE = 'YES' and T.TABLE_TYPE = 'BASE TABLE'
and T.ROW_COUNT > 0
), UNIONED as
(
select listagg(NULL_CHECK, '\nunion all\n') as UNIONED from COLS
)
select replace($$
with NON_NULL_COLUMNS as (
!~UNIONED~!
)
select C.TABLE_CATALOG, C.TABLE_SCHEMA, C.TABLE_NAME, C.COLUMN_NAME from INFORMATION_SCHEMA.COLUMNS C
left join NON_NULL_COLUMNS NN
on C.TABLE_CATALOG = NN.TABLE_CATALOG
and C.TABLE_SCHEMA = NN.TABLE_SCHEMA
and C.TABLE_NAME = NN.TABLE_NAME
and C.COLUMN_NAME = NN.COLUMN_NAME
left join INFORMATION_SCHEMA.TABLES T
on C.TABLE_CATALOG = T.TABLE_CATALOG
and C.TABLE_SCHEMA = T.TABLE_SCHEMA
and C.TABLE_NAME = T.TABLE_NAME
where NN.COLUMN_NAME is null and T.ROW_COUNT > 0
;$$, '!~UNIONED~!', UNIONED) as SQL_TO_RUN from UNIONED
;
You can produce a list of SELECT queries for each column as follows
SELECT CONCAT(CONCAT(CONCAT(CONCAT(CONCAT(CONCAT(CONCAT(CONCAT(CONCAT(CONCAT(CONCAT(CONCAT('SELECT ''', TABLE_NAME), ''', '''), COLUMN_NAME), ''', '), 'COUNT(*) FROM '), TABLE_NAME), ' WHERE '), COLUMN_NAME), ' IS NULL OR '), LEN(COLUMN_NAME)), ' = 0'), ' UNION ')
from information_schema.columns
The result of the above query can then be taken and executed to get the result you need (PS: Remove the UNION on the last row produced from Step 1 before executing)
Hope this helps.
I am getting a permission denied on a table. Trying to find out if the user has permissions or not. Querying pg_tables and svv_tables yields different results. Why?
-- list permissions for user
SELECT
u.usename,
t.schemaname,
t.tablename,
has_schema_privilege(u.usename,t.schemaname,'create') AS user_has_select_permission,
has_schema_privilege(u.usename,t.schemaname,'usage') AS user_has_usage_permission
FROM
pg_user u,
pg_tables t
WHERE
u.usename = 'X'
AND tablename = 'Y'
ORDER BY
t.tablename
, t.schemaname
, usename
;
-- list permissions on table
SELECT DISTINCT
u.usename,
t.table_schema,
t.table_name,
has_table_privilege(u.usename, t.table_schema || '.' || t.table_name, 'select') AS user_has_select_permission,
has_table_privilege(u.usename,t.table_schema || '.' || t.table_name,'insert') AS user_has_insert_permission,
has_table_privilege(u.usename,t.table_schema || '.' || t.table_name,'update') AS user_has_update_permission,
has_table_privilege(u.usename,t.table_schema || '.' || t.table_name,'delete') AS user_has_delete_permission
, has_table_privilege(u.usename,t.table_schema || '.' || t.table_name,'references') AS user_has_references_permission
FROM
pg_user u
CROSS JOIN svv_tables t
WHERE
t.table_schema not like 'pg_temp%' and
t.table_schema not in ('pg_catalog', 'pg_internal') and
t.table_type != 'EXTERNAL TABLE'
and u.usename = 'X'
and t.table_name = 'Y'
order by
usename
, table_schema
, table_name
;
I'd like to retrieve all tables and the associated column values where two of their specific columns (the column names will be passed into) that don't have the exact same content in them.
Here's a more definite break-down of the problem. Suppose, the columns that I need to look into is 'Column_1' and 'Column_2'
First identify from in INFORMATION_SCHEMA which of the tables have both of these columns present in them(possible one sub-query),
And then identify which of these tables don't have exact same content on these 2 columns meaning Column_1 != Column_2.
The following section would retrieve all the tables that has both 'Column_1' and 'Column_2'.
SELECT
TABLE_NAME
FROM
INFORMATION_SCHEMA.TABLES T
WHERE
T.TABLE_CATALOG = 'myDB' AND
T.TABLE_TYPE = 'BASE TABLE'
AND EXISTS (
SELECT T.TABLE_NAME
FROM INFORMATION_SCHEMA.COLUMNS C
WHERE
C.TABLE_CATALOG = T.TABLE_CATALOG AND
C.TABLE_SCHEMA = T.TABLE_SCHEMA AND
C.TABLE_NAME = T.TABLE_NAME AND
C.COLUMN_NAME = 'Column_1')
AND EXISTS
(
SELECT T.TABLE_NAME
FROM INFORMATION_SCHEMA.COLUMNS C
WHERE
C.TABLE_CATALOG = T.TABLE_CATALOG AND
C.TABLE_SCHEMA = T.TABLE_SCHEMA AND
C.TABLE_NAME = T.TABLE_NAME AND
C.COLUMN_NAME = 'Column_2')
As the next step, I tried to use this as a sub-query and have the following at the end but that doesn't work and sql-server returns 'Cannot call methods on sysname'. What would the next step on this? This problem assumes all columns has the exact same Data-type.
WHERE SUBQUERY.TABLE_NAME.Column_1 != SUBQUERY.TABLE_NAME.Column_2
This is what's expected :
Table_Name
Column_Name1
Column_Value_1
Column_Name2
Column_Value_2
Table_A
Column_1
abcd
Column_2
abcde
Table_A
Column_1
qwerty
Column_2
qwert
Table_A
Column_1
abcde
Column_2
eabcde
Table_B
Column_1
zxcv
Column_2
zxcde
Table_C
Column_1
asdfgh
Column_2
asdfghy
Table_C
Column_1
aaaa
Column_2
bbbb
If in fact you want to actually compare values (not length) between two columns in tables that contain those two columns, you will need to generate dynamic SQL and then execute it. This could be done semi-automatically with the following:
DECLARE #SqlTemplate VARCHAR(MAX) =
'UNION ALL'
+ ' SELECT Table_Name = <TNAME>'
+ ', Column_Name1 = <C1NAME>, Column_Value_1 = <C1>'
+ ', Column_Name2 = <C2NAME>, Column_Value_2 = <C2>'
+ ' FROM <T>'
+ ' WHERE ISNULL(<C1>, '(null)') <> ISNULL(<C2>, '(null)')'
SELECT T.TABLE_NAME
, REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(
#SqlTemplate
, '<TNAME>', QUOTENAME(T.TABLE_SCHEMA + '.' + T.TABLE_NAME, ''''))
, '<C1NAME>', QUOTENAME(C1.COLUMN_NAME, ''''))
, '<C2NAME>', QUOTENAME(C2.COLUMN_NAME, ''''))
, '<T>', QUOTENAME(T.TABLE_SCHEMA) + '.' + QUOTENAME(T.TABLE_NAME))
, '<C1>', QUOTENAME(C1.COLUMN_NAME))
, '<C2>', QUOTENAME(C2.COLUMN_NAME))
FROM INFORMATION_SCHEMA.TABLES T
JOIN INFORMATION_SCHEMA.COLUMNS C1
ON C1.TABLE_CATALOG = T.TABLE_CATALOG
AND C1.TABLE_SCHEMA = T.TABLE_SCHEMA
AND C1.TABLE_NAME = T.TABLE_NAME
AND C1.COLUMN_NAME = 'Column_1'
JOIN INFORMATION_SCHEMA.COLUMNS C2
ON C2.TABLE_CATALOG = T.TABLE_CATALOG
AND C2.TABLE_SCHEMA = T.TABLE_SCHEMA
AND C2.TABLE_NAME = T.TABLE_NAME
AND C2.COLUMN_NAME = 'Column_2'
WHERE T.TABLE_CATALOG = 'myDB'
AND T.TABLE_TYPE = 'BASE TABLE'
This would generate sql for each qualifying table of the form:
UNION ALL SELECT Table_Name = 'dbo.Z', Column_Name1 = 'X', Column_Value_1 = [X], Column_Name2 = 'Y', Column_Value_2 = [Y] FROM [dbo].[Z] WHERE ISNULL([X], '(null)') <> ISNULL([Y], '(null)')
After running the above, you would then cut & paste the generated SQL into another query window, remove the initial 'UNION ALL', and then execute the remaining SQL to get the final results.
There are ways of combining all the SQL into a single string and executing it automatically, but your problem sounds like a one-off process that doesn't warrant the extra complexity.
I believe you need to compare the CHARACTER_MAXIMUM_LENGTH or CHARACTER_OCTET_LENGTH metadata values in the INFORMATION_SCHEMA.COLUMNS table instead of using LEN(). This can be done using something like:
SELECT T.TABLE_NAME
, C1.COLUMN_NAME, C1.DATA_TYPE, C1.CHARACTER_MAXIMUM_LENGTH
, C2.COLUMN_NAME, C2.DATA_TYPE, C2.CHARACTER_MAXIMUM_LENGTH
FROM INFORMATION_SCHEMA.TABLES T
JOIN INFORMATION_SCHEMA.COLUMNS C1
ON C1.TABLE_CATALOG = T.TABLE_CATALOG
AND C1.TABLE_SCHEMA = T.TABLE_SCHEMA
AND C1.TABLE_NAME = T.TABLE_NAME
AND C1.COLUMN_NAME = 'Column_1'
JOIN INFORMATION_SCHEMA.COLUMNS C2
ON C2.TABLE_CATALOG = T.TABLE_CATALOG
AND C2.TABLE_SCHEMA = T.TABLE_SCHEMA
AND C2.TABLE_NAME = T.TABLE_NAME
AND C2.COLUMN_NAME = 'Column_2'
WHERE T.TABLE_CATALOG = 'myDB'
AND T.TABLE_TYPE = 'BASE TABLE'
AND C1.CHARACTER_MAXIMUM_LENGTH <> C2.CHARACTER_MAXIMUM_LENGTH
The inner joins both limit results to tables having both columns and retrieve the column metadata. The length compare at the end checks for a mismatch.
This assumes character types. You might also want to check DATA_TYPE consistency ("char" vs "varchar" vs "nvarchar") or some of the other precision and scale values for other non-character data types.
To query the data within the columns you need dynamic SQL. I would advise you not to use INFORMATION_SCHEMA (which is for compatibility only) and instead use sys.tables etc. You don't need to check sys.columns twice, you can use aggregation in the EXISTS subquery to check for multiple columns.
To compare the columns, you can do Column_1 <> Column_2, but that will not deal with nulls correctly. If the columns can be nullable then you should instead use the syntax shown in the code below: NOT EXISTS (SELECT Column_1 INTERSECT SELECT Column_2)
DECLARE #sql nvarchar(max);
SELECT
STRING_AGG(CAST('
SELECT
Table_Name = ' + QUOTENAME(t.name, '''') + ',
Column_1,
Column_2
FROM ' + QUOTENAME(s.name) + '.' + QUOTENAME(t.name) + '
WHERE NOT EXISTS (SELECT Column_1 INTERSECT SELECT Column_2)
' AS nvarchar(max)), '
UNION ALL
' )
FROM sys.tables t
JOIN sys.schemas s ON s.schema_id = t.schema_id
AND s.name = 'myDB'
WHERE EXISTS (SELECT 1
FROM sys.columns c
WHERE c.object_id = t.object_id
AND c.name IN ('Column_1', 'Column_2')
HAVING COUNT(*) = 2
AND COUNT(DISTINCT c.system_type_id) = 1 -- all same type
);
PRINT #sql; -- your friend
EXEC sp_executesql #sql;
I have a table with mixed types of data (real, integrer, character ...) but i would only recover columns that have real values.
I can construct this:
SELECT 'SELECT ' || array_to_string(ARRAY(
select 'o' || '.' || c.column_name
from information_schema.columns as c
where table_name = 'final_datas'
and c.data_type = 'real'), ',') || ' FROM final_datas as o' As sqlstmt
that gives that:
"SELECT o.random,o.struct2d_pred2_num,o.pfam_num,o.transmb_num [...] FROM final_datas as o"
The i would like to create a table with these columns. Of course, do this, doesn't work:
create table table2 as (
SELECT 'SELECT ' || array_to_string(ARRAY(
select 'o' || '.' || c.column_name
from information_schema.columns as c
where table_name = 'final_datas'
and c.data_type = 'real'), ',') || ' FROM final_datas as o' As sqlstmt
)
Suggestions?
You need to generate the whole CREATE TABLE statement as dynamic SQL:
SELECT 'CREATE TABLE table2 AS SELECT ' || array_to_string(ARRAY(
select 'o' || '.' || c.column_name
from information_schema.columns as c
where table_name = 'final_datas'
and c.data_type = 'real'), ',') || ' FROM final_datas as o' As sqlstmt
The result can be run with EXECUTE sqlstmt;
how to get all the tables and structure in a database in a table format using an sql query or stored procedure?
Structure is like below:
Sl No FieldName DataType Size Description
1 UserName varchar 50
This should do the trick. There is a bit more information in here but I think you may find it useful.
Select t.Table_Schema,
t.Table_Name,
c.Column_Name,
IsNull(c.Column_Default, '') as 'Column_Default',
c.Is_Nullable,
c.Data_Type,
IsNull(c.Character_Maximum_Length, IsNull(Numeric_Precision,'') + IsNull(Numeric_Scale, IsNull(DateTime_Precision,''))) as 'Size'
From Information_Schema.Tables t
Join Information_Schema.Columns c on t.Table_Catalog = c.Table_Catalog
And t.Table_Schema = c.Table_Schema
And t.Table_Name = c.Table_Name
Where t.Table_Type = 'BASE TABLE'
Order by t.Table_Schema, t.Table_Name, c.Ordinal_Position