I am having an issue with my postgresql database. I added 5 Tables with a lot of data and a lot of columns. Now I noticed I added the columns with a mix of upper and lowercase letters, which makes it difficult to query them using sqlalchemy or pandas.read_sql_query, because I need double quotes to access them.
Is there a way to change all values in the column names to lowercase letters with a single command?
Im new to SQL, any help is appreciated.
Use an anonymous code block with a FOR LOOP over the table columns:
DO $$
DECLARE row record;
BEGIN
FOR row IN SELECT table_schema,table_name,column_name
FROM information_schema.columns
WHERE table_schema = 'public' AND
table_name = 'table1'
LOOP
EXECUTE format('ALTER TABLE %I.%I RENAME COLUMN %I TO %I',
row.table_schema,row.table_name,row.column_name,lower(row.column_name));
END LOOP;
END $$;
Demo: db<>fiddle
If you wish to simply ensure that the query returns lowercase (without changing the original entries), you can simply input:
select lower(variable) from table;
On the other hand, if you wish to actually change the case in the table itself, you must use an UPDATE command.
UPDATE table SET variable = LOWER(variable);
Something like that should do the trick:
SELECT LOWER(column) FROM my_table;
Related
I got many tables of which some end with digits (date).
Due to technical reasons I had to create a view on every table (just a workaround)
The current state is something like:
begin
for i in (select table_name
from user_tables
where table_name like 'XXX_%'
)
loop
execute immediate 'CREATE VIEW '||replace(i.TABLE_NAME,'XXX_','YYY_')||' AS SELECT * FROM '||i.TABLE_NAME;
end loop;
end;
/
All it does is creating views for every table which begins with 'XXX_'. Its almost what I need.
But now, sometimes tablenames look like "XXX_tablename_20210302", which is a manual backup.
Its not always 8 digits (date), sometimes shorter, soemtimes longer numbers. I would like to avoid all tables, which do have numbers in the end of the tablename (from right to left till first "_" checkup, its a number maybe?)
Does anyone know how to solve it?
Im kind of stuck here.
You can use regular expressions with regexp_like:
create table xxx_t ( c1 int );
create table xxx_t_20210401 ( c1 int );
select table_name from user_tables
where regexp_like ( table_name, '^XXX_.*[^0-9]+$' );
TABLE_NAME
XXX_T
This finds all the tables that
Start with XXX_ XXX_
Followed by any characters .*
That do not end with a digit [^0-9]+$
Lets say we have a table A with 200 different columns. We want to select columns that contain a certain substring (e.g. the "host" substring in "host_id", "host_name", "average_host_rating"), and create a new table B with only those columns and their data imported from a .csv file.
I tried creating the new table manually, however this is not good practice and i want to improve the code by making it valid and functional even if i add more columns to table A.
Creating the table manually:
SELECT
listings.host_id,
listings.host_url ,
..
..
listings.host_name ,
listings.host_since ,
INTO host_table
FROM listings
WHERE TRUE;
Trying to create the table in a better way:
CREATE TABLE B AS
SELECT *
FROM A
WHERE A::text LIKE '%host%'
I expected it to create table B with every column that contains 'host' in its name however it returned an exact copy of table A (and all its data). I tried different ways and methods of creating new tables, however the problem always was that i could not isolate only the columns with the specified substring ('host').
What could be wrong in my syntax, way of thinking or anything else?
Thanks in advance!
You may create and call a function with parameters. The function will dynamically chose the columns from information_schema.columns.
Note that where false is used because you mentioned data will come from a csv file and not from the original table.
create or replace function
fn_gen_tab_text ( curr_tab_in text,tab_text_in TEXT, new_tab_in text)
RETURNS void AS
$body$
declare v_sql TEXT;
BEGIN
select 'CREATE TABLE %I AS select ' || string_agg(column_name,',')
||' from %I where false'
into v_sql from information_schema.columns
where table_name = curr_tab_in
and column_name like '%'||tab_text_in||'%';
EXECUTE format (v_sql,new_tab_in,curr_tab_in);
END $body$ language plpgsql
Call it as
select fn_gen_tab_text('host_table','host','new_table' );
DEMO
I need to get the table names queried in a pl/sql package file.
I know that there is an option for this in Notepad++ by regex but I don't know what regex to apply for get the table names (I understand that must be some regex to take the keyword "FROM" and get the next string after space, I think so).
For the next example code:
CREATE OR REPLACE PACKAGE BODY pac_example AS
FUNCTION f1 RETURN NUMBER IS
BEGIN
SELECT * FROM table1;
RETURN 1;
END f1;
FUNCTION f2 RETURN NUMBER IS
BEGIN
SELECT * FROM table2;
RETURN 1;
END f2;
END pac_example;
And I expect replace all and get the file with only its table names:
table1
table2
If you're interested only in table names that are directly referred from the PACKAGE BODY, a simple and straight-forward method is to query all_dependencies or user_dependencies.
SELECT owner,
referenced_name as table_name
FROM all_dependencies
WHERE type IN (
'PACKAGE BODY'
) AND name IN (
'PAC_EXAMPLE'
) AND referenced_type = 'TABLE';
DEMO
To my knowledge no one has done this with 100% accuracy. The closest you get is the ALL/DBA_DEPENDENIES but does not tell you if the table is accessed in a SELECT, INSERT, UPDATE or DELETE.
It will however resolve synonyms.
The downside of this is that it will not include tables referenced in dynamic SQL.
If you have a database that uses particular naming convention for tables (e.g. Tnnn_XXXXX ) you could do:
SELECT DISTINCT c.text, c.name, c.type, t.table_name
FROM user_source c, user_tables t
WHERE UPPER(t.text) like '%' || t.name_name || '%' -- Maybe REGEXP_LIKE better
ORDER BY 2, 1, 4;
I worked on a project decades ago where they wanted a CRUD matrix of programs (PLSQL, SQL, Oracle Forms/Reports, ProC, ProCOBOL) and what tables each accessed.
The only solution available at the time was for me to write a parser (in C) that parsed the codebase looking for SQL and processing it. Mine even reported columns as well as tables. The C program parsed the source, looking for KEYWORDS and characters to control a state engine. It took a a couple of weeks to refine and get working across all the different codebase types.
By the end, the only thing it could not do was dynamic queries where the table name was built up from variable values. But the workaround here was to capture the tkprof files and process these.
Tragically, I do not have the source code for this anymore.
However, if I were to do it again, I would use Lex/Yacc/Bison to parser SQL and build a system around these tools.
A quick search found this:
https://github.com/jgarzik/sqlfun
https://www.oreilly.com/library/view/flex-bison/9780596805418/ch04.html
Not a small undertaking.
Ctrl+H
Find what: (?:\A(?:(?!FROM).)*|\G)FROM\s+(\w+(?:\s*,\s*\w+)*)(?:(?!FROM).)*
Replace with: " #a space and a double quote
check Wrap around
check Regular expression
CHECK . matches newline
Replace all
Explanation:
(?: # start non capture group
\A # beginning of file
(?:(?!FROM).)* # Tempered greedy token, make sure we haven't FROM before
| # OR
\G # restart from last match position
) # end group
FROM\s+ # literally FROM followed by 1 or more spaces
( # start group 1
\w+ # 1 or more word characters (table name)
(?:\s*,\s*\w+)* # non capture group spaces comma spaces and 1 or more word characters, optional more tables
) # end group
(?:(?!FROM).)* # Tempered greedy token, make sure we haven't FROM
Replacement:
$1 # content of group 1, table name
Screen capture:
You can use following regex to search for table names.
Regex: FROM\s([^;]+)
Replacement: \n%\1%\n
Then follow this answer for replacing other data in file.
the earlier mentioned tables
all_dependencies or user_dependencies
can list out the dependencies as mentioned earlier, but it wont cover the dynamic queries. and the search if done in notepad using key words like 'from' then only tables referred by statements after 'From' statements will be covered.
the below code snippet can be considered for full analysis :- line by line, word by word and analysis the tables (referred to the sample mentioned by you)
declare
l_line varchar2(2000);
ln_start_string number;
ln_last_string number;
ln_string_length number;
l_word varchar2(4000);
l_table_flag varchar(2):='N';
cursor l_pkg_body_cur
is
select TEXT from all_source where upper(name) like upper('pac_example') and type = 'PACKAGE BODY';
-- to get the source compiled in package boby, mentioned the package to be searched here
begin
for rec in l_pkg_body_cur
loop
-- line by line processing
select TRIM(rec.text) into l_line from dual;
ln_string_length := length(l_line);
loop
-- word by word processing
l_table_flag :='N';
select instr(l_line,' ') into ln_last_string from dual;
select substr(l_line,0,ln_last_string) into l_word from dual;
begin
select 'Y' into l_table_flag from all_tables where upper(table_name) like upper(trim(l_word)) and rownum=1; -- to validate it is table or not
exception
when others then
l_table_flag := 'N';
end;
IF l_table_flag = 'Y'
then
dbms_output.put_line(trim(l_word) ); -- table name
end if;
select length (l_word) into ln_start_string from dual;
select trim(substr(replace(l_line,';',null),ln_start_string)) into l_line from dual;
exit when l_line is NULL;
end loop;
end loop;
end;
--output:
Statement processed.
table1
table2
similar way this query can be altered as wanted to find views or synonyms deparated by changing the base table- all_views,all_synonyms, accordingly as per required.
this is like most straight forward approach- might take more processing time depends on the package size
same can be done using UNIX scripting if needed
if needed to check from the file directly UNIX scripting can be used (UTF_file operations can also be used ) to get line by line from the file and have sql session to do the above validation and to display the results
but hopefully this would provide most accurate results.
i want to dynamically delete some tables from database. I select them first, and then i am trying to use delete statement but it doesn't works.
tmp TEXT;
FOR tmp IN SELECT names from Garbage
LOOP
DROP TABLE tmp;
END LOOP;
but unfortuntly i got errors at drop statement. It always trying to delete table "tmp" instead of deleting the value of tmp(which are some strings like "table1" "table2").
You will need a dynamic query to be executed. To do that, you need to construct your query and call EXECUTE:
DECLARE
tmp TEXT;
...
BEGIN
...
FOR tmp IN SELECT names FROM Garbage
LOOP
EXECUTE 'DROP TABLE ' || quote_ident(tmp);
END LOOP;
...
Notice that I used quote_ident to escape the name properly, it is better and safer. But, if you created a table, let's say named MyTable but didn't quoted (double-quotes) it on creation time, PostgreSQL also store its name as mytable, so quote_ident('MyTable') will generate "MyTable" which doesn't exists, you should take care of it (e.g. always use the lowercase names or always quote your IDs).
I want to call create table/ alter table command from a procedure. Is it possible?
My requirement is to change the datatype of a column in all tables. So, I am just getting the column name from user_tab_cols. Now I want to create a temp table which requires create statement .. but i am unable to use that within a proc.
Can anyone please help me out?
I presume from the reference to USER_TAB_COLUMNS that this is Oracle. ALTER and CREATE statements are DDL, which we cannot execute directly in PL/SQL. However, there are a couple of ways around this restriction: EXECUTE IMMEDIATE and DBMS_UTILITY.EXEC_DDL(). I will use EXECUTE IMMEDIATE in the following example.
begin
for lrec in ( select table_name from user_tab_columns
where column_name = 'UNIVERSAL_COLUMN_NAME')
loop
execute immediate 'alter table '||lrec.table_name||
' modify UNIVERSAL_COLUMN_NAME varchar2(255)';
end loop;
end;
Note that the usual restrictions apply: the new datatype has to be compatible with the existing datatype (unless the column is empty), and things are trickier with some specilaized datatypes like CLOBs.
edit
I haven't addressed the CREATE TABLE statement. The principle is the same, it is just longer to type out. Besides, I am not entirely clear how it applies to your prior requirement to change the datatype of those columns.
you can generate the query as string and execute it with 'exec' keyword.