Refactor sql query that uses PostGIS functions - sql

I have a query that looks like this:
SELECT *,
ST_Distance(
ST_GeographyFromText('SRID=4326;POINT(' || users.longitude || ' ' || users.latitude || ')'),
ST_GeographyFromText('SRID=4326;POINT(-84.334078 45.273343)')) as distance
FROM users
WHERE ST_DWithin(
ST_GeographyFromText('SRID=4326;POINT(' || users.longitude || ' ' || users.latitude || ')'),
ST_GeographyFromText('SRID=4326;POINT(-84.334078 45.273343)'),
2000
)
ORDER BY distance ASC;"
I've see some repetitions here. I'm wondering is there any way to make this query more readable?

Lateral join:
select *, ST_Distance(a, b) distance
from
users,
ST_GeographyFromText('SRID=4326;POINT(' || users.longitude || ' ' || users.latitude || ')') a,
ST_GeographyFromText('SRID=4326;POINT(-84.334078 45.273343)') b
where ST_DWithin(a, b, 2000)
order by distance asc;

Related

Reusing alias for an expression in postgresql

I am trying to do this:
select first_name || ' ' || last_name as full_name, Length(full_name) as len from customer order by len
It is not possible;
column "full_name" does not exist
So, I have to do this:
select first_name || ' ' || last_name as full_name, Length(first_name || ' ' || last_name) as len from customer order by len
Does it mean sql engine has to compute expression 'first_name || ' ' || last_name' two times?
As you observe, what you want to do is not possible. Instead, you can use a lateral join to calculate values in the FROM clause:
select v.full_name, Length(v.full_name) as len
from customer c cross join lateral
(values (first_name || ' ' || last_name)
) v(full_name)
order by len;

How to quickly see what columns in a table have data?

We are currently undertaking a testing phase which requires us to see if there is any data in each column for each table. Now, the route that is long and labour-intensive is:
SELECT COUNT(Col1), COUNT(Col2)...FROM TABLE
Is there any easier way to do this? We can go down this route by concatenating each column name from our data lineage document with the COUNT() function, but we have a lot of tables and a lot of columns in each table, making this a bit unfeasible.
Essentially we just need a count of records in each column for each table, without having to write long COUNT(Col) queries.
Thanks
This query will return accurate results if the table statistics were recently gathered with the default value for ESTIMATE_PERCENT:
SELECT utab.table_name
, tcol.column_name
, utab.num_rows
from user_tables utab,
user_tab_cols tcol
where utab.table_name = tcol.table_name
and utab.num_rows > 0
and utab.num_rows = tcol.num_nulls;
You could use a dynamic query to build the queries. This will generate all the queries.
SELECT 'SELECT COUNT(' || t.column_name || ' ) FROM ' || t.owner || '.' || t.table_name || ';' FROM dba_tab_columns t
You can generate all the select statements like so:
SELECT CASE WHEN column_id = 1 AND column_id_desc != 1 THEN 'SELECT ''' || LOWER(owner) || '.' || LOWER(table_name) || ''' table_name, ' || CHR(10) || 'COUNT(' || LOWER(column_name) || ') ' || SUBSTR(LOWER(column_name), 1, 26) || '_cnt,'
WHEN column_id = 1 AND column_id_desc = 1 THEN 'SELECT ''' || LOWER(owner) || '.' || LOWER(table_name) || ''' table_name, ' || CHR(10) || 'COUNT(' || LOWER(column_name) || ') ' || SUBSTR(LOWER(column_name), 1, 26) || '_cnt FROM ' || LOWER(owner) || '.' || LOWER(table_name) || ';'
WHEN column_id_desc = 1 THEN ' COUNT(' || LOWER(column_name) || ') ' || SUBSTR(LOWER(column_name), 1, 26) || '_cnt' || CHR(10) || 'FROM ' || LOWER(owner) || '.' || LOWER(table_name) || ';'
ELSE ' COUNT(' || LOWER(column_name) || ') ' || SUBSTR(LOWER(column_name), 1, 26) || '_cnt,'
END sql_text
FROM (SELECT owner,
table_name,
column_name,
column_id,
row_number() OVER (PARTITION BY owner, table_name ORDER BY column_id DESC) column_id_desc
FROM all_tab_columns)
WHERE <predicates to filter on the tables you're interested in>
ORDER BY owner,
table_name,
column_id;
This goes through all the tables you're interested in plus their columns and outputs text that will, when taken together, form a select statement for each table.
The text that is output in the sql_text column depends on whether the column in the list is the first or last (or both!); this way you get the full statement which queries each table once, rather than one per table and column.
You can then copy and paste the results and run that as a script.
It's can help you
SELECT
a.table_name,
a.column_name
FROM
ALL_TAB_COLUMNS a
WHERE owner = '<your user>'
AND a.SAMPLE_SIZE = a.NUM_NULLS

Oracle SQL: Alternative to aggregate large texts (when exceeding Listagg limit)

I have a simple Select query that aggregates one column containing large texts.
The following worked for me with small texts but I am now exceeding the Listagg character limit (4000 bytes ?).
I am very new to Oracle and couldn't find a proper solution for this online that I could apply here.
Can someone tell me the best alternative to this ?
My Query (simplified):
SELECT
m.S_ID AS SID
, LISTAGG
(
'ITEM NO.: ' || m.ITEM ||
' -nl-ARTICLE: ' || a.ARTICLE ||
' -nl-NET: ' || m.NET ||
' -nl-TAX: ' || NVL(m.TAX, 0) ||
' -nl-GROSS: ' || (m.NET + m.TAX),
' -nl--nl-'
) WITHIN GROUP (ORDER BY m.S_ID) AS Details
/* ... */
FROM
myTable m
/* ... */
Many thanks for any help with this,
Mike
One of possible method.
select xmlagg(xmlelement(xxx,'ITEM NO.: ' || m.ITEM ||
' -nl-ARTICLE: ' || a.ARTICLE ||
' -nl-NET: ' || m.NET ||
' -nl-TAX: ' || NVL(m.TAX, 0) ||
' -nl-GROSS: ' || (m.NET + m.TAX),
' -nl--nl-'||',<-separator').extract('//text()') order m.S_ID).getClobval() from mytable
group by ...
2nd method.
oracle allows to creat own aggregation function user defined aggregation function

Convert SQL query to pivot

I can't seem able to get the logic to convert the following query to a pivot SQL. My table has 20 columns with roles on them, I'd like to convert those columns into rows so, when exported to Excel, I can filter on a single column since the values can be the same on the 20 columns. So far what I've done is convert the 20 columns into a single one and then split that single one into rows:
select distinct TASKID,
regexp_substr(t.roles,'[^|]+', 1, lines.column_value) as role
from (
select TASKID,
TRIM(ROLE1) || '|' ||
TRIM(ROLE2) || '|' ||
TRIM(ROLE3) || '|' ||
TRIM(ROLE4) || '|' ||
TRIM(ROLE5) || '|' ||
TRIM(ROLE6) || '|' ||
TRIM(ROLE7) || '|' ||
TRIM(ROLE8) || '|' ||
TRIM(ROLE9) || '|' ||
TRIM(ROLE10) || '|' ||
TRIM(ROLE11) || '|' ||
TRIM(ROLE12) || '|' ||
TRIM(ROLE13) || '|' ||
TRIM(ROLE14) || '|' ||
TRIM(ROLE15) || '|' ||
TRIM(ROLE16) || '|' ||
TRIM(ROLE17) || '|' ||
TRIM(ROLE18) || '|' ||
TRIM(ROLE19) || '|' ||
TRIM(ROLE20) as roles
from menu_roles
where RLTYPE='58'
) t,
TABLE(CAST(MULTISET(select LEVEL from dual connect by instr(t.roles, '|', 1, LEVEL - 1) > 0) as sys.odciNumberList)) lines
where regexp_substr(t.roles,'[^|]+', 1, lines.column_value) is not null
order by regexp_substr(t.roles,'[^|]+', 1, lines.column_value)
I'd understand that using PIVOT would be more efficient vs concatenating and splitting a string.
Thank you!
You appear to want UNPIVOT:
SELECT task_id,
role
FROM menu_roles
UNPIVOT ( role FOR role_number IN ( ROLE1, ROLE2, ROLE3, ROLE4 /*, ... */ ) );
Or, using UNION ALL:
SELECT task_id, role1 AS role FROM menu_roles
UNION ALL SELECT task_id, role2 AS role FROM menu_roles
UNION ALL SELECT task_id, role3 AS role FROM menu_roles
UNION ALL SELECT task_id, role4 AS role FROM menu_roles
-- ...

SQL optimization with indexes

I have been trying hard to optimize below query using indexes in place of where clause. can any one help on the same. Data volume is very high so I want to optimize it using indexing or anyother way?
Query goes like this:
SELECT *
FROM
(SELECT
a.*,
rownum as row_num
FROM
(SELECT DISTINCT
lot.lot_no AS lotNo, lot.sel as sel,
lot.C_ARRIVAL_NOTICE_NO AS arrNoticeNo,
lot.C_SHIPMENT_DIRECTION AS shipmentDirection,
lot.C_SENDERS_REFERENCE_NUM AS externalReference,
lot.booking_seq AS bookingNo,lot.shipper AS shipperCode,
nvl(lot.shipper_name ,lot.shipper_addr1) AS shipperName,
NVL2 (lot.commdesc, lot.commdesc || ' ', '') || NVL2 (lot.comdesc1,lot. comdesc1 || ' ', '') || NVL2 (lot.comdesc2, lot.comdesc2 || ' ', '') || NVL2 (lot.comdesc3, lot.comdesc3 || ' ', '') || NVL2 (lot.comdesc4, lot.comdesc4 || ' ', '') || NVL2 (lot.comdesc5, lot.comdesc5 || ' ', '') || NVL2 (lot.comdesc6,lot. comdesc6 || ' ', '') || NVL2 (lot.comdesc7,lot. comdesc7 || ' ', '') || NVL2 (lot.comdesc8, lot.comdesc8 || ' ', '') || NVL2 (lot.comdesc9, lot.comdesc9, '') AS description,
lot.lot_qty AS pieces, lot.lot_wght AS weight,
lot.lot_cube AS cube,
to_char(lot.input_date, 'dd-Mon-YYYY') AS lotDate,
lot.e_m AS e_m, lot.warehouse AS warehouse,
to_char(lot.in_date , 'dd-Mon-YYYY') AS inDate,
lot.in_time AS inTime, lot.hold AS hold,
to_char(lot.sail_date, 'dd-Mon-YYYY') AS sailDate,
to_char(lot.sail_date, 'dd-Mon-YYYY') AS etdDate,
lot.container AS container,
lot.oml_bl_no AS billOfLadingNumber,
lot.reference AS fileNumber,
lot.inland_carrier AS trucker,
lot.pro_number AS proNumber,
lot.comments AS exceptions, lot.vessel AS vessel,
lot.cstatus AS status, lot.voyage AS voyage,
(SELECT count(*) FROM tra_shipment_status
WHERE c_reference = lot.lot_no AND i_status_code = '857940') as transmitcount,
(SELECT c_finalcfs_code FROM imp_bl_top
WHERE inv_no = lot.C_Arrival_Notice_No) as cfsCode,
'STI_COUNTRY_US' AS schemaName
FROM
itemdata lot
WHERE
lot.in_date BETWEEN TO_DATE('27-FEB-2017 00:00', 'DD-MON-YYYY HH24:MI')
AND TO_DATE('29-MAR-2017 23:59', 'DD-MON-YYYY HH24:MI')
AND lot.sel IS NOT NULL
AND (lot.C_ARRIVAL_NOTICE_NO IS NOT NULL OR
lot.C_SHIPMENT_DIRECTION ='IO')
AND lot.I_STATUS > 0 AND Lot.Cstatus = 'D'
ORDER BY
lot.lot_no desc, lot.in_date desc) a
WHERE
rownum <= 60)
WHERE
row_num >= 1