Concerned with query size using non-unique join conditions - sql

Question

I have a situation at work. I work in housing. We raise orders to houses (so our contractors can go out and repair the houses).
Orders contain one or more jobs. A dwelling has zero, one or more orders raised against it.
This is a brief data definition. I've simplified the tables - but hopefully you get the idea. An order can contain many jobs, and a property can have many orders.
CREATE TABLE dwellings (
id VARCHAR2(10) PRIMARY KEY NOT NULL,
address VARCHAR2(100) NOT NULL
);
CREATE TABLE orders (
id VARCHAR2(10) PRIMARY KEY NOT NULL,
created_by VARCHAR2(10) NOT NULL,
created_on DATE NOT NULL,
dwelling_id VARCHAR2(10) NOT NULL REFERENCES dwellings(id)
);
CREATE TABLE jobs (
id VARCHAR2(10) PRIMARY KEY NOT NULL,
sor_id VARCHAR2(10) NOT NULL,
order_id VARCHAR2(10) NOT NULL REFERENCES orders(id)
);
And populated:
INSERT INTO dwellings VALUES ('00ABC', '2 The Mews House Little Boston London E1 1EE');
INSERT INTO dwellings VALUES ('5H88H', '3 Electric House Snodsbury S1 1IT');
INSERT INTO orders VALUES ('000001-A', 'CSMITH', DATE '2016-03-10', '00ABC');
INSERT INTO orders VALUES ('000002-A', 'CSMITH', DATE '2016-03-11', '00ABC');
INSERT INTO orders VALUES ('000003-A', 'AJONES', DATE '2016-03-16', '00ABC');
INSERT INTO orders VALUES ('000004-A', 'CSMITH', DATE '2016-03-16', '5H88H');
INSERT INTO jobs VALUES ('001', '000AA0', '000001-A');
INSERT INTO jobs VALUES ('002', '123BB0', '000001-A');
INSERT INTO jobs VALUES ('003', '000AA0', '000002-A');
INSERT INTO jobs VALUES ('004', '787XD7', '000003-A');
INSERT INTO jobs VALUES ('005', '000AA0', '000003-A');
INSERT INTO jobs VALUES ('006', '787XD7', '000004-A');
An analyst wants to know agents who are raising orders that are similar to previous orders. The thing under scrutiny is the SOR_ID, which denotes the type of job. Remember, there is one or more job associated with each order. So the task is: produce a report showing orders that contain one or more duplicate job types to previous orders at the property.
The report I'm building will have these column headings.
Agent Name
Order Id
Address
Previous Order Id
Duplicate Job Types
Here is the start of a query that gets there. I haven't executed it against the database because there are 50,000 properties and 100,000 orders and 200,000 jobs. I'm concerned about the size of the table because I'm joining on columns that are not unique.
select * from orders ord
join orders ord2 on ord.dwelling_id = ord2.dwelling_id --shaky
and ord.id <> ord2.id
and ord.created_on - ord2.created_on between 0 and 90
join jobs job on job.order_id = ord.id
join jobs job2 on job2.order_id = ord2.id
where job.sor_id = job2.sor_id
I'm looking for recommendations for how you might refactor this query into something more manageable (without PLSQL). Note that I haven't used LAG / LEAD and I haven't yet used LISTAGG to collapse the job type codes. That will come later. I'm concerned about how expensive the query is at the moment.

Answer 1

Query:
SELECT o.created_by AS agent_name,
d.address,
LISTAGG( o.id, ',' ) WITHIN GROUP ( ORDER BY o.created_on ) AS order_ids,
j.sor_id AS job_type
FROM dwellings d
INNER JOIN orders o
ON ( o.dwelling_id = d.id )
INNER JOIN jobs j
ON ( j.order_id = o.id )
GROUP BY o.created_by, d.address, j.sor_id
HAVING COUNT(1) > 1;
Output:
AGENT_NAME ADDRESS ORDER_IDS JOB_TYPE
---------- -------------------------------------------- ----------------- ----------
CSMITH 2 The Mews House Little Boston London E1 1EE 000001-A,000002-A 000AA0
Lists the jobs with the different order ids that were of the same type and placed by the same agent at the same address. The orders are listed in chronological order within the comma-separated list.
However, if you want it with your headings then you could do:
SELECT *
FROM (
SELECT o.created_by AS agent_name,
o.id,
d.address,
LAG( o.id ) OVER ( PARTITION BY o.created_by, d.address, j.sor_id
ORDER BY o.created_on
) AS previous_order_id,
j.sor_id AS job_type
FROM dwellings d
INNER JOIN orders o
ON ( o.dwelling_id = d.id )
INNER JOIN jobs j
ON ( j.order_id = o.id )
)
WHERE previous_order_id IS NOT NULL;
Which would output:
AGENT_NAME ID ADDRESS PREVIOUS_ORDER_ID JOB_TYPE
---------- ---------- -------------------------------------------- ----------------- ----------
CSMITH 000002-A 2 The Mews House Little Boston London E1 1EE 000001-A 000AA0
If you want to consider multiple agents then you can remove o.created_by from the GROUP BYor PARTITION BY clauses. For the top query you would then need to use LISTAGG to get all the agents. Like this:
SELECT LISTAGG( o.created_by, ',' ) WITHIN GROUP ( ORDER BY o.created_on ) AS agent_name,
d.address,
LISTAGG( o.id, ',' ) WITHIN GROUP ( ORDER BY o.created_on ) AS order_ids,
j.sor_id AS job_type
FROM dwellings d
INNER JOIN orders o
ON ( o.dwelling_id = d.id )
INNER JOIN jobs j
ON ( j.order_id = o.id )
GROUP BY d.address, j.sor_id
HAVING COUNT(1) > 1;
Or, for the second query, like this:
SELECT *
FROM (
SELECT o.created_by AS agent_name,
o.id,
d.address,
LAG( o.id ) OVER ( PARTITION BY d.address, j.sor_id
ORDER BY o.created_on
) AS previous_order_id,
j.sor_id AS job_type
FROM dwellings d
INNER JOIN orders o
ON ( o.dwelling_id = d.id )
INNER JOIN jobs j
ON ( j.order_id = o.id )
)
WHERE previous_order_id IS NOT NULL;
Both the queries would then also output the order with id 000003-A placed by AJONES.

Answer 2

Changes i would try out:
ord.id <> ord2.id : ord2.id < ord.id (not sure if that's applicable for you)
ord.created_on - ord2.created_on between 0 and 90 : ord2.created_on <= ord.created_on and ord2.created_on >= ord.created_on - 90 (not sure if the RDBMS can do that optimization)
Move job.sor_id = job2.sor_id into the ON clause (But the RDBMS will probably do that for you)
select * from orders ord
join orders ord2
on ord2.dwelling_id = ord.dwelling_id
and ord2.id < ord.id
and ord2.created_on <= ord.created_on
and ord2.created_on >= ord.created_on - 90
join jobs job on job.order_id = ord.id
join jobs job2
on job2.order_id = ord2.id
and job2.sor_id = job.sor_id;
Indexes you will need:
orders(dwelling_id, created_on, id)
jobs(order_id, sor_id)