Optimize SELECT query for working with large database - sql

Question

This is a part of my database:
ID EmployeeID Status EffectiveDate
1 110545 Active 2011-08-01
2 110700 Active 2012-01-05
3 110060 Active 2012-01-05
4 110222 Active 2012-06-30
5 110545 Resigned 2012-07-01
6 110545 Active 2013-02-12
I want to generate records which select Active employees:
ID EmployeeID Status EffectiveDate
2 110700 Active 2012-01-05
3 110060 Active 2012-01-05
4 110222 Active 2012-06-30
So, I tried this query:
SELECT *
FROM Employee AS E
WHERE E.Status='Active' AND
E.EffectiveDate between'2011-08-01' and '2012-07-02'AND NOT
EXISTS(SELECT * FROM Employee AS E2
WHERE E2.EmployeeID = E.EmployeeID AND E2.Status = 'Resigned'
AND E2.EffectiveDate between '2011-08-01' and '2012-07-02'
);
It only works with small amount of data, but got timeout error with large database.
Can you help me optimize this?

Answer 1

This is how I read your request: You want to show active employees. For this to happen, you look at their latest entry, which is either 'Active' or 'Resigned'.
You want to restrict this to a certain time range. That probably means you want to find all employees that became active without becoming immediately inactive again within that time frame.
So, get the latest date per employee first, then stay with those rows in case they are active.
select *
from employee
where (employeeid, effectivedate) in
(
select employeeid, max(effectivedate)
from employee
where effectivedate between date '2011-08-01' and date '2012-07-02'
group by employeeid
)
and status = 'active'
order by employeeid;
The subquery tries to find a time range and then look at each employee to find their latest date within. I'd offer the DBMS this index:
create index idx on employee (effectivedate, employeeid);
The main query wants to find that row again by using employeeid and effectivedate and would then look up the status. The above index could be used again. We could even add the status in order to ease the lookup:
create index idx on employee (effectivedate, employeeid, status);
The DBMS may use this index or not. That's up to the DBMS to decide. I find it likely that it will, for it can be used for all steps in the execution of the query and even contains all columns the query works with, so the table itself wouldn't even have to be read.

Answer 2

I have tried to achieve the above result set using Case Statements.
Hope this helps.
CREATE TABLE employee_test
(rec NUMBER,
employee_id NUMBER,
status VARCHAR2(100),
effectivedate DATE);
INSERT INTO employee_test VALUES(1,110545,'Active',TO_DATE('01-08-2011','DD-MM-YYYY'));
INSERT INTO employee_test VALUES(2,110545,'Active',TO_DATE('05-01-2012','DD-MM-YYYY'));
INSERT INTO employee_test VALUES(3,110545,'Active',TO_DATE('05-01-2012','DD-MM-YYYY'));
INSERT INTO employee_test VALUES(4,110545,'Active',TO_DATE('30-06-2012','DD-MM-YYYY'));
INSERT INTO employee_test VALUES(5,110545,'Resigned',TO_DATE('01-07-2012','DD-MM-YYYY'));
INSERT INTO employee_test VALUES(6,110545,'Active',TO_DATE('12-02-2013','DD-MM-YYYY'));
COMMIT;
SELECT * FROM(
SELECT e.* ,
CASE WHEN (effectivedate BETWEEN TO_DATE('2011-08-01','YYYY-MM-DD') AND TO_DATE('2012-07-02','YYYY-MM-DD') AND status='Active')
THEN 'Y' ELSE 'N' END AS FLAG
FROM Employee_Test e)
WHERE Flag='Y'
;

Answer 3

I'm adding another answer with another interpretation of the request. Just in case :-)
The table shows statuses per employee. An employee can become active, then retired, then active again. But they can not become active and then active again, without becoming retired in between, of course.
We are looking at a time range and want to find all employees that became active but never retired within - no matter whether they became active again after retirement in that period.
This makes this easy. We are looking for employees, that have exactly one row in that time range and that row is active. One way to do this:
select employeeid, any_value(effectivedate), max(status)
from employee
where effectivedate between date '2011-08-01' and date '2012-07-02'
group by employeeid
having max(status) = 'Active'
order by employeeid;
As in my other answer, an appropriate index would be
create index idx on employee (effectivedate, employeeid, status);
as we want to look into the date range and look up the statuses per employee.