Why Postgresql using sec scan in my query - sql

Question

I have a problem with one sql query - it became run too long.
That problem had place one month ago, before everything was OK.
EXPLAIN tell me that Postgresql using Sec scan, well, problem is clear... But why it using sec scan? DB has indexes for all needed keys.
I have spend a lot of time to fix it, but all attemps have failed :(
Please, help!
There is SQL:
EXPLAIN ANALYZE SELECT
t.id,t.category_id,t.category_root_id,t.title,t.deadtime,
t.active,t.initial_cost,t.remote,t.created,t.work_type,
a.city_id,city.city_name_ru AS city_label, curr.short_name AS currency,
l1.val AS root_category_title, l2.val AS category_title, t.service_tasktotop,
t.service_taskcolor,t.service_tasktime, m.name AS metro,
count(tb.id) AS offers_cnt, t.contact_phone_cnt
FROM tasks AS t
LEFT JOIN tasks_address AS a ON ( a.task_id=t.id )
LEFT JOIN geo.cities AS city ON ( a.city_id=city.id_city )
LEFT JOIN catalog_categories AS r ON ( r.id=t.category_root_id )
LEFT JOIN catalog_categories AS c ON ( c.id=t.category_id)
LEFT JOIN localizations AS l1 ON (l1.lang='ru' AND l1.component='catalog' AND l1.subcomponent='categories' AND l1.var=r.name)
LEFT JOIN localizations AS l2 ON (l2.lang='ru' AND l2.component='catalog' AND l2.subcomponent='categories' AND l2.var=c.name)
LEFT JOIN tasks_bets AS tb ON ( tb.task_id=t.id )
LEFT JOIN paym.currencies AS curr ON ( t.currency_id=curr.id )
LEFT JOIN geo.metro AS m ON ( a.metro_id=m.id )
WHERE t.trust_level > 0
AND (a.region_id IN (1, 0) OR a.region_id IS NULL)
AND (a.country_id IN (1, 0) OR a.country_id IS NULL)
AND t.task_type=1
GROUP BY t.id,t.category_id,t.category_root_id,t.title,t.deadtime,t.active,t.initial_cost,t.remote,t.created,t.work_type, a.city_id, city.city_name_ru, curr.short_name, l1.val, l2.val, t.contact_phone_cnt, t.service_tasktotop,t.service_taskcolor,t.service_tasktime, m.name
ORDER BY
CASE
WHEN t.active=1 THEN
CASE
WHEN t.service_tasktotop > 1392025702 THEN 100
ELSE 150
END
WHEN t.active=2 THEN
CASE
WHEN t.service_tasktotop > 1392025702 THEN 200
ELSE 250
END
WHEN t.active=3 THEN 300
WHEN t.active=4 THEN 400
WHEN t.active=5 THEN 500
WHEN t.active=-1 THEN 600
WHEN t.active=-2 THEN 700
WHEN t.active=-3 THEN 800
WHEN t.active=-4 THEN 900
ELSE 1000
END,
CASE
WHEN t.service_tasktotop>1392025702 THEN t.service_tasktotop
ELSE t.created
END
DESC
LIMIT 30 OFFSET 0
There is EXPLAIN dump:
Limit (cost=17101.17..17101.24 rows=30 width=701) (actual time=248.486..248.497 rows=30 loops=1)
-> Sort (cost=17101.17..17156.12 rows=21982 width=701) (actual time=248.484..248.487 rows=30 loops=1)
Sort Key: (CASE WHEN (t.active = 1) THEN CASE WHEN (t.service_tasktotop > 1392025702) THEN 100 ELSE 150 END WHEN (t.active = 2) THEN CASE WHEN (t.service_tasktotop > 1392025702) THEN 200 ELSE 250 END WHEN (t.active = 3) THEN 300 WHEN (t.active = 4) THEN 400 WHEN (t.active = 5) THEN 500 WHEN (t.active = (-1)) THEN 600 WHEN (t.active = (-2)) THEN 700 WHEN (t.active = (-3)) THEN 800 WHEN (t.active = (-4)) THEN 900 ELSE 1000 END), (CASE WHEN (t.service_tasktotop > 1392025702) THEN t.service_tasktotop ELSE t.created END)
Sort Method: top-N heapsort Memory: 35kB
-> GroupAggregate (cost=14363.65..16451.94 rows=21982 width=701) (actual time=212.801..233.808 rows=6398 loops=1)
-> Sort (cost=14363.65..14418.61 rows=21982 width=701) (actual time=212.777..216.111 rows=18347 loops=1)
Sort Key: t.id, t.category_id, t.category_root_id, t.title, t.deadtime, t.active, t.initial_cost, t.remote, t.created, t.work_type, a.city_id, city.city_name_ru, curr.short_name, l1.val, l2.val, t.contact_phone_cnt, t.service_tasktotop, t.service_taskcolor, t.service_tasktime, m.name
Sort Method: quicksort Memory: 6388kB
-> Hash Left Join (cost=2392.33..5939.31 rows=21982 width=701) (actual time=18.986..64.598 rows=18347 loops=1)
Hash Cond: (a.metro_id = m.id)
-> Hash Left Join (cost=2384.20..5628.92 rows=21982 width=681) (actual time=18.866..57.534 rows=18347 loops=1)
Hash Cond: (t.currency_id = curr.id)
-> Hash Left Join (cost=2383.09..5325.56 rows=21982 width=678) (actual time=18.846..50.126 rows=18347 loops=1)
Hash Cond: (t.id = tb.task_id)
-> Hash Left Join (cost=809.08..2760.18 rows=5935 width=674) (actual time=10.987..32.460 rows=6398 loops=1)
Hash Cond: (a.city_id = city.id_city)
-> Hash Left Join (cost=219.25..2029.39 rows=5935 width=158) (actual time=2.883..20.952 rows=6398 loops=1)
Hash Cond: (t.category_root_id = r.id)
-> Hash Left Join (cost=203.42..1969.65 rows=5935 width=125) (actual time=2.719..18.048 rows=6398 loops=1)
Hash Cond: (t.category_id = c.id)
-> Hash Left Join (cost=187.60..1909.91 rows=5935 width=92) (actual time=2.522..15.021 rows=6398 loops=1)
Hash Cond: (t.id = a.task_id)
Filter: (((a.region_id = ANY ('{1,0}'::integer[])) OR (a.region_id IS NULL)) AND ((a.country_id = ANY ('{1,0}'::integer[])) OR (a.country_id IS NULL)))
-> Seq Scan on tasks t (cost=0.00..1548.06 rows=7337 width=84) (actual time=0.008..6.337 rows=7337 loops=1)
Filter: ((trust_level > 0) AND (task_type = 1))
-> Hash (cost=124.49..124.49 rows=5049 width=18) (actual time=2.505..2.505 rows=5040 loops=1)
Buckets: 1024 Batches: 1 Memory Usage: 256kB
-> Seq Scan on tasks_address a (cost=0.00..124.49 rows=5049 width=18) (actual time=0.002..1.201 rows=5040 loops=1)
-> Hash (cost=14.91..14.91 rows=73 width=37) (actual time=0.193..0.193 rows=74 loops=1)
Buckets: 1024 Batches: 1 Memory Usage: 5kB
-> Hash Left Join (cost=6.46..14.91 rows=73 width=37) (actual time=0.113..0.168 rows=74 loops=1)
Hash Cond: ((c.name)::text = (l2.var)::text)
-> Seq Scan on catalog_categories c (cost=0.00..7.73 rows=73 width=17) (actual time=0.001..0.017 rows=74 loops=1)
-> Hash (cost=5.42..5.42 rows=84 width=46) (actual time=0.105..0.105 rows=104 loops=1)
Buckets: 1024 Batches: 1 Memory Usage: 8kB
-> Seq Scan on localizations l2 (cost=0.00..5.42 rows=84 width=46) (actual time=0.005..0.056 rows=104 loops=1)
Filter: (((lang)::text = 'ru'::text) AND ((component)::text = 'catalog'::text) AND ((subcomponent)::text = 'categories'::text))
-> Hash (cost=14.91..14.91 rows=73 width=37) (actual time=0.155..0.155 rows=74 loops=1)
Buckets: 1024 Batches: 1 Memory Usage: 5kB
-> Hash Left Join (cost=6.46..14.91 rows=73 width=37) (actual time=0.085..0.130 rows=74 loops=1)
Hash Cond: ((r.name)::text = (l1.var)::text)
-> Seq Scan on catalog_categories r (cost=0.00..7.73 rows=73 width=17) (actual time=0.002..0.016 rows=74 loops=1)
-> Hash (cost=5.42..5.42 rows=84 width=46) (actual time=0.080..0.080 rows=104 loops=1)
Buckets: 1024 Batches: 1 Memory Usage: 8kB
-> Seq Scan on localizations l1 (cost=0.00..5.42 rows=84 width=46) (actual time=0.004..0.046 rows=104 loops=1)
Filter: (((lang)::text = 'ru'::text) AND ((component)::text = 'catalog'::text) AND ((subcomponent)::text = 'categories'::text))
-> Hash (cost=363.26..363.26 rows=18126 width=520) (actual time=8.093..8.093 rows=18126 loops=1)
Buckets: 2048 Batches: 1 Memory Usage: 882kB
-> Seq Scan on cities city (cost=0.00..363.26 rows=18126 width=520) (actual time=0.002..3.748 rows=18126 loops=1)
-> Hash (cost=1364.56..1364.56 rows=16756 width=8) (actual time=7.844..7.844 rows=16757 loops=1)
Buckets: 2048 Batches: 1 Memory Usage: 655kB
-> Seq Scan on tasks_bets tb (cost=0.00..1364.56 rows=16756 width=8) (actual time=0.005..4.180 rows=16757 loops=1)
-> Hash (cost=1.05..1.05 rows=5 width=9) (actual time=0.008..0.008 rows=5 loops=1)
Buckets: 1024 Batches: 1 Memory Usage: 1kB
-> Seq Scan on currencies curr (cost=0.00..1.05 rows=5 width=9) (actual time=0.003..0.005 rows=5 loops=1)
-> Hash (cost=5.28..5.28 rows=228 width=28) (actual time=0.112..0.112 rows=228 loops=1)
Buckets: 1024 Batches: 1 Memory Usage: 14kB
-> Seq Scan on metro m (cost=0.00..5.28 rows=228 width=28) (actual time=0.004..0.050 rows=228 loops=1)
Total runtime: 248.990 ms
Table:
id serial NOT NULL,
author_id integer DEFAULT 0,
category_id integer DEFAULT 0,
category_root_id integer DEFAULT 0,
title character varying,
description text,
deadtime integer DEFAULT 0,
helper_date integer DEFAULT 0,
active integer DEFAULT 1,
initial_cost integer DEFAULT 0,
conditional integer DEFAULT 0,
remote integer DEFAULT 0,
tariff_id integer DEFAULT 0,
created integer DEFAULT 0,
views integer DEFAULT 0,
accepted_helper_id integer DEFAULT 0,
accept_date integer DEFAULT 0,
auction_bet_id integer DEFAULT 0,
token character varying,
repute2helper text,
repute2author text,
active_dated integer DEFAULT 0,
bot integer,
repute_level integer,
seo_body text,
seo_body_active smallint DEFAULT 0,
service_tasktotop integer DEFAULT 0,
service_taskcolor integer DEFAULT 0,
service_tasktime integer DEFAULT 0,
type_id smallint DEFAULT 1,
partner_id integer NOT NULL DEFAULT 0,
trust_level smallint NOT NULL DEFAULT 1,
trust_date integer NOT NULL DEFAULT 0,
active_cause character varying(1500),
admin_notes text,
currency_id smallint NOT NULL DEFAULT 0,
work_type smallint NOT NULL DEFAULT 0,
helpers_gender smallint NOT NULL DEFAULT 0,
helpers_langs integer[],
service_notifyhelpers integer NOT NULL DEFAULT 0,
contact_phone character varying(50),
contact_email character varying(50),
fastclose smallint NOT NULL DEFAULT 0,
access_code character varying(250),
contact_phone_cnt integer NOT NULL DEFAULT 0,
author_in_task integer NOT NULL DEFAULT 0,
task_type smallint NOT NULL DEFAULT 1
Indexes:
CREATE INDEX tasks_author_in_task_idx
ON tasks
USING btree
(author_in_task );
CREATE INDEX tasks_deadtime_bot_created_active_dated_currency_id_idx
ON tasks
USING btree
(deadtime , bot , created , active_dated , currency_id );
CREATE INDEX tasks_idxs
ON tasks
USING btree
(id , active , category_id , category_root_id , remote , type_id , partner_id , trust_level );
CREATE INDEX tasks_service_tasktotop_service_taskcolor_service_tasktime_idx
ON tasks
USING btree
(service_tasktotop , service_taskcolor , service_tasktime );
CREATE INDEX tasks_task_type_idx
ON tasks
USING btree
(task_type );

Answer 1

i recommend to use explain.depesz.com, with this service it's much easier to see where is the problem.
Here http://explain.depesz.com/s/vHT is your explain, as you can see on stats tab seq scans are not a problem. Only 3.1 % of total runtime duration. On other hand sorting operations take a lot time (67%). Do you really need to sort by so many columns?
Sort Key: t.id, t.category_id, t.category_root_id, t.title, t.deadtime, t.active, t.initial_cost, t.remote, t.created, t.work_type, a.city_id, city.city_name_ru, curr.short_name, l1.val, l2.val, t.contact_phone_cnt, t.service_tasktotop, t.service_taskcolor, t.service_tasktime, m.name
last thing, do you have index on every column which is used with JOINS? look here for my simple example, i'm doing simple left join on table with table itself. First with indexed column, than without index. Look at plans (merge joins vs hash joins) and times. Remember that both table columns used in join should be in some index.
P.S. and always do ANALYZE on table, to be sure if planner has actual statistics!
Table "public.people"
Column | Type | Modifiers
------------+---------+-----------------------------------------------------
id | integer | not null default nextval('people_id_seq'::regclass)
username | text |
department | text |
salary | integer |
deleted | boolean | not null default false
Indexes:
"people_pkey" PRIMARY KEY, btree (id)
"people_department_idx" btree (department)
"people_department_salary_idx" btree (department, salary)
"people_salary_idx" btree (salary)
sebpa=# explain analyze select * from people a left join people b on a.id = b.id where a.salary < 30000;
QUERY PLAN
----------------------------------------------------------------------------------------------------------------------------------------
Merge Left Join (cost=0.57..2540.51 rows=19995 width=82) (actual time=0.022..19.710 rows=19995 loops=1)
Merge Cond: (a.id = b.id)
-> Index Scan using people_pkey on people a (cost=0.29..1145.29 rows=19995 width=41) (actual time=0.011..6.645 rows=19995 loops=1)
Filter: (salary < 30000)
Rows Removed by Filter: 10005
-> Index Scan using people_pkey on people b (cost=0.29..1070.29 rows=30000 width=41) (actual time=0.008..3.769 rows=19996 loops=1)
Total runtime: 20.969 ms
(7 rows)
sebpa=# alter table people drop constraint people_pkey;
ALTER TABLE
sebpa=# vacuum analyze people;
VACUUM
sebpa=# explain analyze select * from people a left join people b on a.id = b.id where a.salary < 30000;
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------
Hash Right Join (cost=1081.94..2829.39 rows=19995 width=82) (actual time=10.767..47.147 rows=19995 loops=1)
Hash Cond: (b.id = a.id)
-> Seq Scan on people b (cost=0.00..581.00 rows=30000 width=41) (actual time=0.001..2.989 rows=30000 loops=1)
-> Hash (cost=656.00..656.00 rows=19995 width=41) (actual time=10.753..10.753 rows=19995 loops=1)
Buckets: 2048 Batches: 2 Memory Usage: 733kB
-> Seq Scan on people a (cost=0.00..656.00 rows=19995 width=41) (actual time=0.007..5.827 rows=19995 loops=1)
Filter: (salary < 30000)
Rows Removed by Filter: 10005
Total runtime: 48.884 ms

Answer 2

OK - so you've run an explain on a complicated query and seen a seq-scan then jumped to conclusions.
Explain output can be tricky to read on a small screen, but there's a nice chap who's built a tool for us. Let's post it to explain.depesz.com
http://explain.depesz.com/s/DTCz
This shows you nicely coloured output. Those sequential scans? Take only milliseconds.
The big time consumer seems to be that sort (151ms by itself). It's sorting 18,000 rows by a lot of fields and uses about 6.4MB of memory to do so.
There's nothing worth concentrating on apart from this sort. There are only three plausible options:
Make sure your work_mem is > 6.4MB for this query (set work_mem=...)
Add an index that matches the fields you want to sort by (might work/might not, but it will be a big index that's expensive to update).
Rewrite the query - use a subquery to filter + group your tasks then join to the other tables. Difficult to say if/how much it will help.
Start with #1 - that'll only take a few minutes to test and is a likely candidate if the query used to be quicker.