Best index configuration for sql statement in sqlite? - sql

Question

I have the following compound sql statement for a lookup and I am trying to understand that are the optimal indexes (indices?) to create, and which ones I should leave out because they aren't needed or if it is counter productive to have multiple.
SELECT items.id, items.standard_part_number,
items.standard_price, items.quantity,
part_numbers.value, items.metadata,
items.image_file_name, items.updated_at
FROM items LEFT OUTER JOIN part_numbers ON items.id=part_numbers.item_id
AND part_numbers.account_id='#{account_id}'
WHERE items.standard_part_number LIKE '#{part_number}%'
UNION ALL
SELECT items.id, items.standard_part_number,
items.standard_price, items.quantity,
part_numbers.value, items.metadata,
items.image_file_name, items.updated_at
FROM items LEFT OUTER JOIN part_numbers ON items.id=part_numbers.item_id
AND part_numbers.account_id='#{account_id}'
WHERE part_numbers.value LIKE '#{part_number}%'
ORDER BY items.standard_part_number
LIMIT '#{limit}' OFFSET '#{offset}'
I have the following indices, some of them may not be necessary or could I be missing an index?... Or worse can having too many be working against the optimal performance configuration?
for items:
CREATE INDEX index_items_standard_part_number ON items (standard_part_number);
for part_numbers:
CREATE INDEX index_part_numbers_item_id ON part_numbers (item_id);
CREATE INDEX index_part_numbers_item_id_and_account_id on part_numbers (item_id,account_id);
CREATE INDEX index_part_numbers_item_id_and_account_id_and_value ON part_numbers (item_id,account_id,value);
CREATE INDEX index_part_numbers_item_id_and_value on part_numbers (item_id,value);
CREATE INDEX index_part_numbers_value on part_numbers (value);
Update:
The schema for the tables listed above
CREATE TABLE accounts (id INTEGER PRIMARY KEY,name TEXT,code TEXT UNIQUE,created_at INTEGER,updated_at INTEGER,company_id INTEGER,standard BOOLEAN,price_list_id INTEGER);
CREATE TABLE items (id INTEGER PRIMARY KEY,standard_part_number TEXT UNIQUE,standard_price INTEGER,part_number TEXT,price INTEGER,quantity INTEGER,unit_of_measure TEXT,metadata TEXT,image_file_name TEXT,created_at INTEGER,updated_at INTEGER,company_id INTEGER);
CREATE TABLE part_numbers (id INTEGER PRIMARY KEY,value TEXT,item_id INTEGER,account_id INTEGER,created_at INTEGER,updated_at INTEGER,company_id INTEGER,standard BOOLEAN);

Answer 1

Outer joins constrain the join order, so you should not use them unless necessary.
In the second subquery, the WHERE part_numbers.value LIKE ... clause would filter out any unmatched records anyway, so you should drop that LEFT OUTER.
SQLite can use at most one index per table per (sub)query.
So to be able to use the same index for both searching and sorting, both operations must use the same collation.
LIKE uses a case-insensitive collation, so the ORDER BY should be declared to use the same (ORDER BY items.standard_part_number COLLATE NOCASE).
This is not possible if the part numbers must be sorted case sensitively.
This is not needed if SQLite does not actually use the same index for both (check with EXPLAIN QUERY PLAN).
In the first subquery, there is no index that could be used for the items.standard_part_number LIKE '#{part_number}%' search.
You would need an index like this (NOCASE is needed for LIKE):
CREATE INDEX iii ON items(standard_part_number COLLATE NOCASE);
In the second subquery, SQLite is likely to use part_numbers as the outer table in the join because it has two filtered columns.
An index for these two searches must look like this (with NOCASE only for the second column):
CREATE INDEX ppp ON part_numbers(account_id, value COLLATE NOCASE);
With all these changes, the query and its EXPLAIN QUERY PLAN output look like this:
EXPLAIN QUERY PLAN
SELECT items.id, items.standard_part_number,
items.standard_price, items.quantity,
part_numbers.value, items.metadata,
items.image_file_name, items.updated_at
FROM items LEFT OUTER JOIN part_numbers ON items.id=part_numbers.item_id
AND part_numbers.account_id='#{account_id}'
WHERE items.standard_part_number LIKE '#{part_number}%'
UNION ALL
SELECT items.id, items.standard_part_number,
items.standard_price, items.quantity,
part_numbers.value, items.metadata,
items.image_file_name, items.updated_at
FROM items JOIN part_numbers ON items.id=part_numbers.item_id
AND part_numbers.account_id='#{account_id}'
WHERE part_numbers.value LIKE '#{part_number}%'
ORDER BY items.standard_part_number COLLATE NOCASE
LIMIT -1 OFFSET 0;
1|0|0|SEARCH TABLE items USING INDEX iii (standard_part_number>? AND standard_part_number<?)
1|1|1|SEARCH TABLE part_numbers USING COVERING INDEX index_part_numbers_item_id_and_account_id_and_value (item_id=? AND account_id=?)
2|0|1|SEARCH TABLE part_numbers USING INDEX ppp (account_id=? AND value>? AND value<?)
2|1|0|SEARCH TABLE items USING INTEGER PRIMARY KEY (rowid=?)
2|0|0|USE TEMP B-TREE FOR ORDER BY
0|0|0|COMPOUND SUBQUERIES 1 AND 2 (UNION ALL)
The second subquery cannot use an index for sorting because part_numbers is not the outer table in the join, but the speedup from looking up both account_id and value through an index is likely to be greater than the slowdown from doing an explicit sorting step.
For this query alone, you could drop all indexes not mentioned here.
If the part numbers can be searched case sensitively, you should remove all the COLLATE NOCASE stuff and replace the LIKE searches with a case-sensitive search (partnum BETWEEN 'abc' AND 'abcz').