Given the OrderLine table below:
OrderID OrderLineID
======= ===========
1 1
1 2
2 3
3 4
1 5
3 6
... ...
... ...
221 123 365 282
What is the most efficient way to find the FIRST OrderLine for each order, given that this information is required to access every now and then by the user?
This is my SQL to find the first OrderLine, but it takes about 3~5 seconds to execute every-time. (about 300k rows)
SELECT OrderID, MIN(OrderLineID)
FROM OrderLine
GROUP BY OrderID
It's very expensive to repeat this every-time when I need to find the first orderline to join with another table. Consider that changing the table structure is not an option, what possible solution do I have to improve this?
Try adding an index by OrderID and OrderLineID.
(You say you can't change the table structure. If you were allowed to change the structure, you could add a flag that identifies the first line of every order, then index by that flag.)
Related
I have relational category & product tables. Categories are hierarchical. I will have queries based on category, for example
select *
from products
where CatId = 3
or
select *
from products
where CatId = 1
I have 6 level of category and 24 million row for products, I have to find fast and optimal solutions. My question is which structure is suitable.
I write some options, feel free to suggest a better alternative.
Current category table:
Id ParentId Name
---------------------
1 null CatA
2 null CatB
3 1 CatAa
4 2 CatBa
Product table option 1
Id Cat Name
------------------
1 3 Product_1
2 4 Product_2
Product table option 2
Id CatLevel1 CatLevel2 ... Name
-------------------------------------
1 1 3 . Product_1
2 2 4 . Product_2
Product table option 3
Id Cats Name
------------------
1 1:3 Product_1
2 2:4 Product_2
Always keep option one, plus some denormalised tables (options two onwards) if you so desire. By keeping option one, you have the source truth to revert to or derive the others from.
Option two is only recommended if the searcher always knows what depth/level to search at. For example, if they know they need Level2=CATAb then it works, but if they don't know CATAb is at level two, they don't know which column to look in. It also relies on knowing how many levels to represent; if you can have a hundred levels, you need a hundred columns, and it's fragile of you need to add more depths. Generally, this doesn't apply and so is generally not a good optimisation.
Option three is a straight no. Never store multiple values in a one field (one column of one row). It will make Efficient searching of that column next to impossible.
The alternative to option three is to have a "link" table. Just two columns, category_id and product_id. Then you list all ancestors of a product, just on different rows.
category_id
product_id
1
1
3
1
2
2
4
2
These are all known as adjacency lists. A different model altogether is Nested Sets. I'm on my phone, and it's hard to describe without lots of formatting, but if you research online you'll find lots of information. They're much harder to comprehend and implement Initially, but very fast at retrieval when specifying a parent.
Your product table option 1 is fine and need no change
product_id,
category_id,
... other attributes
Your problem is in accessing the product based on the category hierarchy - which would make a need of a hierarchical query to get all categories in the tree below your selected category.
Instead of
select * from product where category_id = 1;
you'll need to write an additional hierarchical query to get the whole hierarchy tree
with cat_tree (id) as (
select id
from category where id = 1
UNION ALL
select ca.id
from cat_tree ct
join category ca
on ct.id = ca.parent_id
)
select * from product
where category_id in
(select id from cat_tree);
Which may not be practicable, but you may simplify it by denormalizing the category table
Let's assume your category data is such as
ID PARENT_ID
---------- ----------
1
3 1
5 3
6 3
The query below, which may be implemented as a MATERIALIZED VIEW that is refreshed on each category change pre-calculates all direct and indirect parent and child relations.
The result is
ID CHILD_ID
---------- ----------
1 1
1 3
1 5
1 6
3 3
3 5
3 6
5 5
6 6
E.g. for 1 you get itself, all its child's, their child's etc.
Using this category_denormobject your query can be simplified to
select *
from product
where category_id in
(select child_id from category_denorm where id = 1);
With the following table (assuming it has many other rows and columns), how could I query it while removing duplicates?
order_id
customer_name
amount
bill_type
1
Chris
10
sale
1
Chris
1
tip
1
Chris
10
sale
Note that while all 3 rows are about the same order, only row 3 is a duplicate -- since row 2 tells us about the tips of that order.
Using distinct order_id would remove rows 2 and 3, while I am looking to only remove row 3.
Appreciate any ideas
If you want a new result set, you can use:
select distinct t.*
from t;
I would suggest saving this into a new table, if you need to materialize the result.
I have a table which has a simple parent child structure
products:
- id
- product_id
- time_created
- ... a few other columns
It is a parent if product_id IS NULL. Product id behaves here like parent_id. Data inside looks like this:
id | product_id
1 NULL
2 1
3 1
4 NULL
4 4
This table is updated every night a new versions are added.
Every user is using a lot of these products but only one version. User is notified if new rows are added for an product_id.
He can stop using id:2 and start using id:3. An another user will continue using id:2 etc.
products table is updated every night and it grows pretty fast. There are around 500000 rows at the moment and every night adds around 20000, probably 5-7000000 changes (new rows) per year.
Is there a way to optimize this database/table structure? Should I change anything? Is it a problem to have so much data in one table?
Your question is not clear. The sample data is suggesting that the parent-child relationship is only one level deep. If so, this is not a particularly hard problem. You can create a query to look up the most recent product id for each product -- and I'm assuming this is the one with the maximum id:
select id, product_id,
max(id) over (partition by coalsesce(product_id, id)) as biggest_id
from table t;
This is then a lookup table, to get the biggest id. It would produce:
id | product_id | biggest_id
1 NULL 3
2 1 3
3 1 3
4 NULL 4
4 4 4
If your table has deeper hierarchies, you can solve the problem using recursive CTEs, or by doing the calculation when the table is updated.
I have three tables: store, product, storeproduct.
It doesn't really matter what's in the store and the product table, just know there is a storeID in the store table, and a productID in the product table. However the storeproduct table keeps track of the different products each store has. So the storeproduct table has two columns. The storeID column, and the productID column, both foreign keys from the store and the product table.
Is there a way to put a constraint or check on any of the table to make sure that a store must have more than 0 products, and less than 50 products.
Note: I do not want a select statement to do this. I just want to know if there is a way to put a constraint or a check when creating the tables.
The point of this is so a user cannot insert into the storeproduct table if there are already 50 products(rows) with the same storeID, or delete from the storeproduct table if deleting a row will cause the last row with that storeID to be gone.
The storeproduct table might look like this
storeID productID
1 1
1 2
1 3
2 4
2 5
2 6
2 7
3 4
3 2
3 6
3 1
3 8
Actually, depending on your database you may be able to do this.
Oracle (and maybe others) provide materialized views which you can apply constraints to. So you could create the MV with a column PRODUCTS_IN_STORES (being something like select storeID, count(*) as PRODUCTS_IN_STORES from stores left outer join storeproduct on store.storeid=storeproduct.storeid group by store.storeid .Then put a constraint on it asserting that PRODUCTS_IN_STORES is between 0 and 50 or whatever.
http://www.sqlsnippets.com/en/topic-12896.html
and
http://asktom.oracle.com/pls/asktom/f?p=100:11:0::::P11_QUESTION_ID:21389386132607
Not a complete answer for you, but something to think about and hopefully set you on your way.
I am working with an order system that has two tables Order and OrderLine pretty standard stuff. I want to work out an order line number for the order lines with respect to the order e.g.
Orderid Orderlineid linenumber
1 1 1
2 2 1
2 3 2
3 4 1
4 5 1
4 6 2
The OrderLineId is an identity column. I don't want to store the line number as data in the database for two reasons. First there are already a great many existing orders and lines in the system and retrospectively adding the data is a headache I wish to avoid. Second if the user deletes a line then I would need to recalculate the line numbers for the whole order.
In SQL 2005 I can do this easy peasy using the ROW_NUMBER function.
Select Orderid, OrderLineid, ROW_NUMBER()
OVER(PARTITION BY Orderid ORDER BY Orderlineid) as LineNumber
FROM OrderLine
Is there anyway I can do this in SQL 2000?
The closest I found was a ranking function (see below) but this counts orders not lines.
SELECT x.Ranking, x.OrderId
FROM (SELECT (SELECT COUNT( DISTINCT t1.Orderid) FROM orderline t1 WHERE z.Orderid >= t1.Orderid)AS Ranking, z.orderid
FROM orderline z ) x
ORDER BY x.Ranking
You can use something like this:
select
ol1.orderId,
ol1.orderLineId,
count(*) as lineNumber
from
orderLine ol1
inner join orderLine ol2
on ol1.orderId = ol2.orderId
and ol1.orderLineId >= ol2.orderLineId
group by
ol1.orderId,
ol1.orderLineId
IMHO, calculating it every time you need it may end up being more of a headache than it's worth. And while it may be a pain to update the historical line numbers and save it to the database, you'd only be doing that once... and then would just need to write code to automate it later.
To handle the deletes, you'd just need to add code into the existing delete procedure and have it recalculate the line numbers for it.
An alternative would be to insert your data (without line number) into a #temp table where you add a column with identity(1,1) and then selects everything from the #temp table.