Situation looks like this:
We have product 'A123' and we have to remember lowest price for it.
Prices for one product comes from random number of shops and there is no way to tell when shop x will send us their price for 'A123'.
So I had SQL table with columns:
product_number
price
shop (from which shop this price comes)
An SQL function for updating product price looks like this (this is SQL pseudo-code, syntax doesn't matter):
function update_product(in_shop, in_product_number, in_price)
select price, shop into productRow from products where product_number = in_product_number;
if found then
if (productRow.price > in_price) or (productRow.price < in_price and productRow.shop = in_shop) then
update row with new price and new shop
end if;
else
insert new product that we didn't have before
end if;
the (productRow.price < in_price and productRow.shop = in_shop) condition is to prevent situation like this:
In products table we have
A123 22.5 amazon
then comes info from amazon again:
A123 25 amazon
Thanks to above condition we update price to higher which is correct behavior.
But algorithm fails in this situation: again we have a row in the products table:
A123 22.5 amazon
then comes info from merlin
A123 23 merlin (we don't update because price is higher)
then comes info from amazon
A123 35 amazon
and we update table and now we have:
A123 35 amazon
but this is wrong because merlin earlier has lower price for that product.
Any idea how to avoid this situation?
The only way you are going to solve your problem is keep track of the price per shop and then only return the lowest current price. So for example you would need a table like the one you already have, but when you select out of the table something like:
select min(price)
from products
where product_number = :my_product
Personally if it were me, I would keep a time stamp of when you receive the product price updates so you would be able to ascertain when you got the update.
To make this work you should maintain a table that contains the following:
Product
Supplier
LatestPrice
Then identify the current best supplier by querying that table - you can either do this when requested or when the table is updated either way you simplify the problem at the price of slightly more complex schema and queries
Additional (following comment):
Ok, this is going to mean that you need to store more data - but you don't have a huge amount of choice - the data is required to solve the problem so you either: a) have to update prices from all suppliers concurrently and then choose the best price from that snapshot or b) store the prices as you get them and pick the best price from the data you've got. The former implies a fairly hefty overhead in terms of fetching and processing data whereas the latter is basically a fairly modest storage problem and something any decent databases will cope with easily.
Basically, the problem is that you only store the lowest price from 1 vendor. You have to keep records of prices of all vendors, and use a selection query to select the minimum.
For example, If you have:
A123 22.5 Amazon
and you got:
A123 23 Merlin
You have to insert it, even if it is with higher price, because it's a different vendor. So you'll have:
A123 22.5 Amazon
A123 23 Merlin
When you get the new price from Amazon, for example: 25, you just update it. So you'll get:
A123 25 Amazon
A123 23 Merlin
then select the lowest price, Merlin, in this case.
Related
I have decent Google-Fu skills, but they've utterly failed me on this.
I'm working in PowerPivot and I'm trying to match up a product with a price point in another table. Sounds easy, right? Well each product has several price points, based on the source of the price, with a hierarchy of importance.
For Example:
Product 1 has three prices living in a Pricing Ledger:
Price 1 has an account # of A22011
Price 2 has an account # of B22011
Price 3 has an account # of C22011
Price A overrides Price B which overrides Price C
What I want to do is be able to pull the most relevant price (i.e, that with the highest rank in the hierarchy) when not all price points are being used.
I'd originally used a series of IF statements, but that's when there were only four points. We now have ten points, and that might grow, so the IF statements are an untenable solution.
I'd appreciate any help.
-Thanks-
I have the following use case that OpenRefine seems to be a good candidate to solve. I have an existing, "dirty" product table in my database that looks like this:
id name
51 Product A
52 product-a
53 product B
54 productb
55 produtc
56 productc
I have a new, "clean" product table that looks like this:
id name
1 Product A
2 Product B
3 Product C
I'd like to use OpenRefine's clustering to generate a mapping file, to help me map products from the old table to the new table:
id name old_id
1 Product A 51
1 Product A 52
2 Product B 53
2 Product B 54
3 Product C 55
3 Product C 56
But I can't quite get OpenRefine to do what I want. Any advice for how to achieve this?
As it was already pointed out, there is no direct way to achieve this, but with the help support tables and the cross function, you can get the desired result:
you take the column "name" from the dirty table and clean table, and combine them. Don't worry about the ids at this point.
import them into OpenRefine (e.g. as project "product names")
duplicate the column "name" (the only column so far) and name the new one "name_new".
Cluster the column "name_new" and replace all of the old names with the correct new ones. Some manual adjustments might be required at this point.
Your result should now look like this:
name name_new
Product A Product A
product-a Product A
product B Product B
productb Product B
produtc Product C
productc Product C
Product A Product A
Product B Product B
Product C Product C
import the dirty table as "products" and the clean table as "products clean".
in the project "products" transform the column "name" using
value.cross("product names","name").cells["name_new"].value[0]
rename the column "id" to "old_id"
add a new column based on "name" using
value.cross("products clean","name").cells["id"].value[0]
and save it as "id". The table "products" has now the desired structure.
I hope this helps.
Clustering function is limited to a single column to find similar strings within that column.
OpenRefine doesn't yet have string similarity functions across 2 or more tables or projects (Fuzzy Joins) in the way that your use case presents. You would have to use other tools for this. A common tool that I've seen folks use and express satisfaction with Fuzzy Joining is MS PowerBI (Desktop is Free but has limits on Relationships and Exporting, but Pro version is only $10 a month and canceling anytime) but if you wanted something completely free then a few R packages do this, one of which is https://www.rdocumentation.org/packages/fuzzyjoin/versions/0.1.4
In OpenRefine, we totally want to allow Fuzzy Joins across Projects/datasets in the future and it's on our issue list, but we just haven't had the funding to implement this along with tons of other features we know users would like to see.
I have 2 tables: Customers and Actions, where each customer has uniqe ID (which can be found in each table).
Part of the customers became club members at a specific date (change between the customers). I'm trying to summarize their purchases until that date, and to get those who purchase more than (for example) 200 until they become club members.
For example, I can have the following customer:
custID purchDate purchAmount
1 2015-05-12 100
1 2015-07-12 150
1 2015-12-29 320
Now, assume that custID=1 became a club member at 2015-12-25; in that case, I'd like to get SUM(purchAmount)=250 (pay attention that I'd like to get this customer because 250>200).
I tried the following:
SELECT cust.custID, SUM(purchAmount)totAmount
FROM customers cust
JOIN actions act
ON cust.custID=act.custID
WHERE act.clubMember=1
AND cust.purchDate<act.clubMemberDate
GROUP BY cust.custID
HAVING totAmount>200;
Is it the right way to "attack" this question, or should I use something like while loop over the clubMemberDate (which telling the truth-I don't know how to do)?
I'm working with Teradata.
Your help will be appreciated.
I work for an eCommerce company and we use Microsoft Excel for our inventory database. We currently just keep adding items to this database as we purchase them, without ever removing them. What I would like to do is start removing items as they sell. I am not sure how to attach the file, so if you e-mail me at drenollet#supplykick.com I can send it to you. Below are the following steps:
The Sales tab includes the sales data for the items. I would like to take this data and be able to copy and paste it in a sheet in our Inventory Managment file in excel (a separate file, but I included a sample in the "Database" sheet).
I then need to just use a VLOOKUP formula and the Catalog data to get the Product ID instead of the SKU. (I can do this.)
Then use the copied data in the Sales Tab that is in the Inventory Management file and move the corresponding rows out of the Database file/sheet to the Sold Items sheet.
A few thoughts on specifics:
I want to make sure all the quantities are right. (e.g.1 if we purchased two of an item and only one sold - reducing the quantity in the Database sheet from two down to one.) (e.g.2 If we purchased an item two different times at two different prices and both were purchased in one sale, I would want to make sure both of the rows are moved out of the database).
If you have any thoughts on making sure the quantities are right, let me know. Maybe we need to set all the purchase quantities to one and copy the purchase of a multiple quantity of items X number of times for each one that was purchased.
Would love your input on how to cross this bridge! Let me know if you would like to see the sample file and I can directly e-mail it to you!
Best Regards,
Don Renollet
The best way to do this is to have a sheet called Movements
then you have just rows of entries like
A B C D
----------------------------------------
prodID Movement type Qty Date
123 Purchase 5 08/01/15
789 Sale 2 07/01/15
123 Return 1 06/01/15
456 Sale 1 05/01/15
789 Purchase 10 04/01/15
456 Purchase 5 03/01/15
123 Sale 2 03/01/15
123 Return 1 02/01/15
123 Sale 1 02/01/15
123 Purchase 10 01/01/15
Then at anytime excel can calculate whats in stock using sumifs or similar
=SUMIFS(C:C,A:A,"123",B:B,"Purchase") - Sumif(C:C,A:A,"123",B:B,"Sale")) + Sumif(C:C,A:A,"123",B:B,"Return"))
You should never remove rows from a database like this, you can always do a stock take every so often and restart the database with 1 entry for each item, but aways store the old data elsewhere.
Try not to mix price with quantity if possible, if you need to manage price , consider using a moving average price (MAP)
The task here is to define the optimal (as detailed below) way of ordering items (parts) from suppliers.
The relevant parts of the table schema (with some sample data) are
Items
ID NUMBER
1 Item0001
2 Item0002
3 Item0003
Suppliers
ID NAME DELIVERY DISCOUNT
1 Supplier0001 0 0
2 Supplier0002 0 0.025
3 Supplier0003 20 0
DELIVERY is the delivery charge (in dollars) levied by that supplier on each delivery. DISCOUNT is the settlement discount (as a percentage i.e. 2.5% for ID=2 above) allowed by that supplier for on time payment.
SupplierItems
SUPPLIER_ID ITEM_ID PRICE
1 2 21.67
1 5 45.54
1 7 32.97
This is the many-to-many join between suppliers and items with the price that supplier charges for that item (in dollars). Every item has at least 1 supplier but some have more than one. A supplier may have no items.
PartsRequests
ID ITEM_ID QUANTITY LOCATION_ID ORDER_ID
1 59 4 2 (null)
2 89 5 2 (null)
3 42 4 2 (null)
This table is a request from a field site for parts to be ordered and delivered by the supplier to that site. A delivery of any number of items to a site attracts a delivery charge. When the parts are ordered, the ORDER_ID is inserted into the table so we are only concerned with those where ORDER_ID IS NULL
The question is, what is the optimal way to order these parts for each `LOCATION' where there are 3 optimal solutions that need to be presented to the user for selection.
The combination of orders with the least number of suppliers
The combination of orders with the lowest total cost i.e. The sum of QUANTITY*PRICE for each item plus the DELIVERY for each order summed over all orders ignoring DISCOUNT
As item 2 but accounting for DISCOUNT
Clearly I need to determine the combinations of orders that are available and then determining the optimal ones becomes trivial but I am a bit stuck on an efficient way to deal with building the combinations.
I have built some SQL fiddles in SQL Server 2008 with random data. This one has 100 items, 10 suppliers and 100 requests. This one has 1000 items, 50 suppliers and 250 requests. The table schema is the same.
Update
I reasoned that the solution had to be recursive and I built a nice table valued function to get but I ran into the 32 hard limit on recursion in SQL Server. I was uncomfortable with it anyway because it hinted more of a procedural language solution than a RDMS.
So I am now playing with CTE recursion.
The root query is:
SELECT DISTINCT
'' SOLUTION_ID
,LOCATION_ID
,SUPPLIER_ID
,(subquery I haven't quite worked out) SOLE_SUPPLIER
FROM PartsRequests pr
INNER JOIN
SupplierItems si ON pr.ITEM_ID=si.ITEM_ID
WHERE pr.ORDER_ID IS NULL
This gets all the suppliers that can supply the required items and is certainly a solution, probably not optimal. The subquery sets a flag if the supplier is the sole supplier of any product required for that location; if so they must be part of any solution.
The recursive part is to remove suppliers one by one by means of CTE.SUPPLIER_ID<>CTE.SUPPLIER_ID and add them if they still cover all the items. The SOLUTION_ID will be a CSV list of the suppliers removed, partly to uniquely identify each solution and partly to check against so I get combinations instead of permutations.
Still working on the details, the purpose of this update was to allow the Community to say "Yay, looks like that will work" or, alternatively "You moron, that won't work because ..."
Thanks
This is a more general answer (as in, not sql) as I think solving this problem will require something more powerful. Your first scenario is to select a minimum number of suppliers. This problem can be seen as a set cover problem as you are trying to cover all demands per site with the suppliers. This problem is already NP-complete.
Your third scenario seems to be basically the same as the second. You just have to take the discount into account in the prices, assuming you pay on time for every order.
The second scenario is at least NP-hard as I see a lot of resemblance with the facility location problem. You are trying to decide which suppliers (facilities) to use (open) to cover your orders (demands) based on their prices and delivery costs (opening costs).
Enumerating your possible solutions seems infeasible as with 10 suppliers, you have 2^10 possibilities of using them, further complicated by the distribution of demands internally.
I would suggest some dynamic programming to first select the suppliers that you have to use (=they are the only ones that deliver a specific thing), eliminating some possibilities (if the cost for supplier A +delivery cost A< cost for supplier B) and then trying to expand your set of possible solutions. Linear programming is also a valid train of thought.