My model looks like so: Each Bottle has an attribute name, and a relationship to Brand.
In one of my views, I want to show a user all distinct bottles, and their counts.
A distinct bottle is a bottle that has the same name attribute, and the same Brand relationship.
So this table:
Should display 2 lines instead of 3, with the proper quantities (1 for Eitan, 2 for Almon).
The following line in my views.py:
object = Bottle.objects.filter(brand__business__owner_id=user.id).all().values('name').annotate(Count('brand'))
Produces this when I print object:
<QuerySet [{'name': 'Almon', 'brand__count': 2}, {'name': 'Eitan', 'brand__count': 1}]>
Which seems to be the right direction, but it has two problems:
I lose all other fields (vintage, capacity) except name and brand__count. I can of course explicitly add them to values, but that seems a) upythonic b) that it will group_by these items as well!
My pug template complains: Need 2 values to unpack in for loop; got 1 (this is because I'm iterating through them as a list, and using its index for numbering)
Any help is appreciated!
object = Bottle.objects.filter(brand__business__owner_id=user.id).all().values('name','vintage','capacity').annotate(Count('brand'))
unless you mention the fields to filter as you are mentioning name then how will the query set pass it to you? then do this, like not mentioning any name in the values
object = Bottle.objects.filter(brand__business__owner_id=user.id).all().values().annotate(Count('brand'))
both of this will give you all the fields in Bottle table
Related
I had one simple table ItemMapping, 2 Field, one Field is single Item List Field SingleSelectField With value "A",
Other for Multiple Item List Field MultiSelectField with Value ("B", "C", "D").
I Wanna get This mapping relationship By "B", I tried to set up one dataset, And try some single SuiteQL like before, But I always get empty results returned.
SELECT *
FROM ItemMapping
WHERE ItemMapping.MultiSelectField IN ('B')
Any tips may Help me.
Thank you in advance.
As was pointed out, Marty Zigman's article describes how Boban D. located an undocumented "feature" of SuiteQL which can be used.
I will leave most of the explaining to the article but to summarize, NetSuite automatically creates a relationship table named map_sourcTableId_fieldId which contain two columns: mapone and maptwo. mapone is the record id from the source table and maptwo is record id for the joined table.
This method seems to work well and maybe the most straight forward if you are accustomed to working in SQL.
As an alternative, I constructed a native SuiteScript Query object with a condition on a multiple select field. Then I used the toSuiteQL() method to convert it into SuiteQL to see how NetSuite natively deals with this. What I found was another undocumented "feature". The resulting query used a BUILTIN.MNFILTER function. So for example if you've got a custom transaction body field, custbody_link_type, that is a multiple select field and want to get transactions where one of te values in custbody_link_type is 4 then here is the generated SuiteQL:
SELECT T.tranid, T.custbody_link_types
FROM "transaction" as T
WHERE BUILTIN.MNFILTER(T.custbody_link_types , 'MN_INCLUDE', '', 'FALSE', NULL, 4) = 'T'
And if you want transactions where the custbody_link_types does not contain all of the following: 1, 2, 3 ...
SELECT T.tranid, T.custbody_link_types
FROM "transaction" as T
WHERE BUILTIN.MNFILTER(T.custbody_link_types , 'MN_EXCLUDE_ALL', '', 'FALSE', NULL, 1, 2, 3) = 'T'
OR T.custbody_link_types IS NULL
To wrap it up, the undocumented BUILTIN.MNFILTER function is used by NetSuite's query module to filter multiple select fields. It accepts the multiple select column, the internal string value of the query.Operator enum, some other stuff I don't know anything about, and finally one or more the values to compare. It appears to return a string of either 'T' for when the condition is met otherwise 'F'.
Ultimately, I'm not sure whether this is a "better" way to address the need but I thought it was worth documenting.
I'm extracting a table of 2000+ rows which are park details. One of the columns is JSON type. Image of the table
We have about 15 attributes like this and we also have a documentation of pre-determined codes assigned to each attribute.
Each row in the extracted table has a different set of attributes that you can see in the image. Right now, I have cast(parks.services AS text) AS "details" to get all the attributes for a particular park or extract just one of them using the code below:
CASE
WHEN cast(parks.services AS text) LIKE '%uncovered%' THEN '2'
WHEN cast(parks.services AS text) LIKE '%{covered%' THEN '1' END AS "details"
This time around, I need to extract these attributes by assigning them the codes. As an example, let's just say
Park 1 - {covered, handicap_access, elevator} to be {1,3,7}
Park 2 - {uncovered, always_open, handicap_access} to be {2,5,3}
I have thought of using subquery to pre-assign the codes, but I cannot wrap my head around JSON operators - in fact, I don't know how to extract them on 2000+ rows.
It would be helpful if someone could guide me in this topic. Thanks a lot!
You should really think about normalizing your tables. Don't store arrays. You should add a mapping table to map the parks and the attribute codes. This makes everything much easier and more performant.
step-by-step demo:db<>fiddle
SELECT
t.name,
array_agg(c.code ORDER BY elems.index) as codes -- 3
FROM mytable t,
unnest(attributes) WITH ORDINALITY as elems(value, index) -- 1
JOIN codes c ON c.name = elems.value -- 2
GROUP BY t.name
Extract the array elements into one record per element. Add the WITH ORDINALITY to save the original order.
Join your codes on the elements
Create code arrays. To ensure the correct order, you can use the index values created by the WITH ORDINALITY clause.
I need to build a DB for the following scenario:
I will have an input stream of auctions, and I want to make a price histogram for items on said auction (ie. what they usually go for etc).
The input stream looks something like:
{['item_id': 1, ... 'price': 123, ...],
['item_id': 1, ... 'price': 124, ... modifiers: [1, 2, 3],
['item_id': 1, ... 'price': 125, ... modifiers: [100, 150, 500...],
['item_id': 2, ... 'price': 200, ...],
...}
As you might have noticed, item doesn't only consist of some id, but also of modifiers. Think of it as a car that can be modified with extra stuff (e.g. AC, electronic windows etc).
What would be the most efficient way to store this information? Basically what I want to have is a unique id for each combination that can occur. It's not necessary to store it at all times, but if there is an auction for such an combination, and the combination doesn't exist yet, create it then.
I thought of something like:
base_item:
id
modifier:
id
item:
id (autonumber)
base_item_id
item_modifications:
item_id (FK item.id)
modification_id (FK modifier.id)
item_price_history:
item_id (FK item.id)
price
time
This setup might work. The problem is, imagine I have hundreds of millions of such auctions every day (ie. the auction's information is updated every 20 minutes and it cosists of 2 million auctions in average).
I want to be able to quickly do something like: INSERT INTO item_price_history VALUES (some_item_id, some_price, now()) but in order to do that, I need to find some_item_id. I know base_item_id and modifiers (from auction itself), but doing such call hundreds of millions of times is quite costly I think?
Ie, pseudo code:
for a in auctions:
base_item_id = a['item_id']
modifiers = a['modifiers']
price = a['price']
actual_item_id = some_query(base_item_id, modifiers) #expensive. Can be avoided?
insert_into_histogram(auctual_item_id, price) #expensive but necessary I think
Is there some obvious mistake I'm making in this design?
The schema you describe is the textbook solution.
But wow, that would be a beast to work with. As I understand it, every time you added a price record, you would have to find the item record with that exact set of parameters: no more, no less. And if no such item record existed, you would then have to create the item record. Only then could you add the price record.
While I think one should be very careful about denormalizing, I'd be sorely tempted to denormalize in this case. Namely, it seems to me that in practice, the key to an item record is the combination of the base item id plus the modifiers. I'd be tempted to to create a "modifier string" formed by stringing together codes or IDs for all the modifiers. Of course to be workable they'd have to be strung together in a defined sequence, like you can't have both "1,2" and "2,1". But then you could easily find the desired item record: just have a function that builds the concatenated modifier string, and select item where base_item_id=#base and modifiers=#modifiers. If not found, create the record and all the associated modifier records.
I'd be strongly inclined to make this modifier string be redundant with individual modifier records, but data that is strung together like this is very difficult to process. I mean, if you have a textbook schema like you describe, and someone wants to know prices for cars with air conditioning, it's very easy to select * from price where price.item in (select id from item join modifier on modifier.item_id=item.id where modifier.name='AC'). But try and do that on the concatenated string, say the ID for AC is "17". select blah blah where modifier_string like '%17%' doesn't work: it will find 117 and 171 and so on. like '%,17,%' doesn't work because it won't find it if it's the first or the last. Etc. That's why I routinely tell people NOT to string data together like this in general: create separate records. But if the most common use case is that you want the record with a specific combination of modifiers, creating a redundant modifier string is a plausible denormalization. (And the first time I typed that I accidentally typed 'demoralization', which may have been a Freudian slip.)
For example I want to filter my data based on next filter expression:
lead_veh_of_interest starts with 'BMW 1'
OR
sale_model starts with 'BMW 1'
how can I achieve this?
Are these fields both in the same table? If so you could create a calculated column that performs that conditional. Then filter the view on the resulting calculated column.
For example you could create the following calculated column.
Calculated Column: "Starts With BMW 1"
Equation:
=IF(OR(Left([lead_veh_of_interest starts], 5) = "BMW 1", Left([sale_Model], 5) = "BMW 1"), 1, 0)
Then in the view set the filter so that [Starts With BMW 1] = 1
I found acceptable solution (thanks for all suggestions - it was very helpful).
First of all I redesigned my model and (how #Mike_Honey suggsts me) created dedicated table with consolidated information I want to filter. Next I connected this new table with existing tables and created hierarchy from fields I want to give to the end users for step-wise filtering (previously I split down old fields contained information I want to filter into more granular level). Now it is possible to filter data by any combination of models in any combination of request types (sale, lead, competitor, etc) using hierarchy.
I have a set of data that shows users, collections of fruit they like, and home city:
Alice\tApple:Orange\tSacramento
Bob\tApple\tSan Diego
Charlie\tApple:Pineapple\tSacramento
I would like to create a pig query that correlates the number of users that enjoy tyeps of fruits in different cities, where the results from the query for the data above would look like this:
Apple\tSacramento\t2
Apple\tSan Diego\t1
Orange\tSacramento\t1
Pineapple\tSacramento\t1
The part I can't figure out is how to cross join the split fruit rows with the rest of the data from the same row, so:
Alice\tApple:Orange\tSacramento
becomes:
Alice\tApple\tSacramento
Alice\tOrange\tSacramento
I know I can use TOKENIZE to split the string 'Apple:Orange' into the tuple ('Apple', 'Orange'), but I don't know how to get the cross product of that tuple with the rest of the row ('Alice').
One brute-force solution I came up with is to use the streaming to run the input collection through an external program, and handle the "cross join" to produce multiple rows per row there.
This seems like it should be unnecessary though. Are there better ideas?
You should use FLATTEN, which works great with TOKENIZE to do stuff like this.
b = FOREACH a GENERATE name, FLATTEN(TOKENIZE(fruits)) as fruit, city;
FLATTEN takes a bag and "flattens" it out across different rows. TOKENIZE breaks your fruits out into a bag (not a tuple like you said), and then FLATTEN does the cross-like behavior like you are looking for. I point out that it is a bag and not a tuple, because FLATTEN is overloaded and behaves differently with tuples.
I first learned of the FLATTEN/TOKENIZE technique in the canonical word count example, in which is tokenizes a word, then flattens the words out into rows.