Is it possible to store more than one table's data into a single QVD document?
If so, how can I both store and load multiple tables into/from this QVD?
Storing tables
You cannot store more than one distinct table into a QVD file. The reason for this, is that the QVD format only stores the memory "image" of a single table and as such does not include metadata for any links to other tables.
Therefore, if you wish to store multiple tables in a QVD file you must either:
Store them as separate QVDs
Combine them into a single table (e.g. via JOINs, applymap etc.) and then store the table into a QVD file.
For example, say you have the following schema:
Orders:
LOAD * INLINE [
Product, Country, Value
A, USA, 100
B, UK, 200
C, FR, 300
];
Products:
LOAD * INLINE [
Product, Weight, Colour
A, 10, Red
B, 50, Yellow
C, 70, Green
];
You cannot store this model into a single QVD file. Looking back to our two options:
Store as separate QVDs
You can change the script slightly to:
Orders:
LOAD * INLINE [
Product, Country, Value
A, USA, 100
B, UK, 200
C, FR, 300
];
Products:
LOAD * INLINE [
Product, Weight, Colour
A, 10, Red
B, 50, Yellow
C, 70, Green
];
STORE Orders INTO Orders.qvd (qvd);
STORE Products INTO Products.qvd (qvd);
Combine them into a single table
Depending on your data model, you can use JOINs and other QV functions:
Orders:
LOAD * INLINE [
Product, Country, Value
A, USA, 100
B, UK, 200
C, FR, 300
];
LEFT JOIN
LOAD * INLINE [
Product, Weight, Colour
A, 10, Red
B, 50, Yellow
C, 70, Green
];
STORE Orders INTO Orders.qvd (qvd);
Loading
To load data from your stored QVDs, you simply need to add a LOAD statement to your script, for example:
Orders:
LOAD
*
FROM Orders.qvd (qvd);
Products:
LOAD
*
FROM Products.qvd (qvd);
QlikView will then automatically infer the field links as per your original model, provided that the field names are the same as when you stored the tables.
If the two tables have no fieldnames in common just save it as a qvw and you can store both tables in the dashboard file. But it is proprietary and can only be opened by QlikView, but you seem to have the product already...
Related
I have a table with a large number of columns so I would like to have a filterpane/listbox with all the dimensions and measures allowing the user to select the columns they would like to see in the table.
My table looks like this where the columns are displayed based on the selection:
So something like this:
I tried to do it using this solution but it does not appear to do anything. Can someone tell me where I'm going wrong?/How can I do this?
Here is my script in the load editor where I create the table for the selection:
Selection:
load * inline [
Dimension
Category
Revenue
Year
ProductName
ProductID
Date
];
Here is my condition for each of the columns in the table, using column ProductID as an example:
=SubStringCount(Concat(Measure),1), ProductID)
You can try this:
Set vShowCollumn = Not IsNull(GetFieldSelections($1)) and WildMatch(Concat($2,','),'*$3*');
I_DIMENSIONS:
Load
*
InLine [
_Dimension , _CodDimension
Product , 001
Shop , 002
];
and in the product you choose de number, respondind with your inline, for example:
$(vShowcollumn(_Dimension,_CodDimension,001))
so, when it gonna show only when user click in the dimension.
I want to choose which to extract and show it to each column, I want to have result like this query, but don't want to type this for every single thing like this
Select *,
metrics::json ->> 'spend',
metrics::json ->> 'impressions',
metrics::json ->> 'clicks'
from t1
this show null, How to do if I choose to extract 'reach' and 'clicks',... to column but not all in the json?
select *
from json_to_record('{"reach": 240, "spend": 3.34, "clicks": 10, "frequency": 1.0375}')
as x(a int, b text, d text, e text)
I refer this Stack over flow question
My DEMO
EDIT: I have the main question is: how to choose which to extract
without extract all like the 2nd query? The data have many rows, each row have json, can I do that with Json_to_record ?
If you only want to select partial data from the JSON object, lets say 2 out of 4 keys, you can do so easily by omitting the rest of the keys from the anonymous table declaration. You need to use the JSON keys as column names.
select *
from json_to_record('{"reach": 240, "spend": 3.34, "clicks": 10, "frequency": 1.0375}')
as x(reach int, clicks int)
This allows you to get the columns you need with little writing effort.
https://dbfiddle.uk/?rdbms=postgres_11&fiddle=dd39d912f6e696a8ace3670acf606959
As a_horse_with_no_name said keep the same record names as these in the JSON and use aliases in the select list if needed.
select
reach as a, spend as b, clicks as c, frequency as d
from json_to_record('{"reach": 240, "spend": 3.34, "clicks": 10, "frequency": 1.0375}')
as x(reach integer, spend numeric, clicks integer, frequency numeric);
I've a column called "Description" in a table called "Food" which includes multiple food item names delimited by , such as chicken, soup, bread, coke
How can I extract each item from the column and create multiple rows.
e.g. Currently it's like
{FoodID, FoodName, Description} ==> {123, Meal, "chicken, soup, bread, coke"}
and what I need is
{FoodID, FoodName, Description} ==> {123, Meal, chicken},
{123, Meal, soup},
{123, Meal, bread} etc.
In Redshift, I first did a split of "description" column as
select FoodID, FoodName, Description,
SPLIT_PART(Description, ',',1) AS Item1,
SPLIT_PART(Description, ',',1) AS Item2,
SPLIT_PART(Description, ',',1) AS Item3,.....till Item10
FROM Food
consider that max of 10 items can be there and hence Item10.
What's the best method to convert these columns Item1 to Item10 to store as rows? I tried UNION ALL but it's taking a longer time considering huge load of data.
Your question is answered here in detail specifically for Redshift.
You just need to map your queries to example queries provided over there.
Your query will be something like below.
select (row_number() over (order by true))::int as n into numbers from food limit 100;
This will create numbers table.
Your query would become:
select foodId, foodName, split_part(Description,',',n) as descriptions from food cross join numbers where split_part(Description,',',n) is not null and split_part(Description,',',n) != '';
Now, coming to back to your original question about performance.
it's taking a longer time considering huge load of data.
Considering typical data warehouse use case of high read and seldom write, you should be keeping your typical food data that you have mentioned in stagging table, say stg_food.
You should use following kind of query to do one time insert into actual food table, something like below.
insert into food select foodId, foodName, split_part(Description,',',n) as descriptions from stg_food cross join numbers where split_part(Description,',',n) is not null and split_part(Description,',',n) != '';
This will write one time and make your select queries faster.
My model looks like so: Each Bottle has an attribute name, and a relationship to Brand.
In one of my views, I want to show a user all distinct bottles, and their counts.
A distinct bottle is a bottle that has the same name attribute, and the same Brand relationship.
So this table:
Should display 2 lines instead of 3, with the proper quantities (1 for Eitan, 2 for Almon).
The following line in my views.py:
object = Bottle.objects.filter(brand__business__owner_id=user.id).all().values('name').annotate(Count('brand'))
Produces this when I print object:
<QuerySet [{'name': 'Almon', 'brand__count': 2}, {'name': 'Eitan', 'brand__count': 1}]>
Which seems to be the right direction, but it has two problems:
I lose all other fields (vintage, capacity) except name and brand__count. I can of course explicitly add them to values, but that seems a) upythonic b) that it will group_by these items as well!
My pug template complains: Need 2 values to unpack in for loop; got 1 (this is because I'm iterating through them as a list, and using its index for numbering)
Any help is appreciated!
object = Bottle.objects.filter(brand__business__owner_id=user.id).all().values('name','vintage','capacity').annotate(Count('brand'))
unless you mention the fields to filter as you are mentioning name then how will the query set pass it to you? then do this, like not mentioning any name in the values
object = Bottle.objects.filter(brand__business__owner_id=user.id).all().values().annotate(Count('brand'))
both of this will give you all the fields in Bottle table
I have a columns of IDs with large volume of data and I want to retrieve Unique, duplicates etc in a single query that can display data in below format. Please suggest a query.
ID_Duplicates: a b c
ID_null : w, x, y, z
To search duplicate entries use:
Distinct keyword
Do some programming like:
if(duplicate){
}
else{
}