QlikView: Insert and update based on condition (without timestamp) - qlikview

I have encountered a problem when I am trying to write a script to one version of Insert and Update Incremental Load.
Example: To simplify the example I have made an illustration of how I want the data-set to update. (I leave the code for the discussion)
Illustration:
In the example above, you can see that I both want insert new records and to update records. The condition is that I only want to update records if the new value is greater than the existing record.
For instance, the existing record for ID 2 equals to 0 (Table 1) and since the new record for ID 2 equals to 100 (Table 2) I want to Update that record so the final updated record for ID 2 equals to the highest value of them two (Updated Table). If Table 2 contains a new record I just want to add that record to the final data-set.
Description:
Insert new records
Update records if the value is higher than the existing record
What do you guys think is the best solution for this kind of problem?

Not sure it's the best solution
//QVD
Table:
LOAD * INLINE [
ID, Value, Source
1, 500, 'QVD'
2, 0, 'QVD'
3, 100, 'QVD'
4, 300, 'QVD'
5, 0, 'QVD'
];
//ODBC
Concatenate(Table)
LOAD * INLINE [
ID, Value, Source
2, 100, ODBC
3, 700, ODBC
4, 300, ODBC
6, 500, ODBC
7, 0, ODBC
];
NewTable:
LOAD
ID,
max(Value) as Value
Resident Table
Group by ID
;
drop Table Table;

Related

How to add column with a certain type of number sequence of

There are lots of answers for adding a column to a table with some increment; e.g. start at 1 and increment by 1; e.g. 1, 2, 3 etc.
I need to add a column to tables that starts at 1, and increments by 1, but each number is repeated some number of times before the next number in the sequence.
For example: 100 rows of 1, then 100 rows of 2, etc. until the end of the table.
Options based on what you have said:
Create a view and generate the sequence number using row_number() over ()/100
Create a regular identity column and then create a computed column which is your identity column divided by 100 using integer division.
Create a regular int column and run a manual update on it using row_number() over ()/100.

Can I load only specific data from source file into HIVE table?

I have a pipe delimited text file with 400 values. Out of which I need to load only 40 values at positions[1, 2, 4, 5, 7, 8, 9, 15, 17, 18, 20...] into my Hive table, how can it be achieved?
By the book: create an EXTERNAL table to map your Text file, with 400 columns; create a managed table with 40 columns; then use SQL to INSERT INTO TABLE target SELECT col1, col2, col4, ..., col72 FROM wide_source
Actually, you don't need to map all 400 columns -- stop at the last column that you want to use in SQL and ignore the rest.

Explain Security in OpenERP 7.0 and What is the use of (6,0) and (4)?

I have seen this
eval="[(6, 0, ref('test_security.base_security_access)])]"
and
eval="[(4, [ref('test_security.base_security_access')])]"
in OpenERP 7.0 code.
What is the use of 6,0 and 4 in security and is there any other combination like this, please explain me.
(4, ID) means link to existing record with id = ID which will adds a relationship to existing record.
While (6, 0, [IDs]) means replace the list of linked IDs. First it will unlink/delete existing ids with that record and then link to existing record with each ID in the list of IDs.
For delete existing ids and link ids, it will delete the relationship between the two objects but does not delete the target object itself with (6, 0, [IDs])
For more details, visit here.
Finally I found the answer in ORM write method.
For a many2many field, a list of tuples is expected.
Here is the list of tuple that are accepted, with the corresponding semantics ::
(0, 0, { values }) link to a new record that needs to be created with the given values dictionary
(1, ID, { values }) update the linked record with id = ID (write *values* on it)
(2, ID) remove and delete the linked record with id = ID (calls unlink on ID, that will delete the object completely, and the link to it as well)
(3, ID) cut the link to the linked record with id = ID (delete the relationship between the two objects but does not delete the target object itself)
(4, ID) link to existing record with id = ID (adds a relationship)
(5) unlink all (like using (3,ID) for all linked records)
(6, 0, [IDs]) replace the list of linked IDs (like using (5) then (4,ID) for each ID in the list of IDs)
Example:
[(6, 0, [8, 5, 6, 4])] sets the many2many to ids [8, 5, 6, 4]
For a one2many field, a lits of tuples is expected.
Here is the list of tuple that are accepted, with the corresponding semantics ::
(0, 0, { values }) link to a new record that needs to be created with the given values dictionary
(1, ID, { values }) update the linked record with id = ID (write *values* on it)
(2, ID) remove and delete the linked record with id = ID (calls unlink on ID, that will delete the object completely, and the link to it as well)
Example:
[(0, 0, {'field_name':field_value_record1, ...}), (0, 0, {'field_name':field_value_record2, ...})]
For a many2one field, simply use the ID of target record, which must already exist, or False to remove the link.
For a reference field, use a string with the model name, a comma, and the target object id (example: 'product.product, 5')
A full list of options is in the documentation for the osv class.
(0, 0, { values }) link to a new record that needs to be created with
the given values dictionary
(1, ID, { values }) update the linked record with id = ID (write
values on it)
(2, ID) remove and delete the linked record with id = ID (calls unlink
on ID, that will delete the object completely, and the link to it as
well)
(3, ID) cut the link to the linked record with id = ID (delete the
relationship between the two objects but does not delete the target
object itself)
(4, ID) link to existing record with id = ID (adds a relationship)
(5) unlink all (like using (3,ID) for all linked records)
(6, 0, [IDs]) replace the list of linked IDs (like using (5) then
(4,ID) for each ID in the list of IDs)

Updating multiple rows in SQL

Using phpMyAdmin, I want to update multiple rows of data with some new information.
The 'uid' is unique to each record.
The 'pid' field is the same for all records.
I modified an INSERT INTO statment after I exported the data from another database.
UPDATE `tabme_name` SET (`uid`, `pid`, `title`, `category`) VALUES
(230952, 1902, 112, 27634),
(230953, 1902, 179, 27641),
(230954, 1902, 75, 27630),
(230955, 1902, 38, 27626);
I can only find information on updating one record, or updating records with the same information.
Could somebody show me a correct SQL Statment? I havn't got a WHERE clause in it!?!
Assuming the uid is the pk you must execute multiple updates:
UPDATE tabme_name SET pid = 1902, title =112, category=27634 WHERE uid=230952
UPDATE tabme_name SET pid = 1902, title =179, category=27641 WHERE uid=230953
...

postgresql: how to get primary keys of rows inserted with a bulk copy_from?

The goal is this: I have a set of values to go into table A, and a set of values to go into table B. The values going into B reference values in A (via a foreign key), so after inserting the A values I need to know how to reference them when inserting the B values. I need this to be as fast as possible.
I made the B values insert with a bulk copy from:
def bulk_insert_copyfrom(cursor, table_name, field_names, values):
if not values: return
print "bulk copy from prepare..."
str_vals = "\n".join("\t".join(adapt(val).getquoted() for val in cur_vals) for cur_vals in values)
strf = StringIO(str_vals)
print "bulk copy from execute..."
cursor.copy_from(strf, table_name, columns=tuple(field_names))
This was far faster than doing an INSERT VALUES ... RETURNING id query. I'd like to do the same for the A values, but I need to know the ids of the inserted rows.
Is there any way to execute a bulk copy from in this fashion, but to get the id field (primary key) of the rows that are inserted, such that I know which id associates with which value?
If not, what would the best way to accomplish my goal?
EDIT: Sample data on request:
a_val1 = [1, 2, 3]
a_val2 = [4, 5, 6]
a_vals = [a_val1, a_val2]
b_val1 = [a_val2, 5, 6, 7]
b_val2 = [a_val1, 100, 200, 300]
b_val3 = [a_val2, 9, 14, 6]
b_vals = [b_val1, b_val2, b_val3]
I want to insert the a_vals, then insert the b_vals, using foreign keys instead of references to the list objects.
Generate the IDs yourself.
BEGIN transaction
Lock table a
call nextval() - that's your first ID
generate your COPY with IDs in place
same for table b
call setval() with your final ID + 1
COMMIT transaction
At step 2 you probably want to lock the sequence's relation too. If code calls nextval() and stashes that ID somewhere it might be already in use by the time it uses it.
Slightly off-topic fact: there is a "cache" setting that you can set if you have lots of backends doing lots of inserts. That increments the counter in blocks.
http://www.postgresql.org/docs/9.1/static/sql-createsequence.html
Actually you can do it differently, what you need is:
Start transaction
Create temp table with same (or almost same) schema
COPY data to that temp table
Perform regullar INSERT INTO .. FROM temp_table ... RETURNING id, other_columns
Commit
taken from here (in c#, but algo is the same)