I am trying to update cell contents of a subset of columns in an SQLite Table in Python.
for i in range(len(list)):
if list[i] in other_list:
update_statement = "UPDATE Table \
SET Column_1 = Column_1 || ', ({}, {})', \
Item count = Item count + {}, \
Tot. = {}\
WHERE Column_4 = '{}';".format(part_1, part_2, \
df.iloc[i].at['Count'],\
2*len(other_list)+2, 'Text')
engine.execute(update_statement)
However, I am getting the following syntax error:
OperationalError: (sqlite3.OperationalError) near "count": syntax error
The error arises during the first iteration.
The statement in question is the following:
UPDATE Table
SET Column_1 = Column_1 || ', (part_1, part_2)', Item count = count + 2, Tot. = 6
WHERE Column_4 = 'Text';
Any help will be very appreciated!
Not a SqlLite expert but is Count a keyword in SqlLite? If so maybe alias the Count field. Try:
`UPDATE Table SET Column_1 = Column_1 || ', additional_text', theCount = theCount + 2 AS 'Count', Total = 6`
WHERE Column_4 = 'Text';
`
As pointed out by #forpas, the column name Item count was the problem as it contained a space. Re-defining the database's column name to Item_count fixed the syntax error.
Related
UPDATE PINPOINT_SUPPLEMENT
SET (ATTACHMENT_VALUE,ATTACHMENT_TYPE) = (
SELECT key,'file'
FROM PINPOINT_DOCUMENT
WHERE PINPOINT_SUPPLEMENT.ATTACHMENT_VALUE::integer = PINPOINT_DOCUMENT.DOCUMENT_ID
)
WHERE ATTACHMENT_VALUE IS NULL
Getting Error when i execute this query
ERROR: syntax error at or near "SELECT"
LINE 3: SELECT key,'file
update PINPOINT_SUPPLEMENT
set
ATTACHMENT_VALUE = PINPOINT_DOCUMENT.key,
ATTACHMENT_TYPE = 'file'
from PINPOINT_DOCUMENT
where
PINPOINT_SUPPLEMENT.ATTACHMENT_VALUE::integer = PINPOINT_DOCUMENT.DOCUMENT_ID
and PINPOINT_SUPPLEMENT.ATTACHMENT_VALUE IS NULL
or
update PINPOINT_SUPPLEMENT
set
(ATTACHMENT_VALUE,ATTACHMENT_TYPE) = (PINPOINT_DOCUMENT.key, 'file')
from PINPOINT_DOCUMENT
where
PINPOINT_SUPPLEMENT.ATTACHMENT_VALUE::integer = PINPOINT_DOCUMENT.DOCUMENT_ID
and PINPOINT_SUPPLEMENT.ATTACHMENT_VALUE IS NULL
Support for updating tuples was introduced in Postgres 9.5, so you can't use that syntax with your version.
But as the second value is a constant I don't see a reason to use that syntax to begin with:
UPDATE PINPOINT_SUPPLEMENT
SET ATTACHMENT_VALUE = (SELECT "key"
FROM PINPOINT_DOCUMENT
WHERE PINPOINT_SUPPLEMENT.ATTACHMENT_VALUE::integer = PINPOINT_DOCUMENT.DOCUMENT_ID),
ATTACHMENT_TYPE = 'file'
WHERE ATTACHMENT_VALUE IS NULL
Note, that the sub-query might result in an error if there is more than one ATTACHMENT_VALUE for a document_id!
I want to pivot a table based on a field which can contain "dynamic" values (not always known beforehand).
I can make it work by hard coding the values (which is undesirable):
SELECT *
FROM my_table
pivot(SUM(amount) FOR type_id IN (1,2,3,4,5,20,50,83,141,...));
But I can't make it work using a query to provide the values dynamically:
SELECT *
FROM my_table
pivot(SUM(amount) FOR type_id IN (SELECT id FROM types));
---
090150 (22000): Single-row subquery returns more than one row.
SELECT *
FROM my_table
pivot(SUM(amount) FOR type_id IN (SELECT ARRAY_AGG(id) FROM types));
---
001038 (22023): SQL compilation error:
Can not convert parameter 'my_table.type_id' of type [NUMBER(38,0)] into expected type [ARRAY]
Is there a way to accomplish this?
I don't think it's possible in native SQL, but I wrote an article and published some code showing how my team does this by generating the query from Python.
You can call the Python script directly, passing arguments similar to the options Excel gives you for pivot tables:
python generate_pivot_query.py \
--dbtype snowflake --database mydb \
--host myhost.url --port 5432 \
--user me --password myp4ssw0rd \
--base-columns customer_id \
--pivot-columns category \
--exclude-columns order_id \
--aggfunction-mappings amount=sum \
myschema orders
Or, if you're Airflow, you can use a CreatePivotTableOperator to create tasks directly.
I wrote a Snowflake stored procedure to get dynamics pivots inside Snowflake, check:
https://hoffa.medium.com/dynamic-pivots-in-sql-with-snowflake-c763933987c
3 steps:
Query
Call stored procedure call pivot_prev_results()
Find the results select * from table(result_scan(last_query_id(-2)))
The procedure:
create or replace procedure pivot_prev_results()
returns string
language javascript
execute as caller as
$$
var cols_query = `
select '\\''
|| listagg(distinct pivot_column, '\\',\\'') within group (order by pivot_column)
|| '\\''
from table(result_scan(last_query_id(-1)))
`;
var stmt1 = snowflake.createStatement({sqlText: cols_query});
var results1 = stmt1.execute();
results1.next();
var col_list = results1.getColumnValue(1);
pivot_query = `
select *
from (select * from table(result_scan(last_query_id(-2))))
pivot(max(pivot_value) for pivot_column in (${col_list}))
`
var stmt2 = snowflake.createStatement({sqlText: pivot_query});
stmt2.execute();
return `select * from table(result_scan('${stmt2.getQueryId()}'));\n select * from table(result_scan(last_query_id(-2)));`;
$$;
Inspired by my two predecessors, I created another stored proc that could be called to create even multi-grouped and multi-pivot emulated pivot queries:
create or replace procedure
test_pivot.public.get_full_pivot(
"source" varchar, // fully-qualified 'table/view_name' or full '(subquery)'
"row_headers" varchar, // comma-separated list of 1+ GROUP BY field names
"col_header1" varchar, // first (mandatory) PIVOT field name
"col_header2" varchar, // secondary (optional) PIVOT field name ('' if none)
"agg" varchar, // field name for the aggregate values
"aggf" varchar) // aggregate function (sum, avg, min, max, count...)
returns varchar
language javascript
as $$
// collect all distinct values for a column header field
function get_distinct_values(col_header) {
var vals = [];
if (col_header != '') {
var result = snowflake.execute(
{sqlText: `select distinct ${col_header}\n`
+ `from ${source}\n`
+ `order by ${col_header}`}); // nulls last!
while (result.next())
vals.push(result.getColumnValueAsString(1));
}
return vals;
}
var vals1 = get_distinct_values(col_header1);
var vals2 = get_distinct_values(col_header2);
// create and return the emulated pivot query, for one or two column header values
var query = `select ${row_headers}`;
if (vals2.length == 0)
for (const i in vals1) {
var cond1 = (vals1[i] == 'null'
? `${col_header1} is null` : `to_char(${col_header1})='${vals1[i]}'`);
query += `,\n ${aggf}(iff(${cond1}, ${agg}, null)) as "${vals1[i]}"`;
}
else
for (const i in vals1)
for (const j in vals2) {
var cond1 = (vals1[i] == 'null'
? `${col_header1} is null` : `to_char(${col_header1})='${vals1[i]}'`);
var cond2 = (vals2[j] == 'null'
? `${col_header2} is null` : `to_char(${col_header2})='${vals2[j]}'`);
query += `,\n ${aggf}(iff(${cond1} AND ${cond2}, ${agg}, null)) as "${vals1[i]}+${vals2[j]}"`;
}
query += `\nfrom ${source}\n`
+ `group by ${row_headers}\n`
+ `order by ${row_headers};`; // nulls last!
return query;
$$;
Call with:
call test_pivot.public.get_full_pivot(
'test_pivot.public.demographics',
'country, education', 'status', 'gender', 'sales', 'sum');
to generate the following SQL:
select country, education,
sum(iff(to_char(status)='divorced' AND to_char(gender)='F', sales, null)) as "divorced+F",
sum(iff(to_char(status)='divorced' AND to_char(gender)='M', sales, null)) as "divorced+M",
sum(iff(to_char(status)='married' AND to_char(gender)='F', sales, null)) as "married+F",
sum(iff(to_char(status)='married' AND to_char(gender)='M', sales, null)) as "married+M",
sum(iff(to_char(status)='single' AND to_char(gender)='F', sales, null)) as "single+F",
sum(iff(to_char(status)='single' AND to_char(gender)='M', sales, null)) as "single+M",
sum(iff(status is null AND to_char(gender)='F', sales, null)) as "null+F",
sum(iff(status is null AND to_char(gender)='M', sales, null)) as "null+M"
from test_pivot.public.demographics
group by country, education
order by country, education;
Which may return a result structured like:
I am using following script to generate objects in C# from Oracle and it works ok.
select 'yield return new Question { QuestionID = '||Q.questionid||', Qcode = "'||Q.Qcode||'", QuestionDescription = "'||Q.questiondescription||'", QuestionText = "'||Q.questiontext||'", QuestionCategoryId = '||Q.questioncategoryid||', QuestionTypeID = '||Q.QuestionTypeID||', IsThunderheadOnly = '|| case q.isthunderheadonly when 0 then 'false' else 'true' end ||', DisplayOrder = '||q.displayorder||' };'
from QUESTION q
where
questioncategoryid = 7
However again and again I run into the problem where I cannot || add columns with NULL values and solution to this point was adding those properties manually, which was ok when selecting up to 20 records.
Now I ran into a case of having to select hundreds of records and adding them manually would take substantial amount of time.
How could I modify the script to add (example) MaxValue property if column in the table is NOT NULL but skip it it if it is?
You can skip it with case ... when ... else like you figured out by yourself:
... ||case when A.NEXTQUESTIONID is not null then 'NextQuestionID = '||A.NEXTQUESTIONID||',' else '' end || ...
You can also use the nvl2 function for a shorter solution:
... || nvl2(A.NEXTQUESTIONID, 'NextQuestionID = '||A.NEXTQUESTIONID||',', '') || ...
I am workign with SQL Server 2008 and have to update one field (FIELD_NAM) which contains values of this format:
First value in the field: 'abc', 'efg', 'xyz'
Second value in the field: 'aaaaa', 'bbbb', 'vvvvvv'
I tried with the following statement:
UPDATE Table
SET FIELD = 'ttttt', 'kkkk', 'mmmmmm'
WHERE ID = 1
(only one row and one field/column has to be updated)
The error I got is
Incorrect syntax near 'ttttt'
update "table"
set field = '''ttttt'', ''kkkk'', ''mmmmmm'''
where id = 1
;
UPDATE Table
SET FIELD = 'ttttt\', \'kkkk\', \'mmmmmm'
WHERE ID = 1
OR
UPDATE Table
SET FIELD = 'ttttt'', ''kkkk'', ''mmmmmm'
WHERE ID = 1
Im working on a NHibernate criteria wich i graduatly builds upp depending on input parameters.
I got some problem with the postal section of these paramters.
Since we got a 5 number digit zipcodes the input parameter is a int, but since we in database also accept foreign zipcodes the database saves it as string.
What im trying to replicate in NHibernate Criteria/Criterion is the following where clause.
WHERE
11182 <=
(case when this_.SendInformation = 0 AND dbo.IsInteger(this_.Zipcode) = 1 then
CAST(REPLACE(this_.Zipcode, ' ', '') AS int)
when this_.SendInformation = 1 AND dbo.IsInteger(this_.WorkZipcode) = 1 then
CAST(REPLACE(this_.WorkZipcode, ' ', '') AS int)
when this_.SendInformation = 2 AND dbo.IsInteger(this_.InvoiceZipcode) = 1 then
CAST(REPLACE(this_.InvoiceZipcode, ' ', '') AS int)
else
NULL
end)
What we do is to check where the member contact (this_) has preferenced to get information sent to, then we check the input zipcode as integer against three different columns depending on if the column is convertable to int (IsInteger(expr) function) if column is not convertable we mark the side as NULL
in this case we just check if the zipcode is >= input parameter (reversed in sql code since paramter is first), the goal is to do a between (2 clauses wrapped with 'AND' statement), >= or <=.
UPDATE
Got a hint of success.
Projections.SqlProjection("(CASE when SendInformation = 0 AND dbo.IsInteger(Zipcode) = 1 then CAST(REPLACE(Zipcode, ' ', '') AS int) when SendInformation = 1 AND dbo.IsInteger(WorkZipcode) = 1 then CAST(REPLACE(WorkZipcode, ' ', '') AS int) when SendInformation = 2 AND dbo.IsInteger(InvoiceZipcode) = 1 then CAST(REPLACE(InvoiceZipcode, ' ', '') AS int) else NULL END)"
, new[] { "SendInformation", "Zipcode", "WorkZipcode", "InvoiceZipcode" },
new[] { NHibernateUtil.Int32, NHibernateUtil.String, NHibernateUtil.String, NHibernateUtil.String });
Throw my whole clause in a Projections.SqlProjection, however when i run my code some of my projection is cut (" AS int) else NULL END)" is cut from the end) and makes the sql corrupt.
Is there some kind of limit on this ?
Got it working yesterday.
Projections.SqlProjection worked, however if you don't name the projection as a column it some how cuts some of the TSQL code.
(Case
when x = 1 then 'bla'
when x = 2 then 'bla_bla'
else NULL
END) as foo
when using the last part (as foo) and naming the entire case syntax it works and dont cut anything.
However i dont know why but i could not manage to use the aliases from the other part of the criteria.