Related
My current SQL:
SELECT B.MESSAGENO, B.LINENO, B.LINEDATA
FROM BILL.MESSAGE AS B, BILL.ACTIVITYAS A
WHERE A.MSGNO = D.MESSAGENO AND A.FUPTEAM = 'DBWB'
AND A.ACTIVITY = 'STOPPAY' AND A.STATUS = 'WAIT'
AND A.COMPANY = D.COMPANY
MESSAGENO LINENO LINEDATA
1234567 1 CHEQUE NO : 9999999 RUN NO : 55555
1234567 2 DATE ISSUED: 12/25/2020 AMOUNT : 710.51
1234567 3 PAYEE : LASTNAME, FIRSTNAME
1234567 4 ACCOUNT NO : 12345-67890
1234567 5 USER : USERNAME
there are 550 sets of 5 lines per MESSAGENO
What I am trying to figure out is how I can get something like where LINENO = 1, concatenate LINEDATA so I just get 9999999 as checkno, where LINENO = 2, concatenate LINEDATA so I get 710.51 as amount, where LINENO = 3, concatenate LINEDATA so I get LASTNAME, FIRSTNAME as payee, where LINENO = 4, concatenate LINEDATA so I get LASTNAME, FIRSTNAME as payee, and lastly, the same thing for USERNAME.
I just cannot seems to conceptualize this. Every time I try, my brain starts turning into macaroni. Any help is appreciated.
UPDATED ANSWER, extracts all fields from stored strings:
https://dbfiddle.uk/?rdbms=mysql_8.0&fiddle=e22d26866053ea6088aa78dc23c4d809
Check this fiddle.
It uses a SUBSTRING_INDEX in the internal query to split the fields at the : or by a combination of : and " ". I used the two spaces because I wasn't sure what your actual whitespace was, and when I copied the data out of your post it was just spaces.
Then MAX is used in the outer query to get everything on one line, grouping by the messageNo. Since some lines have two pieces of data to extract, a second string parser was added. Here is the code from the fiddle, for reference. In this case, the table test was created from the output of OP's initial query, since I didn't have enough info to build both tables and completely recreate his output.
SELECT MESSAGENO,
MAX(if(LINENO = 1, extractFirst, null)) as checkNo,
MAX(if(LINENO = 1, extractLast, null)) as runNo,
MAX(if(LINENO = 2, extractFirst, null)) as issued,
MAX(if(LINENO = 2, extractLast, null)) as amount,
MAX(if(LINENO = 3, extractFirst, null)) as payee,
MAX(if(LINENO = 4, extractFirst, null)) as accountNo,
MAX(if(LINENO = 5, extractFirst, null)) as username
FROM (
SELECT MESSAGENO, LINENO,
trim(substring_index(substring_index(LINEDATA, ": ", -2), " ", 1)) as extractFirst,
trim(substring_index(LINEDATA, ":", -1)) as extractLast
FROM test
) z
GROUP BY MESSAGENO
Again, you will be much better off to alter your tables so that you can use simpler queries, as shown in the last fiddle.
===============================================
ORIGINAL ANSWER (demo of string parsing, suggestion for data model change)
You can achieve this with some string parsing BUT ABSOLUTELY DO NOT unless you have no choice. The reason you are having trouble is because your data shouldn't be stored this way.
I've included a fiddle incorporating this case statement and substring_index to extract the data. I have assumed mySQL 8 because you didn't specify SQL version.
https://dbfiddle.uk/?rdbms=mysql_8.0&fiddle=068a49a2c819c08018691e54bcdf91e5
case when LINENO = 1 then trim(substring_index(substring_index(LINEDATA, "RUN NO", 1), ":", -1))
else trim(substring_index(LINEDATA, ":", -1)) end
as LDATA
See this fiddle for the full statement. I have just inserted the data from your join into a test table, instead of trying to recreate all your tables, since I don't have access to all the data I would need for that. In future, set up a fiddle like this one with some representative data and the SQL version, and it will be easier for people to help you.
=========================================
I think this is a better layout for you, with all data stored as the proper type and a field defined for each one and the extra text stripped out:
https://dbfiddle.uk/?rdbms=mysql_8.0&fiddle=11d52189b005740cdc53175466374635
I have a massive database and and need a query to update different tables in the database. I believe this should be easy since the column I'm changing is the same in every table. This is what I have so far;
UPDATE a_assets,
client_notes,
client_pending,
client_task,
country,
document_log,
favlists,
favourites,
g_address,
g_climst,
g_dialog,
g_lang,
g_prdmst,
g_secure,
j_alloc,
logger,
passhistory,
portfolios,
prod_metrics_tank,
product_usage_lists,
region,
reasearch_logger,
search_dataphile,
search_esg,
search_rpm,
sql_workout,
universe_source,
user_jurisdications,
user_languages,
user_universe,
work_group_mappings,
work_groups,
spt_docs
set a_assets.planner = ? ,
client_notes.planner = ?,
client_pending.planner = ?,
client_task.planner = ?,
country.planner = ?,
document_log.planner=?,
favlists.planner=?,
favourites.planner=?,
g_address.planner = ?,
g_climst.planner =?,
g_dialog.planner=?,
g_lang.planner=?,
g_prdmst.planner=?,
j_alloc.planner=?,
logger.planner=?,
passhistory.planner=?,
portfolios.planner=?,
prod_metrics_tank.planner=?,
product_usage_lists.planner=?,
region.planner=?,
reasearch_logger.planner=?,
search_dataphile.planner=?,
search_esg.planner=?,
search_rpm.planner=?,
sql_workout.planner=?,
universe_source.planner=?,
user_jurisdications.planner=?,
user_languages.planner=?,
user_universe.planner=?,
work_group_mappings.planner=?,
work_groups.planner=?,
spt_docs.planner=?,
g_secure.planner = ?
where a_assets.planner = ? ,
client_notes.planner = ?,
client_pending.planner = ?,
client_task.planner = ?,
country.planner = ?,
document_log.planner=?,
favlists.planner=?,
favourites.planner=?,
g_address.planner = ?,
g_climst.planner =?,
g_dialog.planner=?,
g_lang.planner=?,
g_prdmst.planner=?,
g_secure.planner = ?,
j_alloc.planner=?,
logger.planner=?,
passhistory.planner=?,
portfolios.planner=?,
prod_metrics_tank.planner=?,
product_usage_lists.planner=?,
region.planner=?,
reasearch_logger.planner=?,
search_dataphile.planner=?,
search_esg.planner=?,
search_rpm.planner=?,
sql_workout.planner=?,
universe_source.planner=?,
user_jurisdications.planner=?,
user_languages.planner=?,
user_universe.planner=?,
work_group_mappings.planner=?,
work_groups.planner=?,
spt_docs.planner=?
Im not sure why this wont work since all tables are updating their planner column. When this runs I get an: ILLEGAL SYMBOL "token". SOME SYMBOLS THAT MIGHT BE LEGAL ARE: token-list.
What needs to change in the query in order for all the tables to update with the same data.
You can not update more than one table in single update statement.
[Update multiple tables in SQL Server using INNER JOIN
Why do you want to update all tables in one statement? Assuming you are using a recent version of LUW you can actually do the update via transition tables and a cte according to:
with t1 (n) as ( select count(1)
from new table (
update a_assets
set planer = ?
where planer = ?
)
)
, t2 (n) as ( ...
)
, ...
, tn (n) as ( ...
)
select n from t1
union all
select n from t2
...
select n from tn
but I suspect that this is not what you want.
I would suggest that you make a procedure that makes a loop over the tables and then uses execute immediate to fire of the statement(s)
I have loaded the following test data:
name, age,gender
"John", 33,m
"Sam", 33,m
"Julie",33,f
"Jimbo",, m
with schema: name:STRING,age:INTEGER,gender:STRING and I have confirmed that the Jimbo row shows a null for column "age" in the BigQuery Browser Tool > mydataset > Details > Preview section.
When I run this query :
SELECT AVG(age) FROM [peterprivatedata.testpeople]
I get 24.75 which is incorrect. I expected 33 because the documentation for AVG says "Rows with a NULL value are not included in the calculation."
Am I doing something wrong or is this a known bug? (I don't know if there's a public issues list to check). What's the simplest workaround to this?
This is a known bug where we coerce null numeric values to to 0 on import. We're currently working on a fix. These values do however, show up as not not defined (which for various reasons is different from null), so you can check for IS_EXPLICITLY_DEFINED. For example:
SELECT sum(if(is_explicitly_defined(numeric_field), numeric_field, 0)) /
sum(if(is_explicitly_defined(numeric_field), 1, 0))
AS my_avg FROM your_table
Alternately, you could use another column to represent is_null. Then the query would look like:
SELECT sum(if(numeric_field_is_null, 0, numeric_field)) /
sum(if(numeric_field_is_null, 0, 1))
AS my_avg FROM your_table
I have a SQLite database with table myTable and columns id, posX, posY. The number of rows changes constantly (might increase or decrease). If I know the value of id for each row, and the number of rows, can I perform a single SQL query to update all of the posX and posY fields with different values according to the id?
For example:
---------------------
myTable:
id posX posY
1 35 565
3 89 224
6 11 456
14 87 475
---------------------
SQL query pseudocode:
UPDATE myTable SET posX[id] = #arrayX[id], posY[id] = #arrayY[id] "
#arrayX and #arrayY are arrays which store new values for the posX and posY fields.
If, for example, arrayX and arrayY contain the following values:
arrayX = { 20, 30, 40, 50 }
arrayY = { 100, 200, 300, 400 }
... then the database after the query should look like this:
---------------------
myTable:
id posX posY
1 20 100
3 30 200
6 40 300
14 50 400
---------------------
Is this possible? I'm updating one row per query right now, but it's going to take hundreds of queries as the row count increases. I'm doing all this in AIR by the way.
There's a couple of ways to accomplish this decently efficiently.
First -
If possible, you can do some sort of bulk insert to a temporary table. This depends somewhat on your RDBMS/host language, but at worst this can be accomplished with a simple dynamic SQL (using a VALUES() clause), and then a standard update-from-another-table. Most systems provide utilities for bulk load, though
Second -
And this is somewhat RDBMS dependent as well, you could construct a dynamic update statement. In this case, where the VALUES(...) clause inside the CTE has been created on-the-fly:
WITH Tmp(id, px, py) AS (VALUES(id1, newsPosX1, newPosY1),
(id2, newsPosX2, newPosY2),
......................... ,
(idN, newsPosXN, newPosYN))
UPDATE TableToUpdate SET posX = (SELECT px
FROM Tmp
WHERE TableToUpdate.id = Tmp.id),
posY = (SELECT py
FROM Tmp
WHERE TableToUpdate.id = Tmp.id)
WHERE id IN (SELECT id
FROM Tmp)
(According to the documentation, this should be valid SQLite syntax, but I can't get it to work in a fiddle)
One way: SET x=CASE..END (any SQL)
Yes, you can do this, but I doubt that it would improve performances, unless your query has a real large latency.
If the query is indexed on the search value (e.g. if id is the primary key), then locating the desired tuple is very, very fast and after the first query the table will be held in memory.
So, multiple UPDATEs in this case aren't all that bad.
If, on the other hand, the condition requires a full table scan, and even worse, the table's memory impact is significant, then having a single complex query will be better, even if evaluating the UPDATE is more expensive than a simple UPDATE (which gets internally optimized).
In this latter case, you could do:
UPDATE table SET posX=CASE
WHEN id=id[1] THEN posX[1]
WHEN id=id[2] THEN posX[2]
...
ELSE posX END [, posY = CASE ... END]
WHERE id IN (id[1], id[2], id[3]...);
The total cost is given more or less by: NUM_QUERIES * ( COST_QUERY_SETUP + COST_QUERY_PERFORMANCE ). This way, you knock down on NUM_QUERIES (from N separate id's to 1), but COST_QUERY_PERFORMANCE goes up (about 3x in MySQL 5.28; haven't yet tested in MySQL 8).
Otherwise, I'd try with indexing on id, or modifying the architecture.
This is an example with PHP, where I suppose we have a condition that already requires a full table scan, and which I can use as a key:
// Multiple update rules
$updates = [
"fldA='01' AND fldB='X'" => [ 'fldC' => 12, 'fldD' => 15 ],
"fldA='02' AND fldB='X'" => [ 'fldC' => 60, 'fldD' => 15 ],
...
];
The fields updated in the right hand expressions can be one or many, must always be the same (always fldC and fldD in this case). This restriction can be removed, but it would require a modified algorithm.
I can then build the single query through a loop:
$where = [ ];
$set = [ ];
foreach ($updates as $when => $then) {
$where[] = "({$when})";
foreach ($then as $fld => $value) {
if (!array_key_exists($fld, $set)) {
$set[$fld] = [ ];
}
$set[$fld][] = $value;
}
}
$set1 = [ ];
foreach ($set as $fld => $values) {
$set2 = "{$fld} = CASE";
foreach ($values as $i => $value) {
$set2 .= " WHEN {$where[$i]} THEN {$value}";
}
$set2 .= ' END';
$set1[] = $set2;
}
// Single query
$sql = 'UPDATE table SET '
. implode(', ', $set1)
. ' WHERE '
. implode(' OR ', $where);
Another way: ON DUPLICATE KEY UPDATE (MySQL)
In MySQL I think you could do this more easily with a multiple INSERT ON DUPLICATE KEY UPDATE, assuming that id is a primary key keeping in mind that nonexistent conditions ("id = 777" with no 777) will get inserted in the table and maybe cause an error if, for example, other required columns (declared NOT NULL) aren't specified in the query:
INSERT INTO tbl (id, posx, posy, bazinga)
VALUES (id1, posY1, posY1, 'DELETE'),
...
ON DUPLICATE KEY SET posx=VALUES(posx), posy=VALUES(posy);
DELETE FROM tbl WHERE bazinga='DELETE';
The 'bazinga' trick above allows to delete any rows that might have been unwittingly inserted because their id was not present (in other scenarios you might want the inserted rows to stay, though).
For example, a periodic update from a set of gathered sensors, but some sensors might not have been transmitted:
INSERT INTO monitor (id, value)
VALUES (sensor1, value1), (sensor2, 'N/A'), ...
ON DUPLICATE KEY UPDATE value=VALUES(value), reading=NOW();
(This is a contrived case, it would probably be more reasonable to LOCK the table, UPDATE all sensors to N/A and NOW(), then proceed with INSERTing only those values we do have).
A third way: CTE (PostgreSQL, not sure about SQLite3)
This is conceptually almost the same as the INSERT MySQL trick. As written, it works in PostgreSQL 9.6:
WITH updated(id, posX, posY) AS (VALUES
(id1, posX1, posY1),
(id2, posX2, posY2),
...
)
UPDATE myTable
SET
posX = updated.posY,
posY = updated.posY
FROM updated
WHERE (myTable.id = updated.id);
Something like this might work for you:
"UPDATE myTable SET ... ;
UPDATE myTable SET ... ;
UPDATE myTable SET ... ;
UPDATE myTable SET ... ;"
If any of the posX or posY values are the same, then they could be combined into one query
UPDATE myTable SET posX='39' WHERE id IN('2','3','40');
In recent versions of SQLite (beginning from 3.24.0 from 2018) you can use the UPSERT clause. Assuming only existing datasets are updated having a unique id column, you can use this approach, which is similar to #LSerni's ON DUPLICATE suggestion:
INSERT INTO myTable (id, posX, posY) VALUES
( 1, 35, 565),
( 3, 89, 224),
( 6, 11, 456),
(14, 87, 475)
ON CONFLICT (id) DO UPDATE SET
posX = excluded.posX, posY = excluded.posY
I could not make #Clockwork-Muse work actually. But I could make this variation work:
WITH Tmp AS (SELECT * FROM (VALUES (id1, newsPosX1, newPosY1),
(id2, newsPosX2, newPosY2),
......................... ,
(idN, newsPosXN, newPosYN)) d(id, px, py))
UPDATE t
SET posX = (SELECT px FROM Tmp WHERE t.id = Tmp.id),
posY = (SELECT py FROM Tmp WHERE t.id = Tmp.id)
FROM TableToUpdate t
I hope this works for you too!
Use a comma ","
eg:
UPDATE my_table SET rowOneValue = rowOneValue + 1, rowTwoValue = rowTwoValue + ( (rowTwoValue / (rowTwoValue) ) + ?) * (v + 1) WHERE value = ?
To update a table with different values for a column1, given values on column2, one can do as follows for SQLite:
"UPDATE table SET column1=CASE WHEN column2<>'something' THEN 'val1' ELSE 'val2' END"
Try with "update tablet set (row='value' where id=0001'), (row='value2' where id=0002'), ...
I am using multiple IN operators with AND in my sql query where clause as given below...
---
where ID in (1, 3, 234, 2332, 2123, 989) AND tag in ('wow', 'wonderful')
But surprisingly behaviour of result seems to be of OR type rather then AND type. What I mean is it is ignoring AND operator...
Can you please explain me why?
I couldn't reproduce the result using SQL Server 2008.
SELECT * FROM
(
SELECT 0 AS ID, 'wow' as Tag
) X
WHERE ID in (1, 3, 234, 2332, 2123, 989) AND tag in ('wow', 'wonderful')
Result:
No records
SELECT * FROM
(
SELECT 1 AS ID, 'wow' as Tag
) X
WHERE ID in (1, 3, 234, 2332, 2123, 989) AND tag in ('wow', 'wonderful')
Result:
ID Tag
1 wow
Check your code again.