I have the table
| Col1 | Col2 | Col3 |
I want a new table with these values
| Col4 | Col5 | Col2 | Col6 |
With any values of col2 onto the new table
Thanks!
If It Is Just About To Import All Data Of Col2 In Another Table ,Then
Suppose TB1 has Columns (Col1 , Col2 , Col3)
and TB2 has Columns (Col4 , Col5 , Col2 , Col6)
Then Your Quert Would Be Like This ,
Insert INTO TB2(Col2)(Select Col2 From TB1);
else
Please Give More Specification Like You Want To Update Data That is Not In TB2 Like That.
Related
I'm trying to make a select between 2 tables which I cannot join directly ( they don't have the same values ).
My 2 input table look something like this:
Table1:
Col4 | Col3 | Col2 | Col1
ZO123 | ZH10 | Spark111 | 000567
VS561 | JK14 | Dmi563 | 009123
Table2 ( here is the main problem, as we have concatenated values inside the cells):
Col7 | Col6 | Col5 | Col8
000567,009123 | 567,9123 | ZH10,JK14 | Spark111
00657,00896 | 657,896 | PK15,NU85 | Dmi563
Table1 is a temp view as well with 410 million rows. Table2 is a temp view with 221 rows.
Both are temp views built on top of some parquet files.
The query that I currently test and it's taking like 2 hours is:
CREATE OR REPLACE TEMP VIEW Table3 AS SELECT w.Col1 AS Col1
w.Col2 AS Col2
w.Col3 AS Col3
w.Col4 AS Col4
m.Col5 AS Col5,
m.Col6 AS Col6,
m.Col7 AS Col7
m.Col8 AS Col8 FROM Table1 w,Table2 m
WHERE exists (SELECT * FROM Table2 WHERE exists(split(Table2.Col5, ','), x -> x = w.Col3) AND Table2.Col5 = m.Col5 AND exists(split(Table2.Col6, ','), x -> x = udf_function_to_remove_zeros(w.Col1)) and Table2.Col6 = m.Col6 AND exists(split(Table2.Col8, ','), x -> x = w.Col2) AND Table2.Col8 = m.Col8)
The main idea is that I'm trying to retrieve rows from Table1 where the values from rows can be found in the splited values from Table2. Also, all the conditions should be passed with AND.
You are facing performance issues probably because you're:
unnecessarily doing a cartesian product FROM Table1 w,Table2 m
using UDF to remove leading zeros while you can do the same using builtin functions
You can actually write the same query using inner join like this:
SELECT
w.Col1 AS Col1,
w.Col2 AS Col2,
w.Col3 AS Col3,
w.Col4 AS Col4,
m.Col5 AS Col5,
m.Col6 AS Col6,
m.Col7 AS Col7,
m.Col8 AS Col8
FROM
Table1 w
JOIN (
SELECT
Col7,
Col8,
INLINE(ARRAYS_ZIP(SPLIT(Col6, ','), SPLIT(Col5, ','))) AS (Col6, Col5)
FROM
Table2
) m
ON
w.Col3 = m.Col5
AND TRIM(LEADING '0' FROM w.Col1) = m.Col6
AND w.Col2 = m.Col8
Applied to your input example:
spark.createDataFrame([
("ZO123", "ZH10", "Spark111", "000567"), ("VS561", "JK14", "Dmi563", "009123")
], ["Col4", "Col3", "Col2", "Col1"]).createOrReplaceTempView("Table1")
spark.createDataFrame([
("000567,009123", "567,9123", "ZH10,JK14", "Spark111"), ("00657,00896", "657,896", "PK15,NU85", "Dmi563")
], ["Col7", "Col6", "Col5", "Col8"]).createOrReplaceTempView("Table2")
spark.sql(above_query).show()
#+------+--------+----+-----+----+----+-------------+--------+
#| Col1| Col2|Col3| Col4|Col5|Col6| Col7| Col8|
#+------+--------+----+-----+----+----+-------------+--------+
#|000567|Spark111|ZH10|ZO123|ZH10| 567|000567,009123|Spark111|
#+------+--------+----+-----+----+----+-------------+--------+
how to scatter multiple duplicate rows into one row in sql/postgresql.
For example --->
lets i am getting 3 rows of
col1 col2 col3
-------------------
11 test rat
11 test cat
11 test test
I want something like this
col1 col2 col3 col4
------------------------
11 test rat cat
Its the same thing like groupby in lodash. But how do I achieve the same in postgresql query?
You're looking for crosstab
postgres=# create table ab (col1 text, col2 text, col3 text);
CREATE TABLE
postgres=# insert into ab values ('t1','test','cat'),('t1','test','rat'),('t1','test','test');
INSERT 0 3
postgres=# select * from crosstab('select col1,col2,col3 from ab') as (col1 text, col2 text, col3 text, col4 text);
col1 | col2 | col3 | col4
------+------+------+------
t1 | cat | rat | test
(1 row)
Disclosure: I work for EnterpriseDB (EDB)
Does MINUS/EXCEPT command or code workaround analog exist for columns only? Since MINUS/EXCEPT command fine for rows, how about for columns?
Mask-table (physically exist):
id col1 col2 col3 col4 ... colN comment
doesn't A B C D ... Z --alphabet correct sequence
matter
Columns Data Type of col[i] equals to each other.
Incoming select-stream (not a physical table, but represented as table as a result of other complex joined-selection from other tables):
col1 col2 col3 col4 ... colN comment
A B C D ... Z --alphabet correct seq
A C B D ... Z --incorrect
E B C M ... Z --incorrect
...
Z Y X W ... A --full inverse icorrect
Expected output to physical table after processing mask-table to select-stream as insert result:
id col1 col2 col3 col4 ... colN
(auto-
gnrtd)
(null)(null)(null)(null)...(null)
(null) C B (null)...(null)
E (null)(null) M ...(null)
...
Z Y X W ... A
Please note: alphabet is given just as an example. Not the issue-case here. SQL-Logic/command required. Analog of MINUS/EXCEPT, but for columns (DISTINCT? How, if incoming select-stream is a result of other complex joined-select)
What will be the SQL-code for this task? Please, advise.
Another way to do it without CASE statements:
Setup
CREATE TABLE mask (
col1 TEXT,
col2 TEXT,
col3 TEXT,
col4 TEXT,
col5 TEXT
);
INSERT INTO mask SELECT 'A', 'B', 'C', 'D', 'E';
CREATE TABLE your_stream (
col1 TEXT,
col2 TEXT,
col3 TEXT,
col4 TEXT,
col5 TEXT
);
INSERT INTO your_stream
VALUES
('A', 'B', 'C', 'D', 'E'),
('A', 'C', 'B', 'D', 'E'),
('E', 'B', 'C', 'M', 'E');
Query
SELECT
NULLIF(s.col1, m.col1) AS col1,
NULLIF(s.col2, m.col2) AS col2,
NULLIF(s.col3, m.col3) AS col3,
NULLIF(s.col4, m.col4) AS col4,
NULLIF(s.col5, m.col5) AS col5
FROM your_stream s
CROSS JOIN mask m;
Result
| col1 | col2 | col3 | col4 | col5 |
| ---- | ---- | ---- | ---- | ---- |
| null | null | null | null | null |
| null | C | B | null | null |
| E | null | null | M | null |
View on DB Fiddle
I don't see what the connection to a set operation like EXCEPT would be.
Anyway, this is how you could proceed:
INSERT INTO destination (col1, col2, col3, ...)
SELECT CASE WHEN incoming_col1 <> mask.col1
THEN incoming_col1
END,
CASE WHEN incoming_col2 <> mask.col2
THEN incoming_col2
END,
...
FROM mask;
I have three columns in a table.
Requirements: The value of col2 and col3 should make col1.
Below shows the table I have right now, which needs to be change.
col1 col2 col3
AB football
AB football
ER driving
ER driving
TR city
TR city
Below shows the table that needs to be change to
col1 col2 col3
AB_football_1 AB football
AB_football_2 AB football
ER_driving_1 ER driving
ER_driving_2 ER driving
TR_city_1 TR city
TR_city_2 TR city
As you can see in col1, it should take col2, put (underscore), then col3, put (underscore) then increment the number according to the values in col2 and col3.
Can this be approached within CREATE or SELECT or INSERT statement or Trigger Statement, if so any tips would be grateful.
Try as
SELECT col2
||'_'
||col3
||'_'
||rank col1,
col2,
col3
FROM (SELECT col2,
col3,
ROW_NUMBER()
OVER(
PARTITION BY col2, col3
ORDER BY col2) rank
FROM my_table)
Output
+---------------+------+----------+
| COL1 | COL2 | COL3 |
+---------------+------+----------+
| AB_football_1 | AB | football |
| AB_football_2 | AB | football |
| ER_driving _1 | ER | driving |
| ER_driving _2 | ER | driving |
| TR_city _1 | TR | city |
| TR_city _2 | TR | city |
+---------------+------+----------+
/* table is */
col1 col2 col3
test 123
/* Try this query */
UPDATE `demo`
SET `col1` = concat(col2, '_', col3)
/* Output will be */
col1 col2 col3
test_123 test 123
This is easy to do (at SELECT) using row_number() window function , something like this:
select
col2 ||'_'|| col3 ||'_'|| row_number() over(partition by col2, col3 order by col2) as col1,
col2,
col3
from t
My Table data looks like
Col1 | Col2 | Col3
1 | NULL | NULL
NULL | 2 | NULL
NULL | NULL | 3
It is given that for any column there will be only entry. This means that, in the above data, if row1 has value for Col1, then there will be no row with value for Col1. Similarly, if row1 has value for Col1, it will not have value for any other column.
I want to write a query, so that I get only one row out for entire data (leaving NULL values). ie.
Col1 | Col2 | Col3
1 | 2 | 3
The easiest way to do this is using aggregation:
select max(col1) as col1, max(col2) as col2, max(col3) as col3
from t;
select
sum(ifnull(col1,0)) as col1,
sum(ifnull(col2,0)) as col2
sum(ifnull(col3,0)) as col3
from t;
Assuming the table is called tab the following query will work if there are only 3 columns:
select t1.Col1, t2.Col2, t3.Col3
from tab t1, tab t2, tab t3
where t1.Col1 is not null and t2.Col2 is not null and t3.Col3 is not null
The problem is the query will have to alias the table for each additional column. It may not be perfect, but it is a solution.