Firebird CTE result dataset with insert statement - sql

I do have the difficulties with the following code I couldn't find where to place the insert statement in that code where it will insert the returned values from the CTE.
insert into ntm (num,comb,0,0,0,0,0,0,0,0,0,0,0,0)
with CTE as (
SELECT a.ELM_NUM as num,
a.N_1 as N1,
(trim('1.4*'||a.COMBO)||' + ' || '1.6*'||b.COMBO) as com2,
b.ELM_NUM as num2
FROM NTM a cross join ntm b
where ((a.Combo='dead' and b.combo='live') or (a.Combo='dead' and b.combo='live')) and a.ELM_NUM=b.ELM_NUM)
select num, num2, trim(trim(com2)||' + '||trim(d.combo)), N1, 1.4*N1
from CTE cross join ntm d
where (d.elm_num = cte.num) and (position('X',combo)>0 or position('Y',combo)>0 )
error raised by FlameRobin Engine "Token unknown - line 1, column 27
0
CREATE TABLE NTM
(
ELM_NUM Integer NOT NULL,
COMBO Char(40) NOT NULL,
N_1 Double precision,
N_2 Double precision,
V2_1 Double precision,
V2_2 Double precision,
V3_1 Double precision,
V3_2 Double precision,
M2_1 Double precision,
M2_2 Double precision,
M3_1 Double precision,
M3_2 Double precision,
MT_1 Double precision,
MT_2 Double precision
);

You can insert the result of an select statement by adding INSERT INTO <table> (<column>[, <column>]*) before the select statement (including the WITH). So If you have a query:
WITH a AS (
SELECT 4 AS ID, 'Item 4' AS NAME FROM RDB$DATABASE
)
SELECT ID, NAME
FROM a
And you want to insert into a table ITEMS with columns ID and NAME:
INSERT INTO ITEMS (ID, NAME)
WITH a AS (
SELECT 4 AS ID, 'Item 4' AS NAME FROM RDB$DATABASE
)
SELECT ID, NAME
FROM a
This works on Firebird 2.5, but I assume it also works in Firebird 2.1.
Based on your edit:
Your syntax is simply wrong: insert into ntm (num,comb,0,0,0,0,0,0,0,0,0,0,0,0) is invalid: you don't have a columns num and comb in table ntm, and specifying 0 as a column name is illegal. It seems you are confusing the INSERT columnlist with a value-list.
The syntax of an INSERT is:
INSERT INTO <object> [(col [, col …])]
{VALUES (<val> [, <val> …]) | <select_expr>}
You try to use something like:
INSERT INTO <object> (<val> [, <val> …])
<select_expr>
Assuming you only want to insert the num (into ELM_NUM) and the trim(trim(com2)||' + '||trim(d.combo)) (into COMBO) columns from your select, you should use:
insert into ntm (ELM_NUM,COMBO)
with CTE as (
SELECT a.ELM_NUM as num,
a.N_1 as N1,
trim('1.4*'||a.COMBO)||' + ' || '1.6*'||b.COMBO as com2,
b.ELM_NUM as num2
FROM NTM a cross join ntm b
where (a.Combo='dead' and b.combo='live' or a.Combo='dead' and b.combo='live') and a.ELM_NUM=b.ELM_NUM
)
select num, trim(trim(com2)||' + '||trim(d.combo))
from CTE cross join ntm d
where d.elm_num = cte.num and (position('X',combo) > 0 or position('Y',combo) > 0)
Note that I removed the other columns from the select as you are not inserting those (the number of columns in the select list must match the number of columns in the insert column list).
If you want the remaining columns to be 0, you either need to add a DEFAULT 0 to the table definition, or you need to list all columns and add a 0 value in your select list for each of those additional columns.

Related

Return five rows of random DNA instead of just one

This is the code I have to create a string of DNA:
prepare dna_length(int) as
with t1 as (
select chr(65) as s
union select chr(67)
union select chr(71)
union select chr(84) )
, t2 as ( select s, row_number() over() as rn from t1)
, t3 as ( select generate_series(1,$1) as i, round(random() * 4 + 0.5) as rn )
, t4 as ( select t2.s from t2 join t3 on (t2.rn=t3.rn))
select array_to_string(array(select s from t4),'') as dna;
execute dna_length(20);
I am trying to figure out how to re-write this to give a table of 5 rows of strings of DNA of length 20 each, instead of just one row. This is for PostgreSQL.
I tried:
CREATE TABLE dna_table(g int, dna text);
INSERT INTO dna_table (1, execute dna_length(20));
But this does not seem to work. I am an absolute beginner. How to do this properly?
PREPARE creates a prepared statement that can be used "as is". If your prepared statement returns one string then you can only get one string. You can't use it in other operations like insert, e.g.
In your case you may create a function:
create or replace function dna_length(int) returns text as
$$
with t1 as (
select chr(65) as s
union
select chr(67)
union
select chr(71)
union
select chr(84))
, t2 as (select s,
row_number() over () as rn
from t1)
, t3 as (select generate_series(1, $1) as i,
round(random() * 4 + 0.5) as rn)
, t4 as (select t2.s
from t2
join t3 on (t2.rn = t3.rn))
select array_to_string(array(select s from t4), '') as dna
$$ language sql;
And use it in a way like this:
insert into dna_table(g, dna) select generate_series(1,5), dna_length(20)
From the official doc:
PREPARE creates a prepared statement. A prepared statement is a server-side object that can be used to optimize performance. When the PREPARE statement is executed, the specified statement is parsed, analyzed, and rewritten. When an EXECUTE command is subsequently issued, the prepared statement is planned and executed. This division of labor avoids repetitive parse analysis work, while allowing the execution plan to depend on the specific parameter values supplied.
About functions.
This can be much simpler and faster:
SELECT string_agg(CASE ceil(random() * 4)
WHEN 1 THEN 'A'
WHEN 2 THEN 'C'
WHEN 3 THEN 'T'
WHEN 4 THEN 'G'
END, '') AS dna
FROM generate_series(1,100) g -- 100 = 5 rows * 20 nucleotides
GROUP BY g%5;
random() produces random value in the range 0.0 <= x < 1.0. Multiply by 4 and take the mathematical ceiling with ceil() (cheaper than round()), and you get a random distribution of the numbers 1-4. Convert to ACTG, and aggregate with GROUP BY g%5 - % being the modulo operator.
About string_agg():
Concatenate multiple result rows of one column into one, group by another column
As prepared statement, taking
$1 ... the number of rows
$2 ... the number of nucleotides per row
PREPARE dna_length(int, int) AS
SELECT string_agg(CASE ceil(random() * 4)
WHEN 1 THEN 'A'
WHEN 2 THEN 'C'
WHEN 3 THEN 'T'
WHEN 4 THEN 'G'
END, '') AS dna
FROM generate_series(1, $1 * $2) g
GROUP BY g%$1;
Call:
EXECUTE dna_length(5,20);
Result:
| dna |
| :------------------- |
| ATCTTCGACACGTCGGTACC |
| GTGGCTGCAGATGAACAGAG |
| ACAGCTTAAAACACTAAGCA |
| TCCGGACCTCTCGACCTTGA |
| CGTGCGGAGTACCCTAATTA |
db<>fiddle here
If you need it a lot, consider a function instead. See:
What is the difference between a prepared statement and a SQL or PL/pgSQL function, in terms of their purposes?

view key value rows as multi column single row

You have parameters x,y,z stored as key values.
You want to execute an expression z=x+y on those parameters. Expression is stored in another table.
You want to generate an SQL query as simply as possible from the expression.
How can you view those parameter values as a single row with columns (x,y,z) to enable execution of the expression ?
SELECT *
INTO #key_values
FROM
(
SELECT 'x' AS mykey, 2 AS myvalue
UNION ALL
SELECT 'y', 5
UNION ALL
SELECT 'z', 0
) a;
This screams for a PIVOT operator:
;WITH Inputs AS
(
SELECT 'x' AS mykey, 2 AS myvalue
UNION ALL
SELECT 'y', 5
UNION ALL
SELECT 'z', 0
)
SELECT
U.x,
U.y,
U.z,
Result = U.x + U.y
FROM
Inputs AS I
PIVOT (
MAX(I.myvalue) FOR I.mykey IN (x, y, z)
) AS U
Results:
x y z Result
2 5 0 7
You can build any expression you want with the pivoted columns in the SELECT.
If you want to update the z record, you will have to join back to the underlying table since after applying the PIVOT you lose access to original table.
IF OBJECT_ID('tempdb..#Input') IS NOT NULL
DROP TABLE #Input
CREATE TABLE #Input (
mykey VARCHAR(10),
myvalue INT)
INSERT INTO #Input (
mykey,
myvalue)
VALUES
('x', 2),
('y', 5),
('z', 0)
UPDATE I SET
myvalue = R.Result
FROM
#Input AS I
CROSS APPLY (
SELECT
Result = x + y
FROM
#Input AS I
PIVOT (MAX(I.myvalue) FOR I.mykey IN (x, y, z)) AS U
) AS R
WHERE
I.mykey = 'z'
Turn the 3 rows into a single 3 column row using a common table expression and update it to run the expression. So the proposed solution is an updatable cte.
WITH myvalues(x,y,z) AS (
SELECT x.myvalue, y.myvalue, z.myvalue
FROM #key_values AS x
JOIN #key_values AS y ON y.mykey='y' AND x.mykey='x'
JOIN #key_values AS z ON z.mykey='z'
)
UPDATE myvalues SET z=x+y;
SELECT myvalue FROM #key_values WHERE mykey='z';

How to select columns of data in BigQuery that has all NULL values

How to select columns of data in BigQuery that has all NULL values
A B C
NULL 1 NULL
NULL NULL NULL
NULL 2 NULL
NULL 3 NULL
I want to retrieve columns A and C. Please can you help!!
Expanding on my comment on Mikhail's answer, this is what I had in mind. It doesn't require generating a query string, which could be quite long if you have a large number of columns. It compares the count of null values for each column name to the total number of rows in the table to decide if the column should be included in the result.
#standardSQL
WITH `project.dataset.table` AS (
SELECT NULL A, 1 B, NULL C UNION ALL
SELECT NULL, NULL, NULL UNION ALL
SELECT NULL, 2, NULL UNION ALL
SELECT NULL, 3, NULL
)
SELECT null_column
FROM `project.dataset.table` AS t,
UNNEST(REGEXP_EXTRACT_ALL(
TO_JSON_STRING(t),
r'\"([a-zA-Z0-9\_]+)\":null')
) AS null_column
GROUP BY null_column
HAVING COUNT(*) = (SELECT COUNT(*) FROM `project.dataset.table`);
Below is for BigQuery StandardSQL
Simple option:
#standardSQL
WITH `project.dataset.table` AS (
SELECT NULL A, 1 B, NULL C UNION ALL
SELECT NULL, NULL, NULL UNION ALL
SELECT NULL, 2, NULL UNION ALL
SELECT NULL, 3, NULL
)
SELECT COUNT(A) A, COUNT(B) B, COUNT(C) C
FROM `project.dataset.table`
it returns below where 0(zero) indicates that respective column has all NULLs
A B C
0 3 0
If this is "not enough" - below is more "sophisticated" version:
#standardSQL
WITH `project.dataset.table` AS (
SELECT NULL A, 1 B, NULL C UNION ALL
SELECT NULL, NULL, NULL UNION ALL
SELECT NULL, 2, NULL UNION ALL
SELECT NULL, 3, NULL
)
SELECT SPLIT(y, ':')[OFFSET(0)] column
FROM (
SELECT REGEXP_REPLACE(TO_JSON_STRING(t), r'[{}"]', '') x
FROM (
SELECT COUNT(A) A, COUNT(B) B, COUNT(C) C
FROM `project.dataset.table`
) t
), UNNEST(SPLIT(x)) y
WHERE CAST(SPLIT(y, ':')[OFFSET(1)] AS INT64) = 0
it returns result as below - enlisting only columns with all NULLs
column
A
C
Note: for your real table - just remove WITH block and replace project.dataset.table with your real table reference
Also, of course, use real column names
My table has round 700 columns..
Below is an example of how you can easily generate above query for any number of columns.
1. Just run below
2. Copy result - this is a generated query
3. paste generated query into new UI and run it
4. Enjoy (I hope you will) result :o)
Of course, as usually replace project.dataset.table with your real table reference
#standardSQL
SELECT
CONCAT('''
SELECT SPLIT(y, ':')[OFFSET(0)] column
FROM (
SELECT REGEXP_REPLACE(TO_JSON_STRING(t), r'[{}"]', '') x
FROM (
SELECT ''', y,
'''
FROM `project.dataset.table`
) t
), UNNEST(SPLIT(x)) y
WHERE CAST(SPLIT(y, ':')[OFFSET(1)] AS INT64) = 0
'''
)
FROM (
SELECT
STRING_AGG(CONCAT('COUNT(', x, ') ', x), ', ') y
FROM (
SELECT REGEXP_EXTRACT_ALL(REGEXP_REPLACE(TO_JSON_STRING(t), r'[{}]', ''), r'"([\w_]+)":') x
FROM `project.dataset.table` t
LIMIT 1
), UNNEST(x) x
)
Note: please pay attention to query cost - both "generation query" and final query itself will do full scan
You can generate columns list much cheaper off of table schema in any client of your choice
To test / play with it - you can use same dummy data as for initial queries in my answer

How can I use PIVOT to show simultationly average and count in its cells?

Looking at the syntax I get the strong impression, that PIVOT doesn't support anything beyond a single aggregate function to be calculated for a cell.
From statistical view showing just some averages without giving the number of cases an average refers to is very unsatisfying ( that is the polite version ).
Is there some nice pattern to evaluate pivots based on avg and pivots based on count and mix them together to give a nice result?
Yes you need to use the old style cross tab for this. The PIVOT is just syntactic sugar that resolves to pretty much the same approach.
SELECT AVG(CASE WHEN col='foo' THEN col END) AS AvgFoo,
COUNT(CASE WHEN col='foo' THEN col END) AS CountFoo,...
If you have many aggregates you could always use a CTE
WITH cte As
(
SELECT CASE WHEN col='foo' THEN col END AS Foo...
)
SELECT MAX(Foo),MIN(Foo), COUNT(Foo), STDEV(Foo)
FROM cte
Simultaneous.. in its cells. So you mean within the same cell, therefore as a varchar?
You could calc the avg and count values in an aggregate query before using the pivot, and concatenate them together as text.
The role of the PIVOT operator here would only be to transform rows to columns, and some aggregate function (e.g. MAX/MIN) would be used only because it is required by the syntax - your pre-calculated aggregate query would only have one value per pivoted column.
EDIT
Following bernd_k's oracle/mssql solution, I would like to point out another way to do this in SQL Server. It requires streamlining the multiple columns into a single column.
SELECT MODULE,
modus + '_' + case which when 1 then 'AVG' else 'COUNT' end AS modus,
case which when 1 then AVG(duration) else COUNT(duration) end AS value
FROM test_data, (select 1 as which union all select 2) x
GROUP BY MODULE, modus, which
SELECT *
FROM (
SELECT MODULE,
modus + '_' + case which when 1 then 'AVG' else 'COUNT' end AS modus,
case which when 1 then CAST(AVG(1.0*duration) AS NUMERIC(10,2)) else COUNT(duration) end AS value
FROM test_data, (select 1 as which union all select 2) x
GROUP BY MODULE, modus, which
) P
PIVOT (MAX(value) FOR modus in ([A_AVG], [A_COUNT], [B_AVG], [B_COUNT])
) AS pvt
ORDER BY pvt.MODULE
In the example above, AVG and COUNT are compatible (count - int => numeric). If they are not, convert both explicitly to a compatible type.
Note - The first query shows AVG for M2/A as 2, due to integer averaging. The 2nd (pivoted) query shows the actual average taking into account decimals.
Solution for Oracle 11g + :
create table test_data (
module varchar2(30),
modus varchar2(30),
duration Number(10)
);
insert into test_data values ('M1', 'A', 5);
insert into test_data values ('M1', 'A', 5);
insert into test_data values ('M1', 'B', 3);
insert into test_data values ('M2', 'A', 1);
insert into test_data values ('M2', 'A', 4);
select *
FROM (
select *
from test_data
)
PIVOT (
AVG(duration) avg , count(duration) count
FOR modus in ( 'A', 'B')
) pvt
ORDER BY pvt.module;
I do not like the column names containing apostrophes, but the result contains what I want:
MODULE 'A'_AVG 'A'_COUNT 'B'_AVG 'B'_COUNT
------------------------------ ---------- ---------- ---------- ----------
M1 5 2 3 1
M2 2.5 2 0
I really wonder what the Microsoft boys did, when they only allowed one aggregate function within pivot. I call evaluation avgs without accompanying counts statistical lies.
SQL-Server 2005 + (based on Cyberwiki):
CREATE TABLE test_data (
MODULE VARCHAR(30),
modus VARCHAR(30),
duration INTEGER
);
INSERT INTO test_data VALUES ('M1', 'A', 5);
INSERT INTO test_data VALUES ('M1', 'A', 5);
INSERT INTO test_data VALUES ('M1', 'B', 3);
INSERT INTO test_data VALUES ('M2', 'A', 1);
INSERT INTO test_data VALUES ('M2', 'A', 4);
SELECT MODULE, modus, ISNULL(LTRIM(STR(AVG(duration))), '') + '|' + ISNULL(LTRIM(STR(COUNT(duration))), '') RESULT
FROM test_data
GROUP BY MODULE, modus;
SELECT *
FROM (
SELECT MODULE, modus, ISNULL(LTRIM(STR(AVG(duration))), '') + '|' + ISNULL(LTRIM(STR(COUNT(duration))), '') RESULT
FROM test_data
GROUP BY MODULE, modus
) T
PIVOT (
MAX(RESULT)
FOR modus in ( [A], [B])
) AS pvt
ORDER BY pvt.MODULE
result:
MODULE A B
------------------------------ --------------------- ---------------------
M1 5|2 3|1
M2 2|2 NULL

SQL: how to get all the distinct characters in a column, across all rows

Is there an elegant way in SQL Server to find all the distinct characters in a single varchar(50) column, across all rows?
Bonus points if it can be done without cursors :)
For example, say my data contains 3 rows:
productname
-----------
product1
widget2
nicknack3
The distinct inventory of characters would be "productwigenka123"
Here's a query that returns each character as a separate row, along with the number of occurrences. Assuming your table is called 'Products'
WITH ProductChars(aChar, remain) AS (
SELECT LEFT(productName,1), RIGHT(productName, LEN(productName)-1)
FROM Products WHERE LEN(productName)>0
UNION ALL
SELECT LEFT(remain,1), RIGHT(remain, LEN(remain)-1) FROM ProductChars
WHERE LEN(remain)>0
)
SELECT aChar, COUNT(*) FROM ProductChars
GROUP BY aChar
To combine them all to a single row, (as stated in the question), change the final SELECT to
SELECT aChar AS [text()] FROM
(SELECT DISTINCT aChar FROM ProductChars) base
FOR XML PATH('')
The above uses a nice hack I found here, which emulates the GROUP_CONCAT from MySQL.
The first level of recursion is unrolled so that the query doesn't return empty strings in the output.
Use this (shall work on any CTE-capable RDBMS):
select x.v into prod from (values('product1'),('widget2'),('nicknack3')) as x(v);
Test Query:
with a as
(
select v, '' as x, 0 as n from prod
union all
select v, substring(v,n+1,1) as x, n+1 as n from a where n < len(v)
)
select v, x, n from a -- where n > 0
order by v, n
option (maxrecursion 0)
Final Query:
with a as
(
select v, '' as x, 0 as n from prod
union all
select v, substring(v,n+1,1) as x, n+1 as n from a where n < len(v)
)
select distinct x from a where n > 0
order by x
option (maxrecursion 0)
Oracle version:
with a(v,x,n) as
(
select v, '' as x, 0 as n from prod
union all
select v, substr(v,n+1,1) as x, n+1 as n from a where n < length(v)
)
select distinct x from a where n > 0
Given that your column is varchar, it means it can only store characters from codes 0 to 255, on whatever code page you have. If you only use the 32-128 ASCII code range, then you can simply see if you have any of the characters 32-128, one by one. The following query does that, looking in sys.objects.name:
with cteDigits as (
select 0 as Number
union all select 1 as Number
union all select 2 as Number
union all select 3 as Number
union all select 4 as Number
union all select 5 as Number
union all select 6 as Number
union all select 7 as Number
union all select 8 as Number
union all select 9 as Number)
, cteNumbers as (
select U.Number + T.Number*10 + H.Number*100 as Number
from cteDigits U
cross join cteDigits T
cross join cteDigits H)
, cteChars as (
select CHAR(Number) as Char
from cteNumbers
where Number between 32 and 128)
select cteChars.Char as [*]
from cteChars
cross apply (
select top(1) *
from sys.objects
where CHARINDEX(cteChars.Char, name, 0) > 0) as o
for xml path('');
If you have a Numbers or Tally table which contains a sequential list of integers you can do something like:
Select Distinct '' + Substring(Products.ProductName, N.Value, 1)
From dbo.Numbers As N
Cross Join dbo.Products
Where N.Value <= Len(Products.ProductName)
For Xml Path('')
If you are using SQL Server 2005 and beyond, you can generate your Numbers table on the fly using a CTE:
With Numbers As
(
Select Row_Number() Over ( Order By c1.object_id ) As Value
From sys.columns As c1
Cross Join sys.columns As c2
)
Select Distinct '' + Substring(Products.ProductName, N.Value, 1)
From Numbers As N
Cross Join dbo.Products
Where N.Value <= Len(Products.ProductName)
For Xml Path('')
Building on mdma's answer, this version gives you a single string, but decodes some of the changes that FOR XML will make, like & -> &.
WITH ProductChars(aChar, remain) AS (
SELECT LEFT(productName,1), RIGHT(productName, LEN(productName)-1)
FROM Products WHERE LEN(productName)>0
UNION ALL
SELECT LEFT(remain,1), RIGHT(remain, LEN(remain)-1) FROM ProductChars
WHERE LEN(remain)>0
)
SELECT STUFF((
SELECT N'' + aChar AS [text()]
FROM (SELECT DISTINCT aChar FROM Chars) base
ORDER BY aChar
FOR XML PATH, TYPE).value(N'.[1]', N'nvarchar(max)'),1, 1, N'')
-- Allow for a lot of recursion. Set to 0 for infinite recursion
OPTION (MAXRECURSION 365)