Let's say we have a terrible design in BigQuery, which should have never been created that way, like the following:
some_project contains dataset metadata, which contains table metadata. Sample data for some_project.metadata.metadata:
| dataset_id |
| xyz1234567 |
| zzz8562042 |
| vyz0009091 |
For each dataset_id I need to query some_table in this dataset, for example some_project.xyz1234567.some_table.
Is this possible to query these multiple tables in a single query? I'm looking for getting aggregate results for each table.
In other words, I'm trying to say something like that:
SELECT SUM(table.x) from table WHERE table IN
(SELECT CONCAT('some_project.', dataset_id, 'some_table') FROM `some_project.metadata.metadata`)
or
SELECT SUM(table.x) FROM
(SELECT CONCAT('some_project.', dataset_id, 'some_table' as table FROM `some_project.metadata.metadata`)
I know that no one should ever need to do something like this, but the design I described above is something I just have to work with.
You can consider this approach using a temporary table as a SQL cursor alternative with the help of BigQuery looping statements.
You can read row by row and execute the query of each table name.
Here you can see this example:
DECLARE var1 INT64 DEFAULT 1;
DECLARE var2 INT64 DEFAULT 0;
DECLARE str1 string DEFAULT '';
DECLARE str2 string DEFAULT '';
DECLARE str3 string DEFAULT '';
CREATE TEMP TABLE temp_emp AS
SELECT empid,
ename,
deptid,
RANK() OVER(ORDER BY empid) rownum
FROM td.emp1;
SET var2= (SELECT COUNT(*) FROM temp_emp);
WHILE var2<=var1 DO
SET str1 = (SELECT empid FROM temp_emp WHERE rownum = x);
SET str2 = (SELECT empid FROM temp_emp WHERE rownum = x);
SET str3 = (SELECT empid FROM temp_emp WHERE rownum = x);
SET var1=var1+1;
END WHILE;
Following are some of points to be noted.
We are using the SET command to assign value to a variable. It is
SELECT..INTO in the original example.
We are not using open and close cursor.
We are creating a TEMPORARY table in place of cursor declaration.
You can see more documentation in this link.
Try this
declare sql string;
set sql = (
select concat("Select something from
someproject.",datasetid,".sometable"
) from metadata.metadata);
execute immediate sql
EXECUTE_IMMEDIATE
Related
I need assign the following SQL Server's query result value to the variable called #value1
SELECT *
FROM customer
WHERE apo_id = '2589';
How can I do this in SQL Server?
1 - First declare your variable of type table.
declare #value1 table(
--YOUR TABLE DEFINITION ex: ValueId int,
)
2 - Insert into your variable
insert into #value1 select * from customer WHERE apo_id = '2589';
Hope that helps, thanks.
It won't really be a variable but a table because you are selecting multiple fields (e.g. Select *) but, you can select INTO a temporary table like this:
SELECT *
INTO #myTempTable
FROM customer
WHERE apo_id = '2589';
Wasn't sure how to title the question but hopefully this makes sense :)
I have a table (OldTable) with an index and a column of comma separated lists. I'm trying to split the strings in the list column and create a new table with the indexes coupled with each of the sub strings of the string it was connected to in the old table.
Example:
OldTable
index | list
1 | 'a,b,c'
2 | 'd,e,f'
NewTable
index | letter
1 | 'a'
1 | 'b'
1 | 'c'
2 | 'd'
2 | 'e'
2 | 'f'
I have created a function that will split the string and return each sub string as a record in a 1 column table as so:
SELECT * FROM Split('a,b,c', ',', 1)
Which will result in:
Result
index | string
1 | 'a'
1 | 'b'
1 | 'c'
I was hoping that I could use this function as so:
SELECT * FROM Split((SELECT * FROM OldTable), ',')
And then use the id and string columns from OldTable in my function (by re-writing it slightly) to create NewTable. But I as far as I understand sending tables into the function doesn't work as I get: "Subquery returned more than 1 value. ... not premitted ... when the subquery is used as an expression."
One solution I was thinking of would be to run the function, as is, on all the rows of OldTable and insert the result of each call into NewTable. But I'm not sure how to iterate each row without a function. And I can't send tables into the a function to iterate so I'm back at square one.
I could do it manually but OldTable contains a few records (1000 or so) so it seems like automation would be preferable.
Is there a way to either:
Iterate over OldTable row by row, run the row through Split(), add the result to NewTable for all rows in OldTable. Either by a function or through regular sql-transactions
Re-write Split() to take a table variable after all
Get rid of the function altogether and just do it in sql transactions?
I'd prefer to not use procedures (don't know if there is a solutions with them either) mostly because I don't want the functionality inside of the DB to be exposed to the outside. If, however that is the "best"/only way to go I'll have to consider it. I'm quite (read very) new to SQL so it might be a needless worry.
Here is my Split() function if it is needed:
CREATE FUNCTION Split (
#string nvarchar(4000),
#delimitor nvarchar(10),
#indexint = 0
)
RETURNS #splitTable TABLE (id int, string nvarchar(4000) NOT NULL) AS
BEGIN
DECLARE #startOfSubString smallint;
DECLARE #endOfSubString smallint;
SET #startOfSubString = 1;
SET #endOfSubString = CHARINDEX(#delimitor, #string, #startOfSubString);
IF (#endOfSubString <> 0)
WHILE #endOfSubString > 0
BEGIN
INSERT INTO #splitTable
SELECT #index, SUBSTRING(#string, #startOfSubString, #endOfSubString - #startOfSubString);
SET #startOfSubString = #endOfSubString+1;
SET #endOfSubString = CHARINDEX(#delimitor, #string, #startOfSubString);
END;
INSERT INTO #splitTable
SELECT #index, SUBSTRING(#string, #startOfSubString, LEN(#string)-#startOfSubString+1);
RETURN;
END
Hope my problem and attempt was explained and possible to understand.
You are looking for cross apply:
SELECT t.index, s.item
FROM OldTable t CROSS APPLY
(dbo.split(t.list, ',')) s(item);
Inserting in the new table just requires an insert or select into clause.
I have a column, sort_order in a table that contains a string of numbers, a delimiter and some P values:
1150||P||1168||1144||1149||1147||1164||1152||P||1148||1162||1163||P||1156||1157||1154||
I would like to replace any P values in this string with another value from the event_tile_id column of another table.
So far I've drafted this SQL below with no luck. What changes can I make to this Query to get the effect I need?
`SELECT sort_order,
(
REPLACE(sort_order,'P',
(SELECT TOP 1 event_tile_id
FROM daily_email_sales_today)
)
)
as sort_order
FROM daily_email_preview`
Removed "default_SaleID" from Query. Replace should now have 4 arguments.
This is how I would do it.
Since you don't have any joins, why not do a simpler update query using a static value?
DECLARE #update VARCHAR(100)
SET #update = (SELECT TOP 1 event_tile_id FROM daily_email_sales_today)
update daily_email_preview
SET sort_order = replace(sort_order,'P', #update)
Or even,
update daily_email_preview
SET sort_order = replace(sort_order,'P', '<new value>')
Assuming you are using SQL Server.
Along the same thought process as #Eric_Hauenstein if you are running this in a TSQL process:
declare #rSTR as varchar(50)
SELECT TOP 1 #rSTR = event_tile_id FROM daily_email_sales_toda
SELECT sort_order, REPLACE(sort_order,'P', #rSTR) as sort_order
FROM daily_email_preview
I'm using Firebird 2.1.
There is a table: IDs, Labels
There can be multiple labels for the same ID:
10 Peach
10 Pear
10 Apple
11 Apple
12 Pear
13 Peach
13 Apple
Let's say I have a set of labels, ie.: (Apple, Pear, Peach).
How can I write a single select to return all IDs that have all labels associated in a given set? Preferably I'd like to specify the set in a string separated with commas, like: ('Apple', 'Pear', 'Peach') -› this should return ID = 10.
Thanks!
As asked, I'm posting my simpler version of piclrow's answer. I have tested this on my Firebird, which is version 2.5, but the OP (Steve) has tested it on 2.1 and it works as well.
SELECT id
FROM table
WHERE label IN ('Apple', 'Pear', 'Peach')
GROUP BY id
HAVING COUNT(DISTINCT label)=3
This solution has the same disadvantage as pilcrow's... you need to know how many values you are looking for, as the HAVING = condition must match the WHERE IN condition. In this respect, Ed's answer is more flexible, as it splits the concatenated value string parameter and counts the values. So you just have to change the one parameter, instead of the 2 conditions I and pilcrow use.
OTOH, if efficency is of concern, I would rather think (but I am absolutely not sure) that Ed's CTE approach might be less optimizable by the Firebird engine than the one I suggest. Firebird is very good at optimizing queries, but I don't really now if it is able to do so when you use CTE this way. But the WHERE + GROUP BY + HAVING should be optimizable by simply having an index on (id,label).
In conclusion, if execution times are of concern in your case, then you probably need some explain plans to see what is happening, whichever solution you choose ;)
It's easiest to split the string in code and then query
SQL> select ID
CON> from (select ID, count(DISTINCT LABEL) as N_LABELS
CON> from T
CON> where LABEL in ('Apple', 'Pear', 'Peach')
CON> group by 1) D
CON> where D.N_LABELS >= 3; -- We know a priori we have 3 LABELs
ID
============
10
If it is acceptable to create a helper stored procedure that will be called from the primary select then consider the following.
The Helper stored procedure takes in a delimited string along with the delimiter and returns a row for each delimited string
CREATE OR ALTER PROCEDURE SPLIT_BY_DELIMTER (
WHOLESTRING VARCHAR(10000),
SEPARATOR VARCHAR(10))
RETURNS (
ROWID INTEGER,
DATA VARCHAR(10000))
AS
DECLARE VARIABLE I INTEGER;
BEGIN
I = 1;
WHILE (POSITION(:SEPARATOR IN WHOLESTRING) > 0) DO
BEGIN
ROWID = I;
DATA = TRIM(SUBSTRING(WHOLESTRING FROM 1 FOR POSITION(TRIM(SEPARATOR) IN WHOLESTRING) - 1));
SUSPEND;
I = I + 1;
WHOLESTRING = TRIM(SUBSTRING(WHOLESTRING FROM POSITION(TRIM(SEPARATOR) IN WHOLESTRING) + 1));
END
IF (CHAR_LENGTH(WHOLESTRING) > 0) THEN
BEGIN
ROWID = I;
DATA = WHOLESTRING;
SUSPEND;
END
END
Below is the code to call, I am using Execute block to demonstrate passing in the delimited string
EXECUTE BLOCK
RETURNS (
LABEL_ID INTEGER)
AS
DECLARE VARIABLE PARAMETERS VARCHAR(50);
BEGIN
PARAMETERS = 'Apple,Peach,Pear';
FOR WITH CTE
AS (SELECT ROWID,
DATA
FROM SPLIT_BY_DELIMITER(:PARAMETERS, ','))
SELECT ID
FROM TABLE1
WHERE LABELS IN (SELECT DATA
FROM CTE)
GROUP BY ID
HAVING COUNT(*) = (SELECT COUNT(*)
FROM CTE)
INTO :LABEL_ID
DO
SUSPEND;
END
I use a regex in my SQL statements for an app that look like this
SELECT * FROM table WHERE id = {{REPLACEME}}
However, sometimes I'm not giving a parameter to replace that string with. Is there a way to replace it with something that matches anything. I tried *, but that does not work.
SELECT * FROM table WHERE id = id will match all rows that have non-null id
SELECT * FROM table WHERE id = id OR id IS NULL will match all rows.
id is probably a primary key, so you can probably use the former.
Replace {{REPLACEME}} with
[someValidValueForYouIdType] OR 1=1
I can only describe my solution with an example. The #AllRec is a parameter:
Declare #AllRec bit
set #AllRec = {0|1} --as appropriate
SELECT *
FROM table
WHERE
(
id = {{REPLACEME}}
and #AllRec = 0
) OR (
#AllRec = 1
)
In this solution, if #AllRec is 1 then everything is returned, ignoring the id filter. If #AllRec is zero, then the id filter is applied and you get one row. You should be able to quickly adapt this to your current regex solution.
Using the Regex-Replace option opens you up to SQL Injection attacks.
Assuming your language has support for parameterized queries, try this modified version of Jacob's answer:
SELECT * FROM table WHERE (id = #id OR #id IS NULL)
The catch is that you'll always have to provide the #id value.
SELECT field1, field2
FROM dbo.yourTable
WHERE id = isnull(#var, id)
Not sure what language your using, and this code kind of scares me but...
var statement = "SELECT * FROM table";
If REPLACEME is not empty Then
statement += " WHERE id = {{REPLACEME}}"
End If