BigQuery select multiple tables with different column names - google-bigquery

Consider the following BigQuery tables schemas in my dataset my_dataset:
Table_0001: NAME (string); NUMBER (string)
Table_0002: NAME(string); NUMBER (string)
Table_0003: NAME(string); NUMBER (string)
...
Table_0865: NAME (string); CODE (string)
Table_0866: NAME(string); CODE (string)
...
I now want to union all tables using :
select * from `my_dataset.*`
However this will not yield the CODE column of the second set of table. From my understanding, the schema of the first table in the dataset will be adopted instead.
So the result with be something like:
| NAME | NUMBER |
__________________
| John | 123456 |
| Mary | 123478 |
| ... | ...... |
| Abdul | null |
| Ariel | null |
I tried to tap into the INFORMATION_SCHEMA so as to select the two sets of tables separately and then union them:
with t_code as (
select
table_name,
from my_dataset.INFORMATION_SCHEMA.COLUMNS
where column_name = 'CODE'
),
select t.NAME, t.CODE as NUMBER from `my_dataset.*` as t
where _TABLE_SUFFIX in (select * from t_code)
However, still the script will look to the first table of my_dataset for its schema and will return: Error Running Query: Name CODE not found inside t.
So now I'm at a loss: How can I union all my tables without having to union them one by one? ie. how to select CODE as NUMBER in the second set of tables.
Note: Although it seems the question was asked over here, the accepted answer did not seem to actually respond to the question (as far as I'm concerned).

The trick I see you can do is to first gather all codes by running
create table `my_another_dataset.codes` as
select * from `my_dataset.*` where not code is null
Then to do any simple fake update of any just one table with number column - this will make schema with number column default. so now you can gather all numbers
create table `my_another_dataset.numbers` as
select * from `my_dataset.*` where not number is null
Finally then you can do simple union
select * from `my_another_dataset.numbers` union all
select * from `my_another_dataset.codes`
Note: see also my comment below your question

SELECT
borrow.id AS `borrowId`,
IF(borrow.created_date IS NULL, '', borrow.created_date) AS `borrowCreatedDate`,
IF(borrow.return_date IS NULL, '', borrow.return_date) AS `borrowReturnDate`,
IF(borrow.return_date IS NULL, '0', '1') AS `borrowIsReturn`,
IF(person.card_identity IS NULL, '', person.card_identity) AS `personCardIdentity`,
IF(person.fullname IS NULL, '', person.fullname) AS `personFullname`,
IF(person.phone_number IS NULL, '', person.phone_number) AS `personPhoneNumber`,
IF(book.book_name IS NULL, '', book.book_name) AS `bookName`,
IF(book.year IS NULL, '', book.year) AS `bookYear`
FROM tbl_tbl_borrow AS borrow
LEFT JOIN tbl_person AS person
ON person.card_identity = borrow.person_card_identity
LEFT JOIN tbl_book AS book
ON book.unique_id = borrow.book_unique_id
ORDER BY
borrow.return_date ASC, person.fullname ASC;

Related

Return only ALL CAPS strings in BigQuery

Pretty simple question, specific to BigQuery. I'm sure there's a command I'm missing. I'm used to using "collate" in another query which doesn't work here.
email
| -------- |
| eric#email.com |
| JOHN#EMAIL.COM |
| STACY#EMAIL.COM |
| tanya#email.com |
Desired return:
JOHN#EMAIL.COM,STACY#EMAIL.COM
Consider below
select *
from your_table
where upper(email) = email
If applied to sample data in your question - output is
In case you want the output as a comma separated list - use below
select string_agg(email) emails
from your_table
where upper(email) = email
with output
You can use below cte (which is exact data sample from your question) for testing purposes
with your_table as (
select 'eric#email.com' email union all
select 'JOHN#EMAIL.COM' union all
select 'STACY#EMAIL.COM' union all
select 'tanya#email.com'
)

Laravel: Adding multiple count results of sub queries

I wanted to add the count result of more than one queries that belong to different tables.
I am using the below problem as a reference to my actual problem because this problem has already a solution (How do I add two count(*) results together on two different tables?) but I am facing problem in implementing the solution in laravel.
I have two tables: Toys and Games.
+--------------------+------------------+
| Field | Type |
+--------------------+------------------+
| toy_id | int(10) unsigned |
| little_kid_id | int(10) unsigned |
+--------------------+------------------+
+--------------------+------------------+
| Field | Type |
+--------------------+------------------+
| game_id | int(10) unsigned |
| little_kid_id | int(10) unsigned |
+--------------------+------------------+
A little kid can have multiple toys. A little kid can be participating in multiple games at once.
I want a query that will give me the total number of toys + games that a little_kid is involved with.
Basically, I want the sum of these two queries:
SELECT COUNT(*) FROM Toys WHERE little_kid_id = 900;
SELECT COUNT(*) from Games WHERE little_kid_id = 900
The above problem has the following accepted answer
SELECT
(SELECT COUNT(*) FROM Toys WHERE little_kid_id = 900)+
(SELECT COUNT(*) from Games WHERE little_kid_id = 900)
AS SumCount
I wanted to implement the above solution in Laravel.
I have tried the following method but to no avail. It gives syntax error.
$sub=DB::tabel('toys')->select(DB::raw('count(*) as total'))
->where('little_kid_id',$id);
$sub1=DB::tabel('games')->select(DB::raw('count(*) as total'))
->where('little_kid_id',$id);
$result = DB::table( DB::raw("( ({$sub->toSql()})+({$sub1->toSql()}) ) as
total_count") )
->mergeBindings($sub)
->mergeBindings($sub1)
->select('total_count.total')
->get();
I have also tried this method. It works but gives collection but I need an integer value of total count
$result=$sub->unionAll($sub1);
dd($result->get());
In short I wanted to perform the accepted solution of that problem (How do I add two count(*) results together on two different tables?) in laravel.
You can use those codes :
$q = DB::tabel('toys')->select('toy_id','little_kid_id')->where('little_kid_id',900);
$q1 = DB::tabel('games')->select('game_id','little_kid_id')->where('little_kid_id',900)
$data = $q->union($q1)->count();
Don't forget Union require the tables must have the same columns so that I select the columns if your columns will not match each other then don't touch the select statement otherwise feel free to remove the codes
The second way is :
DB::table(DB::raw('(SELECT COUNT(*) as c FROM Toys WHERE little_kid_id = 900) t,(SELECT COUNT(*) as c1 from Games WHERE
little_kid_id = 900) t2'))->selectRaw('t.c+t2.c1 as
SumCount')->toSql(); //change toSql() to get() if you want to get
datas instead of sql code
You can try this. for more information look at https://laravel.com/docs/5.8/queries#raw-expressions. if you want to get request parameters by any user then be confirmed to prevent SQL Injection

What are the technical differences between "select * from table_name" and "select a.* from table_name a"?

This might be a basic question, but I can't find explanations after googling around.
Anyway, a short background story. I have this table that I don't have the permission to alter on DB2:
other_field | date_field | time_field
---------------------------------------
1 | 180101 | 101010
2 | 180102 | 202020
3 | 180103 | 303030
4 | 180104 | 404040
I tried to use:
select *, concat(date_field, time_field) as TIME
from Table_Name
My expected result is displaying something like this:
other_field | date_field | time_field | TIME
--------------------------------------------------------
1 | 180101 | 101010 | 180101101010
2 | 180102 | 102020 | 180102102020
3 | 180103 | 103030 | 180103103030
4 | 180104 | 104040 | 180104104040
But I can't use that query for some reason. It gave me an error ...Token , was not valid. Valid tokens: FROM INTO that basically said a comma (,) after * is invalid.
Then I tried tweaking it a little into:
select a.*, concat(a.date_field, a.time_field) as TIME
from Table_Name a
And it works!
I understand that Table_Name a are often used for joining tables, but I'm curious about the underlying mechanism.
What are the technical differences between using Table_Name and Table_Name a? And what is this a called?
Technically there will be no difference between the op of
SELECT * FROM TAB_NAME and SELECT a,* FROM TAB_NAME a.
Here you are just specifying alias name.
But you can understand the difference when you will try to fetch another column with * from TAB_NAME.
That means if you want to gate data as bellow
SELECT *,COL_1,COL2...
FROM TAB_NAME
or
SELECT *,CONCAT(...)
FROM TAB_NAME
or anything with * you must have to specify the alias name.
But the question is why? Let me try to explain,
As you know here SELECT * means you are trying to select all columns. So, * means "all" and if you are putting * after SELECT clause that means you already have given a command to your system to select all by passing a special character and after that your system can only expect FROM clause instead of any other thing. Because you already told your system/database to select all then there would be nothing left to select and hence your system will always wait for FROM clause. So it will throw an error each and every time.
BUT now the question is, how the bellow query will work internally
SELECT a.*,COL_1,COL2...
FROM TAB_NAME a
or
SELECT a.*,a.COL_1,a.COL2...
FROM TAB_NAME a
or
SELECT a.*,CONCAT(c1,c2)
FROM TAB_NAME a
or
SELECT a.*,CONCAT(a.c1,a.c2)
FROM TAB_NAME a
or anything else like that.
Here your system will understand that you are trying to select all from table a that means you may select any other col/function etc from either table a or any other table. That's the reason why your system/database will allow you to insert other col/func also after a, if required or you can use from clause as well after a.*
Db2 (LUW) 11.1 support this syntax
create table Table_Name (
other_field int not null primary key
, date_field date not null
, time_field time not null
)
;
insert into Table_Name values
(1,'2018-01-01', '10.10.10')
, (2,'2018-01-01', '20.20.20')
, (3,'2018-01-01', '13.13.13')
, (4,'2018-01-01', '14.14.14')
;
select *, timestamp(date_field, time_field) as TIME from Table_Name
;
which will return
OTHER_FIELD DATE_FIELD TIME_FIELD TIME
----------- ---------- ---------- ---------------------
1 2018-01-01 10:10:10 2018-01-01 10:10:10.0
2 2018-01-01 20:20:20 2018-01-01 20:20:20.0
3 2018-01-01 13:13:13 2018-01-01 13:13:13.0
4 2018-01-01 14:14:14 2018-01-01 14:14:14.0
BTW I took the liberty of using sensible data types for your example. Use DATE1, TIME and TIMESAMP (or TIMESTAMP(0)) for working with date and time values...

Empty Char in Where Clause?

I have the following table:
CREATE TABLE SOAUDIT
(SOU_USER CHAR(8 BYTE),
SOU_ORDREF CHAR(8 BYTE),
SOU_TYPE CHAR(1 BYTE),
SOU_DESC CHAR(50 BYTE))
There is a unique index defined on the first three columns (but no primary key, which is something we have no control over).
And in the table there are some records:
| SOU_USER | SOU_ORDREF | SOU_TYPE | SOU_DESC |
|----------|------------|----------|------------------|
| proust | | S | recherche |
| joyce | 12345678 | S | pelurious |
| orwell | 19841984 | T | doubleplusungood |
| camus | 34598798 | P | peiner |
On closer inspection it appears that the value in SOU_ORDREF for user 'proust' is an empty char string of 8 characters.
Now, what I need to be able to do is to query this table based on their unique values (which I will receive from a SQL Server database (just to complicate matters nicely). In the case of SOU_ORDREF the search value will be a blank field:
SELECT *
FROM SOAUDIT
WHERE (SOU_USER, TRIM(SOU_ORDREF), SOU_TYPE)
IN (('proust', null, 'S'))
This doesn't return the record I am looking for.
When I rewrite the query as following:
SELECT *
FROM SOAUDIT
WHERE (SOU_USER, SOU_TYPE)
IN (('proust', 'S'))
AND TRIM(sou_ordref) is null
Then I do get the desired record.
However, I want to be able to pass in more than one record into the WHERE clause so the second version doesn't really help.
Oracle -- by default -- treats empty strings and NULL as the same thing.
This can cause awkward behavior, because comparisons to NULL almost never return true. So a simple expression such as where sou_ordref = '' never returns true, because it is equivalent to where sou_ordref = NULL.
Here is one workaround:
SELECT *
FROM SOAUDIT
WHERE (SOU_USER, COALESCE(TRIM(SOU_ORDREF), ' '), SOU_TYPE) IN
( ('proust', ' ', 'S') )
Note that this replaces the empty string (NULL) with a space. It then compares the results to a space.
Try this way:
SELECT *
FROM test
WHERE SOU_USER = 'proust'
AND SOU_TYPE = 'S'
AND TRIM(sou_ordref) = ''
Since an empty char is different than NULL

Counting occurrences in a table

Lets say I want to count the total number of occurrences of a name contained within a string in a column and display that total next to all occurrences of that name in a new column beside it. For example, if I have:
Name | Home Address | Special ID
==================================
Frank | 152414 | aTRF342
Jane | 4342342 | rRFC432
Mary | 423432 | xTRF353
James | 32111111 | tLZQ399
May | 4302443 | 3TRF322
How would I count the occurrences of special tags like 'TRF', 'RFC', or 'LZQ' so the table looks like this:
Name | Home Address | Special ID | Occurrences
================================================
Frank | 152414 | aTRF342 | 3
Jane | 4342342 | rRFC432 | 1
Mary | 423432 | xTRF353 | 3
James | 32111111 | tLZQ399 | 1
May | 4302443 | 3TRF322 | 3
Currently using Access 2007. Is this even possible using a SQL query?
Using Access 2007, I stored your sample data in a table named tblUser1384831. The query below returns this result set.
Name Home Address Special ID special_tag Occurrences
---- ------------ ---------- ----------- -----------
Frank 152414 aTRF342 TRF 3
Jane 4342342 rRFC432 RFC 1
Mary 423432 xTRF353 TRF 3
James 32111111 tLZQ399 LZQ 1
May 4302443 3TRF322 TRF 3
Although your question has a vba tag, you don't need to use a VBA procedure for this. You can do it with SQL and the Mid() function.
SELECT
base.[Name],
base.[Home Address],
base.[Special ID],
base.special_tag,
tag_count.Occurrences
FROM
(
SELECT
[Name],
[Home Address],
[Special ID],
Mid([Special ID],2,3) AS special_tag
FROM tblUser1384831
) AS base
INNER JOIN
(
SELECT
Mid([Special ID],2,3) AS special_tag,
Count(*) AS Occurrences
FROM tblUser1384831
GROUP BY Mid([Special ID],2,3)
) AS tag_count
ON base.special_tag = tag_count.special_tag;
You would have to GROUP BY the substring of Special ID. In MS Access, you can read about how to compute substrings here.
The problem in your case is that your data in Special ID column does not follow a standard pattern, one which easy to extract via the substring function. You might need to use regular expressions to extract such values, and later apply the GROUP BY to them.
With MSSQL, Oracle, PostgreSQL you would be able to declare a stored procedure (example CLR function in MS SQL Server) that would do this for you. Not sure with MS Access.
you can do something like this:
select Name, [Home Address], [Special ID],
(select count(*) from [your table] where [Special ID] = RemoveNonAlphaCharacters([Special ID]) ) as Occurrences
from [your table]
auxiliar function (got from this link):
Create Function [dbo].[RemoveNonAlphaCharacters](#Temp VarChar(1000))
Returns VarChar(1000)
AS
Begin
While PatIndex('%[^a-z]%', #Temp) > 0
Set #Temp = Stuff(#Temp, PatIndex('%[^a-z]%', #Temp), 1, '')
Return #Temp
End
lets say your first table is called 'table_with_string'
the following code will show the occurance based on the first 3 charecters of string in Special ID column. since it is not clear how exactly you are passing the string to match
select tws.Name,tws.HomeAddress,tws.SpecialID,str_count.Occurrences from
table_with_string tws
left join
(select SpecialID,count(*) from table_with_string where specialID like(substring
(specialid,0,3))
group by specialId) as str_count(id,Occurrences)
on str_count.id=tws.SpecialID
I would suggest doing this explicitly as a join, so you are clear on how it works:
select tws.Name, tws.HomeAddress, tws.SpecialID, str_count.Occurrences
from table_with_string tws
join
(
select substring(spcecialid, 2, 3) as code, count(*) as Occurrences
from table_with_string tws
group by substring(spcecialid, 2, 3)
) s
on s.code = substring(tws.spcecialid, 2, 3)