So I have to search for a bunch of names in a BigQuery table which I collect periodically in another dataset. The dataset is too large at this point containing almost ~60k names and I no longer can do
SELECT * FROM base.table WHERE name LIKE '%name1%' OR name LIKE '%name2%.....
As I tried it using a python script with:
SELECT * FROM base.table WHERE name LIKE({' OR '.join([f'ulv4.full_name LIKE %{name}%' for name in names])})
But the character limit for query exceeds for this many names. I tried looking at solutions like this and other answers to the same question but no answer seems to work for BigQuery Standard SQL. Any help in this regard is highly appreciated.
You should keep the names in another table and then join to it:
SELECT *
FROM base.table t1
WHERE EXISTS (SELECT 1 FROM other.table t2
WHERE t1.name LIKE CONCAT('%', t2.name, '%'));
Then, any record in the base.table would only match if it contains some substring name from the other table.
So I have to search for a bunch of names in a BigQuery table which I collect periodically in another dataset. The dataset is too large at this point containing almost ~60k names and I no longer can do
SELECT * FROM base.table WHERE name LIKE '%name1%' OR name LIKE '%name2%.....
As I tried it using a python script with:
SELECT * FROM base.table WHERE name LIKE({' OR '.join([f'ulv4.full_name LIKE %{name}%' for name in names])})
But the character limit for query exceeds for this many names. I tried looking at solutions like this and other answers to the same question but no answer seems to work for BigQuery Standard SQL. Any help in this regard is highly appreciated.
You should keep the names in another table and then join to it:
SELECT *
FROM base.table t1
WHERE EXISTS (SELECT 1 FROM other.table t2
WHERE t1.name LIKE CONCAT('%', t2.name, '%'));
Then, any record in the base.table would only match if it contains some substring name from the other table.
So there are two tables in a database. I have to find out whichever rows have discrepancy based on certain conditions (in couple of cases that's just equality checking between fields). I report ID of those rows.
The problem is to also include the reasons in another column as to why that ID is reported. Because an id can be fail multiple conditions (like mismatch on two fields), I just wanted to include all of those reasons in another column.
Basic idea is to append all the mismatches in another column.
I've looked at several SO questions but they don't exactly my use case. So now I'm thinking it's not possible with SQL.
I searched Google for "enter dynamic column values based on conditions sql", and hit : SQL Conditional column data return in a select statement : This adds a static column
I also learned it's possible to add another column in SELECT with dynamic content like this:
SELECT id, CASE
WHEN columnname "DEF" then "I" ELSE "YOU" newColumnName
FROM tableName
But I have not been able to find dynamic column value assignment and update SQL. That's the problem.
Expected results:
I just want to be able to concat all the cases "strings" which a record is applicable for.
Do this with the two tables.
So because I have two tables to work with I have to put these conditions in the WHERE sub-clause, and not in the SELECT one.
So, if for ID = 345, column "FOO_MAN" does not match between two tables, and column "BAR_TOO" also does not match between two tables, then?
Then I want my select clause to capture information like this:
ID | REASON
345 | FOO_MAN BAR_TWO
It's probably easier to build this type of query dynamically (e.g. using a stored procedure) based on the conditions you want to test, but here is a small example which shows how it can be done:
SELECT t1.id,
CONCAT_WS(' ',
CASE WHEN t1.foo != t2.foo THEN 'foo' END,
CASE WHEN t1.bar != t2.bar THEN 'bar' END
) AS reason
FROM t1
JOIN t2 ON t2.id = t1.id
WHERE t1.foo != t2.foo OR t1.bar != t2.bar
Output (for my demo on dbfiddle)
id reason
2 foo
4 bar
5 foo bar
I need to replace part of a string with a value from another database table. Actually I need to replace the userids with emails.
DB1.TABLE1
ID|EMAIL
1 |johndoe; janedoe;
2 |otherguy; johndoe;
DB2.TABLE2
ID|USERID |EMAIL
1 |johndoe |johndoe#test.com
2 |janedoe |janedoe#test.com
3 |otherguy|otherguy#test.com
my query
UPDATE
TABLE1
set
EMAIL = TABLE2.EMAIL
from
DB2.TABLE2
where
TABLE1.EMAIL = TABLE2.USERID
How can I specify the "part of the string" thing ?
There are a number of comments about changing your schema...which would be the best way forward.
It looks like what you are storing in table1.email is actually a list of UserId from table2. So you'll need to break out these ids in order to join to the tables together.
If you absolutely must follow this path, then there are existing Q+As on the site that will help you:
(I've taken a leap of faith that you are using SQL server ... but if you search I'm sure you can find similar answers for other RDBMSs)
Turning a Comma Separated string into individual rows
and
Multiple rows to one comma separated value
I guess you need the following
UPDATE TABLE1
SET EMAIL = (
SELECT TABLE2.EMAIL
FROM TABLE2
WHERE TABLE1.EMAIL LIKE TABLE2.USERID + '%');
demo
I am trying to find a way, if possible, to use IN and LIKE together. What I want to accomplish is putting a subquery that pulls up a list of data into an IN statement. The problem is the list of data contains wildcards. Is there any way to do this?
Just something I was curious on.
Example of data in the 2 tables
Parent table
ID Office_Code Employee_Name
1 GG234 Tom
2 GG654 Bill
3 PQ123 Chris
Second table
ID Code_Wildcard
1 GG%
2 PQ%
Clarifying note (via third-party)
Since I'm seeing several responses which don't seems to address what Ziltoid asks, I thought I try clarifying what I think he means.
In SQL, "WHERE col IN (1,2,3)" is roughly the equivalent of "WHERE col = 1 OR col = 2 OR col = 3".
He's looking for something which I'll pseudo-code as
WHERE col IN_LIKE ('A%', 'TH%E', '%C')
which would be roughly the equivalent of
WHERE col LIKE 'A%' OR col LIKE 'TH%E' OR col LIKE '%C'
The Regex answers seem to come closest; the rest seem way off the mark.
I'm not sure which database you're using, but with Oracle you could accomplish something equivalent by aliasing your subquery in the FROM clause rather than using it in an IN clause. Using your example:
select p.*
from
(select code_wildcard
from second
where id = 1) s
join parent p
on p.office_code like s.code_wildcard
In MySQL, use REGEXP:
WHERE field1 REGEXP('(value1)|(value2)|(value3)')
Same in Oracle:
WHERE REGEXP_LIKE(field1, '(value1)|(value2)|(value3)')
Do you mean somethign like:
select * FROM table where column IN (
SELECT column from table where column like '%%'
)
Really this should be written like:
SELECT * FROM table where column like '%%'
Using a sub select query is really beneficial when you have to pull records based on a set of logic that you won't want in the main query.
something like:
SELECT * FROM TableA WHERE TableA_IdColumn IN
(
SELECT TableA_IdColumn FROM TableB WHERE TableA_IDColumn like '%%'
)
update to question:
You can't combine an IN statement with a like statement:
You'll have to do three different like statements to search on the various wildcards.
You could use a LIKE statement to obtain a list of IDs and then use that in the IN statement.
But you can't directly combine IN and LIKE.
Perhaps something like this?
SELECT DISTINCT
my_column
FROM
My_Table T
INNER JOIN My_List_Of_Value V ON
T.my_column LIKE '%' + V.search_value + '%'
In this example I've used a table with the values for simplicity, but you could easily change that to a subquery. If you have a large list (like tens of thousands) then performance might be rough.
select *
from parent
where exists( select *
from second
where office_code like trim( code_wildcard ) );
Trim code_wildcard just in case it has trailing blanks.
You could do the Like part in a subquery perhaps?
Select * From TableA Where X in (Select A from TableB where B Like '%123%')
tsql has the contains statement for a full-text-search enabled table.
CONTAINS(Description, '"sea*" OR "bread*"')
If I'm reading the question correctly, we want all Parent rows that have an Office_code that matches any Code_Wildcard in the "Second" table.
In Oracle, at least, this query achieves that:
SELECT *
FROM parent, second
WHERE office_code LIKE code_wildcard;
Am I missing something?