I'm trying to pull data from a column called file name in which users have to upload the file name with only numericals for eg: 245654, 346595 , 700542. But in few cases i have also noted users where using special characters and aplhabets for e.g. 245654 / Abc, 654658-cgds,78345|ghj. I need to extract all such entries where along with numericals such special characters and alphabets are also noted.
You may use regex like here:
SELECT *
FROM yourTable
WHERE filename ~ '[^0-9]';
The above query will return any record whose file name has one or more non digit characters in it.
Related
How would I compare a file name with a naming convention thats saved as a windows wildcard pattern in a SQL column? For example:
I have a file HCLA_MCLA_20220308 and I want to check if there is a wildcard naming convention that matches this file name in my File_Naming_Convention column of my table. The match should be for "*HCLA_MCLA_**"
HCLA naming convention
All the naming conventions either use * or ?. The stars mean any number of characters before and/or any number or characters after. The ? means only 1 wild character in that position. Im not sure why there are two ** at the end of files in some cases.
My wildcard file naming convention column looks like this:
Section of column
Now I tried a query like below but this doesn't seem to be working. It appears that the wildcard tokens in the SQL table are windows wildcards. In my query I tried to replace them with SQL equivalents.
Select *
from MPM_FTP_Eligibility_Files_List
where replace(File_Naming_Convention,'*','%') like 'HCLA_MCLA_20220308'
Update:
Here is the fix to take care of both * and ?. Needed to swap positions between my file name and like pattern in my query, as jarlh accurately pointed out.
Select * from MPM_FTP_Eligibility_Files_List where 'FCS-CA25_1_20220301_1000N_Cigna_W_Trans.txt' like replace(replace(File_Naming_Convention,'*','%'),'?','_')
does anyone know how to include special characters like $ on a table name or if the table already exist with a $ on its name, how to ingest it on bigquery.
Thank you in advance,
That's not possible.
BigQuery column names must contain only letters, numbers, and undescores.
They must start with either a letter or an underscore.
Link to relevant doc: https://cloud.google.com/bigquery/docs/schemas#column_names
If a table contains special characters, you need to change its name before ingesting to BigQuery ($COLUMN_NAME -> DOLLAR_COLUMN_NAME, maybe?)
I am trying to capture all elements and store in separate column from the below string,seprated via underscores(campaign name for an advertisement) and then I wish to compare it with a master table having the true values to determine how accurate the data is being recorded.
eg: Input :
Expected output is :
My first element extraction was : REGEXP_EXTRACT(campaign_name, r"[^_+]{3}")) as parsed_campaign_agency
I only extracted first 3 letters because according to the naming convention(truth table), the agency name is made of only 3 letters.
Caveat: Some elements can have variable lengths too. eg. The third element "CrossBMC" could be 3 letters in length or more.
I am new to regex and the data lies in a SQL table(in BigQuery) so I thought it could be achieved via SQL's regex_extract but what I am having trouble is to extract all elements at once.
Any help is appreciated :)
If number of underscores constant and knows you can use SUBSTRING_INDEX like:
SELECT
SUBSTRING_INDEX(campaign_name,'_',1) first,
SUBSTRING_INDEX(SUBSTRING_INDEX(campaign_name,'_',2),'_',-1) second,
SUBSTRING_INDEX(SUBSTRING_INDEX(campaign_name,'_',3),'_',-1) third
FROM your_table;
Here you can try an example SQLize.online
I have a SQL database with tables imported from excel csv files. There are some tables that have field names with special characters like a space, /, $, #, % among others.
Is there a way to dynamically look at every field column name in every table and replace the special character with an underscore, some other string (for example: replace % with PCT or # with NBR or $ with CUR) or even delete the character all together?
Thank you in advance.
I am facing an issue while fetching the data via query from a redshift table. For example:
table name: test_users
column names: user_id, userName, userLastName
Now while creating the test_users table it converts the capital letter of the userName column to username and similar with userLastName which will be converted to userlastname.
I have found the way to convert the all columns to capital or in lowercase, but not in the way to get it as it is.
Unfortunately, AWS Redshift does not support case-sensitive identifiers at the time of writing (Feb 2020). And, while Redshift is based on PostgreSQL, AWS has heavily modified it to the point where many assumptions that would be correct for PostgreSQL 8 are not correct for Redshift.
The documentation at https://docs.aws.amazon.com/redshift/latest/dg/r_names.html explicitly states that it downcases identifiers. The relevant paragraph is below, with the critical sentence bolded:
Names identify database objects, including tables and columns, as well as users and passwords. The terms name and identifier can be used interchangeably. There are two types of identifiers, standard identifiers and quoted or delimited identifiers. Identifiers must consist of only UTF-8 printable characters. ASCII letters in standard and delimited identifiers are case-insensitive and are folded to lowercase in the database. In query results, column names are returned as lowercase by default. To return column names in uppercase, set the describe_field_name_in_uppercase configuration parameter to true.
To preserve case:
SET enable_case_sensitive_identifier TO true;
https://docs.aws.amazon.com/redshift/latest/dg/r_enable_case_sensitive_identifier.html
To force returned uppercase fields (for anyone else curious):
SET describe_field_name_in_uppercase TO on;
https://docs.aws.amazon.com/redshift/latest/dg/r_describe_field_name_in_uppercase.html