I have a table with aggregated columns like this in a SQL Server. The goal is: Show me any row that contains Street A or B.
Name
Adresses
Mike
Street A, Street B, Street C
Smith
Street B
Jimmy
Street C
Declare #street table (col nvarchar (50) )
INSERT INTO #street Values ('Street A'), ('Street B' )
SELECT *
FROM Table
WHERE Adresses like '%' + #street + '%'
SELECT *
FROM Table
WHERE Adresses = ( SELECT * FROM #street )
SELECT *
FROM Table
WHERE STRING_SPLIT(Adresses,',') in ( SELECT * FROM #street )
It does not work. I do not get results. The results should be like this:
Name
Adresses
Mike
Street A, Street B, Street C
Smith
Street B
You should get rid of this bad structure and store the data in a better form in future.
Anyway, following your intention to use STRING_SPLIT for that, this would do:
SELECT name, adresses
FROM yourtable
WHERE EXISTS
(SELECT *
FROM STRING_SPLIT(adresses, ',')
WHERE value IN ('Street A', 'Street B'));
You should read the documentation, that's explained there.
Testing with your sample data succeeds: db<>fiddle
I am sure you can use some "contains" functionality on column adresses. Depends if the column is a varchar/text like column, or a array like column.
However, i would instead try to solve this in a more conseptual level.
In relational databases, this would be referred as a "many to one" relationship.
One adress may have only one Persons related to them, while the Person may have listed many Adresses.
Therefore i would make seperate tables. Then you can easily Search the Adresses table for A and B and find related foreign keys to Persons table.
Otherwise, you can insist on using "contains" functionality, but this might get computationally expensive.
I have a table with a field of "content" and a field of a "country".
and I need to check if the country name appears in the text's content.
content:
"La baguette est originaire de France"
"החומוס הטוב ביותר הוא בישראל"
"Michael Phelps earned his final gold medal when the United States
team won the 4×100-meter medley relay."
country:
France
ישראל
United States
using like clause seems to be the right option, but I don't know how to use the values in the country field for this type of check.
Naive approach with dictionary table:
CREATE TABLE country(id INT, name NVARCHAR(100));
INSERT INTO country(id, name) VALUES (1, 'United States'), (2, 'France'), (3, N'ישראל');
SELECT *
FROM mytable m
JOIN country c
ON m.content LIKE '%' + c.name + '%';
Of course it is not bullet-proof solution, because country could be different depending on language and noun case.
EDIT:
If country is present on the same row:
SELECT *,
CASE WHEN m.content LIKE '%' + m.country + '%' THEN 'Present' ELSE 'None' END
FROM mytable m
As you just need to check the data from other table you can just have a subquery or as per your data row by row to do it without subquery like
SELECT Case
when Content Like '%'+country+'%'
then "Yes" else "No" end, content,
country from table ;
or via subquery
SELECT
content,
country from table where content IN
(Select '%'+country+'%' from table )
;
I think you want:
where Content like N'%' + country + N'%'
This would return rows where the country is in the content. You seem to have multiple character sets, so I think nvarchar() is appropriate.
Attached answer using PostgreSQL.
I used the position function but there are similar functions for other databases (instr for Oracle, for example).
select * from $table_name$ a
where position(country in content) > 0;
Most compare two or more columns for the same value between two tables. I need to compare two columns in a table where relative to another column with 3 owner values used. This will be 3 different first names.
I have seen table compares, but this is in one table and data is separated already by a column value. The table is thousands of rows. Owner column is only three different names.
So Joe, Sam and Jim have rows of names. When any of these two people have a common first and last name listed in the row, they will be listed as output with other column data like location and zip. So if Joe and Sam have two common names, the output will list:
Owner,firstname,lastname,location,zip
Joe,Bob,Smith,Dallas,37377
Sam,Bob,Smith,Dallas,37377
Jim had no Bob,Smith,Dallas,37377 so it is not listed in the group with Joe and Sam. In summation, the results will be a few hundred lines of these 3 owners grouped where a common name is found. I will need to use percentages in the query in case some one uses Bobby or Smiths. Therefore a name like Smithson will show up but I will adjust.
I had this all typed out and the page sees it as code so I apologize I had to abbreviate.
The most straightforward thing to use is a self-join, though an EXISTS may be quicker.
First, the self-join:
--note DISTINCT here because if Joe, Bob, and Sam all have records with the name we don't want duplicates
select DISTINCT f.owner, f.firstname, f.lastname, f.location, f.zip
from tablename f
join tablename s on f.firstname = s.firstname and f.lastname = s.lastname and f.owner <> s.owner
Then, with an EXISTS:
select f.owner, f.firstname, f.lastname, f.location, f.zip
from tablename f
WHERE EXISTS(select 1 from tablename s WHERE s.firstname = f.firstname and s.lastname = f.lastname and s.owner <> f.owner)
Of course, if instead of equality you want 'Smith' and 'smithson' to match, you can replace the equality comparison with something like: (f.firstname ilike (s.firstname||'%') OR s.firstname ilike (f.firstname||'%')) you can use that (or any other comparison)
What would be the best approach for comparing the following set of strings in SQL Server?
Purpose: The main purpose of this Stored Procedure is to compare a set of input Names of Customers to names that Exist in the Customer database for a specific accounts. If there is a difference between the input name and the name in the Database this should trigger the updating of the Customer Database with the new name information.
Conditions:
Format of Input: FirstName [MiddleName] LastName
Format of Value in Database: LastName, FirstName MiddleName
The complication arises when names like this are presented,
Example:
Input: Dr. John A. Mc Donald
Database: Mc Donald, Dr. John A.
For last names that consist of 2 or more parts what logic would have to be put into place
to ensure that the lastname in the input is being compared to the lastname in the database and likewise for the first name and middle name.
I've thought about breaking the database values up into a temp HASH table since I know that everything before the ',' in the database is the last name. I could then check to see if the input contains the lastname and split out the FirstName [MiddleName] from it to perform another comparison to the database for the values that come after the ','.
There is a second part to this however. In the event that the input name has a completely New last name (i.e. if the name in the database is Mary Smith but the updated input name is now Mary Mc Donald). In this case comparing the database value of the last name before the ',' to the input name will result in no match which is correct, but at this point how does the code know where the last name even begins in the input value? How does it know that her Middle name isn't MC and her last name Donald?
Has anyone had to deal with a similar problem like this before? What solutions did you end up going with?
I greatly appreciate your input and ideas.
Thank you.
Realistically, it's extremely computationally difficult (if not impossible) to know if a name like "Mary Jane Evelyn Scott" is first-middle-last1-last2, first1-first2-middle-last, first1-first2-last1-last2, or some other combination... and that's not even getting into cultural considerations...
So personally, I would suggest a change in the data structure (and, correspondingly, the application's input fields). Instead of a single string for name, break it into several fields, e.g.:
FullName{
title, //i.e. Dr., Professor, etc.
firstName, //or given name
middleName, //doesn't exist in all countries!
lastName, //or surname
qualifiers //i.e. Sr., Jr., fils, D.D.S., PE, Ph.D., etc.
}
Then the user could choose that their first name is "Mary", their middle name is "Jane Evelyn", and their last name is "Scott".
UPDATE
Based on your comments, if you must do this entirely in SQL, I'd do something like the following:
Build a table for all possible combinations of "lastname, firstname [middlename]" given an input string "firstname [middlename] lastname"
Run a query based on the join of your original data and all possible orderings.
So, step 1. would take the string "Dr. John A. Mc Donald" and create the table of values:
'Donald, Dr. John A. Mc'
'Mc Donald, Dr. John A.'
'A. Mc Donald, Dr. John'
'John A. Mc Donald, Dr.'
Then step 2. would search for all occurrences of any of those strings in the database.
Assuming MSSQL 2005 or later, step 1. can be achieved using some recursive CTE, and a modification of a method I've used to split CSV strings (found here) (SQL isn't the ideal language for this form of string manipulation...):
declare #str varchar(200)
set #str = 'Dr. John A. Mc Donald'
--Create a numbers table
select [Number] = identity(int)
into #Numbers
from sysobjects s1
cross join sysobjects s2
create unique clustered index Number_ind on #Numbers(Number) with IGNORE_DUP_KEY
;with nameParts as (
--Split the name string at the spaces.
select [ord] = row_number() over(order by Number),
[part] = substring(fn1, Number, charindex(' ', fn1+' ', Number) - Number)
from (select #str fn1) s
join #Numbers n on substring(' '+fn1, Number, 1) = ' '
where Number<=Len(fn1)+1
),
lastNames as (
--Build all possible lastName strings.
select [firstOrd]=ord, [lastOrd]=ord, [lastName]=cast(part as varchar(max))
from nameParts
where ord!=1 --remove the case where the whole string is the last name
UNION ALL
select firstOrd, p.ord, l.lastName+' '+p.part
from lastNames l
join nameParts p on l.lastOrd+1=p.ord
),
firstNames as (
--Build all possible firstName strings.
select [firstOrd]=ord, [lastOrd]=ord, [firstName]=cast(part as varchar(max))
from nameParts
where ord!=(select max(ord) from nameParts) --remove the case where the whole string is the first name
UNION ALL
select p.ord, f.lastOrd, p.part+' '+f.firstName
from firstNames f
join nameParts p on f.firstOrd-1 = p.ord
)
--Combine for all possible name strings.
select ln.lastName+', '+fn.firstName
from firstNames fn
join lastNames ln on fn.lastOrd+1=ln.firstOrd
where fn.firstOrd=1
and ln.lastOrd = (select max(ord) from nameParts)
drop table #Numbers
Since I had my share of terrible experience with data from third parties, it is almost guaranteed that the input data will contain lots of garbage not following the specified format.
When trying to match data multipart string data like in your case, I preprocessed both input and our data into something I called "normalized string" using the following method.
strip all non-ascii chars (leaving language-specific chars like "č" intact)
compact spaces (replace multiple spaces with single one)
lower case
split into words
remove duplicates
sort alphabetically
join back to string separated by dashes
Using you sample data, this function would produce:
Dr. John A. Mc Donald ->
a-donald-dr-john-mc Mc Donald, Dr.
John A.-> a-donald-dr-john-mc
Unfortunaly it's not 100% bulletproof, there are cases where degenerated inputs produce invalid matches.
Your name field is bad in the database. Redesign and get rid of it. If you havea a first name, middlename, lastname, prefix and suffix sttructure, you can hava computed filed that has the structure you are using. But it is a very poor way to store data and your first priority should be to stop using it.
Since you have a common customer Id why aren't you matching on that instead of name?
I know that that is not a question... erm anyway HERE is the question.
I have inherited a database that has 1(one) table in that looks much like this. Its aim is to record what species are found in the various (200 odd) countries.
ID
Species
Afghanistan
Albania
Algeria
American Samoa
Andorra
Angola
....
Western Sahara
Yemen
Zambia
Zimbabwe
A sample of the data would be something like this
id Species Afghanistan Albania American Samoa
1 SP1 null null null
2 SP2 1 1 null
3 SP3 null null 1
It seems to me this is a typical many to many situation and I want 3 tables.
Species, Country, and SpeciesFoundInCountry
The link table (SpeciesFoundInCountry) would have foreign keys in both the species and Country tables.
(It is hard to draw the diagram!)
Species
SpeciesID SpeciesName
Country
CountryID CountryName
SpeciesFoundInCountry
CountryID SpeciesID
Is there a magic way I can generate an insert statement that will get the CountryID from the new Country table based on the column name and the SpeciesID where there is a 1 in the original mega table?
I can do it for one Country (this is a select to show what I want out)
SELECT Species.ID, Country.CountryID
FROM Country, Species
WHERE (((Species.Afghanistan)=1)) AND (((Country.Country)="Afghanistan"));
(the mega table is called species)
But using this strategy I would need to do the query for each column in the original table.
Is there a way of doing this in sql?
I guess I can OR a load of my where clauses together and write a script to make the sql, seems inelegant though!
Any thoughts (or clarification required)?
I would use a script to generate all the individual queries, since this is a one-off import process.
Some programs such as Excel are good at mixing different dimensions of data (comparing column names to data inside rows) but relational databases rarely are.
However, you might find that some systems (such as Microsoft Access, surprisingly) have convenient tools which you can use to normalise the data. Personally I'd find it quicker to write the script but your relative skills with Access and scripting might be different to mine.
Why do you want to do it in SQL? Just write a little script that does the conversion.
When I run into these I write a script to do the conversion rather than trying to do it in SQL. It is typically much faster and easier for me. Pick any language you are comfortable with.
If this was SQL Server, you'd use the Unpivot commands, but looking at the tag you assigned it's for access - am I right?
Although there is a pivoting command in access, there is no reverse statement.
Looks like it can be done with a complex join. Check this interesting article for a lowdown on how to unpivot in a select command.
You're probably going to want to create replacement tables in place. The script sort of depends on the scripting language you have available to you, but you should be able to create the country ID table simply by listing the columns of the table you have now. Once you've done that, you can do some string substitutions to go through all of the unique country names and insert into the speciesFoundInCountry table where the given country column is not null.
You could probably get clever and query the system tables for the column names, and then build a dynamic query string to execute, but honestly that will probably be uglier than a quick script to generate the SQL statements for you.
Hopefully you don't have too much dynamic SQL code that accesses the old tables buried in your codebase. That could be the really hard part.
In SQL Server this will generate your custom select you demonstrate. You can extrapolate to an insert
select
'SELECT Species.ID, Country.CountryID FROM Country, Species WHERE (((Species.' +
c.name +
')=1)) AND (((Country.Country)="' +
c.name +
'"))'
from syscolumns c
inner join sysobjects o
on o.id = c.id
where o.name = 'old_table_name'
As with the others I would most likely just do it as a one time quick fix in whatever manner works for you.
With these types of conversions, they are one off items, quick fixes, and the code doesn't have to be elegant, it just has to work. For these types of things I have done it many ways.
If this is SQL Server, you can use the sys.columns table to find all of the columns of the original table. Then you can use dynamic SQL and the pivot command to do what you want. Look those up online for syntax.
I would definitely agree with your suggestion of writing a small script to produce your SQL with a query for every column.
In fact your script could have already been finished in the time you've spent thinking about this magical query (that you would use only one time and then throw away, so what's the use in making it all magicy and perfect)
Sorry, but the bloody posting parser removed the whitespace and formatting on my post. It makes it a log harder to read.
#stomp:
Above the box where you type the answer, there are several buttons. The one that is 101010 is a code sample. You select all your text that is code, and then click that button. Then it doesn't get messed with much.
cout>>"I don't know C"
cout>>"Hello World"
I would use a Union query, very roughly:
Dim db As Database
Dim tdf As TableDef
Set db = CurrentDb
Set tdf = db.TableDefs("SO")
strSQL = "SELECT ID, Species, """ & tdf.Fields(2).Name _
& """ AS Country, [" & tdf.Fields(2).Name & "] AS CountryValue FROM SO "
For i = 3 To tdf.Fields.Count - 1
strSQL = strSQL & vbCrLf & "UNION SELECT ID, Species, """ & tdf.Fields(i).Name _
& """ AS Country, [" & tdf.Fields(i).Name & "] AS CountryValue FROM SO "
Next
db.CreateQueryDef "UnionSO", strSQL
You would then have a view that could be appended to your new design.
When I read the title 'bad BAD database design', I was curious to find out how bad it is. You didn't disappoint me :)
As others mentioned, a script would be the easiest way. This can be accomplished by writing about 15 lines of code in PHP.
SELECT * FROM ugly_table;
while(row)
foreach(row as field => value)
if(value == 1)
SELECT country_id from country_table WHERE country_name = field;
if(field == 'Species')
SELECT species_id from species_table WHERE species_name = value;
INSERT INTO better_table (...)
Obviously this is pseudo code and will not work as it is. You can also populate the countries and species table on the fly by adding insert statements here.
Sorry, I've done very little Access programming but I can offer some guidance which should help.
First lets walk through the problem.
It is assumed that you will typically need to generate multiple rows in SpeciesFoundInCountry for every row in the original table. In other words species tend to be in more then one country. This is actually easy to do with a Cartesian product, a join with no join criteria.
To do a Cartesian product you will need to create the Country table. The table should have the country_id from 1 to N (N being the number of unique countries, 200 or so) and country name. To make life easy just use the numbers 1 to N in column order. That would make Afghanistan 1 and Albania 2 ... Zimbabwe N. You should be able to use the system tables to do this.
Next create a table or view from the original table which contains the species and a sting with a 0 or 1 for each country. You will need to convert the null, not null to a text 0 or 1 and concatenate all of the values into a single string. A description of the table and a text editor with regular expressions should make this easy. Experiment first with a single column and once that's working edit the create view/insert with all of the columns.
Next join the two tables together with no join criteria. This will give you a record for every species in every country, you're almost there.
Now all you have to do is filter out the records which are not valid, they will have a zero in the corresponding location in the string. Since the country table's country_code column has the substring location all you need to do is filter out the records where it's 0.
where substring(new_column,country_code) = '1'
You will still need to create the species table and join to that
where a.species_name = b.species_name
a and b are table aliases.
Hope this help
OBTW,
If you have queries that already run against the old table you will need to create a view which replicates the old tables using the new tables. You will need to do a group by to denormalize the tables.
Tell your users that the old table/view will not be supported in the future and all new queries or updates to older queries will have to use the new tables.
If I ever have to create a truckload of similar SQL statements and execute all of them, I often find Excel is very handy. Take your original query. If you have a country list in column A and your SQL statement in column B, formated as text (in quotes) with cell references inserted where the country appears in the sql
e.g. ="INSERT INTO new_table SELECT ... (species." & A1 & ")= ... ));"
then just copy the formula down to create 200 different SQL statements, copy/paste the column to your editor and hit F5. You can of course do this with as many variables as you want.
When I've been faced with similar problems, I've found it convenient to generate a script that generates SQL scripts. Here's the sample you gave, abstracted to use %PAR1% in place of Afghanistan.
SELECT Species.ID, Country.CountryID
FROM Country, Species
WHERE (((Species.%PAR1%)=1)) AND (((Country.Country)="%PAR1%"))
UNION
Also the key word union has been added as a way to combine all the selects.
Next, you need a list of countries, generated from your existing data:
Afghanistan
Albania
.
,
.
Next you need a script that can iterate through the country list, and for each iteration,
produce an output that substitutes Afghanistan for %PAR1% on the first iteration, Albania for the second iteration and so on. The algorithm is just like mail-merge in a word processor. It's a little work to write this script. But, once you have it, you can use it in dozens of one-off projects like this one.
Finally, you need to manually change the last "UNION" back to a semicolon.
If you can get Access to perform this giant union, you can get the data you want in the form you want, and insert it into your new table.
I would make it a three step process with a slight temporary modification to your SpeciesFoundInCountry table. I would add a column to that table to store the Country name. Then the steps would be as follows.
1) Create/Run a script that walks columns in the source table and creates a record in SpeciesFoundInCountry for each column that has a true value. This record would contain the country name.
2) Run a SQL statement that updates the SpeciesFoundInCountry.CountryID field by joining to the Country table on Country Name.
3) Cleanup the SpeciesFoundInCountry table by removing the CountryName column.
Here is a little MS Access VB/VBA pseudo code to give you the gist
Public Sub CreateRelationshipRecords()
Dim rstSource as DAO.Recordset
Dim rstDestination as DAO.Recordset
Dim fld as DAO.Field
dim strSQL as String
Dim lngSpeciesID as Long
strSQL = "SELECT * FROM [ORIGINALTABLE]"
Set rstSource = CurrentDB.OpenRecordset(strSQL)
set rstDestination = CurrentDB.OpenRecordset("SpeciesFoundInCountry")
rstSource.MoveFirst
' Step through each record in the original table
Do Until rstSource.EOF
lngSpeciesID = rstSource.ID
' Now step through the fields(columns). If the field
' value is one (1), then create a relationship record
' using the field name as the Country Name
For Each fld in rstSource.Fields
If fld.Value = 1 then
with rstDestination
.AddNew
.Fields("CountryID").Value = Null
.Fields("CountryName").Value = fld.Name
.Fields("SpeciesID").Value = lngSpeciesID
.Update
End With
End IF
Next fld
rstSource.MoveNext
Loop
' Clean up
rstSource.Close
Set rstSource = nothing
....
End Sub
After this you could run a simple SQL statement to update the CountryID values in the SpeciesFoundInCountry table.
UPDATE SpeciesFoundInCountry INNER JOIN Country ON SpeciesFoundInCountry.CountryName = Country.CountryName SET SpeciesFoundInCountry.CountryID = Country.CountryID;
Finally, all you have to do is cleanup the SpeciesFoundInCountry table by removing the CountryName column.
****SIDE NOTE: I have found it usefull to have country tables that also include the ISO abbreviations (country codes). Occassionally they are used as Foreign Keys in other tables so that a join to the Country table does not have to be included in queries.
For more info: http://en.wikipedia.org/wiki/Iso_country_codes
This is (hopefully) a one-off exercise, so an inelegant solution might not be as bad as it sounds.
The problem (as, I'm sure you're only too aware!) is that at some point in your query you've got to list all those columns. :( The question is, what is the most elegant way to do this? Below is my attempt. It looks unwieldy because there are so many columns, but it might be what you're after, or at least it might point you in the right direction.
Possible SQL Solution:
/* if you have N countries */
CREATE TABLE Country
(id int,
name varchar(50))
INSERT Country
SELECT 1, 'Afghanistan'
UNION SELECT 2, 'Albania',
UNION SELECT 3, 'Algeria' ,
UNION SELECT 4, 'American Samoa' ,
UNION SELECT 5, 'Andorra' ,
UNION SELECT 6, 'Angola' ,
...
UNION SELECT N-3, 'Western Sahara',
UNION SELECT N-2, 'Yemen',
UNION SELECT N-1, 'Zambia',
UNION SELECT N, 'Zimbabwe',
CREATE TABLE #tmp
(key varchar(N),
country_id int)
/* "key" field needs to be as long as N */
INSERT #tmp
SELECT '1________ ... _', 'Afghanistan'
/* '1' followed by underscores to make the length = N */
UNION SELECT '_1_______ ... ___', 'Albania'
UNION SELECT '__1______ ... ___', 'Algeria'
...
UNION SELECT '________ ... _1_', 'Zambia'
UNION SELECT '________ ... __1', 'Zimbabwe'
CREATE TABLE new_table
(country_id int,
species_id int)
INSERT new_table
SELECT species.id, country_id
FROM species s ,
#tmp t
WHERE isnull( s.Afghanistan, ' ' ) +
isnull( s.Albania, ' ' ) +
... +
isnull( s.Zambia, ' ' ) +
isnull( s.Zimbabwe, ' ' ) like t.key
My Suggestion
Personally, I would not do this. I would do a quick and dirty solution like the one to which you allude, except that I would hard-code the country ids (because you're only going to do this once, right? And you can do it right after you create the country table, so you know what all the IDs are):
INSERT new_table SELECT Species.ID, 1 FROM Species WHERE Species.Afghanistan = 1
INSERT new_table SELECT Species.ID, 2 FROM Species WHERE Species.Albania= 1
...
INSERT new_table SELECT Species.ID, 999 FROM Species WHERE Species.Zambia= 1
INSERT new_table SELECT Species.ID, 1000 FROM Species WHERE Species.Zimbabwe= 1