Excluding Data from SQL Results based on DATE - sql

I have the following query built: My problem is that if I get a date that is past today's date in the policy_exp_date I need to exclude everything that is insplans or clientplans related and still return everything else specified.
To summarize: If Policy_Exp_DATE > TODAY then pull * else exclude insplans.* or clientplans.*
select clients.CLIENT_ID_1 as SourceMRN,
'' AS HMOMEMBER,
'' as IDXMRN,
replace (clients.CLIENT_ID_2,'-','') as PatientSSN,
clients.LAST_NAME as LASTNAME,
clients.FIRST_NAME as FIRSTNAME,
clients.MIDDLE_NAME as MIDDLENAME,
'' as PTMothersMaiden,
convert (varchar,clients.BIRTH_DATE,112) as DOB,
clients.sex as GENDER,
'' as PTNICKNAME,
'' as RACE,
clients.ADDRESS_1,
clients.ADDRESS_2,
clients.CITY,
clients.STATE,
clients.ZIP_CODE,
'' as FLD18,
'' as FLD19,
clients.PHONE_1 as PTHOMEPHONE,
'' as PTCELL,
'' as PTEMAIL,
'' as PTDAYPHONE,
(case WHEN [CLIENTS].LANGUAGE_ID LIKE '0' THEN 'ENGLISH'
when [clients].LANGUAGE_ID like '1' then 'SPANISH' END) as LANGUAGE,
'' as MaritalStatus,
'' as Religion,
'' as Ethnicity,
'' as PtExpDate,
'' as PtExpIndex,
'' as LASTUPDATEDATE,
convert (varchar,clientlog.FNDATE,112) as LASTUPDATEDATE,
'' as PCPNAME,
'' as VIP,
'' as FLD33,
'' as PREFCONTACT,
'' as FLD35,
'' as GUARANTORNAME,
'' AS GUARADDRESS1,
'' AS GUARADDRESS2,
'' AS GUARCITY,
'' AS GUARSTATE,
'' AS GUARZIP,
'' AS GUARHOMEPHONE,
'' AS GUARWORKPHONE,
'' AS GUARRELATIONSHIP,
'' as GUARSSN,
'' as GUAREMPNAME,
'' as FLD47,
'' AS GUAREMPADDRESS1,
'' as GUAREMPADDRESS2,
'' as GUAREMPCITY,
'' AS GUAREMPSTATE,
'' AS GUAREMPZIP,
'' AS GUAREMPPHONE,
'' AS GUAREMPSTATUS,
'' AS FLD55,
'' AS FLD56,
'' AS FLD57,
'' AS FLD58,
'' AS FLD59,
'' AS FLD60,
'' AS FLD61,
'' AS FLD62,
'' AS FLD63,
'' AS FLD64,
'' AS FLD65,
insplans.DESCRIPTION_UPPER AS FINANCIAL_CLASS,
'' as PAYERID#,
insplans.DESCRIPTION_UPPER as PAYERNAME,
'' as PAYERADDRESS2,
'' AS PAYERCITY,
'' AS PAYERSTATE,
'' AS PAYERZIP,
'' AS PAYERCONTACTNBR,
clientplans.GROUP_NO AS GROUPNUMBER,
convert (varchar,clientplans.START_DATE,112) as POLICY_EFF_DATE,
convert (varchar,clientplans.END_DATE,112) as POLICY_EXP_DATE,
clientplans.SEQUENCE_NUM,
(clients.LAST_NAME +'^' + clients.FIRST_NAME)as INSURED_NAME,
'' as INSURED_DOB,
'' AS INSURED_ADD1,
'' AS INSURED_ADD2,
'' AS INSURED_CITY,
'' AS INSURED_STATE,
'' AS INSURED_ZIP,
'' AS INSURED_PLAN_NBR,
clientplans.POLICY_NO AS POLICY_NBR,
'' AS INSURED_GENDER,
'' AS INSURED_HOMEPHONE,
'' AS INSURED_EMP_NAME,
'' AS INSURED_RELATION,
'' AS ACCIDENT_DATE,
'' AS ACCIDENT_CODE,
'' AS ACCIDENT_STATE,
'' AS FLD94,
'' AS CLINICAL_REMINDER,
'' AS APPT_REMINDER,
'' AS INS_INST,
'' AS SET_ID,
'' AS COPAY,
'CC' AS SOURCE_PREFIX
FROM project.dbo.CLIENTS
join CLIENTPLANS
on clients.RECORD_ID = clientplans.CLIENT_ID
join INSPLANS
on insplans.RECORD_ID = clientplans.PLAN_ID
join (SELECT MAX(fndate) FNDATE,client_id
from CLIENTLOG
group by CLIENT_ID)CLIENTLOG
on clientlog.CLIENT_ID = clients.RECORD_ID
where clients.RECORD_ID = clientplans.CLIENT_ID

Would something like work?
FROM project.dbo.CLIENTS
join (SELECT MAX(fndate) FNDATE,client_id
from CLIENTLOG
group by CLIENT_ID)CLIENTLOG
on clientlog.CLIENT_ID = clients.RECORD_ID
Left Outer join CLIENTPLANS
on clients.RECORD_ID = clientplans.CLIENT_ID
And clientplans.END_DATE > GetDate()
Left Outer join INSPLANS
on clientplans.PLAN_ID = insplans.RECORD_ID
Also make sure to remove the where clause or you will be removing a lot of the records.

Related

SQL Nested Replace Order

Dumb question, but can't seem to find a quick answer... I am creating UDF to replace name prefix with a nested replace... something like this:
REPLACE(
REPLACE(
REPLACE(
REPLACE(
REPLACE(
REPLACE(
REPLACE(
REPLACE(
#Name
, 'Mr ', '')
, 'Mrs ', '')
, 'Ms ', '')
, 'Dr ', '')
, 'Mr. ', '')
, 'Mrs. ', '')
, 'Ms. ', '')
, 'Dr. ', '')
but want to replace those containing a '.' BEFORE those without. What's the order/hierarchy of nested replace? TIA, Danny
Your inner most Replace will always run first on the character string. So from inner to outer is the order in which they will be executed. I suggest though trying a more elegant approach perhaps through the use of regex and case statements so it might be easier to maintain rather than just having a ton of nested replaces which don't necessarily all do work.
I suggest using a function like this in SQL:
CREATE FUNCTION [dbo].[ReplaceNamePrefix]
(#value NVARCHAR(50)
)
RETURNS NVARCHAR(50)
AS
BEGIN
DECLARE #RETURN NVARCHAR(50);
SET #RETURN = #value;
SET #RETURN = REPLACE(#RETURN, 'Mrs.', '');
SET #RETURN = REPLACE(#RETURN, 'Mrs', '');
SET #RETURN = REPLACE(#RETURN, 'Mr.', '');
SET #RETURN = REPLACE(#RETURN, 'Mr', '');
SET #RETURN = REPLACE(#RETURN, 'Ms.', '');
SET #RETURN = REPLACE(#RETURN, 'Ms', '');
SET #RETURN = REPLACE(#RETURN, 'Dr. ', '');
SET #RETURN = REPLACE(#RETURN, 'Dr', ''); ');
SET #RETURN = RTRIM(LTRIM(#RETURN));
RETURN #RETURN;
END;
Use the following command to get your goal:
SELECT [dbo].[ReplaceNamePrefix]('Dr. Danny Dennison');
the result is: Danny Dennison
But if you insist on using your own code, use the following:
SELECT REPLACE( REPLACE( REPLACE( REPLACE( REPLACE( REPLACE( REPLACE( REPLACE( 'MR Danny Dennison', 'Mrs.', '' ), 'Mrs', '' ), 'Mr.', '' ), 'Mr', '' ), 'Ms.', '' ), 'Ms', '' ), 'Dr. ', '' ), 'Dr', '' );

How to append data to a JSON object during a select?

I'm in the progress of migrating data from an old system. I have some boolean columns, which can be True, False or Null. Each column has a certain meaning. In the new database, I use a JSON Data object. What I would like to achieve is this:
OldTable:
ValueA, ValueB, ValueC, ValueD
Null , True , False , True
I create now Json data using a select on my old table and FOR JSON to get the Json object. The object should look something like this:
{
myObject: {
data: "valueB, ValueD"
}
}
My problem is the appending the values to the same "data" field.
So right now I can only do it with single values like this:
SELECT CASE WHEN ValueA == True THEN 'ValueA' END 'myObject.data',
CASE WHEN ValueB == True THEN 'ValueB' END 'myObject.data'
FROM myLegacyTable FOR JSON Path
This would obviously overwrite, what ever is in myObject.data, whenever both values are true. JSON_MODIFY seems not to be an option, as I'm not working on an existing Json object, but creating a new one. Maybe someone else has an idea?
You can just build your json string with a CASE WHEN. If you want to handle a variable number of columns you'll probably need dynamic TSQL.
This is a simple static version that should do the job:
declare #tmp table(ValueA bit, ValueB bit, ValueC bit, ValueD bit)
insert into #tmp values (null, 1 ,0 , 1)
SELECT JSON_QUERY('{"data":"' +
stuff(
case when ValueA = 1 then ',ValueA' else '' end
+ case when ValueB = 1 then ',ValueB' else '' end
+ case when ValueC = 1 then ',ValueC' else '' end
+ case when ValueD = 1 then ',ValueD' else '' end
,1,1,'')
+ '"}') as myObject
FROM #tmp FOR JSON Path, without_array_wrapper
Result:
I don't know how your source data is really stored but here is an example of what you need to do if this was all in SQL Server
data.property is not seperate items so don't split it into columns. It's just a single concatenated string. So concatenate everything up and remove the trailing comma.
DECLARE #MyTable TABLE (
RowID INT,
ValueA VARCHAR(5),
ValueB VARCHAR(5),
ValueC VARCHAR(5),
ValueD VARCHAR(5)
)
INSERT INTO #MyTable (RowID,ValueA,ValueB,ValueC,ValueD) VALUES
(1,NULL,'True','False','True'),
(2,'False',NULL,'True','True'),
(3,NULL,'False',NULL,NULL)
SELECT
ISNULL(
RTRIM(REVERSE(STUFF(REVERSE
(
CASE WHEN ValueA='True' THEN 'ValueA, ' ELSE '' END +
CASE WHEN ValueB='True' THEN 'ValueB, ' ELSE '' END +
CASE WHEN ValueC='True' THEN 'ValueC, ' ELSE '' END +
CASE WHEN ValueD='True' THEN 'ValueD, ' ELSE '' END
), 2, 1, ''))),'') AS 'myObject.data'
FROM #MyTable
FOR JSON Path
The huge construct required to remove trailing , is from here:
Remove the last character in a string in T-SQL?
Result:
[
{
"myObject": {
"data": "ValueB, ValueD"
}
},
{
"myObject": {
"data": "ValueC, ValueD"
}
},
{
"myObject": {
"data": ""
}
}
]

Sql concatenate result on case

I have a table like this
Nomeutente|data |Controllo
----------|----------|---------
utente1 |11-11-2016|prova1
utente1 |11-11-2016|prova4
utente1 |11-11-2016|prova3
utente2 |11-11-2016|ricontrollo
utente2 |11-11-2016|ricontrollo2
utente2 |11-11-2016|ricontrollo3
utente3 |11-11-2016|ricontrollo3
utente4 |11-11-2016|ricontrollo3
and with case i create a query like a pivot
Select
BASE.data,
Max(Case BASE.Nomeutente When 'utente1' Then base.controllo Else ''
End) As utente1,
Max(Case BASE.Nomeutente When 'utente2' Then base.controllo Else ''
End) As utente2,
Max(Case BASE.Nomeutente When 'utente3' Then base.controllo Else ''
End) As utente3,
Max(Case BASE.Nomeutente When 'utente4' Then base.controllo Else ''
End) As utente4,
From
(Select
Nomeutente,
data,
controllo
From PROVA) As BASE
Group By
base.data
but I want in case that insert all controllo value like
Nomeutente|data |Controllo
----------|----------|---------
utente1 |11-11-2016| prova1,prova4,prova3
utente2 |11-11-2016| ricontrollo,ricontrollo2,ricontrollo3
utente3 |11-11-2016| ricontrollo3
utente4 |11-11-2016| ricontrollo3
what kind of query can I create for make this on postgresql 7.4 ?
First create aggregate (only once):
create aggregate textcat_all(
basetype = text,
sfunc = textcat,
stype = text,
initcond = ''
);
Then you can run:
select
Nomeutente,
"data",
textcat_all(Controllo || ',') as Controllo
from
<table_name>
group by
Nomeutente, "data";
Recommeded reading:
https://www.postgresql.org/docs/7.4/static/sql-createaggregate.html
https://www.postgresql.org/support/versioning/

Issue with using nested case conditions in Oracle SQL query

i have a scenario where in , i need to calculate an employeee salary using a package , which gives me salary remaining for permanent employee if there is some prepayment and then convert the calculated value to a currency specified by the employee using another package.
How i have been doing this is:
CASE
WHEN ( ai.employee_type = 'PERMANENT' and ai.payment_status_flag='Y')
THEN EMP_UTILS_PKG.get_pay_amount_remaining( ai.employee_id )
ELSE ail.amount_remaining
END amount_remaining,
CASE
WHEN ( ai.salary_currency_code = NVL(find_currency('EMP_COMMON_CURRENCY'),ai.EMP_CURRENCY_CODE))
THEN CASE
WHEN ( ai.employee_type = 'PERMANENT' and ai.payment_status_flag='Y')
THEN EMP_UTILS_PKG.get_pay_amount_remaining( ai.employee_id )
ELSE ail.amount_remaining
END
ELSE EMP_API.convert_closest_amount_sql( x_from_currency => ai.EMP_CURRENCY_CODE ,x_to_currency => find_currency('EMP_COMMON_CURRENCY') ,x_amount =>( CASE
WHEN ( ai.employee_type = 'PERMANENT' and ai.payment_status_flag='Y')
THEN EMP_UTILS_PKG.get_pay_amount_remaining( ai.employee_id )
ELSE ail.amount_remaining
END) ,x_max_roll_days => -1)
END EMP_CONVERTED_AMOUNT_REMAINING,
I have repeated below condition 3 times in my query:
WHEN ( ai.employee_type = 'PERMANENT' and ai.payment_status_flag='Y')
THEN EMP_UTILS_PKG.get_pay_amount_remaining( ai.employee_id )
ELSE ail.amount_remaining
Is there a simpler approach for this, can we avoid repeating code like this?
You already use the function EMP_UTILS_PKG.get_pay_amount_remaining so why not create a PL/SQL function that encapsulates the logic and just call that in your multiple places.
You can use an inline view in the FROM clause to simplify this. Use the joins properly in your live code.
SELECT amount_remaining
, CASE
WHEN ai.salary_currency_code = NVL(find_currency('EMP_COMMON_CURRENCY')
,ai.EMP_CURRENCY_CODE)
THEN amount_remaining
ELSE EMP_API.convert_closest_amount_sql(
x_from_currency => ai.EMP_CURRENCY_CODE
,x_to_currency => find_currency('EMP_COMMON_CURRENCY')
,x_amount => amount_remaining
,x_max_roll_days => -1)
END EMP_CONVERTED_AMOUNT_REMAINING,
FROM (SELECT CASE
WHEN ai.employee_type = 'PERMANENT'
AND ai.payment_status_flag='Y'
THEN EMP_UTILS_PKG.get_pay_amount_remaining( ai.employee_id )
ELSE ail.amount_remaining
END amount_remaining
FROM table_name);

SQL: parse the first, middle and last name from a fullname field

How do I parse the first, middle, and last name out of a fullname field with SQL?
I need to try to match up on names that are not a direct match on full name. I'd like to be able to take the full name field and break it up into first, middle and last name.
The data does not include any prefixes or suffixes. The middle name is optional. The data is formatted 'First Middle Last'.
I'm interested in some practical solutions to get me 90% of the way there. As it has been stated, this is a complex problem, so I'll handle special cases individually.
Here is a self-contained example, with easily manipulated test data.
With this example, if you have a name with more than three parts, then all the "extra" stuff will get put in the LAST_NAME field. An exception is made for specific strings that are identified as "titles", such as "DR", "MRS", and "MR".
If the middle name is missing, then you just get FIRST_NAME and LAST_NAME (MIDDLE_NAME will be NULL).
You could smash it into a giant nested blob of SUBSTRINGs, but readability is hard enough as it is when you do this in SQL.
Edit-- Handle the following special cases:
1 - The NAME field is NULL
2 - The NAME field contains leading / trailing spaces
3 - The NAME field has > 1 consecutive space within the name
4 - The NAME field contains ONLY the first name
5 - Include the original full name in the final output as a separate column, for readability
6 - Handle a specific list of prefixes as a separate "title" column
SELECT
FIRST_NAME.ORIGINAL_INPUT_DATA
,FIRST_NAME.TITLE
,FIRST_NAME.FIRST_NAME
,CASE WHEN 0 = CHARINDEX(' ',FIRST_NAME.REST_OF_NAME)
THEN NULL --no more spaces? assume rest is the last name
ELSE SUBSTRING(
FIRST_NAME.REST_OF_NAME
,1
,CHARINDEX(' ',FIRST_NAME.REST_OF_NAME)-1
)
END AS MIDDLE_NAME
,SUBSTRING(
FIRST_NAME.REST_OF_NAME
,1 + CHARINDEX(' ',FIRST_NAME.REST_OF_NAME)
,LEN(FIRST_NAME.REST_OF_NAME)
) AS LAST_NAME
FROM
(
SELECT
TITLE.TITLE
,CASE WHEN 0 = CHARINDEX(' ',TITLE.REST_OF_NAME)
THEN TITLE.REST_OF_NAME --No space? return the whole thing
ELSE SUBSTRING(
TITLE.REST_OF_NAME
,1
,CHARINDEX(' ',TITLE.REST_OF_NAME)-1
)
END AS FIRST_NAME
,CASE WHEN 0 = CHARINDEX(' ',TITLE.REST_OF_NAME)
THEN NULL --no spaces # all? then 1st name is all we have
ELSE SUBSTRING(
TITLE.REST_OF_NAME
,CHARINDEX(' ',TITLE.REST_OF_NAME)+1
,LEN(TITLE.REST_OF_NAME)
)
END AS REST_OF_NAME
,TITLE.ORIGINAL_INPUT_DATA
FROM
(
SELECT
--if the first three characters are in this list,
--then pull it as a "title". otherwise return NULL for title.
CASE WHEN SUBSTRING(TEST_DATA.FULL_NAME,1,3) IN ('MR ','MS ','DR ','MRS')
THEN LTRIM(RTRIM(SUBSTRING(TEST_DATA.FULL_NAME,1,3)))
ELSE NULL
END AS TITLE
--if you change the list, don't forget to change it here, too.
--so much for the DRY prinicple...
,CASE WHEN SUBSTRING(TEST_DATA.FULL_NAME,1,3) IN ('MR ','MS ','DR ','MRS')
THEN LTRIM(RTRIM(SUBSTRING(TEST_DATA.FULL_NAME,4,LEN(TEST_DATA.FULL_NAME))))
ELSE LTRIM(RTRIM(TEST_DATA.FULL_NAME))
END AS REST_OF_NAME
,TEST_DATA.ORIGINAL_INPUT_DATA
FROM
(
SELECT
--trim leading & trailing spaces before trying to process
--disallow extra spaces *within* the name
REPLACE(REPLACE(LTRIM(RTRIM(FULL_NAME)),' ',' '),' ',' ') AS FULL_NAME
,FULL_NAME AS ORIGINAL_INPUT_DATA
FROM
(
--if you use this, then replace the following
--block with your actual table
SELECT 'GEORGE W BUSH' AS FULL_NAME
UNION SELECT 'SUSAN B ANTHONY' AS FULL_NAME
UNION SELECT 'ALEXANDER HAMILTON' AS FULL_NAME
UNION SELECT 'OSAMA BIN LADEN JR' AS FULL_NAME
UNION SELECT 'MARTIN J VAN BUREN SENIOR III' AS FULL_NAME
UNION SELECT 'TOMMY' AS FULL_NAME
UNION SELECT 'BILLY' AS FULL_NAME
UNION SELECT NULL AS FULL_NAME
UNION SELECT ' ' AS FULL_NAME
UNION SELECT ' JOHN JACOB SMITH' AS FULL_NAME
UNION SELECT ' DR SANJAY GUPTA' AS FULL_NAME
UNION SELECT 'DR JOHN S HOPKINS' AS FULL_NAME
UNION SELECT ' MRS SUSAN ADAMS' AS FULL_NAME
UNION SELECT ' MS AUGUSTA ADA KING ' AS FULL_NAME
) RAW_DATA
) TEST_DATA
) TITLE
) FIRST_NAME
It's difficult to answer without knowing how the "full name" is formatted.
It could be "Last Name, First Name Middle Name" or "First Name Middle Name Last Name", etc.
Basically you'll have to use the SUBSTRING function
SUBSTRING ( expression , start , length )
And probably the CHARINDEX function
CHARINDEX (substr, expression)
To figure out the start and length for each part you want to extract.
So let's say the format is "First Name Last Name" you could (untested.. but should be close) :
SELECT
SUBSTRING(fullname, 1, CHARINDEX(' ', fullname) - 1) AS FirstName,
SUBSTRING(fullname, CHARINDEX(' ', fullname) + 1, len(fullname)) AS LastName
FROM YourTable
Alternative simple way is to use parsename :
select full_name,
parsename(replace(full_name, ' ', '.'), 3) as FirstName,
parsename(replace(full_name, ' ', '.'), 2) as MiddleName,
parsename(replace(full_name, ' ', '.'), 1) as LastName
from YourTableName
source
Reverse the problem, add columns to hold the individual pieces and combine them to get the full name.
The reason this will be the best answer is that there is no guaranteed way to figure out a person has registered as their first name, and what is their middle name.
For instance, how would you split this?
Jan Olav Olsen Heggelien
This, while being fictious, is a legal name in Norway, and could, but would not have to, be split like this:
First name: Jan Olav
Middle name: Olsen
Last name: Heggelien
or, like this:
First name: Jan Olav
Last name: Olsen Heggelien
or, like this:
First name: Jan
Middle name: Olav
Last name: Olsen Heggelien
I would imagine similar occurances can be found in most languages.
So instead of trying to interpreting data which does not have enough information to get it right, store the correct interpretation, and combine to get the full name.
Unless you have very, very well-behaved data, this is a non-trivial challenge. A naive approach would be to tokenize on whitespace and assume that a three-token result is [first, middle, last] and a two-token result is [first, last], but you're going to have to deal with multi-word surnames (e.g. "Van Buren") and multiple middle names.
This query is working fine.
SELECT name
,Ltrim(SubString(name, 1, Isnull(Nullif(CHARINDEX(' ', name), 0), 1000))) AS FirstName
,Ltrim(SUBSTRING(name, CharIndex(' ', name), CASE
WHEN (CHARINDEX(' ', name, CHARINDEX(' ', name) + 1) - CHARINDEX(' ', name)) <= 0
THEN 0
ELSE CHARINDEX(' ', name, CHARINDEX(' ', name) + 1) - CHARINDEX(' ', name)
END)) AS MiddleName
,Ltrim(SUBSTRING(name, Isnull(Nullif(CHARINDEX(' ', name, Charindex(' ', name) + 1), 0), CHARINDEX(' ', name)), CASE
WHEN Charindex(' ', name) = 0
THEN 0
ELSE LEN(name)
END)) AS LastName
FROM yourtableName
Are you sure the Full Legal Name will always include First, Middle and Last? I know people that have only one name as Full Legal Name, and honestly I am not sure if that's their First or Last Name. :-) I also know people that have more than one Fisrt names in their legal name, but don't have a Middle name. And there are some people that have multiple Middle names.
Then there's also the order of the names in the Full Legal Name. As far as I know, in some Asian cultures the Last Name comes first in the Full Legal Name.
On a more practical note, you could split the Full Name on whitespace and threat the first token as First name and the last token (or the only token in case of only one name) as Last name. Though this assumes that the order will be always the same.
This Will Work in Case String Is FirstName/MiddleName/LastName
Select
DISTINCT NAMES ,
SUBSTRING(NAMES , 1, CHARINDEX(' ', NAMES) - 1) as FirstName,
RTRIM(LTRIM(REPLACE(REPLACE(NAMES,SUBSTRING(NAMES , 1, CHARINDEX(' ', NAMES) - 1),''),REVERSE( LEFT( REVERSE(NAMES), CHARINDEX(' ', REVERSE(NAMES))-1 ) ),'')))as MiddleName,
REVERSE( LEFT( REVERSE(NAMES), CHARINDEX(' ', REVERSE(NAMES))-1 ) ) as LastName
From TABLENAME
Like #1 said, it's not trivial. Hyphenated last names, initials, double names, inverse name sequence and a variety of other anomalies can ruin your carefully crafted function.
You could use a 3rd party library (plug/disclaimer - I worked on this product):
http://www.melissadata.com/nameobject/nameobject.htm
I would do this as an iterative process.
1) Dump the table to a flat file to work with.
2) Write a simple program to break up your Names using a space as separator where firsts token is the first name, if there are 3 token then token 2 is middle name and token 3 is last name. If there are 2 tokens then the second token is the last name. (Perl, Java, or C/C++, language doesn't matter)
3) Eyeball the results. Look for names that don't fit this rule.
4) Using that example, create a new rule to handle that exception...
5) Rinse and Repeat
Eventually you will get a program that fixes all your data.
Here's a stored procedure that will put the first word found into First Name, the last word into Last Name and everything in between into Middle Name.
create procedure [dbo].[import_ParseName]
(
#FullName nvarchar(max),
#FirstName nvarchar(255) output,
#MiddleName nvarchar(255) output,
#LastName nvarchar(255) output
)
as
begin
set #FirstName = ''
set #MiddleName = ''
set #LastName = ''
set #FullName = ltrim(rtrim(#FullName))
declare #ReverseFullName nvarchar(max)
set #ReverseFullName = reverse(#FullName)
declare #lengthOfFullName int
declare #endOfFirstName int
declare #beginningOfLastName int
set #lengthOfFullName = len(#FullName)
set #endOfFirstName = charindex(' ', #FullName)
set #beginningOfLastName = #lengthOfFullName - charindex(' ', #ReverseFullName) + 1
set #FirstName = case when #endOfFirstName <> 0
then substring(#FullName, 1, #endOfFirstName - 1)
else ''
end
set #MiddleName = case when (#endOfFirstName <> 0 and #beginningOfLastName <> 0 and #beginningOfLastName > #endOfFirstName)
then ltrim(rtrim(substring(#FullName, #endOfFirstName , #beginningOfLastName - #endOfFirstName)))
else ''
end
set #LastName = case when #beginningOfLastName <> 0
then substring(#FullName, #beginningOfLastName + 1 , #lengthOfFullName - #beginningOfLastName)
else ''
end
return
end
And here's me calling it.
DECLARE #FirstName nvarchar(255),
#MiddleName nvarchar(255),
#LastName nvarchar(255)
EXEC [dbo].[import_ParseName]
#FullName = N'Scott The Other Scott Kowalczyk',
#FirstName = #FirstName OUTPUT,
#MiddleName = #MiddleName OUTPUT,
#LastName = #LastName OUTPUT
print #FirstName
print #MiddleName
print #LastName
output:
Scott
The Other Scott
Kowalczyk
If you are trying to parse apart a human name in PHP, I recommend Keith Beckman's nameparse.php script.
Copy in case site goes down:
<?
/*
Name: nameparse.php
Version: 0.2a
Date: 030507
First: 030407
License: GNU General Public License v2
Bugs: If one of the words in the middle name is Ben (or St., for that matter),
or any other possible last-name prefix, the name MUST be entered in
last-name-first format. If the last-name parsing routines get ahold
of any prefix, they tie up the rest of the name up to the suffix. i.e.:
William Ben Carey would yield 'Ben Carey' as the last name, while,
Carey, William Ben would yield 'Carey' as last and 'Ben' as middle.
This is a problem inherent in the prefix-parsing routines algorithm,
and probably will not be fixed. It's not my fault that there's some
odd overlap between various languages. Just don't name your kids
'Something Ben Something', and you should be alright.
*/
function norm_str($string) {
return trim(strtolower(
str_replace('.','',$string)));
}
function in_array_norm($needle,$haystack) {
return in_array(norm_str($needle),$haystack);
}
function parse_name($fullname) {
$titles = array('dr','miss','mr','mrs','ms','judge');
$prefices = array('ben','bin','da','dal','de','del','der','de','e',
'la','le','san','st','ste','van','vel','von');
$suffices = array('esq','esquire','jr','sr','2','ii','iii','iv');
$pieces = explode(',',preg_replace('/\s+/',' ',trim($fullname)));
$n_pieces = count($pieces);
switch($n_pieces) {
case 1: // array(title first middles last suffix)
$subp = explode(' ',trim($pieces[0]));
$n_subp = count($subp);
for($i = 0; $i < $n_subp; $i++) {
$curr = trim($subp[$i]);
$next = trim($subp[$i+1]);
if($i == 0 && in_array_norm($curr,$titles)) {
$out['title'] = $curr;
continue;
}
if(!$out['first']) {
$out['first'] = $curr;
continue;
}
if($i == $n_subp-2 && $next && in_array_norm($next,$suffices)) {
if($out['last']) {
$out['last'] .= " $curr";
}
else {
$out['last'] = $curr;
}
$out['suffix'] = $next;
break;
}
if($i == $n_subp-1) {
if($out['last']) {
$out['last'] .= " $curr";
}
else {
$out['last'] = $curr;
}
continue;
}
if(in_array_norm($curr,$prefices)) {
if($out['last']) {
$out['last'] .= " $curr";
}
else {
$out['last'] = $curr;
}
continue;
}
if($next == 'y' || $next == 'Y') {
if($out['last']) {
$out['last'] .= " $curr";
}
else {
$out['last'] = $curr;
}
continue;
}
if($out['last']) {
$out['last'] .= " $curr";
continue;
}
if($out['middle']) {
$out['middle'] .= " $curr";
}
else {
$out['middle'] = $curr;
}
}
break;
case 2:
switch(in_array_norm($pieces[1],$suffices)) {
case TRUE: // array(title first middles last,suffix)
$subp = explode(' ',trim($pieces[0]));
$n_subp = count($subp);
for($i = 0; $i < $n_subp; $i++) {
$curr = trim($subp[$i]);
$next = trim($subp[$i+1]);
if($i == 0 && in_array_norm($curr,$titles)) {
$out['title'] = $curr;
continue;
}
if(!$out['first']) {
$out['first'] = $curr;
continue;
}
if($i == $n_subp-1) {
if($out['last']) {
$out['last'] .= " $curr";
}
else {
$out['last'] = $curr;
}
continue;
}
if(in_array_norm($curr,$prefices)) {
if($out['last']) {
$out['last'] .= " $curr";
}
else {
$out['last'] = $curr;
}
continue;
}
if($next == 'y' || $next == 'Y') {
if($out['last']) {
$out['last'] .= " $curr";
}
else {
$out['last'] = $curr;
}
continue;
}
if($out['last']) {
$out['last'] .= " $curr";
continue;
}
if($out['middle']) {
$out['middle'] .= " $curr";
}
else {
$out['middle'] = $curr;
}
}
$out['suffix'] = trim($pieces[1]);
break;
case FALSE: // array(last,title first middles suffix)
$subp = explode(' ',trim($pieces[1]));
$n_subp = count($subp);
for($i = 0; $i < $n_subp; $i++) {
$curr = trim($subp[$i]);
$next = trim($subp[$i+1]);
if($i == 0 && in_array_norm($curr,$titles)) {
$out['title'] = $curr;
continue;
}
if(!$out['first']) {
$out['first'] = $curr;
continue;
}
if($i == $n_subp-2 && $next &&
in_array_norm($next,$suffices)) {
if($out['middle']) {
$out['middle'] .= " $curr";
}
else {
$out['middle'] = $curr;
}
$out['suffix'] = $next;
break;
}
if($i == $n_subp-1 && in_array_norm($curr,$suffices)) {
$out['suffix'] = $curr;
continue;
}
if($out['middle']) {
$out['middle'] .= " $curr";
}
else {
$out['middle'] = $curr;
}
}
$out['last'] = $pieces[0];
break;
}
unset($pieces);
break;
case 3: // array(last,title first middles,suffix)
$subp = explode(' ',trim($pieces[1]));
$n_subp = count($subp);
for($i = 0; $i < $n_subp; $i++) {
$curr = trim($subp[$i]);
$next = trim($subp[$i+1]);
if($i == 0 && in_array_norm($curr,$titles)) {
$out['title'] = $curr;
continue;
}
if(!$out['first']) {
$out['first'] = $curr;
continue;
}
if($out['middle']) {
$out['middle'] .= " $curr";
}
else {
$out['middle'] = $curr;
}
}
$out['last'] = trim($pieces[0]);
$out['suffix'] = trim($pieces[2]);
break;
default: // unparseable
unset($pieces);
break;
}
return $out;
}
?>
Get a sql regex function. Sample: http://msdn.microsoft.com/en-us/magazine/cc163473.aspx
Extract names using regular expressions.
I recommend Expresso for learnin/building/testing regular expressions. Old free version, new commercial version
I once made a 500 character regular expression to parse first, last and middle names from an arbitrary string. Even with that honking regex, it only got around 97% accuracy due to the complete inconsistency of the input. Still, better than nothing.
Subject to the caveats that have already been raised regarding spaces in names and other anomalies, the following code will at least handle 98% of names. (Note: messy SQL because I don't have a regex option in the database I use.)
**Warning: messy SQL follows:
create table parsname (fullname char(50), name1 char(30), name2 char(30), name3 char(30), name4 char(40));
insert into parsname (fullname) select fullname from ImportTable;
update parsname set name1 = substring(fullname, 1, locate(' ', fullname)),
fullname = ltrim(substring(fullname, locate(' ', fullname), length(fullname)))
where locate(' ', rtrim(fullname)) > 0;
update parsname set name2 = substring(fullname, 1, locate(' ', fullname)),
fullname = ltrim(substring(fullname, locate(' ', fullname), length(fullname)))
where locate(' ', rtrim(fullname)) > 0;
update parsname set name3 = substring(fullname, 1, locate(' ', fullname)),
fullname = ltrim(substring(fullname, locate(' ', fullname), length(fullname)))
where locate(' ', rtrim(fullname)) > 0;
update parsname set name4 = substring(fullname, 1, locate(' ', fullname)),
fullname = ltrim(substring(fullname, locate(' ', fullname), length(fullname)))
where locate(' ', rtrim(fullname)) > 0;
// fullname now contains the last word in the string.
select fullname as FirstName, '' as MiddleName, '' as LastName from parsname where fullname is not null and name1 is null and name2 is null
union all
select name1 as FirstName, name2 as MiddleName, fullname as LastName from parsname where name1 is not null and name3 is null
The code works by creating a temporary table (parsname) and tokenizing the fullname by spaces. Any names ending up with values in name3 or name4 are non-conforming and will need to be dealt with differently.
As everyone else says, you can't from a simple programmatic way.
Consider these examples:
President "George Herbert Walker Bush" (First Middle Middle Last)
Presidential assassin "John Wilkes Booth" (First Middle
Last)
Guitarist "Eddie Van Halen" (First Last Last)
And his mom probably calls him Edward Lodewijk Van Halen (First
Middle Last Last)
Famed castaway "Mary Ann Summers" (First First Last)
New Mexico GOP chairman "Fernando C de Baca" (First Last Last Last)
I'm not sure about SQL server, but in postgres you could do something like this:
SELECT
SUBSTRING(fullname, '(\\w+)') as firstname,
SUBSTRING(fullname, '\\w+\\s(\\w+)\\s\\w+') as middle,
COALESCE(SUBSTRING(fullname, '\\w+\\s\\w+\\s(\\w+)'), SUBSTRING(fullname, '\\w+\\s(\\w+)')) as lastname
FROM
public.person
The regex expressions could probably be a bit more concise; but you get the point. This does by the way not work for persons having two double names (in the Netherlands we have this a lot 'Jan van der Ploeg') so I'd be very careful with the results.
We of course all understand that there's no perfect way to solve this problem, but some solutions can get you farther than others.
In particular, it's pretty easy to go beyond simple whitespace-splitters if you just have some lists of common prefixes (Mr, Dr, Mrs, etc.), infixes (von, de, del, etc.), suffixes (Jr, III, Sr, etc.) and so on. It's also helpful if you have some lists of common first names (in various languages/cultures, if your names are diverse) so that you can guess whether a word in the middle is likely to be part of the last name or not.
BibTeX also implements some heuristics that get you part of the way there; they're encapsulated in the Text::BibTeX::Name perl module. Here's a quick code sample that does a reasonable job.
use Text::BibTeX;
use Text::BibTeX::Name;
$name = "Dr. Mario Luis de Luigi Jr.";
$name =~ s/^\s*([dm]rs?.?|miss)\s+//i;
$dr=$1;
$n=Text::BibTeX::Name->new($name);
print join("\t", $dr, map "#{[ $n->part($_) ]}", qw(first von last jr)), "\n";
The biggest problem I ran into doing this was cases like "Bob R. Smith, Jr.". The algorithm I used is posted at http://www.blackbeltcoder.com/Articles/strings/splitting-a-name-into-first-and-last-names. My code is in C# but you could port it if you must have in SQL.
The work by #JosephStyons and #Digs is great! I used parts of their work to create a new function for SQL Server 2016 and newer. This one also handles suffixes, as well as prefixes.
CREATE FUNCTION [dbo].[NameParser]
(
#name nvarchar(100)
)
RETURNS TABLE
AS
RETURN (
WITH prep AS (
SELECT
original = #name,
cleanName = REPLACE(REPLACE(REPLACE(REPLACE(LTRIM(RTRIM(#name)),' ',' '),' ',' '), '.', ''), ',', '')
)
SELECT
prep.original,
aux.prefix,
firstName.firstName,
middleName.middleName,
lastName.lastName,
aux.suffix
FROM
prep
CROSS APPLY (
SELECT
prefix =
CASE
WHEN LEFT(prep.cleanName, 3) IN ('MR ', 'MS ', 'DR ', 'FR ')
THEN LEFT(prep.cleanName, 2)
WHEN LEFT(prep.cleanName, 4) IN ('MRS ', 'LRD ', 'SIR ')
THEN LEFT(prep.cleanName, 3)
WHEN LEFT(prep.cleanName, 5) IN ('LORD ', 'LADY ', 'MISS ', 'PROF ')
THEN LEFT(prep.cleanName, 4)
ELSE ''
END,
suffix =
CASE
WHEN RIGHT(prep.cleanName, 3) IN (' JR', ' SR', ' II', ' IV')
THEN RIGHT(prep.cleanName, 2)
WHEN RIGHT(prep.cleanName, 4) IN (' III', ' ESQ')
THEN RIGHT(prep.cleanName, 3)
ELSE ''
END
) aux
CROSS APPLY (
SELECT
baseName = LTRIM(RTRIM(SUBSTRING(prep.cleanName, LEN(aux.prefix) + 1, LEN(prep.cleanName) - LEN(aux.prefix) - LEN(aux.suffix)))),
numParts = (SELECT COUNT(1) FROM STRING_SPLIT(LTRIM(RTRIM(SUBSTRING(prep.cleanName, LEN(aux.prefix) + 1, LEN(prep.cleanName) - LEN(aux.prefix) - LEN(aux.suffix)))), ' '))
) core
CROSS APPLY (
SELECT
firstName =
CASE
WHEN core.numParts <= 1 THEN core.baseName
ELSE LEFT(core.baseName, CHARINDEX(' ', core.baseName, 1) - 1)
END
) firstName
CROSS APPLY (
SELECT
remainder =
CASE
WHEN core.numParts <= 1 THEN ''
ELSE LTRIM(SUBSTRING(core.baseName, LEN(firstName.firstName) + 1, 999999))
END
) work1
CROSS APPLY (
SELECT
middleName =
CASE
WHEN core.numParts <= 2 THEN ''
ELSE LEFT(work1.remainder, CHARINDEX(' ', work1.remainder, 1) - 1)
END
) middleName
CROSS APPLY (
SELECT
lastName =
CASE
WHEN core.numParts <= 1 THEN ''
ELSE LTRIM(SUBSTRING(work1.remainder, LEN(middleName.middleName) + 1, 999999))
END
) lastName
)
GO
SELECT * FROM dbo.NameParser('Madonna')
SELECT * FROM dbo.NameParser('Will Smith')
SELECT * FROM dbo.NameParser('Neil Degrasse Tyson')
SELECT * FROM dbo.NameParser('Dr. Neil Degrasse Tyson')
SELECT * FROM dbo.NameParser('Mr. Hyde')
SELECT * FROM dbo.NameParser('Mrs. Thurston Howell, III')
Check this query in Athena for only one-space separated string (e.g. first name and middle name combination):
SELECT name, REVERSE( SUBSTR( REVERSE(name), 1, STRPOS(REVERSE(name), ' ') ) ) AS middle_name
FROM name_table
If you expect to have two or more spaces, you can easily extend the above query.
Based on #hajili's contribution (which is a creative use of the parsename function, intended to parse the name of an object that is period-separated), I modified it so it can handle cases where the data doesn't containt a middle name or when the name is "John and Jane Doe". It's not 100% perfect but it's compact and might do the trick depending on the business case.
SELECT NAME,
CASE WHEN parsename(replace(NAME, ' ', '.'), 4) IS NOT NULL THEN
parsename(replace(NAME, ' ', '.'), 4) ELSE
CASE WHEN parsename(replace(NAME, ' ', '.'), 3) IS NOT NULL THEN
parsename(replace(NAME, ' ', '.'), 3) ELSE
parsename(replace(NAME, ' ', '.'), 2) end END as FirstName
,
CASE WHEN parsename(replace(NAME, ' ', '.'), 3) IS NOT NULL THEN
parsename(replace(NAME, ' ', '.'), 2) ELSE NULL END as MiddleName,
parsename(replace(NAME, ' ', '.'), 1) as LastName
from {#YourTableName}
Employee table has column "Name" and we had to split it into First, Middle and Last Name. This query will handle to keep middle name as null if name column has value of two words like 'James Thomas'.
UPDATE Employees
SET [First Name] = CASE
WHEN (len(name) - len(Replace(name, '.', ''))) = 2
THEN PARSENAME(Name, 3)
WHEN (len(name) - len(Replace(name, '.', ''))) = 1
THEN PARSENAME(Name, 2)
ELSE PARSENAME(Name, 1)
END
,[Middle Name] = CASE
WHEN (len(name) - len(Replace(name, '.', ''))) = 2
THEN PARSENAME(Name, 2)
ELSE NULL
END
,[Last Name] = CASE
WHEN (len(name) - len(Replace(name, '.', ''))) = 2
THEN PARSENAME(Name, 1)
WHEN (len(name) - len(Replace(name, '.', ''))) = 1
THEN PARSENAME(Name, 1)
ELSE NULL
END GO
UPDATE Employee
SET [Name] = Replace([Name], '.', ' ') GO
I wanted to post an update to the suggestion by hajili, but this response was too long for a comment on that suggestion.
Our issue was "Lastname,Firstname Middlename" with some last name's with a space in them.
So we came up with:
,FullName = CUST.FULLNAME
,LastName = PARSENAME(REPLACE(CUST.FULLNAME, ',', '.'),2)
,FirstName = (CASE WHEN PARSENAME(REPLACE(CUST.FULLNAME, ',', '.'),1) LIKE '% %' THEN PARSENAME(REPLACE(PARSENAME(REPLACE(CUST.FULLNAME, ',', '.'),1), ' ', '.'),2) ELSE PARSENAME(REPLACE(CUST.FULLNAME, ',', '.'),1) END)
,MiddleName = (CASE WHEN PARSENAME(REPLACE(CUST.FULLNAME, ' ', '.'),1) LIKE '%,%' THEN NULL ELSE PARSENAME(REPLACE(CUST.FULLNAME, ' ', '.'),1) END)
SELECT SUBSTRING_INDEX(name, ' ', 1) as fname, SUBSTRING_INDEX(SUBSTRING_INDEX(name, ' ', 2), ' ', -1) as mname, SUBSTRING_INDEX(name, ' ', -1) as lname FROM Person
If the “fullname” column is in “Last, First - Middle” format (it usually isn’t, but let’s imagine it is), then this works. Done in My SQL. In the first line, the “inner” SUBSTRING_INDEX() gets everything from the left up to ‘ - ‘, which is “Last, First”; then the “outer” SUBSTRING_INDEX() gets everything from the right up to ‘, ‘ from this new “Last, First” string, which is “First”.
The second line gets the piece from the right up to ‘ - ‘, which is “Middle”.
The third line gets the first string from the left up to the ‘, ‘.
SUBSTRING_INDEX(SUBSTRING_INDEX(fullname, ' - ', 1), ', ', -1) AS First,
SUBSTRING_INDEX(fullname, ' - ', -1), AS Middle,
SUBSTRING_INDEX(fullname, ', ', 1) AS Last,
Name,
Case when (DATALENGTH(NAME)-DATALENGTH(REPLACE(NAME,' ','')))=2 then
SUBSTRING(Name,CharIndex(' ',NAME,(CharIndex(' ',NAME)+1)),LEN(NAME))
else
SUBSTRING(Name_Line1,CharIndex(' ',NAME,(CharIndex(' ',NAME))),LEN(NAME))
end As Last_name,
Case when (DATALENGTH(NAME)-DATALENGTH(REPLACE(NAME,' ','')))=2 then
SUBSTRING(Name,CharIndex(' ',NAME,(CharIndex(' ',NAME))),(CharIndex(' ',NAME)+1))
else ''
end As Middle_name