Detecting a sub-string - sql

Question

I have a lot of data that I need to sort through and one of the fields contains both the make/model of the vehicle as well as the reg, sometimes separated by a dash (-) sometimes however it is not. Here is an example of such a string:
VehicleModel - TU69YUP
VehicleModel - TU69 YUP
VehicleModel TU69YUP
VehicleModel TU69 YUP
There are also some other variations but they are the main ones I have encountered. Is there a way that I can reliably go through all of the data and separate the vehicle reg from the model?
The data is currently contained within a Paradox database which I have no problem going through. I do not have a list of all of the vehicle models and names that are contained within the database, likewise, I also do not have a list of the licence plates.
The project is written in Delphi/SQL so I would prefer to stick with either one of these if at all possible.

Answer 1

Trouble ahead
If that field was originally entered by a user in the form that you're now seeing, then we can assume there was no validation, the original program would simply store whatever the user entered. If that's the case, you can't get 100% accuracy: human beings will always make mistakes, intentionally or unintentionally. Expect this kinds of human errors:
Missing fields (ie: registration only, no vehicle information - or the other way around)
Meaningless duplication of words (example: "Ford Ford K - TU 69 YUP")
Missing letters, duplicated letters, extra garbage letters. Example: "For K - T69YUP"
Wrong order of fields
Other small errors you can't even dream of.
Plain garbage that not even a human would make sense of.
You might have guessed I'm a bit pessimistic when dealing with human-entered data straight into text fields. I had the distinct misfortune to deal with a database where all data was text and there was no validation: Can you guess the kind of nonsense people typed in unvalidated date fields that allowed free user input?
The plan
Things aren't as dark as they seem, you can probably "fix" lots of things. The trick here is making sure you only fix data that's unambiguous and let a human sift through the rest of the stuff. The easiest way to do that is to do something like this:
Look at the data you have and wasn't automatically fixed yet. Figure out a rule that unambiguously applies to lots of records.
Apply the unambiguous rule.
Repeat until only a few records are left. Those should be fixed by hand, because they resisted all automatic methods that were applied.
The implementation
I strongly recommend using regular expressions for all the tests, because you'll surely end up implementing lots of different tests, and regular expressions can easily "express" the slight variations in search text. For example the following reg-ex can parse all 4 of your examples and give the correct result:
(.*?)(\ {1,3}-\ {1,3})?(\b[A-Z]{2}\ {0,2}[0-9]{2}\ {0,3}[A-Z]{3}\b)
If you've never worked with regular expressions before, that single expressions looks unintelligible, but it's in fact very simple. This is not a reg-ex question so I'm not going into any details. I'd rather explain how I've come up with the idea.
First of all, if the text includes vehicle registration numbers, those numbers will be in a very strict format: they'd be easy to match. Given your example I assume all registration numbers are of the form:
LLNNLLL
where "L" is a letter and "N" is a number. My regex is rigid in it's interpretation of it: it wants exactly two uppercase letters, followed by a small number of spaces (or no space), followed by exactly two digits, followed by a small number of spaces (or no space), finally followed by exactly 3 uppercase letters. The part of the regex that deals with that is:
[A-Z]{2}\ {0,2}[0-9]{2}\ {0,3}[A-Z]{3}
The rest of the regex makes sure the registration number isn't found embedded into other words, deals with grouping text into capture groups and creates an "lazy capture group" for the VehicleModel.
If I were to implement this myself, I'd probably write a "master" function and a number of simpler "case" functions, each function dealing with one kind of variation in user input. Example:
// This function does a validation of the extracted data. For example it validates the
// Registration number, using other, more precise criteria. The parameters are VAR so the
// function may normalize the results.
function ResultsAreValid(var Make, Registration:string): Boolean;
begin
Result := True; // Only you know what your data looks like and how it can be validated.
end;
// This is a case function that deals with a very rigid interpretation of user data
function VeryStrictInterpretation(const Text:string; out Make, Registration: string): Boolean;
var TestMake, TestReg: string;
// regex engine ...
begin
Result := False;
if (your condition) then
if ResultsAreValid(TestMake, TestReg) then
begin
Make := TestMake;
Registration := TestReg;
Result := True;
end;
end;
// Master function calling many different implementations that each deal with all sorts
// of variations of input. The most strict function should be first:
function MasterTest(const Text:string; out Make, Registration: string): Boolean;
begin
Result := VeryStrictInterpretation(Text, Make, Registration);
if not Result then Result := SomeOtherImplementation(Text, Make, Registration);
if not Result then Result := ThirdInterpretation(Text, Make, Registration);
end;
The idea here is to try to make multiple SIMPLE procedures, that each understands one kind of input in an unambiguous way; And make sure each step doesn't return false positives! And finally don't forget, a human should deal with the last few cases, so don't aim for a fix-it-all solution.

Answer 2

Well assuming that they are of the same format. Word[space]Word
Then you can iterate through them all, and if you encounter a whitespace without a dash, insert a dash. Then split as normal.

Answer 3

Here is a code example.
It will check for the - and also remove possible spaces in the license number.
Note : (as commented by Ken White), if the vehicle contains a space, this will have to be handled as well.
type
EMySplitError = class(Exception);
procedure SplitVehicleAndLicense( s : String; var vehicle,license : String);
var
p : Integer;
begin
vehicle := '';
license := '';
p := Pos('-',s);
if (p = 0) then
begin // No split delimiter
p := Pos(' ',s);
if (p > 0) then
begin
vehicle := Trim(Copy(s,1,p-1));
license := Trim(Copy(s,p+1,Length(s)));
end
else
Raise EMySplitError.CreateFmt('Not a valid vehicle/license name:%s',[s]);
end
else
begin
vehicle := Trim(Copy( s,1,p-1));
license := Trim(Copy( s,p+1,Length(s)));
end;
// Trim spaces in license
repeat
p := Pos(' ',license);
if (p <> 0) then Delete(license,p,1);
until (p = 0);
end;