Regular expression parsing pseudo-columnized data in SQL - sql

Question

I'm working on a project where we're trying to parse information out of reroute advisories issued by the FAA. These advisories are issued as free text with a loosely-identified structure, with the goal being to allow viewers to print them out on a single sheet of paper.
The area of most interest is the final portion of the advisory that contains specific information related to a given reroute - the origin and destination airport as well as the specific required route that applies to any flight between them. Here's an example:
ORIG DEST ROUTE
---- --------------- ---------------------------
FRG MCO TPA PIE SRQ WAVEY EMJAY J174 SWL CEBEE
FMY RSW APF WETRO DIW AR22 JORAY HILEY4
What I'd like to do is be able to parse this into three entries like this:
ORIG: FRG
DEST: MCO TPA PIE SRQ FMY RSW APF
ROUTE: WAVEY EMJAY J174 SWL CEBEE WETRO DIW AR22 JORAY HILEY4
Here are the three code segments I'm currently using to parse this portion of the advisories:
Origin
regexp_substr(route_1,'^(([A-Z0-9]|\(|\)|\-)+\s)+')
Destination
regexp_substr(route_1,'(([A-Z0-9]|\(|\)|\-)+\s)+',1,2)
Route String
regexp_substr(route_1, '\s{2,}>?([A-Z0-9]|>|<|\s|:)+<?$')
While these expressions can deal with the majority of situations where the Origin and Destination portions are all on the first line, they cannot deal with the example provided earlier. Does anyone know how I might be able to successfully parse the text in my original example?

Answer 1

Proof of concept.
Input file is a plain text file (with no tabs, only spaces). Vertically it is divided into three columns, of fixed width: 20 characters, 20 characters, whatever is left (till the end of line).
The first two rows are always populated, with the headers and -----. These two rows can be ignored. Then the rest of the file (reading down the page) are "grouped" by the latest non-empty string in the ORIG column.
The input file looks like this:
ORIG DEST ROUTE
---- --------------- ---------------------------
FRG MCO TPA PIE SRQ WAVEY EMJAY J174 SWL CEBEE
FMY RSW APF WETRO DIW AR22 JORAY HILEY4
ABC SFD RRE BAC TRIO SBL CRT
POLDA FARM OLE BID ORDG BALL
BINT LFV
YYT PSS TRI BABA TEN NINE FIVE
COL DMV
SAL PRT DUW PALO VR22 NOL3
Notice the empty lines between blocks, the empty DEST in one block (I handle that, although perhaps in the OP's problem that is not possible), and the different number of rows used by DEST and ROUTE in some cases.
The file name is inp.txt and it resides in a directory which I have made known to Oracle: create directory sandbox as 'c:\app\sandbox'. (First I had to grant create any directory to <myself>, while logged in as SYS.)
The output looks like this:
ORIG DEST ROUTE
----- --------------------------- ------------------------------------------------------
FRG MCO TPA PIE SRQ FMY RSW APF WAVEY EMJAY J174 SWL CEBEE WETRO DIW AR22 JORAY HILEY4
ABC SFD RRE BAC TRIO SBL CRT
POLDA FARM OLE BID ORDG BALL BINT LFV
YYT PSS TRI BABA COL DMV TEN NINE FIVE
SAL PRT DUW PALO VR22 NOL3
I did this in two steps. First I created a helper table, INP, with four columns (RN number, ORIG varchar2(20), DEST varchar2(20), ROUTE varchar2(20)) and I imported from the text file through a procedure. Then I processed this further and used the output to populate the final table. It is very unlikely that this is the most efficient way to do this (and perhaps there are very good reasons not to do it the way I did); I have no experience with UTL_FILE and importing text files into Oracle in general. I did this for two reasons: to learn, and to show it can be done.
The procedure to import the text file into the helper table:
Create or Replace PROCEDURE read_inp is
f UTL_FILE.FILE_TYPE;
s VARCHAR2(200);
rn number := 1;
BEGIN
f := UTL_FILE.FOPEN('SANDBOX','inp.txt','r', 200);
LOOP
BEGIN
UTL_FILE.GET_LINE(f,s);
INSERT INto inp (rn, orig, dest, route)
VALUES
(rn, trim(substr(s, 1, 20)), trim(substr(s, 21, 20)), trim(substr(s, 41)));
END;
rn := rn + 1;
END LOOP;
exception
when no_data_found then
utl_file.fclose(f);
END;
/
exec read_inp
/
And the further processing (after creating the REROUTE table):
create table reroute ( orig varchar2(20), dest varchar2(4000), route varchar2(4000) );
insert into reroute
select max(orig),
trim(listagg(dest , ' ') within group (order by rn)),
trim(listagg(route, ' ') within group (order by rn))
from (
select rn, orig, dest, route, count(orig) over (order by rn) as grp
from inp
where rn >= 3
)
group by grp
;