Text data extraction between keywords in a string - dataframe

Question

I have text data that looks like the following after extracting from a file and cleaning. I want to put the data into a pandas dataframe where the columns are ('EXAMINATION', 'TECHNIQUE', 'COMPARISON', 'FINDINGS', 'IMPRESSION'), and each cell in each row contains the extracted data related to the column name (i.e. the keyword).
'FINAL REPORT EXAMINATION: CHEST PA AND LAT INDICATION: F with new onset ascites eval for infection TECHNIQUE: Chest PA and lateral COMPARISON: None FINDINGS: There is no focal consolidation pleural effusion or pneumothorax Bilateral nodular opacities that most likely represent nipple shadows The cardiomediastinal silhouette is normal Clips project over the left lung potentially within the breast The imaged upper abdomen is unremarkable Chronic deformity of the posterior left sixth and seventh ribs are noted IMPRESSION: No acute cardiopulmonary process'
For example, under the column TECHNIQUE there should be a cell containing "Chest PA and lateral", and under the column IMPRESSION, there should be a cell containing "No acute cardiopulmonary process".

Answer 1

Solution as follows, please note the following assumptions:
Keywords as presented are located in that order within the sample text.
The keywords are not contained within the text to be extracted.
Each keyword is followed by a ": " (the colon and whitespace is removed).
Solution
import pandas as pd
sample = "FINAL REPORT EXAMINATION: CHEST PA AND LAT INDICATION: F with new onset ascites eval for infection TECHNIQUE: Chest PA and lateral COMPARISON: None FINDINGS: There is no focal consolidation pleural effusion or pneumothorax Bilateral nodular opacities that most likely represent nipple shadows The cardiomediastinal silhouette is normal Clips project over the left lung potentially within the breast The imaged upper abdomen is unremarkable Chronic deformity of the posterior left sixth and seventh ribs are noted IMPRESSION: No acute cardiopulmonary process"
keywords = ["EXAMINATION", "TECHNIQUE", "COMPARISON", "FINDINGS", "IMPRESSION"]
# Create function to extract text between each of the keywords
def extract_text_using_keywords(clean_text, keyword_list):
extracted_texts = []
for prev_kw, current_kw in zip(keyword_list, keyword_list[1:]):
prev_kw_index = clean_text.index(prev_kw)
current_kw_index = clean_text.index(current_kw)
extracted_texts.append(clean_text[prev_kw_index + len(prev_kw) + 2:current_kw_index])
# Extract the text after the final keyword in keyword_list (i.e. "IMPRESSION")
if current_kw == keyword_list[-1]:
extracted_texts.append(clean_text[current_kw_index + len(current_kw) + 2:len(clean_text)])
return extracted_texts
# Extract text
result = extract_text_using_keywords(sample, keywords)
# Create pandas dataframe
df = pd.DataFrame([result], columns=keywords)
print(df)
# To append future results to the end of the pandas df you can use
# df.loc[len(df)] = result
Output
EXAMINATION TECHNIQUE COMPARISON FINDINGS IMPRESSION
0 CHEST PA AND LAT INDICATION: F with new onset ... Chest PA and lateral None There is no focal consolidation pleural effusi... No acute cardiopulmonary process

Answer 2

It looks like the input is organized such that EXAMINATION, TECHNIQUE, etc. occur in that order.
One approach is to iterate over pairs of strings and use .split() to select content between them. Here is one approach:
import pandas as pd
data = 'FINAL REPORT EXAMINATION: CHEST PA AND LAT INDICATION: F with new onset ascites eval for infection TECHNIQUE: Chest PA and lateral COMPARISON: None FINDINGS: There is no focal consolidation pleural effusion or pneumothorax Bilateral nodular opacities that most likely represent nipple shadows The cardiomediastinal silhouette is normal Clips project over the left lung potentially within the breast The imaged upper abdomen is unremarkable Chronic deformity of the posterior left sixth and seventh ribs are noted IMPRESSION: No acute cardiopulmonary process'
strings = ('EXAMINATION','TECHNIQUE', 'COMPARISON','FINDINGS', 'IMPRESSION', '')
out = {}
for s1, s2 in zip(strings, strings[1:]):
if not s2:
text = data.split(s1)[1]
else:
text = data.split(s1)[1].split(s2)[0]
out[s1] = [text]
print(pd.DataFrame(out))
Which results in:
EXAMINATION TECHNIQUE COMPARISON FINDINGS IMPRESSION
0 : CHEST PA AND LAT INDICATION: F with new onse... : Chest PA and lateral : None : There is no focal consolidation pleural effu... : No acute cardiopulmonary process