stacking cnn output layer with xgboost. Data prep gives OOM error - tensorflow

Question

I have trained a cnn model and I am trying to stack the output layer to an xgboost regressor to reduce mape. I am getting OOM error in Sagemaker training job when I try to include the input data (in npy format) with the cnn output layer and save it as csv - so this can be input to xgboost. When I try to run this in Sagemaker notebook instance the kernel dies. The training input npy file is around 42gb and I have tried these instances : ml.m5d.24xlarge, ml.r5.24xlarge
Here is my code I am running in notebook:
'''
import numpy as np
import tensorflow as tf
import boto3
from io import BytesIO
from keras.models import load_model
from keras import backend as K
client = boto3.client("s3")
bucket = <bucket_name>
key = '/path/cnn_model.h5'
client.download_file(bucket, key, 'cnn_model.h5')
cnn_model = load_model("cnn_model.h5")
def read_s3_npy(s3_uri, arg = False):
bytes = BytesIO()
bytes_.seek(0)
parsed_s3 = urlparse(s3_uri)
obj = client.get_object(Bucket=parsed_s3.netloc, key = parsed_s3.path[1:])
return np.load(BytesIO(obj['Body'].read()), allow_pickle=arg)
x_train_path = <path in s3>+'x_train.npy'
y_train_path = <path in s3>+'y_train.npy'
x_train = read_s3_npy(x_train_path)
y_train = read_s3_npy(y_train_path)
last_layer_op = K.function([cnn_model.layers[0].input], [cnn_model.layers[-2].output])
train_layer = last_layer_op([x_train, 1])[0]
'''