Download a page doesn't return a status code - scrapy

Question

I have found a page I need to download that doesn't include an http status code in the returned headers. I get the error: ParseError: ('non-integer status code', b'Tag: "14cc1-5a76434e32f9e"') which is obviously accurate. But otherwise the returned data is complete.
I'm just trying to save the page content manually in a call back: afilehandle.write(response.body) sort of thing. It's a pdf. Is there a way I can bypass this and still get the contents of the page?
The returned example that also crashed fiddler. The first thing in the header is Tag.
Tag: "14cc1-5a76434e32f9
e"..Accept-Ranges: bytes
..Content-Length: 85185.
.Keep-Alive: timeout=15,
max=100..Connection: Ke
ep-Alive..Content-Type:
application/pdf....%PDF-
1.4.%ÓôÌá.1 0 obj.<<./Cr
eationDate(D:20200606000
828-06'00')./Creator(PDF
sharp 1.50.4740 \(www.pd
fsharp.com\))./Producer(
PDFsharp 1.50.4740 \(www
.pdfsharp.com\)).>>.endo
bj.2 0 obj.<<./Type/Cata
log./Pages 3 0 R.>>.endo
bj.3 0 obj.<<./Type/Page
s./Count 2./Kids[4 0 R 8
0 R].>>.endobj.4 0 obj.
<<./Type/Page./MediaBox[
0 0 612 792]./Parent 3 0
R./Contents 5 0 R./Reso
urces.<<./ProcSet [/PDF/
Text/Ima.... etc
Note: For any not familiar with PDF file structure %PDF-1.4 and everything after is the correct format for a PDF document. Chrome downloads the PDF just fine even with the bad headers.

Answer 1

In the end, I modified the file twisted/web/_newclient.py directly to not throw the error, and use a weird status code that I could identify:
def statusReceived(self, status):
parts = status.split(b' ', 2)
if len(parts) == 2:
version, codeBytes = parts
phrase = b""
elif len(parts) == 3:
version, codeBytes, phrase = parts
else:
raise ParseError(u"wrong number of parts", status)
try:
statusCode = int(codeBytes)
except ValueError:
# Changes were made here
version = b'HTTP/1.1' #just assume it is what it should be
statusCode = 5200 # deal with invalid status codes later
phrase = b'non-integer status code' # sure, pass on the error message
# and commented out the line below.
# raise ParseError(u"non-integer status code", status)
self.response = Response._construct(
self.parseVersion(version),
statusCode,
phrase,
self.headers,
self.transport,
self.request,
)
And I set the spider to accept that status code.
class MySpider(Spider):
handle_httpstatus_list = [5200]
However, in the end I discovered the target site behaved correctly when accessed via https, so I ended up rolling back all the above changes.
Note the above hack would work, until you updated the library, at which point you would need to reapply the hack. But it could possibly get it done if you are desparate.