when enabling errors to bigquery I do not receive the bad record number - google-bigquery

Question

I'm using bigquery command line tool to upload these records:
{name: "a"}
{name1: "b"}
{name: "c"}
.
➜ ~ bq load --source_format=NEWLINE_DELIMITED_JSON my_dataset.my_table ./names.json
this is the result I get:
Upload complete.
Waiting on bqjob_r7fc5650eb01d5fd4_000001560878b74e_1 ... (2s) Current status: DONE
BigQuery error in load operation: Error processing job 'my_dataset:bqjob...4e_1': JSON table encountered too many errors, giving up.
Rows: 2; errors: 1.
Failure details:
- JSON parsing error in row starting at position 5819 at file:
file-00000000. No such field: name1.
when I use bq --format=prettyjson show -j <jobId> I get:
{
"status": {
"errorResult": {
"location": "file-00000000",
"message": "JSON table encountered too many errors, giving up. Rows: 2; errors: 1.",
"reason": "invalid"
},
"errors": [
{
"location": "file-00000000",
"message": "JSON table encountered too many errors, giving up. Rows: 2; errors: 1.",
"reason": "invalid"
},
{
"message": "JSON parsing error in row starting at position 5819 at file:
file-00000000. No such field: name1.",
"reason": "invalid"
}
],
"state": "DONE"
}
}
As you can see I receive an error which tells me in what line I had an error. : Rows: 2; errors: 1
Now I'm trying to enable errors by using max_bad_errors
➜ ~ bq load --source_format=NEWLINE_DELIMITED_JSON --max_bad_records=3 my_dataset.my_table ./names.json
here is what I receive:
Upload complete.
Waiting on bqjob_...ce1_1 ... (4s) Current status: DONE
Warning encountered during job execution:
JSON parsing error in row starting at position 5819 at file: file-00000000. No such field: name1.
when I use bq --format=prettyjson show -j <jobId> I get:
{
.
.
.
"status": {
"errors": [
{
"message": "JSON parsing error in row starting at position 5819 at file: file-00000000. No such field: name1.",
"reason": "invalid"
}
],
"state": "DONE"
},
}
when I check - it actually uploads the good records to the table and ignores the bad record,
but now I do not know in what record the error was.
Is this a big query bug?
can it be fixed so I receive record number also when enabling bad records?

Answer 1

Yes this is what max_bad_records does. If the number of errors is below max_bad_records the load will succeed. The error message tells you the start position of the failed line, 5819, and the file name, file-00000000. The file name is changed since you're doing an upload and load.
The previous "Rows: 2; errors: 1" means 2 rows are parsed and there is 1 error. It's not always the 2nd row in the file. A big file can be processed by many workers in parallel. Worker n starts processing at position xxxx, parsed two rows, and found an error. It'll also report the same error message and apparently 2 doesn't mean the 2nd row in the file. And it doesn't make sense for worker n to scan the file from beginning to find out which line it starts with. Instead, it'll just report the start position of the line.