I have a JSON that I want to insert into BQ. The column data type is STRING. Here is the sample JSON value.
"{\"a\":\"#\",\"b\":\"b value\"}"
This is a bulk load from a CSV file.
The error I'm getting is
Error: Data between close double quote (\") and field separator."; Reason: "invalid"},Invalid {Location: ""; Message: "Error while reading data, error message: CSV processing encountered too many errors, giving up. Rows: 0; errors: 1; max bad: 0; error percent: 0"; Reason: "invalid"}
Thanks!
I think there is an issue with how you escape the double quotes.
I could reproduce the issue you describe, and fixed it by escaping the double quotes with " instead of a backslash \:
"{""a"":""#"",""b"":""b value""}"
This information is well-hidden in the doc there (in the "Quote" section):
For example, if you want to escape the default character ' " ', use ' "" '.
Related
I am trying to push data from Databricks into SQL. However, I get the following error:
I noticed in the file that I am processing that one of the columns has the following as a value:
I have tried to filter out the records by using the following:
df = df.filter(df.COLUMN != "\N")
However, when the above runs, I get the error message idenitfied above. Is there some way to filter out values that have an escape character in them?
I would really appreciate any help. Thank you.
As the error suggested, you need to escape the backslash \
df.filter(df.value != "\\N")
Getting this error when trying to import data from txt to my Db
17 values were expected, but found 13. (near "(" at position 20627)
The line of data where the problem appears is this:
('35','835','835','32','57804','Ex sit at id et.','Fanopoiia','14660.36347','10','Funk','Mixanikos','835','Duchess; 'and most of 'em do.' 'I don't see any.','1982-07-02 141548','7','Nai','1973-01-25 060239'),
You may escape the apostrophes with double qoutes: ''
Check How to insert a value that contains an apostrophe (single quote)?
Select * from mytable where field=
'ce7bd3d4-dbdd-407e-a3c3-ce093a65abc9;cdb597073;7cf6cda5fc'
Getting Below Error while running above query in Hive
FAILED: ParseException line 1:92 character '' not supported here
<EOF> here means End Of File. When you get an "unexpected End Of File" error it means the parser reached the end of the query unexpectedly. This typically happens when the parser is expecting to find a closing character, such as when you have started a string with ' or " but have not closed the string (with the closing ' or ").
When you come across these types of errors it is good to check that your query can be parsed correctly. In addition, the error gives you the location where the parser failed: line 1:92 in this case. You can usually look at this location (character 92 of the query) and work backwards to find the problem character.
Try adding the database name to the "from" statement as below.
Select * from my_db_name.mytable where field= 'ce7bd3d4-dbdd-407e-a3c3-
ce093a65abc9;cdb597073;7cf6cda5fc';
Hive uses the default database when no database was previously specified.
Errors were reported when my program tried to upload a .csv file, via job upload to BigQuery:
Job failed while writing to Bigquery. invalid: Too many errors encountered. Limit is: 0. at
Error: [REASON] invalid [MESSAGE] Data between close double quote (") and field separator: field starts with: <N> [LOCATION] File: 0 / Line:21470 / Field:2
Error: [REASON] invalid [MESSAGE] Too many errors encountered. Limit is: 0. [LOCATION]
I traced back to my file and did find the specified line like:
3D0F92F8-C892-4E6B-9930-6FA254809E58~"N" STYLE TOWING~1~0~5.7.1512.441~10.20.10.25:62342~MSSqlServer: N_STYLE on localhost~3~2015-12-17 01:56:41.720~1~<?xml version="1
The delimiter was set to be ~ , then why the double quote or maybe <N> is a problem?
The specification for csv says that if there is a quote in the field, then the entire field should be quoted. As in a,b,"c,d", which would have only three fields, since the third comma is quoted. The csv parser gets confused when there is data after a closing quote but before the next delimiter, as in a,b,"c,d"e.
You can fix this by specifying a custom quote character, since it sounds like you don't need a quote char at all, so you could just set it to something that you'll never see, like \0 or |. You're already setting configuration.load.delimiter, just set configuration.load.quote as well.
I have a string that is read from an excel file. String is ¬©K!n?8)©}gH"$.F!r'&(®
I keep getting the following error. When running the insert statement
ERROR [42601] [IBM][CLI Driver][DB2/NT] SQL0007N The character "\" following " K!n?8) }gH""$.F!r'" is not valid. SQLSTATE=42601
How do I do this insert with the \ character in the string? The Character causing the problem might also be the &.
Its a DB2 Database
Thanks
It depends on which interface you are sending the insert statement.
In my experience, it is most probably the typewriter single quote character (') which makes the problem.
INSERT INTO TEMP.CHAR_TEST
VALUES ('¬©K!n?8)©}gH"$.F!r''&(®');
works fine for me, but notice that I have changed ' to '' (insert another single quote after the existing one).