I am facing this error when running my unit test to insert some rows into my bigquery table today :
Caused by: java.io.IOException: insufficient data written
at sun.net.www.protocol.http.HttpURLConnection$StreamingOutputStream.close(HttpURLConnection.java:3213)
at com.google.api.client.http.javanet.NetHttpRequest.execute(NetHttpRequest.java:81)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:960)
at com.google.api.client.googleapis.media.MediaHttpUploader.executeCurrentRequest(MediaHttpUploader.java:482)
at com.google.api.client.googleapis.media.MediaHttpUploader.executeCurrentRequestWithBackOffAndGZip(MediaHttpUploader.java:504)
at com.google.api.client.googleapis.media.MediaHttpUploader.executeUploadInitiation(MediaHttpUploader.java:456)
at com.google.api.client.googleapis.media.MediaHttpUploader.upload(MediaHttpUploader.java:348)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:418)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:343)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:460)
I thought it was due to the new version of google-http-client (1.16.0.rc) because i updated just before running the test. But rollbacking to 1.15.0-rc has no effect.
Any idea ?
Me too. Also, it seems like a sign that Bigquery just stops receiving any data. Because if you query you table by count(*) after this exception, result won't change anymore. If I keep my program running for a while, it will give me errors such as:
javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1343)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1371)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1355)
waiting for answers...
These errors usually happen when there is a communication failure, specially for large files.
The way to avoid it is to use resumable upload.
Related
I've had this query in BigQuery that I have been updating every day for the last few months. It's been fine - some occasional errors, but retrying has solved the problem.
Bet last few days I am getting the error: The job encountered an error during execution. Retrying the job may solve the problem.
The error description says that it's an external error, so how can I fix that?
I have been retrying (with rather long pauses in between), but I still get the error.
JobID example: bquxjob_152ced5d_169917f0145
Does anyone have any idea what's going on? Is there any data/time limitations I might encounter (but why just the last few days then)?
You can use CGP stackdriver to monitor your BigQuery process using this URL
Interesting information you can find among others are the queryTime heatmap and the Slot usage which might help you understand your problems better
On the subject of the external table usage, you can use Google transfer (See this link for details) to schedule a repeated transfer from CSV to BigQuery table.
The below Image show you how to get to the transfer set up page from the webUI
I encountered this dreadfully useless error in a scheduled query. It was working great and then one day it stopped working at all and has been failing ever since without any other explanation. The StackDriver (now "Logs Explorer") showed nothing more enlightening:
jobStatus: {
errorResult: {
code: 14
message: "Error encountered during execution. Retrying may solve the problem."
}
errors: [
0: {
code: 14
message: "Error encountered during execution. Retrying may solve the problem."
}
]
jobState: "DONE"
}
Figuring out the actual issue takes a long time because scheduled queries start slowly since they use BATCH priority. What I found in my case was that the partitioned table and "Partition field" setting in the scheduled query was the culprit. I dropped the table and removed the partition field and voila the thing works again (although far from ideal since I need partitioning).
I hope this helps someone else running up against that useless error but in any case, I hope the good folks working on BigQuery find a better error to bubble up.
I ran into this problem when replacing the contents of a partitioned table. Two retries did not help. When I removed the --range_partitioning from the command the update was processed correctly. The table remained partitioned.
So there seems to be an issue about updates to partitioned tables, and when that is the cause these errors might not benefit from retry. I don't know whether there are other causes of this error.
This kind of issue probably has a lot to do with BigQuery quota errors : https://cloud.google.com/bigquery/docs/troubleshoot-quotas#ts-number-column-partition-quota, as mentionned by other answers, such as the 4000 partitions-by-table quota.
On attempting to execute a simple query on a table (dimensions: 1,131,714,069 rows by 22 columns), I am running into the error:
[Amazon][JDBC](12080) Statement object has been closed.
Research online has unfortunately not provided much insight into this error.
I will not encounter this error each time I execute a query; so far it seems that its occurrence is unpredictable. The query that most recently caused this error looked was a very simple SELECT ... FROM ... WHERE with no subqueries and only one condition in the WHERE clause.
The query was busy for about 22 minutes before failing, however after waiting a few minutes, then running it again, it completed successfully in a matter of seconds. That being said, this kind of unpredictability and unreliability is exactly what I'm trying to prevent against.
If it helps, the IDE that I am using to connect to my Redshift database is TeamSQL.
What could be causing this error, and what steps could I take to prevent it?
I have a designed a report which works well. I am trying to add another sql query in it. Previewing the sql query works without any hiccups.
But when I try to add the parameter for the same query, I get two different errors depending on which mysql-connector I am using.
Earlier I was using the mysql-connector-java-5.0.8-bin and the error
was
org.pentaho.reporting.engine.classic.core.ReportDataFactoryException: Failed at query: (a few lines down )
Caused by: java.sql.SQLException: Stopped by user.
Then I changed the mysql-connector to mysql-connector-java-5.1.36-bin. The error changed to :-
org.pentaho.reporting.engine.classic.core.ReportDataFactoryException: Failed at query:
Caused by: com.mysql.jdbc.exceptions.MySQLTimeoutException: Statement cancelled due to timeout or client request
Any suggestions would be helpful. I am using Pentaho 5.3.0.0-213 on Windows 8.1 Pro. Although the same problem exists when I run Pentaho 5.3.0.0-213 on Ubuntu 14.04
Thanks
I am executing a insert into ... select ... from ... where ... SQL and got following error using Oracle:
java.sql.SQLException: ORA-12801: error signaled in parallel query server P004
ORA-01555: snapshot too old: rollback segment number 32 with name "_SYSSMU32_2039035886$" too small
I read the following doc: http://www.dba-oracle.com/t_ora_12801_parallel_query.htm and http://www.dba-oracle.com/t_ora_01555_snapshot_old.htm
Saying ORA-12801 is caused by no enough processors to support parallel query. ORA-01555 error relates to insufficient undo storage or a too small value for the undo_retention parameter.
But how can I check related parameters to avoid such issue recur?
ORA-12801 is a generic error message and we must check the second message on the error stack to find the real error. From the manual:
ORA-12801: error signaled in parallel query server string
Cause: A parallel query server reached an exception condition.
Action: Check the following error message for the cause, and consult your error manual for the appropriate action.
There are literally thousands of different reasons for an ORA-12801 error, and that error almost never has anything to do with not enough processors. This is an example of how the site you linked to often contains bad or outdated information. Maybe 17 processes was "a lot" 17 years ago but it's not today. Unfortunately, that site is often the first result from Google.
For troubleshooting your second error, ORA-01555, check the UNDO retention, which is the amount of time in seconds, like this:
select value from v$parameter where name = 'undo_retention'
The amount of space available for the UNDO tablespace is also relevant:
select round(sum(maxbytes)/1024/1024/1024) gb
from dba_data_files
where tablespace_name like '%UNDO%';
Once again, see the manual for more information on the parameter.
I'm simply doing a select * from one of my tables. It is a daily table containing yesterday's data and was working fine yesterday and earlier today. It is suddenly returning "Error: Unexpected. Please try again."
Are there any production issues currently? The job id for one of these failed queries is job_Tj_vZhAqfZH9jpW52M8IUld4XKE
A configuration change this morning caused a problem querying from tables that had been written to via streaming. The issue should be fixed now. Please ping this thread if you continue to see problems.