I am running a Redshift cluster (version 1.0.39380) that fails when running a VACUUM DELETE ONLY <table> query.
This is the output I'm getting.
Query 1 ERROR: ERROR: Assert
DETAIL:
-----------------------------------------------
error: Assert
code: 1000
context: state->m_tbl_rowid_encoding_type == EncodeRawAuto || (state->m_tbl_rowid_encoding_type == EncodeFor64Auto && gconf_enable_rowid_compression_with_az64 && gconf_enable_vacuum_for_rowid_compressed_table) - Tableid108790 is rowid compressed with0 encoding and
query: 1531591
location: xen_vacuum.cpp:4772
process: query1_24_1531591 [pid=16344]
-----------------------------------------------
To give a bit more context, this is a regular job that runs 3 times a week. Other queries have been running fine on the same table, also on other tables.
The cluster is not particularly under pressure (not much as usual).
What may be the cause of this error?
Related
I was running into troubles today while running Airflow and airflow-dbt-python. I tried to debug a bit using the logs and the error shown in the logs was this one:
[2022-12-27, 13:53:53 CET] {functions.py:226} ERROR - [0m12:53:53.642186 [error] [MainThread]: Encountered an error:
Database Error
Expecting value: line 2 column 5 (char 5)
Quite a weird one.
Possibly check your credentials file that allows DBT to run queries on your DB (in our case we run DBT with BigQuery), in our case the credentials file was empty. We even tried to run DBT directly in the worker instead of running it through airflow, giving as a result exactly the same error. Unfortunately this error is not really explicit.
I am executing dbt run -s model_name on CLI and the task completes successfully. However, when I run the exact same command on dbt cloud, I get this error:
Syntax or semantic analysis error thrown in server while executing query.
Error message from server: org.apache.hive.service.cli.HiveSQLException:
Error running query: org.apache.spark.sql.AnalysisException: cannot
resolve '`pv.meta.uuid`' given input columns: []; line 6 pos 4;
\n'Project ['pv.meta.uuid AS page_view_uuid#249595,
'pv.user.visitorCookieId AS (80) (SQLExecDirectW)")
it looks like it fails recognizing 'pv.meta.uuid' syntax which extract data from a json format. It is not clear to me what is going on. Any thoughts? Thank you!
I have 14 scheduled queries that run hourly from Google Sheets but they fail half of the time. I don't understand the error status though as the queries do run successfully half of the time. The error reads:
Error status: Invalid value Error while reading table: tester-253410.test1.Table_View_2_Agent_Targets, error message: Failed to read the spreadsheet. Errors: Deadline=118.888051456s; JobID: tester-253410:5e59a150-0000-2421-b469-001a1144591c
Is there anything that I can try?
I have two tables. Table 1 is a large one, and Table 2 is a small one. I would like to extract data from Table 1 if values in Table1.column1 matches those in Table2.column1. Both Table 1 and Table 2 have column, column1. Here is my code.
select *
from Table1
where condition1
and condition2
and column1 in (select column1 from Table2)
Condition 1 and Condition 2 are meant to restrict the size of the table to be extracted. Not sure if this actually works. Then I got execution error, return code 1. I am on Hue platform.
EDIT
As suggested by #yammanuruarun, I tried the following code.
SELECT *
FROM
(SELECT *
FROM Table1
WHERE condition1
AND condition2) t1
INNER JOIN Table2 ON t1.column1 = t2.column1
Then, I got the following error.
Error while processing statement: FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.tez.TezTask. Application
application_1580875150091_97539 failed 2 times due to AM Container for
appattempt_1580875150091_97539_000002 exited with exitCode: 255 Failing this
attempt.Diagnostics: [2020-02-07 14:35:53.944]Exception from container-launch.
Container id: container_e1237_1580875150091_97539_02_000001 Exit code: 255
Exception message: Launch container failed Shell output: main : command provided 1
main : run as user is hive main : requested yarn user is hive Getting exit code
file... Creating script paths... Writing pid file... Writing to tmp file /disk-
11/hadoop/yarn/local/nmPrivate/application_1580875150091_97539/container_e1237_1580875150091_97539_02_000001/container_e1237_1580875150091_97539_02_000001.pid.tmp
Writing to cgroup task files... Creating local dirs... Launching container...
Getting exit code file... Creating script paths... [2020-02-07 14:35:53.967]Container exited with a non-zero exit code 255. Error file: prelaunch.err. Last 4096 bytes of prelaunch.err : Last 4096 bytes of stderr :
Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in
thread "IPC Server idle connection scanner for port 26888" Halting due to Out Of
Memory Error... Halting due to Out Of Memory Error... Halting due to Out Of Memory
Error...
Halting due to Out Of Memory Error... Halting due to Out Of Memory Error...
Halting due to Out Of Memory Error... Halting due to Out Of Memory Error...
Halting due to Out Of Memory Error... [2020-02-07 14:35:53.967]Container exited
with a non-zero exit code 255. Error file: prelaunch.err. Last 4096 bytes of prelaunch.err : Last 4096 bytes of stderr :
Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in
thread "IPC Server idle connection scanner for port 26888" Halting due to Out Of Memory Error... Halting due to Out Of Memory Error...
Halting due to Out Of Memory Error... Halting due to Out Of Memory Error...
Halting due to Out Of Memory Error... Halting due to Out Of Memory Error...
Halting due to Out Of Memory Error... Halting due to Out Of Memory Error...
For more detailed output, check the application tracking page: http://dcwipphm12002.edc.nam.gm.com:8088/cluster/app/application_1580875150091_97539 Then click on links to logs of each attempt. . Failing the application.
Looks like it is a memory error. Is there any way I could optimize my query?
Using the shakespeare public dataset, I tried to run the following (full code of the query, plus error):
` bq query "SELECT word FROM publicdata:samples.shakespeare WHERE word = 'huzzah' IGNORE CASE"
Waiting on bqjob_ref3f8f63522c642_0000014452358cb2_1 ... (0s) Current status: DONE
Bigquery service returned an invalid reply in query operation: Not enough rows returned by
server for job 'test-rich-app:bqjob_ref3f8f63522c642_0000014452358cb2_1'.
Please make sure you are using the latest version of the bq tool and try again. If this problem
persists, you may have encountered a bug in the bigquery client. Google engineers monitor and
answer questions on Stack Overflow, with the tag google-bigquery:
https://stackoverflow.com/questions/ask?tags=google-bigquery
Please include a brief description of the steps that led to this issue, as well as the
following information:
========================================
== Platform ==
CPython:2.7.2:Darwin-12.5.0-x86_64-i386-64bit
== bq version ==
v2.0.17
== Command line ==
['/Users/rich/google-cloud-sdk/platform/bigquery/bq.py', '--credential_file', '/Users/richmorrow/.config/gcloud/legacy_credentials/rich#quicloud.com/singlestore.json', 'query', "SELECT word FROM publicdata:samples.shakespeare WHERE word = 'huzzah' IGNORE CASE"]
== UTC timestamp ==
2014-02-21 02:10:46
== Error trace ==
File "/Users/rich/google-cloud-sdk/platform/bigquery/bq.py", line 783, in RunSafely
return_value = self.RunWithArgs(*args, **kwds)
File "/Users/rich/google-cloud-sdk/platform/bigquery/bq.py", line 1134, in RunWithArgs
max_rows=self.max_rows)
File "/Users/rich/google-cloud-sdk/platform/bigquery/bigquery_client.py", line 804, in ReadSchemaAndJobRows
return reader.ReadSchemaAndRows(start_row, max_rows)
File "/Users/rich/google-cloud-sdk/platform/bigquery/bigquery_client.py", line 2095, in ReadSchemaAndRows
'Not enough rows returned by server for %r' % (self,))
Unexpected exception in query operation: Not enough rows returned by server for job
'test-rich-app:bqjob_ref3f8f63522c642_0000014452358cb2_1'`
This issue happens with google cloud SDK and BigQuery CLI v2.0.15
Works fine with: BigQuery CLI v2.0.11
Sounds like you're using an old version of bq. The bq tool is now distributed as part of the google cloud SDK... you can install it here:
https://developers.google.com/cloud/sdk/
Essentially the issue was that when queries returned 0 rows, the old version of bq failed with the error you're seeing.