After installing Magento 2.4.1 version(Enterprise version)on windows and After applying the di compile command and while applying re-indexing I am getting the following error even with the fresh Magento setup due to which I am facing the keep on uploading page issue while saving anything at the backend:-
Product Flat Data index exception: SQLSTATE[42S22]: Column not found:
1054 Unknown column 'row_id' in 'field list', query was: INSERT INTO
catalog_product_entity_tmp_indexer (row_id, entity_id,
type_id, attribute_set_id, created_at, has_options,
required_options, sku, updated_at) SELECT e.row_id,
e.entity_id, e.type_id, e.attribute_set_id,
e.created_at, e.has_options, e.required_options,
e.sku, e.updated_at FROM catalog_product_entity AS e WHERE
(e.created_in <= 1) AND (e.updated_in > 1)
Moreover, there is no table named catalog_product_entity_tmp_indexer in the database.
Related
I am trying to run the query on EMR/EMR Notebooks (Spark with Scala) -
SELECT max(version), max(timestamp) FROM (DESCRIBE HISTORY delta.`s3://a/b/c/d`)
But I am getting the following error -
The same query works fine on Databricks.
Another doubt that I have is - why does the colour of s3 location change post //.
So I tried to break the above query and only run the Describe HISTORY query. And for some reason it says -
Error Log -
An error was encountered:
org.apache.spark.sql.AnalysisException: Table or view not found: HISTORY;
at org.apache.spark.sql.catalyst.analysis.package$AnalysisErrorAt.failAnalysis(package.scala:47)
at org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveRelations$.org$apache$spark$sql$catalyst$analysis$Analyzer$ResolveRelations$$lookupTableFromCatalog(Analyzer.scala:835)
at org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveRelations$.resolveRelation(Analyzer.scala:787)
at org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveRelations$$anonfun$apply$8.applyOrElse(Analyzer.scala:817)
at org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveRelations$$anonfun$apply$8.applyOrElse(Analyzer.scala:810)
at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper$$anonfun$resolveOperatorsUp$1$$anonfun$apply$1.apply(AnalysisHelper.scala:90)
at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper$$anonfun$resolveOperatorsUp$1$$anonfun$apply$1.apply(AnalysisHelper.scala:90)
at org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(TreeNode.scala:71)
at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper$$anonfun$resolveOperatorsUp$1.apply(AnalysisHelper.scala:89)
at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper$$anonfun$resolveOperatorsUp$1.apply(AnalysisHelper.scala:86)
at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper$.allowInvokingTransformsInAnalyzer(AnalysisHelper.scala:194)
at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper$class.resolveOperatorsUp(AnalysisHelper.scala:86)
at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.resolveOperatorsUp(LogicalPlan.scala:30)
at org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveRelations$.apply(Analyzer.scala:810)
at org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveRelations$.apply(Analyzer.scala:756)
at org.apache.spark.sql.catalyst.rules.RuleExecutor$$anonfun$execute$1$$anonfun$apply$1$$anonfun$2.apply(RuleExecutor.scala:92)
at org.apache.spark.sql.catalyst.rules.RuleExecutor$$anonfun$execute$1$$anonfun$apply$1$$anonfun$2.apply(RuleExecutor.scala:92)
at org.apache.spark.sql.execution.QueryExecutionMetrics$.withMetrics(QueryExecutionMetrics.scala:141)
at org.apache.spark.sql.catalyst.rules.RuleExecutor$$anonfun$execute$1$$anonfun$apply$1.apply(RuleExecutor.scala:91)
at org.apache.spark.sql.catalyst.rules.RuleExecutor$$anonfun$execute$1$$anonfun$apply$1.apply(RuleExecutor.scala:88)
at scala.collection.LinearSeqOptimized$class.foldLeft(LinearSeqOptimized.scala:124)
at scala.collection.immutable.List.foldLeft(List.scala:84)
at org.apache.spark.sql.catalyst.rules.RuleExecutor$$anonfun$execute$1.apply(RuleExecutor.scala:88)
at org.apache.spark.sql.catalyst.rules.RuleExecutor$$anonfun$execute$1.apply(RuleExecutor.scala:80)
at scala.collection.immutable.List.foreach(List.scala:392)
at org.apache.spark.sql.catalyst.rules.RuleExecutor.execute(RuleExecutor.scala:80)
at org.apache.spark.sql.catalyst.analysis.Analyzer.org$apache$spark$sql$catalyst$analysis$Analyzer$$executeSameContext(Analyzer.scala:164)
at org.apache.spark.sql.catalyst.analysis.Analyzer$$anonfun$execute$1.apply(Analyzer.scala:156)
at org.apache.spark.sql.catalyst.analysis.Analyzer$$anonfun$execute$1.apply(Analyzer.scala:156)
at org.apache.spark.sql.catalyst.analysis.AnalysisContext$.withLocalMetrics(Analyzer.scala:104)
at org.apache.spark.sql.catalyst.analysis.Analyzer.execute(Analyzer.scala:155)
at org.apache.spark.sql.catalyst.analysis.Analyzer$$anonfun$executeAndCheck$1.apply(Analyzer.scala:126)
at org.apache.spark.sql.catalyst.analysis.Analyzer$$anonfun$executeAndCheck$1.apply(Analyzer.scala:125)
at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper$.markInAnalyzer(AnalysisHelper.scala:201)
at org.apache.spark.sql.catalyst.analysis.Analyzer.executeAndCheck(Analyzer.scala:125)
at org.apache.spark.sql.execution.QueryExecution.analyzed$lzycompute(QueryExecution.scala:76)
at org.apache.spark.sql.execution.QueryExecution.analyzed(QueryExecution.scala:74)
at org.apache.spark.sql.execution.QueryExecution.assertAnalyzed(QueryExecution.scala:66)
at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:80)
at org.apache.spark.sql.SparkSession.table(SparkSession.scala:630)
at org.apache.spark.sql.execution.command.DescribeColumnCommand.run(tables.scala:714)
at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:70)
at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:68)
at org.apache.spark.sql.execution.command.ExecutedCommandExec.executeCollect(commands.scala:79)
at org.apache.spark.sql.Dataset$$anonfun$6.apply(Dataset.scala:196)
at org.apache.spark.sql.Dataset$$anonfun$6.apply(Dataset.scala:196)
at org.apache.spark.sql.Dataset$$anonfun$52.apply(Dataset.scala:3391)
at org.apache.spark.sql.execution.SQLExecution$.org$apache$spark$sql$execution$SQLExecution$$executeQuery$1(SQLExecution.scala:83)
at org.apache.spark.sql.execution.SQLExecution$$anonfun$withNewExecutionId$1$$anonfun$apply$1.apply(SQLExecution.scala:94)
at org.apache.spark.sql.execution.QueryExecutionMetrics$.withMetrics(QueryExecutionMetrics.scala:141)
at org.apache.spark.sql.execution.SQLExecution$.org$apache$spark$sql$execution$SQLExecution$$withMetrics(SQLExecution.scala:178)
at org.apache.spark.sql.execution.SQLExecution$$anonfun$withNewExecutionId$1.apply(SQLExecution.scala:93)
at org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:200)
at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:92)
at org.apache.spark.sql.Dataset.org$apache$spark$sql$Dataset$$withAction(Dataset.scala:3390)
at org.apache.spark.sql.Dataset.<init>(Dataset.scala:196)
at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:81)
at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:644)
... 50 elided
Caused by: org.apache.spark.sql.catalyst.analysis.NoSuchTableException: Table or view 'history' not found in database 'default';
at org.apache.spark.sql.hive.client.HiveClient$$anonfun$getTable$1.apply(HiveClient.scala:81)
at org.apache.spark.sql.hive.client.HiveClient$$anonfun$getTable$1.apply(HiveClient.scala:81)
at scala.Option.getOrElse(Option.scala:121)
at org.apache.spark.sql.hive.client.HiveClient$class.getTable(HiveClient.scala:81)
at org.apache.spark.sql.hive.client.HiveClientImpl.getTable(HiveClientImpl.scala:84)
at org.apache.spark.sql.hive.HiveExternalCatalog.getRawTable(HiveExternalCatalog.scala:141)
at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$getTable$1.apply(HiveExternalCatalog.scala:723)
at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$getTable$1.apply(HiveExternalCatalog.scala:723)
at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:98)
at org.apache.spark.sql.hive.HiveExternalCatalog.getTable(HiveExternalCatalog.scala:722)
at org.apache.spark.sql.catalyst.catalog.ExternalCatalogWithListener.getTable(ExternalCatalogWithListener.scala:138)
at org.apache.spark.sql.catalyst.catalog.SessionCatalog.lookupRelation(SessionCatalog.scala:706)
at org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveRelations$.org$apache$spark$sql$catalyst$analysis$Analyzer$ResolveRelations$$lookupTableFromCatalog(Analyzer.scala:832)
UPDATED (18-Feb-2021) -> What I have tried till now.
Query Using Spark Sql -
spark.sql("SELECT max(version), max(timestamp) FROM (DESCRIBE HISTORY delta.s3://a/b/c/d)")
But this Didnt work. Same Error.
Create Spark Session with -
spark.sql.extensions=io.delta.sql.DeltaSparkSessionExtension
and spark.sql.catalog.spark_catalog=org.apache.spark.sql.delta.catalog.DeltaCatalog.
But its throwing the same error.
UPDATE 2 (18-Feb-2021) :- Trying the approach as mentioned by #alex.
Using PySpark.
It was working partly and but not completely.
Thanks in Advance.
Per documentation, to get support for DESCRIBE HISTORY you need to configure Spark SQL Extensions and Catalog by passing 2 properties (see docs):
spark.sql.extensions to value io.delta.sql.DeltaSparkSessionExtension
spark.sql.catalog.spark_catalog to value org.apache.spark.sql.delta.catalog.DeltaCatalog
Update:
For Spark 2.4.x, the Delta 0.6.1 should be used, and its documentation has following code snippet to activate extensions:
spark.sparkContext._jvm.io.delta.sql.DeltaSparkSessionExtension() \
.apply(spark._jsparkSession.extensions())
spark = SparkSession(spark.sparkContext, spark._jsparkSession.cloneSession())
I own a Magento CE 1.7 based website where we keep getting variations of the following error, usually after the daily morning site reindex:
"SQL ERROR: SQLSTATE[42S22]: Column not found: 1054 Unknown column 'main_table.data' in 'field list'
SQL QUERY: SELECT DISTINCT main_table.data, main_table.lifetime, main_table.expire, main_table.priority, additional_table.*, IFNULL(al.value, main_table.frontend_label) AS store_label FROM eav_attribute AS main_table
INNER JOIN catalog_eav_attribute AS additional_table ON additional_table.attribute_id = main_table.attribute_id
LEFT JOIN eav_attribute_label AS al ON al.attribute_id = main_table.attribute_id AND al.store_id = 1 WHERE (main_table.entity_type_id (...)"
I have a website down detector set up so that I can know immediately when the site is unavailable. However it obviously doesn't detect whenever the site is unusable due to errors such as “SQL ERROR: SQLSTATE[42S22]”. That means that my team must be constantly manually (!!!) monitoring if the site is indeed working properly (by clicking on product and category links)!
Usually we surpass the error by just cleaning the cache or performing a new reindex.
My questions are the following:
1 - Is there a way of automatically perform a detection of this type of errors in Magento so that - if it happens - a cache cleaning (or a site reindex) is immediately run and an alert is sent to the webmaster?
2 – If such an error is detected, is there a way of it not be shown to the person accessing the site? That is, if the error is detected, is it possible to immediately display a message (“We’ll be back soon”) while the cache is being cleaned or the site reindexed?
I will be grateful for any help you can provide.
Thank you!
I'm running scrapy and inserting the result into mysql database. The spider doesn't finish successfully and gives me this error:
Exception twisted._threads._ithreads.AlreadyQuit: AlreadyQuit()
I'm not sure why workers die/quit.
Edit:
Basically I used this code to insert into a table that has one field with unique index on it.
Here's the whole error that I got:
mysql_exceptions.IntegrityError: (1062, "Duplicate entry 'www.example.com' for key 'idx_url'")
2016-02-01 03:22:07 [twisted] CRITICAL:
Exception twisted._threads._ithreads.AlreadyQuit: AlreadyQuit() in > ignored
but I got this error after running for a while (sometimes close to the end)
I have single node cassandra installion with 0.8.2 . I have created some column families with cassandra-cli like
create column family demo;
Now i have to use secondary indexes over this column family.For that i need to upgrade the schema.When i try to upgrade this with cassandra-cli like
update column family demo with comparator=BytesType and column_metadata=[{column_name: col1, validation_class: UTF8Type, index_type: KEYS}];
i get the following error message
org.apache.cassandra.db.marshal.MarshalException: cannot parse 'col1'
I have tried using the bytes('col1') , assume keyword ,ascii and utf8 none of them works.
as hex bytes
While the same thing works perfectly fine with cassandra 0.8.4
You answered your own question. It's a bug in 0.8.2 and you should upgrade. (To the latest 0.8 release, which is 0.8.7 at this time.)
Warning: Cannot modify header information - headers already sent by (output started at /home/sites/superallan.com/public_html/includes/common.inc:2561) in drupal_send_headers() (line 1040 of /home/sites/superallan.com/public_html/includes/bootstrap.inc).
PDOException: SQLSTATE[42S02]: Base table or view not found: 1146 Table 'web247-sa_admin.field_data_field_embedcode' doesn't exist: SELECT field_data_field_embedcode0.entity_type AS entity_type, field_data_field_embedcode0.entity_id AS entity_id, field_data_field_embedcode0.revision_id AS revision_id, field_data_field_embedcode0.bundle AS bundle FROM {field_data_field_embedcode} field_data_field_embedcode0 WHERE (field_data_field_embedcode0.deleted = :db_condition_placeholder_0) AND (field_data_field_embedcode0.bundle = :db_condition_placeholder_1) LIMIT 10 OFFSET 0; Array ( [:db_condition_placeholder_0] => 1 [:db_condition_placeholder_1] => blog ) in field_sql_storage_field_storage_query() (line 569 of /home/sites/superallan.com/public_html/modules/field/modules/field_sql_storage/field_sql_storage.module).
As I understand it Drupal is looking for a data field that I deleted. I thought maybe it had got corrupted and Drupal couldn't find it to delete it properly. In phpMyAdmin it doesn't exist so how can I get Drupal to recognize it's not longer there and stop it showing this error at the bottom of every page?
You can see it on this page: http://superallan.com/404
Have you tried clearing the site cache, or uninstalling the module that provides the field? It looks like a reference to the field's SQL data is sticking around in your database, which is of course causing the error you posted.
This worked for me:
DELETE FROM field_config WHERE deleted = 1;
DELETE FROM field_config_instance WHERE deleted = 1;
I received no adverse effect from removing all the things that were already deleted.
Source:
http://digcms.com/remove-field_deleted_data-drupal-database/