I am attempting to completely rewrite footer.liquid. Unfortunately, there are pre-existing fields which I am no longer using which are causing errors to be thrown when I attempt to save my file. For example:
Error: New schema is incompatible with the current setting value. Invalid type value for block 'linklist'. Type must be defined in schema.
I understand that this is because I am not using an "id": "linklist" in my new schema. Is there a way to flush out these pre-existing schema types so that I don't have to include it in my schema?
If you delete the section in the theme editor and then readd the section after, that should solve the problem.
Alternatively, you could find the settings_data.json and remove the current values associated with your footer section.
Related
I am using DataFlow's WriteToBigQuery with CREATE_IF_NEEDED, and thus have to specify the schema.
I define the schema in the beginning of my code (outside the actual pipeline), but since I need the flag --save_main_session, I get the same error as here, which explains that the schema cannot be passed along with the pipeline since a BigQuery schema definition is not pickleable.
The solution mentioned on the page is not an option for me (disable the --save_main_session flag), and thus the other option to specify the schema is through a string.
However, I need to set some fields to REQUIRED. Is there a way to do this with the string schema definition?
As you can see from bigquery.py the conversion from a string schema to a TableSchema is quite straightforward and does indeed set the mode to NULLABLE. Perhaps you can create the TableSchema with REQUIRED fields based on this code snippet.
Every morning, an automatic job creates a new table from an Avro file. In the afternoon, I would need to append some data to this table from a Query.
When trying to do so, I get the following error:
Error: Invalid schema update. Field chn has changed mode from REQUIRED to NULLABLE
I noticed that I can change the property of the field chn from REQUIRED to NULLABLE in the BigQuery Web UI and then it works fine, but I would have to do it manually everyday which is not what I am looking for.
Is there a way to "cast" the field as REQUIRED during the append query ?
Or during the first import from the Avro file, force the field to be NULLABLE and not REQUIRED ?
Thanks !
The feature that allows relaxing a field as part of a query or a load job will be available in production shortly. I will update this answer when it goes live (likely within a week).
Update: 08/25/2016
You can supply schemaUpdateOptions in load or query job configuration.
Multiple options can be provided.
It allows the schema of the destination table to be updated as a side effect of the load or query job. Schema update options are supported in two cases:
When writeDisposition is WRITE_APPEND
When writeDisposition is WRITE_TRUNCATE and the destination table is a partition of a table, specified by partition decorators
For non-partitioned tables, WRITE_TRUNCATE will always overwrite the schema.
The following values are supported:
ALLOW_FIELD_ADDITION: allow adding a nullable field to the schema
ALLOW_FIELD_RELAXATION: allow relaxing a required field in the original schema to nullable
NOTE: This doesn't currently work with schema auto-detection. We plan to support that soon.
I'm getting the following error:
Error during parsing. repetition constraint is more restrictive: can not merge type required binary MyTime into optional binary MyTime.
Maybe one of the files is corrupted but I don't know how to skip it.
Thanks
This happens when reading multiple parquet files that have slightly different metadata in their schemas. Either you have a mixed collection of files in a single directory or you are giving the LOAD statement a glob and the resulting collection of files is mixed in this respect.
Rather than specifying the schema in an AS() clause or making a bare call to the loader function the solution is to override the schema in the loader function's argument like this:
data = LOAD 'data'
USING parquet.pig.ParquetLoader( 'n1:int, n2:float, n3:double, n4:long')
Otherwise the loader function infers the schema from the first file it encounters which then conflicts with one of the others.
If you have still have trouble try using type bytearray in the schema specification and then cast to the desired types in a subsequent FOREACH.
According to the Parquet source code there is another argument to the loader function that allows columns to be specified by position rather than name (the default) but I have not experimented with that.
I have a directory containing multiple xlsx files and what I want to do is to insert the data from the files in to a database.
So far I have solved this by using tFileList -> tFileInputExcel -> tPostgresOutput
My problem begins when one of this files doesn't match the defined schema and returns an error resulting on a interruption of a workflow.
What I need to figure out is if it's possible skip that file (moving it to another folder for instance) and continuing iterating the rest of existing files.
If I check the option "Die on error" the process ends and doesn't process the rest of the files.
I would approach this by making your initial input schema on the tFileInputExcel be all strings.
After reading the file I would then validate the schema using a tSchemaComplianceCheck set to "Use another schema for compliance check".
You should be able to then connect a reject link from the tSchemaComplianceCheck to a tFileCopy configured to move the file to a new directory (if you want it to move it then just tick "Remove source file").
Here's a quick example:
With the following set as the other schema for the compliance check (notice how it now checks that id and age are Integers):
And then to move the file:
Your main flow from the tSchemaComplianceCheck can carry on using just strings if you are inserting into a database. You might want to use a tConvertType to change things back to the correct data types after this if you are doing any processing that requires proper data types or you are using your tPostgresOutput component to create the table as well.
Using SSRS2005, can you pull in values specified in the SSRS Web.Config file (think appSettings)? I'm needing to build up a dynamic hyperlink in a series of reports and would like this to be based on a value set in a config file.
At present, it seems my only options are to:
Update the hardcoded variable in each report that is building up the links blech
Store the value in a dedicated configuration table and then query that
I could live with #2 above, but would like to avoid having to create yet another table to store this information.
I don't believe there is an out of the box way to access files from reports. Here are a few other options to add to your list:
Create a [custom assembly](http://msdn.microsoft.com/en-us/library/ms153561(SQL.90).aspx) that either has the values or looks them up in some configuration file (i.e. xml file). Going down this path will probably open up areas where you need to modify the [security settings](http://msdn.microsoft.com/en-us/library/ms155108(SQL.90).aspx).
To build off of your configuration table idea... one thing we ended up having to do was create a fairly generic table that represented the appSettings format of a .NET config file. The appSettings is nothing more than a name/value pair. As a table, this pretty much translates to two fields (name, value). We then use this table to hold all sorts of config based options that would otherwise belong in a appSettings section of a config file.
I hope this provides some help. Good luck!