To be able to use conditionals in DeletionPolicy you have to use the AWS::LanguageExtensions transform. How would I add this to a Serverless template yaml-file?
Related
I'm running a simple penguin pipeline in interactive mode with a split train/eval, the transform step run but i can't get post_transform_statistics artifacts.
Inside the dedicated artifacts folder /tmp/tfx-penguin_custom_INTERACTIVE-nq5dn56x/Transform/post_transform_stats/5, i have just one FeaturesStats.pb inside, but not subfolders Split-train and Split-eval with a FeaturesStats.pb inside each.
However, I have the subfolders inside artifacts dedicated to transformed examples (/tmp/tfx-penguin_custom_INTERACTIVE-nq5dn56x/Transform/transformed_examples/5/).
Here is how i define the transform components by explicitly providing splits and also disable_statistics=False:
transform = tfx.components.Transform(
examples=example_gen.outputs['examples'],
schema=schema_gen.outputs['schema'],
disable_statistics=False,
splits_config= transform_pb2.SplitsConfig(
analyze=['train'], transform=['train', 'eval']),
module_file=_transformer_module_file)
I went to the docstring and even the __init__ of the component https://github.com/tensorflow/tfx/blob/master/tfx/components/transform/component.py, it seems there is nothing i would have forgotten or mistaken but i was very disturbed to read following comment with an untraceable location for stats....
disable_statistics: If True, do not invoke TFDV to compute pre-transform
and post-transform statistics. When statistics are computed, they will
will be stored in the `pre_transform_feature_stats/` and
`post_transform_feature_stats/` subfolders of the `transform_graph`
export.
For now, the workaround is to explicitly disable stats in the transform component and define next to it, a dedicated statistics components to work on transformed features splits but it would have been great to have the splits statistics inside transform component directly.
Thanks for any help
This is expected as StatisticsGen in Transform is currently working on the entire transform dataset regardless of split/span.
To generate separate statistics for different splits, please use StatisticsGen component.
Thank you!
I have a custom section in serverless.common.yml, which is common to all the services. Some services use some additional properties in their serverless.yml appart from the defined in the file mentioned before.
My idea would be to perform something like:
custom:
- ${file(../serverless.common.yml):custom}
- myServiceCustomProperty: 1
But this does not work. Any idea about how can I achieve that behaviour?
Thank you
- ${file(../serverless.common.yml):custom} is going to dump the array from common so that wont merge. With this approach you need to add each property from the common/custom section individually:
custom:
- foo: ${file(../serverless.common.yml):custom.foo}
- myServiceCustomProperty: 1
or
custom: ${file(../serverless.common.yml):custom}
Alternatively if your setup is complex you can write a serverless.js file instead which allows JS processing. I personally use that to programically merge a dozen yml file - for example when I run sls offline I want to add/remove a bunch of stuff so that's a pretty powerful approach.
Let's say I have JSON configuration files where multiple entries have placeholders like these:
"fooUri": "https://{DOMAIN}/api/fooResource",
"userId": "{USER_GROUP}_{USER_NAME}"
I'd like to replace the {PLACEHOLDERS} with values from other configuration sources, e.g. environmental variables.
Right now, the easiest way I see is to:
Build an IConfiguration from the JSON files.
Build a separate IConfiguration from environmental variables.
Recursively iterate through all values in the first configuration and replace the placeholders with values from the second.
Is there a simpler way to do this easily with the Microsoft.Extensions.Configuration API?
Support for automatic replacement in case of JSON file change would also be nice.
I want to parse a SQL statements (ANSI SQL or HiveQL) into equivalent AST. When I try to parse statements with “lateral view explode” keywords in it, which is a valid HiveQL syntax, Babel fails with ParseException. Adding these as keywords to the default list of keywords for Babel also does not help. Can someone point me to an example where something similar has been done.
Calcite does support the lateral keyword, but it does not support the "view explode" keywords.:
https://github.com/apache/calcite/blob/master/core/src/main/codegen/templates/Parser.jj#L2083
You could extend the parser, and might be able to use the free-marker support to skip the unsupported keywords ( I haven't tried it myself ):
https://calcite.apache.org/docs/adapter.html#extending-the-parser
However, if you need to access it through the corresponding SqlNode implementation, then it will require contribution given you would need to modify the core module.
more about parser.jj:
https://stackoverflow.com/a/44467850/1332098
Is there any API to explore the definition of a feature (like the reflexion for code)?
My goal is for example to be able to do:
var myFeature = GetMyFeature("my feature guid")
var contentTypes = myFeature.Definition.ContentTypes
I know I can parse the xml files of the feature, but I'd like to avoid it.
Yes, you want to look at the SPFeature and SPFeatureDefinition types in the server object model.