I have a simple flow consisting of
GenerateFlowFile ----> PutS3Object ----> Wait
And the generated flow files are getting stored in the bucket correctly.
Now I want to add Metadata to my flow file.
If I add a property "Test1" to PutS3Object, it shows up as "X-Amz-Meta-Test1" in the metadata of the object.
But if I add a property "Test2" in GenerateFlowFile it doesn't show up in metadata.
I tried adding "Test2" as s3.usermetadata.Test2 but it still didn't work.
Is there a way to pass all the flow files attributes as metadata without explicitly adding properties in the PutS3Object.
PutS3Object only inserts metadata values that you have set as Dynamic Properties on the PutS3Object processor itself. Please see the docs link and look at the Dynamic Properties section.
PutS3Object does not just stick any Attribute you set as metadata, otherwise you would end up with potentially hundreds of metadata entries that you aren't interested in. The only Attribute it reads by default is filename - please see the Reads Attributes section of the docs.
If you have an existing Attibute, and you want to push the value of this Attribute into the metadata, you must add a Dynamic Property to PutS3Object and reference the value of the Attribute.
E.g. you have an Attribute called file_author with a value Steve and you want the S3 object to have the metadata field author with the value Steve:
You would add a Dynamic Property to PutS3Object with a name of author and a value of ${file_author}.
Edit:
You could fork PutS3Object into a custom processor to add the dynamic functionality you want, but I would recommend just using the standard PutS3Object config and manually configuring the Attributes you want.
Related
In a dataflow job written in Kotlin
using a PubSub subscription as input i receive a Proto object (Event) and map this object to Strings.
My pipeline has type:
PCollection<KV<Event, String>>
These strings are the lines of a file that must be written in GCS.
The Event Object has a "Id" that must be used to set the filename, and a "name" to set the folder.
Is it possible using FileIO ?
pipeline.apply(
FileIO.writeDynamic<String, String>()
.to("gs://my-bucket")
// withNaming?
)
My goal is to write the right lines in the right files, based on the information in the Event object
File-names can be customized by providing a FileNaming implementation to the withNaming() API.
However this currently does not support mapping input elements directly to final file names. Input elements can be mapped to groups using the dynamic destinations API and for each group you can provide a file-naming strategy.
To fully customize naming using input element values you might need to implement a new sink transform.
I need to insert some data from an XML document into an SQL table using a stored procedure. This should be done using the WCF-SQL adapter on a send port in Biztalk. I've set up the connection properties on the adapter, but I am missing the message part.
I'm currently using the below code with some hardcoded values inside a body template:
<SaveDocument xmlns="http://schemas.microsoft.com/Sql/2008/05/Procedures/dbo">
<conversationID>123456</conversationID>
<dataType>OIO</dataType>
<fromID></fromID>
<toID></toID>
<msgInfoExtension><![CDATA[<infoExt><fileInfo fileName="ublinvoice.xml" encoding="utf-8" /></infoExt>]]></msgInfoExtension>
<msgBody>
<bts-msg-body xmlns="http://www.microsoft.com/schemas/bts2007" encoding="string"/>
</msgBody>
<msgBodyBin></msgBodyBin>
</SaveDocument>
Instead of the hardcoded values, I need to grab some of the promoted properties in an XML document, but I'm unsure how to correctly go about doing this.
In the MS documentation I found this page, which describes the structure of the request: https://learn.microsoft.com/en-us/biztalk/adapters-and-accelerators/adapter-sql/message-schemas-for-procedures-and-functions
Distinguish the properties that you want to set in the SaveDocument schema
Create a map that populates the fields, the ones you want to set from the Promoted properties, just fill them with some constants from the map.
In an Orchestration have a Construct Message that contains both the map shape pointing to the above map and a message assignment shape.
In the message assignment shape have some statements like saveSG.conversationID = sourceMSG(PropmotedProperty).
P.S. I think you have manually tried to create the SaveDocument schema. This is probably not correct. The correct way is to create the stored procedure and then use the Generate Instance dialogue in Visual Studio, and to configure that to auto generate the schema for you, and as a bonus, it even generates a binding file containing the settings you need for your send port.
You might be able to use the often overlook Property Demotion feature.
It's basically the reverse of Promotion and is done by the XmlAssembler component.
How to access the corresponding FieldDefinition, if I have only the jcr property name at hand.
Is there any desired functionality,like find a field definition by a jcr property name ? If not, how would I access configured field definitions in java code ?
Scenario : I need to determine, if a given jcr property name was configured to be an i18n-capable field in definition ?
The answer is you can never be sure if property was configured to be i18n-able (or have any other trait) at the time it was last edited as definition might have changes since.
However to achieve what you want (obtain the definition), what you need to do is:
get a parent node of the property in question,
get value of it's mgnl:template property. That's your templateId,
use the templateId to obtain template from the registry,
read value of dialog property of the template, that's your dialogId
use the dialogId to obtain dialog from the registry
scan tabs in the dialog to find property definition with name of the jcr property you have started with
I have a SoapUI project which contains around 60 plus services. Each service requires some input which will be changed for every execution. So I have created certain Global Properties and assign some values to that properties.
I have to use these properties values in my SoapUI request ( i.e. JSON Format request ).
If it is groovy script means, I will use like this.
String HTiC_Username = com.eviware.soapui.model.propertyexpansion.PropertyExpansionUtils.globalProperties['HTiC_Username'].value;
But, how to get the value of the Global Property in the request?
Hope you understand my question. Please provide proper guidance.
Thanks
To dynamically "expand" (i.e. substitute) the value of a property into a test step, the following syntax is used: ${#scope#propertyName}
...where 'scope' refers to the level at which the property has been defined (e.g. Global, Project, TestSuite, TestCase).
So to expand a property named username defined as a Global property, for example, the following code can be used directly within a Request Test Step (e.g within a JSON body, or header value, etc):
${#Global#username}
To access the same property value within a Groovy Test Step, you can use the following syntax:
context.expand('${#scope#propertyName}')
...as in the following example:
context.expand('${#Global#username}')
What we did was the following:
created a test data file to store all the specific input data for the different services (testdata.properties)
Example content of testdata.properties:
Billing_customerID=1234567
OtherService_paymentid=12121212
....
create a SoupUi global parameter (File/Preferences/Global properties): testdata_filepath=C:\...
For specific services we added a Properties test step. You can specify the "Load from" field to our new global parameter: ${#Global#testdata_filepath} Now you can use the Load button to load parameters.
Finally you can reference the parameter in your xml in the following format: ${Properties#Billing_customerID}
Example content of a service with parameter:
...
<BillingCustomerIdentification>
<BillingCustomerID>${#Properties#Billing_customerID}</BillingCustomerID>
</BillingCustomerIdentification>
...
To set up your projects in this manner also helps to automate service tests eg. using Hudson (see my previous SO answer).
If it is too heavy and automation is not a target, you can simply use ${#Global#someinputvariable} format in your xml ;-)
I have a custom content type with 3 FileFields (plone.app.blob.field.FileField) and I want to get their url's, so i can put them on my custom view and people will be able to download these files.
However, when using Clouseau to test and debug, I call :
context.getFirst_file().absolute_url()
Where getFirst_file() is the accessor to the first file (field called 'first_file').
The url returned is 'http://foo/.../eat.00001', where 'eat.00001' is the object of my custom type that contains the file fields...
The interesting thing is, if I call:
context.getFirst_file().getContentType()
It returns 'application/pdf', which is correct since it's a pdf file.
I'm pretty lost here, any help is appreciated. Thanks in advance!
File fields do not support a absolute_url method; instead, through acquisition you inherit the method from the object itself, hence the results you see. Moreover, calling getFirst_field() will return the actual downloadable contents of the field, not the field itself which could provide such information.
Instead, you should use the at_download script appended to the object URL, followed by the field id:
First File
You can also re-use the Archetypes widget for the field, by passing the field name to the widget method:
<metal:field use-macro="python:context.widget('first_field', mode='view')">
First File
</metal:field>
This will display the file size, icon (if available), the filename and the file mime type.
In both these examples, I assumed the name of the field is 'first_field'.