The issue I am facing is similar to the one described here:
https://github.com/spring-cloud/spring-cloud-dataflow/issues/1826
This seems to have been closed with a code change.
But I am not sure if this change applied to deployment properties also.
I tried separating out the properties with a space, but it didn't work.
Here is my stream definition sample:
stream create --name testFileStream --definition "file --directory=/input --mode=ref --filename-pattern=TEST*.DAT | tasklaunchprocessor --task.launch.request.task-name=test-task --task.launch.request.arg-expressions=localFilePath=payload.substring(1,payload.length()-1) --task.launch.request.deployment-properties=app.test.input.fileArchiveDirectory=/archive, app.test.fileOutputDirectory=/output, app.test.expiryNotificationDays=14,30 app.disclosure.spring.batch.job.names=testFileLoadAndProcess | dataflow-tasklauncher --server-uri=http://scdf-spring-cloud-dataflow-server:8080"
For the property 'app.test.expiryNotificationDays', I have a comma separated list of values (14,30 in this case).
And I separated each property by a space.
But when I try to create this stream through SCDF CLI, it throws an error:
Found unexpected data after stream definition: 'app'
It is pointing to the blank space. So, looks like blank space between each deployment property is not acceptable.
Is there any way of passing two values (separated by a comma) to a property?
Related
I have a CSV file with attribute name EpochWithMicroSec and value like 1512520846 and I want to append 000 at the end of value like this 1512520846000.
I am using following configuration please help to fix this issue.
TIA
My configuration is appending 000 at the last attribute of my CSV file. How to map it to attribute EpochWithMicroSec attribute?
As you are changing the existing field value then use
Replacement Value Strategy
Literal Value
Update Record Configs:
Add Dynamic property
/EpochWithMicroSec
${field.value:append('000')}
If you're only working on the attribute itself, you should use UpdateAttribute rather than ReplaceText (the latter works on the flow file contents, not attributes). In UpdateAttribute you can add a user-defined property "EpochWithMicroSec" with the same Expression you have in your ReplaceText: ${EpochWithMicroSec:append('000')}
I am trying to import a file, source here and selections here (select all fields and select "Pilkkueroteltu (otsikollinen)" and then click Jatka to download), with two header rows, " as text qualifier, comma as a field separator and with UTF-8 format. I am unable to do this in Micsosoft Server SQL Management Studio. I will focus now only on the text qualifier where " does not work (only reading the first quote as text qualifier).
where I am unable to specify the column separator, no idea why this is occurring.
Update 1
Refresh/Reset buttons fixed the initial preview but I am getting the following preview error in the step Select Source Tables and Views later.
Update 2
I get the LocaleID error The LocaleID 11 is not installed on this system.
(SQL Server Import and Export Wizard). I am getting the same error despite Locale/Code page settings, what is causing this?
How to specify the text qualifier in the MSMM?
I tried to replicate your scene. Very first, I had to delete first heading entry eg : "Kuntien avainluvut 1987-2016"
Please see : sample image
Column delimited is: ,
Might not be accurate answer or different from something which you expect, but by applying above settings, I could import data through SSMS2012
edit : based on comments.
Here is the detailed steps :
next,
next, you need to change column width of first column as it gave me data truncate error
next,
I have also got a dtsx package for the same, but I don't know how can I share it with you here.
I'm trying to pass a parameter into the File List component through input port 0. All of my attempts thus far have been met with an error,
Input edge has no effect. Disconnect edge or use metadata fields as parameters in Target URL, Source path or Target path.
Ideally, I would like the Target URL to be something along the lines of http://${S3_ACCESS_KEY}:${S3_SECRET_KEY}#${MY_BUCKET}.s3.amazonaws.com/reports/${port:0.value}/*_interestingReport.csv where ${port:0.value} is the value passed in from the input port.
What is the correct way to use data coming in on input port 0?
The way how passing parameters from input edge for File List (but other file components as well) works, is that you use the name of the metadata column from an input edge and enclose it between ${ and }.
So if the metadata on the edge have a field called directory, which contains the dir you want to use, this is the way how to do it.
http://${S3_ACCESS_KEY}:${S3_SECRET_KEY}#${MY_BUCKET}.s3.amazonaws.com/reports/${directory}/*_interestingReport.csv
Let me show you an example of a very simple graph which uses a 'Data Generator' that creates the flow and sends it as the input of a 'File List' component'.
http://www.filedropper.com/inputportfilelist_1
As you can see the way the input field is referenced is '${DATA_SOURCE_DIR}/${fileDir}/', being 'fileDir' the only field contained in the metadata of the link that connects both components. It'll basically list the files located in ${PROJECT}\data\source\manifests.
I hope this helps.
I have written a sample smooks config using fixed length reader[1] with many fields.
I see a weird issue here. That is, if i have the last field AnnualReportReturnedDate[8] in the configuration it does not return any sets. that is my set is null.
So, i checked one by one by adding all fields and my configuration works fine. When i try to add AnnualReportReturnedDate[8] filed, it is not working.(taht is empty set is returned) Further if i change the length of that filed to 2/5 it works..but it is not working fro length 8. That is ,
if i defined AnnualReportReturnedDate[2]--works, AnnualReportReturnedDate[8]--Not works.
Why do i get this type of annoying issue?
[1]
<smooks-resource-list xmlns="http://www.milyn.org/xsd/smooks-1.1.xsd"
xmlns:fl="http://www.milyn.org/xsd/smooks/fixed-length-1.3.xsd">
<fl:reader
fields="RecordId[10]?trim,CompactedCorpName[60]?trim,FileNumber[9]?trim,CorporationName[60]?trim,...........,AnnualReportReturnedDate[8]?trim,AnnualReportReturnedCode[2]?trim"
skipLines="1"></fl:reader>
</smooks-resource-list>
The issue here is, in my file, in the middle it had a line, which did not have enough length of characters. SoO, smooks skipped the whole file to process.
I'm using TortoiseSVN 1.6.12, and seeing something very strange behaviour on property substitution. I have some svn:keyword properties (configured via TSVN) like this:
Author, LastChangedBy, Date, DateLastChanged
which I've applied recursively across every file in the codeset
I then did a simple test on a text file like this
Some text
$Author$
$LastChangedBy$
$Date$
$LastChangedDate$
When I commit my changes, the Author and LastChangedBy properties are substituted but not the Date or LastChangedDate ones. I did some experimenting around combinations and it appears that either the author properties are set, or the date ones (but never both). So it must be doing some validation based on property groups. (In TSVN, you can't simply created another svn:keywords entry, you're stuck with one).
Has anyone ever encountered this and/or is there a workaround?
The problem you have is simply based on that SVN only replaces keywords which are known to SVN.
You are using the following list of keywords set:
Author, LastChangedBy, Date, DateLastChanged
but you have placeholders set in your text file:
$Author$
$LastChangedBy$
$Date$
$LastChangedDate$
the known keywords are the following:
URL, HeadURL
Author, LastChangedBy
Date, LastChangedDate
Rev, Revision
LastChangedRevision
Id
Header
The problem you have that svn:keywords must exactly represents the keywords you would like to replace with values. But be aware that keywords are case sensitive. Furthermore you have defined a keyword "DateLastChanged" which does simply not exist and will of course not be replaced by SVN, cause it's unknown by SVN. On the other hand i assume you have a typo in your svn:keywords contents. may be you can copy&past the output of
svn pl . -v filename
on command line on that file. One point i missed before have you separated the keywords with a space?