I have created data flow which will transfer SQL server data from DB to DB in nifi
and there a datetime column and it also contains some null value in source table, now i want to transfer that column with datetime value and also null values same time to the datetime column in target table.
I'm unable to find right expression to put it in update attributes processor.
I'm using below expression in update process:
sql.args.1.name : CR_LINE_CMTD_START_DT
sql.args.1.type : 93
sql.args.1.value :${CR_LINE_CMTD_START_DT:toDate("yyyy-MM-dd HH:mm:ss.SSS"):toNumber()}
Please help to fix this
Thanks
sam
Are you getting errors when a null value comes along, or for every input?
In UpdateAttribute there is an Advanced UI (click the Advanced button on the bottom left corner of the dialog) where you can update the value of sql.args.value.1 based on conditions. So you could use ${CR_LINE_CMTD_START_DT:notNull()} as one condition, where the action is what you have above. The you could add a condition for ${CR_LINE_CMTD_START_DT:isNull()} to set the value to ${CR_LINE_CMTD_START_DT} (or perhaps leave the value blank if you can).
Please see the UpdateAttribute "Additional Details" page for more information on the Advanced usage. Also as of NiFi 1.2.0 (not yet released at the time of this answer), NIFI-3206 will add an ifElse() function to Expression Language, which should make it easier to do inline.
Related
The problem was to add a new field to existing datasource and fill it with some default value.
I have tried so via this aticle
But the actual result is that new column was added, but it filled with a null value.
Where I was wrong and can I fix it in the same way?
It would be hard to tell without looking at how you have added the new column in your ingestions spec.
I will suggest using druid unified console data loader UI > and parse your input data > define the additional column under transform section. The advantage of data loader UI is that you can preview the transformed result immediately and once the workflow is completed you will get an ingestions spec and can submit it from there itself.
Eg-
Transformation example
From the documentation here: https://druid.apache.org/docs/latest/design/segments.html#different-schemas-among-segments
Maybe the result is related to the fact that existing segments don't have the new field and therefore show null.
I'm new to PDI and Kettle, and what I thought was a simple experiment to teach myself some basics has turned into a lot of frustration.
I want to check a database to see if a particular record exists (i.e. vendor). I would like to get the name of the vendor from reading a flat file (.CSV).
My first hurdle selecting only the vendor name from 8 fields in the CSV
The second hurdle is how to use that vendor name as a variable in a database query.
My third issue is what type of step to use for the database lookup.
I tried a dynamic SQL query, but I couldn't determine how to build the query using a variable, then how to pass the desired value to the variable.
The database table (VendorRatings) has 30 fields, one of which is vendor. The CSV also has 8 fields, one of which is also vendor.
My best effort was to use a dynamic query using:
SELECT * FROM VENDORRATINGS WHERE VENDOR = ?
How do I programmatically assign the desired value to "?" in the query? Specifically, how do I link the output of a specific field from Text File Input to the "vendor = ?" SQL query?
The best practice is a Stream lookup. For each record in the main flow (VendorRating) lookup in the reference file (the CSV) for the vendor details (lookup fields), based on its identifier (possibly its number or name or firstname+lastname).
First "hurdle" : Once the path of the csv file defined, press the Get field button.
It will take the first line as header to know the field names and explore the first 100 (customizable) record to determine the field types.
If the name is not on the first line, uncheck the Header row present, press the Get field button, and then change the name on the panel.
If there is more than one header row or other complexities, use the Text file input.
The same is valid for the lookup step: use the Get lookup field button and delete the fields you do not need.
Due to the fact that
There is at most one vendorrating per vendor.
You have to do something if there is no match.
I suggest the following flow:
Read the CSV and for each row look up in the table (i.e.: the lookup table is the SQL table rather that the CSV file). And put default upon not matching. I suggest something really visible like "--- NO MATCH ---".
Then, in case of no match, the filter redirect the flow to the alternative action (here: insert into the SQL table). Then the two flows and merged into the downstream flow.
I've created a SSRS report to examine whether certain fields contain a specific value using a simple case, when statement with the values returned being 'Yes' or 'No'. I'd like the cells to be coloured yellow with the value is a 'Yes'. (See Sample Table).
I know I can do this using the function IIF(Fields!Result1.Value="Yes","Yellow","Transparent") but for a table with multiple columns (>60), is there a way to copy the background colour formatting without having to write the function for each column?
If you want to compare the value of each text box/cell with "YES" then you can use the ME.Value reference.
There is little documentation on ME (https://msdn.microsoft.com/en-us/library/dd255285.aspx?f=255&MSPPError=-2147217396)
=IIF(ME.Value = "Yes", "Yellow", "Transparent")
To access the value of the current text box, you can use the Visual
Basic built-in global Me.Value or simply Value. In report functions
such as First and aggregate functions, use the fully qualified syntax.
Unfortunately, ME does not seem to be fully implemented - it causes errors when used in places that were not created to use ME correctly. An ACTION (i.e. Go To URL) that uses ME will fail with an error of an unknown reference.
I have a feature in my application where a user can opt to UNsubscribe itself. When this happens, there is no change made to the other fields in database and only user's Subscription flag is unchecked. I am doing this by getting a datarow and settingone of the field values and then updating the datarow in the table.
This is working fine for all of the recently created records, however some of the old records have a blank value set for a date field which is now mandatory. Hence when I try to unsubscribe old record of this type, it tries to retrieve null value from the database and then update the same back resulting in an error - "Cannot insert null value in Date Field"
First, if you have a table with a null value in a mandatory column, you would be better served to run an update query on the table that will enter a default value for this mandatory/required column. Wayne G. Dunn mentioned this in the comments to your question. For example:
UPDATE [TABLE1] SET [datefield] = '4/28/1949' where [datefield] = null
**Be sure to backup your data prior to running this kind of query and test with a test table before you execute the query on production data.
Second, if you are doing an update to a row and setting the checkbox value to true, you can review any columns that are required and set their default value in the update query. This makes the query more complex and more difficult to support.
I advise you go with the first option and run an update query on the table. The second option adds unnecessary code to your query that updates the checkbox.
If you post some query examples and table structure, you will get more specific answers. You may consider editing your question to include these details.
I have a few reports built using Report Builder 3 for MSSQL 2008 Reporting Services.
Some fields in my report are showing "#Error", instead of this I want to show only a simple "-". Is there any built-in function or custom code to overcome this?
I'd still really like to see your formula but you seem determined not to show it, so I'll take a wild stab at answering without it. I imagine that you are doing something like dividing the field on the current row by the field on the previous row. However, this would give you Infinity on the first line rather than #Error so there is something else going on. But let's run with this anyway since we don't have your formula.
The most common way to solve this is to check for Nothing being returned for the Previous function, usually indicating that you are on the first row (assuming your field always has data). This has the advantage of also working on fields that are not guaranteed to have a value.
=IIF(IsNothing(Previous(Fields!MyField.Value)), "-", Fields!MyField.Value / Previous(Fields!MyField.Value))
Here is another way you could do it using the row number, which will always check for the first row regardless:
=IIF(RowNumber(Nothing) = 1, "-", Fields!MyField.Value / Previous(Fields!MyField.Value))
This assumes that the error is being caused by the Value formula and not by some other mechanism such as applying an expression to other properties like Format, Color which is invalid when there is no previous row.