Mule 4.4 DataWeave append counter to file - mule

Question

I am reading in a file (see below). The example file has 13 rows.
A|doe|chemistry|100|A|
B|shea|maths|90|A|
C|baba|physics|80|B|
D|doe|chemistry|100|A|
E|shea|maths|90|A|
F|baba|physics|80|B|
G|doe|chemistry|100|A|
H|shea|maths|90|A|
I|baba|physics|80|B|
J|doe|chemistry|100|A|
K|shea|maths|90|A|
L|baba|physics|80|B|
M|doe|chemistry|100|A|
Then iterating over these rows using a for each ( batch size 5 ) and then calling a REST API
Depending on REST API response ( success or failure ) I am writing payloads to respective success / error files.
I have mocked the called API such that first batch of 5 records will fail and rest of the files will succeed.
While writing to success / error files am using the following transformation :
output application/csv quoteValues=true,header=false,separator="|"
---
payload
All of this works fine.
Success log file:
"F"|"baba"|"physics"|"80"|"B"
"G"|"doe"|"chemistry"|"100"|"A"
"H"|"shea"|"maths"|"90"|"A"
"I"|"baba"|"physics"|"80"|"B"
"J"|"doe"|"chemistry"|"100"|"A"
"K"|"shea"|"maths"|"90"|"A"
"L"|"baba"|"physics"|"80"|"B"
"M"|"doe"|"chemistry"|"100"|"A"
Error log file:
"A"|"doe"|"chemistry"|"100"|"A"
"B"|"shea"|"maths"|"90"|"A"
"C"|"baba"|"physics"|"80"|"B"
"D"|"doe"|"chemistry"|"100"|"A"
"E"|"shea"|"maths"|"90"|"A"
Now what I want to do is append the row/line number to each of these files so when this goes to production , whoever is monitoring these files can easily understand and correlate with the original file .
So as an example in case of error log file ( the first batch failed which is rows 1 to 5 ) I want to append these numbers to each of the rows:
"1"|"A"|"doe"|"chemistry"|"100"|"A"
"2"|"B"|"shea"|"maths"|"90"|"A"
"3"|"C"|"baba"|"physics"|"80"|"B"
"4"|"D"|"doe"|"chemistry"|"100"|"A"
"5"|"E"|"shea"|"maths"|"90"|"A"
Not sure what I should write in DataWeave to achieve this?

Answer 1

Inside the ForEach scope, you have access to the counter vars.counter (or whatever name you've chosen since it's configurable).
You will need to iterate over each chunk of records for adding the position for each one. You can use something like:
%dw 2.0
output application/csv quoteValues=true,header=false,separator="|"
var batchSize = 5
---
payload map ({
counter: batchSize * (vars.counter - 1) + ($$ + 1)
} ++ $
)
Or if you prefer to use the update function (this will add the record counter at the last column instead though):
%dw 2.0
output application/csv quoteValues=true,header=false,separator="|"
var batchSize = 5
---
payload map (
$ update {
case .counter! -> batchSize * (vars.counter - 1) + ($$ + 1)
}
)
Remember to replace the batchSize variable from this code with the same value you're using in the ForEach scope (if it's parameterised, it would be better).
Edit 1 -
Clarification: the - 1 and + 1 are because both indexes (the counter from the For Each scope and the $$ from the map) are zero-based.

Answer 2

Just another workaround and to simplify without using any external variables. The script can be split into two; 1st is for Error group and 2nd is for Success.
%dw 2.0
output application/csv quoteValues=true,header=false,separator="|"
// Will be used for creating a counter for Error group
var errorIdx = 1
// Will be used for creating a counter for Success group
var successIdx = 6
---
//errorItems for the first 5 rows
(payload[0 to 4] map (items,idx) -> (({"0":(idx) + errorIdx} ++ items)))
++
//successItems from 6 and remaining items.
(payload[5 to -1] map (items,idx) -> (({"0":(idx) + successIdx} ++ items)))
DataWeave Inline Variables:
errorIdx is a pointer for starting the error counter
successIdx is a pointer for starting the success counter
This will extract from index 0 to 4 element:
payload[0 to 4]
This will extract from index 5 to remaining elements:
payload[5 to -1]