Hey guys I am kinda new to Nextflow I would like to parse a channel that is a list of tuples that look like that:
[ID, [[Type1, file, file], [[Type2, file, file],(...)]
I would like to filter it to contain only tuples with Type1 to get:
[ID, [[Type1, file, file]]
What would be the best approach? I tried .filter() however obviously it returns a full list as soon as it detects Type1 without removing Type2.
You can use the map operator and the Groovy findAll() method with a closure to find the tuples where "Type1" is the first element. For example:
workflow {
...
your_channel.map { id, the_list ->
tuple( id, the_list.findAll { it.first() == "Type1" } )
}
}
Related
I have a map<String, MyObject> where the values in the map could have same values for some variables (e.g., name parameter in my example). I would appreciate any solution using streams to remove entries with same name parameter on the value and keep only one of them with minimum id.
data class MyObject(val id: Int, val name: String)
For instance my map could be:
[
"first" to MyObject(1, "Alice"),
"second" to MyObject(2, "Bob"),
"third" to MyObject(3, "Alice")
]
and the expected output is:
[
"first" to MyObject(1, "Alice"),
"second" to MyObject(2, "Bob")
]
where the entry with key third is removed because the value has the same name as the first entry.
First, we need to identify all of the duplicate candidates. We can do that with groupBy, which works on any iterable (and Map is iterable with iteratee type Entry<K, V>).
myMap.entries
.groupBy({ entry => entry.value.name })
This produces a value of type Map<String, List<Entry<String, MyObject>>>.
Now, for each value in the map, we want to choose the element in the list with the smallest ID. We can select the minimum element by some condition using minBy and can do that to each element of a map with mapValues.
myMap.entries
.groupBy({ entry => entry.value.name })
.mapValues({ entry => entry.value.minBy({ it.value.id })!! })
(Note: groupBy always produces nonempty lists, since it's partitioning a set, so we can confidently !! assert that a minimum exists)
Finally, this returns a Map<String, Entry<String, MyObject>>, and you probably want to eliminate the excess Map layer.
myMap.entries
.groupBy({ entry -> entry.value.name })
.mapValues({ entry -> entry.value.minBy({ it.value.id })!! })
.values
.associate({ it.key to it.value })
Try it online!
There are multiple ways to do this using pure Kotlin, here is one relying on the fact that hash-maps do not allow duplicates. I am sure there are better solutions out there:
values.toList()
// If you care about the smaller id number value
// then sort by descending so they replace larger values.
.sortedByDescending { it.second.id }
// Will replace duplicates by hashing technique
.associateBy { it.second.name }
// Back to the same data structure
.map { it.value.first to it.value.second }.toMap()
Try it online!
I'm trying to parse data from certain server, and need to export it to excel or csv.
before I export, I need to do some post processing such as merging values between parsed data.
for example,
There are two series out of all data.
series#1 - {Name: "MATH", Student:"Zay",Id:"MATH-123", Date:"12/25/2022", Status:"Good"}
series#2 - {Name: "MATH", Student:"Zay",Id:"MATH-124", Date:"12/26/2022", Status:"Bad"}
What I want to do is,
I want to update series#1's Status to
{Name: "MATH", Student:"Zay", Id:"MATH-123,MATH-124", Date:"12/25/2022,12/26/2022", Status:"Bad"}
Id, Date ==> combining with ","
Status ==> changing to latest result
Now I'm using Filter method of DataFrames,
type MyDataSet struct{
Name string
Student string
Id string
Date string
Status string
}
totalDF:=series1_result //overall result dataframe
df := dataframe.LoadStructs(series2_result) //new dataframe which needs to be compared to previous data `totalDF`
length := df.Nrow()
for i:=0;i<length;i++ {
name:=df.Subset(i).Col("Name")
student:=df.Subset(i).Col("Student")
query:= totalDF.Filter(
dataframe.F {
Colname:"Name",
Comparator:series.Eq,
Comparando:name,
},
).Filter(
dataframe.F {
Colname:"Student",
Comparator:series.Eq,
Comparando:student,
}
)
if query.Nrow()==0 {
totalDF = totalDF.Concat(df.Subset(i))
} else {
newDF:= dataframe.LoadStructs([]MyDataSet{
{
Name:query.Col("Name").String(),
Student:query.Col("Student").String(),
Id:query.Col("Id").String()+","+df.Subset(i).Col("Id").String(),
Date:query.Col("Date").String()+","+df.Subset(i).Col("Date").String(),
Status:df.Subset(i).Col("Status").String(),
}
})
query.Set(
series.Ints([]int{0}, newDF) //It's not updated as query was not an pointer
)
}
}
Even though I updated values on result of query, it's not updated on totalDF
How can I query the data from totalDF and update the data on that totalDF?
How can I get index number of Filtered item using Filter function?
Should I implement search function instead of using Filter function?
I would really appreciate it if you could help me.
Thanks everyone!
Merry Christmas!
*I tried to find out from official docs.
*But every method returns Value, not Pointer.
I have a json with several keys being a number instead of a fixed string. Is there any way I could bypass them in order to access the nested values?
{
"55568509":{
"registers":{
"001":{
"isPlausible":false,
"deviceNumber":"55501223",
"register":"001",
"readingValue":"5295",
"readingDate":"2021-02-25T00:00:00.000Z"
}
}
}
}
My expected output here would be 5295, but since 59668509 can vary from json to json, JSON_QUERY(data, '$."59668509".registers."001".readingValue) would not be an option. I'm not able to use regexp here because this is only a part of the original json, which contains more than this.
UPDATE: full json with multiple occurrences:
This is how my whole json looks like. I would like all the readingValue in brackets, in the example below, my expected output would be [32641, 00964].
WITH test_table ( data ) AS (
SELECT
'{
"session":{
"sessionStartDate":"2021-02-26T12:03:34+0000",
"interactionDate":"2021-02-26T12:04:19+0000",
"sapGuid":"369F01DFXXXXXXXXXX8553F40CE282B3",
"agentId":"USER001",
"channel":"XXX",
"bpNumber":"5551231234",
"contractAccountNumber":"55512312345",
"contactDirection":"",
"contactMethod":"Z08",
"interactionId":"5550848784",
"isResponsibleForPayingBill":"Yes"
},
"payload":{
"agentId":"USER001",
"contractAccountNumber":"55512312345",
"error":{
"55549271":{
"registers":{
"001":{
"isPlausible":false,
"deviceNumber":"55501223",
"register":"001",
"readingValue":"32641",
"readingDate":"2021-02-26T00:00:00.000Z"
}
},
"errors":[
{
"contractNumber":"55501231",
"language":"EN",
"errorCode":"62",
"errorText":"Error Text1",
"isHardError":false
},
{
"contractNumber":"55501232",
"language":"EN",
"errorCode":"62",
"errorText":"Error Text2",
"isHardError":false
}
],
"bpNumber":"5557273667"
},
"55583693":{
"registers":{
"001":{
"isPlausible":false,
"deviceNumber":"555121212",
"register":"001",
"readingValue":"00964",
"readingDate":"2021-02-26T00:00:00.000Z"
}
},
"errors":[
],
"bpNumber":"555123123"
}
}
}
}'
FROM
dual
)
SELECT
JSON_QUERY(data, '$.payload.error.*.registers.*[*].readingValue') AS reading_value
FROM
test_table;
UPDATE 2:
Solved, this would do the trick, upvoting the first comment.
JSON_QUERY(data, '$.payload.error.*.registers.*.readingValue' WITH WRAPPER) AS read_value
As I explained in the comment to your question, if you are getting that result from the JSON you posted, you are not using JSON_QUERY(); you must be using JSON_VALUE(). Either that, or there's something else you didn't share with us.
In any case, let's say you are using JSON_VALUE() with the arguments you showed. You are asking, how can you modify the path so that the top-level attribute name is not hard-coded. That is trivial: use asterisk (*) instead of the hard-coded name. (This would work the same with JSON_QUERY() - it's about JSON paths, not the specific function that uses them.)
with test_table (data) as (
select
'{
"59668509":{
"registers":{
"001":{
"isPlausible":false,
"deviceNumber":"40157471",
"register":"001",
"readingValue":"5295",
"readingDate":"2021-02-25T00:00:00.000Z"
}
}
}
}' from dual
)
select json_value (data, '$.*."registers"."001"."readingValue"'
returning number) as reading_value
from test_table
;
READING_VALUE
-------------
5295
As an aside that is not related to your question in any way: In your JSON you have an object with a single attribute named "registers", whose value is another object with a single attribute "001", and in turn, this object has an attribute named "register" with value "001". Does that make sense to you? It doesn't to me.
We are developing filters functionality using Storefront API. We can write query with operators as given below to search within parent key such as "product_type" but we couldn't find way to search within child key.
.products(first: 10, query: "tag:blue AND product_type:sneaker") { $0
...
}
So can anyone help us to search within child key's such as variant selected options?
We are expecting query format like:
.products(first: 10, query: "selectedProductVarient.option1:M") { $0
...
}
You can't add a query parameter to a nested connection, but the productVariants() query argument takes all of those filter parameters:
{
productVariants(first: 10, query: "tag:foo product_type:bar option_1:baz") {
nodes {
id
title
product {
productType
tags
}
}
pageInfo {
hasNextPage
endCursor
}
}
}
The option_1 parameter is the name of the option ('size', 'color', etc.), not the value assigned to the variant. You might be better off querying the title.
Let's say I have a list of Strings in Kotlin: stringList: MutableList<String>
Then it is is easy to sort such list in case insensitive order by doing this:
stringList.sortWith(String.CASE_INSENSITIVE_ORDER)
But how would I sort a list of objects in case insensitive order? For example: places: MutableList<Place>
Where Place is a simple class with 2 fields - name: String and id: Int, and I would like to sort these Places by the name field.
I tried to do something like this: places.sortedWith(compareBy { it.name }) but this solution doesn't take letter case into account.
It looks like compareBy might be able to take a Comparator as an argument, see the documentation here: https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.comparisons/compare-by.html
Try:
places.sortWith(compareBy(String.CASE_INSENSITIVE_ORDER, { it.name }))
to sort the array in place, or you can assign it to a new variable using
val newArray = places.sortedWith(compareBy(String.CASE_INSENSITIVE_ORDER, { it.name }))
Sort Ascending - case insensitive:
myList.sortedBy { it.name?.toLowerCase() }
Sort Descending - case insensitive:
myList.sortedByDescending { it.name?.toLowerCase() }