I'd like to use the filter function in Coll. However I'm getting an error that says that there's something wrong with parsing:
Code:
val isNftInDataInputBox: Boolean =
dataInputBox.tokens
.filter(token => token._1 == outProfileBox.R5[Coll[Byte]].get)
.nonEmpty
Error:
Invalid declaration of lambda Ident(token,NoType) => Some(EQ(Select(Ident(token,NoType),_1,None),Select(ApplyTypes(Select(Ident(outProfileBox,NoType),R5,None),Vector(Coll[SByte$])),get,None)))
sigmastate.lang.syntax.ParserException:
line 61: .filter(token => token._1 == outProfileBox.R5[Coll[Byte]].get)
Is Filter allowed in ErgoScript?
Is this the correct documentation for Colls?
https://github.com/ScorexFoundation/sigmastate-interpreter/blob/fada073b82a16a928c457693b888da4c0310aca6/library/src/main/scala/special/collection/impl/CollsImpl.scala
I was able to filter it this way:
val filteredNFTToken: Coll[(Coll[Byte], Long)] = dataInputBox.tokens
.filter{
(token: (Coll[Byte], Long)) => token._1 == outProfileBox.R5[Coll[Byte]].get
}
val isNftInDataInputBox: Boolean = filteredNFTToken.size == 1
However, when I try
filteredNFTToken.NonEmpty
it fails with
Cannot find method 'nonEmpty' in in the object Ident(filteredNFTToken,NoType) of Product type with methods List(SMethod(sigmastate.SCollection$#25a5c8e,size,(Coll[IV]) => SInt$,1,FixedCost(14),MethodIRInfo(None,None,None),Some(OperationInfo(Some(SizeOf$(177)),The size of the collection in elements.,ArrayBuffer(ArgInfo(this,this instance)))),None), SMethod(sigmastate.SCollection$#25a5c8e,getOrElse,[IV](Coll[IV],SInt$,IV) => IV,2,FixedCost(30),MethodIRInfo(Some(<function1>),None,None),Some(OperationInfo(Some(ByIndex$(178)),Return the element of collection if \lst{index} is in range \lst{0 .. size-1},ArrayBuffer(ArgInfo(this,this instance), ArgInfo(index,index of the element of this collection), ArgInfo(default,value to return when \lst{index} is out of range)))),None), SMethod(sigmastate.SCollection$#25a5c8e,map,[IV,OV](Coll[IV],(IV) => OV) => Coll[OV],3,PerItemCost(20,1,10),MethodIRInfo(None,None,None),Some(OperationInfo(Some(MapCollection$(173)), Builds a new collection by applying a function to all elements of this collection.
Returns a new collection of type \lst{Coll[B]} resulting from applying the given function
\lst{f} to each element of this collection and collecting the results.
The documentation for available methods is 1.
The NonEmpty of nonEmpty methods are not available on Coll type.
Note, the names are case sensitive.
So, your solution is the right way to go.
Related
I using bind variable to do a batch update using below code
`var balanceUpdate = dslContext.batch(
dslContext.update(BALANCE)
.set(BALANCE.BALANCE, (BigDecimal) null)
.where(BALANCE.ID.eq((String) null)));
balances.forEach(balance -> {
balanceUpdate.bind(
balance.getAmount()
balance.Id);
});
int[] execute = balanceUpdate.execute();
`
Above code work well, but now i want to use bind with array of arguments like
var balanceUpdate = dslContext.batch(
dslContext.update(BALANCE)
.set(BALANCE.BALANCE, (BigDecimal) null)
.where(BALANCE.ID.eq((String) null)));
var arguments = balances.stream()
.map(balance ->
new Object[] {
balance.getAmount(),
balance.Id
}).collect(Collectors.toList());
int[] execute = balanceUpdate.bind(arguments).execute();
I get exception
java.lang.ArrayStoreException: arraycopy: element type mismatch: can not cast one of the elements of java.lang.Object[] to the type of the destination array, java.math.BigDecimal
at org.jooq_3.14.4.ORACLE12C.debug(Unknown Source)
at java.base/java.util.Arrays.copyOf(Arrays.java:3722)
at org.jooq.tools.Convert.convertArray(Convert.java:357)
at org.jooq.tools.Convert.convertArray(Convert.java:345)
at org.jooq.tools.Convert$ConvertAll.from(Convert.java:603)
at org.jooq.tools.Convert.convert0(Convert.java:392)
at org.jooq.tools.Convert.convert(Convert.java:384)
at org.jooq.tools.Convert.convert(Convert.java:458)
at org.jooq.tools.Convert.convertArray(Convert.java:363)
at org.jooq.tools.Convert.convertArray(Convert.java:345)
at org.jooq.tools.Convert$ConvertAll.from(Convert.java:614)
at org.jooq.tools.Convert.convert0(Convert.java:392)
at org.jooq.tools.Convert.convert(Convert.java:384)
at org.jooq.tools.Convert.convert(Convert.java:458)
at org.jooq.impl.AbstractDataType.convert(AbstractDataType.java:534)
at org.jooq.impl.DefaultDataType.convert(DefaultDataType.java:86)
at org.jooq.impl.DSL.val(DSL.java:24409)
at org.jooq.impl.DSL.val(DSL.java:24377)
at org.jooq.impl.Tools.field(Tools.java:1794)
at org.jooq.impl.Tools.fields(Tools.java:1865)
at org.jooq.impl.BatchSingle.executePrepared(BatchSingle.java:226)
at org.jooq.impl.BatchSingle.execute(BatchSingle.java:170)
According docs it should work ? Atleast it works without explicit casting when using jdbc. Is there anyway to get it work to sent bind variables only once instead of many times like in first example?
I think you're calling the wrong BatchBindStep.bind(Object...) method, or at least not in the way you're expecting. There's currently no overload accepting a collection of type List<Object[]>. So, what you should do instead is create an Object[][] type for your bind variable sets:
Object[][] arguments = balances
.stream()
.map(balance -> new Object[] {
balance.getAmount(),
balance.Id
}).toArray();
React.useEffect(_ => {
ReactNativeAsyncStorage.getItem("jwt")
|> Js.Promise.then_(jwt => {Js.log(jwt)});
None;
});
Error:
This expression has type unit but an expression was expected of type
Js.Promise.t('a) = Js.Promise.t('a)
I am using https://github.com/reason-react-native/async-storage with https://github.com/react-native-community/async-storage
The type of Js.Promise.then_ is
('a => t('b), t('a)) => t('b)
which means the function passed to it must return a, and that then_ itself will return that promise.
You're making two mistakes in your code:
You are not returning a promise from the function you pass to then_
You are not handling the promise returned by then_.
The following fixes both:
React.useEffect(_ => {
let _: Js.Promise.t(unit) =
Js.Prmoise.(
ReactNativeAsyncStorage.getItem("jwt")
|> then_(jwt => resolve(Js.log(jwt)))
);
None;
});
let _: Js.Promise.t(unit) = ... uses the wildcard pattern to discard the result of the following expression. The type of the result is Js.Result.t(unit), which is annotated to protect against accidental partial application.
resolve(Js.log(jwt)) will return a promise with the result of calling Js.log, which is unit, hence why the resulting promise has the type Js.Promise.t(unit).
See the Reason documentation for more on how to use promises.
Here is my problem, I have a map of Map[Array[String],String], and I want to pass that into a UDF.
Here is my UDF:
def lookup(lookupMap:Map[Array[String],String]) =
udf((input:Array[String]) => lookupMap.lift(input))
And here is my Map variable:
val srdd = df.rdd.map { row => (
Array(row.getString(1),row.getString(5),row.getString(8)).map(_.toString),
row.getString(7)
)}
Here is how I call the function:
val combinedDF = dftemp.withColumn("a",lookup(lookupMap))(Array($"b",$"c","d"))
I first got an error about immutable array, so I changed my array into immutable type, then I got an error about type mismatch. I googled a bit, apparently I can't pass in non-column type directly into a UDF. Can somebody help? Kudos.
Update: So I did convert everything to a wrapped array. Here is what I did:
val srdd = df.rdd.map{row => (WrappedArray.make[String](Array(row.getString(1),row.getString(5),row.getString(8))),row.getString(7))}
val lookupMap = srdd.collectAsMap()
def lookup(lookupMap:Map[collection.mutable.WrappedArray[String],String]) = udf((input:collection.mutable.WrappedArray[String]) => lookupMap.lift(input))
val combinedDF = dftemp.withColumn("a",lookup(lookupMap))(Array($"b",$"c",$"d"))
Now I am having an error like this:
required: Map[scala.collection.mutable.WrappedArray[String],String]
-ksh: Map[scala.collection.mutable.WrappedArray[String],String]: not found [No such file or directory]
I tried to do something like this:
val m = collection.immutable.Map(1->"one",2->"Two")
val n = collection.mutable.Map(m.toSeq: _*)
but then I just got back to the error of column type.
First, you have to pass a Column as an argument of the UDF; Since you want this argument to be an array, you should use the array function in org.apache.spark.sql.functions, which creates an array Column from a series of other Columns. So the UDF call would be:
lookup(lookupMap)(array($"b",$"c",$"d"))
Now, since array columns are deserialized into mutable.WrappedArray, in order for the map lookup to succeed you'd best make sure that's the type used by your UDF:
def lookup(lookupMap: Map[mutable.WrappedArray[String],String]) =
udf((input: mutable.WrappedArray[String]) => lookupMap.lift(input))
So altogether:
import spark.implicits._
import org.apache.spark.sql.functions._
// Create an RDD[(mutable.WrappedArray[String], String)]:
val srdd = df.rdd.map { row: Row => (
mutable.WrappedArray.make[String](Array(row.getString(1), row.getString(5), row.getString(8))),
row.getString(7)
)}
// collect it into a map (I assume this is what you're doing with srdd...)
val lookupMap: Map[mutable.WrappedArray[String], String] = srdd.collectAsMap()
def lookup(lookupMap: Map[mutable.WrappedArray[String],String]) =
udf((input: mutable.WrappedArray[String]) => lookupMap.lift(input))
val combinedDF = dftemp.withColumn("a",lookup(lookupMap)(array($"b",$"c",$"d")))
Anna your code for srdd/lookupmap is of type org.apache.spark.rdd.RDD[(Array[String], String)]
val srdd = df.rdd.map { row => (
Array(row.getString(1),row.getString(5),row.getString(8)).map(_.toString),
row.getString(7)
)}
Where as in lookup method you are expecting a Map as a parameter
def lookup(lookupMap:Map[Array[String],String]) =
udf((input:Array[String]) => lookupMap.lift(input))
That is the reason why you are getting type mismatch error.
First make srdd from RDD[tuple] to a RDD[Map] and then try converting the RDD to Map to resolve this error.
val srdd = df.rdd.map { row => Map(
Array(row.getString(1),row.getString(5),row.getString(8)).map(_.toString) ->
row.getString(7)
)}
My apologies if this is a simple basic info that I should be knowing. This is the first time I am trying to use Java 8 streams and other features.
I have two ArrayLists containing same type of objects. Let's say list1 and list2. Let's say the lists has Person objects with a property "employeeId".
The scenario is that I need to merge these lists. However, list2 may have some objects that are same as in list1. So I am trying to remove the objects from list2 that are same as in list1 and get a result list that then I can merge in list1.
I am trying to do this with Java 8 removeIf() and stream() features. Following is my code:
public List<PersonDto> removeDuplicates(List<PersonDto> list1, List<PersonDto> list2) {
List<PersonDto> filteredList = list2.removeIf(list2Obj -> {
list1.stream()
.anyMatch( list1Obj -> (list1Obj.getEmployeeId() == list2Obj.getEmployeeId()) );
} );
}
The above code is giving compile error as below:
The method removeIf(Predicate) in the type Collection is not applicable for the arguments (( list2Obj) -> {})
So I changed the list2Obj at the start of "removeIf()" to (<PersonDto> list2Obj) as below:
public List<PersonDto> removeDuplicates(List<PersonDto> list1, List<PersonDto> list2) {
List<PersonDto> filteredList = list2.removeIf((<PersonDto> list2Obj) -> {
list1.stream()
.anyMatch( list1Obj -> (list1Obj.getEmployeeId() == list2Obj.getEmployeeId()) );
} );
}
This gives me an error as below:
Syntax error on token "<", delete this token for the '<' in (<PersonDto> list2Obj) and Syntax error on token(s), misplaced construct(s) for the part from '-> {'
I am at loss on what I really need to do to make it work.
Would appreciate if somebody can please help me resolve this issue.
I've simplified your function just a little bit to make it more readable:
public static List<PersonDto> removeDuplicates(List<PersonDto> left, List<PersonDto> right) {
left.removeIf(p -> {
return right.stream().anyMatch(x -> (p.getEmployeeId() == x.getEmployeeId()));
});
return left;
}
Also notice that you are modifying the left parameter, you are not creating a new List.
You could also use: left.removeAll(right), but you need equals and hashcode for that and it seems you don't have them; or they are based on something else than employeeId.
Another option would be to collect those lists to a TreeSet and use removeAll:
TreeSet<PersonDto> leftTree = left.stream()
.collect(Collectors.toCollection(() -> new TreeSet<>(Comparator.comparing(PersonDto::getEmployeeId))));
TreeSet<PersonDto> rightTree = right.stream()
.collect(Collectors.toCollection(() -> new TreeSet<>(Comparator.comparing(PersonDto::getEmployeeId))));
leftTree.removeAll(rightTree);
I understand you are trying to merge both lists without duplicating the elements that belong to the intersection. There are many ways to do this. One is the way you've tried, i.e. remove elements from one list that belong to the other, then merge. And this, in turn, can be done in several ways.
One of these ways would be to keep the employee ids of one list in a HashSet and then use removeIf on the other list, with a predicate that checks whether each element has an employee id that is contained in the set. This is better than using anyMatch on the second list for each element of the first list, because HashSet.contains runs in O(1) amortized time. Here's a sketch of the solution:
// Determine larger and smaller lists
boolean list1Smaller = list1.size() < list2.size();
List<PersonDto> smallerList = list1Smaller ? list1 : list2;
List<PersonDto> largerList = list1Smaller ? list2 : list1;
// Create a Set with the employee ids of the larger list
// Assuming employee ids are long
Set<Long> largerSet = largerList.stream()
.map(PersonDto::getEmployeeId)
.collect(Collectors.toSet());
// Now remove elements from the smaller list
smallerList.removeIf(dto -> largerSet.contains(dto.getEmployeeId()));
The logic behind this is that HashSet.contains will take the same time for both a large and a small set, because it runs in O(1) amortized time. However, traversing a list and removing elements from it will be faster on smaller lists.
Then, you are ready to merge both lists:
largerList.addAll(smallerList);
I am trying to eliminate hardcoded property names in my code and stuck with indexes for nested Arrays.
string version looks pretty simple:
DocumentCaollection.EnsureIndex("ChangesList._id");
but i have no idea how to write this with IndexKeysBuilder:
new IndexKeysBuilder<TransactionEntity>().Ascending(x => x.ChangesList. ???? )
For example, this code compiles but fails on runtime:
new IndexKeysBuilder<TransactionEntity>().Ascending(x => x.ChangesList.Select(y => y.Id))
with following exception:
Unable to determine the serialization information for the expression: (TransactionEntity x) => Enumerable.Select(x.ChangesList, (TransactionItem y) => y.Id)