There are 2 completable futures cf1 and cf2 defined as follows:
CompletableFuture<Boolean> cf1 = CompletableFuture.completedFuture(true);
CompletableFuture<Boolean> cf2 = CompletableFuture.completedFuture(true);
Technically, one could do:
var result1 = cf1.get();
var result2 = cf2.get();
assertThat(result1).isEqualTo(result2);
For example, if there was only one future, we could do the following:
assertThat(cf1)
.succeedsWithin(Duration.ofSeconds(1))
.isEqualTo(true);
Is there a more idiomatic way to compare the two futures against each other? Note that while the example here uses CompletableFuture<Boolean>, the Boolean can be replaced with any class.
If you are interested only in value comparison, passing cf2.get() as argument of isEqualTo should be enough:
CompletableFuture<Boolean> cf1 = CompletableFuture.completedFuture(true);
CompletableFuture<Boolean> cf2 = CompletableFuture.completedFuture(true);
assertThat(cf1)
.succeedsWithin(Duration.ofSeconds(1))
.isEqualTo(cf2.get());
The only downside is that get() can potentially throw ExecutionException and InterruptedException so they need to be declared in the test method signature.
If type-specific assertions are needed, succeedsWithin(Duration, InstanceOfAssertFactory) can help:
assertThat(cf1)
.succeedsWithin(Duration.ofSeconds(1), InstanceOfAssertFactories.BOOLEAN)
.isTrue(); // hardcoded check to show type-specific assertion
What about composing the two futures into one that completes successfully with a value of true if and only if both individual futures complete successfully with the same value? You could e.g. use thenCompose followed by thenApply:
CompletableFuture<Boolean> bothEqual = cf1.thenCompose(b1 -> cf2.thenApply(b2 -> b1 == b2));
If the sequential execution from this solution is problematic, you can parallelize by implementing a helper function alsoApply and use that one instead of thenCompose. See this answer for more details.
Related
How to get the string "hi" from the CharBuffer? toString() does not seem to work.
val a = CharBuffer.allocate(10);
a.put('h');
a.put('i');
val b = a.toString();
Variable states after running the code above:
CharBuffer is pretty low-level and really meant for I/O stuff, so it may seem illogical at first. In your example it actually returned a string containing remaining 8 bytes that you didn't set. To make it return your data you need to invoke flip() like this:
val a = CharBuffer.allocate(10);
a.put('h');
a.put('i');
a.flip()
val b = a.toString();
You can find more in the docs of the Buffer
For more typical use cases it is much easier to use StringBuilder:
val a = StringBuilder()
a.append('h')
a.append('i')
val b = a.toString()
Or even use a Kotlin util that wraps StringBuilder:
val b = buildString {
append('h')
append('i')
}
I want to fill two variables in the same line, but I don't know the best way to do it at kotlin
var a:String? = null
var b:String? = null
a, b = "Text"
Not possible in Kotlin (unless you are ready to resort to some contrived constructs with repetition as described in other answers and comments). You cannot even write
a = b = "Text"
because weirdly enough, assignments are not expressions in Kotlin (as opposed to almost everything else like if, return, throw, swicth, etc., which are expressions in Kotlin, but not in Java, for example).
So, if you want to assign exactly the same value without repetition (of the assigned value), you'll have to write
a = "Text"
b = a
Note, that there is also an also function (pun intended), so technically you can write the following if you really want to stay on one line
a = "Text".also { b = it }
but I doubt it is really worth it.
var a: String? = null; var b: String? = null
or
var (a: String?, b: String?) = null to null
But please don't ever do so
Simply create an inline array, iterate through and assign values.
arrayListOf(a, b, c, d).forEach { it = "Text" }
My apologies if this is a simple basic info that I should be knowing. This is the first time I am trying to use Java 8 streams and other features.
I have two ArrayLists containing same type of objects. Let's say list1 and list2. Let's say the lists has Person objects with a property "employeeId".
The scenario is that I need to merge these lists. However, list2 may have some objects that are same as in list1. So I am trying to remove the objects from list2 that are same as in list1 and get a result list that then I can merge in list1.
I am trying to do this with Java 8 removeIf() and stream() features. Following is my code:
public List<PersonDto> removeDuplicates(List<PersonDto> list1, List<PersonDto> list2) {
List<PersonDto> filteredList = list2.removeIf(list2Obj -> {
list1.stream()
.anyMatch( list1Obj -> (list1Obj.getEmployeeId() == list2Obj.getEmployeeId()) );
} );
}
The above code is giving compile error as below:
The method removeIf(Predicate) in the type Collection is not applicable for the arguments (( list2Obj) -> {})
So I changed the list2Obj at the start of "removeIf()" to (<PersonDto> list2Obj) as below:
public List<PersonDto> removeDuplicates(List<PersonDto> list1, List<PersonDto> list2) {
List<PersonDto> filteredList = list2.removeIf((<PersonDto> list2Obj) -> {
list1.stream()
.anyMatch( list1Obj -> (list1Obj.getEmployeeId() == list2Obj.getEmployeeId()) );
} );
}
This gives me an error as below:
Syntax error on token "<", delete this token for the '<' in (<PersonDto> list2Obj) and Syntax error on token(s), misplaced construct(s) for the part from '-> {'
I am at loss on what I really need to do to make it work.
Would appreciate if somebody can please help me resolve this issue.
I've simplified your function just a little bit to make it more readable:
public static List<PersonDto> removeDuplicates(List<PersonDto> left, List<PersonDto> right) {
left.removeIf(p -> {
return right.stream().anyMatch(x -> (p.getEmployeeId() == x.getEmployeeId()));
});
return left;
}
Also notice that you are modifying the left parameter, you are not creating a new List.
You could also use: left.removeAll(right), but you need equals and hashcode for that and it seems you don't have them; or they are based on something else than employeeId.
Another option would be to collect those lists to a TreeSet and use removeAll:
TreeSet<PersonDto> leftTree = left.stream()
.collect(Collectors.toCollection(() -> new TreeSet<>(Comparator.comparing(PersonDto::getEmployeeId))));
TreeSet<PersonDto> rightTree = right.stream()
.collect(Collectors.toCollection(() -> new TreeSet<>(Comparator.comparing(PersonDto::getEmployeeId))));
leftTree.removeAll(rightTree);
I understand you are trying to merge both lists without duplicating the elements that belong to the intersection. There are many ways to do this. One is the way you've tried, i.e. remove elements from one list that belong to the other, then merge. And this, in turn, can be done in several ways.
One of these ways would be to keep the employee ids of one list in a HashSet and then use removeIf on the other list, with a predicate that checks whether each element has an employee id that is contained in the set. This is better than using anyMatch on the second list for each element of the first list, because HashSet.contains runs in O(1) amortized time. Here's a sketch of the solution:
// Determine larger and smaller lists
boolean list1Smaller = list1.size() < list2.size();
List<PersonDto> smallerList = list1Smaller ? list1 : list2;
List<PersonDto> largerList = list1Smaller ? list2 : list1;
// Create a Set with the employee ids of the larger list
// Assuming employee ids are long
Set<Long> largerSet = largerList.stream()
.map(PersonDto::getEmployeeId)
.collect(Collectors.toSet());
// Now remove elements from the smaller list
smallerList.removeIf(dto -> largerSet.contains(dto.getEmployeeId()));
The logic behind this is that HashSet.contains will take the same time for both a large and a small set, because it runs in O(1) amortized time. However, traversing a list and removing elements from it will be faster on smaller lists.
Then, you are ready to merge both lists:
largerList.addAll(smallerList);
I'm generating small dataFrames in for loop. At each round of for loop, I pass the generated dataFrame to a function which returns double. This simple process (which I thought could be easily taken care of by garbage collector) blow up my memory. When I look at Spark UI at each round of for loop it adds a new "SQL{1-500}" (my loop runs 500 times). My question is how to drop this sql object before generating a new one?
my code is something like this:
Seq.fill(500){
val data = (1 to 1000).map(_=>Random.nextInt(1000))
val dataframe = createDataFrame(data)
myFunction(dataframe)
dataframe.unpersist()
}
def myFunction(df: DataFrame)={
df.count()
}
I tried to solve this problem by dataframe.unpersist() and sqlContext.clearCache() but neither of them worked.
You have two places where I suspect something fishy is happening:
in the definition of myFunction : you really need to put the = before the body of the definition. I had typos like that compile, but produce really weird errors (note I changed your myFunction for debugging purposes)
it is better to fill your Seq with something you know and then apply foreach or some such
(You also need to replace random.nexInt with Random.nextInt, and also, you can only create a DataFrame from a Seq of a type that is a subtype of Product, such as tuple, and need to use sqlContext to use createDataFrame)
This code works with no memory issues:
Seq.fill(500)(0).foreach{ i =>
val data = {1 to 1000}.map(_.toDouble).toList.zipWithIndex
val dataframe = sqlContext.createDataFrame(data)
myFunction(dataframe)
}
def myFunction(df: DataFrame) = {
println(df.count())
}
Edit: parallelizing the computation (across 10 cores) and returning the RDD of counts:
sc.parallelize(Seq.fill(500)(0), 10).map{ i =>
val data = {1 to 1000}.map(_.toDouble).toList.zipWithIndex
val dataframe = sqlContext.createDataFrame(data)
myFunction(dataframe)
}
def myFunction(df: DataFrame) = {
df.count()
}
Edit 2: the difference between declaring function myFunction with = and without = is that the first is (a usual) function definition, while the other is procedure definition and is only used for methods that return Unit. See explanation. Here is this point illustrated in Spark-shell:
scala> def myf(df:DataFrame) = df.count()
myf: (df: org.apache.spark.sql.DataFrame)Long
scala> def myf2(df:DataFrame) { df.count() }
myf2: (df: org.apache.spark.sql.DataFrame)Unit
In Scalding, suppose you have a TypedPipe[Long] or ValuePipe[Long]. How would you go about checking whether they are empty in the most elegant/efficient way?
Currently testing the following:
val isTPEmpty: Boolean = typePipe.equals(TypedPipe.empty)
val isVPEmpty: Boolean = valuePipe.equals(EmptyValue)
Or, to make it more generic:
def isTypedPipeEmpty[A](typedPipe: TypedPipe[A]): Boolean = {
val emptyTP: TypedPipe[A] = TypedPipe.empty
typedPipe.equals(emptyTP)
}
UPDATE: this doesn't work (will return false for an empty TypedPipe). Appreciate any inputs.
After speaking to several people on this, there is no straight solution simply because a TypedPipe is distributed, and checking whether it is empty is "expensive", therefore one should avoid this as much as possible.
If you absolutely have no choice, what worked for me was something "ugly" as creating a temporary empty TypedPipe, then calling mapWithValue on my ValuePipe, and if it is empty do X, otherwise do Y. Something like:
TypedPipe.from(List()).mapWithValue(valuePipe) { case (temp, valuePipe) => if (valuePipe.isEmpty) doX else doY }
But again, cumbersome.