I try to get a simple SCIO code running.
case class Foo(first: String, second: String, third: String)
Trying to use Foo in a SCollection it leads to an error:
Cannot find an implicit Coder instance for type:
>> (String, my.Foo)
...
<redacted prose>
...
There is a lot written in the error message.
I cannot seem to grasp, though how all that text provides any information to what I am supposed to do around Coders to solve this problem.
Can anyone shed some light how one would solve this.
Bonus points for not using implicits.
Here is a minimalistic example
case class DemoEvents (reverse_geo_code_failed:String
,eventType:String
,city:String
,area:String
)
class DoFnExample extends DoFn [String, DemoEvents]{
// `#ProcessElement` is called once per element.
#ProcessElement
private[example] def processElement(c: DoFn[String, DemoEvents]#ProcessContext): Unit ={
//val t = DemoEvents("BNG1111","pickup","ORR","1111")
implicit val formats: DefaultFormats.type = DefaultFormats
c.output(parse(c.element()).extract[DemoEvents])
}
}
val eventStream: SCollection[String] = sc.pubsubSubscription[String](args("pubsub-subscription-name"))
.withFixedWindows(Duration.standardSeconds(120L))
val events: SCollection[DemoEvents] = eventStream.applyTransform(ParDo.of(new DoFnExample()))
Related
I am learning Kotlin (coming from JS/TS background) and wondering if it is possible to access properties of a data class dynamically. For e.g.
data class Obj (private val name: String){}
val obj = Obj(name="sel")
would it be possible to do now:
val getValueOfField = "name"
println(obj[getValueOfField])
When I tried, it failed but also noticed type inference is an issue. When I did:
Obj::class.declaredMemberProperties.contains(getValueOfField)
to check if the field exists, it still failed to compile due to data inference issue.
No dynamic access is available without using reflection as the return type of such an operation would be unknown and so type safety would be lost.
You can do destructuring though which is almost as flexible:
data class Name (
val firstName: String,
val secondName: String
){}
fun main() {
val (name, _) = Name("Nie", "Selam")
println("$name is learning Kotlin") // prints Nie is learning Kotlin
}
See Kotlin playground.
I was following a tutorial for learning kotlin and ran into this example.
open class AquariumPlant(val color: String, private val size: Int)
class GreenLeafyPlant(size: Int) : AquariumPlant("green", size)
fun AquariumPlant.print() = println("AquariumPlant")
fun GreenLeafyPlant.print() = println("GreenLeafyPlant")
val plant = GreenLeafyPlant(size = 10)
plant.print()
println("\n")
val aquariumPlant: AquariumPlant = plant
aquariumPlant.print() // what will it print?
Well this apparently prints "Aquarium Plant" instead of "GreenLeafyPlant". I was a bit confused by this so I tested this out with this little snippet of code.
open class Aquarium {
open fun printSize() {
println("hello")
}
}
class TowerTank: Aquarium() {
override fun printSize() {
println("rawr")
}
}
fun main() {
towerTank = TowerTank()
(towerTank as Aquarium).printSize()
}
So this prints "rawr" and not "hello". My question is why doesn't it print "hello"? Aren't these two examples contradicting themselves? How does the function extensions create this difference in behaviour? Sorry if this may seem like a dumb question, I'm new to Kotlin as you can probably tell.
To understand this we need to understand how extensions work. Extensions don't magically add new members to existing classes. This is technically impossible both in Java and Kotlin. Instead, they work as good old static utility functions in Java. Accessing them as members is just a syntactic sugar.
First example is really similar to these functions:
fun print(plant: AquariumPlant) = println("AquariumPlant")
fun print(plant: GreenLeafyPlant) = println("GreenLeafyPlant")
To make it even more clear, we can rename these functions:
fun printAquariumPlant(plant: AquariumPlant) = println("AquariumPlant")
fun printGreenLeafyPlant(plant: GreenLeafyPlant) = println("GreenLeafyPlant")
Now, it is pretty clear that if we have object like this:
val aquariumPlant: AquariumPlant = GreenLeafyPlant(size = 10)
Then we can only invoke printAquariumPlant() function with it and it will print AquariumPlant, not GreenLeafyPlant. Despite the fact aquariumPlant is actually a GreenLeafyPlant object.
If we move one step back and rename them again to just print, nothing will really change. aquariumPlant variable is of type AquariumPlant (even if it contains GreenLeafyPlant object), so the compiler chooses print(AquariumPlant) function.
This is why we say extensions are resolved statically. Compiler decides which function to call at compile time. Virtual functions are resolved at runtime, taking into consideration the real type of the object.
I am trying to use
Arrow Either results instead of try-catch, but have gone too deep down the rabbit hole. 🙄
I have been trying to use Either<Problem,Value> as my functional return types, where Problem is like
sealed interface Problem
data class Caught(val cause: Throwable): Problem
data class DataKeyDisabled(val uuid: UUID, val cause: String): Problem
data class SubscriberNotFound(val uuid: UUID, val cause: String): Problem
data class NotEncrypted(val field: String): Problem
where the use case looks like
when (val result = transform(...)) {
is Right -> {}
is Left -> when (val problem = result.value) {
is Caught -> {}
is DataKeyDisabled -> {}
is SubscriberNotFound -> {}
is NotEncrypted -> {}
// else -> {} not needed...
}
}
But, there are really three types of problems, and I don't want to have to exhaust all the choices all the time.
Problem -> Caught
KeyProblem -> Caught, DataKeyDisabled, SubscriberNotFound
DataProblem -> Caught, DataKeyDisabled, SubscriberNotFound, NotEncrypted
For example, I want to have something like
sealed interface Problem
sealed interface KeyProblem : Problem
sealed interface DataProblem : KeyProblem
data class NotHandled(val cause: Throwable): Problem
data class DataKeyDisabled(val uuid: UUID, val cause: String): KeyProblem
data class SubscriberNotFound(val uuid: UUID, val cause: String): KeyProblem
data class NotEncrypted(val cause: String) : DataProblem
And I want to be able to have some code like
fun bar(input: Either<Problem,String>) : Either<KeyProblem,String> {
val something = when (input) {
is Right -> {}
is Left -> {
when (val problem = input.value) {
is NotHandled -> {}
is DataKeyDisabled -> {}
is SubscriberNotFound -> {}
is NotEncrypted -> {}
}
}
}
}
But Kotlin complains about NotHandled, DataKeyDiabled, and SubscriberNotFound are not a DataProblem
In some cases, I want to return a KeyProblem so I can drop the NotEncrypted case from the when, and in some cases I want to return only a Problem such that the only case is NotHandled.
I do not know how to express this in Kotlin. I suspect it is not possible to express this in Kotlin, so if someone tells me it is impossible, that is a solution.
I am thinking it was a bad decision to replace try-catch with Arrow Either. If so, someone please tell me so.
I wanted to stick to Functional Reactive Programming paradigms, where try-catch does not work, but with Kotlin coroutines it sort of does work. 🤔
It seems to me, the problem with sealed things is that when using when you can only have one level of inheritance, and no more?
Maybe I am just looking at the whole problem the wrong way... help... please...
So my solution is to give up on trying to use Arrow Either and Kotlin sealed classes instead of using standard
try {
// return result
}
catch {
// handle or rethrow
}
finally {
// clean up
}
While I have been trying to practice Reactive and non-blocking programming for years, this was easy in Scala, but it's not easy in Kotlin.
After watching enough Java Project Loom videos, I am by far convinced this is the best way to go because exception handling just works... I could use Kotlin Coroutines because they also preserve correct exception handling, and may do that temporarily, but in the long run, Virtual Threads and Structured Concurrency are the way to go.
I hate using these words, but I am making a 'paradigm shift' back to cleaner code, retreating from this rabbit hole I have gone down.
It seems like you are going too far to re-use your error-types, when in fact your functions have different return-types and things that can go wrong. The simplest and cleanest solution in my opinion is to declare both the happy-case and error-case types per function. Then it should be very easy to only handle the cases than can actually go wrong per function.
For example if you have a function getPerson, you would declare the data class Person as the right value, and a GetPersonError as the left value, where the GetPersonError is an interface with only the relevant errors, like so:
private fun getPerson(identifier: String): Either<GetPersonError, Person> {...}
data class Person(name: String, ....)
sealed interface GetPersonError
sealed class PersonNotFoundError(): GetPersonError
sealed class InvalidIdentifierError(): GetPersonError
This does require you to write more code than reusing the same Problem-class for multiple functions, but the code becomes very readable and easy to change, which is much more difficult to achieve when reusing a lot of code.
I have simple three functions returning arrow-kt data types
fun validate(input): Validated<Error, Input> = ...
fun fetch(input): Option<Error, InputEntity> = ...
fun performAction(inputEntity): Either<Error, Output> = ...
And want to chain something like this (can use any available function instead of map)
validate(input)
.map{fetch(it)}
.map{performAction(it)}
Only solution I could come up with is to replace Validated and Option with Either and chain using flatMap. Is there any better functional way to make it work without updating the existing functions?
👋 What #pablisco described is correct, but you can keep it simpler by using some syntax extensions we provide to convert from one type to the other. Note that both options are correct, but Monad Transformers can be a bit convoluted and too powerful, and they're also prone to get removed from Arrow soon, once we finally figure out our delimited continuations style completely. But that is out of scope here. Here is how you could solve it by using the extensions I mentioned:
import arrow.core.*
import arrow.core.extensions.fx
sealed class Error {
object Error1 : Error()
object Error2 : Error()
}
data class InputEntity(val input: String)
data class Output(val input: InputEntity)
fun validate(input: String): Validated<Error, InputEntity> = InputEntity(input).valid()
fun fetch(input: String): Option<InputEntity> = InputEntity(input).some()
fun performAction(inputModel: InputEntity): Either<Error, Output> = Output(inputModel).right()
fun main() {
val input = "Some input"
Either.fx<Error, Output> {
val validatedInput = !validate(input).toEither()
val fetched = !fetch(validatedInput.input).toEither { Error.Error1 /* map errors here */ }
!performAction(fetched)
}
}
Hope it was useful 👍
What you are looking for is called a Monad Transformer. In Arrow, you may have seen them already, they end with a T at the end. Like OptionT or EitherT.
There are some good examples here for EitherT:
https://arrow-kt.io/docs/0.10/arrow/mtl/eithert/
And here for OptionT:
https://arrow-kt.io/docs/0.10/arrow/mtl/optiont/
The idea would be that to choose what your final value is going to be (let's say Either) and using an FX block you can then use EitherT to convert the other types to an Either.
I have an immutable object:
class Foo(
val name: String,
val things: List<Thing>
)
A third party lib creates the Foo object with some 'null' Thing objects.
I am creating a new object:
val foo = thirdPartyGetFoo()
val filteredFoo = Foo(foo.name, foo.things.filterNotNull())
That works, however AndroidStudio greys out the filterNotNull function call and presents a warning:
Useless call on collection type: The inspection reports filter-like
calls on already filtered collections.
Is this the right way to filter that list? Should I ignore the warning or is there a better way?
You do not specify what library creates the object with nulls. Some deserialization libraries can use static factory methods which you could configure, and then have the factory method strip the null. For example, if this were Jackson you would simply:
class Foo(val name: String, val things: List<Thing>) {
companion object {
#JsonCreator
#JvmName("createFromNullable")
fun create(name: String, things: List<Thing?>) = Foo(name, things.filterNotNull())
fun create(name: String, things: List<Thing>) = Foo(name, things)
}
}
Then...
val goodFoo = jacksonObjectMapper().readValue<Foo>(someJsonWithNulls)
Maybe your library has options that are similar?
If not, and you don't have 100 of these things with this problem, I would probably create a temporary class to hold the results and convert that to the final class:
open class FooNullable(val name: String, open val things: List<Thing?>) {
open fun withoutNulls(): Foo = Foo(name, things.filterNotNull())
}
class Foo(name: String, override val things: List<Thing>) : FooNullable(name, things) {
override fun withoutNulls(): Foo = this
}
Then you can deserialize into FooNullable and just call withoutNulls() to get the other flavor that is clean. And if you accidentally call it on one without nulls already, it just does nothing.
val goodFoo = Foo("", emptyList<Thing>())
val alsoGoodFoo = goodFoo.withoutNulls() // NOOP does nothing
val badFoo = thirdPartyGetFoo()
val betterFoo = badFoo.withoutNulls() // clean up the instance
val safeFoo = thirdPartyGetFoo().withoutNulls() // all at once!
Not the cleanest, but does work. The downsides is this second step, although it looks like you were already planning on doing that anyway. But this model is safer than what you proposed since you KNOW which type of object you have and therefore you continue to be typesafe and have the compiler helping you avoid a mistake.
You don't have to use inheritance as in the above example, I was just trying to unify the API in case there was a reason to have either version in hand and know which is which, and also act upon them in a similar way.