I have a SAX parser reading an XML file (specifically, a .xlsx file) and returning the content as a list of Row objects: it's roughly like this
fun readExcelContent(data: InputStream) {
val pkg = OPCPackage.open(file)
val reader = XSSFReader(pkg)
val sst = reader.sharedStringsTable
val parser = XMLHelper.newXMLReader()
val handler = ExcelSheetHandler(sst)
parser.contentHandler = handler
val sheet = reader.sheetsData.next()
val source = InputSource(sheet)
parser.parse(source)
return handler.content
}
Where ExcelSheetHandler is a class that extends DefaultHandler and takes care of filling up a list:
class ExcelSheetHandler(sst: SharedStringsTable): DefaultHandler() {
private val content = mutableListOf<Row>()
#Throws(SAXException::class)
override fun endElement(uri: String?, localName: String?, name: String) {
// If it's the end of a content element, add a row to content
}
}
It's basically a slight modification of the event model example in the Apache POI howto.
I was wondering if there was a way to have readExcelContent return an asynchronous object, such as a flow, and emit the rows to its client as soon as they are read rather than having to wait for the whole file to be processed.
I'd prefer kotlinx.coroutines.Channel over kotlinx.coroutines.Flow for this use case since this is a hot stream of data triggered by the parse() method. Here's what Kotlin Language Guide states.
Flows are cold streams similar to sequences — the code inside a flow
builder does not run until the flow is collected
Here's a quick implementation that you can try.
class ExcelSheetHandler : DefaultHandler() {
private val scope = CoroutineScope(Dispatchers.Default)
private val rows = Channel<Row>()
override fun endDocument() {
// To avoid suspending forever!
rows.close()
}
#Throws(SAXException::class)
override fun endElement(uri: String?, localName: String?, name: String) {
readRow(uri, localName, name)
}
private fun readRow(uri: String?, localName: String?, name: String) = runBlocking {
// If it's the end of a content element, add a row to content
rows.send(row)
}
// Client code - if it needs to be somewhere else
// you can expose a reference to Channel object
private fun processRows() = scope.launch {
for(row in rows) {
// Do something
println(row)
}
}
}
Related
I'm not super sure what I'm doing here so go easy on me:
I'm making a wordle clone and the word that is to be guessed is stored as a string in a pre-populated room database which I am trying to retrieve to my ViewModel and currently getting:
"StandaloneCoroutine{Active}#933049a"
instead of the actual data.
I have tried using LiveData which only returned null which as far as I'm aware is because it was not observed.
Switched to coroutines which seemed to make more sense if my UI doesn't need the data anyway.
I ended up with this so far:
DAO:
#Dao
interface WordListDao {
#Query("SELECT word FROM wordlist WHERE used = 0 ORDER BY id DESC LIMIT 1")
suspend fun readWord(): String
// tried multiple versions here only string can be converted from Job
// #Query("SELECT * FROM wordlist WHERE used = 0 ORDER BY id DESC LIMIT 1")
// fun readWord(): LiveData<WordList>
// #Query("SELECT word FROM wordlist WHERE used = 0 ORDER BY id DESC LIMIT 1")
// fun readWord(): WordList
}
repository:
class WordRepository(private val wordListDao: WordListDao) {
//val readWordData: String = wordListDao.readWord()
suspend fun readWord(): String {
return wordListDao.readWord()
}
}
model:
#Entity(tableName = "wordlist")
data class WordList(
#PrimaryKey(autoGenerate = true)
val id: Int,
val word: String,
var used: Boolean
)
VM:
class HomeViewModel(application: Application) : ViewModel() {
private val repository: WordRepository
private var word: String
init {
val wordDb = WordListDatabase.getDatabase(application)
val wordDao = wordDb.wordlistDao()
repository = WordRepository(wordDao)
word = viewModelScope.launch {
repository.readWord()
}.toString()
Log.d("TAG", ": $word") // does nothing?
}
println(word) // StandaloneCoroutine{Active}#933049a
}
This is the only way that I have managed to not get the result of:
Cannot access database on the main thread
There is a better way to do this, I just can't figure it out.
You can access the return value of repository.readWord() only inside the launch block.
viewModelScope.launch {
val word = repository.readWord()
Log.d("TAG", ": $word") // Here you will get the correct word
}
If you need to update you UI when this word is fetched from database, you need to use an observable data holder like a LiveData or StateFlow.
class HomeViewModel(application: Application) : ViewModel() {
private val repository: WordRepository
private val _wordFlow = MutableStateFlow("") // A mutable version for use inside ViewModel
val wordFlow = _word.asStateFlow() // An immutable version for outsiders to read this state
init {
val wordDb = WordListDatabase.getDatabase(application)
val wordDao = wordDb.wordlistDao()
repository = WordRepository(wordDao)
viewModelScope.launch {
_wordFlow.value = repository.readWord()
}
}
}
You can collect this Flow in your UI layer,
someCoroutineScope {
viewModel.wordFlow.collect { word ->
// Update UI using this word
}
}
Edit: Since you don't need the word immediately, you can just save the word in a simple global variable for future use, easy.
class HomeViewModel(application: Application) : ViewModel() {
private lateinit var repository: WordRepository
private lateinit var word: String
init {
val wordDb = WordListDatabase.getDatabase(application)
val wordDao = wordDb.wordlistDao()
repository = WordRepository(wordDao)
viewModelScope.launch {
word = repository.readWord()
}
// word is not available here, but you also don't need it here
}
// This is the function which is called when user types a word and presses enter
fun submitGuess(userGuess: String) {
// You can access the `word` here and compare it with `userGuess`
}
}
The database operation will only take a few milliseconds to complete so you can be sure that by the time you actually need that original word, it will have been fetched and stored in the word variable.
(Now that I'm at a computer I can write a bit more.)
The problems with your current code:
You cannot safely read from the database on the main thread synchronously. That's why the suspend keyword would be used in your DAO/repository. Which means, there is no way you can have a non-nullable word property in your ViewModel class that is initialized in an init block.
Coroutines are asychronous. When you call launch, it is queuing up the coroutine to start its work, but the launch function returns a Job, not the result of the coroutine, and your code beneath the launch call continues on the same thread. The code inside the launch call is sent off to the coroutines system to be run and suspend calls will in most cases, as in this case, be switching to background threads back and forth. So when you call toString() on the Job, you are just getting a String representation of the coroutine Job itself, not the result of its work.
Since the coroutine does its work asynchronously, when you try to log the result underneath the launch block, you are logging it before the coroutine has even had a chance to fetch the value yet. So even if you had assigned the result of the coroutine to some String variable, it would still be null by the time you are logging it.
For your database word to be usable outside a coroutine, you need to put it in something like a LiveData or SharedFlow so that other places in code can subscribe to it and do something with the value when it arrives.
SharedFlow is a pretty big topic to learn, so I'll just use LiveData for the below samples.
One way to create a LiveData using your suspend function to retrieve the word is to use the liveData builder function, which returns a LiveData that uses a coroutine under the hood to get the value to publish via the LiveData:
class HomeViewModel(application: Application) : ViewModel() {
private val repository: WordRepository = WordListDatabase.getDatabase(application)
.wordDb.wordlistDao()
.let(::WordRepository)
private val word: LiveData<String> = liveData {
repository.readWord()
}
val someLiveDataForUi: LiveData<Something> = Transformations.map(word) { word ->
// Do something with word and return result. The UI code can
// observe this live data to get the result when it becomes ready.
}
}
To do this in a way that is more similar to your code (just to help with understanding, since this is less concise), you can create a MutableLiveData and publish to the LiveData from your coroutine.
class HomeViewModel(application: Application) : ViewModel() {
private val repository: WordRepository
private val word = MutableLiveData<String>()
init {
val wordDb = WordListDatabase.getDatabase(application)
val wordDao = wordDb.wordlistDao()
repository = WordRepository(wordDao)
viewModelScope.launch {
word.value = repository.readWord()
}
}
val someLiveDataForUi: LiveData<Something> = Transformations.map(word) { word ->
// Do something with word and return result. The UI code can
// observe this live data to get the result when it becomes ready.
}
}
If you're not ready to dive into coroutines yet, you can define your DAO to return a LiveData instead of suspending. It will start reading the item from the database and publish it through the live data once it's ready.
#Dao
interface WordListDao {
#Query("SELECT word FROM wordlist WHERE used = 0 ORDER BY id DESC LIMIT 1")
fun readWord(): LiveData<String>
}
class HomeViewModel(application: Application) : ViewModel() {
private val repository: WordRepository = WordListDatabase.getDatabase(application)
.wordDb.wordlistDao()
.let(::WordRepository)
private val word: LiveData<String> = repository.readWord()
//...
}
The return value is as expected, because launch does always return a Job object representing the background process.
I do not know how you want to use the String for, but all operations which should be done after receiving the String must be moved inside the Coroutine or in a function which is called from the Coroutine.
viewModelScope.launch {
val word = repository.readWord()
// do stuff with word
// switch to MainThread if needed
launch(Dispatchers.Main){}
}
I'm trying to build a class where certain values are Observable but also Serializable.
This obviously works and the serialization works, but it's very boilerplate-heavy having to add a setter for every single field and manually having to call change(...) inside each setter:
interface Observable {
fun change(message: String) {
println("changing $message")
}
}
#Serializable
class BlahVO : Observable {
var value2: String = ""
set(value) {
field = value
change("value2")
}
fun toJson(): String {
return Json.encodeToString(serializer(), this)
}
}
println(BlahVO().apply { value2 = "test2" })
correctly outputs
changing value2
{"value2":"test2"}
I've tried introducing Delegates:
interface Observable {
fun change(message: String) {
println("changing $message")
}
#Suppress("ClassName")
class default<T>(defaultValue: T) {
private var value: T = defaultValue
operator fun getValue(observable: Observable, property: KProperty<*>): T {
return value
}
operator fun setValue(observable: Observable, property: KProperty<*>, value: T) {
this.value = value
observable.change(property.name)
}
}
}
#Serializable
class BlahVO : Observable {
var value1: String by Observable.default("value1")
fun toJson(): String {
return Json.encodeToString(serializer(), this)
}
}
println(BlahVO().apply { value1 = "test1" }) correctly triggers change detection, but it doesn't serialize:
changing value1
{}
If I go from Observable to ReadWriteProperty,
interface Observable {
fun change(message: String) {
println("changing $message")
}
fun <T> look(defaultValue: T): ReadWriteProperty<Observable, T> {
return OP(defaultValue, this)
}
class OP<T>(defaultValue: T, val observable: Observable) : ObservableProperty<T>(defaultValue) {
override fun setValue(thisRef: Any?, property: KProperty<*>, value: T) {
super.setValue(thisRef, property, value)
observable.change("blah!")
}
}
}
#Serializable
class BlahVO : Observable {
var value3: String by this.look("value3")
fun toJson(): String {
return Json.encodeToString(serializer(), this)
}
}
the result is the same:
changing blah!
{}
Similarly for Delegates.vetoable
var value4: String by Delegates.vetoable("value4", {
property: KProperty<*>, oldstring: String, newString: String ->
this.change(property.name)
true
})
outputs:
changing value4
{}
Delegates just doesn't seem to work with Kotlin Serialization
What other options are there to observe a property's changes without breaking its serialization that will also work on other platforms (KotlinJS, KotlinJVM, Android, ...)?
Serialization and Deserialization of Kotlin Delegates is not supported by kotlinx.serialization as of now.
There is an open issue #1578 on GitHub regarding this feature.
According to the issue you can create an intermediate data-transfer object, which gets serialized instead of the original object. Also you could write a custom serializer to support the serialization of Kotlin Delegates, which seems to be even more boilerplate, then writing custom getters and setters, as proposed in the question.
Data Transfer Object
By mapping your original object to a simple data transfer object without delegates, you can utilize the default serialization mechanisms.
This also has the nice side effect to cleanse your data model classes from framework specific annotations, such as #Serializable.
class DataModel {
var observedProperty: String by Delegates.observable("initial") { property, before, after ->
println("""Hey, I changed "${property.name}" from "$before" to "$after"!""")
}
fun toJson(): String {
return Json.encodeToString(serializer(), this.toDto())
}
}
fun DataModel.toDto() = DataTransferObject(observedProperty)
#Serializable
class DataTransferObject(val observedProperty: String)
fun main() {
val data = DataModel()
println(data.toJson())
data.observedProperty = "changed"
println(data.toJson())
}
This yields the following result:
{"observedProperty":"initial"}
Hey, I changed "observedProperty" from "initial" to "changed"!
{"observedProperty":"changed"}
Custom data type
If changing the data type is an option, you could write a wrapping class which gets (de)serialized transparently. Something along the lines of the following might work.
#Serializable
class ClassWithMonitoredString(val monitoredProperty: MonitoredString) {
fun toJson(): String {
return Json.encodeToString(serializer(), this)
}
}
fun main() {
val monitoredString = obs("obsDefault") { before, after ->
println("""I changed from "$before" to "$after"!""")
}
val data = ClassWithMonitoredString(monitoredString)
println(data.toJson())
data.monitoredProperty.value = "obsChanged"
println(data.toJson())
}
Which yields the following result:
{"monitoredProperty":"obsDefault"}
I changed from "obsDefault" to "obsChanged"!
{"monitoredProperty":"obsChanged"}
You however lose information about which property changed, as you don't have easy access to the field name. Also you have to change your data structures, as mentioned above and might not be desirable or even possible. In addition, this work only for Strings for now, even though one might make it more generic though.
Also, this requires a lot of boilerplate to start with. On the call site however, you just have to wrap the actual value in an call to obs.
I used the following boilerplate to get it to work.
typealias OnChange = (before: String, after: String) -> Unit
#Serializable(with = MonitoredStringSerializer::class)
class MonitoredString(initialValue: String, var onChange: OnChange?) {
var value: String = initialValue
set(value) {
onChange?.invoke(field, value)
field = value
}
}
fun obs(value: String, onChange: OnChange? = null) = MonitoredString(value, onChange)
object MonitoredStringSerializer : KSerializer<MonitoredString> {
override val descriptor: SerialDescriptor = PrimitiveSerialDescriptor("MonitoredString", PrimitiveKind.STRING)
override fun serialize(encoder: Encoder, value: MonitoredString) {
encoder.encodeString(value.value)
}
override fun deserialize(decoder: Decoder): MonitoredString {
return MonitoredString(decoder.decodeString(), null)
}
}
I have a class that calls functions depending on events. Events are emitted from sockets. I should catch these events, parse JSON and respond (call a corresponding function). For instance, {"event_name": "message", "data": {"text": "dfgfdgfdg", "sender": "dsfdsfs"}}
fun listener(jsonString: String, methodsMap: Map<String, () -> Unit>) {
val json = JSONObject(jsonString)
val data = json.getJSONObject("data")
when (json.get("event_name")) {
"update" -> {
val count = data.getInt("count")
methodsMap["update"]?.invoke(count) // 1 parameter.
}
"message" -> {
val message = data.getString("text")
val sender = data.getString("sender")
methodsMap["message"]?.invoke(message, sender) // 2 parameters.
}
}
}
So, I cannot create one method that calls functions with different parameters. How to do this?
Since you are already have if-then logic in listener, having the functions in a Map is of questionable value and it forces you to to deal with the fact that your functions are of different types. If it is parametrisation of listener you are after, perhaps this (simplified example code that skips JSON) is sufficient:
class UpdateHandler {
fun update(n: Int) = println("update ( $n )")
}
class MessageHandler {
fun message(s1: String, s2: String) = println("message ( $s1 $s2 )")
}
fun listener(jsonString: String, updateF: (Int) -> Unit, messageF: (String, String) -> Unit) {
when (jsonString) {
"update" -> updateF(73)
"message" -> messageF("message", "sender")
}
}
fun main() {
val updateHandler = UpdateHandler()
val messageHandler = MessageHandler()
val listener = { json: String -> listener(json, updateHandler::update, messageHandler::message) }
listener("update") // prints: update ( 73 )
listener("message")// prints: message ( message sender )
}
First, I wanted to use a list of parameters in each function, but it leads to poor type verification during compilation. Also I wanted to assign vararg instead of List, but couldn't.
fun listener(jsonString: String, methodsMap: Map<String, (List<Any>) -> Unit>) {
...
methodsMap["update"]?.invoke(listOf(count)) // 1 parameter.
...
methodsMap["message"]?.invoke(listOf(message, sender)) // 2 parameters.
}
This is a poor solution. Bugs may occur, we should remember to change methodsMap in every class that uses listener when we change any event.
Second, I tried to use sealed classes. This is not so simple.
Third, I tried to use interface. We know that callbacks are usually made with interfaces. We can even merge interfaces in Kotlin. So, this can be a solution to a problem (but not to a question).
fun listener(jsonString: String, callback: EventListener) {
val json = JSONObject(jsonString)
val data = json.getJSONObject("data")
when (json.get("event_name")) {
"update" -> {
val count = data.getInt("count")
callback.onUpdate(count)
}
"message" -> {
val text = data.getString("text")
val sender = data.getString("sender")
callback.onNewMessage(text, sender)
}
}
}
interface EventListener {
fun onUpdate(count: Int)
fun onNewMessage(text: String, sender: String)
}
Then we can call listener outside of the class and pass any callbacks we like.
i was wondering if it is possible to send messages from the backend (for example a running task that receives information from an external system) to the UI. In my case it needs to be a specific session (no broadcast) and only on a specific screen
plan B would be polling the backend frequently but i was hoping to get something more "realtime"
I was trying to work something out like this, but i keep getting a NotSerializableException.
#Push
class StorageAccess : Screen(), MessageListener {
#Inject
private lateinit var stationWSService: StationWebSocketService
#Inject
private lateinit var notifications: Notifications
#Subscribe
private fun onInit(event: InitEvent) {
}
#Subscribe("stationPicker")
private fun onStationPickerValueChange(event: HasValue.ValueChangeEvent<StorageUnit>) {
val current = AppUI.getCurrent()
current.userSession.id ?: return
val prevValue = event.prevValue
if (prevValue != null) {
stationWSService.remove(current.userSession.id)
}
val value = event.value ?: return
stationWSService.listen(current.userSession.id, value, this)
}
override fun messageReceived(message: String) {
val current = AppUI.getCurrent()
current.access {
notifications.create().withCaption(message).show()
}
}
#Subscribe
private fun onAfterDetach(event: AfterDetachEvent) {
val current = AppUI.getCurrent()
current.userSession.id ?: return
stationWSService.remove(current.userSession.id)
}
}
-- The callback interface
interface MessageListener : Serializable {
fun messageReceived(message: String);
}
-- The listen method of my backend service
private val listeners: MutableMap<String, MutableMap<UUID, MessageListener>> = HashMap()
override fun listen(id: UUID, storageUnit: StorageUnit, callback: MessageListener) {
val unitStationIP: String = storageUnit.unitStationIP ?: return
if (!listeners.containsKey(unitStationIP))
listeners[unitStationIP] = HashMap()
listeners[unitStationIP]?.set(id, callback)
}
The Exception i get is NotSerializableException: com.haulmont.cuba.web.sys.WebNotifications which happens during adding the listener to the backend: stationWSService.listen(current.userSession.id, value, this)
as far as i understand this is the place where the UI sends the information to the backend - and with it the entire status of the class StorageAccess, including all its members.
is there an elegant solution to this?
regards
There is an add-on that solves exactly this problem: https://github.com/cuba-platform/global-events-addon
I am trying to implement a QueryBus. Basically, I want to register a list of QueryHandlers. Each QueryHandler implements a handle method defined by an interface. Each QueryHandler is associated to a Query. I want to be able to retrieve a QueryHandler using the Query and call handle on it.
The thing is the handle has to be generic because each QueryHandler handles a Query differently. They all take a dedicated Query and may return whatever they want.
interface Query<R>
interface QueryHandler<R, Q : Query<R>> {
fun handle(query: Q): R
fun listenTo(): String
}
// DTOs
data class BookDto(val name: String)
// List books query
data class ListBooksQuery(val page: Int = 1): Query<List<BookDto>>
class ListBooksQueryHandler: QueryHandler<List<BookDto>, ListBooksQuery> {
override fun handle(query: ListBooksQuery): List<BookDto> {
return listOf(BookDto("Dune"), BookDto("Dune II"))
}
override fun listenTo(): String = ListBooksQuery::class.toString()
}
// Get book query
data class GetBookQuery(val name: String): Query<BookDto?>
class GetBookQueryHandler: QueryHandler<BookDto?, GetBookQuery> {
override fun handle(query: GetBookQuery): BookDto {
return BookDto("Dune")
}
override fun listenTo(): String = GetBookQuery::class.toString()
}
// Run it!
fun main(args: Array<String>) {
// Initializing query bus
val queryHandlers = mapOf(
with(ListBooksQueryHandler()) {this.listenTo() to this},
with(GetBookQueryHandler()) {this.listenTo() to this}
)
val command = ListBooksQuery()
val result = queryHandlers[command::class.toString()].handle(command)
// Should print the list of BookDto
print(result)
}
I don't even know if its possible, to be honest.
UPDATE 1:
I changed the usage example in the main to show what I am really trying to do. The List was for (bad?) demonstration purpose. I want to store the QueryHandlers and retrieve them from a map.
Additional resources:
Here is what I really want to do:
https://gist.github.com/ValentinTrinque/76b7a32221884a46e657090b9ee60193
UPDATE I've read your gist and tried to come up with a solution that will provide a clean interface to the user of the QueryBusMiddleware.
Note that I used objects instead of classes for the QueryHandler implementations, which felt more natural to me (since there is only one possible entry in the map for each Query implementation).
interface Query<R>
interface QueryHandler<R, Q: Query<R>> {
fun handle(query: Q): R
fun listenTo(): String
}
// DTOs
data class BookDto(val name: String)
// List books query
data class ListBooksQuery(val page: Int = 1): Query<List<BookDto>>
object ListBooksQueryHandler: QueryHandler<List<BookDto>, ListBooksQuery> {
override fun handle(query: ListBooksQuery): List<BookDto> {
return listOf(BookDto("Dune"), BookDto("Dune II"))
}
override fun listenTo(): String = ListBooksQuery::class.toString()
}
// Get book query
data class GetBookQuery(val name: String): Query<BookDto?>
object GetBookQueryHandler: QueryHandler<BookDto?, GetBookQuery> {
override fun handle(query: GetBookQuery): BookDto {
return BookDto("Dune")
}
override fun listenTo(): String = GetBookQuery::class.toString()
}
// Run it!
fun main(args: Array<String>) {
// Initializing query bus
val queryHandlers = listOf(
ListBooksQueryHandler,
GetBookQueryHandler
)
val dispatcher: QueryBusMiddleware = QueryDispatcherMiddleware(queryHandlers)
// Calling query bus
val query = ListBooksQuery()
// Result should be List<BookDto>
val result = dispatcher.dispatch(query)
print(result)
}
interface QueryBusMiddleware {
fun <R, Q : Query<R>> dispatch(query: Q): R
}
class QueryDispatcherMiddleware constructor(handlers: List<QueryHandler<*, *>>) : QueryBusMiddleware {
private val handlers = HashMap<String, QueryHandler<*, *>>()
init {
handlers.forEach { handler -> this.handlers[handler.listenTo()] = handler }
}
override fun <R, Q : Query<R>> dispatch(query: Q): R {
val queryClass = query::class.toString()
val handler = handlers[queryClass] ?: throw Exception("No handler listen to the query: $queryClass")
return handler::class.members.find { it.name == "handle" }!!.call(handler, query) as R
}
}