I'm building an object detection application (in Kotlin, for Android). The application uses CameraX to build a camera preview and Google ML to provide machine learning expertise. Just for reference; I used this CameraX documentation and this this Google ML Kit documentation.
I'm currently attempting to print Log.d("TAG", "onSuccess" + it.size) to my IDE console in order to determine if .addonSuccessListener is actually running. If it does, it should print something along the lines of onSuccess1. However, this isn't the case. Infact, it isn't even printing the Log statement from the .addOnFailureListener either, which really confuses me as I'm not even entirely sure the objectDetector code is even running. What really puzzles me is that I have relatively completed the same project in Java and have not faced this issue.
I did have someone point out that within my YourImageAnalyzer.kt class, if mediaImage is null, then I won't see anything logging. However, upon my own debugging (this is actually my very first time debugging), I was unable to find out if my first sentence of this paragraph is true or not. I suppose this issue may provide a lead into how I'll resolve this issue, and also learn how to properly debug.
Here is my YourImageAnalyzer.kt class, and I will also add the code for my MainActivity.kt class below as well.
YourImageAnalyzer.kt
private class YourImageAnalyzer : ImageAnalysis.Analyzer {
override fun analyze(imageProxy: ImageProxy) {
val mediaImage = imageProxy.image
if (mediaImage != null) {
val image =
InputImage.fromMediaImage(mediaImage, imageProxy.imageInfo.rotationDegrees)
val localModel = LocalModel.Builder()
.setAssetFilePath("mobilenet_v1_0.75_192_quantized_1_metadata_1.tflite")
.build()
val customObjectDetectorOptions =
CustomObjectDetectorOptions.Builder(localModel)
.setDetectorMode(CustomObjectDetectorOptions.STREAM_MODE)
.enableClassification()
.setClassificationConfidenceThreshold(0.5f)
.setMaxPerObjectLabelCount(3)
.build()
val objectDetector =
ObjectDetection.getClient(customObjectDetectorOptions)
objectDetector //Here is where the issue stems, with the following listeners
.process(image)
.addOnSuccessListener {
Log.i("TAG", "onSuccess" + it.size)
for (detectedObjects in it)
{
val boundingBox = detectedObjects.boundingBox
val trackingId = detectedObjects.trackingId
for (label in detectedObjects.labels) {
val text = label.text
val index = label.index
val confidence = label.confidence
}
}
}
.addOnFailureListener { e -> Log.e("TAG", e.getLocalizedMessage()) }
.addOnCompleteListener { it -> imageProxy.close() }
}
}
}
MainActivity.kt
class MainActivity : AppCompatActivity() {
private lateinit var cameraProviderFuture: ListenableFuture<ProcessCameraProvider>
override fun onCreate(savedInstanceState: Bundle?) {
cameraProviderFuture = ProcessCameraProvider.getInstance(this)
super.onCreate(savedInstanceState)
setContentView(R.layout.activity_main)
cameraProviderFuture.addListener(Runnable {
val cameraProvider = cameraProviderFuture.get()
bindPreview(cameraProvider)
}, ContextCompat.getMainExecutor(this))
}
fun bindPreview(cameraProvider: ProcessCameraProvider) {
val previewView = findViewById<PreviewView>(R.id.previewView)
var preview : Preview = Preview.Builder()
.build()
var cameraSelector : CameraSelector = CameraSelector.Builder()
.requireLensFacing(CameraSelector.LENS_FACING_BACK)
.build()
preview.setSurfaceProvider(previewView.surfaceProvider)
var camera = cameraProvider.bindToLifecycle(this as LifecycleOwner, cameraSelector, preview)
}
}
You are not binding your ImageAnalysis use case. Something in the line of:
val imageAnalysis = ImageAnalysis.Builder()
.setTargetResolution(Size(1280, 720))
.setBackpressureStrategy(ImageAnalysis.STRATEGY_KEEP_ONLY_LATEST)
.setOutputImageFormat(ImageAnalysis.OUTPUT_IMAGE_FORMAT_RGBA_8888)
.build()
and then;
imageAnalysis.setAnalyzer(executor, YourImageAnalyzer())
cameraProvider.bindToLifecycle(this as LifecycleOwner, cameraSelector, imageAnalysis, preview)
Also a suggestion as a bonus:
You should get your LocalModel.Builder() out of analyze as this is called each time an image arrives. You do not need to execute this code piece each time as it will make your analysis slower.
So move this code:
val localModel = LocalModel.Builder()
.setAssetFilePath("mobilenet_v1_0.75_192_quantized_1_metadata_1.tflite")
.build()
to just below of the class private class YourImageAnalyzer : ImageAnalysis.Analyzer {.
Related
viewModelScope blocks UI in Jetpack Compose
I know viewModelScope.launch(Dispatchers.IO) {} can avoid this problem, but how to use viewModelScope.launch(Dispatchers.IO) {}?
This is my UI level code
#Composable
fun CountryContent(viewModel: CountryViewModel) {
SingleRun {
viewModel.getCountryList()
}
val pagingItems = viewModel.countryGroupList.collectAsLazyPagingItems()
// ...
}
Here is my ViewModel, Pager is my pagination
#HiltViewModel
class CountryViewModel #Inject constructor() : BaseViewModel() {
var countryGroupList = flowOf<PagingData<CountryGroup>>()
private val config = PagingConfig(pageSize = 26, prefetchDistance = 1, initialLoadSize = 26)
fun getCountryList() {
countryGroupList = Pager(config) {
CountrySource(api)
}.flow.cachedIn(viewModelScope)
}
}
This is the small package
#Composable
fun SingleRun(onClick: () -> Unit) {
val execute = rememberSaveable { mutableStateOf(true) }
if (execute.value) {
onClick()
execute.value = false
}
}
I don't use Compose much yet, so I could be wrong, but this stood out to me.
I don't think your thread is being blocked. I think you subscribed to an empty flow before replacing it, so there is no data to show.
You shouldn't use a var property for your flow, because the empty original flow could be collected before the new one replaces it. Also, it defeats the purpose of using cachedIn because the flow could be replaced multiple times.
You should eliminate the getCountryList() function and just directly assign the flow. Since it is a cachedIn flow, it doesn't do work until it is first collected anyway. See the documentation:
It won't execute any unnecessary code unless it is being collected.
So your view model should look like:
#HiltViewModel
class CountryViewModel #Inject constructor() : BaseViewModel() {
private val config = PagingConfig(pageSize = 26, prefetchDistance = 1, initialLoadSize = 26)
val countryGroupList = Pager(config) {
CountrySource(api)
}.flow.cachedIn(viewModelScope)
}
}
...and you can remove the SingleRun block from your Composable.
You are not doing anything that would require you to specify dispatchers. The default of Dispatchers.Main is fine here because you are not calling any blocking functions directly anywhere in your code.
I use Jetpack Compose and have 2 screens. When I open second screen and back to the fisrt, flow variable calling again and ui updated again. But, I don't understand why... When I use liveData was working perfect.
My code with LiveData:
class MainViewModel(private val roomRepository: Repository, private val sPref:SharedPreferences) : ViewModel() {
val words: LiveData<List<WordModel>> by lazy {
roomRepository.getAllWords()
}
...
}
MainScreen.kt:
#ExperimentalMaterialApi
#Composable
fun MainScreen(viewModel: MainViewModel) {
...
val words: List<WordModel> by viewModel
.words
.observeAsState(listOf())
...
WordList(
words = words,
onNoticeClick = { viewModel.onWordClick(it) },
state = textState,
lazyState = viewModel.listState!!
)
...
}
#Composable
private fun WordList(
words: List<WordModel>,
onNoticeClick: (WordModel) -> Unit,
state: MutableState<TextFieldValue>,
lazyState: LazyListState
) {
var filteredCountries: List<WordModel>
LazyColumn(state = lazyState) {
val searchedText = state.value.text
filteredCountries = if (searchedText.isEmpty()) {
words
} else {
words.filter {
it.word.lowercase().contains(searchedText) || it.translation.lowercase()
.contains(searchedText)
}
}
items(count = filteredCountries.size) { noteIndex ->
val note = filteredCountries[noteIndex]
Word(
word = note,
onWordClick = onNoticeClick
)
}
}
}
WordDao.kt:
#Dao
interface WordDao {
#Query("SELECT * FROM WordDbModel")
fun getAll(): LiveData<List<WordDbModel>>
}
RoomRepositoryImpl.kt:
class RoomRepositoryImpl(
private val wordDao: WordDao,
private val noticeDao: NoticeDao,
private val dbMapper: DbMapper
) : Repository {
override fun getAllWords(): LiveData<List<WordModel>> =
Transformations.map(wordDao.getAll()) {dbMapper.mapWords(it)}
...
}
DbMapperImpl.kt:
class DbMapperImpl: DbMapper {
...
override fun mapWords(words: List<WordDbModel>): List<WordModel> =
words.map { word -> mapWord(word, listOf<NoticeModel>()) }
}
My code with Flow, which calling every time when open the first screen:
class MainViewModel(private val roomRepository: Repository, private val sPref:SharedPreferences) : ViewModel() {
val words: Flow<List<WordModel>> = flow {
emitAll(repository.getAllWords())
}
}
MainScreen.kt:
#ExperimentalMaterialApi
#Composable
fun MainScreen(viewModel: MainViewModel) {
...
val words: List<WordModel> by viewModel
.words
.collectAsState(initial = listOf())
...
}
WordDao.kt:
#Dao
interface WordDao {
#Query("SELECT * FROM WordDbModel")
fun getAll(): Flow<List<WordDbModel>>
}
RoomRepositoryImpl.kt:
class RoomRepositoryImpl(
private val wordDao: WordDao,
private val noticeDao: NoticeDao,
private val dbMapper: DbMapper
) : Repository {
override fun getWords(): Flow<List<WordModel>> = wordDao.getAll().map { dbMapper.mapWords(it) }
}
And my router from MainRouting.kt:
sealed class Screen {
object Main : Screen()
object Notice : Screen()
object Your : Screen()
object Favorites : Screen()
}
object MainRouter {
var currentScreen: Screen by mutableStateOf(Screen.Main)
var beforeScreen: Screen? = null
fun navigateTo(destination: Screen) {
beforeScreen = currentScreen
currentScreen = destination
}
}
And MainActivity.kt:
class MainActivity : ComponentActivity() {
...
#Composable
#ExperimentalMaterialApi
private fun MainActivityScreen(viewModel: MainViewModel) {
Surface {
when (MainRouter.currentScreen) {
is Screen.Main -> MainScreen(viewModel)
is Screen.Your -> MainScreen(viewModel)
is Screen.Favorites -> MainScreen(viewModel)
is Screen.Notice -> NoticeScreen(viewModel = viewModel)
}
}
}
...
}
Perhaps someone knows why a new drawing does not occur with liveData (or, it is performed so quickly that it is not noticeable that it is), but with Flow the drawing of the list is visible.
You're passing the viewModel around, which is a terrible practice in a framework like Compose. The Model is like a waiter. It hangs around you, serves you water, does its job while you make the order. As you get distracted talking, it leaves. When it comes back, it is not the same waiter you had earlier. It wears the same uniform, with the same characteristics, but is still essentially a different object. When you pass the model around, it gets destroyed in the process of navigation. In case of flow, you are getting biased. Notice how you manually do a lazy initialization for the LiveData, but a standard proc. for Flow? Seems like the only logical reason for your observed inconsistency. If you want to use Flow in your calls instead of LiveData, just convert it at the site of initialization in the ViewModel. Your symptoms should go away.
I run into a problem during writing tests for a viewModel. The problem occurred when I was trying to verify LiveData that is updated with channelFlow flow on Dispatchers.IO.
I created a simple project to show the issue.
There is a data provider class that is providing 10 numbers:
As it is, the numbers variable in the test is empty and the test fails. I know it is a problem with coroutine dispatchers.
val numbersFlow: Flow<Int> = channelFlow {
var i = 0
while (i < 10) {
delay(100)
send(i)
i++
}
}.flowOn(Dispatchers.IO)
a simple viewModel that is collecting data:
class NumbersViewModel: ViewModel() {
private val _numbers: MutableLiveData<IntArray> = MutableLiveData(IntArray(0))
val numbers: LiveData<IntArray> = _numbers
val dataProvider = NumbersProvider()
fun startCollecting() {
viewModelScope.launch(Dispatchers.Main) {
dataProvider.numbersFlow
.onStart { println("start") }
.onCompletion { println("end") }
.catch { exception -> println(exception.message.orEmpty())}
.collect { data -> onDataRead(data) }
}
}
fun onDataRead(data: Int) {
_numbers.value = _numbers.value?.plus(data)
}
}
and the test:
class NumbersViewModelTest {
#get:Rule
var instantTaskExecutorRule = InstantTaskExecutorRule()
#get:Rule
var mainCoroutineRule = MainCoroutineRule()
private lateinit var viewModel: NumbersViewModel
#Before
fun setUp() {
viewModel = NumbersViewModel()
}
#Test
fun `provider_provides_10_values`() {
viewModel.startCollecting()
mainCoroutineRule.advanceTimeBy(2000)
val numbers = viewModel.numbers.value
assertThat(numbers?.size).isEqualTo(10)
}
}
There is a common solution with changing the main dispatcher for test usage but... is there any good solution for dealing with the IO one?
I found a solution with injecting dispatchers everywhere - similarly to how I would inject NumbersProvider using Hilt in a real app - and that enables injecting our test dispatcher when we need it. It works but now I have to inject dispatchers everywhere in the code and I don't really like that if it only serves to solve the testing problem
I tried another solution and created a Singleton which makes all the standard dispatchers available in the production code and which I can configure for tests (by setting every dispatcher to the test one). I like how the resulting source code looks more - there is no additional code in viewModels and data providers but there is this singleton and everyone shouting 'Don't use singletons'
Is there any better option to correctly test code with coroutines?
I am writing a Kotlin app and using Firebase for authentication.
As onActivityResult is now depraceted, I am trying to migrate my app to use registerForActivityResult. I have a link to Google account feature, that starts with the Google sign-in flow, as shown here. My code:
private fun initGoogleSignInClient() =
activity?.let {
// Configure Google Sign In
val gso =
GoogleSignInOptions.Builder(GoogleSignInOptions.DEFAULT_SIGN_IN)
.requestIdToken(getString(R.string.default_web_client_id))
.requestEmail()
.build()
// Build a GoogleSignInClient with the options specified by gso.
viewModel.googleSignInClient = GoogleSignIn.getClient(it, gso)
}
private fun showLinkWithGoogle() =
startActivityForResult(viewModel.googleSignInClient.signInIntent, RC_LINK_GOOGLE)
Where initGoogleSignInClient is called in the fragment's onCreateView, and showLinkWithGoogle is called when the user taps the button on the screen. This workes perfectly.
I looked for an example using registerForActivityResult, and the best one I found was at the bottom of this page. I added this code:
private val linkWithGoogle =
registerForActivityResult(ActivityResultContracts.StartIntentSenderForResult()) {
viewModel.handleGoogleResult(it.data)
}
private fun showLinkWithGoogle() =
linkWithGoogle.launch(IntentSenderRequest.Builder(viewModel.googleSignInClient.signInIntent))
But realized that IntentSenderRequest.Builder needs an IntentSender and not an Intent. I haven't found any example of how to build an IntentSender from an Intent, nor a way to get one from my GoogleSignInClient.
Could anyone please provide a full example of using registerForActivityResult(ActivityResultContracts.StartIntentSenderForResult())?
Thank you very much!
For this use-case, you don't need an ActivityResultContracts of type StartIntentSenderForResult but one of type StartActivityForResult. Here is an example (since you did not provide your full implementation):
Fragment
private val googleRegisterResult =
registerForActivityResult(ActivityResultContracts.StartActivityForResult()) { result ->
result.checkResultAndExecute {
val task = GoogleSignIn.getSignedInAccountFromIntent(data)
val account = task.getResult(ApiException::class.java)
loginViewModel.onEvent(LoginRegistrationEvent.SignInWithGoogle(account))
}.onFailure { e -> toast("Error: ${e.message}") }
}
override fun onViewCreated(view: View, savedInstanceState: Bundle?) {
super.onViewCreated(view, savedInstanceState)
myGoogleSignInButton.setOnClickListener {
googleRegisterResult.launch(viewModel.googleSignInClient.signInIntent)
}
}
Then, in your viewmodel, you can handle the login as you would usually do, the only difference is, that you no longer need an RC_SIGN_IN
ViewModel Example
class YourViewModel : ViewModel() {
fun onEvent(event: LoginRegistrationEvent) {
when(event) {
is LoginRegistrationEvent.SignInWithGoogle -> {
viewModelScope.launch {
val credential = GoogleAuthProvider.getCredential(event.account.idToken)
Firebase.auth.signInWithCredential(credential).await()
}
}
}
}
}
To make my life easier, I created an extension function, that checks, if the login was successful and then executes a block of code (in this case, getting the account), while caching any exceptions. Futhermore, inside your block, you have access to an instance of ActivityResult as this:
inline fun ActivityResult.checkResultAndExecute(block: ActivityResult.() -> Unit) =
if (resultCode == Activity.RESULT_OK) runCatching(block)
else Result.failure(Exception("Something went wrong"))
i was wondering if it is possible to send messages from the backend (for example a running task that receives information from an external system) to the UI. In my case it needs to be a specific session (no broadcast) and only on a specific screen
plan B would be polling the backend frequently but i was hoping to get something more "realtime"
I was trying to work something out like this, but i keep getting a NotSerializableException.
#Push
class StorageAccess : Screen(), MessageListener {
#Inject
private lateinit var stationWSService: StationWebSocketService
#Inject
private lateinit var notifications: Notifications
#Subscribe
private fun onInit(event: InitEvent) {
}
#Subscribe("stationPicker")
private fun onStationPickerValueChange(event: HasValue.ValueChangeEvent<StorageUnit>) {
val current = AppUI.getCurrent()
current.userSession.id ?: return
val prevValue = event.prevValue
if (prevValue != null) {
stationWSService.remove(current.userSession.id)
}
val value = event.value ?: return
stationWSService.listen(current.userSession.id, value, this)
}
override fun messageReceived(message: String) {
val current = AppUI.getCurrent()
current.access {
notifications.create().withCaption(message).show()
}
}
#Subscribe
private fun onAfterDetach(event: AfterDetachEvent) {
val current = AppUI.getCurrent()
current.userSession.id ?: return
stationWSService.remove(current.userSession.id)
}
}
-- The callback interface
interface MessageListener : Serializable {
fun messageReceived(message: String);
}
-- The listen method of my backend service
private val listeners: MutableMap<String, MutableMap<UUID, MessageListener>> = HashMap()
override fun listen(id: UUID, storageUnit: StorageUnit, callback: MessageListener) {
val unitStationIP: String = storageUnit.unitStationIP ?: return
if (!listeners.containsKey(unitStationIP))
listeners[unitStationIP] = HashMap()
listeners[unitStationIP]?.set(id, callback)
}
The Exception i get is NotSerializableException: com.haulmont.cuba.web.sys.WebNotifications which happens during adding the listener to the backend: stationWSService.listen(current.userSession.id, value, this)
as far as i understand this is the place where the UI sends the information to the backend - and with it the entire status of the class StorageAccess, including all its members.
is there an elegant solution to this?
regards
There is an add-on that solves exactly this problem: https://github.com/cuba-platform/global-events-addon