MLKit pose detection - google-mlkit

cloned an APP from github that is supposed to take a video and estimate a person's pose using MLKIT.
The thing is the app I think fills up the memory buffers and crashes itself and I don't know how to get a result, if anyone could give me some tips on how to avoid this crash on Kotlin and thank you.
Log.d("LOG:", "Here")
val bm = mTextureView.bitmap
if (bm != null) {
Log.d("LOG:", "not null")
bm.compress(Bitmap.CompressFormat.JPEG, 50,out)
val inputImage = InputImage.fromBitmap(
BitmapFactory.decodeStream( ByteArrayInputStream(out.toByteArray())), 0)
poseDetector.process(inputImage)
.addOnSuccessListener { pose ->
if (parentLayout.childCount > 4) parentLayout.removeViewAt(4)
imgView.setImageBitmap(bm)
if (pose.allPoseLandmarks.isNotEmpty()) {
val draw = Draw(applicationContext, pose)
parentLayout.addView(draw)
var content : String = ""
pose.allPoseLandmarks.forEach {
content = content + (it.position3D.x) + (",") + (it.position3D.y) + (",") + (it.position3D.z) + (",") + (it.inFrameLikelihood) + (",")
}
println(content)
content += "\n"
fileOut.appendText(content)
Log.d("LOG:", "Success")
}
}
.addOnFailureListener { Log.d("LOG:", "Failure") }
} else Log.d("LOG:", "null")

The app may be crashing because the variable mTextureView is null. The bitmap property is called on it, but it is not initialized.
It is possible that mTextureView is not being initialized before it is used in the code, or it is not being correctly passed to this function. You should check if mTextureView is initialized or not, if not, you should initialize it before using it in the code.
Another possible reason is that the variable 'out' is not initialized, it might be causing the crash.
It's also possible that there is an issue with the ByteArrayInputStream or BitmapFactory.decodeStream calls, which could be causing a crash if the input data is invalid or in an unexpected format.
It would be helpful to check the logcat for the exact error message when the app crashes. It would give more information about the cause of the crash.
It would be best to check the initialization of the variables and make sure that they are not null before using them in the code.

Related

Using require() later in code and should one handle any exceptions thrown thereby

I have a kotlin class with a method
loadElements(e: Iterable<Int>) {
}
This then constructs a new copy of that Iterable as an ArrayList<Int> within the object.
It is a requirement that all the elements in that ArrayList<Int> be non-negative. It is considered a breach of contract by the caller if that is not met. I've been led to believe that "breach of contract" is something to be tested by require(), whereas check() is for testing logic internal to that method. Is this correct ?
All the examples I have seen, have the require() as the very first lines of code within the method. Is it, however, acceptable to run require() in a loop, like this ?
public fun loadElements(e: Iterable<Int>) {
elementArray.clear()
e.forEach {
require(it>=0)
elementArray.add(it)
moduleCount += it
}
if (elementCount %2 == 1)
elementArray.add(0)
check(elementCount %2 == 0)
computeInternalSizes()
}
Thing is, this means that part of the object's internals may already be set-up by the time the require() breach is detected: i.e., moduleCount will be wrong and computeInternalSizes() will never get called.
Now, of course I could just use a separate pass, with the first one checking for the require() condition, and then doing all the real work thereafter. This would mean that if the input came in as a Sequence<Int>, it would be forced to be terminal and multi-iterable.
If the require() throws, I would like to assume that the program cannot continue because a design error has occurred somewhere. However, if someone traps the resultant exception, and continues, I will end-up with an incoherent object state.
What is best practice for handling conditions where incoming parameter breaches won't be noticed until some significant unrewindable work has been done ?
I tried using a separate pass for checking for non-negativity. This worked perfectly well but, given that it could be coming from a Sequence or similar, I don't want to have to build the whole sequence, and then traverse that sequence again.
I tried using check(). This works, but it just shows up as an inconsistency in object state, rather than flagging up the incoming parameter validation, which is making a breach of contract look like an internal design fault, and just delaying the inevitable.
I've tried putting try/catch/finally all over the place, but this is an excessive amount of code for such a simple thing.
I'm not even sure if a program should attempt recovery if a require() fails.
In general you avoid situations like this, by reducing the scope of mutability in your code.
The difference between require and check is mostly a convention. They throw different Exceptions, namely IllegalArgumentException and IllegalStateException respectively. As the type of the Exceptions suggest, former is suited for validating the (user) input to a method whereas the latter is designed to check intermediate states during the runtime.
Exceptions in Kotlin should be handled as such, being an Exception that should not occur regularly. See also the Kotlin documentation why there are no checked exceptions in Kotlin.
You did not write the name of your surrounding Kotlin class, thus I'll call it Foo for the time being.
Rather than providing a function on Foo, that mutates the internal state of Foo, you could create new instances of Foo based on the Iterable<Int> / Sequence<Int>. This way, you only ever have an Foo object when its in a valid state.
private class Foo(source: Iterable<Int>) {
private val elementArray = ArrayList<Int>()
private val moduleCount: Int
init {
var internalCount = 0
for (count in source) {
require(count > 0)
elementArray.add(count)
internalCount += count
}
moduleCount = internalCount
if (elementArray.size % 2 == 1) {
elementArray.add(0)
}
check(elementArray.size % 2 == 0)
// ...
}
}
Alternatively, if you want / need to keep the interface as described in your question but also avoid the invalid state, you could make use of an intermediate copy.
As you're copying the incoming Iterable<Int> / Sequence<Int> into an ArrayList<Int> I assume you're not working with very large collections.
private class Foo(source: Iterable<Int>) {
private val elementArray = ArrayList<Int>()
private var moduleCount = 0
public fun loadElements(source: Iterable<Int>) {
val internalCopy = ArrayList<Int>()
for (count in source) {
require(count >= 0)
internalCopy.add(count)
}
elementArray.clear()
for (count in internalCopy) {
elementArray.add(count)
moduleCount += count
}
if (elementArray.size % 2 == 1) {
elementArray.add(0)
}
check(elementArray.size % 2 == 0)
// ...
}
}

Smart cast to 'Bitmap!' is impossible, because 'bitmap1' is a mutable property that could have been changed by this time

bitmap1 = Bitmap.createScaledBitmap(
bitmap1, // <---- error is here
(width.toInt()),
(height.toInt()),
false)
numberOfInvaders ++
I also used bitmap2 and bitmap 1 in another class :
if (uhOrOh) {
canvas.drawBitmap(Invader.bitmap1, // <--- error is here
invader.position.left,
invader.position.top,
paint)
} else {
canvas.drawBitmap(Invader.bitmap2, // <---- and here
invader.position.left,
invader.position.top,
paint)
}
here its says : Type mismatch,
Required:Bitmap Found: Bitmap?
Yup, that's true :) You cannot use value like this, because it can be null at some point.
createScaledBitmap requires nonnullable Bitmap, but there is no guarantee that bitmap you use won't be null at the moment of calling given function.
So, what you can do?
Before the call check if bitmap is not null:
if (bitmap != null) { /* code here, still requires !! operator */ }
In multithreaded environment there is a risk that during execution of code block a value will change anyway, so you can use let function with ?. operator (basically the same operator like ., but executes only if value is not null). The block code will be invoked with an effectively final argument which is an instance you use to call this method, in this case "bitmap", called "context object", accessible via it keyword:
bitmap?.let { /* code here, bitmap is passed as effectively final, so for sure it's not null */ }
There other way would be !! operator (but it can finish with NPE exception, if value is null). Use only if you are sure that this value at that moment won't be null, otherwise you can crash your application.
Also, you can use ?: operator - this will take first value if not null, otherwise the second. It's quite nice, because you can use for example default value. Also, you can throw exception there ;)
bitmap ?: throw IllegalStateException("bitmap is null") // exception
bitmap ?: DEFAULT_BITMAP // default bitmap, if any
In this case you will get exception but with very communicative message (instead of just NPE).
bitmap1 = Bitmap.createScaledBitmap(
bitmap1!!, // !! <--- helps
(width.toInt()),
(height.toInt()),
false)
numberOfInvaders ++
if (uhOrOh) {
canvas.drawBitmap(Invader.bitmap1!!, // here
invader.position.left,
invader.position.top,
paint)
} else {
canvas.drawBitmap(Invader.bitmap2!!, // and here too
invader.position.left,
invader.position.top,
paint)
}

Having Trouble Getting the UIDocumentBrowserController to open docs in a Document based app

I've been working on a new Document-based app, and was super glad about the new UIDocumentBrowserController...trying to roll my own solution for the document browser UI was tricky!
I'm having some trouble getting the browser to open documents after they've been created.
What happens now is that when I choose to create a new document in the document browser, the document is created and opened as expected, although an error message is logged. However, after the doc is closed, I cannot reopen the file, either immediately or upon subsequent launches, even though the document is displayed. However, a weird clue here is that if I stop running the app after creating the document, but without adding new information to it (triggering the save cycle), and run the project again, I can open the file correctly. Whuch makes me think that there's something in the way the files are being saved that is the issue.
(Note: At this phase, I'm working on getting the local, non/icloud implentation working, before I move on to the icloud implementation.)
Here is the error message at any point in the code whenthe document is saved to disk (or at least most of the time!):
2017-06-20 13:21:58.254938-0500 Sermon Design 2 iOS[22454:5000138] [default] [ERROR] Could not get attribute values for item file:///Users/stevenhovater/Library/Developer/CoreSimulator/Devices/9A4364F2-B3A1-4AD9-B680-FB4BC876C707/data/Containers/Data/Application/DD534ED8-C4A3-40FE-9777-AED961976878/Documents/Untitled-9.sermon. Error: Error Domain=NSFileProviderInternalErrorDomain Code=1 "The reader is not permitted to access the URL." UserInfo={NSLocalizedDescription=The reader is not permitted to access the URL.}
I suspect that the issue lies somewher in my document types plists, which I've tried to set up by imitating the setup in the video for wwdc 2017 session 229.
My docs are encapuslated by an NSData object, using what I take to be a pretty standard subclass implentation of UIDocument. (I'm omitting the code to generate the thumbnails)
override func contents(forType typeName: String) throws -> Any {
print("Saving Document Changes")
if sermon != nil {
let newData = NSKeyedArchiver.archivedData(withRootObject: sermon!)
return newData
} else {
let newData = NSKeyedArchiver.archivedData(withRootObject: Sermon())
return newData
}
}
override func fileAttributesToWrite(to url: URL, for saveOperation: UIDocumentSaveOperation) throws -> [AnyHashable : Any] {
let thumbnail:UIImage = self.createThumbnail()
let thumbnaildict = [URLThumbnailDictionaryItem.NSThumbnail1024x1024SizeKey : thumbnail]
let dict = [URLResourceKey.thumbnailDictionaryKey:thumbnaildict]
return dict
}
override func load(fromContents contents: Any, ofType typeName: String?) throws {
guard let newSermon:Sermon = NSKeyedUnarchiver.unarchiveObject(with: contents as! Data) as? Sermon else{
throw documentErrors.invalidFile
}
self.sermon = newSermon
}
In my subclass of UIDocumentBrowserViewController, Here is my code for getting a local filename and for creating the new document.
func documentBrowser(_ controller: UIDocumentBrowserViewController, didRequestDocumentCreationWithHandler importHandler: #escaping (URL?, UIDocumentBrowserViewController.ImportMode) -> Void) {
var newDocumentURL: URL? = nil
print("creating new local document")
guard let target = self.newLocalFilename() else {
return
}
let targetSuffix = target.lastPathComponent
let tempURL = URL(fileURLWithPath: NSTemporaryDirectory() + targetSuffix)
let newDocument:SDDocument = SDDocument(fileURL: tempURL)
newDocument.sermon = Sermon()
/
newDocument.save(to: tempURL, for: .forCreating) { (saveSuccess) in
/
guard saveSuccess else {
/
importHandler(nil, .none)
return
}
/
newDocument.close(completionHandler: { (closeSuccess) in
/
guard closeSuccess else {
/
importHandler(nil, .none)
return
}
/
importHandler(tempURL, .move)
})
}
}
func newLocalFilename() -> URL? {
let fileManager = FileManager()
guard let baseURL = self.localDocumentsDirectoryURL.appendingPathComponent("Untitled")
else {return nil}
var target = baseURL.appendingPathExtension(DocumentBrowserViewController.documentExtension)
var nameSuffix = 2
while fileManager.fileExists(atPath: target.path) {
target = URL(fileURLWithPath: baseURL.path + "-\(nameSuffix).\(DocumentBrowserViewController.documentExtension)")
nameSuffix += 1
}
let targetSuffix = target.lastPathComponent
print("Target name: \(targetSuffix)")
print("new url: \(target)")
return target
}
After four or five hours of work banging my head against this problem, I discovered a simple solution: don't test in the Simulator. I switched to testing on my device and instantly everything started working as advertised.
[I can't speak from experience here, but it may be that the "doesn't work in the Simulator" problem is confined to Sierra, but that the Simulator does work in High Sierra. This would explain why some users see this issue and others don't, and especially why Apple seems blissfully unaware of it in the WWDC video.]
I had exactly the same issue when I was trying to save to NSTemporaryDirectory().
If you instead save to the documents directory ([[[NSFileManager defaultManager] URLsForDirectory:NSDocumentDirectory inDomains:NSUserDomainMask] firstObject]), it appears to work fine!
Update: it looks like this issue is fixed in iOS 11 beta 3, and you can now save newly created documents to NSTemporaryDirectory() correctly.
Here is my current theory.
This error
Error Domain=NSFileProviderInternalErrorDomain Code=1 "The reader is not permitted to access the URL."
shows up when first creating a UIDocument at a new URL using -initWithFileURL. It's basically saying "this URL doesn't exist yet," but in a way that makes it sound more like a permissions issue.
As far as I can tell, it doesn't prevent you from saving, opening, editing, or closing the file. So I think it's just a superfluous error that Apple should have taken out.
I found that the error happens on simulator when LSSupportsOpeningDocumentsInPlace property is set to YES in the info.plist.
Turn this property to NO, then it starts working, in my case.
On the real device, it works anyway without having error.
Had similar problem, and tried the method for providing a default Core Location in the Schemes settings and it works now. The method was mentioned in this answer: IOS 9 Error Domain=kCLErrorDomain Code=0 "(null)"

How do I properly handle these exceptions?

In the following method, I need to handle if a null gets passed in but I am unsure how. How do I handle this so this method will pass a JUnit test?
public Album(String name) {
if(name==null){
// what do I do here?
}
this.name = name;
this.images = new ArrayList<Image>();
}
And in this method, how do I handle if the index passed in is either negative or greater than the size of the array list it is getting from?
public Image getImage(int index) {
if (index < 0 || index > images.size()) {
// how to handle here?
}
return images.get(index);
}
I've tried various things for both, but these methods keep failing their JUnit tests. Thank you for any input you can give me.
if(name==null){
throw new IllegalArgumentException("name");
}
although personally I don't see anything wrong with throwing NullPointerException directly.
if (index < 0 || index > images.size()) {
throw new ArrayIndexOutOfBounds(index);
}
But you know what? images.get(index) will throw it as well, so don't bother checking it first. Less code to read and test. In fact you already made a mistake. Your precondition will let index equal to images.size(). There should have been index >= images.size().

Grails 1.3.6 - Object fields seem to lose their value

I am a newbie to Grails and using an ancient version of Grails (1.3.6/8). I have an object with some variables that aren't holding on to the stuff I assign them.
class NiftyController {
try
{
SomeGrid someGrid = new SomeGrid()
def selectedDate = params.specifiedDate
...
someGrid.selectedDate = selectedDate
someGrid.longDate = Calendar.getInstance().getTimeInMillis()
println someGrid.selectedDate // prints, say, 08/06/2012
println someGrid.longDate // prints, say, 1302558890256
....
doSomeWork(someGrid)
}
def doSomeWork = { SomeGrid someGrid ->
println someGrid.selectedDate // prints '' (empty)
println someGrid.longDate // prints 8 - the number for the current month.
}
}
I am totally scratching my head - I work with Java and this type of loss of assigned variable data is new and makes no sense to me at all. Can anyone figure out what is going on here? I would guess it is some kind of visibility problem unique to Grails, but I can't nail it.
Any help is greatly appreciated!
If doSomeWork doesn't itself need to be a controller action, I would make it a normal method instead of a closure:
private doSomeWork(SomeGrid someGrid) {
//...
}
With it as a closure, Grails may be treating the SomeGrid parameter as a command object and trying to data-bind it directly from the params.