I want to read csv file to List of strings. I also want it to trim whitespaces and allow trailing comma. My csv file looks like this:
abcdefghijklmno ,
pqrstuwzabcdefh,
ijklmnopqrstuwyz,
zabcdefghijklmopqr,
And when I try to parse it using my method:
fun parseCsv(inputStream: InputStream): List<String> {
val mapper = CsvMapper().enable(CsvParser.Feature.TRIM_SPACES).enable(CsvParser.Feature.ALLOW_TRAILING_COMMA)
val schema = CsvSchema
.emptySchema()
.withLineSeparator("\n")
.withColumnSeparator(',')
.withQuoteChar('"')
return try {
mapper
.readerFor(String::class.java)
.with(schema)
.readValues<String>(inputStream)
.readAll()
} catch (e: IOException ) {
logger.error("Error parsing file", e)
emptyList()
} catch (e: NoSuchElementException) {
throw FileParsingError(e)
}
}
I get:
expected == actual
| | |
| | [abcdefghijklmno, ]
| false
[abcdefghijklmno, pqrstuwzabcdefh, ijklmnopqrstuwyz, zabcdefghijklmopqr]
How to fix it ?
This worked for me:
return mapper.readerForListOf(Map.class)
.with(schema)
.with(CsvParser.Feature.WRAP_AS_ARRAY) // <-- This.
.readTree(content);
Related
I am using import java.net.http.HttpClient
and my code is as follows:
try {
val response = httpClient.send(httpRequest, BodyHandlers.ofString())
...
when (response.statusCode()) {
200 -> {
result = decodedResponse
} else -> {
val errorResponse = Json.decodeFromString<ErrorObject>(response.body())
throw handleCustomError(errorResponse.error, errorResponse.error_description)
}
}
return result
} catch (ex: Exception) {
throw Exception("Service is unavailable!")
}
my handleCustomError iterates over the different types of status codes 401,403,404 etc and throws user friendly exception. But I dont think that code is ever reached. Instead I see the generic exception thrown by the catch block.
How can I make sure to iterate over the different status code?
I am using CameraX (Beta) and using Analyzer with an executor to analyze the image and draw bounds/overlay on it. Based on the documentation, I need to close the image proxy to continue to get the images into the analyze function. When I add the imageProxy.close() call, it fails with Image Already closed error. What am I missing here?
Analyzer Code:
private val successListener = OnSuccessListener<List<DetectedObject>> { papers ->
isAnalyzing.set(false)
val rectPoints= mutableListOf<Rect>()
Log.d(TAG," overlayRef Info: ${overlayRef.get()}")
for (paper in papers) {
val bounds = paper.boundingBox
val paperId = paper.trackingId
rectPoints+=bounds
Log.d(TAG, "Successful Paper Analysis - Bounds of the paper: $bounds")
Log.d(TAG," Labels found on the paper: ${paper.labels}")
}
Log.d(TAG, "Invoking pointsRectListener for : $rectPoints")
pointsRectListener?.invoke(rectPoints)
}
private val failureListener = OnFailureListener { e ->
isAnalyzing.set(false)
Log.e(TAG, "Paper analysis failure.", e)
}
#SuppressLint("UnsafeExperimentalUsageError")
override fun analyze(imageProxy: ImageProxy) {
val mediaImage = imageProxy?.image ?: return
Log.d(TAG,"entered analysis..analysis in progress?$isAnalyzing.get()")
if (!isAnalyzing.get()){
isAnalyzing.set(true)
Log.d(TAG,"No other analysis in progress..so starting analysis now")
val currentTimestamp = System.currentTimeMillis()
if (currentTimestamp - lastAnalyzedTimestamp >= TimeUnit.SECONDS.toMillis(1)) {
currentTimestamp - lastAnalyzedTimestamp
analysisSizeListener?.invoke(Size(imageProxy.width, imageProxy.height))
val image = InputImage.fromMediaImage(mediaImage, imageProxy.imageInfo.rotationDegrees)
objectDetector.process(image)
.addOnSuccessListener(successListener)
.addOnFailureListener(failureListener)
}
}
imageProxy.close()
}
Code where I am instantiating and binding to lifecycle
paperAnalyzer=ImageAnalysis.Builder()
.setTargetAspectRatio(screenAspectRatio)
.setBackpressureStrategy(ImageAnalysis.STRATEGY_KEEP_ONLY_LATEST)
.setTargetRotation(rotation)
.build()
.also {
it.setAnalyzer(cameraExecutor, PaperAnalyzer( WeakReference(overlayView)).apply {
pointsRectListener = { rectPoints ->
overlayView.rectPoints = rectPoints
}
analysisSizeListener = {
updateOverlayTransform(overlayView, it)
}
}
)
}
cameraProvider.unbindAll()
try {
camera = cameraProvider.bindToLifecycle(
this, cameraSelector, preview, imageCapture, paperAnalyzer)
// Attach the viewfinder's surface provider to preview use case
preview?.setSurfaceProvider(viewFinder.createSurfaceProvider())
} catch (exc: Exception) {
Log.e(TAG, "Use case binding failed", exc)
}
It seems like the logic you're trying to accomplish looks like this.
fun analyze(imageProxy) {
if (isAnalyzing) {
imageProxy.close() // 1
} else {
val image = imageInput.fromMediaImage(...)
objectDetector.process(image)
.addOnSuccessListener(OnSuccessListener { result ->
// Do something with the result
imageProxy.close() // 2
isAnalyzing.set(false)
})
.addOnFailureListener(OnFailureListener { exception ->
// Do something with the exception
imageProxy.close() // 3
isAnalyzing.set(false)
})
}
}
I might be wrong, but it looks like you're doing 1, but not 2 and 3. Could you update your code to follow this pattern and see if you're still encountering the issue.
I'm new in kotlin world. So I have some problem. I'm using ktor framework and try to use ktor-locations (https://ktor.io/servers/features/locations.html#route-classes)
And as example
#Location("/show/{id}")
data class Show(val id: Int)
routing {
get<Show> { show ->
call.respondText(show.id)
}
}
Everything is good, when I try to get /show/1
But if route will be /show/test there is NumberFormatException, cause DefaultConversionService try to convert id to Int and can't do it.
So my question is, how can I catch this exception and return Json with some error data. For example, if not using locations I can do smt like this
routing {
get("/{id}") {
val id = call.parameters["id"]!!.toIntOrNull()
call.respond(when (id) {
null -> JsonResponse.failure(HttpStatusCode.BadRequest.value, "wrong id parameter")
else -> JsonResponse.success(id)
})
}
}
Thx for help!
You can do a simple try-catch in order to catch the parsing exception which is thrown when a string can not be converted to an integer.
routing {
get("/{id}") {
val id = try {
call.parameters["id"]?.toInt()
} catch (e : NumberFormatException) {
null
}
call.respond(when (id) {
null -> HttpStatusCode.BadRequest
else -> "The value of the id is $id"
})
}
}
Other way of handling exception is to use StatusPages module:
install(StatusPages) {
// catch NumberFormatException and send back HTTP code 400
exception<NumberFormatException> { cause ->
call.respond(HttpStatusCode.BadRequest)
}
}
This should work with using Location feature. Please note that Location is experimental above ktor version 1.0.
I have a large stream of text coming back from REST web service and I would like to write it directly to file. What is the simplest way of doing this?
I have written the following function extension that WORKS. But I can't help thinking that there is a cleaner way of doing this.
Note: I was hoping to use try with resources to auto close the stream and file
fun File.copyInputStreamToFile(inputStream: InputStream) {
val buffer = ByteArray(1024)
inputStream.use { input ->
this.outputStream().use { fileOut ->
while (true) {
val length = input.read(buffer)
if (length <= 0)
break
fileOut.write(buffer, 0, length)
}
fileOut.flush()
}
}
}
You can simplify your function by using the copyTo function:
fun File.copyInputStreamToFile(inputStream: InputStream) {
this.outputStream().use { fileOut ->
inputStream.copyTo(fileOut)
}
}
My proposition is:
fun InputStream.toFile(path: String) {
File(path).outputStream().use { this.copyTo(it) }
}
without closing current stream
InputStream.toFile("/path/filename")
also, do not forget to handle exceptions, for example if write permission is denied :)
I suggest to make like this:
fun InputStream.toFile(path: String) {
use { input ->
File(path).outputStream().use { input.copyTo(it) }
}
}
and then to use like:
InputStream.toFile("/some_path_to_file")
You needs to do like this
#Throws
fun copyDataBase() {
var myInput = context.getAssets().open(DB_NAME)
var outFileName = DB_PATH + DB_NAME
var fileOut: OutputStream = FileOutputStream(outFileName)
val buffer: ByteArray = ByteArray(1024)
var length: Int? = 0
while (true) {
length = myInput.read(buffer)
if (length <= 0)
break
fileOut.write(buffer, 0, length)
}
fileOut.flush()
fileOut.close()
myInput.close()
throw IOException()
}
What appears to have worked for me is this:
fun fileCopyer(localFileA: File, localFileB: File) {
var output = localFileA.inputStream()
output.copyTo(localFileB.outputStream())
output.close()
}
Trying to parse xml and I'm having trouble with my UDF returning a tuple. Following the example from http://verboselogging.com/2010/03/31/writing-user-defined-functions-for-pig
pig script
titles = FOREACH programs GENERATE (px.pig.udf.PARSE_KEYWORDS(program))
AS (root_id:chararray, keyword:chararray);
here is the output schema code:
override def outputSchema(input: Schema): Schema = {
try {
val s: Schema = new Schema
s.add(new Schema.FieldSchema("root_id", DataType.CHARARRAY))
s.add(new Schema.FieldSchema("keyword", DataType.CHARARRAY))
return s
}
catch {
case e: Exception => {
return null
}
}
}
I'm getting this error
pig script failed to validate: org.apache.pig.impl.logicalLayer.FrontendException:
ERROR 0: Given UDF returns an improper Schema.
Schema should only contain one field of a Tuple, Bag, or a single type.
Returns: {root_id: chararray,keyword: chararray}
Update Final Solution:
In java
public Schema outputSchema(Schema input) {
try {
Schema tupleSchema = new Schema();
tupleSchema.add(input.getField(1));
tupleSchema.add(input.getField(0));
return new Schema(new Schema.FieldSchema(getSchemaName(this.getClass().getName().toLowerCase(), input),tupleSchema, DataType.TUPLE));
} catch (Exception e) {
return null;
}
}
You will need to add your s schema instance variable to another Schema object.
Try returning a new Schema(new FieldSchema(..., input), s, DataType.TUPLE)); like in the template below:
Here is my answer in Java (fill out your variable names):
#Override
public Schema outputSchema(Schema input) {
Schema tupleSchema = new Schema();
try {
tupleSchema.add(new FieldSchema("root_id", DataType.CHARARRAY));
tupleSchema.add(new FieldSchema("keyword", DataType.CHARARRAY));
return new Schema(new FieldSchema(getSchemaName(this.getClass().getName().toLowerCase(), input), tupleSchema, DataType.TUPLE));
} catch (FrontendException e) {
e.printStackTrace();
return null;
}
}
Would you try:
titles = FOREACH programs GENERATE (px.pig.udf.PARSE_KEYWORDS(program));
If that doesn't error, then try:
titles = FOREACH TITLES GENERATE
$0 AS root_id
,$1 AS keyword
;
And tell me the error?