I have a large stream of text coming back from REST web service and I would like to write it directly to file. What is the simplest way of doing this?
I have written the following function extension that WORKS. But I can't help thinking that there is a cleaner way of doing this.
Note: I was hoping to use try with resources to auto close the stream and file
fun File.copyInputStreamToFile(inputStream: InputStream) {
val buffer = ByteArray(1024)
inputStream.use { input ->
this.outputStream().use { fileOut ->
while (true) {
val length = input.read(buffer)
if (length <= 0)
break
fileOut.write(buffer, 0, length)
}
fileOut.flush()
}
}
}
You can simplify your function by using the copyTo function:
fun File.copyInputStreamToFile(inputStream: InputStream) {
this.outputStream().use { fileOut ->
inputStream.copyTo(fileOut)
}
}
My proposition is:
fun InputStream.toFile(path: String) {
File(path).outputStream().use { this.copyTo(it) }
}
without closing current stream
InputStream.toFile("/path/filename")
also, do not forget to handle exceptions, for example if write permission is denied :)
I suggest to make like this:
fun InputStream.toFile(path: String) {
use { input ->
File(path).outputStream().use { input.copyTo(it) }
}
}
and then to use like:
InputStream.toFile("/some_path_to_file")
You needs to do like this
#Throws
fun copyDataBase() {
var myInput = context.getAssets().open(DB_NAME)
var outFileName = DB_PATH + DB_NAME
var fileOut: OutputStream = FileOutputStream(outFileName)
val buffer: ByteArray = ByteArray(1024)
var length: Int? = 0
while (true) {
length = myInput.read(buffer)
if (length <= 0)
break
fileOut.write(buffer, 0, length)
}
fileOut.flush()
fileOut.close()
myInput.close()
throw IOException()
}
What appears to have worked for me is this:
fun fileCopyer(localFileA: File, localFileB: File) {
var output = localFileA.inputStream()
output.copyTo(localFileB.outputStream())
output.close()
}
Related
I have a CLI with multiple sub-commands, some of the sub-commands have an optional flag -f with which an input file can be specified, e.g.
#CommandLine.Command(name = "get", description = ["Get something"])
class GetUserCommand: Runnable {
#Option(names = ["-f", "--file"], description = ["Input file"])
var filename: String? = null
override fun run() {
var content = read_file(filename)
}
}
#CommandLine.Command(name = "query", description = ["Query something"])
class QueryUserCommand: Runnable {
#Option(names = ["-f", "--file"], description = ["Input file"])
var filename: String? = null
override fun run() {
var content = read_file(filename)
}
}
The input file format can be different from command to command. Ideally, I'd like to parse the file automatically if it was specified as an argument.
Also the file content can be different on each command (but will be a specific format, CSV or JSON).
For example I'd like to have something like this
data class First(val col1, val col2)
data class Second(val col1, val col2, val col3)
class CustomOption(// regular #Option parameters, targetClass=...) {
// do generic file parsing
}
#CommandLine.Command(name = "get", description = ["Get something"])
class GetUserCommand: Runnable {
#CustomOption(names = ["-f", "--file"], description = ["Input file"], targetClass=First))
var content: List<First> = emptyList()
override fun run() {
// content now contains the parse file
}
}
#CommandLine.Command(name = "query", description = ["Query something"])
class QueryUserCommand: Runnable {
#CustomOption(names = ["-f", "--file"], description = ["Input file"], targetClass=Second))
var content: List<Second> = emptyList()
override fun run() {
// content now contains the parse file
}
}
Would anyone have an idea if this is possible or how to do it?
To rephrase the question: how to do additional processing of input parameters during the parsing process rather than during the command execution?
(Note that the OP did not specify why this is desirable. I assume the goal is either to leverage picocli's error reporting, or to encapsulate the parsing logic somewhere for easier testing and reuse. The OP may want to expand on the underlying goal if the solution below is not satisfactory.)
One idea is to use picocli's custom parameter processing.
It is possible to specify a IParameterConsumer for an option that will process the parameter for that option.
So, for example when the user specifies get -f somefile, the custom parameter consumer will be responsible for processing the somefile argument. An implementation can look something like this:
// java implementation, sorry I am not that fluent in Kotlin...
class FirstConsumer implements IParameterConsumer {
public void consumeParameters(Stack<String> args,
ArgSpec argSpec,
CommandSpec commandSpec) {
if (args.isEmpty()) {
throw new ParameterException(commandSpec.commandLine(),
"Missing required filename for option " +
((OptionSpec) argSpec).longestName());
}
String arg = args.pop();
First first = parseFile(new File(arg), commandSpec);
List<String> list = argSpec.getValue();
list.add(first);
}
private First parseFile(File file,
ArgSpec argSpec,
CommandSpec commandSpec) {
if (!file.isReadable()) {
throw new ParameterException(commandSpec.commandLine(),
"Cannot find or read file " + file + " for option " +
((OptionSpec) argSpec).longestName());
}
// other validation...
// parse file contents...
// finally, return the result...
return new First(...);
}
}
Once the parameter consumer classes are defined, you can use them as follows:
#Command(name = "get", description = ["Get something"])
class GetUserCommand: Runnable {
#Option(names = ["-f", "--file"], description = ["Input file"],
parameterConsumer = FirstConsumer::class))
var content: List<First> = emptyList()
override fun run() {
// content now contains the parsed file
}
}
I'm nearly done creating an app that sends txt files, containing scanned data, to an ftp server.
The issue that I'm currently struggling with is: what if my Wi-Fi has terrible signal or no signal at all.
I noticed that my 'isOnline()' check works fine and that if there is no internet, it alerts the user. However a few hours ago I tested the app in the basement and noticed that when the Wi-Fi signal has no bars, it still sends the data but it gets lost somewhere along the way.
Currently the flow of the data is as follow:
user presses 'send'
check internet and if true, continue
clear content list and send
the content to viewmodel
viewmodel checks internet again before
creating the txt files
txt files get sent via FTP code below.
private fun sendTXT(result: String) {
try {
val name = "00_VER${LocalDateTime.now().format(fileNameFormatter)}.txt"
val path = getApplication<Application>().applicationContext.filesDir.path
.toString() + name
val f = File(path)
val isNewFileCreated: Boolean = f.createNewFile()
if (isNewFileCreated) {
f.writeText(result, Charsets.UTF_8)
}
val ftpClient = FTPClient()
ftpClient.addProtocolCommandListener(PrintCommandListener(PrintWriter(System.out)))
ftpClient.connect("xxx.xx.xxx.xx", 21)
val reply: Int = ftpClient.replyCode
if (!FTPReply.isPositiveCompletion(reply)) {
ftpClient.disconnect()
throw IOException("Exception in connecting to FTP Server")
}
if (ftpClient.login("username", "pass")) {
ftpClient.enterLocalPassiveMode()
ftpClient.setFileType(FTP.BINARY_FILE_TYPE)
val inp = FileInputStream(f)
var directory = "/files/input"
ftpClient.changeWorkingDirectory(directory)
val result = ftpClient.storeFile(name, inp)
inp.close()
if (result) {
ftpClient.logout()
ftpClient.disconnect()
f.delete()
}
}
} catch (e: Exception) {
e.printStackTrace()
}
}
fun isOnline(context: Context): Boolean {
var result = false
val connectivityManager =
context.getSystemService(Context.CONNECTIVITY_SERVICE) as ConnectivityManager
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.M) {
val networkCapabilities = connectivityManager.activeNetwork ?: return false
val actNw =
connectivityManager.getNetworkCapabilities(networkCapabilities) ?: return false
result = when {
actNw.hasTransport(NetworkCapabilities.TRANSPORT_WIFI) -> true
actNw.hasTransport(NetworkCapabilities.TRANSPORT_CELLULAR) -> true
actNw.hasTransport(NetworkCapabilities.TRANSPORT_ETHERNET) -> true
else -> false
}
} else {
connectivityManager.run {
connectivityManager.activeNetworkInfo?.run {
result = when (type) {
ConnectivityManager.TYPE_WIFI -> true
ConnectivityManager.TYPE_MOBILE -> true
ConnectivityManager.TYPE_ETHERNET -> true
else -> false
}
}
}
}
return result
}
I'm stuck at finding a way to make sure the data gets to the server. Is there a more advanced way to check for internet connectivity?
I was considering adding all the scan objects as JSON to sharedpreferences, and if at the end of the day the user notices a scan didn't make it through, they can look up the missing scan and resend it.
However this seems very unconventional and I'm pretty sure there must be a better way to handle things.
I'm using Chronicle Queue as a DataStore which will be written to once but read many many times. I'm trying to get the best performance (time to read x number of records).
My data set (for my test) is about 3 million records , where each record consists of a bunch of longs and doubles. I initially started with "Highest-level" API which was obviously slow , then self-describing" data as mentioned in this Chronicle Documentation and finally using "raw data" which gave the best performance.
Code as below:(Corresponding write() code is omitted for brevity)
public List<DomainObject> read()
{
final ExcerptTailer tailer = _cq.createTailer();
List<DomainObject> result = new ArrayList<>();
for (; ; ) {
try (final DocumentContext ctx = tailer.readingDocument()) {
Wire wire = ctx.wire();
if(wire != null) {
wire.readBytes(in -> {
final long var1= in.readLong();
final int var2= in.readInt();
final double var3= in.readDouble();
final int var4= in.readInt();
final double var5= in.readDouble();
final int var6= in.readInt();
final double var7= in.readDouble();
result.add(DomainObject.create(var1, var2, var3, var4, var5, var6, var7);
});
}else{
return result;
}
}
}
}
However to improve my Application performance ,I started using ByteBuffer instead of a "DomainObject" and thus modified by read method as below:
public List<ByteBuffer> read()
{
final ExcerptTailer tailer = _cq.createTailer();
List<ByteBuffer> result = new ArrayList<>();
for (; ; ) {
try (final DocumentContext ctx = tailer.readingDocument()) {
Wire wire = ctx.wire();
if(wire != null) {
ByteBuffer bb = ByteBuffer.allocate(56);
wire.readBytes(in -> {
in.read(bb); });
result.add(bb);
}else{
return result;
}
}
}
}
Above code listing took an average of 550 ms vs 270ms for the first listing.
I also tried using Bytes.elasticByteBuffer as mentioned in this post but it was way slower
I'm guessing the second code listing is slower because it has to loop through the entire byte array.
So my question is - Is there a more performant way to read bytes from Chronicle Queue into a ByteBuffer? My data will always be 56 bytes with 8 bytes for each data item.
I suggest you use Chronicle-Bytes instead of raw ByteBuffer. Chronicle's Bytes class is a wrapper on top of ByteBuffer but much easier to use.
The problem with your code is you create a bunch of objects instead of stream-processing. I suggest you read with something like:
public void read(Consumer<Bytes> consumer) {
final ExcerptTailer tailer = _cq.createTailer();
for (; ; ) {
try (final DocumentContext ctx = tailer.readingDocument()) {
if (ctx.isPresent()) {
consumer.accept(ctx.wire().bytes());
} else {
break;
}
}
}
}
And your writing method could look like:
public void write(BytesMarshallable o) {
try (DocumentContext dc = _cq.acquireAppender().writingDocument()) {
o.writeMarshallable(dc.wire().bytes());
}
}
And then your consumer could be like:
private BytesMarshallable reusable = new BusinessObject(); //your class here
public accept(Bytes b) {
reusable.readMarshallable(b);
// your business logic here
doSomething(reusable);
}
this is MainActivity.kt before
var spannable = SpannableStringBuilder("$noColorText$coloredText")
spannable.setSpan(
ForegroundColorSpan(ContextCompat.getColor(textView.context, R.color.mainGreen)),
noColorText.length, spannable.length,
Spannable.SPAN_EXCLUSIVE_INCLUSIVE
)
spannable.setSpan(
StyleSpan(BOLD),
noColorText.length, spannable.length,
Spannable.SPAN_EXCLUSIVE_INCLUSIVE
)
textView.text = spannable
Here's my approach so far.
Extension.kt
// TODO: e.g: "string".putSpans(start, end, flags) { ForgroundColorSpan(color), StyleSpan(BOLD) }
fun String.putSpans(vararg flags: Int.() -> Unit, spanBuilder: SpannableStringBuilder.() -> Unit):
SpannableStringBuilder = SpannableStringBuilder(this).apply(spanBuilder)
MainActivity.kt
// TODO: Change SpannableBuilder to be modular (without, reinput duplicate args)
val resultSpan = "$noColorText$coloredText ".putSpans {
setSpan(ForegroundColorSpan(ContextCompat.getColor(textView.context, R.color.mainGreen)),
noColorText.length, this.length, // this is duplicate
Spannable.SPAN_EXCLUSIVE_INCLUSIVE) // this is duplicate
setSpan(StyleSpan(BOLD),
noColorText.length, this.length, // this is duplicate
Spannable.SPAN_EXCLUSIVE_INCLUSIVE) // this is duplicate
}
textView.text = resultSpan
Is this possible to create extension like this
"string".putSpans(start, end, flags) { ForgroundColorSpan(color), StyleSpan(BOLD) }
so we don't have to use duplicate start, end, also flags argument, but open for modification, e.g:
"string".putSpans(start, end, flags) { // for default value
span(ForgroundColorSpan(color), diffStart, diffEnd),
span(StyleSpan(BOLD), diffFlags)
}
You can use extensions included in core-ktx which simplify using, more specifically, building SpannedString in kotlin as so:
buildSpannedString {
bold {
append("hitherejoe")
}
}
I guess you would use it like this:
buildSpannedString {
bold {
inSpans(ForegroundColorSpan(ContextCompat.getColor(textView.context, R.color.mainGreen))) {
append("string")
}
}
}
See androidx.text package for reference.
I took the example from this Medium post by Joe Birch.
I have a simple bean, like that:
package models;
import play.data.validation.Constraints;
public class Upload
{
#Constraints.Required
#Constraints.MinLength(4)
#Constraints.MaxLength(40)
public String name;
#Constraints.Required
public String inputFile;
}
and form, like that:
#form(action = routes.Application.submit(), 'enctype -> "multipart/form-data") {
#inputText(
uploadForm("name"),
'_label -> "Name"
)
#inputFile(
uploadForm("inputFile"),
'_label -> "Queries"
)
}
What is the best way to validate inputFile?
Is it possible do to that with annotations?
#Required constraint does not work at all.
I want it to be selected + add some limitation on size.
make your form like:
<input type="file" name="inputFile">
In you submit method add this:
// from official documentation
public static Result submit() {
MultipartFormData body = request().body().asMultipartFormData();
FilePart file = body.getFile("inputFile");
if (inputFile != null) {
String fileName = picture.getFilename();
String contentType = picture.getContentType();
File file = picture.getFile();
// method the check size
if(!validateFileSize){
return redirect(routes.Application.index()); // error in file size
}
return ok("File uploaded");
} else {
// here comes the validation
flash("error", "Missing file");
return redirect(routes.Application.index());
}
}
Something like the following, maybe?
MultipartFormData body = request().body().asMultipartFormData();
if (!body.getFiles().isEmpty()) {
// do your work
}