Ktor server Progress on incoming files - ktor

I have a ktor server running on android and I would like to display a progress bar for files being uploaded to the server via POST.
In my each part handling code I have something like this:
call.receiveMultipart().forEachPart { part ->
when (part) {
is PartData.FileItem -> {
val input = part.provider()
// Read from input by chunks
}
else -> {}
}
}
but I the when part is called after the file is uploaded to the server.
I had a look at (custom) Plugins and Logging to see if I can get any progress update on incoming bytes but no luck.

Related

File Uploading service gets failed from android whereas works with IOS

I had created the WCF service for file uploading. Its working fine when the service hits from web application or from IOS device. But its throwing an exception when it comes from Android device.
I tried to multiparse the streamdata. Its throwing an exception as like file unavailable.
public OASIS.Entity.Shared.UserFileUpload FileUpload(Stream data, string UploadMode)
{
OASIS.Entity.Shared.UserFileUpload userFileUpload = new Entity.Shared.UserFileUpload();
try
{
MultipartParser parser = new MultipartParser(data);
string fileName = string.Empty;
string filePath = string.Empty;
string allowedExtensions = string.Empty;
int allowedFileSizeMB = 0;
if (parser.FileAvailable)
{
// File Available for IOS / Web application.
// userFileUpload
}
else
{
// From android device file is getting not available.
}
}
catch (Exception exp)
{
OASIS.Utility.ExceptionManager.HandleException(exp);
userFileUpload = null;
}
return userFileUpload;
}
Expecting it should get work for android device too.
By default, WCF does not support form data files, so it looks like you are using MultipartParser to convert form data (data from a file stream uploaded through a form-data).
If this class can handle data submitted in IOS, it should also be able to handle data submitted through forms in Andriod, after all, the HTTP protocol is cross-platform.
thereby I would like to know, how do you upload data in the Andriod system?
By adding breakpoint debugging, can you use this class to parse form data properly?
I suggest you handle the form-data by creating the service with asp.net WebAPI.
https://learn.microsoft.com/en-us/aspnet/web-api/overview/advanced/sending-html-form-data-part-2
Feel free to let me know if there is anything I can help with.

Spring Integration - how to delegate error handling to separate thread

My Spring Integration Flow looks like the below,
Read file from directory
Add a header called errorChannel:'exceptionChannel' to the message
Perform business logic
Print output to a 'success directory'
If there is an error on step 3, the exception is sent to the exceptionChannel,and is written to a 'failed directory'.
I want the error flow to be delegated to a separate thread.
What I have:
If there are 5 files and the third file has an error,
-> 2 files are written to the success directory
-> 1 file is written to the failed directory.
-> The flow stops after the error file.
What I need:
If there are 5 files and the third file has an error,
-> First 2 files must be written to success directory
-> 3rd file must be written to failed directory
-> Last 2 files must be written to success directory
Code for Success flow:
#Bean(name="exceptionChannel")
MessageChannel exceptionChannel() {
return MessageChannels.direct()
.get();
}
#Bean
public IntegrationFlow migrateInputToOutput() {
return IntegrationFlows.from(Files.inboundAdapter(new File(INPUT_DIR))),
.enrichHeaders(h -> h.header("errorChannel", "exceptionChannel", true))
.handle(someBusinessLogic) // ANY EXCEPTION HERE IS SENT TO exceptionChannel
.handle(Files.outboundAdapter(new File(OUTPUT_SUCCESS_DIR))
.get();
}
Code for handling error:
#Bean
public IntegrationFlow handleErrorInMigration() {
return IntegrationFlows.from("exceptionChannel"),
.handle(errorLogicToPrintException)
.handle(Files.outboundAdapter(new File(OUTPUT_ERROR_DIR))
.get();
}
The Files.inboundAdapter() is pollable source, so there is somewhere something like Poller configured in your code. That one has a errorChannel() option. It is really would be better to do that way. If your poller is global one and you don't want to modify it, it is better to have a poller configured exactly for this endpoint:
IntegrationFlows.from(Files.inboundAdapter(new File(INPUT_DIR)),
e -> e.poller(p -> p.fixedDelay().errorChannel(exceptionChannel())))
This way you won't effect all other polling endpoints.
To move your error handling to different thread, you need to consider to make that exceptionChannel as an ExecutorChannel:
#Bean(name="exceptionChannel")
MessageChannel exceptionChannel() {
return MessageChannels.executor(myExecutor)
.get();
}

Task Module call from Ms Teams in Bot Framework

I am looking to open a task module (Pop up - iframe with audio/video) in my bot that is connected to Teams channel. I am having issues following the sample code provided on the GitHub page.
I have tried to follow the sample and incorporate to my code by did not succeed.
In my bot.cs file I am creating card action of invoke type:
card.Buttons.Add(new CardAction("invoke", TaskModuleUIConstants.YouTube.ButtonTitle, null,null,null,
new Teams.Samples.TaskModule.Web.Models.BotFrameworkCardValue<string>()
{
Data = TaskModuleUIConstants.YouTube.Id
}));
In my BotController.cs that inherits from Controller
[HttpPost]
public async Task PostAsync()
{
// Delegate the processing of the HTTP POST to the adapter.
// The adapter will invoke the bot.
await _adapter.ProcessAsync(Request, Response, _bot);
}
public async Task<HttpResponseMessage> Post([FromBody] Activity activity)
{
if (activity.Type == ActivityTypes.Invoke)
{
return HandleInvokeMessages(activity);
}
return new HttpResponseMessage(HttpStatusCode.Accepted);
}
private HttpResponseMessage HandleInvokeMessages (Activity activity)
{
var activityValue = activity.Value.ToString();
if (activity.Name == "task/fetch")
{
var action = Newtonsoft.Json.JsonConvert.DeserializeObject<Teams.Samples.TaskModule.Web.Models.BotFrameworkCardValue<string>>(activityValue);
Teams.Samples.TaskModule.Web.Models.TaskInfo taskInfo = GetTaskInfo(action.Data);
Teams.Samples.TaskModule.Web.Models.TaskEnvelope taskEnvelope = new Teams.Samples.TaskModule.Web.Models.TaskEnvelope
{
Task = new Teams.Samples.TaskModule.Web.Models.Task()
{
Type = Teams.Samples.TaskModule.Web.Models.TaskType.Continue,
TaskInfo = taskInfo
}
};
return msg;
}
return new HttpResponseMessage(HttpStatusCode.Accepted);
}
There is more code as per the GitHub sample but I won't paste it here. Can someone point me into the correct direction ?
I have got to the stage that it is displaying a pop up window but the content and title comes from manifest file instead of creating actual iframe also no video is rendering. My goal is to render video within my teams using iframe container.
The important part from the sample:
This sample is deployed on Microsoft Azure and you can try it yourself by uploading Task Module CSharp.zip to one of your teams and/or as a personal app. (Sideloading must be enabled for your tenant; see step 6 here.) The app is running on the free Azure tier, so it may take a while to load if you haven't used it recently and it goes back to sleep quickly if it's not being used, but once it's loaded it's pretty snappy.
So,
Your Teams Admin MUST enable sideloading
Your bot MUST be sideloaded into Teams
The easiest way to do this would be download the sample manifest, open it in App Studio, then edit your bot information in. You then need to make sure Domains and permissions > Valid Domains are set for your bot. Also ensure you change the Tabs URLs to your own.
You also need to make sure that in your Tasks, the URLs they call ALL use https and not http. If anywhere in the chain is using http (like if you're using ngrok and http://localhost), it won't work.

Streaming S3 object to VertX Http Server Response

The title basically explains itself.
I have a REST endpoint with VertX. Upon hitting it, I have some logic which results in an AWS-S3 object.
My previous logic was not to upload to S3, but to save it locally. So, I can do this at the response routerCxt.response().sendFile(file_path...).
Now that the file is in S3, I have to download it locally before I could call the above code.
That is slow and inefficient. I would like to stream S3 object directly to the response object.
In Express, it's something like this. s3.getObject(params).createReadStream().pipe(res);.
I read a little bit, and saw that VertX has a class called Pump. But it is used by vertx.fileSystem() in the examples.
I am not sure how to plug the InputStream from S3'sgetObjectContent() to the vertx.fileSystem() to use Pump.
I am not even sure Pump is the correct way because I tried to use Pump to return a local file, and it didn't work.
router.get("/api/test_download").handler(rc -> {
rc.response().setChunked(true).endHandler(endHandlr -> rc.response().end());
vertx.fileSystem().open("/Users/EmptyFiles/empty.json", new OpenOptions(), ares -> {
AsyncFile file = ares.result();
Pump pump = Pump.pump(file, rc.response());
pump.start();
});
});
Is there any example for me to do that?
Thanks
It can be done if you use the Vert.x WebClient to communicate with S3 instead of the Amazon Java Client.
The WebClient can pipe the content to the HTTP server response:
webClient = WebClient.create(vertx, new WebClientOptions().setDefaultHost("s3-us-west-2.amazonaws.com"));
router.get("/api/test_download").handler(rc -> {
HttpServerResponse response = rc.response();
response.setChunked(true);
webClient.get("/my_bucket/test_download")
.as(BodyCodec.pipe(response))
.send(ar -> {
if (ar.failed()) {
rc.fail(ar.cause());
} else {
// Nothing to do the content has been sent to the client and response.end() called
}
});
});
The trick is to use the pipe body codec.

Grails how to post out to someone else's API

I am writing a Grails app, and I want the controller to hit some other API with a POST and then use the response to generate the page my user sees. I am not able to Google the right terms to find anything about posting to another page and receiving the response with Grails. Links to tutorials or answers like "Thats called..." would me much appreciated.
Seems like you are integrating with some sort of RESTful web service. There is REST client plugin, linked here.
Alternatively, its quite easy to do this without a plugin, linked here.
I highly recommend letting your controller just be a controller. Abstract your interface with this outside service into some class like OtherApiService or some sort of utility. Keep all the code that communicates with this outside service in one place; that way you can mock your integration component and make testing everywhere else easy. If you do this as a service, you have room to expand, say in the case you want to start storing some data from the API in your own app.
Anyway, cutting and posting from the linked documentation (the second link), the following shows how to send a GET to an API and how to set up handlers for success and failures, as well as dealing with request headers and query params -- this should have everything you need.
#Grab(group='org.codehaus.groovy.modules.http-builder', module='http-builder', version='0.5.0-RC2' )
import groovyx.net.http.*
import static groovyx.net.http.ContentType.*
import static groovyx.net.http.Method.*
def http = new HTTPBuilder( 'http://ajax.googleapis.com' )
// perform a GET request, expecting JSON response data
http.request( GET, JSON ) {
uri.path = '/ajax/services/search/web'
uri.query = [ v:'1.0', q: 'Calvin and Hobbes' ]
headers.'User-Agent' = 'Mozilla/5.0 Ubuntu/8.10 Firefox/3.0.4'
// response handler for a success response code:
response.success = { resp, json ->
println resp.statusLine
// parse the JSON response object:
json.responseData.results.each {
println " ${it.titleNoFormatting} : ${it.visibleUrl}"
}
}
// handler for any failure status code:
response.failure = { resp ->
println "Unexpected error: ${resp.statusLine.statusCode} : ${resp.statusLine.reasonPhrase}"
}
}
You might also want to check out this, for some nifty tricks. Is has an example with a POST method.