The api is streaming large volume of data. after performing validation and opening up the connection to backend then creating the jdbc statement, we return httpresponse ok status with the header. The problem we see is that when the streaming breaks, the client does not get error code and only thing we can do is just close the channel.
Here is how we send the status back at the begining;
HttpResponse response = new DefaultHttpResponse(HTTP_1_1, OK);
response.headers().set(CONTENT_TYPE, MimeTypes.TEXT_JSON_UTF_8);
response.setChunked(true);
response.headers().set(Names.TRANSFER_ENCODING, Values.CHUNKED);
Channel ch = ctx.getChannel();
// Write the initial line and the header.
ch.write(response);
When anything fails during the streaming, the error is captured by the catch block;
} catch (Exception e) {
ctx.getChannel().close();
String msg = "Error while streaming dynamic content from backend datasource " + ((DatasetDynamic) datasets[0]).getDbDataSourceName();
error(e, msg);
debug("uriNodes=" + this.uriNodes + "; params=" + this.params);
throw new Exception(msg, e);
} finally {
As you see in the catch block, to notify the client, something went wrong, all it is doing is;ctx.getChannel().close();
Is there anyway we can send proper httpresponse with error back to the client?
Looks like you can send httpresponse anytime thru the channel, it does not have to be as header;
} catch (Exception e) {
// Any exception here means an error in the middle of chunk streaming, we
// send back http 500 to the client to inform the failure
ResponseErrorStatus status = new ResponseErrorStatus(ServiceErrorStatus.INTERNAL_SERVER_ERROR,
" error: " + e.getMessage() + (e.getCause() != null ? (" - caused by: " + e.getCause().getMessage()) : ""));
HttpResponse response = new DefaultHttpResponse(HTTP_1_1, status.serviceErrorStatus.httpStatus);
Channel ch = ctx.getChannel();
ch.write(response);
Related
I am trying to upload file to Dropbox using dropbox java sdk
I created a app in dropbox, gave permission to write, app status is Development.
The code works till "write mode done" logger, it fails after that. fileStream is of type InputStream
DbxClientV2 client = createClient();
String path = "/" + fileName.toString();
try {
System.out.println("STARTING...............");
UploadBuilder ub = client.files().uploadBuilder(path);
System.out.println("ub done");
ub.withMode(WriteMode.ADD);
System.out.println("write mode done");
FileMetadata metadata = ub.uploadAndFinish(fileStream);
System.out.println("File uploaded");
System.out.println(metadata.toStringMultiline());
;
System.out.println("FINISHED...............");
} catch (IOException e) {
e.printStackTrace();
}
Exception I get is
com.dropbox.core.BadResponseException: Bad JSON: expected object value.
at [Source: ; line: 1, column: 1]
at com.dropbox.core.DbxRequestUtil.unexpectedStatus(DbxRequestUtil.java:347)
at com.dropbox.core.DbxRequestUtil.unexpectedStatus(DbxRequestUtil.java:324)
at com.dropbox.core.DbxUploader.finish(DbxUploader.java:268)
at com.dropbox.core.DbxUploader.uploadAndFinish(DbxUploader.java:126)
.
.
.
Caused by: com.fasterxml.jackson.core.JsonParseException: expected object value.
at [Source: ; line: 1, column: 1]
at com.dropbox.core.stone.StoneSerializer.expectStartObject(StoneSerializer.java:91)
at com.dropbox.core.ApiErrorResponse$Serializer.deserialize(ApiErrorResponse.java:53)
I'm actually developing a project that read data from 19 PLCs Siemens S1500 and 1 modicon. I have used the scraper tool following this tutorial:
PLC4x scraper tutorial
but when the scraper is working for a little amount of time I get the following exception:
I have changed the scheduled time between 1 to 100 and I always get the same exception when the scraper reach the same number of received messages.
I have tested if using PlcDriverManager instead of PooledPlcDriverManager could be a solution but the same problem persists.
In my pom.xml I use the following dependency:
<dependency>
<groupId>org.apache.plc4x</groupId>
<artifactId>plc4j-scraper</artifactId>
<version>0.7.0</version>
</dependency>
I have tried to change the version to an older one like 0.6.0 or 0.5.0 but the problem still persists.
If I use the modicon (Modbus TCP) I also get this exception after a little amount of time.
Anyone knows why is happening this error? Thanks in advance.
Edit: With the scraper version 0.8.0-SNAPSHOT I continue having this problem.
Edit2: This is my code, I think the problem can be that in my scraper I am opening a lot of connections and when it reaches 65526 messages it fails. But since all the processing is happenning inside the lambda function and I'm using a PooledPlcDriverManager, I think the scraper is using only one connection so I dont know where is the mistake.
try {
// Create a new PooledPlcDriverManager
PlcDriverManager S7_plcDriverManager = new PooledPlcDriverManager();
// Trigger Collector
TriggerCollector S7_triggerCollector = new TriggerCollectorImpl(S7_plcDriverManager);
// Messages counter
AtomicInteger messagesCounter = new AtomicInteger();
// Configure the scraper, by binding a Scraper Configuration, a ResultHandler and a TriggerCollector together
TriggeredScraperImpl S7_scraper = new TriggeredScraperImpl(S7_scraperConfig, (jobName, sourceName, results) -> {
LinkedList<Object> S7_results = new LinkedList<>();
messagesCounter.getAndIncrement();
S7_results.add(jobName);
S7_results.add(sourceName);
S7_results.add(results);
logger.info("Array: " + String.valueOf(S7_results));
logger.info("MESSAGE number: " + messagesCounter);
// Producer topics routing
String topic = "s7" + S7_results.get(1).toString().substring(S7_results.get(1).toString().indexOf("S7_SourcePLC") + 9 , S7_results.get(1).toString().length());
String key = parseKey_S7("s7");
String value = parseValue_S7(S7_results.getLast().toString(),S7_results.get(1).toString());
logger.info("------- PARSED VALUE -------------------------------- " + value);
// Create my own Kafka Producer
ProducerRecord<String, String> record = new ProducerRecord<String, String>(topic, key, value);
// Send Data to Kafka - asynchronous
producer.send(record, new Callback() {
public void onCompletion(RecordMetadata recordMetadata, Exception e) {
// executes every time a record is successfully sent or an exception is thrown
if (e == null) {
// the record was successfully sent
logger.info("Received new metadata. \n" +
"Topic:" + recordMetadata.topic() + "\n" +
"Partition: " + recordMetadata.partition() + "\n" +
"Offset: " + recordMetadata.offset() + "\n" +
"Timestamp: " + recordMetadata.timestamp());
} else {
logger.error("Error while producing", e);
}
}
});
}, S7_triggerCollector);
S7_scraper.start();
S7_triggerCollector.start();
} catch (ScraperException e) {
logger.error("Error starting the scraper (S7_scrapper)", e);
}
So in the end indeed it was the PLC that was simply hanging up the connection randomly. However the NiFi integration should have handled this situation more gracefully. I implemented a fix for this particular error ... could you please give version 0.8.0-SNAPSHOT a try (or use 0.8.0 if we happen to have released it already)
I am wondering how can I get the HTTP Status returned after a WebException was thrown. I am calling for example a RestAPI to get a token and the Server returns a 401 and a Body in json Format telling me that access is denied. I would like to get the 401 but have not found a way to only get 401.
Catch ex As WebException
Dim resp = New StreamReader(ex.Response.GetResponseStream()).ReadToEnd()
Dim errorNumber As Integer = CInt(ex.Status)
Console.WriteLine(ex.Message & " " & errorNumber)
Console.WriteLine(resp & " ")
Return resp
Below is the console output I have for my code:
CInt(ex.Status) = "7" and the
ex.message = "The remote server returned an error: (401) Unauthorized."
What I am looking for is to get the 401 or whatever the Server sends which would be equal to response.StatusCode
I actually found a way to access the 401 directly
Dim ExResponse = TryCast(ex.Response, HttpWebResponse)
Console.WriteLine(ExResponse.StatusCode)
I have the same problem and I realise a few things while I search for a solution.
WebExceptionStatus enum is not equivalent to http status code that the API you call returned. Instead it is a enum of possible error that may occour during a http call.
The WebExceptionStatus error code that will be returned when you receive an error (400 to 599) from your API is WebExceptionStatus.ProtocolError aka number 7 as int.
When you need to get the response body or the real http status code returned from the api, first you need to check if WebExceptionStatus.Status is WebExceptionStatus.ProtocolError. Then you can get the real response from WebExceptionStatus.Response and read its content.
This is a C# code but you folow the same logic in VB.Net
try
{
...
}
catch (WebException webException)
{
if (webException.Status == WebExceptionStatus.ProtocolError)
{
var httpResponse = (HttpWebResponse)webException.Response;
var responseText = "";
using (var content = new StreamReader(httpResponse.GetResponseStream()))
{
responseText = content.ReadToEnd(); // Get response body as text
}
int statusCode = (int)httpResponse.StatusCode; // Get the status code
}
// Handle other webException.Status errors
}
I'm calling an api using ESP8266 WiFi Chip. Im declaring a global variable to assign the token sent by the server to it. The variable assignment is made inside a while loop. In this loop i'm reading the contents of the reply "the token" and saving it in the global variable.
But when i print out the result outside the while loop to variable is empty. But inside the loop it is not.
String deviceToken;
WiFiClient client;
if (!client.connect(hostIp, 80)){
Serial.println("Connection to server failed.");
Serial.print(client.connect(hostIp, 80));
return;
}
String authURL = "/api/";
authURL += "startauth/";
authURL += serialValue;
Serial.print("Requesting URL: ");
Serial.println(authURL);
// This will send auth request to the server
client.print(String("GET ") + authURL + " HTTP/1.1\r\n" +
"Host: " + host + "\r\n" +
"Connection: close\r\n\r\n");
unsigned long timeout = millis();
while (client.available() == 0) {
if (millis() - timeout > 5000) {
Serial.println(">>> Client Timeout !");
client.stop();
return;
}
}
// Read Auth reply from server and print them to Serial
while(client.available()){
String token = client.readStringUntil('\r');
deviceToken = token;
writeDeviceToken(token.c_str()); //Function to write data in eeprom
Serial.print(deviceToken); //NOT EMPTY
}
Serial.print("Device Token:" + deviceToken); //EMPTY VARIABLE
In your while loop you are looping over the HTTP headers and the data line by line and overwrite deviceToken each time. This is ok, except that each line ends with \r\n and not only \r - (also, you probably only want to write the token to the eeprom, not every header).
So the last token that gets read is a single \n which is then put into deviceToken. You will need to read that extra byte after each line.
An alternative way would be to read until you reach \r\n\r\n (end of header block), then read the size (28 hex - this is a chunked transfer), and then you can read and store the token you want. Otherwise your token would be that last 0, or the newline following it.
Also keep in mind that there is no guarantee that the reply will allways be chunked (which is actually rather strange in this case), and you should be prepared for a 'normal' reply too.
An easy way to fix this, is to send a HTTP/1.0 request instead of HTTP/1.1 (as far as that server supports it). Since HTTP/1.0 doesn't support/allow chunked transfers you will always have a 'clean' deviceToken in the reply.
Thanks to Danny_ds instructions i was able to solve my issue and thought of sharing the working code in here as a reference.
// Read Auth reply from server and print them to Serial
while(client.available()){
char c = client.read();
response += c;
}
String testStatus = response;
if((strstr(testStatus.c_str(),"HTTP/1.1 500")) != NULL) {
Serial.println("Found reponse 500");
return;
} else if ((strstr(testStatus.c_str(),"HTTP/1.1 200")) != NULL) {
Serial.println("Found reponse 200");
//Grabing the token after the header
String token = response.substring(response.indexOf("\r\n\r\n") + 8 );
//Removing the \r in end of token
token.remove(40);
deviceToken = token;
writeDeviceToken(deviceToken.c_str());
delay(500);
Serial.print("Device Token:" + deviceToken);
return;
}
I'm new to grails so hoping someone will be patient and give me a hand. I have a controller that creates a PDF. If the user clicks more then one time before the PDF is created I get the following error. Below is the code for the creation of the PDF.
2016-03-09 09:32:11,549 ERROR errors.GrailsExceptionResolver - SocketException occurred when processing request: [GET] /wetlands-form/assessment/f3458c91-3435-4714-a0e0-3b24de238671/assessment/pdf
Connection reset by peer: socket write error. Stacktrace follows:
java.net.SocketException: Connection reset by peer: socket write error
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:113)
at java.net.SocketOutputStream.write(SocketOutputStream.java:159)
at mdt.wetlands.AssessmentController$_closure11$$EPeyAg3t.doCall(AssessmentController.groovy:300)
at grails.plugin.cache.web.filter.PageFragmentCachingFilter.doFilter(PageFragmentCachingFilter.java:195)
at grails.plugin.cache.web.filter.AbstractFilter.doFilter(AbstractFilter.java:63)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
2016-03-09 09:32:11,549 ERROR errors.GrailsExceptionResolver - IllegalStateException occurred when processing request: [GET] /wetlands-form/assessment/f3458c91-3435-4714-a0e0-3b24de238671/assessment/pdf
getOutputStream() has already been called for this response. Stacktrace follows:
org.codehaus.groovy.grails.web.pages.exceptions.GroovyPagesException: Error processing GroovyPageView: getOutputStream() has already been called for this response
at grails.plugin.cache.web.filter.PageFragmentCachingFilter.doFilter(PageFragmentCachingFilter.java:195)
at grails.plugin.cache.web.filter.AbstractFilter.doFilter(AbstractFilter.java:63)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.lang.IllegalStateException: getOutputStream() has already been called for this response
at C__MDTDATA_gg_workspace_new_wetlands_grails_app_views_error_gsp.run(error.gsp:1)
... 5 more
2016-03-09 09:32:11,549 ERROR [/wetlands-form].[grails] - Servlet.service() for servlet grails threw exception
java.lang.IllegalStateException: getOutputStream() has already been called for this response
PDF CODE VIA rendering plugin
def pdf = {
def assessment = lookupAssessment()
if (!assessment){
return
}
// Trac 219 Jasper report for PDF output
Map reportParams = [:]
def report = params.report
def printType = params.printType
def mitigationType = params.mitigationType
def fileName
def fileType
fileType = 'PDF'
def reportDir =
grailsApplication.mainContext.servletContext.getRealPath(""+File.separatorChar+"reports"+File.separatorChar)
def resolver = new SimpleFileResolver(new File(reportDir))
reportParams.put("ASSESS_ID", assessment.id)
reportParams.put("RUN_DIR", reportDir+File.separatorChar)
reportParams.put("JRParameter.REPORT_FILE_RESOLVER", resolver)
reportParams.put("_format", fileType)
reportParams.put("_file", "assessment")
println params
def reportDef = jasperService.buildReportDefinition(reportParams, request.getLocale(), [])
def file = jasperService.generateReport(reportDef).toByteArray()
// Non-inline reports (e.g. PDF)
if (!reportDef.fileFormat.inline && !reportDef.parameters._inline)
{
response.setContentType("APPLICATION/OCTET-STREAM")
response.setHeader("Content-disposition", "attachment; filename=" + assessment.name + "." + reportDef.fileFormat.extension);
response.contentType = reportDef.fileFormat.mimeTyp
response.characterEncoding = "UTF-8"
response.outputStream << reportDef.contentStream.toByteArray()
}
else
{
// Inline report (e.g. HTML)
render(text: reportDef.contentStream, contentType: reportDef.fileFormat.mimeTyp, encoding: reportDef.parameters.encoding ? reportDef.parameters.encoding : 'UTF-8');
}
}
This is the WORD code.
def word = {
def assessment = lookupAssessment()
if (!assessment){
return
}
// get the assessment's data as xml
def assessmentXml = g.render(template: 'word', model: [assessment:assessment]).toString()
// open the Word template
def loader = new LoadFromZipNG()
def template = servletContext.getResourceAsStream('/word/template.docx')
WordprocessingMLPackage wordMLPackage = (WordprocessingMLPackage)loader.get(template)
// get custom xml piece from Word template
String itemId = '{44f68b34-ffd4-4d43-b59d-c40f7b0a2880}' // have to pull up part by ID. Watch out - this may change if you muck with the template!
CustomXmlDataStoragePart customXmlDataStoragePart = wordMLPackage.getCustomXmlDataStorageParts().get(itemId)
CustomXmlDataStorage data = customXmlDataStoragePart.getData()
// and replace it with our assessment's xml
ByteArrayInputStream bs = new ByteArrayInputStream(assessmentXml.getBytes())
data.setDocument(bs) // needs java.io.InputStream
// that's it! the data is in the Word file
// but in order to do the highlighting, we have to manipulate the Word doc directly
// gather the list of cells to highlight
def highlights = assessment.highlights()
// get the main document from the Word file as xml
MainDocumentPart mainDocPart = wordMLPackage.getMainDocumentPart()
def xml = XmlUtils.marshaltoString(mainDocPart.getJaxbElement(), true)
// use the standard Groovy tools to handle the xml
def document = new XmlSlurper(keepWhitespace:true).parseText(xml)
// for each value in highlight list - find node, shade cell and add bold element
highlights.findAll{it != null}.each{highlight ->
def tableCell = document.body.tbl.tr.tc.find{it.sdt.sdtPr.alias.'#w:val' == highlight}
tableCell.tcPr.shd[0].replaceNode{
'w:shd'('w:fill': 'D9D9D9') // shade the cell
}
def textNodes = tableCell.sdt.sdtContent.p.r.rPr
textNodes.each{
it.appendNode{
'w:b'() // bold element
}
}
}
// here's a good way to print out xml for debugging
// System.out.println(new StreamingMarkupBuilder().bindNode(document.body.tbl.tr.tc.find{it.sdt.sdtPr.alias.#'w:val' == '12.1.1'}).toString())
// or save xml to file for study
// File testOut = new File("C:/MDTDATA/wetlands-trunk/xmlout.xml")
// testOut.setText(new StreamingMarkupBuilder().bindNode(document).toString())
// get the updated xml back in the Word doc
Object obj = XmlUtils.unmarshallFromTemplate(new StreamingMarkupBuilder().bindNode(document).toString(), null);
mainDocPart.setJaxbElement((Object)obj)
File file = File.createTempFile('wordexport-', '.docx')
wordMLPackage.save(file)
response.setHeader('Content-Type', 'application/vnd.openxmlformats-officedocument.wordprocessingml.document;')
response.setHeader('Content-Disposition', "attachment; filename=${assessment.name.encodeAsURL()}.docx")
response.setHeader('Content-Length', "${file.size()}")
response.outputStream << file.readBytes()
response.outputStream.flush()
file.delete()
}
// for checking XML during development
def word2 = {
def assessment = lookupAssessment()
if (!assessment){
return
}
render template: 'word', model: [assessment:assessment]
}
You need to catch the exception, if you wish to not do anything with it then as below in the catch nothing going on.. after it has gone through try and catch if still no file we know something has gone wrong so we render another or same view with error this time. After this it returns so it won't continue to your other bit which checks report type i.e. pdf or html
..
//declare file (def means it could be any type of object)
def file
//Now when you expect unexpected behaviour capture it with a try/catch
try {
file = jasperService.generateReport(reportDef).toByteArray()
}catch (Exception e) {
//log.warn (e)
//println "${e} ${e.errors}"
}
//in your scenario or 2nd click the user will hit the catch segment
//and have no file produced that would be in the above try block
//this now says if file == null or if file == ''
// in groovy !file means capture if there nothing defined for file
if (!file) {
//render something else
render 'a message or return to page with error that its in use or something gone wrong'
//return tells your controller to stop what ever else from this point
return
}
//so what ever else would occur will not occur since no file was produced
...
Now a final note try/catches are expensive and should not be used everywhere. If you are expecting something then deal with the data. In scenarios typically like this third party api where you have no control i.e. to make the unexpected expected then you fall back to these methods
1- Client Side : Better is to disable button on first click and wait for response from Server.
2- Catch Exception and do nothing or just print error log.
// get/set parameters
def file
def reportDef
try{
reportDef = jasperService.buildReportDefinition(reportParams, request.getLocale(), [])
file = jasperService.generateReport(reportDef).toByteArray()
}catch(Exception e){
// print log or do nothing
}
if (file){
// render file according to your conditions
}
else {
// render , return appropriate message.
}
Instead of catching Exception, Its better to catch IOException. Otherwise you will be eating all other exceptions as well. Here is how i handled it.
private def streamFile(File file) {
def outputStream
try {
response.contentType = "application/pdf"
response.setHeader "Content-disposition", "inline; filename=${file.name}"
outputStream = response.outputStream
file.withInputStream {
response.contentLength = it.available()
outputStream << it
}
outputStream.flush()
}
catch (IOException e){
log.info 'Probably User Cancelled the download!'
}
finally {
if (outputStream != null){
try {
outputStream.close()
} catch (IOException e) {
log.info 'Exception on close'
}
}
}
}