I am requesting files from an IBM HTTP server via a Websphere App Server (7FP19). For most files I get the content-length header but for some, not. I discovered that when I set the last-modified value in the request to '0' then I get the content-length for all files.
This seems a bit wierd to me. Does anyone know why this might be or is it just a coincidence?
Here is some code:
connection = (HttpURLConnection) url.openConnection();
for (String value : cookies.values()) {
connection.addRequestProperty("Cookie", value); //$NON-NLS-1$
}
connection.setDoOutput(true);
connection.setRequestProperty("User-Agent", USER_AGENT); //$NON-NLS-1$
//connection.setIfModifiedSince(localLastModified);
connection.setIfModifiedSince(0);
OutputStreamWriter wr = new OutputStreamWriter(connection.getOutputStream());
wr.write(post);
wr.flush();
wr.close();
....
// set file attributes
long remoteDate = connection.getLastModified();
if(rc == 304)
data.lastModified = localLastModified;
else
data.lastModified = remoteDate;
data.retCode = connection.getResponseCode();
data.contentType = connection.getContentType();
data.contentEncoding = connection.getContentEncoding();
int expectedLength = connection.getContentLength();
if(expectedLength < 0) {
log.warn("Expected length: " + expectedLength);
}
UPDATE
this was running on Wesphere FP19. I returned to FP15 and the problem was gone. The length is always returned.
Are you just getting HTTP_NOT_MODIFIED/304 back which has no body and no C-L header? This seems to be working as expected.
Related
I am currently using HttpDeclarePushto exploit the Server Push feature in HTTP/2.
I am able to successfully create all the parameters that this function accepts. But the issue is when HttpDeclarePushexecutes it returns a value of 1229 (ERROR_CONNECTION_INVALID) - https://learn.microsoft.com/en-us/windows/desktop/debug/system-error-codes--1000-1299-.
On further investigation I found that the HttpHeaderConnection in _HTTP_HEADER_ID (https://learn.microsoft.com/en-us/windows/desktop/api/http/ne-http-_http_header_id) is actually passed in the function as 'close'. That implies that on every request response the server closes the connection and that is also happening in my case, I checked it in the log.
Here is the code.
class http2_native_module : public CHttpModule
{
public:
REQUEST_NOTIFICATION_STATUS OnBeginRequest(IN IHttpContext * p_http_context, IN IHttpEventProvider * p_provider)
{
HTTP_REQUEST_ID request_id;
const HTTPAPI_VERSION version = HTTPAPI_VERSION_2;
auto pHttpRequest = p_http_context->GetRequest();
auto phttpRequestRaw = pHttpRequest->GetRawHttpRequest();
HANDLE p_req_queue_handle = nullptr;
auto isHttp2 = phttpRequestRaw->Flags;
try {
const auto request_queue_handle = HttpCreateRequestQueue(version, nullptr, nullptr, NULL, &p_req_queue_handle);
const auto verb = phttpRequestRaw->Verb;
const auto http_path = L"/polyfills.0d74a55d0dbab6b8c32c.js"; //ITEM that I want to PUSH to client
const auto query = nullptr;
request_id = phttpRequestRaw->RequestId;
auto headers = phttpRequestRaw->Headers;
auto connId = phttpRequestRaw->ConnectionId;
WriteEventViewerLog(L"OnBeginRequest - Entering HTTPDECLAREPUSH");
headers.KnownHeaders[1].pRawValue = NULL;
headers.KnownHeaders[1].RawValueLength = 0;
const auto is_success = HttpDeclarePush(p_req_queue_handle, request_id, verb, http_path, query, &headers);
sprintf_s(szBuffer, "%lu", is_success);
Log("is_success value", szBuffer); //ERROR CODE 1229 here
HttpCloseRequestQueue(p_req_queue_handle);
}
catch (std::bad_alloc & e)
{
auto something = e;
}
return RQ_NOTIFICATION_CONTINUE;
}
I even tried to update the header connection value as below but it still gives me 1229.
headers.KnownHeaders[1].pRawValue = NULL;
headers.KnownHeaders[1].RawValueLength = 0;
I understand from https://http2.github.io/http2-spec/ that HTTP/2 actually ignores the content in HTTP HEADERs and uses some other mechanism as part of its FRAME.
This brings us to the next question on how we can keep the connection OPEN and is it something related to the FRAME (similar to HEADER) that HTTP2 uses, if so, how C++ or rather Microsoft helps us to play and exploit with the FRAME in HTTP2?
I need to upload a file using an API.
I tried REST CLIENT and didn't find any options.
Tried with HTTP POST and that responded with 415.
Please suggest how to accomplish this
Error 415 is “Unsupported media type”.
You may need to change the media type of the request or check whether that type of file us accepted by the remote server.
https://en.m.wikipedia.org/wiki/List_of_HTTP_status_codes
This solution uses only standard classes of jre 7. Add a step Modified Java Script Value in your transformation. You will have to add two columns in the flow: URL_FORM_POST_MULTIPART_COLUMN and FILE_URL_COLUMN, you can add as many files as you want, you will just have to call outputStreamToRequestBody.write more times.
try
{
//in this step you will need to add two columns from the previous flow -> URL_FORM_POST_MULTIPART_COLUMN, FILE_URL_COLUMN
var serverUrl = new java.net.URL(URL_FORM_POST_MULTIPART_COLUMN);
var boundaryString = "999aaa000zzz09za";
var openBoundary = java.lang.String.format("\n\n--%s\nContent-Disposition: form-data\nContent-Type: text/xml\n\n" , boundaryString);
var closeBoundary = java.lang.String.format("\n\n--%s--\n", boundaryString);
// var netIPSocketAddress = java.net.InetSocketAddress("127.0.0.1", 8888);
// var proxy = java.net.Proxy(java.net.Proxy.Type.HTTP , netIPSocketAddress);
// var urlConnection = serverUrl.openConnection(proxy);
var urlConnection = serverUrl.openConnection();
urlConnection.setDoOutput(true); // Indicate that we want to write to the HTTP request body
urlConnection.setRequestMethod("POST");
//urlConnection.addRequestProperty("Authorization", "Basic " + Authorization);
urlConnection.addRequestProperty("Content-Type", "multipart/form-data; boundary=" + boundaryString);
var outputStreamToRequestBody = urlConnection.getOutputStream();
outputStreamToRequestBody.write(openBoundary.getBytes(java.nio.charset.StandardCharsets.UTF_8));
outputStreamToRequestBody.write(java.nio.file.Files.readAllBytes(java.nio.file.Paths.get(FILE_URL_COLUMN)));
outputStreamToRequestBody.write(closeBoundary.getBytes(java.nio.charset.StandardCharsets.UTF_8));
outputStreamToRequestBody.flush();
var httpResponseReader = new java.io.BufferedReader(new java.io.InputStreamReader(urlConnection.getInputStream()));
var lineRead = "";
var finalText = "";
while((lineRead = httpResponseReader.readLine()) != null) {
finalText += lineRead;
}
var status = urlConnection.getResponseCode();
var result = finalText;
var time = new Date();
}
catch(e)
{
Alert(e);
}
I solved this by using the solution from http://www.dietz-solutions.com/2017/06/pentaho-data-integration-multi-part.html
Thanks Ben.
He's written a Java class for Multi-part Form submission. I extendd by adding a header for Authorization...
I am trying to make http post requests from within Illustrator ExtendScript (via BridgeTalk) and for the most part it is working. However, the documentation on using HttpConnection is non-existent and I am trying to figure out how to set http-headers. The HttpConnection object has both a requestheaders and responseheaders property so I suspect it is possible.
By default, the post requests are being sent with the Content-Type header "text/html", and I would like to override it so that I can use either "application/x-www-form-urlencoded" or "multipart/form-data".
Here is what I have so far:
var http = function (callback) {
var bt = new BridgeTalk();
bt.target = 'bridge' ;
var s = '';
s += "if ( !ExternalObject.webaccesslib ) {\n";
s += " ExternalObject.webaccesslib = new ExternalObject('lib:webaccesslib');\n";
s += "}\n";
s += "var html = '';\n";
s += "var http = new HttpConnection('http://requestb.in/1mo0r1z1');\n";
s += "http.method = 'POST';\n";
s += "http.requestheaders = 'Content-Type, application/x-www-form-urlencoded'\n";
s += "http.request = 'abc=123&def=456';\n";
s += "var c=0,t='';for(var i in http){t+=(i+':'+http[i]+'***');c++;}t='BEFORE('+c+'):'+t;alert(t);\n"; // Debug: to see what properties and values exist on the http object
s += "http.response = html;\n";
s += "http.execute() ;\n";
s += "http.response;\n";
s += "var t='AFTER:';for(var i in http){t+=(i+':'+http[i]+'***');}alert(t);\n"; // Debug: to see what properties and values have been set after executing
bt.body = s;
bt.onResult = function (evt) {
callback(evt);
};
bt.onError = function (evt) {
callback(evt);
};
bt.send();
};
Things to note:
If I try setting the requestheaders properties like in my code above, the request fails. If I comment it out, the request succeeds. The default value for requestheaders is undefined.
Examining the http object after a successful request, shows the reponseheaders properties to be set to: "Connection, keep-alive,Content-Length, 2,Content-Type, text/html; charset=utf-8,Date, Wed, 24 Jun 2015 09:45:40 GMT,Server, gunicorn/18.0,Sponsored-By, https://www.runscope.com,Via, 1.1 vegur". Before the request executes, the responseheaders is set to undefined.
If anyone could help me set the request headers (in particular the Content-Type header), I would be eternally grateful!
Solved it!
The key for setting the content-type header is to set the http.mime property as follows:
s += "http.mime = 'application/x-www-form-urlencoded';\n";
Also for completeness, you can add your own custom headers as follows:
s += "http.requestheaders = ['My-Sample-Header', 'some-value'];\n";
(It turns out the headers is an array which takes the format [key1, value1, key2, value2, .......])
We are trying to build a jmeter testcase which does the following:
login to a system
obtain some information and check whether correct.
Where we are facing issues is because there is a captcha while logging into the system. What we had planned to do was to download the captcha link and display, and wait for user to type in the value. Once done, everything goes as usual.
We couldnt find any plugin that can do the same? Other than writing our own plugin, is there any option here?
I was able to solve it myself. The solution is as follows:
Create a JSR223 PostProcessor (using Groovy)
more practical CAPTCHA example with JSESSIONID handling and proxy setting
using image.flush() to prevent stale CAPTCHA image in dialog box
JSR223 Parameters for proxy connection setting:
Parameters: proxy 10.0.0.1 8080
In it, the following code displays the captcha and waits for user input
import java.awt.Image;
import java.awt.Toolkit;
import javax.swing.Icon;
import javax.swing.JOptionPane;
import org.apache.jmeter.threads.JMeterContextService;
import org.apache.jmeter.threads.JMeterContext;
import org.apache.jmeter.protocol.http.control.CookieManager;
import org.apache.jmeter.protocol.http.control.Cookie;
URL urlTemp ;
urlTemp = new URL( "https://your.domainname.com/endpoint/CAPTCHACode");
HttpURLConnection myGetContent = null;
if(args[0]=="proxy" ){
Proxy proxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress(args[1], Integer.parseInt(args[2])));
myGetContent = (HttpURLConnection) urlTemp.openConnection(proxy);
}else{
myGetContent = (HttpURLConnection) urlTemp.openConnection();
}
// false for http GET
myGetContent.setDoOutput(false);
myGetContent.connect();
int status = myGetContent.getResponseCode();
log.info("HTTP Status Code: "+Integer.toString(status));
if (status == HttpURLConnection.HTTP_OK) {
//We have 2 Set-Cookie headers in response message but 1 Set-Cookie entry in Map
String[] parts2;
for (Map.Entry<String, List<String>> entries : myGetContent.getHeaderFields().entrySet()) {
if( entries.getKey() == "Set-Cookie" ){
for (String value : entries.getValue()) {
if ( value.contains("JSESSIONID") == true ){
String[] parts = value.split(";",2);
log.info("Response header: "+ entries.getKey() + " - " + parts[0] );
JMeterContext context = JMeterContextService.getContext();
CookieManager manager = context.getCurrentSampler().getCookieManager();
parts2 = parts[0].split("=",2)
Cookie cookie = new Cookie("JSESSIONID",parts2[1],"your.domainname.com","/endpoint",true,0, true, true, 0);
manager.add(cookie);
log.info( cookie.toString() );
log.info("CookieCount "+ manager.getCookieCount().toString() );
}
}
}
}//end of outer for loop
if ( parts2.find() == null ) {
throw new Exception("The Response Header not contain Set-Cookie:JSESSIONID= .");
}
}else{
throw new Exception("The Http Status Code was ${status} , not expected 200 OK.");
}
BufferedInputStream bins = new BufferedInputStream(myGetContent.getInputStream());
String destFile = "number.png";
File f = new File(destFile);
if(f.exists() ) {
boolean fileDeleted = f.delete();
log.info("delete file ... ");
log.info(String.valueOf(fileDeleted));
}
FileOutputStream fout =new FileOutputStream(destFile);
int m = 0;
byte[] bytesIn = new byte[1024];
while ((m = bins.read(bytesIn)) != -1) {
fout.write(bytesIn, 0, m);
}
fout.close();
bins.close();
log.info("File " +destFile +" downloaded successfully");
Image image = Toolkit.getDefaultToolkit().getImage(destFile);
image.flush(); // release the prior cache of Captcha image
Icon icon = new javax.swing.ImageIcon(image);
JOptionPane pane = new JOptionPane("Enter Captcha", 0, 0, null);
String captcha = pane.showInputDialog(null, "Captcha", "Captcha", 0, icon, null, null);
captcha = captcha.trim();
captcha = captcha.replaceAll("\r\n", "");
log.info(captcha);
vars.put("captcha", captcha);
myGetContent.disconnect();
By vars.put method we can use the captcha variable in any way we want. Thank you everyone who tried to help.
Since CAPTHA used to detect non-humans, JMeter will always fail it.
You have to make a workaround in your software: either disable captcha requesting or print somewhere on page correct captcha. Of course, only for JMeter tests.
Dirty workaround? Print the captcha value in alt image for the tests. And then you can retrieve the value and go on.
I have written an ActiveX control which supports drag-drop of email attachments and disk files and uploads files to a web server.
I used the samples available at this link for Uploading files
Upload files with HTTPWebrequest (multipart/form-data)
I am sending data in chunks by setting the following properties
wr = (HttpWebRequest)WebRequest.Create(UploadUrl);
wr.ContentType = "multipart/form-data; boundary=" + boundary;
wr.Method = "POST";
wr.ContentLength = contentLength;
wr.AllowWriteStreamBuffering = false;
wr.Timeout = 600000;
wr.KeepAlive = false;
wr.ReadWriteTimeout = 600000;
wr.ProtocolVersion = HttpVersion.Version10;
wr.Credentials = System.Net.CredentialCache.DefaultCredentials;
wr.SendChunked = true;
wr.UserAgent = "Mozilla/3.0 (compatible; My Browser/1.0)";
rs = wr.GetRequestStream();
With the above settings I am getting an error (411) Length Required.
After reading the following article I realized, I dont need to set Content-Length property when I set SendChunked = true;
http://en.wikipedia.org/wiki/Chunked_transfer_encoding
But the Microsoft example code here doesn't do so
http://msdn.microsoft.com/en-us/library/system.net.httpwebrequest.sendchunked.aspx
After further digging I came to know that Chunked encoding is supported in HTTP version 1.1 only. So I changed the property as follows
wr.ProtocolVersion = HttpVersion.Version11;
Now I don't see that 411 error any more.
Now, can someone with better knowledge verify my understanding here and please let me know if I am doing right.
Thanks
Ravi.
They are both just mechanisms to let the receiver know when it has reached the end of the transfer. If you want to use Content-Length, it is pretty simple. Just take your encoded byte array of POST data, and use the Length property.
ASCIIEncoding encoding = new ASCIIEncoding ();
byte[] postDataByteArray = encoding.GetBytes (postData);
wr.ContentLength = postDataByteArray.Length;