USing Wicket 6.17 and servlet 2.5, I have a form that allows file upload, and also has ReCaptcha (using Recaptcha4j). When the form has ReCaptcha without file upload, it works properly using the code:
final HttpServletRequest servletRequest = (HttpServletRequest ) ((WebRequest) getRequest()).getContainerRequest();
final String remoteAddress = servletRequest.getRemoteAddr();
final String challengeField = servletRequest.getParameter("recaptcha_challenge_field");
final String responseField = servletRequest.getParameter("recaptcha_response_field");
to get the challenge and response fields so that they can be validated.
This doesn't work when the form has the file upload because the form must be multipart for the upload to work, and so when I try to get the parameters in that fashion, it fails.
I have pursued trying to get the parameters differently using ServletFileUpload:
ServletFileUpload fileUpload = new ServletFileUpload(new DiskFileItemFactory(new FileCleaner()) );
String response = IOUtils.toString(servletRequest.getInputStream());
and
ServletFileUpload fileUpload = new ServletFileUpload(new DiskFileItemFactory(new FileCleaner()) );
List<FileItem> requests = fileUpload.parseRequest(servletRequest);
both of which always return empty.
Using Chrome's network console, I see the values that I'm looking for in the Request Payload, so I know that they are there somewhere.
Any advice on why the requests are coming back empty and how to find them would be greatly appreciated.
Update: I have also tried making the ReCaptcha component multipart and left out the file upload. The result is still the same that the response is empty, leaving me with the original conclusion about multipart form submission being the problem.
Thanks to the Wicket In Action book, I have found the solution:
MultipartServletWebRequest multiPartRequest = webRequest.newMultipartWebRequest(getMaxSize(), "ignored");
// multiPartRequest.parseFileParts(); // this is needed since Wicket 6.19.0+
IRequestParameters params = multiPartRequest.getRequestParameters();
allows me to read the values now using the getParameterValue() method.
Related
I'm trying to set up an API that generate PDF from web page (provided as URL). The API is gotenberg from thecodingmachine. I have it on Docker, it works just fine, I can't generate PDF through http request send with curl (for now I'm just trying to make it work, so I use the request provided as example in the documentation)
Now I am trying to make it work with my groovy/grails app. So I'm using the java tools to make the request.
Now here is my problem : the PDF file I get is blank (my app opend directly in my browser). It do has the right content, if I open it with the text editor, it's not empty, and it has almost the same content as the one I make using the curl request (which isn't blank).
I am 99% sure the problem come from the encoding. I tried changing the InputStreamReader encoding parameter, but it doesn't change anything. Here I put "X-MACROMAN" because that the encoding inside the pdf file that isn't blank, but it still doesn't change.
Here is my code :
static def execute(def apiURL)
{
def httpClient = HttpClients.createDefault()
// Request parameters and other properties.
def request = new HttpPost(apiURL)
MultipartEntityBuilder builder = MultipartEntityBuilder.create()
builder.addTextBody("remoteURL", 'https://google.com')
builder.addTextBody("marginTop", '0')
builder.addTextBody("marginBottom", '0')
builder.addTextBody("marginLeft", '0')
builder.addTextBody("marginRight", '0')
HttpEntity multipart = builder.build()
request.setEntity(multipart)
def response = httpClient.execute(request)
BufferedReader rd = new BufferedReader(
new InputStreamReader(response.getEntity().getContent(), "X-MACROMAN"))
StringBuffer result = new StringBuffer()
String line = ""
Boolean a = Boolean.FALSE
while ((line = rd.readLine()) != null) {
if(!a){
a = Boolean.TRUE
}
else {
result.append("\n")
}
result.append(line)
}
return result
I am 99% sure the problem come from the encoding. I tried changing the InputStreamReader encoding parameter, but it doesn't change anything. Here I put "X-MACROMAN" because that the encoding inside the pdf file that isn't blank, but it still doesn't change.
Did I made myself clear ? And does those who understands has any ideas why my PDFs are blank ?
I made a custom editor plugin, in a Seam 2.2.2 project, which makes file upload this way:
1) config the editor to load my specific xhtml upload page;
2) call the following method inside this page, and return a javascript callback;
public String sendImageToServer()
{
HttpServletRequest request = ServletContexts.instance().getRequest();
try
{
List<FileItem> items = new ServletFileUpload(new DiskFileItemFactory()).parseRequest(request);
processItems(items);//set the file data to specific att
saveOpenAttachment();//save the file to disk
}
//build callback
For this to work I have to put this inside components.xml:
<web:multipart-filter create-temp-files="false"
max-request-size="1024000" url-pattern="*"/>
The attribute create-temp-files do not seems to matter whatever its value.
But url-pattern has to be "" or "/myUploadPage.seam", any other value makes the item list returns empty. Does Anyone know why?
This turns into a problem because when I use a url-pattern that work to this case, every form with enctype="multipart/form-data" in my application stops to submit data. So I end up with other parts of the system crashing.
Could someone help me?
To solve my problem, I changed the solution to be like Seam multipart filter handle requests:
ServletRequest request = (ServletRequest) FacesContext.getCurrentInstance().getExternalContext().getRequest();
try
{
if (!(request instanceof MultipartRequest))
{
request = unwrapMultipartRequest(request);
}
if (request instanceof MultipartRequest)
{
MultipartRequest multipartRequest = (MultipartRequest) request;
String clientId = "upload";
setFileData(multipartRequest.getFileBytes(clientId));
setFileContentType(multipartRequest.getFileContentType(clientId));
setFileName(multipartRequest.getFileName(clientId));
saveOpenAttachment();
}
}
Now I handle the request like Seam do, and do not need the web:multipart-filter config that was breaking other types of request.
I run a web service where I convert a file from one file format into another. The conversion logic is already functioning but now, I want to query this logic via Jersey. Whenever file upload via Jersey is addressed in tutorials / questions, people describe how to do this using multipart form data. I do however simply want to send and return a single file and skip the overhead of sending multiple parts. (The webservice is triggered by another machine which I control so there is no HTML form involved.)
My question is how would I achieve something like the following:
#POST
#Path("{sessionId"}
#Consumes("image/png")
#Produces("application/pdf")
public Response put(#PathParam("sessionId") String sessionId,
#WhatToPutHere InputStream uploadedFileStream) {
return BusinessLogic.convert(uploadedFile); // returns StreamingOutput - works!
}
How do I get hold of the uploadedFileStream (It should be some annotation, I guess which is of course not #WhatToPutHere). I figured out how to directly return a file via StreamingOutput.
Thanks for any help!
You do not have to put anything in the second param of the function; just leave it un-annoted.
The only thing you have to be carefull is to "name" the resource:
The resource should have an URI like: someSite/someRESTEndPoint/myResourceId so the function should be:
#POST
#Path("{myResourceId}")
#Consumes("image/png")
#Produces("application/pdf")
public Response put(#PathParam("myResourceId") String myResourceId,
InputStream uploadedFileStream) {
return BusinessLogic.convert(uploadedFileStream);
}
If you want to use some kind of SessionID, I'd prefer to use a Header Param... something like:
#POST
#Path("{myResourceId}")
#Consumes("image/png")
#Produces("application/pdf")
public Response put(#HeaderParam("sessionId") String sessionId,
#PathParam("myResourceId") String myResourceId,
InputStream uploadedFileStream) {
return BusinessLogic.convert(uploadedFileStream);
}
I'm struggling with the file upload on form:update with spring roo.
For the creation part, I used a form:multi tag provided by Jose Delgado here. The custom form:multi tag adds the enctype="multipart/form-data" to the form and that works fine.
The problem is when you want to offer the file upload capabilities to your update form. Spring Roo (maybe it's spring mvc, i don't know) will, by default, set enctype="application/x-www-form-urlencoded" to the update form (form:update tag). If I set the enctype attribute to enctype="multipart/form-data" in the upload form, the server will execute "create" method of the controller instead of the "udpate" method when the form is submitted...
Any idea how we could (simply) work around that? I spent quite some time on it already and I'm finding myself out of inspiration (maybe it's because it's the end of the day, also :).
Thanks for your help,
Kind Regards
OK...It seems that there's a little problem with the RequestMapping.
For whatever reason, the method parameters is set to "POST" when the multipart attribute is set to "true" in the form:update tag.
As a workaround, I check the _method parameter at the beginning of the create method. If it's set to "PUT", I return the value of the update method.
#RequestMapping(method = RequestMethod.POST, produces = "text/html")
public String create(#Valid ActionRequest actionRequest, BindingResult bindingResult, Model uiModel, HttpServletRequest httpServletRequest) {
// Work around dispatcher bug: if the multipart attribute of the form is set to true,
// submission of the update form routes to create method
String toto = httpServletRequest.getParameter("_method");
if(httpServletRequest.getParameter("_method").equals("PUT")){
return this.update(actionRequest,bindingResult,uiModel,httpServletRequest);
}
...
}
I am new to Web Crawling, and I am using HttpWebRequest to crawl data from sites.
As of now I was successfully able to crawl and get data from my wordpress site. This data was a simple user profile data. (like name, email, AIM id etc...)
Now as an exercise I want to crawl wikipedia, where I will search using the value entered into textbox at my end and then crawl wikipedia with the search value and get the appropriate title(s) from the search.
Now I have the following doubts/difficulties.
Firstly, is this even possible ? I have heard that wiki has robot.txt setup to block this. Though I have heard this only from a friend and hence not sure.
I am using the same procedure I used earlier, but I am not getting the required results.
Thanks !
Update :
After some explanation and help from #svick, I tried the below code, but still not able to get any value (see last line of code, there I am expecting an html markup of the search result page)
string searchUrl = "http://en.wikipedia.org/w/index.php?search=Wikipedia&title=Special%3ASearch";
var postData = new StringBuilder();
postData.Append("search=" + model.Query);
postData.Append("&");
postData.Append("title" + "Special:Search");
byte[] data2 = Crawler.GetEncodedData(postData.ToString());
var webRequest = (HttpWebRequest)WebRequest.Create(searchUrl);
webRequest.Method = "POST";
webRequest.UserAgent = "Crawling HW (http://yassershaikh.com/contact-me/)";
webRequest.AllowAutoRedirect = false;
ServicePointManager.Expect100Continue = false;
Stream requestStream = webRequest.GetRequestStream();
requestStream.Write(data2, 0, data2.Length);
requestStream.Close();
var responseCsv = (HttpWebResponse)webRequest.GetResponse();
Stream response = responseCsv.GetResponseStream();
// Todo Parsing
var streamReader = new StreamReader(response);
string val = streamReader.ReadToEnd();
// val is empty !! <-- this is my problem !
and here is my GetEncodedData method defination.
public static byte[] GetEncodedData(string postData)
{
var encoding = new ASCIIEncoding();
byte[] data = encoding.GetBytes(postData);
return data;
}
Pls help me on this.
You probably don't need to use HttpWebRequest. Using WebClient (or HttpClient if you're on .Net 4.5) will be much easier for you.
robots.txt doesn't actually block anything. If something doesn't support it (and .Net doesn't support it), it can access anything.
Wikipedia does block requests that don't have their User-Agent header set. And you should use an informative User-Agent string with your contact information.
A better way to access Wikipedia is to use its API, rather than scraping. This way, you will get an answer that's specifically meant to be read by a custom applications, formatted as XML or JSON. There are also dumps containing all information from Wikipedia available for download.
EDIT: The problem with your newly posted code is that your query returns a 302 Moved Temporarily response to the searched article, if it exists. Either remove the line that forbids AllowAutoRedirect, or add &fulltext=Search to your query, which will mean you won't get redirected.