I've installed / configured Magento 1.9. CE on my local Ubuntu 14 machine by following the guide at https://www.digitalocean.com/community/tutorials/how-to-install-and-configure-magento-on-ubuntu-14-04. I've setup my Magento base as http://localhost.
I then tried accessing the APIs from Java using Scribe. This is the code I have right now:
public final class MagentoAuth {
/**
* #param args
*/
public static void main(String[] args) {
final String MAGENTO_API_KEY = "abc";
final String MAGENTO_API_SECRET = "xyz";
final String MAGENTO_REST_API_URL = "http://localhost/api/rest";
// three-legged oauth
OAuthService service = new ServiceBuilder()
.provider(MagentoThreeLeggedOAuth.class)
.apiKey(MAGENTO_API_KEY)
.apiSecret(MAGENTO_API_SECRET)
.debug()
.build();
System.out.println("" + service.getVersion());
Scanner in = new Scanner(System.in);
System.out.println("Magento's OAuth Workflow");
System.out.println();
// Obtain the Request Token
System.out.println("Fetching the Request Token...");
Token requestToken = service.getRequestToken();
System.out.println("Got the Request Token!");
System.out.println();
System.out.println("Fetching the Authorization URL...");
String authorizationUrl = service.getAuthorizationUrl(requestToken);
System.out.println("Got the Authorization URL!");
System.out.println("Now go and authorize Main here:");
System.out.println(authorizationUrl);
System.out.println("And paste the authorization code here");
System.out.print(">>");
Verifier verifier = new Verifier(in.nextLine());
System.out.println();
System.out.println("Trading the Request Token for an Access Token...");
Token accessToken = service.getAccessToken(requestToken, verifier);
System.out.println("Got the Access Token!");
System.out.println("(if your curious it looks like this: "
+ accessToken + " )");
System.out.println();
OAuthRequest request = new OAuthRequest(Verb.GET, MAGENTO_REST_API_URL+ "/products?limit=2");
service.signRequest(accessToken, request);
Response response = request.send();
System.out.println();
System.out.println(response.getCode());
System.out.println(response.getBody());
System.out.println();
}
}
When I run this program, I'm able to successfully get the token, which I then verify from the Magento Admin panel. But then when I paste the token, I get the following error:
404
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>404 Not Found</title>
</head><body>
<h1>Not Found</h1>
<p>The requested URL /api/rest/products was not found on this server.</p>
<hr>
<address>Apache/2.4.7 (Ubuntu) Server at localhost Port 80</address>
</body></html>
Now, I saw on many forums that adding a block to their Apache conf file worked. I have the following in my conf file (/etc/apache2/sites-available/magento.conf):
<VirtualHost *:80>
DocumentRoot /var/www/html
<Directory /var/www/html/>
Options Indexes FollowSymLinks MultiViews
AllowOverride All
</Directory>
</VirtualHost>
However, I was not sure if that is the place to add. So I also added the <Directory> block in my apache.conf file (/etc/apache2/apache.conf) as well and restarted apache2 server. However, this doesn't help me either. I still get the 404 error. What am I doing wrong here?
Not sure if this has anything to do with the problem ,but I added the sample data provided on Magento's website after installing Magento (even though they have specifically mentioned not to do so) because of which all my default data such as admin user etc were deleted from the database. So I had to add the admin user manually myself in the database. Would this have changed something else?
======================== EDIT ===========================
I edited my .htaccess file and added the following:
## you can put here your magento root folder
## path relative to web root
RewriteBase /var/www/html/
Now, when I try to run my code, it fails even before getting the token with this error:
response status code: 403
response body: <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>403 Forbidden</title>
</head><body>
<h1>Forbidden</h1>
<p>You don't have permission to access /var/www/html/index.php
on this server.</p>
<hr>
<address>Apache/2.4.7 (Ubuntu) Server at localhost Port 80</address>
</body></html>
Thanks.
Make sure to use Options -MultiViews
i'm trying to request a GET via HTTPS trough a Proxy. The Proxy answers with 400:Bad Request. I sniffed the data with wireshark and i have seen, that the headers are not set. Because of security, i replaced some Values with <> Brackets. Can anybody help?
This is a part of my implementation:
String urlString = ctx.getUrl();
HttpHost target;
CloseableHttpClient httpclient;
HttpClientContext localContext;
try
{
CredentialsProvider credsProvider = new BasicCredentialsProvider();
int proxyport = Integer.parseInt(ctx.getProxyPort());
credsProvider.setCredentials(
new AuthScope(<MyProxyUrl>, <MyProxyPort>),
new UsernamePasswordCredentials(<MyProxyUser>, <MyProxyPassword>));
httpclient = HttpClients.custom().setDefaultCredentialsProvider(credsProvider).build();
target = new HttpHost(urlString, 443, "https");
HttpHost proxy = new HttpHost(<MyProxyUser>, <MyProxyPassword>);
RequestConfig config = RequestConfig.custom().setProxy(proxy).build();
request = new HttpGet("/");
request.setURI(new URI(urlString));
//this method sets different header
HttpProperty.setHeaders(request, ctx);
request.setConfig(config);
HttpResponse response = httpclient.execute(target, request);
I have traced the headers in the request with and printed name/value with the output below:
Header[] headers = request.getAllHeaders();
Header Name:Proxy-Connection
Header Value:close
Header Name:Proxy-Authorization
Header Value:Basic
Header Name:User-Agent
Header Value: MyDevice
Header Name:Accept
Header Value:text/html
At least this is the response:
Content length responesCode: 0 400
From Wireshark sniffing i found, that the headers which are defintly inside the request are not set when sending CONNECT.
I attached this picture:
EDIT:
Thank you Damian Nikodem,
but i found the solution.
The first request is always sent without user headers.
I changed two things, and the proxy authorization works:
request = new HttpGet(urlString);
HttpResponse response = httpclient.execute(request);
I am assuming that you are attempting to communicate with a SCADA or production management system. From what I can see you are sending plain HTTP to your target server via the proxy, while trying to connect via HTTPS.
I couldn't see the response in your wireshark dump, but it most likely contains a error message that looks something like this:
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html>
<head>
<title>400 Bad Request</title>
</head>
<body>
<h1>Bad Request</h1>
<p>Your browser sent a request that this server could not understand.<br />
Reason: You're speaking plain HTTP to an SSL-enabled server port.<br />
Instead use the HTTPS scheme to access this URL, please.<br />
<blockquote>Hint: <b>https://???/</b></blockquote>
</p>
</body>
</html>
P.S. You should edit your image a little bit more because it shows a LOT of information about your employer/customer..
#Thorgas, I don't understand what you sad about "found the solution". Do you mean that you found out how to add extra headers in the CONNECT with HttpClinet? I'm also confusing about this.
But i found out that using HttpsUrlconnection in android doesn't have this kind of problem.
URLConnection urlConnection = url.openConnection(proxy);
HttpsURLConnection httpsUrlConnection = (HttpsURLConnection) urlConnection;
httpsUrlConnection.setRequestMethod("GET");
httpsUrlConnection.setRequestProperty("HEADER1","HEADER1Content");
httpsUrlConnection.connect();
I need to wrap this up but not sure how :-)
Basically, I'm using jQuery forms to upload a file to a WCF REST service.
Client looks like:
<script type="text/javascript">
$(function () {
$('form').ajaxForm(
{
url: '/_vti_bin/VideoUploadService/VideoUploadService.svc/Upload/theFileName',
type: 'post'
})
});
</script>
<form id="fileUploadForm" name="fileUploadForm" method="post" action="/_vti_bin/VideoUploadService/VideoUploadService.svc/Upload" enctype="multipart/form-data">
<input type="text" name="filename" />
<input type="file" id="postedFile" name="postedFile" />
<input type="submit" id="fileUploadSubmit" value="Do Upload" />
</form>
While server relevant snippets are
[WebInvoke(UriTemplate = "Upload/{fileName}", ResponseFormat = WebMessageFormat.Json)]
void Upload(string fileName, Stream fileStream);
public void Upload(string fileName, Stream stream)
{
// just write stream to disk...
}
Problem is that what I write to disk looks like this, not the content of the file (in my case, "0123456789"):
-----------------------------7dc7e131201ea
Content-Disposition: form-data; name="MSOWebPartPage_PostbackSource"
// bunch of same stuff here
-----------------------------7dc7e131201ea
Content-Disposition: form-data; name="filename"
fdfd
-----------------------------7dc7e131201ea
Content-Disposition: form-data; name="postedFile"; filename="C:\Inter\Garbage\dummy.txt"
Content-Type: text/plain
0123456789
-----------------------------7dc7e131201ea--
Question - should I be manually parsing what I get above and extract the last part which corresponds to the uploaded file (not elegant)? Or there is a way to apply attribute, content type, whatever setting, in order to get in the stream JUST the uploaded file's content?
I'm suspect there is, but I'm missing it... any help?
Thanks much!
Take a look at 2 articles about Form file upload using ASP.NET Web API:
http://www.asp.net/web-api/overview/working-with-http/sending-html-form-data,-part-1
http://www.asp.net/web-api/overview/working-with-http/sending-html-form-data,-part-2
ASP.NET Web API already has classes (like MultipartFormDataStreamProvider) to parse multipart requests and also to save data from them. Why not to use these classes to solve your problem
UPD In case of .NET 3.5 code and hosting:
Think that MultipartFormDataStreamProvider class and it's assembly aren't for .NET 3.5. In these case you should use some other library/class to parse multipart or do it manually.
Try following project (MIT license) and file HttpMultipartParser.cs from this project:
https://bitbucket.org/lorenzopolidori/http-form-parser/src
I'm using varnish to cache our pages. When we get a 503 -- which happens a little too often -- I'd like to put some sort of page tracking on there. I'd like to place the GA code in there. I can't seem to find any instance of anyone else doing this. Has anyone done this? Is there some sort of T&C violation in doing this?
For Varnish you can use vcl_error to include your own responses (that have the Google Analytics code).
Edit: I have not tested any of these. They are just examples.
An example:
sub vcl_error {
set obj.http.Content-Type = "text/html; charset=utf-8";
if (obj.status == 503) {
synthetic {"
<html>
<head>
<title></title>
</head>
<body>
<h1>Error</h1>
<p>Something</p>
<!-- ANALYTICS CODE -->
</body>
</html>
"};
return(deliver);
}
}
Alternatively you can add your own pages from the file system by using vmod (which is included as standard with version 3.*+).
# Add this to the top of your config
import std;
# vcl_error
sub vcl_error {
set obj.http.Content-Type = "text/html; charset=utf-8";
if (obj.status == 503) {
set obj.http.error503 = std.fileread("/path/to/errors/503.html");
synthetic obj.http.error503;
return(deliver);
}
}
I'm trying to start a Selenium test with a POST request to my application.
Instead of a simple open(/startpoint)
I would like to do something like open(/startpoint, stuff=foo,stuff2=bar)
Is there any way to do that?
I'm asking this because the original page which posts to this start point depends on external providers that are often offline (development environment) and so will often fail too early (and are not the subject of the test)
I guess sending data as GET would work too. I would just prefer using a POST method.
If you are using Python selenium bindings, nowadays, there is an extension to selenium - selenium-requests:
Extends Selenium WebDriver classes to include the request function
from the Requests library, while doing all the needed cookie and
request headers handling.
Example:
from seleniumrequests import Firefox
webdriver = Firefox()
response = webdriver.request('POST', 'url here', data={"param1": "value1"})
print(response)
Short answer: No.
But you might be able to do it with a bit of filthing. If you open up a test page (with GET) then evaluate some JavaScript on that page you should be able to replicate a POST request. See JavaScript post request like a form submit to see how you can replicate a POST request in JavaScript.
Hope this helps.
One very practical way to do this is to create a dummy start page for your tests that is simply a form with POST that has a single "start test" button and a bunch of <input type="hidden"... elements with the appropriate post data.
For example you might create a SeleniumTestStart.html page with these contents:
<body>
<form action="/index.php" method="post">
<input id="starttestbutton" type="submit" value="starttest"/>
<input type="hidden" name="stageid" value="stage-you-need-your-test-to-start-at"/>
</form>
</body>
In this example, index.php is where your normal web app is located.
The Selenium code at the start of your tests would then include:
open /SeleniumTestStart.html
clickAndWait starttestbutton
This is very similar to other mock and stub techniques used in automated testing. You are just mocking the entry point to the web app.
Obviously there are some limitations to this approach:
data cannot be too large (e.g. image data)
security might be an issue so you need to make sure that these test files don't end up on your production server
you may need to make your entry points with something like php instead of html if you need to set cookies before the Selenium test gets going
some web apps check the referrer to make sure someone isn't hacking the app - in this case this approach probably won't work - you may be able to loosen this checking in a dev environment so it allows referrers from trusted hosts (not self, but the actual test host)
Please consider reading my article about the Qualities of an Ideal Test
I used driver.execute_script() to inject an html form into the page, and then submit it. It looks like this:
def post(path, params):
driver.execute_script("""
function post(path, params, method='post') {
const form = document.createElement('form');
form.method = method;
form.action = path;
for (const key in params) {
if (params.hasOwnProperty(key)) {
const hiddenField = document.createElement('input');
hiddenField.type = 'hidden';
hiddenField.name = key;
hiddenField.value = params[key];
form.appendChild(hiddenField);
}
}
document.body.appendChild(form);
form.submit();
}
post(arguments[1], arguments[0]);
""", params, path)
# example
post(path='/submit', params={'name': 'joe'})
If you'd like, you can just add it the function to \selenium\webdriver\chrome\webdriver.pyand then use it in your code with driver.post()
Selenium IDE allows you to run Javascript using storeEval command. Mentioned above solution works fine if you have test page (HTML, not XML) and you need to perform only POST request.
If you need to make POST/PUT/DELETE or any other request then you will need another approach:
XMLHttpRequest!
Example listed below has been tested - all methods (POST/PUT/DELETE) work just fine.
<!--variables-->
<tr>
<td>store</td>
<td>/your/target/script.php</td>
<td>targetUrl</td>
</tr>
<tr>
<td>store</td>
<td>user=user1&password</td>
<td>requestParams</td>
</tr>
<tr>
<td>store</td>
<td>POST</td>
<td>requestMethod</td>
</tr>
<!--scenario-->
<tr>
<td>storeEval</td>
<td>window.location.host</td>
<td>host</td>
</tr>
<tr>
<td>store</td>
<td>http://${host}</td>
<td>baseUrl</td>
</tr>
<tr>
<td>store</td>
<td>${baseUrl}${targetUrl}</td>
<td>absoluteUrl</td>
</tr>
<tr>
<td>store</td>
<td>${absoluteUrl}?${requestParams}</td>
<td>requestUrl</td>
</tr>
<tr>
<td>storeEval</td>
<td>var method=storedVars['requestMethod']; var url = storedVars['requestUrl']; loadXMLDoc(url, method); function loadXMLDoc(url, method) { var xmlhttp = new XMLHttpRequest(); xmlhttp.onreadystatechange=function() { if (xmlhttp.readyState==4) { if(xmlhttp.status==200) { alert("Results = " + xmlhttp.responseText);} else { alert("Error!"+ xmlhttp.responseText); }}}; xmlhttp.open(method,url,true); xmlhttp.send(); }</td>
<td></td>
</tr>
Clarification:
${requestParams} - parameters you would like to post (e.g. param1=value1¶m2=value3¶m1=value3)
you may specify as many parameters as you need
${targetUrl} - path to your script (if your have page located at http://domain.com/application/update.php then targetUrl should be equal to /application/update.php)
${requestMethod} - method type (in this particular case it should be "POST" but can be "PUT" or "DELETE" or any other)
Selenium doesn't currently offer API for this, but there are several ways to initiate an HTTP request in your test. It just depends what language you are writing in.
In Java for example, it might look like this:
// setup the request
String request = "startpoint?stuff1=foo&stuff2=bar";
URL url = new URL(request);
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setRequestMethod("POST");
// get a response - maybe "success" or "true", XML or JSON etc.
InputStream inputStream = connection.getInputStream();
BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(inputStream));
String line;
StringBuffer response = new StringBuffer();
while ((line = bufferedReader.readLine()) != null) {
response.append(line);
response.append('\r');
}
bufferedReader.close();
// continue with test
if (response.toString().equals("expected response"){
// do selenium
}
Well, i agree with the #Mishkin Berteig - Agile Coach answer. Using the form is the quick way to use the POST features.
Anyway, i see some mention about javascript, but no code. I have that for my own needs, which includes jquery for easy POST plus others.
Basically, using the driver.execute_script() you can send any javascript, including Ajax queries.
#/usr/local/env/python
# -*- coding: utf8 -*-
# proxy is used to inspect data involved on the request without so much code.
# using a basic http written in python. u can found it there: http://voorloopnul.com/blog/a-python-proxy-in-less-than-100-lines-of-code/
import selenium
from selenium import webdriver
import requests
from selenium.webdriver.common.proxy import Proxy, ProxyType
jquery = open("jquery.min.js", "r").read()
#print jquery
proxy = Proxy()
proxy.proxy_type = ProxyType.MANUAL
proxy.http_proxy = "127.0.0.1:3128"
proxy.socks_proxy = "127.0.0.1:3128"
proxy.ssl_proxy = "127.0.0.1:3128"
capabilities = webdriver.DesiredCapabilities.PHANTOMJS
proxy.add_to_capabilities(capabilities)
driver = webdriver.PhantomJS(desired_capabilities=capabilities)
driver.get("http://httpbin.org")
driver.execute_script(jquery) # ensure we have jquery
ajax_query = '''
$.post( "post", {
"a" : "%s",
"b" : "%s"
});
''' % (1,2)
ajax_query = ajax_query.replace(" ", "").replace("\n", "")
print ajax_query
result = driver.execute_script("return " + ajax_query)
#print result
#print driver.page_source
driver.close()
# this retuns that from the proxy, and is OK
'''
POST http://httpbin.org/post HTTP/1.1
Accept: */*
Referer: http://httpbin.org/
Origin: http://httpbin.org
X-Requested-With: XMLHttpRequest
User-Agent: Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/538.1 (KHTML, like Gecko) PhantomJS/2.0.0 Safari/538.1
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
Content-Length: 7
Cookie: _ga=GAxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx; _gat=1
Connection: Keep-Alive
Accept-Encoding: gzip, deflate
Accept-Language: es-ES,en,*
Host: httpbin.org
None
a=1&b=2 <<---- that is OK, is the data contained into the POST
None
'''
from selenium import webdriver
driver = webdriver.Firefox()
driver.implicitly_wait(12)
driver.set_page_load_timeout(10)
def _post_selenium(self, url: str, data: dict):
input_template = '{k} <input type="text" name="{k}" id="{k}" value="{v}"><BR>\n'
inputs = ""
if data:
for k, v in data.items():
inputs += input_template.format(k=k, v=v)
html = f'<html><body>\n<form action="{url}" method="post" id="formid">\n{inputs}<input type="submit" id="inputbox">\n</form></body></html>'
html_file = os.path.join(os.getcwd(), 'temp.html')
with open(html_file, "w") as text_file:
text_file.write(html)
driver.get(f"file://{html_file}")
driver.find_element_by_id('inputbox').click()
_post_selenium("post.to.my.site.url", {"field1": "val1"})
driver.close()