A friend is trying to run a script to check which notebooks are using the most memory, but their server is password protected. I'm trying to figure out how to configure authentication using urllib2 since I don't believe there is a username, only a password.
The #aiguofer answer did not work for me because jupyter now uses '_xsrf' in cookie. The follwoing woked for me:
s = requests.Session()
url='http://127.0.0.1:8888/login/'
resp=s.get(url)
xsrf_cookie = resp.cookies['_xsrf']
params={'_xsrf':xsrf_cookie,'password': password}
s.post(url, data=params)
After that s can be used to call the apis.
After digging into the notebook code and through some trial and error, I figured out how to do this (and I switched to using requests).
I can't guarantee that this is the best way to do it, but it certainly worked for me. I actually set my vars elsewhere in the code but included here for completeness
import requests
hostname = '127.0.0.1'
port = '8888'
password = 'mypassword'
base_url = 'http://{0}:{1}/'.format(hostname, port)
h = {}
if password:
r = requests.post(base_url + 'login', params={
'password': password
})
h = r.request.headers
sessions = requests.get(base_url + 'api/sessions', headers=h).json()
I believe this works because when you hit the /login endpoint, it redirects you with the right headers set. I guess requests keeps the headers of the redirect, so we can reuse those for the other call. It might be better to extract only the cookies and use those, but this works for now :)
It seems there are some changes with new version. url '/login' does not work for me, I need to add next parameters
url='http://localhost:8050/login?next=%2F'
For the login request. The rest just like Hassan answer
i found when use jupyter put api upload file response 403,
need add "X-XSRFToken" header can solve it..
data= json.dumps({
"name": "test.jpg",
"path": "path",
"type":"file",
"format": "base64",
"content": "base64 data"
})
headers["X-XSRFToken"] = xsrf_cookie
s.put(url, data=data, headers=headers)
Related
Hello all good people.
I have tested everything that I can find on internet and nothing is working to fix this problem. I'm really hoping that someone here can help me solve this.
When i try to do "patch" request from backend to my flask API I get this error (GET, DELETE & PUT are working fine):
Access to fetch at 'https://MYAPI-NOTREALURL.com' from origin
'https://MYBACKEND-NOTREALURL.com' has been blocked by CORS policy:
Response to preflight request doesn't pass access control check: The
'Access-Control-Allow-Origin' header contains multiple values
'https://MYBACKEND-NOTREALURL.com, *', but only one is allowed. Have
the server send the header with a valid value, or, if an opaque
response serves your needs, set the request's mode to 'no-cors' to
fetch the resource with CORS disabled.
This is how my code for API is written:
from flask_cors import CORS, cross_origin
from flask import render_template, redirect, flash, request, url_for, jsonify, session, make_response
from flask_restful import Api, Resource, reqparse
import requests
app = Flask(__name__)
cors = CORS(app, resources={r"/*": {"origins": "*"}})
api = Api(app)
class ordersByID(Resource):
def get(self,ID_ORDER):
****
return jsonify(data)
def patch(self,ID_ORDER):
req321 = request.form
result = updateOrder(req321,ID_ORDER)
return result
def delete(self,ID_ORDER):
****
return result
def put(self,ID_ORDER):
****
return result
api.add_resource(ordersByID, "/orders/id/<string:ID_ORDER>")
if __name__ == '__main__':
app.run(debug=True)
I have tested everything that I can find on internet and nothing is working when trying to do patch request. I'm doing patch request with fetch from popup window.
<form action="{{ **https://MYAPI-NOTREALURL.com** }}" id="popupForm" method="patch" onsubmit="formFetch(event,this,'patch')">
You can check javascript code under.
function formFetch(e,form,method) {
result = fetch(form.action, {method:method, body: new FormData(form)})
.then(response => response.json())
.then(data => document.getElementById('submitedFormConfirmationText').innerHTML = data['DB_Result']
);
e.preventDefault();
document.getElementById('submitedFormConfirmation').style.display = 'inline';
};
I really hope that someone can help me solve this problem without needing to redo whole code?
I managed to solve this.
For some strange reason "patch" with small letters was working on local but when deployed it did not work.
Changing method from "patch" to "PATCH" solved this problem.
var url = "https://web-site_name/page/?format=json&var_data-organization_dates&xlsexport=true";
var payload =
{
"login" : "login",
"password" : "pass",
};
var options =
{
"method" : "post",
"payload" : payload,
"followRedirects" : false
};
var login = UrlFetchApp.fetch("https://web-site_name/page/" , options);
var sessionDetails = login.getAllHeaders()['Set-Cookie'];
Logger.log(login.getAllHeaders());
here is the part of the code I try to use, to automate export of the data from web-site, i do have proper login and password and able to download file in json (opened in xsl) manually, I've got the address to the downloaded file in network in developer tools, but i have a problem on the first stage - when trying to authorize to the web-site - access denied. I've tried the code, given in answers on stackoverflow, but it still doesn't work.
How to make an url fetch request correctly, depends on the website you want to access and the authentication they uses
In the simplest case, your website requires HTTP basic authentification, in this case the correct syntax would be
var authHeader = 'Basic ' + Utilities.base64Encode(login + ':' + pass);
var options = {
headers: {Authorization: authHeader}
}
If your website uses a different authentication form, you might need to provide an access token.
In any case: the authentication credentials go into headers, not into payload!
payload is the data that you want to post = upload to the website.
If you want export data from the website - that is download data - you do not need a payload and the correct method would be get, not post. Btw., if the method is get, you do not need to specify it.
Please see here for more information and samples.
I need to build webcrawler for internal usage and I need to login into administration area. I'm trying to use requests lib, tried this ways:
import urllib.parse
import requests
base_url = "https://target.url"
data = ({'login': 'login', 'pass': 'password'})
params = urllib.parse.urlencode(data)
r = requests.post(base_url, data=params)
print(r.text)
and
import requests
base_url = "https://target.url"
r = requests.post(base_url, auth=('login', 'password')
print(r.text)
but in both cases r.text returns me login page content same as if I try to get any other page after auth code:
req = requests.get("https://target.url/smth")
What I lose sight of? I have ideas:
chain of hidden redirections from https://target.url to real login page, so I send auth info to wrong url
I don't send additional required info (like cookies e.g.)
Could you please comment? How can I gather required for login information?
In my case problem was in 'Referer' parameter in headers, which is required but wasn't specified
I'm having trouble with tastypie and posting data to it. I only am able to retrieve a 401 error code.
For clarification, I am able to successfully retrieve data from the tastypie api.
Attached are the code snippets, and maybe someone can help me out get behind this.
Before I get started, a little background: I am using a custom authorization class.
class CustomAuthorization(Authorization):
def is_authorized(self, request, object=None):
if request.user.username == 'custom_user':
return True
return False
Here is the actual resource:
class CustomObjectResource(ModelResource):
class Meta:
queryset = CustomObject.objects.all()
authentication = ApiKeyAuthentication()
authorization = CustomAuthorization()
list_allowed_methods = ['get', 'post', ]
detail_allowed_methods = ['get', 'post', 'put']
include_resource_uri = False
resource_name = 'customobject'
always_return_data = True
def obj_create(self, bundle, request=None, **kwargs):
try:
print "request"
except:
raise BadRequest('I couldnt save your information.')
return True
I know the obj_create method is bogus, but it should still be called and do something, or is this already the issue?
The following curl command is used to post the data to the tastypie API.
curl --dump-header - -H "Content-Type: application/json" -X POST --data '{"body": "This will prbbly be my lst post.", "pub_date": "2011-05-22T00:46:38", "slug": "another-post", "title": "Another Post"}' http://local.com:8000/api/v1/customobject/?format=json&username=custom_user&api_key=123456789012345
The api_key is correct, but bogus in this case!
As previously mentioned, the get method works but the post just wont work.
Anyone have an idea on how to solve this or have a workaround?
I would try a couple of things to debug this issue.
1) Try adding: allowed_methods = ['get', 'post', 'put']
2) Add print statements in the custom_authorization to check if that is causing the problems due to the request.user.username being different.
3) Do (2) in the source of APIKeyAuthentication too.
This should be sufficient for you to debug the issue.
Remember to remove the print statements once youre done!
Best of luck.
This is COULD be due to a known issue. On the background tastypie at the moment converts the POST to PUT and as Nikunj pointed since in list_allowed_methods you don't have PUT the POST gets blocked too... Not sure there though cause you should get method not allowed in that case. I would suggest debug in the method "is_authorized" and check what is happening there.
Whats is the python urllib equivallent of
curl -u username:password status="abcd" http://example.com/update.json
I did this:
handle = urllib2.Request(url)
authheader = "Basic %s" % base64.encodestring('%s:%s' % (username, password))
handle.add_header("Authorization", authheader)
Is there a better / simpler way?
The trick is to create a password manager, and then tell urllib about it. Usually, you won't care about the realm of the authentication, just the host/url part. For example, the following:
password_mgr = urllib2.HTTPPasswordMgrWithDefaultRealm()
top_level_url = "http://example.com/"
password_mgr.add_password(None, top_level_url, 'user', 'password')
handler = urllib2.HTTPBasicAuthHandler(password_mgr)
opener = urllib2.build_opener(urllib2.HTTPHandler, handler)
request = urllib2.Request(url)
Will set the user name and password to every URL starting with top_level_url. Other options are to specify a host name or more complete URL here.
A good document describing this and more is at http://www.voidspace.org.uk/python/articles/urllib2.shtml#id6.
Yes, have a look at the urllib2.HTTP*AuthHandlers.
Example from the documentation:
import urllib2
# Create an OpenerDirector with support for Basic HTTP Authentication...
auth_handler = urllib2.HTTPBasicAuthHandler()
auth_handler.add_password(realm='PDQ Application',
uri='https://mahler:8092/site-updates.py',
user='klem',
passwd='kadidd!ehopper')
opener = urllib2.build_opener(auth_handler)
# ...and install it globally so it can be used with urlopen.
urllib2.install_opener(opener)
urllib2.urlopen('http://www.example.com/login.html')