Whats is the python urllib equivallent of
curl -u username:password status="abcd" http://example.com/update.json
I did this:
handle = urllib2.Request(url)
authheader = "Basic %s" % base64.encodestring('%s:%s' % (username, password))
handle.add_header("Authorization", authheader)
Is there a better / simpler way?
The trick is to create a password manager, and then tell urllib about it. Usually, you won't care about the realm of the authentication, just the host/url part. For example, the following:
password_mgr = urllib2.HTTPPasswordMgrWithDefaultRealm()
top_level_url = "http://example.com/"
password_mgr.add_password(None, top_level_url, 'user', 'password')
handler = urllib2.HTTPBasicAuthHandler(password_mgr)
opener = urllib2.build_opener(urllib2.HTTPHandler, handler)
request = urllib2.Request(url)
Will set the user name and password to every URL starting with top_level_url. Other options are to specify a host name or more complete URL here.
A good document describing this and more is at http://www.voidspace.org.uk/python/articles/urllib2.shtml#id6.
Yes, have a look at the urllib2.HTTP*AuthHandlers.
Example from the documentation:
import urllib2
# Create an OpenerDirector with support for Basic HTTP Authentication...
auth_handler = urllib2.HTTPBasicAuthHandler()
auth_handler.add_password(realm='PDQ Application',
uri='https://mahler:8092/site-updates.py',
user='klem',
passwd='kadidd!ehopper')
opener = urllib2.build_opener(auth_handler)
# ...and install it globally so it can be used with urlopen.
urllib2.install_opener(opener)
urllib2.urlopen('http://www.example.com/login.html')
Related
I am fetching google photos from my account using Google Photo API. Now there is a need for me to execute that php file via terminal, but the problem is that I can't authenticate with Google API in doing so. Is there a way to do this, and if yes, then how shall it be done?
Yes, it is possible, you need an interactive login for the first authentication but then you can save the token and refresh it automatically as required.
I have implemented this class in Python to do just that.
from requests.adapters import HTTPAdapter
from requests_oauthlib import OAuth2Session
from pathlib import Path
from urllib3.util.retry import Retry
from typing import List, Optional
from json import load, dump, JSONDecodeError
import logging
log = logging.getLogger(__name__)
# OAuth endpoints given in the Google API documentation
authorization_base_url = "https://accounts.google.com/o/oauth2/v2/auth"
token_uri = "https://www.googleapis.com/oauth2/v4/token"
class Authorize:
def __init__(
self, scope: List[str], token_file: Path,
secrets_file: Path, max_retries: int = 5
):
""" A very simple class to handle Google API authorization flow
for the requests library. Includes saving the token and automatic
token refresh.
Args:
scope: list of the scopes for which permission will be granted
token_file: full path of a file in which the user token will be
placed. After first use the previous token will also be read in from
this file
secrets_file: full path of the client secrets file obtained from
Google Api Console
"""
self.max_retries = max_retries
self.scope: List[str] = scope
self.token_file: Path = token_file
self.session = None
self.token = None
try:
with secrets_file.open('r') as stream:
all_json = load(stream)
secrets = all_json['installed']
self.client_id = secrets['client_id']
self.client_secret = secrets['client_secret']
self.redirect_uri = secrets['redirect_uris'][0]
self.token_uri = secrets['token_uri']
self.extra = {
'client_id': self.client_id,
'client_secret': self.client_secret}
except (JSONDecodeError, IOError):
print('missing or bad secrets file: {}'.format(secrets_file))
exit(1)
def load_token(self) -> Optional[str]:
try:
with self.token_file.open('r') as stream:
token = load(stream)
except (JSONDecodeError, IOError):
return None
return token
def save_token(self, token: str):
with self.token_file.open('w') as stream:
dump(token, stream)
self.token_file.chmod(0o600)
def authorize(self):
""" Initiates OAuth2 authentication and authorization flow
"""
token = self.load_token()
if token:
self.session = OAuth2Session(self.client_id, token=token,
auto_refresh_url=self.token_uri,
auto_refresh_kwargs=self.extra,
token_updater=self.save_token)
else:
self.session = OAuth2Session(self.client_id, scope=self.scope,
redirect_uri=self.redirect_uri,
auto_refresh_url=self.token_uri,
auto_refresh_kwargs=self.extra,
token_updater=self.save_token)
# Redirect user to Google for authorization
authorization_url, _ = self.session.authorization_url(
authorization_base_url,
access_type="offline",
prompt="select_account")
print('Please go here and authorize,', authorization_url)
# Get the authorization verifier code from the callback url
response_code = input('Paste the response token here:')
# Fetch the access token
self.token = self.session.fetch_token(
self.token_uri, client_secret=self.client_secret,
code=response_code)
self.save_token(self.token)
# note we want retries on POST as well, need to review this once we
# start to do methods that write to Google Photos
retries = Retry(total=self.max_retries,
backoff_factor=0.1,
status_forcelist=[500, 502, 503, 504],
method_whitelist=frozenset(['GET', 'POST']),
raise_on_status=False)
self.session.mount('https://', HTTPAdapter(max_retries=retries))
I have just tried
===============================
# Give user the file requested
url = "http://superhost.gr/data/files/%s" % realfile
username = getpass.getuser()
password = getpass.getpass()
r = requests.get( url, auth = (username, password) ) # ask user for authentication data
r.raise_for_status()
===============================
as well as input() for both user & pass combo but iam not getting in chrome the basic pop-up HTTP auth window.
Any idea why?
How can i ASK the user for http auth data and store them isntead of giving them to the script?
You can try to use below code to ask user for credentials and send them via HTTP request
from requests.auth import HTTPBasicAuth
import requests
import sys
username = sys.argv[1]
password = sys.argv[2]
realfile = sys.argv[3]
url = "http://superhost.gr/data/files/%s" % realfile
r = requests.get( url, auth=HTTPBasicAuth(username, password))
print(r.status_code)
This script user can run as python script.py username password filename
# Give user the file requested
print('''<meta http-equiv="refresh"
content="5;url=http://superhost.gr/data/files/%s">''' % realfile)
authuser = os.environ.get( 'REMOTE_USER', 'Άγνωστος' )
print( authuser )
================================
Trying this, feels liek i'm almost there except that when printing the value of authuser variable it default to "Άγνωστος" meaning not there.
is there any other way i can grab what the user gave a auth login info?
If someone could point me in the right direction I would be very grateful. I am trying to use the requests module to interact with Reddit's APIs, but for some reason I am getting HTTP status 403 codes. I am able to successfully log in I believe (since I get HTTP 200), but I cannot perform a successful API request. I am very new to this so please use small words. Thanks.
import requests
import json
from pprint import pprint
# URLs
url_base = r'https://www.reddit.com'
url_login = '/api/login'
url_me = '/api/v1/me'
# Credentials
user = '__kitten_mittens__'
passwd = 'password'
params = {'user': user,
'passwd': passwd,
'api_type': 'json',}
headers = {'user-agent': '/u/__kitten_mittens__ practice api bot'}
# Set up the session/headers
client = requests.session()
client.headers = headers
response = client.post(url_base + url_login, data = params)
j = json.loads(response.content.decode('utf-8'))
mh = j['json']['data']['modhash']
print("The modhash for {USER} is {mh}".format(USER=user, mh=mh))
# THIS CODE GIVES HTTP STATUS CODE 403
response = client.get(url_base + url_me)
me_json = json.loads(r.content.decode('utf-8'))
pprint(me_json)
EDIT: I fixed the missing single quote from the password field. That was not part of the problem.
I need to build webcrawler for internal usage and I need to login into administration area. I'm trying to use requests lib, tried this ways:
import urllib.parse
import requests
base_url = "https://target.url"
data = ({'login': 'login', 'pass': 'password'})
params = urllib.parse.urlencode(data)
r = requests.post(base_url, data=params)
print(r.text)
and
import requests
base_url = "https://target.url"
r = requests.post(base_url, auth=('login', 'password')
print(r.text)
but in both cases r.text returns me login page content same as if I try to get any other page after auth code:
req = requests.get("https://target.url/smth")
What I lose sight of? I have ideas:
chain of hidden redirections from https://target.url to real login page, so I send auth info to wrong url
I don't send additional required info (like cookies e.g.)
Could you please comment? How can I gather required for login information?
In my case problem was in 'Referer' parameter in headers, which is required but wasn't specified
I am developing mobile apps using rhodes. I want to access private repo of github. I am having only username and password.
How to get token of given username and password.
Once you have only login and password you can use them using basic auth. First of all, check if this code shows you json data of desired repo. Username and password must be separated by a colon.
curl -u "user:pwd" https://api.github.com/repos/user/repo
If succeeded you should consider doing this request from code.
import urllib2
import json
from StringIO import StringIO
import base64
username = "user#example.com"
password = "naked_password"
req = urllib2.Request("https://api.github.com/repos/user/repo")
req.add_header("Authorization", "Basic " + base64.urlsafe_b64encode("%s:%s" % (username, password)))
req.add_header("Content-Type", "application/json")
req.add_header("Accept", "application/json")
res = urllib2.urlopen(req)
data = res.read()
repository = json.load(StringIO(data))
You should use oauth instead: http://developer.github.com/v3/oauth/
Github users can create Personal Access Tokens at their application settings. You can use this token as an alternative to username/password in basic http authentication to call the API or to access private repositories on the github website.
Simply use a client that supports basic http authentication. Set the username equal to the token, and the password equal to x-oauth-basic. For example with curl:
curl -u <token>:x-oauth-basic https://api.github.com/user
See also https://developer.github.com/v3/auth/.
Send A POST request to /authorizations
With headers
Content-Type: application/json
Accept: application/json
Authorization: Basic base64encode(<username>:<password>)
But remember to take Two factor Authentication in mind
https://developer.github.com/v3/auth/#working-with-two-factor-authentication
Here You will receive a token which can be used for further request
Follow this guide at help.github.com. It describes how to find your api-token (it's under "Account Settings" > "Account Admin") and configuring git so it uses the token.
Here is the code to use GitHub Basic Authentication in JavaScript
let username = "*******";
let password = "******";
let auth = "Basic " + new Buffer(username + ":" + password).toString("base64");
var options = {
host: 'api.github.com',
path: '/search/repositories?q=google%20maps%20api',
method: 'GET',
headers: {
'user-agent': 'node.js',
"Authorization": auth
}
};
var request = https.request(options, function (res) {
}));