I'm trying to get JSON file via get method in RESTClient.
Right now I'm trying
def url = 'http://urlurlurl'
def username = 'username'
def password = 'password'
def restClient = new RESTClient(url)
restClient.auth.basic(username, password)
render restClient
When I see what I get from restClient, is just prints
'groovyx.net.http.RESTClient#65333e2e'
Which is hard to understand.
Given that the url is a endpoint of a API get method, and contains JSON file, how can I retrieve JSON file so I can parse it and use that JSON file?
Also I'm trying this too
def url = 'http://urlurlurl'
def username = 'username'
def password = 'password'
def restClient = new RESTClient(url)
restClient.auth.basic(username, password)
//Adding get method
def jsonData = restClient.get(/* what value should I put in here?? */)
This gives me a forbbiden error that says:
Error 500: Internal Server Error
URI: JsonRender
Class: groovyx.net.http.HttpResponseException
Message: Forbidden
Any suggestions? Examples that uses get method in RESTClient will be nice.
The url should be the base url for your api. For example if we want to search some data from an api which complete url is http://localhost:9200/user/app/_search. So, we have base url as http://localhost:9200/ and the path to api is user/app/_search. Now the request looks like this
def client = new RESTClient( 'http://localhost:9200/' )
def resp = client.get( path : 'user/app/_search')
log.debug (resp.getContentAsString())
Hope this will work out.
Thanks,
Related
I am using a nodemcu esp8266 and the plan is to make a spotify api request to automate playing some songs, and I have a fully functional python script that does exactly that, but when I tried to convert it to upython it failed. I have now spent hours on this and have figured out that the problem is that the urequest for some reason only works on specific websites, for example if I try:
import urequests as requests
x = requests.get('https://www.spotify.com')
print(x.status_code)
i get a 302
but when I try:
import urequests as requests
x = requests.get('https://www.google.com')
print(x.status_code)
I get a 200. That problem percists with a few websites and with no valid response my urequests.put command does not return the things I need... any ideas how to fix it?
thank you in advance.
this is the code I am trying to run:
import urequests as requests
import ujson as json
refresh_token = "xxxxx"
base_64 = "xxxxx"
class Refresh:
def __init__(self):
self.refresh_token = refresh_token
self.base_64 = base_64
def refresh(self):
query = "https://accounts.spotify.com/api/token"
payload={"grant_type": "refresh_token", "refresh_token": refresh_token}
headers={'Authorization': 'Basic %s' % base_64}
data = (json.dumps(payload)).encode()
response = requests.post(query,
data=data,
headers=headers)
print(response.status_code)
print(response.reason)
a = Refresh()
a.refresh()
There are several things going on here that are going to cause your problems.
First, you're trying to send JSON data to the /api/token endpoint:
data = (json.dumps(payload)).encode()
response = requests.post(query,
data=data,
headers=headers)
...but according to the documentation, this endpoint excepts application/x-www-form-urlencoded data.
Second, while the documentation suggests you need to send a basicAuthorization header, in my experiments this evening I wasn't able to get that to work...but I was able to successfully refresh a token if I included the client id and secret in the request body.
You can see my forum post with this question here.
With that in mind, the following code seems to work on my esp8266 running micropython 1.18:
import urequests as requests
refresh_token = "..."
client_id = "..."
client_secret = "..."
class Refresh:
def __init__(self, refresh_token, client_id, client_secret):
self.refresh_token = refresh_token
self.client_id = client_id
self.client_secret = client_secret
def refresh(self):
url = "https://accounts.spotify.com/api/token"
data = "&".join(
[
"grant_type=refresh_token",
f"refresh_token={self.refresh_token}",
f"client_id={self.client_id}",
f"client_secret={self.client_secret}",
]
)
headers = {
"content-type": "application/x-www-form-urlencoded",
}
response = requests.post(url, data=data, headers=headers)
print(data)
print(response.status_code)
print(response.reason)
print(response.text)
a = Refresh(refresh_token, client_id, client_secret)
a.refresh()
This is the message I get when I am trying to get the token data using web.contents query from "VMware vRealize Automation API":
There was an error when processing the data in the dataset.
Please try again later or contact support. If you contact support, please provide these details.
Data source error
{"error":{"code":"ModelRefresh_ShortMessage_ProcessingError","pbi.error":{"code":"ModelRefresh_ShortMessage_ProcessingError","parameters":
{},"details":[{"code":"Message","detail":{"type":1,"value":"Web.Contents failed to get contents from 'https://xxxxxxxxx.com/identity/api/tokens'
(404): Not Found"}}],"exceptionCulprit":1}}}
Table: GetToken.
The url passed to the first parameter of Web.Contents (authUrl = "https://xxxxxxxxx.com/identity/api/tokens") is accessible but always return the HTTP ERROR 405, probably
because this API uses a a JSON object in the request body parameter with the users credentials to obtain the Response.
API
My query
The main issues:
Your API uses HTTP POST verses GET, so you need to set Options[Content]
You can get refresh errors on the service unless you use Options[RelativePath]
You can "opt-in" to handling errors for specific HTTP Status codes, combined with Value.MetaData you get more detailed error messages.
Let it generate JSON for you from records and lists by using Query or Content parameters see: docs: Web.Contents
This is equivalent to your curl POST request
let
BaseUrl = "https://www.example.com",
Options = [
RelativePath = "/identity/api/tokens",
Headers = [
Accept="application/json"
],
Content = [
username = "username",
password = "password",
tenant = "tenant"
],
ManualStatusHandling = {400, 405}
],
// wrap 'Response' in 'Binary.Buffer' if you are using it multiple times
response = Web.Contents(BaseUrl, Options),
buffered = Binary.Buffer(response),
response_metadata = Value.Metadata(response),
status_code = response_metadata[Response.Status],
from_json = Json.Document(final_result)
in
from_json
I have related Web.Contents examples here, like Chaining Web.Contents requests: ninmonkeys.com/Power-Query-Custom-Functions-Cheat-Sheet
I am trying to obtain an authentication ticket using a POST request with 3 parameters(user,pass,realm) to access Proxmox API Server to be parsed for further queries.
As I am writing the code in Groovy Script for a parameter in a Jenkins job, I am not getting much help in terms of errors. I have tried the POST request on insomnia and it has no problems.
I am still very new to GroovyScript any pointers in the right direction is much appreciated.
def url = new URL("https://$HOST/api2/json/access/ticket")
def connection = url.openConnection()
connection.setDoOutput(true)
connection.setRequestMethod("POST")
connection.setRequestProperty("Content-Type", "application/json")
connection.setRequestProperty('Username', '$USER')
connection.setRequestProperty('Password', '$PASS')
connection.setRequestProperty('Realm', '$REALM')
def requestCode = connection.getResponseCode
Not need connection.setRequestProperty('Realm', '$REALM'), using Username#realm
try this, write to API:
def url = new URL("https://$HOST/api2/json/access/ticket")
def connection = url.openConnection()
connection.setDoOutput(true)
connection.setRequestMethod("POST")
connection.setRequestProperty("Content-Type", "application/json")
connection.setRequestProperty('Password', '$PASS')
connection.setRequestProperty('Username', '$USER' + '#' + '$REALM')
def requestCode = connection.getResponseCode
For example:
the url is http://www.wandoujia.com/search?key=saber
It will be redirected to the new url http://www.wandoujia.com/search/3161097853842468421.
I want to get the new url in the process_request() of scrapy RedirectMiddleware?
Following is my code:
class RedirectMiddleware(object):
def process_request(self, request, spider):
new_url = request.url
logging.debug('new_url = ' + new_url)
logging.debug('****************************')
patterns = spider.request_pattern
logging.debug(patterns)
for pattern in patterns:
obj = re.match(pattern, new_url)
if obj:
return Request(new_url)
ps:the request.url is the old url. I want to get the new url correctly.
Try replacing the default middleware with something like this (the method you're looking for is the process_response because the response "contains the redirection")
class CustomRedirectMiddleware(RedirectMiddleware):
def process_response(self, request, response, spider):
redirected = super(CustomRedirectMiddleware, self).process_response(
request, response, spider)
if isinstance(redirected, request.__class__):
print("Original url: <{}>".format(request.url))
print("Redirected url: <{}>".format(redirected.url))
return redirected
return response
Django 1.9.6
I'd like to write some unit test for checking redirection.
Could you help me understand what am I doing wrongly here.
Thank you in advance.
The test:
from django.test import TestCase
from django.core.urlresolvers import reverse
from django.http.request import HttpRequest
from django.contrib.auth.models import User
class GeneralTest(TestCase):
def test_anonymous_user_redirected_to_login_page(self):
user = User(username='anonymous', email='vvv#mail.ru', password='ttrrttrr')
user.is_active = False
request = HttpRequest()
request.user = user
hpv = HomePageView()
response = hpv.get(request)
self.assertRedirects(response, reverse("auth_login"))
The result:
ERROR: test_anonymous_user_redirected_to_login_page (general.tests.GeneralTest)
Traceback (most recent call last):
File "/home/michael/workspace/photoarchive/photoarchive/general/tests.py", line 44, in test_anonymous_user_redirected_to_login_page
self.assertRedirects(response, reverse("auth_login"))
File "/home/michael/workspace/venvs/photoarchive/lib/python3.5/site-packages/django/test/testcases.py", line 326, in assertRedirects
redirect_response = response.client.get(path, QueryDict(query),
AttributeError: 'HttpResponseRedirect' object has no attribute 'client'
Ran 3 tests in 0.953s
What pdb says:
-> self.assertRedirects(response, reverse("auth_login"))
(Pdb) response
<HttpResponseRedirect status_code=302, "text/html; charset=utf-8", url="/accounts/login/">
You need to add a client to the response object. See the updated code below.
from django.test import TestCase, Client
from django.core.urlresolvers import reverse
from django.http.request import HttpRequest
from django.contrib.auth.models import User
class GeneralTest(TestCase):
def test_anonymous_user_redirected_to_login_page(self):
user = User(username='anonymous', email='vvv#mail.ru', password='ttrrttrr')
user.is_active = False
request = HttpRequest()
request.user = user
hpv = HomePageView()
response = hpv.get(request)
response.client = Client()
self.assertRedirects(response, reverse("auth_login"))
Looks like you are directly calling your view's get directly rather than using the built-in Client. When you use the test client, you get your client instance back in the response, presumably for cases such as this where you want to check/fetch a redirect.
One solution would be to use the client to fetch the response from your view. Another is to stick a client in the response as mentioned above.
A third option is tell assertRedirects not to fetch the redirect. There is no need for client if you don't ask the assertion to fetch the redirect. That's done by adding fetch_redirect_response=False to your assertion.