The soup object that I get from one url looks like:
<html><body><p>[
{
"id": "...",
"Version": "...",
"Product": ".",
},
{
"id": "...",
"Version": "...",
"Product": ".",
}]
</p></body></html>
My goal is to parse the objects and store them in an array.
I tried
for res in soup.find("body").find_all('p'):
print(res)
But the single res contains all objects instead of being a separate object.
How can I parse each object separately?
Here a example :
from bs4 import BeautifulSoup
import ast
html_string = """
<html><body><p>[
{
"id": "...",
"Version": "...",
"Product": ".",
},
{
"id": "...",
"Version": "...",
"Product": ".",
}
]
</p></body></html>
"""
soup = BeautifulSoup(html_string, "html.parser")
samples = soup.find_all("p")
for row in samples:
liststring=row.get_text()
# convert this string into list
res = ast.literal_eval(liststring)
print(res)
for x in res:
print(x)
Related
I would like to get the value from the response based on a condition and store it to a variable.
In the below JSON, I would like to store the value when the name matches to something I prefer. Is there a way to achieve this using Karate API?
{
"results": [
{
"name": "Sample1",
"email": "sample1#text.com",
"id": "U-123"
},
{
"name": "Sample2",
"email": "sample2#text.com",
"id": "U-456"
},
{
"name": "Sample3",
"email": "sample3#text.com",
"id": "U-789"
}
]
}
So after reading the comment, my interpretation is to "find" the id where name is Sample2. Easy, just use a filter() operation, refer the docs: https://github.com/karatelabs/karate#jsonpath-filters
Instead of using a filter, I'm using the JS Array find() method as a very concise example below:
* def response =
"""
{ "results": [
{ "name": "Sample1", "email": "sample1#text.com", "id": "U-123" },
{ "name": "Sample2", "email": "sample2#text.com", "id": "U-456" },
{ "name": "Sample3", "email": "sample3#text.com", "id": "U-789" }
]
}
"""
* def id = response.results.find(x => x.name == 'Sample2').id
* match id == 'U-456'
Take some time to understand how it works. Talk to someone who knows JS if needed.
Writing test for a little API. Test for GET method working, but for create an error is being called. What could be the problem? i may guess the wrong data format is using.
class CoursesTest(APITestCase):
def setUp(self):
self.course_url = reverse('course-group')
User.objects.create(username='test111', password='123456')
def test_courses_post(self):
data = {
"name": "Blasssbla",
"description": "blabla",
"logo": "img",
"category": {
"name": "Baling",
"imgpath": "img"
},
"contacts": [
{
"status": 1
}
],
"branches": [
{
"latitude": "2131ssss2321",
"longitude": "12321321",
"address": "Osssssh"
}
]
}
self.response = self.client.post(self.course_url, data)
self.assertEqual(self.response.status_code, status.HTTP_201_CREATED)
Error:
AssertionError: Test data contained a dictionary value for key 'category', but multipart uploads do not support nested data. You may want to consider using format='json' in this test case.
If your test data is JSON, you should add format="json" to self.client.post.
class CoursesTest(APITestCase):
def setUp(self):
self.course_url = reverse('course-group')
User.objects.create(username='test111', password='123456')
def test_courses_post(self):
data = {
"name": "Blasssbla",
"description": "blabla",
"logo": "img",
"category": {
"name": "Baling",
"imgpath": "img"
},
"contacts": [
{
"status": 1
}
],
"branches": [
{
"latitude": "2131ssss2321",
"longitude": "12321321",
"address": "Osssssh"
}
]
}
self.response = self.client.post(self.course_url, data, format="json")
self.assertEqual(self.response.status_code, status.HTTP_201_CREATED)
I'm using Karate loops to generate dynamic Json.
One of my test data contains array within an object. When I use karate loop on this data, the array is returned as an object.
* def fun = function(i){ return { "name": "userName"+ i, "email": "Tester#test.com", "id": "ID"+i, "testSheet": [{"sheetId" : "123"}]} }
* def jsonBody = karate.repeat(3, fun)
* print jsonBody
I'm expecting below:
[
{
"name": "userName0",
"email": "Tester#test.com",
"id": "ID1",
"testSheet": [
{
"sheetId": "123"
}
]
}
]
But it returns this
[
{
"name": "userName0",
"email": "Tester#test.com",
"id": "ID1",
"testSheet": {
"0": {
"sheetId": "123"
}
}
}
]
Expected path: jsonBody[0].testSheet[0].sheetId
Actual path: jsonBody[0].testSheet.0.sheetId
Unfortunately this is a bug, which will be fixed in the next release: https://github.com/intuit/karate/issues/1187
This is the work-around. Define the array part outside the JS block for now, and use copy:
* def testSheet = [{"sheetId" : "123"}]
* def fun = function(i){ return { "name": "userName"+ i, "email": "Tester#test.com", "id": "ID"+i, "testSheet": testSheet } }
* copy jsonBody = karate.repeat(3, fun)
* print jsonBody
In the below JSON response, I need to extract the 'cid' for the record that has the 'nationalityDecription' as 'USA'. By using this query as a reference, I used the below loc in the karate feature file, but 1st line itself fails with syntax error(tried different combinations). For now, I'm using the custom javascript as a workaround which is working fine. I need help to check if i'm missing anything in syntax. Thanks
Response:
{
"header": {
"Id": "12345678",
"timeStamp": "2018-09-17T10:09:812.000"
},
"dataRecords": [
{
"cid": "31H678",
"cidMeta": "00",
"nationalityDecription": "CHINA"
},
{
"cid": "31S421",
"cidMeta": "01",
"nationalityDecription": "USA"
}
]
}
Feature file:
* def record= $response.dataRecords[?(#.nationalityDecription=='USA')]
* def cid = record.cid
* def response = { "header": { "Id": "12345678", "timeStamp": "2018-09-17T10:09:812.000" }, "dataRecords": [ { "cid": "31H678", "cidMeta": "00", "nationalityDecription": "CHINA" }, { "cid": "31S421", "cidMeta": "01", "nationalityDecription": "USA" } ] }
* def cid = get[0] response.dataRecords[?(#.nationalityDecription=='USA')].cid
* match cid == '31S421'
Part of json object as below, need to extract questionId and check whether it is number:
{
"rootProductQuestionId": "17",
"questionAnswers": [
{
"question": "",
"questionId": "17",
"answer": ""
}
],
"productId": "8",
"notes": ""
}
You have an option for isNumber() check, something like:
import groovy.json.jsonSlurper
def jsonSlurper = new JsonSlurper()
def json = jsonSlurper.parseText('{"rootProductQuestionId":"17","questionAnswers":[{"question":"","questionId":"17","answer":""}],"productId":"8","notes":""}')
return json.questionAnswers.questionId[0].isNumber()