I'm attempting to use the wikipedia API. However, I'm finding that I can only retrieve recent changes for english pages. For example:
https://en.wikipedia.org/w/api.php?action=query&list=recentchanges&rctitle=Battle%20of%20Palma&format=json
returns:
{
"batchcomplete": "",
"query": {
"recentchanges": [{
"type": "edit",
"ns": 0,
"title": "Battle of Palma",
"pageid": 67226819,
"revid": 1061578435,
"old_revid": 1049512969,
"rcid": 1455128770,
"timestamp": "2021-12-22T15:19:02Z"
}]
}
}
However, the same request with the same page in Spanish returns an empty array:
https://en.wikipedia.org/w/api.php?action=query&list=recentchanges&rctitle=Batalla%20de%20Palma&format=json
{"batchcomplete":"","query":{"recentchanges":[]}}
Even though both of the pages exist:
English: https://en.wikipedia.org/wiki/Battle_of_Palma
Spanish: https://es.wikipedia.org/wiki/Batalla_de_Palma
I was using the incorrect API call. The above provides any changes across wikipedia. I needed to use the history call found here: https://www.mediawiki.org/wiki/API:REST_API/Reference#Get_page_history
Related
shopware 6 admin api patch - why it's failing? I get error "Only single write operations are supported"
Following is api for rule-condition entity in the database, I update it with Id.
For same api get method is working!
url: api/rule-condition/dbb0d904c7c14860a9a94cf26b94eca6
method: patch
json body
[
{
"op": "replace",
"path": "/data/attributes/value/email",
"value": "test#gmail.com"
}
]
response:
{
"errors": [
{
"code": "0",
"status": "400",
"title": "Bad Request",
"detail": "Only single write operations are supported. Please send the entities one by one or use the /sync api endpoint.",
.......
I also tried changing json body to following
{
"data": {
"attributes": {
"value": {
"email": "test#gmail.com"
}
}
} }
Still it's not updating. Can somebody check and let me know what am i missing?
Documentation I followed:
https://shopware.stoplight.io/docs/admin-api/ZG9jOjEyMzA4NTQ5-writing-entities
This website has all apis and example methods. https://swagger.docs.fos.gg/,
rule-condition entity can also be found there.
Btw : I used postman for testing api
You're passing an array of objects in the request body, suggesting you want to update multiple records, but the endpoint only supports updating a single record. The correct payload in your case should look like this:
{
"value": {
"operator": "=",
"email": "test#gmail.com"
}
}
Notice that value is a json field and not only includes a single value. The exact content and the names of the properties of value depend on the type of condition used and usually it also includes the operator used in the condition.
I'm using the categorymembers generator in Wikipedia API to retrieve all pages in a category efficiently, for example:
https://en.wikipedia.org/w/api.php?action=query&format=json&prop=revisions&generator=categorymembers&rvprop=content&rvslots=*&gcmtitle=Category%3AAirports%20in%20Chad&gcmlimit=5000
(sandbox link)
But some articles are missing content, for instance page id 40376021 "Tchagen Airport":
"40376021": {
"pageid": 40376021,
"ns": 0,
"title": "Tchagen Airport"
},
However, when I retrieve the individual page, it does have content:
https://en.wikipedia.org/w/api.php?action=query&format=json&prop=revisions&titles=Tchagen%20Airport&formatversion=2&rvprop=content&rvslots=*
(sandbox link)
{
"batchcomplete": true,
"query": {
"pages": [
{
"pageid": 40376021,
"ns": 0,
"title": "Tchagen Airport",
"revisions": [
{
"slots": {
"main": {
"contentmodel": "wikitext",
"contentformat": "text/x-wiki",
"content": "page content here (long, ommited)"
}
}
}
]
}
]
}
}
I don't know what's going on here. It's not a recently created page so it can't be a caching issue. How to ensure that content for all pages can be seen in bulk results?
You only see the content of the first 50 pages.
Thus, it's best practice to set gcmlimit=50 in your request:
https://en.wikipedia.org/w/api.php?action=query&format=json&prop=revisions&generator=categorymembers&rvprop=content&rvslots=*&gcmtitle=Category%3AAirports%20in%20Chad&gcmlimit=50
In the response, you will see a continue>gcmcontinue value. Use this value in a next request to get the content of another 50 pages:
https://en.wikipedia.org/w/api.php?action=query&format=json&prop=revisions&generator=categorymembers&rvprop=content&rvslots=*&gcmtitle=Category%3AAirports%20in%20Chad&gcmlimit=50&gcmcontinue=page|482a402a042a3a4c48464c50011001dcc2dc0a|21079773
If the response does not contain a continue value, you know that you retrieved all data.
I am working on a research publication and collaboration project that has a literature search feature in it.
Google Scholar seems like it will work since it is an open source tool but, when I researched Google Scholar, I could not find any information about it having an API.
Please let me know if there is any API for Google Scholar that is valid.
TIA.
There's no official Google Scholar API. There are third-party solutions like free scholarly Python package which supports profile, author, cite and organic results (search_pubs seems to be the method to get organic results, although method name confuses me).
Note that by using scholarly constantly without a requests rate limit, Google may block your IP (mentioned by #RadioControlled). Use it wisely.
Alternatively, there's a Google Scholar API from SerpApi which is a paid API with a free plan that supports organic, cite, profile, author results and bypasses all the blocks on SerpApi backend so it won't block your IP.
Example code to parse profile results using scholarly using search_by_keyword method:
import json
from scholarly import scholarly
# will paginate to the next page by default
authors = scholarly.search_keyword("biology")
for author in authors:
print(json.dumps(author, indent=2))
# part of the output:
'''
{
"container_type": "Author",
"filled": [],
"source": "SEARCH_AUTHOR_SNIPPETS",
"scholar_id": "LXVfPc8AAAAJ",
"url_picture": "https://scholar.google.com/citations?view_op=medium_photo&user=LXVfPc8AAAAJ",
"name": "Eric Lander",
"affiliation": "Broad Institute",
"email_domain": "",
"interests": [
"Biology",
"Genomics",
"Genetics",
"Bioinformatics",
"Mathematics"
],
"citedby": 552013
}
... other author results
'''
Example code to parse organic results using Google Scholar Profile Results API from SerpApi:
import json
from serpapi import GoogleScholarSearch
# search parameters
params = {
"api_key": "Your SerpApi API key",
"engine": "google_scholar_profiles",
"hl": "en", # language
"mauthors": "biology" # search query
}
search = GoogleScholarSearch(params)
results = search.get_dict()
# only first page results
for result in results["profiles"]:
print(json.dumps(result, indent=2))
# part of the output:
'''
{
"name": "Masatoshi Nei",
"link": "https://scholar.google.com/citations?hl=en&user=VxOmZDgAAAAJ",
"serpapi_link": "https://serpapi.com/search.json?author_id=VxOmZDgAAAAJ&engine=google_scholar_author&hl=en",
"author_id": "VxOmZDgAAAAJ",
"affiliations": "Laura Carnell Professor of Biology, Temple University",
"email": "Verified email at temple.edu",
"cited_by": 384074,
"interests": [
{
"title": "Evolution",
"serpapi_link": "https://serpapi.com/search.json?engine=google_scholar_profiles&hl=en&mauthors=label%3Aevolution",
"link": "https://scholar.google.com/citations?hl=en&view_op=search_authors&mauthors=label:evolution"
},
{
"title": "Evolutionary biology",
"serpapi_link": "https://serpapi.com/search.json?engine=google_scholar_profiles&hl=en&mauthors=label%3Aevolutionary_biology",
"link": "https://scholar.google.com/citations?hl=en&view_op=search_authors&mauthors=label:evolutionary_biology"
},
{
"title": "Molecular evolution",
"serpapi_link": "https://serpapi.com/search.json?engine=google_scholar_profiles&hl=en&mauthors=label%3Amolecular_evolution",
"link": "https://scholar.google.com/citations?hl=en&view_op=search_authors&mauthors=label:molecular_evolution"
},
{
"title": "Population genetics",
"serpapi_link": "https://serpapi.com/search.json?engine=google_scholar_profiles&hl=en&mauthors=label%3Apopulation_genetics",
"link": "https://scholar.google.com/citations?hl=en&view_op=search_authors&mauthors=label:population_genetics"
},
{
"title": "Phylogenetics",
"serpapi_link": "https://serpapi.com/search.json?engine=google_scholar_profiles&hl=en&mauthors=label%3Aphylogenetics",
"link": "https://scholar.google.com/citations?hl=en&view_op=search_authors&mauthors=label:phylogenetics"
}
],
"thumbnail": "https://scholar.googleusercontent.com/citations?view_op=small_photo&user=VxOmZDgAAAAJ&citpid=3"
}
... other results
'''
There is a dedicated Scrape historic Google Scholar results using Python blog post of mine at SerpApi which shows how to scrape historic 2017-2021 Organic, Cite Google Scholar results to CSV, SQLite.
Disclaimer, I work for SeprApi
A quick search shows that others are trying to implement such APIs, but Google does not provide one. It is not clear whether this is legal, see for instance
How to get permission from Google to use Google Scholar Data, if needed?.
I am new to Dialogflow so my question may be too simple. However, I do not understand what is the purpose of naming actions in Dialogflow. I have watched videos on youtube and people in them are using actions when they have a webhook. For example they may have an if condition in their source code
(e.g. in python
if action == 'action_name':
...
)
which executes something particular in this case.
However the json output which is retrieved by the source code has the following form:
{
"id": "123d9e8e-314f-451b-8b15-5e3b55baa980",
"timestamp": "2018-03-16T17:03:05.987Z",
"lang": "en",
"result": {
"source": "agent",
"resolvedQuery": "Hello",
"action": "input.welcome",
"actionIncomplete": false,
"parameters": {},
"contexts": [],
"metadata": {
"intentId": "effe6b2b-3372-4f89-882f-ff937b2b2abb",
"webhookUsed": "false",
"webhookForSlotFillingUsed": "false",
"intentName": "Welcome"
},
"fulfillment": {
"speech": "Hello, how can I help you?",
"messages": [
{
"type": 0,
"speech": "Hello, how can I help you?"
}
]
},
"score": 1
},
"status": {
"code": 200,
"errorType": "success",
"webhookTimedOut": false
},
"sessionId": "491d57cb-0af2-45ac-a658-9e47ec6658ce",
"alternativeResultsFromKnowledgeService": {}
}
Since the json data contains the IntentName why to bother naming an unique action for this specific intent when you can get directly the name of the intent in your json?
I tend to think of this in two ways, depending on exactly what I'm building. (Or sometimes a combination of these two ways.)
The Intent Name is a human-usable name, while the Action is something that is more intended for use by the webhook and more directly maps to a function.
Since you can have more than one Intent use the same Action, it can be convenient to map a few different ways the user may say something (and the parameters they may send along with them) to the same method. While you could do that by listing all the different Intent names in your code, it is easier to do that on the Dialogflow side.
In truth - use whatever works best for you. I tend to name my Intents and my Actions very similarly, but do branching based on what makes the most sense for the code (which sometimes also includes other values that may be sent).
I'm using Google Custom Search API to search for images. The search result is always 0 when I request for images. I'm following the documentation at the link below:
https://developers.google.com/custom-search/json-api/v1/reference/cse/list
according to the docs, by specifying searchType=images, the api only looks for images.
Here's what my url looks like:
https://www.googleapis.com/customsearch/v1?key=[API_Key]&cx=017576662512468239146:omuauf_lfve&searchType=image&q=cars
and the result looks like below:
{
"kind": "customsearch#search",
"url": {
"type": "application/json",
"template": "https://www.googleapis.com/customsearch/v1?q={searchTerms}&num={count?}&start={startIndex?}&lr={language?}&safe={safe?}&cx={cx?}&cref={cref?}&sort={sort?}&filter={filter?}&gl={gl?}&cr={cr?}&googlehost={googleHost?}&c2coff={disableCnTwTranslation?}&hq={hq?}&hl={hl?}&siteSearch={siteSearch?}&siteSearchFilter={siteSearchFilter?}&exactTerms={exactTerms?}&excludeTerms={excludeTerms?}&linkSite={linkSite?}&orTerms={orTerms?}&relatedSite={relatedSite?}&dateRestrict={dateRestrict?}&lowRange={lowRange?}&highRange={highRange?}&searchType={searchType}&fileType={fileType?}&rights={rights?}&imgSize={imgSize?}&imgType={imgType?}&imgColorType={imgColorType?}&imgDominantColor={imgDominantColor?}&alt=json"
},
"queries": {
"request": [
{
"title": "Google Custom Search - cars",
"totalResults": "0",
"searchTerms": "cars",
"count": 10,
"inputEncoding": "utf8",
"outputEncoding": "utf8",
"safe": "off",
"cx": "017576662512468239146:omuauf_lfve",
"searchType": "image"
}
]
},
"searchInformation": {
"searchTime": 0.049329,
"formattedSearchTime": "0.05",
"totalResults": "0",
"formattedTotalResults": "0"
}
}
If I remove searchType from the request, I get results back in the form of web pages. What is wrong here?
Your Custom Search Engine might have Image Search disabled. The CSE API returns 0 results if the searchType requested is disabled.
You can enable it by visiting https://cse.google.com/cse/all, opening your search engine, and switching the "Image Search" toggle to ON.
I'm 2 years delayed but i found the solution.
you need to go at your Custom Search Engine Dashboard and turn on the property "Search the entire Web" Or something like that.
When you get that property activated you must be able to see results