Is there a way to search and retrieve the results from On-Line Encyclopedia of Integer Sequences (http://oeis.org) programmatically?
I have searched their site and the results are always returned in html. They do not seem to provide an API but in the policy statement they say its acceptable to access the database programmatically. But how to do it without screen scraping?
Thanks a lot for your help.
The OEIS now provides several points of access, not just ones using their internal format. These seem largely undocumented, so here are all of the endpoints that I have found:
https://oeis.org/search?fmt=json&q=<sequenceTerm>&start=<itemToStartAt>
Returns a JSON formatted response of the results found from the sequenceTerm given. If too many results were returned, count will be > 0 whilst results will be null. If no results were returned, count will be 0. itemToStartAt is used for pagination of results, as only a maximum of 10 are ever returned. This starts at 0. If you wanted to return a second page of results, this would equal 10. Information about what each of the entries means can be found here.
https://oeis.org/search?fmt=text&q=<sequenceTerm>&start=<itemToStartAt>
Exactly the same arguments as before, however this returns it in the OEIS internal format. Which is largely written about here. Unless your project requires it, I'd highly recommend using the JSON format over this.
https://oeis.org/search?fmt=<json|text>&q=id:A<sequenceNumber>
Will return a single result if the sequenceNumber is found. This is the suggested method for obtaining single sequences, as it appears to be far more optimised than some of the alternative methods that can be used as queries. Requests often take under a second. Alternative search query methods can be found on this page.
https://oeis.org/A<sequenceNumber>/graph?png=1
This endpoint can be used to grab the images used to graph the data points. Alternatively, setting png to equal to zero returns the HTML page containing a graph of it.
https://oeis.org/recent.txt
This returns a list of recently updated entries in the OEIS internal format. There are no parameters available, or JSON format, as this seems like a static text file that is simply being served to the client. Due to the length of replies from the OEIS database (for some sequences replies can take above five seconds), I'd highly recommend heavily caching requests and using the above endpoint to update them when they change.
A URL of the form http://oeis.org/search?fmt=text&q=2,5,14,50,233 gives a nicely formatted text output.
But it seems there is no way to get a single sequence in text form.
If you happen to use Mathematica, it sounds like the following notebook might help. It allows you to specify a sequence and automatically import a detailed list of matching entries from the OEIS:
http://www.brotherstechnology.com/math/oeis_mathematica.html
It looks like direct use of their CGI program is the only API they provide.
URL for Searching the Database
https://oeis.org/search?q=id:A000032&fmt=text
gives the plain text form of an entry in their internal format
https://oeis.org/eishelp1.html
Related
I am developing a web application (a dashboard of some sort), and on some pages I have a couple of data tables. Each data table has its own page number, filter and sort. Although the data is fetched asynchronously in the background (hence the page does not need to reload), but I need to store these information (page, filter, sort) so it can be persistent for example during a page refresh. As far as I know, there are two ways to store these information:
In the session storage (local storage is not an option, because I don't want these information to be persistent between sessions)
In the query string
Up until now I used to store these information in the query string because I thought it has some advantages: e.g. the user can copy, bookmark, or share the URL with a colleague to discuss some data which they find interesting (for example on the 10th page of a table after performing some filtering and sorting). With the session storage I cannot do this
But now I am extending the abilities of the filter function, and now the information is quite long to use in the query string, and may even exceed the limit (I think 2048 characters, right?). And also there may be more than a couple of tables on each page, and therefore the query string would even become longer.
So first I wanted to know what is the best practice in this situation
And second, is that feature (being able to copy/bookmark/share the page as is) really that important, or not?
Note: please note that the information that I'm talking about is nothing secret or sensitive. It's just table page number, table filter and table sort
The 2048 limit seems that's not really an actual limit, see What is the maximum possible length of a query string?
About the best approach - I've personally always hated that most "new websites" do not support the feature you care about, so I'd personally encourage you to support it!
And finally about the exact mechanism, firstly the query string approach will keep working for a lot longer than 2048 characters, but I can see that copy&pasting it might be unwieldy and depending on the media to share the URL it can introduce mistakes.
So, from the user's perspective, I think the best experience would be given by storing those searches on the backend side and enabling a shorter URL for permalinking/sharing/bookmarking.
This new URL could be obtained by the user via a specific UI button (Share/Permalink) so you save the search in that moment and return the URL, or (best experience but harder and costlier to implement) you can be saving it continuously and sending back to the UI the generated URL and use Javascript to replace the URL for the nicer version (either always or just when people copy it).
Also consider: it may be good enough to just keep using the query string :-)
I am new to MarkLogic and now I am trying to get some specific result of searching query.
More specifically, searching some word through search API and supposed to get a result of documents which include specific word.
No header information, no rank or any other meta data, just want to get documents as a result.
Is there any way to just one request and get documents as a result?
Or do I need to write some code to get specific result.
I'll be appreciated if you help me.
Thanks
If you are accessing MarkLogic from outside, I'd have a look at a POST call to /v1/search with an Accept header of multipart/mixed. Details should be described here: https://docs.marklogic.com/REST/POST/v1/search
If running inside MarkLogic, you could consider using the low-level cts:search, which indeed returns documents directly. Keep in mind though that it won't paginate results, and it is usually unwise to return more than about 50 to 100 documents at once. It would just hog memory, and not allow for parallel processing.
HTH!
I'm trying to work with MusicBrainz's API but I'm having some issues with the results of the search endpoint.
Let's have an example searching for Who's Who? - SIZE020 - Klack (Mix Two)
Searching from their site leads to this page, with an almost correct first result (probably because the 100% correct infos are not on the database at all).
Using the API leads to different situations which are causing some issues.
I made some different attempts with no success, even if I think I know enough of Lucene's syntax to write a successful query for this service.
Take 1 - empty results with the query "Who's Who? - SIZE020 - Klack (Mix Two)"
Take 2 - completely wrong results with the query Who's+Who%3F+-+SIZE020+-+Klack+(Mix+Two) (same result with the unescaped ? character)
Take 3 - empty results with the query "Who's" AND "Who?" AND "SIZE020" AND "Klack" AND "Mix" AND "Two"
Now, I know that SIZE020 shouldn't be in the query, but I don't want to deal with file names on client side so I'm just pushing the query to their service hoping that everything will work. And it works, but only if I query the service through their website, making me think that my query syntax is wrong and leaving me clueless.
Do you have any hint on why I get different results between website and xml API?
EDIT: as a side question, given a random file name, what's the better way to submit the query? I'm getting good result using the web version and submitting the typical mp3 filenames (like artist_-_title_(version).mp3 but I'm not getting anything good from my client.
Searching via the web service always uses the "indexed search with advanced query syntax" search method, this can't be changed.
We are trying to implement a complex search functionality in Rest service using coldfusion 10.
Something like projectid=1 and active=1 and (ManagerName contains John or ManagerName contains alfred)
One way of doing this is ?projectid=1&active=1&ManagerName=[John,Alfred]. However this does not serve my purpose as the ManagerName search will not return the required result. Also, as the number of search filter grows, the query string becomes tough to handle.
I tried to get an xml (with all search filter) as an input through HTTP Get Request, but that did not help, as GetHTTPRequestData() does not reflect the xml content.
Is there any means to pass an xml/json through an HTTP Get Request??
Will it be a bad practice, if xml is passed by a HTTP Post Request??
Are there any other options to pass complex filter param to a REST service??
i have gone through a lot of post on the site for similar question, but still could not find a solution to my problem.
GET should be idempotent it should not modify the state of the Resource. strictly limit get usage to Read operations
your POST triggers Resource Creation i.e along with payload (xml/json).It is very bad practice to use POST for search.
Also you should take-care of the Cache-Control as your GET requests might get cached and if your search is real time you might get stale data.
You can take example as stackoverflow itself
https://stackoverflow.com/questions/tagged/rest?sort=newest&pagesize=30
In above URl, path elements questions , tagged, rest derive subset of question resources.
The query parameters suggest filtering of those which met the creiteria.
I'm writing a webapp that uses the YouTube Code API to do specific types of searches. In this case, I'm trying to search for all videos that match a query, and which were uploaded between two dates. This document says I can use published-min and published-max parameters, while this one says I can use updated-min and updated-max.
Both of these parameter sets cause YouTube to return an error:
published-min returns "This service does not support the 'published-min parameter"
updated-min returns "This service does not support the 'updated-max' parameter"
With neither returns a correct result set.
How can I limit my result set to hits within a specified date range?
The Reference Guide for YouTube's Data API doesn't list anything that would suggest the possibility to filter on time interval in general.
The published-min argument is only advertised in the "User activity feeds" section which is something different and probably not the thing you wanted. Or is it?
The updated-min argument in your link is referenced in a generic gdata context. It looks like they intended to describe all the things common to all the specialized APIs, but somehow updated-min isn't available everywhere.
When it comes to your very problem. I would suggest sorting on time (orderby=published) and do the filtering on the client side. I know this is not the optimal way, but the only one I can see with what Google gives us.
youtube api v3 supports publishedAfter and publishedBefore parameters with search results. For example:
https://www.googleapis.com/youtube/v3/search?key={{YOUKEY}}&channelId={{CHANNELID}}&part=snippet,id&order=date&maxResults=50&publishedAfter=2014-09-21T00:00:00Z&publishedBefore=2014-09-22T02:00:00Z