I am trying to get the twitter name, website, and markets for any cryptocurrency listed on coinmarketcap.
For example:
https://coinmarketcap.com/currencies/bitcoin shows all of the data that I need but how would I parse the data listed on that page to get the twitter name and website associated with bitcoin?
Don't try to parse the data on that page if you are trying to use it in an application. If you are doing it for a one time data collection then maybe that is OK. There is absolutely no guarantee that the format will remain consistent or that the information will even be there tomorrow.
You should try to find an API that gives you the information that you are looking for. An API is a contract that is expected to be honored and therefore is reliable. A quick look at CoinMarketCap's API doesn't appear to have the info you are looking for but maybe another one exists that does.
If you were to parse the HTML you could write a regex for the specific thing you are looking for. For example if you want to get the website you could write a regex that would pick up the pattern:
Website
the capture group ([^"]+) is the website in this example. You could do something like that for every element you want to get.
Related
I am trying to use the OpenAlex API: https://docs.openalex.org/api
This API provides access to a catalog of scholarly papers, authors, institutions, etc...
It is easy enough to make a query for an institution's information such. Here is an example from the API docs:
https://api.openalex.org/I19820366
I am trying to figure out how to get a specific institution's ID.
In the docs, there is a statement that this ID number is Microsoft Academic Graph's institutional ID: https://www.microsoft.com/en-us/research/project/microsoft-academic-graph/
But I have been unable to figure out anything in Microsoft Academic Graph either.
How can I find the institutional ID for a specific institution?
I am a beginner, and I also just started to retrieve data from the OpenAlex API. For me, the easiest way was either to use the ROR id or the GRID id, which you can both look up for any institution either here: https://www.grid.ac/ or here: https://ror.org/
Then you use either ROR or GRID-ID as an identifier (https://docs.openalex.org/about-the-data/institution#ids) and that identifier as a filter, as specified in the API documentation.
Be aware, that except for the Institution-ID, that you want to find, all the other institutional IDs, like ROR or GRID, have to be put in your request as a full URL. Take the example of the Johns Hopkins University. It's not enough to put their ROR like this: "00za53h95", you have to put in the API request like that: "https://ror.org/00za53h95" (without the quotes) or else it won't work. In my example, a request could look like this:
https://api.openalex.org/institutions/https://ror.org/00za53h95
This will deliver a nice json file with all the info you need, including the institution's ID in the database. Save the information as a file by using a cURL GET request or just do it via your browser and get the result as a webpage, both works. If you do the latter, you should follow the suggestion of the OpenAlex team and install a browser plugin like JSONVue, that will make the experience of reading the result on your screen so much better.
Hope that helps.
You can use the OpenAlex API to search for the ID like this:
https://api.openalex.org/institutions?filter=display_name.search:University%20of%20Virginia
In addition to Heather's comment, I'd like to add that we can now go to https://explore.openalex.org/ and search for any entity. Start typing "Johns Hopkins University" and you'll get to this page: https://explore.openalex.org/institutions/I145311948 which has all the identifiers (including the openalex id I145311948) and additional information about this institution.
QuickBooks Online (QBO) uses a URL format like qbo.intuit.com/app/timeactivity?txnId=123 to point to, in this example, a TimeActivity.
However, in the API, resources are referenced by entityId (returned as just Id when querying via the TimeActivity API), which is different from txnId.
In my time tracking web app, I have a feature that exports time to QBO as TimeActivities. I'd like to provide users with direct links from the time entered in my app to the corresponding TimeActivity in QBO—is there any way to do so?
The answer to this is that it's not currently possible because there's multiple base URLs (I think they call them "realms") in use for QuickBooks Online (qbo.intuit.com is only one of them). Because you do not receive this information as part of linking to QBO via OAuth, there's unfortunately no way to construct proper links.
Yes, when creating a successful TimeActivity you will receive back an Id, which I assume you are persisting.
That Id can be used to query QBO in a simple GET request.
<baseURL>/company/{companyID}/timeactivity/{timeactivityId}
I'm looking to somehow tell users to upload a picture and do something, either tag or # our company so we are notified in real time, i just need to fetch the url of the photo that was uploaded and tagged or # or ANYTHING.
the hashtag search doesn't seem to work correctly, i've tried multiple hashtags and they wont retrieve them in any type of real time or at all.
the real-time subscription only notifies me that there is a new photo tagged that i subscribed to, so i have api search for a list of new media with that tag, which goes back to point 1, its not accurate or reliable.
so do i have any options at all on instagram?
If not do I have any options on facebook? any real time keyword search to grab the url of the photo they uploaded with a unique tag.
Check out https://github.com/chapmanu/hummingbird, it might be what you're looking for.
My experience with the Instagram Real-Time API is, that it is rather reliable.
When you get a real-time ping and then call the endpoint for your hashtag, the results are ordered newest first.
You can then paginate until you hit the last known result.
Maybe this tutorial helps: http://www.mtrinitaria.com/mykhel/tutorial-instagram-real-time-photo-update-api-with-php/
I have to get events from Wikipedia. suppose I type Brazilian Grand Prix, output should not be what Brazilian Grand Prix is but it should be where it is being held in upcoming days/weeks.I have to implement it on asp.net web application. User input should be only event name and he will get output in xml/json format. Please tell me if anyone know how to do this?
we can see all events here: http://en.wikipedia.org/wiki/Portal:Current_events
But I don't know how to get it programmatically.
I'm not completely sure, but it seems to me that you want to get a list of pages whose names are in the form Portal:Current events/2012 November 25, depending on what exactly do you consider “current”. Then search the code for each of those pages for what you want. If you get a match, output the line that matched.
Probably the easiest way to get the code of a page is to use action=raw. Another option is to use the API.
I have been looking for a while at how to retrieve financial quotes in c#, in this case, Canadian mutual funds.
It seems there are 2 main sources of information for this, Yahoo and Google! Also there seems to be 2 preferred methods, API and HTML scrapping. My preference would be for an API method, but I am open to any suggestion.
I have found the Yahoo api to be very nice to use, however although it works for stocks like "MSFT" it fails to retrieve the data for mutual funds like "RBCCANADIANI.TO". For some reason this symbol only works on the web site itself.
Google's API seems to requires to have a portfolio set up and login in order to retrieve quotes, I would prefer to avoid logins and use a completly opened api if possible.
HTML scraping introduce a totally new set of complications, asking the user to locate on an html page where the price is and such. Note that Google's html (http://www.google.com/finance?q=MUTF_CA:RBF556) returns the right page, but a look at the html source shows that the price is not in an easily identifiable tag (id not set!).
Has anybody tried similar things, I have the feeling I am missing something here :)
Many thanks
I am having success with the following YQL query:
select LastTradePriceOnly from yahoo.finance.quotes where symbol in ("F0CAN05NGC.TO")
This is for Royal Global Precious Metals fund.
Mutual funds have a LastTradePriceOnly field but no Bid field like stocks have. I got the symbol from the Symbol Lookup on the Yahoo finance page.
If you are looking for pricing data I would suggest a service like IQFeed. I have used them as a client for some time and they do have mutual fund data (although I don't use it). If you don't want to pay for it you will probably have to scrape it.