Right, this is really working on my nerves, but Instagram has to do something about their bloody documentation.
I am already trying for a week to live update my website with new instagram posts without refreshing the page. Twitter was fairly easy, but instagram is just one big mess. Basically I use the realtime Instagram API, the callback and all that stuff is working fine, but thanks to Instagram it does not return me an ID from the post that is new, the callback only returns some basic stuff:
[{"changed_aspect": "media", "object": "tag", "object_id": "nofilter", "time": 1391091743, "subscription_id": xxxxx, "data": {}}]
with this data you are nothing, except for the Tag, but I knew the tag before this callback too so doesn't matter. It actually only tells me that there is a new post. I have tried doing the same request as when the page loads, when this callback occurs, and get the posts that are newer than those already on the page. Unfortunately I have not succeeded in this yet. I have picked the ID from the last posted instagram post, and checked if it is in the callback request, and it's not.
What am I doing wrong?
I'd appreciate some help, thanks!
Edit:
I'd like to note that this is not only a problem with the realtime api, but also with the normal API. I just don't know how to compare data so I don't get duplicates in my database(normal api), or on my website (realtime). I can't find any tutorial or documentation (Yes, I might be blind), that explains to me how to compare data. I can only find the min_id and max_id, but no explanation what these id's contains. I checked these id's with id's from results, and they do not match. It's not an ID from a media item.
I also checked the next_url, and in my logic thinking, this should be a URL to the next page (like Twitter).
Am I looking at this all wrong?
Ok strike my old answer, I changed the way I do this. Here's how I'll do it now.
I still wait for 10 hits on my Real-time subscription, when I reach 10 I send off a new thread (if one is not already running).
The sync thread queries my DB for a value, I need the last min_tag_id I used. Then I query:
https://api.instagram.com/v1/tags/*/media/recent?access_token=*&min_tag_id=*
Try it out here: https://api.instagram.com/v1/tags/montreal/media/recent?access_token=*
You'll get 20 results, and a min_tag_id value. Append that to your url, you'll see you get no results. Wait a couple of seconds and refresh. Eventually you'll get some media, and a new min_tag_id.
(You can ignore the "next_url" value they give you, you won't be using that).
Basically you only need to store that min_tag_id and query until you have no more results, that means you're done then.
When you get a subscription push, you need to query that endpoint (tag / recent).
I normally start an synchronous thread to perform this so I can answer in under 2 seconds to Instagram.
Then you parse that endpoint and look for a "next url" value.
Keep querying that end point, parsing the media and going to the next url until you find your stop condition.
For me I try to match 10 consecutive records in my DB. Basically from the tag, I store media when then meet my business rules.
The Instagram documentation is accurate and actually well written.
The realtime API is working correctly. As stated in the documentation:
The changed data is not included in the payload, so it is up to you
how you'd like to fetch the new data. For example, you may decide only
to fetch new data for specific users, or after a certain number of
photos have been posted.
http://instagram.com/developer/realtime/
You only receive a notification that an update has happened to your subscribed object. It is up to you to call the API to find out what that data is.
You can call the /tags/[tag-name]/media/recent with an access token that you have previously stored on your own server or DB. Then, you should be able to compare the data returned from that endpoint with any data you have retrieved prior, and just pull the objects that you do not yet have.
Related
I've tried searching through Stack overflow as well as the soundcloud API reference but couldn't find any answer. I'm trying to retrieve the date a user liked a song on Soundcloud. I tried two parts of the API:
- /users/{id}/favorites: This will return all tracks liked by the user, but no date when it was liked.
- /me/activities: This returns track reposts, comments and seemingly tracks that show up on the stream. Sadly there doesn't seem to be a type track-like, but maybe I'm doing something wrong here or need to pass additional parameters.
I'm having the user log into Soundcloud before utilizing the API.
Any chance someone knows if retrieving the track like date via the API is possible?
Thanks!
With the public API available right now, this is not possible.
You have to build it on your own and track the changes in a different, your own database.
Example call from the website when you lookup your likes, i dont know what you will see with a pro account:
https://api-v2.soundcloud.com/users/soundcloud:users:1672444/stats/timeseries/likes?from=1442016000000&to=1442534400000&resolution=day&tracks=soundcloud%3Atracks%3A30984358&client_id=02gUJC0hH2ct1EGOcYXQIzRFU91c72Ea&app_version=39e9f65
Response:
[{"user_urn":"soundcloud:users:1672444","timeseries":[{"time":1442016000000,"count":1},{"time":1442534400000,"count":1}]},{"user_urn":"soundcloud:users:1672444","track_urn":"soundcloud:tracks:30984358","timeseries":[{"time":1442016000000,"count":1},{"time":1442534400000,"count":1}]}]
Here is and older, related answer about playback stats:
How to fetch playback statistics
I'm looking to somehow tell users to upload a picture and do something, either tag or # our company so we are notified in real time, i just need to fetch the url of the photo that was uploaded and tagged or # or ANYTHING.
the hashtag search doesn't seem to work correctly, i've tried multiple hashtags and they wont retrieve them in any type of real time or at all.
the real-time subscription only notifies me that there is a new photo tagged that i subscribed to, so i have api search for a list of new media with that tag, which goes back to point 1, its not accurate or reliable.
so do i have any options at all on instagram?
If not do I have any options on facebook? any real time keyword search to grab the url of the photo they uploaded with a unique tag.
Check out https://github.com/chapmanu/hummingbird, it might be what you're looking for.
My experience with the Instagram Real-Time API is, that it is rather reliable.
When you get a real-time ping and then call the endpoint for your hashtag, the results are ordered newest first.
You can then paginate until you hit the last known result.
Maybe this tutorial helps: http://www.mtrinitaria.com/mykhel/tutorial-instagram-real-time-photo-update-api-with-php/
Context: let's say we want to retrieve whole list of Starred repositories by given User periodically (ones per day, hour or few minutes).
There are at least 2 approaches to do that:
execute GET to https://api.github.com/users/evereq/starred and use Url with rel='next' in 'Link' Response Headers to get next page Url (we should do that till we get no "next" page in response, mean that we reach the end). Seems that is recommended approach (by Github).
iterating 'page' parameter (from 1 to infinite) using GET to https://api.github.com/users/evereq/starred?page=XXX till you get 0 results in response. Ones you get 0 results, you finish (not recommended because for example instead of page numbers Github can move to "hash" values. Github already did it for some API operations.).
Now, let's say we want to make sure we use Conditional Requests (see https://docs.github.com/en/rest/overview/resources-in-the-rest-api#conditional-requests) to save our API usage limits (and traffic, trees in the world, etc.).
So we add for example 'If-None-Match' to our Requests Headers and check if response Status is 304 (Not Modified). If so, it means that nothing was changed from our last request. That works OK.
The issue however that what we have in 1) and 2) above, related to the way how we detect when to stop is NOT working anymore when you use Conditional Requests!
I.e. with approach 1), you don't get Link Response Headers at all when you use Conditional Requests.
So you will need to execute one more request with page bigger than page for which you already have ETag and see that it return 0 results and than you know you are done. That way you basically "waste" one request to Github API (because it miss Conditional Requests Headers).
Same with approach 2), you basically have 0 responses in every request with status 304... So again, to know you are done, you need to make at least one additional request which do return 0 results.
So the question is: when we do conditional requests with the fact that Github API does not send back Link Response Header (at least with queries using ETag which result Status 304) how could we know when to stop paging? Is it a bug in Github API implementation OR I miss something?
We don't know maximum page number, so to get when to stop we should execute one more "waste" request and check if we get 0 results back!
I also can't found how to query Github for total count of starred repositories (so I can calculate how many pages I should iterate in advice), same as responses does not include something like "X-Total-Count" so I know when to stop using simple math for pages count.
Any ideas how to save that one ('end') request and still use Conditional Requests?
If you do one request per day, it's OK to accept such waste, but what if you do such request ones per minute? You will quickly use all your API usage Limits!
UPDATE
Well, after a few more tests, I see now following "rule" (can't however found it anywhere in the docs, so note sure if its rule or just assumption): if user star something new, result for EVERY requested page contains different ETag value compared to previous and does not have status 304 anymore! That means that it's enough to just request first page and check for status. if its 304 (not modified), we do NOT need to check next pages, ie we are DONE as nothing was changed in any page. Is it correct approach or just coincidence?
We indeed return pagination relations in the Link response header when the content has changed 1. Since we don't support a since parameter for that call, you'll need to sort by most recent results and maintain a client-side cursor for the last known ID or timestamp (based on your sort criteria) and stop paging when it shows up in your paginated results. Conditional requests will just let you know if Page 1 has changed.
We haven't settled on a way to return counts on our listing methods, but a really low-tech solution is to set the page size to 1, grab the rel=last Link relation and check its page parameter value.
Hope that helps.
For some reason, when searching on one specific Twitter user, the search API return nothing. (ie http://search.twitter.com/search.atom?q=+from%3ATWITTERHANDLE_A) TWITTERHANDLE_A here is the Twitter account name. This user has been active for over a month, has had many RTs, #s and has sent such tweets out as well.
Meanwhile, I created a new Twitter account - we will call it TWITTERHANDLE_B. Immediately after I created the account, I sent 1 tweet and performed the same search as above (http://search.twitter.com/search.atom?q=+from%3ATWITTERHANDLE_B)
The tweet was returned.
Is there ANYWAY to find out if and/or why a particular user would be blocked from search results? Thanks so much for any help... I'm going crazy here. Twitter's documentation just says sometimes a user's tweets won't be searched!
The documentation you refer to has a link to this page:
http://support.twitter.com/forums/10713/entries/42646
The bottom entry addresses your issue, hopefully you can fix it that way.
Bear in mind the search API only grabs results from the last couple of weeks, so if the user hasn't tweeted in a while then there won't be any results.
You could try not using the Search API and grab the results directly:
http://twitter.com/statuses/user_timeline/-username-.json
Works fine for http requests at least... >_<
Let's say user A follows user B, and B follows A. I want to know the exact date A started following B and viceversa.
Is this information stored on twitter? Can I retrieve it using the API?
To clear out: The point of this question is finding a way to know who followed who first.
(I'm assuming both A and B deleted the notification e-mails)
No Ignacio, you can't. You just can know who follows who but not the date the follow started.
Looking at the API, there's is no way, there are two calls to get the followers:
User Methods/statuses/followers
and
Social Graph Methods/followers/ids
Neither of them returns dates or even a serial that would let you see who started following first. Really, there's no indication that twitter is internally storing this information, neither in the API nor Twitter's web interface.
This is a very old question, but perhaps some might be interested to know that while you cannot get the date at which someone started following, you can at least infer an "earliest possible following date" from the fact that the list of followers is ordered according to date, and the fact that follower objects come with a created_at timestamp.
Here's a Python function for calculating an "earliest possible following date": https://github.com/BernhardClemm/twitter-follow-dates
Of course Twitter stores it, because Twitter sorts followers and following lists by the date ;)
It is possible to do this, but impractical. When you call the followers API you can page the results. Each returned object contains next_cursor and prev_cursor items. These refer to the first and last records in the next and previous pages. These values are time based and can be used to calculate the time that the respective users followed you.
It follows that, if you set the page size to 1, you can walk through the list of follower IDs one at a time and the next_cursor value will allow you to derive the follow time for the next record.
This is reasonably simple to implement, however, in practice, you'll very quickly hit Twitter's API rate limit.