I am working on a little app which will give an option to the user to select From and To dates and the app will return tweets they had in the particular time span.
Twitter is offering User Stream API that returns 200 records per request. My account has 6790 tweets and I need to calculate their favorite and retweets.
6790 will result approx 33 requests if I go for all tweets and how much time will it take? Further, will it cause some issue due to twitter rate-limit?
If somebody can please guide me on how can I fetch the data?
Thanks
You don't need to stream for this.
If you want to get the user's favourites, use the favorites/list
For example, to get my favourites, call
https://api.twitter.com/1.1/favorites/list.json?screen_name=edent&count=200
If the user has authenticated with you, it's easy to get their retweets with statuses/retweets_of_me
Related
I'm hitting the 100 seconds per user site restricted google custom search quota. I have unlimited queries plan and am getting billed. I don't pass in the userId so I'm not sure how it's limiting me. From the console I see the site restricted rate as 100 per user per 100 seconds. Can I just feed in random strings as the user id to help solve this? I've read the docs and they're not useful and have found other people ask this without an answer.
Help would be appreciated!
Thank you
If you don't pass a user ID, then all of your queries are considered to be coming from a single user.
Passing a user ID with your requests will apply the capping correctly per user.
See the "Capping API Usage" doc for more info.
I know you must be thinking that its impossible or its been asked already.
But I have 2 queries. The first is that by using Twitter API, using an access token of my own profile, can I get all (more than 100) the retweets of my own tweet? Not someone else's, as all previous questions at stackoverflow have been asked.
Because there's a difference in private and public tweet and getting data related to it.
Secondly, if we cannot get more than 100 retweets, then how does this app Pickaw (formerly Twrench) https://pickaw.com/en gets all the retweets even if they're more than 500 and the corresponding data?
Any ideas?
The Twitter API only provides access to up to 100 Retweeters (IDs for users that RT'd a Tweet), regardless of whether this is your owned Tweet, or another one.
As for a specific app, it is only possible to speculate, unless the source code was Open Source and available. I would suggest there are two ways to get all the Retweets:
pay for premium or enterprise search access, and use the advanced PowerTrack rules to find Retweets of a specific Tweet ID;
use the Account Activity API webhooks to track whenever a user's Tweet is Retweeted. Not that this would only work in a real-time tracking case; you wouldn't be able to check historical Tweets.
I'm using this YQL command to access stock quote information in an XML format. The problem is it keeps timing out and rejecting after a bunch of hits. I think I need to plug in my API key so it doesn't think it's bots.
I'm using SharePoint to process the XML.
http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20yahoo.finance.quotes%20where%20symbol%20in%20(%22AAPL%22)&env=store://datatables.org/alltableswithkeys
This is what it says for limits.
Per application limit (identified by your Access Key): 100,000 calls per day.
Per IP limits: /v1/public/: 1,000 calls per hour; /v1/yql/: 10,000 calls per hour.
I'm trying to get that per application limit of 100,000 calls per day. Or I guess that 10,000 calls per hour is also good. Any suggestions? Thanks for any help.
You need to use OAuth and go through the complicated procedure of exchanging tokens. Once you have an access_token and a token_secret, you can use them to make authenticated requests until they expire.
The full workflow is summarized here.
You'll need to go through the step-by-step procedure outlined in the link above, but essentially what you'll be doing is:
Get a request_token from here.
Redirect user to a Yahoo authorization page.
Retrieve the oauth_verifier that comes in the query string when the user is redirected back to your page.
Exchange the request_token and oauth_verifier for an access_token and token_secret.
You can then use the access_token and token_secret to make authenticated requests.
Since you're using .Net, you can make your life easier by using this sample code posted here.
hope you have used Yahoo.inc to get your access token
so just use
$session = YahooSession::requireSession(CONSUMER_KEY,CONSUMER_SECRET, APP_ID);
$mails=$session->query('your query limit 10');
after feting access
I want to know how many followers my followers have (so it's recursively).
The Problem is obviously the API limit.
I read out my follower ids with http://api.twitter.com/1/followers/ids.json?screen_name=username
I request http://twitter.com/users/show.xml?user_id=the_id, but if I got many followers, I quickly get blocked by API limitations.
Is there a way to bypass this? For example, would this work for Twitter users with more than 6000 (or even infinite?) followers, if I ask to authenticate with their twitter account? Or is there another API call which may perform more requests?
If you use statuses/followers for your self, and page thru the results.
http://dev.twitter.com/doc/get/statuses/followers
http://twitapi.com/explore/statuses-followers/
This will give you information on 100 followers per api call. In that result amongst other things is each followers, follower and following count.
6000 will only take 60 calls.
As this needs to be processed while authenticated you will also get 300 api calls per user.
Good luck.
There is currently no way to combine multiple API calls to Twitter into one, like Facebook allows you to.
So there probably is no way to do this efficiently, you just have to spread out the calls over multiple hours.
Assume there's a mobile app and a server.
I have question about rate limiting and hoping someone can give some advice on a design as I'm banging my head on how to navigate around rate limit. There must be something I"m missing because the 150 unauthenticated rate limit per IP per hour is extremely low.
Imagine the scenario I want to build is the following (simplified into a trivial example for this discusion). Assume user is signed into Twitter for this entire discussion to remove discussion about oAuth.
Mobile talks to our service to show users twitter friends list. Every time the mobile app is loaded, it will show the entire friends list, and highlighting the new friends that were added within the last 2 days.
That's it. But the trick is that I want to ensure that the friends list is always up to date in the client, which means our server has to have the most recent up to date friends list.
Periodically, I want my server to automatically scan the Twitter friends list for every user of my app to see if new friends have been added.
Our initial design was getting our server to do all the work with this flow:
New User signs in on client, gives access token to server
Server makes call to Twitter REST APIs to get initial friends lists
Server stores the Twitter Friends IDs and shows responds to the client with that list.
Periodically (e.g. every 48 hours), server checks Twitter REST APIs for friends list for each user and compares it to our cached Twitter friends list we have for them to see who is new and to highlight in the mobile app.
The good thing about this is that all the interaction with twitter to get friends list, compare and peridiocally refresh is on the server. Mobile client just makes a single call to my server and gets friends list.
The problem with this design is that it will work for a single user, but since the rate limit is 150 per hour on un-authenticated calls, I will hit my limit as soon as 151 users user my service (which has a fixed IP).
The only solution I can see is to have the client do the work for each user, then send me the friends list which my server caches. This takes care of Step #2 above. However, for Step #4, I'd have to build something into the client to auto refresh twitter friends and send back to the server.
This is super clumsy to have the client involved at all in this Twitter friends list operation.
At first I thought I was crazy and the public unauthenticated APIs like getting friends lists wouldn't be subject to rate limiting. However, according to their docs, it is.
Am I missing something obvious or is the only way to solve this is to put heavy logic into the client?
With whitelisting gone for those that aren't grandfathered or Twitter business partners, I don't think you have any alternative but to have your mobile app do the Twitter API calls from the handset.
Having the handset call Twitter isn't a bad thing by any means. Pretty much every Twitter client in the world does it. One benefit will be that the user will be authenticated to Twitter, and thus her full 350 calls per hour will be available to you. Keep in mind, however, that you should minimize your calls since the user may have other Twitter-aware applications installed on her handset eating into your call allotment, and vice versa.
Now to the solution. The way I would implement your use case would be to first fetch the complete list of friends for your user by calling the friends/ids method.
http://api.twitter.com/1/friends/ids.json?screen_name=yourUsersName
The above call will return the most recent 5,000 friend IDs, in order followed, for #yourUsersName. If you want to fetch more friend IDs than the first 5,000, you'll need to specify the cursor parameter to initiate paging.
Next, I would check the latest list of friends we just fetched against the list on the handset, syncing them by removing any IDs that are no longer present, while adding any that are new.
If we only need the friend IDs, then we're done at a cost of one API call per 5,000 friend IDs. If, however, we need to get user info for these new friends as well, then I would call users/lookup and pass in the list of all new users that we discovered while syncing friend IDs. You can request up to 100 user objects at a time.
http://api.twitter.com/1/users/lookup.json?user_id=123123,5235235,456243,4534563
You user must be authenticated in order to make the above request, but the call can fetch any Twitter user profiles you wish -- not just those that are friends of the authenticated user.
So, let's say for example that a user has 2,500 friends and has never used your app before. In that case, she would burn one call to fetch all of the friend IDs, and 25 calls for her friends' information. That's not too bad to get the app populated with data.
Subsequent calls should be more streamlined with probably only two calls burned (one for the IDs, and one to get the new friends).
Finally, once the data has been updated on the handset, the deltas for the IDs and user data can be gathered up and pushed to your server.
It may even be possible that your server application won't even have to interface with Twitter at all, and that should alleviate the 150 user limit you are encountering.
Some final notes:
Be sure to note in your app's privacy policy that you sync your user's friend list with your server.
I recommend specifying JSON as the return format for all Twitter API calls. It is a much more lightweight document format than XML, and you will typically transfer only about 1/3 to 1/2 as much data over the wire.
Pick a Twitter framework appropriate for your mobile device and your programming language. Twitter access is a commodity these days, and there's little to no reason to reinvent how to access the Twitter API.
I answered a similar question about an approach for efficiently fetching followers here.
Since you are making request on behalf of users you should make those requests be authenticated as those users. Then requests will count against each users own pool of 350 requests/hour.