Is there a way to use the Twitter API to get a user's tweets for a specific time range?
It doesn't seem to be in statuses/user_timeline.
You can use the since and until operator.
https://api.twitter.com/1.1/statuses/user_timeline.json?include_entities=true&include_rts=true&screen_name=your_screen_name&since:2011-05-16&until:2011-08-16
But it will do you little good because tweets are searchable in a small time window. To get around this check out several resources,
Snapbird - https://github.com/remy/snapbird
and
20 ways of searching old tweets
Related
Is it possible to count how many times some URL was twitted in some period of time?
You can use the Twitter Search API and search for URL's, then examine the Tweet objects to find the ones in your date range. Use the GET search/tweets call. This is limited to the past 6-7 days in their index.
You can also write an app to watch the Twitter stream for the URL over a period of time, then count the occurrences. This can be done using the Twitter Streaming API using the POST statuses/filter call. Specify the string you are watching for and matching results will stream.
In Ruby, for example, you can do this using the Twitter Gem. It's best to search for known URL-shortened versions as well as the full URL. You can control this somewhat by registering the URL with several known URL shorteners in advance, so you know what to watch for.
I wrote a little Script using Python and Tweepy to save the tweets for a list of users and also to get some basic properties for those accounts.
Somehow the number of tweets stated in the user profile under statuses_count
(for an example of the json description of an account:
https://api.twitter.com/1/users/show.json?screen_name=TwitterAPI&include_entities=true )
does not match the number of tweets i get when iterating through the tweets of the same users profile.
I am aware of the fact, that twitter limits the number of tweets per user available through the API to 3200 and even does not guarantee this number, but this behavior does even occur with users who have well less than 3200 tweets
My question is, whether this difference is common and why this happens?
Is this just an issue of the twitter API, is it caused by deleted tweets (maybe they still count for statuses_count but can not be fetched anymore?), ...?
Thanks!
Thomas
I haven't messed with the Twitter API in several months, but I remember back when I was working with it I found inconsistencies due to retweets not showing up when iterating tweets, but getting counted in the number of Tweets. This seems to corroborate that, but its several months old and things may have changed since then.
Make sure include_rts is set to true, t, or 1 (in addition to specifying the same for include_entities, which you have done). When these aren't included by default (e.g. user lists) then you can get fewer tweets than what you specified with count.
The Twitter API documentation isn't clear on what the defaults are so it's safer to explicitly specify these optional parameters. And since you're specifically working with the user timeline you might also want exclude_replies turned off.
With https://dev.twitter.com/docs/api/1/get/statuses/user_timeline I can get 3,200 most recent tweets. However, certain sites like http://www.mytweet16.com/ seems to bypass the limit, and my browse through the API documentation could not find anything.
How do they do it, or is there another API that doesn't have the limit?
You can use twitter search page to bypass 3,200 limit. However you have to scroll down many times in the search results page. For example, I searched tweets from #beyinsiz_adam. This is the link of search results:
https://twitter.com/search?q=from%3Abeyinsiz_adam&src=typd&f=realtime
Now in order to scroll down many times, you can use the following javascript code.
var myVar=setInterval(function(){myTimer()},1000);
function myTimer() {
window.scrollTo(0,document.body.scrollHeight);
}
Just run it in the FireBug console. And wait some time to load all tweets.
The only way to see more is to start saving them before the user's tweet count hits 3200. Services which show more than 3200 tweets have saved them in their own dbs. There's currently no way to get more than that through any Twitter API.
http://www.quora.com/Is-there-a-way-to-get-more-than-3200-tweets-from-a-twitter-user-using-Twitters-API-or-scraping
https://dev.twitter.com/discussions/276
Note from that second link: "…the 3,200 limit is for browsing the timeline only. Tweets can always be requested by their ID using the GET statuses/show/:id method."
I've been in this (Twitter) industry for a long time and witnessed lots of changes in Twitter API and documentation. I would like to clarify one thing to you. There is no way to surpass 3200 tweets limit. Twitter doesn't provide this data even in its new premium API.
The only way someone can surpass this limit is by saving the tweets of an individual Twitter user.
There are tools available which claim to have a wide database and provide more than 3200 tweets. Few of them are followersanalysis.com, keyhole.co which I know of.
You can use a tool I wrote that bypasses the limit.
It saves the Tweets in a JSON format.
https://github.com/pauldotknopf/twitter-dump
You can use a Python library snscrape to do it. Or you can use ExportData tool to get all tweets for the user, which returns already preprocessed CSV and spreadsheet files. The first option is free, but has less information and requires more manual work.
My site needs the Twitter equivalent of the Facebook Comments plugin... You may be aware of the fact that Twitter only searches back about a week, so comments would be lost just a few days after being made! Is there a service that would allow me to show the tweets that contain the given page's unique #hashtag? I'm looking to put together a system that searches for the hashtag and puts those tweets up for display, even if the tweets are old. Does anyone know how to make this possible? I'm not that good of a programmer, FYI...
No. The Twitter Search API will not let you search that far back.
The Search API is not complete index of all Tweets, but instead an index of recent Tweets. At the moment that index includes between 6-9 days of Tweets.
You cannot use the Search API to find Tweets older than about a week.
From https://dev.twitter.com/docs/using-search
I'm building a Twitter Application to show specific tweets (that matching pre defined criteria). I used a good library to grab the tweets and before showing them to the user I the tweets must get stored in a local database, so that I have more data and amazing statistics (ego? huh) to be calculated and shown to the user.
The problem is that tweets are not stored in the hashtag, so if I search for the hashtag one week later I will not be able to find the tweets, so I must have a way to show the tweets from the database instead of Twitter API. I decided that I will show data from database when the last tweet from a hashtag (in the database) is stored before than three days or more. when the last tweet is stored in less than three days, then I will ask Twitter to show the tweets.
So I'm asking you if you have an idea how to show tweets from database since my library depends on JSON (or consider it XML). Any ideas?
Store the tweets in CouchDB. If you use twitter streaming api or search api, that should be the most straightforward way for "saving" tweets.