My site needs the Twitter equivalent of the Facebook Comments plugin... You may be aware of the fact that Twitter only searches back about a week, so comments would be lost just a few days after being made! Is there a service that would allow me to show the tweets that contain the given page's unique #hashtag? I'm looking to put together a system that searches for the hashtag and puts those tweets up for display, even if the tweets are old. Does anyone know how to make this possible? I'm not that good of a programmer, FYI...
No. The Twitter Search API will not let you search that far back.
The Search API is not complete index of all Tweets, but instead an index of recent Tweets. At the moment that index includes between 6-9 days of Tweets.
You cannot use the Search API to find Tweets older than about a week.
From https://dev.twitter.com/docs/using-search
Related
I am trying to get as much data as a I can out of the Twitter API for an academic research project. Even though I only have access to the Standard API the data should be as accurate as possible. I am building myself a "wrapper" around Twarc and other utilities in Python that gets me most of the data I want in just the format I need. A big problem was getting all the replies, but I was able to solve it with a bit of trickery: Searching from the tweet in question onwards and then checking if the tweets in the obtained sample have the original tweet ID in "in_reply_to_tweet_id". Rinse and repeat with those newly obtained tweets.
Then I noticed the new moderation feature Twitter implemented in March. Now the moderated comments under "More replies" do not show up in my search output.
Example: https://twitter.com/NDRreporter/status/1113353224730365952
I find all replies except the following: Under "More replies" ("Mehr Antworten" in German), there is a reply chain started by a extreme right leaning (possibly troll) account ("#Der Steuerzahler") that got moderated and shoved down there. This does not show up in API searches, even if I let the code iterate for over an hour just looking for replies to this particular original tweet.
My question is pretty general: Aside from getting replies as they come in (i.e. before they are moderated) via Filter API, is it possible to find these moderated tweets via the Standard Search API? Not looking for a ready-made solution, general pointers suffice. If I can't find them via Search, then I obviously won't try it with that anymore.
Thanks in advance.
With https://dev.twitter.com/docs/api/1/get/statuses/user_timeline I can get 3,200 most recent tweets. However, certain sites like http://www.mytweet16.com/ seems to bypass the limit, and my browse through the API documentation could not find anything.
How do they do it, or is there another API that doesn't have the limit?
You can use twitter search page to bypass 3,200 limit. However you have to scroll down many times in the search results page. For example, I searched tweets from #beyinsiz_adam. This is the link of search results:
https://twitter.com/search?q=from%3Abeyinsiz_adam&src=typd&f=realtime
Now in order to scroll down many times, you can use the following javascript code.
var myVar=setInterval(function(){myTimer()},1000);
function myTimer() {
window.scrollTo(0,document.body.scrollHeight);
}
Just run it in the FireBug console. And wait some time to load all tweets.
The only way to see more is to start saving them before the user's tweet count hits 3200. Services which show more than 3200 tweets have saved them in their own dbs. There's currently no way to get more than that through any Twitter API.
http://www.quora.com/Is-there-a-way-to-get-more-than-3200-tweets-from-a-twitter-user-using-Twitters-API-or-scraping
https://dev.twitter.com/discussions/276
Note from that second link: "…the 3,200 limit is for browsing the timeline only. Tweets can always be requested by their ID using the GET statuses/show/:id method."
I've been in this (Twitter) industry for a long time and witnessed lots of changes in Twitter API and documentation. I would like to clarify one thing to you. There is no way to surpass 3200 tweets limit. Twitter doesn't provide this data even in its new premium API.
The only way someone can surpass this limit is by saving the tweets of an individual Twitter user.
There are tools available which claim to have a wide database and provide more than 3200 tweets. Few of them are followersanalysis.com, keyhole.co which I know of.
You can use a tool I wrote that bypasses the limit.
It saves the Tweets in a JSON format.
https://github.com/pauldotknopf/twitter-dump
You can use a Python library snscrape to do it. Or you can use ExportData tool to get all tweets for the user, which returns already preprocessed CSV and spreadsheet files. The first option is free, but has less information and requires more manual work.
Is there a way to use the Twitter API to get a user's tweets for a specific time range?
It doesn't seem to be in statuses/user_timeline.
You can use the since and until operator.
https://api.twitter.com/1.1/statuses/user_timeline.json?include_entities=true&include_rts=true&screen_name=your_screen_name&since:2011-05-16&until:2011-08-16
But it will do you little good because tweets are searchable in a small time window. To get around this check out several resources,
Snapbird - https://github.com/remy/snapbird
and
20 ways of searching old tweets
I'm building a Twitter Application to show specific tweets (that matching pre defined criteria). I used a good library to grab the tweets and before showing them to the user I the tweets must get stored in a local database, so that I have more data and amazing statistics (ego? huh) to be calculated and shown to the user.
The problem is that tweets are not stored in the hashtag, so if I search for the hashtag one week later I will not be able to find the tweets, so I must have a way to show the tweets from the database instead of Twitter API. I decided that I will show data from database when the last tweet from a hashtag (in the database) is stored before than three days or more. when the last tweet is stored in less than three days, then I will ask Twitter to show the tweets.
So I'm asking you if you have an idea how to show tweets from database since my library depends on JSON (or consider it XML). Any ideas?
Store the tweets in CouchDB. If you use twitter streaming api or search api, that should be the most straightforward way for "saving" tweets.
For some reason, when searching on one specific Twitter user, the search API return nothing. (ie http://search.twitter.com/search.atom?q=+from%3ATWITTERHANDLE_A) TWITTERHANDLE_A here is the Twitter account name. This user has been active for over a month, has had many RTs, #s and has sent such tweets out as well.
Meanwhile, I created a new Twitter account - we will call it TWITTERHANDLE_B. Immediately after I created the account, I sent 1 tweet and performed the same search as above (http://search.twitter.com/search.atom?q=+from%3ATWITTERHANDLE_B)
The tweet was returned.
Is there ANYWAY to find out if and/or why a particular user would be blocked from search results? Thanks so much for any help... I'm going crazy here. Twitter's documentation just says sometimes a user's tweets won't be searched!
The documentation you refer to has a link to this page:
http://support.twitter.com/forums/10713/entries/42646
The bottom entry addresses your issue, hopefully you can fix it that way.
Bear in mind the search API only grabs results from the last couple of weeks, so if the user hasn't tweeted in a while then there won't be any results.
You could try not using the Search API and grab the results directly:
http://twitter.com/statuses/user_timeline/-username-.json
Works fine for http requests at least... >_<