I'm working on a project accessing Twitter's Streaming API with HBC.
I'm storing keywords for Twitters Streaming API (filter) in a file and now I'm looking for a way to close and reconnect to Twitter each time the file changes.
I googled with no useful result.
Any idea how I could manage this task?
Don't do this Twitter doesn't like reconnects in stream API, they will ban your application.
If you have to often change filter parameters better use Rest Api search tweets endpoint
Related
I'm trying to fetch a tweet's video url using API V2.
Using API V1.1 I can use
https://api.twitter.com/1.1/statuses/show.json?id=<ID>&include_entities=true
and get the direct mp4 urls in the response at
extended_entities.media[0].video_info.variants
But using API V2 I can't seem to find any ways to get those, I was only able to get the video thumbnail but not the actual video.
I've tried it with both the lookup and the recent search endpoints but couldn't find a way to do that.
In lookup docs twitter says:
This endpoint was recently graduated from Twitter Developer Labs, and is the replacement of v1.1 statuses/show, v1.1 statuses/lookup, and Labs Tweet lookup. If you are currently using any of these endpoints, you can use our migration materials to start working with this new endpoint.
but that doesn't seem to be the case, is that feature not included ? Also if there is any other way to be able to embed a twitter video in a web page (without the tweet text) that'd be helpful.
The videos are not currently available in the Twitter API v2 at this time. This is a known request.
For those who still looking for an answer, you can do it on the V2 by adding the following parameters :
expansions=attachments.media_keys
media.fields=variants
Your final link should give https://api.twitter.com/2/users/{{uid}}/tweets?expansions=attachments.media_keys&media.fields=variants
Source : https://developer.twitter.com/en/docs/twitter-api/data-dictionary/object-model/media
Progress on twitter-api-v2 media handling, Maybe this will allow the video URLs too in the response.
https://twittercommunity.com/t/v1-1-media-endpoints-available-for-essential-access-in-the-twitter-api-v2/171664
We're building an app that analyzes twitter feed using streaming api. But it will only start analyzing after app starts. We need to download historical twitter data using API to initialize our system.
As per documentation twitter API only has until query parameter.
Is there a way to download /keep downloading the twits for given keyworkds with from to dates ?
We're using python.
As per documentation twitter API, query method that you mentioned provides "since_id" parameter. If you want to keep downloading tweetsas they come then use "until" parameter for first time n then "since _id" whenever you want to update
Has anyone found a way to retrieve the activity feed on Steam for a specific user to post on a website similar to tweets? I'm adding an activity feed to my website, but really the only thing I'm most active in is Steam, so it will get stale pretty quickly without Steam in there. I've looked at the web API, but it doesn't specify if I can grab my full feed and post it or if I can just grab certain stats for specific games or not. I've tried to find an RSS feed for my activity but had no luck so far, that would definitely be the preferable format.
I've just looked at this and you can't grab the web feed directly from the site on RSS or JSON. The Web API is meant for developers of Steam applications so that they can get at the player information. For that, you need an API Key, which is provided by Steam. It is not a casual web interface like you might find on Google.
The nasty solution is to HTML scrape the page. I used Yahoo! Pipes to scrape the page (and automatically update) but ultimately decided that was entirely too dirty as it assumes that the Steam pages won't change.
A bit too late, but I'm also searching for this kind of RSS feed. I think I will end up creating a RSS bot to parse the AJAX response used to fetch the activity feed:
http://steamcommunity.com/id/[your username]/ajaxgetusernews
This URL doesn't work out of the box, I think we have to pass some cookies to get access to this page to make Steam think the bot is logged in as a normal user. It returns the HTML markup used to render the activity feed, and a URL to fetch the next batch of activities.
Be advised that this HTML markup is hard to parse because it is inconsistent.
Is there any way to programatically access, through the Twitter API or otherwise, the data from the list of handles from the similar_to page? (e.g. https://twitter.com/#!/similar_to/aplusk).
Scraping as always is least desirable with dynamic javascript sites like Twitter.
It's there, it's just currently undocumented:
https://api.twitter.com/1/users/recommendations.json?user_id=783214
Looking at the request, these parameters are supported:
connections: Includes extended network connections in the data structure. For example, connections=true
limit: Limit the number of entries. For example, limit=3
user_id: The user ID to get recommendations for. For example, user_id=783214
Twitter Web options:
cursor: Unclear. I'd assume this removes the selection cursor in Twitter Web. For example, cursor=-1
display_location: The location to display the results (for Twitter Web). For example, display_location=st-view-all-stream
pc: Unclear. I'd assume this is just user-agent checking. For example, pc=true
It doesn't require authentication to hit against. I'm not sure if this is rate-limited, because I can't find any further information at the time of this writing.
While no documentation has been given by Twitter on this API, it's in current use by Twitter Web and Twitter Mobile for iOS, and it otherwise appears to be production-ready. Obviously, no guarantees can be had until Twitter officially releases it.
Good luck!
There's also a thing you may remember since theres no API 1.0 you cannot make requests from javascript, jquery, ajax or client languages to json. You can only make requests just with server languages like PHP to json in API 1.1.
But let's think that you want to make your application more dynamic, if we cannot make request to twitter directly, the solution is that you have to use PHP auth with Twitter and then in your own page (with the response of the php page) make the Javascript, ajax, and you can make it dynamic like API 1.0.
my own application Twitter API
I'd like to pull all of a user's tweets. I could do this the hard way (manually scraping twitter) or the easy way: using their api. The problem with the easy (api) way is that I seem to be limited to the 200 most recent tweets. What's a simple way to get all tweets?
Thanks
Yes you can get up to 3,200 historical tweets by requesting as follows...
Make a request to:
http://api.twitter.com/1/statuses/user_timeline.format
And use the count parameter 200 and iterate through the page parameter from page 1 to 16 or until there are no more tweets.
Thats the only thing you can currently do because Twitter specifically say they prevent this in their API Doc...
https://apiwiki.twitter.com/Things-Every-Developer-Should-Know#6Therearepaginationlimits
I would add, please don't screen-scrape because it will cause undue load on Twitter and in bulk requests it would probably get your server blocked from accessing Twitter.
you can make sure you get all future tweets by subscribing to your Twitter RSS feed with Google Reader. Then you can use their infinite scrolling feature to look back to the first tweet tracked.