twitter stream API track all tweet posted by user who registered in twitter application - api

I am creating the application which need to track all tweets from user who registered to my application, i tried to track those with streaming API , there are public API, user API , and site API,
in those API it just have an option to follow the user ID by add the comma separated user ID
https://dev.twitter.com/streaming/overview/request-parameters#follow
but i think it is not flexible, if there are a new user registered , i need to rebuild the HTTP request , and also if there are so many users try to listen this stream and query will be so long,
it will be
https://stream.twitter.com/1.1/statuses/filter.json?follow=[user1],[user2],[user3]........[userN],
i afraid the query wont fit, i just need a parameter to filter all user who registered in my application such as, for example.
https://stream.twitter.com/1.1/statuses/filter.json?application=[applicationID]
but i think twitter dev does not provide it
so, is there any way to filter stream by application ID?

I didn't see anything like tracking by application id. If your query become too complex (too many follows/keywords), public streaming api will reject it,
and you can't open more than 2 connections with user stream. So, last solution is using Site Stream, -> you can open as many user connections as you have users registered to your app.
BUT the docs says :
"Site Streams is currently in a closed beta. Applications are no
longer being accepted."
Contact twitter to be sure

Arstechnica has a very interesting article about it. Take a look at this code and the link in the end of this post
If you are using python pycurl will do the job. Its provides a way to execute a function for every little piece of data received.
import pycurl, json
STREAM_URL = "http://chirpstream.twitter.com/2b/user.json"
USER = "YOUR_USERNAME"
PASS = "XXXXXXXXX"
userlist = ['user1',...,'userN']
def on_receive(self, data):
self.buffer += data
if data.endswith("rn") and self.buffer.strip():
content = json.loads(self.buffer)
self.buffer = ""
if "text" in content and content['user'] in userlist:
#do stuff
conn = pycurl.Curl()
conn.setopt(pycurl.USERPWD, "%s:%s" % (USER, PASS))
conn.setopt(pycurl.URL, STREAM_URL)
conn.setopt(pycurl.WRITEFUNCTION, on_receive)
conn.perform()
You can find more information here Real time twitter stream api

Related

How to look up YouTube Live Chat Ban ID to delete it

The YouTube Live Streaming API has the ability to ban and un-ban a user from chat and provides the following method specifically to perform the un-banning:
https://developers.google.com/youtube/v3/live/docs/liveChatBans/delete
The API requires you to pass the id parameter that identifies the chat ban to remove and states that the value uniquely identifies both the ban and the chat. However, there does not appear any way to look up the chat bans for a channel, other than when you get the ID back initially when the ban occurs.
So is there no way to un-ban a user via the API long after the ban has occurred and you no longer have the ban ID?
Actually, LiveChatBanId is a base64-encoded string containing
banned user channel id
owner channel id
some random bytes?
Here is some python code that will generate it for you
import base64
banned_user_id = 'banned user id'
owner_channel_id = 'channel owner id'
decoded = f"\x08\x01\x12\x18{banned_user_id}\x1a8\n\r\n\x0b7cHT2DHCA7s*'\n\x18{owner_channel_id}\x12\x0b7cHT2DHCA7s".encode('utf-8')
encoded = base64.b64encode(decoded)
print(encoded)
Fill in banned_user_id and owner_channel_id
You may need to invoke the "LiveChatBans: delete" method twice, that's how Google works

How can i know that my Youtube API Data is correct?

I having some trouble to understand something related to the API of youtube
So my code is basically very simple:
name = input("enter the username: ")
key = "MY API KEY"
data = urllib.request.urlopen("https://www.googleapis.com/youtube/v3/channels?
part=statistics&forUsername="+name+"&key="+key).read()
subs = json.loads(data)["items"][0]["statistics"]["subscriberCount"]
print(name + " has " + "{:,d}".format(int(subs)) + " subscribers!🎉")
just yelling the number of subscribers after giving specific YouTube Username:
The thing is that some Usernames(for example: Vsuase/Veritasium/Unbox Therapy ) which have many subs and the API-URL giving me wrong Data
Vsause - in return giving me back 72 subs
Veritasium/Unbox Therapy - not giving my any number at all
BUT, a channel "Computerphile" giving me that exact same subs they have.
How come that few Usernames work and few do not??
I tested in both, using the try-it functionality available in the YouTube Data API - Official Documentation and in the Google API Explorer and in both sites the results are closely1 similar.
For example, when the statistics of the YouTube user vsauce is requested vía YouTube API, the value in subscriberCount is 14220819 and checking his YouTube channel it says: 14,220,825.
Here is the example for request the statistics of the YouTube user vsauce (using the try-it)
And here is the demo for request the statistics of the YouTube user vsauce (using the Google API Explorer).
I didn't see any differences in the values in subscriberCount by requesting the other channels you mentioned in your question.
1 You need consider that some channels has more changes in the quantity of subscribers than others and such results vary too in the responses of the API.
For some reason, if you change in the URL from forUsernae= --> id=
it gives you the correct numbers.
TED channel:
https://www.googleapis.com/youtube/v3/channels?part=statistics&id=UCAuUUnT6oDeKwE6v1NGQxug&key=AIzaSyDjnINqqxQlIg4kbXoPDVYOhHNfdmDbb9I

What data can I save from the spotify API?

I'm building a website and I'm using the Spotify API as a music library. I would like to add more filters and order options to search traks than the api allows me to so I was wondering what track/song data can I save to my DB from the API, like artist name or popularity.
I would like to save: Name, Artists, Album and some other stuff. Is that possible or is it against the terms and conditions?
Thanks in advance!
Yes, it is possible.
Data is stored in Spotify API in endpoints.
Spotify API endpoint reference here.
Each endpoint deals with the specific kind of data being requested by the client (you).
I'll give you one example. The same logic applies for all other endpoints.
import requests
"""
Import library in order to make api calls.
Alternatively, ou can also use a wrapper like "Spotipy"
instead of requesting directely.
"""
# hit desired endpoint
SEARCH_ENDPOINT = 'https://api.spotify.com/v1/search'
# define your call
def search_by_track_and_artist(artist, track):
path = 'token.json' # you need to get a token for this call
# endpoint reference page will provide you with one
# you can store it in a file
with open(path) as t:
token = json.load(t)
# call API with authentication
myparams = {'type': 'track'}
myparams['q'] = "artist:{} track:{}".format(artist,track)
resp = requests.get(SEARCH_ENDPOINT, params=myparams, headers={"Authorization": "Bearer {}".format(token)})
return resp.json()
try it:
search_by_track_and_artist('Radiohead', 'Karma Police')
Store the data and process it as you wish. But you must comply with Spotify terms in order to make it public.
sidenote: Spotipy docs.

I am trying to use Yodlee/executeUserSearchRequest as a RESTful request and need an answer on how to call

I am working with the Yodlee services in c# and using the RESTful api. So far I have successfully connected and logged in with my CobrandSession and UserSessionToken in the development environment. I used the sample apps provided in c# and with some advice from shreyans i got an app working. What I got working was
1) Get YodleeAuthentication
2) Get UserAuthentication
3) Get ItemSummaries
I am now trying to get the full transaction details for each of the Items (i.e. collections of accounts that are an Item)
reading the Docs here https://developer.yodlee.com/Indy_FinApp/Aggregation_Services_Guide/REST_API_Reference/executeUserSearchRequest it states that I need to call executeUserSearchRequest and then paginate through the results using the getUserTransactions. So I am stuck at this point. I dont really want a search which has parameters I just want ALL transactions for this account that I can see.
However, I am using the variables as defined in that page :-
var request = new RestRequest("/jsonsdk/TransactionSearchService/executeUserSearchRequest", Method.POST);
request.AddParameter("cobSessionToken", param.CobSessionToken);
request.AddParameter("userSessionToken", param.UserSessionToken);
request.AddParameter("transactionSearchRequest.containerType", param.ContainerType);
request.AddParameter("transactionSearchRequest.higherFetchLimit", param.HigherFetchLimit);
request.AddParameter("transactionSearchRequest.lowerFetchLimit", param.LowerFetchLimit);
request.AddParameter("transactionSearchRequest.resultRange.endNumber", param.EndNumber);
request.AddParameter("transactionSearchRequest.resultRange.startNumber", param.StartNumber);
request.AddParameter("transactionSearchRequest.searchFilter.currencyCode", param.CurrencyCode);
request.AddParameter("transactionSearchRequest.searchFilter.postDateRange.fromDate", param.FromDate);
request.AddParameter("transactionSearchRequest.searchFilter.postDateRange.toDate", param.ToDate);
request.AddParameter("transactionSearchRequest.searchFilter.transactionSplitType.splitType", param.SplitType);
request.AddParameter("transactionSearchRequest.ignoreUserInput", param.IgnoreUserInput);
request.AddParameter("transactionSearchRequest.searchFilter.itemAcctId", param.ItemAcctId);
var response = RestClientUtil.GetBase().Execute(request);
var content = response.Content;
return new YodleeServiceResultDto(content);
As per the response from shreyans in this posting Getting Error "Any one of [**] of transactionSearchFilter cannot be NULL OR Invalid Values I am not putting in the ClientId and the ClientName
The documentation doesn't specify the format of the dates but the example seems to tell me that its american date format. And specifies a parameter saying IgnoreUserinput, but doesnt have a parameter for user input so this is confusing
When I make a call using this format I get an error response
var getSearchResult = yodleeExecuteUserSearchRequest.Go(yodleeExecuteUserSearchRequestDto);
getSearchResult.Result="
{"errorOccured":"true","exceptionType":"Exception Occured","refrenceCode":"_60ecb1d7-a4c4-4914-b3cd-49182518ca5d"}"
But I get no error message in this and I have no idea what I have done wrong or where to look up this error, can somebody who has used Yodlee REST Api point me in the right direction as I need to get this researched quickly....
thanks your your help, advice, corrections and pointers....
Here is the list of parameters which you can try
1) For a specific ItemAccountId all transactions
transactionSearchRequest.containerType=all
transactionSearchRequest.higherFetchLimit=500
transactionSearchRequest.lowerFetchLimit=1
transactionSearchRequest.resultRange.startNumber=1
transactionSearchRequest.resultRange.endNumber=500
transactionSearchRequest.searchClients.clientId=1
transactionSearchRequest.searchClients.clientName=DataSearchService
transactionSearchRequest.searchFilter.currencyCode=USD
transactionSearchRequest.searchClients=DEFAULT_SERVICE_CLIENT
transactionSearchRequest.ignoreUserInput=true
transactionSearchRequest.ignoreManualTransactions=false
transactionSearchRequest.searchFilter.transactionSplitType=ALL_TRANSACTION
transactionSearchRequest.searchFilter.itemAccountId.identifier=10000353
2) For a Specific account (itemAccountId) with start and end dates
transactionSearchRequest.containerType=all
transactionSearchRequest.higherFetchLimit=500
transactionSearchRequest.lowerFetchLimit=1
transactionSearchRequest.resultRange.startNumber=1
transactionSearchRequest.resultRange.endNumber=500
transactionSearchRequest.searchClients.clientId=1
transactionSearchRequest.searchClients.clientName=DataSearchService
transactionSearchRequest.searchFilter.currencyCode=USD
transactionSearchRequest.searchClients=DEFAULT_SERVICE_CLIENT
transactionSearchRequest.ignoreUserInput=true
transactionSearchRequest.ignoreManualTransactions=false
transactionSearchRequest.searchFilter.transactionSplitType=ALL_TRANSACTION
transactionSearchRequest.searchFilter.itemAccountId.identifier=10000353
transactionSearchRequest.searchFilter.postDateRange.fromDate=08-01-2013
transactionSearchRequest.searchFilter.postDateRange.toDate=10-31-2013

Disqus - How to pass current logged in user?

I am using Disqus API to fetch details of the logged in user. I am not sure how to pass the current logged in user.
I have both api_key(public) and remote_auth and I am using Jquery ajax to send api request over http.
If I do something like this,
https://disqus.com/api/3.0/users/details.json?api_key=[apikey]
It says "You must either provide a user or authenticate the user." Now I have the loggedin users remote_auth.
FYI: This is how I am creating the remote_auth. Example User Id: 3096795, email = "a#a.com", Name="Test". Now when this user logs in to the website, it logs in to Disqus as well. I can see this user in http://disqus.com/api/sso/users/ with id = 3096795.
I have couple of questions:
1) Can I use jquery ajax to send a authenticated user and get user details? Or this can be done only via Server side? (Java/Php)
2) If I pass ?remote_auth=[remote_auth] as a query string, will it work?
3) if yes, remote_auth value has spaces in between HMAC->SHA1(secret_key, message + ' ' + timestamp) so how can I pass it as query string parameter?
4) If no, then how to pass a user to the listActivity.json endpoint? If I am passing the userid, then it returns me some other user and not the user I created.
The below request returns a different user.
https://disqus.com/api/3.0/users/details.json?api_key=[apikey]&user=3096795
How can I ensure the userid I am passing is unique and not already taken by a different disqus account?
Am I missing something?
Your remote_auth is a form of authentication, just like access_token, so you'll want to pass that in your request as remote_auth=<YOUR_PAYLOAD>.
If you pass "user=" that ID would have to be the Disqus user ID, which isn't the same as your remote_auth ID. Your remote_auth is a form of authentication, just like the access_token. However, keep in mind that we don't return as many details for SSO users as authenticated Disqus users. This is because the details are managed by you, the SSO site owner.
To answer your other questions:
You can use the client-side API to get these details, but we recommend the server-side API + caching the results to avoid bumping into API limits.
URL-encode the payload and this will work
Easier using https://github.com/anthavio/disquo
DisqusApplicationKeys keys = new DisqusApplicationKeys("...api_key...", "...secret_key...", "...access_token...");
DisqusApi disqus = new DisqusApi(keys);
//SSO is available only to premium accounts
SsoAuthData ssoauth = new SsoAuthData("custom-12345-id", "Firstname", "Surname");
//SSO User identity is used to create post
disqus.posts().create(ssoauth, keys.getApiSecret(), threadId, "Hello world " + new Date(), null);