Iam a RoR developer and I want to calculate DAU for my website. By DAU I mean:
DAU- Total Number of users who either came to my website and logged in or who came to the website and were already logged in.
I implemented Devise in my website because it tracks current_sign_in_at. And later found that these attributes mean as:
current_sign_in_at - The latest time the person logged in to my website.
(Using this I can get the count of people who came to my website and logged in.)
But I am missing the count of people who came to the website and were already logged in.
One of the way I found to implement it is trigger a function in delayed job which updates the timestamp whenever the root action is called and home page of my website loads. But since my website gets around 1000 - 2000 hits per minute I guess this function will become the bottleneck.
Can anyone please suggest me the best way to implement it.
Many Thanks.
Possible solutions:
Use a service like Google Analytics(custom variables), KISSmetrics(visitor identity) or Mixpanel(distinct_id) and send them the user id and you can find daily active users.
You can do something like what you mentioned, store the last active time for the user in a database each time you get a request, if you get 1000-2000 hits per minute, you might want to store it in an in memory store like redis instead. If users will be coming to your website from different paths and not just root url you could create a after_filter in the application controller which updates the redis store with the current time for the user.
If you are interesting in looking at more then just the daily active users in future, it is worthwhile to create a log for your users in another database. Again I'd suggest an before or after_filter in application_controller which queues a job to store time, user_id, any other data you might be interested in. I'm using a mongo db separate from my ActiveRecord MySQL database to do this.
Related
I'm making a simple analytics admin assuming I'm the owner of a shopping mall as my first project
I've looked up a lot about it but, I couldn't find any information. What I've found out so far is that most existing marketing toole uses cookie and they caclulates it in some ways like they set some cookie with current time and set another cookie when users move to another page on website.
My website is made of AJAX and expressjs with mysql and I'll send user's average duration time to my admin page
Could anyone tell me how to track duration time?
I'm definitely not distributing the idea but it is only for my team prject
This could be accomplished with a combination of localStorage (optionally) and onbeforeunload.
Set the current time when a user first gets to site
Store it in localstorage
When they are about to leave, onbeforeunload, you can save / calculate the duration
https://developer.mozilla.org/en-US/docs/Web/API/Storage/LocalStorage
https://developer.mozilla.org/en-US/docs/Web/API/WindowEventHandlers/onbeforeunload
I'm using a google events tracker like this:
_gaq.push(['_setAccount', 'UA-1422398-23']);
_gaq.push(['_trackEvent', 'BookingRequest', 'Parent Name', $('#parent_fname').val()+' '.$('#parent_lname').val()]);
In this case, let's say I can track how many requests were submitted.. is there a way to track these specific users, and see the traffic source, and if it's Google, then what keywords they searched my website by?
Basically I want to see the people booking online, and see how and where they got me... and hoping there's a better and more elegant way than to have a field that asks "What did you type into Google to find us?"
Putting fname and lname into GA is a violation of Google Analytics terms of service. See number 7 PRIVACY.
Custom vars can be used to determine user type and/or login status. See Session-level Custom Variables
For example, if your website offers users the ability to login, you can use a custom variable scoped to the session level for user login status. In that way, you can segment visits by those from logged in members versus anonymous visitors.
But, don't try to put the username in GA either:
While the username or user ID is not directly PII, if it is used to tie to a person from a backend system…that’s a violation of the Terms of Service.
To see the traffic source of the events, go to Content, Events, select the event you want, and add the secondary dimension of Traffic Source (just above event title in report).
Here is a Custom Report for Event Labels that drills down by source then keyword. Just log into GA and go to the URL.
Our site uses Facebook connect. When a new user signs up we ask for permission to pull their interest data, their list of friends, and their friends' interests. Fetching this data used to be a very quick process (couple seconds). Over the last week or so, the time to fetch this data has increase to 10+ seconds. According to Facebook insights, our site is not being throttled. We didn't make any changes to our site.
Anyone else experiencing this issue with Facebook? Have any ideas for how to address it?
Thanks!
As of 1/26 at 7:55 PM EST, the live status page doesn't indicate any irregular activity.
Sometimes this occurs because a user simply has a lot of likes and interests. I would recommend making this operation asynchronous following a flow something like this:
User connects with your app
Get the access token and store it in a queue that a background process can access.
Get all the information you need immediately to make the app work.
Some time later
In a background process, grab an access token from the queue, parse it and handle it however you'd like.
A simpler, although less stable option, is redirecting the user to a page upon installation which makes an AJAX request to that page telling it to download the information from the graph. This keeps the response time low, but does require your user to have Javascript enabled and for them to stay on the destination page long enough for the request to be created.
Assume there's a mobile app and a server.
I have question about rate limiting and hoping someone can give some advice on a design as I'm banging my head on how to navigate around rate limit. There must be something I"m missing because the 150 unauthenticated rate limit per IP per hour is extremely low.
Imagine the scenario I want to build is the following (simplified into a trivial example for this discusion). Assume user is signed into Twitter for this entire discussion to remove discussion about oAuth.
Mobile talks to our service to show users twitter friends list. Every time the mobile app is loaded, it will show the entire friends list, and highlighting the new friends that were added within the last 2 days.
That's it. But the trick is that I want to ensure that the friends list is always up to date in the client, which means our server has to have the most recent up to date friends list.
Periodically, I want my server to automatically scan the Twitter friends list for every user of my app to see if new friends have been added.
Our initial design was getting our server to do all the work with this flow:
New User signs in on client, gives access token to server
Server makes call to Twitter REST APIs to get initial friends lists
Server stores the Twitter Friends IDs and shows responds to the client with that list.
Periodically (e.g. every 48 hours), server checks Twitter REST APIs for friends list for each user and compares it to our cached Twitter friends list we have for them to see who is new and to highlight in the mobile app.
The good thing about this is that all the interaction with twitter to get friends list, compare and peridiocally refresh is on the server. Mobile client just makes a single call to my server and gets friends list.
The problem with this design is that it will work for a single user, but since the rate limit is 150 per hour on un-authenticated calls, I will hit my limit as soon as 151 users user my service (which has a fixed IP).
The only solution I can see is to have the client do the work for each user, then send me the friends list which my server caches. This takes care of Step #2 above. However, for Step #4, I'd have to build something into the client to auto refresh twitter friends and send back to the server.
This is super clumsy to have the client involved at all in this Twitter friends list operation.
At first I thought I was crazy and the public unauthenticated APIs like getting friends lists wouldn't be subject to rate limiting. However, according to their docs, it is.
Am I missing something obvious or is the only way to solve this is to put heavy logic into the client?
With whitelisting gone for those that aren't grandfathered or Twitter business partners, I don't think you have any alternative but to have your mobile app do the Twitter API calls from the handset.
Having the handset call Twitter isn't a bad thing by any means. Pretty much every Twitter client in the world does it. One benefit will be that the user will be authenticated to Twitter, and thus her full 350 calls per hour will be available to you. Keep in mind, however, that you should minimize your calls since the user may have other Twitter-aware applications installed on her handset eating into your call allotment, and vice versa.
Now to the solution. The way I would implement your use case would be to first fetch the complete list of friends for your user by calling the friends/ids method.
http://api.twitter.com/1/friends/ids.json?screen_name=yourUsersName
The above call will return the most recent 5,000 friend IDs, in order followed, for #yourUsersName. If you want to fetch more friend IDs than the first 5,000, you'll need to specify the cursor parameter to initiate paging.
Next, I would check the latest list of friends we just fetched against the list on the handset, syncing them by removing any IDs that are no longer present, while adding any that are new.
If we only need the friend IDs, then we're done at a cost of one API call per 5,000 friend IDs. If, however, we need to get user info for these new friends as well, then I would call users/lookup and pass in the list of all new users that we discovered while syncing friend IDs. You can request up to 100 user objects at a time.
http://api.twitter.com/1/users/lookup.json?user_id=123123,5235235,456243,4534563
You user must be authenticated in order to make the above request, but the call can fetch any Twitter user profiles you wish -- not just those that are friends of the authenticated user.
So, let's say for example that a user has 2,500 friends and has never used your app before. In that case, she would burn one call to fetch all of the friend IDs, and 25 calls for her friends' information. That's not too bad to get the app populated with data.
Subsequent calls should be more streamlined with probably only two calls burned (one for the IDs, and one to get the new friends).
Finally, once the data has been updated on the handset, the deltas for the IDs and user data can be gathered up and pushed to your server.
It may even be possible that your server application won't even have to interface with Twitter at all, and that should alleviate the 150 user limit you are encountering.
Some final notes:
Be sure to note in your app's privacy policy that you sync your user's friend list with your server.
I recommend specifying JSON as the return format for all Twitter API calls. It is a much more lightweight document format than XML, and you will typically transfer only about 1/3 to 1/2 as much data over the wire.
Pick a Twitter framework appropriate for your mobile device and your programming language. Twitter access is a commodity these days, and there's little to no reason to reinvent how to access the Twitter API.
I answered a similar question about an approach for efficiently fetching followers here.
Since you are making request on behalf of users you should make those requests be authenticated as those users. Then requests will count against each users own pool of 350 requests/hour.
I'm working on an app that allows users to search for a particular friend on Twitter (and eventually Facebook) and then send them a message (sort of).
My problem is, the API limits me to only getting 100 friends per request. For a user with a lot of friends, this could take many requests (even if I cache it) and will make my app hit the rate limit pretty quickly.
Is there an official (or unofficial) Twitter API for searching for only your friends?
The solution I have implemented for now is this: whenever a user logs in, iterate through each 100 block of friends and put them in the Rails.cache. They stay there until the user logs out and logs back in. Now that I know that the API requests are counted against the logged in user, I shouldn't need to worry about hitting the rate limit API since each user will have 350 requests per hour.
However, I have found a few problems with this, and I have a few thoughts on solutions:
Problem: We are storing a large amount of data to cache someone's friends.
Solution: It would be best if we could cache all twitter users who are friends of one of our users in one object (or hash) and also cache only the IDs of the friends for each user (which can be grabbed with far less API calls). This would create a bit of a slowdown, but would be far less storage required. Then, whenever a user logs in, we would simply update the global friend cache with any changes (i.e. picture, name, etc.).
Problem: My application still has to store this and figure out how to parse it; it's not very organized.
Solution: Extract this functionality into a new application that creates a better API for searching. If I accomplish this, I'll post an update here with a link.