Now, I can get prices of apps using the search and lookup APIs. But I don't know how to monitor the prices. Should I check all the apps using API every day or even every some hours? It seems to be a huge task.
And here's another question. How can I get info of "all" the apps since the APIs needs keyword or id parameter.
You would want to index them initially, then index the most popular keywords that you're looking for, so you can search for those and update the prices if necessary. In terms of actually scanning the entire app store every few hours, that seems a bit much. Can you expand on what you'll be doing with this information?
Related
I’m trying to build a mobile app that involves users following each other. I’ve seen posts (here) that say it is a cardinal sin to store a users’ followees and followers as a list in a SQL database as each “cell” should only store one discrete value.
However, is this the case for noSQL, document-based databases? What are the pros and cons of storing followers and followees as a list in the user document, vs storing it in a separate collection?
The only ones i can see now is that retrieving the follower/followee data (could be?) faster for the former method as you don’t have to index the entire follower/followee collection, unlike the latter method (or is the time difference negligible?). On the other hand, one would require 2 writes every time someone follows/unfollows another user, which may be disadvantageous for billing in cloud databases, but might not be a problem if the database is hosted locally (?)
I’m very new to working with databases so I’m hoping to get some insight from more experienced people about long term/large scale effects of this choice. Thanks!
I've developed a react native application where users can login, chose different items from a list, see the details to the item (profile) and add/delete/edit different posts (attached to one item).
Now my user base has grown and therefor I have decided to introduce new database tables in order to log each action my users do to analyze the accumulated data later and optimize for example the usability etc.
My first question is: Is there any convention or standard that lists the data to be collected in such a case (like logtime, action, ...)? Don't want to lose any useful data because I've noticed the value of it too late.
And: In which time intervals should an app send the users logdata to my remote server (async requests after each action, daily, before logout...)? Is there any gold standard?
Actually it's more about how much data you would like to collect and if it is matching with your privacy terms and conditions. If you're going to store the data on some server other than yours to analyse it, it is highly recommended that you don' refer to user ids there, clearly for privacy reasons.
About when is the right time to log data, again it depends of the data you would like to track, if you are tracking how many minutes they spend on a screen or how they interact with some posts, you may need to send those regularly to your server depending on your needs: whether you want to analyse the data instantly to improve the user experience (show more relevant posts) or just to use the data later. If the data you need to analyse is not really that much, you can do it after each call, if you're planning to track huge amounts of data that you don't need right away, probably you could choose to send the data at time frames where you don't have a big load on your server (to save bandwidth, you can choose night time (it's a little bit more complicated than that))
With Sitecore Content Search configuration is it possible to support the addition of a field which is populated with a value at search time, not index time? The population would be from an in-memory data structure for performance.
Essentially without re-indexing the values need to be updated/accessed, examples for this real time field would be Facebook Likes, In Stock, or Real Time Pricing. This data would then be used for faceting such as items with a range of Facebook likes, in-stock versus out-of-stock, or real time price facets.
The content search api does the searching on an iindexable, so I would look into that - you'd probably have to implement this interface yourself.
More info here:
http://www.sitecore.net/learn/blogs/technical-blogs/sitecore-7-development-team/posts/2013/04/sitecore-7-search-operations-explained.aspx
If you need to search on data that is not in the index I would question whether sitecore search is the best option here. If the data needs to be searched in real time then maybe a database would suffice.
If the data set is large and you need realtime access then maybe a nosql database such as MongoDB might be the right choice. Hope this has given you some ideas and you reach a solution
You can leverage the Sitecore dynamic index. The idea is to query your "large" index from within your in-memory index which you'll use dynamically. The implementation is relatively easy.
More info: http://www.sitecore.net/en-gb/learn/blogs/technical-blogs/sitecore-7-development-team/posts/2013/04/sitecore-7-dynamic-indexes.aspx
I've been using the google distance matrix API, and so far it works very well! However, I've noticed that google put a limit of 1000 requests per day. I am making a program for a website with about 1000 users, and the way I plan to search through all the users would lead to a lot of requests every time someone searched for nearby people.
I was planning to save everyone's location in a database, and then when someone entered their own location, it would compare that location to each and every one in the database. This would lead to potentially hundreds/thousands of requests every time someone searched.
So would there be a better way to do this?
This sounds like a heavy request method you currently doing. And even if you buy the commercial solution I fear you will loose money very quickly.
Instead I suggest that you host a routing server on your own (e.g. GraphHopper where I am the author) and then you can query this until the CPU melts. Also for only a city the hardware requirements are very low.
I'm implementing a Leaderboard into my django web app and don't know the best way to do it. Currently, I'm just using SQL to order my users and, from that, make a Leaderboard, however, this creates two main problems:
Performance is shocking. I've only tried scaling it to a few hundred users but I can tell calculating ranking is slow and excessive caching is annoying since I need users to see their ranking after they are added to the Leaderboard.
It's near-impossible to tell a user what position they are without performing the whole Leaderboard calculation again.
I haven't deployed but I estimate about 5% updates to Leaderboard vs 95% reading (probably more, actually) the Leaderboard. So my latest idea is to calculate a Leaderboard again each time a user is added, with a position field I can easily sort by, and no need to re-calculate to display a user's ranking.
However, could this be a problem if multiple users are committing at the same time, will locking be enough or will rankings stuff up? Additionally, I plan to put this on a separate database solely for these leaderboards, which is the best? I hear good things about redis...
Any better ways to solve this problem? (anyone know how SO makes their leaderboards?)
I've written a number of leaderboards libraries that would help you out there. The one that would be of immediate use is python-leaderboard, which is based on the reference implementation leaderboard ruby gem. Using Redis sorted sets, your leaderboard will be ranked in real-time and there is a specific section on the leaderboard page with respect to performance metrics for inserting a large number of members in a leaderboard at once. You can expect to rank 1 million members in around 30 seconds if you're pipelining writes.
If you're worried about the data changing too often in real-time, you could operate Redis in a master-slave configuration and have the leaderboards pull data from the slave, which would only poll periodically from the master.
Hope this helps!
You will appreciate the concept of sorted sets in Redis.
Don't miss the paragraph which describes your problem :D
Make a table that stores user id and user score. Just pull the leader board using
ORDER BY user_score DESC
and join the Main table for the User name or whatever else you need.
Unless the total number of tests is a variable in your equation, the calculation from your ranking system should stay the same for each user so just update individual entries.