Insert and Select Data in Redis - redis

I need to enter the below data in Redis:
Atlanta_96_Bronze Ana_Moser Ida_Alvares Ana_Paula Hilma_Caldeira Leila_Barros Virna_Dias Marcia_Fu Ericleia_Bodziak Ana_Flavia_Sanglard Fernanda_Venturini Fofao_Helia_Souza Sandra_Suruagy
Sidney_00_Bronze Elisangela_Oliveira Erika_Coimbra Fofao_Helia_Souza Janina_Conceicao Karin_Rodrigues Katia_Lopes Kely_Fraga Leila_Barros Raquel_Silva Ricarda_Lima Virna_Dias Walewska_Oliveira
Pequim_08_Gold Marianne_Steinbrecher Fofao_Helia_Souza Paula_Pequeno Walewska_Oliveira Thaisa_Menezes Valeska_Menezes Welissa_Gonzaga Fabiana_Oliveira Fabiana_Claudino Sheilla_Castro Jaqueline_Carvalho Carolina_Albuquerque
Londres_12_Gold Fabiana_Claudino Dani_Lins Paula_Pequeno Adenizia_Silva Thaisa_Menezes Jaqueline_Carvalho Fernanda_Ferreira Tandara_Caixeta Natalia_Pereira Sheilla_Castro Fabiana_Oliveira Fernanda_Garay
And then perform the following queries:
Which players won gold and silver medals?
Which players won two gold medals?
Which players only won medal in '96?
Which players were in the 96, 00 and 08 Olympics?
Which players were only in the 12 olympics?
But I never touched Redis, I came from a relational world, I need help.

Redis doesn't really seem like the right tool for the job as you've described it, as Redis is more of a cache than a database. There's no concept of running a "query" in Redis.
If you must use Redis, I would recommend storing the data in multiple sets to facilitate getting to the answers you need.
You might make a set for each type of medal, for example (a set of gold medalists, a set of silver medalists, and a set of bronze medalists). Then you could ask for the union of the gold and silver sets (Redis's SUNION operator) to get the answer to your first question.
You might also make a set for each year, so that you could retrieve information by year (for your last three questions).
In some cases, there may be no way around doing some coding to refine the results to give you exactly the answers you need.

Related

IEXCloud "change percent" statistics make no sense

The results of stats calls to IEX Cloud make no sense.
Docs are at https://iexcloud.io/docs/api/#stats-basic but they don't go into the details. Just saying a field exists seems to be enough for them.
Here is a result for today for Agilent from an API call:
{"companyName":"Agilent Technologies Inc.","marketcap":48702022828,"week52high":179.35,"week52low":104.59,"week52highSplitAdjustOnly":179.57,"week52lowSplitAdjustOnly":105.19,"week52change":0.4686617632157193,"sharesOutstanding":302722668,"float":0,"avg10Volume":1289493,"avg30Volume":1291085,"day200MovingAvg":152.69,"day50MovingAvg":156.41,"employees":16400,"ttmEPS":3.21,"ttmDividendRate":0.7744734957186514,"dividendYield":0.004813982444795198,"nextDividendDate":"","exDividendDate":"2021-10-04","nextEarningsDate":"2021-11-22","peRatio":49.19396245236364,"beta":0.9917301080890041,"maxChangePercent":6.000809392433486,"year5ChangePercent":2.617394432702253,"year2ChangePercent":1.1149775855495814,"year1ChangePercent":0.4686617632157193,"ytdChangePercent":0.3655365001761244,"month6ChangePercent":0.2405176122966075,"month3ChangePercent":0.011620302832134222,"month1ChangePercent":0.0709625882039675,"day30ChangePercent":0.05432859296153092,"day5ChangePercent":0.028578735374976016}
For example "year5ChangePercent" with value 2.617. Agilent is up roughly 400% in the last 5 years, from about $44 to $160. So how on earth did they come up with 2.617?

How to find results being between two values of different keys with Redis?

I'm creating a game matchmaking system using Redis based on MMR, which is a number that pretty much sums up the skill of a player. Therefore the system can match him/her with others who are pretty much with the same skill.
For example if a player with MMR of 1000 joins the queue, system will try to find other ppl with MMR of range 950 to 1050 to match with this player. But if after one minute it cannot find any player with given stats it will scale up the range to 900 to 1100 (a constant threshold).
What I want to do is really easy with relational database design but I can't figure out how to do it with Redis.
The queue table implementation would be like this:
+----+---------+------+-------+
| ID | USER_ID | MMR | TRIES |
+----+---------+------+-------+
| 1 | 50 | 1000 | 1 |
| 2 | 70 | 1500 | 1 |
| 3 | 350 | 1200 | 1 |
+----+---------+------+-------+
So when a new player queues up, it will check it's MMR against other players in the queue if it finds one between 5% Threshold it will match the two players if not it will add the new player to the table and wait for new players to queue up to compare or to pass 1 minute and the cronjob increment the tries and retry to match players.
The only way I can imagine is to use two separate keys for the low and high of each player in the queue like this
MatchMakingQueue:User:1:Low => 900
MatchMakingQueue:User:1:High => 1100
but the keys will be different and I can't get for example all users in between range of low of 900 to high of 1100!
I hope I've been clear enough any help would be much appreciated.
As #Guy Korland had suggested, a Sorted Set can be used to track and match players based on their MMR, and I do not agree with the OP's "won't scale" comment.
Basically, when a new player joins, the ID is added to a zset with the MMR as its score.
ZADD players:mmr 1000 id:50
The matchmaking is made for each user, e.g. id:50 with the following query:
ZREVRANGEBYSCORE players:mmrs 1050 950 LIMIT 0 2
A match is found if two IDs are returned and at least one of them is different than that of the new player. To make the match, both IDs (the new player's and the matched with one) need to be removed from the set - I'd use a Lua script to implement this piece of logic (matching and removing) for atomicity and communication reduction, but it can be done in the client as well.
There are different ways to keep track of the retries, but perhaps the simplest one is to use another Sorted Set, where the score is that metric.
The following pseudo Redis Lua code is a minimal example of the approach:
local kmmrs, kretries = KEYS[1], KEYS[2]
local id = ARGV[1]
local mmr = redis.call('ZSCORE', kmmrs, id)
local retries = redis.call('ZSCORE', kretries, id)
local min, max = mmr*(1-0.05*retries), mmr*(1+0.05*retries)
local candidates = redis.call('ZREVRANGEBYSCORE', kmmrs, max, min, 'LIMIT', 0, 2)
if #candidates < 2 then
redis.call('ZINCRBY', kretries, 1, id)
return nil
end
local reply
if candidates[1] ~= id then
reply = candidates[1]
else
reply = candidates[2]
end
redis.call('ZREM', kmmrs, id, reply)
redis.call('ZREM', kretries, id, reply)
return reply
Let me get the problem right! Your problem is that you want to find all the users in a given range of MMR value. What if You make other users say that "I'm falling in this range".
Read about Redis Pub/Sub.
When a user joins in, publish its MMR to rest of the players.
Write the code on the user side to check if his/her MMR is falling in the range.
If it is, user will publish back to a channel that it is falling in that range. Else, user will silently discard the message.
repeat these steps if you get no response back in 1 minute.
You can make one channel (let's say MatchMMR) for all users to publish MMR for match request which should be suscribed by all the users. And make user specific channel in case somebody has a MMR in the calculated range.
Form you published messages such that you can send all the information like "retry count", "match range percentage", "MMR value" etc. so that your code at user side can calculate if it is the right fit for the MMR.
Read mode about redis Pub/Sub at: https://redis.io/topics/pubsub

NYT article search API not returning results for certain queries

I have a set of queries and I am trying to get web_urls using the NYT article search API. But I am seeing that it works for q2 below but not for q1.
q1: Seattle+Jacob Vigdor+the University of Washington
q2: Seattle+Jacob Vigdor+University of Washington
If you paste the url below with your API key in the web browser, you get an empty result.
Search request for q1
api.nytimes.com/svc/search/v2/articlesearch.json?q=Seattle+Jacob%20Vigdor+the%20University%20of%20Washington&begin_date=20170626&api-key=XXXX
Empty results for q1
{"response":{"meta":{"hits":0,"time":27,"offset":0},"docs":[]},"status":"OK","copyright":"Copyright (c) 2013 The New York Times Company. All Rights Reserved."}
Instead if you paste the following in your web browser (without the article 'the' in the query) you get non-empty results
Search request for q2
api.nytimes.com/svc/search/v2/articlesearch.json?q=Seattle+Jacob%20Vigdor+University%20of%20Washington&begin_date=20170626&api-key=XXXX
Non-empty results for q2
{"response":{"meta":{"hits":1,"time":22,"offset":0},"docs":[{"web_url":"https://www.nytimes.com/aponline/2017/06/26/us/ap-us-seattle-minimum-wage.html","snippet":"Seattle's $15-an-hour minimum wage law has cost the city jobs, according to a study released Monday that contradicted another new study published last week....","lead_paragraph":"Seattle's $15-an-hour minimum wage law has cost the city jobs, according to a study released Monday that contradicted another new study published last week.","abstract":null,"print_page":null,"blog":[],"source":"AP","multimedia":[],"headline":{"main":"New Study of Seattle's $15 Minimum Wage Says It Costs Jobs","print_headline":"New Study of Seattle's $15 Minimum Wage Says It Costs Jobs"},"keywords":[],"pub_date":"2017-06-26T15:16:28+0000","document_type":"article","news_desk":"None","section_name":"U.S.","subsection_name":null,"byline":{"person":[],"original":"By THE ASSOCIATED PRESS","organization":"THE ASSOCIATED PRESS"},"type_of_material":"News","_id":"5951255195d0e02550996fb3","word_count":643,"slideshow_credits":null}]},"status":"OK","copyright":"Copyright (c) 2013 The New York Times Company. All Rights Reserved."}
Interestingly, both queries work fine on the api test page
http://developer.nytimes.com/article_search_v2.json#/Console/
Also, if you look at the article below returned by q2, you see that the query term in q1, 'the University of Washington' does occur in it and it should have returned this article.
https://www.nytimes.com//aponline//2017//06//26//us//ap-us-seattle-minimum-wage.html
I am confused about this behaviour of the API. Any ideas what's going on? Am I missing something?
Thank you for all the answers. Below I am pasting the answer I received from NYT developers.
NYT's Article Search API uses Elasticsearch. There are lots of docs online about the query syntax of Elasticsearch (it is based on Lucene).
https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-query-string-query.html#query-string-syntax
If you want articles that contain "Seattle", "Jacob Vigdor" and "University of Washington", do
"Seattle" AND "Jacob Vigdor" AND "University of Washington"
or
+"Seattle" +"Jacob Vigdor" +"University of Washington"
I think you need to change encoding of spaces (%20) to + (%2B):
In your example,
q=Seattle+Jacob%20Vigdor+the%20University%20of%20Washington
When I submit from the page on the site, it uses %2B:
q=Seattle%2BJacob+Vigdor%2Bthe+University+of+Washington
How are you URL encoding? One way to fix it would be to replace your spaces with + before URL encoding.
Also, you may need to replace %20 with +. There are various schemes for URL encoding, so the best way would depend on how you are doing it.

How to generate combination with pig

I have a map like this
{Tim, [Badminton, Basketball]}
{Viola, [Badminton, Baseball]}
{David, [Basketball]}
....
I use pig to find which games can they play together
for example, Tim and Viola can play Badminton together
Tim, Viola, David can not play together
I also need to find what combination can play more than N types of ball games.
How can I do that?
It's straightforward if you change the way you present the data.
At the moment, you have :
{Tim, [Badminton, Basketball]}
{Viola, [Badminton, Baseball]}
Now, let consider you flat your map games and to have a two-columns dataset :
{Tim, Badminton}
{Tim, Basketball}
{Viola, Badminton}
{Viola, Baseball}
You group on the second column and you will immediatly have the persons that can to play together.
There is also the possibility to use DataFu Bag's join BagLeftOuterJoin. However, on your example, it may not worth it.

Related jobs in JSprit

Is it possible to handle relations between jobs (in this case services) in jsprit?
For instance one job has to start at the same time with one or more other jobs.
Or one job must not start before the end of another job (normal sequence).
If not, do you know other java libraries that can handle such kind of restrictions?
Thank you!
Yes you can handle such relations with jsprit.
It is a bit of coding, but - I think - easy to implement. To illustrate it, I prepared you five examples that are based on the Vehicle Routing Problems (VRP) developed by Christofides-Mingozzi-Toth's* (first benchmarking instance). Follow the links behind the individual headlines and you get to the code. Note that the first activity in a route is marked with a triangle. The labels denote jobIds.
noConstraints
capacityConstraints
job 13 and 21 in same route
job 13 and 21 in same route AND 21 before 13
job 13 and 21 in same route AND 13 right after 21
Visit http://jsprit.github.io/ and you get to a number of code examples and docs.
*N. Christofides, A. Mingozzi, and P. Toth. The vehicle routing problem. In N. Christofides, A. Mingozzi, P. Toth, and C. Sandi, editors, Combinatorial Optimization. John Wiley, Chichester, 1979.