I receive exchange rate from an external web service and I log the response received like below (note both line contain data from a single response):
com.test.Currency#366c1a1e[Id=<Null>,Code=<Null>,Feedcode=Gbparslite,Rate=<Null>,Percentaqechangetrigger=<Null>,Bid=93.4269,Offer=93.43987,Mustinvertprice=False],
com.test.Currency#54acb93a[Id=<Null>,Code=<Null>,Feedcode=Gbphkdlite,Rate=<Null>,Percentaqechangetrigger=<Null>,Bid=10.04629,Offer=10.04763,Mustinvertprice=False],
I want to set up an alert which triggers when the last x (x=5) values received did not changed.
Assuming you're looking to alert when a particular field doesn't change after 5 events, you can try the following.
index=data | head 5 | stats dc(Bid) as dv
Then alert if dv equals 1. dc(Bid) calculates the number of unique values of Bid, in this case, over the last 5 events. If there is no difference, they will be 1. If there are multiple values, dc(Bid) will be greater than 1
Related
I'm creating a game matchmaking system using Redis based on MMR, which is a number that pretty much sums up the skill of a player. Therefore the system can match him/her with others who are pretty much with the same skill.
For example if a player with MMR of 1000 joins the queue, system will try to find other ppl with MMR of range 950 to 1050 to match with this player. But if after one minute it cannot find any player with given stats it will scale up the range to 900 to 1100 (a constant threshold).
What I want to do is really easy with relational database design but I can't figure out how to do it with Redis.
The queue table implementation would be like this:
+----+---------+------+-------+
| ID | USER_ID | MMR | TRIES |
+----+---------+------+-------+
| 1 | 50 | 1000 | 1 |
| 2 | 70 | 1500 | 1 |
| 3 | 350 | 1200 | 1 |
+----+---------+------+-------+
So when a new player queues up, it will check it's MMR against other players in the queue if it finds one between 5% Threshold it will match the two players if not it will add the new player to the table and wait for new players to queue up to compare or to pass 1 minute and the cronjob increment the tries and retry to match players.
The only way I can imagine is to use two separate keys for the low and high of each player in the queue like this
MatchMakingQueue:User:1:Low => 900
MatchMakingQueue:User:1:High => 1100
but the keys will be different and I can't get for example all users in between range of low of 900 to high of 1100!
I hope I've been clear enough any help would be much appreciated.
As #Guy Korland had suggested, a Sorted Set can be used to track and match players based on their MMR, and I do not agree with the OP's "won't scale" comment.
Basically, when a new player joins, the ID is added to a zset with the MMR as its score.
ZADD players:mmr 1000 id:50
The matchmaking is made for each user, e.g. id:50 with the following query:
ZREVRANGEBYSCORE players:mmrs 1050 950 LIMIT 0 2
A match is found if two IDs are returned and at least one of them is different than that of the new player. To make the match, both IDs (the new player's and the matched with one) need to be removed from the set - I'd use a Lua script to implement this piece of logic (matching and removing) for atomicity and communication reduction, but it can be done in the client as well.
There are different ways to keep track of the retries, but perhaps the simplest one is to use another Sorted Set, where the score is that metric.
The following pseudo Redis Lua code is a minimal example of the approach:
local kmmrs, kretries = KEYS[1], KEYS[2]
local id = ARGV[1]
local mmr = redis.call('ZSCORE', kmmrs, id)
local retries = redis.call('ZSCORE', kretries, id)
local min, max = mmr*(1-0.05*retries), mmr*(1+0.05*retries)
local candidates = redis.call('ZREVRANGEBYSCORE', kmmrs, max, min, 'LIMIT', 0, 2)
if #candidates < 2 then
redis.call('ZINCRBY', kretries, 1, id)
return nil
end
local reply
if candidates[1] ~= id then
reply = candidates[1]
else
reply = candidates[2]
end
redis.call('ZREM', kmmrs, id, reply)
redis.call('ZREM', kretries, id, reply)
return reply
Let me get the problem right! Your problem is that you want to find all the users in a given range of MMR value. What if You make other users say that "I'm falling in this range".
Read about Redis Pub/Sub.
When a user joins in, publish its MMR to rest of the players.
Write the code on the user side to check if his/her MMR is falling in the range.
If it is, user will publish back to a channel that it is falling in that range. Else, user will silently discard the message.
repeat these steps if you get no response back in 1 minute.
You can make one channel (let's say MatchMMR) for all users to publish MMR for match request which should be suscribed by all the users. And make user specific channel in case somebody has a MMR in the calculated range.
Form you published messages such that you can send all the information like "retry count", "match range percentage", "MMR value" etc. so that your code at user side can calculate if it is the right fit for the MMR.
Read mode about redis Pub/Sub at: https://redis.io/topics/pubsub
I have a search query like below.
index = abc_dev sourcetype = data RequestorSystem = * Description="Request Receieved from Consumer Service"
OR Description="Total Time taken in sending response"
| dedup TId
| eval InBoundCount=if(Description="Request Receieved from Consumer Service",1,0)
| eval OutBoundCount=if(Description="Total Time taken in sending response",1,0)
| stats sum(InBoundCount) as "Inbound Count",sum(OutBoundCount) as "Outbound Count"
I am not sure why inbound count is always showing as 0, outbound count works perfectly
There is a typo in your eval InBoundCount=... Received is spelled wrong, and if your events have it spelled correctly it won't match!
If that's not the case:
try running the query for both counts separately and make sure you are getting events. Also, posting some example input events will make our answer be more precise.
Splunk queries are joined by an implicit AND which means that your OR needs to either be included in parenthesis or (if you are using Splunk 6.6 or newer) use the IN keyword like so:
index = abc_dev sourcetype = data RequestorSystem = *
Description IN ("Request Receieved from Consumer Service", "Total Time taken in sending response")
Using IN is more portable in case you want add other strings later on. With some tweaks, you could even use a variation of stats count by Description with this.
I have a server with 2 APIs: /migrate/start and /migrate/end
For each request, I log the userID (field usrid="") of the user using my service to be migrated and the api called (field api="").
Users call /migrate/start, then call /migrate/end. I would like to write a slunk query to list the userIDs that are being migrated, i.e. those that called /migrated/start but have yet to call /migrate/end. How would I write that query?
Thank you
Assuming you have only 2 api calls (start/end) in the logs, you can use a stats command to do this.
| your_search
| stats values(api) as api by usrid
| where api!="/migrate/end"
This clubs all api calls done per user and removes the ones which have called /migrate/end
The general method is to get all the start and end events and match them up by user ID. Take the most recent event for each user and throw out the ones that are "migrate/end". What's left are all the in-progress migrations. Something like this:
index = foo (api="/migrate/start" OR api="/migrate/end")
| stats latest(api) by usrid
| where api="/migrate/start"
I wanted to print number of lines for an event in Splunk after querying it
Ex: index=* host=* source=*application* this query is giving all the events but I want to print/get number of lines for each and every event. I tried with len() of Splunk query but it didnt work for me
Actually every event contains 1 line only.
Even though size of the event is large but it is 1 json response only, so its treating complete response as 1 line response.
we can query based on characters
index=index-source host=hs* source=*appname* "TXNEnd" "SUCCESS" NOT "truncated" |where len(_raw) > 3000 |stats count by methodPath
I have JMeter plan that starts with a single JDBC sampler query that captures session ID from the Teradata database (SELECT SESSION;). Same plan also has large number of JDBC samplers with complicated queries producing large output that I don't want to include in the report.
If I configure summary report and tick Save Response Data (XML) then the output from all sampler queries will be saved
How do I add only first query result (it's a single integer) into the test summary report and ignore results from all other queries? For example is there a way to set responseData = false after the first query output is captured?
Maybe sample_variables property can help?
Define something in "Variable Names" section of the JDBC Request, i.e. put session reference name there like:
Add the next line to user.properties file (lives in Jmeter's "bin" folder)
sample_variables=session_1
or alternatively pass it via -J command-line argument like:
jmeter -Jsample_variables=session_1 -n -t /path/to/testplan.jmx -l /path/to/results.csv
You need to use session_1 not session. As per JDBC Request Sampler documentation:
If the Variable Names list is provided, then for each row returned by a Select statement, the variables are set up with the value of the corresponding column (if a variable name is provided), and the count of rows is also set up. For example, if the Select statement returns 2 rows of 3 columns, and the variable list is A,,C, then the following variables will be set up:
A_#=2 (number of rows)
A_1=column 1, row 1
A_2=column 1, row 2
C_#=2 (number of rows)
C_1=column 3, row 1
C_2=column 3, row 2
So given your query returns only 1 row containing 1 integer - it will live in session_1 JMeter Variable. See Debugging JDBC Sampler Results in JMeter article for comprehensive information on working with database query results in JMeter.
When test completes you'll see an extra column in .jtl results file holding your "session" value:
Although not exactly solving your question as posted, I will suggest a workaround, using a "scope" of a listener (i.e. listener will only record items on the same or lower level than a listener itself). Specifically: have two Summary Reports: one on the level of test, the other (together with the sampler whose response you want to record) under a controller. For example:
here I have samplers 1, 2, 3, 4. I only want to save response data from sampler 2. So
Summary Report - Doesn't save responses is on global level, and it's configured to not save any response data. It only saves what I want to save for all samplers.
Summary Report - Saves '2' only is configured to save response data in XML format. But because this instance of Summary Report is under the same controller as sampler 2, but other samplers (1, 3, 4) are on higher level, it will only record responses of sampler 2.
So it doesn't exactly allow you to save response data from one sampler into the same file as all other Summary Report data. But at least you can filter which responses you are saving.
May be you can try assertion for ${__threadNum}
i.e. set condition for assertion as "${__threadNum}=1" and set your listner's "Log/display only" option as "successes"
This way it should log only the first response from samplers.