Splunk search if message is x for more than 5 minutes - splunk

I have two specific messages in splunk data that I'm searching for per user.
on-screen
off-screen
Anyone know how I can search in splunk for a user that is message="off-screen" for more than 5 minutes with a query checking every 2 minutes ?
index="document" (message="off-screen")
My query will be ran every 2 minutes so I want to check for the event with message off-screen. Then next time around check if 5 minutes have elapsed since the on-screen message was fired and that no on-screen event was fired in that time period for that user.
Is this possible ?

If you want to find off-screen messages that don't have an on-screen message within 5 minutes, then you can use a transaction. Let's say your raw data is:
| makeresults count=10
| streamstats count
| eval _time=_time-(count*60)
| eval message=case(count=1,"on-screen",count=2,"on-screen",count=5,"off-screen",count=8,"off-screen",count=9,"on-screen",count=10,"on-screen")
| eval user=case(count=1,"Alice",count=2,"Bob",count=5,"Alice",count=8,"Bob",count=9,"Alice",count=10,"Bob")
| where NOT isnull(user)
| table _time user message
That would look like this:
_time
user
message
2021-05-28 13:57:50
Alice
on-screen
2021-05-28 13:56:50
Bob
on-screen
2021-05-28 13:53:50
Alice
off-screen
2021-05-28 13:50:50
Bob
off-screen
2021-05-28 13:49:50
Alice
on-screen
2021-05-28 13:48:50
Bob
on-screen
You need a transaction that gathers the user's cooresponding on-screen and off-screen messages as long as they are within 5 minutes. But you need to keep the orphans where the off-screen message doesn't have a cooresponding on-screen message. Then you filter out the transactions that have both and you get just the orphans:
message="off-screen" OR message="on-screen"
| transaction user maxpause=5m keeporphans=true startswith="message=off-screen" endswith="message=on-screen"
| where mvcount(message)<2
| table _time user message
That would produce this output:
_time
user
message
2021-05-28 13:50:50
Bob
off-screen
Here is a runnable example:
| makeresults count=10
| streamstats count
| eval _time=_time-(count*60)
| eval message=case(count=1,"on-screen",count=2,"on-screen",count=5,"off-screen",count=8,"off-screen",count=9,"on-screen",count=10,"on-screen")
| eval user=case(count=1,"Alice",count=2,"Bob",count=5,"Alice",count=8,"Bob",count=9,"Alice",count=10,"Bob")
| where NOT isnull(user)
| table _time user message
| search message="off-screen" OR message="on-screen"
| transaction user maxpause=5m keeporphans=true startswith="message=off-screen" endswith="message=on-screen"
| where mvcount(message)<2
| table _time user message

Related

Show the sum of an event per day by user in Splunk

I want to be able to show the sum of an event (let's say clicks) per day but broken down by user type. The results I'm looking for will look like this:
User Role
01/01
01/02
01/03
...
Guest
500
450
348
55
Admin
220
200
150
75
Here is my initial start but I'm unsure how to do the pivots on this to produce a table and visual chart
earliest=-30d index=* role=Guest OR role=Admin | count clicks as clickCount | ...
I'm unsure on how to both only count by day but then also only count by role to render them as shown above. Thanks for the help in advance.
You can create a timechart by day and then untable, convert the _time into a day field with formatted mm/dd value, and then construct an xyseries with the rows as columns and the day as the header:
| timechart span=1d count by role as "User Role"
| untable _time name value
| eval day=strftime(_time, "%m/%d")
| xyseries name day value

Database Table Design / Setup

I'm trying to setup a PostgreSQL database for a calendar app, and was wondering what would be the preferred way to set up one of the tables.
If I want to have to have multiple users, would the best way to setup the table be:
UserID | Start Time | End Time | Activity |
Or would it be better to do:
User ID | Activity | 8am | 8:30am | 9am | ...| 12am | ... | 7:30am |
The time granularity could be 15 min or 5 min too if that would be the better choice.
The first way would likely be cleaner, but ideally I don't want to let things overlap, but the second way may be more difficult to work with?
Your 1st example is better. But, minimum of 3 tables.
User - you define it.
Timetable: RowID | UserID | Start Time (datetime) | End Time (datetime) | ActivityID
Activity - you define what activity details go in.

How to join between table DurationDetails and Table cost per program

How to design database for tourism company to calculate cost of flight and hotel per every program tour based on date ?
what i do is
Table - program
+-----------+-------------+
| ProgramID | ProgramName |
+-----------+-------------+
| 1 | Alexia |
| 2 | Amon |
| 3 | Sfinx |
+-----------+-------------+
every program have more duration may be 8 days or 15 days only
it have two periods only 8 days or 15 days .
so that i do duration program table have one to many with program .
Table - ProgramDuration
+------------+-----------+---------------+
| DurationNo | programID | Duration |
+------------+-----------+---------------+
| 1 | 1 | 8 for Alexia |
| 2 | 1 | 15 for Alexia |
+------------+-----------+---------------+
And same thing to program amon program and sfinx program 8 and 15 .
every program 8 or 15 have fixed details for every day as following :
Table Duration Details
+------+--------+--------------------+-------------------+
| Days | Hotel | Flight | transfers |
+------+--------+--------------------+-------------------+
| Day1 | Hilton | amsterdam to luxor | airport to hotel |
| Day2 | Hilton | | AbuSimple musuem |
| Day3 | Hilton | | |
| Day4 | Hilton | | |
| Day5 | Hilton | Luxor to amsterdam | |
+------+--------+--------------------+-------------------+
every program determine starting by flight date so that
if flight date is 25/06/2017 for program alexia 8 days it will be as following
+------------+-------+--------+----------+
| Date | Hotel | Flight | Transfer |
+------------+-------+--------+----------+
| 25/06/2017 | 25 | 500 | 20 |
| 26/06/2017 | 25 | | 55 |
| 27/06/2017 | 25 | | |
| 28/06/2017 | 25 | | |
| 29/06/2017 | 25 | 500 | |
+------------+-------+--------+----------+
And this is actually what i need how to make relations ship to join costs with program .
for flight and hotel costs as above ?
for 5 days cost will be 1200
25 is cost per day for hotel Hilton
500 is cost for flight
20 and 55 is cost per transfers
image display what i need
relation between duration and cost
Truthfully, I don't fully understand exactly what you're trying to accomplish. Your description is not clear, your tables seem to be missing information / contain information that should not be in your tables, and the way that I'm understanding your description doesn't really make sense based on the UI screenshot that you shared.
It looks like you're working on an application for a travel agency which will allow agents to create an itinerary for a trip. They can give this trip a name (so if a particular package is a hit with customers, they can just offer the "Alexa" package), and the utility will calculate the total estimated cost of the trip. If I understand correctly, the trips will be either 8, or 15 days long.
Personally, I would delete the "ProgramDuration" table altogether. If there are two versions of the Alexa trip at index 1, then you're going to run into all manners of issues. I can get into the details of why this is a bad idea, but unless you're really hung up on having this ProgramDuration table, it's not worth the time. You should add a "duration" field to your "program" table, and assign a new ProgramID for each different duration version of the "Alexa" program.
Your table "Duration details" also misses the mark. Your fields in this table will make it harder to add new features to your application down the line. You should have a field "ProgramID," which we will use to join this table against the program table later. You should have a field "Day" which obviously indicates the day in the itinerary. You should have only one more field "ItemID." We're going to use the "ItemID" field to join your itinerary against a new items table we're going to create.
Your items table is where you define all of the items that can possibly appear in an itinerary. Your current itinerary table has three possible "types" of expenses, flights, hotels, and transfers. What if your travel agents want to start adding meal expenditures into their itineraries / budgets? What about activities that cost money? What about currency exchange fees? What about items that your clientele will need before their trip (wall adapters, luggage, etc.)? In your items table, you will have fields for an ItemID, ItemName, ItemUnitPrice, and ItemType. A possible item is as follows:
ItemID: 1, ItemName: Night At The Hilton, ItemUnitPrice: 300, ItemType: Lodging
Using the "SELECT [Column] AS [Alias]" syntax with some CTEs or subqueries and the JOIN operator, we can easily reconstitute a table that looks like your "Program Duration Details" table, but we will be afforded considerably more flexibility to add or remove things later down the line.
In the interests of security and programmability, I would also add a table called "ItemTypeTable" with a single field "TypeName." You can use this table to prevent unauthorized users from defining new item types, and you can use this table to create drop down menus, navigation, and all manners of other useful features. There might be cleaner implementations, but this shouldn't represent a serious performance or size hit.
All in all, at the risk of being somewhat rude, it seems like you're trying to take on a rather large, sophisticated task with a very rudimentary understanding of basic relational database design and implementation. If you are doing this in a professional context, I would strongly encourage you to consider consulting with another professional that may be more experienced in this area.

Access Query: get difference of dates with a twist

I'm going to do my best to explain this so I apologize in advance if my explanation is a little awkward. If I am foggy somewhere, please tell me what would help you out.
I have a table filled with circuits and dates. Each circuit gets trimmed on a time cycle of about 36 months or 48 months. I have a column that gives me this info. I have one record for every time the a circuit's trim cycle has been completed. I am attempting to link a known circuit outage list, to a table with their outage data, to a table with the circuit's trim history. The twist is the following:
I only want to get back circuits that have exceeded their trim cycles by 6 months. So I would need to take all records for a circuit, look at each individual record, find the most recent previous record relative to the record currently being examined (I will need every record examined invididually), calculate the difference between the two records in months, then return only the records that exceeded 6 months of difference between any two entries for a given feeder.
Here is an example of the data:
+----+--------+----------+-------+
| ID | feeder | comp | cycle |
| 1 | 123456 | 1/1/2001 | 36 |
| 2 | 123456 | 1/1/2004 | 36 |
| 3 | 123456 | 7/1/2007 | 36 |
| 4 | 123456 | 3/1/2011 | 36 |
| 5 | 123456 | 1/1/2014 | 36 |
+----+--------+----------+-------+
Here is an example of the result set I would want (please note: cycle can vary by circuit, so the value in the cycle column needs to be in the calculation to determine if I exceeded the cycle by 6 months between trimmings):
+----+--------+----------+-------+
| ID | feeder | comp | cycle |
| 3 | 123456 | 7/1/2007 | 36 |
| 4 | 123456 | 3/1/2011 | 36 |
+----+--------+----------+-------+
This is the query I started but I'm failing really hard at determining how to make the date calculations correctly:
SELECT temp_feederList.Feeder, Temp_outagesInfo.causeType, Temp_outagesInfo.StormNameThunder, Temp_outagesInfo.deviceGroup, Temp_outagesInfo.beginTime, tbl_Trim_History.COMP, tbl_Trim_History.CYCLE
FROM (temp_feederList
LEFT JOIN Temp_outagesInfo ON temp_feederList.Feeder = Temp_outagesInfo.Feeder)
LEFT JOIN tbl_Trim_History ON Temp_outagesInfo.Feeder = tbl_Trim_History.CIRCUIT_ID;
I wasn't really able to figure out where I need to go from here to get that most recent entry and perform the mathematical comparison. I've never been asked to do SQL this complex before, so I want to thank all of you for your patience and any assistance you're willing to lend.
I'm making some assumptions, but this uses a subquery to give you rows in the feeder list where the previous completed date was greater than the number of months ago indicated by the cycle:
SELECT tbl_Trim_History.ID, tbl_Trim_History.feeder,
tbl_Trim_History.comp, tbl_Trim_History.cycle
FROM tbl_Trim_History
WHERE tbl_Trim_History.comp>
(SELECT Max(DateAdd("m", tbl_Trim_History.cycle, comp))
FROM tbl_Trim_History T2
WHERE T2.feeder = tbl_Trim_History.feeder AND
T2.comp < tbl_Trim_History.comp)
If you needed to check for longer than 36 months you could add an arbitrary value to the months calculated by the DateAdd function.
Also I don't know if the value of cycle specified the number of month from the prior cycle or the number of months to the next one. If the latter I would change tbl_Trim_History.cycle in the DateAdd function to just cycle.
SELECT tbl_trim_history.ID, tbl_trim_history.Feeder,
tbl_trim_history.Comp, tbl_trim_history.Cycle,
(select max(comp) from tbl_trim_history T
where T.feeder=tbl_trim_history.feeder and
t.comp<tbl_trim_history.comp) AS PriorComp,
IIf(DateDiff("m",[priorcomp],[comp])>36,"x") AS [Select]
FROM tbl_trim_history;
This query identifies (with an X in the last column) the records from tbl_trim_history that exceed the cycle time - but as noted in the comments I'm not entirely sure if this is what you need or not, or how to incorporate the other 2 tables. Once you see what it is doing you can modify it to only keep the records you need.

CQRS and Race : how to handle race requirements

While there are articles saying that race conditions do not occur in business world, and it the solution that what we need to look, I am not sure it is the case.
I have a need of capacity and do event ticketing. When the demand for the event is high there are many concurrent bookingCommands that come in the same microsecond. The traditional way to do this is to use locking to prevent RACE conditions. Otherwise it ends up selling tickets for seats that are not available which is a strict business no-no.
Below table shows the sequence of steps that occur concurrently.
Time | Total Capacity | Consumed | Available | Customer1 | customer2
1 | 100 | 99 | 1 |seat available?| -
2 | | | | apply | seat available ?
3 | | | | event handle | apply
4 | | 100 | 0 | update state | event handle
5 | | 101 | -1 | | update state
If "selling tickets for seats that are not available is a strict business no-no." then model it this way. What this requirement tells you is that "selling/reserving a seat" and "number of seats available" should end up in the same transaction and be consistent. You can't take reservation and fire an event to change the number of available seats, it has to be in single transaction. This way when you try to decrease "number of seats available" (Time-5 from your table) you will receive optimistic concurrency exception, because someone modified it in the meantime. Then you can try to process it again and this time number of available seats has been exhausted so you can publish "application/reservation rejected" event and notify the user.
Project "a CQRS Journey" is something you should have a look at:
The reference implementation will be a conference management system
that you will be able to easily deploy and run in your own
environment. This will enable you to explore and experiment with a
realistic application built following a CQRS-based approach.
Especially have a look at SeatsAvailability.MakeReservation and SeatsAvailabilityHandler.Handle(MakeSeatReservation command)