Splunk query to create a table view? - splunk

This event is printed eveytime UserPin AreaCode AreaNum Sector Short Sem are unique for each userid and come only inside User Login successfully message with timestamp
"message":" *** User Login successfully credentials userid 2NANO-323254-7654-4 UserPin - 287654 AreaCode - 98765 AreaNum - 98765 Sector - 87612345 Short Sem - ZEB"
Below these two event are only printed when certain conditions are meet. I am very new in Splunk like a naive, how can we write a Splunk query such that take out the userid with UserPin AreaCode AreaNum Sector Short Sem which have the below printed event then only create a table with userid. If below two message are not printed with userid from above message then we should not consider the userid
"message": "User Failed to login userid - 2NANO-323254-7654-4"
"message": "User is from stackoverflow group, on XZ ABCE for userid - 2NAN0-323254-7654-4"
this is table structure where i want to fill values
UserId | UserPin | AreaCode | AreaNum | Sector | Short_Sem
I am very new in splunk can someone guide how to start to build where to look for the thing. Any hint or demo will work. Thank you
Example
"message":" *** User Login successfully credentials userid 2NANO-323254-7654-4 UserPin - 287654 AreaCode - 98765 AreaNum - 98765 Sector - 87612345 Short Sem - ZEB"
"message": "User Failed to login userid - 2NANO-323254-7654-4"
"message": "User is from stackoverflow group, on XZ ABCE for userid - 2NAN0-323254-7654-4"
"message":" *** User Login successfully credentials userid 2ABDO-54312-7654-4 UserPin - 287654 AreaCode - 98765 AreaNum - 98765 Sector - 87612345 Short Sem - ZEB"
"message":" *** User Login successfully credentials userid 2COMA-765234-8653-4 UserPin - 287654 AreaCode - 98765 AreaNum - 98765 Sector - 87612345 Short Sem - ZEB"
So we consider first only because that userid have has two more event with same userid and associated all the event have timestamp
UserId | UserPin| AreaCode | AreaNum | Sector | Short_Sem
2NANO-323254-7654-4 | 287654 | 98765 | 98765 | 87612345 | ZEB

This question is an expansion of your question at how to write splunk query to create a table view so the answer is an expansion of that answer.
First, use rex to extract the desired fields. Then the stats command will group the results by userid. Finally, use the table command to display the fields.
| makeresults
| eval data="\"message\":\" *** User Login successfully credentials userid 2NANO-323254-7654-4 UserPin - 287654 AreaCode - 98765 AreaNum - 98765 Sector - 87612345 Short Sem - ZEB\"
\"message\": \"User Failed to login userid - 2NANO-323254-7654-4\"
\"message\": \"User is from stackoverflow group, on XZ ABCE for userid - 2NAN0-323254-7654-4\"
\"message\":\" *** User Login successfully credentials userid 2ABDO-54312-7654-4 UserPin - 287654 AreaCode - 98765 AreaNum - 98765 Sector - 87612345 Short Sem - ZEB\"
\"message\":\" *** User Login successfully credentials userid 2COMA-765234-8653-4 UserPin - 287654 AreaCode - 98765 AreaNum - 98765 Sector - 87612345 Short Sem - ZEB\""
| eval data=split(data,"
")
| mvexpand data
| eval _raw=data
```Everything above is for demo purposes only```
```Extract fields```
| rex "message\":\s*\"\s*(?:\*+\s)?(?<msg>.*?)(?:userid|,)"
| rex "userid\s(?:-\s)?(?<userid>\S+)"
| rex "UserPin - (?<UserPin>\S+) AreaCode - (?<AreaCode>\S+) AreaNum - (?<AreaNum>\S+) Sector - (?<Sector>\S+) Short Sem - (?<Short_Sem>\S+)"
```Clean up the fields```
| eval userid=trim(userid,"\""), Short_Sem=trim(Short_Sem, "\"")
```Group results```
| stats values(*) as * by userid
```Filter events```
| search (msg="*User Failed to login*") OR (msg="*User is from stackoverflow group*")
| rename userid as UserId
| table UserId UserPin AreaCode AreaNum Sector Short_Sem

Related

How to make SQL SERVER Partition when there is no Group identifier on the original table?

I have the following table:
Name
Rol
FirstDate
Alice
Leader
01-01-2020
Bob
Follower
01-05-2020
Charles
Follower
03-06-2020
Art
Leader
04-01-2021
Will
Leader
05-01-2022
Susy
Follower
06-01-2023
I want to get this:
Name
Rol
GroupId
MemberId
Alice
Leader
1
1
Bob
Follower
1
2
Charles
Follower
1
3
Art
Leader
2
1
Will
Leader
3
1
Susy
Follower
3
2
Whenever the Rol shows a Leader, I want a new group. Inside each group, I want a MemberId.
I have tried to solve this using PARTITION OVER.
The above table is not the real table, but it serves the purpose of what I am trying to achieve.

Splunk search if message is x for more than 5 minutes

I have two specific messages in splunk data that I'm searching for per user.
on-screen
off-screen
Anyone know how I can search in splunk for a user that is message="off-screen" for more than 5 minutes with a query checking every 2 minutes ?
index="document" (message="off-screen")
My query will be ran every 2 minutes so I want to check for the event with message off-screen. Then next time around check if 5 minutes have elapsed since the on-screen message was fired and that no on-screen event was fired in that time period for that user.
Is this possible ?
If you want to find off-screen messages that don't have an on-screen message within 5 minutes, then you can use a transaction. Let's say your raw data is:
| makeresults count=10
| streamstats count
| eval _time=_time-(count*60)
| eval message=case(count=1,"on-screen",count=2,"on-screen",count=5,"off-screen",count=8,"off-screen",count=9,"on-screen",count=10,"on-screen")
| eval user=case(count=1,"Alice",count=2,"Bob",count=5,"Alice",count=8,"Bob",count=9,"Alice",count=10,"Bob")
| where NOT isnull(user)
| table _time user message
That would look like this:
_time
user
message
2021-05-28 13:57:50
Alice
on-screen
2021-05-28 13:56:50
Bob
on-screen
2021-05-28 13:53:50
Alice
off-screen
2021-05-28 13:50:50
Bob
off-screen
2021-05-28 13:49:50
Alice
on-screen
2021-05-28 13:48:50
Bob
on-screen
You need a transaction that gathers the user's cooresponding on-screen and off-screen messages as long as they are within 5 minutes. But you need to keep the orphans where the off-screen message doesn't have a cooresponding on-screen message. Then you filter out the transactions that have both and you get just the orphans:
message="off-screen" OR message="on-screen"
| transaction user maxpause=5m keeporphans=true startswith="message=off-screen" endswith="message=on-screen"
| where mvcount(message)<2
| table _time user message
That would produce this output:
_time
user
message
2021-05-28 13:50:50
Bob
off-screen
Here is a runnable example:
| makeresults count=10
| streamstats count
| eval _time=_time-(count*60)
| eval message=case(count=1,"on-screen",count=2,"on-screen",count=5,"off-screen",count=8,"off-screen",count=9,"on-screen",count=10,"on-screen")
| eval user=case(count=1,"Alice",count=2,"Bob",count=5,"Alice",count=8,"Bob",count=9,"Alice",count=10,"Bob")
| where NOT isnull(user)
| table _time user message
| search message="off-screen" OR message="on-screen"
| transaction user maxpause=5m keeporphans=true startswith="message=off-screen" endswith="message=on-screen"
| where mvcount(message)<2
| table _time user message

Recording earliest login time for each day

I need to return the earliest login time per day for a single username. However, some returns do not match the login from that date. Query below:
index=app_redacted_int_* sourcetype="redacted" SessionState="Active" UserName=ABCDE123
| rex field=UserRealName "(?<IDNUM>\d+$)"
| bucket _time span=1d as day
| eval day=strftime(_time,"%F")
| stats earliest(SessionStateChangeTime) as SesssionStateChangeTime by day IDNUM UserRealName UserName
Results:
day IDNUM UserRealName UserName SessionStateChangeTime
2020-07-23 123 John Smith ABCDE123 7/22/2020 09:48:52
2020-07-24 123 John Smith ABCDE123 7/23/2020 12:47:13
2020-07-25 123 John Smith ABCDE123 7/24/2020 07:23:01
2020-07-27 123 John Smith ABCDE123 7/27/2020 07:54:34
2020-07-28 123 John Smith ABCDE123 7/27/2020 07:54:34
2020-07-29 123 John Smith ABCDE123 7/28/2020 07:32:04
As you can see, some days are returning their earliest login as a login from the previous day. I need the dates on the left side and the right side to be matching, and I need this all together in one query, I already know how to do it one query at a time. Thanks for taking your time to help! It is greatly appreciated!
It would appear that on those dates you've binned, the earliest login time was from an earlier day
It appears you've conflated multiple dates in the data into expecting them to be "the same"
I would strongly suspect that SesssionStateChangeTime is not the field you want to look at - at least, not in the manner you're trying to now

SQL: Return first instance of an access request

I rarely use SQL and I'm having trouble with this task.
This table has all the information I need.
A person can request access to a sector of a building, and for that 2 types of approbations may be needed.
I have to discover the data about the first request a user did for each building it has access
If user Jon requested access to sector "A" and 2 weeks later for sector "B", I just have to return the information about
who approved sector's "A" and when.
I can find out the which sector was requested first and the date. But I don't know how to return the approvers since they are in another
row. I think the key here is to use the ticket number.
Records
ID EVENTDATE TICKET USER ACTION EVENT APPROVER BUILDING SECTOR STATUS
15 7/1/2015 12:25 17C9F862 4003321 New access request Started - OHIO IT_2 Running
14 7/1/2015 12:41 17C9F862 4003321 Approved Manager approval 4001719 OHIO IT_2 Running
12 7/1/2015 15:29 17C9F862 4003321 - Finished - OHIO IT_2 Finished: Approved
13 7/1/2015 15:29 17C9F862 4003321 Approved Director Approval 4003468 OHIO IT_2 Running
10 7/1/2015 20:57 897B9A0A 4003321 New access request Started - OHIO DEVELOPMENT Running
11 7/1/2015 20:57 F3DCFB96 4003321 New access request Started - OHIO INFRA_2 Running
9 7/1/2015 20:58 897B9A0A 4003321 Approved Manager approval 4001719 OHIO DEVELOPMENT Running
8 7/1/2015 20:58 F3DCFB96 4003321 Approved Manager approval 4001719 OHIO INFRA_2 Running
7 7/1/2015 21:01 F3DCFB96 4003321 Approved Director Approval 4001547 OHIO INFRA_2 Running
6 7/1/2015 21:01 F3DCFB96 4003321 - Finished - OHIO INFRA_2 Finished: Approved
4 7/1/2015 21:03 897B9A0A 4003321 - Finished - OHIO DEVELOPMENT Finished: Approved
5 7/1/2015 21:03 897B9A0A 4003321 Approved Director Approval 4001549 OHIO DEVELOPMENT Running
3 7/1/2015 21:22 3E18483E 4003321 Approval not needed Finished - OHIO IT_1 Finished: Approved
2 7/2/2015 9:48 F902EB9C 4003321 Approval not needed Finished - UTAH FINANCE Finished: Approved
1 7/2/2015 11:08 C186101C 4003321 Approval not needed Finished - OHIO INFRA_1 Finished: Approved
Desired Result:
USER Manager Approver Director Approver BUILDING SECTOR DATE
4003321 4001719 4003468 OHIO IT_2 7/1/2015 15:29
4003321 - - UTAH FINANCE 7/2/2015 9:48
SQL used
SELECT SEL.USER
, SEL.BUILDING
, SEL.SECTOR
, SEL.EVENTDATE
, SEL.TICKET
FROM ACCESSREQUESTS SEL
INNER JOIN
(
SELECT USER
, BUILDING
, SECTOR
, MIN(EVENTDATE) as data
FROM ACCESSREQUESTS
WHERE EVENT = 'Finished'
AND STATUS = 'Finished: Approved'
AND USER != 'null'
GROUP BY USER
, BUILDING
, SECTOR
ORDER BY USER
) RE ON RE.DATA = SEL.EVENTDATE
AND SEL.STATUS = 'Finished: Approved'
Sorry, I couldn't really tell from your sample data where one column's data ended and the next one started so I may have some columns wrong, but try adding in this:
left join (select
ticket
, approver
from accessrequests
where event = 'Approved Director Approval') as Director_Approval
on Director_Approval.ticket = sel.ticket
left join (select
ticket
, approver
from accessrequests
where event = 'Approved Manager Approval') as Manager_Approval
on Manager_Approval.ticket = sel.ticket
Then in your select add just add in
isnull(Manager_approval.approver,'') as Manager_approver,
isnull(Director_approval.approver,'') as director_approver
You may need to tack it into your subquery there, but that should hopefully point you in the right direction :)

Adding a type of heading to a query

Hey all, im not sure if this is possible but i am looking to add a heading to a query's output. The best way to describe this is just by showing what i would like the output to look like:
[heading here]
<skip row>
[Another Header here]
MemebersID | FirstName | LastName | Email | OrderDate
05466 Bob Barker Bob#aol.com 2010-05-11
05946 Jan Lowers JLo#aol.com 2010-09-11
06456 Tony Montana TonyM#aol.com 2010-01-11
So here's what it would look like with the headers in place:
Top Users Records
The Memebers
MemebersID | FirstName | LastName | Email | OrderDate
05466 Bob Barker Bob#aol.com 2010-05-11
05946 Jan Lowers JLo#aol.com 2010-09-11
06456 Tony Montana TonyM#aol.com 2010-01-11
Is that possible as the output of a query using MS SQL 2005?
#KM
Because the report looks like that when all said and done....
Top Users Records
The Memebers
MemebersID | FirstName | LastName | Email | OrderDate
05466 Bob Barker Bob#aol.com 2010-05-11
05946 Jan Lowers JLo#aol.com 2010-09-11
06456 Tony Montana TonyM#aol.com 2010-01-11
The Users
MemebersID | FirstName | LastName | Email | OrderDate | Bill
05466 Bob Barker Bob#aol.com 2010-05-11 $5.66
05946 Jan Lowers JLo#aol.com 2010-09-11 $95.05
06456 Tony Montana TonyM#aol.com 2010-01-11 $9.02
Any help would be great! Thanks :)
Adding formatting headings to the query is really an application issue, with not much you can do in TSQL, this is about the best you can do:
PRINT '[heading here]'
PRINT ' '
PRINT '[Another Header here]'
SELECT ....