I have a Splunk log which contains a message at different time stamp with some case number
"message":"Welcome home user case num 1ABCD-201901-765-2 UserId - 1203 XV - 543 UserAd - 76542 Elect - 5789875 Later Code - QWERZX"
In below log few log message also get printed at different timestamp if certain conditions are met
"message":"Passed First class case num 1ABCD-201901-765-2"
"message":"Failed First class case num 1ABCD-201901-765-2"
"message":"Passed Second class case num 1ABCD-201901-765-2"
"message":"Fully Failed case num 1ABCD-201901-765-2"
"message":"Saved case num 1ABCD-201901-765-2"
"message":"Not saved case num 1ABCD-201901-765-2"
"message":"Not user to us case num 1ABCD-201901-765-2"
I want to create a table in Splunk dashboard to view using Splunk query with these columns list all the case numbers with the details
Case Num | XV | UserId | UserAd | Elect | Later Code | Passed First class | Passed Second class | Failed First class | Saved | Not saved | Not user to us
How to print true and false for these columns Passed First class | Passed Second class | Failed First class | Saved | Not saved | Not user to us I want to check for each case num whether the case num is present in those logs if its present then print true for that column else false
I'm going to presume you have no field extractions yet built (except for message) for the sample data you provided, and that - as provided - it's in the correct format (though, since it seems to be missing timestamps, I can tell something is likely amiss)
This should get you down the right road:
index=ndx sourcetype=srctp message=*
| rex field=message "Passed (?<passed_attempt>\w+)"
| rex field=message "Failed (?<failed_attempt>\w+)"
| rex field=message "case num (?<case_num>\S+)"
| rex field=message "(?<saved>Not saved)"
| rex field=message "(?<saved>Saved)"
| rex field=message "UserId - (?<userid>\w+)"
| rex field=message "XV - (?<xv>\w+)"
| rex field=message "UserAd - (?<userad>\w+)"
| rex field=message "Elect - (?<elect>\w+)"
| rex field=message "Later Code - (?<later_code>\w+)"
| fields passed_attempt failed_attempt _time case_num xv userid elect later_code saved userad
| stats max(_time) as _time values(*) as * by userid case_num
I've used separate regular expressions to pull the fields because they're easier to read - they may (or may not) be more performant to combine.
Related
I am new to Splunk queries and I am not able to figure out how to extract multiple values from same event. I am working with events that look like this :
...
starting count: 12345678
ending count: 12347890
total time: ...
....
I want to extract the values associated with "starting count" and "ending count" and create a chart comparing these two values.
So far I am able to extract one set of value using this query
rex field=_raw "starting count: (?<StartCount>\d+)"
But how can I extract two different values and compare? Thanks in advance.
If you are going to make a chart, does that means you have multiple events and each event contains a starting count and ending count?
If so, extract the starting count and the ending count with a rex (just like you suggested) and then eval the difference. Somthing like:
| rex field=_raw "starting count: (?<StartCount>\d+)"
| rex field=_raw "ending count: (?<EndCount>\d+)"
| eval difference=EndCount-StartCount
| table _time StartCount EndCount difference
Here is a "run anywhere" version that makes it's own test data:
| makeresults count=2
| streamstats count
| eval _raw=if(count=1,"starting count: 12345678 ending count: 12346789 total time: ...","starting count: 12347890 ending count: 12349999 total time: ...")
| eval _time=if(count=1,_time-1,_time)
| rex field=_raw "starting count: (?<StartCount>\d+)"
| rex field=_raw "ending count: (?<EndCount>\d+)"
| eval difference=EndCount-StartCount
| table _time StartCount EndCount difference
I am trying to merge Splunk search query with a database query result set. Basically I have a Splunk dbxquery 1 which returns userid and email from database as follows for a particualr user id:
| dbxquery connection="CMDB009" query="SELECT dra.value, z.email FROM DRES_PRINTABLE z, DRES.CREDENTIAL bc, DRES.CRATTR dra WHERE z.userid = bc.drid AND z.drid = dra.dredid AND dra.value in ('xy67383') "
Above query outputs
VALUE EMAIL
xv67383 xyz#test.com
Another query is a Splunk query 2 that provides the user ids as follows:
index=index1 (host=xyz OR host=ABC) earliest=-20m#m
| rex field=_raw "samlToken\=(?>user>.+?):"
| join type=outer usetime=true earlier=true username,host,user
[search index=index1 source="/logs/occurences.log" SERVER_SERVER_CONNECT NOT AMP earliest=#w0
| rex field=_raw "Origusername\((?>username>.+?)\)"
| rex field=username"^(?<user>,+?)\:"
| rename _time as epoch1]
| "stats count by user | sort -count | table user
This above query 2 returns a column called user but not email.
What I want to do is add a column called email from splunk dbxquery 1 for all matching rows by userid in output of query 1. Basically want to add email as additional field for each user returned in query 2.
What I tried so far is this but it does not give me any results. Any help would be appreciated.
index=index1 (host=xyz OR host=ABC) earliest=-20m#m
| rex field=_raw "samlToken\=(?>user>.+?):"
| join type=outer usetime=true earlier=true username,host,user
[search index=index1 source="/logs/occurences.log" SERVER_SERVER_CONNECT NOT AMP earliest=#w0
| rex field=_raw "Origusername\((?>username>.+?)\)"
| rex field=username"^(?<user>,+?)\:"
| rename _time as epoch1]
| "stats count by user | sort -count
| table user
| map search="| | dbxquery connection=\"CMDB009\" query=\"SELECT dra.value, z.email FROM DRES_PRINTABLE z, DRES.CREDENTIAL bc, DRES.CRATTR dra WHERE z.userid = bc.drid AND z.drid = dra.dredid AND dra.value in ('$user'):\""
Replace $user with $user$ in the map command. Splunk uses a $ on each end of a token.
The username field is not available at the end of the query because the stats command stripped it out. The only fields available after stats are the ones mentioned in the command (user and count in this case). To make the username field available, add it to the stats command. That may, however, change your results.
| rex field=_raw "samlToken\=(?<user>.+?):"
| join type=outer usetime=true earlier=true username,host,user
[search index=index1 source="/logs/occurences.log" SERVER_SERVER_CONNECT NOT AMP earliest=#w0
| rex field=_raw "Origusername\((?<username>.+?)\)"
| rex field=username"^(?<user>,+?)\:"
| rename _time as epoch1]
| stats count by user, username | sort -count
| table user, username
| map search="| dbxquery connection=\"CMDB009\" query=\"SELECT dra.value, z.email FROM DRES_PRINTABLE z, DRES.CREDENTIAL bc, DRES.CRATTR dra WHERE z.userid = bc.drid AND z.drid = dra.dredid AND dra.value in ('$user'):\""```
I am a newbie to SplunK.
I am trying to pull the Memory % of my Linux hosts which belong to a particular group called Database_hosts.
I am able to get the Memory % of a particular host if I provide that explicitly as host="host01.example.com" however, I'm looking to run this query against multiple hosts.
Multiple hosts which belong to Database_hosts group I can extract from the inputlookup cmdb_host.csv in Splunk.
Now, I can extract the hosts from inputlookup cmdb_host.csv where it contains the hosts in name field but I am clueless how to put my second query into my first query ie sourcetype=top pctMEM=* host="host01.example.com"
Both the queries working independently though.
My first query:
sourcetype=top pctMEM=* host="host01" OR host="host02"
| multikv
| dedup host
| rex field=pctMEM "(?<usage>\d+)"
| where usage> 40
| table host pctMEM
Result on run:
and this is my second query:
| inputlookup cmdb_host.csv
| search support_group="Database_hosts" NOT (fqdn IN("ap*", "aw*",""))
| table name
Result on run:
How I can use my second query output field name into first query's host= field?
Any help will be much appreciated.
EDIT: just tried but no luck:
sourcetype=top pctMEM=* host="[inputlookup cmdb_host.csv where support_group="Database_hosts" | table name]
| multikv
| dedup name
| rex field=pctMEM "(?<usage>\d+)"
| where usage>20
| table name pctMEM
You're very close. If you run the subsearch (the part inside square brackets) by itself and add | format then you'll see what is returned to the main search. It'll look something like ((name=host01) OR (name=host02)). Combining that with the main search produces:
sourcetype=top pctMEM=* host=((name=host01) OR (name=host02))
| multikv
| dedup name
| rex field=pctMEM "(?<usage>\d+)"
| where usage>20
| table name pctMEM
which won't work. It can be fixed by renaming name to host in the subsearch and letting the subsearch create the expression.
sourcetype=top pctMEM=* [|inputlookup cmdb_host.csv where support_group="Database_hosts"
| return 100 host=name]
| multikv
| dedup name
| rex field=pctMEM "(?<usage>\d+)"
| where usage>20
| table name pctMEM
The return command tells Splunk to return up to 100 results and rename name to host. It's equivalent to fields name | rename name as host | format.
I have ported a database to a new laptop by using an sql dump from the old computer and then executing this dump on the new computer.
Thea data was copied correctly, and all the SQL statements regarding the tables work perfectly.
For creating new relations i need to know the values of some of the sequences in the DB:
italo=> \ds
List of relations
Schema | Name | Type | Owner
--------+--------------------------+----------+-------
public | Conjugation_conjType_seq | sequence | italo
public | Conjugation_idConj_seq | sequence | italo
public | NounItalian_idNounIt_seq | sequence | italo
public | VerbItalian_idVerbIt_seq | sequence | italo
public | times_idTime_seq | sequence | italo
(5 rows)
When i use currval() in a select statement i get an error:
italo=> SELECT currval('VerbItalian_idVerbIt_seq');
ERROR: relation "verbitalian_idverbit_seq" does not exist
LINE 1: SELECT currval('VerbItalian_idVerbIt_seq');
When i look at a sequence in phppgadmin, i get the same error message about the missing relation followed by a column view of the sequence. The error message reads
SQL error:
ERROR: relation "public.verbitalian_idverbit_seq" does not exist
In statement:
SELECT pg_catalog.has_sequence_privilege('public.VerbItalian_idVerbIt_seq','SELECT,USAGE')
Name | Start value | Last value | Increment by | Max value | Min value | Cache value | Log count | Can cycle? | Will increment last value before returning next value (is_called)?
VerbItalian_idVerbIt_seq | 100 | | 1 | 9223372036854775807 | 1 | 1 | | No | No
Alter Set value Increment value Restart Reset Show all sequences
(the values for "Last value" and "Log count" are empty)
When i click on "Set value", i can enter a value for "Last value", and there is small message "Sequence value set.", but in the view nothing has changed.
The behavior is the same for all 5 sequences.
What do i have to do to fix my sequences so that i can use currval() on them?
EDIT: when i create a new sequence from scratch, it behaves as expected
Because of the usage of UpperCase characters, you have to use double quotes " around the identifiers:
SELECT currval('"VerbItalian_idVerbIt_seq"');
I have a some fields like this:
Group_servers|Name_server|Status**
Group1| server1|OK
Group1| server2|OK
Group2| server1|OK
Group2| server1|No data
Group2| server1|Yellow
Group2| server1|
I want to get the result as shown below
Group_servers|Status
Group1|OK
Group1| No data
Сonditions for the formation of status groups are as follows:
1. If at least one server in the group has the status "No data" or the field is empty, the status for the group is " No data"
2. If at least one server in the group has the "Yellow" status, the status for the group is " Yellow"
3. If all servers in the group have the status "OK", the status for the group is " OK"
Here are two ways, one may be clearer than the other
| fillnull value="No data" Status | stats values(Status) as StatusList by Group_servers will give you something like the following
Group_servers|StatusList
------------------------
Group1 |OK
------------------------
Group2 |No data
|Yellow
------------------------
You can then use mvfind to determine what values are present for each group.
| fillnull value="No data" Status | stats values(Status) as StatusList by Group_servers | eval Status=if(isnotnull(mvfind(StatusList,"No data")),"NoData",( isnotnull(mvfind(StatusList,"Yellow")),"Yellow","OK"))
As an alternative, you could do something like the following, which just assigns a numeric score to each Status, and then gets the minimum for each group.
eval status_code=case(Status="OK",2, Status="Yellow",1,1==1,0) | stats min(status_code) as min_status_code by Group_servers | eval Status=case(min_status_code=2,"OK",min_status_code=1,"Yellow",1==1,"No data")