I have a very strange issue, in the same event there are two different values for the same field in the below format a.b.c="" and a.b.c="qwe123df".I need to get the second value but when listing, first value is getting selected which is empty.Is there some way to get the non-empty value for this field? Remember '.' means concatenate in Splunk.I have tried to use rex but no luck.
It's the field multi valued? If so, you can use eval field=mvindex(a.b.c,1) (multi value fields start at 0, so this will get the 2nd value)
Alternatively, you can use rex to only match for at least one character.
rex field=_raw "a.b.c=\"(?<value>.+)\""
Related
I want to extract 4 values out of one field, called msg, from a Splunk query; and the msg is in the form of:
msg: "Service call successful k1=v1 k2=v2 k3=v3 k4=v4 k5=v5 something else can be ignored"
keys are always static but values are not, for instance, v2 could be XXX or XXYYZZ; similarly possible values for v3 just have unpredictable length.
I query to get some sample results and hope to use Field Extractor to generate a regex, but the regex generated can't get all the values out and I guess it's probably because values are not having the same length?
Do I need to change my logging format by separating each key=value using a common? Or I am not using the field extractor correctly?
[Update1]: A few sample data:
msg:Service call successful k1=XXX k2=BBBB k3=Something I made up k4=YYYNNN k5=do not need to retrieve this value
msg:Service call successful k1=SSSSSS k2=AAA k3=This could contain space and comma, like this one k4=YYYNNM k5=can be ignored
I could change the logging format if it makes easier to query and extract fields. Will adding a separator like dot or pipe help?
Normally Splunk will pull key-value pairs out automatically
However, when it doesn't, go try your regular expression(s) on regex101 - the field extractor is often a good[ish] start, but rarely creates efficient (or complete) regular expressions
An inline version of this would be as follows (presuming the "value" half of the key-value pair is contiguous characters):
| rex field=_raw "k1=(?<k1>\S+)\s+k2=(?<k2>\S+)\s+k3=(?<k3>\S+)\s+k4=(?<k4>\S+)\s+k5=(?<k5>\S+)"
Normally I prefer to do sequential rex calls, in case something's out of order or missing, but if your data's consistent, this will work
Once you have it the way you want it, update your props.conf and transforms.conf as appropriate for the sourcetype
EDIT for updated sample data / comment response:
...
| rex field=_raw "k3=(?<k3>.+)\s+k4="
| rex field=_raw "k4=(?<k4>.+)\s+k5="
...
Have the following query index=app (splunk_server_group=bex OR splunk_server_group=default) sourcetype=rpm-web* host=rpm-web* "CACHE_NAME=RATE_SHOPPER" method = GET | stats count(eval(searchmatch("true))) as Hit, count(eval(searchmatch("found=false"))) as Miss
Need to make a pie chart of two values "Hit and Miss rates"
The field where it is possible to distinguish the values is Message=[CACHE_NAME=RATE_SHOPPER some_other_strings method=GET found=false]. or found can be true
With out knowing the structure of your data it's harder to say what exactly you need todo but,
Pie charts is a single data series so you need to use a transforming command to generate a single series. PieChart Doc
if you have a field that denotes a hit or miss (You could use an Eval statement to create one if you don't already have this) you can use it to create the single series like this.
Lets say this field is called result.
|stats count by result
Here is a link to the documentation for the Eval Command
Good luck, hope you can get the results your looking for
Since you seem to be concerned only about whether "found" equals either "hit" or "miss", try this:
index=app (splunk_server_group=bex OR splunk_server_group=default) sourcetype=rpm-web* host=rpm-web* "CACHE_NAME=RATE_SHOPPER" method=GET found IN("hit","miss")
| stats count by found
Pie charts require a single field so it's not possible to graph the Hit and Miss fields in a pie. However, if the two fields are combined into one field with two possible values, then it will work.
index=app (splunk_server_group=bex OR splunk_server_group=default) sourcetype=rpm-web* host=rpm-web* "CACHE_NAME=RATE_SHOPPER" method = GET
| eval result=if(searchmatch("found=true"), "Hit", "Miss")
| stats count by result
I'm new to Splunk and need some help with the following:
authIndexValue[] is an array that will hold at least one value
I want to access its value from inside a case in an eval statement but I get
this error: Unknown search command '0'.
I also tried http.request.queryParameters.authIndexValue{} with no luck
Below the eval line:
..search
| eval EventType=case(http.request.queryParameters.authIndexValue[0]==Login_FooBar, "LOGIN")
How can I achieve this?
Thanks.
Splunk doesn't have "arrays"
It has multivalue fields
To access/do anything with them, you need to use multivalue commands/functions
multivalue eval functions - https://docs.splunk.com/Documentation/Splunk/latest/SearchReference/MultivalueEvalFunctions
multivalue stats functions - https://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Multivaluefunctions
In your example, you'd want to do something like the following:
<search>
| eval EventType=case(mvindex(http.request.queryParameters.authIndexValue,0)==Login_FooBar,"LOGIN", mvindex(http.request.queryParameters.authIndexValue,0)==Login_BarFoo,"not a good login",1=1,"error state")
I want to extract a string from a string...and use it under a field named source.
I tried writing like this bu no good.
index = cba_nemis Status: J source = *AAP_ENC_UX_B.* |eval plan=upper
(substr(source,57,2)) |regex source = "AAP_ENC_UX_B.\w+\d+rp"|stats
count by plan,source
for example..
source=/p4products/nemis2/filehandlerU/encpr1/log/AAP_ENC_UX_B.az_in_aza_277U_ rp-20190722-054802.log
source=/p4products/nemis2/filehandlerU/encpr2/log/AAP_ENC_UX_B.oh_in_ohf_ed_ph_ld-20190723-034121.log
I want to extract the string \
AAP_ENC_UX_B.az_in_aza_277U_ rp from 1st
and
AAP_ENC_UX_B.oh_in_ohf_ed_ph_ld from 2nd.
and put it under the column source along with the counts..
I want results like...
source counts
AAP_ENC_UX_B.az_in_aza_277U_ rp 1
AAP_ENC_UX_B.oh_in_ohf_ed_ph_ld 1
You can use the [rex][1] command that extracts a new field from an existing field by applying a regular expression.
...search...
| rex field=source ".+\/(?<source_v2>[\.\w\s]+)-.+"
| stats count by plan, source_v2
Be careful, though: I called the new field source_v2, what you were asking would rewrite the existing source field without you explicitly requesting this. Just change source_v2 to source in my code in case this is what you want.
The search takes this new source_v2 field into account. Try and see if this is what you need. You can tweak it easily to get your expected results.
I'm trying to do an ad hoc search of records that contain duplicate values in the first and second elements of a multivalued UniData field. I was hoping something like this would work but I'm not having any luck.
LIST PERSON WITH EVAL "STATUS[1] = STATUS[2]"
After some testing it looks like I stumbled across a way of reading the field right to left that many characters. Interesting but not useful for what I need.
LIST PERSON NAME EVAL "NAME[3]" COL.HDG 'Last3'
PERSON Name Last3
0001 Smith ith
Any ideas on how to correctly select on specific field elements?
Apparently the EXTRACT function will let me specify an element but I still can't get a selection on it to work properly.
LIST PERSON STATUS EVAL "EXTRACT(STATUS,1,2,0)" COL.HDG 'Status2'
PERSON STATUS Status2
0001 Added Processed
Processed
I would use eval with #RECORD placeholder with the dynamic array notation as such (assuming that STATUS is in Attribute 11.
Edit:
Previous answer was how I would do this in UniVerse
SELECT PERSON WITH EVAL "#RECORD<11,1>" EQ EVAL "#RECORD<11,2>"
Script Wolf's more better way that works in UniVerse and UniData.
SELECT PERSON WITH EVAL "EXTRACT(#RECORD,11,1,0)" EQ EVAL "EXTRACT(#RECORD,11,2,0)"
Good Luck.