In Splunk I'm trying to extract multiple parameters and values that do not equal a specific word from a string. For example:
Anything in this field that does not equal "negative", extract the parameter and value:
Field:
field={New A=POSITIVE, New B=NEGATIVE, New C=POSITIVE, New D=BAD}
Result:
New A=POSITIVE
New C=POSITIVE
New D=BAD
Try this search. It uses a regular expression to extract parameters and values where the value is not "NEGATIVE".
index=foo
| rex field=field max_match=0 "(?<result>New \w=(?!NEGATIVE)\w+)"
| mvexpand result
To extract the actual field/value pairs, add this to the end of Rich's solution
| rename _raw as _orig_raw, result AS _raw
| extract pairdelim="," kvdelim="=" clean_keys=true
| rename _orig_raw as _raw
Related
I have an array with a list of unique literal strings (ids) and I want to use the "in" keyword to test for set membership. I've used the following query, the ephemeral field "id" extracts the id from the message.
fields #timestamp,#message, #logStream
| filter #message like /mutation CreateOrder/
| parse #message 'Parameters: *}], "id"=>"*"}}, "graphql"*' as rest_of_message, id
| parse #message '"variables"=>{"createOrderInput"=>*}, "graphql"' as variables
| filter id in ["182841661","182126710"]
| sort #timestamp desc
| limit 10000
| display id, variables
It was my assumption that it would match any message whose ephemeral field "id" matches any of the literal ids in the array. However, it's only matching the message that contain the first literal id in the array.
I've searched for both ids using the "like" key word and they both come up in the selected period.
Is it possible to do what I want to do? Is there a better way of doing it?
I've got a bunch of fields which are double quoted with delimiters but for the life of me, I'm unable to get any regex to pull out what I need.
In short - the delimiters can be in any order and I just need the value that's between the double quotes after each delimiter. Some sample data is below, can anyone help with what regex might extract each value? I've tried
'delimiter_1=\\W+\\w+'
but I only seem to get the first word after the delimiter (unfortunately - they do have spaces in the value)
some content delimiter_1="some value" delimiter_2="some other value" delimiter_4="another value" delimiter_3="the last value"
The problem is returning a varying numbers of values from the regex function. For example, if you know that there will 4 delimiters, then you can use REGEXP_SUBSTR for each match, but if the text will have varying delimiters, this approach doesn't work.
I think the best solution is to write a function to parse the text:
create or replace function superparser( SRC varchar )
returns array
language javascript
as
$$
const regexp = /([^ =]*)="([^"]*)"/gm;
const array = [...SRC.matchAll(regexp)]
return array;
$$;
Then you can use LATERAL FLATTEN to process the returning values from the function:
select f.VALUE[1]::STRING key, f.VALUE[2]::STRING value
from values ('some content delimiter_1="some value" delimiter_2="some other value" delimiter_4="another value" delimiter_3="the last value"') tmp(x),
lateral flatten( superparser(x) ) f;
+-------------+------------------+
| KEY | VALUE |
+-------------+------------------+
| delimiter_1 | some value |
| delimiter_2 | some other value |
| delimiter_4 | another value |
| delimiter_3 | the last value |
+-------------+------------------+
i am trying to extract matched strings from the multivalue field and display in another column. I have tried various options to split the field by delimiter and then mvexpand and then user where/search to pull those data. I was trying to find if there is an easier way to do this without all this hassle in SPLUNK query.
Example: Lets say i have below multi-value column1 field with data separated by delimiter comma
column1 = abc1,test1,test2,abctest1,mail,send,mail2,sendtest2,new,code,results
I was splitting this column using delimiter |eval column2=split(column1,",") and using regex/where/search to search for data with *test* in this column and return results, where i was able to pull the results but the column1 still shows all the values abc1,test1,test2,abctest1,mail,send,mail2,sendtest2,new,code,results , what i want is either to trim1 column1 to show only words match with test or show those entries in new column2 which should only show this words test1,test2,abctest1,sendtest2 as they were only matching *test*.
I would appreciate your help, thanks.
Found the answer after posting this question, its just using exiting mvfilter function to pull the match resutls.
column2=mvfilter(match(column1,"test"))
| eval column2=split(column1,",") | search column2="*test*"
doesn't work, as the split creates a multi-value field, which is a single event containing a single field containing many values. The search for *test* will still find that event, even though it contains abc1, etc... as there is at least one field that is *test*.
What you can use is the mvfilter command to narrow down the multi-value field to the events you are after.
| eval column2=split(column1,",") | eval column2=mvfilter(match(column2,".*test.*"))
Alternatively to this approach, you can use a regular expression to extract what you need.
| rex field=column1 max_match=0 "(<?column2>[^,]*test[^,]*)"
Regardless, at the end, you would need to use mvjoin to join your multiple values into a single string
| eval column2=mvjoin(column2, ",")
Is it possible to create a metric that extracts a numeric value from a string in Cloudwatch logs so I can graph / alarm it?
For example, the log may be:
20190827 1234 class: File size: 64MB
I realize I can capture the space delimited fields by using a filter pattern like: [date, time, class, word1, word2, file_size]
And file_size will be "64MB", but how do I convert that to a numeric 64 to be graphed?
Bonus question, is there any way of matching "File size:" as one field instead of creating a field for each space delimited word?
Use abs to cast to number, or any other numberic function
Using Glob Expressions
fields #message
| parse #message "File size: *MB" as size
| filter abs(size)<64
| limit 20
Using Regular Expressions
fields #message
| parse #message /File size:\s+(?<size>\d+)MB/
| filter abs(size)<64
| limit 20
To learn how glob or regular expression can be used, see Cloud Watch Logs Query Syntax
I have a field where all values have the following format:
Knowledge:xyz,id:2907129
The id number always changes, however, all I want is the value of xyz.
I used the following to remove "Knowledge:"e
eval url=replace (url, "Open_KnowledgeZone:", "")
For the id portion, using ",id*" did not work within the eval replace function.
You'll want to use a regex. Something like:
rex field=url "(?<=Knowledge:)(?<AnyFieldName>.*)(?=,)"
Where <AnyFieldName> is the name you want the result field to be. This will select all characters after "Knowledge:" and before the ",".
Here is the regex in action outside of Splunk:
https://regex101.com/r/ofW0a1/1