Splunk to take the second queries result(field) into first query for Percentage Memory of Linux host - splunk

I am a newbie to SplunK.
I am trying to pull the Memory % of my Linux hosts which belong to a particular group called Database_hosts.
I am able to get the Memory % of a particular host if I provide that explicitly as host="host01.example.com" however, I'm looking to run this query against multiple hosts.
Multiple hosts which belong to Database_hosts group I can extract from the inputlookup cmdb_host.csv in Splunk.
Now, I can extract the hosts from inputlookup cmdb_host.csv where it contains the hosts in name field but I am clueless how to put my second query into my first query ie sourcetype=top pctMEM=* host="host01.example.com"
Both the queries working independently though.
My first query:
sourcetype=top pctMEM=* host="host01" OR host="host02"
| multikv
| dedup host
| rex field=pctMEM "(?<usage>\d+)"
| where usage> 40
| table host pctMEM
Result on run:
and this is my second query:
| inputlookup cmdb_host.csv
| search support_group="Database_hosts" NOT (fqdn IN("ap*", "aw*",""))
| table name
Result on run:
How I can use my second query output field name into first query's host= field?
Any help will be much appreciated.
EDIT: just tried but no luck:
sourcetype=top pctMEM=* host="[inputlookup cmdb_host.csv where support_group="Database_hosts" | table name]
| multikv
| dedup name
| rex field=pctMEM "(?<usage>\d+)"
| where usage>20
| table name pctMEM

You're very close. If you run the subsearch (the part inside square brackets) by itself and add | format then you'll see what is returned to the main search. It'll look something like ((name=host01) OR (name=host02)). Combining that with the main search produces:
sourcetype=top pctMEM=* host=((name=host01) OR (name=host02))
| multikv
| dedup name
| rex field=pctMEM "(?<usage>\d+)"
| where usage>20
| table name pctMEM
which won't work. It can be fixed by renaming name to host in the subsearch and letting the subsearch create the expression.
sourcetype=top pctMEM=* [|inputlookup cmdb_host.csv where support_group="Database_hosts"
| return 100 host=name]
| multikv
| dedup name
| rex field=pctMEM "(?<usage>\d+)"
| where usage>20
| table name pctMEM
The return command tells Splunk to return up to 100 results and rename name to host. It's equivalent to fields name | rename name as host | format.

Related

how to write splunk query to create a dashboard

I have a Splunk log which contains a message at different time stamp with some case number
"message":"Welcome home user case num 1ABCD-201901-765-2  UserId - 1203 XV - 543 UserAd - 76542 Elect - 5789875 Later Code - QWERZX"
In below log few log message also get printed at different timestamp if certain conditions are met
"message":"Passed First class case num 1ABCD-201901-765-2"
"message":"Failed First class case num 1ABCD-201901-765-2"
"message":"Passed Second class case num 1ABCD-201901-765-2"
"message":"Fully Failed case num 1ABCD-201901-765-2"
"message":"Saved case num 1ABCD-201901-765-2"
"message":"Not saved case num 1ABCD-201901-765-2"
"message":"Not user to us case num 1ABCD-201901-765-2"
I want to create a table in Splunk dashboard to view using Splunk query with these columns list all the case numbers with the details
Case Num | XV | UserId | UserAd | Elect | Later Code | Passed First class | Passed Second class | Failed First class | Saved | Not saved | Not user to us
How to print true and false for these columns  Passed First class | Passed Second class | Failed First class | Saved | Not saved | Not user to us I want to check for each case num whether the case num is present in those logs if its present then print true for that column else false
I'm going to presume you have no field extractions yet built (except for message) for the sample data you provided, and that - as provided - it's in the correct format (though, since it seems to be missing timestamps, I can tell something is likely amiss)
This should get you down the right road:
index=ndx sourcetype=srctp message=*
| rex field=message "Passed (?<passed_attempt>\w+)"
| rex field=message "Failed (?<failed_attempt>\w+)"
| rex field=message "case num (?<case_num>\S+)"
| rex field=message "(?<saved>Not saved)"
| rex field=message "(?<saved>Saved)"
| rex field=message "UserId - (?<userid>\w+)"
| rex field=message "XV - (?<xv>\w+)"
| rex field=message "UserAd - (?<userad>\w+)"
| rex field=message "Elect - (?<elect>\w+)"
| rex field=message "Later Code - (?<later_code>\w+)"
| fields passed_attempt failed_attempt _time case_num xv userid elect later_code saved userad
| stats max(_time) as _time values(*) as * by userid case_num
I've used separate regular expressions to pull the fields because they're easier to read - they may (or may not) be more performant to combine.

Splunk dbxquery merge with splunk search

I am trying to merge Splunk search query with a database query result set. Basically I have a Splunk dbxquery 1 which returns userid and email from database as follows for a particualr user id:
| dbxquery connection="CMDB009" query="SELECT dra.value, z.email FROM DRES_PRINTABLE z, DRES.CREDENTIAL bc, DRES.CRATTR dra WHERE z.userid = bc.drid AND z.drid = dra.dredid AND dra.value in ('xy67383') "
Above query outputs
VALUE EMAIL
xv67383 xyz#test.com
Another query is a Splunk query 2 that provides the user ids as follows:
index=index1 (host=xyz OR host=ABC) earliest=-20m#m
| rex field=_raw "samlToken\=(?>user>.+?):"
| join type=outer usetime=true earlier=true username,host,user
[search index=index1 source="/logs/occurences.log" SERVER_SERVER_CONNECT NOT AMP earliest=#w0
| rex field=_raw "Origusername\((?>username>.+?)\)"
| rex field=username"^(?<user>,+?)\:"
| rename _time as epoch1]
| "stats count by user | sort -count | table user
This above query 2 returns a column called user but not email.
What I want to do is add a column called email from splunk dbxquery 1 for all matching rows by userid in output of query 1. Basically want to add email as additional field for each user returned in query 2.
What I tried so far is this but it does not give me any results. Any help would be appreciated.
index=index1 (host=xyz OR host=ABC) earliest=-20m#m
| rex field=_raw "samlToken\=(?>user>.+?):"
| join type=outer usetime=true earlier=true username,host,user
[search index=index1 source="/logs/occurences.log" SERVER_SERVER_CONNECT NOT AMP earliest=#w0
| rex field=_raw "Origusername\((?>username>.+?)\)"
| rex field=username"^(?<user>,+?)\:"
| rename _time as epoch1]
| "stats count by user | sort -count
| table user
| map search="| | dbxquery connection=\"CMDB009\" query=\"SELECT dra.value, z.email FROM DRES_PRINTABLE z, DRES.CREDENTIAL bc, DRES.CRATTR dra WHERE z.userid = bc.drid AND z.drid = dra.dredid AND dra.value in ('$user'):\""
Replace $user with $user$ in the map command. Splunk uses a $ on each end of a token.
The username field is not available at the end of the query because the stats command stripped it out. The only fields available after stats are the ones mentioned in the command (user and count in this case). To make the username field available, add it to the stats command. That may, however, change your results.
| rex field=_raw "samlToken\=(?<user>.+?):"
| join type=outer usetime=true earlier=true username,host,user
[search index=index1 source="/logs/occurences.log" SERVER_SERVER_CONNECT NOT AMP earliest=#w0
| rex field=_raw "Origusername\((?<username>.+?)\)"
| rex field=username"^(?<user>,+?)\:"
| rename _time as epoch1]
| stats count by user, username | sort -count
| table user, username
| map search="| dbxquery connection=\"CMDB009\" query=\"SELECT dra.value, z.email FROM DRES_PRINTABLE z, DRES.CREDENTIAL bc, DRES.CRATTR dra WHERE z.userid = bc.drid AND z.drid = dra.dredid AND dra.value in ('$user'):\""```

Combining two rowsets in ADLA without join on clause

I've got two types of input files I'm loading into an ADLA job. In one, I've got a bunch of data (left) and in another, I've got a list of values that are important to me (right).
As an example here, let's say I'm using the following in my "left" rowset:
| ID | URL |
|----|-------------------------|
| 1 | https://www.google.com/ |
| 2 | https://www.yahoo.com/ |
| 3 | https://www.hotmail.com/|
I'll have something like the following in my right rowset:
| ID | Name | Regex | Exceptions | Other Lookup Val |
|----|-------|-------------|------------|------------------|
| 1 | ThisA | /[a-z]{3,}/ | abc | 091238 |
| 2 | ThatA | /[a-z]{3,}/ | xyz | lksdf9 |
| 3 | OtherA| /[a-z]{3,}/ | def | 098143 |
As each are loaded via an EXTRACT statement, both are in separate rowsets. Ideally, I'd like to be able to load all the values for both rowsets and loop through the right one to run a series of calculations against the left one to find a match per various business rules. Notably, there's no value to simply join on, nor is it a simple Regex evaluation, but rather something a bit more involved. Thus, the output might just look something like the "left" rowset:
| ID | URL |
|----|-------------------------|
| 1 | https://www.google.com/ |
| 3 | https://www.hotmail.com/|
Now, a COMBINER is the only UDO I see that accepts two rowsets, but the U-SQL syntax requires that I do some sort of join statement here. There's no common identifier between each of the rowsets though, so there's nothing to join on, which suddenly makes this seem less ideal. Of the attribute options defined at https://learn.microsoft.com/en-us/azure/data-lake-analytics/data-lake-analytics-u-sql-programmability-guide#use-user-defined-combiners, I'd like to specify this as a Full because I'd need each of the left values available to evaluate against each of the right ones, but again, no shared identifier to do this on.
I then tried to use a REDUCER that accepted an IRowset in the IReducer constructor as a parameter, then tried to just pass the rowset in from the U-SQL, but it didn't like that syntax.
Is there any way to perform this custom combining in a manner that doesn't require a JOIN ON clause?
It sounds like you may be able to use an IProcessor. This would allow you to analyze each row in the RIGHT set and add a column (with a value based on your business rules) that you can subsequently use to join to the LEFT set.
[Adding a bit more detail]: You could also do this twice, once for the left and once for the right to create an artificial join column, like row_number or some such.

How to spool three columns from a table and check the summation of the third column in UNIX shell script

I have created a query which results three columns. I am able to fetch the details in the spool file and based on that i am checking a condition that the sum of all the values (numerical) from the third column is 0 or not. If not 0, then in the mail the complete result should come.
Issues i am facing are:
1) When i am writing simple SELECT query for the three columns, the results are not coming as three columns and single row for single record. But it is displaying as one row for each column value.
i.e. in TOAD the result is as:
|Column_name_1 | Column_name_2 | Column_name_3 |
+--------------+-----------------+----------------+
| text_1 | text_2 | num_1 |
| text_3 | text_4 | num_2 |
But in the spool file, i am getting result as--
|text_1 |
|text_2 |
|num_1 |
| text_3 |
| text_4 |
| num_2 |
2) The other issue is i am not getting any header in the spool file.
Can anyone please look into this and let me know how to proceed.
Try adding SET RECSEP OFF to fix the issue 1, which will solve your problem for record seperation.
Add SET HEADING ON to print the column headers.
See this link for a learning.

How to get numbers arranged right to left in sql server SELECT statements

When performing SELECT statements including number columns (prices, for example), the result always is left to right ordered, which reduces the readability. Therefore I'm searching a method to format the output of number columns right to left.
I already tried to use something like
SELECT ... SPACE(15-LEN(A.Nummer))+A.Nummer ...
FROM Artikel AS A ...
which gives close results, but depending on font not really. An alternative would be to replace 'SPACE()' with 'REPLICATE('_',...)', but I don't really like the underscores in output.
Beside that this formula will crash on numbers with more digits than 15, therefore I searched for a way finding the maximum length of entries to make it more save like
SELECT ... SPACE(MAX(A.Nummer)-LEN(A.Nummer))+A.Nummer ...
FROM Artikel AS A ...
but this does not work due to the aggregate character of the MAX-function.
So, what's the best way to achieve the right-justified order for the number-columns?
Thanks,
Rainer
To get you problem with the list box solved have a look at this link: http://www.lebans.com/List_Combo.htm
I strongly believe that this type of adjustment should be made in the UI layer and not mixed in with data retrieval.
But to answer your original question i have created a SQL Fiddle:
MS SQL Server 2008 Schema Setup:
CREATE TABLE dbo.some_numbers(n INT);
Create some example data:
INSERT INTO dbo.some_numbers
SELECT CHECKSUM(NEWID())
FROM (VALUES (1),(1),(1),(1),(1),(1),(1),(1),(1),(1))X(x);
The following query is using the OVER() clause to specify that the MAX() is to be applied over all rows. The > and < that the result is wrapped in is just for illustration purposes and not required for the solution.
Query 1:
SELECT '>'+
SPACE(MAX(LEN(CAST(n AS VARCHAR(MAX))))OVER()-LEN(CAST(n AS VARCHAR(MAX))))+
CAST(n AS VARCHAR(MAX))+
'<'
FROM dbo.some_numbers SN;
Results:
| COLUMN_0 |
|---------------|
| >-1486993739< |
| > 1620287540< |
| >-1451542215< |
| >-1257364471< |
| > -819471559< |
| >-1364318127< |
| >-1190313739< |
| > 1682890896< |
| >-1050938840< |
| > 484064148< |
This query does a straight case to show the difference:
Query 2:
SELECT '>'+CAST(n AS VARCHAR(MAX))+'<'
FROM dbo.some_numbers SN;
Results:
| COLUMN_0 |
|---------------|
| >-1486993739< |
| >1620287540< |
| >-1451542215< |
| >-1257364471< |
| >-819471559< |
| >-1364318127< |
| >-1190313739< |
| >1682890896< |
| >-1050938840< |
| >484064148< |
With this query you still need to change the display font to a monospaced font like COURIER NEW. Otherwise, as you have noticed, the result is still misaligned.