Syntax highlighting of Splunk search results is missing when grouping events via Transactions command - splunk
I use List view in Splunk website.
Normally in the Splunk search results, the fields are highlighted with colours(red and green). Also, if one event is coloured with white background then the next event is coloured with dark background to make it readable.
Here is the Screenshot where I blurred the data as it has company information:
But when I use transactions command to group events together then there is no syntax highlighting available even though I am using List view. But, the 1st group of events are coloured together in light background and the second group of events are coloured together in dark background so that works fine.
Command:-
application_name=appname | transaction startswith="This is the start of the transaction" endswith="This is the end of the transaction"
Screenshot:
1 transaction Splunk event taken from search result:
{"cf_app_id":"uuid","cf_app_name":"app-name","deployment":"cf","event_type":"LogMessage","info_splunk_index":"splunk-index","ip":"ipaddr","message_type":"OUT","msg":"2022-12-22 19:11:30.242 DEBUG [app-name,02c11142eee3be456dc30ddb1b234d5f,f20222ba46461ea9] 28 --- [nio-8080-exec-1] classname : This is the start of the transaction","origin":"rep","source_instance":"0","source_type":"APP/PROC/WEB","timestamp":1671732690242714069}
{"cf_app_id":"uuid","cf_app_name":"app-name","deployment":"cf","event_type":"LogMessage","info_splunk_index":"splunk-index","ip":"ipaddr","message_type":"OUT","msg":"2022-12-22 19:11:30.242 DEBUG [app-name,02c11142eee3be456dc30ddb1b234d5f,f20222ba46461ea9] 28 --- [nio-8080-exec-1] classname : app log text","origin":"rep","source_instance":"0","source_type":"APP/PROC/WEB","timestamp":1671732690243292964}
{"cf_app_id":"uuid","cf_app_name":"app-name","deployment":"cf","event_type":"LogMessage","info_splunk_index":"splunk-index","ip":"ipaddr","message_type":"OUT","msg":"2022-12-22 19:11:30.242 DEBUG [app-name,02c11142eee3be456dc30ddb1b234d5f,f20222ba46461ea9] 28 --- [nio-8080-exec-1] classname : another app log","origin":"rep","source_instance":"0","source_type":"APP/PROC/WEB","timestamp":1671732690243306564}
{"cf_app_id":"uuid","cf_app_name":"app-name","deployment":"cf","event_type":"LogMessage","info_splunk_index":"splunk-index","ip":"ipaddr","message_type":"OUT","msg":"2022-12-22 19:11:30.242 DEBUG [app-name,02c11142eee3be456dc30ddb1b234d5f,f20222ba46461ea9] 28 --- [nio-8080-exec-1] classname : {\"data\":{\"fields\":[{\"__typename\":\"name\",\"field\":\"value\",\"field2\":\"value2\",\"field3\":\"value 3\",\"field4\":\"value4\",\"field5\":\"value5\",\"field6\":\"value6\",\"field7\":\"value7\",\"field8\":null,\"field9\":\"value9\",\"field10\":null,\"field11\":111059.0,\"field12\":111059.0,\"field13\":null,\"field14\":\"value14\",\"field15\":\"2018-10-01\",\"field16\":null,\"field17\":false,\"field18\":{\"field19\":\"value19\",\"fieldl20\":\"value20\",\"field21\":2.6,\"field22\":\"2031-10-31\",\"field23\":\"2017-11-06\"},\"field24\":{\"field25\":\"\",\"field26\":\"\"},\"field27\":{\"field28\":{\"field29\":0.0,\"field30\":0.0,\"field31\":240.63,\"field32\":\"2022-12-31\",\"field33\":0.0,\"field34\":\"9999-10-31\"}},\"field35\":[{\"field36\":{\"field37\":\"value37\"}},{\"field38\":{\"field39\":\"value39\"}}],\"field40\":{\"__typename\":\"value40\",\"field41\":\"value41\",\"field42\":\"value 42\",\"field43\":111059.0,\"field44\":\"2031-04-01\",\"field45\":65204.67,\"field46\":null,\"field47\":\"value47\",\"field48\":\"value48\",\"field49\":null,\"field50\":\"value50\",\"field51\":null,\"field52\":null}},{\"__typename\":\"value53\",\"field54\":\"value54\",\"field55\":\"value55\",\"field56\":\"value56\",\"field57\":\"value57\",\"field58\":\"value58\",\"field59\":\"9\",\"field60\":\"value60\",\"field61\":null,\"field62\":\"value62\",\"field63\":null,\"field64\":88841.0,\"field65\":38841.0,\"field66\":null,\"field67\":\"value67\",\"field68\":\"2018-10-01\",\"field69\":null,\"field70\":false,\"field71\":{\"field72\":\"value72\",\"field73\":\"value73\",\"field74\":2.6,\"field75\":\"2031-10-31\",\"field76\":\"2017-11-06\"},\"field77\":{\"field78\":\"\",\"field79\":\"\"},\"field80\":{\"field81\":{\"field82\":0.0,\"field83\":0.0,\"field84\":84.16,\"field85\":\"2022-12-31\",\"field86\":0.0,\"field87\":\"9999-10-31\"}},\"field88\":[{\"field89\":{\"field90\":\"value90\"}},{\"field91\":{\"field92\":\"value92\"}}],\"field93\":null},{\"__typename\":\"value94\",\"field95\":\"value95\",\"field96\":\"value96\",\"field97\":\"value97\",\"field98\":\"value98\",\"field99\":\"value99\",\"field100\":\"1\",\"field101\":\"value101\",\"field102\":null,\"field103\":\"value103\",\"field104\":\"359\",\"field105\":88025.0,\"field106\":79316.87,\"field107\":\"309\",\"field108\":\"value108\",\"field109\":\"2018-10-01\",\"field110\":\"2048-09-30\",\"field111\":false,\"field112\":{\"field113\":\"value113\",\"field114\":\"value114\",\"field115\":2.35,\"field116\":\"2031-10-31\",\"field117\":\"2017-11-06\"},\"field118\":{\"field119\":\"\",\"field120\":\"\"},\"field121\":{\"field122\":{\"field123\":341.58,\"field124\":0.0,\"field125\":155.33,\"field126\":\"2022-12-31\",\"field127\":186.25,\"field128\":\"2022-12-31\"}},\"field129\":[{\"field130\":{\"field131\":\"value131\"}},{\"field132\":{\"field133\":\"value133\"}}],\"field134\":null}]}}","origin":"rep","source_instance":"0","source_type":"APP/PROC/WEB","timestamp":1671732690243306564}
{"cf_app_id":"uuid","cf_app_name":"app-name","deployment":"cf","event_type":"LogMessage","info_splunk_index":"splunk-index","ip":"ipaddr","message_type":"OUT","msg":"2022-12-22 19:11:30.242 DEBUG [app-name,02c11142eee3be456dc30ddb1b234d5f,f20222ba46461ea9] 28 --- [nio-8080-exec-1] classname : This is the end of the transaction","origin":"rep","source_instance":"0","source_type":"APP/PROC/WEB","timestamp":1671732690870483226}
So even though I am using the List view it feels like I am seeing the Raw view.
Is there something I can change in the search query so that the syntax highlighting is applied for the search results when searching via transactions command?
EDIT:
The initial issue is resolved but that lead to another annoyance.
A transaction command is used to group Splunk events. In my case, each Splunk event is a JSON. So I converted all Splunk events of the transaction into JSON array and now the Syntax highlighting is applied. But the problem is that I need to manually expand each JSON using the plus icon. Is there an expand all option?
Syntax highlighting is not standard for all Splunk events. It is, however, done for JSON events. The output of the transaction command is not valid JSON so no syntax highlighting is done.
I want to preface this by saying, this wouldn't be good practice.
But regardless you could do the following.
| eval _raw=_raw+","
| transaction id
| eval _raw="["+rtrim(_raw,",")+"]"
Where the transaction line would be your own transaction command.
The reason for this behaviour, is that syntax highlights will just check that if _raw is a valid json(or xml) and highlight the _raw if it is.
when grouping events with the transaction command, the _raw becomes the multiple events appended onto one another. this does not produce a valid json therefore it won't be highlighted.
These eval function manipulate the events such that after the transaction they produce a valid json as it's a list of the individual json events.
If you are not using Json. and instead XML, you may be able to use a very similar method, but I can't tell due to the blurred pictures, looks like it is however.
Related
How do I transform array in search or elsewhere in dashboard
I have a search that is working fine
index=event_db environment=prod release = 2020150015
| timechart count as Events
However, I'd like to modify this to search for any release in an array of releases. I'm aware of the "in" operator.
The catch is that the array of releases I've been provided ("Releases") is formatted slightly differently like so:
[ver2020.15.0015, ver2020.15.0016, ver2020.22.0019] // in general, many more than 3!
Is there a way to use the in operator and some mapping to get
release in
[2020150015, 2020150016, 2020220019] ?
Can this be put in the search?
This is part of a panel so if it's simpler I could have code elsewhere to convert [ver2020.15.0015, ver2020.15.0016, ver2020.22.0019] into [2020150015, 2020150016, 2020220019]
However, as mentioned I'm a newbie so my knowledge of where to put code to transform an array is limited :)
I have a fieldset section and a panel with a query in it.
The "Releases" array is populated in the fieldset section as so:
<input type="text" token="Releases">
<label>Release or Releases</label>
<default>*</default>
</input>
The user enters ver2020.15.0015 or perhaps ver2020.15.*.
I can't just have the user enter 2020150015 as the ver2020.15.0015 format is used elsewhere.
Perhaps there's a way to create new field Releases_Alt right after getting this?
Let me know of any other info I can provide. As I said, I'm new to Splunk so I'm still struggling with terminology.
Try this query. It uses a subsearch to build the IN argument. Subsearches in Splunk run before the main search and the output of the subsearch replaces the subsearch itself. index=event_db environment=prod release IN ( [ | makeresults | eval Releases=replace ($Releases|s$, "[ver\.]+","") | return $Releases ] ) | timechart count as Events The makeresults command is there because even subsearches have to start with a generating command. makeresults creates a "dummy" event that allows other commands to work. The eval command does the work of converting release versions into the desired format. Note the use of |s with the Releases token. This construct ensures the contents of the token are enclosed in quotation marks, which is expected by the replace function. Finally, the return command with $ returns the results of the eval, but without the field name itself. Without it, the subsearch would return releases="2020150015, 2020150016, 2020220019", which wouldn't work.
UniQuery LIST without page breaks
This should be simple but it's just not working. I have a list of records that I want to display in the terminal without anything besides the list of record keys. No Headers, page gaps, etc. The problem is I can't get rid of the page gaps that keep chopping up my list. SORT INVENTORY HDR.SUPP COL.HDR.SUPP NO.PAGE NO.SPLIT COUNT.SUP
You need to change your TERM settings, ignore the 'type' option. Increase the TERM 'B' (HELP TERM) to the total number of records you are returning. For example if the report contains 100 records, you enter: : TERM [to get default settings] : TERM 80,100,132,60 : run LIST/SORT : TERM A,B,C,D [ use values from first TERM ] You need to run the term command prior to running your select.
Just in case you're trying to get this list of bare keys so you can copy/paste it somewhere else (like Excel), I'll throw out another option that may be helpful and save some scrolling: :sort INVENTORY TO DELIM 9 /tmp/inventory.txt Will run your command and dump the output (tab-delimited, though that's irrelevant with only one field) to a file in the tmp directory. :UDT.OPTIONS 91 ON Is helpful if you're dumping dates, numbers or anything that needs output formatting. UDT Options Command reference describes that: UDT.OPTIONS 91 U_LIST_TO_CONV affects saved queries on data that is defined in the dictionary with a conversion code. For example, when a date is defined as D4, the internal date is 9611, which the conversion code translates as 04/24/94. UniData does not convert the data before it saves UniQuery results to an ASCII file. With UDT.OPTIONS 91, you can force the conversion before UniData saves the ASCII file.
Whats wrong with Neo4j 2.0 Query?
I am trying to understand why the data is not showing up in my query. I was wondering if there is any way to troubleshoot whats going on.
Here is the current issue:
I have populated some data from existing test database to check the performance with a relation like this : (e:Event)-[:FOR_USER]->(u:User) when I get all the users and look at the property, I can see the data, but when I query the users using same data it says 0 records found.
Below image shows the 2 query:
Can some one please help me understand how to debug such issue in neo4j
EDIT
Issue is that the Browser is somehow truncating the multiple spaces in the result. Like in this case "User-May<space>1 2013 1:18AM" was displayed on both webadmin and new browser, but in reality it should have been "User-May<space><space>1 2013<space><space>1:18AM"
So no matter what I do I can't query the value as looks like duplicate space is truncated somewhere.
Tabular data as Micheal suggested is as below
{"id":"75307","labels":["User"],"properties":{"Name":"User-May 1 2013 1:18AM"}}
and what we are seeing is User-May 1 2013 1:18AM
Regards
Kiran
Use the following Cypher syntax in the browser:
MATCH (user:User { Name: "User-May 1 2013 1:18AM" })
RETURN user.Name as Name
As far as the rendering of multiple spaces being trimmed, that is a browser specific functionality. See screenshot below for example:
The text itself is preserved as it is returned from the Neo4j server. As you can see when I analyze the HTML element of the browser using Firebug, the redundant spaces are indeed there.
So again, this doesn't seem to be a bug with Neo4j, it's how the browser you are using renders the text. The browser expects redundant spaces to be encoded as like so: "Testing testing" which is HTML encoded as Testing testing
SharePoint Workflow Error: "Unable to transform the input lookup data into the requested type" BUT only on New Item Creation
FYI to start, I am aware of how to properly set up an update to a lookup, and am 99% positive I've done this correctly. I know this because When I set the workflow to automatically start when an Item is Changed, then it works perfectly. But when I simply change this setting so it will automatically start on New Item Creation, it Cancels the workflow and I get a "Coercion Failed: Unable to transform the input lookup data into the requested type." If both options are checked then it fails on creation, but simply clicking edit on the item properties, and the "Save" makes it work. The workflow is on a Document Library and works as follows; User selects the Work Task LookUp from a dropdown in the edit properties form after uploading, and then Saves the item (adding it to the document library). The workflow is suppose to then look at the Work Task LookUp selected, and pull the Account and Effective Date-Type lookUp ID's that Work Task item has, and sets the Document's identical fields to the same value. Here is the code for the workflow if it helps; If Current Item: Parent Task is not empty If Current Item: Sub Task is not empty Log Both are empty to workflow history list Then Set Account to Work Tasks:Account The Log Set Account to workflow history list Then Set Effective Date and Type to WorkTasks: Effective Date and Type The Log Set EffDateType to the workflow history list This is all done in one step. I also added additional steps to test if the account and effective date type fields have been set properly, and if not to set them again. But everytime I run the workflow on change and it works, it always correctly sets these fields based upon the first Step (posted above) and the additional check logs to the history that they are not needed. As an example, The lookUp for Integer for Tasks:Account is set to work as follows; Date Source: Work Tasks (a list) Field from Source: Account (a lookup) Return Field as: Lookup ID (as Integer) Find the List Item Field: Title (from the Work Tasks list) Value: Current Item: Parent Task (Which is a look up of the "Title" Field from Work Tasks List, and is set to return the Value as a LookUp Value (As Text)) The Effective Date and Type setting is pretty much identical. So anyone have any insight? I've tried running it as an impersonated Step, setting a workflow pause (for 1 minute), changing the lookup types incase I messed it up to start with, but ultimately the above workflow DOES work, but only when I set it to "Automatically start on the Change (edit) of an Item", NOT "Automatically start on New Item Creation" like I need to to do. Oh yes, fyi, I am using SPServices CascadingDropDown on the Work Task and Sub Task fields of the doc Library form, but I honestly do not believe this has anything to do with my issue. UPDATE: I've talked with another developer, and he believes it is due to the issue that the workflow is occuring too quickly, before the item creates an ID for itself, which it needs to conduct the lookUps. He had me add another "Pause Workflow" to the very top of my workflow code (above the If conditions) and set it for 1 minute. It then worked properly. Downside is we want this to labeling to occur as close to item creation as possible. Because a view of the library relies on grouping based upon Account and Effective Date and Type. To add to this downer, Microsoft's Pause Workflow only allows for 1 minute or more, and then the timer used for this is often off, resulting in a pause longer than that. So far, every test is currently showing 2 minutes minimum on the pause. A possible alternative solution for instantaniously populate the fileds is to use Javascript and SPServices to do the lookUp to the Task list to pull the account and effective date - type fields and then populate, but my Javascript is not very strong and I would need help doing this. If anyone has any suggestions, I would appreciate them.
(Answered in a question edit. Converted to a community wiki answer. See Question with no answers, but issue solved in the comments (or extended in chat) ) The OP wrote: I don't know if it is the ID for the item after further testing. I changed the start of the workflow to wait until a field in the item changes. I set it to wait until the ID field is not 0 (since you cannot set to null), and it still does not work. 6/14/2012 4:13 PM Comment System Account Waiting on ID 6/14/2012 4:13 PM Comment System Account Waiting complete on ID 6/14/2012 4:13 PM Error System Account Coercion Failed: Unable to transform the input lookup data into the requested type. I have tried other fields as well, like document ID value is not empty, and it will wait, log it finishing the wait, and then fail. UPDATE This issue has something to do with the Parent Task field. I have solved the issue without having to wait for a period of time by setting the change from above to wait until the Parent Task field is not empty. It then completes the workflow fine. Anyone know why there is a delay though? I've solved it, but still don't fully understand what takes it so long. The main fault has been solved (hence the answer), and the remaining point about the reasons for the delay would probably be a discussion point or not specific enough for SO. Any further clarification can be edited in here.
Formatting output so that Intellij Idea shows diffs for two texts
I would like to be able to print in the logs a message for which intellij idea would present a nice way of comparing two objects (strings). This happens automatically for the error message logged by a failed junit assert:
assertEquals("some\nString", "another\nString");
=>
org.junit.ComparisonFailure: <Click to see difference>
at org.junit.Assert.assertEquals(Assert.java:123)
at org.junit.Assert.assertEquals(Assert.java:145)
at com.something.DummyTest.testDummy(DummyTest.java:89)
The <Click to see difference> entry is actually displayed as a link in the output window of the Intellij Idea. When you click on the link, a compare window opens which shows the two values (just like you would compare two files).
Simply throwing an exception is not acceptable because I would like to log multiple objects to compare. I already tried logging a text, but I wasn't able to convince idea to compare the two texts.
IntelliJ IDEA is using the hardcoded regular expression. If the text matches the pattern, it will suggest to click to view the difference. The pattern is: expected:<bla-blah> but was:<blah-blah-blah> Output should match the format of assertEquals or assertThat. The exact patterns are somewhat scattered around the code in IDEA, but some are e.g. here.
I had the same Problem and found the solution in https://github.com/joel-costigliola/assertj-core/issues/1364#issuecomment-440800958 You should throw an org.junit.ComparisonFailure. Then IntelliJ will display the <Click to see difference>