I'm facing a very strange issue in my Splunk search. I have a data input coming from a REST API that returns a multi-level (nested) JSON response:
The entity node has several nodes, each node represents one access point. Each access point contains a field called ipAddress.
This API is being called every 5 min and response stored in Splunk. When I do a search to get the list of IP Addresses from one event I don't get all of them. For some reason, is like Splunk is reading only the first seven nodes inside entity, because when I do:
source="rest://AccessPointDetailsAPI" | head 1
Splunk shows only the following values on the field (7 values although there are around 27):
I'm using demo license if that matters. Why I cannot see all values ? If I change my search to look for a specific iPAddress on the response but not on the list it won't return records.
Thanks and regards,
I think I understand the problem now. So the event is a big json and Splunk is not properly parsing all fields on the big json.
We need to tell splunk to parse the specific field we need with spath and specifying the field:
yoursearch | spath output=myIpAddress path=queryResponse.entity{}.accessPointDetailsDTO.ipAddress | table myIpAddress
http://docs.splunk.com/Documentation/Splunk/5.0.4/SearchReference/Spath
But I think also is important to analyze if maybe the data input needs to be divided in multiple events rather than a single huge event.
Related
I have podio data with more number of column, but we need to fetch 5-6 column data through API. I attached column name screenshot. If we need only for example order id, city, country then how to write API query?
/item/app/{app_id}/filter/
If it is right, how to write query with selected column name with GET/POST.
The filter endpoint uses a POST body to filter which records to return, not which fields/columns to return. It is not possible to specify which fields/columns to return with an API call according to this SO thread from an old Podio support person.
If you are looking to remove fields from the query to reduce your datasource size within Klipfolio, I would recommend returning the API call in CSV format instead of JSON. Klipfolio support documents how to do this HERE by performing a GET operation and adding /csv to the end of the URL.
https://api.podio.com/item/app/Your-APP-ID/csv/
Good day everyone,
we are trying to have through the use of the integrations of the Apigee service of google all the rows in a bigquery table that have a certain value in a field.
this operation is quite easy to do, but when we have more than 200 lines as a result, problems arise.
The problem is that using the integration to connect to BigQuery I am not returning any listEntitiesPageToken value and not even any listEntitiesNextPageToken value
so i can't figure out how i can go about navigating the result pages
Has anyone had the same problem? What do you suggest?
In the tutorial: "https://cloud.google.com/apigee/docs/api-platform/integration/connectors-task#configure-the-connectors-task" is write : "For example, if you are expecting 1000 records in your result set, you can set the listEntitiesPageSize to 100. So when the Connectors task runs for the first time, it returns the first 100 records, the next 100 records in the second run and so on."
And there is a tip: "Use the listEntitiesPageSize parameter in conjunction with the listEntitiesPageToken parameter to navigate through the pages."
I used the tutorial to understand how to use the task for loop and I understood that I should create a "subintegration" which must be called by a "main integration" for each element present in a list / array.
But what what can i do since these tokens are empty?
I load JSON reports into Splunk and those reports have many arrays. When I search:
source=test| search "behavior.system.processes.process{}.fileactivities.fileCreated.call{}.path"="C:\\Windows*"
I often like to show the matching data. I use table to do so:
source=test| search "behavior.system.processes.process{}.fileactivities.fileCreated.call{}.path"="C:\\Windows*" | table "behavior.system.processes.process{}.fileactivities.fileCreated.call{}.path"
However, the issue is that this shows me all fileCreated of the matching event and not only the one starting with C:\Windows.
How do I filter that?
#joe-jeff
I posted answer on answers.splunk.com. Please check below link.
https://answers.splunk.com/answers/745093/only-select-matching-json-data.html
How can we obtain the output of the Freebase topic api (used like this from the web api) from offline dumps of the id: /m/09937 ?
Actually, I would like to obtain similar (if not the same) output without querying the web.
Collect all the triples (lines) for the desired subject (first column value)
Filter out unneeded stuff like /base and /user properties in second column
Sort by property name (second column value)
Construct JSON with a hierarchy to match type/property occurrences and populate it with the values from the third column
For extra credit, process the URI templates for key/value pairs to construct links to other web sites and attribution templates to create the corresponding attribution strings.
Of course, the whole point of the Topic API is to save you having to go to all this trouble.
The are some articles are written in several parts,
for example, I got those articles from IBM developer works:
Distributed data processing with
Hadoop, Part 1:Getting started
Distributed data processing with
Hadoop, Part 2:Going further
Distributed data processing with
Hadoop, Part 3: Application
development
I will index those three articles separately. And some one search certain keywords, it is possible the part3 is on the top of hit whle part1 is on the 32th. Therefor, if I list results page by page, the part1 and part3 will display on different page.
How can I make sure the hitted documents in the same series displayed together?
I guess in SQL, we can use "group by".
I believe what you are asking for is Field Collapsing, which is currently a trunk feature in Solr, and will be incorporated into the next Solr version.
If you want to roll your own, One possible way to do this is:
Add a "series id" field to each document that is a member of a series. You will have to ensure that this gets incremented for every new series.
Make an initial query to Lucene, and get a hit list.
For each hit, check to see if it has a series id; If it does, make another query by the series id in order to retrieve all the members of the series.
An alternative is to store the ids of all the series members in a field inside each member's document.