Create custom Kibana dashboard with custom query - apache

I have 2 log files, 1 for Apache and another for a custom app. To query the logs for errors, the Apache log has log level tag as ERROR while the custom app has a response code tag 500.
I have a dashboard that shows the total errors in Apache log that matches tag = Error and second for total errors that match response code = 500.
My challenge is how to combine both errors as one rather than separating it. I have searched online for documentation on how to define query as global error = total with tag = error in apache log TYPE & total with response = 500 in xxx log type.
Thanks

Assuming you don't want to somehow correlate Apache and application logs but simply want all events that match either condition, i.e. the sum of both types of errors, this works (adjust field names to taste):
(type:apache AND tags:error) OR (type:custom AND response_code:500)

Related

JMeter variable in GET request failing

I have a GET Request that is returning an XML that contains a TicketName. I have setup the Regular Expression Extractor with Debug Sampler. It is picking up the TicketName as required and is displaying it in the View Results Tree, with the correct name variable name ticketID_g1.
However when I pass that variable to the next GET request the test plan fails with Non HTTP response message: Socket closed.
The thing is that the GET request looks find when I look at the request tab in the Results Tree.
I have changed my regular expression a number of times with each one extracting the TicketName properly but each time I apply it as a variable the GET request fails. However if I copy the request showing in the Results Tree Request Tab and paste it directly into my browser I get the desired result.
I have been through the manuals and on-line tutorials and it appears that I am doing everything right but obviously I am missing something.
The 1st GET Request returns an XML that contains name="2019-05-09-16-59-54cmrpip000613_EDASERVE" needsPrompt
I am using the following regular expression to extract the name for my variable ticketID
name="([^"]+)" needsPrompt - This works
The Results Tree is showing the following response from the Debug Sampler -
ticketID_g1=2019-05-09-16-59-54cmrpip000613_EDASERVE
When I pass the ticketID variable to the next GET request
//localhost:8080/ibi_apps/rs?IBIRS_action=getReport&IBIRS_ticketName=${ticketID_g1}cmrpip000589_EDASERVE&IBIRS_service=defer
The Response tab in the Results Tree for the second GET request is showing that the request is good but is failing.
GET http://localhost:8080/ibi_apps/rs?IBIRS_action=getReport&IBIRS_ticketName=2019-05-09-16-59-54cmrpip000613_EDASERVE&IBIRS_service=defer
What I am expecting is that this second GET will run with the variable and return a report but is throwing the Non HTTP response message: Socket closed error.
You have below variable which is capturing ticket id.
ticketID_g1=2019-05-09-16-59-54cmrpip000613_EDASERVE
But, in the below request you are passing the same which also have repeated content that is
"cmrpip000589_EDASERVE"
Request:-//localhost:8080/ibi_apps/rs?IBIRS_action=getReport&IBIRS_ticketName=${ticketID_g1}cmrpip000589_EDASERVE&IBIRS_service=defer
Please pass the ticketID variable correctly and hopefully it solves the issue. If I am correct you request should look like:-
Request:-//localhost:8080/ibi_apps/rs?IBIRS_action=getReport&IBIRS_ticketName=${ticketID_g1}&IBIRS_service=defer

Counting the number of response codes in JMeter 4.0

I run some load tests (all endpoints) and we do have a known issue in our code: if multiple POST requests are sent in the same time we do get a duplicate error based on a timestamp field in our database.
All I want to do is to count timeouts (based on the message received "Service is not available. Request timeout") in a variable and accept this as a normal behavior (don't fail the tests).
For now I've added a Response Assertion for this (in order to keep the tests running) but I cannot tell if or how many timeout actually happen.
How can I count this?
Thank you
I would recommend doing this as follows:
Add JSR223 Listener to your Test Plan
Put the following code into "Script" area:
if (prev.getResponseDataAsString().contains('Service is not available. Request timeout')) {
prev.setSampleLabel('False negative')
}
That's it, if sampler will contain Service is not available. Request timeout in the response body - JMeter will change its title to False negative.
You can even mark it as passed by adding prev.setSuccessful(false) line to your script. See Apache Groovy - Why and How You Should Use It article fore more information on what else you can do with Groovy in JMeter tests
If you just need to find out the count based on the response message then you can save the performance results in a csv file using simple data writer (configure for csv only) and then filter csv based on the response message to get the required count. Or you can use Display only "errors" option to get all the errors and then filter out based on the expected error message.
If you need to find out at the runtime then you can use aggregate report listener and use "Errors" checkbox to get the count of failure but this will include other failures also.
But, if you need to get the count at the run time to use it later then it is a different case. I am assuming that it is not the case.
Thanks,

Extract report results with CloudConnect

I would like to extract raw report results within the CloudConnect process.
So far I have managed to get response from the raw report API end point - https://secure.gooddata.com/gdc/app/projects/{project_id}/execute/raw/
This response contains URI to the file and if I put that URI to browser, file is uploaded.
I have tried passing this URI to the following readers without success:
CSV Reader produces the following error:
------------------- Error details ------------------
Component [CSV Reader:CSV_READER] finished with status ERROR.
Parsing error: Unexpected end of file in record 1, field 1 ("date"),
metadata "outOfStock";
value: Raw record data is not available, please turn on verbose mode.
File Download - I don't know how to pass the URI through the port to "URL to Downlaod" parameter.
HTTP Connector again I don't see how to pass URI from the port.
What is the way to do this?
EDIT
If I use the HTTP Connector as suggested by #Filip, I get the following error:
Error details:
Component [HTTP connector:HTTP_CONNECTOR] finished with status ERROR. hostname in
certificate didn't match: xxx.com != secure.gooddata.com OR secure.gooddata.com
I have tried setting header to X-GDC-CHECK-DOMAIN: false with no effect.
The HTTP connector is the right component to go with. Leave the URL property empty and use the component’s property called “Input mapping”, where in the graphic editor you can assign the input edge field to the URL field.
Solution from GoodData support:
HTTP connector can be also used, but it is very complex, because
logging in to GoodData has to be created. REST connector has it built
in.
If you want to run the example graph, you have to be logged in in
CloudConnect with a user who has access to the project from where you
would like to export the report. You also have to change URL to
the one of white-labeled account in both REST connector components and change project
and report definition in the first REST connector.
So the graph that works looks like this:
Here are the main fields that you will need to set for each element:
Get Results URI - set params for POST request:
Request URL = https://secure.gooddata.com/gdc/app/projects/${GDC_PROJECT_ID}/execute/raw/
Request Body =
{
"report_req": {
"reportDefinition": "gdc/md/${GDC_PROJECT_ID}/obj/${OBJECT_ID}"
}
}
Get URI from Response - just map uri value to corresponding field:
<Mapping cloverField="uri" xpath="uri"/>
Load Results - make sure it is connected to metadata with two fields, one for response with data, other to pass through the uri.
Load Results - you will need to exclude uri field to process the data:
Exclude Fields = uri

SoapUI dataSource illegal character in authority

I am trying to use an external dataSource in SoapUI to send some basic GET http requests to a number of nodes, and i get "Illegal character in authority at index 7".
What i have setup.
1x dataSource (external file > excel):
The nodes setup appears to be correct (its called "nodes") > column required is called "node".
Getting the rows from the datafile from the dataSource options appear to be working correctly.
1x HTTP request
GET request, URL is: http://${nodes#node}:2040/api/doSometimes
I know i need to add the loop at the end, however the HTTP request isn't working with the first node yet, so i'll do the loop once the request works.
The error i get when trying to run the HTTP request:
Sun Aug 10 11:20:18 IDT 2014:ERROR:An error occurred [Illegal character in authority at index 7: http://XXX.XXX.XXX.XXX /api/doSomething], see error log for details (where XXX.XXX.XXX.XXX is the first ADDRESS from the nodes#node file). -- Also notice its missing the port.
The error log sais: Sun Aug 10 11:29:25 IDT 2014:ERROR:java.lang.NullPointerException
Clarification: we do not have a WSDL available, however the service does reply to different queries. /api/sendID WILL return the ID. I want to get all IDS from all NODES in the file.
Any ideas what i can do to mend this ?
Used a preset REST request which included the required parameters. Used standard testsuite > added the request port directly in the datasource #node which appeared to be causing the issue.

Apache JMeter : How to compare responses of same HTML request during loop?

Please reply me with the solution for following problem:
With Apache JMeter I have ran the 2000 requests per 2 minutes with 10 loops.
The request is for checking hotel availability for mentioned start and end dates.
Following 3 important parameters are there:
Hotel id, Start date and End date
With the help of 'CSV data set Config' I have stored input from 2 text files (1 text file has hotel ids and another having start,End dates). and used variables into http requests.
By taking listeners View Result Tree and summary report, I checked the responses of each request. For some requests I am getting blank responses, as hotel is not available. Now I want to find out the exact count of the blank responses. Please let me know if anybody has solution for it.
Thanks in advance
You can use i.e. Size Assertion or Response Assertion to mark blank requests as failed ones.
If you just need just number and don't want to record failures on blank responses I would suggest using a Beanshell Post processor as follows.
Define a User Defined Variable called i.e. blank with the value of 0
Add a Beanshell Post Processor to each request which may produce blank response
At the end add a Beanshell Sampler which will report the final variable
Example Post Processor code:
int blank = Integer.parseInt(vars.get("blank")); //get current "blank" variable value
String response = new String(data); //get response data as string
if (response.length() == 0) // if response length equals zero (feel free to update this as requred)
{
blank++; // increment "blank" variable
}
vars.put("blank", String.valueOf(blank)); // update "blank" variable value
Example final Beanshell sampler code:
log.info("Pages with blank response count: " + vars.get("blank")); // print current "blank" variable value to "jmeter.log" file
You should see something like below in jmeter.log file:
2014/05/15 18:36:34 INFO - jmeter.util.BeanShellTestElement: Pages
with blank response count: 15
2014/05/15 18:36:34 INFO -
jmeter.threads.JMeterThread: Thread finished: Thread Group 1-1
2014/05/15 18:36:34 INFO - jmeter.engine.StandardJMeterEngine:
Notifying test listeners of end of test
2014/05/15 18:36:34 INFO -
jmeter.gui.util.JMeterMenuBar: setRunning(false,local)
See How to use BeanShell: JMeter's favorite built-in component guide for JMeter extension with scripting walkthrough and a kind of Beanshell cookbook.