I am trying below in Karate.
I have a json schema ( for response validation) in .json file. There are few REGEXs that are common in many schemas. I want to extract them into one common file as key value pairs and use it across other schemas. Is it possible? if so, how can I do that? Is templating allowed in json schema?
Example:
Sample Json Schema File ( sample-response.json):
{
"response": {
"name": "#string",
"amount": "#regex ^(-?)([0]|[1-9][0-9]{0,15})[.][0-9]{2}$"
}
}
Feature File
Feature: Example feature
Background:
* def sampleResponse = read('./sample-response.json');
Scenario: Example scenario
When url 'https://someurl.com'
And method get
Then status 200
And match response == sampleResponse
What would I like to Do?
I would like to store the amount regex in json file as a reusable variable and use templating in json file to replace it.
Is it possible?
{
"response": {
"name": "#string",
"amount": "{{get regex from this template}}"
}
}
Yes. Embedded expressions work even when reading files.
So do this:
{
"response": {
"name": "#string",
"amount": "#(amount)"
}
}
And then do this:
Background:
* def amount = 100
* def sampleResponse = read('sample-response.json')
If you want the amount to come from another JSON file, why not, say this below is data.json:
{ "amount": 100 }
Then you do this:
Background:
* def data = read('data.json')
# you don't need the next line if you use "data.amount" as the embedded expression
* def amount = data.amount
* def sampleResponse = read('sample-response.json')
Related
I`m new to nifi and I want to convert big amount of json data to csv format.
This is what I am doing at the moment but it is not the expected result.
These are the steps:
processes to create access_token and send request body using InvokeHTTP(This part works fine I wont name the processes since this is the expected result) and getting the response body in json.
Example of json response:
[
{
"results":[
{
"customer":{
"resourceName":"customers/123456789",
"id":"11111111"
},
"campaign":{
"resourceName":"customers/123456789/campaigns/222456422222",
"name":"asdaasdasdad",
"id":"456456546546"
},
"adGroup":{
"resourceName":"customers/456456456456/adGroups/456456456456",
"id":"456456456546",
"name":"asdasdasdasda"
},
"metrics":{
"clicks":"11",
"costMicros":"43068982",
"impressions":"2079"
},
"segments":{
"device":"DESKTOP",
"date":"2021-11-22"
},
"incomeRangeView":{
"resourceName":"customers/456456456/incomeRangeViews/456456546~456456456"
}
},
{
"customer":{
"resourceName":"customers/123456789",
"id":"11111111"
},
"campaign":{
"resourceName":"customers/123456789/campaigns/222456422222",
"name":"asdasdasdasd",
"id":"456456546546"
},
"adGroup":{
"resourceName":"customers/456456456456/adGroups/456456456456",
"id":"456456456546",
"name":"asdasdasdas"
},
"metrics":{
"clicks":"11",
"costMicros":"43068982",
"impressions":"2079"
},
"segments":{
"device":"DESKTOP",
"date":"2021-11-22"
},
"incomeRangeView":{
"resourceName":"customers/456456456/incomeRangeViews/456456546~456456456"
}
},
....etc....
]
}
]
Now I am using:
===>SplitJson ($[].results[])==>JoltTransformJSON with this spec:
[{
"operation": "shift",
"spec": {
"customer": {
"id": "customer_id"
},
"campaign": {
"id": "campaign_id",
"name": "campaign_name"
},
"adGroup": {
"id": "ad_group_id",
"name": "ad_group_name"
},
"metrics": {
"clicks": "clicks",
"costMicros": "cost",
"impressions": "impressions"
},
"segments": {
"device": "device",
"date": "date"
},
"incomeRangeView": {
"resourceName": "keywords_id"
}
}
}]
==>> MergeContent( here is the problem which I don`t know how to fix)
Merge Strategy: Defragment
Merge Format: Binary Concatnation
Attribute Strategy Keep Only Common Attributes
Maximum number of Bins 5 (I tried 10 same result)
Delimiter Strategy: Text
Header: [
Footer: ]
Demarcator: ,
What is the result I get?
I get a json file that has parts of the json data
Example: I have 50k customer_ids in 1 json file so I would like to send this data into big query table and have all the ids under the same field "customer_id".
The MergeContent uses the split json files and combines them but I will still get 10k customer_ids for each file i.e. I have 5 files and not 1 file with 50k customer_ids
After the MergeContent I use ==>>ConvertRecord with these settings:
Record Reader JsonTreeReader (Schema Access Strategy: InferSchema)
Record Writer CsvRecordWriter (
Schema Write Strategy: Do Not Write Schema
Schema Access Strategy: Inherit Record Schema
CSV Format: Microsoft Excel
Include Header Line: true
Character Set UTF-8
)
==>>UpdateAttribute (custom prop: filename: ${filename}.csv) ==>> PutGCSObject(and put the data into the google bucket (this step works fine- I am able to put files there))
With this approach I am UNABLE to send data to big query(After MergeContent I tried using PutBigQueryBatch and used this command in bq sheel to get the schema I need:
bq show --format=prettyjson some_data_set.some_table_in_that_data_set | jq '.schema.fields'
I filled all the fields as needed and Load file type: I tried NEWLINE_DELIMITED_JSON or CSV if I converted it to CSV (I am not getting errors but no data is uploaded into the table)
)
What am I doing wrong? I basically want to map the data in such a way that each fields data will be under the same field name
The trick you are missing is using Records.
Instead of using X>SplitJson>JoltTransformJson>Merge>Convert>X, try just X>JoltTransformRecord>X with a JSON Reader and a CSV Writer. This skips a lot of inefficiency.
If you really need to split (and you should avoid splitting and merging unless totally necessary), you can use MergeRecord instead - again with a JSON Reader and CSV Writer. This would make your flow X>Split>Jolt>MergeRecord>X.
I am trying to create my request body dynamically from an external json file.
I want to update few values and keep the remaining ones same as received from the json.
The idea here is to keep one maintainable json file and manipulate it at run time to execute various scenarios.
Here's my feature file:
* def myJson = read('testFile.json')
* def requestBody = { "product": "#(myJson.product)", "properties": { "make": "#(brand)", "color": "#(myJson.color)" }
When request requestBody
And method post
Then status 200
Examples:
| brand |
| honda |
Contents of testFile.json are -
{
"product": "car",
"properties": {
"make": "brand",
"color": "red"
}
}
The problem is that whenever there is nested json object, those fields won't keep the value from json. If the value is passed from the feature file as an example, then it gets evaluated correctly. Here's how the request body gets passed in the service call-
{
"product": "car",
"properties": {
"make": "honda",
"color": null
}
}
I need the color key's value to be taken from myJson i.e. red but it get evaluated as null.
Shouldn't it be:
"color": "#(myJson.properties.color)"
Here I would like to clarify about creating dynamic example table for a dynamic JSON index size
My JSON looks like
Env - Dev - 2 servers
"response": {
"abc": [{
"status": "pass"
.
.
},
{
"status": "pass"
.
.
}
]
}
Env - Uat - 3 servers
{
"response": {
"abc": [{
"status": "pass"
},
{
"status": "pass"
},
{
"status": "pass"
}
]
}
}
My scenario outline looks like
Scenario Outline: validating .....
When def result = callonce read('featurefilename#tagname')
Then print result
And print <status>
And print ...
And match ....
Examples:
|result.response.abc|
Errors for the above:
1) * dynamic expression evaluation failed:result.response.abc
2) com.intuit.karate.karateExpresion: ---- javascript evaluation failed result.response.abc, ReferenceError:"result" is not defined in at line number 1
Note - If I move step 'When def result = callonce read('featurefilename#tagname') to background it's working as expected but I can't use background in my feature file due to other factors.
Thanks in advance
Instead of providing index in a table you can leverage Dynamic Scenario Outline feature in karate.
In this case you you can pass the variable as a input to Examples. If the JSON provided above is from variable result then,
Examples:
| result.response.abc |
Refer the docs for more insights.
* def mpRequestJson =
"""
{
"entity": '<entity>',
"consent": {
"PHONE": <updategetPhonePref>,
"EMAIL": true,
"POST": false,
"SMS": <updategetSMSPref>
},
"notices": [{
"title": "Privacy policy",
"version": "NA"
}],
"source": "web"
}
"""
Given path '<entity>'
And request mpRequestJson
When method PUT
Then status 200
And match $.consent.PHONE == '<updategetPhonePref>'
And match $.consent.SMS == '<updategetSMSPref>'
Examples:
|entity | updategetPhonePref|updategetSMSPref|
|xyz| #(updategetPhonePref)|#(updategetSMSPref)|
If i want to store the JSON request in a JSON file rather than in the feature file, what should be my JSON file?
In the JSON use embedded expressions, e.g.
entity: '#(entity)'
Then you can read it from a file:
* def mpRequestJson = read('my.json')
But before the read you should initialize variables that have to be substituted. So you will have some extra lines.
* def entity = '<entity>'
One way to reduce the extra lines is to create a temp JSON:
* def data = { entity: '<entity'>, phone: '<updategetPhonePref>' }
And then you can do this in the JSON:
entity: '#(data.entity)'
Read the docs on data driven tests also please.
I am facing few issues while using relative path and mapping test data from a JSON file. I am having JSON POST request and a test data file in JSON format.
This is the test data I am using.
{
"name": "Test Data",
"description": "Information's mainly related with Users",
"testData": [
{
"Scenario1": {
"givenName": "Joseph",
"familyName": "George",
"addressType": "Current",
"lineOne": "BNRA-222, Kowdiar lane",
"cityName": "Trivandrum",
"countryID": "India",
"postcode": "695006"
}
},
{
"Scenario2": {
"givenName": "Sreenath",
"familyName": "Bhasi",
"addressType": "Current",
"lineOne": "HSE-123, Karyavatom",
"cityName": "Trivandrum",
"countryID": "India",
"postcode": "695552"
}
}
]
}
This is the feature file
Feature: Test using the Data from a JSON file
Background:
* def baseJsonRequest = read('../requests/jsonrequest.json')
* def baseData = read('../data/sampledata.json')
* def endPointURL = endPointURI + path
Scenario: A sample scenario to test the data parametrization
Given url endPointURL
And request baseJsonRequest
* set baseJsonRequest.autoRequest.applicants.applicant.specifiedPerson.givenName = baseData.testData[*].Scenario1.givenName
* set baseJsonRequest.autoRequest.applicants.applicant.specifiedPerson.familyName = baseData.testData[*].Scenario1.familyName
* set baseJsonRequest.autoRequest.applicants.applicant.specifiedPerson.residenceAddress.addressType = baseData.testData[*].Scenario1.addressType
* set baseJsonRequest.autoRequest.applicants.applicant.specifiedPerson.residenceAddress.lineOne = baseData.testData[*].Scenario1.lineOne
* set baseJsonRequest.autoRequest.applicants.applicant.specifiedPerson.residenceAddress.cityName = baseData.testData[*].Scenario1.cityName
* set baseJsonRequest.autoRequest.applicants.applicant.specifiedPerson.residenceAddress.countryID = baseData.testData[*].Scenario1.countryID
* set baseJsonRequest.autoRequest.applicants.applicant.specifiedPerson.residenceAddress.postcode = baseData.testData[*].Scenario1.postcode
My Questions are:
I am not able to give relative path on both sides. The relative path is returning me a json array. For eg I cannot use $..Scenario1.givenName, which makes me write longer paths.
To include this mapping on every scenario will be practically difficult. How can we implement a parameterized solution for that. What will better way? Can I invoke the data reading using a feature file and pass the informations to another feature. If that's possible then I need to parameterize . How to do that?
Or do I need to use a java class to read the JSON file?
Yes, the moment you have a wildcard in JsonPath, it returns an array. Anyway, 2 points that should help here straight away:
you can move repeating nested paths into a table-set
you can refer to a nested chunk of JSON by assigning to a variable
So this should be the way to go:
* def first = get[0] baseData.testData[*].Scenario1
* set baseJsonRequest.autoRequest.applicants.applicant.specifiedPerson
| path | value |
| familyName | first.familyName |
| residenceAddress.addressType | first.addressType |
I would try to not use wildcards as far as possible, for e.g.
* def first = $baseData.testData[0].Scenario1
Hope this helps !