create relationships between nodes in parallel - cypher

Using cypher and neo4j 2.0.
Given two sets of node ids (of equal length) and a set of weights, I'd like to create a relationship between the corresponding nodes and set the weight as a property. For example if I have the following three lists:
node list 1: (101, 201, 301)
node list 2: (102, 202, 302)
weights: (0.1, 0.6, 0.25)
I would like to create the following representation
101 - knows {w : .1} - 102
201 - knows {w : .6} - 202
301 - knows {w : .25} - 302
but NOT, for example, 101 - knows - 302
I can do this by iterating over my parameters and then creating the individual queries.
Is there a way to batch run this, passing my lsits as parameters and asking
cypher to match the nodes & properties in order?
I thought for a moment that using parameters in the following fashion would work, but it instead creates all permutations of relationships (as expected) and assigns as the entire list of weights as a property to each relationship.
{
"query":
"START a1=node({starts}), a2=node({ends})
CREATE UNIQUE a1-[r:knows {w : {weights}}]-a2
RETURN type(r), r.w, a1.name, a2.name",
"params": {
"starts" : [101, 201, 301],
"ends" : [102, 202, 302],
"weights" : [0.1, 0.6, 0.25]
}
}

How large are your lists in real life?
I'd probably send in one triple at a time.
Otherwise you should be able to use a collection and foreach to do what you want:
START a1=node({starts}), a2=node({ends})
FOREACH(w in filter(w in weights : head(w)=id(a1) AND head(tail(w))=id(a2)) :
CREATE UNIQUE a1-[r:knows {w : last(w)}]-a2
)
"params": {
"starts" : [101, 201, 301],
"ends" : [102, 202, 302],
"weights" : [[101,102,0.1], [201,202,0.6], [301,302,0.25]]
}

Related

How to iterate over a dynamic array of objects and use each object as a parameter in test?

I started my adventure with Karate a month ago. I have a simple GET test called getAllCars.feature showing a list of cars currently available:
[
{
"brandName": "BMW",
"id": 1,
"winterTires": false,
"modelName": "X5"
},
{
"brandName": "Opel",
"id": 34,
"winterTires": true,
"modelName": "Insignia"
},
{
"brandName": "Mercedes-Benz",
"id": 36,
"winterTires": true,
"modelName": "GLE Coupe"
},
{
"brandName": "Huydai",
"id": 251,
"winterTires": false,
"modelName": "i30"
}
]
I have to use each id as a parameter for the next feature file, the problem is, the list of cars is dynamic, ids don't repeat and I will have to use this list of ids for several other feature files. I managed to create a helper getCarIds.feature, which creates an array of objects "carId": "#number":
Feature: Get car IDs
Scenario: Get car IDs
* call read('classpath:x/automation/cars/getAllCars.feature')
* def carIds = $response[*].id
* def carFeeder = karate.mapWithKey(carIds, 'carId')
The following getCarParameters.feature has to iterate over the array from getCarIds.feature and pass each id as a parameter to get a response with performance parameters of each car and I don't know how to use each id separately as a parameter (keeping in mind that the list of ids is changing):
Feature: Get parameters of each car
Scenario: Get parameters for each car
* call read('classpath:x/automation/cars/getCarIds.feature')
Given url carUrl + '/carparameters'
And param carId =
When method GET
Then status 200
I managed to do it when passing the values from getCarIds.feature to getCarParameters.feature like described here by adding following line to getCarIds.feature:
* call read('classpath:x/automation/cars/getCarParameters.feature') carFeeder
but several other tests require car ids. I need getCarIds.feature to be reusable, so I would have to retrieve data from feature file, which creates the array with ids, instead of passing it to the GET feature and apparently it isn't so easy. Maybe my approach is completely wrong.
I think this is a valid case for karate.callSingle(): https://github.com/karatelabs/karate#karatecallsingle
So you can actually stick this in any feature and it is guaranteed to execute only once across your test suite. If the data is indeed something used by a majority of your test suite, you could even do this initialization in karate-config.js.
So this should work. First the reusable feature common.feature. Instead of the hard-coded response, you know how to make an actual HTTP request.
#ignore
Feature:
Scenario:
* def response =
"""
[
{
"brandName": "BMW",
"id": 1,
"winterTires": false,
"modelName": "X5"
},
{
"brandName": "Opel",
"id": 34,
"winterTires": true,
"modelName": "Insignia"
}
]
"""
* print 'getting car ids'
* def carIds = response.map(x => ({ id: x.id }))
Note the use of the JS map() function above, which I have started to recommend instead of JsonPath.
And here is a feature that uses the above. This uses the new #setup annotation that makes it easy to "loop": https://github.com/karatelabs/karate#setup
You can try this example quickly, and watch it make 2 requests using a param id from the loop.
Feature:
#setup
Scenario:
* def data = karate.callSingle('call-single-common.feature').carIds
Scenario Outline:
* url 'https://httpbin.org/get'
* param id = id
* method get
Examples:
| karate.setup().data |
There are other ways to loop, refer: https://github.com/karatelabs/karate#data-driven-features

How to loop new request according to response data list from previous step [duplicate]

I am using Karate version 0.8.0.1 and I want to perform following steps to test some responses.
I make a Get to web service 1
find the value for currencies from the response of web service 1 using jsonpath: $.currencies
Step 2 gives me following result: ["USD","HKD","SGD","INR","GBP"]
Now I use Get method for web service 2
From the response of web service 2 I want to get the value of price field with json-path something like below(passing the values from step 3 above):
$.holding[?(#.currency=='USD')].price
$.holding[?(#.currency=='HKD')].price
$.holding[?(#.currency=='SGD')].price
$.holding[?(#.currency=='INR')].price
$.holding[?(#.currency=='GBP')].price
So there are so many currencies but I want to verify price for only the currencies returned by web service 1(which will be always random) and pass it on to the the output of web service 2 to get the price.
Once i get the price I will match each price value with the value returned from DB.
I am not sure if there is any simple way in which I can pass the values returned by service 1 into the json-path of service 2 one by one and get the results required. Any suggestions for doing this will be helpful As this will be the case for most of the web services I will be automating.
There are multiple ways to do this in Karate. The below should give you a few pointers. Note how there is a magic variable _$ when you use match each. And since you can reference any other JSON in scope, you have some very powerful options.
* def expected = { HKD: 1, INR: 2, USD: 3}
* def response1 = ['USD', 'HKD', 'INR']
* def response2 = [{ currency: 'INR', price: 2 }, { currency: 'USD', price: 3 }, { currency: 'HKD', price: 1 }]
* match response2[*].currency contains only response1
* match each response2 contains { price: '#(expected[_$.currency])' }
You probably already have seen how you can call a second feature file in a loop which may be needed for your particular use case. One more piece of the puzzle may be this - it is very easy to transform any JSON array into the form Karate expects for calling a feature file in a loop:
* def response = ['USD', 'HKD', 'INR']
* def data = karate.map(response, function(x){ return { code: x } })
* match data == [{code: 'USD'}, {code: 'HKD'}, {code: 'INR'}]
EDIT - there is a short-cut to convert an array of primitives to an array of objects now: https://stackoverflow.com/a/58985917/143475
Also see this answer: https://stackoverflow.com/a/52845718/143475

How to loop through karate response array and pass this in json path of another web service response

I am using Karate version 0.8.0.1 and I want to perform following steps to test some responses.
I make a Get to web service 1
find the value for currencies from the response of web service 1 using jsonpath: $.currencies
Step 2 gives me following result: ["USD","HKD","SGD","INR","GBP"]
Now I use Get method for web service 2
From the response of web service 2 I want to get the value of price field with json-path something like below(passing the values from step 3 above):
$.holding[?(#.currency=='USD')].price
$.holding[?(#.currency=='HKD')].price
$.holding[?(#.currency=='SGD')].price
$.holding[?(#.currency=='INR')].price
$.holding[?(#.currency=='GBP')].price
So there are so many currencies but I want to verify price for only the currencies returned by web service 1(which will be always random) and pass it on to the the output of web service 2 to get the price.
Once i get the price I will match each price value with the value returned from DB.
I am not sure if there is any simple way in which I can pass the values returned by service 1 into the json-path of service 2 one by one and get the results required. Any suggestions for doing this will be helpful As this will be the case for most of the web services I will be automating.
There are multiple ways to do this in Karate. The below should give you a few pointers. Note how there is a magic variable _$ when you use match each. And since you can reference any other JSON in scope, you have some very powerful options.
* def expected = { HKD: 1, INR: 2, USD: 3}
* def response1 = ['USD', 'HKD', 'INR']
* def response2 = [{ currency: 'INR', price: 2 }, { currency: 'USD', price: 3 }, { currency: 'HKD', price: 1 }]
* match response2[*].currency contains only response1
* match each response2 contains { price: '#(expected[_$.currency])' }
You probably already have seen how you can call a second feature file in a loop which may be needed for your particular use case. One more piece of the puzzle may be this - it is very easy to transform any JSON array into the form Karate expects for calling a feature file in a loop:
* def response = ['USD', 'HKD', 'INR']
* def data = karate.map(response, function(x){ return { code: x } })
* match data == [{code: 'USD'}, {code: 'HKD'}, {code: 'INR'}]
EDIT - there is a short-cut to convert an array of primitives to an array of objects now: https://stackoverflow.com/a/58985917/143475
Also see this answer: https://stackoverflow.com/a/52845718/143475

Dse Graph loader duplicate edges

I have the following csv files:
one is with the person and the other one is with the addresses and one with person address connection (one row on each file plus header). For testing purpose at first run I have:
config create_schema: true, load_new: true, load_threads: 3
The import is a success with the vertices and edges. (two vertices and one edge between them)
Now when I run the same script(same data, same input script) but with different config
config create_schema: false, load_new: false, load_threads: 3
It seems that the nodes didn’t change but I have a duplicate edge for the nodes. (two vertices and two edges between the same nodes)
this is the code that i run:
inputfiledir = 'data/'
personInput = File.csv(inputfiledir + 'sna_person_test.csv').delimiter(',')
addressInput = File.csv(inputfiledir + 'sna_address_test.csv').delimiter(',')
personAddressInput = File.csv(inputfiledir + 'san_person_address_test.csv').delimiter(',')
load(personInput).asVertices {
label "person"
key "id"
}
load(addressInput).asVertices {
label "address"
key "id"
}
load(personAddressInput).asEdges {
label "has_address"
outV "person_id", {
label "person"
key "id"
}
inV "address_id", {
label "address"
key "id"
}
}
Is there a way to avoid this ?
Thanks
This is due to edges not having an Id, which leads to Graph Loader not having a way to determine if an edge is in fact a duplicate. This will cause subsequent loads to duplicate the edges, but not the vertices.

Elasticsearch bulk/batch indexing with python requests module

I have a smallish (~50,00) array of json dictionaries that I want to store/index in ES. My preference is to use python, since the data I want to index is coming from a csv file, loaded and converted to json via python. Alternatively, I would like to skip the step of converting to json, and simply use the array of python dictionaries I have. Anyway, a quick search revealed the bulk indexing functionality of ES. I want to do something like this:
post_url = 'http://localhost:9202/_bulk'
request.post(post_url, data = acc ) # acc a python array of dictionaries
or
post_url = 'http://localhost:9202/_bulk'
request.post(post_url, params = acc ) # acc a python array of dictionaries
both request give a [HTTP 500 error]
My understanding is that you have to have one "command" per line (index, create, delete...) and then some of them (like index) takes a row of data on the next line like so
{'index': ''}\n
{'your': 'data'}\n
{'index': ''}\n
{'other': 'data'}\n
NB the new-lines, even on the last row.
Empty index objects like above works if you POST to ../index/type/_bulk or else you need to specify index and type I think, have not tried that.
You the following function will do it:
def post_request(self, endpoint, data):
endpoint = 'localhost:9200/_bulk'
response = requests.post(endpoint, data=data, headers={'content-type':'application/json', 'charset':'UTF-8'})
return response
As data you need to pass a String such:
{ "index" : { "_index" : "test-index", "_type" : "_doc", "_id" : "1681", "routing" : 0 }}
{ "field1" : ... , ..., "fieldN" : ... }
{ "index" : { "_index" : "test-index", "_type" : "_doc", "_id" : "1684", "routing" : 1 }}
{ "field1" : ... , ..., "fieldN" : ... }
Make sure you add a "\n" at the end of each line.
I don't know much about Python, but did you look at Pyes?
Bulk is supported in Pyes.