How to use dynamic values for Karate Features - testing

I've a need where I should use dynamic values in the features of my karate tests.
I've came accross with some of the questions and answers like this: How to read input data from an excel spreadsheet and pass it JSON payload in karate framework?
But no mather what how hard I try, I couln't make it happen. I believe I should share the code parts that I am trying to use, so that a discussion can start.
I have a SOAP request for creating new users as below:
<?xml version="1.0" encoding="utf-8"?>
<soapenv:Envelope xxxxxx>
<soapenv:Header/>
<soapenv:Body>
<int:createSubscriber soapenv:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">
<custBean xxxxx>
<accountNumber xsi:type="xsd:string">#(accountNo)</accountNumber>
<custName xsi:type="xsd:string" xs:type="type:string">Xbox</custName>
</custBean>
<addSubscriberBean xxxxx>7
<subscriberID xsi:type="xsd:string">#(subsID)</subscriberID>
<password xsi:type="xsd:string" xs:type="type:string">0</password>
<areaID xsi:type="xsd:string" xs:type="type:string">1</areaID>
<lineOfCredit xsi:type="xsd:int" xs:type="type:int"></lineOfCredit>
<creditCycle xsi:type="xsd:int" xs:type="type:int"></creditCycle>
<points xsi:type="xsd:int" xs:type="type:int"></points>
<bandwidth xsi:type="xsd:int" xs:type="type:int"></bandwidth>
<physicalPorts xsi:type="xsd:string" xs:type="type:string">8080</physicalPorts>
<mobilePhoneNo xsi:type="xsd:string" xs:type="type:string">#(mobile)</mobilePhoneNo>
<stbCount xsi:type="xsd:int" xs:type="type:int">5</stbCount>
<oTTCount xsi:type="xsd:int" xs:type="type:int">10</oTTCount>
<subscriptionType xsi:type="xsd:string" xs:type="type:string">#(subsType)</subscriptionType>
</addSubscriberBean>
<sequenceID xxxxx>1234567840123422700</sequenceID>
</int:createSubscriber>
</soapenv:Body>
As you've seen, I have some variables which are going to be given from outside, and those are: accountNo, subsID, subsType and mobile.
Now, I have a feature file where I make a call to a SOAP service by using above file. I am assigning new values to all variables of request, so that I can create new users all the time.
Here is the example:
Feature: Create Subscriber Feature End-To-End Scenario
Background:
* url SOAP_CREATE_SUBSCRIBER_HOST
* def accountNumber = '789'
* def subscriberID = '456'
* def userMsisdn = '123'
* def subscriptionType = 'ASD'
* def createUser = read('create-user-soap.xml') # This is the above one
* replace createUser
| token | value |
| #(accountNo) | accountNumber |
| #(subsID) | subscriberID |
| #(mobile) | userMsisdn |
| #(subsType) | subscriptionType |
Scenario: Create Subscriber
Given request createUser
When soap action SOAP_CREATE_SUBSCRIBER_HOST
Then status 200
And match //returnCode == 0
And match //returnMessage == 'The operation succeeded.'
However, I need to create bunch of users, so I need to use dynamic variables to call my .xml file too many times.
I checked the docs and answer here: How to read input data from an excel spreadsheet and pass it JSON payload in karate framework?
But couldn't locate it in my situation.
Thanks in advance.
EDIT:
I am aware of the situation that I need to use table or json or csv or excel kind of data holder to use it late, so below is my users table. Just don't know how to implement it to my feature file so that it can create too many users.
* table userstable
| accountNo | subsID | mobile | subsType |
| '113888572' | '113985218890' | '1135288836' | 'asd' |
| '113888573' | '113985218891' | '1135288837' | 'qwe' |
| '113888582' | '113985218810' | '1135288846' | 'asd' |
| '883889572' | '883985219890' | '8835298836' | 'qwe' |
| '773888572' | '773985218890' | '7735288836' | 'asd' |
| '663888572' | '663985218890' | '6635288836' | 'qwe' |
| '553888572' | '553985218890' | '5535288836' | 'asd' |
| '443888572' | '443985218890' | '4435288836' | 'qwe' |
| '333888572' | '333985218890' | '3335288836' | 'asd' |
| '223888572' | '223985218890' | '2235288836' | 'qwe' |
| '165488572' | '175585218890' | '1114788836' | 'asd' |
EDIT 2:
After a deep dive into the some answers and reading lots of docs, I've encounter with the solution below. There should be a .feature file where you place your create method to fire up the single user creation mechanism. It is going to look like this:
#ignore
Feature: re-usable feature to create a single user
Background:
* url SOAP_CREATE_SUBSCRIBER_HOST
Scenario: Create single user
* match __arg == bulkusers[__loop]
* def createUser = read('xxxx')
Given request createUser
When soap action SOAP_CREATE_SUBSCRIBER_HOST
And request { accountNo: '#(accountNo)', subsID: '#(subsID)', mobile: '#(mobile)', subsType: '#(subsType)' }
Then status 200
So above code can be placed as a template in your mind. On the other hand, we need another**.feature** file to call that template. And it is going to look like this:
Feature: call template feature.
Background:
* url SOAP_CREATE_SUBSCRIBER_HOST
Scenario: Use bulkusers table to create default users
* table bulkusers
| accountNo | subsID | mobile | subsType |
| '131451715' | '133451789134' | '5335167897' | 'asd' |
| '122452715' | '123452789124' | '5334287897' | 'qwe' |
| '124453715' | '123453789114' | '5334367817' | 'asd' |
* def result = call read('user-create.feature') bulkusers
* def created = $result[*].response
* match result[*].__loop == [0, 1, 2]
* match created[*].name == $bulkusers[*].name
* def createUser = read('xxx')
What this code achieve is, it is packing up the bulkusers table with user-create.feature, thus, user-create.feature template class being called recursively till the number of the table variables ends, with the bulkusers variables.

I am providing a simplified example below, but am sure you will find the answers to your questions here. It is easy to loop over data and build XML in Karate using the karate.set(varName, xPath, value) API:
* table users
| accountNo | subsID | mobile | subsType |
| '113888572' | '113985218890' | '1135288836' | 'asd' |
| '113888573' | '113985218891' | '1135288837' | 'qwe' |
| '113888582' | '113985218810' | '1135288846' | 'asd' |
* def xml = <users></users>
* def fun =
"""
function(u, i) {
var base = '/users/user[' + (i + 1) + ']/';
karate.set('xml', base + 'account', u.accountNo);
karate.set('xml', base + 'mobile', u.mobile);
karate.set('xml', base + 'type', u.subsType);
}
"""
* eval karate.forEach(users, fun)
* match xml ==
"""
<users>
<user>
<account>113888572</account>
<mobile>1135288836</mobile>
<type>asd</type>
</user>
<user>
<account>113888573</account>
<mobile>1135288837</mobile>
<type>qwe</type>
</user>
<user>
<account>113888582</account>
<mobile>1135288846</mobile>
<type>asd</type>
</user>
</users>
"""

Related

Iterate Over Many XML Attributes In SQL

I have a XML file that has a series of attributes. The attributes look something like the list below:
<Summary>
<MyAttributes AT001="ABC" AT002="123" AT003="456" AT004="DEF" ... />
</Summary>
I need to iterate over the attributes and add them into a SQL table that looks something like this:
Name
Value
AT001
ABC
AT002
123
AT003
456
AT004
DEF
...
...
Because the attribute list isn't fixed, I need to iterate over all the attributes to ensure each attribute gets added.
I typically can figure out how to do things in SQL, but this one has me stumped!
It is not clear what SQL you are using.
Here is how to do it in MS SQL Server by using its T-SQL and XQuery methods.
SQL
DECLARE #xml XML =
N'<Summary>
<MyAttributes AT001="ABC" AT002="123" AT003="456" AT004="DEF" />
</Summary>';
SELECT c.value('local-name(.)', 'VARCHAR(30)') AS attr_name
, c.value('.', 'VARCHAR(30)') AS attr_value
FROM #xml.nodes('/Summary/MyAttributes/#*') AS t(c);
Output
+-----------+------------+
| attr_name | attr_value |
+-----------+------------+
| AT001 | ABC |
| AT002 | 123 |
| AT003 | 456 |
| AT004 | DEF |
+-----------+------------+

How better I can optimize this Kusto Query to get my logs

I have below query which I am running and getting logs for Azure K8s, but its takes hour to generate the logs and i am hoping there is a better way to write what i have already written. Can some Kusto experts advice here as how can I better the performance?
AzureDiagnostics
| where Category == 'kube-audit'
| where TimeGenerated between (startofday(datetime("2022-03-26")) .. endofday(datetime("2022-03-27")))
| where (strlen(log_s) >= 32000
and not(log_s has "aksService")
and not(log_s has "system:serviceaccount:crossplane-system:crossplane")
or strlen(log_s) < 32000
| extend op = parse_json(log_s)
| where not(tostring(op.verb) in ("list", "get", "watch"))
| where substring(tostring(op.responseStatus.code), 0, 1) == "2"
| where not(tostring(op.requestURI) in ("/apis/authorization.k8s.io/v1/selfsubjectaccessreviews"))
| extend user = op.user.username
| extend decision = tostring(parse_json(tostring(op.annotations)).["authorization.k8s.io/decision"])
| extend requestURI = tostring(op.requestURI)
| extend name = tostring(parse_json(tostring(op.objectRef)).name)
| extend namespace = tostring(parse_json(tostring(op.objectRef)).namespace)
| extend verb = tostring(op.verb)
| project TimeGenerated, SubscriptionId, ResourceId, namespace, name, requestURI, verb, decision, ['user']
| order by TimeGenerated asc
You could try starting your query as follow.
Please note the additional condition at the end.
AzureDiagnostics
| where TimeGenerated between (startofday(datetime("2022-03-26")) .. endofday(datetime("2022-03-27")))
| where Category == 'kube-audit'
| where log_s hasprefix '"code":2'
I assumed that code is integer, in case it is string, use the following (added qualifier)
| where log_s has prefix '"code":"2'

Difference between scenario and scenaio out line

i have created below feature and calling this in another feature file
#ignore
Feature: re-usable feature to create a single order
Scenario Outline: Create multiple users and verify their id, name and age
Given url 'https://arid-stage.****/sun-api//user/****'
And request { locale:'',offerId:'',operationType:'',paidTermDuration:'',paidTermDurationUnit:'',paymentCategory:'',storeOrderId:'',userId:'' }
When method post
Then status 200
Examples:
| locale | offerId | operationType| paidTermDuration |paidTermDurationUnit | paymentCategory |storeOrderId|userId |
| en_us | 7777777 | CREATE | 30 | MONTH | VENDOR_PAYMENT | localDate | 42DC198E5ABCE1430A494128 |
In other feature, i'm calling feature -> * def result = call read('redeem-create.feature')
Questions:
This will be executed only when with text Scenario Outline and i remove and update as Scenario, this will not get executed.
When to use scenario outline and scenario
Any suggestions/ ideas
Why don't you read the documentation: https://github.com/intuit/karate#data-driven-tests
And also look at this example for a comparison: examples.feature

Transitive SQL query on same table

Hey. consider d following table and data...
in_timestamp | out_timestamp | name | in_id | out_id | in_server | out_server | status
timestamp1 | timestamp2 | data1 |id1 | id2 | others-server1 | my-server1 | success
timestamp2 | timestamp3 | data1 | id2 | id3 | my-server1 | my-server2 | success
timestamp3 | timestamp4 | data1 | id3 | id4 | my-server2 | my-server3 | success
timestamp4 | timestamp5 | data1 | id4 | id5 | my-server3 | others-server2 | success
the above data represent log of a execution flow of some data across servers.
e.g. some data has flowed from some 'outside-server1' to bunch of 'my-servers' and finally to destined 'others-server2'.
Question :
1) I need to give this log in representable form to client where he doesn't need to know anything about the bunch of 'my-servers'. All i am supposed to give is timestamp of the data entered my infrastructure and when it left; drilling down to following info.
in_timestamp (of 'others_server1' to 'my-server1')
out_timestamp (of 'my-server3' to 'others-server2')
name
status
I want to write sql for the same! Can someone help?
NOTE : there might not be 3 'my-servers' all the time. It differs from situation to situation. e.g. there might be 4 'my-server' involved for, say, data2!
2) Are there any other alternatives to SQL? I mean stored procs/etc?
3) Optimizations? (The records are huge in number! As of now, it is around 5 million a day. And we are supposed to show records that are upto a week old.)
In advance, THANKS FOR THE HELP! :)
WITH RECURSIVE foo AS
(
SELECT *, in_timestamp AS timestamp1, 1 AS hop, ARRAY[in_id] AS hops
FROM log_parsing.log_of_sent_mails
WHERE in_server = 'other-server1'
UNION ALL
SELECT t_alias2.*, foo.timestamp1, foo.hop + 1, hops || in_id
FROM foo
JOIN log_parsing.log_of_sent_mails t_alias2
ON t_alias2.in_id = (foo.t_alias1).out_id
)
SELECT *
FROM foo
ORDER BY
hop DESC
LIMIT 1
Your table has a heirarchical structure (adjacency lists). This can be queried efficiently in PostgreSQL v8.4 and later using recursive CTEs. Quassnoi has written a blog post about how to implement it. It is a quite complex query that you need to write but he explains it well with examples very similar to what you need. Especially if you look at his last example, he demonstrates a query than gets the complete path from the first node to the last by using an array.
One way of doing it - if the data is STABLE (e.g. never changes onc inserted) is to compute the transitive relationships ON THE FLY (e.g. via a trigger or by the app which does the insertion) at the insert time.
E.g. you have a new column "start_ts" in your table; when you insert a record:
in_timestamp | out_timestamp | name | in_id | out_id | in_server | out_server | status
timestamp3 | timestamp4 | data1 | id3 | id4 | my-server2 | my-server3 | success
... then your logic automatically finds the record with name=data1 and out_id=id3 and clones its start_ts into the newly inserted record. You may need some special logic around propagating last status as well depending on how you compute those transitive values.
BTW, you need not necessarily have to look up the previous (name=data1 and out_id=id3) record - you can persist the start_ts value in the data record's meta data itself while processing.
Then the final report is simply select start_ts, out_ts from T where out_server=others_server2 (of course more complicated as far as out_server and status, but still a single simple select)
A second option is of course the more straightforward loop computing the resulting report - google or "stack" (is that an accepted verb now?) for SQL BFS implementations if you're not sure how.
#Other Readers :
Refer to 1st answer posted by Mark Byers first. I used 'answering' rather than 'commenting' his post since i needed to use tables/links etc. which is not available while commenting on answers. :)
#Mark Byers :
Thanks for the link... It really helped me and i was able to figure out the way to generate the path between the servers... Have a look # what i was able to do.
in_id | in_timestamp | out_timestmap | name | hops_count | path |
id1 | timestamp1 | timestamp2 | data1 | 1 | {id1} |
id2 | timestamp2 | timestamp3 | data1 | 2 | {id1,id2} |
id3 | timestamp3 | timestamp4 | data1 | 3 | {id1,id2,id3} |
id4 | timestamp4 | timestamp2 | data1 | 4 | {id1,id2,id3,id4} |
* path is generated using 'in_id'
I used the following query...
WITH RECURSIVE foo AS
(
SELECT t_alias1, 1 AS hops_count, ARRAY[in_id] AS hops
FROM log_parsing.log_of_sent_mails t_alias1
WHERE in_server = 'other-server1'
UNION ALL
SELECT t_alias2, foo.hops_count + 1 AS hops_count, hops || in_id
FROM foo
JOIN log_parsing.log_of_sent_mails t_alias2
ON t_alias2.in_id = (foo.t_alias1).out_id
)
SELECT (foo.t_alias1).in_id,
(foo.t_alias1).name,
(foo.t_alias1).in_timestamp,
hops_count,
hops::VARCHAR AS path
FROM foo ORDER BY hops
But i could not reach the ultimate stage yet. Here is what i wish to get ultimately...
in_id | in_timestamp | out_timestmap | name | hops_count | path |
id4 | timestamp1 | timestamp5 | data1 | 4 | {id1,id2,id3,id4}|
* observe the timestamp. Its required since i do not wish the client to know about the internal infrastructure. So for him the time-lag between timestamp1 and timestamp5 is what matters.
Any clue how possibly i could achieve it!?
p.s. I would try contacting Quassnoi too. :)

Assistance with SQL multi-table query - returning duplicate results

We use an online project management system, and I'm trying to extend it somewhat.
It has the following tables of interest:
todo_itemStatus:
+--------------+-----------------------+------+-----+---------------------+----------------+
| Field | Type | Null | Key | Default | Extra |
+--------------+-----------------------+------+-----+---------------------+----------------+
| itemStatusId | bigint(20) unsigned | NO | PRI | NULL | auto_increment |
| itemId | int(10) unsigned | NO | MUL | 0 | |
| statusDate | datetime | NO | | 0000-00-00 00:00:00 | |
| statusKey | tinyint(3) unsigned | NO | | 0 | |
| memberId | mediumint(8) unsigned | NO | | 0 | |
+--------------+-----------------------+------+-----+---------------------+----------------+
This table keeps track of when a task is complete, and also keeps the status of all task changes.
There's then a project table, and an 'item' (or task) table.
I basically want to be able to extract a list of projects, with details on the percentage of tasks complete. However, for now I'd be happy if I could just list each task in a project with details on whether they're complete.
As far as I'm aware, the best way to get the most recent status of a task is to choose an todo_itemStatus where the statusDate is the newest, or the itemStatusId is the largest whilst itemId equals the task I'm interested.
I tried a query like this:
<pre>
select todo_item.itemId, todo_item.title, todo_itemStatus.statusKey, todo_itemStatus.statusDate
from todo_item, todo_project, todo_itemStatus
where todo_item.projectId = todo_project.projectId
and todo_project.projectId = 13
and todo_itemStatus.itemId = todo_item.itemId
and todo_itemStatus.statusDate = (
select MAX(todo_itemStatus.statusDate)
from todo_itemStatus key1 where todo_itemStatus.itemId = key1.itemId);
</pre>
However, this yields all status updates with output like this:
+--------+-----------------------------------------------------------------------------+-----------+---------------------+
| itemId | title | statusKey | statusDate |
+--------+-----------------------------------------------------------------------------+-----------+---------------------+
| 579 | test complete item - delete me | 1 | 2009-07-28 13:04:38 |
| 579 | test complete item - delete me | 0 | 2009-07-28 14:12:12 |
+--------+-----------------------------------------------------------------------------+-----------+---------------------+
Which isn't what I want, as I only want one task entry returning with the statusKey / statusDate from the most recent entry in the todo_itemStatus table.
I know I've been a bit vague in my description, but I didn't want to write a massively long message. I can provide much more detail if necessary.
Please can someone suggest what I'm doing wrong? It's been a long time since I've done any real database stuff, so I'm a bit unsure what I'm doing wrong here...
Many thanks!
Dave
You should look into using the DISTINCT keyword (Microsoft SQL Server)
EDIT: I've just re-read your question and I think that the GROUP BY clause is more suited in this situation. You should read http://www.xaprb.com/blog/2006/12/07/how-to-select-the-firstleastmax-row-per-group-in-sql/, however essentially what you need to do is first select the columns that you are interested in using a GROUP BY clause:
SELECT todo_itemStatus.itemStatusId, MAX(todo_itemStatus.statusDate)
FROM todo_item, todo_project, todo_itemStatus
WHERE todo_item.projectId = todo_project.projectId
AND todo_itemStatus.itemId = todo_item.itemId
AND todo_project.projectId = 13
GROUP BY itemStatusId
We then self-join to this set of id's to get the rest of the columns we are interested in:
SELECT
todo_item.itemId,
todo_item.title,
todo_itemStatus.statusKey,
todo_itemStatus.statusDate
FROM todo_item
JOIN todo_itemStatus
ON todo_itemStatus.itemId = todo_item.itemId
JOIN
(SELECT todo_itemStatus.itemStatusId, MAX(todo_itemStatus.statusDate)
FROM todo_item, todo_project, todo_itemStatus
WHERE todo_item.projectId = todo_project.projectId
AND todo_itemStatus.itemId = todo_item.itemId
AND todo_project.projectId = 13
GROUP BY itemStatusId) AS x
ON todo_itemStatus.itemStatusId = x.itemStatusId
I've experimented some more and the following query does what I want:
select todo_item.itemId, todo_item.title, todo_itemStatus.statusKey, todo_itemStatus.statusDate from todo_itemStatus, todo_item where todo_item.itemId = todo_itemStatus.itemId and todo_item.projectId = 13 and todo_itemStatus.statusDate = (select MAX(status.statusDate) from todo_itemStatus as status where status.itemId = todo_item.itemId);
So I'm now happy. Thanks for all the help and the suggestions.
Dave.