Unable to find the duplicate elements from API response - selenium

We were testing an API and recently got an issue, some of the customers cannot log in to the website.
We found the issue and it is because of Duplicate keys in the API response, it is giving response even if the API contains the duplicate key.
So tests are not helping for the duplicate key conditions,So can anyone please help me or guide how I can find whether there is a duplicate element in the API response.
Tool: postman
Below is the sample API output,
In the below JSON output from API we can find there are duplicates for "operatingSystem",like this duplicate key is coming for different elements.
Since there is no way to debug the API for a while due to some reasons,so need to find out these duplicate cases.
Any idea or suggestions will be much appreciated.Thanks in advance.
JSON
eg: {
"code": 2,
"deviceId": "ID",
"deviceName": "Test",
"platform": "x64",
"operatingSystem": "test",
"operatingSystem": "test",
"gde": 000,
"productVersion": "0.0",
"build": "00000",
"receipt": null
}
How could we handle such a situation. Do we have any method to automate/test this case?

Here's something you can try although it's a bit convoluted. pm.response.json() will normalize the response and remove any duplicates i.e. you won't be able to detect any. So what you can do is take the response in text then manipulate it into a list and look for duplicates there. I used a map object so that if the map already contains a given key then set a flag and fail the test.
This is not thoroughly tested but it should give you an idea or at least a starting point to tackle the problem:
var jsonBody = pm.response.text();
var str = jsonBody.substring(1, jsonBody.length-1);
var keyArr = str.split(",");
var keyMap = {};
var foundDups = false;
for (var i = 0; i < keyArr.length; i++) {
var key = keyArr[i].split(":")[0];
if(!(key in keyMap)) {
keyMap[key] = key;
console.log("added key " + key);
}
else {
console.log("found duplicate: " + key);
foundDups = true;
break;
}
}
pm.test("Look for dups", function() {
pm.expect(foundDups).to.eql(false);
});

Related

API call from google sheets is too long

I am trying to write a script in google sheets to update my 3commas bots. The API requires a number of mandatory fields are passed even when there's only 1 item that needs to be updated.
The code I have is below and it uses the values already read from the platform updating only the base_order_volume value. This works perfectly except for when the pairs value is long (more the 2k chars) and then I get an error from the UrlFetchApp call because the URL is too long.
var sheet = SpreadsheetApp.getActiveSheet();
var key = sheet.getRange('F4').getValue();
var secret = sheet.getRange('F5').getValue();
var baseUrl = "https://3commas.io";
var editBots = "/ver1/bots/"+bots[botCounter].id+"/update";
var patchEndPoint = "/public/api"+editBots+"?";
.
.
[loop around values in sheet]
.
.
var BaseOrder=Number(sheet.getRange(rowCounter,12).getValue().toFixed(2));
var botParams = {
"name": bots[botCounter].name,
"pairs": bots[botCounter].pairs,
"max_active_deals": bots[botCounter].max_active_deals,
"base_order_volume": BaseOrder,
"take_profit": Number(bots[botCounter].take_profit),
"safety_order_volume": bots[botCounter].safety_order_volume,
"martingale_volume_coefficient": bots[botCounter].martingale_volume_coefficient,
"martingale_step_coefficient": Number(bots[botCounter].martingale_step_coefficient),
"max_safety_orders": bots[botCounter].max_safety_orders,
"active_safety_orders_count": Number(bots[botCounter].active_safety_orders_count),
"safety_order_step_percentage": Number(bots[botCounter].safety_order_step_percentage),
"take_profit_type": bots[botCounter].take_profit_type,
"strategy_list": bots[botCounter].strategy_list,
"bot_id": bots[botCounter].id
};
var keys = Object.keys(botParams);
var totalParams = keys.reduce(function(q, e, i) {
q += e + "=" + encodeURIComponent(JSON.stringify(botParams[e])) + (i != keys.length - 1 ? "&" : "");
return q;
},endPoint);
var signature = Utilities.computeHmacSha256Signature(totalParams, secret);
signature = signature.map(function(e) {return ("0" + (e < 0 ? e + 256 : e).toString(16)).slice(-2)}).join("");
var headers = {
'APIKEY': key,
'Signature': signature,
};
try {
var params = {
'method': 'PATCH',
'headers': headers,
'muteHttpExceptions': true
};
var response = JSON.parse(UrlFetchApp.fetch(baseUrl + totalParams, params).getContentText());
I have tried to set the botParams as a payload in the params but when I do the signature is incorrect.
I anyone knows how to use sheets to make a call using extensive length of parameters I'd appreciate any help at all
Some sample data for the bots array would be
{
"name": "TestBot",
"base_order_volume": 0.001,
"take_profit": 1.5,
"safety_order_volume": 0.001,
"martingale_volume_coefficient": 2,
"martingale_step_coefficient": 1,
"max_safety_orders": 1,
"active_safety_orders_count": 1,
"safety_order_step_percentage": 2.5,
"take_profit_type": "total",
"stop_loss_percentage": 0,
"cooldown": 0,
"pairs": ["BTC_ADA","BTC_TRX"],
"trailing_enabled":"true",
"trailing_deviation":0.5,
"strategy_list": [{"strategy":"cqs_telegram"}]
}
Thanks in advance
I'd consider using a Cloud Function to either do the heavy lifting, or, if you're worried about costs, use it as a proxy. You can then call the cloud function from Google Sheets. Cloud Functions can be written in whatever language you're most comfortable with, including Node.
Check the GCP pricing calculator to see what the cost would be. For many cases it would be completely free.
This should give you a sense of how to use cloud functions for CSV creation:
https://codelabs.developers.google.com/codelabs/cloud-function2sheet#0
Here is a SO question with an answer that explains how to query cloud functions with authentication.

Karate - how to access an array element by UUID during a 'retry until' statement

I have an endpoint which returns this JSON response:
{
"jobs": [
{
"name": "job1",
"id": "d6bd9aa1-0708-436a-81fd-cf22d5042689",
"status": "pending"
},
{
"name": "job2",
"id": "4fdaf09f-51de-4246-88fd-08d4daef6c3e",
"status": "pending"
}
]
I would like to repeatedly GET call this endpoint until the job I care about ("job2") has a "status" of "completed", but I'd like to check this by using a UUID stored in a variable from a previous call.
i.e. by doing something like this:
#NB: code for previous API call is executed
* def uuidVar = response.jobRef
#NB: uuidVar equates to '4fdaf09f-51de-4246-88fd-08d4daef6c3e' for this scenario
* configure retry = { count: 5, interval: 10000 }
Given path /blah
And retry until response.jobs[?(#.id==uuidVar)].status == 'completed'
When method GET
Could anyone suggest the correct syntax for the retry until?
I've tried referencing the fantastic Karate docs & examples (in particular, js-arrays.feature) and some questions on SO (including this one: Karate framework retry until not working as expected) but sadly I haven't been able to get this working.
I also tried using karate.match here as suggested in the link above, but no cigar.
Apologies in advance if I am missing something obvious.
First I recommend you read this answer on Stack Overflow, it is linked from the readme actually, and is intended to be the definitive reference. Let me know if it needs to be improved: https://stackoverflow.com/a/55823180/143475
Short answer, you can't use JsonPath in the retry until expression, it has to be pure JavaScript.
While you can use karate.jsonPath() to bridge the worlds of JsonPath and JS, JsonPath can get very hard to write and comprehend. Which is why I recommend using karate.filter() to do the same thing, but break down the steps into simple, readable chunks. Here is what you can try in a fresh Scenario:. Hint, this is a good way to troubleshoot your code without making any "real" requests.
* def getStatus = function(id){ var temp = karate.filter(response.jobs, function(x){ return x.id == id }); return temp[0].status }
* def response =
"""
{
"jobs": [
{
"name": "job1",
"id": "d6bd9aa1-0708-436a-81fd-cf22d5042689",
"status": "pending"
},
{
"name": "job2",
"id": "4fdaf09f-51de-4246-88fd-08d4daef6c3e",
"status": "pending"
}
]
}
"""
* def selected = '4fdaf09f-51de-4246-88fd-08d4daef6c3e'
* print getStatus(selected)
So if you have getStatus defined up-front, you can do this:
* retry until getStatus(selected) == 'completed'
Note you can use multiple lines for a JS function if you don't like squeezing it all into one line, or even read it from a file.

Community Connector getData() Request only uses the first two schema fields, not all four

I am building a Community Connector between Google Data Studio and SpyFu.com, in order to funnel SEO information for a specific url into the GDS Dashboard.
However, My getData() request only contains the first two fields from my Schema. As you can see, I have four listed in the code. The result is only the first two fields in the schema are printed to GDS.
I've been through tutorials, official documentation, YouTube videos, looked this issue up on google and checked out the community resources on GitHub.
//Step Two: Define getConfig()
function getConfig(request) {
var cc = DataStudioApp.createCommunityConnector();
var config = cc.getConfig();
config.newInfo()
.setId('instructions')
.setText('Give me SpyFu information on the following domain:');
config.newTextInput()
.setId('domain')
.setName('Enter the domain to search')
.setHelpText('e.g. ebay.com')
.setPlaceholder('ebay.com');
config.newTextInput()
.setId('SECRET_KEY')
.setName('Enter your API Secret Key')
.setHelpText('e.g. A1B2C3D4')
.setPlaceholder('A1B2C3D4');
config.setDateRangeRequired(false);
return config.build();
}
//Step Three: Define getSchema()
function getFields(request) {
var cc = DataStudioApp.createCommunityConnector();
var fields = cc.getFields();
var types = cc.FieldType;
var aggregations = cc.AggregationType;
fields.newDimension()
.setId('Keyword')
.setName('Keywords')
.setDescription('The keywords most often attributed to this domain.')
.setType(types.TEXT);
fields.newMetric()
.setId('Rank')
.setName('Rankings')
.setDescription('The ranking of the target site keyword on the Google Search Page.')
.setType(types.NUMBER);
fields.newMetric()
.setId('Local_Monthly_Searches')
.setName('Local Searches per Month')
.setDescription('Number of times, locally, that people have searched for this term within in the last month.')
.setType(types.NUMBER);
fields.newMetric()
.setId('Global_Monthly_Searches')
.setName('Global Searches per Month')
.setDescription('Number of times, globally, that people have searched for this term within in the last month.')
.setType(types.NUMBER);
return fields;
}
function getSchema(request) {
var fields = getFields(request).build();
return { schema: fields };
}
//Step Four: Define getData()
function responseToRows(requestedFields, response, domain) {
// Transform parsed data and filter for requested fields
return response.map(function(Array) {
var row = [];
requestedFields.asArray().forEach(function (field) {
switch (field.getId()) {
case 'Keyword':
return row.push(Array.term);
case 'Rank':
return row.push(Array.position);
case 'Local_Monthly_Searches':
return row.push(Array.exact_local_monthly_search_volume);
case 'Global_Monthly_Searches':
return row.push(Array.exact_global_monthly_search_volume);
case 'domain':
return row.push(domain);
default:
return row.push('');
}
});
return { values: row };
});
}
function getData(request) {
console.log("Request from Data Studio");
console.log(request);
var requestedFieldIds = request.fields.map(function(field) {
return field.name;
});
var requestedFields = getFields().forIds(requestedFieldIds);
// Fetch data from API
var url = [
'https://www.spyfu.com/apis/url_api/organic_kws?q='
+ request.configParams.domain
+ '&r=20'
+ '&p=[1 TO 10]'
+ '&api_key='
+ request.configParams.SECRET_KEY,
];
try {
var response = UrlFetchApp.fetch(url.join(''));
} catch (e) {
DataStudioApp.createCommunityConnector()
.newUserError()
.setDebugText('Failed URL Fetch Attempt. Exception details: ' + e)
.setText('There was an error accessing this domain. Try again later, or file an issue if this error persists.')
.throwException();
}
console.log("Response from API");
console.log(response);
//Parse data from the API
try {
var parsedResponse = JSON.parse(response);
} catch (e) {
DataStudioApp.createCommunityConnector()
.newUserError()
.setDebugText('Error parsing the JSON data. Exception details: ' + e)
.setText('There was an error parsing the JSON data. Try again later, or file an issue if this error persists.')
.throwException();
}
var rows = responseToRows(requestedFields, parsedResponse);
return {
schema: requestedFields.build(),
rows: rows
};
}
I need the GDS to post four columns of data. They are, "Keyword", "Rank", "Local Monthly Searches" and "Global Monthly searches".
I cannot figure out how to create a "fixed schema" so that the system always prints these four columns of data at every request. The tutorials and various documentation say it's possible, but not how to do it. Please help!
The number of metrics initially called up by the Google Community Connector is handled from the front-end, via Google Data Studio.
The back-end system (the Connector) only initially posts the default dimension and default metric. Getting the rest of the schemas to post should be handled when you are building a report on Google Data Studio. Simply click on the data set, select "data" on the right-hand menu, scroll down to either Metrics or Dimensions, and pick the ones you wish to add to the current set.
Note that these are the fields you established earlier in the coding process, when you were setting up your schemas.
Here, you're filtering your defined schema for fields that are present on the request object received by getData().
var requestedFieldIds = request.fields.map(function(field) {
return field.name;
});
var requestedFields = getFields().forIds(requestedFieldIds);
The visualization in Google Data Studio that is the catalyst for the request will determine which fields are requested.

Getting all bids from each Header bidding partners

We are implementing some header bidding partners on our wrapper using prebid. Is it possible to get all bids from each ssp.
Any help appreciated.
If you’re asking about demand, this is dependent on each SSP. For example there may be a segment pixel or placement in one SSP that will always give you a $10 bid, but that wouldnt apply to the other SSPs.
If your asking about getting data on all the bids, you may want to check out pbjs.getBidResponses() which returns an object with the ad units and bids
Heres a sample response from pbjs.getBidResponses() which can then be used however you'd need that data:
{
"div-id-one": {
"bids": [
{
"bidderCode": "appnexus",
"width": 970,
"height": 250,
"statusMessage": "Bid available",
"adId": "1293a95bb3e9615",
"mediaType": "banner",
"creative_id": 77765220,
"cpm": 0.7826,
"adUrl": "https://...",
"requestId": "57f961f3-a32b-45df-a180-9d5e53fb9070",
"responseTimestamp": 1513707536256,
"requestTimestamp": 1513707535321,
"bidder": "appnexus",
"adUnitCode": "div-id-one",
"timeToRespond": 935,
"pbLg": "0.50",
"pbMg": "0.70",
"pbHg": "0.78",
"pbAg": "0.75",
"pbDg": "0.78",
"pbCg": "0.78",
"size": "970x250",
"adserverTargeting": {
"hb_bidder": "appnexus",
"hb_adid": "1293a95bb3e9615",
"hb_pb": "0.78",
"hb_size": "970x250"
}
}
]
},
"div-id-two": {
"bids": []
}
}
Theres also a great example on prebid.org on how to output this to console.table that could be helpful as well:
var responses = pbjs.getBidResponses();
var output = [];
for (var adunit in responses) {
if (responses.hasOwnProperty(adunit)) {
var bids = responses[adunit].bids;
for (var i = 0; i < bids.length; i++) {
var b = bids[i];
output.push({
'adunit': adunit, 'adId': b.adId, 'bidder': b.bidder,
'time': b.timeToRespond, 'cpm': b.cpm, 'msg': b.statusMessage
});
}
}
}
if (output.length) {
if (console.table) {
console.table(output);
} else {
for (var j = 0; j < output.length; j++) {
console.log(output[j]);
}
}
} else {
console.warn('NO prebid responses');
}
There is also a chrome extensions called Prebid helper that do the same as console snippet but with less clicks.
However that is useful for initial setup debug. If you need to gather aggregated data on all demand partners - bids, timeouts, wins etc. You will need to run third party wrapper analytics or use analytics adapter. It's not free but it usually is priced depending on your load on the analytics server. For example https://headbidder.net/pricing
Try out the Chrome Extension called Adwizard. It's been built to debug prebid setups. Shows you all networks and bids per adunit. CPM and Size included.
https://chrome.google.com/webstore/detail/adwizard/kndnhcfdajkaickocacghchhpieogbjh/?ref=stackoverflow

How to pass same parameter with different value

I am trying the following API using Alamofire, but this API has multiple "to" fields. I tried to pass an array of "to" emails as parameters. It shows no error but did not send to all emails. API is correct, I tested that from terminal. Any suggestions will be cordially welcomed.
http -a email:pass -f POST 'sampleUrl' from="email#email.com" to="ongkur.cse#gmail.com" to="emailgmail#email.com" subject="test_sub" bodyText="testing hello"
I am giving my code:
class func sendMessage(message:MessageModel, delegate:RestAPIManagerDelegate?) {
let urlString = "http://localhost:8080/app/user/messages"
var parameters = [String:AnyObject]()
parameters = [
"from": message.messageFrom.emailAddress
]
var array = [String]()
for to in message.messageTO {
array.append(to)
}
parameters["to"] = array
for cc in message.messageCC {
parameters["cc"] = cc.emailAddress;
}
for bcc in message.messageBCC {
parameters["bcc"] = bcc.emailAddress;
}
parameters["subject"] = message.messageSubject;
parameters["bodyText"] = message.bodyText;
Alamofire.request(.POST, urlString, parameters: parameters)
.authenticate(user: MessageManager.sharedInstance().primaryUserName, password: MessageManager.sharedInstance().primaryPassword)
.validate(statusCode: 200..<201)
.validate(contentType: ["application/json"])
.responseJSON {
(_, _, jsonData, error) in
if(error != nil) {
println("\n sendMessage attempt json response:")
println(error!)
delegate?.messageSent?(false)
return
}
println("Server response during message sending:\n")
let swiftyJSONData = JSON(jsonData!)
println(swiftyJSONData)
delegate?.messageSent?(true)
}
}
First of all if you created the API yourself you should consider changing the API to expect an array of 'to' receivers instead of multiple times the same parameter name.
As back2dos states it in this answer: https://stackoverflow.com/a/1898078/672989
Although POST may be having multiple values for the same key, I'd be cautious using it, since some servers can't even properly handle that, which is probably why this isn't supported ... if you convert "duplicate" parameters to a list, the whole thing might start to choke, if a parameter comes in only once, and suddendly you wind up having a string or something ...
And I think he's right.
In this case I guess this is not possible with Alamofire, just as it is not possible with AFNetworking: https://github.com/AFNetworking/AFNetworking/issues/21
Alamofire probably store's its POST parameter in a Dictionary which doesn't allow duplicate keys.