I have a "copy data" activity making a REST call to get some data in json format. The data should then be transfered to a SQL database.
The problem is I can only get a certain amount of data for the specified HTTP request. For this I need to implement pagination rules, and I have tried to understand this documentation: https://learn.microsoft.com/en-us/azure/data-factory/connector-rest#pagination-support
The HTTP response returns the absolute URL for the next request in the header, with the field named "Link". As I can see in the documentation, it is supposed to be possible to get the value from "Link" and put it into a pagination rule.
As stated in the documentation:
Next request’s absolute or relative URL = header value in current response headers
It says supported pagination keys are AbsoluteUrl, with the value of this should be set like this:
Headers.response_header OR Headers['response_header']
Where the response_header is defined like this in the docs:
"response_header" is user-defined which references one header name in the current HTTP response, the value of which will be used to issue next request.
What I can't seem to understand is how this "response_header" can be set to reference the HTTP response header value of "Link".
You need to replace the 'response_header' placeholder with your header name.
In your case - 'Link'
Or in code editor:
"paginationRules": {
"AbsoluteUrl": "Headers['Link']"
}
Related
I already read the documentation and I think I am making the simplest request in the correct way, but it always returns only the IDs, instead of all the fields of the games
Documentation example: Documentation Example
The request header is fine. I know this because I can get the expected request if fields = * as querystring
this is my request:
You have to provide the fields you want inside the body.
Like this:
fields age_ratings,aggregated_rating,aggregated_rating_count,alternative_names,artworks,bundles,category,checksum,collection,cover,created_at,dlcs,expansions,external_games,first_release_date,follows,franchise,franchises,game_engines,game_modes,genres,hypes,involved_companies,keywords,multiplayer_modes,name,parent_game,platforms,player_perspectives,rating,rating_count,release_dates,screenshots,similar_games,slug,standalone_expansions,status,storyline,summary,tags,themes,total_rating,total_rating_count,updated_at,url,version_parent,version_title,videos,websites;"
Scenario : I need to verify the logs response on server on the basis of Tracking-Id.
I had passed the tracking-id using 'header.js' file, here i define metod which get the unique UUID for every request and passed in header.
Now I need that header value to passed in some method to get logs only for specific Tracking-Id
Is there any way to achieve this in karate?
Yes, you can use karate.set('someVarName', uuidValue) in JS and then back in the feature you will be able to: * print someVarName.
EDIT: please see this commit for an example: link
I'm using JMeters to automate API testing to/from our database using basic CRUD methodology. After the record creation, I'm trying to perform 3 different types of Reads (think CRRRUD). :)
Read 1 - Retrieve by ID
HTTP (GET) the base URL is appended with the saved record ID.
http..../crud/tableName/${newRecordId}
This returns
Read 2 - Retrieve by field type with no defined value
HTTP (POST) the base URL is extended with a "search" as the end. A Key is defined ("name") with no value.
http..../crud/tableName/search
Parameter Name = name
Value = {undefined}
This returns all records within the table whose field ("name") is not null.
Read 3 - Retrieve by field type with a defined value
HTTP (POST) the base URL is extended with a "search" as the end. A Key is defined ("name") with the value generated during the creation request.
http..../crud/tableName/search
Parameter Name = name
Value = Metropolis
This, too, returns ALL of the records within the table instead of just the record(s) whose name = Metropolis.
This 3rd Retrieve works properly when using a REST client, (e.g., Postman, Advanced REST Client, etc.) when defining the Key|Value pair as "name|Metropolis". So it must be something within JMeter that I'm missing.
Any insight is appreciated!
There is only one correct answer: compare what's being sent by REST Client and JMeter using sniffer tool (i.e. Wireshark), detect the differences and configure JMeter accordingly.
Just a guess: given you mentioned "API" an "REST" words maybe you need to pass it as JSON like:
{"name":"Metropolis"}
as a single parameter value
or in formatted way (in case if server checks Content-Length and/or wants the request to be properly formatted)
You may also need to add HTTP Header Manager in order to send Content-Type header with the value of application/json. See Testing SOAP/REST Web Services Using JMeter guide for more details.
When I fetch from a dStore the URL looks something like this
http://localhost/rest/dojo?department=sales
which works fine. If I then click in the header of the dGrid the sent URL looks like this.
http://localhost/rest/dojo?department=sales&sort(+id)&limit(25)
Shouldn't it send &sort=+id&limit=25? I am using Java and Spring for the backend and it expects the parameters to be formatted this way. Right now I can not receive the extra parameters. Is there a way to get it to send the parameters the way Spring is expecting them?
sort(...) and limit(...) are the default behaviors of dstore/Request (which Rest extends), but these can be customized via sortParam for sort, and useRangeHeaders or rangeStartParam and rangeCountParam for range.
For example, to result in &sort=+id&limit=25 as you requested, you could set up your store as follows:
var store = new Rest({
target: '...',
sortParam: 'sort',
rangeStartParam: 'offset',
rangeCountParam: 'limit'
});
I've additionally assumed above that offset is the GET parameter you'd want to use to indicate what record to start at when requesting ranges. Generally if you're not using range headers (useRangeHeaders defaults to false) and you want to set a count GET parameter, you'll also need to set a start GET parameter.
These properties are listed in the Request Store documentation.
6.30.15 - HOW CAN I MAKE THIS QUESTION BETTER AND MORE HELPFUL TO OTHERS? FEEDBACK WOULD BE HELPFUL. THANKS!
I need to send a content-range header to a dojo/dgrid request:
I cannot find any examples of HOW to do this. I'm not exactly sure where this setting goes (Content-Range: items 0-9/*). I have been given a great linkheaderpagination example on this question: Django Rest Framework Pagination Settings - Content-Range But I don't know how to make this work to produce a Content-Range response. Any takers or does anyone know of any good resources or examples??
UPDATE: I am trying to create pagination in Dojo/grid. I have am using a server-side api (Django Rest Framework) to supply to data to the Dojo/Dgrid. Django Rest Framework does not automatically sent content-range headers when it gets a response from Dojo. Dojo sends a range request when formatted to have pagination. I don't know now to configure the Django Rest Framework API to send a content-range header when it gets a request from Dojo. Unfortunately, I'm trying to do something very specific and just general settings on either side doesn't work.
Including Content-Range Header in response:
You just need to create a headers dictionary with Content-Range as the key and value as how many items are being returned and how many total items exist.
For example:
class ContentRangeHeaderPagination(pagination.PageNumberPagination):
"""
A custom Pagination class to include Content-Range header in the
response.
"""
def get_paginated_response(self, data):
"""
Override this method to include Content-Range header in the response.
For eg.:
Sample Content-Range header value received in the response for
items 11-20 out of total 50:
Content-Range: items 10-19/50
"""
total_items = self.page.paginator.count # total no of items in queryset
item_starting_index = self.page.start_index() - 1 # In a page, indexing starts from 1
item_ending_index = self.page.end_index() - 1
content_range = 'items {0}-{1}/{2}'.format(item_starting_index, item_ending_index, total_items)
headers = {'Content-Range': content_range}
return Response(data, headers=headers)
Suppose this is the header received:
Content-Range: items 0-9/50
This indicates that first 10 items are returned out of total 50.
Note: You can also use * instead of total_items if calculating total is expensive.
Content-Range: items 0-9/* # Use this if total is expensive to calculate
If you are talking about providing Content-Range in the response, I mentioned in an answer to another SO question (which I believe may also have originated from your team?) that there is one alternative to this header: if your response format is an object (not just an array of items), it can specify a total property indicating the total number of items instead.
Again looking for a few minutes at the DRF documentation, it appears it should be possible to customize the format of the response.
Based on the docs and reading the source of LimitOffsetPagination (which you want to be using to work with dstore, as already discussed in a previous question), if I had to take a wild guess, you should be able to do the following server-side:
class CustomPagination(pagination.LimitOffsetPagination):
def get_paginated_response(self, data):
return Response(OrderedDict([
('total', self.count),
('next', self.get_next_link()),
('previous', self.get_previous_link()),
('items', data)
]))
This purposely assigns the count to total and the data to items to align with the expectations of dstore/Request. (next and previous are entirely unnecessary as far as dstore is concerned, so you could take or leave them, depending if you have any use for them elsewhere.)