REST Fetching data with GET not possible due to exceed header size limit - api

I am having a dilema. I need to fetch data for some products by Id, these products which are selected can vary from a couple to thousands.
I see and tested that GET is not possible due to exceeding the HeaderSizeLimit of 8192.
I had discussions with colleagues and changed to POST and the ids are in the body. Everything works but have a lot of discussions about this. Have you encountered something like this? What was your approach?

First question for me is, do you really pass all those ids in a single request? How is this list of IDs generated in the first place? Could the server know this list in advance?
For example, if the list of IDs is obtained by doing a search query on the same server, perhaps that search query can already emit the list of entities.
I find that in most cases this can be avoided, but there's some exceptions.
If you find that you can't avoid this, I would suggest you use the new http QUERY method instead of of POST, but POST should be fine too as a fallback.

Related

How to fetch results from an offset when the API doesn't support offset (HERE Maps API)

I have a search functionality that gets data from HERE API's Search endpoint. I maintain records of each search's results so I can add metadata that I need for my own purposes and also so I can provide results without always going back to HERE API. The problem I have is with paginating, specifically with providing a starting index when fetching results from HERE. Similar to how Algolia does it, I want to be able to search for a term and begin with the results at a certain index, the offset. HERE API apparently doesn't allow this at all. The closest it comes to such a feature is that it provides the URL for the next search, as described here. This is limited because it doesn't allow me to start the search results at a particular index that I specify. So essentially I want to know if there's a "standard" way of getting such functionality even when it's not provided by the API.
My own solution
The HERE API provides a size parameter that allows specifying the total number of results that I want, so I can specify a larger size than I need, and basically use code to start the results from my desired index. But this feels a bit hacky, and I wonder if there's a better/more established way of doing this.
Happy to listen to any ideas! Thanks. :)
Such a kind of an 'offset' for starting the paging after a specific number of results is indeed not supported by the Places API itself.
You have to set up a workaround within your application.

Picking up of multiple IDs together or individual ID in selenium

Which is correct way to pick elements from HTML : pick up all ids on a page together or picking each ID individually?
Let me explain the scenario.
I have a HTML page with information associated to specific IDs, but IDs sometimes change according to the section they are being displayed into.
Now which approach is better, either to pick the all the ID's from the page and then adding checks to get the value or picking up of separate ID individually.
I also need to work with many IDs (20 out of 100) on the page at same time.
Please suggests.
Thanks
If an element is static, in that it will not change throughout the course of your test then there is no problem selecting it at the beginning. However i definetly feel that attempting to select all your elements at the beginnning is a bad idea. What happens if an element is not available until you have completed some action, NoSuchElementException. Or what about if you select an element and when you go to use it the element cant be found, StaleElementException.
So you definetly need to find a balance and select your elements at the aprropriate time. I saw some other suggestion above to use xpath as it will never change, again thats not true.It is also recommended in the documentation to first attempt to select by id and then name. Im sure there are alot of reasons for this but one that immediatley comes to mind is readibility, speed is also a factor.

Providing complex filtering REST API [duplicate]

This question already has answers here:
REST and complex search queries
(5 answers)
Closed 10 years ago.
So I am building a RESTful (as RESTful as I can) API with the Laravel 4 PHP Framework. Right now I have dozens of API calls working and I have a process for being to do limit, ordering, and do simple filtering. Here would be an example of one of the calls:
/api/v1/users?limit=10&offset=10&firstName=John&order[]=createdTimestamp desc
This would return the 11th through 20th users that have a first name of John ordered by the createdTimestamp in descending order. The simple filtering here can only does exact matches (=). Now I also want to be able to provide a more complex filtering system through the REST API that supports the ability to specific the equality match type that way they could do a != or > or LIKE, etc... The issue is that I don't know if I am going to be able to provide this type of filtering through a normal query string.
What is the best way to provide this complex filtering through a REST API? Is doing through a POST still considered the best way even though it is not "truly" RESTful (even though this would prevent issues of the user trying to run a long query that exceeds the URI character length limit that some browsers have)?
#ryanzec
Now I also want to be able to provide a more complex filtering system
through the REST API that supports the ability to specific the
equality match type that way they could do a != or > or LIKE, etc...
The issue is that I don't know if I am going to be able to provide
this type of filtering through a normal query string.
It's not possible with simple query string(well, maybe it's possible but is very hard to encode such logic properly in query string). You need to define custom query format and use POST to submit such query. Server may respond with:
"201 Created" status and "Location" header field indicating query resource if there was no such query before; or
"303 See Other" and "Location" header field indicating already existing query resource.
Is doing through a POST still considered the best way even though it
is not "truly" RESTful
I do not know who said this, but it's wrong. There is nothing wrong with using POST for such purposes.
Use forms in your collection resource responses to tell the client how to search the collections. See my answer to REST and complex search queries for examples.

Specify items per page

I'm trying to query Picasa Web Albums to pull album/photo data (obviously!) however initially I only need to pull the first 4 photos. Is there any way to limit a particular field or specify the items per page?
I've already accomplished something similar with Facebook's Graph API however I'm unable to find anything similar for Picasa. The only items I can find related to limiting the response is specifying which fields, but nothing related to the number of rows.
Use max-results argument. See doc here.
https://developers.google.com/picasa-web/docs/2.0/reference#Parameters

Is it possible to retrieve arbitrary number of items from reddit using API?

I am writing a small assistant app to read (well, filter/rank) /r/programming/ for me, because it has so many damn posts, and because certain area of my coding skills was getting rusty and it sounds like good exercise.
I am getting items from "new" page of the subreddit using json api; however it only returns 25 items per request (which is the page size), so to retrieve items for, say, last week, I need to make dozens of requests. As the mandated request interval is 2s, it is painful.
I wonder if there's some way to retrieve more items? Query string parameters for standard html gets also work for json gets, but I cannot find one for page size.
EDIT: for posterity, the paramrter name is "limit", although that too is capped at 100
for posterity, the parameter name is "limit", although that too is capped at 100