Yesterday I went to add pagination to a chat service. I've implemented pagination a hundred times before using either page/per_page or start/end arguments, and I've always been aware that there is the potential for the order or offsets to change between page loads, but in the case of this chat service, that is much more likely to happen. My first approach was to use the updated_at column and have subsequent requests just pass something like "updated_after" but in this case, we often import multiple rooms at once that could easily have the same updated_at (down to the millisecond) for the same user.
One solution I found was to use an update count, or index that gets incremented (table wide) each time a row is updated. I've implemented it using a postgres SEQUENCE column that I set it to nextval('room_update_count') each time the room is updated (this could be a trigger but I wanted more control). Then I can just sort on that column and use a simple update_index < $next to get the following page.
However, I haven't been able to find any references to any technique like this. It seems to work perfectly but I'm skeptical that I'm not seeing some issue with it. I feel like I can't be the first person to come up with this idea, and that if I could just think of the correct name for it I would find other examples of it.
Is this a known technique? What are other ways to solve the pagination offset issue?
Related
I am having a dilema. I need to fetch data for some products by Id, these products which are selected can vary from a couple to thousands.
I see and tested that GET is not possible due to exceeding the HeaderSizeLimit of 8192.
I had discussions with colleagues and changed to POST and the ids are in the body. Everything works but have a lot of discussions about this. Have you encountered something like this? What was your approach?
First question for me is, do you really pass all those ids in a single request? How is this list of IDs generated in the first place? Could the server know this list in advance?
For example, if the list of IDs is obtained by doing a search query on the same server, perhaps that search query can already emit the list of entities.
I find that in most cases this can be avoided, but there's some exceptions.
If you find that you can't avoid this, I would suggest you use the new http QUERY method instead of of POST, but POST should be fine too as a fallback.
I have a search functionality that gets data from HERE API's Search endpoint. I maintain records of each search's results so I can add metadata that I need for my own purposes and also so I can provide results without always going back to HERE API. The problem I have is with paginating, specifically with providing a starting index when fetching results from HERE. Similar to how Algolia does it, I want to be able to search for a term and begin with the results at a certain index, the offset. HERE API apparently doesn't allow this at all. The closest it comes to such a feature is that it provides the URL for the next search, as described here. This is limited because it doesn't allow me to start the search results at a particular index that I specify. So essentially I want to know if there's a "standard" way of getting such functionality even when it's not provided by the API.
My own solution
The HERE API provides a size parameter that allows specifying the total number of results that I want, so I can specify a larger size than I need, and basically use code to start the results from my desired index. But this feels a bit hacky, and I wonder if there's a better/more established way of doing this.
Happy to listen to any ideas! Thanks. :)
Such a kind of an 'offset' for starting the paging after a specific number of results is indeed not supported by the Places API itself.
You have to set up a workaround within your application.
I was searching for a way to re-order my records, like blog posts, for instance.
One of the solutions I have found is to self-reference to refer to the previous (or next) value, like in a linked list (https://softwareengineering.stackexchange.com/a/375246). However, this requires the client-side (a web service or perhaps a mobile app) to implement the linked-list travesal logic to derive the order.
Is there a way to do this at the database level?
The reason for this is that if you are deriving the order at the client-side, then if you want to display only the first 10 records, you would have to retrieve all the records anyway.
EDIT
It seems the blog posts example was a very bad example, sorry. I was thinking of blog posts as they are displayed on an admin dashboard, and the user can re-order the position they are displayed by dragging and dropping. Hope this is more clear.
EDIT 2
I guess, generally, what I'm really asking is, how can one implement and query a tree-like structure in SQL
Which is correct way to pick elements from HTML : pick up all ids on a page together or picking each ID individually?
Let me explain the scenario.
I have a HTML page with information associated to specific IDs, but IDs sometimes change according to the section they are being displayed into.
Now which approach is better, either to pick the all the ID's from the page and then adding checks to get the value or picking up of separate ID individually.
I also need to work with many IDs (20 out of 100) on the page at same time.
Please suggests.
Thanks
If an element is static, in that it will not change throughout the course of your test then there is no problem selecting it at the beginning. However i definetly feel that attempting to select all your elements at the beginnning is a bad idea. What happens if an element is not available until you have completed some action, NoSuchElementException. Or what about if you select an element and when you go to use it the element cant be found, StaleElementException.
So you definetly need to find a balance and select your elements at the aprropriate time. I saw some other suggestion above to use xpath as it will never change, again thats not true.It is also recommended in the documentation to first attempt to select by id and then name. Im sure there are alot of reasons for this but one that immediatley comes to mind is readibility, speed is also a factor.
I'm currently designing an API and I came a cross a little problem:
How should a URL of a RESTful API look like when you should be able to identify an item by either an ID or a slug?
I could think of three options:
GET /items/<id>
GET /items/<slug>
This requires that the slug and the ID are distinguishable, which is not necessarily given in this case. I can't think of a clean solution for this problem, except you do something like this:
GET /items/id/<id>
GET /items/slug/<slug>
This would work fine, however this is not the only place I want to identify items by either a slug or an ID and it would soon get very ugly when one wants to implement the same approach for the other actions. It's just not very extendable, which leads us to this approach:
GET /items?id=<id>
GET /items?slug=<slug>
This seems to be a good solution, but I don't know if it is what one would expect and thus it could lead to frustrating errors due to incorrect use. Also, it's not so easy - or let's say clean - to implement the routing for this one. However, it would be easily extendable and would look very similar to the method for getting multiple items:
GET /items?ids=<id:1>,<id:2>,<id:3>
GET /items?slugs=<slug:1>,<slug:2>,<slug:3>
But this has also a downside: What if someone wants to identify some of the items he want to fetch with IDs, but the others with a slug? Mixing these identifiers wouldn't be easy to achieve with this.
What is the best and most widely-accepted solution for these problems?
In general, what matters while designing such an API?
Of the three I prefer the third option, it's not uncommon to see that syntax; e.g. parts of Twitter's API allow that syntax:
https://dev.twitter.com/rest/reference/get/statuses/show/id
A fourth option is a hybrid approach, where you pick one (say, ID) as the typical access method for single items, but also allow queries based on the slug. E.g.:
GET /items/<id>
GET /items?slug=<slug>
GET /items?id=<id>
Your routing will obvious map /items/id to /items?id=
Extensible to multiple ids/slugs, but still meets the REST paradigm of matching URIs to the underlying data model.