How to build pagination API using Hapi JS and MYSQL - hapi.js

I would like to develop an API using HAPI JS and MYSQL using limit and offset parameters
Could anyone please suggest me resources on how to develop this.
Thanks.
Example: GET /schools?offset=10&limit=20

Paging depends on very basic principles.
First, you need to know in a particular query, which you'll show in the page you have, how many rows do you have?
For example, you listing blog post on your page, get total row
SELECT count(*) FROM blog_post_table; // cosnt numRows = Query....,
Let's say it's 168;
Now, you need to decide how many rows will be shown in a page. On your example, there is a query parameter "limit=20" which answers our question.
const perPage = request.query.limit || 20;
Based on the limit parameter in the query string or at least 20 items per page, validation is your homework, Joi.
Then, we should calculate how many pages will be there based on our perPage information.
const totalPages = Match.ceil(numRows / perPage);
It results in 9, we used Math.ceil, you get it I assume, you can't divide 168 with 20 because there are 8 remaining items. The result w 8.4, means we need one more page to show remaining 8 items.
So far so good, now we have to calculate our offset, where we start from reading data. In the first page, of course, it starts from 0, but what about second or fourth or ninth?
Now we need a current page parameter, and we should carry this information in query strings (I discarded your offset=10 definition on purpose), let's say there is parameter in query string named page
const currentPage = request.query.page || 1; // easy, yet harmful, it needs validation
Now we know our current page, now we can calculate our limit based on those figures.
const offset = perPage * (currentPage - 1);
To realize, put numbers on the places
const offset = 20 * (1-1); // page 1 offset = 0;
const offset = 20 * (2-1); // page 2 offset = 20;
.
.
.
const offset = 20 * (9-1); // page 9 offset = 160;
Now we can make our main query based on our calculations.
const sql = `SELECT * FROM blog_post_table LIMIT ${perPage} OFFSET ${offset}`;
Fetch rows and send to your page. Some additional information will be useful if you want to show a proper paging interface.
Let's say your response data is like this;
const response = {
'results': ['array of elements from db'],
'currentPage': currentPage, // learn object literals, it unnecessary,
'nextPage': currentPage + 1, // don't be that naive, validate this,
'lastPage': totalPages, // it is obvious
'previousPage': currentPage - 1, // what if we're in the very first page, then we can go to 0, validate!
}
BONUS:
Little code that generates previous and next pages, not a perfect solution just made it up quickly.
const getOtherPages = (totalPages, currentPage) => {
previousPage = null
nextPage = null
if currentPage > totalPages or currentPage < 1:
return previousPage, nextPage
if currentPage > 1:
previousPage = currentPage - 1
if (currentPage + 1) < totalPages:
nextPage = currentPage + 1
return [previousPage, nextPage]
}
// Usage
const [previousePage, nextPage] = getOtherPages(totalPages, currentPage);

Related

Get total results for revisions (RavenDB Client API)

I am trying to display document revisions in a paginated list. This will only work when I know the total results count so I can calculate the page count.
For common document queries this is possible with session.Query(...).Statistics(out var stats).
My current solution is the following:
// get paged revisions
List<T> items = await session.Advanced.Revisions.GetForAsync<T>(id, (pageNumber - 1) * pageSize, pageSize);
double totalResults = items.Count;
// get total results in case items count equals page size
if (pageSize <= items.Count || pageNumber > 1)
{
GetRevisionsCommand command = new GetRevisionsCommand(id, 0, 1, true);
session.Advanced.RequestExecutor.Execute(command, session.Advanced.Context);
command.Result.Results.Parent.TryGet("TotalResults", out totalResults);
}
Problem with this approach is that I need an additional request just to get the TotalResults property which indeed has already been returned by the first request. I just don't know how to access it.
You are right the TotalResults is returned from server but not parsed on the client side.
I opened an issue to fix that: https://issues.hibernatingrhinos.com/issue/RavenDB-15552
You can also get the total revisions count for document by using the /databases/{dbName}/revisions/count?id={docId} endpoint, client code for example:
var docId = "company/1";
var dbName = store.Database;
var json = await store.GetRequestExecutor().HttpClient
.GetAsync(store.Urls.First() + $"/databases/{dbName}/revisions/count?id={docId}").Result.Content.ReadAsStringAsync();
using var ctx = JsonOperationContext.ShortTermSingleUse();
using var obj = ctx.ReadForMemory(json, "revision-count");
obj.TryGet("RevisionsCount", out long revisionsCount);
Another way could be getting all the revisions:
var revisions = await session.Advanced.Revisions.GetForAsync<Company>(docId, 0, int.MaxValue);
Then using the revisions.Count as the total count.

How to select efficiently from a long list of options in react-select

My use case is to allow the user to select a ticker from a long list of about 8000 companies. I fetch all the companies when the component mounts, so I don't really need the async feature of react-select. The problem really is displaying and scrolling through the 8000 items (as described in several open issues like this one).
My thought is why display 8000 entries when the user can't do anything meaningful with such a big list anyway. Instead why not show a maximum of 5 matches. As the user types more, the matches keep getting better. Specifically:
When the input is blank, show no options
When the input is a single character, there will still be hundreds of matches, but show only the first 5
As the user keeps on typing, the number of matches will reduce, but still limited to 5. However they will be more relavant.
I am not seeing this solution mentioned anywhere, so was wondering if it makes sense. Also wanted to find out what's the best way to implement it with react-select. I have tried the following two approaches - can you think of a better way:
Approach 1: Use Async React Select
Although I don't need async fetching, I can use this feature to filter down the options. It seems to work very well:
const filterCompanies = (value: string) => {
const inputValue = value.trim().toLowerCase();
const inputLength = inputValue.length;
let count = 0;
return inputLength === 0
? []
: companies.filter(company => {
const keep =
count < 5 &&
(company.ticker.toLowerCase().indexOf(inputValue) >= 0 ||
company.name.toLowerCase().indexOf(inputValue) >= 0);
if (keep) {
count += 1;
}
return keep;
});
};
const promiseOptions = (inputValue: string) =>
Promise.resolve(filterCompanies(inputValue));
return (
<AsyncSelect<Company>
loadOptions={promiseOptions}
value={selectedCompany}
getOptionLabel={option => `${option.ticker} - ${option.name}`}
getOptionValue={option => option.ticker}
isClearable={true}
isSearchable={true}
onChange={handleChange}
/>
);
Approach 2: Use filterOption
Here I am using the filterOption to directly filter down the list. However it does not work very well - the filterOption function is very myopic - it gets only one candidate option at a time and needs to decide if that matches or not. Using this approach I cannot tell whether I have crossed the limit of showing 5 options or not. Net result: with blank input I am showing all 8000 options, as user starts typing, the number of options is reduced but still pretty large - so the sluggishness is still there. I would have thought that filterOption would be the more direct approach for my use case but it turns out that it is not as good as the async approach. Am I missing something?
const filterOption = (candidate: Option, input: string) => {
const { ticker, name } = candidate.data;
const inputVal = input.toLowerCase();
return (
ticker.toLowerCase().indexOf(inputVal) >= 0 ||
name.toLowerCase().indexOf(inputVal) >= 0
);
};
return (
<ReactSelect
options={companies}
value={selectedCompany}
filterOption={filterOption}
getOptionLabel={option => `${option.ticker} - ${option.name}`}
getOptionValue={option => option.ticker}
isClearable={true}
isSearchable={true}
onChange={handleChange}
/>
);
you can try using react-window to replace the menulist component
ref : https://github.com/JedWatson/react-select/issues/3128#issuecomment-431397942

TYPO3: Get the values in fluid from inline elements

I managed to create an own inline content element (on tt_content table) but when i try to get the values on the frontend via fluid, i get nothing.
I debugged the {data} variable and on the column that my data are saved, there is an integer. I suppose it reads the number of the content elements which were created on the foreign table (accordion). How can i get those values?
At this point the {data} variables reads the tt_content table and the column that has the integer reads the number of content elements on the table accordion.
I suppose no code is necessary. If it is necessary, feel free to comment the part of the code you would like to review.
Best regards
You need to add a DataProcessor to your TypoScript creating the content element, which fetch your accordion records. Example:
tt_content {
yourContentElementName < lib.contentElement
yourContentElementName.templateName = YourContentElementName
yourContentElementName.dataProcessing {
10 = TYPO3\CMS\Frontend\DataProcessing\DatabaseQueryProcessor
10 {
if.isTrue.field = fieldInTtContentWithInteger
table = your_accordion_table
pidInList = this
where.field = uid
where.intval = 1
where.dataWrap = field_pointing_to_ttcontent_record = |
as = accordions
}
}
}

Why can I not use Continuation when using a proxy class to access MS CRM 2013?

So I have a standard service reference proxy calss for MS CRM 2013 (i.e. right-click add reference etc...) I then found the limitation that CRM data calls limit to 50 results and I wanted to get the full list of results. I found two methods, one looks more correct, but doesn't seem to work. I was wondering why it didn't and/or if there was something I'm doing incorrectly.
Basic setup and process
crmService = new CrmServiceReference.MyContext(new Uri(crmWebServicesUrl));
crmService.Credentials = System.Net.CredentialCache.DefaultCredentials;
var accountAnnotations = crmService.AccountSet.Where(a => a.AccountNumber = accountNumber).Select(a => a.Account_Annotation).FirstOrDefault();
Using Continuation (something I want to work, but looks like it doesn't)
while (accountAnnotations.Continuation != null)
{
accountAnnotations.Load(crmService.Execute<Annotation>(accountAnnotations.Continuation.NextLinkUri));
}
using that method .Continuation is always null and accountAnnotations.Count is always 50 (but there are more than 50 records)
After struggling with .Continutation for a while I've come up with the following alternative method (but it seems "not good")
var accountAnnotationData = accountAnnotations.ToList();
var accountAnnotationFinal = accountAnnotations.ToList();
var index = 1;
while (accountAnnotationData.Count == 50)
{
accountAnnotationData = (from a in crmService.AnnotationSet
where a.ObjectId.Id == accountAnnotationData.First().ObjectId.Id
select a).Skip(50 * index).ToList();
accountAnnotationFinal = accountAnnotationFinal.Union(accountAnnotationData).ToList();
index++;
}
So the second method seems to work, but for any number of reasons it doesn't seem like the best. Is there a reason .Continuation is always null? Is there some setup step I'm missing or some nice way to do this?
The way to get the records from CRM is to use paging here is an example with a query expression but you can also use fetchXML if you want
// Query using the paging cookie.
// Define the paging attributes.
// The number of records per page to retrieve.
int fetchCount = 3;
// Initialize the page number.
int pageNumber = 1;
// Initialize the number of records.
int recordCount = 0;
// Define the condition expression for retrieving records.
ConditionExpression pagecondition = new ConditionExpression();
pagecondition.AttributeName = "address1_stateorprovince";
pagecondition.Operator = ConditionOperator.Equal;
pagecondition.Values.Add("WA");
// Define the order expression to retrieve the records.
OrderExpression order = new OrderExpression();
order.AttributeName = "name";
order.OrderType = OrderType.Ascending;
// Create the query expression and add condition.
QueryExpression pagequery = new QueryExpression();
pagequery.EntityName = "account";
pagequery.Criteria.AddCondition(pagecondition);
pagequery.Orders.Add(order);
pagequery.ColumnSet.AddColumns("name", "address1_stateorprovince", "emailaddress1", "accountid");
// Assign the pageinfo properties to the query expression.
pagequery.PageInfo = new PagingInfo();
pagequery.PageInfo.Count = fetchCount;
pagequery.PageInfo.PageNumber = pageNumber;
// The current paging cookie. When retrieving the first page,
// pagingCookie should be null.
pagequery.PageInfo.PagingCookie = null;
Console.WriteLine("#\tAccount Name\t\t\tEmail Address");while (true)
{
// Retrieve the page.
EntityCollection results = _serviceProxy.RetrieveMultiple(pagequery);
if (results.Entities != null)
{
// Retrieve all records from the result set.
foreach (Account acct in results.Entities)
{
Console.WriteLine("{0}.\t{1}\t\t{2}",
++recordCount,
acct.EMailAddress1,
acct.Name);
}
}
// Check for more records, if it returns true.
if (results.MoreRecords)
{
// Increment the page number to retrieve the next page.
pagequery.PageInfo.PageNumber++;
// Set the paging cookie to the paging cookie returned from current results.
pagequery.PageInfo.PagingCookie = results.PagingCookie;
}
else
{
// If no more records are in the result nodes, exit the loop.
break;
}
}

NHibernate.Search - SQL Server 2005 - hitting max parameter limit 2100 !

I am using NHibernate.Search libraries in my project for free text search. Recently when I started getting more than 2100 results, I started getting max parameter length error from SQL Server.
Does NHibernate.Search take care of such situation ? Any workaround anyone ?
You could modify NHibernate.Search code to take care of this, or, use custom paging, IE get number of hits for your search, then page nhibernate search results accordingly.
public IList<TEntity> Search<TEntity>(Query query, bool? active, string orderBy)
{
var search = NHibernate.Search.Search.CreateFullTextSession(this.session);
var total = search.CreateFullTextQuery(query, typeof(TEntity)).ResultSize;
var first = 0;
var l = new List<TEntity>();
while (total > 0)
{
l.AddRange(search.CreateFullTextQuery(query, typeof(TEntity))
.SetFirstResult(first)
.SetMaxResults(1000)
.List<TEntity>());
first += 1000;
total -= 1000;
}
return l;
}
See : IFullTextQuery - exception if there are too may objects