CGridview With Custom Dataprovider That Doesn't Load All Records From Source (Lazy Loading) - yii

Question

I'm a 5-day Yii fan trying to learn the ropes of this great framework. I would really appreciate it if you could bear with me reading through my problem and help me out.
So here goes, I'm retrieving a list of products sold by a particular merchant of an online marketplace through its web service. When I query the web service like this
api.marketplace.com?merchant_id=1234
I'll get back the response in JSON which consists of two main pieces of data
The number of products sold by Merchant 1234 (total_products)
The list (array) of the products sold (list_product). Each element in this array represents one distinct Product object with two fields: product_id and product_name respectively.
So if total_products = 934, there'll be 934 elements in the array.
When I put it in codes, my derived DataProvider looks like this
class ProductDataProvider extends CDataProvider /* or should I extend other better class? */ {
public function __construct($config = array()) {
foreach($config as $key=>$value)
$this->$key=$value;
}
protected function fetchData() {
$request = "api.marketplace.com?merchant_id=1234";
$response = retrieveProductsAndProcessJSONResponseIntoAppropriateDataStructure($request);
$this->setTotalItemCount($response->total_products);
if(($pagination=$this->getPagination())!==false)
$pagination->setItemCount($this->getTotalItemCount());
return $response->list_products;
}
protected function calculateTotalItemCount() {
return $this->getTotalItemCount();
}
}
My model
class Product extends CModel {
public $product_id;
public $product_name;
public function display() {
return new ProductDataProvider(array(
'pagination'=>array(
'pageSize'=>10,
));
}
}
and my View
$this->widget('zii.widgets.grid.CGridView', array(
'id'=>'product-grid',
'dataProvider'=>$model->display(),
'columns'=>array(
'product_id',
'product_name',
),
));
This works well with pagination and all that. However retrieving all 934 products is taking a bit too long. We can do this to the web service.
api.marketplace.com?merchant_id=1234&page_size=10&page_no=1
This will only retrieve the first 10 records. We can play around with the query string variable page_no to retrieve subsequent pages of products in groups of 10. The total_products returned from the web service will still be 934.
So my question is, how do I put this into play so that the ProductDataProvider only retrieves 10 product at any one time and load it into CGridView? The CGridView will have to know that there are a total of 934 products to enable pagination. User browsing to other pages of the CGridView will trigger the ProductDataProvider to retrieve the next 10 records corresponding to the page number being viewed. I hope that my long winded question is comprehensible.

Answer 1

I had the same issue. So, I think:
Firstly, you should implement all 3 abstract methods from CDataProvider. In your class fetchKeys() is not implemented.
Secondly, pagination logic should be put in fetchData. For instance:
protected function fetchData() {
$request = "api.marketplace.com?merchant_id=1234";
// page size
$request .= "&page_size=".$this->pagination->pageSize;
// current page
$request .= "&page_no=".$this->pagination->currentPage;
$response = retrieveProductsAndProcessJSONResponseIntoAppropriateDataStructure($request);
return $response->list_products;
}
To use your new custom data provider write smth like that:
$dataProvider = new CustomDataProvider();
$pages = new CPagination($dataProvider->getTotalItemCount());
$pages->pageSize = 40; // page size
$dataProvider->pagination = $pages;
Data providers get total number of items using calculateTotalItemCount(). So, in addition, you should write there some logic to provide total item count. Guess, you should exploit another method in your API for that purpose. Remember, that if you use pagination, your data provider will do 2 api requests: the first for total item count and the second for current page items. Some can say that only one request is enough, but in this case, that request will return all data (934 elements). So heavy.
That's all you need.

Answer 2

I think that you need to create your own custom data provider. See the Class Reference for CDataProvider
There are three implemented data providers in Yii. CActiveDataProvider, CArrayDataProvider, and CSqlDataProvider. You can use those as a model for your custom data provider.