Concrete5 attribute counts - optimization

Question

We built a site with Concrete5 that was originally developed in Joomla. Our job was to bring over everything and Concrete5-ize it. A major part of this site is about 1200 audio teachings, with each teaching having various attributes, such as topic, author, program, location, etc.
Some teachings might have more than one attribute assigned, say multiple keywords or topics.
I would like to give counts to all of the attributes so that the visitor can see how many teachings are by a certain author, or how many are on a particular topic at a glance, ie:
Ethics (20)
Fear (42)
Gratitude (55)
My original code turned out to have way too much overheard to be practical for so many teachings and so many attributes. Basically, I ran through and for each attribute, I did a lookup for the total count based on the PageList count. We're talking hundreds of lookups with each page load. Turning on cache didn't seem to help here.
Are there any other strategies that have proved successful for aggregating counts for attributes over a large-ish number of pages?
Here is the site for reference: http://everydayzen.org/teachings/

Answer 1

I typically say "don't access the database directly; use the API", but I think you should be using the DB here.
Check out the [Collection|File]SearchIndexAttributes table. (I'm not sure if teachings are files or pages. If pages, you'll need to reindex them regularly, via the job in the dashboard.) Looking at the index table will be a lot easier than joining in the most-recent version in the attribute value table. Once you see that table, you can do some simple GROUPing within SQL.
If you want to use the API, you could do it as you do today as a batch, do the appropriate calculations, and then cache it.
There's no reason caching shouldn't work but the first hit (when the cache is cold) will, of course, take the full amount of time. You should cache my IndexAttributes idea (a full table read and looping isn't trivial), but at least with a cold cache that should take a fraction of a second vs 10 or more seconds that hundreds of page list calls could take.

Answer 2

I've done something similar on a job site for Concrete5, by showing the counts of each Department that jobs fall in.
i.e. HR (32), Sales (12) and so on
This is the code taken from a Helper that acheives this (this is just the related functions included):
<?php
class JobHelper {
/**
* GetDepartmentJobsCount
* Returns array of Department names with job count based on input Pages
* #param Array(Pages) - Result of a PageList->getPages
* #return Array
*/
public function getDepartmentJobsCount($pages) {
$depts = $this->getDepartments();
$cj = $this->setCounts($depts);
$cj = $this->setAttributeCounts($cj, $pages,'job_department');
return $cj;
}
/**
* GetDepartments
* Return all available Departments
* #return Array(Page)
*/
public function getDepartmentPages(){
$pld = new PageList();
$pld->filterByPath('/working-lv'); //the path that your Teachings all sit under
$pld->setItemsPerPage(0);
$res = $this->getPage();
$depts = array();
foreach($res as $jp){
$depts[$jp->getCollectionName()] = $jp;
}
ksort($depts);
return $depts;
}
/**
* PopulateCounts
* Returns array of page names and counts
* #param Array - Array to feed from
* #return Array
*/
public function setCounts($v){
foreach($v as $w){
$a[$w]['count'] = 0;
}
return $a;
}
/**
* PopulateCounts
* Returns array of page names, with counts added from attribute, and paths
* #param Array - Array to add counts and paths in to
* #param Array(Pages) - Pages to run through
* #param String - Attribute to also add to counts
* #param String - Optional - Job Search parameter, leave blank to get Page URL
* #return Array
*/
public function setAttributeCounts($cj, $pages, $attr){
foreach($pages as $p) {
$pLoc = explode('|',$p->getAttribute($attr)); // Our pages could have multiple departments pipe separated
foreach($pLoc as $locName){
$cj[$locName]['count']++;
}
}
return $cj;
}
You can then do the following from a PageList template
$jh = Loader::helper('job');
$deptCounts = $jh->getDepartmentJobsCount($pages);
foreach($deptCounts as $dept => $data) {
echo $dept . '(' . $data['count] . ')';
}