ColdFusion and loading message during loading big data - sql

Question

I have a cfm page with several queries (one cfinvoke in a cfloop from a query). When the page is loading, the loading of data take a long time before to be completely displayed.
Data are displayed in an html table thanks to the cfloop. I saw that the display is progressively displayed.
I would like to know how to improve that with a lazy loading (with a loading message or icon).
my script:
<!-------------------------------- GET DATA ON THE MEETING ----------------------------- START --->
<cftry>
<cfquery name="qry_reunion" datasource="#application.datasource#">
SELECT
R.*
FROM REUNION R
LEFT JOIN MODULES_V M on M.MOD_ID = R.I_MODULE_SIC_ID
INNER JOIN APP_GEN_CODE_T B on B.ST_CODE = R.REUN_CNRE_ST_CODE
INNER JOIN APP_GEN_CODE_T D on d.ST_CODE = R.CSRE_ST_CODE
where R.EXRC_NUMREXRC = '#form.year_source#'
and B.ST_CLASS_CODE = 'BUD_CNRE'
and d.ST_CLASS_CODE = 'BUD_CSRE'
<cfif #Searchstatut# is not "">
and R.CSRE_ST_CODE = '#Searchstatut#'
</cfif>
<cfif #Searchnumber# is not "">
and R.REUN_NUMREURO = '#Searchnumber#'
</cfif>
ORDER BY R.REUN_NUMRREUN
</cfquery>
<cfcatch type="database">
<!--- PUT THE VARIABLE errorDBquery TO 1 --> Disable the Save button of the form) --->
<cfset SESSION.errorDBquery = 1>
</cfcatch>
</cftry>
<!-------------------------------- GET DATA ON THE MEETING ------------------------------ END --->
<cfset reunionIdList= "0" />
<cfif isdefined("qry_reunion.recordcount")>
<!--- Create the list of meeting number --->
<cfloop from="1" to="#qry_reunion.recordcount#" index="i">
<!--- create a var with all REUN_NUMRREUN used in this SCRIPT --->
<cfscript>
reunionIdList &= ",";
reunionIdList &= #qry_reunion.REUN_NUMRREUN[i]#;
</cfscript>
</cfloop>
<cfif #qry_reunion.RECORDCOUNT# is 0 AND #searchorg# is not "">
<br><br><cfoutput>#application.ui_lib_Result#</cfoutput>
<cfelse>
<br>
<table class="dataTable">
<thead>
<cfoutput>
<tr>
<th title="#application.ui_lib_numero#">#application.ui_lib_numero#</th>
<th title="#application.ui_lib_organisateur#">#application.ui_lib_organisateur#</th>
<th title="#application.ui_lib_objet#">#application.ui_lib_objet#</th>
<th title="#application.ui_lib_statut#">#application.ui_lib_statut#</th>
<th title="#application.ui_lib_ville#">#application.ui_lib_ville#</th>
<th title=""> </th>
</tr>
</cfoutput>
</thead>
<tbody>
<!-------------------------------- Retrieve info on all unit ------------------------------- START --->
<cfinvoke component="service/meetings" method="getMeetingsUnitLabel" returnvariable="meetingsUnit">
<cfinvokeargument name="meetingsNumber" value="#reunionIdList#">
</cfinvoke>
<cfloop from="1" to="#qry_reunion.recordcount#" index="i">
<cfoutput>
<form action="reunion_status.cfm?mode=generalite_sta" method="post">
<tr>
<td class="cc">#qry_reunion.REUN_NUMREURO[i]#</td>
<td class="cc">
<!-------------------------------- Retrieve info on unit and check that the org_id is still active ------------------------------- START --->
<cfinvoke component="service/units" method="getUnitVersionFromMeetingOrgId" returnvariable="meetingUnitVersion">
<cfinvokeargument name="meetingOrgId" value="#qry_reunion.ORG_ID[i]#">
<cfinvokeargument name="meetingId" value="#qry_reunion.REUN_NUMRREUN[i]#">
<cfinvokeargument name="unitsArray" value="#SerializeJSON(meetingsUnit.VALUES)#">
</cfinvoke>
<cfif StructIsEmpty(meetingUnitVersion) eq false >
<cfif #meetingUnitVersion.VALUES.OBSOLETE# EQ 0>
<cfset unit ="#meetingUnitVersion.VALUES.ORG_CD#"/>
<cfelse>
<cfset unit ="#meetingUnitVersion.VALUES.ORG_CD# (obsolete)" />
</cfif>
#unit#
<cfif #meetingUnitVersion.VALUES.OBSOLETE# EQ 1 AND #meetingUnitVersion.VALUES.IS_ESTAT# eq 1>
<!--- Dsiplay a tooltip when unit is archived --->
<span id="obsoleteMeetingUnit_#i#" style="line-height:30px;">
<img class='my-tooltip' src='pictures/questionmark.gif' style='vertical-align:middle;'>
</span>
<cfset REUN_DATECREA = #DateFormat(qry_reunion.DATECREA[i], "dd/mm/yyyy")# />
<script>
var #toScript(meetingUnitVersion.VALUES.UNIT_VALID_FROM, "unitValidFrom")#;
var #toScript(meetingUnitVersion.VALUES.UNIT_VALID_TO, "unitValidTo")#;
var #toScript(meetingUnitVersion.VALUES.ORG_CD, "unitLabel")#;
var #toScript(REUN_DATECREA, "meetingCreationDate")#;
var #toScript(i, "i")#;
$("##obsoleteMeetingUnit_" + i).tooltipster({
contentAsHTML: true,
interactive: true,
animation: 'grow',
content: $("<span> was " + unitLabel + " <em>(valid from " + unitValidFrom + " to " + unitValidTo +")</em> when the meeting has been created on " + meetingCreationDate +".</span>"),
theme: 'tooltipster-shadow'
});
</script>
</cfif>
<cfelse>
Error
</cfif>
</td>
<td class="cc">#qry_reunion.REUN_OBJET[i]#</td>
<td class="cc">#qry_reunion.NATURE[i]#</td>
<td class="cc">#qry_reunion.VIL_CD[i]#</td>
<td class="cc">
<input type="hidden" name="REUN_NUMRREUN" value="#qry_reunion.REUN_NUMRREUN[i]#">
<input name="#application.ui_lib_chgt_status#" type="submit" value="#application.ui_lib_chgt_status#">
</td>
</tr>
</form>
</cfoutput>
</cfloop>
</tbody>
</table>
</cfif>
</cfif>
Some additional information:
In my DB there is a view of all Units containing current and the history of each unit.
A meeting is linked to one unit but the unit can have several records in the view. And I need to get the correct row of the unit linked to each meeting
I need to display the list of meetings in function of the filters selected in the different drop-down lists.
I create a function getMeetingsUnitLabel for getting data of all units from the list of meetings in parameter. it's allows me to use that for 1 or several meetings.
The second function getUnitVersionFromMeetingOrgId retrieve the correct record on the version of the unit used when the meeting has been created (it used the parameters retrieved from the previous function.
Thank you in advance for your help,
Seb

Answer 1

(Too long for a comment.)
First thing: Read about SQL injection (use <cfqueryparam>) and DOM XSS (use encodeForHtml()). Your page is full of vulnerabilities and it doesn't matter if that's just some intranet/backoffice because security is relevant everywhere.
Second thing: Learn when to use # and when you don't need them.
Finally: Measure the performance of each section. There is getTickCount() to get a rough timing delta for code execution (just don't use it for micro benchmarking). Generally you want to avoid calling heavy functions in a loop (see Big O notation), especially database/IO calls. Retrieve and process data once and then work with the resultset instead of calling subfunctions for each result. You did not tell us what your functions (getMeetingsUnitLabel and getUnitVersionFromMeetingOrgId) actually do, nor how your query performs, so it's hard to recommend anything specific right now.
Also: Consider moving the JavaScript blocks within your loop to a single script block at the end (after the loop). It will be easier to maintain that instead of fiddling around in each table row.

Answer 2

There is a lot of stuff going on here. Let me cover what stands out to me.
Using cfoutput
...
<cfloop from="1" to="#qry_reunion.recordcount#" index="i">
<cfoutput>
...
<tr>
<td class="cc">#qry_reunion.REUN_NUMREURO[i]#</td>
...
Is the the ColdFusion way of getting data out of a DB. It looks like how other languages do it. It should really look like this
...
<cfoutput query="qry_reunion">
...
<tr>
<td class="cc">#REUN_NUMREURO#</td>
...
Some places I have worked at really like scoping query variables. In that case it would be
...
<cfoutput query="qry_reunion">
...
<tr>
<td class="cc">#qry_reunion.REUN_NUMREURO#</td>
...
Nested calls
<cfloop
...
<cfinvoke component="service/units" method="getUnitVersionFromMeetingOrgId" returnvariable="meetingUnitVersion">
...
This is just asking to be slow. I would create an SQL user defined function or a view to do whatever this is. I want to put the data collecting work on the DB because the DB knows more about collecting and grouping and joining data than ColdFusion
Tooltip
<span id="obsoleteMeetingUnit_#i#" style="line-height:30px;">
<img class='my-tooltip' src='pictures/questionmark.gif' style='vertical-align:middle;'>
</span>
...
var #toScript(i, "i")#;
$("##obsoleteMeetingUnit_" + i).tooltipster({
contentAsHTML: true,
interactive: true,
animation: 'grow',
content: $("<span> was " + unitLabel + " <em>(valid from " + unitValidFrom + " to " + unitValidTo +")</em> when the meeting has been created on " + meetingCreationDate +".</span>"),
theme: 'tooltipster-shadow'
});
</script>
All this is way to verbose. Not to mention it is probably real slow to become interactive. If you need a fancy tool tip. The pattern should look like this:
<span class="custom_tooltip"
data-unitlabel = "#EncodeForHTMLAttribute(unitLabel)#"
data-unitvalidfrom = "#EncodeForHTMLAttribute(unitValidFrom)#"
data-unitvalidto = "#EncodeForHTMLAttribute(unitValidTo)#"
data-dtCreate = "#EncodeForHTMLAttribute(meetingCreationDate)#"
></span>
Then hook in some document.ready stuff and have it apply the tooltip stuff after everything has loaded.
Conclusion
The amount of HTML generated is about twice as big as it needs to be. All of this should be much smaller and quicker.