Loop through variable - sql

I need to loop through variable and write returned values into DB.
Here is my code so far:
<cffunction name="csv">
<cffile action="read" file="test.csv" variable="csvfile" charset="utf-8">
<cfset csv_query = app.csvToQuery.CSVToQuery(CSV = csvFile.Trim()) />
<cfloop query="csv_query">
<cfset getInfo = app.directory.searchAll(
findNo = "#replace(csv_query.column_1, "test", "", 'all')#",
findBy = "Null"
)>
<!--- <cfdump var="#getInfo#"> --->
<cfoutput><cfdump var="#getInfo.NAME#"></cfoutput>
<cftry>
<cfquery datasource="#app.dsn#">
INSERT INTO WRITEINFO (
Name,
) VALUES (
'#val(getInfo.NAME)#',
)
</cfquery>
<cfcatch type="database">
</cfcatch>
</cftry>
</cfloop>
</cffunction>
It writes into DB, but it only writes the first value it gets. I am already looping through query, and when I dumped the values(<cfdump var="#getInfo.NAME#">), I can see all the values I needed, but they're just not inserted into DB

It looks as if this fails silently. You catch database errors but don't do anything in case of an error. One insert statement works. On other inserts it might fail (e.g. unique index violation, "Name" inserted to long for the column in the table, etc.). Maybe get rid of the try/catch block (at least temporarily) and see what happens.

Related

ColdFusion count the amount without using SQL

I am trying to count the amount without using SQL count() code
Is there a way to extract the results including the number amount over ColdFusion?
Select test
From ......
Where .....
<Cfif not test.RecordCount>
No records found
</Cfif>
<cfoutput query="test">#amount#</cfoutput>
It should display like this
TEST CountOutput
TEST A: 22
TEST B: 32
TEST B: 1
TEST C: 23
(From comments ...)
Why can't you use SQL for this? Unless there's a valid reason you can't use COUNT, it's by far the simplest way to aggregate. You don't need to know the specific values in the database table in order to perform a COUNT. Just do:
SELECT column, count(*) AS someAlias
FROM tableName
GROUP BY column
Yes, it is possible to do the same in ColdFusion code, but again unless there's a specific reason to do so, it's more efficient to use SQL.
<cfset data = {} />
<cfset names = {} />
<cfloop query="test">
<cfif not structKeyExists(data, test.name)>
<cfset data[test.name] = 0 />
<!--- if you need to preserve original case of name --->
<cfset names[test.name] = test.name />
</cfif>
<cfset data[test.name] += 1 />
</cfloop>
<cfoutput>
<cfloop collection="#data#" item="key">
#names[key]#: #data[key]#<br />
</cfloop>
</cfloop>
To get the total count without using SQL, Use
QUERYVARIABLE.RECORDCOUNT
Based on the condition used in the SQL query it will give you the count.
FOR Example,
<cfquery name="test">
SELECT * FROM database WHERE name=testA
</cfquery>
<cfdump var="#test.RECORDCOUNT#" />
Use this within a function, call it and store the return value in a stack or an array.
I strongly recommend you use a SQL Count(). I cannot imagine why you would want to do it inside a CFLoop or CFOutput, but it is possible if you use the Group attribute, and count each row that way.
Did you know that after you execute CFQuery against your SQL server to get all those detail records, you can run a "query of queries" summarize the SQL data. This article seems to explain it:
https://www.quackit.com/coldfusion/tutorial/coldfusion_query_of_queries.cfm

Build struct and then populate with the values from query?

I have task to update/insert some fields in two different tables. Before I run my update I have to grab values from my query and put them in structure where one of my values should be the key. My query looks like this:
<cfquery name="getRecords" datasource="test">
Select
s.ID
,f.USER_ID
,s.USER_NUMBER
,s.STATUS
,f.DINING
From USERS s
Left Outer Join FIELDS f ON s.ID = f.USER_ID
</cfquery>
I need USER_NUMBER to use as a key in my structure and store all other values from the query above. I will use this structure to compare values from my other list and then build final list that I will use for update/insert. I tried something like the code below, but it did not work:
Here is a stand alone example using a manual query:
<cfset getRecords = queryNew("")>
<cfset queryAddColumn(getRecords, "ID", [1,2,3])>
<cfset queryAddColumn(getRecords, "USER_ID", ["userA","userB","userC"])>
<cfset queryAddColumn(getRecords, "STATUS", ["Active","Active","Active"])>
<cfset queryAddColumn(getRecords, "DINING", ["X","Y","Z"])>
<cfset myStruct = StructNew()>
<cfloop query="getRecords">
<cfset myStruct = [key:#USER_NUMBER#{
id:#ID#
,userid:##USER_ID
,status:#STATUS#
,dining:#DINING#
}]>
</cfloop>
If anyone can help with this code please let me know. I usually use arrays but this time I have to use struct because of some other reasons. Thank you.
I believe you just need to move your key up a level so that the loop doesn't overwrite the values. so try something like:
<cfset myStruct = StructNew()>
<cfloop query="getRecords">
<cfset myStruct[getRecords.USER_ID] = {
id:getRecords.ID,
userid:getRecords.USER_ID,
status:getRecords.STATUS,
dining:getRecords.DINING
}>
</cfloop>
Then to access the variables you can use something like:
<cfoutput>#htmlEditFormat(myStruct[1].dining)#</cfoutput>

reading Excel (93-97) sheet with more than 65536 rows using cfspreadsheet

Hi I have to deal with data that is more than 65536 rows. and hence it comes into 2 different Excel Sheets, named as "Details" and "Details_1".
Basically whats happening is uploading Excel sheets, and using "cfspreadsheet" to "read" this data.
Once read this is inserted into SQL table.
I am using a component-function to read these 1/2 sheets. The idea is cfif Query recordcount () from "cfspreadsheet" is more than 65533, then read the second sheet too. Then use QoQ and UNION ALL to create a combined query.(Most of the cases there are just 1 sheet but in some cases it is more than 2 sheets.)
It worked fine for till for some time. then all of a sudden it stopped working. I am not sure about wrong/error had crept into it that is causing it to stop. the following is my code
<cftry>
<cfset fileEXCL = "#ExpandPath('../folder')#/#arguments.xclfile#" />
<!---when there e 2 Sheets --->
<!---get info from sheet1 as a "query1"--->
<cfspreadsheet action="read" src="#fileEXCL#" sheet="1" query="Query1" headerrow="1" />
<!--- recordcount for "sheet1" as "count1"--->
<cfset count1 =#Query1.recordcount#>
<!--- case when excel has more than 65533 rows
;THIS IMPLIES THAT THERE 2 SHEETS)--->
<cfif count1 gt 65533>
<!--- take info from sheet 2 as a "query2" and count as "count2"--->
<cfspreadsheet action="read" src="#fileEXCL#" sheet="2" query="Query2" headerrow="1" />
<cfset count2 =#Query2.recordcount#>
<!---club both query's using QoQ and call it "excelQuery"--->
<cfquery dbtype="query" name="excelQuery">
SELECT * FROM Query1
UNION ALL
SELECT * FROM Query2
</cfquery>
<!---total record count for "sheet1" & "sheet2"--->
<cfset rowCount =#excelQuery.recordcount#>
<cfelse>
<!---this case there is just 1 query "Query1" ;rename it "excelQuery"--->
<cfquery dbtype="query" name="excelQuery">
SELECT * FROM Query1
</cfquery>
<!--- recordcount for "sheet1"--->
<cfset rowCount =#excelQuery.recordcount#>
</cfif>
<cflog file="Collections" application="yes" text="#Session.user_info.uname# logged in. Data file #fileEXCL# read. Recordcount:#rowCount#" type="Information">
<cfset ins =insertUserLog("#Session.user_name#","#Session.user_code#","file #fileEXCL# read. ","Recordcount:#rowCount#","")>
<cfcatch type="any" >
<cflog file="Collections" application="yes" text="Error in reading Data file #fileEXCL#." type="Error">
<cfset ins =insertUserLog("#Session.user_name#","#Session.user_code#","error file","failed","#cfcatch.Message#")>
<cfreturn 1>
</cfcatch>
</cftry>
**
I have done the following:-
a) tried to dump individual Query's Query1 and Query2 of each sheet !
Still if its more than 65536 rows the IE page hangs up
Unable to read both the sheets 1 and 2.
b) I have placed t he error handling to catch specific Errors such as "Database"
c) When i reduce the number of rows below 65536 or remove the Sheet with larger number of Rows it works.
As i said earlier it was a code that was working and went kaput all of a sudden.
**
Perhaps the issue is with JVM's heap size? You might try increasing the maximum heap size, if your environment can handle it.

fixing "more columns in insert than values specified"

A user got this error
DIAGNOSTICS: Error Executing Database Query. [Macromedia][SQLServer
JDBC Driver][SQLServer]There are more columns in the INSERT statement
than values specified in the VALUES clause. The number of values in
the VALUES clause must match the number of columns specified in the
INSERT statement. The error occurred on line 252. MESSAGE: Error
Executing Database Query. ROOT CAUSE:
coldfusion.tagext.sql.QueryTag$DatabaseQueryException: Error Executing
Database Query. Server: server2
There should be around 60 values/columns. Possible solution is adding <CFIF isDefined("VARIABLES.xxx")> to each of the values in INSERT and VALUES but this would take a long time. Is there a better way? Here is some of the code
<CFQUERY name="addAgency" datasource="#REQUEST.dsn#">
INSERT INTO agency (agency_name, code, address1, address2, city_id, postal, phone, alternativePhone, fax, provincialRegistration,
<!--- contact, ---> firstname, lastname, username, password, relationship_id, editdate, adddate, email, URL_link,Airport_id,
header, teaser, full_desc, <CFIF isDefined("VARIABLES.specialty")>specialty,</CFIF> hours_1, hours_2, hours_3, hours_4,
hours_5, hours_6, hours_7, merchant_ID,
region_ID, fcApproved, agencyType_ID, agencystatus, crossview, insp_Approved, rbc_Approved,
branch_number, gp_entity_ID,
<CFIF VARIABLES.alias1 gt 100> alias1,</CFIF>
<CFIF VARIABLES.alias2 gt 100> alias2,</CFIF>
<CFIF Trim(VARIABLES.alias3) NEQ ""> alias3,</CFIF>
default_agent_id,LOCAL_PASSWORD,LOCAL_USERNAME,NATIONAL_PASSWORD,NATIONAL_USERNAME,VACATIONCLUB_PASSWORD,VACATIONCLUB_USERNAME,toll_free_number,display_toll_free ,MARKETINGREGIONID, searchPreference, bookingOption,customer_can_choose
<CFIF isDefined("variables.logo_filedata")>,logo_fileName,logo_fileData,logo_mimeType</CFIF>
,gds, pseudo_city, acv_affiliate_code, vip_ext,owner_manager_title_en,owner_manager_title_fr)
VALUES (<CFQUERYPARAM value="#Trim(sanitize(VARIABLES.agency_name))#" cfsqltype = "cf_sql_varchar" maxlength="150">,
<CFQUERYPARAM value="#Trim(sanitize(VARIABLES.code))#" cfsqltype = "cf_sql_varchar" maxlength="50">,
<CFQUERYPARAM value="#Trim(sanitize(VARIABLES.address1))#" cfsqltype = "cf_sql_varchar" maxlength="255">,
<CFQUERYPARAM value="#Trim(sanitize(VARIABLES.address2))#" cfsqltype = "cf_sql_varchar" maxlength="255">,
<CFQUERYPARAM value="#sanitize(VARIABLES.city_id,true,true,false,true,false)#" cfsqltype = "cf_sql_integer">,
<CFQUERYPARAM value="#Trim(sanitize(VARIABLES.postal))#" cfsqltype = "cf_sql_varchar" maxlength="10">,
<CFQUERYPARAM value="#Trim(sanitize(VARIABLES.phone))#" cfsqltype = "cf_sql_varchar" maxlength="20">,
<CFQUERYPARAM value="#Trim(sanitize(VARIABLES.fax))#" cfsqltype = "cf_sql_varchar" maxlength="20">,
<!--- <CFIF IsDefined("VARIABLES.contact")>#Trim(VARIABLES.contact)#, <CFELSE> ' ', </CFIF> --->
<CFQUERYPARAM value="#Trim(sanitize(VARIABLES.firstname))#" cfsqltype = "cf_sql_varchar" maxlength="40">,
<CFQUERYPARAM value="#Trim(sanitize(VARIABLES.lastname))#" cfsqltype = "cf_sql_varchar" maxlength="40">,
'-', '-',
<!--- Dummy un/pw --->
<CFQUERYPARAM value="#sanitize(VARIABLES.relationship_ID,true,true,false,true,false)#" cfsqltype = "cf_sql_integer">,
#CreateODBCDateTime(Now())#,
#CreateODBCDateTime(Now())#,
<CFQUERYPARAM value="#Trim(sanitize(VARIABLES.email))#" cfsqltype = "cf_sql_varchar" maxlength="40">,
Each of your s in the "columns" area MUST be matched with corollary CFIF in the "values" section as long as you are doing it this way. While there are other ways of doing it, they are not necessarily less messy. You could, for example:
set defaults for all columns with some cfparams so you did not have to have any "cfifs"
Write multiple "cleaner" queries
Move the whole thing to a stored procedure and pass what you have into it - the SP would then worry about the logic.
Save a small subset of values then "update" based on your cfif logic (don't like that one!).
As you can see they all have a downside.
Elaboration on 1)
Instead of doing something arduous like:
<CFIF isDefined("VARIABLES.specialty")> Specialty,</cfif>
Before you start your query make sure you have a value for all the variables you need using cfparam as in:
<Cfparam name="Specialty" default=""/>
(of course the default for specialty might be 0 or whatever).
The cfparam tag checks to see IF the variable exist and if it does not exist it creates it with the default you specify. This would mean you can "guarantee" that the variable will indeed exist - meaning you can eliminate the CFIF statement in your query.
One cavaet has to do with NULLs in the database. If you are depending on nulls (meaning if specialty does not exist you wish for the column to remain null), then you will need to add the "null" attribute to your cfqueryparam as in:
<CFQUERYPARAM
value="#Trim(sanitize(VARIABLES.specialty))#"
cfsqltype = "cf_sql_varchar"
null="#mynullfunction(variables.specialty)#"/>,
Note the "mynullfunction" - I ususally create a utility UDF for this that simply returns YES if the value is blank or zero (or whatever I define) and "no" if it's populated.
The best solution is to use the null attribute of cfqueryparam for columns where a default has not be set in the database, this means you can avoid adding conditions to the INSERT INTO and keeps the VALUE to one line per column. When a default does exist for a column then you need to use the long winded condition. This is the simplest, least obtrusive change.
Example where Alias1 has a database default, speciality does not.
` <cfset Variables.IsValidAlias1 = Variables.alias1 gt 100>
<cfquery name="addAgency" datasource="#REQUEST.dsn#">
INSERT INTO agency (agency_name,
specialty
<cfif Variables.IsValidAlias1>
, Alias1
</cfif>)
VALUES (<cfqueryparam value="#Variables.agency_name#" cfsqltype="cf_sql_varchar" maxlength="150">,
<cfqueryparam value="#Variables.specialty#" cfsqltype="cf_sql_varchar" null="#StructKeyExists(Variables,'Specialty') EQ false#" maxlength="199">
<cfif Variables.IsValidAlias1>
,<cfqueryparam value="#Variables.Alias1#" cfsqltype="cf_sql_varchar" maxlength="199">
</cfif>)
</cfquery>`
As an aside you should use StructKeyExists() instead of IsDefined as the former is faster and can be clearer to read. You should put this code inside a function, and that function inside a component so it can be re-used. This is assuming your using CF6 or later.
I would avoid setting empty string defaults for each field or setting values to NULL for non-existent keys. The reason is that you may now (or decide to have in the future) default values in your database columns. As the resulting NULL values would be explicit, they would override your database defaults. If you had passed in an empty string, this might be valid but as a default it likely is not.
My main advice would be to be keep each column on one line so that you easily see that you are being consistent in your conditionals. As Mark said, you must be careful to have the exact same conditionals in the "columns" area as the "values" area of your query.
Somewhat as an aside, I have a free tool called DataMgr that you could use to manage all of this sort of thing for most simple queries.
http://www.bryantwebconsulting.com/docs/datamgr/replace-sql-inserts-and-updates.cfm
http://www.bryantwebconsulting.com/docs/datamgr/getting-started.cfm
The main thing, however, is to be consistent in your conditionals and to format your query in such a way as to make it obvious when you are not.

SQL Array being returned as number, not String in ColdFusion

In Coldfusion, I'm using a cfc that binds one select box to another (basically, choose a State from one box, the second box is populated with County names.) The value for the County box is a 5-digit number WHICH IS FORMATTED AS TEXT (ie. the value comes from a text field.)
The problem is that I'm finding that if the value of the selected county id starts with a '0', it's been cut off.
So I get stuff like:
ID County
11223 A
2300 B (should be 02300)
Can someone help make sure that leading 0s are not cut off?
Here's the select boxes on the page:
<!--- State Name options --->
<b>State:</b><br />
<cfselect bind="cfc:states.getStates()" bindonload="true" name="search_state" id="search_state" value="StateUSAbb" display="StateName">
</cfselect><br />
<!--- County Name options --->
<b>County:</b><br />
<cfselect bind="cfc:states.getCounties({search_state})" name="search_county" id="search_county" value="FIPS_County" display="CountyName">
</cfselect>
I hate pasting the whole .cfc but pay attention to the latter part, particularly the cfloop which uses a cfset to populate array RESULT:
<cfcomponent output="false">
<!--- Get array of media types --->
<cffunction name="getStates" access="remote" returnType="array">
<!--- Define variables --->
<cfset var data="">
<cfset var result=ArrayNew(2)>
<cfset var i=0>
<!--- Get data --->
<cfquery name="data" datasource="bridges">
SELECT DISTINCT tblLoc.StateUSAbb, lkuState.StateName
FROM lkuState INNER JOIN tblLoc ON lkuState.FIPS_State = tblLoc.FIPS_State
WHERE (lkuState.StateName <> 'New Brunswick')
UNION
SELECT '' AS StateUSAbb, ' ALL' AS StateName
FROM lkuState
ORDER BY StateName
</cfquery>
<!--- Convert results to array --->
<cfloop index="i" from="1" to="#data.RecordCount#">
<cfset result[i][1]=data.StateUSAbb[i]>
<cfset result[i][2]=data.StateName[i]>
</cfloop>
<!--- And return it --->
<cfreturn result>
</cffunction>
<!--- Get counties by state --->
<cffunction name="getCounties" access="remote" returnType="array">
<cfargument name="stateabb" type="string" required="true">
<!--- Define variables --->
<cfset var data="">
<cfset var result=ArrayNew(2)>
<cfset var i=0>
<!--- Get data --->
<cfquery name="data" datasource="bridges">
SELECT '' AS FIPS_COUNTY, ' ALL' as CountyName
FROM lkuCnty
UNION
SELECT FIPS_County, CountyName
FROM lkuCnty
WHERE StateAbb = '#ARGUMENTS.stateabb#'
ORDER BY CountyName
</cfquery>
<!--- Convert results to array --->
<cfloop index="i" from="1" to="#data.RecordCount#">
<cfset result[i][1]=data.FIPS_County[i]>
<cfset result[i][2]=data.CountyName[i]>
</cfloop>
<!--- And return it --->
<cfreturn result>
</cffunction>
</cfcomponent>
If the data is a fixed length you can use NumberFormat to force leading zero's. In general, CF is typeless so there must be some underlying conversion happening that is causing the data to get corrupt. You might try forcing the value toString(), or to debug add something like a single quote as the first character in the column value (eg. SELECT '''' + FIPS_County, '''' + CountyName FROM lkuCnty) to see if they keep all their characters.
[Update]
Based on your comments about how SQL is not returning 5 char, use this updated query to go from INT to VARCHAR with leading zeros.
SELECT DISTINCT
RIGHT('00000' + CONVERT(VARCHAR(5),StateUSAbb),5),
lkuState.StateName
FROM lkuState INNER JOIN tblLoc ON lkuState.FIPS_State = tblLoc.FIPS_State
WHERE (lkuState.StateName <> 'New Brunswick')
UNION
SELECT '' AS StateUSAbb,
' ALL' AS StateName
FROM lkuState
ORDER BY StateName
append a space to the end of the number, then CF will treat it as a string and no leading 0 will be chopped.
simple workaround: <cfset result[i][1]=data.StateUSAbb[i] & " ">
btw, you know that query object is supported for populating cfselect right? So you don't even need the loop. You can do the same workaround but in SQL inside your cfquery
UPDATE: anyway, the idea is that if u want to preserve the leading 0 and keep using CF's built in serializeJSON() or calling the cfc remote method in JSON style (which will internally invoke serializeJSON(), you can append a space so CF will treat it as a string and leading 0 will be preserved. If your script somehow must need "012345" with no trailing space, then look for another JSON seralizer from riaforge or cflib.
You're sure the data returned from your query is a text string which contains the leading zero, rather than just the integer value? Regardless, I think Zachary's suggestion of NumberFormat(x, "00000") is the way to go.