Implementation of pagination in coldfusion - sql

We are trying to implement pagination in coldfusion.So that I need to make changes in sql query to fetch only a perticular number of data and when clicking on next button, next set of data should be queried.
SELECT History.*,User.FirstName,User.LastName
FROM History
LEFT OUTER JOIN User ON History.UserID = User.UserID
WHERE History.UserID = <CFQUERYPARAM VALUE="#UserID#" CFSQLTYPE="CF_SQL_INTEGER">
AND Type IS NOT NULL
AND SubType IS NOT NULL
ORDER BY #OrderBy# #sort#
Can anyone help me to do the same with above given query.
example query :
SELECT * FROM (
SELECT ROW_NUMBER() OVER(ORDER BY SearchID) AS NUMBER,
* FROM search_history
WHERE UserID=111
AND Date >= #DateVal#
) AS TBL
WHERE NUMBER BETWEEN ((#pageNum# - 1) * #MaxRows# + 1) AND (#pageNum# * #MaxRows#)
ORDER BY #OrderBy# #sort#

First, this is an issue with SQL Server and not ColdFusion. You're just generating the query inline with ColdFusion. It would be better if you create this query as a stored procedure, which by nature would perform better than an inline query.
Second, this answer is the basis for the example I'm posting. SQL Server famously has not supported OFFSET and LIMIT, which makes paging dead simple. These have been in MySQL and postgreSql forever.
Third, as of SQL Server 2012, there is now support for OFFSET and FETCH (essentially LIMIT). Here is a good example of how that works.
This example uses SQL #parameters. Ideally you would declare these in a stored procedure, but you can declare and set them in your inline query. Just make sure to use cfqueryparam to protect against SQL Injection attacks.
SELECT *
FROM (
SELECT
h.column1
, h.column2
, u.FirstName
, u.LastName
, SELECT ROW_NUMBER() OVER ( ORDER BY #ORDER_BY #SORT_ORDER ) AS RowNum
FROM
dbo.History h
INNER JOIN
dbo.User u ON h.UserID = u.UserID
WHERE
h.UserID = #USER_ID
AND
h.Type IS NOT NULL
AND
h.SubType IS NOT NULL
) AS HistorySearch
WHERE
RowNum >= #ROW_START
AND
RowNum < #ROW_END
ORDER BY
RowNum
You'll want to calculate the values of ROW_START and ROW_END based on the page number and a page size variable.
Finally, don't use SELECT * in your main select query. It's a bad practice.

Depending on how much data you are returning you can do your query normally and do pagination on the output with cold fusion. And if you can avoid using * in the query that is a good idea.
<cfparam name="StartAt" default="1">
<cfparam name="MaxRows" default="100">
<cfoutput name="QueryName" startrow="#StartAt#" maxrows="#maxRows#" >
The variables MaxRows and StartAt would be set in your form, and passed when the user clicks Next or previous.
If you have a very large result set then limiting the data returned would be a better idea as you have put forward.

Related

SQL injection mid query

I would like to improve my knowledge about the possible SQL injection attacks that exist. I know that parameterization completely avoids SQL injection risk and should therefore be applied everywhere. However, when someone asks me how it can be exploited, I like to have an answer.
I know how a basic SQL injection attack works. For example a website has a page website.com/users/{id} where id is the primary key of the user. If we trust the input completely and simply pass the id parameter to the query being executed, this can have dire consequences. In the case of website.com/users/1 the query becomes SELECT * FROM [User] WHERE [Id] = 1. However, in the case of website.com/users/1;DROP TABLE User the query becomes SELECT * FROM [User] WHERE [Id] = 1;DROP TABLE User, leading to the nasty result.
But, pretty much all SQL injection attacks I read about count on the WHERE clause being present right before the injection. Almost always, the injection works in some form of ;Injected statement--.
My question is, if it is also possible to perform a SQL injection attack given a query like the one below? Or in a broader sense: does the entire statement have to compile for a SQL injection attack to be possible, or will any error in the statement cause the attack to fail? If the answer is different per DBMS, please specify the DBMS.
In the query below, the injection is supposed to happen in the CHARINDEX('input', [Name]) > 0 where input is copied from a user's input.
SELECT
*
FROM (
SELECT TOP 10
*
FROM
[User]
WHERE
CHARINDEX('input', [Name]) > 0
) AS [User]
LEFT JOIN
[Setting] ON [Setting].[UserId] = [User].[Id]
The furthest I got myself was with the query below, but the error it returns, Missing end comment mark '*/', seems to be completely blocking any attack.
SELECT
*
FROM (
SELECT TOP 10
*
FROM
[User]
WHERE
CHARINDEX('input', '') > 0) AS [User];DROP TABLE [NonExistentTable]/*, [Name]) > 0
) AS [User]
LEFT JOIN
[Setting] ON [Setting].[UserId] = [User].[Id]
The resulting SQL has to be accepted by the particular DBMS for injection to occur, which generally means it needs to be valid SQL, but there are usually ways of crafting the input to make it valid regardless of the SQL in question.
If a line comment isn't enough, an extra statement can be added; if multiple statements aren't allowed, a UNION can be used; and so on.
The exact details vary, but with enough knowledge of the query (e.g. through error details leaking to the user) or lucky guesses, something can usually be crafted that is to the attacker's advantage.
In your example, consider this input, which simply repeats parts of the existing query:
nonsense', [Name]) > 0
)
) AS [User];
Drop Table [User];
SELECT
*
FROM (
SELECT TOP 10
*
FROM
[User]
WHERE
CHARINDEX('nonsense
Which results in the following SQL:
SELECT
*
FROM (
SELECT TOP 10
*
FROM
[User]
WHERE
CHARINDEX('nonsense', [Name]
)
) AS [User];
Drop Table [User];
SELECT
*
FROM (
SELECT TOP 10
*
FROM
[User]
WHERE
CHARINDEX('nonsense', [Name]) > 0
) AS [User]
LEFT JOIN
[Setting] ON [Setting].[UserId] = [User].[Id]
SQL injection normally happens where some kind of string concatenation/insertion operation is involved. It does not have to be the WHERE clause. Also, generally speaking, the attacker is not interested in dropping the tables, he wants information. What if input is replaced by this:
', '') > 0 UNION ALL SELECT TABLE_NAME, COLUMN_NAME FROM INFORMATION_SCHEMA.COLUMNS WHERE COLUMN_NAME = 'password' --
Assuming that the result from select are displayed somehow and error messages are also shown, it'll take a few minutes for the attacker to determine the number and position of , NULL he should add before the query actually returns the name of table and column he wants to probe in the next stage.

Is there a better way to write this gross SQL?

So I'm creating a query for a report that could have several optional filters. I've only included client and station here to keep it simple. Each of these options could be an include or an exclude and could contain NULL, 1, or multiple values. So I split the varchar into a table before joining it to the query.
This test takes about 15 minutes to execute, which... just won't do :p Is there a better way? We have similar queries written with dynamic sql, and I was trying to avoid that, but maybe there's no way around it for this?
DECLARE
#ClientsInc VARCHAR(10) = 'ABCD, EFGH',
#ClientsExc VARCHAR(10) = NULL,
#StationsInc VARCHAR(10) = NULL,
#StationsExc VARCHAR(10) = 'SomeStation'
SELECT *
INTO #ClientsInc
FROM dbo.StringSplit(#ClientsInc, ',')
SELECT *
INTO #ClientsExc
FROM dbo.StringSplit(#ClientsExc, ',')
SELECT *
INTO #StationsInc
FROM dbo.StringSplit(#StationsInc, ',')
SELECT *
INTO #StationsExc
FROM dbo.StringSplit(#StationsExc, ',')
SELECT [some stuff]
FROM media_order mo
LEFT JOIN #ClientsInc cInc WITH(NOLOCK) ON cInc.Value = mo.client_code
LEFT JOIN #ClientsExc cExc WITH(NOLOCK) ON cExc.Value = mo.client_code
LEFT JOIN #StationsInc sInc WITH(NOLOCK) ON sInc.Value = mo.station_name
LEFT JOIN #StationsExc sExc WITH(NOLOCK) ON sExc.Value = mo.station_name
WHERE ((#ClientsInc IS NOT NULL AND cInc.Value IS NOT NULL)
OR (#ClientsExc IS NOT NULL AND cExc.Value IS NULL)
)
AND ((#StationsInc IS NOT NULL AND sInc.Value IS NOT NULL)
OR (#StationsExc IS NOT NULL AND sExc.Value IS NULL)
)
First of all, I always tend to mention Erland Sommarskog's Dynamic Search Conditions in such cases.
However, you already seem to be aware of the two options: one is dynamic SQL. The other is usually the old trick and (#var is null or #var=respective_column). This trick, however, works only for one value per variable.
Your solution indeed seems to work for multiple values. But in my opinion, you are trying too hard to avoid dynamic sql. Your requirements are complex enough to guarantee it. And remember, usually, dynamic sql is harder for you to code, but easier for the server in complex cases - and this one certainly is. Making a performance guess is always risky, but I would guess an improvement in this case.
I would use exists and not exists:
select ...
from media_order mo
where
(
#ClientsInc is null
or exists (
select 1
from string_split(#ClientsInc, ',')
where value = mo.client_code
)
)
and not exist (
select 1
from string_split(#ClientsExc, ',')
where value = mo.client_code
)
and (
#StationsInc is null
or exists (
select 1
from string_split(#StationsInc, ',')
where value = mo.station_name
)
)
and not exist (
select 1
from string_split(#StationsExc, ',')
where value = mo.station_name
)
Notes:
I used buil-in function string_split() rather than the custom splitter that you seem to be using. It is available in SQL Server 2016 and higher, and returns a single column called value. You can change that back to your customer function if you are running an earlier version
as I understand the logic you want, "include" parameters need to be checked for nullness before using exists, while it is unnecessary for "exclude" variables

selecting minimum value depending on other value

Is there any better way for below sql query? Don't want to drop and create temporary table just would like to do it in 1 query.
I am trying to select minimum value for price depending if its order sell where obviously price is higher then in buy and it just shows 0 results when I try it.
DROP TABLE `#temporary_table`;
CREATE TABLE `#temporary_table` (ID int(11),region int(11),nazwa varchar(100),price float,isBuyOrder int,volumeRemain int,locationID int,locationName varchar(100),systemID int,security_status decimal(1,1));
INSERT INTO `#temporary_table` SELECT * FROM hauling WHERE isBuyOrder=0 ORDER BY ID;
SELECT * FROM `#temporary_table`;
SELECT * FROM `#temporary_table` WHERE (ID,price) IN (select ID, MIN(price) from `#temporary_table` group by ID) order by `ID`
UPDATE: when I try nvogel answer and checked profiling thats what I get:
Any chance to optimize this or different working way with 700k rows database?
Try this:
SELECT *
FROM hauling AS h
WHERE isBuyOrder = 0
AND price =
(SELECT MIN(t.price)
FROM hauling AS t
WHERE t.isBuyOrder = 0
AND t.ID = h.ID);
You don't need a temporary table at all. You can basically use your current logic:
SELECT h.*
FROM hauling h
WHERE h.isBuyOrder = 0 AND
(h.id, h.price) IN (SELECT h2.id, MIN(h2.price)
FROM hauling h2
WHERE h2.isBuyOrder = 0
)
ORDER BY h.id
There are many other ways to write similar logic. However, there is no need to rewrite the logic; you just need to include the comparison on isBuyOrder in the subquery.
Note that not all databases support IN with tuples. If your database does not provide such support, then you would need to rewrite the logic.

Sql Server IN Clause Issue

Writing a stored procedure that will have multiple input parameters. The parameters may not always have values and could be empty. But since the possibility exists that each parameter may contain values I have to include the criterion that utilizing those parameters in the query.
My query looks something like this:
SELECT DISTINCT COUNT(*) AS SRM
FROM table p
WHERE p.gender IN (SELECT * FROM Fn_SplitParms(#gender)) AND
p.ethnicity IN (SELECT * FROM Fn_SplitParms(#race)) AND
p.marital_status IN (SELECT * FROM Fn_SplitParms(#maritalstatus))
So my problem is if #gender is empty(' ') the query will return data where gender field is empty when I really want to just ignore p.gender all together. I don't want to have to accomplish this task using IF/ELSE conditional statements because they would be too numerous.
Is there any way to use CASE with IN for this scenario? OR
Is there other logic that I'm just not comprehending that will solve this?
Having trouble finding something that works well...
Thanks!
Use or:
SELECT DISTINCT COUNT(*) AS SRM
FROM table p
WHERE
(p.gender IN (SELECT * FROM Fn_SplitParms(#gender)) OR #gender = '')
AND (p.ethnicity IN (SELECT * FROM Fn_SplitParms(#race)) OR #race = '')
AND (p.marital_status IN (SELECT * FROM Fn_SplitParms(#maritalstatus)) OR #maritalstatus = '')
You might also want to consider table-valued parameters (if using SQL Server 2008 and up) - these can sometimes make the code simpler, since they are treated as tables (which in your case, may be empty) and you can join - plus no awkward split function required.

sql server-query optimization with many columns

we have "Profile" table with over 60 columns like (Id, fname, lname, gender, profilestate, city, state, degree, ...).
users search other peopel on website. query is like :
WITH TempResult as (
select ROW_NUMBER() OVER(ORDER BY #sortColumn DESC) as RowNum, profile.id from Profile
where
(#a is null or a = #a) and
(#b is null or b = #b) and
...(over 60 column)
)
SELECT profile.* FROM TempResult join profile on TempResult.id = profile.id
WHERE
(RowNum >= #FirstRow)
AND
(RowNum <= #LastRow)
sql server by default use clustered index for execution query. but total execution time is over 300. we test another solution such as multi column index in all columns in where clause but total execution time is over 400.
do you have any solution to make total execution time lower than 100.
we using sql server 2008.
Unfortunately I don't think there is a pure SQL solution to your issue. Here are a couple alternatives:
Dynamic SQL - build up a query that only includes WHERE clause statements for values that are actually provided. Assuming the average search actually only fills in 2-3 fields, indexes could be added and utilized.
Full Text Search - go to something more like a Google keyword search. No individual options.
Lucene (or something else) - Search outside of SQL; This is a fairly significant change though.
One other option that I just remembered implementing in a system once. Create a vertical table that includes all of the data you are searching on and build up a query for it. This is easiest to do with dynamic SQL, but could be done using Table Value Parameters or a temp table in a pinch.
The idea is to make a table that looks something like this:
Profile ID
Attribute Name
Attribute Value
The table should have a unique index on (Profile ID, Attribute Name) (unique to make the search work properly, index will make it perform well).
In this table you'd have rows of data like:
(1, 'city', 'grand rapids')
(1, 'state', 'MI')
(2, 'city', 'detroit')
(2, 'state', 'MI')
Then your SQL will be something like:
SELECT *
FROM Profile
JOIN (
SELECT ProfileID
FROM ProfileAttributes
WHERE (AttributeName = 'city' AND AttributeValue = 'grand rapids')
AND (AttributeName = 'state' AND AttributeValue = 'MI')
GROUP BY ProfileID
HAVING COUNT(*) = 2
) SelectedProfiles ON Profile.ProfileID = SelectedProfiles.ProfileID
... -- Add your paging here
Like I said, you could use a temp table that has attribute name/values:
SELECT *
FROM Profile
JOIN (
SELECT ProfileID
FROM ProfileAttributes
JOIN PassedInAttributeTable ON ProfileAttributes.AttributeName = PassedInAttributeTable.AttributeName
AND ProfileAttributes.AttributeValue = PassedInAttributeTable.AttributeValue
GROUP BY ProfileID
HAVING COUNT(*) = CountOfRowsInPassedInAttributeTable -- calculate or pass in
) SelectedProfiles ON Profile.ProfileID = SelectedProfiles.ProfileID
... -- Add your paging here
As I recall, this ended up performing very well, even on fairly complicated queries (though I think we only had 12 or so columns).
As a single query, I can't think of a clever way of optimising this.
Provided that each column's check is highly selective, however, the following (very long winded) code, might prove faster, assuming each individual column has it's own separate index...
WITH
filter AS (
SELECT
[a].*
FROM
(SELECT * FROM Profile WHERE #a IS NULL OR a = #a) AS [a]
INNER JOIN
(SELECT id FROM Profile WHERE b = #b UNION ALL SELECT NULL WHERE #b IS NULL) AS [b]
ON ([a].id = [b].id) OR ([b].id IS NULL)
INNER JOIN
(SELECT id FROM Profile WHERE c = #c UNION ALL SELECT NULL WHERE #c IS NULL) AS [c]
ON ([a].id = [c].id) OR ([c].id IS NULL)
.
.
.
INNER JOIN
(SELECT id FROM Profile WHERE zz = #zz UNION ALL SELECT NULL WHERE #zz IS NULL) AS [zz]
ON ([a].id = [zz].id) OR ([zz].id IS NULL)
)
, TempResult as (
SELECT
ROW_NUMBER() OVER(ORDER BY #sortColumn DESC) as RowNum,
[filter].*
FROM
[filter]
)
SELECT
*
FROM
TempResult
WHERE
(RowNum >= #FirstRow)
AND (RowNum <= #LastRow)
EDIT
Also, thinking about it, you may even get the same result just by having the 60 individual indexes. SQL Server can do INDEX MERGING...
You've several issues imho. One is that you're going to end up with a seq scan no matter what you do.
But I think your more crucial issue here is that you've an unnecessary join:
SELECT profile.* FROM TempResult
WHERE
(RowNum >= #FirstRow)
AND
(RowNum <= #LastRow)
This is a classic "SQL Filter" query problem. I've found that the typical approaches of "(#b is null or b = #b)" & it's common derivatives all yeild mediocre performance. The OR clause tends to be the cause.
Over the years I've done a lot of Perf/Tuning & Query Optimisation. The Approach I've found best is to generate Dynamic SQL inside a Stored Proc. Most times you also need to add "with Recompile" on the statement. The Stored Proc helps reduce potential for SQL injection attacks. The Recompile is needed to force the selection of indexes appropriate to the parameters you are searching on.
Generally it is at least an order of magnitude faster.
I agree you should also look at points mentioned above like :-
If you commonly only refer to a small subset of the columns you could create non-clustered "Covering" indexes.
Highly selective (ie:those with many unique values) columns will work best if they are the lead colum in the index.
If many colums have a very small number of values, consider using The BIT datatype. OR Create your own BITMASKED BIGINT to represent many colums ie: a form of "Enumerated datatyle". But be careful as any function in the WHERE clause (like MOD or bitwise AND/OR) will prevent the optimiser from choosing an index. It works best if you know the value for each & can combine them to use an equality or range query.
While often good to find RoWID's with a small query & then join to get all the other columns you want to retrieve. (As you are doing above) This approach can sometimes backfire. If the 1st part of the query does a Clustred Index Scan then often it is faster to get the otehr columns you need in the select list & savethe 2nd table scan.
So always good to try it both ways & see what works best.
Remember to run SET STATISTICS IO ON & SET STATISTICS TIME ON. Before running your tests. Then you can see where the IO is & it may help you with index selection, for the mose frequent combination of paramaters.
I hope this makes sense without long code samples. (it is on my other machine)