I have a client talking with server program(using sqlite3 as the storage) which needs to support paging. I am thinking about how to implement that. One approach:
1) user request page 1
a. execute query from sqlite3
b. return the first page range items to client
2) user request page N
a. execute query from sqlite3
b. return the N page range itmes to client
So in my approach every time user requests data I will re issue the query and get the specified elements, which seems waste of time...(comparing with caching all the items one time and just give the portion user asks, but this approach is more complex, as I need to timeout the cache - user is using browser I don't know when the user is loggout and the cache is useless)
Any better ideas?
Maybe you could use the LIMIT… OFFET… capability of SQLite.
Taken from the select syntax documentation:
Instead of a separate OFFSET clause, the LIMIT clause may specify two scalar expressions separated by a comma. In this case, the first expression is used as the OFFSET expression and the second as the LIMIT expression. This is counter-intuitive, as when using the OFFSET clause the second of the two expressions is the OFFSET and the first the LIMIT. This is intentional - it maximizes compatibility with other SQL database systems.
Related
There is a query like:
select * from tablename where username='value1' and password='value2';
If I set to the fields the following:
username ='admin' and password ='admin';
Then I sign in into the website as administrator.
Now, If I wanted to SQL inject my query, I would enter to the username field the value 'or 1=1, after which the query would be executed like:
select * from tablename where username ='' or 1=1
Assuming everything after this the query is executed successfully.
My question is based on above example, what user we will be logged in as?
As:
1. Admin
2. Or first row in table?
3. Or any other user and how?
That is just a SQL query, it does not login or do any other application functionality. What is done with the data retrieved is entirely dependent on the specific application.
The code may happily consume the first row in the resulting recordset and assume that is the user to be logged in. It may also throw an exception, e.g., if the query is being done with LINQ and .SingleOrDefault() is used. Without seeing the application code, there is no way to know.
All rows in the tablename table will be returned to whatever is running this query. The order in which these rows are returned isn't well defined (and tables don't have an order, so your guess of "first row" is wrong for a number of reasons)
We'd then need to see the consuming code to know what happens - it might take the first row it's given, it might take the last row it's given (different runs of this query could have different first and last rows, because as I said, the order isn't well defined). It may attempt to (in some manner) merge all of the resulting rows together.
You shouldn't try to reason about what happens when your code is subject to SQL injection, you should just apply adequate defenses (e.g. parameterized queries) so you don't have to think about it again.
For example, lets say, for the sake of argument, that this query always returned the rows in some particular order (so long as the moon is full), such that the lowest UserID (or similar) is the first row, and that the consuming code uses the first returned row and ignores other rows. So you decide to "cunningly" create a dummy user with UserID 0 which can't do anything and warns you of an attack.
Well, guess what - all the attacker has to do is inject an ORDER BY CASE WHEN UserName='Admin' THEN 0 ELSE 1 END into the query - and bingo, the first row returned is now guaranteed to be the Admin user.
I'm building an interface on Flask using SQLAlchemy and part of it is a search API. Essentially a type-ahead input is calling the server with its value (for example an email) and then the server is performing a SQLalchemy query using .like in a filter like below
q = session.query(User).filter(User.email.like('%'+term+'%')).all()
This query isn't really returning anything useful and after the first few characters, nothing at all. But if I perform the same query with the term hardcoded, like so:
q = session.query(User).filter(User.email.like('%mysearchterm%')).all()
It will return results perfectly fine, so there's something with how I'm putting the term into the like() method but I really can't figure out what the issue is. The term is coming in from an ajax POST and the value is there on the server-side, just .like() isn't using it correctly.
By "nothing useful" I mean that the first sets of results coming back have nothing to do with the actual term being inputted, after a term of length higher than 3-4 no results are returned back despite matching items existing in the DB.
Any help greatly appreciated.
Issues been resolved. This query sat inside of a larger query builder function which applied limit and offset to the query later in the function, because the limit and offset were higher than the amount of results back the returned result set was empty.
Can chock this one up to bad human error and lack of sleep.
I'm trying to learn the hibernate criteria API but I'm puzzled by the criteria method setFirstResult.
I don't understand why I would want to use it except in the rarest of circumstances. It seems to me that when I retrieve information from a database, I'm only interested in establishing some criteria and then executing the query against the criteria. Why do I care from which index number in the database the results should be read. It is not something I normally do when I write sql queries yet I see this method all over the hibernate literature. Is this method something I always have to invoke when writing Hibernate queries or can I safely ignore it?
Thank you,
Elliott
This is typically used when displaying paginated results of a query. The first page goes from 0 to 19, the second page from 20 to 39, etc.
Well I use it in a bunch of places.. its unfortunate or outright lucky/dumb that you have run into a case where you needed to page your results in which case you generally right queries that pick from one index to another. consider the case where you want to display the audit log of an app that is stored for every write action on the page. in that case you will show the 20 results based on which page the user is and what field the audit log is sorted on.
I have a user list table on my listing page. The data needs to be paged, so how can I make SQL page the data for me (ie. pull the data in sets of 10 records from the table)?
Informix has clauses analogous to, but different from, LIMIT and OFFSET:
SELECT SKIP n LIMIT m ...
You can use FIRST in place of LIMIT.
See the IDS 11.70 InfoCenter, or similar locations for earlier versions of IDS.
Now that you pointed out that you are using Informix, the LIMIT clause will not work. Are you able to instead place your selection into an array and call for the desired data from the array?
What's the equivalent of mysql Limit in ms access. TOP is not sufficient since I'm going to use it for pagination.
Thanks
There isn't one. Your best bet is to add an ID column as a primary key (if you don't already have one) and chunk output by looping through:
SELECT * FROM table
WHERE id >= offset AND id <= offset + chunk_size - 1
until you get all the rows.
Curiously, there are a few references in Microsoft documentation to a LIMIT TO nn ROWS syntax for the Access Database Engine:
ACC2002: Setting ANSI 92 Compatibility in a Database Does Not Allow DISTINCT Keyword in Aggregate Functions
About ANSI SQL query mode (MDB)
However, actual testing seems to confirm that this syntax has never existed in a release version of the Access Database Engine. Perhaps this is one of those features that the SQL Server team wanted to put into Jet 4.0 but were ordered to rollback by the Windows team? Whatever, it seem we must simply put it down to a bad documentation error that Microsoft won't take the time to correct :(
If you need to do pagination on the server** side then I suggest you consider a more capable, modern SQL product with better documentation ;)
** conceptually, that is: the Access Database Engine is not a server DBMS.
Since it doesn't appear that you have any type of sequencial unique key number for these rows, you'll need to create a ranking column: How to Rank Records Within a Query
You need to determine how many rows at a time you will return N = (10, 25,100).
You need to keep track of what "page" the user is on and the values of the first and last rank.
Then when you make the call for the next page it is either the next N rows that are > or < the first and last ranks (depending if the users is going to the previous or next page.).
I'm sure there is a way to calculate the last page, first page, etc.
Only way to achive paging SQL similar to Limit statement by using TOP keywords is as follows:
First Step:
sql = "select top "&LESS_COUNT&" * from (SELECT top "&(PAGE_COUNT*getPage)&" * FROM (SELECT "&COLUMNS&" FROM "&TABLENAME&") AS TBL "&getWhere&getOrderby("asc")&") as TBL "&getOrderby("desc")
Second step:
sql = "SELECT TOP "&PAGE_COUNT&" * FROM (" & sql & ") as TBL "&getOrderby("asc")
To summarize; you should re-order and make the results upside down for 3 times.
port your project to PHP & MySQL. Better support for these type of actions and queries and much much better online documentation. As a 16 year veteran DB developer, I have grown to dispise MS Access and MS SQL with a passion unmatched by anything else. This is due exclusively to their lack of support and documentation.