Negative query issue in Lucene - lucene

Take a look at this query :
-HOSTNAME:ram AND SEVERITY:information
The above query isn't giving me the intended results. I know we need to append a * : * to a query that it is totally negative, also I found that the above case could be solved by appending a * : * to the HOSTNAME search criteria, but it might be a serious performance crunch, since my application gets search queries from the end user directly.
How do I go about this and help me arrive at an optimal solution please!

Your query should work OK as written, However, including only negative terms does not work:
-HOSTNAME:ram AND -SEVERITY:ERROR
as there is nothing to subtract from (at least, that is the way I think of it).

Related

Looking for an explanation of this attempted SQL injection query

Looking through my logs I found the following query string as an attempt to perform a SQL injection, probably from an automated tool:
(select*from(select+sleep(10)union/**/select+1)a)
From what I can tell, it’s attempting a timing based attack to see if any of the tables in my database start with “a” - the sleep function will only run if the union query matches something? But I am a bit confused about other parts of the attack:
Why are there plus signs between parts of the query?
Why is there a comment as part of the query string?
Would be interested in any answers - I’m fairly certain my site hasn’t been compromised as I haven’t scanned further activity on that query and can’t get it to execute myself, so just wondering if my intuition was correct. Cheers!
I don't know what the point of this is, nor what the point is of trying to figure out the point. Injections are easier to block than to reverse engineer, and the latter doesn't contribute much to the former.
The point of the + and the /**/ are probably pretty much the same, they separate tokens without the use of whitespace. Presumably someone thinks whitespace is going to trigger some kind of alarm or blockage.
The 'a' is just an alias, and is probably there to avoid the error 'ERROR: subquery in FROM must have an alias'
This won't work in stock PostgreSQL because there is no function spelled sleep. They might be targeting a different DBMS, or maybe PostgreSQL with a specific app/framework in use which creates its own sleep function.
The sleep is probably there in case the system doesn't return meaningful messages to the end user. If it takes 10 seconds to get a response, then you know the sleep got executed. If it immediately returns, you know it didn't execute, but don't know why it didn't.
This is meant to detect a SQL injection (probably through an HTML parameter) via a timing attack. The inserted comments (as other people have mentioned) are meant to remove whitespace while still allowing the query to parse in an attempt to fool custom (badly designed) sanitization. The "+" is likely meant to be decoded into a space after passing through HTML decoding.
If you replace the whitespace and add indentation it's easier to see what's going on:
select * <-- match any number of columns on the original query
from
(select <-- nested sub-query in the from clause
sleep(10) <-- timing attack meant to detect whether the SQL ran
union <-- not sure why the union is needed
select 1) a <-- alias the subquery to "a"
) <-- close off matching parens in injected SQL?
I don't think this is attempting to look for tables that start with a, simply run a sleep on a possible recursive query, which could cause your database trouble, if a bunch of them execute.
The + signs are likely an attempt to do some string concatenation... That would be my guess
Regardless I would strongly look at tracing back where this originated from and sanitizing your inputs on your site so raw inputs ( potential sql ) is not being dropped into queries.

Lucene data range search

I'm using Umbraco v7.2 for a site, and have run into a highly entertaining issue trying to search for things using the External Searcher by a date of ranges.
If I perform a Lucene search using the examine management search tools in the backoffice, I get results using this query:
{(+__NodeTypeAlias:bookingperiod)} AND startDate:2016-03-01T00\:00\:00
Subsequently, I KNOW that I can get results that include this date in a range. However, what's highly entertaining, quite puzzling and really rather frustrating, is that if I use a range query, I get no results. Here's the syntax:
{(+__NodeTypeAlias:bookingperiod)} AND +(startDate:[2016-02-28T00:00:00 TO 2016-03-20T00:00:00])
Now, in the interests of clarity, I've tried escaping the colon characters in the dates, the dashes in the dates and both, but it makes no difference at all. Can anyone explain to me where I'm going wrong?
Thanks!
I ran into this issue a while back, not sure why though, but changing to the format :"yyyyMMddHHmmss" helped, might be something with the parser.
So the query becomes:
+__NodeTypeAlias:bookingperiod AND +startDate:[20160228000000 TO 20160320000000]

Pagination with SQLite using LIMIT

I'm writing my own SQLIteBrowser and I have one final problem, which apparently is quite often discussed on the web, but it doesn't seem to have a good general solution.
So currently I store the SQL which the user entered. Whenever I need to fetch rows, I execute the SQL by adding "Limit n, m` at the end of the SQL.
For normal SQLs, which I mostly use, this seems good enough. However if I want to use limit myself in the query, this will obviously give an error, because the resulting sql can look like this:
select * from table limit 30 limit 1,100
which is obviously wrong. Is there some better way to do this?
My idea was, that I could scan the SQL and check if there is a limit clause already used and then ignore it. Of course it's not as simlpe as that, because if I have an sql like this:
select * from a where a.b = ( select x from z limit 1)
it obviously should still apply my limit in such a case, so I could scan the string from the end and look if there is a limit somehwere. My question now is, how feasable this is. As I don't know who the SQL parser works, I'm not sure if LIMIT has to be at the end of SQL or if there can be other commands at the end as well.
I tested it with order byand group by and I get SQL errors if limit is not at the end, so my assumption seems to be true.
I found now a much better solution which is quite simple and doesn't require me to parse the SQL.
The user can enter an arbitrary sql. The result is loaded into a table. Since we don't want to load the whole result at once, as this can return millions of records, only N records are retriueved. When the user scroll to the bottom of the table the next N items are fetched and loaded into the table.
The solution is, to wrapt the SQL into an outer sql with my page size limits.
select * from (arbitrary UserSQL) limit PageSize, CurrentOffset
I tested it with SQLs I regularly use, and this seem to work quite nicely and is also fast enough for my purpose.
However, I don't know wether SQLite has a mechanism to fetch the new rows faster, or if the sql has to be rerun every time. In that case it might not be a good solution fo rrealy complex queries with a long response time.

Is this possible with the SELECT LIKE specification?

Looking at this link: SQL SELECT LIKE
What if you were searching for a name that starts with H and ends with dinger?
Would I use:
SELECT NAME LIKE
'H_dinger'
'H...dinger' or
'H%dinger' ?
I'll assume H_dinger would think there is only 1 character in between, but I don't know what it is -- so I'm searching for it.
H...dinger isn't valid.
And H%dinger seems like it would check it all, but on the site, that isn't even listed?
You would use %, which is the variable-sized wildcard.
But you need to get the syntax right, such as with:
select NAME from TABLE where NAME like 'H%dinger'
Keep in mind that queries using % may be a performance issue (depending on how it's used and the DBMS engine). It can prevent the efficient use of indexes to speed up queries. It probably won't matter for small tables but it's something to keep in mind if you ever need to scale.

Maximum values in wherein clause of mysql

Do anyone knows about how many values I can give in a where in clause? I get 25000 values in a where in clause and mysql is unable to execute. Any thoughts? Awaiting for your thoughts
Although this is old, it still shows up in search results so is worth answering.
There is no hard-coded maximum in MySQL for the length of a query. This includes all parts of the query such as the WHERE clause.
However, there is a value called max_allowed_packet which determines the largest query you can run on the MySQL server process. It isn't to do with the number of elements in the query, but the total length of the query. So
SELECT * FROM mytable WHERE mycol IN (1,2,3);
is less likely to hit the limit than
SELECT * FROM mytable WHERE mycal IN ('This string','That string','Tother string');
The value of max_allowed_packet is configurable from server to server. But almost certainly, if you find yourself hitting the limit because you're writing SQL statements of epic length (rather than dealing with binary data which is a legitimate reason to hit it), then you need to re-think your SQL.
I think that if this restriction is a problem then you're doing something wrong.
Perhaps you could store the data from your where clause in a table and then join with it. This would probably be more efficient.
I think it is something with execution time.
I think you are doing soemthing like this: Correct me if i am wrong:
Select FROM table WHERE V1='A1' AND V2='A1' AND V3='A3' AND ... Vn='An'
There is always a efficient way how you can do your SELECT in your database. Working in a database is importent to keep in your mind that seconds are very importent.
If you can share how your query is look like, then we can help you making a efficient SELECT statement.
I wish u succes