I'm trying to convert some searching C# code over to an old SQL project but I don't know how to:
split strings in SQL
perform a "WHERE LIKE IN (#subTerms)"
Edit: I'm using MS SQL Server 2012.
Sorry I thought my question had enough information, the corresponding SQL table would look like:
TableName: Products
Columns: ID (int), Name (varchar)
Data:
MyProduct
My Stuff
Super Product
My SQL would look something like:
DECLARE #term varchar;
SET #term = "My Product"
DECLARE #subTerms varchar/array;
SET #subTerms = ??? (Split Term by ' ')
SELECT *
FROM Products
WHERE Name LIKE IN (#subTerms)
The term I have provided should pull all of the records available in the database.
Hard to say exactly, based on what you've written, but it should be something like this:
SELECT * FROM Items WHERE Name in ('MyProduct', 'My Stuff', 'Super Product')
By the way, I've found it LINQPad to be very useful in situations like this: you can point it at a database, write a LINQ query, and then click the "SQL" tab to see what SQL was produced for that query. The SQL tends to be overly verbose, but you can usually get a pretty good idea of what it's doing and come up with a simplified version yourself.
Update
Based on the updated question, it's more clear what you want to do. I would recommend that you create a full-text search capable field, and use FREETEXT to query it. This is exactly the sort of thing that full-text search was made for.
Any text, including words, phrases or sentences, can be entered. Matches are generated if any term or the forms of any term is found in the full-text index.
Related
I have a repository of SQL queries and I want to understand which queries use certain tables or fields.
Let's say I want to understand what queries use the email field, how can I write it?
Example SQL query:
select
users.email as email_user
,users.email as email_user_too
,email as email_user_too_2
email as email_user_too_3,
back_email as wrong_email -- wrong field
from users
So to state the problem more accurately, you are sorting through a list of SQL queries [as text], and you now need to find the queries that use certain fields using SQL & RegEx (Regular Expressions) in PostgreSQL. (please tag the question so that StackOverflow indexes your question correctly, more importantly, readers have more context about the question)
PostgreSQL has Regular Expression support OOTB (Out Of The Box). So we skip exploring other ways to do this. (If you are reading this as Microsoft SQL Server person, then I strongly suggest you to have a read of this brilliant article on Microsoft's website on defining a Table-Valued UDF (User Defined Function))
The simplest way I could think of to approach your problem, is to throw away what we don't want out of the query text first, and then filter out what's left.
This way, after throwing away the stuff you don't need, you will be left with a set of "tokens" that you can easily filter, and I'm putting token in quotes since we are not really parsing the SQL language, but if we did that would be the first step: to extract tokens.. (:
Take this query for example:
With Queries (
Id
, QueryText
) As (
values (1, 'select
users.email as email_user
,users.email as email_user_too
,email as email_user_too_2,
email as email_user_too_3,
back_email as wrong_email -- wrong field
from users')
)
Select QueryText
, found
From (
Select Id
, QueryText
, regexp_split_to_table (QueryText, '(--[\s\w]+|select|from|as|where|[ \s\n,])') As found
From Queries
) As Result
Where found != ''
And found = 'back_email'
I have sourced the concept of a "query repository" with a WITH statement for ease of doing the pseudo-code.
I have also selected few words/characters to split QueryText with. Like select, where etc. We don't need these in our 'found' set.
And in the end, as you can see above, I simply used found as what's left and filtered it with the field name you are looking for. (Assuming that you know the field you are looking for)
You could improve upon the RegEx I did, or change the method as you wish to make it better. But I think the general concept addresses what you need to achieve. One problem I can see with my solution right off the bat is the fact that you can search for anything really, not just names of the selected fields - which begs the question, why use RegEx, and not Like statements? But again, as I mentioned, you can improve upon the RegEx and address specific requirements you may have. Using Like might limit you in that direction. (In other words, only you know what's good for you. I can't say that from here.)
You can play with the query online here: db-fiddle query and use https://regex101.com/ for testing your RegEx.
Disclaimer I'm not a PostgreSQL developer. There must be other, perhaps better ways of doing this. (:
Hello I want to display results from unrelated tables where a text string exists in a column which is common to all tables in the database.
I can get the desired result with this:
SELECT *
FROM Table1
WHERE Title LIKE '%Text%'
UNION
SELECT *
FROM Table2
WHERE Title LIKE '%Text%'`
However my question is is there a more efficient way to go about this as I need to search dozens of tbls. Thanks for any help you can give!
ps the system I am using supports most dialects but would prefer to keep it simple with SQL Server as that is what I am used to.
There is a SP script you can find online called SearchAllTables (http://vyaskn.tripod.com/search_all_columns_in_all_tables.htm).
When you call it pass in the string, it will return the tables and columns as well as the full string.
You can modify it to work with other datatypes quite easily. It's a fantastic resource for tasks exactly like yours.
I'm still a bit of a noob, so pardon if this question is a bit obvious. I did search for an answer but either couldn't understand how the answers I found applied, or simply couldn't find an answer.
I have a massive database housed on a DB2 for i server which I'm accessing using SQL through SQLExplorer (based on Squirrel SQL). The tables are very poorly documented and the first order of business is figuring out how to find my way around.
I want to write a simple query that does this:
1) Allows me to search the entire database looking for tables that include a column called "Remarks" (which contains field descriptions).
2) I then want it to search that column for a keyword.
3) I want a table returned that includes the names of the tables that include that keyword (just the name, I can look up the table alphabetically later and look inside if I need to.)
I need this search to be super lightweight, and I'm hoping the concept I describe will achieve that. Anything that eats up a lot of resources will likely anger the sys admin for the server.
Just to show I have tried (and that I am a complete noob), here's what I've got so far.
SELECT *
FROM <dbname>
WHERE Remarks LIKE '<keyword>'
Feel free to mock, I told you I'm an idiot :-).
Any help? Perhaps at least a push in the right direction?
PS - I can't seem to find a search function in SQLExplorer, if someone knows if I can perhaps use a simple search or filter to accomplish this same goal...that would be great.
You can query the system catalog to identify the tables:
SELECT TABLE_SCHEMA, TABLE_NAME, COLUMN_NAME
FROM QSYS2.SYSCOLUMNS WHERE UPPER(DBILFL) = 'REMARKS'
And then query each table individually:
SELECT * FROM TABLE_SCHEMA.TABLE_NAME WHERE Remarks LIKE '%<keyword>%'
See the LIKE predicate for details of the pattern expression.
Normally i use something like this
SELECT TABLE_SCHEMA, TABLE_NAME
,COLUMN_NAME,SYSTEM_COLUMN_NAME,COLUMN_HEADING
,DATA_TYPE, "LENGTH",NUMERIC_SCALE
FROM QSYS2.SYSCOLUMNS
WHERE UPPER(COLUMN_NAME) LIKE '%REMARK%'
#JamesA, i'm at V6R1, by default, normal user are not authorized to object QADBIFLD in QSYS
Generally, many if not most IBM i shops (especially those that use RPG) stick to 10 (or less) character schema names & table names, and have a 10 (or less) character names for 'system' column names, even if longer column names are also provided. Column text generally describes each field.
SELECT SYSTEM_TABLE_SCHEMA, SYSTEM_TABLE_NAME
,SYSTEM_COLUMN_NAME,
,DATA_TYPE, "LENGTH",NUMERIC_SCALE
,CHAR(COLUMN_TEXT)
FROM QSYS2.SYSCOLUMNS
WHERE UPPER(COLUMN_NAME) LIKE '%REMARK%'
I'm building an abstract gem. i need a sql query that looks like this
SELECT * FROM my_table WHERE * LIKE '%my_search%'
is that possible?
edit:
I don't care about querys performance because it's a feature function of a admin panel, which is used once a month. I also don't know what columns the table has because it's so abstract. Sure i could use some rails ActiveRecord functions to find all the columns but i hoped to avoid adding this logic and just using the *. It's going to be a gem, and i can't know what db is going to be used with it. Maybe there is a sexy rails function that helps me out here.
As I understand the question, basically you are trying to build a sql statement which should check for a condition across all columns in that table. A dirty hack, but this generates the required Sql.
condition_string = MyTable.column_names.join(' LIKE ? OR ')
MyTable.all(:conditions => [condition_string, '%my_search%'])
However, this is not tested. This might work.
* LIKE '...' isn't valid according to the SQL standards, and not supported by any RDBMS I'm aware of. You could try using a function like CONCAT to make the left argument of LIKE, though performance won't be good. As for SELECT *, it's generally something to be avoided.
No, SQL does not support that syntax.
To search all columns you need to use procedures or dynamic SQL. Here's another SO question which may help:
SQL: search for a string in every varchar column in a database
EDIT: Sorry, the question I linked to is looking for a field name, not the data, but it might help you write some dynamically SQL to build the query you need.
You didn't say which database you are using, as there might be a vendor specific solution.
Its only an Idea, but i think it worth testing!
It depends on your DB you can get all Columns of a table, in MSSQL for example you can use somethink like:
select name from syscolumns where id=object_id('Tablename')
Under Oracle guess its like:
select column_name from USER_TAB_COLUMNS where TABLE_NAME = 'Tablename'
and then you will have to go through these columns usign a procedure and maby a cursor so you can check for each Column if the data your searching for is in there:
if ((select count(*) from Tablename where Colname = 'searchingdata') > 0)
then keep the results in a separated table(ColnameWhereFound, RecNrWhereFound).
The matter of Datatye may be an Issue if you try to compare strings with numbers, but if you notice for instance under SQL-Server the syscolumns table contains a column called "usertype" which contains a number seems to refer to the Datatype stored in the Columne, like 2 means string and 7 means int, and 2 means smallint, guess Oracle would have something similar too.
Hope this helps.
I am trying to implement a feature similar to the "Related Questions" on Stackoverflow.
How do I go about writing the SQL statement that will search the Title and Summary field of my database for similar questions?
If my questions is: "What is the SQL used to do a search similar to "Related Questions" on Stackoverflow".
Steps that I can think of are;
Strip the quotation marks
Split the sentence into an array of words and run a SQL search on each word.
If I do it this way, I am guessing that I wouldn't get any meaningful results. I am not sure if Full Text Search is enabled on the server, so I am not using that. Will there be an advantage of using Full Text Search?
I found a similar question but there was no answer: similar question
Using SQL 2005
Check out this podcast.
One of our major performance
optimizations for the “related
questions” query is removing the top
10,000 most common English dictionary
words (as determined by Google search)
before submitting the query to the SQL
Server 2008 full text engine. It’s
shocking how little is left of most
posts once you remove the top 10k
English dictionary words. This helps
limit and narrow the returned results,
which makes the query dramatically
faster.
They probably relate based on tags that are added to the questions...
After enabling Full Text search on my SQL 2005 server, I am using the following stored procedure to search for text.
ALTER PROCEDURE [dbo].[GetSimilarIssues]
(
#InputSearch varchar(255)
)
AS
BEGIN
-- SET NOCOUNT ON added to prevent extra result sets from
-- interfering with SELECT statements.
SET NOCOUNT ON;
DECLARE #SearchText varchar(500);
SELECT #SearchText = '"' + #InputSearch + '*"'
SELECT PostId, Summary, [Description],
Created
FROM Issue
WHERE FREETEXT (Summary, #SearchText);
END
I'm pretty sure it would be most efficient to implement the feature based on the tags associated with each post.
It's probably done using a full text search which matches like words/phrases. I've used it in MySQL and SQL Server with decent success with out of the box functionality.
You can find more on MySQL full text searches at:
http://dev.mysql.com/doc/refman/5.1/en/fulltext-search.html
Or just google Full Text search and you will find a lot of information.
It looks keyword based on the title you enter, queried against titles and content of other questions. Probably easier (and more appropriate) to do in Lucene (or similar) then in a relational database.
I'd say it's probably a full text search on the question title and the question content and answers as well using the individual words (not the whole title) you enter. Then, using the ranking features of full-text, the top 10 or so questions that rank the highest are displayed.
As tydok pointed out, it looks like they are using full-text searching (I couldn't imagine any other way).
Here's the MSDN reference on Full-Text Searching, nailing the specific query used probably isn't going to happen.
The SQL very well may be just "SELECT * FROM questions;". I find it hard to imagine that the algorithm for finding similar questions is implemented in SQL.