In the process of fixing a poorly imported database with issues caused by using the wrong database encoding, or something like that.
Anyways, coming back to my question, in order to fix this issues I'm using a query of this form:
UPDATE table_name SET field_name =
replace(field_name,’search_text’,'replace_text’);
And thus, if the table I'm working on has multiple columns I have to call this query for each of the columns. And also, as there is not only one pair of things to run the find and replace on I have to call the query for each of this pairs as well.
So as you can imagine, I end up running tens of queries just to fix one table.
What I was wondering is if there is a way of either combine multiple find and replaces in one query, like, lets say, look for this set of things, and if found, replace with the corresponding pair from this other set of things.
Or if there would be a way to make a query of the form I've shown above, to run somehow recursively, for each column of a table, regardless of their name or number.
Thank you in advance for your support,
titel
Let's try and tackle each of these separately:
If the set of replacements is the same for every column in every table that you need to do this on (or there are only a couple patterns), consider creating a user-defined function that takes a varchar and returns a varchar that just calls replace(replace(#input,'search1','replace1'),'search2','replace2') nested as appropriate.
To update multiple columns at the same time you should be able to do UPDATE table_name SET field_name1 = replace(field_name1,...), field_name2 = replace(field_name2,...) or something similar.
As for running something like that for every column in every table, I'd think it would be easiest to write some code which fetches a list of columns and generates the queries to execute from that.
I don't know of a way to automatically run a search-and-replace on each column, however the problem of multiple pairs of search and replace terms in a single UPDATE query is easily solved by nesting calls to replace():
UPDATE table_name SET field_name =
replace(
replace(
replace(
field_name,
'foo',
'bar'
),
'see',
'what',
),
'I',
'mean?'
)
If you have multiple replaces of different text in the same field, I recommend that you create a table with the current values and what you want them replaced with. (Could be a temp table of some kind if this is a one-time deal; if not, make it a permanent table.) Then join to that table and do the update.
Something like:
update t1
set field1 = t2.newvalue
from table1 t1
join mycrossreferncetable t2 on t1.field1 = t2.oldvalue
Sorry didn't notice this is MySQL, the code is what I would use in SQL Server, my SQL syntax may be different but the technique would be similar.
I wrote a stored procedure that does this. I use this on a per database level, although it would be easy to abstract it to operate globally across a server.
I would just paste this inline, but it would seem that I'm too dense to figure out how to use the markdown deal, so the code is here:
http://www.anovasolutions.com/content/mysql-search-and-replace-stored-procedure
Related
Thank you for checking my question out!
I'm trying to write a query for a very specific problem we're having at my workplace and I can't seem to get my head around it.
Short version: I need to be able to target columns by their name, and more specifically by a part of their name that will be consistent throughout all the columns I need to combine or compare.
More details:
We have (for example), 5 different surveys. They have many questions each, but SOME of the questions are part of the same metric, and we need to create a generic field that keeps it. There's more background to the "why" of that, but it's pretty important for us at this point.
We were able to kind of solve this with either COALESCE() or CASE statements but the challenge is that, as more surveys/survey versions continue to grow, our vendor inevitably generates new columns for each survey and its questions.
Take this example, which is what we do currently and works well enough:
CASE
WHEN SURVEY_NAME = 'Service1' THEN SERV1_REC
WHEN SURVEY_NAME = 'Notice1' THEN FNOL1_REC
WHEN SURVEY_NAME = 'Status1' THEN STAT1_REC
WHEN SURVEY_NAME = 'Sales1' THEN SALE1_REC
WHEN SURVEY_NAME = 'Transfer1' THEN Null
ELSE Null
END REC
And also this alternative which works well:
COALESCE(SERV1_REC, FNOL1_REC, STAT1_REC, SALE1_REC) as REC
But as I mentioned, eventually we will have a "SALE2_REC" for example, and we'll need them BOTH on this same statement. I want to create something where having to come into the SQL and make changes isn't needed. Given that the columns will ALWAYS be named "something#_REC" for this specific metric, is there any way to achieve something like:
COALESCE(all columns named LIKE '%_REC') as REC
Bonus! Related, might be another way around this same problem:
Would there also be a way to achieve this?
SELECT (columns named LIKE '%_REC') FROM ...
Thank you very much in advance for all your time and attention.
-Kendall
Table and column information in Db2 are managed in the system catalog. The relevant views are SYSCAT.TABLES and SYSCAT.COLUMNS. You could write:
select colname, tabname from syscat.tables
where colname like some_expression
and syscat.tabname='MYTABLE
Note that the LIKE predicate supports expressions based on a variable or the result of a scalar function. So you could match it against some dynamic input.
Have you considered storing the more complicated properties in JSON or XML values? Db2 supports both and you can query those values with regular SQL statements.
I'm building an abstract gem. i need a sql query that looks like this
SELECT * FROM my_table WHERE * LIKE '%my_search%'
is that possible?
edit:
I don't care about querys performance because it's a feature function of a admin panel, which is used once a month. I also don't know what columns the table has because it's so abstract. Sure i could use some rails ActiveRecord functions to find all the columns but i hoped to avoid adding this logic and just using the *. It's going to be a gem, and i can't know what db is going to be used with it. Maybe there is a sexy rails function that helps me out here.
As I understand the question, basically you are trying to build a sql statement which should check for a condition across all columns in that table. A dirty hack, but this generates the required Sql.
condition_string = MyTable.column_names.join(' LIKE ? OR ')
MyTable.all(:conditions => [condition_string, '%my_search%'])
However, this is not tested. This might work.
* LIKE '...' isn't valid according to the SQL standards, and not supported by any RDBMS I'm aware of. You could try using a function like CONCAT to make the left argument of LIKE, though performance won't be good. As for SELECT *, it's generally something to be avoided.
No, SQL does not support that syntax.
To search all columns you need to use procedures or dynamic SQL. Here's another SO question which may help:
SQL: search for a string in every varchar column in a database
EDIT: Sorry, the question I linked to is looking for a field name, not the data, but it might help you write some dynamically SQL to build the query you need.
You didn't say which database you are using, as there might be a vendor specific solution.
Its only an Idea, but i think it worth testing!
It depends on your DB you can get all Columns of a table, in MSSQL for example you can use somethink like:
select name from syscolumns where id=object_id('Tablename')
Under Oracle guess its like:
select column_name from USER_TAB_COLUMNS where TABLE_NAME = 'Tablename'
and then you will have to go through these columns usign a procedure and maby a cursor so you can check for each Column if the data your searching for is in there:
if ((select count(*) from Tablename where Colname = 'searchingdata') > 0)
then keep the results in a separated table(ColnameWhereFound, RecNrWhereFound).
The matter of Datatye may be an Issue if you try to compare strings with numbers, but if you notice for instance under SQL-Server the syscolumns table contains a column called "usertype" which contains a number seems to refer to the Datatype stored in the Columne, like 2 means string and 7 means int, and 2 means smallint, guess Oracle would have something similar too.
Hope this helps.
so i have a table lets say call it "tbl.items" and there is a column "title" in "tbl.items" i want to loop through each row and for each "title" in "tbl.items" i want to do following:
the column has the datatype nvarchar(max) and contains a string...
filter the string to remove words like in,out, where etc (stopwords)
compare the rest of the string to a predefined list and if there is a match perform some action which involves inserting data in other tables as well..
the problem is im ignotent when it comes to writing T-sql scripts, plz help and guide me how can i achieve this?
whether it can be achieved by writing a sql script??
or i have to develope a console application in c# or anyother language??
im using mssql server 2008
thanks in advance
You want a few things. First, look up SQL Server's syntax for functions, and write something like this:
-- Warning! Code written off the top of my head,
-- don't expect this to work w/copy-n-paste
create function removeStrings(#input nvarchar(4000))
as begin
-- We're being kind of simple-minded and using strings
-- instead of regular expressions, so we are assuming a
-- a space before and after each word. This makes this work better:
#input = ' ' + #input
-- Big list of replaces
#input = replace(' in ','',#input)
#input = replace(' out ','',#input)
--- more replaces...
end
Then you need your list of matches in a table, call this "predefined" with a column "matchString".
Then you can retrieve the matching rows with:
select p.matchString
from items i
join predefined p
on removeStrings(i.title) = p.matchString
Once you have those individual pieces working, I suggest a new question on what particular process you may be doing with them.
Warning: Not knowing how many rows you have or how often you have to do this (every time a user saves something? Once/day?), this will not exactly be zippy, if you know what I mean. So once you have these building blocks in hand, there may also be a follow-up question for how and when to do it.
I have a script that updates itself every week. I've got a warning from my hosting that I've been overloading the server with the script. The problem, I've gathered is that I use too many UPDATE queries (one for each of my 8000+ users).
It's bad coding, I know. So now I need to lump all the data into one SQL query and update it all at once. I hope that is what will fix my problem.
A quick question. If I add purely add UPDATE queries separated by a semicolon like this:
UPDATE table SET something=3 WHERE id=8; UPDATE table SET something=6 WHERE id=9;
And then update the database with one large SQL code as opposed to querying the database for each update, it will be faster right?
Is this the best way to "bunch" together UPDATE statements? Would this significantly reduce server load?
Make a delimited file with your values and use your equivalent of MySQL's LOAD DATA INFILE. This will be significantly faster than an UPDATE.
LOAD DATA INFILE '/path/to/myfile'
REPLACE INTO TABLE thetable(field1,field2, field3)
//optional field and line delimiters
;
Your best bet is to batch these statements by your "something" field:
UPDATE table SET something=3 WHERE id IN (2,4,6,8)
UPDATE table SET something=4 WHERE id IN (1,3,5,7)
Of course, knowing nothing about your requirements, there is likely a better solution out there...
It will improve IO since there is only one round trip, but the database "effort" will be the same.
A curiosity of SQL is that the following integer expression
(1 -abs(sign(A - B))) = 1 if A == B and 0 otherwise. For convenience lets call this expression _eq(A,B).
So
update table set something = 3*_eq(id,8) + 6* _eq(id,9)
where id in (8,9);
will do what you want with a single update statement.
I am doing a simple SELECT statement in an Oracle DB and need to select the columns in a somewhat-specific order. Example:
Table A has 100 attributes, one of which is "chapter" that occurs somewhere in the order of columns in the table. I need to select the data with "chapter" first and the remaining columns after in no particular order. Essentially, my statement needs to read something like:
SELECT a.chapter, a. *the remaining columns* FROM A
Furthermore, I cannot simply type:
SELECT a.chapter, a.*
because this will select "chapter" twice.
I know the SQL statement seems simple, but if I know how to solve this problem, I can extrapolate this thought into more complicated areas. Also, let's assume that I can't just scroll over to find the "chapter" column and drag it to the beginning.
Thanks.
You should not select * in a program. As your schema evolves it will bring in things you do not know yet. Think about what happens when someone add a column with the whole book in it? The query you thought would be very cheap suddenly starts to bring in megabytes of data.
That means you have to specify every column you need.
Your best bet is just to select each column explicitly.
A quickie way to get around this would be SELECT a.chapter AS chapterCol, a.* FROM table a; This means there will be one column name chapterCol (assuming there's not a column already there named chapterCol. ;))
If your going to embed the 'SELECT *' into program code, then I would strongly recommend against doing that. As noted by the previous authors, your setting up the code to break if a column is ever added to (or removed from) the table. The simple advice is don't do it.
If your using this in development tools (viewing the data, and the like). Then, I'd recommend creating a view with the specific column order you need. Capture the output from 'SELECT COLUMN_NAME FROM ALL_TAB_COLUMNS' and create a select statement for the view with the column order you need.
This is how I would build your query without having to type all the names in, but with some manual effort.
Start with "Select a.chapter"
Now perform another select on your data base as follows :
select ','|| column_name
from user_tab_cols
where table_name = your_real_table_name
and column_name <> 'CHAPTER';
now take the output from that, in a cut-and-paste manner and append it to what you started with. Now run that query. It should be what you asked for.
Ta-da!
Unless you have a very good reason to do so, you should not use SELECT * in queries. It will break your application every time the schema changes.