Remove html entities from a databases - sql

Question

Due to errors of my predecessors a (MySQL) database I would like to use contains a lot of HTML entities (e.g. € instead of €).
As the database should contain raw data (a database shouldn't have anything to do with HTML) I want to remove them from the DB and store it in proper UTF8, the collocation is already that.
What would be a good way to fix this? The only thing I can think of is to write a PHP script that gets all the data, runs it through html_entity_decode() and writes it back. It's doable since it's a one-time-operation and the DB is only about 100MB large, but it's still less than optimal.
Any ideas?

Answer 1

Since no-one could provide a satisfying SQL-only solution, I solved it with a script similar to this one.
Note that it only works if all the tables you use it on have a primary key, but this will usually be the case
<?php
// Specify which columns need to be de-entitiezed
$affected = array(
'table1' => array('column1', 'column2'),
'table2' => array('column1', 'column2'),
);
// Make database connection
$db = new PDO("mysql:dbname=yourdb;host=yourhost", "user", "pass");
foreach($affected as $table => $columns){
// Start a transaction for each table
$db->beginTransaction();
// Find the table primary key. PHP5.4 syntax!
$pk = $db->query("SHOW INDEX FROM " . $table . " WHERE Key_name = 'PRIMARY'")->fetch()[0];
foreach($columns as $column){
// Construct a prepared statement for this column
$ps = $db->prepare("UPDATE " . $table . " SET " . $column . " . = ? WHERE " . $pk . " = ?");
// Go through all rows
foreach( $db->query("SELECT " . $column . ", " . $pk . " FROM " . $table) as $row){
$row[0] = html_entity_decode($row[0]); // Actual processing
$ps->execute($row);
}
}
// Everything went well for this table, commit
$db->commit();
}
?>

Answer 2

I tnink u need to create a mysql procedure. (with SELECT loop and update replace) REPLACE(TextString, ''','"') ;

Answer 3

Depending on the database (Oracle, MySql, etc) and whether you can take it offline you might be able to export all the DDL and data as a large SQL script (containing INSERTs for all the tables). Then you could do a standard search/replace using sed:
sed -i 's/€/€/g' script.sql
then drop the database or truncate the tables and recreate it using the script.

Answer 4

Ultimately I think you are going to have to resort to PHP at some stage, converting a lot of these entites in SQL is going to invole a huge amount of desicion logic.
However, One approach I can think of if you must use SQL, is to create a user defined function, that esentially has a huge case statement in (Or lots of if/then's) :
http://dev.mysql.com/doc/refman/5.0/en/case-statement.html
Then you should simply be able to do something like:
SELECT col1,col2,col3,mtuserdecodefunction(column-with-entities-in) FROM mytable
Which should in theory return you a cleaned table.