I had my fair share of questions and I got great answers from all of you.
I have a lot of queries running on a DB2 server and I decided to run them from a Batch file for ease of use. No problems here, except the fact that one of them consists of a multitude of
replace statements where I change local chars to Latin chars.
(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE('ä', 'AE'), 'Ä', 'AE'), 'ö', 'OE'), 'Ö', 'OE')
The problem is that by running this from a .bat file from the DB2CLP I get a "+" sign added right before some of the special characters so some get replaced, and others do not.
I have tried to replace them with the HEX code, but I still get some issues with HEX codes containing two sequential letters that give a parsing error.
Is there a better way to do this, maybe with a mapping table?
Thanks to advise,
Doru
Related
I am trying to pull 'COURSE_TITLE' column value from 'PS_TRAINING' table in PeopleSoft and writing into UTF-8 text file to get loaded into Workday system. The file is erroring out while loading because of bad characters(Ã â and many more) present in the column. I have used a procedure which will convert non-ascii value into space. But because of this procedure, the 'Course_Title' which are written in non-english language like Chinese, Korean, Spanish also replacing with spaces.
I even tried using regular expressions (``regexp_like(course_title, 'Ã) only to find bad characters but since the table has hundreds of thousands of rows, it would be difficult to find all bad characters. Please suggest a way to solve this.
If you change your approach, this may work.
Define what you want, and retrieve it.
select *
from PS_TRAINING
where not regexp_like(course_title, '[0-9A-Za-z]')```
If you take out too much data, just add it to the regex
I have a big sporadic sql scripts and need to find and replace a few values in it. I am trying to pass my values in REGEX to Notepad++ but I can't seem to make it work. To be more specific, I have around 50 script, each with 5000 lines, and I need to look for a list of values, e.g. "[dbo].[livesales]" "[dbo].[CreditCards]" in all my scripts separately. I undertand that I need either run this separately against each script or merger them all into one file, but I need the proper REGEX command for it. I need to include square bracket and dots as well. I end up building this but it doesn't work for me:
^(?=.*\b[dbo].[LiveSales]\b)(?=.*\b[dbo].[CreditCards]\b).+$
enter image description here
thanks in advance,
I wouldn't bother using word boundaries, as square brackets in SQL Server are pretty ubiquitous for database object names (e.g. database and column names). I suggest the following pattern:
\[dbo\]\.\[(?:LiveSales|CreditCards)\]
Demo
The major changes I have made include not using word boundaries, escaping the [ and ] brackets (since square bracket is a regex metacharacter with a special meaning), and also not try to match the entire input. Presumably you want to find all such occurrences, and so don't bother trying to scope your pattern with ^ and $.
I am trying to do SQL Injection testing but I am currently testing a command line that separates parameters by spaces, so I'm trying to write a sql statement without any spaces. I've gotten it down to:
create table"aab"("id"int,"notes"varchar(100))
But I cannot figure out how to get rid of the space between CREATE and TABLE. The same would apply obviously for DROP and TABLE, etc.
Does anyone have any ideas? This is for Microsoft SQL Server 2014. Thanks!
[Update]: We are evaluating a third party product for vulnerabilities. I am not doing this to test my own code for weaknesses.
You can write comments between lines instead of spaces in many cases. So /**/ instead of spaces.
Sure it is possible to write some pretty elaborate statements without spaces.
Here is one.
select'asdf'as[asdf]into[#MyTable]
You can even do things like execute sp_executesql without spaces.
exec[sp_executesql]N'select''asdf''as[asdf]into[#MyTable]'
This is not possible, you have to check every argument to make sure they are as intended.
If they are supposed to be numbers, make sure they are numbers, is they are supposed to be a string that may contain specific caracters (like ' or ,) you should escape them when executing the request.
There should be a dedicated mechanism in your programmation langage to take care of hat (like PreparedStatement in Java)
You can also using brackets () for every functions without spaces
SELECT(COUNT(id))FROM(users)where(id>5)
When I execute sql on my csv sometimes in the results we get symbols like Â. I found the entry and rewrote the numbers out and fixed it. But I am curious as to what would do that?
and in the future can I add some sort of character encode checker to make sure it spits out normally?
What do you mean by 'normal'? You tend to see those characters when you have utf8-encoded characters stored in a latin1-encoded table. Or the other way around. I can't remember. I saw this in an old WordPress installation I was trying to migrate and upgrade. Check your table encodings.
Working for a considerable time on cracking some sales data, I came across an error which started to bug me for so real, eating my time of work. After so much of an effort, I was so fed up and nearly to give up on un-importable records.
The scenario:
Bulk sales data comes on txt/csv format needs to be imported to SQL database and then matched with Address History information available on a combination of tables by verifying strings directly from field to field.
If codes matched, need to run a script to update few tables with data. If not matched, need to insert a whole bunch of data in to different tables to create ID which requires for the final sales import.
Most of the was matching, except for few which was giving the trouble. I just needed to import those to history tables. Then started the problems, even though, I updated them, i couldn't match them.
After some much of frustrated hours, I just asked my girl-friend to check when there any error in the string, I worked with.
The string is "Bramhall Stockport" to be matched to "Bramhall Stockport". For SQL script, these two strings are not matching.
I bet if you copy and paste on your table this would match, coz now this is txt format.
Then, Ana figured the error (She is not a computer geek, Architecture Masters), by simply coping and pasting on Microsoft Word 2007.
Screenshot: http://www.contentbcc.com/Anushka/sql_xls.png
Do you see the difference? First is in the txt/csv file and second on the SQL table.
In the first one, you have three regular spaces (ascii 20). In the second one, you have a regular space followed by a non-breaking space, (unicode 0xA0). In excel you can do a search and replace with ALT+0160 as the search and a space character as the replacement to fix it.