When I execute sql on my csv sometimes in the results we get symbols like Â. I found the entry and rewrote the numbers out and fixed it. But I am curious as to what would do that?
and in the future can I add some sort of character encode checker to make sure it spits out normally?
What do you mean by 'normal'? You tend to see those characters when you have utf8-encoded characters stored in a latin1-encoded table. Or the other way around. I can't remember. I saw this in an old WordPress installation I was trying to migrate and upgrade. Check your table encodings.
Related
I am trying to pull 'COURSE_TITLE' column value from 'PS_TRAINING' table in PeopleSoft and writing into UTF-8 text file to get loaded into Workday system. The file is erroring out while loading because of bad characters(Ã â and many more) present in the column. I have used a procedure which will convert non-ascii value into space. But because of this procedure, the 'Course_Title' which are written in non-english language like Chinese, Korean, Spanish also replacing with spaces.
I even tried using regular expressions (``regexp_like(course_title, 'Ã) only to find bad characters but since the table has hundreds of thousands of rows, it would be difficult to find all bad characters. Please suggest a way to solve this.
If you change your approach, this may work.
Define what you want, and retrieve it.
select *
from PS_TRAINING
where not regexp_like(course_title, '[0-9A-Za-z]')```
If you take out too much data, just add it to the regex
I am trying to do SQL Injection testing but I am currently testing a command line that separates parameters by spaces, so I'm trying to write a sql statement without any spaces. I've gotten it down to:
create table"aab"("id"int,"notes"varchar(100))
But I cannot figure out how to get rid of the space between CREATE and TABLE. The same would apply obviously for DROP and TABLE, etc.
Does anyone have any ideas? This is for Microsoft SQL Server 2014. Thanks!
[Update]: We are evaluating a third party product for vulnerabilities. I am not doing this to test my own code for weaknesses.
You can write comments between lines instead of spaces in many cases. So /**/ instead of spaces.
Sure it is possible to write some pretty elaborate statements without spaces.
Here is one.
select'asdf'as[asdf]into[#MyTable]
You can even do things like execute sp_executesql without spaces.
exec[sp_executesql]N'select''asdf''as[asdf]into[#MyTable]'
This is not possible, you have to check every argument to make sure they are as intended.
If they are supposed to be numbers, make sure they are numbers, is they are supposed to be a string that may contain specific caracters (like ' or ,) you should escape them when executing the request.
There should be a dedicated mechanism in your programmation langage to take care of hat (like PreparedStatement in Java)
You can also using brackets () for every functions without spaces
SELECT(COUNT(id))FROM(users)where(id>5)
I had my fair share of questions and I got great answers from all of you.
I have a lot of queries running on a DB2 server and I decided to run them from a Batch file for ease of use. No problems here, except the fact that one of them consists of a multitude of
replace statements where I change local chars to Latin chars.
(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE('ä', 'AE'), 'Ä', 'AE'), 'ö', 'OE'), 'Ö', 'OE')
The problem is that by running this from a .bat file from the DB2CLP I get a "+" sign added right before some of the special characters so some get replaced, and others do not.
I have tried to replace them with the HEX code, but I still get some issues with HEX codes containing two sequential letters that give a parsing error.
Is there a better way to do this, maybe with a mapping table?
Thanks to advise,
Doru
What's the simplest way to write and, then, read Emoji symbols in Oracle table?
Currently I have this situation:
iOS client pass encoded Emojis: One%20more%20time%20%F0%9F%98%81%F0%9F%98%94%F0%9F%98%8C%F0%9F%98%92. For example, %F0%9F%98%81 means 😁;
Column type is nvarchar2(2000), so when view saved text via Oracle SQL Developer it looks like: One more time ????????.
This seems more a client problem than a database problem. Certain iOs programs are capable of interpreting that string and show an image instead of that string.
SQL Developer does not do that.
As long as the data stored in the database is the same as the data retrieved from the database, you have no problem.
After all, we do BASE64 encoding/decoding of the text. It’s suitable for small texts.
In MySQL the character set needs to be set to UTF-16 to be able to save emojis, I assume Oracle would need the same ch
I have generated the Create statement for a SQL Server view.
Pretty standard, although there is a some replacing happening on a varchar column, such as:
select Replace(txt, '�', '-')
What the heck is '�'?
When I run that against a row that contains that character, I am seeing the literal '?' being replaced.
Any ideas? Do I need some special encoding in my editor?
Edit
If it helps the end point is a Google feed.
You need to read the script in the same encoding as that in which it was written. Even then, if your editor's font doesn't include a glyph for the character, it may still not display correctly.
When the script was created, did you choose an encoding, or accept the default? If the later, you need to find out which encoding was used. UTF-8 is likely.
However, in this case, the character may not be a mis-representation. Unicode replacement character explains that this character is used as a replacement for some other character that cannot be represented. It's possible in your case that the code you are looking at is simply saying, if we have some data that could not be represented, treat it as a hyphen instead. In other words, this may be nothing to do with the script generation/viewing process, but rather a deliberate piece of code.