I'm converting some sensitive data from a low-security encryption to a higher security encryption (specifically, from CFMX_COMPAT to AES with a 256-bit key). I intend to encode my AES-encrypted strings using Hex, and CFMX_COMPAT is extremely likely to use special characters, so finding records that aren't yet converted should be as simple as (pseudocode):
select from table where column has at least one character not in [A-Z0-9]
Is this possible in SQL? If so, how?
I found this documentation, but I had no idea it was possible in a simple LIKE clause. Awesome!
select top 10 foo
from bar
where foo like '%[^A-Z0-9]%'
Related
I currently have a column in a DB2 table which is being passed through web calls and procedure by a character-encrypted value. It is type CHARACTER(13) with a CSSID for encryption.
This has become a huge pain to accommodate through multiple APIs but was initially intended to allow us a unique ID to use in calls that wasn't the primary key.
In DB2-400, what would be the next best thing as far as a 13 or more character string that is unique and randomly created upon insert, but doesn't require decryption (just a plain string)?
Is there a commonly-gravitated-to method for this? We aren't passing secure data, so there's no need for encryption, but we just want a randomly created and unique character
Try hex(generate_unique()). It's unique CHAR(26) string.
Or to_char(timestamp(generate_unique()), 'YYYYMMDDHH24MISSFF6'). You may play with format of the to_char function as well. May be useful to use, let's say, reverse format like FF6SSMIHH24DDMMYYYY to avoid unique index page contention upon heavy insert activity.
This is a comment that doesn't fit in the comments section.
I don't have access to a DB2-400 (anymore), but I tested the code below in DB2 10.5 for Linux.
create sequence seq1;
select concat('A', varchar_format(next value for seq1, '000000000000')) as my_id
from sysibm.sysdummy1;
Result, if you run it 4 times in a row:
A0000000000001
A0000000000002
A0000000000003
A0000000000004
Maybe there's something equivalent in DB2-400.
Sounds like you might be using GENERATE_UNIQUE()
GENERATE_UNIQUE function returns a bit data character string 13 bytes
long (CHAR(13) FOR BIT DATA)
Doesn't really have anything to do with encryption...
And pretty much the ideal solution in my opinion generating a unique value other than a simple numeric identity. So what the problem you are having?
I'm importing data from one system to another. The former keys off an alphanumeric field whereas the latter requires a numeric integer field. I'd like to find or write a function that I can feed the alphanumeric value to and have it return a number that would be unique to the value passed in.
My first thought was to do a hash, but of course the result of any built in hashes are going to contains letters and plus it's technically possible (however unlikely) that a hash may not be unique.
My first question is whether there is anything built in to sql that I'm overlooking, and short of that I'd like to hear suggestions on the easiest way to implement such a function.
Here is a function which will probably convert from base 10 (integer) to base 36 (alphanumeric) and back again:
https://www.simple-talk.com/sql/t-sql-programming/numeral-systems-and-numbers-conversion-in-sql/
You might find the resultant number is too big to be held in an integer though.
You could concatenate the ascii values of each character of your string and cast the result as a bigint.
If the original data is known to be integers you can use cast:
SELECT CAST(varcharcol AS INT) FROM Table
I want to store MAC addresses in one of my database tables, what data-type should I use? Reading articles on google, I have seen Binary(8) mentioned a few times. Is this the correct way?
Also, this does not make sense to me, as MAC addresses are six groups of two hexadecimal digits, wouldn't you use Binary(6)?
I wouldn't use Binary at all.
I would use CHAR(12).
Though this really depends on what you use the data for - if this is for display only, you can simply use the textual representation.
For easier performaing binary operations you can store them into Binary(6)
You can use the following built in function to view the Hex readable value of the binary data:
select top 10 master.sys.fn_varbintohexstr(mac) from macaddresses
and to convert the hexadecimal text into binary:
select CONVERT(binary(6), 'AABBCCDDEEFF', 2);
MAC address is a sequence of 6 hexadecimal numbers. That's why it would be efficient to store it as a number. Use BIGINT.
Also what's the vb.net function that will map all those different characters into their most standard form.
For example, tolower would map A and a to the same character right?
I need the same function for these characters
german
ß === s
Ü === u
Χιοσ == Χίος
Otherwise, sometimes I insert Χιοσ and latter when I insert Χίος mysql complaints that the ID already exist.
So I want to create a unique ID that maps all those strange characters into a more stable one.
For the encoding aspect of the thing, look at String.Normalize. Notice also its overload that specifies a particular normal form to which you want to convert the string, but the default normal form (C) will work just fine for nearly everyone who wants to "map all those different characters into their most standard form".
However, things get more complicated once you move into the database and deal with collations.
Unicode normalization does not ever change the character case. It covers only cases where the characters are basically equivalent - look the same1, mean the same thing. For example,
Χιοσ != Χίος,
The two sigma characters are considered non-equivalent, and the accented iota (\u1F30) is equivalent to a sequence of two characters, the plain iota (\u03B9) and the accent (\u0313).
Your real problem seems to be that you are using Unicode strings as primary keys, which is not the most popular database design practice. Such primary keys take up more space than needed and are bound to change over time (even if the initial version of the application does not plan to support that). Oh, and I forgot their sensitivity to collations. Instead of identifying records by Unicode strings, have the database schema generate meaningless sequential integers for you as you insert the records, and demote the Unicode strings to mere attributes of the records. This way they can be the same or different as you please.
It may still be useful to normalize them before storing for the purpose of searching and safer subsequent processing; but the particular case insensitive collation that you use will no longer restrict you in any way.
1Almost the same in case of compatibility normalization as opposed to canonical normalization.
I have two SQLite tables, that I would love to join them on a name column. This column contains accented characters, so I am wondering how can I compare them for join. I would like the accents dropped for the comparison to work.
You can influence the comparison of characters (such as ignoring case, ignoring accents) by using a Collation. SQLLite has only a few built in collations, although you can add your own.
SqlLite, Data types, Collating Sequences
SqlLite, Define new Collating Sequence
EDIT:
Given that it seems doubtful if Android supports UDFs and computed columns, here's another approach:
Add another column to your table, normalizedName
When your app writes out rows to your table, it normalizes name itself, removing accents and performing other changes. It saves the result in normalizedName.
You use normalizedName in your join.
As the normalization function is now in java, you should have few restrictions in coding it. Several examples for removing accents in java are given here.
There is an easy solution, but not very elegant.
Use the REPLACE function, to remove your accents. Exemple:
SELECT YOUR_COLUMN FROM YOUR_TABLE WHERE replace(replace(replace(replace(replace(replace(replace(replace(
replace(replace(replace( lower(YOUR_COLUMN), 'á','a'), 'ã','a'), 'â','a'), 'é','e'), 'ê','e'), 'í','i'),
'ó','o') ,'õ','o') ,'ô','o'),'ú','u'), 'ç','c') LIKE 'SEARCH_KEY%'
Where SEARCH_KEY is the key word that you wanna find on the column.
As mdma says, a possible solution would be a User-Defined-Function (UDF). There is a document here describing how to create such a function for SQLite in PHP. You could write a function called DROPACCENTS() which drops all the accents in the string. Then, you could join your column with the following code:
SELECT * FROM table1
LEFT JOIN table2
ON DROPACCENTS(table1.column1) = DROPACCENTS(table2.column1)
Much similar to how you would use the UCASE() function to perform a case-insensitive join.
Since you cannot use PHP on Android, you would have to find another way to create the UDF. Although it has been said that creating a UDF is not possible on Android, there is another Stack Overflow article claiming that a content provider could do the trick. The latter sounds slightly complicated, but promising.
Store a special "neutral" column without accented characters and compare / search only this column.