We have a column of type varchar(25) in a SQL Server table that mistakenly had plain text values inserted when they should have been encrypted with AES. We are going to remove the plain text values from the database. The plan was to verify the block size of the field, though this would cause some unencrypted values to be left. Is there any other criteria I can check to reliably identify valid encrypted data?
We need it to be a T-SQL only solution.
Update
Just dug a little deeper, it's getting the values back from a web service. This web service encrypts them using AES in ASP.Net. It takes the returned byte array and then it uses this method to conver the byte array to a string:
static public string ByteArrToString(byte[] byteArr)
{
byte val;
string tempStr = "";
for (int i = 0; i <= byteArr.GetUpperBound(0); i++)
{
val = byteArr[i];
if (val < (byte)10)
tempStr += "00" + val.ToString();
else if (val < (byte)100)
tempStr += "0" + val.ToString();
else
tempStr += val.ToString();
}
return tempStr;
}
For clarity, I should say I did not originally write this code!
Cheers
Not really, especially since the encoding method doesn't look normal to me. It is more common to base64 encode the data which makes it very distinctive. It really depends what the unencrypted data consists of as to how easily it is to determine whether the data is encrypted or not - for instance, is it words, numbers, does it have spaces etc (since the encoded data has no spaces for instance).
It looks like your encoded data will all be numeric represented as a string so depending on length of data, you could see if your column will cast to a BIGINT.
Not sure the best way off the top of my head but there is an answer here that might help you "try cast" in T-SQL StackOverflow-8453861
Related
For more control and clarity of the unpack ISO8583 message , I wanted to print the length of each field, my first attempt was this:
.getFieldPackager(i).getLength()
In a code like this:
Gp=new GenericPackager("./iso87ascii.xml");
message.setPackager(Gp);
for (int i = 0; i <= 128; i++) {
if (message.hasField(i)) {
System.out.println(message.getPackager().getFieldDescription(message, i) +
Gp.getFieldPackager(i).getLength()+
message.getString(i));
}
For fields with variable lengths, it shows the maximum length, not the actual length.
Can any body help?
Edit:
As stated in the comment, We can calculate the length of field:
message.getString(n).getLength()
but we can not figure out what length Jpos has as the length of the field, and if JPos is wrong, there is no way to control it.
We only need the first number of the field, which represents the length of the field, with Jpos terms
You are printing out the configuration of the jPOS Packager, you won’t have addressability to data and its field lengths until you parse and unpack a message using a packager into an ISOMsg Object using message.unpack() - once you unpack() you can get the length of the data using message.getString(n).getLength() or equivalent.
Looking at https://developers.google.com/protocol-buffers/docs/proto3#scalar it appears that string and bytes types don't limit the length? Does it mean that we're expected to specify the length of transmitted string in a separate field, e.g. :
message Person {
string name = 1;
int32 name_len = 2;
int32 user_id = 3;
...
}
The wire type used for string/byte is Length-delimited. This means that the message includes the strings length. How this is made available to you will depend upon the language you are using - for example the table says that in C++ a string type is used so you can call name.length() to retrieve the length.
So there is no need to specify the length in a separate field.
One of the things that I wished GPB did was allow the schema to be used to set constraints on such things as list/array length, or numerical value ranges. The best you can do is to have a comment in the .proto file and hope that programmers pay attention to it!
Other serialisation technologies do do this, like XSD (though often the tools are poor), ASN.1 and JSON schema. It's very useful. If GPB added these (it doesn't change wire formats), GPB would be pretty well "complete".
I would like to have a function in sql that encrypts a given varchar value and returns it as a varchar and Vice-versa. I have checked out the following functions of sql such as :
ENCRYPTBYPASSPHRASE ENCRYPTBYKEY ENCRYPTBYCERT
I have also checked out other questions here but I am unable to find (or maybe understand) the solution I am looking for. What I want to do is pass the encrypted string to a URL as query string (I would like this to do this from Db) which does not have any / so that it does not mess with the URL routes.
Step 1: I made random string generator. (Works)
DECLARE #RandomString VARCHAR (500) = ( SELECT RandomString
FROM dbo.SfRandomStringGenerator (64, 1) );
Example string:
X922t1N2udpdi30HZN9W4U9N997UatHZMJKWvI4si0w9g9q6FA3Lqd8NxCJXAe5D
Step 2: I would like to encrypt the string from Step 1 so that the Output is also in String format BUT the Encryption methods that I have come across all return VARBINARY which is not what I Want.
Declare #EncryptedString Nvarchar(MAX)=dbo.SfEncrypt(#RandomString)
Select #EncryptedString
Example string:
WDkyMnQxTjJ1ZHBkaTMwSFpOOVc0VTlOOTk3VWF0SFpNSktXdkk0c2kwdzlnOXE2RkEzTHFkOE54Q0pYQWU1RA
Step 3: I would also like to be able to decrypt the encrypted string.
Declare #DecryptedString Nvarchar(MAX)=dbo.SfDecrypt(#RandomString)
Select #DecryptedString
Example string:
X922t1N2udpdi30HZN9W4U9N997UatHZMJKWvI4si0w9g9q6FA3Lqd8NxCJXAe5D
Thus far I have not been able to get the encryption I want.
Any help or pointers towards the solution would be helpful. Thanks.
Read the VARBINARY result into a byte[] array.
Convert the byte[] array into a Base 64 string using System.Convert.ToBase64String()
Base64 includes the + and / characters which have special meaning in a url path, so they must be encoded.
UrlEncode the base 64 string using System.Web.HttpUtility.UrlEncode(string) to encode any + or / characters with %2B or %2F.
You should then be able to add the url-encoded string to your url.
On the url endpoint, reverse the procedure to obtain the original encrypted byte[] array.
EDIT: Be aware that using a query string can come up against a length limit, of the order of 2048 characters, so if your data is larger than this, you may find that the server will refuse to handle the request.
In that case, consider using POST to send your data, and supply the encrypted data in the body of the message request.
EDIT in response to comment:
I figured that you would be processing the result of your "do encryption" query before sending it to a server as a url, and so the fact that the SQL encryption functions return varbinary should not have presented a problem.
IF you are happy that you can actually do encryption and decryption in TSQL, then I'll refer you to this SO post which offers a way to do Base-64 encoding and decoding in TSQL care of the xml query api. Do your encryption to a varbinary field, run it through the example to generate a base64 varchar, and then use TSQL REPLACE to convert '+' to '%2B' and '/' to '%2F'
You'll then have a varchar value of a url-encoded, base-64 encoded, encrypted representation of your input data, safe for transmission as a query string.
I am trying to return the 2 byte WORD Hex value of a string character which is not typically English. Basically the Unicode representation. Using vb.net
Ex:
FF5F = ((
FF06 = &
These are represented in unicode standard 6.2. I do not have the ability to display some of the foreign language characters displayed in this set.
So would like for my string character to be converted to this 2 byte value. I haven't been able to find a function in .net to do this.
The code is currently nothing more than a for loop cycling through the string characters, so no sample progress.
I have tried the AscW and ChrW functions but they do not return the 2byte value. ASCII does not seem to be reliable above 255.
If necessary I could isolate the possible languages being tested so that only one language is considered through the comparisons, although an English character is always possible.
Any guidance would be appreciated.
I think you could convert your string to a byte array, which, would look something like this in C#:
static byte[] GetBytes(string str)
{
byte[] bytes = new byte[str.Length * sizeof(char)];
System.Buffer.BlockCopy(str.ToCharArray(), 0, bytes, 0, bytes.Length);
return bytes;
}
From that you can just grab to two first bytes from the array, and there you go, you have them.
If you want to show them on a screen, I guess you should probably convert them to hex or some such displayable format.
I've stolen this from the question here.
A collegaue assisted in developing a solution. Each character of the string is converted to character array, and then to an unsigned integer, which is then converted to Hex.
lt = myString
Dim sChars() As Char = lt.ToCharArray
For Each c As Char In sChars
Dim intVal As UInteger = AscW(c)
Debug.Print(c & "=" & Hex(intVal))
Next
Note the AscW function... AscW returns the Unicode code point for the input character. This can be 0 through 65535. The returned value is independent of the culture and code page settings for the current thread. http://msdn.microsoft.com/en-us/library/zew1e4wc(v=vs.90).aspx
I then compare the resulting Hex to the spec for reporting.
I'm using this code to return some string from a tcpclient but when the string comes back it has a leading " character in it. I'm trying to remove it but the Len() function is reading the number of bytes instead of the string itself. How can I alter this to give me the length of the string as I would normally use it and not of the array underlying the string itself?
Dim bytes(tcpClient.ReceiveBufferSize) As Byte
networkStream.Read(bytes, 0, CInt(tcpClient.ReceiveBufferSize))
' Output the data received from the host to the console.'
Dim returndata As String = Encoding.ASCII.GetString(bytes)
Dim LL As Int32 = Len(returndata)
Len() reports the number of bytes not the number of characters in the string.
Your code is currently somewhat broken. The answer is tcpClient.ReceiveBufferSize, regardless of how much data you actually received - because you're ignoring the return value from networkStream.Read. It could be returning just a few bytes, but you're creating a string using the rest of the bytes array anyway. Always check the return value of Stream.Read, because otherwise you don't know how much data has actually been read. You should do something like:
Dim bytesRead = networkStream.Read(bytes, 0, CInt(tcpClient.ReceiveBufferSize))
' Output the data received from the host to the console.'
Dim returndata As String = Encoding.ASCII.GetString(bytes, 0, bytesRead)
Now, ASCII always has a single character per byte (and vice versa) so the length of the string will be exactly the same as the length of the data you received.
Be aware that any non-ASCII data (i.e. any bytes over 127) will be converted to '?' by Encoding.ASCII.GetString. You may also get control characters. Is this definitely ASCII text data to start with? If it's not, I'd recommend hex-encoding it or using some other option to dump the exact data in a non-lossy way.
You could try trimming the string inside the call to Len():
Dim LL As Int32 = Len(returndata.Trim())
If Len reports the number of bytes and it doesn't match the number of characters, then I can think of two possibilities:
There are more chars being sent than you think (ie, that extra character is actually being sent)
The encoding is not ASCII, so there can be more than one byte per char (and one of them is that 'weird' character, that is the character is being sent and is not 'wrong data'). Try to find out if the data is really ASCII encoded, if not, change the call accordingly.
When I read you correctly, you get a single quotation mark at the beginning, right?
If you get that one consistently why not just subtract one from the string length? Or use a substring from the second character:
Len(returndata.Substring(1)
And I don't quite understand what you mean with »the length of the string as I would normally use it and not of the array underlying the string itself«. You have a string. Any array which might represent that string internally is entirely implementation-dependent and nothing you should see or rely on. Or am I getting you wrong here. The string is what you are using normally. I mean, if that's not what you do, then why not take the length of the string after processing it into something you would normally use?
Maybe I am missing something here, but what is wrong with String.Length?
Dim LL As Int32 = returndata.Length