I have, in a database, records that are serialized PHP strings that I must obfuscate emails if there are any. The simplest record is like {s:20:"pika.chu#pokemon.com"}. It is basically saying: this is a string of length 20 which is pika.chu#pokemon.com. This field can be kilobytes long with lot of emails (or none) and sometimes it is empty.
I wish I could use a SQL regular expression function to obfuscate the user part of the email while preserving the length of the string in order not to break the PHP serialization. The example email above shall be turned into {s:20:"xxxxxxxx#pokemon.com"} where the number of x matches the length of pika.chu.
Any thoughts?
Here is a more complete example of what can be found as serialized PHP:
a:4:{s:7:"locales";a:3:{i:0;s:5:"fr_FR";i:1;s:5:"de_DE";i:2;s:5:"en_US";}s:9:"publisher";s:18:"john#something.com";s:7:"authors";a:2:{i:0;s:21:"william#something.com";i:1;s:19:"debbie#software.org";}s:12:"published_at";O:8:"DateTime":3:{s:4:"date";s:26:"2022-01-26 13:05:26.531289";s:13:"timezone_type";i:3;s:8:"timezone";s:3:"UTC";}}
I tried to do it using native functions but it not worked because functions like REGEXP_REPLACE don't let you manipulate the match to get the size of it, for example.
Instead, I've created a UDF to do that:
CREATE TEMP FUNCTION hideEmail(str STRING)
RETURNS STRING
LANGUAGE js AS """
return str
.replace(/([a-zA-Z.0-9_\\+-:]*)#/g, function(txt){return '*'.repeat(txt.length-1)+"#";})
""";
select hideEmail('a:4:{s:7:"locales";a:3:{i:0;s:5:"fr_FR";i:1;s:5:"de_DE";i:2;s:5:"en_US";}s:9:"publisher";s:18:"john#something.com";s:7:"authors";a:2:{i:0;s:21:"william#something.com";i:1;s:19:"debbie#software.org";}s:12:"published_at";O:8:"DateTime":3:{s:4:"date";s:26:"2022-01-26 13:05:26.531289";s:13:"timezone_type";i:3;s:8:"timezone";s:3:"UTC";}}')
Result:
a:4:{s:7:"locales";a:3:{i:0;s:5:"fr_FR";i:1;s:5:"de_DE";i:2;s:5:"en_US";}s:9:"publisher";s:18:"****#something.com";s:7:"authors";a:2:{i:0;s:21:"*******#something.com";i:1;s:19:"******#software.org";}s:12:"published_at";O:8:"DateTime":3:{s:4:"date";s:26:"2022-01-26 13:05:26.531289";s:13:"timezone_type";i:3;s:8:"timezone";s:3:"UTC";}}
I tried to match the sql values string (0),(5),(12),... or (0,11),(122,33),(4,51),... or (0,121,12),(31,4,5),(26,227,38),... and so on with the regular expression
\(\s*\d+\s*(\s*,\s*\d+\s*)*\)(\s*,\s*\(\s*\d+\s*(\s*,\s*\d+\s*)*\))*
and it works. But...
How can I ensure that the regex does not match a values string like (0,12),(1,2,3),(56,7) with different number of columns?
Thanks in advance...
As i mentioned in comment to the question, the best way to check if input string is valid: contains the same count of numbers between brackets, is to use client side programm, but not clear SQL.
Implementation:
List<string> s = new List<string>(){
"(0),(5),(12)", "(0,11),(122,33),(4,51)",
"(0,121,12),(31,4,5),(26,227,38)","(0,12),(1,2,3),(56,7)"};
var qry = s.Select(a=>new
{
orig = a,
newst = a.Split(new string[]{"),(", "(", ")"},
StringSplitOptions.RemoveEmptyEntries)
})
.Select(a=>new
{
orig = a.orig,
isValid = (a.newst
.Sum(b=>b.Split(new char[]{','},
StringSplitOptions.RemoveEmptyEntries).Count()) %
a.newst.Count()) ==0
});
Result:
orig isValid
(0),(5),(12) True
(0,11),(122,33),(4,51) True
(0,121,12),(31,4,5),(26,227,38) True
(0,12),(1,2,3),(56,7) False
Note: The second Select statement gets the modulo of sum of comma instances and the count of items in string array returned by Split function. If the result isn't equal to zero, it means that input string is invalid.
I strongly believe there's a simplest way to achieve that, but - at this moment - i don't know how ;)
:(
Unless you add some more constraints, I don't think you can solve this problem only with regular expressions.
It isn't able to solve all of your string problems, just as it cannot be used to check that the opening and closing of brackets (like "((())()(()(())))") is invalid. That's a more complicated issue.
That's what I learnt in class :P If someone knows a way then that'd be sweet!
I'm sorry, I spent a bit of time looking into how we could turn this string into an array and do more work to it with SQL but built in functionality is lacking and the solution would end up being very hacky.
I'd recommend trying to handle this situation differently as large scale string computation isn't the best way to go if your database is to gradually fill up.
A combination of client and serverside validation can be used to help prevent bad data (like the ones with more numbers) from getting into the database.
If you need to keep those numbers then you could rework your schema to include some metadata which you can use in your queries, like how many numbers there are and whether it all matches nicely. This information can be computed inexpensively from your server and provided to the database.
Good luck!
I tested this query in my database, and it works fine:
select * from variables where value = 'commas-:-)';
I get a result. Now, I stored the value in a variable and use the query class.
$value = 'commas-:-)' <<< this is passed as a parameter
$query = "select * from variables where value = '$value'";
$this->db->query($query);
Now, this query works for every other value except for this one - but what's odd is that if I PRINT out the exact query (print_r of $query) and execute it on the database, it returns the correct result. So I'm left to think that the query class is screwing with my query, which it shouldn't because everything is properly escaped and $value is a string literal.
What is going on?
$sql = "SELECT * FROM variables WHERE value = ?";
$this->db->query($sql, array('commas-:-)'));
More info
$get_data = $this->db->from('variables')
->where('value', $value)
->get();
Hope this will work...!
try to use these things for checking the queries
echo $this->db->last_query();
print_r($this->db->result_array($get_data));
I found the issue - it was the rerouting function that was causing the mishap. More specifically, the segment filtering function within the route folder in the system core.
This is what happened:
I created an anchor with the encoded value (commas:-)) and I configured the route to reroute the uri to a function I had in my controller. Each time I clicked the link, the value gets passed, and (supposedly) rerouted to the function. Which it did, for almost all the values I used. Except this one.
1st assumption: the db query function is escaping the values. But I turned off the escape, as well as checked the query by printing. The value was correct. I then tried other query formats, and still no results. Conclusion: There's nothing wrong with the database query functions.
2nd assumption: the data must be corrupt - although the value is correct (I'm getting commas:-)), it's not returning anything except when I type in the value manually. So I tested this:
I created a seperate value, and set it equals to the one I typed in(the one that works). I then printed the original value(one passed) and the newly created value using VAR_DUMP.
Turns out, the argument value (one that doesn't work) is a string with length 14 whereas my new variable was a string with a length of 10. WTF? Conclusion: Something occured during the rerouting / passing process that changed the variable.
I went back to the config folder, and replace the variable $i in the reroute to the literal string value commas:-). And guess what? It worked perfectly. And just to make sure it wasn't the regex, I wrote my own custom regex and it matched fine, but the value was still being changed. So I decided to get under the hood.
I traced the URI manipulation in the routes class to the _explode_segment() function, which was used to perform the regex and analyse the uri for other variables. It also did this thing ...
_filter_uri($str)
for each part of the uri segment that was matched.
What did it do? It replaces programmable characters like ( and ) with their HTML ENTITY. Now, if you don't know, html entities have long lengths than url encoding. LOL. So what happened was this:
Original segment : commas-%3A-%29 <- very nice!
Filtered segment : commas-%3A-) <- NOOOOOOOOO! (the right paren encoded with ).)
urldecode(")") = string(4)
urldecode("%29") = string(1)
Fail.
or WIN?!
Im just wondering if it is possible to put a variable in a pattern match in Lua. Like something similar to the following:
var = "hello"
pattern = string.match(datasource, "(var)%s(a%+)")
The reason I need to do this is because the variable "var" will change periodically. (it will be in a loop)
Cheers in advance
Lua doesn't handle string interpolation inside of the quotes. Instead, you'll need to concatenate the parts with the var as a var reference and the rest as quote strings.
"("..var..")%s(a%+)" starts with a "(" as a string literal, concatenates the variable, then finishes off the rest of the string with a string literal.
Use "("..var..")%s(a%+)" instead.
I needed the same thing I think, a variable in a pattern match, but the above solution didn't work for me. I'm posting my solution in case it helps someone, didn't find anything else on the net like it.
I read a ': ' delimited file (name: tel) and want to search by name in the file and have the name and telephone number as answer.
local FileToSearch = h:read'*a' -- Read all the file
var = io.read() -- ask the name
string.gmatch(FileToSearch, ''..var..': '..'%d+') -- search for name, concatenate with number
For some reason the argument I pass to my if statement is not true for even though it should be heres my code:
if (currentAttribute == cBusName)
{
NSLog(#"currentAttribute == cBusName");
}
current attribute and cBusName are both NSMutableStrings an both equal "1" but the NSLog never outputs the string in the console is there something I am missing???
The == operator is comparing that those objects are the same object (IE they point to the same address in memory), not that their values are the same.
Try
if ([currentAttribute isEqualToString: cBusName])
{
NSLog(#"currentAttribute == cBusName");
}
which compares the values of the two strings, not their location in memory.
String objects cannot be compared
using the equality (==) operator. The
reason for this is that any attempt to
perform a comparison this way will
simply compare whether the two string
objects are located at the same memory
location. Let's take a look at this
via an example:via an example:
from http://www.techotopia.com/index.php/Working_with_String_Objects_in_Objective-C, search for Comparing Strings