I have, in a database, records that are serialized PHP strings that I must obfuscate emails if there are any. The simplest record is like {s:20:"pika.chu#pokemon.com"}. It is basically saying: this is a string of length 20 which is pika.chu#pokemon.com. This field can be kilobytes long with lot of emails (or none) and sometimes it is empty.
I wish I could use a SQL regular expression function to obfuscate the user part of the email while preserving the length of the string in order not to break the PHP serialization. The example email above shall be turned into {s:20:"xxxxxxxx#pokemon.com"} where the number of x matches the length of pika.chu.
Any thoughts?
Here is a more complete example of what can be found as serialized PHP:
a:4:{s:7:"locales";a:3:{i:0;s:5:"fr_FR";i:1;s:5:"de_DE";i:2;s:5:"en_US";}s:9:"publisher";s:18:"john#something.com";s:7:"authors";a:2:{i:0;s:21:"william#something.com";i:1;s:19:"debbie#software.org";}s:12:"published_at";O:8:"DateTime":3:{s:4:"date";s:26:"2022-01-26 13:05:26.531289";s:13:"timezone_type";i:3;s:8:"timezone";s:3:"UTC";}}
I tried to do it using native functions but it not worked because functions like REGEXP_REPLACE don't let you manipulate the match to get the size of it, for example.
Instead, I've created a UDF to do that:
CREATE TEMP FUNCTION hideEmail(str STRING)
RETURNS STRING
LANGUAGE js AS """
return str
.replace(/([a-zA-Z.0-9_\\+-:]*)#/g, function(txt){return '*'.repeat(txt.length-1)+"#";})
""";
select hideEmail('a:4:{s:7:"locales";a:3:{i:0;s:5:"fr_FR";i:1;s:5:"de_DE";i:2;s:5:"en_US";}s:9:"publisher";s:18:"john#something.com";s:7:"authors";a:2:{i:0;s:21:"william#something.com";i:1;s:19:"debbie#software.org";}s:12:"published_at";O:8:"DateTime":3:{s:4:"date";s:26:"2022-01-26 13:05:26.531289";s:13:"timezone_type";i:3;s:8:"timezone";s:3:"UTC";}}')
Result:
a:4:{s:7:"locales";a:3:{i:0;s:5:"fr_FR";i:1;s:5:"de_DE";i:2;s:5:"en_US";}s:9:"publisher";s:18:"****#something.com";s:7:"authors";a:2:{i:0;s:21:"*******#something.com";i:1;s:19:"******#software.org";}s:12:"published_at";O:8:"DateTime":3:{s:4:"date";s:26:"2022-01-26 13:05:26.531289";s:13:"timezone_type";i:3;s:8:"timezone";s:3:"UTC";}}
Related
I need to execute a SQL query, which converts a String column to a Array and then validate the size of that array
I was able to do it easily with postgresql:
e.g.
select
cardinality(string_to_array('a$b','$')),
cardinality(string_to_array('a$b$','$')),
cardinality(string_to_array('a$b$$$$$','$')),
But for some reason trying to convert String on vertica to array is not that simple, Saw this links:
https://www.vertica.com/blog/vertica-quick-tip-dynamically-split-string/
https://forum.vertica.com/discussion/239031/how-to-create-an-array-in-vertica
And much more that non of them helped.
I also tried using:
select REGEXP_COUNT('a$b$$$$$','$')
But i get an incorrect value - 1.
How can i Convert String to array on Vertica and gets his Length ?
$ has a special meaning in a regular expression. It represents the end of the string.
Try escaping it:
select REGEXP_COUNT('a$b$$$$$', '[$]')
You could create a UDx scalar function (UDSF) in Java, C++, R or Python. The input would be a string and the output would be an integer. https://www.vertica.com/docs/9.2.x/HTML/Content/Authoring/ExtendingVertica/UDx/ScalarFunctions/ScalarFunctions.htm
This will allow you to use language specific array logic on the strings passed in. For example in python, you could include this logic:
input_list = input.split("$")
filtered_input_list = list(filter(None, input_list))
list_count = len(filtered_input_list)
These examples are a good starting point for writing UDx's for Vertica. https://github.com/vertica/UDx-Examples
I wasn't able to convert to an array - but Im able to get the length of the values
What i do is convert to Rows an use count - its not best performance wise
But with this way Im able to do also manipulation like filtering of each value between delimiter - and i dont need to use [] for characters like $
select (select count(1)
from (select StringTokenizerDelim('a$b$c','$') over ()) t)
Return 3
I have a column called "Bakery Activity" whose values are all JSONs that look like this:
{"flavors": [
{"d4js95-1cc5-4asn-asb48-1a781aa83": "chocolate"},
{"dc45n-jnsa9i-83ysg-81d4d7fae": "peanutButter"}],
"degreesToCook": 375,
"ingredients": {
"d4js95-1cc5-4asn-asb48-1a781aa83": [
"1nemw49-b9s88e-4750-bty0-bei8smr1eb",
"98h9nd8-3mo3-baef-2fe682n48d29"]
},
"numOfPiesBaked": 1,
"numberOfSlicesCreated": 6
}
I'm trying to extract the number of pies baked with a regex function in Tableau. Specifically, this one:
REGEXP_EXTRACT([Bakery Activity], '"numOfPiesBaked":"?([^\n,}]*)')
However, when I try to throw this calculated field into my text table, I get an error saying:
ERROR: function regexp_matches(jsonb, unknown) does not exist;
Error while executing the query
Worth noting is that my data source is PostgreSQL, which Tableau regex functions support; not all of my entries have numOfPiesBaked in them; when I run this in a simulator I get the correct extraction (actually, I get "numOfPiesBaked": 1" but removing the field name is a problem for another time).
What might be causing this error?
In short: Wrong data type, wrong function, wrong approach.
REGEXP_EXTRACT is obviously an abstraction layer of your client (Tableau), which is translated to regexp_matches() for Postgres. But that function expects text input. Since there is no assignment cast for jsonb -> text (for good reasons) you have to add an explicit cast to make it work, like:
SELECT regexp_matches("Bakery Activity"::text, '"numOfPiesBaked":"?([^\n,}]*)')
(The second argument can be an untyped string literal, Postgres function type resolution can defer the suitable data type text.)
Modern versions of Postgres also have regexp_match() returning a single row (unlike regexp_matches), which would seem like the better translation.
But regular expressions are the wrong approach to begin with.
Use the simple json/jsonb operator ->>:
SELECT "Bakery Activity"->>'numOfPiesBaked';
Returns '1' in your example.
If you know the value to be a valid integer, you can cast it right away:
SELECT ("Bakery Activity"->>'numOfPiesBaked')::int;
I found an easier way to handle JSONB data in Tableau.
Firstly, make a calculated field from the JSONB field and convert the field to a string by using str([FIELD_name]) command.
Then, on the calculated field, make another calculated field and use function:
REGEXP_EXTRACT([String_Field_Name], '"Key_to_be_extracted":"?([^\n,}]*)')
The required key-value pair will form the second caluculated field.
I am working in an ABAP program and I have a question.
For example in C# when we have a String variable: string name; , and we want this to be filled with some data from a textbox but also add some ohter text.
For example:
string name = "Hello: " + textBox1.text;,
And I want to ask you how can I do this in ABAP ??? How to add text plus the text written from a Parameter type C?
CONCATENATE and the concatenate operator && will do it as answered by Jagger and vwegert. To do it with string expressions, you use the below where name is the screen field or whatever that has the name in it (it doesn't need to be a field-symbol):
greeting = |Hello: { <name> }|.
String expressions are extremely useful as they can be used to build up complex values without declaring extra variables - e.g. they can passed as directly as function module or method parameters without first assigning to a local variable.
You can either use the CONCATENATE keyword or -- in newer releases -- string expressions. Be sure to check the online documentation and sample programs available using the transaction ABAPDOCU, it will save you a ton of seemingly basic questions.
The equivalent operator is &&.
So in your case it would be:
name = 'Hello: ' && textBox1->text.
I have this string:
201057&channelTitle=null_JS
I want to be able to cut out the '201057' and make it a new variable. But I don't always know how long the digits will be, so can I somehow use the '&' as a reference?\
myDigits substring(0, position of &)?
Thanks
Sure, you can split the string along the &.
String s = "201057&channelTitle=null_JS";
String[] parts = s.split("&");
String newVar = parts[0];
The expected result here is
parts[0] = "201057";
parts[1] = "channelTitle=null_JS";
In production code you chould check of course the length of the parts array, in case no "&" was present.
Several programming languages also support the useful inverse operation
String s2 = parts.join("&"); // should have same value like s
Alas this one is not part of the Java standard libs, but e.g. Apache Commons Lang features it.
Always read the API first. There is an indexOf method in String that will return you the first index of the character/String you gave it.
You can use myDigits.substring(0, myDigits.indexOf('&');
However, if you want to get all of the arguments in the query separately, then you should use mvw's answer.
I got a list of strings. And I want to check for every string in there. Sometimes, a string can have the suffix _anim(X) where X is an integer. If such string has that kind of suffix, I need to check for all other strings that have the same "base" (the base being the part without suffix) and finally group such strings and send them to my function.
So, given the next list:
Man_anim(1)
Woman
Man_anim(3)
Man_anim(2)
My code would discover the base Man has a special suffix, and will then generate a new list grouping all Man objects and arrange them depending on the value inside parenthesis. The code is supposed to return
Man_anim(1)
Man_anim(2)
Man_anim(3)
And send such list to my function for further processing.
My problem is, how can I check for the existence of such suffix, and afterwards, check for the value inside parenthesis?
If you know that the suffix is going to be _anim(X) every time (obviously, with X varying) then you can use a regular expression:
Regex.IsMatch(value, #"_anim\(\d+\)$")
If the suffix isn't at least moderately consistent, then you'll have to look into data structures, like Suffix Trees, which you can use to determine common structures in strings.