I have a column called "Bakery Activity" whose values are all JSONs that look like this:
{"flavors": [
{"d4js95-1cc5-4asn-asb48-1a781aa83": "chocolate"},
{"dc45n-jnsa9i-83ysg-81d4d7fae": "peanutButter"}],
"degreesToCook": 375,
"ingredients": {
"d4js95-1cc5-4asn-asb48-1a781aa83": [
"1nemw49-b9s88e-4750-bty0-bei8smr1eb",
"98h9nd8-3mo3-baef-2fe682n48d29"]
},
"numOfPiesBaked": 1,
"numberOfSlicesCreated": 6
}
I'm trying to extract the number of pies baked with a regex function in Tableau. Specifically, this one:
REGEXP_EXTRACT([Bakery Activity], '"numOfPiesBaked":"?([^\n,}]*)')
However, when I try to throw this calculated field into my text table, I get an error saying:
ERROR: function regexp_matches(jsonb, unknown) does not exist;
Error while executing the query
Worth noting is that my data source is PostgreSQL, which Tableau regex functions support; not all of my entries have numOfPiesBaked in them; when I run this in a simulator I get the correct extraction (actually, I get "numOfPiesBaked": 1" but removing the field name is a problem for another time).
What might be causing this error?
In short: Wrong data type, wrong function, wrong approach.
REGEXP_EXTRACT is obviously an abstraction layer of your client (Tableau), which is translated to regexp_matches() for Postgres. But that function expects text input. Since there is no assignment cast for jsonb -> text (for good reasons) you have to add an explicit cast to make it work, like:
SELECT regexp_matches("Bakery Activity"::text, '"numOfPiesBaked":"?([^\n,}]*)')
(The second argument can be an untyped string literal, Postgres function type resolution can defer the suitable data type text.)
Modern versions of Postgres also have regexp_match() returning a single row (unlike regexp_matches), which would seem like the better translation.
But regular expressions are the wrong approach to begin with.
Use the simple json/jsonb operator ->>:
SELECT "Bakery Activity"->>'numOfPiesBaked';
Returns '1' in your example.
If you know the value to be a valid integer, you can cast it right away:
SELECT ("Bakery Activity"->>'numOfPiesBaked')::int;
I found an easier way to handle JSONB data in Tableau.
Firstly, make a calculated field from the JSONB field and convert the field to a string by using str([FIELD_name]) command.
Then, on the calculated field, make another calculated field and use function:
REGEXP_EXTRACT([String_Field_Name], '"Key_to_be_extracted":"?([^\n,}]*)')
The required key-value pair will form the second caluculated field.
Related
We have data created by an external source (i.e. I cannot just change the names used so it works) -- the datetime field is named #timestamp and I cannot figure out how to address that element within a transformation expression.
Sample data is available on Vega.GitHub.IO and in a Gist with the data -- I added the "timestamp" element to verify the issue I am experiencing is related to the at sign in the name. Using the 'timestamp' data field, I am able to transform and graph the data as desired:
But I have been unable to use the #timestamp field. I get a parse failure if I use "expr": "datetime(datum.#timestamp)" and an invalid date if I use "expr": "datetime('datum.#timestamp')". Attempting to escape the at sign (or the quotes) gives me parse errors as well. How exactly can I use the data element named #timestamp in the expression?
I finally figured this out -- I don't think it has anything to do with Vega but, rather, is a JavaScript limitation. I can use the array subscript method of accessing the data and the dates parse.
I have a bigquery code.
CREATE TEMP FUNCTION to_struct_attributes(input STRING)
RETURNS STRUCT<status_code STRING, created_time TIMESTAMP>
LANGUAGE js AS """
let res = JSON.parse(input);
res['created_time'] = Date(res['created_time'])
return res;
""";
SELECT
5 AS ID,
to_struct_attributes(
TO_JSON_STRING(
STRUCT(
TIMESTAMP(PARSE_TIMESTAMP('%Y%m%d%H%M%S', '20220215175959','America/Los_Angeles')) AS created_time
)
)
) AS ATTRIBUTES;
When I execute this, I'm getting the following error:
Failed to coerce output value "2022-02-16 01:59:59+00" to type TIMESTAMP
I feel this is quite strange, since BigQuery should be able to interpret it correctly and I haven't had this issue with any other datatypes. Also, if I do:
SELECT TIMESTAMP("2022-02-16 01:59:59+00")
It returns:
2022-02-16 01:59:59 UTC
So BigQuery can indeed parse it correctly. I'm not sure why it doesn't happen for the UDF. On searching the internet, I found this question and as the answer suggests, if I change the return statement to:
return Date(res.created_time);
It resolves the issue. But for a project of mine, doing it for every timestamp is not feasible due to the high number of struct columns.
So, I wanted to know if someone has a better alternative to it?
PS : I have removed a lot of non-essential parts from the above example, so this might look a bit abstract. Also, the actual use-case is a bit different and complex that's why I need that JS udf.
The best way to do what you want is to implement the following code.
return Date(res.created_time);
This happens when you pass a TIMESTAMP to a UDF, it is represented as a DATE object, as stated in the documentation. This is like a return of a TIMESTAMP from a JavaScript UDF, where you need to construct and return a DATE object.
Is there any way of programmatically getting the value of a Text Symbol at runtime?
The scenario is that I have a simple report that calls a function module. I receive an exported parameter in variable LV_MSG of type CHAR1. This indicates a certain status message created in the program, for instance F (Fail), X (Match) or E (Error). I currently use a CASE statement to switch on LV_MSG and fill another variable with a short description of the message. These descriptions are maintained as text symbols that I retrieve at compile time with text-MS# where # is the same as the possible returns of LV_MSG, for instance text-MSX has the value "Exact Match Found".
Now it seems to me that the entire CASE statement is unnecessary as I could just assign to my description variable the value of the text symbol with ID 'MS' + LV_MSG (pseudocode, would use CONCATENATE). Now my issue is how I can find a text symbol based on the String representation of its ID at runtime. Is this even possible?
If it is, my code would look cleaner and I wouldn't have to update my actual code when new messages are added in the function module, as I would simply have to add a new text symbol. But would this approach be any faster or would it in fact degrade the report's performance?
Personally, I would probably define a domain and use the fixed values of the domain to represent the values. This way, you would even get around the string concatenation. You can use the function module DD_DOMVALUE_TEXT_GET to easily access the language-dependent text of a domain value.
To access the text elements of a program, use a function module like READ_TEXT_ELEMENTS.
Be aware that generic programming like this will definitely slow down your program. Whether it would make your code look cleaner is in the eye of the beholder - if the values change rarely, I don't see why a simple CASE statement should be inferior to some generic text access.
Hope I understand you correctly but here goes. This is possible with a little trickery, all the text symbols in a report are defined as variables in the program (with the name text-abc where abc is the text ID). So you can use the following:
data: lt_all_text type standard table of textpool with default key,
lsr_text type ref to textpool.
"Load texts - you will only want to do this once
read textpool sy-repid into lt_all_text language sy-langu.
sort lt_all_Text by entry.
"Find a text, the field KEY is the text ID without TEXT-
read table lt_all_text with key entry = i_wanted_text
reference into lsr_text binary search.
If you want the address you can add:
field-symbols: <l_text> type any.
data l_name type string.
data lr_address type ref to data.
concatenate 'TEXT-' lsr_text->key into l_name.
assign (l_name) to <l_text>.
if sy-subrc = 0.
get reference of <l_text> into lr_address.
endif.
As vwegert pointed out this is probably not the best solution, for error handling rather use message classes or exception objects. This is useful in other cases though so now you know how.
I have two datasets in my report and data is being displayed through a table. When I give expression like below:
=Format(Fields!InvDt.Value, "dsRepSalesReport_tblPrintSalesReport","dd/MMMyyyy")
It says there is Syntax error. If I remove dsRepSalesReport_tblPrintSalesReport part, there is no error.
1) Please advise how to wite the expression in format with aggregate expression.
2) If I write expression without dsRepSalesReport_tblPrintSalesReport part, my table repeats data and shows for all invoice. But when I add aggregate part, dsRepSalesReport_tblPrintSalesReport
Table just shows one value several times.
Please advise how to handel with these two issues.
Thanks
The method signature for Format is:
Public Shared Function Format(
ByVal Expression As Object,
Optional ByVal Style As String = ""
) As String
So that means you can't just specify the field and the Scope as in your first example; the first of the two arguments must return one value only.
In your example, you could use something like:
=Format(First(Fields!InvDt.Value, "dsRepSalesReport_tblPrintSalesReport"), "dd/MMMyyyy")
Which will format the first value in the specified Scope.
Another option would be to just set the value as required in the report then use the Format property:
It's difficult to answer your second question without knowing what your data/required results are... If you update the question with some simplified sample data to illustrate the actual issue you're facing that would be helpful.
I wanted to use Date.UTC to store dates and datetimes in postgresql 9.2 "json" field, but of course it fails:
hp=> update formapp_record set data='{"dt": Date.UTC(120, 10, 2)}' where id=17;
ERROR: invalid input syntax for type json
LINE 1: update formapp_record set data='{"dt": Date.UTC(120, 10, 2)}...
^
DETAIL: Token "Date" is invalid.
CONTEXT: JSON data, line 1: {"dt": Date...
It is possible to store the UTC timestamp directly, but then how could the decoder know that the value should decode to a date or datetime instead of an int ?
It is also possible to store the Date.UTC call as string as such:
update formapp_record set data='{"dt": "Date.UTC(120, 10, 2)"}' where id=17;
While that works, it requires 0. checking if the string starts with Date.UTC and 1. use eval in plv8
A solution would be to store some metadata like:
update formapp_record set data='{"dt": {"_type": "date", "_value": [120, 10, 2]}}' where id=17;
But that's not very "standard", it's even "hackish".
What's your take on this matter ?
Alas, json doesn't know anything about dates.
I'd store an ISO 8601 date as a string. Yes, it's a pain. Yes, it means there's no nice standard way to tell "this is a date" vs "this is a string". IMO it's less painful than most of the other options, though.
A possible solution is to use Postgres's row_to_json function and just store your dates as timestamps and extract them to json as required. However, Tobe Hede wrote some json functions for Postgres that may help and seem to be alot more complete then the 2 native options that Postgres has made available for 9.2
See post How do I query using fields inside the new PostgreSQL JSON datatype? for the thread.