is there a way to add an additional column when specifying BQ table schema?
What I mean - apart from the field type, optionality and description I also would like to add another column - additional metadata. For example a semantic column name. Do not want to use the Description one, as that would be for an actual field definition.
thank you!
Related
Our database has a field with JSON data that we'd like to use in reports. E.g.
{
owner_type: "USER",
updated_at: 1641996749092389600,
version_no: 1,
entity_type: "INDIVIDUAL",
country:"ES",
}
How can one create dynamic fields in Superset, e.g. to expose owner_type as its own field?
I'm coming from tools like Snowflake and Zoho Analytics where you could build Views, Dynamic Tables and Formula Fields based on aggregated raw data.
You can add columns to your table in Superset. Hover on 'Sources' on the header and select 'Tables'. Then from there, choose the option to edit the record of your table. In that you can add a calculated column/custom column.
To add a column for owner_type, lets name the custom column as owner_type. Fill the datatype for the new column as VARCHAR(100). Choose the table from the dropdown. In the expression, put json_column->"$.owner_type" and then hit save. This expression is for MySQL database. You can find the expression to parse JSON in your particular DB.
I created a table in SAP via se11, then I used the table maintenance generator.
Now I edit the table via sm30:
The second and the third column: Both have the heading "Feldname".
The first "Feldname" column is called COLUMN_NAME and its data element is "Fieldname".
The second "Feldname" column is called AUTH_FIELD and its data element is "XUFIELD"
I would like to see the column names which I gave the columns in se16 (COLUMN_NAME, AUTH_FIELD) in the heading.
How to prevent the table maintenance generator from giving other names in the headings?
Option 1 - use custom data elements:
Instead of using Fieldname and XUFIELD data elements, you can create your custom data elements and give them what header you would like.
(You will have to regenerate table maintenance)
Option 2 - editing screen
When generated the table maintenance, you supplied a function group and a screen number.
Go to SE80 -> Function Groups -> <function_group_supplied> -> screens -> <screen_supplied>.
Then edit it as you want.
Note: Modifying a generated object is considered risky. Your customized changes might be overwritten in a future regeneration.
Add custom data elements with suitable descriptions. Let the new data elements refer to the original ones (resp. the domains) to avoid having to reinvent everything.
Data element descriptions can be translated.
You can set different descriptions for different lengths, e.g. "Field" for the narrow column with length 10, and "Field name" for a wide label with length 30.
Regenerating the maintenance screen won't accidentally delete the changed descriptions.
I am trying Sense/Net Community edition features.
I defined and installed content type called "Vacation Request" successfully.
I want to know what is the purpose of "mapping" field in table: SchemaPropertyTypes
Many thanks,
I really, really hope you are asking this only out of curiosity, and not because you want to change something manually in the db - because it is not recommended :). Please always access the content repository through the API, do not query or modify the db directly.
Property types and values
Simple property values (like int or short text values) are stored in the FlatProperties table. This is a fixed-with table, containing a predefined number of columns dedicated to different types (e.g. x pieces of string columns, y pieces of int columns - see column names in the table).
Property definitions are stored in the SchemaPropertyTypes table, as you have found out.
The zero-based Mapping field in the SchemaPropertyTypes table defines the column index in the FlatProperties table for a particular property. E.g. a value of a string property with mapping 6 will be stored in the FlatProperties table's 'nvarchar_7' column (note the index is shifted by one, because the column name index is one-based).
If you take a look at the PropertyInfoView view (not table), it may help clarifying this: the last column of the view is a computed column that displays the column name that you can look up in the flat properties table.
(there are other useful SQL views there that display data in a more readable way)
Property 'overflow'
It is possible to register more properties of the same type (e.g. int) than can fit in one row in the FlatProperties table. Solution: Sense/Net stores these nodes in multiple rows - this is why there is a Page column there.
Although MS SQL Server supports a huge number of columns for some time, this design has been kept for compatibility reasons.
This is why you see mapping values in PropertyInfoView like 249 with column name nvarchar_10: the value is stored on page 3, which means that content occupies 3 records in the FlatProperties table.
'Other' property types
You may have noticed that in case of reference or long text properties there is no mapping. This is because we do not store them in the FlatProperties table, they have their own tables like ReferenceProperties or TextPropertiesNText.
In my Qlik View document I want to change the Current Selections information to use the Label applied to the field rather than the table.field format.
For example
PartsTable.PartNo
Would be
Part Number
Unfortunately there's no way to do this in the Current Selections object. However, you may have some alternatives depending on your requirements.
First of all, if you're just happy with seeing the current selections as text (rather than having the functionality of the Current Selections object), you can create a Text object and use the expression:
=replace(GetCurrentSelections(), 'PartsTable.PartNo', 'Part Number')
This will then look something like the below:
The other alternative is to use the RENAME statement in the load script after all your table loads are complete. RENAME allows you to rename a single field or a collection of fields by using a mapping table. The syntax for a single field is shown below:
RENAME FIELD oldname to newname
If you should need to rename more than one field at a time, you can expand this to:
RENAME FIELD oldname1 to newname1, oldname2 to newname2,...
More detail on the syntax including using a mapping table can be found in the QlikView installed help file.
For your example, I put together a small demo:
QUALIFY *;
PartsTable:
LOAD * INLINE [
PartNo
100
200
300
];
UNQUALIFY *;
RENAME FIELD PartsTable.PartNo to [Part Number];
This then results in:
RENAME is similar to the alias (AS) statement, except that you can first load all of your data, and then do the rename at the end. This will then rename your field so that it appears under its new name in any front-end controls (e.g. Current Selections etc.) However, this may not be suitable for you if you already have an existing field named Part Number in your script.
I've created a derived column that translates a 1 to an 'M' and a 2 to 'F'. i.e. a gender indicator. The derived column feeds into a Fuzzy Lookup transformation and then to a conditional split. The problem is the derived field does not show up in any of the downstream components. In the Fuzzy Lookup transform the "Pass Through" checkbox is checked for the derived column, but in the following Conditional Split transform the column does not show up at all. Funny thing is that the _Similarity_Gender_Derived does show up in the column list for the conditional split.
Hopefully someone else has seen this type of behavior.
Thanks - Mr. Do
Right click on the Fuzzy Lookup task and select Show Advanced Editor.
Go to the "Input and Output Properties" tab.
Expand the "Output" item, and then the "Output Columns" item.
Is your derived column listed there?
If it is, it should also show up on the available input columns of the Conditional Split task. If not ...
Right click on the Derived Column task and select Show Advanced Editor.
Go to the "Input and Output Properties" tab.
Expand the "Derived Column Output" item, and then the "Output Columns" item, and select your derived gender column.
Note its LineageID attribute.
Repeat the earlier steps to get the Fuzzy Lookup's Output Columns.
Hit the "Add Column" button. Name the column the same name as your derived column, and in the "SourceInputColumnLineageID" attribute, enter the LineageID you noted earlier.
Alternate answer: is your derived column creating an all new column, or simply replacing your existing "1/2" column? In the Derived Column Editor, check your "Derived Column" .. umm .. column. If you are just replacing your existing column with the new value (instead of adding a new column) you may just be looking in the wrong place.
Thanks for the response. Turns out that issue had to do with some corruption with the meta data. I ended up going back into the Derived Column Transform, renamed the column in error, then added a new derived column with the old name. I saved the transform, and then removed the original column. That fixed the problem.
Thanks for the responses.
Did you add the Derived Column into an already existing transform chain?
If you did then there's a good chance that one of the transforms further down the queue is set to not pass on this newly derived column. Check all the transforms below and make sure that you're derived column is set to be passed through.