I am facing an issue related to olap and kettle. Thing is I am using pentaho DI(kettle) to export data from my database to olap star schema(facts & dimensions). Now when it comes to timestamp I am able to retreive months,years and days from aw timestamp using calculator in kettle . Now What I need is to convert month numeric value to its coresponding month name (like replace 1 with JAN and so on). How this can be acheived with kettle . Please suggest me .
In PDI/Kettle, I've found it tricky to set the value of one field based on the value of another field that is a different data type. A JavaScript or Formula step will do it, but in this case I think I'd use a Stream Lookup because your lookup values are few and fixed.
Use a Data Grid step with an Integer column for the month number (1, 2, 3 ...) and a string column with the month name (JAN, FEB, MAR ...). Use this step as the 'Lookup step' in your Stream Lookup. Then just retrieve the month name into your data flow. This should be pretty fast, which is good if you're working with typical data warehouse volumes.
As Brian said you can also use the Modified JavaScript step to perform the conversion. Here is how you can do it with that:
var month_names=new Array();
month_names[1]="Jan";
month_names[2]="Feb";
month_names[3]="Mar";
month_names[4]="Apr";
month_names[5]="May";
month_names[6]="Jun";
month_names[7]="Jul";
month_names[8]="Aug";
month_names[9]="Sep";
month_names[10]="Oct";
month_names[11]="Nov";
month_names[12]="Dec";
var month_short = month_names[month_num];
Related
I have a SQL query that creates a table, and every month 2 new columns will be added for that table related to the current month.
I have tried without success to set up a flat table (visual) in Qlik that will automatically expand every month to include these table. Is there a way to do this, and i so please point me in the right direction.
You can have a look at CrossTable prefix.
This prefix allows a wide table to be converted to a long table.
So if we have data like this:
After running the following script:
CrossTable:
CrossTable(Month, Sales)
LOAD Item,
[2022-10],
[2022-11],
[2022-12],
[2023-01],
[2023-02],
[2023-03],
[2023-04]
FROM
[C:\Users\User1\Documents\SO_75447715.xlsx]
(ooxml, embedded labels, table is Sheet1);
The final data will looks like below. As you can see there are only 3 columns. All xls month columns (after Item) are now collapsed under one field - Month and all the values are collapsed under Sales column.
Having the data in this format then allows creating "normal" charts with adding Month column as dimension and use sum(Sales) as an expression.
P.S. If you dont want to manage the new columns being added then the script can be:
CrossTable(Month, Sales)
LOAD
Item,
*
FROM
...
Am a newbie in Microsoft Access 2016 making a database system for processing loans.
I have made two fields:
ID e.g: 001 with datatype number and
NRC (This is the official National Registration Card number in my country e.g: "123456/78/1" )
I now want to generate a new field:
LoanNumber from the fields 1. ID and 2. NRC mentioned above.
From the examples above, I want this new field to be composed with two parts concatenated together; that is "ID-first part of NRC before the first '/' ".
e.g: LoanNumber : 001-123456
What code in the expression builder will will help me achieve this?
A number datatype value cannot be saved with preceding zeros. Use Format function to manipulate number. This can be done in a query or textbox but not in table because Format function is not available for Calculated field type.
Format([ID], "000-") & Left([NRC], 6)
I wants to log transformation start time and end time into table. But I am getting error Field [start_date] is required and couldn't be found!.
Following steps I did.
Step 1 : Get Transformation name and system date from Get System Data as
Transformation Start_Date.
Step 2 : Use Table Input to get count of records in table A.
Step 3 : Use Filter to check if table A is empty (Count = 0), if empty then
copy of data from table B to Table A.
Step 4 : IF empty then control goes to Table Input to select all
data from table B.
Step 5 : Use Table Output To insert data from Table Input.
Step 6 : Get system date from Get System Data as transformation End_date.
Step 7 : Use Table Output step to insert data into log table, Into this step I
am inserting Transformation name,Start Date and End Date.
Can someone let me know where I am wrong. I am not able to get Start Date at the end of transformation. Following is the Diagram.
Transformation Diagram
Table Input step ignores records that were generated before. In your diagram "Get_Transformation_name_and_start_time" generates a single row that is passed to the next step (the Table Input one) and then it's not propagated any further.
You can use a single "Get System Info" step at the end of your transformation to obtain start/end date (in your diagram that would be Get_Transformation_end_time 2). To get transformation start date you can use "with system date (fixed)" value. It will return the system time, determined at the start of the transformation, that will be common for all rows. You can use "system date (variable)" as an end timestamp (in case of more than one record you'll have to take max of these values).
It's probably worth looking at standard Pentaho logging options: http://wiki.pentaho.com/display/EAI/.08+Transformation+Settings#.08TransformationSettings-Logging , you can set-up a DB conection and a table that will store transformation execution data "out of the box".
How to write this expression in PowerBI
select distinct([date]),Temperature from Device47A8F where Temperature>25
Totally new to PowerBI. Is there any tool that can change the query from sql to PowerBI expression?
I have tried so many type of different type of expressions but getting error, Most of the time I am getting this:
The expression refers to multiple columns. Multiple columns cannot be converted to a scalar value.
Need help, Thanks.
After I posted my answer, wondered if your expected result is get only one date by temperature, In other words, without repeated dates in your result set.
A side note: select distinct([date]),Temperature from Device47A8F where Temperature>25 returns repeated dates since DISTINCT keyword evaluate distinct columns values specified in the SELECT statement, it doesn't return distinct values in a specific column even if you surround it with parenthesis.
Now what brings us here. What I can see in your error is that you are trying to use a table-valued (produces a table with multiple columns) expression in a measure which only accepts scalar-valued (calculate only one value).
Supposing you have a table like this:
Running your SQL query you will get the highlighted in yellow rows:
You can see 01/09/2016 date is repeated. If you want to create a measure you have to define what calculation you want to show for temperature. i.e, average, max or min etc.
In the below expression is being calculated the maximum temperature greater than 25 per date:
MaxTempGreaterThan25 =
CALCULATE ( MAX ( Device47A8F[Temperature] ), Device47A8F[Temperature] > 25 )
In this case the measure MaxTempGreaterThan25 is calculated per date.
If you don't want to produce a measure but a table. In the Power BI Tool bar select Modeling tab and click New Table icon.
Use this expression:
MyTemperatureTable =
FILTER ( Device47A8F, Device47A8F[Temperature] > 25 )
It should produce a new table named MyTemperatureTable like this:
I recommend you learn some basics about DAX, it is pretty different from SQL / T-SQL and there are things you can't do depending on your model and data.
Let me know if this helps.
You probably don't need to write any code if your objective is to show the result in a Power BI visual e.g. a table. Power BI naturally aggregates data if the datatype is numeric (e.g. Temperature).
I would just add a Table visual on a Report page and add the Date and Temperature columns to it. Then in Visualizations / Fields / Values I would click the little down-arrow on the Temperature field and set the Aggregation e.g. Maximum. Then in Visualizations / Fields / Filters I would click the little down-arrow on the Temperature field and set the Filter e.g. is greater than: 25
Hard-coded solutions are unlikely to survive the next question from your users e.g. "but what if I want to see Temperature > 24? Or 20? Or 30?"
Data - from table (58 columns), in this table 4 columns will use in conditional filter.
1 column - bit data type(Color_Ind)
1 column - bit data type(Type_ind)
2 columns - varchar data type(Region,State)
process steps
1) OLE DB connection to extract all data
2) use Data Conversion to convert bit type columns into string data type.
3) use conditional filter to filter the data into 5 different category.
Probelm:-
- If I use color_ind and Type_ind it will not create my desired output, but If I use it without these columns and only use Region and state it will create my desired output.
Could you please tell me how to use bit data type into condtional filter in SSIS?
Appreciate your help.
Thanks
I got the answare from http://social.msdn.microsoft.com/Forums/en-US/sqlintegrationservices/thread/a0c4dbe0-bbe1-421f-b2fe-b72dce4d224e post...
Thanks for your efforts.