Hi, I have a PowerBI report which has 1 Static column Object1 and Value's as Dynamic column. I want to add a Calculated Column which calculates the difference between LAST 2 columns, this is to calculate increase in the Sales for last month. Any idea how this can be done in PowerBI using DAX or Power Query? Thanks
This is a bit clunky, but I think it does what you want.
#"Unpivoted Columns" = Table.UnpivotOtherColumns(PreviousStepNameHere, {"Object1"}, "Attribute", "Value"),
#"Filtered Last 2" = Table.SelectRows(#"Unpivoted Columns", each List.Contains(List.LastN(#"Unpivoted Columns"[Attribute], 2), [Attribute])),
#"Added Custom" = Table.AddColumn(#"Filtered Last 2", "Custom", each if List.Contains(List.LastN(#"Unpivoted Columns"[Attribute], 1), [Attribute]) then [Value] else -[Value]),
#"Grouped Rows" = Table.Group(#"Added Custom", {"Object1"}, {{"Value", each List.Sum([Custom]), type number}}),
#"Added Custom1" = Table.AddColumn(#"Grouped Rows", "Attribute", each "Calculated_Column_Difference_Last2_Columns"),
#"Appended Query" = Table.Combine({#"Unpivoted Columns", #"Added Custom1"}),
#"Pivoted Column" = Table.Pivot(#"Appended Query", List.Distinct(#"Appended Query"[Attribute]), "Attribute", "Value")
Unpivot should preserve the column order. You filter the last two and switch the sign of the 2nd to last to get the difference when you group and sum. Add the desired column name as a custom column named Attribute. Append that back to your original unpivoted table and then re-pivot.
Related
This might be simple, but I can't wrap my head around it... I need a Custom Column in Power Query to return data from a specific column in another row
Currently I have location data for all employee ID numbers, but for some, the location is blank. In this data, in any given employee's row, there is also their manager's ID#.
What I need is a custom row that returns the employee's manager's location IF the employee's location is blank. For now, I am not looking to fix manager's that also do not have a location, if the manager's location is blank, I am ok with the Employee's pulling blanks in these cases only.
Any help would be greatly appreciated.
Merge the table on top of itself using the manager ID column on top matched to the employee ID column on bottom.
Expand [x] location using the arrows atop the new column.
Add column ... custom column ... with formula
=if [location] =null then [putnameofnewcolumnyouexpandedhere] else [location]
Right click remove extra columns
let Source = Excel.CurrentWorkbook(){[Name="Table2"]}[Content],
#"Merged Queries" = Table.NestedJoin(Source, {"ManagerID"}, Source, {"ID"}, "Source", JoinKind.LeftOuter),
#"Expanded Source" = Table.ExpandTableColumn(#"Merged Queries", "Source", {"Location"}, {"Manager.Location"}),
#"Added Custom" = Table.AddColumn(#"Expanded Source", "CombinedLocation", each if [Location]=null then [Manager.Location] else [Location]),
#"Removed Columns" = Table.RemoveColumns(#"Added Custom",{"Location", "Manager.Location"})
in #"Removed Columns"
You can do something like this:
let
Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
AddLocation
= Table.AddColumn(
Source,
"LocationCleaned",
(a) =>
if a[location] = null then
Table.SelectRows(Source, each [emp ID] = a[manager] )[location]{0}
else
a[location]
)
in
AddLocation
I have a call dataset. Looks like this
If a call about a certain member comes in within 30 days of an "original call", that call is considered a callback. I need some logic or Power Query magic to handle this dataset using this logic. So the end result should look like this
Right now, I have the table left joined to itself which gives me every possible combination. I thought I could do something with that but it's proven difficult and when I have over 2 million unique case keys, the duplicates kill run time and overload memory. Any suggestions? I'd prefer to do the manipulation in Power Query editor but can do it in SQL. Plz and thank you.
I think you can do this in Power Query, but I have no idea how it will run with two million records.
It may be able to be sped up with judicious use of the Table.Buffer function. But give it a try as written first.
The code should be reasonably self-documenting
Group by Member ID
For each Member ID, create a table from a list of records which is created using the stated logic.
expand the tables
Mark the rows to be deleted by shifting up the Datediff column by one and applying appropriate logic to the Datediff and shifted columns.
Code assumes that the dates for each Member ID are in ascending order. If not, an extra sorting step would need to be added
Try this M code. (Change the Source line to be congruent with your own data source).
Edit:
Code edited to allow for multiple call backs from an initial call
let
//Change next line to be congruent with your actual data source
Source = Excel.CurrentWorkbook(){[Name="Table3"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{
{"Case key", type text}, {"Member ID", Int64.Type}, {"Call Date", type date}}),
//Group by Member ID
// then create tables with call back date using the stated logic
#"Grouped Rows" = Table.Group(#"Changed Type", {"Member ID"}, {
{"Call Backs",(t)=>Table.FromRecords(
List.Generate(
()=>[ck=t[Case key]{0}, cd=t[Call Date]{0}, cb = null, df=null, idx=0],
each [idx] < Table.RowCount(t),
each [ck=if Duration.Days(t[Call Date]{[idx]+1} - [cd]) < 30
then [ck] else t[Case key]{[idx]+1},
cd=if Duration.Days(t[Call Date]{[idx]+1} - [cd]) < 30
then [cd] else t[Call Date]{[idx]+1},
cb = if Duration.Days(t[Call Date]{[idx]+1} - [cd]) < 30
then t[Call Date]{[idx]+1} else null,
df = if Duration.Days(t[Call Date]{[idx]+1} - [cd]) < 30
then Duration.Days(t[Call Date]{[idx]+1} - [cd]) else null,
idx = [idx]+1],
each Record.FromList({[ck],[cd],[cb],[df]},{"Case key","Call Date","Call Back Date", "Datediff"}))
)}
}),
#"Expanded Call Backs" = Table.ExpandTableColumn(#"Grouped Rows", "Call Backs",
{"Case key", "Call Date", "Call Back Date", "Datediff"},
{"Case key", "Call Date", "Call Back Date", "Datediff"}),
#"Shifted Datediff" = Table.FromColumns(
Table.ToColumns(#"Expanded Call Backs") & {
List.RemoveFirstN(#"Expanded Call Backs"[Datediff]) & {null}},
type table[Member ID=Int64.Type, Case key=text, Call Date=date, Call Back Date=date, Datediff=Int64.Type, shifted=Int64.Type ]),
#"Filter" = Table.SelectRows(#"Shifted Datediff", each [shifted]=null or [Datediff]<>null),
#"Removed Columns" = Table.RemoveColumns(Filter,{"shifted"})
in
#"Removed Columns"
Example with multiple callbacks
Think you can do this with Lead function.
here is the fiddle https://dbfiddle.uk/?rdbms=oracle_11.2&fiddle=f7cabdbe4d1193e5f0da6bd6a4571b96
select
a.*,
LEAD(CallDate, 1) OVER (
Partition by memberId
ORDER BY
CallDate
) AS "CallbackDate",
LEAD(CallDate, 1) OVER (
Partition by memberId
ORDER BY
CallDate
) - a.calldate AS DateDiff
from
mytable a
Eg:
Department
Customer Name
CSE
A
CSE
B
IT
D
CSE
A
ECE
C
EEE
B
ECE
F
Output:
Department
Customer Name
CSE
A,B
IT
D
ECE
C,F
EEE
B
Try this DAX measure
Measure =
CONCATENATEX ( VALUES ( tbl[Customer] ), tbl[Customer], "," )
You can do it in Power Query by adding these below 5 steps to your table's Advance Query Editor
Please adjust the PREVIOUS_STEP_NAME in the code as per your last step name.
let
//............,
//............ all previous steps,
#"Removed Duplicates" = Table.Distinct(#"PREVIOUS_STEP_NAME"),
#"Grouped Rows" = Table.Group(#"Removed Duplicates", {"Department"}, {{"all name", each _, type table [Department=nullable text, Customer Name=nullable text]}}),
#"Added Custom" = Table.AddColumn(#"Grouped Rows", "Custom", each [all name][Customer Name]),
#"Extracted Values" = Table.TransformColumns(#"Added Custom", {"Custom", each Text.Combine(List.Transform(_, Text.From), ", "), type text}),
#"Removed Columns" = Table.RemoveColumns(#"Extracted Values",{"all name"})
in
#"Removed Columns"
Here is the output-
But, if you wants to keep your base table unchanged, just create a new table pointing the base table as source and perform those above steps.
I am using Power bi and I would like to calculate the working hours of my employees based on the time they entered and left the company.
Here is a sample of data
the supposed total hours should be 7 hours and 8 minutes.
SO, the time where the employee was out and do nothing. In my case, I should exclude the time starting from OUT 2:04:20 to IN 3:09:46 about 1 hour.
You can calculate the next out for each in and take the difference:
select t.employee_id,
sum(datediff(second, local_time, next_local_time)) as diff_seconds
from (select t.*,
lead(local_time) over (partition by employee_id order by local_time) as next_local_time
from t
) t
where action = 'IN'
group by t.employee_id;
Note: This assumes that INs and OUTs are interleaved, so there are no rows with the same action in a row.
This also gives the result in seconds -- which can be converted to decimal hours or any other particular format you want.
In Power BI/Power Query, assuming your table is the same structure as your post you can create a new table with the following script. In this script yourSource is reference to your table.
let
#"Sorted Rows" = Table.Sort(yourSource,{{"EmployeeID", Order.Ascending}, {"LOCAL_TIME", Order.Ascending}}),
#"Added Index" = Table.AddIndexColumn(#"Sorted Rows", "Index", 1, 1, Int64.Type),
#"Added Custom1" = Table.AddColumn(#"Added Index", "Employee_In_Out_Index", each if [Action] = "IN" then [Index] else null),
#"Filled Down" = Table.FillDown(#"Added Custom1",{"Employee_In_Out_Index"}),
#"Removed Columns" = Table.RemoveColumns(#"Filled Down",{"Index"}),
#"Reordered Columns" = Table.ReorderColumns(#"Removed Columns",{"Employee_In_Out_Index", "EmployeeID", "Action", "LOCAL_TIME"}),
#"Pivoted Column" = Table.Pivot(#"Reordered Columns", List.Distinct(#"Reordered Columns"[Action]), "Action", "LOCAL_TIME"),
#"Added Custom" = Table.AddColumn(#"Pivoted Column", "Time Difference", each [OUT] - [IN])
in
#"Added Custom"
In the new table, each record is for one in-out. There is a Time Difference column here which you can use in your calculations.
I have in Power Query a Column "% sum of all". I need to create a custom column "Sum Consecutive" that each row has as value the "% sum of all" of the current row + the value of "Sum Consecutive" of the previous row.
Current row situation
New Custom Column Expectation
You can see two images that show the current situation and the next situation I need in the Power Query.
Can you please help me find a code/command to create this new column like that?
Although there are similar solved questions in DAX, I still need to keep editing the file after that, so it should be performed in M language in power query.
Thank you!
Not sure how performant my approaches are. I would think both should be reasonably efficient as they only loop over each row in the table once (and "remember" the work done in the previous rows). However, maybe the conversion to records/list and then back to table is slow for large tables (I don't know).
Approach 1: Isolate the input column as a list, transform the list by cumulatively adding, put the transformed list back in the table as a new column.
let
someTable = Table.FromColumns({List.Repeat({0.0093}, 7) & List.Repeat({0.0086}, 7) & {0.0068, 0.0068}}, {"% of sum of all"}),
listToLoopOver = someTable[#"% of sum of all"],
cumulativeSum = List.Accumulate(List.Positions(listToLoopOver), {}, (listState, currentIndex) =>
let
numberToAdd = listToLoopOver{currentIndex},
sum = try listState{currentIndex - 1} + numberToAdd otherwise numberToAdd,
append = listState & {sum}
in
append
),
backToTable = Table.FromColumns(Table.ToColumns(someTable) & {cumulativeSum}, Table.ColumnNames(someTable) & {"Cumulative sum"})
in
backToTable
Approach 2: Convert the table to a list of records, loop over each record and add a new field (representing the new column) to each record, then convert the transformed list of records back into a table.
let
someTable = Table.FromColumns({List.Repeat({0.0093}, 7) & List.Repeat({0.0086}, 7) & {0.0068, 0.0068}}, {"% of sum of all"}),
listToLoopOver = Table.ToRecords(someTable),
cumulativeSum = List.Accumulate(List.Positions(listToLoopOver), {}, (listState, currentIndex) =>
let
numberToAdd = Record.Field(listToLoopOver{currentIndex}, "% of sum of all"),
sum = try listState{currentIndex - 1}[Cumulative sum] + numberToAdd otherwise numberToAdd, // 'try' should only be necessary for first item
recordToAdd = listToLoopOver{currentIndex} & [Cumulative sum = sum],
append = listState & {recordToAdd}
in
append
),
backToTable = Table.FromRecords(cumulativeSum)
in
backToTable
I couldn't find a function in the reference for M/Power Query that sums a list cumulatively.