The image above shows the current structure of the time series data that I am working with. It has many columns of time series data which are identified by the customer id in the header row. In order to use this data in a pivot table for analysis, I'd like to convert it to a format like the image below:
Here the customer id becomes a dimension that describes the time series data.
Since this is a large data set, it would be a huge time sink to manually transform the data into the desired format. Also, I do not have fancy add-ins like Power Pivot or Power Query...
Using Excel VBA, how can I write a macro to handle this task?
Thanks,
Solution to problem can be found here: http://www.excel.solutions/2014/03/unpivot-excel-data
Start off by pressing keys ALT > D > P to open the Pivot Table Wizard dialog box:
Choose the ‘Multiple consolidation ranges’ option, then click ‘Next’
In step 2a of the wizard, choose the ‘I will create the page fields’ option, and click ‘Next’
Now we need to add our crosstab data range as a data source for this pivot table. Enter / select the appropriate range, then click ‘Add’. Then click ‘Next’.
Choose a location for the intermediate pivot table (it’s a good idea to use a new worksheet, as we can simply delete the entire worksheet when we’re finished). Then click ‘Finish’.
We now have an ‘intermediate’ pivot table, which looks very similar to our raw data, but has some grand totals. Now we want to drill into the source data for this pivot table, by double clicking on the overall Grand Total value – the cell intersection of the Grand Total column, and Grand Total row:
By double clicking to drill into the grand total data source, another worksheet is created, containing a table with our unpivotted data:
Related
Im trying to use MS Excel Power Query to get value from SQL DB based on item in each row.
My Excel Table has the following in A1, B1:
Date: =TODAY()
A2, B2 has the following headers and A3 has the fruits list. C2 - F2 contains other information. Hence, the val needs to be populated in col. B
Fruit, Val
Apple
Orange
Banana
The SQL query looks like below:
select val
from MY_TABLE
WHERE fruit = ?
AND date = ?
The ? is the parameter and it links to cell $B$1 (date), $A3 (the first item in the fruit list)
I am using the ODBC data connection where I input my query and insert the final parameter as ?
Then from editing the connections > properties, I change the parameters under the 'Definition' tab, selecting the appropriate cells.
But when I drag this to the next cell, it doesn't update. I tried changing $A3 to $A4, but once again the value is returned in cell B3 only.
Any idea how I can update this for each row?
I know I could use the MS SQL data connection where I can use a query like
SELECT val
FROM MY_TABLE
WHERE fruit IN (
'Apple',
'Orange',
'Banana'
)
But the excel sheet is used by many people and hence, the fruits list is updated at regular intervals. So using a static query is not ideal.
What im trying to achieve is that whenever the fruits list gets updated, the user can choose to flash fill to the next cell, which will update the Col B, by referencing the equivalent cell A.
I was not able to reproduce the exact problem you are facing with creating dynamic SQL query parameters in this way (for some reason the Parameters button under the Definition tab is greyed out, I am using Excel for Microsoft 365 on Windows 10). Anyway, if you were to succeed in doing this, wouldn't you end up with a unique query for each cell? I would imagine that would hurt performance when clicking on Data > Refresh All.
In any case, I believe one of the reasons for using Power Query is to have it write SQL queries for you: Power Query Editor > Query Settings pane on the right > Applied Steps, right-click on the last one and click on View Native Query to see the SQL query being sent to the server. As you further process the data, this underlying SQL query will be automatically edited depending on the statements supported by query folding. Of course, the connector needs to support this, so I suggest using the MS SQL Server connector. Note that sometimes the View Native Query option is greyed out but query folding is still taking place, the only way to know for sure is by using a profiling tool on the database.
Here is a way to use Power Query so that the whole Val column gets updated in a single data refresh.
Click on cell B1 and name it cellDate by using in the name box left of the formula bar, then right-click on cell B1 > Get Data from Table/Range... to open the Power Query Editor.
Replace the content of the Power Query Editor formula bar with this:
= Date.From(Excel.CurrentWorkbook(){[Name="cellDate"]}[Content][Column1]{0})
You now have a query that returns the date from cell B1. Now click on the query that contains the table you are importing from the database (named Fruits in this example). Filter the Date column using the drop-down list and select any random date.
In the formula bar, replace #date(2021, 9, 10) with cellDate. Now every time you change the date in cell B1 and refresh the data, this filter will be updated. If you are ignoring Privacy Level settings or using a Public Privacy Level for your workbook, this filter step should be folded to the data source.
Close and load these queries as connections only.
Select the range of cells containing the fruit names, create a Table, name it listFruits and right-click > Get Data from Table/Range... to open the Power Query Editor.
In the Query List on the left, right-click on listFruits > Duplicate. Rename it as listFruitsValues. On the Home tab > Merge Queries. Select Fruits as the second table and click on the Fruit column in each table. Select as Join Kind: Left Outer (all from first, matching from second), then click on OK. Note that from this step onwards, the query is not folded back to the data source.
Click on the expand button of the Fruits column, select only the Val column, uncheck Use original column name as prefix, then OK. Remove the Fruit column.
This is what the Power Query Editor window should look like at this stage.
Now you can load the listFruitsValues query in the worksheet next to the Fruit table. Here is what is that looks like with the default table formatting.
Now if any edit is made to the date and/or the list of fruits, clicking on Data > Refresh All will update the Val column accordingly.
On a final note, I would suggest considering a different approach if the source table filtered for the date (i.e. Fruits in this example) is not too large. The issue with the approach presented above is that the users need to click on the Refresh All button after every edit of the fruit list. This can be avoided by simply loading the Fruits query in a separate worksheet and using the following formula to populate the Val column:
=XLOOKUP(A4,Fruits[Fruit],Fruits[Val])
By creating a single Table with the Fruit and Val columns, the values are instantaneously updated when changes are made to the list of fruits and the Fruits query only needs to be refreshed when the date is changed.
I am having an issue running a label filter on a column (classic view) within my pivot table. I am trying to weed out the items in the data that are less than 180 days old. I can't adjust the data because it is used to feed multiple worksheets and I don't wish to copy the data and manipulate it for just this worksheet. Below is the line that I am running.
ActiveSheet.PivotTables("PivotTable5").PivotFields("Ageing").PivotFilters.Add _
Type:=xlCaptionIsBetween, Value1:="180", Value2:="9999"
This is a "between" filter. I have tried greater than and greater than or equal to but the same result occurs. After this line is run, I can click into the pivot table soft/filter down arrow and see that the filter is there as a between and even the days are inputted but the data does not reflect this. If I hit "OK" on the filter it then applies itself but I do not wish to have to do this manually.
Yes, "PivotTable5" is the pivot table in question. Can anyone help me automate this process?
Created a Pivot Table in Excel 2007, and it seems I can only make it do one type of calculation at a time.
Is it possible to provide an average in one column, and a sum in the next?
When I click on "Value Field Settings", choosing any of the options applies it to every column in the Pivot Table, can't figure out how to apply it to only one column.
You can have a different calculation.
When you drop a field into the data section of the pivot change the calculation to sum.
Then drag the same field across again this time change it to average
Currently we use an excel spreadsheet with 4 tabs (as tab for each team), each tab is a planner/diary that week, each day has 4 "slots" for jobs that are planned in.
The people type in the information for each job into one of the 4 slotsfor the day. Each slot contains information in different cells; Job No., Address, Duration, Notes, etc...
The issue/time consuming aspect of this method is when jobs have to be moved about between teams or moved to a different day it involves lots of copying, pasting and deleting.
What I am trying to achieve is some form of drag/drop interface (similar to the way the calendar in Outlook works), where the people who use the spreadsheet can input details for the job in a form and then have a drag an drop "tile" to be able to drag into one of the "slots.
The data contained in these slots needs to be able to produce a report which is a linear list of all the jobs planned that week.
I am at a loss to figure out what method I should use to create this, it seems it is beyond the reach of Excel from what I can find, also beyond the reach of Access, so I suspect it will involve some form of programming. I have basic understanding of VBA, but that's about it.
Can anyone make any suggestions.
Thanks
Working with data in this format will always be hard and I would not reccomend adding any drag and drop until you have sorted out the structure of your data.
You might like to consider another approach, which involves restructuring your data to make it easier to work with...
If you had one sheet with the following columns:
Teamname
Date (instead of Week Number and Day of week,)
Timeslot
JobNo
Address
Duration
Notes
Etc
Then each row would represent a job and have details of when it is occurring.
This is way a database would store the information about the jobs that are happening.
To change when a job is happening, all you have to do is update the date (and timeslot) of the row.
You could then create another worksheet for each team that uses excel formulas to lookup jobs happening this week and display them in a column for each day - like you have now. These sheets would be for display purposes only.
(You would use formulas like VLOOKUP etc)
If you want to change the date of a job you then simply go to the worksheet with the long list and change the date.
Having your data in this long list (ie normalised) format will give you many benefits. For instance you can summerise the report of the data using pivot tables.
If you add filters to the long list (SEE Ribbon>>Data>filters) you can order the data and reduce the list to only show jobs on a selected date, or for a selected team etc...
This is definitely the way forwards. Having restructured the data in this way, you could then add flashy ways to change when a job is taking place by adding buttons on the team worksheets or something.
PART2:
So in you ling list I would recommend excel CONCATENATE function to add an extra column with the text you want displayed elsewhere in your "calendar". #COLUMNA
I would also create another column which is a unique identifier for the row. This will probably be the "Date in YYYMMDD format & timeslot in HHMI format" concatenated together.
In the calendar view sheets say you need a column to display each day of the week with a row for each time slot: You need to
enter a StartDate of the period you want to view
Set up a column for each day (with a formula to calculate the date of the day (ie StartDate, StartDate+1 etc). The date is displayed as a column header (perhaps with a format that shows the dayname, but is stores the date)
each row will have a column that has the timeperiod name and each row will have a value entered. The value is the same as those used in the long list's timeperiod column.
When a user double clicks on a cell you can use the worksheet event
Worksheet_BeforeDoubleClick(ByVal Target As Range, Cancel As Boolean)
in the VBA code, for this event you need to:
see if the cell double clicked is one you want to respond to
if it is find the date and timeperiod from the corresponding column and row header.
use VBA to activate the long list sheet and probably filter to only show the jobs taking place that day or perhaps that week. Then select (or highlight?) the job that was double clicked.
This is quite easy once you know VBA.
There is a lot to know though I would:
use a named range to define the row with the date and another to define the column with the timeperiod
use a named range to define the cells that will respond to a double click
use range functions in VBA like intersection and the entirerow, entirecolumn range properties to find the date and timeperiod of the double clicked cell.
use the macro record to write the initial code that will do the filter.
adjust the code to be well written and to find the date and timeperiod as required.
Create a sheet for all departments? Or just create one sheet where the dept can be chosen.
You could get clever and allow the user to select two cells in the calendar and provide buttons that do things like:
swap the jobs
If they select many cells then a button to Goto the list and only show jobs that were selected in the calendar.
Perhaps the user could select one job and buttons could move the job back or forwards 1,2,7,14 days.
Note that you can use the long list to record other details of the job like time taken, customer, start time, end time etc...
See about Pivot tables here which might give you some ideas about how you can summarise you data to give info like jobs per day, total value of jobs by day, etc...
Before you tie yourself in knots using Excel, you should always consider using Access.... as with someone who know what they are doing, access will deliver a lot more in a shorter period of time and can do nice things like print one page per job etc etc...
(Do you need a contractor?)
mr#HarveyFrench.co.uk
I have a list named Employee Dates, this list contains the columns:
Employee | CPR Completed | CPR Required | ETC
These columns keep going on for all of the training courses required for our employees with alternating columns for completed and required dates. I am using a workflow to calculate all of the required dates of training from the completed dates.
What I desire to do is make another list that will look at ALL of the columns for the required dates and find the soonest ones and populate that list with the soonest dates and from which column it was pulled from.
Any help as to how to approach this? I have been trying to use queries in Access and also some of the custom view settings in SharePoint Designer but no luck so far.
You could try an Excel table (they also have these functions in access if I recall, but I avoid access like a plague). To connect Excel to share point follow the steps in this article:
support.office.com
Ok, now that we are connected you should see all of the columns and values in excel. Next up we need to find the min date (easy) and then get the associated column name (a little harder).
Min Date: The formula should be something along the lines of =min(B1,B3,B5), jut type in =Min( and then CTRL-click on the columns you want to consider for the row. When your done close with ). After wards double click on the square in the bottom right corner and it will do the same logic for all of the rows.
Column Name: A little more difficult, use the min value from the prior column as the lookup value for VlookUp to get the column name. After wards double click on the square in the bottom right corner and it will do the same logic for all of the rows. I'd explain VlookUp, but I'd run out of characters and attention span long before I got to the relevant parts, and excel functions does a fine job of getting you the basics.
Anyway hope that helps,