Is there any way to print a textual representation of a tf-slim model along the lines of what nolearn offers:
## Layer information
name size total cap.Y cap.X cov.Y cov.X filter Y filter X field Y field X
-------------- ---------- ------- ------- ------- ------- ------- ---------- ---------- --------- ---------
input 1x144x192 27648 100.00 100.00 100.00 100.00 144 192 144 192
Conv2DLayer 12x144x192 331776 100.00 100.00 2.08 1.56 3 3 3 3
Conv2DLayer 12x144x192 331776 60.00 60.00 3.47 2.60 3 3 5 5
MaxPool2DLayer 12x72x96 82944 60.00 60.00 3.47 2.60 3 3 5 5
...
DenseLayer 7 7 100.00 100.00 100.00 100.00 144 192 144 192
EDIT:
I can use something like this to print the info for a given layer:
print("%s: %s" % (layer.name, layer.get_shape()))
what I would need to complete the table, would be some way to crawl or walk up the "layer stack" (i.e. get from a given layer to the incoming / input layer(s).
It is not textual representation that you seek, but maybe TensorBoard will suffice? You can visualize whole computation graph and monitor your model using this tool.
https://www.tensorflow.org/how_tos/summaries_and_tensorboard/
Related
I would like to create pie charts according to their respective rows such that each pie chart contain the 3 different columns in their respective years
I manage to create the pie charts but they are all squeezed together in one graph, how can I separate them?
this is my dataset:
sector year Total in Practice (OT) Total in Practice (SLP) Total in Practice (SLP)
0 2014 123 400 123
1 2015 234 456 123
2 2016 345 484 345
3 2017 345 539 566
4 2018 453 565 123
5 2019 454 598 234
6 2020 453 626 243
7 2021 755 682 243
this is my code:
df_all.T.plot.pie(df_all,subplots=True, figsize=(10, 3))
and this is how my plot end up as
I have the following data frames:
print(df)
id_code turnover costs
001 100 200
002 100 200
003 100 200
004 100 200
print(df_db)
Description Code1, Code2, ... CodeN
Retail 001 002 ... nan
Wholesale 003 nan ... nan
Supply 004 nan ... nan
And I would like to create the following final_df, adding a column representing the description in df_db; basically, if the id_code is present in a row of df_db, merge the values:
print(final_df)
id_code turnover costs Description
001 100 200 Retail
002 100 200 Retail
003 100 200 Wholesale
004 100 200 Supply
I tried with pd pivot but it does not report the desired result. How can I obtain final_df?
Use DataFrame.melt + Series.map
if there are no duplicate codes in df_db:
mapper=df_db.melt('Description').set_index('value')['Description']
df['Description']=df['id_code'].map(mapper)
print(df)
id_code turnover costs Description
0 1 100 200 Retail
1 2 100 200 Retail
2 3 100 200 Wholesale
3 4 100 200 Supply
Detail:
print(mapper)
value
1 Retail
3 Wholesale
4 Supply
2 Retail
5 Wholesale
6 Supply
Name: Description, dtype: object
We use melt before merge
final_df=df.merge(df_db.melt('Description').drop('variable',1),left_on='id_code',right_on='value').\
drop('value',1)
Out[157]:
id_code turnover costs Description
0 1 100 200 Retail
1 2 100 200 Retail
2 3 100 200 Wholesale
3 4 100 200 Supply
I am using Keras to create an LSTM neural-network that can predict the concentration in the blood of a certain drug. I have a dataset with time stamps on which a drug dosage was administered and when the concentration in the blood was measured. These dosage and measurement time stamps are disjoint. Furthermore several other variables are measured at all time steps (both dosage and measurements). These variables are the input for my model along with the dosages (0 when no dosage was given at time t). The observed concentration in the blood is the response variable.
I have normalized all input features using the MinMaxScaler().
Q1:
Now I am wondering, do I need to normalize the time variable that corresponds with all rows as well and give it as input to the model? Or can I leave this variable out since the time steps are equally spaced?
The data looks like:
PatientID Time Dosage DosageRate DrugConcentration
1 0 100 12 NA
1 309 100 12 NA
1 650 100 12 NA
1 1030 100 12 NA
1 1320 NA NA 12
1 1405 100 12 NA
1 1812 90 8 NA
1 2078 90 8 NA
1 2400 NA NA 8
2 0 120 13.5 NA
2 800 120 13.5 NA
2 920 NA NA 16
2 1515 120 13.5 NA
2 1832 120 13.5 NA
2 2378 120 13.5 NA
2 2600 120 13.5 NA
2 3000 120 13.5 NA
2 4400 NA NA 2
As you can see, the time between two consecutive dosages and measurements differs for a patient and between patients, which makes the problem difficult.
Q2:
One approach I can think of is aggregating on measurements intervals and taking the average dosage and SD between two measurements. Then we only predict on time stamps of which we know the observed drug concentration. Would this work, or would we lose to much information?
Q3
A second approach I could think of is create new data points, so that all intervals between dosages are the same and set the dosage and dosage rate at those time points to zero. The disadvantage is then, that we can only calculate the error on the time stamps on which we know the observed drug concentration. How should we tackle this?
My granny has some strange ideas. Every birthday she takes me shopping.
She has some strict rules. If I buy a present less than $20 she wont contribute anything. If I spend over $20 she will contribute up to $30.
So if a present costs $27 she would contribute $7.
That now leaves me with $23 to spend on extra presents that birthday; the same rules as above apply on any additional presents.
Once the $30 are spent there are no more contributions from granny and I must pay the rest myself.
Here is an example table of my 11th, 12th and 13th birthday.
DollarsSpent granny_pays
BirthDayAge PresentNum
11 1 25.00 5.00 -- I used up $5
2 100.00 25.00 -- I used up last $20
3 10.00 0.00
4 50.00 0.00
12 1 39.00 19.00 -- I used up $19 only $11 left
2 7.00 0.00
3 32.00 11.00 -- I used up the last $11 despite $12 of $32 above the $20 starting point
4 19.00 0.00
13 1 21.00 1.00 -- used up $1
2 27.00 7.00 -- used up $7, total used up $8 and never spent last $22
So in pandas I have gotten this far.
import pandas as pd
granny_wont_pay_first = 20.
granny_limit = 30.
df = pd.DataFrame({'BirthDayAge' : ['11','11','11','11','12','12','12','12','13','13']
,'PresentNum' : [1,2,3,4,1,2,3,4,1,2]
,'DollarsSpent' : [25.,100.,10.,50.,39.,7.,32.,19.,21.,27.]
})
df.set_index(['BirthDayAge','PresentNum'],inplace=True)
df['granny_pays'] = df['DollarsSpent'] - granny_wont_pay_first
df['granny_limit'] = granny_limit
df['zero'] = 0.0
df['granny_pays'] = df[['granny_pays','zero','granny_limit']].apply(np.median,axis=1)
df.drop(['granny_limit','zero'], axis=1, inplace=True)
print df.head(len(df))
And this is the output. Using the median on the 3 numbers is a nice way to work out what granny will contribute.
The problem is that you can see each present is treated in isolation and I don't correctly erode my $30 each present within each BirthDayAge.
DollarsSpent granny_pays
BirthDayAge PresentNum
11 1 25.00 5.00
2 100.00 30.00 -- should be 25.0
3 10.00 0.00
4 50.00 30.00 -- should be 0.0
12 1 39.00 19.00
2 7.00 0.00
3 32.00 12.00 -- should be 11.0
4 19.00 0.00
13 1 21.00 1.00
2 27.00 7.00
Trying to think of a nice pandas way to do this erosion.
Hopefully no loops please.
I don't know if there is a more concise way, but this should work and does avoid loops as requested.
df['per_gift'] = df.DollarsSpent - 20
df['per_gift'] = np.where( df.per_gift > 0, df.per_gift, 0 )
df['per_bday'] = df.groupby('BirthDayAge').per_gift.cumsum()
df['per_bday'] = np.where( df.per_bday > 30, 30, df.per_bday )
df['granny_pays'] = df.groupby('BirthDayAge').per_bday.diff()
df['granny_pays'] = df.granny_pays.fillna(df.per_bday)
Note that 'per_gift' ignores the maximum subsidy of $30 and 'per_bday' is the cumulative subsidy (capped at $30) per 'BirthDayAge'.
BirthDayAge DollarsSpent PresentNum per_gift per_bday granny_pays
0 11 25 1 5 5 5
1 11 100 2 80 30 25
2 11 10 3 0 30 0
3 11 50 4 30 30 0
4 12 39 1 19 19 19
5 12 7 2 0 19 0
6 12 32 3 12 30 11
7 12 19 4 0 30 0
8 13 21 1 1 1 1
9 13 27 2 7 8 7
I need a solution similar to this:
DAX running total (or count) across 2 groups
However slightly more complex.
I have the following:
(apologies for the layout - i can't post pictures)
Name Date Monthly Rev Total Rev Margin( % Rev)
Proj 1 1/08/2014 0 7000 15%
Proj 1 1/09/2014 1000 7000 15%
Proj 1 1/10/2014 1000 7000 15%
Proj 1 1/11/2014 1000 7000 15%
Proj 1 1/12/2014 0 7000 15%
Proj 1 1/01/2015 0 7000 15%
Proj 1 1/02/2015 2000 7000 15%
Proj 1 1/03/2015 2000 7000 15%
Proj 2 1/11/2014 0 16000 10%
Proj 2 1/12/2014 1500 16000 10%
Proj 2 2/12/2014 1500 16000 10%
Proj 2 3/12/2014 1500 16000 10%
Proj 2 4/12/2014 1500 16000 10%
Proj 2 5/12/2014 2000 16000 10%
Proj 2 6/12/2014 2000 16000 10%
Proj 2 7/12/2014 0 16000 10%
Proj 2 8/12/2014 2000 16000 10%
Proj 2 9/12/2014 2000 16000 10%
Proj 2 10/12/2014 2000 16000 10%
Monthly rev is the revenue received in a month, total is the total project value and margin is the percentage of revenue. The table is linked to a dates table by Date.
I need to show margin by date (there are other descriptive columns in the table for slicing) however the margin calc is not straightforward.
In an excel table it would look something like this:
Cumm simple margin | Completion| Cumm complex margin | Margin earnt
0 0% 0 0
150 20% 30 30
300 40% 120 90
450 60% 270 150
450 60% 270 0
450 60% 270 0
750 80% 600 330
1050 100% 1050 450
0 0% 0 0
150 11% 17 17
300 22% 67 50
450 33% 150 83
600 44% 267 117
800 56% 444 178
1000 67% 667 222
1000 67% 667 0
1200 78% 933 267
1400 89% 1244 311
1600 100% 1600 356
Where:
Simple margin is calculated on a cumulative basis as % of monthly Rev
Percentage complete of the project is calculated based on "active" months where revenue is earned
Cumulative simple margin is multiplied by the % complete
Actual margin earned in a particular month is the difference between two months.
Note that Monthly revenue is not necessarily continuous.
No idea how to recreate this in power pivot, any suggestions would be well received.
Cheers
Assuming
That your Project 2 data should run monthly from 1/11/2015 to 1/09/2015 (rather than individual December dates)
You have your data in a table called 'ProjectMargins'
Your DateDim table is called 'Reporting Dates'
Then these are the DAX Measures you need (although there may be simpler methods for achieving these results):
[MonthlyRev]:=SUM(ProjectMargins[Monthly Rev])
[ActiveMonth]:=CALCULATE(COUNTROWS('ProjectMargins'),FILTER('ProjectMargins',[MonthlyRev]>0))
[AllActiveMonths]:=CALCULATE([ActiveMonth],ALL('Reporting Dates'[Date]))
[Completion]:=DIVIDE(CALCULATE([ActiveMonth],FILTER(ALL('Reporting Dates'[Date]),'Reporting Dates'[Date] <= MAX(ProjectMargins[Date]))),[AllActiveMonths])
If you need to calculate TotalRev, from your Monthly Rev, Rather than it appearing in the original source table:
[TotalRev]:=IF(ISBLANK(MAX(ProjectMargins[Margin( % Rev)])),BLANK(),CALCULATE([MonthlyRev],ALL('Reporting Dates'[Date])))
[Rev%]:=MAX(ProjectMargins[Margin( % Rev)])
[Cumm Simple Margin]:=CALCULATE([MonthlyRev]*[Rev%],FILTER(ALL('Reporting Dates'[Date]),'Reporting Dates'[Date] <= MAX(ProjectMargins[Date])))
[Cumm Complex Margin]:=[Completion]*[Cumm Simple Margin]
[Previous Month Cumm Complex]:=CALCULATE([Cumm Complex Margin], DATEADD('Reporting Dates'[Date],-1,MONTH))
[Margin Earnt]:=IF([Cumm Complex Margin]>0,[Cumm Complex Margin]-[Previous Month Cumm Complex],BLANK())
NOTE: This assumes that the margin is never negative.
Ensure that the date field from the DateDim table is used in your pivot, not the date field from the Fact table.