Related
I try to explain my problems but perhaps there are to many so I dont know where to start. And I am running out of time :(
I have tested the ability of fungi to alter plastic surfaces after 2 diff timepoints and in two batches. Method of surface investigation was ATR-FT-IR. I now have spectral IR data from 4 different substrates, each exposed to 5 diff fungi for two diff times. Every sample was measured 10 times (very mostly but sadly not always). Logically, I was running control samples (no fungi and no treatment, sample treated but without fungi), also for the two diff batches. SO- for each Substrate, I come up with around 140 columns and 1820 rows. I shrunk the data to respective means and standard deviations with excel and imported it as .xlsx- because .csv absolutely failed and i could figure out why ?! Catastrophe.
> head(pet)
Wavenumbers MEAN_PET_untreated SD_PET_untreated MEAN_c_PET_B1_AL1 SD_PET_B1_AL1 MEAN_c_PET_B1_AL2 SD_c_PET_B1_AL2
1 3997.805 8.021747e-05 0.0003198024 -5.862401e-05 0.0002445300 0.0001309613 0.0004636534
2 3995.877 7.575977e-05 0.0003168603 -4.503153e-05 0.0002384142 0.0001185064 0.0004360579
3 3993.948 7.713719e-05 0.0003169468 -3.218230e-05 0.0002414230 0.0001145128 0.0004352532
4 3992.020 7.847460e-05 0.0003191443 -3.255098e-05 0.0002519945 0.0001258732 0.0004388980
5 3990.091 7.835603e-05 0.0003159916 -4.792059e-05 0.0002617358 0.0001325122 0.0004465352
6 3988.163 7.727790e-05 0.0003063113 -6.286794e-05 0.0002593732 0.0001297744 0.0004532126
My goal was a multiplot, showing averaged spectral data with geom_path and geom_ribbons per fungus, yielding 5 elements per plot (substrate pur, controle t1, controle t2, fungi treat 1, fungi treat 2). The dataset is really large so I had problems to handle it and created these plots manually, so NOT by faceting.
F4<-ggplot(pet)+
geom_errorbar(aes(x = Wavenumbers, y = MEAN_c_PET_B2_AL2, ymin = MEAN_c_PET_B2_AL2 - SD_c_PET_B2_AL2, ymax = MEAN_c_PET_B2_AL2 + SD_c_PET_B2_AL2, group=1), alpha= .1, stat="identity", position = "identity", colour="red")+
geom_path(aes(x = Wavenumbers, y = MEAN_c_PET_B2_AL2), stat="identity", group= 1, colour= "red")+
geom_errorbar(aes(x = Wavenumbers, y = MEAN_c_PET_B2_AL1 ,ymax = MEAN_c_PET_B2_AL1 + SD_c_PET_B2_AL1, ymin = MEAN_c_PET_B2_AL1 - SD_c_PET_B2_AL1, group=1), alpha= .1, stat="identity", position = "identity", colour="purple")+
geom_path(aes(x = Wavenumbers, y = MEAN_c_PET_B2_AL1), stat="identity", group= 1, colour= "purple")+
geom_errorbar(aes(x = Wavenumbers, y = MEAN_PET_untreated, ymax = MEAN_PET_untreated + SD_PET_untreated, ymin = MEAN_PET_untreated - SD_PET_untreated, group=1), alpha= .1, stat="identity", position = "identity", colour="yellow")+
geom_path(aes(x = Wavenumbers, y = MEAN_PET_untreated), stat="identity", group= 1, colour= "yellow")+
geom_errorbar(aes(x = Wavenumbers, y = MEAN_F4_PET_B2_AL1, ymax = MEAN_F4_PET_B2_AL1 + SD_F4_PET_B2_AL1, ymin = MEAN_F4_PET_B2_AL1 - SD_F4_PET_B2_AL1, group=1), alpha= .1, stat="identity", position = "identity", colour="orange")+
geom_path(aes(x = Wavenumbers, y = MEAN_F4_PET_B2_AL1), stat="identity", group= 1, colour= "orange")+
geom_errorbar(aes(x = Wavenumbers, y = MEAN_F4_PET_B2_AL2, ymax = MEAN_F4_PET_B2_AL2 + SD_F4_PET_B2_AL2, ymin = MEAN_F4_PET_B2_AL2 - SD_F4_PET_B2_AL2, group=1), alpha= .1, stat="identity", position = "identity", colour="darkgreen")+
geom_path(aes(x = Wavenumbers, y = MEAN_F4_PET_B2_AL2), stat="identity", group= 1, colour= "darkgreen")+xlab(NULL)+ylab(NULL)+
scale_x_reverse(limits=c(4000 , 500))
So far I summarized the diff ggplots with:
pets<-grid.arrange(F1, F2, F7,F4, F19, ncol = 1, nrow = 5)
ggsave("Multi.pdf", width = 210, height = 297, units = "mm", pets)
This is nearly fine, not elegant and very complicated, but I wont give up at this stage of work as it costed me a whole week. Sadly, I am not really happy with the design, not even to say, I can not use this like it is. Currently, I try to find solutions regarding:
a) Getting rid of empty grid areas left and right to the plotted values. I use scale_x_reverse(limits=c(4000 , 500)), but the range is extended to both sides on the x axis.
b) Creating manually a legend, because even if it would be possible to do this via shared.legend or whatever, it would always yield to many elements. I only want 5 elements with the always repeating, same colors (red=substrate pure, orange= cT_t1, yellow= cT_t2, green= f_t1, purple = f_t2)
c) creating manually a y-labeling (Absorbance), spanning invisible over all plots (vertically)- I tried to label only the 3. plot in the middle, but this leads to a indentation of this plot and the ones above and below appear more left-ragged. If this would be possible, I could use the direct labeling for indicating the respective fungus (e.g. F4).
d) creating a global x labeling- because if I label only the last element, the height of the last plot is reduced by the height of the label.
e) Give it an overall name.
What makes me nervous, too, is that I get an error only for geom_path, telling me that 1 row was removed. But shouldnt this affect also the geom_ribbon? Has it something to do with the fact that I have to call ribbon BEFOR I call geom_path? Otherwise, the lines would have been hidden by the ribbon.
Removed 1 row(s) containing missing values (geom_path).
Also, I am a wondering about the long duration of code execution. 1 element needs 20 seconds, the whole plot 2 minutes to compute. But at least, it is not collapsing like Excel did before- inclusively data loss. Is it normal for such huge datasets? Or could it indicate a very problematic problem?
Ok, finally I hope someone is out there, having had similar work-around-solutions. Because, like I said, I am not willing to spend another week to tidyr or reshape or mutate or whatever.
Thanx in advance! :)
I am trying to plot a bar plot, but it looks really bad.
plt.style.use('ggplot')
x = ['High School or Below', 'College', 'Bachelor', 'Master or Above']
y = [maleDataFrame["Education"].str.contains("High School or Below").sum(),
maleDataFrame["Education"].str.contains("College").sum(),
maleDataFrame["Education"].str.contains("Bachelor").sum(),
maleDataFrame["Education"].str.contains("Master or Above").sum()]
x_pos = [i for i, _ in enumerate(x)]
plt.bar(x_pos, y, color=['blue','red','green','yellow'])
plt.xlabel("Type of Education")
plt.ylabel("Level of Education Male")
customTitle = ""
plt.title(customTitle)
plt.xticks(x_pos, x)
How can I fix this?
You can add back the line that defines y to reproduce it using the data you have. Protip: Use bar(x, y, width=30) to modify the width of the bar as per your requirement.
Modified your code to:
plt.style.use('ggplot')
x = ['High School \nor Below', 'College', 'Bachelor', 'Master or \nAbove']
y = [10, 20, 30, 40]
plt.bar(x, y, color=['blue','red','green','yellow'])
plt.xlabel("Type of Education")
plt.ylabel("Level of Education Male")
customTitle = ""
plt.title(customTitle)
To give:
I've created a plot with geom_line and geom_ribbon (image 1) and the result is okay, but for the sake of aesthetics, I'd like the line and ribbon to be smoother. I know I can use geom_smooth for the line (image 2), but I'm not sure if it's possible to smooth the ribbon.I could create a geom_smooth line for the top and bottom lines of the ribbon (image 3), but is there anyway to fill in the space between those two lines?
A principled way to achieve what you want is to fit a GAM model to your data using the gam() function in mgcv and then apply the predict() function to that model over a finer grid of values for your predictor variable. The grid can cover the span defined by the range of observed values for your predictor variable. The R code below illustrates this process for a concrete example.
# load R packages
library(ggplot2)
library(mgcv)
# simulate some x and y data
# x = predictor; y = response
x <- seq(-10, 10, by = 1)
y <- 1 - 0.5*x - 2*x^2 + rnorm(length(x), mean = 0, sd = 20)
d <- data.frame(x,y)
# plot the simulated data
ggplot(data = d, aes(x,y)) +
geom_point(size=3)
# fit GAM model
m <- gam(y ~ s(x), data = d)
# define finer grid of predictor values
xnew <- seq(-10, 10, by = 0.1)
# apply predict() function to the fitted GAM model
# using the finer grid of x values
p <- predict(m, newdata = data.frame(x = xnew), se = TRUE)
str(p)
# plot the estimated mean values of y (fit) at given x values
# over the finer grid of x values;
# superimpose approximate 95% confidence band for the true
# mean values of y at given x values in the finer grid
g <- data.frame(x = xnew,
fit = p$fit,
lwr = p$fit - 1.96*p$se.fit,
upr = p$fit + 1.96*p$se.fit)
head(g)
theme_set(theme_bw())
ggplot(data = g, aes(x, fit)) +
geom_ribbon(aes(ymin = lwr, ymax = upr), fill = "lightblue") +
geom_line() +
geom_point(data = d, aes(x, y), shape = 1)
This same principle would apply if you were to fit a polynomial regression model to your data using the lm() function.
Here is my code so far, my question is how do I make the white grid lines thicker and keep them 25 pixels apart? Please help
def grid():
picture = makeEmptyPicture(365,365,black)
w = getWidth(picture)
h = getHeight(picture)
vertical = 25
horizontal = 25
for y in range(0,h):
for x in range(0,w):
if (x % horizontal ==0 or y % vertical == 0):
px = getPixel(picture,x,y)
setColor(px, white)
show(picture)
return picture
To make your lines bigger try drawing 2 lines at the same. For example
px1 = getPixel(picture, x, y+1)
px2 = getPixel(picture, x, y+1)
setColor(px1, white)
setColor(px2, white)
This will make the line bigger along the X-axis.
As for keeping the lines 25 pixels apart try incrementing your outer loop by 26 after every iteration.
I asked this question yesterday about storing a plot within an object. I tried implementing the first approach (aware that I did not specify that I was using qplot() in my original question) and noticed that it did not work as expected.
library(ggplot2) # add ggplot2
string = "C:/example.pdf" # Setup pdf
pdf(string,height=6,width=9)
x_range <- range(1,50) # Specify Range
# Create a list to hold the plot objects.
pltList <- list()
pltList[]
for(i in 1 : 16){
# Organise data
y = (1:50) * i * 1000 # Get y col
x = (1:50) # get x col
y = log(y) # Use natural log
# Regression
lm.0 = lm(formula = y ~ x) # make linear model
inter = summary(lm.0)$coefficients[1,1] # Get intercept
slop = summary(lm.0)$coefficients[2,1] # Get slope
# Make plot name
pltName <- paste( 'a', i, sep = '' )
# make plot object
p <- qplot(
x, y,
xlab = "Radius [km]",
ylab = "Services [log]",
xlim = x_range,
main = paste("Sample",i)
) + geom_abline(intercept = inter, slope = slop, colour = "red", size = 1)
print(p)
pltList[[pltName]] = p
}
# close the PDF file
dev.off()
I have used sample numbers in this case so the code runs if it is just copied. I did spend a few hours puzzling over this but I cannot figure out what is going wrong. It writes the first set of pdfs without problem, so I have 16 pdfs with the correct plots.
Then when I use this piece of code:
string = "C:/test_tabloid.pdf"
pdf(string, height = 11, width = 17)
grid.newpage()
pushViewport( viewport( layout = grid.layout(3, 3) ) )
vplayout <- function(x, y){viewport(layout.pos.row = x, layout.pos.col = y)}
counter = 1
# Page 1
for (i in 1:3){
for (j in 1:3){
pltName <- paste( 'a', counter, sep = '' )
print( pltList[[pltName]], vp = vplayout(i,j) )
counter = counter + 1
}
}
dev.off()
the result I get is the last linear model line (abline) on every graph, but the data does not change. When I check my list of plots, it seems that all of them become overwritten by the most recent plot (with the exception of the abline object).
A less important secondary question was how to generate a muli-page pdf with several plots on each page, but the main goal of my code was to store the plots in a list that I could access at a later date.
Ok, so if your plot command is changed to
p <- qplot(data = data.frame(x = x, y = y),
x, y,
xlab = "Radius [km]",
ylab = "Services [log]",
xlim = x_range,
ylim = c(0,10),
main = paste("Sample",i)
) + geom_abline(intercept = inter, slope = slop, colour = "red", size = 1)
then everything works as expected. Here's what I suspect is happening (although Hadley could probably clarify things). When ggplot2 "saves" the data, what it actually does is save a data frame, and the names of the parameters. So for the command as I have given it, you get
> summary(pltList[["a1"]])
data: x, y [50x2]
mapping: x = x, y = y
scales: x, y
faceting: facet_grid(. ~ ., FALSE)
-----------------------------------
geom_point:
stat_identity:
position_identity: (width = NULL, height = NULL)
mapping: group = 1
geom_abline: colour = red, size = 1
stat_abline: intercept = 2.55595281266726, slope = 0.05543539319091
position_identity: (width = NULL, height = NULL)
However, if you don't specify a data parameter in qplot, all the variables get evaluated in the current scope, because there is no attached (read: saved) data frame.
data: [0x0]
mapping: x = x, y = y
scales: x, y
faceting: facet_grid(. ~ ., FALSE)
-----------------------------------
geom_point:
stat_identity:
position_identity: (width = NULL, height = NULL)
mapping: group = 1
geom_abline: colour = red, size = 1
stat_abline: intercept = 2.55595281266726, slope = 0.05543539319091
position_identity: (width = NULL, height = NULL)
So when the plot is generated the second time around, rather than using the original values, it uses the current values of x and y.
I think you should use the data argument in qplot, i.e., store your vectors in a data frame.
See Hadley's book, Section 4.4:
The restriction on the data is simple: it must be a data frame. This is restrictive, and unlike other graphics packages in R. Lattice functions can take an optional data frame or use vectors directly from the global environment. ...
The data is stored in the plot object as a copy, not a reference. This has two
important consequences: if your data changes, the plot will not; and ggplot2 objects are entirely self-contained so that they can be save()d to disk and later load()ed and plotted without needing anything else from that session.
There is a bug in your code concerning list subscripting. It should be
pltList[[pltName]]
not
pltList[pltName]
Note:
class(pltList[1])
[1] "list"
pltList[1] is a list containing the first element of pltList.
class(pltList[[1]])
[1] "ggplot"
pltList[[1]] is the first element of pltList.
For your second question: Multi-page pdfs are easy -- see help(pdf):
onefile: logical: if true (the default) allow multiple figures in one
file. If false, generate a file with name containing the
page number for each page. Defaults to ‘TRUE’.
For your main question, I don't understand if you want to store the plot inputs in a list for later processing, or the plot outputs. If it is the latter, I am not sure that plot() returns an object you can store and retrieve.
Another suggestion regarding your second question would be to use either Sweave or Brew as they will give you complete control over how you display your multi-page pdf.
Have a look at this related question.