Issue with trend line and confidence interval

Question

[pdhrati02]

Hi all, I am trying to plot a line graph (more line a trend chart for each bacteria of interest) using geom_smooth. However when using geom_smooth with se=T, I don't get any confidence interval around the line. So I tried geom_ribbon + geon_line but this gives edgy lines instead of smooth wavy ones. The code I am using is as follows: To use geom_ribbon, the lwl and upl was calculated as follows:

###
loess_mod <- loess(Abundance.mean ~ Time2, DF_Bacteroides_vulgatus)
pred <- predict(loess_mod, DF_Bacteroides_vulgatus, se=TRUE)
DF_Bacteroides_vulgatus$lwl <- pred$fit-1.96*pred$se.fit
DF_Bacteroides_vulgatus$upl <- pred$fit+1.96*pred$se.fit

##plot

Bacter <- ggplot(DF_Bacter, aes(x = Time2, y = Abundance.mean, group = Group, color = Group)) +
  geom_smooth(se=TRUE)+ 
  #stat_smooth(geom = 'line', alpha = 0.5, se = TRUE, level = 0.999, aes(ymin=Abundance.mean*0.95, ymax=Abundance.mean*1.05), fill="grey70") + 
  #geom_ribbon(aes(ymin=lwl, ymax=upl, group = Group), fill="grey70")+
  geom_point()+
  #geom_line()+
  theme_bw()+
  xlab("Timepoint (weeks)") + ylab(" abundance") + 
  ggtitle("Bacter") + 
  theme(plot.title = element_text(hjust = 0.5, size = 10), 
                                          panel.grid.major = element_blank(), 
                                          panel.grid.minor = element_blank(),
                                          axis.line = element_line(colour = "black"))

Bacter

Can someone help me with how can I obtain a smooth line graph with confidence interval for each line (group in my case)?

A minimal dataset to reproduce the above is as follows:

Group Time2 Abundance.mean lwl upl
C_b 1 0 -0.963105217808425 1.92997815898489
C_b 4 0 -1.00394435506332 1.88913902172999
C_b 8 0 0.872531880230793 3.7656152570241
C_b 24 0 -0.268735609965284 2.62434776682803
C_b 104 1.1898 0.758031565571598 3.65111494236491
C_nb 1 0 -0.963105217808425 1.92997815898489
C_nb 4 0.081532 -1.00394435506332 1.88913902172999
C_nb 8 3.47626470588235 0.872531880230793 3.7656152570241
C_nb 24 1.66671294117647 -0.268735609965284 2.62434776682803
C_nb 104 1.84441333333333 0.758031565571598 3.65111494236491
V_nb 1 1.45030941176471 -0.963105217808425 1.92997815898489
V_nb 4 1.24626 -1.00394435506332 1.88913902172999
V_nb 8 3.480956 0.872531880230793 3.7656152570241
V_nb 24 1.86670529411765 -0.268735609965284 2.62434776682803
V_nb 104 3.57950642857143 0.758031565571598 3.65111494236491

Any help would be appreciated, Thank you Best DP

Answer 1

[dariober]

The problem, I think, is that you have only a handful of data points per group and ggplot tries to fit a loess model that is too complex. In fact, you should see some warnings. Try overriding ggplot's default and use a simpler fit. For example:

Bacter <- ggplot(DF_Bacter, aes(x = Time2, y = Abundance.mean, group = Group, color = Group)) +
  geom_point() +
  geom_smooth(se=TRUE, method= 'lm')+ 
  ...etc

However, this won't look particularly good because the points are quite scattered at least for some groups.

[pdhrati02]

Hi, you are absolutely right, I am getting some warning with ggplot. Also using lm is not giving a particularly good looking plot. I will try to use all data points and see if it works or might have to use geom_ribbon I guess. Thank you for your help

Answer 2

[Jeremy]

If you can use your raw data instead of the mean, you can use the following code:

graph <- ggplot(DF_Bacter, aes(x = Time2, y = Abundance, group = Group, color = Group)) +


geom_smooth(se = TRUE)

[pdhrati02]

Hi Jeremy, thank you for your help, I tried suing the raw data as you suggested. However the prediction plotting with loess in this case gives me a plot with curve dipping to negative (below zero), but I have no negative abundance in the data. Thank you very much for your help

[Jeremy]

Try setting the method to generalized additive model (GAM).

gam = ggplot(df, aes(x = Time2, y = Abundance, group = Group, color = Group)) +


geom_smooth(method = 'gam', se = TRUE)

[pdhrati02]

Hi Jeremy, Thank you for your suggestion, I tried it but it throws several warnings and error (x has insufficient unique values to support 10 knots: reduce k) . I will try working around this and see if it works. Thank you very much once again.