Diagnosing errors

library(bycatch)
library(loo)
set.seed(123)

In some cases, models might produce warnings about high Pareto K statistics (> 0.7). These may reflect a model that is too flexible for the data being used, in which case the model might need to be changed. In some cases, ‘moment matching’ may be used to improve diagnostics; an example of this is below.

Load data

# replace this with your own data frame
d = data.frame("Year"= 2002:2014, 
  "Takes" = c(0, 0, 0, 0, 0, 0, 0, 0, 1, 3, 0, 0, 0),
  "expansionRate" = c(24, 22, 14, 32, 28, 25, 30,  7, 26, 21, 22, 23, 27),
  "Sets" = c(391, 340, 330, 660, 470, 500, 330, 287, 756, 673, 532, 351, 486))

Simple model with constant bycatch, no covariates

This model has a constant bycatch rate and Poisson distribution,

fit = fit_bycatch(Takes ~ 1, data=d, time="Year", effort="Sets", family="poisson",
  time_varying = FALSE)

If the ‘fit’ object produced errors about the Pareto K statistics being high, we could use moment matching in the loo package,

loo_stats <- loo(fit$fitted_model, moment_match = TRUE)

## Warning: Some Pareto k diagnostic values are slightly high. See help('pareto-k-diagnostic') for details.

print(loo_stats)

## 
## Computed from 1500 by 13 log-likelihood matrix
## 
##          Estimate   SE
## elpd_loo    -11.0  5.8
## p_loo         1.7  1.3
## looic        22.0 11.6
## ------
## Monte Carlo SE of elpd_loo is 0.1.
## 
## Pareto k diagnostic values:
##                          Count Pct.    Min. n_eff
## (-Inf, 0.5]   (good)     12    92.3%   381       
##  (0.5, 0.7]   (ok)        1     7.7%   473       
##    (0.7, 1]   (bad)       0     0.0%   <NA>      
##    (1, Inf)   (very bad)  0     0.0%   <NA>      
## 
## All Pareto k estimates are ok (k < 0.7).
## See help('pareto-k-diagnostic') for details.

Eric J. Ward, Jason E. Jannot

2023-07-05

Load data

Simple model with constant bycatch, no covariates