Extending nlmixr2est: custom estimation methods and behavior hooks

nlmixr2est provides the estimation routines for the nlmixr2 ecosystem. This vignette is for package developers who want to plug their own behavior into that machinery. There are three extension points, from highest-level to lowest:

Add a new estimation method (est = "myMethod") via S3 dispatch.
Intercept an estimation so a standard method transparently does something extra, without a new est name.
Contribute to the objective function from compiled code (C function-pointer linkage), e.g. to add likelihood terms during a FOCEi/SAEM fit.

The lowest-level solve hooks (injecting parameters or derivative forcing into the rxode2 solve itself) live in rxode2; see its “Solve-time hooks for package developers” article.

1. Adding a new estimation method

nlmixr2() dispatches on the est string through the nlmixr2Est() S3 generic: the estimation environment is given the class c(est, "nlmixr2Est"), so est = "myMethod" calls nlmixr2Est.myMethod(). A method needs three pieces:

A control constructor myMethodControl(...) returning a classed list.
getValidNlmixrCtl.myMethod(control) – validate/coerce the user’s control (called before dispatch).
nlmixr2Est.myMethod(env, ...) – do the fit and return an nlmixr2FitData.

By the time your method runs, the estimation environment env carries everything you need:

`env` slot	contents
`env$ui`	the model (an `rxUi`); `env$ui$iniDf` has the estimates
`env$data`	the (validated) data.frame
`env$control`	your validated control
`env$table`	the `tableControl()` for output tables
`class(env)[1]`	the est name (`"myMethod"`)
`env$.nlmixr2Dots`	any extra named `nlmixr2()` arguments (see section 2)

The existing methods are the best templates – a population optimizer such as bobyqa (R/bobyqa.R) is the simplest, and saem/focei show the full pattern. Skeleton:

myMethodControl <- function(maxeval = 1000L, ...) {
  .ctl <- list(maxeval = as.integer(maxeval), ...)
  class(.ctl) <- "myMethodControl"
  .ctl
}

#' @exportS3Method nlmixr2est::getValidNlmixrCtl
getValidNlmixrCtl.myMethod <- function(control) {
  .ctl <- control[[1]]
  if (is.null(.ctl)) .ctl <- myMethodControl()
  if (!inherits(.ctl, "myMethodControl")) {
    stop("est = 'myMethod' needs control = myMethodControl(...)", call. = FALSE)
  }
  .ctl
}

#' @exportS3Method nlmixr2est::nlmixr2Est
nlmixr2Est.myMethod <- function(env, ...) {
  .ui   <- env$ui
  .data <- env$data
  .ctl  <- env$control
  ## ... run the fit, build the result ...
  ## return an nlmixr2FitData (see nlmixr2CreateOutputFromUi / the existing methods)
}

Optional method attributes (mirroring the built-ins) advertise capabilities to the dispatcher, e.g. attr(nlmixr2Est.myMethod, "covPresent") <- TRUE.

nlmixr2AllEst() lists every registered method, so once your package is loaded est = "myMethod" is available like any built-in.

2. Intercepting an estimation

Sometimes you do not want a new est; you want a model fit with a standard estimator to transparently do something extra. For example, nlmixr2nn lets a model that contains a neural-network term be fit with est = "focei" and trains the network as a side effect – with no est = "nn".

Register an interceptor – a function(env) consulted at the top of nlmixr2Est(), before the standard method dispatches:

nlmixr2est::registerEstInterceptor("myPkg", function(env) {
  if (!shouldClaim(env$ui)) return(NULL)         # decline -> normal fit proceeds
  ## claim it: do the work and return a finalized nlmixr2FitData
  runMyThing(env)
})

Returning NULL declines – the next interceptor, then the ordinary method, runs. Returning a fit claims the estimation.
Interceptors are re-entrancy guarded: while a claimed interceptor runs, interceptors are suppressed, so it may call nlmixr2() internally (e.g. with the requested est) and get the ordinary method.
Register in your package’s .onLoad() and removeEstInterceptor("myPkg") in .onUnload().

Passing options to your interceptor

Any extra named argument to nlmixr2() is stashed on env$.nlmixr2Dots, so an interceptor can be configured without wrapping the standard control. nlmixr2nn uses this so nn = nnControl(...) rides alongside the ordinary est/control:

nlmixr2(model, data, "focei", foceiControl(), nn = nnControl(...))
#                                             ^ arrives as env$.nlmixr2Dots$nn

3. Contributing to the objective (compiled linkage)

The deepest hook lets a compiled plugin add terms to the FOCEi/SAEM objective during the fit – per observation, inside the parallel subject loop, with no R callback (R is not thread-safe). This is how nlmixr2nn feeds a network’s objective cotangents into a running FOCEi fit.

nlmixr2est exposes a registry of C callables via a function-pointer table (nlmixr2est::.nlmixr2estLikContribPtrs()), consumed downstream through the header inst/include/nlmixr2estLikContrib.h. A contributor supplies plain-C entry points (a per-subject begin/obs/end bundle and/or an EM per-subject log-likelihood) that read the per-observation prediction f, residual variance R, and dv, and add into the accumulators (llik, dLL/deta, …). Registration is via nlmixrRegisterLikContrib() / nlmixrRegisterEmLik(); the registry is cycled in series, so several packages can contribute at once.

Because the registry holds raw function pointers, the lifecycle matters in both directions: install the pointer table on every .onLoad() (a reloaded nlmixr2est hands back new addresses), and unregister every bundle in your .onUnload() – nlmixr2est cannot tell that your DLL has gone away, so a bundle left behind is called on the next objective evaluation.

This is a C-linkage API rather than an R one – see inst/include/nlmixr2estLikContrib.h for the exact struct and the nlmixr2nn package (src/*Contrib.c) for a worked consumer. The mechanism follows rxode2’s function-pointer-table convention described in the rxode2 hooks article.

1. Adding a new estimation method

2. Intercepting an estimation

Passing options to your interceptor

3. Contributing to the objective (compiled linkage)

See also