Residuals¶

Residuals - the difference between a predicted quantity, such as concentration, and its observed value - indicate the error (or noise) model on the observations, for which there are three popular choices.

Table 23 Residuals vs IPRED for the additive (left), proportional (centre) and additive+proportional (right) noise models¶

Additive

Proportional

Additive+Proportional

In the Additive Noise model, noise has a fixed standard deviation (or variance) that is independent of the predicted value.

In the Proportional Noise model, the level (variance) of noise increases with the magnitude of the predicted variable.

In the Additive+Proportional Noise model, there is an element of both: additive noise that ensures valid likelihoods when the predicted variable is zero; and a proportional component that better reflects a prediction-dependent noise level.

Weighted Residuals (WRES)¶

To achieve a more intuitive and easy-to interpret diagnostic, we can scale residuals by the variance we expect given the predicted value.

Table 24 Weighted Residuals vs PRED for the additive (left), proportional (centre) and additive+proportional noise models¶

Additive

Proportional

Additive+Proportional

Residuals should have a mean of zero and a variance of one when normalized in this way, and if they do not follow the standard normal distribution this could be an indication that the model is misspecified. (Higher order moments of the distribution such as skew and kurtosis should also be close to their standardized values of zero and three, respectively.)

Conditional Weighted Residuals¶

It was noted by Hooker et al. [HookerEtAl2007], however, that WRES can sometimes imply that a misspecified model is better than a correctly specified one. This problem arises because (in Nonmem) the WRES are computed from the FO statistics even when FOCE was used to fit the model to the data.

Here we reproduce Hooker’s example model in which the response of the disease to the drug is limited through an Emax equation, generating data for a population of 200 individuals with fixed observation time points and inter-subject variability in Emax and C50:

GEN_EFFECTS:
    POP: |
        f[EMAX] = 100
        f[C50] = 20
        f[Hill] = 4.5
    ID: |
        c[ID] = sequential(200)
        t[OBS] = [ 0.001, 1.0, 2.0, 4.0, 8.0, 16.0, 32.0, 64, 96, 120, 150, 180, 240, 320, 420, 525 ]
        f[EMAX_isv] = 0.5
        r[EMAX_isv] ~ norm(0, f[EMAX_isv])
        f[C50_isv] = 0.5
        r[C50_isv] ~ norm(0, f[C50_isv])

In order to compare the input concentration with the Emax response for fixed drug concentrations, we simply treat the observation times as if they were drug concentrations in the PREDICTIONS block and assign the corresponding Emax response to the output column, c[Effect]:

PREDICTIONS: |
    p[Drug] = c[TIME]
    p[Effect] = m[EMAX] * p[Drug]**m[Hill]/(p[Drug]**m[Hill] + m[C50]**m[Hill])
    effect_var = m[ANOISE_var]
    c[Effect] ~ norm(p[Effect], effect_var)

We then apply a PoPy fit for two models: one with a correctly specified Hill coefficient of 4.5 (sum_link_cwres_emax_correct_model_tut),

FIT_EFFECTS:
    POP: |
        f[EMAX] ~ P 100
        f[C50] ~ P 20
        f[Hill] ~ P 4.5  # Correct Hill coefficient
    ID: |
        f[EMAX_isv] ~ P 0.5
        r[EMAX_isv] ~ norm(0, f[EMAX_isv])
        f[C50_isv] ~ P 0.5
        r[C50_isv] ~ norm(0, f[C50_isv])

and one with a misspecified Hill coefficient of 1.0 (sum_link_cwres_emax_misspecified_model_tut).

FIT_EFFECTS:
    POP: |
        f[EMAX] ~ P 100
        f[C50] ~ P 20
        f[Hill] = 1.0  # Incorrect Hill coefficient
    ID: |
        f[EMAX_isv] ~ P 0.5
        r[EMAX_isv] ~ norm(0, f[EMAX_isv])
        f[C50_isv] ~ P 0.5
        r[C50_isv] ~ norm(0, f[C50_isv])

Table 25 Fitted predictions for the correctly specified (left) and misspecified (right) models¶

Correctly specified model

Misspecified model

The figure above shows the predictions fitted to the data, where it is clear that the correctly specified model has achieved a better fit to the data.

As noted by Hooker, however, the popular WRES diagnostic for the misspecified model has a distribution whose parameters are closer to the standard normal (mean and skew of zero, standard deviation of one, kurtosis of three) compared with the correctly specified model after fitting with FOCE:

Table 26 WRES vs. PRED for the correctly specified (left) and misspecified (right) models¶

Correctly specified model

Misspecified model

To address this anomaly, Hooker suggested a new diagnostic - the Conditional Weighted Residual (CWRES) - that translated and scaled residuals with values more suited to an FOCE-fitted model.

When using this diagnostic rather than WRES, the moments of the error distribution suggest that the correctly specified model is a better fit to the data than the misspecified mode (with the exception of Standard Deviation which is close to one in both cases):

Table 27 CWRES vs IPRED for the correctly specified (left) and misspecified (right) models¶

Correctly specified model

Misspecified model

We therefore suggest using CWRES rather than WRES for diagnostics by requesting this plot (‘{CWRES}_vs_IPRED’) in the OUTPUT_SCRIPTS block of the model file:

OUTPUT_SCRIPTS:
    GEN: {output_mode: run, grph_list: []}
    FIT:
        output_mode: run
        grph_list: ['{OBS}_vs_TIME', '{WRES}_vs_PRED', '{CWRES}_vs_IPRED']
    COMP: {output_mode: run}
    TUTSUM: {output_mode: run}