Generate data and Fit using Simple PopPK Model¶
PoPy provides a method to simulate, analyse and compare results in a single script, which is ideal for generating tutorials or illustrative examples. Here we will demonstrate a Tut Script using the same compartment model as used in Fitting a Simple PopPK Model using PoPy, see Figure Fig. 3:-
Note
See the Simple Tut Example obtained by the PoPy developers for this example, including input script and output data file.
A Tut Script can be used as a theoretical tool to investigate identifiability of PK/PD models, because the true f[X]
parameters and underlying structure of the data are known. Unfortunately this is never the case in a real life analysis.
This documentation makes extensive use of tut_scripts to create examples to illustrate different Principles of Pharmacokinetics.
Running the Tutorial Script¶
This tutorial example requires a single input file:-
c:\PoPy\examples\tut_example1.pyml
Open a PoPy Command Prompt to set up the PoPy environment in this folder:-
c:\PoPy\examples\
With the PoPy environment enabled, you can open the script using:-
$ popy_edit tut_example1.pyml
Again, with the PoPy environment enabled, call popy_run on the Tut Script from the command line:-
$ popy_run tut_example1.pyml
When the tut script has completed, you can view the output of the fit using popy_view, by typing the following command:-
$ popy_view tut_example1.pyml.html
Note the extra ‘.html’ extension in the above command. This command opens a local .html file in your web browser to summarise the result of the generating process.
You can compare your local html output with the pre-computed documentation output, see Simple Tut Example. You should expect some minor numerical differences when comparing results with the documentation.
Syntax of Tut Script¶
The major structural difference between a Gen Script or Fit Script and a Tut Script is that the tut_script has separate GEN_EFFECTS and FIT_EFFECTS sections to describe both the generating and fitting effects. The GEN_EFFECTS section for this tutorial example is as follows:-
GEN_EFFECTS:
POP: |
c[AMT] = 100.0
f[KE] = 0.1
f[PNOISE] = 0.05
f[KE_isv] = 0.03
ID: |
c[ID] = sequential(20)
t[DOSE] = 1.0
t[OBS] ~ unif(1.0, 50.0; 5)
r[KE] ~ norm(0, f[KE_isv])
The FIT_EFFECTS section for this tutorial example is as follows:-
FIT_EFFECTS:
POP: |
f[KE] ~ unif(0.001, 100) 0.05
f[PNOISE] ~ unif(0.001, 100) 0.1
f[KE_isv] ~ unif(0.001, 100) 0.1
ID: |
r[KE] ~ norm(0, f[KE_isv])
The GEN_EFFECTS get copied into the Gen Script and renamed EFFECTS. Similarly the FIT_EFFECTS get copied into the Fit Script and also renamed EFFECTS. From the examples above you can see that the GEN_EFFECTS->POP section has:-
f[X] = true_value
Whereas the FIT_EFFECTS->POP section has:-
f[X] ~ unif(0.001, 100) starting_value
Reflecting the fact that the f[X]
are known constants for a Gen Script, but are unknown values to be estimated in a Fit Script, with lower and upper limits of [0.001, 100]. Note in PoPy you can also use ‘P’ meaning positive, as a shortcut for ‘~unif(0.0,+inf)’.
The GEN_EFFECTS->POP level has this extra line:-
c[AMT] = 100.0
This sets the dose amount to be 100.0 units for all individuals.
The GEN_EFFECTS->ID level also contains these extra lines:-
c[ID] = sequential(20)
t[DOSE] = 1.0
t[OBS] ~ unif(1.0, 50.0; 5)
These lines are passed to the Gen Script and generate 20 individuals all with a dose at time 1.0 and 5 observations uniformally sampled in the time interval [1.0, 50.0]. See EFFECTS with two levels from a gen_script for more information on the Gen Script syntax within a EFFECTS section.
Summary of Tut Results¶
See Simple Tut Example for example HTML outputs generated by the PoPy developers.
On your local machine, the Tut Script generates an output folder containing four new scripts:-
simple_tut_example.pyml_output/
simple_tut_example_gen.pyml
simple_tut_example_fit.pyml
simple_tut_example_comp.pyml
simple_tut_example_tutsum.pyml
See Files Generated by Tut Script for more info. The purpose of each of theses scripts is as follows:-
Script | Purpose | Documentation |
---|---|---|
*_gen.pyml | Generate synthetic data set from model | Gen Script |
*_fit.pyml | Fit model to synthetic data set | Fit Script |
*_comp.pyml | Compare gen model and fit model to synthetic data set | Comp Script |
*_tutsum.pyml | Summary of generating and fitting and comparison results | TutSum Script |
These four scripts are run in order.
The Gen Script uses a EFFECTS structure similar to a Fit Script, but with some extra commands to generate new data rows, see Syntax of Tut Script above. Alternatively see Generate a Two Compartment PopPK Data Set, for a longer explanation of how a Gen Script works.
The Fit Script here is very similar to the PK/PD model described in Fitting a Simple PopPK Model using PoPy. Therefore here we will focus on the Comp Script outputs. To generate the comp output, you need this entry in your Tut Script OUTPUT_SCRIPTS section:-
OUTPUT_SCRIPTS:
COMP: {output_mode: run}
Otherwise the TutSum Script will have no comp output to summarise. The comp outputs are PK curves from the fitted and generated f[X]
parameters and the associated objective function values. The simplest Comp Script output is a visual comparison of the true and fitted f[X]
PK curves and the synthetic generated data, see Table 13.
The solid blue lines in Table 13 show the predicted PK curves for the fitted model f[X]
values. The dotted blue lines show the PK curves for the true f[X]
values that were used to generated the data set (in the Gen Script). The blue dots are the target c[DV_CENTRAL]
values from the data file.
The target c[DV_CENTRAL]
values have measurement noise added, so blue dot data points do not lie exactly on the true f[X]
curves. The graphs show that the PK curves for the fitted f[X]
are almost identical to the true f[X]
curves, this is to be expected as the model only contains a single model parameter m[KE]
and we have 5 observations per individual.
If the Comp Script has been run, the TutSum Script outputs convenient tables to compare the initial, fitted and true f[X]
values, see Table 14, Table 15 and Table 16.
Name | Initial | Fitted | True | Abs. Error | Prop. Error |
---|---|---|---|---|---|
f[KE] | 0.05 | 0.0999 | 0.1 | 1.30e-04 | 0.13% |
Name | Initial | Fitted | True | Abs. Error | Prop. Error |
---|---|---|---|---|---|
f[KE_isv] | 0.1 | 0.0183 | 0.03 | 1.17e-02 | 39.01% |
Name | Initial | Fitted | True | Abs. Error | Prop. Error |
---|---|---|---|---|---|
f[PNOISE] | 0.1 | 0.0503 | 0.05 | 3.36e-04 | 0.67% |
Table 14 shows that the f[KE]
parameter is recovered reasonably well, in the sense that the fitted value 0.106 is close to the true generating value 0.1 starting from an initial value of 0.05. Similarly the fitted f[KE_isv]
and f[PNOISE]
parameters are close to the true generating values.
The objective function Comp Script computes the objective function given the synthetic data and the true generating f[X]
parameters (the r[X]
are re-optimised). In this case the true f[X]
ObjV is:-
-44.20
The Comp Script also computes he ObjV for the fitted f[X]
and optimised r[X]
, which is as follows:-
-48.43
The lower objective value for the fitted f[X]
is quite common, because the fitted f[X]
can take advantage of noise in the generated synthetic data set. If the size of the synthetic data set is increased, then it is likely that the f[KE]
, f[KE_isv]
and f[PNOISE]
parameters will move closer to the true generating values and the ObjVs will also converge.
In this simple example the parameters are very easy to identify. For a more challenging example see Generate data and Fit using a Two Compartment Model.
Generate multiple data sets and Fit using Simple PopPK Model¶
The tutorial example above generates a single data set from user specified true f[X]
values. In PoPy it is possible to generalise this approach and sample true f[X]
values multiple times to create multiple data sets. Then fit the same model to each data set.
In this section we walk through how to generate multiple data sets using a MTut Script, using the same simple one compartment model as shown in Fig. 3.
Running the MTut Script¶
This multi tutorial example makes use of a single script file:-
c:\PoPy\examples\mtut_example1.pyml
Open a PoPy Command Prompt to setup the PoPy environment in this folder:-
c:\PoPy\examples\
With the PoPy environment enabled, you can open the script using:-
$ popy_edit mtut_example1.pyml
Again, with the PoPy environment enabled, call popy_run on the MTut Script from the command line:-
$ popy_run mtut_example1.pyml
Running a MTut Script can take a considerable amount of time, as it is equivalent to running a Tut Script multiple times. However in this toy example we only run the fit/gen cycle 30 times and only a small number of the f[X]
parameters are estimated.
Syntax of MTut Script¶
The MTut Script specifies the number of populations to sample as follows:-
OUTPUT_OPTIONS: {n_pop_samples: 30}
The MTut Script encodes both the data generation and fitting in the GEN_EFFECTS and FIT_EFFECTS sections, like a Tut Script. The syntax is the same. In this example the GEN_EFFECTS section is as follows:-
GEN_EFFECTS:
POP: |
c[AMT] = 100.0
# f[KE] = 0.1
f[KE] ~ unif(0.05,0.15)
# f[PNOISE] = 0.05
f[PNOISE] ~ unif(0.02,0.08)
# f[KE_isv] = 0.03
f[KE_isv] ~ unif(0.01,0.05)
ID: |
c[ID] = sequential(20)
t[DOSE] = 1.0
t[OBS] ~ unif(1.0, 50.0; 5)
r[KE] ~ norm(0, f[KE_isv])
And the FIT_EFFECTS section is as follows:-
FIT_EFFECTS:
POP: |
# f[KE] ~ unif(0.001, 100) 0.05
f[KE] ~ P 0.1
# f[PNOISE] ~ unif(0.001, 100) 0.1
f[PNOISE] ~ P 0.05
# f[KE_isv] ~ unif(0.001, 100) 0.1
f[KE_isv] ~ P 0.03
ID: |
r[KE] ~ norm(0, f[KE_isv])
Here the generated values of f[KE]
, f[PNOISE]
and f[KE_isv]
are sampled from uniform distributions. The fitting process is initialised with (constant) values in the centre of the uniform distribution, used to sample the generating fixed effect values.
Summary of MTut Results¶
The MTut Script should generate an output folder containing three new scripts:-
mtut_example1.pyml_output/
mtut_example1_mgen.pyml
mtut_example1_mfit.pyml
mtut_example1_mcomp.pyml
The purpose of each of theses scripts is as follows:-
Script | Purpose | Documentation |
---|---|---|
*_mgen.pyml | Generate multiple synthetic data sets from model | MGen Script |
*_mfit.pyml | Fit model to multiple synthetic data sets | MFit Script |
*_mcomp.pyml | Compare gen model and fit model f[X] |
MComp Script |
The MGen Script is very similar to the Gen Script described in Syntax of Tut Script and the MFit Script is very similar to the Fit Script described in Fitting a Simple PopPK Model using PoPy. See Files Generated by MTut Script for more info.
Here we mainly discuss the MComp Script outputs, which processes the results of MGen Script and MFit Script. The simplest output is a visual comparison of the true and fitted f[X]
values as shown in Table 18.
In Table 18 the blue dots are a scatter plot of fitted f[X]
vs true f[X]
. The green dots are initial f[X]
vs true f[X]
. For example in the case of f[KE]
the true values are sampled as follows:-
f[KE] ~ unif(0.05,0.15)
i.e. the true values are uniformly sampled in the range [0.05,0.15]. The fitted f[KE]
parameters are modelled as follows:-
f[KE] ~ P0.1
The initial values for f[KE]
are always 0.1, see green dots in a horizontal line on the left graph in Table 18. The ‘P’ specifies that the fitting value of f[KE]
is restricted to positive numbers. The final fitting values are the blue dots on the left graph in Table 18. For f[KE]
the blue dots are clustered along the black 45 degree line. Hence fitting for f[KE]
works well, this agrees with the initial findings in Fitting a Simple PopPK Model using PoPy. The blue dots for f[KE_isv]
and f[PNOISE]
(centre and right graphs in Table 18) are not as tightly clustered around the 45 degree line, indicating these parameters are harder to identify than f[KE]
. However both f[KE_isv]
and f[PNOISE]
show a reasonable correlation between the true and fitted values.
Note it might well be possible to carry out a more statistical analysis of correlation between the true and fitted f[X]
, for example finding a line of best fit through the scatter plot data. The MComp Script outputs are all saved to .csv files, see Files Generated by MComp Script, which could easily be loaded into R or other statistical packages for further analysis.
Next Steps¶
You can see more complicated examples of using Tut Script and MTut Script:-
- Generate data and Fit using a Two Compartment Model
- Generate multiple data sets and Fit using a Two Compartment Model
Or see Typical Workflows for a general overview of using PoPy for PK/PD modelling.