.. _create_scripts:

Creating Your First |popy| Scripts
###################################

Some |popy| commands (|eg| ``info``, ``validate``) take no arguments; 
others (|eg| ``activate``) take a single specific argument; 
the most important commands, however, operate on a structured text file
that we refer to as a *script*
and that implements a specific *action* 
(|eg| simulate data, fit model parameters, plot graphs).

This script contains metadata about the experiment 
along with a definition of the mathematical model 
and parameters of the code that will operate on it (|eg| optimizer tolerances).

To ease in the creation of these scripts |popy| comes with 
a command, ``create``, that creates a template script for each action 
and that you can customize to suit your needs.

.. code-block:: console

    popy create <action> <filename>

where ``<action>`` is the action whose template script you want to create,
and ``<filename>`` is the name of the file to create.

These built in scripts are designed to get you started with using a 
particular type of |script_file|. 
    
See :ref:`popy_create` documentation.


Creating a ``fit`` Script
============================

One of the actions that |popy| provides is ``fit``
that estimates ('fits') model parameters given a model and observed data. 
To create a ``fit`` script, :ref:`open_a_popy_command_prompt` and type:

.. code-block:: console

    popy create fit my_first_fit_script.pyml

where

* :ref:`popy_create` is one of the :ref:`popy_tools`
* the ``fit`` option requests a :ref:`fit_script` to be created and 
* the :ref:`fit_script` written to a file called ``my_first_fit_script.pyml``.

This will create a brief script that is short but functional.
Pass additional arguments to ``popy create``
to create a more verbose example script that 
includes all possible options (``-a``)
with comments (``-c``) that explain each option
and added spaces (``-s``) and line breaks (``-l``) for an easier read:

.. code-block:: console

    popy create fit my_first_fit_script.pyml -acsl

(See :ref:`popy_create` or type ``popy_create -h`` 
for an explanation of the available command line arguments.)


Message Logging
================

You may notice that, in addition to the requested file, 
:ref:`popy_create` also creates a file called 
``my_first_fit_script.pyml.create.main.log``.
This text file contains a record of the messages output 
by the :ref:`popy_create` tool which, in this case, is very short. 

All of the :ref:`popy_tools` create between one and three log files
and these files act as part of the audit trail for your experiments.

* The ``main`` log file is always created and contains messages generated as part of the normal running process
* A ``warning`` log file contains details of warnings (if any occurred) that were raised and that suggest changes to the script might be beneficial
* An ``error`` log file contains details of errors (if any occurred) that prevented |popy| from completing its operation

For example, if you try running the created fit_script:

.. code-block:: console

    popy run my_fit_script.pyml

the output on the screen should display an error:

.. code-block::

    CAST_ERROR= ERROR in value_record: ROOT->FILE_PATHS->input_data_file
    ERROR when casting input file element
    input file path = <current working directory>/my_pkpd_model_gen.pyml_output/generated_data/cx_obs_params.csv
    NOT present on file system.
    
telling you that the data to which the model should be fitted cannot be found
in the expected location.

Furthermore, the ``my_first_fit_script.pyml.run.main.log`` file 
will declare that an error has occurred,
and a new ``my_first_fit_script.pyml.run.error.log`` file will include details
of the error that may be useful for debugging.

The amount of detail given in log files can be controlled with another
command line argument (``-v``) to each |popy| command that supports it.


Creating a ``tut`` Script
============================

The created ``fit`` script did not run because there was no data to which 
we could fit the model.

We could solve this problem using another |popy| action, ``gen``, that
generates synthetic data from a mathematical model and user-given parameters
that define a population of individuals. For learning purposes, however,
|popy| also comes with a "tutorial" action (``tut``) that completes four steps:

#. Create and run a ``gen`` script to generate a population of individuals 
   and simulate observations from a mathematical model
#. Create and run a ``fit`` script that estimates the (known) model parameters
   from (inaccurate) initial estimates and the simulated data
#. Create and run a ``comp`` script that compares the estimated 
   model parameters with the known parameters used to simulate the data
#. Create and run a ``tutsum`` script that summarizes the results of the 
   experiment as an HTML report for easy reading

In this way, ``tut`` demonstrates almost all of |popy|'s capabilities in a 
single script for ease of learning, and you should create one now by running

.. code-block:: console

    popy create tut my_first_tut_script.pyml

in the |popy| terminal. 

Because the ``tut`` action contains all of the steps
we need, you should be able to run the script to completion straight away
with no reported errors:

.. code-block:: console

    popy run my_first_tut_script.pyml

(As an aside, note that the created script contains the action it performs in 
its metadata so you do not need to specify the action to ``popy run``, only
to ``popy create`` where the action is not yet known.)

Let us now look closer at what is happening "under the hood" and the results
that a ``tut`` script produces.
