.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "auto_examples/bayesian_swissmetro/plot_b04_validation.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        :ref:`Go to the end <sphx_glr_download_auto_examples_bayesian_swissmetro_plot_b04_validation.py>`
        to download the full example code.

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_auto_examples_bayesian_swissmetro_plot_b04_validation.py:


4. Out-of-sample validation
===========================

Example of the out-of-sample validation of a logit model in a Bayesian estimation context.

Michel Bierlaire, EPFL
Thu Oct 30 2025, 16:40:12

.. GENERATED FROM PYTHON SOURCE LINES 11-18

.. code-block:: Python


    from biogeme.bayesian_estimation import BayesianResults
    from biogeme.biogeme import BIOGEME
    from biogeme.expressions import Beta
    from biogeme.models import loglogit
    from biogeme.validation import ValidationResult








.. GENERATED FROM PYTHON SOURCE LINES 19-20

See the data processing script: :ref:`swissmetro_data`.

.. GENERATED FROM PYTHON SOURCE LINES 20-34

.. code-block:: Python

    from swissmetro_data import (
        CAR_AV_SP,
        CAR_CO_SCALED,
        CAR_TT_SCALED,
        CHOICE,
        SM_AV,
        SM_COST_SCALED,
        SM_TT_SCALED,
        TRAIN_AV_SP,
        TRAIN_COST_SCALED,
        TRAIN_TT_SCALED,
        database,
    )








.. GENERATED FROM PYTHON SOURCE LINES 35-36

Parameters to be estimated.

.. GENERATED FROM PYTHON SOURCE LINES 36-42

.. code-block:: Python

    asc_car = Beta('asc_car', 0, None, None, 0)
    asc_train = Beta('asc_train', 0, None, None, 0)
    asc_sm = Beta('asc_sm', 0, None, None, 1)
    b_time = Beta('b_time', 0, None, None, 0)
    b_cost = Beta('b_cost', 0, None, None, 0)








.. GENERATED FROM PYTHON SOURCE LINES 43-44

Definition of the utility functions.

.. GENERATED FROM PYTHON SOURCE LINES 44-48

.. code-block:: Python

    v_train = asc_train + b_time * TRAIN_TT_SCALED + b_cost * TRAIN_COST_SCALED
    v_swissmetro = asc_sm + b_time * SM_TT_SCALED + b_cost * SM_COST_SCALED
    v_car = asc_car + b_time * CAR_TT_SCALED + b_cost * CAR_CO_SCALED








.. GENERATED FROM PYTHON SOURCE LINES 49-50

Associate utility functions with the numbering of alternatives.

.. GENERATED FROM PYTHON SOURCE LINES 50-52

.. code-block:: Python

    v = {1: v_train, 2: v_swissmetro, 3: v_car}








.. GENERATED FROM PYTHON SOURCE LINES 53-54

Associate the availability conditions with the alternatives.

.. GENERATED FROM PYTHON SOURCE LINES 54-56

.. code-block:: Python

    av = {1: TRAIN_AV_SP, 2: SM_AV, 3: CAR_AV_SP}








.. GENERATED FROM PYTHON SOURCE LINES 57-59

Definition of the model. This is the contribution of each
observation to the log likelihood function.

.. GENERATED FROM PYTHON SOURCE LINES 59-61

.. code-block:: Python

    logprob = loglogit(v, av, CHOICE)








.. GENERATED FROM PYTHON SOURCE LINES 62-63

Create the Biogeme object.

.. GENERATED FROM PYTHON SOURCE LINES 63-66

.. code-block:: Python

    the_biogeme = BIOGEME(database, logprob)
    the_biogeme.model_name = 'b04validation'








.. GENERATED FROM PYTHON SOURCE LINES 67-68

Estimate the parameters.

.. GENERATED FROM PYTHON SOURCE LINES 68-75

.. code-block:: Python

    try:
        results = BayesianResults.from_netcdf(
            filename=f'saved_results/{the_biogeme.model_name}.nc'
        )
    except FileNotFoundError:
        results = the_biogeme.bayesian_estimation()





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    load finished in 4385 ms (4.39 s)




.. GENERATED FROM PYTHON SOURCE LINES 76-84

The validation consists in organizing the data into several slices
of about the same size, randomly defined. Each slice is considered
as a validation dataset. The model is then re-estimated using all
the data except the slice, and the estimated model is applied on the
validation set (i.e. the slice). The value of the log likelihood for
each observation in the validation set is reported in a
dataframe. As this is done for each slice, the output is a list of
dataframes, each corresponding to one of these exercises.

.. GENERATED FROM PYTHON SOURCE LINES 84-91

.. code-block:: Python

    validation_results: list[ValidationResult] = the_biogeme.validate(results, slices=5)

    for slide in validation_results:
        print(
            f'Log likelihood for {slide.simulated_values.shape[0]} validation data: '
            f'{slide.simulated_values.iloc[:, 0].sum()}'
        )




.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    Log likelihood for 1354 validation data: -1045.4505746905775
    Log likelihood for 1354 validation data: -1071.522163749923
    Log likelihood for 1354 validation data: -1073.554194770125
    Log likelihood for 1353 validation data: -1053.902652288213
    Log likelihood for 1353 validation data: -1097.3034038689275





.. rst-class:: sphx-glr-timing

   **Total running time of the script:** (0 minutes 30.493 seconds)


.. _sphx_glr_download_auto_examples_bayesian_swissmetro_plot_b04_validation.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: plot_b04_validation.ipynb <plot_b04_validation.ipynb>`

    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: plot_b04_validation.py <plot_b04_validation.py>`

    .. container:: sphx-glr-download sphx-glr-download-zip

      :download:`Download zipped: plot_b04_validation.zip <plot_b04_validation.zip>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_