.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "auto_examples/programmers/plot_optimization.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code. .. rst-class:: sphx-glr-example-title .. _sphx_glr_auto_examples_programmers_plot_optimization.py: biogeme.optimization ==================== Examples of use of several functions. This is designed for programmers who need examples of use of the functions of the module. The examples are designed to illustrate the syntax. They do not correspond to any meaningful model. Michel Bierlaire Sun Jun 29 2025, 18:10:12 .. GENERATED FROM PYTHON SOURCE LINES 15-38 .. code-block:: Python from IPython.core.display_functions import display import biogeme.biogeme_logging as blog from biogeme.biogeme import BIOGEME from biogeme.data.swissmetro import ( CAR_AV_SP, CAR_CO_SCALED, CAR_TT_SCALED, CHOICE, SM_AV, SM_COST_SCALED, SM_TT_SCALED, TRAIN_AV_SP, TRAIN_COST_SCALED, TRAIN_TT_SCALED, read_data, ) from biogeme.expressions import Beta from biogeme.models import loglogit from biogeme.results_processing import get_pandas_estimated_parameters from biogeme.version import get_text .. GENERATED FROM PYTHON SOURCE LINES 39-40 Version of Biogeme. .. GENERATED FROM PYTHON SOURCE LINES 40-42 .. code-block:: Python print(get_text()) .. rst-class:: sphx-glr-script-out .. code-block:: none biogeme 3.3.1 [2025-09-03] Home page: http://biogeme.epfl.ch Submit questions to https://groups.google.com/d/forum/biogeme Michel Bierlaire, Transport and Mobility Laboratory, Ecole Polytechnique Fédérale de Lausanne (EPFL) .. GENERATED FROM PYTHON SOURCE LINES 43-45 The logger sets the verbosity of Biogeme. By default, Biogeme is quite silent and generates only warnings. To have more information about what it happening behind the scene, the level should be set to `blog.INFO`. .. GENERATED FROM PYTHON SOURCE LINES 45-47 .. code-block:: Python logger = blog.get_screen_logger(level=blog.INFO) .. GENERATED FROM PYTHON SOURCE LINES 48-49 Read the data file .. GENERATED FROM PYTHON SOURCE LINES 49-50 .. code-block:: Python the_data = read_data() .. GENERATED FROM PYTHON SOURCE LINES 51-52 Parameters to be estimated: alternative specific constants .. GENERATED FROM PYTHON SOURCE LINES 52-55 .. code-block:: Python asc_car = Beta('asc_car', 0, None, None, 0) asc_train = Beta('asc_train', 0, None, None, 0) .. GENERATED FROM PYTHON SOURCE LINES 56-59 The constant associated with Swissmetro is normalized to zero. It does not need to be defined at all. Here, we illustrate the fact that setting the last argument of the `Beta` function to 1 fixes the parameter to its default value (here, 0). .. GENERATED FROM PYTHON SOURCE LINES 59-61 .. code-block:: Python asc_sm = Beta('asc_sm', 0, None, None, 1) .. GENERATED FROM PYTHON SOURCE LINES 62-63 Coefficients of the attributes .. GENERATED FROM PYTHON SOURCE LINES 63-67 .. code-block:: Python b_time = Beta('b_time', 0, None, None, 0) b_cost = Beta('b_cost', 0, None, None, 0) .. GENERATED FROM PYTHON SOURCE LINES 68-69 Definition of the utility functions. .. GENERATED FROM PYTHON SOURCE LINES 69-73 .. code-block:: Python v_train = asc_train + b_time * TRAIN_TT_SCALED + b_cost * TRAIN_COST_SCALED v_sm = asc_sm + b_time * SM_TT_SCALED + b_cost * SM_COST_SCALED v_car = asc_car + b_time * CAR_TT_SCALED + b_cost * CAR_CO_SCALED .. GENERATED FROM PYTHON SOURCE LINES 74-75 Associate utility functions with the numbering of alternatives. .. GENERATED FROM PYTHON SOURCE LINES 75-77 .. code-block:: Python v = {1: v_train, 2: v_sm, 3: v_car} .. GENERATED FROM PYTHON SOURCE LINES 78-79 Associate the availability conditions with the alternatives. .. GENERATED FROM PYTHON SOURCE LINES 79-81 .. code-block:: Python av = {1: TRAIN_AV_SP, 2: SM_AV, 3: CAR_AV_SP} .. GENERATED FROM PYTHON SOURCE LINES 82-84 Definition of the model. This is the contribution of each observation to the log likelihood function. .. GENERATED FROM PYTHON SOURCE LINES 84-87 .. code-block:: Python log_probability = loglogit(v, av, CHOICE) .. GENERATED FROM PYTHON SOURCE LINES 88-89 **scipy**: this is the optimization algorithm from scipy. .. GENERATED FROM PYTHON SOURCE LINES 89-106 .. code-block:: Python my_biogeme_scipy = BIOGEME( the_data, log_probability, save_iterations=False, generate_html=False, generate_yaml=False, optimization_algorithm='scipy', ) my_biogeme_scipy.model_name = 'simple_example_scipy' print(my_biogeme_scipy) results_scipy = my_biogeme_scipy.estimate( starting_values={'asc_train': 0, 'b_time': 0, 'b_cost': 0, 'asc_car': 0} ) pandas_parameters_scipy = get_pandas_estimated_parameters( estimation_results=results_scipy ) display(pandas_parameters_scipy) .. rst-class:: sphx-glr-script-out .. code-block:: none Biogeme parameters read from biogeme.toml. simple_example_scipy: database [swissmetro]{'log_like': LogLogit[choice=CHOICE]U=(1:((Beta('asc_train', 0, None, None, 0) + (Beta('b_time', 0, None, None, 0) * TRAIN_TT_SCALED)) + (Beta('b_cost', 0, None, None, 0) * TRAIN_COST_SCALED)), 2:((Beta('asc_sm', 0, None, None, 1) + (Beta('b_time', 0, None, None, 0) * SM_TT_SCALED)) + (Beta('b_cost', 0, None, None, 0) * SM_COST_SCALED)), 3:((Beta('asc_car', 0, None, None, 0) + (Beta('b_time', 0, None, None, 0) * CAR_TT_SCALED)) + (Beta('b_cost', 0, None, None, 0) * CAR_CO_SCALED)))av=(1:TRAIN_AV_SP, 2:SM_AV, 3:CAR_AV_SP)} Starting values for the algorithm: {'asc_train': 0, 'b_time': 0, 'b_cost': 0, 'asc_car': 0} Optimization algorithm: scipy Minimize with tol 1e-07 Optimization algorithm has converged. Algorithm: scipy.optimize Cause of termination: CONVERGENCE: NORM OF PROJECTED GRADIENT <= PGTOL Number of iterations: 11 Number of function evaluations: 12 Optimization time: 0:00:00.132374 Calculate second derivatives and BHHH Name Value Robust std err. Robust t-stat. Robust p-value 0 asc_train -0.652239 0.054394 -11.991019 0.000000 1 b_time -1.278942 0.065598 -19.496761 0.000000 2 b_cost -0.789791 0.050965 -15.496743 0.000000 3 asc_car 0.016228 0.037088 0.437559 0.661706 .. GENERATED FROM PYTHON SOURCE LINES 107-108 Here are the messages generated by the optimization algorithm .. GENERATED FROM PYTHON SOURCE LINES 108-112 .. code-block:: Python for k, v in results_scipy.optimization_messages.items(): print(f'{k}:\t{v}') .. rst-class:: sphx-glr-script-out .. code-block:: none Algorithm: scipy.optimize Cause of termination: CONVERGENCE: NORM OF PROJECTED GRADIENT <= PGTOL Number of iterations: 11 Number of function evaluations: 12 Optimization time: 0:00:00.132374 .. GENERATED FROM PYTHON SOURCE LINES 113-114 **Newton with trust region** .. GENERATED FROM PYTHON SOURCE LINES 114-131 .. code-block:: Python my_biogeme_tr_newton = BIOGEME( the_data, log_probability, save_iterations=False, generate_html=False, generate_yaml=False, optimization_algorithm='TR-newton', ) my_biogeme_tr_newton.model_name = 'simple_example_tr_newton' print(my_biogeme_tr_newton) results_tr_newton = my_biogeme_tr_newton.estimate( starting_values={'asc_train': 0, 'b_time': 0, 'b_cost': 0, 'asc_car': 0} ) pandas_parameters_tr_newton = get_pandas_estimated_parameters( estimation_results=results_tr_newton ) display(pandas_parameters_tr_newton) .. rst-class:: sphx-glr-script-out .. code-block:: none Biogeme parameters read from biogeme.toml. simple_example_tr_newton: database [swissmetro]{'log_like': LogLogit[choice=CHOICE]U=(1:((Beta('asc_train', -0.6522385388419308, None, None, 0) + (Beta('b_time', -1.2789417543268635, None, None, 0) * TRAIN_TT_SCALED)) + (Beta('b_cost', -0.7897905924800291, None, None, 0) * TRAIN_COST_SCALED)), 2:((Beta('asc_sm', 0, None, None, 1) + (Beta('b_time', -1.2789417543268635, None, None, 0) * SM_TT_SCALED)) + (Beta('b_cost', -0.7897905924800291, None, None, 0) * SM_COST_SCALED)), 3:((Beta('asc_car', 0.016228046732543577, None, None, 0) + (Beta('b_time', -1.2789417543268635, None, None, 0) * CAR_TT_SCALED)) + (Beta('b_cost', -0.7897905924800291, None, None, 0) * CAR_CO_SCALED)))av=(1:TRAIN_AV_SP, 2:SM_AV, 3:CAR_AV_SP)} Starting values for the algorithm: {'asc_train': 0, 'b_time': 0, 'b_cost': 0, 'asc_car': 0} Optimization algorithm: Newton with trust region [TR-newton] ** Optimization: Newton with trust region Optimization algorithm has converged. Relative gradient: 2.2660610701087138e-07 Cause of termination: Relative gradient = 2.3e-07 <= 6.1e-06 Number of function evaluations: 53 Number of gradient evaluations: 36 Number of hessian evaluations: 18 Number of iterations: 17 Optimization time: 0:00:00.353333 Calculate second derivatives and BHHH Name Value Robust std err. Robust t-stat. Robust p-value 0 asc_train -0.652240 0.054394 -11.991046 0.000000 1 b_time -1.278939 0.065598 -19.496745 0.000000 2 b_cost -0.789789 0.050965 -15.496741 0.000000 3 asc_car 0.016227 0.037088 0.437528 0.661728 .. GENERATED FROM PYTHON SOURCE LINES 132-133 Here are the messages generated by the optimization algorithm .. GENERATED FROM PYTHON SOURCE LINES 133-136 .. code-block:: Python for k, v in results_tr_newton.optimization_messages.items(): print(f'{k}:\t{v}') .. rst-class:: sphx-glr-script-out .. code-block:: none Relative gradient: 2.2660610701087138e-07 Cause of termination: Relative gradient = 2.3e-07 <= 6.1e-06 Number of function evaluations: 53 Number of gradient evaluations: 36 Number of hessian evaluations: 18 Number of iterations: 17 Optimization time: 0:00:00.353333 .. GENERATED FROM PYTHON SOURCE LINES 137-138 **Newton/BFGS with trust region for simple bounds** .. GENERATED FROM PYTHON SOURCE LINES 140-143 This is the default algorithm used by Biogeme. It is the implementation of the algorithm proposed by `Conn et al. (1988) `_. .. GENERATED FROM PYTHON SOURCE LINES 143-160 .. code-block:: Python my_biogeme_simple_bounds = BIOGEME( the_data, log_probability, save_iterations=False, generate_html=False, generate_yaml=False, optimization_algorithm='automatic', ) my_biogeme_simple_bounds.model_name = 'simple_example_simple_bounds' print(my_biogeme_simple_bounds) results_simple_bounds = my_biogeme_simple_bounds.estimate( starting_values={'asc_train': 0, 'b_time': 0, 'b_cost': 0, 'asc_car': 0} ) pandas_parameters_simple_bounds = get_pandas_estimated_parameters( estimation_results=results_simple_bounds ) display(pandas_parameters_simple_bounds) .. rst-class:: sphx-glr-script-out .. code-block:: none Biogeme parameters read from biogeme.toml. simple_example_simple_bounds: database [swissmetro]{'log_like': LogLogit[choice=CHOICE]U=(1:((Beta('asc_train', -0.6522395400465253, None, None, 0) + (Beta('b_time', -1.2789389133282347, None, None, 0) * TRAIN_TT_SCALED)) + (Beta('b_cost', -0.7897894350204051, None, None, 0) * TRAIN_COST_SCALED)), 2:((Beta('asc_sm', 0, None, None, 1) + (Beta('b_time', -1.2789389133282347, None, None, 0) * SM_TT_SCALED)) + (Beta('b_cost', -0.7897894350204051, None, None, 0) * SM_COST_SCALED)), 3:((Beta('asc_car', 0.01622690225771413, None, None, 0) + (Beta('b_time', -1.2789389133282347, None, None, 0) * CAR_TT_SCALED)) + (Beta('b_cost', -0.7897894350204051, None, None, 0) * CAR_CO_SCALED)))av=(1:TRAIN_AV_SP, 2:SM_AV, 3:CAR_AV_SP)} Starting values for the algorithm: {'asc_train': 0, 'b_time': 0, 'b_cost': 0, 'asc_car': 0} As the model is not too complex, we activate the calculation of second derivatives. To change this behavior, modify the algorithm to "simple_bounds" in the TOML file. Optimization algorithm: hybrid Newton/BFGS with simple bounds [simple_bounds] ** Optimization: Newton with trust region for simple bounds Iter. asc_train b_time b_cost asc_car Function Relgrad Radius Rho 0 -0.76 -0.77 -0.7 -0.29 8.8e+03 0.04 10 1.1 ++ 1 -0.66 -1.2 -0.77 -0.0015 8.7e+03 0.0064 1e+02 1.1 ++ 2 -0.65 -1.3 -0.79 0.016 8.7e+03 0.00012 1e+03 1 ++ 3 -0.65 -1.3 -0.79 0.016 8.7e+03 4e-08 1e+03 1 ++ Optimization algorithm has converged. Relative gradient: 3.954408093567457e-08 Cause of termination: Relative gradient = 4e-08 <= 6.1e-06 Number of function evaluations: 13 Number of gradient evaluations: 9 Number of hessian evaluations: 4 Algorithm: Newton with trust region for simple bound constraints Number of iterations: 4 Proportion of Hessian calculation: 4/4 = 100.0% Optimization time: 0:00:00.304598 Calculate second derivatives and BHHH Name Value Robust std err. Robust t-stat. Robust p-value 0 asc_train -0.652239 0.054394 -11.991022 0.000000 1 b_time -1.278941 0.065598 -19.496759 0.000000 2 b_cost -0.789790 0.050965 -15.496743 0.000000 3 asc_car 0.016228 0.037088 0.437556 0.661708 .. GENERATED FROM PYTHON SOURCE LINES 161-162 Here are the messages generated by the optimization algorithm .. GENERATED FROM PYTHON SOURCE LINES 162-165 .. code-block:: Python for k, v in results_simple_bounds.optimization_messages.items(): print(f'{k}:\t{v}') .. rst-class:: sphx-glr-script-out .. code-block:: none Relative gradient: 3.954408093567457e-08 Cause of termination: Relative gradient = 4e-08 <= 6.1e-06 Number of function evaluations: 13 Number of gradient evaluations: 9 Number of hessian evaluations: 4 Algorithm: Newton with trust region for simple bound constraints Number of iterations: 4 Proportion of Hessian calculation: 4/4 = 100.0% Optimization time: 0:00:00.304598 .. GENERATED FROM PYTHON SOURCE LINES 166-170 When the second derivatives are too computationally expensive to calculate, it is possible to avoid calculating them at each successful iteration. The parameter `second_derivatives` allows to control that. .. GENERATED FROM PYTHON SOURCE LINES 172-192 .. code-block:: Python my_biogeme_simple_bounds_no_hessian = BIOGEME( the_data, log_probability, save_iterations=False, generate_html=False, generate_yaml=False, optimization_algorithm='simple_bounds', second_derivatives=0, ) my_biogeme_simple_bounds_no_hessian.model_name = ( 'simple_example_simple_bounds_no_hessian' ) print(my_biogeme_simple_bounds_no_hessian) results_simple_bounds_no_hessian = my_biogeme_simple_bounds_no_hessian.estimate( starting_values={'asc_train': 0, 'b_time': 0, 'b_cost': 0, 'asc_car': 0} ) pandas_parameters_simple_bounds_no_hessian = get_pandas_estimated_parameters( estimation_results=results_simple_bounds_no_hessian ) display(pandas_parameters_simple_bounds_no_hessian) .. rst-class:: sphx-glr-script-out .. code-block:: none Biogeme parameters read from biogeme.toml. simple_example_simple_bounds_no_hessian: database [swissmetro]{'log_like': LogLogit[choice=CHOICE]U=(1:((Beta('asc_train', -0.652238664271019, None, None, 0) + (Beta('b_time', -1.2789413398819158, None, None, 0) * TRAIN_TT_SCALED)) + (Beta('b_cost', -0.7897904566401142, None, None, 0) * TRAIN_COST_SCALED)), 2:((Beta('asc_sm', 0, None, None, 1) + (Beta('b_time', -1.2789413398819158, None, None, 0) * SM_TT_SCALED)) + (Beta('b_cost', -0.7897904566401142, None, None, 0) * SM_COST_SCALED)), 3:((Beta('asc_car', 0.01622793815045202, None, None, 0) + (Beta('b_time', -1.2789413398819158, None, None, 0) * CAR_TT_SCALED)) + (Beta('b_cost', -0.7897904566401142, None, None, 0) * CAR_CO_SCALED)))av=(1:TRAIN_AV_SP, 2:SM_AV, 3:CAR_AV_SP)} Starting values for the algorithm: {'asc_train': 0, 'b_time': 0, 'b_cost': 0, 'asc_car': 0} Optimization algorithm: hybrid Newton/BFGS with simple bounds [simple_bounds] ** Optimization: BFGS with trust region for simple bounds Iter. asc_train b_time b_cost asc_car Function Relgrad Radius Rho 0 -1 -1 -1 1 1e+04 0.2 1 0.19 + 1 -0.61 -2 0 0 9.4e+03 0.16 1 0.18 + 2 0.065 -1.2 -1 0.058 9.1e+03 0.1 1 0.24 + 3 -0.93 -1.7 -1.1 0.44 9e+03 0.058 1 0.11 + 4 -0.93 -1.7 -1.1 0.44 9e+03 0.058 0.5 -0.46 - 5 -0.56 -1.5 -0.88 -0.057 8.7e+03 0.036 0.5 0.87 + 6 -0.56 -1.5 -0.88 -0.057 8.7e+03 0.036 0.25 -1.8 - 7 -0.56 -1.5 -0.88 -0.057 8.7e+03 0.036 0.12 -0.89 - 8 -0.44 -1.4 -1 0.068 8.7e+03 0.016 0.12 0.14 + 9 -0.56 -1.5 -0.88 0.14 8.7e+03 0.023 0.12 0.4 + 10 -0.44 -1.4 -0.77 0.085 8.7e+03 0.013 0.12 0.44 + 11 -0.44 -1.4 -0.77 0.085 8.7e+03 0.013 0.062 -1.4 - 12 -0.5 -1.4 -0.74 0.15 8.7e+03 0.0069 0.062 0.46 + 13 -0.56 -1.4 -0.81 0.085 8.7e+03 0.0038 0.062 0.72 + 14 -0.62 -1.3 -0.74 0.023 8.7e+03 0.0042 0.062 0.37 + 15 -0.62 -1.3 -0.74 0.023 8.7e+03 0.0042 0.031 -0.031 - 16 -0.63 -1.3 -0.77 0.054 8.7e+03 0.0046 0.031 0.28 + 17 -0.64 -1.3 -0.79 0.023 8.7e+03 0.0015 0.031 0.78 + 18 -0.64 -1.3 -0.79 0.023 8.7e+03 0.0015 0.016 -0.63 - 19 -0.64 -1.3 -0.79 0.016 8.7e+03 0.00083 0.016 0.64 + 20 -0.64 -1.3 -0.79 0.016 8.7e+03 0.00083 0.0078 -0.6 - 21 -0.65 -1.3 -0.79 0.021 8.7e+03 0.00059 0.0078 0.29 + 22 -0.65 -1.3 -0.79 0.014 8.7e+03 0.00042 0.0078 0.42 + 23 -0.65 -1.3 -0.79 0.014 8.7e+03 0.00042 0.0039 -0.18 - 24 -0.65 -1.3 -0.79 0.017 8.7e+03 0.00024 0.0039 0.4 + 25 -0.65 -1.3 -0.79 0.017 8.7e+03 0.00024 0.002 -0.0014 - 26 -0.65 -1.3 -0.79 0.015 8.7e+03 0.00011 0.002 0.64 + 27 -0.65 -1.3 -0.79 0.015 8.7e+03 0.00011 0.00098 0.0081 - 28 -0.65 -1.3 -0.79 0.016 8.7e+03 4.8e-05 0.00098 0.62 + 29 -0.65 -1.3 -0.79 0.016 8.7e+03 1.9e-05 0.00098 0.23 + 30 -0.65 -1.3 -0.79 0.016 8.7e+03 1.9e-05 0.00047 -1.6 - 31 -0.65 -1.3 -0.79 0.016 8.7e+03 8.8e-06 0.00047 0.79 + 32 -0.65 -1.3 -0.79 0.016 8.7e+03 8.8e-06 0.00023 -4.4 - 33 -0.65 -1.3 -0.79 0.016 8.7e+03 8.8e-06 0.00012 -1.4 - 34 -0.65 -1.3 -0.79 0.016 8.7e+03 8.8e-06 5.8e-05 -0.53 - 35 -0.65 -1.3 -0.79 0.016 8.7e+03 6.8e-06 5.8e-05 0.38 + 36 -0.65 -1.3 -0.79 0.016 8.7e+03 2.2e-06 5.8e-05 0.66 + Optimization algorithm has converged. Relative gradient: 2.1737457232057418e-06 Cause of termination: Relative gradient = 2.2e-06 <= 6.1e-06 Number of function evaluations: 84 Number of gradient evaluations: 47 Number of hessian evaluations: 0 Algorithm: BFGS with trust region for simple bound constraints Number of iterations: 37 Proportion of Hessian calculation: 0/23 = 0.0% Optimization time: 0:00:00.190814 Calculate second derivatives and BHHH Name Value Robust std err. Robust t-stat. Robust p-value 0 asc_train -0.652188 0.054394 -11.990015 0.000000 1 b_time -1.278993 0.065599 -19.497136 0.000000 2 b_cost -0.789796 0.050966 -15.496685 0.000000 3 asc_car 0.016248 0.037088 0.438089 0.661322 .. GENERATED FROM PYTHON SOURCE LINES 193-194 Here are the messages generated by the optimization algorithm .. GENERATED FROM PYTHON SOURCE LINES 194-196 .. code-block:: Python for k, v in results_simple_bounds_no_hessian.optimization_messages.items(): print(f'{k}:\t{v}') .. rst-class:: sphx-glr-script-out .. code-block:: none Relative gradient: 2.1737457232057418e-06 Cause of termination: Relative gradient = 2.2e-06 <= 6.1e-06 Number of function evaluations: 84 Number of gradient evaluations: 47 Number of hessian evaluations: 0 Algorithm: BFGS with trust region for simple bound constraints Number of iterations: 37 Proportion of Hessian calculation: 0/23 = 0.0% Optimization time: 0:00:00.190814 .. rst-class:: sphx-glr-timing **Total running time of the script:** (0 minutes 3.516 seconds) .. _sphx_glr_download_auto_examples_programmers_plot_optimization.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: plot_optimization.ipynb ` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: plot_optimization.py ` .. container:: sphx-glr-download sphx-glr-download-zip :download:`Download zipped: plot_optimization.zip ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_