Note
Go to the end to download the full example code.
biogeme.results_processing¶
Examples of use of several functions.
This is designed for programmers who need examples of use of the functions of the module. The examples are designed to illustrate the syntax. They do not correspond to any meaningful model.
Michel Bierlaire Mon Oct 7 18:46:39 2024
import pandas as pd
from IPython.core.display_functions import display
from biogeme.biogeme import BIOGEME
from biogeme.database import Database
from biogeme.expressions import Beta, Variable, exp
from biogeme.results_processing import (
EstimateVarianceCovariance,
EstimationResults,
calc_p_value,
compile_estimation_results,
get_f12,
get_html_estimated_parameters,
get_latex_estimated_parameters,
get_pandas_correlation_results,
get_pandas_estimated_parameters,
)
from biogeme.version import get_text
Version of Biogeme.
print(get_text())
biogeme 3.3.1 [2025-09-03]
Home page: http://biogeme.epfl.ch
Submit questions to https://groups.google.com/d/forum/biogeme
Michel Bierlaire, Transport and Mobility Laboratory, Ecole Polytechnique Fédérale de Lausanne (EPFL)
Definition of a database
df = pd.DataFrame(
{
'Person': [1, 1, 1, 2, 2],
'Exclude': [0, 0, 1, 0, 1],
'Variable1': [1, 2, 3, 4, 5],
'Variable2': [10, 20, 30, 40, 50],
'Choice': [1, 2, 3, 1, 2],
'Av1': [0, 1, 1, 1, 1],
'Av2': [1, 1, 1, 1, 1],
'Av3': [0, 1, 1, 1, 1],
}
)
display(df)
Person Exclude Variable1 Variable2 Choice Av1 Av2 Av3
0 1 0 1 10 1 0 1 0
1 1 0 2 20 2 1 1 1
2 1 1 3 30 3 1 1 1
3 2 0 4 40 1 1 1 1
4 2 1 5 50 2 1 1 1
my_data = Database('test', df)
Definition of various expressions
Variable1 = Variable('Variable1')
Variable2 = Variable('Variable2')
beta1 = Beta('beta1', -1.0, -3, 3, 0)
beta2 = Beta('beta2', 2.0, -3, 10, 0)
likelihood = -(beta1**2) * Variable1 - exp(beta2 * beta1) * Variable2 - beta2**4
simul = beta1 / Variable1 + beta2 / Variable2
dict_of_expressions = {'log_like': likelihood, 'beta1': beta1, 'simul': simul}
Creation of the BIOGEME object
my_biogeme = BIOGEME(my_data, dict_of_expressions, bootstrap_samples=100)
my_biogeme.model_name = 'simple_example'
results = my_biogeme.estimate(run_bootstrap=True)
print(results)
Bootstraps: 0%| | 0/100 [00:00<?, ?it/s]
0%| | 0/100 [00:00<?, ?it/s]
1%| | 1/100 [00:00<00:11, 8.54it/s]
3%|▎ | 3/100 [00:00<00:07, 12.52it/s]
6%|▌ | 6/100 [00:00<00:06, 13.95it/s]
10%|█ | 10/100 [00:00<00:06, 13.09it/s]
14%|█▍ | 14/100 [00:01<00:05, 14.42it/s]
18%|█▊ | 18/100 [00:01<00:05, 15.73it/s]
22%|██▏ | 22/100 [00:01<00:04, 16.18it/s]
26%|██▌ | 26/100 [00:01<00:04, 16.38it/s]
30%|███ | 30/100 [00:01<00:04, 16.67it/s]
34%|███▍ | 34/100 [00:02<00:03, 16.81it/s]
36%|███▌ | 36/100 [00:02<00:03, 17.31it/s]
38%|███▊ | 38/100 [00:02<00:03, 16.54it/s]
42%|████▏ | 42/100 [00:02<00:03, 16.87it/s]
46%|████▌ | 46/100 [00:02<00:03, 16.97it/s]
50%|█████ | 50/100 [00:03<00:02, 19.07it/s]
54%|█████▍ | 54/100 [00:03<00:02, 18.51it/s]
56%|█████▌ | 56/100 [00:03<00:02, 18.57it/s]
58%|█████▊ | 58/100 [00:03<00:02, 17.59it/s]
62%|██████▏ | 62/100 [00:03<00:01, 19.87it/s]
64%|██████▍ | 64/100 [00:03<00:02, 17.45it/s]
68%|██████▊ | 68/100 [00:04<00:01, 17.34it/s]
72%|███████▏ | 72/100 [00:04<00:01, 17.57it/s]
76%|███████▌ | 76/100 [00:04<00:01, 17.36it/s]/Users/bierlair/python_envs/venv313/lib/python3.13/site-packages/joblib/externals/loky/process_executor.py:782: UserWarning: A worker stopped while some jobs were given to the executor. This can be caused by a too short worker timeout or by a memory leak.
warnings.warn(
80%|████████ | 80/100 [00:04<00:01, 12.98it/s]
82%|████████▏ | 82/100 [00:06<00:03, 5.33it/s]
84%|████████▍ | 84/100 [00:06<00:02, 5.71it/s]
88%|████████▊ | 88/100 [00:06<00:01, 7.63it/s]
92%|█████████▏| 92/100 [00:06<00:00, 10.02it/s]
94%|█████████▍| 94/100 [00:07<00:00, 10.34it/s]
98%|█████████▊| 98/100 [00:07<00:00, 11.64it/s]
100%|██████████| 100/100 [00:07<00:00, 13.30it/s]
Results for model simple_example
Nbr of parameters: 2
Sample size: 5
Excluded data: 0
Final log likelihood: -67.06549
Akaike Information Criterion: 138.131
Bayesian Information Criterion: 137.3499
Dump results on a file
the_yaml_file = my_biogeme.yaml_filename
results.dump_yaml_file(filename=the_yaml_file)
print(the_yaml_file)
simple_example~02.yaml
Results can be imported from a file previously generated
read_results = EstimationResults.from_yaml_file(filename=the_yaml_file)
print(read_results)
Results for model simple_example
Nbr of parameters: 2
Sample size: 5
Excluded data: 0
Final log likelihood: -67.06549
Akaike Information Criterion: 138.131
Bayesian Information Criterion: 137.3499
Results can be formatted in LaTeX
print(get_latex_estimated_parameters(estimation_results=read_results))
\begin{tabular}{rlr@{.}lr@{.}lr@{.}lr@{.}l}
& & \multicolumn{2}{l}{} & \multicolumn{2}{l}{Bootstrap} & \multicolumn{4}{l}{} \\
Parameter & & \multicolumn{2}{l}{Coeff.} & \multicolumn{2}{l}{Asympt.} & \multicolumn{4}{l}{} \\
number & Description & \multicolumn{2}{l}{estimate} & \multicolumn{2}{l}{std. error} & \multicolumn{2}{l}{$t$-stat} & \multicolumn{2}{l}{$p$-value} \\
\hline
0 & beta1 & -1&27 & 0&014 & -91&2 & 0&0 \\
1 & beta2 & 1&25 & 0&0609 & 20&5 & 0&0 \\
\end{tabular}
Results can be formatted in HTML
print(get_html_estimated_parameters(estimation_results=read_results))
<table border="1">
<tr class=biostyle><th>Id</th><th>Name</th><th>Value</th><th>Bootstrap std err.</th><th>Bootstrap t-stat.</th><th>Bootstrap p-value</th></tr>
<tr class=biostyle><td>0</td><td>beta1</td><td>-1.27</td><td>0.014</td><td>-91.2</td><td>0</td></tr>
<tr class=biostyle><td>1</td><td>beta2</td><td>1.25</td><td>0.0609</td><td>20.5</td><td>0</td></tr>
</table>
General statistics, including a suggested format.
statistics = read_results.get_general_statistics()
display(statistics)
{'Number of estimated parameters': '2', 'Sample size': '5', 'Excluded observations': '0', 'Init log likelihood': '-67.06549', 'Final log likelihood': '-67.06549', 'Likelihood ratio test for the init. model': '-0', 'Rho-square for the init. model': '0', 'Rho-square-bar for the init. model': '-0.0298', 'Akaike Information Criterion': '138.131', 'Bayesian Information Criterion': '137.3499', 'Final gradient norm': '1.1521E-07', 'Bootstrapping time': '0:00:07.518484'}
Estimated parameters as pandas dataframe
pandas_parameters = get_pandas_estimated_parameters(estimation_results=read_results)
display(pandas_parameters)
Name Value Bootstrap std err. Bootstrap t-stat. Bootstrap p-value
0 beta1 -1.273264 0.013957 -91.229031 0.0
1 beta2 1.248769 0.060945 20.489987 0.0
Correlation results
pandas_correlation = get_pandas_correlation_results(estimation_results=read_results)
display(pandas_correlation)
First parameter Second parameter ... Bootstrap t-stat. Bootstrap p-value
0 beta2 beta1 ... 53.669506 0.0
[1 rows x 6 columns]
Obtain the values of the parameters
beta_values = read_results.get_beta_values()
display(beta_values)
{'beta1': -1.273263987213694, 'beta2': 1.2487688099301162}
Obtain the value of one or several specific parameters
some_beta_values = read_results.get_beta_values(my_betas=['beta2'])
display(some_beta_values)
{'beta2': 1.2487688099301162}
Variance-covariance matrix (Rao-Cramer)
rao_cramer = read_results.get_variance_covariance_matrix(
variance_covariance_type=EstimateVarianceCovariance.RAO_CRAMER
)
display(rao_cramer)
[[0.01325818 0.00167146]
[0.00167146 0.00719613]]
Variance-covariance matrix (robust)
robust = read_results.get_variance_covariance_matrix(
variance_covariance_type=EstimateVarianceCovariance.ROBUST
)
display(robust)
[[0.00018835 0.00081089]
[0.00081089 0.00349115]]
Variance-covariance matrix (bootstrap)
bootstrap = read_results.get_variance_covariance_matrix(
variance_covariance_type=EstimateVarianceCovariance.BOOTSTRAP
)
display(bootstrap)
[[0.00019479 0.00085044]
[0.00085044 0.00371433]]
Draws for sensitivity analysis are generated using bootstrapping. Any indicator can be generated by the model for each draw, and its empirical distribution can be investigated.
read_results.get_betas_for_sensitivity_analysis(['beta1', 'beta2'], size=10)
[{'beta1': -1.282393736283639, 'beta2': 1.209195603860136}, {'beta1': -1.282393736283639, 'beta2': 1.209195603860136}, {'beta1': -1.282393736283639, 'beta2': 1.209195603860136}, {'beta1': -1.2611085813010015, 'beta2': 1.3007517002098443}, {'beta1': -1.273263987213694, 'beta2': 1.2487688099301162}, {'beta1': -1.282393736283639, 'beta2': 1.209195603860136}, {'beta1': -1.273263987213694, 'beta2': 1.2487688099301162}, {'beta1': -1.2611085813010015, 'beta2': 1.3007517002098443}, {'beta1': -1.2573978799513064, 'beta2': 1.3165120810083486}, {'beta1': -1.2981035069734936, 'beta2': 1.1393434553785933}, {'beta1': -1.243923633974187, 'beta2': 1.373502065868922}, {'beta1': -1.3170661906585204, 'beta2': 1.0497320768282947}, {'beta1': -1.2690260405244644, 'beta2': 1.2669668838392423}, {'beta1': -1.2611085813405174, 'beta2': 1.3007516144406395}, {'beta1': -1.282393736283639, 'beta2': 1.209195603860136}, {'beta1': -1.287332492839996, 'beta2': 1.1875193084801916}, {'beta1': -1.273263987213694, 'beta2': 1.2487688099301162}, {'beta1': -1.2649797742013058, 'beta2': 1.2842631765105266}, {'beta1': -1.2649797742013058, 'beta2': 1.2842631765105266}, {'beta1': -1.269026040455536, 'beta2': 1.2669668836379888}, {'beta1': -1.2777126116342759, 'beta2': 1.2295568140731905}, {'beta1': -1.2873325027193596, 'beta2': 1.1875194143603178}, {'beta1': -1.2573978799513064, 'beta2': 1.3165120810083486}, {'beta1': -1.2873324919847893, 'beta2': 1.187519300827712}, {'beta1': -1.2573978799513064, 'beta2': 1.3165120810083486}, {'beta1': -1.2649797727418584, 'beta2': 1.284263172308228}, {'beta1': -1.2690260405244644, 'beta2': 1.2669668838392423}, {'beta1': -1.2504079893334659, 'beta2': 1.3461108405007625}, {'beta1': -1.2649797742013058, 'beta2': 1.2842631765105266}, {'beta1': -1.269026040455536, 'beta2': 1.2669668836379888}, {'beta1': -1.2777126116342759, 'beta2': 1.2295568140731905}, {'beta1': -1.2873325027193596, 'beta2': 1.1875194143603178}, {'beta1': -1.298103506705179, 'beta2': 1.1393434542527645}, {'beta1': -1.282393821462994, 'beta2': 1.2091960928148573}, {'beta1': -1.3103131424440955, 'beta2': 1.0825801152308525}, {'beta1': -1.2732639874634077, 'beta2': 1.2487688091207687}, {'beta1': -1.298103506705179, 'beta2': 1.1393434542527645}, {'beta1': -1.2690260404854072, 'beta2': 1.2669668816538486}, {'beta1': -1.2573978799513064, 'beta2': 1.3165120810083486}, {'beta1': -1.2649797727418584, 'beta2': 1.284263172308228}, {'beta1': -1.2690260405244644, 'beta2': 1.2669668838392423}, {'beta1': -1.2504079893334659, 'beta2': 1.3461108405007625}, {'beta1': -1.273263987213694, 'beta2': 1.2487688099301162}, {'beta1': -1.2649797742013058, 'beta2': 1.2842631765105266}, {'beta1': -1.2777126116342759, 'beta2': 1.2295568140731905}, {'beta1': -1.292558011774817, 'beta2': 1.1643229811019848}, {'beta1': -1.2690260405244644, 'beta2': 1.2669668838392423}, {'beta1': -1.2925579238204843, 'beta2': 1.1643222008081031}, {'beta1': -1.282393736283639, 'beta2': 1.209195603860136}, {'beta1': -1.282393736283639, 'beta2': 1.209195603860136}, {'beta1': -1.298103506705179, 'beta2': 1.1393434542527645}, {'beta1': -1.2573978838405362, 'beta2': 1.3165118203767316}, {'beta1': -1.2690260405244644, 'beta2': 1.2669668838392423}, {'beta1': -1.2611085813405174, 'beta2': 1.3007516144406395}, {'beta1': -1.2925578214686664, 'beta2': 1.164322217510277}, {'beta1': -1.2823937457787886, 'beta2': 1.2091956298296247}, {'beta1': -1.298103506705179, 'beta2': 1.1393434542527645}, {'beta1': -1.2690260404854072, 'beta2': 1.2669668816538486}, {'beta1': -1.2538348107653448, 'beta2': 1.331612380127896}, {'beta1': -1.2732639365473521, 'beta2': 1.2487701544564327}, {'beta1': -1.2649797742013058, 'beta2': 1.2842631765105266}, {'beta1': -1.2649797742013058, 'beta2': 1.2842631765105266}, {'beta1': -1.2777126116342759, 'beta2': 1.2295568140731905}, {'beta1': -1.273263987607468, 'beta2': 1.248768812102508}, {'beta1': -1.2611085813010015, 'beta2': 1.3007517002098443}, {'beta1': -1.26497977397441, 'beta2': 1.2842631509636993}, {'beta1': -1.2573978799513064, 'beta2': 1.3165120810083486}, {'beta1': -1.2873324919847893, 'beta2': 1.187519300827712}, {'beta1': -1.2538348107653448, 'beta2': 1.331612380127896}, {'beta1': -1.2732639365473521, 'beta2': 1.2487701544564327}, {'beta1': -1.2925578214686664, 'beta2': 1.164322217510277}, {'beta1': -1.2732640875536856, 'beta2': 1.248769699775698}, {'beta1': -1.273263987213694, 'beta2': 1.2487688099301162}, {'beta1': -1.2690260405244644, 'beta2': 1.2669668838392423}, {'beta1': -1.2538348107653448, 'beta2': 1.331612380127896}, {'beta1': -1.2732639365473521, 'beta2': 1.2487701544564327}, {'beta1': -1.2649797742013058, 'beta2': 1.2842631765105266}, {'beta1': -1.2823937666074936, 'beta2': 1.2091966201417774}, {'beta1': -1.273263987213694, 'beta2': 1.2487688099301162}, {'beta1': -1.287332533620085, 'beta2': 1.1875198317198776}, {'beta1': -1.2649797742013058, 'beta2': 1.2842631765105266}, {'beta1': -1.2573978830118717, 'beta2': 1.3165118359003405}, {'beta1': -1.273263987213694, 'beta2': 1.2487688099301162}, {'beta1': -1.2925578214686664, 'beta2': 1.164322217510277}, {'beta1': -1.2649797742013058, 'beta2': 1.2842631765105266}, {'beta1': -1.2873324919775402, 'beta2': 1.1875193007862883}, {'beta1': -1.2690260405244644, 'beta2': 1.2669668838392423}, {'beta1': -1.2823937485848418, 'beta2': 1.2091958966297875}, {'beta1': -1.2690260405244644, 'beta2': 1.2669668838392423}, {'beta1': -1.2573978810083073, 'beta2': 1.3165119010872348}, {'beta1': -1.273263987213694, 'beta2': 1.2487688099301162}, {'beta1': -1.2649797742013058, 'beta2': 1.2842631765105266}, {'beta1': -1.2777126116342759, 'beta2': 1.2295568140731905}, {'beta1': -1.273263987607468, 'beta2': 1.248768812102508}, {'beta1': -1.282393736283639, 'beta2': 1.209195603860136}, {'beta1': -1.2777126118096789, 'beta2': 1.229556814569854}, {'beta1': -1.282393736283639, 'beta2': 1.209195603860136}, {'beta1': -1.2777126118096789, 'beta2': 1.229556814569854}, {'beta1': -1.3103131424440955, 'beta2': 1.0825801152308525}, {'beta1': -1.257397883841623, 'beta2': 1.316511820405938}]
Results can be produced in the ALOGIT F12 format
print(get_f12(estimation_results=read_results))
simple_example
From biogeme 3.3.1 2025-09-03 02:00:59
END
0 beta1 F -1.273263987214e+00 +1.395678510435e-02
0 beta2 F +1.248768809930e+00 +6.094531842799e-02
-1
5 0 0 -6.706549047946e+01
0 0 2025-09-03 02:00:59
99981
Miscellaneous functions
Likelihood ratio test. Let’s first estimate a constrained model
beta2_constrained = Beta('beta2_constrained', 2.0, -3, 10, 1)
likelihood_constrained = (
-(beta1**2) * Variable1
- exp(beta2_constrained * beta1) * Variable2
- beta2_constrained**4
)
my_biogeme_constrained = BIOGEME(my_data, likelihood_constrained)
my_biogeme_constrained.model_name = 'simple_example_constrained'
results_constrained = my_biogeme_constrained.estimate()
print(results_constrained.short_summary())
Bootstraps: 0%| | 0/100 [00:07<?, ?it/s]
Results for model simple_example_constrained
Nbr of parameters: 1
Sample size: 5
Excluded data: 0
Final log likelihood: -114.7702
Akaike Information Criterion: 231.5403
Bayesian Information Criterion: 231.1498
We can now perform a likelihood ratio test.
test_results = results.likelihood_ratio_test(results_constrained, 0.95)
print(test_results.message)
print(f'Statistic: {test_results.statistic}')
print(f'Threshold: {test_results.threshold}')
H0 can be rejected at level 95.0%
Statistic: 95.40936413216045
Threshold: 0.003932140000019531
Calculation of the \(p\)-value
calc_p_value(1.96)
np.float64(0.04999579029644097)
Compilation of results
dict_of_results = {'Model A': read_results, 'Model B': the_yaml_file}
df = compile_estimation_results(dict_of_results)
display(df)
( Model A Model B
Number of estimated parameters 2 2
Sample size 5 5
Final log likelihood -67.06549 -67.06549
Akaike Information Criterion 138.131 138.131
Bayesian Information Criterion 137.3499 137.3499
beta1 (t-test) -1.27 (-92.8) -1.27 (-92.8)
beta2 (t-test) 1.25 (21.1) 1.25 (21.1), {'Model A': 'Model A', 'Model B': 'Model B'})
Total running time of the script: (0 minutes 7.982 seconds)