Assisted specification

Example of the estimation of several versions of the model using assisted specification algorithm. The catalog of specifications is defined in Specification of a catalog of models . All specifications are estimated. Have a look at Assisted specification for an example where the number of specifications is too high to be enumerated.

author:

Michel Bierlaire, EPFL

date:

Wed Apr 12 16:58:49 2023

import biogeme.biogeme_logging as blog
from biogeme.results import compile_estimation_results
from biogeme.multiobjectives import loglikelihood_dimension
from biogeme.assisted import AssistedSpecification
from plot_b21multiple_models_spec import the_biogeme, PARETO_FILE_NAME

logger = blog.get_screen_logger(blog.INFO)
logger.info('Example b21multipleModels')
income_segmentation=INCOME: [{0: 'inc-zero', 1: 'inc-under50', 2: 'inc-50-100', 3: 'inc-100+', 4: 'inc-unknown'}] ref: inc-zero
Total number of possible specifications: 36
Example b21multipleModels

Creation of the object capturing the assisted specification algorithm. Its constructor takes three arguments:

  • the biogeme object containing the specifications and the database,

  • an object defining the objectives to minimize. Here, we use the opposite of the log lieklihood and the number of estimated parameters.

  • the name of the file where the estimated are saved, and organized into a Pareto set.

assisted_specification = AssistedSpecification(
    biogeme_object=the_biogeme,
    multi_objectives=loglikelihood_dimension,
    pareto_file_name=PARETO_FILE_NAME,
)
Unable to read file b21multiple_models.pareto. Pareto set empty.

The algorithm is run.

non_dominated_models = assisted_specification.run()
File biogeme.toml has been parsed.
*** Estimate b21multiple_models_000000
Optimization algorithm: hybrid Newton/BFGS with simple bounds [simple_bounds]
** Optimization: Newton with trust region for simple bounds
Iter.         ASC_CAR       ASC_TRAIN          B_COST          B_TIME     Function    Relgrad   Radius      Rho
    0           -0.49           -0.92           -0.88           -0.67      5.4e+03      0.041       10      1.1   ++
    1           -0.18           -0.73              -1            -1.2      5.3e+03     0.0072    1e+02      1.1   ++
    2           -0.16            -0.7            -1.1            -1.3      5.3e+03    0.00018    1e+03        1   ++
    3           -0.16            -0.7            -1.1            -1.3      5.3e+03    1.1e-07    1e+03        1   ++
default_specification=ASC:no_seg;B_COST:no_seg;TRAIN_TT:linear
We consider all possible combinations of the catalogs.
Model 0/36
File biogeme.toml has been parsed.
*** Estimate b21multiple_models_000000
Optimization algorithm: hybrid Newton/BFGS with simple bounds [simple_bounds]
** Optimization: Newton with trust region for simple bounds
Iter.         ASC_CAR      ASC_CAR_GA       ASC_TRAIN    ASC_TRAIN_GA          B_COST B_COST_inc-100+ B_COST_inc-50-1 B_COST_inc-unde B_COST_inc-unkn          B_TIME     Function    Relgrad   Radius      Rho
    0           -0.27           -0.11           -0.78            0.18           -0.34           -0.14           -0.13          -0.042          -0.021              -1      5.4e+03      0.036       10        1   ++
    1          -0.072          -0.068           -0.98             2.4            -1.5            0.65            0.41          -0.066            0.77            -1.6        5e+03      0.028    1e+02     0.94   ++
    2           -0.07            -0.2            -1.1             2.1            -1.6            0.66            0.28           -0.76            0.78            -1.7        5e+03    0.00071    1e+03        1   ++
    3           -0.07            -0.2            -1.1             2.1            -1.6            0.66            0.28           -0.76            0.78            -1.7        5e+03    5.1e-06    1e+03        1   ++
Model 1/36
File biogeme.toml has been parsed.
*** Estimate b21multiple_models_000000
Optimization algorithm: hybrid Newton/BFGS with simple bounds [simple_bounds]
** Optimization: Newton with trust region for simple bounds
Iter.         ASC_CAR    ASC_CAR_male       ASC_TRAIN  ASC_TRAIN_male          B_COST       B_COST_GA          B_TIME     Function    Relgrad   Radius      Rho
    0           -0.32           -0.12           -0.37           -0.59           -0.89           -0.15              -1      5.3e+03      0.036       10        1   ++
    1           -0.17            0.22            0.41            -1.3           -0.94           -0.88            -1.7      5.2e+03     0.0094    1e+02        1   ++
    2           -0.18            0.25            0.42            -1.3           -0.96              -1            -1.7      5.2e+03    0.00022    1e+03        1   ++
    3           -0.18            0.25            0.42            -1.3           -0.96              -1            -1.7      5.2e+03    2.3e-07    1e+03        1   ++
Model 2/36
File biogeme.toml has been parsed.
*** Estimate b21multiple_models_000000
Optimization algorithm: hybrid Newton/BFGS with simple bounds [simple_bounds]
** Optimization: Newton with trust region for simple bounds
Iter.         ASC_CAR    ASC_CAR_male       ASC_TRAIN  ASC_TRAIN_male          B_COST          B_TIME     Function    Relgrad   Radius      Rho
    0           -0.52          -0.039           -0.33           -0.81           -0.89           -0.63      5.3e+03      0.043       10      1.1   ++
    1           -0.47            0.28           0.038            -1.2              -1            -1.2      5.2e+03     0.0093    1e+02      1.1   ++
    2           -0.46            0.31           0.089            -1.2            -1.1            -1.2      5.2e+03    0.00033    1e+03        1   ++
    3           -0.46            0.31           0.089            -1.2            -1.1            -1.2      5.2e+03    4.1e-07    1e+03        1   ++
Model 3/36
Model 4/36
File biogeme.toml has been parsed.
*** Estimate b21multiple_models_000000
Optimization algorithm: hybrid Newton/BFGS with simple bounds [simple_bounds]
** Optimization: Newton with trust region for simple bounds
Iter.         ASC_CAR       ASC_TRAIN          B_COST       B_COST_GA          B_TIME     Function    Relgrad   Radius      Rho
    0            -0.5           -0.92            -0.9           0.097           -0.67      5.4e+03      0.041       10      1.1   ++
    1           -0.13           -0.71           -0.98           -0.74            -1.2      5.3e+03     0.0075    1e+02      1.1   ++
    2          -0.097           -0.68              -1              -1            -1.3      5.3e+03     0.0002    1e+03        1   ++
    3          -0.097           -0.68              -1              -1            -1.3      5.3e+03    1.2e-06    1e+03        1   ++
Model 5/36
File biogeme.toml has been parsed.
*** Estimate b21multiple_models_000000
Optimization algorithm: hybrid Newton/BFGS with simple bounds [simple_bounds]
** Optimization: Newton with trust region for simple bounds
Iter.         ASC_CAR      ASC_CAR_GA       ASC_TRAIN    ASC_TRAIN_GA          B_COST B_COST_inc-100+ B_COST_inc-50-1 B_COST_inc-unde B_COST_inc-unkn          B_TIME     Function    Relgrad   Radius      Rho
    0           -0.41           -0.23              -1            0.85           -0.75           -0.21           -0.31           -0.11          -0.052            -0.7      5.2e+03      0.049       10      1.1   ++
    1           -0.28           -0.19            -1.2             1.9            -1.5            0.54            0.19           -0.54            0.71            -1.1        5e+03      0.012    1e+02      1.1   ++
    2           -0.26           -0.21            -1.3               2            -1.6            0.59            0.23           -0.78            0.78            -1.2        5e+03     0.0006    1e+03        1   ++
    3           -0.26           -0.21            -1.3               2            -1.6            0.59            0.23           -0.78            0.78            -1.2        5e+03    1.8e-06    1e+03        1   ++
Model 6/36
File biogeme.toml has been parsed.
*** Estimate b21multiple_models_000000
Optimization algorithm: hybrid Newton/BFGS with simple bounds [simple_bounds]
** Optimization: Newton with trust region for simple bounds
Iter.         ASC_CAR      ASC_CAR_GA    ASC_CAR_male       ASC_TRAIN    ASC_TRAIN_GA  ASC_TRAIN_male          B_COST          B_TIME     Function    Relgrad   Radius      Rho
    0           -0.46           -0.21           -0.15           -0.51               1            -0.8           -0.88            -0.6      5.1e+03      0.045       10      1.1   ++
    1            -0.6           -0.37            0.36           -0.54             1.7           -0.96            -1.1            -1.1        5e+03      0.012    1e+02      1.1   ++
    2           -0.61           -0.41            0.41           -0.53             1.9            -1.1            -1.1            -1.2      4.9e+03     0.0007    1e+03        1   ++
    3           -0.61           -0.41            0.41           -0.53             1.9            -1.1            -1.1            -1.2      4.9e+03    4.3e-06    1e+03        1   ++
Model 7/36
File biogeme.toml has been parsed.
*** Estimate b21multiple_models_000000
Optimization algorithm: hybrid Newton/BFGS with simple bounds [simple_bounds]
** Optimization: Newton with trust region for simple bounds
Iter.         ASC_CAR       ASC_TRAIN          B_COST          B_TIME     Function    Relgrad   Radius      Rho
    0            -0.3           -0.74           -0.39              -1      5.5e+03      0.044       10        1   ++
    1         -0.0041           -0.54           -0.93            -1.6      5.3e+03     0.0052    1e+02      1.1   ++
    2          0.0019           -0.51              -1            -1.7      5.3e+03    0.00015    1e+03        1   ++
    3          0.0019           -0.51              -1            -1.7      5.3e+03    1.4e-07    1e+03        1   ++
Model 8/36
File biogeme.toml has been parsed.
*** Estimate b21multiple_models_000000
Optimization algorithm: hybrid Newton/BFGS with simple bounds [simple_bounds]
** Optimization: Newton with trust region for simple bounds
Iter.         ASC_CAR      ASC_CAR_GA       ASC_TRAIN    ASC_TRAIN_GA          B_COST       B_COST_GA          B_TIME     lambda_time     Function    Relgrad   Radius      Rho
    0           -0.45            -0.2           -0.73            0.39           -0.73           -0.15              -1             1.6      5.5e+03       0.12        1     0.83    +
    1           -0.19           -0.33            -1.1             1.4            -1.1            -0.2            -1.3            0.91      5.1e+03      0.022       10      1.1   ++
    2           0.034            -1.1           -0.87               2            -1.1             1.3            -1.9            0.26        5e+03      0.013       10     0.81    +
    3          -0.066           -0.95              -1               2            -1.1               1            -1.7            0.36        5e+03     0.0012    1e+02        1   ++
    4          -0.068           -0.89              -1               2            -1.1            0.92            -1.7            0.38        5e+03    3.3e-05    1e+03        1   ++
    5          -0.068           -0.89              -1               2            -1.1            0.92            -1.7            0.38        5e+03    8.2e-08    1e+03        1   ++
Model 9/36
File biogeme.toml has been parsed.
*** Estimate b21multiple_models_000000
Optimization algorithm: hybrid Newton/BFGS with simple bounds [simple_bounds]
** Optimization: Newton with trust region for simple bounds
Iter.         ASC_CAR       ASC_TRAIN          B_COST          B_TIME     lambda_time     Function    Relgrad   Radius      Rho
    0            -0.5           -0.71           -0.82              -1             1.7      5.6e+03       0.15        1     0.77    +
    1           -0.35           -0.77            -1.3            -1.7            0.69      5.4e+03      0.049       10        1   ++
    2           0.017           -0.47              -1            -1.7            0.53      5.3e+03      0.002    1e+02     0.96   ++
    3         -0.0043           -0.48            -1.1            -1.7            0.51      5.3e+03    1.3e-05    1e+03        1   ++
    4         -0.0043           -0.48            -1.1            -1.7            0.51      5.3e+03    1.8e-09    1e+03        1   ++
Model 10/36
File biogeme.toml has been parsed.
*** Estimate b21multiple_models_000000
Optimization algorithm: hybrid Newton/BFGS with simple bounds [simple_bounds]
** Optimization: Newton with trust region for simple bounds
Iter.         ASC_CAR      ASC_CAR_GA    ASC_CAR_male       ASC_TRAIN    ASC_TRAIN_GA  ASC_TRAIN_male          B_COST B_COST_inc-100+ B_COST_inc-50-1 B_COST_inc-unde B_COST_inc-unkn          B_TIME     Function    Relgrad   Radius      Rho
    0           -0.47           -0.21           -0.15           -0.51               1           -0.82           -0.69           -0.15           -0.32          -0.094          -0.041           -0.59      5.1e+03      0.045       10      1.1   ++
    1           -0.62           -0.29            0.38           -0.57             1.8           -0.94            -1.4            0.51            0.15           -0.37            0.71            -1.1      4.9e+03      0.012    1e+02      1.1   ++
    2           -0.65           -0.33            0.44           -0.58             1.9            -1.1            -1.5            0.55            0.16            -0.6            0.75            -1.2      4.9e+03    0.00074    1e+03        1   ++
    3           -0.65           -0.33            0.44           -0.58             1.9            -1.1            -1.5            0.55            0.16            -0.6            0.75            -1.2      4.9e+03    5.9e-06    1e+03        1   ++
Model 11/36
File biogeme.toml has been parsed.
*** Estimate b21multiple_models_000000
Optimization algorithm: hybrid Newton/BFGS with simple bounds [simple_bounds]
** Optimization: Newton with trust region for simple bounds
Iter.         ASC_CAR      ASC_CAR_GA    ASC_CAR_male       ASC_TRAIN    ASC_TRAIN_GA  ASC_TRAIN_male          B_COST       B_COST_GA          B_TIME     Function    Relgrad   Radius      Rho
    0           -0.46           -0.19           -0.15           -0.51               1            -0.8           -0.88         -0.0059            -0.6      5.1e+03      0.045       10      1.1   ++
    1           -0.61           -0.93            0.36           -0.54             1.7           -0.97            -1.1            0.93            -1.1      4.9e+03      0.012    1e+02      1.1   ++
    2           -0.62           -0.98            0.41           -0.54             1.9            -1.1            -1.1            0.89            -1.2      4.9e+03     0.0007    1e+03        1   ++
    3           -0.62           -0.98            0.41           -0.54             1.9            -1.1            -1.1            0.89            -1.2      4.9e+03    4.4e-06    1e+03        1   ++
Model 12/36
File biogeme.toml has been parsed.
*** Estimate b21multiple_models_000000
Optimization algorithm: hybrid Newton/BFGS with simple bounds [simple_bounds]
** Optimization: Newton with trust region for simple bounds
Iter.         ASC_CAR    ASC_CAR_male       ASC_TRAIN  ASC_TRAIN_male          B_COST       B_COST_GA          B_TIME     Function    Relgrad   Radius      Rho
    0           -0.52           -0.04           -0.33           -0.81           -0.91            0.11           -0.63      5.3e+03      0.043       10      1.1   ++
    1           -0.42            0.29           0.052            -1.1           -0.98           -0.72            -1.2      5.2e+03     0.0097    1e+02      1.1   ++
    2           -0.41            0.31            0.11            -1.2              -1              -1            -1.3      5.2e+03    0.00036    1e+03        1   ++
    3           -0.41            0.31            0.11            -1.2              -1              -1            -1.3      5.2e+03    9.1e-07    1e+03        1   ++
Model 13/36
File biogeme.toml has been parsed.
*** Estimate b21multiple_models_000000
Optimization algorithm: hybrid Newton/BFGS with simple bounds [simple_bounds]
** Optimization: Newton with trust region for simple bounds
Iter.         ASC_CAR    ASC_CAR_male       ASC_TRAIN  ASC_TRAIN_male          B_COST B_COST_inc-100+ B_COST_inc-50-1 B_COST_inc-unde B_COST_inc-unkn          B_TIME     Function    Relgrad   Radius      Rho
    0           -0.56           0.012           -0.36           -0.78              -1            0.16           -0.03            0.12            0.29           -0.63      5.3e+03      0.043       10      1.1   ++
    1           -0.48             0.3           0.017            -1.1            -1.4            0.49            0.17           -0.28            0.72            -1.2      5.2e+03     0.0097    1e+02      1.1   ++
    2           -0.47            0.33           0.069            -1.2            -1.5            0.53            0.18           -0.41            0.78            -1.3      5.2e+03    0.00037    1e+03        1   ++
    3           -0.47            0.33           0.069            -1.2            -1.5            0.53            0.18           -0.41            0.78            -1.3      5.2e+03    5.1e-07    1e+03        1   ++
Model 14/36
File biogeme.toml has been parsed.
*** Estimate b21multiple_models_000000
Optimization algorithm: hybrid Newton/BFGS with simple bounds [simple_bounds]
** Optimization: Newton with trust region for simple bounds
Iter.         ASC_CAR       ASC_TRAIN          B_COST B_COST_inc-100+ B_COST_inc-50-1 B_COST_inc-unde B_COST_inc-unkn          B_TIME     lambda_time     Function    Relgrad   Radius      Rho
    0           -0.48           -0.72           -0.74           -0.31           -0.26          -0.092          -0.047              -1             1.6      5.6e+03       0.13        1     0.81    +
    1           -0.22           -0.64           -0.91            0.12           -0.47           -0.36          -0.013            -1.6            0.65      5.3e+03      0.038       10        1   ++
    2           0.012           -0.49            -1.5             0.6            0.27           -0.59             0.8            -1.7            0.51      5.3e+03    0.00074    1e+02        1   ++
    3          0.0042           -0.49            -1.6            0.62            0.28           -0.65            0.81            -1.7             0.5      5.3e+03      7e-06    1e+03        1   ++
    4          0.0042           -0.49            -1.6            0.62            0.28           -0.65            0.81            -1.7             0.5      5.3e+03    1.3e-09    1e+03        1   ++
Model 15/36
File biogeme.toml has been parsed.
*** Estimate b21multiple_models_000000
Optimization algorithm: hybrid Newton/BFGS with simple bounds [simple_bounds]
** Optimization: Newton with trust region for simple bounds
Iter.         ASC_CAR      ASC_CAR_GA    ASC_CAR_male       ASC_TRAIN    ASC_TRAIN_GA  ASC_TRAIN_male          B_COST B_COST_inc-100+ B_COST_inc-50-1 B_COST_inc-unde B_COST_inc-unkn          B_TIME     Function    Relgrad   Radius      Rho
    0           -0.34           -0.18           -0.16           -0.44            0.47            -0.6           -0.69           -0.28           -0.25          -0.075          -0.042              -1      5.2e+03      0.036       10        1   ++
    1           -0.37           -0.29            0.34           -0.24               2           -0.98            -1.5             0.6            0.22            -0.3            0.76            -1.6      4.9e+03      0.018    1e+02        1   ++
    2           -0.43           -0.34            0.41           -0.25               2            -1.1            -1.5            0.63            0.21           -0.59            0.78            -1.7      4.9e+03     0.0011    1e+03        1   ++
    3           -0.43           -0.34            0.42           -0.25             2.1            -1.2            -1.6            0.63            0.21           -0.61            0.78            -1.7      4.9e+03    9.2e-06    1e+04        1   ++
    4           -0.43           -0.34            0.42           -0.25             2.1            -1.2            -1.6            0.63            0.21           -0.61            0.78            -1.7      4.9e+03    9.6e-08    1e+04        1   ++
Model 16/36
File biogeme.toml has been parsed.
*** Estimate b21multiple_models_000000
Optimization algorithm: hybrid Newton/BFGS with simple bounds [simple_bounds]
** Optimization: Newton with trust region for simple bounds
Iter.         ASC_CAR      ASC_CAR_GA    ASC_CAR_male       ASC_TRAIN    ASC_TRAIN_GA  ASC_TRAIN_male          B_COST          B_TIME     lambda_time     Function    Relgrad   Radius      Rho
    0           -0.45           -0.24           -0.25           -0.46            0.57           -0.64              -1           -0.92             1.9      5.5e+03       0.22        1     0.73    +
    1           -0.47           -0.36            0.12           -0.56             1.6            -1.1           -0.95              -1             1.2        5e+03      0.044       10      1.2   ++
    2           -0.47           -0.36            0.12           -0.56             1.6            -1.1           -0.95              -1             1.2        5e+03      0.044      1.6      -11    -
    3           -0.47           -0.36            0.12           -0.56             1.6            -1.1           -0.95              -1             1.2        5e+03      0.044      0.8    -0.46    -
    4           -0.48            -0.4             0.3           -0.55             1.9            -1.2            -1.1            -1.5            0.42      4.9e+03      0.017        8     0.99   ++
    5           -0.42           -0.45            0.41           -0.21             1.9            -1.1            -1.1            -1.7            0.33      4.9e+03    0.00079       80     0.98   ++
    6           -0.42           -0.45            0.41           -0.21             1.9            -1.1            -1.1            -1.7            0.33      4.9e+03    3.4e-06       80        1   ++
Model 17/36
File biogeme.toml has been parsed.
*** Estimate b21multiple_models_000000
Optimization algorithm: hybrid Newton/BFGS with simple bounds [simple_bounds]
** Optimization: Newton with trust region for simple bounds
Iter.         ASC_CAR      ASC_CAR_GA    ASC_CAR_male       ASC_TRAIN    ASC_TRAIN_GA  ASC_TRAIN_male          B_COST       B_COST_GA          B_TIME     lambda_time     Function    Relgrad   Radius      Rho
    0           -0.44           -0.24           -0.24           -0.46            0.57           -0.65              -1           -0.17           -0.92             1.9      5.5e+03       0.22        1     0.73    +
    1           -0.47           -0.33            0.13           -0.55             1.6            -1.1           -0.94           -0.19              -1             1.2        5e+03      0.046       10      1.2   ++
    2           -0.47           -0.33            0.13           -0.55             1.6            -1.1           -0.94           -0.19              -1             1.2        5e+03      0.046      1.8      -14    -
    3           -0.47           -0.33            0.13           -0.55             1.6            -1.1           -0.94           -0.19              -1             1.2        5e+03      0.046     0.88    -0.88    -
    4           -0.46           -0.35            0.34           -0.56             1.9            -1.2            -1.2           -0.16            -1.6            0.36      4.9e+03      0.023      8.8     0.91   ++
    5           -0.43            -1.3            0.41           -0.21             1.9            -1.1            -1.1             1.6            -1.7            0.34      4.9e+03     0.0036      8.8     0.82    +
    6           -0.42            -1.2            0.41           -0.22               2            -1.2            -1.1             1.2            -1.7            0.33      4.9e+03    0.00065       88      1.2   ++
    7           -0.42            -1.1            0.41           -0.22               2            -1.2            -1.1            0.98            -1.7            0.33      4.9e+03    0.00011  8.8e+02      1.1   ++
    8           -0.42            -1.1            0.41           -0.22               2            -1.2            -1.1            0.98            -1.7            0.33      4.9e+03    5.3e-06  8.8e+02        1   ++
Model 18/36
File biogeme.toml has been parsed.
*** Estimate b21multiple_models_000000
Optimization algorithm: hybrid Newton/BFGS with simple bounds [simple_bounds]
** Optimization: Newton with trust region for simple bounds
Iter.         ASC_CAR    ASC_CAR_male       ASC_TRAIN  ASC_TRAIN_male          B_COST B_COST_inc-100+ B_COST_inc-50-1 B_COST_inc-unde B_COST_inc-unkn          B_TIME     lambda_time     Function    Relgrad   Radius      Rho
    0           -0.43           -0.22           -0.44           -0.66              -1           -0.42           -0.35           -0.11           -0.06            -0.9               2      5.7e+03       0.27        1     0.68    +
    1           -0.52            0.77            0.56            -1.4           -0.96             0.3           -0.49           -0.51           0.066            -1.6             1.1      5.3e+03      0.049        1     0.84    +
    2           -0.14            0.38            0.66            -1.2            -1.6            0.62            0.25           -0.36            0.96            -2.2            0.34      5.1e+03      0.029        1     0.87    +
    3            -0.3            0.32            0.33            -1.3            -1.5            0.59            0.23           -0.37            0.84            -1.7            0.42      5.1e+03     0.0026       10     0.96   ++
    4            -0.3            0.32            0.35            -1.3            -1.6            0.59            0.23           -0.37            0.85            -1.7            0.45      5.1e+03    7.5e-05    1e+02     0.98   ++
    5            -0.3            0.32            0.35            -1.3            -1.6            0.59            0.23           -0.37            0.85            -1.7            0.45      5.1e+03    9.2e-08    1e+02        1   ++
Model 19/36
File biogeme.toml has been parsed.
*** Estimate b21multiple_models_000000
Optimization algorithm: hybrid Newton/BFGS with simple bounds [simple_bounds]
** Optimization: Newton with trust region for simple bounds
Iter.         ASC_CAR    ASC_CAR_male       ASC_TRAIN  ASC_TRAIN_male          B_COST          B_TIME     lambda_time     Function    Relgrad   Radius      Rho
    0           -0.45           -0.24           -0.43           -0.62              -1           -0.89             1.8      5.6e+03        0.2        1     0.72    +
    1           -0.65            0.58            0.39            -1.5              -1            -1.6            0.85      5.2e+03      0.028       10     0.94   ++
    2           -0.21             0.3            0.48            -1.3            -1.1            -1.9            0.42      5.1e+03     0.0096    1e+02     0.95   ++
    3           -0.29             0.3            0.36            -1.3            -1.1            -1.7            0.45      5.1e+03     0.0004    1e+03     0.98   ++
    4           -0.29             0.3            0.36            -1.3            -1.1            -1.7            0.45      5.1e+03    6.7e-07    1e+03        1   ++
Model 20/36
File biogeme.toml has been parsed.
*** Estimate b21multiple_models_000000
Optimization algorithm: hybrid Newton/BFGS with simple bounds [simple_bounds]
** Optimization: Newton with trust region for simple bounds
Iter.         ASC_CAR    ASC_CAR_male       ASC_TRAIN  ASC_TRAIN_male          B_COST       B_COST_GA          B_TIME     lambda_time     Function    Relgrad   Radius      Rho
    0           -0.44           -0.23           -0.43           -0.62              -1           -0.18           -0.89             1.9      5.6e+03        0.2        1     0.72    +
    1           -0.63            0.58            0.39            -1.5           -0.97           -0.42            -1.6            0.86      5.2e+03      0.029       10     0.94   ++
    2           -0.14             0.3            0.51            -1.3              -1              -1            -1.9             0.4      5.1e+03      0.011    1e+02     0.93   ++
    3           -0.23            0.31            0.39            -1.3              -1            -1.1            -1.7            0.44      5.1e+03    0.00049    1e+03     0.98   ++
    4           -0.23            0.31            0.39            -1.3              -1            -1.1            -1.7            0.44      5.1e+03    1.9e-06    1e+03        1   ++
Model 21/36
File biogeme.toml has been parsed.
*** Estimate b21multiple_models_000000
Optimization algorithm: hybrid Newton/BFGS with simple bounds [simple_bounds]
** Optimization: Newton with trust region for simple bounds
Iter.         ASC_CAR       ASC_TRAIN          B_COST       B_COST_GA          B_TIME     lambda_time     Function    Relgrad   Radius      Rho
    0            -0.5           -0.71           -0.82           -0.17              -1             1.7      5.6e+03       0.15        1     0.77    +
    1           -0.33           -0.78            -1.3           -0.38            -1.6            0.69      5.4e+03      0.049       10        1   ++
    2           0.085           -0.44           -0.96            -1.2            -1.7            0.53      5.3e+03      0.002    1e+02     0.96   ++
    3            0.06           -0.46              -1            -1.1            -1.7            0.51      5.3e+03      1e-05    1e+03        1   ++
    4            0.06           -0.46              -1            -1.1            -1.7            0.51      5.3e+03    7.2e-10    1e+03        1   ++
Model 22/36
File biogeme.toml has been parsed.
*** Estimate b21multiple_models_000000
Optimization algorithm: hybrid Newton/BFGS with simple bounds [simple_bounds]
** Optimization: Newton with trust region for simple bounds
Iter.         ASC_CAR      ASC_CAR_GA       ASC_TRAIN    ASC_TRAIN_GA          B_COST          B_TIME     Function    Relgrad   Radius      Rho
    0           -0.27           -0.12           -0.76            0.19           -0.37              -1      5.4e+03      0.044       10        1   ++
    1          -0.072           -0.11           -0.97             2.4           -0.96            -1.6        5e+03      0.027    1e+02     0.94   ++
    2          -0.067           -0.29              -1             2.1            -1.1            -1.7        5e+03    0.00066    1e+03        1   ++
    3          -0.067           -0.29              -1             2.1            -1.1            -1.7        5e+03    2.1e-06    1e+03        1   ++
Model 23/36
File biogeme.toml has been parsed.
*** Estimate b21multiple_models_000000
Optimization algorithm: hybrid Newton/BFGS with simple bounds [simple_bounds]
** Optimization: Newton with trust region for simple bounds
Iter.         ASC_CAR    ASC_CAR_male       ASC_TRAIN  ASC_TRAIN_male          B_COST          B_TIME     Function    Relgrad   Radius      Rho
    0           -0.33           -0.13           -0.37           -0.59           -0.89              -1      5.3e+03      0.036       10        1   ++
    1           -0.22            0.22             0.4            -1.3              -1            -1.6      5.2e+03     0.0091    1e+02        1   ++
    2           -0.24            0.25             0.4            -1.3              -1            -1.7      5.2e+03    0.00022    1e+03        1   ++
    3           -0.24            0.25             0.4            -1.3              -1            -1.7      5.2e+03    2.4e-07    1e+03        1   ++
Model 24/36
File biogeme.toml has been parsed.
*** Estimate b21multiple_models_000000
Optimization algorithm: hybrid Newton/BFGS with simple bounds [simple_bounds]
** Optimization: Newton with trust region for simple bounds
Iter.         ASC_CAR       ASC_TRAIN          B_COST B_COST_inc-100+ B_COST_inc-50-1 B_COST_inc-unde B_COST_inc-unkn          B_TIME     Function    Relgrad   Radius      Rho
    0           -0.29           -0.75           -0.36           -0.14           -0.13          -0.045          -0.023              -1      5.5e+03      0.035       10        1   ++
    1          0.0049           -0.54            -1.4             0.6            0.37           -0.17            0.73            -1.6      5.3e+03     0.0064    1e+02      1.1   ++
    2           0.011           -0.51            -1.5            0.63            0.28           -0.64            0.75            -1.7      5.3e+03    0.00037    1e+03        1   ++
    3           0.011           -0.51            -1.5            0.63            0.28           -0.64            0.75            -1.7      5.3e+03    2.2e-06    1e+03        1   ++
Model 25/36
File biogeme.toml has been parsed.
*** Estimate b21multiple_models_000000
Optimization algorithm: hybrid Newton/BFGS with simple bounds [simple_bounds]
** Optimization: Newton with trust region for simple bounds
Iter.         ASC_CAR    ASC_CAR_male       ASC_TRAIN  ASC_TRAIN_male          B_COST B_COST_inc-100+ B_COST_inc-50-1 B_COST_inc-unde B_COST_inc-unkn          B_TIME     Function    Relgrad   Radius      Rho
    0           -0.32           -0.13           -0.37            -0.6           -0.72           -0.28           -0.27          -0.082          -0.045              -1      5.3e+03      0.036       10        1   ++
    1           -0.22            0.24            0.38            -1.3            -1.4            0.56            0.22           -0.22            0.76            -1.6      5.2e+03     0.0092    1e+02        1   ++
    2           -0.25            0.27            0.38            -1.3            -1.5             0.6            0.23           -0.36            0.81            -1.7      5.2e+03    0.00022    1e+03        1   ++
    3           -0.25            0.27            0.38            -1.3            -1.5             0.6            0.23           -0.36            0.81            -1.7      5.2e+03    2.4e-07    1e+03        1   ++
Model 26/36
File biogeme.toml has been parsed.
*** Estimate b21multiple_models_000000
Optimization algorithm: hybrid Newton/BFGS with simple bounds [simple_bounds]
** Optimization: Newton with trust region for simple bounds
Iter.         ASC_CAR      ASC_CAR_GA       ASC_TRAIN    ASC_TRAIN_GA          B_COST       B_COST_GA          B_TIME     Function    Relgrad   Radius      Rho
    0           -0.27           -0.12           -0.76            0.19           -0.36          -0.089              -1      5.4e+03      0.044       10        1   ++
    1           -0.08           -0.61           -0.98             2.4           -0.98            0.71            -1.6        5e+03      0.027    1e+02     0.93   ++
    2          -0.071           -0.87            -1.1             2.1            -1.1            0.92            -1.7        5e+03    0.00067    1e+03        1   ++
    3          -0.071           -0.87            -1.1             2.1            -1.1            0.92            -1.7        5e+03      2e-06    1e+03        1   ++
Model 27/36
File biogeme.toml has been parsed.
*** Estimate b21multiple_models_000000
Optimization algorithm: hybrid Newton/BFGS with simple bounds [simple_bounds]
** Optimization: Newton with trust region for simple bounds
Iter.         ASC_CAR       ASC_TRAIN          B_COST       B_COST_GA          B_TIME     Function    Relgrad   Radius      Rho
    0            -0.3           -0.74           -0.39          -0.095              -1      5.5e+03      0.043       10        1   ++
    1           0.037           -0.52           -0.88           -0.53            -1.6      5.3e+03     0.0053    1e+02      1.1   ++
    2           0.061           -0.48           -0.95           -0.97            -1.7      5.3e+03    0.00033    1e+03      1.1   ++
    3           0.063           -0.48           -0.95            -1.1            -1.7      5.3e+03    1.1e-05    1e+04        1   ++
    4           0.063           -0.48           -0.95            -1.1            -1.7      5.3e+03    1.7e-08    1e+04        1   ++
Model 28/36
File biogeme.toml has been parsed.
*** Estimate b21multiple_models_000000
Optimization algorithm: hybrid Newton/BFGS with simple bounds [simple_bounds]
** Optimization: Newton with trust region for simple bounds
Iter.         ASC_CAR       ASC_TRAIN          B_COST B_COST_inc-100+ B_COST_inc-50-1 B_COST_inc-unde B_COST_inc-unkn          B_TIME     Function    Relgrad   Radius      Rho
    0           -0.49           -0.92              -1             0.2         -0.0026           0.026            0.21           -0.66      5.4e+03      0.041       10      1.1   ++
    1           -0.17           -0.74            -1.4            0.52            0.22           -0.53            0.69            -1.2      5.3e+03     0.0075    1e+02      1.1   ++
    2           -0.15           -0.71            -1.5            0.56            0.24           -0.66            0.75            -1.3      5.3e+03     0.0002    1e+03        1   ++
    3           -0.15           -0.71            -1.5            0.56            0.24           -0.66            0.75            -1.3      5.3e+03    1.5e-07    1e+03        1   ++
Model 29/36
File biogeme.toml has been parsed.
*** Estimate b21multiple_models_000000
Optimization algorithm: hybrid Newton/BFGS with simple bounds [simple_bounds]
** Optimization: Newton with trust region for simple bounds
Iter.         ASC_CAR      ASC_CAR_GA    ASC_CAR_male       ASC_TRAIN    ASC_TRAIN_GA  ASC_TRAIN_male          B_COST       B_COST_GA          B_TIME     Function    Relgrad   Radius      Rho
    0           -0.34           -0.18           -0.16           -0.43            0.47           -0.59           -0.76           -0.12              -1      5.2e+03      0.042       10        1   ++
    1           -0.36           -0.91            0.33           -0.21               2              -1              -1            0.89            -1.6      4.9e+03      0.018    1e+02        1   ++
    2           -0.39           -0.99            0.37            -0.2               2            -1.2            -1.1            0.91            -1.7      4.9e+03     0.0011    1e+03        1   ++
    3           -0.39           -0.99            0.38            -0.2               2            -1.2            -1.1            0.91            -1.7      4.9e+03    8.4e-06    1e+04        1   ++
    4           -0.39           -0.99            0.38            -0.2               2            -1.2            -1.1            0.91            -1.7      4.9e+03    5.4e-10    1e+04        1   ++
Model 30/36
File biogeme.toml has been parsed.
*** Estimate b21multiple_models_000000
Optimization algorithm: hybrid Newton/BFGS with simple bounds [simple_bounds]
** Optimization: Newton with trust region for simple bounds
Iter.         ASC_CAR      ASC_CAR_GA    ASC_CAR_male       ASC_TRAIN    ASC_TRAIN_GA  ASC_TRAIN_male          B_COST B_COST_inc-100+ B_COST_inc-50-1 B_COST_inc-unde B_COST_inc-unkn          B_TIME     lambda_time     Function    Relgrad   Radius      Rho
    0           -0.43           -0.25           -0.22           -0.48            0.63           -0.69              -1           -0.42           -0.35           -0.11           -0.06           -0.93               2      5.6e+03       0.29        1     0.69    +
    1           -0.35           -0.33            0.26           -0.46             1.6            -1.1            -0.7            0.05           -0.38           -0.23         -0.0044           -0.92             1.5      5.1e+03      0.079       10      1.1   ++
    2           -0.35           -0.33            0.26           -0.46             1.6            -1.1            -0.7            0.05           -0.38           -0.23         -0.0044           -0.92             1.5      5.1e+03      0.079        5 -1.4e+03    -
    3           -0.35           -0.33            0.26           -0.46             1.6            -1.1            -0.7            0.05           -0.38           -0.23         -0.0044           -0.92             1.5      5.1e+03      0.079      2.5      -23    -
    4           -0.35           -0.33            0.26           -0.46             1.6            -1.1            -0.7            0.05           -0.38           -0.23         -0.0044           -0.92             1.5      5.1e+03      0.079      1.2     -1.8    -
    5           -0.44           -0.41            0.53           -0.31             2.1           -0.96            -1.3           -0.14           -0.44           -0.44           0.074              -2            0.26      4.9e+03      0.022      1.2     0.69    +
    6           -0.45           -0.36            0.44           -0.26               2            -1.1            -1.5            0.66            0.23           -0.58            0.89            -1.7            0.31      4.9e+03     0.0046       12     0.91   ++
    7           -0.45           -0.37            0.45           -0.26               2            -1.1            -1.6            0.63            0.22           -0.59            0.82            -1.7            0.33      4.9e+03    9.7e-05  1.2e+02        1   ++
    8           -0.45           -0.37            0.45           -0.26               2            -1.1            -1.6            0.63            0.22           -0.59            0.82            -1.7            0.33      4.9e+03    1.4e-06  1.2e+02        1   ++
Model 31/36
File biogeme.toml has been parsed.
*** Estimate b21multiple_models_000000
Optimization algorithm: hybrid Newton/BFGS with simple bounds [simple_bounds]
** Optimization: Newton with trust region for simple bounds
Iter.         ASC_CAR      ASC_CAR_GA       ASC_TRAIN    ASC_TRAIN_GA          B_COST          B_TIME     lambda_time     Function    Relgrad   Radius      Rho
    0           -0.45           -0.21           -0.73            0.39           -0.73              -1             1.6      5.5e+03       0.12        1     0.83    +
    1            -0.2           -0.35            -1.1             1.4            -1.1            -1.3             0.9      5.1e+03      0.022       10      1.1   ++
    2           0.037           -0.36           -0.87               2            -1.1            -1.9            0.26        5e+03      0.012       10     0.82    +
    3          -0.061           -0.31              -1               2            -1.1            -1.7            0.36        5e+03     0.0012    1e+02        1   ++
    4          -0.064           -0.31              -1               2            -1.1            -1.7            0.38        5e+03    3.1e-05    1e+03        1   ++
    5          -0.064           -0.31              -1               2            -1.1            -1.7            0.38        5e+03    3.6e-09    1e+03        1   ++
Model 32/36
File biogeme.toml has been parsed.
*** Estimate b21multiple_models_000000
Optimization algorithm: hybrid Newton/BFGS with simple bounds [simple_bounds]
** Optimization: Newton with trust region for simple bounds
Iter.         ASC_CAR      ASC_CAR_GA       ASC_TRAIN    ASC_TRAIN_GA          B_COST B_COST_inc-100+ B_COST_inc-50-1 B_COST_inc-unde B_COST_inc-unkn          B_TIME     lambda_time     Function    Relgrad   Radius      Rho
    0           -0.43           -0.19           -0.73            0.37           -0.67           -0.28           -0.23          -0.082          -0.042              -1             1.6      5.4e+03        0.1        1     0.86    +
    1           -0.19           -0.32            -1.1             1.4           -0.84           -0.12           -0.37           -0.21          -0.031            -1.3            0.83      5.1e+03      0.023       10      1.1   ++
    2          0.0088           -0.24           -0.93             2.1            -1.6            0.63            0.26           -0.57            0.83            -1.9            0.28        5e+03     0.0077    1e+02     0.91   ++
    3          -0.065           -0.22              -1             2.1            -1.6            0.67            0.28           -0.78            0.84            -1.7            0.36        5e+03    0.00067    1e+03        1   ++
    4          -0.068           -0.22              -1             2.1            -1.6            0.66            0.28           -0.79            0.84            -1.7            0.37        5e+03    9.8e-06    1e+04        1   ++
    5          -0.068           -0.22              -1             2.1            -1.6            0.66            0.28           -0.79            0.84            -1.7            0.37        5e+03      4e-10    1e+04        1   ++
Model 33/36
File biogeme.toml has been parsed.
*** Estimate b21multiple_models_000000
Optimization algorithm: hybrid Newton/BFGS with simple bounds [simple_bounds]
** Optimization: Newton with trust region for simple bounds
Iter.         ASC_CAR      ASC_CAR_GA       ASC_TRAIN    ASC_TRAIN_GA          B_COST          B_TIME     Function    Relgrad   Radius      Rho
    0           -0.36           -0.25           -0.88            0.66              -1           -0.79      5.2e+03      0.047       10      1.1   ++
    1           -0.28           -0.28            -1.2             1.9            -1.1            -1.1      5.1e+03      0.014    1e+02      1.1   ++
    2           -0.25            -0.3            -1.3               2            -1.1            -1.2      5.1e+03     0.0007    1e+03        1   ++
    3           -0.25            -0.3            -1.3               2            -1.1            -1.2      5.1e+03    2.4e-06    1e+03        1   ++
Model 34/36
File biogeme.toml has been parsed.
*** Estimate b21multiple_models_000000
Optimization algorithm: hybrid Newton/BFGS with simple bounds [simple_bounds]
** Optimization: Newton with trust region for simple bounds
Iter.         ASC_CAR      ASC_CAR_GA    ASC_CAR_male       ASC_TRAIN    ASC_TRAIN_GA  ASC_TRAIN_male          B_COST          B_TIME     Function    Relgrad   Radius      Rho
    0           -0.34           -0.18           -0.17           -0.44            0.47           -0.59           -0.76              -1      5.2e+03      0.044       10        1   ++
    1           -0.36           -0.35            0.33           -0.21               2              -1              -1            -1.6      4.9e+03      0.018    1e+02        1   ++
    2           -0.39           -0.42            0.37            -0.2               2            -1.2            -1.1            -1.7      4.9e+03     0.0011    1e+03        1   ++
    3           -0.39           -0.42            0.38            -0.2               2            -1.2            -1.1            -1.7      4.9e+03    8.5e-06    1e+04        1   ++
    4           -0.39           -0.42            0.38            -0.2               2            -1.2            -1.1            -1.7      4.9e+03    5.5e-10    1e+04        1   ++
Model 35/36
File biogeme.toml has been parsed.
*** Estimate b21multiple_models_000000
Optimization algorithm: hybrid Newton/BFGS with simple bounds [simple_bounds]
** Optimization: Newton with trust region for simple bounds
Iter.         ASC_CAR      ASC_CAR_GA       ASC_TRAIN    ASC_TRAIN_GA          B_COST       B_COST_GA          B_TIME     Function    Relgrad   Radius      Rho
    0           -0.43           -0.22              -1             0.9           -0.93          -0.072            -0.7      5.2e+03      0.046       10      1.1   ++
    1           -0.28           -0.87            -1.2             1.9            -1.1            0.98            -1.1      5.1e+03      0.011    1e+02      1.1   ++
    2           -0.25           -0.87            -1.3               2            -1.1            0.89            -1.2        5e+03    0.00055    1e+03        1   ++
    3           -0.25           -0.87            -1.3               2            -1.1            0.89            -1.2        5e+03    1.5e-06    1e+03        1   ++
Pareto file has been updated: b21multiple_models.pareto
Before the algorithm: 1 models, with 1 Pareto.
After the algorithm: 36 models, with 8 Pareto.
VNS algorithm completed. Postprocessing of the Pareto optimal solutions
Pareto set initialized from file with 36 elements [8 Pareto] and 0 invalid elements.
File biogeme.toml has been parsed.
*** Initial values of the parameters are obtained from the file __b21multiple_models_000000.iter
Parameter values restored from __b21multiple_models_000000.iter
Optimization algorithm: hybrid Newton/BFGS with simple bounds [simple_bounds]
** Optimization: Newton with trust region for simple bounds
Iter.         ASC_CAR      ASC_CAR_GA    ASC_CAR_male       ASC_TRAIN    ASC_TRAIN_GA  ASC_TRAIN_male          B_COST          B_TIME     lambda_time     Function    Relgrad   Radius      Rho
    0           -0.51            -0.7            0.47           -0.42             2.1              -1              -1            -1.6             0.3      4.9e+03      0.014       10     0.99   ++
    1           -0.42           -0.41            0.41           -0.22               2            -1.1            -1.1            -1.7            0.34      4.9e+03    0.00042    1e+02     0.99   ++
    2           -0.42           -0.41            0.41           -0.22               2            -1.1            -1.1            -1.7            0.34      4.9e+03    3.6e-06    1e+02        1   ++
Results saved in file b21multiple_models_000000.html
Results saved in file b21multiple_models_000000.pickle
File biogeme.toml has been parsed.
*** Initial values of the parameters are obtained from the file __b21multiple_models_000001.iter
Cannot read file __b21multiple_models_000001.iter. Statement is ignored.
Optimization algorithm: hybrid Newton/BFGS with simple bounds [simple_bounds]
** Optimization: Newton with trust region for simple bounds
Iter.         ASC_CAR      ASC_CAR_GA    ASC_CAR_male       ASC_TRAIN    ASC_TRAIN_GA  ASC_TRAIN_male          B_COST B_COST_inc-100+ B_COST_inc-50-1 B_COST_inc-unde B_COST_inc-unkn          B_TIME     lambda_time     Function    Relgrad   Radius      Rho
    0           -0.51           -0.73            0.39           -0.68             2.1              -1           -0.88          -0.028           -0.43           -0.23          -0.012            -1.4            0.32      4.9e+03      0.013       10     0.99   ++
    1           -0.45           -0.33            0.44           -0.26               2            -1.1            -1.5            0.56            0.15           -0.52            0.76            -1.7            0.35      4.9e+03     0.0019    1e+02        1   ++
    2           -0.45           -0.37            0.45           -0.26               2            -1.1            -1.6            0.63            0.21           -0.59            0.82            -1.7            0.33      4.9e+03    4.1e-05    1e+03        1   ++
    3           -0.45           -0.37            0.45           -0.26               2            -1.1            -1.6            0.63            0.21           -0.59            0.82            -1.7            0.33      4.9e+03    4.6e-07    1e+03        1   ++
Results saved in file b21multiple_models_000001.html
Results saved in file b21multiple_models_000001.pickle
File biogeme.toml has been parsed.
*** Initial values of the parameters are obtained from the file __b21multiple_models_000002.iter
Cannot read file __b21multiple_models_000002.iter. Statement is ignored.
Optimization algorithm: hybrid Newton/BFGS with simple bounds [simple_bounds]
** Optimization: Newton with trust region for simple bounds
Iter.         ASC_CAR       ASC_TRAIN          B_COST          B_TIME     Function    Relgrad   Radius      Rho
    0           -0.19           -0.62            -1.1            -1.2      5.3e+03      0.017       10      1.1   ++
    1           -0.16            -0.7            -1.1            -1.3      5.3e+03    0.00069    1e+02        1   ++
    2           -0.16            -0.7            -1.1            -1.3      5.3e+03    1.2e-06    1e+02        1   ++
Results saved in file b21multiple_models_000002.html
Results saved in file b21multiple_models_000002.pickle
File biogeme.toml has been parsed.
*** Initial values of the parameters are obtained from the file __b21multiple_models_000003.iter
Cannot read file __b21multiple_models_000003.iter. Statement is ignored.
Optimization algorithm: hybrid Newton/BFGS with simple bounds [simple_bounds]
** Optimization: Newton with trust region for simple bounds
Iter.         ASC_CAR      ASC_CAR_GA       ASC_TRAIN    ASC_TRAIN_GA          B_COST          B_TIME     Function    Relgrad   Radius      Rho
    0           -0.22           -0.15           -0.48               1           -0.93            -1.7      5.2e+03      0.049       10      1.1   ++
    1           -0.06           -0.28           -0.96             2.1            -1.1            -1.6        5e+03      0.013    1e+02        1   ++
    2          -0.065           -0.27              -1             2.1            -1.1            -1.7        5e+03    0.00034    1e+03        1   ++
    3          -0.065           -0.27              -1             2.1            -1.1            -1.7        5e+03    5.6e-07    1e+03        1   ++
Results saved in file b21multiple_models_000003.html
Results saved in file b21multiple_models_000003.pickle
File biogeme.toml has been parsed.
*** Initial values of the parameters are obtained from the file __b21multiple_models_000004.iter
Cannot read file __b21multiple_models_000004.iter. Statement is ignored.
Optimization algorithm: hybrid Newton/BFGS with simple bounds [simple_bounds]
** Optimization: Newton with trust region for simple bounds
Iter.         ASC_CAR      ASC_CAR_GA    ASC_CAR_male       ASC_TRAIN    ASC_TRAIN_GA  ASC_TRAIN_male          B_COST       B_COST_GA          B_TIME     lambda_time     Function    Relgrad   Radius      Rho
    0           -0.26           -0.88            0.16            -1.2               2           -0.33              -1          -0.015            -1.2            0.69        5e+03      0.024       10     0.94   ++
    1           -0.37            -1.4            0.39          -0.068             1.9            -1.3            -1.1             1.7            -1.7             0.2      4.9e+03     0.0075       10     0.82    +
    2           -0.42            -1.2            0.42           -0.21               2            -1.2            -1.1             1.3            -1.7            0.33      4.9e+03     0.0012    1e+02      1.1   ++
    3           -0.42            -1.1            0.41           -0.22               2            -1.2            -1.1               1            -1.7            0.33      4.9e+03    0.00022    1e+03      1.1   ++
    4           -0.42              -1            0.41           -0.22               2            -1.2            -1.1            0.92            -1.7            0.33      4.9e+03    1.6e-05    1e+04        1   ++
    5           -0.42              -1            0.41           -0.22               2            -1.2            -1.1            0.92            -1.7            0.33      4.9e+03      9e-08    1e+04        1   ++
Results saved in file b21multiple_models_000004.html
Results saved in file b21multiple_models_000004.pickle
File biogeme.toml has been parsed.
*** Initial values of the parameters are obtained from the file __b21multiple_models_000005.iter
Cannot read file __b21multiple_models_000005.iter. Statement is ignored.
Optimization algorithm: hybrid Newton/BFGS with simple bounds [simple_bounds]
** Optimization: Newton with trust region for simple bounds
Iter.         ASC_CAR       ASC_TRAIN          B_COST          B_TIME     lambda_time     Function    Relgrad   Radius      Rho
    0          -0.053           -0.42            -1.1            -1.6             0.4      5.3e+03      0.018       10        1   ++
    1        -0.00093           -0.47            -1.1            -1.7            0.52      5.3e+03     0.0011    1e+02     0.98   ++
    2         -0.0046           -0.48            -1.1            -1.7            0.51      5.3e+03    1.6e-05    1e+03        1   ++
    3         -0.0046           -0.48            -1.1            -1.7            0.51      5.3e+03    2.8e-09    1e+03        1   ++
Results saved in file b21multiple_models_000005.html
Results saved in file b21multiple_models_000005.pickle
File biogeme.toml has been parsed.
*** Initial values of the parameters are obtained from the file __b21multiple_models_000006.iter
Cannot read file __b21multiple_models_000006.iter. Statement is ignored.
Optimization algorithm: hybrid Newton/BFGS with simple bounds [simple_bounds]
** Optimization: Newton with trust region for simple bounds
Iter.         ASC_CAR      ASC_CAR_GA       ASC_TRAIN    ASC_TRAIN_GA          B_COST          B_TIME     lambda_time     Function    Relgrad   Radius      Rho
    0           -0.14           -0.19           -0.75               1              -1            -1.5            0.28      5.1e+03      0.025       10      1.1   ++
    1          -0.063            -0.3           -0.97               2            -1.1            -1.7            0.42        5e+03     0.0067    1e+02     0.99   ++
    2          -0.063           -0.31              -1               2            -1.1            -1.7            0.38        5e+03    8.5e-05    1e+03        1   ++
    3          -0.063           -0.31              -1               2            -1.1            -1.7            0.38        5e+03    6.6e-08    1e+03        1   ++
Results saved in file b21multiple_models_000006.html
Results saved in file b21multiple_models_000006.pickle
File biogeme.toml has been parsed.
*** Initial values of the parameters are obtained from the file __b21multiple_models_000007.iter
Cannot read file __b21multiple_models_000007.iter. Statement is ignored.
Optimization algorithm: hybrid Newton/BFGS with simple bounds [simple_bounds]
** Optimization: Newton with trust region for simple bounds
Iter.         ASC_CAR      ASC_CAR_GA    ASC_CAR_male       ASC_TRAIN    ASC_TRAIN_GA  ASC_TRAIN_male          B_COST          B_TIME     Function    Relgrad   Radius      Rho
    0           -0.45           -0.71            0.42           -0.47             2.3              -1            -0.9            -1.5      4.9e+03      0.024       10        1   ++
    1           -0.39           -0.39            0.38            -0.2               2            -1.2            -1.1            -1.7      4.9e+03     0.0011    1e+02        1   ++
    2           -0.39           -0.39            0.38            -0.2               2            -1.2            -1.1            -1.7      4.9e+03      3e-06    1e+02        1   ++
Results saved in file b21multiple_models_000007.html
Results saved in file b21multiple_models_000007.pickle
Pareto: 8
Condidered: 36
Removed: 10
summary, description = compile_estimation_results(
    non_dominated_models, use_short_names=True
)
print(summary)
                                   Model_000000     Model_000001     Model_000002      Model_000003     Model_000004         Model_000005     Model_000006     Model_000007
Number of estimated parameters                9               13                4                 6               10                    5                7                8
Sample size                                6768             6768             6768              6768             6768                 6768             6768             6768
Final log likelihood               -4881.916954     -4862.364941     -5331.252007      -5021.233909     -4879.461203         -5292.095411     -4995.755387     -4900.883369
Akaike Information Criterion        9781.833907      9750.729881     10670.504014      10054.467818      9778.922406         10594.190822     10005.510775      9817.766739
Bayesian Information Criterion      9843.213555      9839.389373     10697.783858      10095.387584      9847.122015         10628.290626     10053.250501      9872.326426
ASC_CAR (t-test)                -0.417  (-4.23)   -0.453  (-4.4)  -0.155  (-2.66)  -0.0646  (-1.25)  -0.422  (-4.29)  -0.00462  (-0.0963)  -0.064  (-1.22)  -0.389  (-3.95)
ASC_CAR_GA (t-test)             -0.447  (-2.19)  -0.371  (-1.84)                    -0.268  (-1.35)   -1.03  (-2.56)                       -0.313  (-1.59)  -0.415  (-2.02)
ASC_CAR_male (t-test)             0.412  (3.95)    0.449  (4.17)                                       0.413  (3.95)                                          0.377  (3.65)
ASC_TRAIN (t-test)              -0.219  (-2.42)  -0.261  (-2.82)  -0.701  (-8.49)    -1.05  (-14.6)   -0.22  (-2.44)      -0.485  (-7.53)   -1.03  (-13.9)  -0.203  (-2.23)
ASC_TRAIN_GA (t-test)              1.96  (21.1)     1.99  (21.1)                       2.13  (24.3)     1.96  (21.2)                          2.04  (22.8)     2.03  (22.4)
ASC_TRAIN_male (t-test)          -1.15  (-13.4)   -1.12  (-12.9)                                      -1.15  (-13.4)                                          -1.2  (-14.1)
B_COST (t-test)                    -1.09  (-15)   -1.58  (-5.84)   -1.08  (-15.9)      -1.07  (-15)    -1.1  (-15.1)       -1.08  (-15.9)    -1.1  (-14.8)   -1.06  (-15.2)
B_TIME (t-test)                  -1.69  (-21.2)   -1.71  (-21.3)   -1.28  (-12.3)    -1.68  (-21.5)    -1.7  (-21.3)       -1.67  (-21.9)   -1.67  (-21.3)    -1.7  (-21.5)
lambda_time (t-test)              0.334  (4.54)     0.329  (4.5)                                       0.334  (4.55)          0.51  (6.6)    0.382  (5.18)
B_COST_inc-100+ (t-test)                           0.629  (2.29)
B_COST_inc-50-100 (t-test)                         0.215  (0.69)
B_COST_inc-under50 (t-test)                      -0.588  (-1.08)
B_COST_inc-unknown (t-test)                        0.817  (2.56)
B_COST_GA (t-test)                                                                                     0.915  (1.85)

Explanation of the short names of the model.

for k, v in description.items():
    if k != v:
        print(f'{k}: {v} AIC={summary.at["Akaike Information Criterion", k]}')
Model_000000: ASC:MALE-GA;B_COST:no_seg;TRAIN_TT:boxcox AIC=9781.833907090044
Model_000001: ASC:MALE-GA;B_COST:INCOME;TRAIN_TT:boxcox AIC=9750.72988109646
Model_000002: ASC:no_seg;B_COST:no_seg;TRAIN_TT:linear AIC=10670.504013896403
Model_000003: ASC:GA;B_COST:no_seg;TRAIN_TT:log AIC=10054.467818331683
Model_000004: ASC:MALE-GA;B_COST:GA;TRAIN_TT:boxcox AIC=9778.922405513185
Model_000005: ASC:no_seg;B_COST:no_seg;TRAIN_TT:boxcox AIC=10594.19082155567
Model_000006: ASC:GA;B_COST:no_seg;TRAIN_TT:boxcox AIC=10005.510774519298
Model_000007: ASC:MALE-GA;B_COST:no_seg;TRAIN_TT:log AIC=9817.76673883066

Total running time of the script: (0 minutes 12.576 seconds)

Gallery generated by Sphinx-Gallery