Translated MDCEV forecasting

Michel Bierlaire, EPFL Fri Jul 25 2025, 17:34:35

Forecasting with a MDCEV model and the “translated utility” specification.

Example: translated utility
Forecasting observation 0 / 2 [10 draws]
============ Comparison ===================
Brute force: {1: '2.78', 2: '480', 3: '12.1', 4: '5.57'} objective 60.1, constraint 500, choice set {1, 2, 3, 4}
Analytical:  {1: '2.77', 2: '480', 3: '12.1', 4: '5.56'} objective 60.1, constraint 500, choice set {1, 2, 3, 4}
============ Comparison ===================
Brute force: {1: '24.2', 2: '358', 3: '100', 4: '16.9'} objective 36.1, constraint 500, choice set {1, 2, 3, 4}
Analytical:  {1: '24.2', 2: '358', 3: '100', 4: '16.9'} objective 36.1, constraint 500, choice set {1, 2, 3, 4}
============ Comparison ===================
Brute force: {1: '18.7', 2: '277', 3: '182', 4: '22.4'} objective 47.6, constraint 500, choice set {1, 2, 3, 4}
Analytical:  {1: '18.7', 2: '276', 3: '182', 4: '22.4'} objective 47.6, constraint 500, choice set {1, 2, 3, 4}
============ Comparison ===================
Brute force: {1: '19', 2: '415', 3: '46.3', 4: '19.9'} objective 42, constraint 500, choice set {1, 2, 3, 4}
Analytical:  {1: '19', 2: '415', 3: '46.3', 4: '19.9'} objective 42, constraint 500, choice set {1, 2, 3, 4}
============ Comparison ===================
Brute force: {1: '11.8', 2: '395', 3: '77.5', 4: '15.7'} objective 46.7, constraint 500, choice set {1, 2, 3, 4}
Analytical:  {1: '11.8', 2: '395', 3: '77.5', 4: '15.7'} objective 46.7, constraint 500, choice set {1, 2, 3, 4}
============ Comparison ===================
Brute force: {1: '36.1', 2: '367', 3: '78.8', 4: '17.8'} objective 43.4, constraint 500, choice set {1, 2, 3, 4}
Analytical:  {1: '36.1', 2: '367', 3: '78.8', 4: '17.8'} objective 43.4, constraint 500, choice set {1, 2, 3, 4}
============ Comparison ===================
Brute force: {1: '20.5', 2: '439', 3: '29', 4: '11.6'} objective 45.5, constraint 500, choice set {1, 2, 3, 4}
Analytical:  {1: '20.5', 2: '439', 3: '29', 4: '11.6'} objective 45.5, constraint 500, choice set {1, 2, 3, 4}
============ Comparison ===================
Brute force: {1: '27.6', 2: '395', 3: '66.3', 4: '11.2'} objective 39.6, constraint 500, choice set {1, 2, 3, 4}
Analytical:  {1: '27.6', 2: '395', 3: '66.3', 4: '11.2'} objective 39.6, constraint 500, choice set {1, 2, 3, 4}
============ Comparison ===================
Brute force: {1: '28.7', 2: '381', 3: '78', 4: '12.5'} objective 38.1, constraint 500, choice set {1, 2, 3, 4}
Analytical:  {1: '28.7', 2: '381', 3: '78', 4: '12.4'} objective 38.1, constraint 500, choice set {1, 2, 3, 4}
============ Comparison ===================
Brute force: {1: '113', 2: '320', 3: '53.5', 4: '13'} objective 49.3, constraint 500, choice set {1, 2, 3, 4}
Analytical:  {1: '113', 2: '320', 3: '53.5', 4: '13'} objective 49.3, constraint 500, choice set {1, 2, 3, 4}
Forecasting observation 1 / 2 [10 draws]
============ Comparison ===================
Brute force: {1: '9.92', 2: '405', 3: '78.3', 4: '7.23'} objective 65.4, constraint 500, choice set {1, 2, 3, 4}
Analytical:  {1: '9.92', 2: '405', 3: '78.3', 4: '7.23'} objective 65.4, constraint 500, choice set {1, 2, 3, 4}
============ Comparison ===================
Brute force: {1: '21.4', 2: '381', 3: '74.3', 4: '23.7'} objective 48.9, constraint 500, choice set {1, 2, 3, 4}
Analytical:  {1: '21.4', 2: '381', 3: '74.3', 4: '23.7'} objective 48.9, constraint 500, choice set {1, 2, 3, 4}
============ Comparison ===================
Brute force: {1: '22.5', 2: '393', 3: '73.1', 4: '11.1'} objective 57.6, constraint 500, choice set {1, 2, 3, 4}
Analytical:  {1: '22.5', 2: '393', 3: '73.1', 4: '11.1'} objective 57.6, constraint 500, choice set {1, 2, 3, 4}
============ Comparison ===================
Brute force: {1: '9.09', 2: '420', 3: '65.6', 4: '5.47'} objective 61.8, constraint 500, choice set {1, 2, 3, 4}
Analytical:  {1: '9.08', 2: '420', 3: '65.6', 4: '5.46'} objective 61.8, constraint 500, choice set {1, 2, 3, 4}
============ Comparison ===================
Brute force: {1: '23', 2: '355', 3: '104', 4: '18.5'} objective 48.6, constraint 500, choice set {1, 2, 3, 4}
Analytical:  {1: '23', 2: '355', 3: '104', 4: '18.5'} objective 48.6, constraint 500, choice set {1, 2, 3, 4}
============ Comparison ===================
Brute force: {1: '31.6', 2: '288', 3: '169', 4: '11.4'} objective 45.5, constraint 500, choice set {1, 2, 3, 4}
Analytical:  {1: '31.6', 2: '288', 3: '169', 4: '11.4'} objective 45.5, constraint 500, choice set {1, 2, 3, 4}
============ Comparison ===================
Brute force: {1: '19.8', 2: '282', 3: '190', 4: '9.02'} objective 54.8, constraint 500, choice set {1, 2, 3, 4}
Analytical:  {1: '19.8', 2: '282', 3: '190', 4: '9.01'} objective 54.8, constraint 500, choice set {1, 2, 3, 4}
============ Comparison ===================
Brute force: {1: '16.4', 2: '449', 3: '28.7', 4: '5.57'} objective 57.3, constraint 500, choice set {1, 2, 3, 4}
Analytical:  {1: '16.4', 2: '449', 3: '28.7', 4: '5.57'} objective 57.3, constraint 500, choice set {1, 2, 3, 4}
============ Comparison ===================
Brute force: {1: '10.8', 2: '364', 3: '112', 4: '12.9'} objective 57.2, constraint 500, choice set {1, 2, 3, 4}
Analytical:  {1: '10.8', 2: '364', 3: '112', 4: '12.9'} objective 57.2, constraint 500, choice set {1, 2, 3, 4}
============ Comparison ===================
Brute force: {1: '30.8', 2: '376', 3: '81.9', 4: '11.8'} objective 46.7, constraint 500, choice set {1, 2, 3, 4}
Analytical:  {1: '30.8', 2: '376', 3: '81.8', 4: '11.8'} objective 46.7, constraint 500, choice set {1, 2, 3, 4}
Forecasting observation 0 / 2 [2000 draws]
Forecasting observation 1 / 2 [2000 draws]
Execution time for 2000 draws with brute force algorithm: 96.8 seconds
Forecasting observation 0 / 2 [2000 draws]
Forecasting observation 1 / 2 [2000 draws]
Execution time for 2000 draws with analytical algorithm: 102 seconds
                 1            2            3            4
count  2000.000000  2000.000000  2000.000000  2000.000000
mean     23.463093   390.937666    72.478533    13.120709
std      21.129312    60.108549    50.665825     7.465697
min       0.000000    19.840805     0.000000     1.048504
25%      11.534350   361.038890    39.266556     8.890361
50%      18.304842   401.869975    60.505406    11.728940
75%      28.031979   431.889220    93.504950    15.493649
max     247.619824   497.404069   475.337145   106.050983
                 1            2            3            4
count  2000.000000  2000.000000  2000.000000  2000.000000
mean     23.461268   390.935089    72.482305    13.121338
std      21.112487    60.133128    50.688009     7.466033
min       0.000000    19.847112     0.000000     1.049500
25%      11.527212   361.043798    39.244188     8.890699
50%      18.304658   401.870233    60.494902    11.728213
75%      28.033698   431.893025    93.473112    15.494610
max     247.013712   497.402713   475.328315   106.052853
                 1            2             3            4
count  2000.000000  2000.000000  2.000000e+03  2000.000000
mean     25.705314   369.569321  9.501463e+01     9.710732
std      22.826667    70.791691  6.447994e+01     5.279167
min       0.000000     6.954474  5.896093e-15     0.177216
25%      13.027729   336.426502  5.265620e+01     6.561885
50%      20.271775   381.228643  8.100198e+01     8.676587
75%      30.512608   417.415730  1.180033e+02    11.640300
max     285.268057   499.822784  4.872200e+02    77.574175
                 1            2            3            4
count  2000.000000  2000.000000  2000.000000  2000.000000
mean     25.699210   369.598989    94.990795     9.711006
std      22.793118    70.732524    64.432790     5.279133
min       0.000000     6.960521     0.000000     0.177458
25%      13.028637   336.411070    52.657862     6.560871
50%      20.270946   381.224179    80.996537     8.677453
75%      30.503319   417.416712   118.013863    11.642826
max     285.271021   499.822543   487.209871    77.589522

import sys
import time

import numpy as np
import pandas as pd
from IPython.core.display_functions import display

import biogeme.biogeme_logging as blog
from biogeme.database import Database
from biogeme.results_processing import EstimationResults
from process_data import database
from translated_specification import the_translated

logger = blog.get_screen_logger(level=blog.INFO)
logger.info('Example: translated utility')

result_file = 'saved_results/translated.yaml'
try:
    results = EstimationResults.from_yaml_file(filename=result_file)
except FileNotFoundError as e:
    print(e)
    print(f'File {result_file} is missing.')
    sys.exit()

the_translated.estimation_results = results

# %
# We apply the model only on the first two rows of the database.
two_rows_of_database: Database = database.extract_rows([0, 1])

# %
budget_in_hours = 500

# %
# # Validation

# %
# As the implementation is still experimental, we compare the result obtained by the bruteforce algorithm and
# the analytical algorithm for a few draws.

# Note that minor discrepancies between the outcome of the two algorithms are likely to occur, due to numerical
# imprecision, inevitable in finite arithmetic.

# However, if there are major differences, it should be reported.

# %
number_of_draws = 10

# %
# We generate the draws
epsilons = [
    np.random.gumbel(
        loc=0, scale=1, size=(number_of_draws, the_translated.number_of_alternatives)
    )
    for _ in range(two_rows_of_database.num_rows())
]

# %
# We first compare the results obtained from the brute force and the analytical algorithms, for each draw.
the_translated.validate_forecast(
    database=two_rows_of_database, total_budget=budget_in_hours, epsilons=epsilons
)

# %
# # Forecasting
# We use a larger number of draws to obtain the forecast.

# %
number_of_draws = 2000

# %
# We generate the draws
epsilons = [
    np.random.gumbel(
        loc=0, scale=1, size=(number_of_draws, the_translated.number_of_alternatives)
    )
    for _ in range(two_rows_of_database.num_rows())
]

# %
# First, the brute force algorithm.
start_time = time.time()
optimal_consumptions_brute_force: list[pd.DataFrame] = the_translated.forecast(
    database=two_rows_of_database,
    total_budget=budget_in_hours,
    epsilons=epsilons,
    brute_force=True,
)
end_time = time.time()

# %
print(
    f'Execution time for {number_of_draws} draws with brute force algorithm: {end_time-start_time:.3g} seconds'
)

# %
# Then, the analytical algorithm.
start_time = time.time()
optimal_consumptions_analytical: list[pd.DataFrame] = the_translated.forecast(
    database=two_rows_of_database,
    total_budget=budget_in_hours,
    epsilons=epsilons,
    brute_force=False,
)
end_time = time.time()

# %
print(
    f'Execution time for {number_of_draws} draws with analytical algorithm: {end_time-start_time:.3g} seconds'
)

# %
# Results for the first observation, brute force method
display(optimal_consumptions_brute_force[0].describe())

# %
# Results for the first observation, analytical method
display(optimal_consumptions_analytical[0].describe())

# %
# Results for the second observation, brute force method
display(optimal_consumptions_brute_force[1].describe())

# %
# Results for the second observation, analytical method
display(optimal_consumptions_analytical[1].describe())