Generalized translated MDCEV forecasting

Michel Bierlaire, EPFL Fri Jul 25 2025, 17:05:32

Forecasting with a MDCEV model and the “generalized translated utility” specification.

Example: generalized translated utility
Forecasting observation 0 / 2 [10 draws]
============ Comparison ===================
Brute force: {1: '9.66', 2: '455', 3: '20.8', 4: '14.7'} objective 248, constraint 500, choice set {1, 2, 3, 4}
Analytical:  {1: '9.66', 2: '455', 3: '20.8', 4: '14.7'} objective 248, constraint 500, choice set {1, 2, 3, 4}
============ Comparison ===================
Brute force: {1: '20.1', 2: '440', 3: '29.6', 4: '10.6'} objective 199, constraint 500, choice set {1, 2, 3, 4}
Analytical:  {1: '20.1', 2: '440', 3: '29.4', 4: '10.6'} objective 199, constraint 500, choice set {1, 2, 3, 4}
============ Comparison ===================
Brute force: {1: '28.7', 2: '419', 3: '39.6', 4: '12.9'} objective 209, constraint 500, choice set {1, 2, 3, 4}
Analytical:  {1: '28.7', 2: '419', 3: '39.6', 4: '12.9'} objective 209, constraint 500, choice set {1, 2, 3, 4}
============ Comparison ===================
Brute force: {1: '107', 2: '359', 3: '24.2', 4: '9.82'} objective 237, constraint 500, choice set {1, 2, 3, 4}
Analytical:  {1: '107', 2: '359', 3: '24.2', 4: '9.82'} objective 237, constraint 500, choice set {1, 2, 3, 4}
============ Comparison ===================
Brute force: {1: '12.9', 2: '445', 3: '24.9', 4: '17.6'} objective 195, constraint 500, choice set {1, 2, 3, 4}
Analytical:  {1: '12.9', 2: '445', 3: '24.9', 4: '17.6'} objective 195, constraint 500, choice set {1, 2, 3, 4}
============ Comparison ===================
Brute force: {1: '18.6', 2: '425', 3: '19.8', 4: '36.3'} objective 217, constraint 500, choice set {1, 2, 3, 4}
Analytical:  {1: '18.6', 2: '425', 3: '19.8', 4: '36.3'} objective 217, constraint 500, choice set {1, 2, 3, 4}
============ Comparison ===================
Brute force: {1: '14.5', 2: '429', 3: '46.8', 4: '9.31'} objective 200, constraint 500, choice set {1, 2, 3, 4}
Analytical:  {1: '14.5', 2: '429', 3: '46.7', 4: '9.3'} objective 200, constraint 500, choice set {1, 2, 3, 4}
============ Comparison ===================
Brute force: {1: '17.4', 2: '412', 3: '56.9', 4: '14'} objective 198, constraint 500, choice set {1, 2, 3, 4}
Analytical:  {1: '17.4', 2: '412', 3: '56.8', 4: '14'} objective 198, constraint 500, choice set {1, 2, 3, 4}
============ Comparison ===================
Brute force: {1: '27', 2: '445', 3: '20.7', 4: '7.25'} objective 227, constraint 500, choice set {1, 2, 3, 4}
Analytical:  {1: '27', 2: '445', 3: '20.7', 4: '7.25'} objective 227, constraint 500, choice set {1, 2, 3, 4}
============ Comparison ===================
Brute force: {1: '10.7', 2: '432', 3: '45.9', 4: '11.3'} objective 220, constraint 500, choice set {1, 2, 3, 4}
Analytical:  {1: '10.7', 2: '432', 3: '46.5', 4: '11.2'} objective 220, constraint 500, choice set {1, 2, 3, 4}
Forecasting observation 1 / 2 [10 draws]
============ Comparison ===================
Brute force: {1: '44.1', 2: '407', 3: '32.4', 4: '16'} objective 212, constraint 500, choice set {1, 2, 3, 4}
Analytical:  {1: '44', 2: '408', 3: '32.5', 4: '16'} objective 212, constraint 500, choice set {1, 2, 3, 4}
============ Comparison ===================
Brute force: {1: '5.52', 2: '481', 3: '11.6', 4: '2.3'} objective 337, constraint 500, choice set {1, 2, 3, 4}
Analytical:  {1: '5.54', 2: '481', 3: '11.6', 4: '2.3'} objective 337, constraint 500, choice set {1, 2, 3, 4}
============ Comparison ===================
Brute force: {1: '11.1', 2: '439', 3: '41.4', 4: '8.39'} objective 210, constraint 500, choice set {1, 2, 3, 4}
Analytical:  {1: '11.1', 2: '439', 3: '41.5', 4: '8.39'} objective 210, constraint 500, choice set {1, 2, 3, 4}
============ Comparison ===================
Brute force: {1: '35.1', 2: '440', 3: '21.4', 4: '3.83'} objective 257, constraint 500, choice set {1, 2, 3, 4}
Analytical:  {1: '35.2', 2: '440', 3: '21.3', 4: '3.85'} objective 257, constraint 500, choice set {1, 2, 3, 4}
============ Comparison ===================
Brute force: {1: '30.1', 2: '436', 3: '18', 4: '15.5'} objective 357, constraint 500, choice set {1, 2, 3, 4}
Analytical:  {1: '30.1', 2: '436', 3: '18', 4: '15.5'} objective 357, constraint 500, choice set {1, 2, 3, 4}
============ Comparison ===================
Brute force: {1: '5.43', 2: '461', 3: '26.3', 4: '6.89'} objective 239, constraint 500, choice set {1, 2, 3, 4}
Analytical:  {1: '5.42', 2: '461', 3: '26.3', 4: '6.89'} objective 239, constraint 500, choice set {1, 2, 3, 4}
============ Comparison ===================
Brute force: {1: '26.7', 2: '422', 3: '44.5', 4: '7.27'} objective 225, constraint 500, choice set {1, 2, 3, 4}
Analytical:  {1: '26.7', 2: '421', 3: '44.5', 4: '7.33'} objective 225, constraint 500, choice set {1, 2, 3, 4}
============ Comparison ===================
Brute force: {1: '82.6', 2: '373', 3: '34.8', 4: '10.1'} objective 193, constraint 500, choice set {1, 2, 3, 4}
Analytical:  {1: '82.7', 2: '373', 3: '34.7', 4: '10.1'} objective 193, constraint 500, choice set {1, 2, 3, 4}
============ Comparison ===================
Brute force: {1: '17', 2: '418', 3: '57.5', 4: '7.27'} objective 253, constraint 500, choice set {1, 2, 3, 4}
Analytical:  {1: '17', 2: '418', 3: '57.8', 4: '7.23'} objective 253, constraint 500, choice set {1, 2, 3, 4}
============ Comparison ===================
Brute force: {1: '6.02', 2: '471', 3: '19.5', 4: '3.54'} objective 287, constraint 500, choice set {1, 2, 3, 4}
Analytical:  {1: '6.01', 2: '471', 3: '19.6', 4: '3.54'} objective 287, constraint 500, choice set {1, 2, 3, 4}
Forecasting observation 0 / 2 [2000 draws]
Forecasting observation 1 / 2 [2000 draws]
Execution time for 2000 draws with brute force algorithm: 82.1 seconds
Forecasting observation 0 / 2 [2000 draws]
Forecasting observation 1 / 2 [2000 draws]
Execution time for 2000 draws with analytical algorithm: 1.44 seconds
                 1            2            3            4
count  2000.000000  2000.000000  2000.000000  2000.000000
mean     24.032551   428.404582    34.346774    13.216093
std      21.811145    32.910363    15.720443     6.684630
min       0.000000    25.892336     0.000000     0.788066
25%      11.948309   412.744821    24.106598     9.007174
50%      19.195159   431.204879    32.971702    11.978946
75%      29.731847   449.062779    42.359255    15.910329
max     454.098273   498.200864   118.398159    70.595082
                 1            2            3            4
count  2000.000000  2000.000000  2000.000000  2000.000000
mean     24.030018   428.413673    34.335463    13.220846
std      21.811841    32.909148    15.715213     6.705124
min       0.000000    25.878668     0.000000     0.788133
25%      11.959329   412.754323    24.073680     9.007265
50%      19.188856   431.241445    32.897403    11.983777
75%      29.722530   449.065144    42.405045    15.922564
max     454.039249   498.200881   118.507541    70.603279
                 1            2            3             4
count  2000.000000  2000.000000  2000.000000  2.000000e+03
mean     27.013683   422.162289    40.678776  1.014525e+01
std      23.845361    33.914691    16.542748  5.202817e+00
min       0.000000   135.398497     0.000000  6.307005e-14
25%      13.693753   405.681751    30.189296  6.804690e+00
50%      21.805729   426.438840    38.900127  9.166401e+00
75%      32.531595   444.113758    49.304937  1.231941e+01
max     320.406226   500.000000   153.835152  6.041570e+01
                 1            2            3            4
count  2000.000000  2000.000000  2000.000000  2000.000000
mean     26.968284   422.214718    40.671969    10.145030
std      23.550148    33.700964    16.550481     5.202286
min       0.000000   135.111423     0.000000     0.000000
25%      13.688121   405.601475    30.137566     6.812101
50%      21.798767   426.466832    38.870147     9.176289
75%      32.506006   444.138825    49.313833    12.328314
max     320.476069   500.000000   153.746128    60.422742

import sys
import time

import numpy as np
import pandas as pd
from IPython.core.display_functions import display

import biogeme.biogeme_logging as blog
from biogeme.database import Database
from biogeme.results_processing import EstimationResults
from generalized_specification import the_generalized
from process_data import database

logger = blog.get_screen_logger(level=blog.INFO)
logger.info('Example: generalized translated utility')

result_file = 'saved_results/generalized.yaml'
try:
    results = EstimationResults.from_yaml_file(filename=result_file)
except FileNotFoundError as e:
    print(e)
    print(f'File {result_file} is missing.')
    sys.exit()

the_generalized.estimation_results = results

# %
# We apply the model only on the first two rows of the database.
two_rows_of_database: Database = database.extract_rows([0, 1])

# %
budget_in_hours = 500

# %
# # Validation

# %
# As the implementation is still experimental, we compare the result obtained by the bruteforce algorithm and
# the analytical algorithm for a few draws.

# Note that minor discrepancies between the outcome of the two algorithms are likely to occur, due to numerical
# imprecision, inevitable in finite arithmetic.

# However, if there are major differences, it should be reported.

# %
number_of_draws = 10

# %
# We generate the draws
epsilons = [
    np.random.gumbel(
        loc=0, scale=1, size=(number_of_draws, the_generalized.number_of_alternatives)
    )
    for _ in range(two_rows_of_database.num_rows())
]

# %
# We first compare the results obtained from the brute force and the analytical algorithms, for each draw.
the_generalized.validate_forecast(
    database=two_rows_of_database, total_budget=budget_in_hours, epsilons=epsilons
)

# %
# # Forecasting
# We use a larger number of draws to obtain the forecast.

# %
number_of_draws = 2_000

# %
# We generate the draws
epsilons = [
    np.random.gumbel(
        loc=0, scale=1, size=(number_of_draws, the_generalized.number_of_alternatives)
    )
    for _ in range(two_rows_of_database.num_rows())
]

# %
# First, the brute force algorithm.
start_time = time.time()
optimal_consumptions_brute_force: list[pd.DataFrame] = the_generalized.forecast(
    database=two_rows_of_database,
    total_budget=budget_in_hours,
    epsilons=epsilons,
    brute_force=True,
)
end_time = time.time()

# %
print(
    f'Execution time for {number_of_draws} draws with brute force algorithm: {end_time-start_time:.3g} seconds'
)

# %
# Then, the analytical algorithm.
start_time = time.time()
optimal_consumptions_analytical: list[pd.DataFrame] = the_generalized.forecast(
    database=two_rows_of_database,
    total_budget=budget_in_hours,
    epsilons=epsilons,
    brute_force=False,
)
end_time = time.time()

# %
print(
    f'Execution time for {number_of_draws} draws with analytical algorithm: {end_time-start_time:.3g} seconds'
)

# %
# Results for the first observation, brute force method
display(optimal_consumptions_brute_force[0].describe())

# %
# Results for the first observation, analytical method
display(optimal_consumptions_analytical[0].describe())

# %
# Results for the second observation, brute force method
display(optimal_consumptions_brute_force[1].describe())

# %
# Results for the second observation, analytical method
display(optimal_consumptions_analytical[1].describe())