Comparison of execution timesΒΆ

Michel Bierlaire Tue Jul 2 14:48:52 2024

import pandas as pd
from IPython.core.display_functions import display

Version 3.2.14 and 3.3.0 of Biogeme have been used to calculate a logit, a crossed-nested logit and a logit mixtures model on the same computer (MacBook Pro Apple M2 Max 64 GB).

Raw data

running_times = [
    # Logit 3.2.14
    ('logit', '3.2.14', 'Function only', None, 0.132),
    ('logit', '3.2.14', 'Function and gradient', None, 0.135),
    ('logit', '3.2.14', 'Function, gradient and hessian', None, 0.141),
    # Logit new
    ('logit', '3.3.0', 'Function only', 'with JIT', 0.0022),
    ('logit', '3.3.0', 'Function only', 'without JIT', 0.0161),
    ('logit', '3.3.0', 'Function and gradient', 'with JIT', 0.000819),
    ('logit', '3.3.0', 'Function and gradient', 'without JIT', 0.0445),
    ('logit', '3.3.0', 'Function, gradient and hessian', 'with JIT', 0.00258),
    ('logit', '3.3.0', 'Function, gradient and hessian', 'without JIT', 0.136),
    # CNL 3.2.14
    ('CNL', '3.2.14', 'Function only', None, 0.408),
    ('CNL', '3.2.14', 'Function and gradient', None, 0.362),
    ('CNL', '3.2.14', 'Function, gradient and hessian', None, 0.324),
    # CNL new
    ('CNL', '3.3.0', 'Function only', 'with JIT', 0.0034),
    ('CNL', '3.3.0', 'Function only', 'without JIT', 0.147),
    ('CNL', '3.3.0', 'Function and gradient', 'with JIT', 0.00848),
    ('CNL', '3.3.0', 'Function and gradient', 'without JIT', 0.525),
    ('CNL', '3.3.0', 'Function, gradient and hessian', 'with JIT', 0.121),
    ('CNL', '3.3.0', 'Function, gradient and hessian', 'without JIT', 1.79),
    # Mixtures 100 draws 3.2.14
    ('mixtures_100', '3.2.14', 'Function only', None, 1.38),
    ('mixtures_100', '3.2.14', 'Function and gradient', None, 1.65),
    ('mixtures_100', '3.2.14', 'Function, gradient and hessian', None, 2.12),
    # Mixtures 100 draws new
    ('mixtures_100', '3.3.0', 'Function only', 'with JIT', 0.0476),
    ('mixtures_100', '3.3.0', 'Function only', 'without JIT', 0.0503),
    ('mixtures_100', '3.3.0', 'Function and gradient', 'with JIT', 0.0185),
    ('mixtures_100', '3.3.0', 'Function and gradient', 'without JIT', 0.116),
    ('mixtures_100', '3.3.0', 'Function, gradient and hessian', 'with JIT', 0.16),
    ('mixtures_100', '3.3.0', 'Function, gradient and hessian', 'without JIT', 0.586),
    # Mixtures 1000 draws new
    ('mixtures_1000', '3.3.0', 'Function only', 'with JIT', 0.0836),
    ('mixtures_1000', '3.3.0', 'Function only', 'without JIT', 0.32),
    ('mixtures_1000', '3.3.0', 'Function and gradient', 'with JIT', 0.199),
    ('mixtures_1000', '3.3.0', 'Function and gradient', 'without JIT', 0.61),
    ('mixtures_1000', '3.3.0', 'Function, gradient and hessian', 'with JIT', 1.63),
    ('mixtures_1000', '3.3.0', 'Function, gradient and hessian', 'without JIT', 4.84),
]
df = pd.DataFrame(
    running_times, columns=['Model', 'Version', 'Computation', 'JIT', 'Time_sec']
)
display(df)
            Model Version  ...          JIT  Time_sec
0           logit  3.2.14  ...         None  0.132000
1           logit  3.2.14  ...         None  0.135000
2           logit  3.2.14  ...         None  0.141000
3           logit   3.3.0  ...     with JIT  0.002200
4           logit   3.3.0  ...  without JIT  0.016100
5           logit   3.3.0  ...     with JIT  0.000819
6           logit   3.3.0  ...  without JIT  0.044500
7           logit   3.3.0  ...     with JIT  0.002580
8           logit   3.3.0  ...  without JIT  0.136000
9             CNL  3.2.14  ...         None  0.408000
10            CNL  3.2.14  ...         None  0.362000
11            CNL  3.2.14  ...         None  0.324000
12            CNL   3.3.0  ...     with JIT  0.003400
13            CNL   3.3.0  ...  without JIT  0.147000
14            CNL   3.3.0  ...     with JIT  0.008480
15            CNL   3.3.0  ...  without JIT  0.525000
16            CNL   3.3.0  ...     with JIT  0.121000
17            CNL   3.3.0  ...  without JIT  1.790000
18   mixtures_100  3.2.14  ...         None  1.380000
19   mixtures_100  3.2.14  ...         None  1.650000
20   mixtures_100  3.2.14  ...         None  2.120000
21   mixtures_100   3.3.0  ...     with JIT  0.047600
22   mixtures_100   3.3.0  ...  without JIT  0.050300
23   mixtures_100   3.3.0  ...     with JIT  0.018500
24   mixtures_100   3.3.0  ...  without JIT  0.116000
25   mixtures_100   3.3.0  ...     with JIT  0.160000
26   mixtures_100   3.3.0  ...  without JIT  0.586000
27  mixtures_1000   3.3.0  ...     with JIT  0.083600
28  mixtures_1000   3.3.0  ...  without JIT  0.320000
29  mixtures_1000   3.3.0  ...     with JIT  0.199000
30  mixtures_1000   3.3.0  ...  without JIT  0.610000
31  mixtures_1000   3.3.0  ...     with JIT  1.630000
32  mixtures_1000   3.3.0  ...  without JIT  4.840000

[33 rows x 5 columns]

Function evaluation

def generate_speedup_sentence(model_name, computation_type):
    old_time = df.query(
        'Model == @model_name and Version == "3.2.14" and Computation == @computation_type'
    )['Time_sec'].values[0]

    new_time = df.query(
        'Model == @model_name and Version == "3.3.0" and Computation == @computation_type and JIT == "with JIT"'
    )['Time_sec'].values[0]

    speedup = old_time / new_time
    return (
        f'For a {model_name} model, version 3.3.0 of Biogeme with JIT is '
        f'{speedup:.1f} times faster than version 3.2.14 for {computation_type.lower()}.'
    )

Generate all speedup sentences

unique_models = df['Model'].unique()
unique_computations = df['Computation'].unique()

for model in unique_models:
    for computation in unique_computations:
        try:
            sentence = generate_speedup_sentence(model, computation)
            print(sentence)
        except IndexError:
            # Skip combinations where data is missing
            continue
For a logit model, version 3.3.0 of Biogeme with JIT is 60.0 times faster than version 3.2.14 for function only.
For a logit model, version 3.3.0 of Biogeme with JIT is 164.8 times faster than version 3.2.14 for function and gradient.
For a logit model, version 3.3.0 of Biogeme with JIT is 54.7 times faster than version 3.2.14 for function, gradient and hessian.
For a CNL model, version 3.3.0 of Biogeme with JIT is 120.0 times faster than version 3.2.14 for function only.
For a CNL model, version 3.3.0 of Biogeme with JIT is 42.7 times faster than version 3.2.14 for function and gradient.
For a CNL model, version 3.3.0 of Biogeme with JIT is 2.7 times faster than version 3.2.14 for function, gradient and hessian.
For a mixtures_100 model, version 3.3.0 of Biogeme with JIT is 29.0 times faster than version 3.2.14 for function only.
For a mixtures_100 model, version 3.3.0 of Biogeme with JIT is 89.2 times faster than version 3.2.14 for function and gradient.
For a mixtures_100 model, version 3.3.0 of Biogeme with JIT is 13.2 times faster than version 3.2.14 for function, gradient and hessian.

Total running time of the script: (0 minutes 0.031 seconds)

Gallery generated by Sphinx-Gallery