.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "auto_examples/sampling/plot_b01logit.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        :ref:`Go to the end <sphx_glr_download_auto_examples_sampling_plot_b01logit.py>`
        to download the full example code.

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_auto_examples_sampling_plot_b01logit.py:


Logit
=====

Estimation of a logit model using sampling of alternatives.

:author: Michel Bierlaire
:date: Wed Nov  1 17:39:47 2023

.. GENERATED FROM PYTHON SOURCE LINES 11-29

.. code-block:: Python


    import pandas as pd
    from biogeme.sampling_of_alternatives import (
        SamplingContext,
        ChoiceSetsGeneration,
        GenerateModel,
        generate_segment_size,
    )
    import biogeme.biogeme_logging as blog
    import biogeme.biogeme as bio
    from compare import compare
    from specification_sampling import V, combined_variables
    from alternatives import (
        alternatives,
        ID_COLUMN,
        partitions,
    )





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    Number of asian restaurants: 33




.. GENERATED FROM PYTHON SOURCE LINES 30-32

.. code-block:: Python

    logger = blog.get_screen_logger(level=blog.INFO)








.. GENERATED FROM PYTHON SOURCE LINES 33-35

The data file contains several columns associated with synthetic
choices. Here we arbitrarily select `logit_4`.

.. GENERATED FROM PYTHON SOURCE LINES 35-37

.. code-block:: Python

    CHOICE_COLUMN = 'logit_4'








.. GENERATED FROM PYTHON SOURCE LINES 38-44

.. code-block:: Python

    SAMPLE_SIZE = 10
    PARTITION = 'asian'
    MODEL_NAME = f'logit_{PARTITION}_{SAMPLE_SIZE}_alt'
    FILE_NAME = f'{MODEL_NAME}.dat'
    OBS_FILE = 'obs_choice.dat'








.. GENERATED FROM PYTHON SOURCE LINES 45-49

.. code-block:: Python

    the_partition = partitions.get(PARTITION)
    if the_partition is None:
        raise ValueError(f'Unknown partition: {PARTITION}')








.. GENERATED FROM PYTHON SOURCE LINES 50-52

.. code-block:: Python

    segment_sizes = generate_segment_size(SAMPLE_SIZE, the_partition.number_of_segments())








.. GENERATED FROM PYTHON SOURCE LINES 53-55

.. code-block:: Python

    observations = pd.read_csv(OBS_FILE)








.. GENERATED FROM PYTHON SOURCE LINES 56-68

.. code-block:: Python

    context = SamplingContext(
        the_partition=the_partition,
        sample_sizes=segment_sizes,
        individuals=observations,
        choice_column=CHOICE_COLUMN,
        alternatives=alternatives,
        id_column=ID_COLUMN,
        biogeme_file_name=FILE_NAME,
        utility_function=V,
        combined_variables=combined_variables,
    )








.. GENERATED FROM PYTHON SOURCE LINES 69-71

.. code-block:: Python

    logger.info(context.reporting())





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    Size of the choice set: 100
    Main partition: 2 segment(s) of size 33, 67
    Main sample: 10: 5/33, 5/67
 




.. GENERATED FROM PYTHON SOURCE LINES 72-74

.. code-block:: Python

    the_data_generation = ChoiceSetsGeneration(context=context)








.. GENERATED FROM PYTHON SOURCE LINES 75-77

.. code-block:: Python

    the_model_generation = GenerateModel(context=context)








.. GENERATED FROM PYTHON SOURCE LINES 78-80

.. code-block:: Python

    biogeme_database = the_data_generation.sample_and_merge(recycle=False)





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    Generating 10 alternatives for 10000 observations 

      0%|          | 0/10000 [00:00<?, ?it/s]
      1%|▏         | 133/10000 [00:00<00:07, 1325.02it/s]
      3%|▎         | 273/10000 [00:00<00:07, 1367.65it/s]
      4%|▍         | 415/10000 [00:00<00:06, 1391.39it/s]
      6%|▌         | 561/10000 [00:00<00:06, 1418.21it/s]
      7%|▋         | 705/10000 [00:00<00:06, 1424.02it/s]
      8%|▊         | 850/10000 [00:00<00:06, 1431.95it/s]
     10%|▉         | 995/10000 [00:00<00:06, 1434.86it/s]
     11%|█▏        | 1141/10000 [00:00<00:06, 1440.42it/s]
     13%|█▎        | 1287/10000 [00:00<00:06, 1444.24it/s]
     14%|█▍        | 1434/10000 [00:01<00:05, 1449.63it/s]
     16%|█▌        | 1579/10000 [00:01<00:05, 1447.92it/s]
     17%|█▋        | 1725/10000 [00:01<00:05, 1449.75it/s]
     19%|█▊        | 1872/10000 [00:01<00:05, 1454.61it/s]
     20%|██        | 2019/10000 [00:01<00:05, 1457.68it/s]
     22%|██▏       | 2165/10000 [00:01<00:05, 1438.65it/s]
     23%|██▎       | 2309/10000 [00:01<00:05, 1406.30it/s]
     24%|██▍       | 2450/10000 [00:01<00:05, 1399.83it/s]
     26%|██▌       | 2591/10000 [00:01<00:05, 1390.57it/s]
     27%|██▋       | 2731/10000 [00:01<00:05, 1389.14it/s]
     29%|██▊       | 2872/10000 [00:02<00:05, 1393.76it/s]
     30%|███       | 3012/10000 [00:02<00:05, 1383.84it/s]
     32%|███▏      | 3151/10000 [00:02<00:04, 1385.18it/s]
     33%|███▎      | 3290/10000 [00:02<00:04, 1375.19it/s]
     34%|███▍      | 3428/10000 [00:02<00:04, 1372.56it/s]
     36%|███▌      | 3566/10000 [00:02<00:04, 1372.29it/s]
     37%|███▋      | 3705/10000 [00:02<00:04, 1375.80it/s]
     38%|███▊      | 3844/10000 [00:02<00:04, 1379.30it/s]
     40%|███▉      | 3983/10000 [00:02<00:04, 1380.56it/s]
     41%|████      | 4122/10000 [00:02<00:04, 1377.72it/s]
     43%|████▎     | 4260/10000 [00:03<00:04, 1375.19it/s]
     44%|████▍     | 4399/10000 [00:03<00:04, 1377.73it/s]
     45%|████▌     | 4537/10000 [00:03<00:03, 1374.31it/s]
     47%|████▋     | 4676/10000 [00:03<00:03, 1377.29it/s]
     48%|████▊     | 4814/10000 [00:03<00:03, 1377.04it/s]
     50%|████▉     | 4952/10000 [00:03<00:03, 1368.55it/s]
     51%|█████     | 5090/10000 [00:03<00:03, 1370.71it/s]
     52%|█████▏    | 5228/10000 [00:03<00:03, 1365.39it/s]
     54%|█████▎    | 5365/10000 [00:03<00:03, 1362.10it/s]
     55%|█████▌    | 5502/10000 [00:03<00:03, 1359.52it/s]
     56%|█████▋    | 5638/10000 [00:04<00:03, 1356.92it/s]
     58%|█████▊    | 5775/10000 [00:04<00:03, 1358.17it/s]
     59%|█████▉    | 5911/10000 [00:04<00:03, 1356.86it/s]
     60%|██████    | 6047/10000 [00:04<00:02, 1357.19it/s]
     62%|██████▏   | 6183/10000 [00:04<00:02, 1354.27it/s]
     63%|██████▎   | 6320/10000 [00:04<00:02, 1356.64it/s]
     65%|██████▍   | 6458/10000 [00:04<00:02, 1360.86it/s]
     66%|██████▌   | 6595/10000 [00:04<00:02, 1355.89it/s]
     67%|██████▋   | 6731/10000 [00:04<00:02, 1350.63it/s]
     69%|██████▊   | 6870/10000 [00:04<00:02, 1360.48it/s]
     70%|███████   | 7007/10000 [00:05<00:02, 1355.43it/s]
     71%|███████▏  | 7143/10000 [00:05<00:02, 1350.47it/s]
     73%|███████▎  | 7280/10000 [00:05<00:02, 1356.24it/s]
     74%|███████▍  | 7416/10000 [00:05<00:01, 1344.12it/s]
     76%|███████▌  | 7551/10000 [00:05<00:01, 1341.40it/s]
     77%|███████▋  | 7687/10000 [00:05<00:01, 1345.57it/s]
     78%|███████▊  | 7822/10000 [00:05<00:01, 1343.59it/s]
     80%|███████▉  | 7960/10000 [00:05<00:01, 1354.25it/s]
     81%|████████  | 8096/10000 [00:05<00:01, 1351.67it/s]
     82%|████████▏ | 8232/10000 [00:05<00:01, 1350.14it/s]
     84%|████████▎ | 8369/10000 [00:06<00:01, 1354.40it/s]
     85%|████████▌ | 8505/10000 [00:06<00:01, 1353.60it/s]
     86%|████████▋ | 8641/10000 [00:06<00:01, 1352.96it/s]
     88%|████████▊ | 8777/10000 [00:06<00:00, 1350.07it/s]
     89%|████████▉ | 8913/10000 [00:06<00:00, 1350.64it/s]
     91%|█████████ | 9053/10000 [00:06<00:00, 1362.44it/s]
     92%|█████████▏| 9190/10000 [00:06<00:00, 1354.36it/s]
     93%|█████████▎| 9326/10000 [00:06<00:00, 1347.15it/s]
     95%|█████████▍| 9463/10000 [00:06<00:00, 1353.14it/s]
     96%|█████████▌| 9601/10000 [00:06<00:00, 1359.32it/s]
     97%|█████████▋| 9738/10000 [00:07<00:00, 1359.82it/s]
     99%|█████████▉| 9878/10000 [00:07<00:00, 1370.32it/s]
    100%|██████████| 10000/10000 [00:07<00:00, 1334.90it/s]
    Define new variables 

    Defining new variables...:   0%|          | 0/10 [00:00<?, ?it/s]
    Defining new variables...:  20%|██        | 2/10 [00:00<00:00, 14.35it/s]
    Defining new variables...:  40%|████      | 4/10 [00:00<00:00, 14.00it/s]
    Defining new variables...:  60%|██████    | 6/10 [00:00<00:00, 13.78it/s]
    Defining new variables...:  80%|████████  | 8/10 [00:00<00:00, 13.53it/s]
    Defining new variables...: 100%|██████████| 10/10 [00:00<00:00, 13.36it/s]
    Defining new variables...: 100%|██████████| 10/10 [00:00<00:00, 13.55it/s]
    File logit_asian_10_alt.dat has been created. 




.. GENERATED FROM PYTHON SOURCE LINES 81-83

.. code-block:: Python

    logprob = the_model_generation.get_logit()








.. GENERATED FROM PYTHON SOURCE LINES 84-87

.. code-block:: Python

    the_biogeme = bio.BIOGEME(biogeme_database, logprob)
    the_biogeme.modelName = MODEL_NAME





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    Default values of the Biogeme parameters are used. 
    File biogeme.toml has been created 




.. GENERATED FROM PYTHON SOURCE LINES 88-89

Calculate the null log likelihood for reporting.

.. GENERATED FROM PYTHON SOURCE LINES 89-91

.. code-block:: Python

    the_biogeme.calculate_null_loglikelihood({i: 1 for i in range(SAMPLE_SIZE)})





.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    np.float64(-23025.850929942502)



.. GENERATED FROM PYTHON SOURCE LINES 92-93

Estimate the parameters

.. GENERATED FROM PYTHON SOURCE LINES 93-95

.. code-block:: Python

    results = the_biogeme.estimate(recycle=False)





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    As the model is not too complex, we activate the calculation of second derivatives. If you want to change it, change the name of the algorithm in the TOML file from "automatic" to "simple_bounds" 
    *** Initial values of the parameters are obtained from the file __logit_asian_10_alt.iter 
    Cannot read file __logit_asian_10_alt.iter. Statement is ignored. 
    As the model is not too complex, we activate the calculation of second derivatives. If you want to change it, change the name of the algorithm in the TOML file from "automatic" to "simple_bounds" 
    Optimization algorithm: hybrid Newton/BFGS with simple bounds [simple_bounds] 
    ** Optimization: Newton with trust region for simple bounds 
    Iter.    beta_chinese  beta_ethiopian     beta_french     beta_indian   beta_japanese     beta_korean   beta_lebanese   beta_log_dist    beta_mexican      beta_price     beta_rating     Function    Relgrad   Radius      Rho      
        0            0.04           -0.08            0.13            0.36               1            0.44          -0.046           -0.67            0.84            -0.4            0.46      1.9e+04      0.065       10        1   ++ 
        1            0.58             0.4             0.6            0.87             1.1            0.69            0.64           -0.58             1.1            -0.4             0.7      1.8e+04     0.0099    1e+02      1.1   ++ 
        2            0.61            0.44            0.64            0.92             1.2            0.71            0.65           -0.59             1.2           -0.41            0.75      1.8e+04    0.00032    1e+03        1   ++ 
        3            0.61            0.44            0.64            0.92             1.2            0.71            0.65           -0.59             1.2           -0.41            0.75      1.8e+04    3.6e-07    1e+03        1   ++ 
    Results saved in file logit_asian_10_alt.html 
    Results saved in file logit_asian_10_alt.pickle 




.. GENERATED FROM PYTHON SOURCE LINES 96-98

.. code-block:: Python

    print(results.short_summary())





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    Results for model logit_asian_10_alt
    Nbr of parameters:              11
    Sample size:                    10000
    Excluded data:                  0
    Null log likelihood:            -23025.85
    Final log likelihood:           -18451.88
    Likelihood ratio test (null):           9147.951
    Rho square (null):                      0.199
    Rho bar square (null):                  0.198
    Akaike Information Criterion:   36925.75
    Bayesian Information Criterion: 37005.06





.. GENERATED FROM PYTHON SOURCE LINES 99-102

.. code-block:: Python

    estimated_parameters = results.get_estimated_parameters()
    estimated_parameters






.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>Value</th>
          <th>Rob. Std err</th>
          <th>Rob. t-test</th>
          <th>Rob. p-value</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>beta_chinese</th>
          <td>0.607892</td>
          <td>0.050236</td>
          <td>12.100600</td>
          <td>0.0</td>
        </tr>
        <tr>
          <th>beta_ethiopian</th>
          <td>0.436107</td>
          <td>0.050743</td>
          <td>8.594396</td>
          <td>0.0</td>
        </tr>
        <tr>
          <th>beta_french</th>
          <td>0.637537</td>
          <td>0.061926</td>
          <td>10.295114</td>
          <td>0.0</td>
        </tr>
        <tr>
          <th>beta_indian</th>
          <td>0.917637</td>
          <td>0.043002</td>
          <td>21.339266</td>
          <td>0.0</td>
        </tr>
        <tr>
          <th>beta_japanese</th>
          <td>1.152123</td>
          <td>0.046626</td>
          <td>24.709975</td>
          <td>0.0</td>
        </tr>
        <tr>
          <th>beta_korean</th>
          <td>0.713678</td>
          <td>0.042509</td>
          <td>16.788924</td>
          <td>0.0</td>
        </tr>
        <tr>
          <th>beta_lebanese</th>
          <td>0.655038</td>
          <td>0.062660</td>
          <td>10.453805</td>
          <td>0.0</td>
        </tr>
        <tr>
          <th>beta_log_dist</th>
          <td>-0.586987</td>
          <td>0.014885</td>
          <td>-39.435942</td>
          <td>0.0</td>
        </tr>
        <tr>
          <th>beta_mexican</th>
          <td>1.181418</td>
          <td>0.036514</td>
          <td>32.355573</td>
          <td>0.0</td>
        </tr>
        <tr>
          <th>beta_price</th>
          <td>-0.408117</td>
          <td>0.012796</td>
          <td>-31.894969</td>
          <td>0.0</td>
        </tr>
        <tr>
          <th>beta_rating</th>
          <td>0.751644</td>
          <td>0.015430</td>
          <td>48.714276</td>
          <td>0.0</td>
        </tr>
      </tbody>
    </table>
    </div>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 103-105

.. code-block:: Python

    df, msg = compare(estimated_parameters)








.. GENERATED FROM PYTHON SOURCE LINES 106-108

.. code-block:: Python

    print(df)





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

                  Name  True Value  Estimated Value    T-Test
    0      beta_rating        0.75         0.751644 -0.106559
    1       beta_price       -0.40        -0.408117  0.634324
    2     beta_chinese        0.75         0.607892  2.828786
    3    beta_japanese        1.25         1.152123  2.099197
    4      beta_korean        0.75         0.713678  0.854456
    5      beta_indian        1.00         0.917637  1.915316
    6      beta_french        0.75         0.637537  1.816087
    7     beta_mexican        1.25         1.181418  1.878265
    8    beta_lebanese        0.75         0.655038  1.515505
    9   beta_ethiopian        0.50         0.436107  1.259136
    10   beta_log_dist       -0.60        -0.586987 -0.874276




.. GENERATED FROM PYTHON SOURCE LINES 109-110

.. code-block:: Python

    print(msg)




.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    Parameters not estimated: ['mu_asian', 'mu_downtown']





.. rst-class:: sphx-glr-timing

   **Total running time of the script:** (0 minutes 11.010 seconds)


.. _sphx_glr_download_auto_examples_sampling_plot_b01logit.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: plot_b01logit.ipynb <plot_b01logit.ipynb>`

    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: plot_b01logit.py <plot_b01logit.py>`

    .. container:: sphx-glr-download sphx-glr-download-zip

      :download:`Download zipped: plot_b01logit.zip <plot_b01logit.zip>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_