.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "auto_examples/latent/plot_b00factor_analysis.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        :ref:`Go to the end <sphx_glr_download_auto_examples_latent_plot_b00factor_analysis.py>`
        to download the full example code.

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_auto_examples_latent_plot_b00factor_analysis.py:


Factor analysis
===============

Preliminary analysis of the indicators using factor analysis

:author: Michel Bierlaire, EPFL
:date: Thu Apr 13 16:39:54 2023

.. GENERATED FROM PYTHON SOURCE LINES 11-18

.. code-block:: Python


    import sys
    import pandas as pd
    import numpy as np
    from IPython.core.display_functions import display
    from biogeme.data.optima import data_file_path








.. GENERATED FROM PYTHON SOURCE LINES 19-24

The following package can be installed using

`pip install factor_analyzer`

See https://github.com/EducationalTestingService/factor_analyzer

.. GENERATED FROM PYTHON SOURCE LINES 24-30

.. code-block:: Python

    try:
        from factor_analyzer import FactorAnalyzer
    except ModuleNotFoundError:
        print('Use "pip install factor_analyzer" to install a requested package')
        sys.exit()








.. GENERATED FROM PYTHON SOURCE LINES 31-32

We first extract the columns containing the indicators

.. GENERATED FROM PYTHON SOURCE LINES 32-91

.. code-block:: Python

    variables = [
        'Envir01',
        'Envir02',
        'Envir03',
        'Envir04',
        'Envir05',
        'Envir06',
        'Mobil01',
        'Mobil02',
        'Mobil03',
        'Mobil04',
        'Mobil05',
        'Mobil06',
        'Mobil07',
        'Mobil08',
        'Mobil09',
        'Mobil10',
        'Mobil11',
        'Mobil12',
        'Mobil13',
        'Mobil14',
        'Mobil15',
        'Mobil16',
        'Mobil17',
        'Mobil18',
        'Mobil19',
        'Mobil20',
        'Mobil21',
        'Mobil22',
        'Mobil23',
        'Mobil24',
        'Mobil25',
        'Mobil26',
        'Mobil27',
        'ResidCh01',
        'ResidCh02',
        'ResidCh03',
        'ResidCh04',
        'ResidCh05',
        'ResidCh06',
        'ResidCh07',
        'LifSty01',
        'LifSty02',
        'LifSty03',
        'LifSty04',
        'LifSty05',
        'LifSty06',
        'LifSty07',
        'LifSty08',
        'LifSty09',
        'LifSty10',
        'LifSty11',
        'LifSty12',
        'LifSty13',
        'LifSty14',
    ]

    indicators = pd.read_csv(data_file_path, sep='\t', usecols=variables)








.. GENERATED FROM PYTHON SOURCE LINES 92-93

Negative values are missing values.

.. GENERATED FROM PYTHON SOURCE LINES 93-96

.. code-block:: Python

    indicators[indicators <= 0] = np.nan
    indicators = indicators.dropna(axis=0, how='any')








.. GENERATED FROM PYTHON SOURCE LINES 97-98

We perform the factor analysis

.. GENERATED FROM PYTHON SOURCE LINES 98-101

.. code-block:: Python

    fa = FactorAnalyzer(rotation='varimax')
    fa.fit(indicators)






.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <style>#sk-container-id-1 {
      /* Definition of color scheme common for light and dark mode */
      --sklearn-color-text: black;
      --sklearn-color-line: gray;
      /* Definition of color scheme for unfitted estimators */
      --sklearn-color-unfitted-level-0: #fff5e6;
      --sklearn-color-unfitted-level-1: #f6e4d2;
      --sklearn-color-unfitted-level-2: #ffe0b3;
      --sklearn-color-unfitted-level-3: chocolate;
      /* Definition of color scheme for fitted estimators */
      --sklearn-color-fitted-level-0: #f0f8ff;
      --sklearn-color-fitted-level-1: #d4ebff;
      --sklearn-color-fitted-level-2: #b3dbfd;
      --sklearn-color-fitted-level-3: cornflowerblue;

      /* Specific color for light theme */
      --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));
      --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, white)));
      --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));
      --sklearn-color-icon: #696969;

      @media (prefers-color-scheme: dark) {
        /* Redefinition of color scheme for dark theme */
        --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));
        --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, #111)));
        --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));
        --sklearn-color-icon: #878787;
      }
    }

    #sk-container-id-1 {
      color: var(--sklearn-color-text);
    }

    #sk-container-id-1 pre {
      padding: 0;
    }

    #sk-container-id-1 input.sk-hidden--visually {
      border: 0;
      clip: rect(1px 1px 1px 1px);
      clip: rect(1px, 1px, 1px, 1px);
      height: 1px;
      margin: -1px;
      overflow: hidden;
      padding: 0;
      position: absolute;
      width: 1px;
    }

    #sk-container-id-1 div.sk-dashed-wrapped {
      border: 1px dashed var(--sklearn-color-line);
      margin: 0 0.4em 0.5em 0.4em;
      box-sizing: border-box;
      padding-bottom: 0.4em;
      background-color: var(--sklearn-color-background);
    }

    #sk-container-id-1 div.sk-container {
      /* jupyter's `normalize.less` sets `[hidden] { display: none; }`
         but bootstrap.min.css set `[hidden] { display: none !important; }`
         so we also need the `!important` here to be able to override the
         default hidden behavior on the sphinx rendered scikit-learn.org.
         See: https://github.com/scikit-learn/scikit-learn/issues/21755 */
      display: inline-block !important;
      position: relative;
    }

    #sk-container-id-1 div.sk-text-repr-fallback {
      display: none;
    }

    div.sk-parallel-item,
    div.sk-serial,
    div.sk-item {
      /* draw centered vertical line to link estimators */
      background-image: linear-gradient(var(--sklearn-color-text-on-default-background), var(--sklearn-color-text-on-default-background));
      background-size: 2px 100%;
      background-repeat: no-repeat;
      background-position: center center;
    }

    /* Parallel-specific style estimator block */

    #sk-container-id-1 div.sk-parallel-item::after {
      content: "";
      width: 100%;
      border-bottom: 2px solid var(--sklearn-color-text-on-default-background);
      flex-grow: 1;
    }

    #sk-container-id-1 div.sk-parallel {
      display: flex;
      align-items: stretch;
      justify-content: center;
      background-color: var(--sklearn-color-background);
      position: relative;
    }

    #sk-container-id-1 div.sk-parallel-item {
      display: flex;
      flex-direction: column;
    }

    #sk-container-id-1 div.sk-parallel-item:first-child::after {
      align-self: flex-end;
      width: 50%;
    }

    #sk-container-id-1 div.sk-parallel-item:last-child::after {
      align-self: flex-start;
      width: 50%;
    }

    #sk-container-id-1 div.sk-parallel-item:only-child::after {
      width: 0;
    }

    /* Serial-specific style estimator block */

    #sk-container-id-1 div.sk-serial {
      display: flex;
      flex-direction: column;
      align-items: center;
      background-color: var(--sklearn-color-background);
      padding-right: 1em;
      padding-left: 1em;
    }


    /* Toggleable style: style used for estimator/Pipeline/ColumnTransformer box that is
    clickable and can be expanded/collapsed.
    - Pipeline and ColumnTransformer use this feature and define the default style
    - Estimators will overwrite some part of the style using the `sk-estimator` class
    */

    /* Pipeline and ColumnTransformer style (default) */

    #sk-container-id-1 div.sk-toggleable {
      /* Default theme specific background. It is overwritten whether we have a
      specific estimator or a Pipeline/ColumnTransformer */
      background-color: var(--sklearn-color-background);
    }

    /* Toggleable label */
    #sk-container-id-1 label.sk-toggleable__label {
      cursor: pointer;
      display: block;
      width: 100%;
      margin-bottom: 0;
      padding: 0.5em;
      box-sizing: border-box;
      text-align: center;
    }

    #sk-container-id-1 label.sk-toggleable__label-arrow:before {
      /* Arrow on the left of the label */
      content: "▸";
      float: left;
      margin-right: 0.25em;
      color: var(--sklearn-color-icon);
    }

    #sk-container-id-1 label.sk-toggleable__label-arrow:hover:before {
      color: var(--sklearn-color-text);
    }

    /* Toggleable content - dropdown */

    #sk-container-id-1 div.sk-toggleable__content {
      max-height: 0;
      max-width: 0;
      overflow: hidden;
      text-align: left;
      /* unfitted */
      background-color: var(--sklearn-color-unfitted-level-0);
    }

    #sk-container-id-1 div.sk-toggleable__content.fitted {
      /* fitted */
      background-color: var(--sklearn-color-fitted-level-0);
    }

    #sk-container-id-1 div.sk-toggleable__content pre {
      margin: 0.2em;
      border-radius: 0.25em;
      color: var(--sklearn-color-text);
      /* unfitted */
      background-color: var(--sklearn-color-unfitted-level-0);
    }

    #sk-container-id-1 div.sk-toggleable__content.fitted pre {
      /* unfitted */
      background-color: var(--sklearn-color-fitted-level-0);
    }

    #sk-container-id-1 input.sk-toggleable__control:checked~div.sk-toggleable__content {
      /* Expand drop-down */
      max-height: 200px;
      max-width: 100%;
      overflow: auto;
    }

    #sk-container-id-1 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {
      content: "▾";
    }

    /* Pipeline/ColumnTransformer-specific style */

    #sk-container-id-1 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {
      color: var(--sklearn-color-text);
      background-color: var(--sklearn-color-unfitted-level-2);
    }

    #sk-container-id-1 div.sk-label.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {
      background-color: var(--sklearn-color-fitted-level-2);
    }

    /* Estimator-specific style */

    /* Colorize estimator box */
    #sk-container-id-1 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {
      /* unfitted */
      background-color: var(--sklearn-color-unfitted-level-2);
    }

    #sk-container-id-1 div.sk-estimator.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {
      /* fitted */
      background-color: var(--sklearn-color-fitted-level-2);
    }

    #sk-container-id-1 div.sk-label label.sk-toggleable__label,
    #sk-container-id-1 div.sk-label label {
      /* The background is the default theme color */
      color: var(--sklearn-color-text-on-default-background);
    }

    /* On hover, darken the color of the background */
    #sk-container-id-1 div.sk-label:hover label.sk-toggleable__label {
      color: var(--sklearn-color-text);
      background-color: var(--sklearn-color-unfitted-level-2);
    }

    /* Label box, darken color on hover, fitted */
    #sk-container-id-1 div.sk-label.fitted:hover label.sk-toggleable__label.fitted {
      color: var(--sklearn-color-text);
      background-color: var(--sklearn-color-fitted-level-2);
    }

    /* Estimator label */

    #sk-container-id-1 div.sk-label label {
      font-family: monospace;
      font-weight: bold;
      display: inline-block;
      line-height: 1.2em;
    }

    #sk-container-id-1 div.sk-label-container {
      text-align: center;
    }

    /* Estimator-specific */
    #sk-container-id-1 div.sk-estimator {
      font-family: monospace;
      border: 1px dotted var(--sklearn-color-border-box);
      border-radius: 0.25em;
      box-sizing: border-box;
      margin-bottom: 0.5em;
      /* unfitted */
      background-color: var(--sklearn-color-unfitted-level-0);
    }

    #sk-container-id-1 div.sk-estimator.fitted {
      /* fitted */
      background-color: var(--sklearn-color-fitted-level-0);
    }

    /* on hover */
    #sk-container-id-1 div.sk-estimator:hover {
      /* unfitted */
      background-color: var(--sklearn-color-unfitted-level-2);
    }

    #sk-container-id-1 div.sk-estimator.fitted:hover {
      /* fitted */
      background-color: var(--sklearn-color-fitted-level-2);
    }

    /* Specification for estimator info (e.g. "i" and "?") */

    /* Common style for "i" and "?" */

    .sk-estimator-doc-link,
    a:link.sk-estimator-doc-link,
    a:visited.sk-estimator-doc-link {
      float: right;
      font-size: smaller;
      line-height: 1em;
      font-family: monospace;
      background-color: var(--sklearn-color-background);
      border-radius: 1em;
      height: 1em;
      width: 1em;
      text-decoration: none !important;
      margin-left: 1ex;
      /* unfitted */
      border: var(--sklearn-color-unfitted-level-1) 1pt solid;
      color: var(--sklearn-color-unfitted-level-1);
    }

    .sk-estimator-doc-link.fitted,
    a:link.sk-estimator-doc-link.fitted,
    a:visited.sk-estimator-doc-link.fitted {
      /* fitted */
      border: var(--sklearn-color-fitted-level-1) 1pt solid;
      color: var(--sklearn-color-fitted-level-1);
    }

    /* On hover */
    div.sk-estimator:hover .sk-estimator-doc-link:hover,
    .sk-estimator-doc-link:hover,
    div.sk-label-container:hover .sk-estimator-doc-link:hover,
    .sk-estimator-doc-link:hover {
      /* unfitted */
      background-color: var(--sklearn-color-unfitted-level-3);
      color: var(--sklearn-color-background);
      text-decoration: none;
    }

    div.sk-estimator.fitted:hover .sk-estimator-doc-link.fitted:hover,
    .sk-estimator-doc-link.fitted:hover,
    div.sk-label-container:hover .sk-estimator-doc-link.fitted:hover,
    .sk-estimator-doc-link.fitted:hover {
      /* fitted */
      background-color: var(--sklearn-color-fitted-level-3);
      color: var(--sklearn-color-background);
      text-decoration: none;
    }

    /* Span, style for the box shown on hovering the info icon */
    .sk-estimator-doc-link span {
      display: none;
      z-index: 9999;
      position: relative;
      font-weight: normal;
      right: .2ex;
      padding: .5ex;
      margin: .5ex;
      width: min-content;
      min-width: 20ex;
      max-width: 50ex;
      color: var(--sklearn-color-text);
      box-shadow: 2pt 2pt 4pt #999;
      /* unfitted */
      background: var(--sklearn-color-unfitted-level-0);
      border: .5pt solid var(--sklearn-color-unfitted-level-3);
    }

    .sk-estimator-doc-link.fitted span {
      /* fitted */
      background: var(--sklearn-color-fitted-level-0);
      border: var(--sklearn-color-fitted-level-3);
    }

    .sk-estimator-doc-link:hover span {
      display: block;
    }

    /* "?"-specific style due to the `<a>` HTML tag */

    #sk-container-id-1 a.estimator_doc_link {
      float: right;
      font-size: 1rem;
      line-height: 1em;
      font-family: monospace;
      background-color: var(--sklearn-color-background);
      border-radius: 1rem;
      height: 1rem;
      width: 1rem;
      text-decoration: none;
      /* unfitted */
      color: var(--sklearn-color-unfitted-level-1);
      border: var(--sklearn-color-unfitted-level-1) 1pt solid;
    }

    #sk-container-id-1 a.estimator_doc_link.fitted {
      /* fitted */
      border: var(--sklearn-color-fitted-level-1) 1pt solid;
      color: var(--sklearn-color-fitted-level-1);
    }

    /* On hover */
    #sk-container-id-1 a.estimator_doc_link:hover {
      /* unfitted */
      background-color: var(--sklearn-color-unfitted-level-3);
      color: var(--sklearn-color-background);
      text-decoration: none;
    }

    #sk-container-id-1 a.estimator_doc_link.fitted:hover {
      /* fitted */
      background-color: var(--sklearn-color-fitted-level-3);
    }
    </style><div id="sk-container-id-1" class="sk-top-container"><div class="sk-text-repr-fallback"><pre>FactorAnalyzer(rotation=&#x27;varimax&#x27;, rotation_kwargs={})</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class="sk-container" hidden><div class="sk-item"><div class="sk-estimator fitted sk-toggleable"><input class="sk-toggleable__control sk-hidden--visually" id="sk-estimator-id-1" type="checkbox" checked><label for="sk-estimator-id-1" class="sk-toggleable__label fitted sk-toggleable__label-arrow fitted">&nbsp;FactorAnalyzer<span class="sk-estimator-doc-link fitted">i<span>Fitted</span></span></label><div class="sk-toggleable__content fitted"><pre>FactorAnalyzer(rotation=&#x27;varimax&#x27;, rotation_kwargs={})</pre></div> </div></div></div></div>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 102-103

We obtain the factor loadings and label them

.. GENERATED FROM PYTHON SOURCE LINES 103-107

.. code-block:: Python

    labeled_results = pd.DataFrame(fa.loadings_)
    labeled_results.index = variables
    display(labeled_results)





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

                      0         1         2
    Envir01   -0.564729 -0.002045  0.111001
    Envir02   -0.407355  0.047089  0.105001
    Envir03    0.413744  0.135357  0.089509
    Envir04    0.339893  0.164578  0.027584
    Envir05   -0.293116  0.082449  0.117659
    Envir06   -0.397525 -0.000947  0.142948
    Mobil01    0.070801  0.094246  0.348101
    Mobil02    0.161354  0.221123  0.350501
    Mobil03   -0.034115  0.120752  0.377315
    Mobil04   -0.158243 -0.018616  0.160603
    Mobil05   -0.174380  0.146278  0.352953
    Mobil06    0.274974  0.163948  0.116831
    Mobil07    0.230149 -0.018872  0.140380
    Mobil08    0.380445  0.242195  0.165503
    Mobil09   -0.286631 -0.006629  0.233548
    Mobil10    0.341252  0.126686  0.191877
    Mobil11    0.482423  0.061773  0.101396
    Mobil12    0.291986  0.345614 -0.043070
    Mobil13    0.293047 -0.005639  0.040887
    Mobil14    0.476652  0.171167  0.070976
    Mobil15   -0.165495 -0.042412  0.151499
    Mobil16    0.459362  0.066581  0.077292
    Mobil17    0.431262  0.098147  0.208418
    Mobil18    0.333607  0.214557  0.060210
    Mobil19   -0.235192 -0.035251  0.368719
    Mobil20    0.025723 -0.060406  0.412330
    Mobil21    0.147848  0.092530  0.295888
    Mobil22    0.178539  0.182377  0.078155
    Mobil23   -0.070000 -0.026870  0.243105
    Mobil24   -0.310479  0.065048  0.299017
    Mobil25    0.193832  0.355736  0.188833
    Mobil26    0.045875  0.082315  0.417643
    Mobil27    0.083807  0.096861  0.368456
    ResidCh01 -0.021483  0.565421 -0.028667
    ResidCh02 -0.345073  0.325007  0.170992
    ResidCh03  0.151949  0.141772  0.103544
    ResidCh04  0.026371  0.413732  0.115635
    ResidCh05 -0.088138  0.606071 -0.058620
    ResidCh06 -0.024728  0.441251 -0.100628
    ResidCh07  0.117831 -0.375275  0.263202
    LifSty01  -0.111344  0.116332  0.148828
    LifSty02   0.182615  0.224647  0.108381
    LifSty03  -0.015983  0.033632  0.223094
    LifSty04   0.061664  0.064185  0.161211
    LifSty05   0.017626  0.193170  0.276527
    LifSty06   0.046335  0.310471  0.258672
    LifSty07   0.157569  0.447161  0.108234
    LifSty08  -0.066643 -0.151222  0.250315
    LifSty09   0.105715  0.302478  0.283061
    LifSty10   0.122540  0.403303  0.160987
    LifSty11   0.084352  0.244044  0.185830
    LifSty12  -0.231626  0.178191  0.071661
    LifSty13  -0.188736  0.014129  0.063625
    LifSty14   0.035538 -0.002001  0.181145




.. GENERATED FROM PYTHON SOURCE LINES 108-110

When we print the results, we keep only loadings that are 0.4 or
higher, in absolute value.

.. GENERATED FROM PYTHON SOURCE LINES 110-114

.. code-block:: Python


    labeled_results = labeled_results.astype('object')
    labeled_results[(labeled_results <= 0.4) & (labeled_results >= -0.4)] = ''








.. GENERATED FROM PYTHON SOURCE LINES 115-116

Results.

.. GENERATED FROM PYTHON SOURCE LINES 116-117

.. code-block:: Python

    display(labeled_results)




.. rst-class:: sphx-glr-script-out

 .. code-block:: none

                      0         1         2
    Envir01   -0.564729                    
    Envir02   -0.407355                    
    Envir03    0.413744                    
    Envir04                                
    Envir05                                
    Envir06                                
    Mobil01                                
    Mobil02                                
    Mobil03                                
    Mobil04                                
    Mobil05                                
    Mobil06                                
    Mobil07                                
    Mobil08                                
    Mobil09                                
    Mobil10                                
    Mobil11    0.482423                    
    Mobil12                                
    Mobil13                                
    Mobil14    0.476652                    
    Mobil15                                
    Mobil16    0.459362                    
    Mobil17    0.431262                    
    Mobil18                                
    Mobil19                                
    Mobil20                         0.41233
    Mobil21                                
    Mobil22                                
    Mobil23                                
    Mobil24                                
    Mobil25                                
    Mobil26                        0.417643
    Mobil27                                
    ResidCh01            0.565421          
    ResidCh02                              
    ResidCh03                              
    ResidCh04            0.413732          
    ResidCh05            0.606071          
    ResidCh06            0.441251          
    ResidCh07                              
    LifSty01                               
    LifSty02                               
    LifSty03                               
    LifSty04                               
    LifSty05                               
    LifSty06                               
    LifSty07             0.447161          
    LifSty08                               
    LifSty09                               
    LifSty10             0.403303          
    LifSty11                               
    LifSty12                               
    LifSty13                               
    LifSty14                               





.. rst-class:: sphx-glr-timing

   **Total running time of the script:** (0 minutes 1.016 seconds)


.. _sphx_glr_download_auto_examples_latent_plot_b00factor_analysis.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: plot_b00factor_analysis.ipynb <plot_b00factor_analysis.ipynb>`

    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: plot_b00factor_analysis.py <plot_b00factor_analysis.py>`

    .. container:: sphx-glr-download sphx-glr-download-zip

      :download:`Download zipped: plot_b00factor_analysis.zip <plot_b00factor_analysis.zip>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_