# Multinomial Logit

This is a step-by-step guide on how to estimate Multinomial Logit models using the `xlogit` package. You can interactively execute the code in this guide by opening it Google Colab using the following link:

[](https://colab.research.google.com/github/arteagac/xlogit/blob/master/examples/multinomial_model.ipynb)

## Install `xlogit` package

Install `xlogit` using `pip` as shown below.

```python
!pip install xlogit
```

    Collecting xlogit
      Downloading https://files.pythonhosted.org/packages/60/5f/9bc576d180c366af77bc04e268536e9e34be23c52a520918aa0cb56b438e/xlogit-0.1.3-py3-none-any.whl
    Requirement already satisfied: numpy>=1.13.1 in /usr/local/lib/python3.7/dist-packages (from xlogit) (1.19.5)
    Requirement already satisfied: scipy>=1.0.0 in /usr/local/lib/python3.7/dist-packages (from xlogit) (1.4.1)
    Installing collected packages: xlogit
    Successfully installed xlogit-0.1.3

## Route Choice Dataset

This dataset contains choices of 151 commuters among three Home-to-work route alternatives. The three alternatives are arterial, rural, and freeway roads. This dataset was taken from Example 13.1 of the book "Statistical and econometric methods for transportation data analysis" [(Washintong et. al., 2011) ](https://engineering.purdue.edu/~flm/StatEconBook.htm).

### Read data

We start by importing the data using pandas and renaming the columns of interest (choice, distance, male, and vehicle model). In addition, we create a column with the name of the alternatives and a column that uniquely identifies every observation in the dataset. Note that this dataset is long format.

```python
import pandas as pd
import numpy as np
df = pd.read_csv("https://engineering.purdue.edu/~flm/StatEcon-Files/Ex13-1.txt",
                 sep="\t", header=None, prefix="x")
df.rename(columns={'x0': 'choice', 'x6': 'dist', 'x10': 'male', 'x14': 'vehmodel'},
          inplace=True)  # Rename columns of interest
df['alt'] = np.tile(['arterial', 'rural', 'freeway'], len(df)//3)  # Add column with alternatives
df['ids'] = np.repeat(np.arange(len(df)//3), 3)  # Add column with unique ids
df['vehage'] = 86 - df['vehmodel']
df
```

      choice
      x1
      x2
      x3
      x4
      x5
      dist
      x7
      x8
      x9
      male
      x11
      x12
      x13
      vehmodel
      x15
      x16
      alt
      ids
      vehage

      0
      1
      1
      0
      0
      460
      14
      48
      0
      0
      2
      0
      1
      0
      1
      86
      0
      28
      arterial
      0
      0

      1
      0
      0
      1
      0
      440
      7
      44
      0
      0
      2
      0
      1
      0
      1
      86
      0
      28
      rural
      0
      0

      2
      0
      0
      0
      1
      130
      7
      61
      0
      0
      2
      0
      1
      0
      1
      86
      0
      28
      freeway
      0
      0

      3
      1
      1
      0
      0
      595
      13
      59
      1
      0
      2
      1
      1
      0
      2
      85
      0
      27
      arterial
      1
      1

      4
      0
      0
      1
      0
      515
      13
      70
      1
      0
      2
      1
      1
      0
      2
      85
      0
      27
      rural
      1
      1

      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...

      448
      0
      0
      1
      0
      325
      10
      70
      0
      1
      2
      1
      1
      0
      1
      84
      1
      24
      rural
      149
      2

      449
      1
      0
      0
      1
      200
      5
      74
      0
      1
      2
      1
      1
      0
      1
      84
      1
      24
      freeway
      149
      2

      450
      0
      1
      0
      0
      900
      14
      51
      1
      0
      2
      1
      1
      0
      3
      83
      1
      18
      arterial
      150
      3

      451
      0
      0
      1
      0
      550
      7
      47
      1
      0
      2
      1
      1
      0
      3
      83
      1
      18
      rural
      150
      3

      452
      1
      0
      0
      1
      220
      7
      64
      1
      0
      2
      1
      1
      0
      3
      83
      1
      18
      freeway
      150
      3

453 rows × 20 columns

### Create model specification

The following code creates the model specification by including additional variables in the dataframe to accomodate the specification needs. The newly added variables are simply the product of existing variables with dummy variables for the different alternatives. Note that the specification in the code below corresponds to the following utility maximization formulation:

\begin{equation}
\begin{array}{}
V_{arterial} & = & \quad \beta_3DIST_{arterial} \\
V_{rural} & = \beta_1ASC_{rural} & +  \beta_4DIST_{rural} & + \beta_6VEHAGE_{rural}  \\
V_{freeway} & = \beta_2ASC_{freew} & +  \beta_5DIST_{freeway} & +  \beta_7{VEHAGE_{freeway}} & + \beta_8MALE_{freeway}
\end{array}
\end{equation}

```python
# Alternative specific constants
df['asc_rural'] = np.ones(len(df)) * (df['alt'] == 'rural')
df['asc_freeway'] = np.ones(len(df)) * (df['alt'] == 'freeway')

# Distance
df['dist_arterial'] = df['dist'] * (df['alt'] == 'arterial')
df['dist_rural'] = df['dist'] * (df['alt'] == 'rural')
df['dist_freeway'] = df['dist'] * (df['alt'] == 'freeway')

# Vehicle age
df['vehage_rural'] = df['vehage'] * (df['alt'] == 'rural')
df['vehage_freeway'] = df['vehage'] * (df['alt'] == 'freeway')

# Male driver
df['male_freeway'] = df['male'] * (df['alt'] == 'freeway')
```

### Estimate model

After creating the model specification, we can use `xlogit` to estimate the model as follows:

```python
from xlogit import MultinomialLogit
varnames=['asc_rural', 'asc_freeway', 'dist_arter', 'dist_rural',
          'dist_freew', 'vehage_rural', 'vehage_freew', 'male_freew']
model = MultinomialLogit()
model.fit(X=df[varnames], y=df['choice'], varnames=varnames,
          ids=df['ids'], alts=df['alt'])
model.summary()
```

    Estimation time= 0.0 seconds
    ---------------------------------------------------------------------------
    Coefficient              Estimate      Std.Err.         z-val         P>|z|
    ---------------------------------------------------------------------------
    asc_rural               2.8131275     1.1504189     2.4453072        0.0416 *
    asc_freeway            -2.6868817     2.2817329    -1.1775619         0.398
    dist_arterial          -0.1229102     0.0240053    -5.1201358      4.14e-06 ***
    dist_rural             -0.1773579     0.0279818    -6.3383224       1.3e-08 ***
    dist_freeway           -0.0956391     0.0369681    -2.5870738        0.0295 *
    vehage_rural            0.1236721     0.0535597     2.3090535         0.057 .
    vehage_freeway          0.2268642     0.0755401     3.0032267       0.00969 **
    male_freeway            0.5990000     0.6202114     0.9657996         0.499
    ---------------------------------------------------------------------------
    Significance:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

    Log-Likelihood= -94.440
    AIC= 204.881
    BIC= 229.019

## Swissmetro Dataset

In this example, we will estimate a Multinomial Logit where each alternative is defined with a different utility specification. The swissmetro dataset is an SP/RP survey dataset popularly used in Biogeme and Pylogit examples. The dataset is available at http://transp-or.epfl.ch/data/swissmetro.dat and [Bierlaire et. al., (2001)](https://transp-or.epfl.ch/documents/proceedings/BierAxhaAbay01.pdf) provides a detailed discussion of the data as wells as its context and collection process. . Note that the dataset is available in wide format; therefore, we need to convert it to long format for `xlogit`.

### Read data

The dataset is imported to the Python environment using `pandas`. Then, two types of samples, ones with a trip purpose different to commute or business and ones with an unknown choice, are filtered out. The original dataset contains 10,729 records, but after filtering, 6,768 records remain for following analysis. Finally, a new column that uniquely identifies each sample is added to the dataframe and the `CHOICE` column, which originally contains a numerical coding of the choices, is mapped to a description that is consistent with the alternatives in the column names.

```python
import pandas as pd
import numpy as np

df_wide = pd.read_table("http://transp-or.epfl.ch/data/swissmetro.dat", sep='\t')

# Keep only observations for commute and business purposes that contain known choices
df_wide = df_wide[(df_wide['PURPOSE'].isin([1, 3]) & (df_wide['CHOICE'] != 0))]

df_wide['custom_id'] = np.arange(len(df_wide))  # Add unique identifier
df_wide['CHOICE'] = df_wide['CHOICE'].map({1: 'TRAIN', 2:'SM', 3: 'CAR'})
df_wide
```

      GROUP
      SURVEY
      SP
      ID
      PURPOSE
      FIRST
      TICKET
      WHO
      LUGGAGE
      AGE
      MALE
      INCOME
      GA
      ORIGIN
      DEST
      TRAIN_AV
      CAR_AV
      SM_AV
      TRAIN_TT
      TRAIN_CO
      TRAIN_HE
      SM_TT
      SM_CO
      SM_HE
      SM_SEATS
      CAR_TT
      CAR_CO
      CHOICE
      custom_id

      0
      2
      0
      1
      1
      1
      0
      1
      1
      0
      3
      0
      2
      0
      2
      1
      1
      1
      1
      112
      48
      120
      63
      52
      20
      0
      117
      65
      SM
      0

      1
      2
      0
      1
      1
      1
      0
      1
      1
      0
      3
      0
      2
      0
      2
      1
      1
      1
      1
      103
      48
      30
      60
      49
      10
      0
      117
      84
      SM
      1

      2
      2
      0
      1
      1
      1
      0
      1
      1
      0
      3
      0
      2
      0
      2
      1
      1
      1
      1
      130
      48
      60
      67
      58
      30
      0
      117
      52
      SM
      2

      3
      2
      0
      1
      1
      1
      0
      1
      1
      0
      3
      0
      2
      0
      2
      1
      1
      1
      1
      103
      40
      30
      63
      52
      20
      0
      72
      52
      SM
      3

      4
      2
      0
      1
      1
      1
      0
      1
      1
      0
      3
      0
      2
      0
      2
      1
      1
      1
      1
      130
      36
      60
      63
      42
      20
      0
      90
      84
      SM
      4

      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...

      8446
      3
      1
      1
      939
      3
      1
      7
      3
      1
      5
      1
      2
      0
      1
      2
      1
      1
      1
      108
      13
      30
      50
      17
      30
      0
      130
      64
      TRAIN
      6763

      8447
      3
      1
      1
      939
      3
      1
      7
      3
      1
      5
      1
      2
      0
      1
      2
      1
      1
      1
      108
      12
      30
      53
      16
      10
      0
      80
      80
      TRAIN
      6764

      8448
      3
      1
      1
      939
      3
      1
      7
      3
      1
      5
      1
      2
      0
      1
      2
      1
      1
      1
      108
      16
      60
      50
      16
      20
      0
      80
      64
      TRAIN
      6765

      8449
      3
      1
      1
      939
      3
      1
      7
      3
      1
      5
      1
      2
      0
      1
      2
      1
      1
      1
      128
      16
      30
      53
      17
      30
      0
      80
      104
      TRAIN
      6766

      8450
      3
      1
      1
      939
      3
      1
      7
      3
      1
      5
      1
      2
      0
      1
      2
      1
      1
      1
      108
      13
      60
      53
      21
      30
      0
      100
      80
      TRAIN
      6767

6768 rows × 29 columns

### Reshape data

Given that `xlogit` requires the dataset to be provided in the long format, we reshape the dataset using the `wide_to_long` utility provided by `xlogit`. This function takes as input the column that uniquely identifies each sample (`id_col`), the name of column to save the alternatives (`alt_name`), the list of alternatives (`alt_list`), the columns that vary across alternatives (`varying`), and whether the alternative names are prefix in the column names (`alt_is_prefix`).The `wide_to_long` method fills with `NaN` the columns that do not have certain alternatives (e.g. `SEATS` and `HE`). Depending on your specification needs, you can ignore the `NaN` or replace them with zeros. In this case we replaced them with zeros using the `empty_val` parameter. Additional details about the `wide_to_long` function can be found in the [xlogit's documentation](https://xlogit.readthedocs.io/en/latest/api/utils.html).

```python
from xlogit.utils import wide_to_long

df = wide_to_long(df_wide, id_col='custom_id', alt_name='alt', sep='_',
                  alt_list=['TRAIN', 'SM', 'CAR'], empty_val=0,
                  varying=['TT', 'CO', 'HE', 'AV', 'SEATS'], alt_is_prefix=True)
df
```

      custom_id
      alt
      TT
      CO
      HE
      AV
      SEATS
      GROUP
      SURVEY
      SP
      ID
      PURPOSE
      FIRST
      TICKET
      WHO
      LUGGAGE
      AGE
      MALE
      INCOME
      GA
      ORIGIN
      DEST
      CHOICE

      0
      0
      CAR
      117
      65
      0
      1
      0
      2
      0
      1
      1
      1
      0
      1
      1
      0
      3
      0
      2
      0
      2
      1
      SM

      1
      0
      SM
      63
      52
      20
      1
      0
      2
      0
      1
      1
      1
      0
      1
      1
      0
      3
      0
      2
      0
      2
      1
      SM

      2
      0
      TRAIN
      112
      48
      120
      1
      0
      2
      0
      1
      1
      1
      0
      1
      1
      0
      3
      0
      2
      0
      2
      1
      SM

      3
      1
      CAR
      117
      84
      0
      1
      0
      2
      0
      1
      1
      1
      0
      1
      1
      0
      3
      0
      2
      0
      2
      1
      SM

      4
      1
      SM
      60
      49
      10
      1
      0
      2
      0
      1
      1
      1
      0
      1
      1
      0
      3
      0
      2
      0
      2
      1
      SM

      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...

      20299
      6766
      SM
      53
      17
      30
      1
      0
      3
      1
      1
      939
      3
      1
      7
      3
      1
      5
      1
      2
      0
      1
      2
      TRAIN

      20300
      6766
      TRAIN
      128
      16
      30
      1
      0
      3
      1
      1
      939
      3
      1
      7
      3
      1
      5
      1
      2
      0
      1
      2
      TRAIN

      20301
      6767
      CAR
      100
      80
      0
      1
      0
      3
      1
      1
      939
      3
      1
      7
      3
      1
      5
      1
      2
      0
      1
      2
      TRAIN

      20302
      6767
      SM
      53
      21
      30
      1
      0
      3
      1
      1
      939
      3
      1
      7
      3
      1
      5
      1
      2
      0
      1
      2
      TRAIN

      20303
      6767
      TRAIN
      108
      13
      60
      1
      0
      3
      1
      1
      939
      3
      1
      7
      3
      1
      5
      1
      2
      0
      1
      2
      TRAIN

20304 rows × 23 columns

We then scale some variables as per the [examples in Biogeme and Pylogit](https://https://github.com/timothyb0912/pylogit/blob/master/examples/notebooks/Main%20PyLogit%20Example.ipynb). The time and headway variables are converted to hours, the price is scaled, and the new variables `single_luggage`, `free ticket`, `multiple_luggage`, `regular_class` and `train_survey` are created to accommodate the model specification as shown below:

```python
# Scale travel time and headway variables to hours
df['time'] = df['TT'] / 60.0
df['headway'] = df['HE'] / 60.0
df['cost'] = df['CO'] / 100.0

# We set the cost as zero for individuals with an annual pass paid by employer
annual_pass = (df['alt'].isin(['TRAIN', 'SM'])) & ((df["GA"] == 1) | (df["WHO"] == 2))
df["cost"] = df["cost"] * (~annual_pass)

#Travellers carrying only single luggage
df["single_luggage"] = (df["LUGGAGE"] == 1).astype(int)

#Travellers carrying more than one luggage
df["multiple_luggage"] = (df["LUGGAGE"] == 3).astype(int)

# Travellers travelling in classes other than First class
df["regular_class"] = 1 - df["FIRST"]

# Travellers who responded to the survey while on a train
df["train_survey"] = 1 - df["SURVEY"]
df
```

      custom_id
      alt
      TT
      CO
      HE
      AV
      SEATS
      GROUP
      SURVEY
      SP
      ID
      PURPOSE
      FIRST
      TICKET
      WHO
      LUGGAGE
      AGE
      MALE
      INCOME
      GA
      ORIGIN
      DEST
      CHOICE
      time
      headway
      cost
      single_luggage
      multiple_luggage
      regular_class
      train_survey

      0
      0
      CAR
      117
      65
      0
      1
      0
      2
      0
      1
      1
      1
      0
      1
      1
      0
      3
      0
      2
      0
      2
      1
      SM
      1.950000
      0.000000
      0.65
      0
      0
      1
      1

      1
      0
      SM
      63
      52
      20
      1
      0
      2
      0
      1
      1
      1
      0
      1
      1
      0
      3
      0
      2
      0
      2
      1
      SM
      1.050000
      0.333333
      0.52
      0
      0
      1
      1

      2
      0
      TRAIN
      112
      48
      120
      1
      0
      2
      0
      1
      1
      1
      0
      1
      1
      0
      3
      0
      2
      0
      2
      1
      SM
      1.866667
      2.000000
      0.48
      0
      0
      1
      1

      3
      1
      CAR
      117
      84
      0
      1
      0
      2
      0
      1
      1
      1
      0
      1
      1
      0
      3
      0
      2
      0
      2
      1
      SM
      1.950000
      0.000000
      0.84
      0
      0
      1
      1

      4
      1
      SM
      60
      49
      10
      1
      0
      2
      0
      1
      1
      1
      0
      1
      1
      0
      3
      0
      2
      0
      2
      1
      SM
      1.000000
      0.166667
      0.49
      0
      0
      1
      1

      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...

      20299
      6766
      SM
      53
      17
      30
      1
      0
      3
      1
      1
      939
      3
      1
      7
      3
      1
      5
      1
      2
      0
      1
      2
      TRAIN
      0.883333
      0.500000
      0.17
      1
      0
      0
      0

      20300
      6766
      TRAIN
      128
      16
      30
      1
      0
      3
      1
      1
      939
      3
      1
      7
      3
      1
      5
      1
      2
      0
      1
      2
      TRAIN
      2.133333
      0.500000
      0.16
      1
      0
      0
      0

      20301
      6767
      CAR
      100
      80
      0
      1
      0
      3
      1
      1
      939
      3
      1
      7
      3
      1
      5
      1
      2
      0
      1
      2
      TRAIN
      1.666667
      0.000000
      0.80
      1
      0
      0
      0

      20302
      6767
      SM
      53
      21
      30
      1
      0
      3
      1
      1
      939
      3
      1
      7
      3
      1
      5
      1
      2
      0
      1
      2
      TRAIN
      0.883333
      0.500000
      0.21
      1
      0
      0
      0

      20303
      6767
      TRAIN
      108
      13
      60
      1
      0
      3
      1
      1
      939
      3
      1
      7
      3
      1
      5
      1
      2
      0
      1
      2
      TRAIN
      1.800000
      1.000000
      0.13
      1
      0
      0
      0

20304 rows × 30 columns

### Create model specification

By operating the dataframe columns, highly-flexible utility specifications can be modeled in `xlogit`. As shown below, alternative specific constants or coefficients can be included in the specification by strategically creating new columns and setting their values depending on the alternative. This flexibility allows even the specification of one or multiple coefficients per alternative or group of alternatives.

```python
# Create model specification
# Alternative Specific Constants
df['asc_train'] = np.ones(len(df))*(df['alt'] == 'TRAIN')
df['asc_sm'] = np.ones(len(df))*(df['alt'] == 'SM')

# Travel cost (One coefficient per alternative)
df['cost_train'] = df['cost']*(df['alt'] == 'TRAIN')
df['cost_sm'] = df['cost']*(df['alt'] == 'SM')
df['cost_car'] = df['cost']*(df['alt'] == 'CAR')

# Travel time (One coefficient for train and sm and other for car)
df['time_train_sm'] = df['time']*((df['alt'] == 'TRAIN') | (df['alt'] == 'SM'))
df['time_car'] = df['time']*(df['alt'] == 'CAR')

# Headway (One coefficient per alternative, except for car)
df['headway_train'] = df['headway']*(df['alt'] == 'TRAIN')
df['headway_sm'] = df['headway']*(df['alt'] == 'SM')

# Seat config (Coefficient only for swissmetro)
df['seatconf_sm'] = df['SEATS']*(df['alt'] == 'SM')

# Train Survey (Coefficient only for swissmetro)
df['survey_train_sm'] = df['train_survey']* ((df['alt'] == 'TRAIN') | (df['alt'] == 'SM'))

# Regular class (Coefficient only for swissmetro)
df['regular_class_sm'] = df['regular_class']*(df['alt'] == 'TRAIN')

# Luggage (Coefficient only for car)
df['single_lug_car'] = df['single_luggage']*(df['alt'] == 'CAR')
df['multip_lug_car'] = df['multiple_luggage']*(df['alt'] == 'CAR')
```

### Estimate model

The swissmetro dataset contains unbalanced choice situations across individuals (i.e., some individuals do not have observations for all alternatives). The `avail` option enables estimation for such datasets. `avail` takes the values that indicate the availability of each alternative across individuals.
Once the model specification is complete, the model is estimated as follows:

```python
from xlogit import MultinomialLogit

varnames=['asc_train', 'asc_sm', 'time_train_sm', 'time_car', 'cost_train',
          'cost_sm', 'cost_car', 'headway_train', 'headway_sm', 'seatconf_sm',
          'survey_train_sm', 'regular_class_sm', 'single_lug_car',
          'multip_lug_car']
model = MultinomialLogit()
model.fit(X=df[varnames],
          y=df['CHOICE'],
          varnames=varnames,
          alts=df['alt'],
          ids=df['custom_id'],
          avail=df['AV'])
model.summary()
```

    Estimation time= 0.1 seconds
    ---------------------------------------------------------------------------
    Coefficient              Estimate      Std.Err.         z-val         P>|z|
    ---------------------------------------------------------------------------
    asc_train              -1.2929512     0.1237556   -10.4476139      2.43e-24 ***
    asc_sm                 -0.5026152     0.1032927    -4.8659312      5.87e-06 ***
    time_train_sm          -0.6990098     0.0396510   -17.6290608      8.13e-67 ***
    time_car               -0.7229887     0.0442625   -16.3340968      1.18e-57 ***
    cost_train             -0.5618773     0.0807075    -6.9618968      2.59e-11 ***
    cost_sm                -0.2816843     0.0417373    -6.7489902       1.1e-10 ***
    cost_car               -0.5139009     0.0970406    -5.2957304      6.66e-07 ***
    headway_train          -0.3143519     0.0505955    -6.2130370      3.49e-09 ***
    headway_sm             -0.3773753     0.1652542    -2.2836046        0.0589 .
    seatconf_sm            -0.7824379     0.0758912   -10.3100010      9.91e-24 ***
    survey_train_sm         2.5424946     0.0921336    27.5957408      1.5e-157 ***
    regular_class_sm        0.5650259     0.0652226     8.6630441      4.93e-17 ***
    single_lug_car          0.4227658     0.0611684     6.9115077      3.66e-11 ***
    multip_lug_car          1.4141058     0.2373032     5.9590672      1.62e-08 ***
    ---------------------------------------------------------------------------
    Significance:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

    Log-Likelihood= -5159.258
    AIC= 10346.517
    BIC= 10441.996

The estimates are identical to those provided by PyLogit (Brathwaite and Walker 2018) as shown in [this link](https://github.com/timothyb0912/pylogit/blob/master/examples/notebooks/Main%20PyLogit%20Example.ipynb)

## Fishing Dataset

The following example illustrates the estimation of a Multinomial Logit model for choices of 1,182 individuals for sport fishing modes using `xlogit`. The goal is to analyze the market shares of four alternatives (i.e., beach, pier, boat, and charter) based on their cost and fish catch rate. [Cameron (2005)](http://cameron.econ.ucdavis.edu/mmabook/mma.html) provides additional details about this dataset. The following code illustrates how to use `xlogit` to estimate the model parameters.

### Read data

The data to be analyzed can be imported to Python using any preferred method. In this example, the data in CSV format was imported using the popular `pandas` Python package. However, it is worth highlighting that `xlogit` does not depend on the `pandas` package, as `xlogit` can take any array-like structure as input. This represents an additional advantage because `xlogit` can be used with any preferred dataframe library, and not only with `pandas`.

```python
import pandas as pd
df = pd.read_csv("https://raw.github.com/arteagac/xlogit/master/examples/data/fishing_long.csv")
df
```

      id
      alt
      choice
      income
      price
      catch

      0
      1
      beach
      0
      7083.33170
      157.930
      0.0678

      1
      1
      boat
      0
      7083.33170
      157.930
      0.2601

      2
      1
      charter
      1
      7083.33170
      182.930
      0.5391

      3
      1
      pier
      0
      7083.33170
      157.930
      0.0503

      4
      2
      beach
      0
      1249.99980
      15.114
      0.1049

      ...
      ...
      ...
      ...
      ...
      ...
      ...

      4723
      1181
      pier
      0
      416.66668
      36.636
      0.4522

      4724
      1182
      beach
      0
      6250.00130
      339.890
      0.2537

      4725
      1182
      boat
      1
      6250.00130
      235.436
      0.6817

      4726
      1182
      charter
      0
      6250.00130
      260.436
      2.3014

      4727
      1182
      pier
      0
      6250.00130
      339.890
      0.1498

4728 rows × 6 columns

### Estimate the model

Once the data is in the `Python` environment, `xlogit` can be used to fit the model, as shown below. The `MultinomialLogit` class is imported from `xlogit`, and its constructor is used to initialize a new model. The `fit` method estimates the model using the input data and estimation criteria provided as arguments to the method's call. The arguments of the `fit` methods are described in [`xlogit`'s documentation](https://https://xlogit.readthedocs.io/en/latest/api/).

```python
from xlogit import MultinomialLogit

varnames = ['income','price', 'catch']
model = MultinomialLogit()
model.fit(X=df[varnames],
          y=df['choice'],
          varnames=varnames,
          isvars=['income'],
          ids=df['id'],
          alts=df['alt'],
          fit_intercept=True)
model.summary()
```

    Estimation time= 0.0 seconds
    ---------------------------------------------------------------------------
    Coefficient              Estimate      Std.Err.         z-val         P>|z|
    ---------------------------------------------------------------------------
    _intercept.boat         0.5273413     0.2017396     2.6139700        0.0264 *
    _intercept.charter      1.6943827     0.2096186     8.0831712       1.2e-14 ***
    _intercept.pier         0.7779899     0.2051062     3.7931076      0.000622 ***
    income.boat             0.0000894     0.0000473     1.8906643         0.134
    income.charter         -0.0000333     0.0000480    -0.6936462         0.627
    income.pier            -0.0001276     0.0000467    -2.7335425        0.0192 *
    price                  -0.0251161     0.0015240   -16.4800784      5.89e-54 ***
    catch                   0.3578374     0.0985344     3.6315992       0.00113 **
    ---------------------------------------------------------------------------
    Significance:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

    Log-Likelihood= -1215.138
    AIC= 2446.275
    BIC= 2486.875

## Heating Dataset

For this example, we use the Heating dataset from R's mlogit package (Croissant 2020), which contains choice of heating systems in California house. The dataset contains 90 observations, with 8 explanatory variables and more information can be found in https://cran.r-project.org/web/packages/mlogit/vignettes/e1mlogit.html.

### Read data

```python
import pandas as pd
import numpy as np

df_wide = pd.read_csv("https://raw.github.com/arteagac/xlogit/master/examples/data/heating_wide.csv")
df_wide
```

      idcase
      depvar
      ic.gc
      ic.gr
      ic.ec
      ic.er
      ic.hp
      oc.gc
      oc.gr
      oc.ec
      oc.er
      oc.hp
      income
      agehed
      rooms
      region

      0
      1
      gc
      866.00
      962.64
      859.90
      995.76
      1135.50
      199.69
      151.72
      553.34
      505.60
      237.88
      7
      25
      6
      ncostl

      1
      2
      gc
      727.93
      758.89
      796.82
      894.69
      968.90
      168.66
      168.66
      520.24
      486.49
      199.19
      5
      60
      5
      scostl

      2
      3
      gc
      599.48
      783.05
      719.86
      900.11
      1048.30
      165.58
      137.80
      439.06
      404.74
      171.47
      4
      65
      2
      ncostl

      3
      4
      er
      835.17
      793.06
      761.25
      831.04
      1048.70
      180.88
      147.14
      483.00
      425.22
      222.95
      2
      50
      4
      scostl

      4
      5
      er
      755.59
      846.29
      858.86
      985.64
      883.05
      174.91
      138.90
      404.41
      389.52
      178.49
      2
      25
      6
      valley

      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...

      895
      896
      gc
      766.39
      877.71
      751.59
      869.78
      942.70
      142.61
      136.21
      474.48
      420.65
      203.00
      6
      20
      4
      mountn

      896
      897
      gc
      1128.50
      1167.80
      1047.60
      1292.60
      1297.10
      207.40
      213.77
      705.36
      551.61
      243.76
      7
      45
      7
      scostl

      897
      898
      gc
      787.10
      1055.20
      842.79
      1041.30
      1064.80
      175.05
      141.63
      478.86
      448.61
      254.51
      5
      60
      7
      scostl

      898
      899
      gc
      860.56
      1081.30
      799.76
      1123.20
      1218.20
      211.04
      151.31
      495.20
      401.56
      246.48
      5
      50
      6
      scostl

      899
      900
      gc
      893.94
      1119.90
      967.88
      1091.70
      1387.50
      175.80
      180.11
      518.68
      458.53
      245.13
      2
      65
      4
      ncostl

900 rows × 16 columns

### Reshape data

The dataset is available in wide format. Since `xlogit` requires the data in long format, we convert it as shown below:

```python
from xlogit.utils import wide_to_long
df = wide_to_long(df_wide, id_col='idcase', alt_name='alt', varying=['ic', 'oc'],
                  alt_list=['ec', 'er', 'gc', 'gr', 'hp'], sep='.', empty_val=0)
df
```

      idcase
      alt
      ic
      oc
      depvar
      income
      agehed
      rooms
      region

      0
      1
      ec
      859.90
      553.34
      gc
      7
      25
      6
      ncostl

      1
      1
      er
      995.76
      505.60
      gc
      7
      25
      6
      ncostl

      2
      1
      gc
      866.00
      199.69
      gc
      7
      25
      6
      ncostl

      3
      1
      gr
      962.64
      151.72
      gc
      7
      25
      6
      ncostl

      4
      1
      hp
      1135.50
      237.88
      gc
      7
      25
      6
      ncostl

      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...

      4495
      900
      ec
      967.88
      518.68
      gc
      2
      65
      4
      ncostl

      4496
      900
      er
      1091.70
      458.53
      gc
      2
      65
      4
      ncostl

      4497
      900
      gc
      893.94
      175.80
      gc
      2
      65
      4
      ncostl

      4498
      900
      gr
      1119.90
      180.11
      gc
      2
      65
      4
      ncostl

      4499
      900
      hp
      1387.50
      245.13
      gc
      2
      65
      4
      ncostl

4500 rows × 9 columns

### Estimate the model

We now import `MultinomialLogit` from xlogit and estimate the model as shown below:

```python
from xlogit import MultinomialLogit

varnames = ['ic', 'oc']
model = MultinomialLogit()
model.fit(X=df[varnames],
          y=df['depvar'],
          varnames=varnames,
          alts=df['alt'],
          ids=df['idcase'])
model.summary()
```

    Estimation time= 0.0 seconds
    ---------------------------------------------------------------------------
    Coefficient              Estimate      Std.Err.         z-val         P>|z|
    ---------------------------------------------------------------------------
    ic                     -0.0062318     0.0003516   -17.7222802      2.16e-59 ***
    oc                     -0.0045800     0.0003208   -14.2767999      9.14e-41 ***
    ---------------------------------------------------------------------------
    Significance:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

    Log-Likelihood= -1095.237
    AIC= 2194.474
    BIC= 2204.079

Note that these results are identical to the ones estimated by the R mlogit package: https://cran.r-project.org/web/packages/mlogit/vignettes/e1mlogit.html

## References

- Bierlaire, M. (2018). PandasBiogeme: a short introduction. EPFL (Transport and Mobility Laboratory, ENAC).

- Brathwaite, T., & Walker, J. L. (2018). Asymmetric, closed-form, finite-parameter models of multinomial choice. Journal of Choice Modelling, 29, 78–112.

- Cameron, A. C., & Trivedi, P. K. (2005). Microeconometrics: methods and applications. Cambridge university press.

- Croissant, Y. (2020). Estimation of Random Utility Models in R: The mlogit Package. Journal of Statistical Software, 95(1), 1-41.

- Washington, S., Karlaftis, M., Mannering, F., & Anastasopoulos, P. (2020). Statistical and econometric methods for transportation data analysis. Chapman and Hall/CRC.


```python
# zero divisor (영인자)
# AB = 0 A \neq 0, B \neq 0
import sympy as sm
M1 = sm.Matrix([[1,1],[2,2]])
M2 = sm.Matrix([[1,1],[-1,-1]])
M1*M2
```

$\displaystyle \left[\begin{matrix}0 & 0\\0 & 0\end{matrix}\right]$

### 행고정: 행벡터, 열고정: 열벡터
> ### $
\left [
\begin{array}{}
a_{11} & a_{12} & a_{13} & a_{14} & a_{15} \\
a_{21} & a_{22} & a_{23} & a_{24} & a_{25} \\
a_{31} & a_{32} & a_{33} & a_{34} & a_{35} \\
a_{41} & a_{42} & a_{43} & a_{44} & a_{45} \\
a_{51} & a_{52} & a_{53} & a_{54} & a_{55} \\
\end{array}
\right ]
\left [
\begin{array}{}
b_{11} & b_{12} & b_{13} \\
b_{21} & b_{22} & b_{23} \\
b_{31} & b_{32} & b_{33} \\
b_{41} & b_{42} & b_{43} \\
b_{51} & b_{52} & b_{53} \\
\end{array}
\right ]
=
\left [
\begin{array}{}
(AB)_{11} & (AB)_{12} & (AB)_{13} \\
(AB)_{21} & (AB)_{22} & (AB)_{23} \\
(AB)_{31} & (AB)_{32} & (AB)_{33} \\
(AB)_{41} & (AB)_{42} & (AB)_{43} \\
(AB)_{51} & (AB)_{52} & (AB)_{53} \\
\end{array}
\right ]
\\
(AB)_{11} = a_{11} b_{11} + a_{12} b_{21} + a_{13} b_{31} + a_{14} b_{41} + a_{15} b_{51}
= \sum_{j=1}^{5}a_{1j}b_{j1}
\\
(AB)_{21} = a_{21} b_{11} + a_{22} b_{21} + a_{23} b_{31} + a_{24} b_{41} + a_{25} b_{51}
= \sum_{j=1,k=1}^{5}a_{2j}b_{j1}
\\
(AB)_{31} = a_{31} b_{11} + a_{32} b_{21} + a_{33} b_{31} + a_{34} b_{41} + a_{35} b_{51}
= \sum_{j=1,k=1}^{5}a_{3j}b_{j1}
\\
(AB)^{1} = a_{jk}b_{k1} \\
(AB)^{2} = a_{jk}b_{k2} \\
(AB)^{i} = a_{jk}b_{ki} \\
\because A^{j}(\text{ 열벡터:열고정행더미}) = \sum_{k}a_{kj} \quad \because A_{i} = \sum_{k}a_{ik} \\
\therefore (AB)^{i} = b_{ji}A^{j} \\
$

$
AB_{12} = a_{11} b_{12} + a_{12} b_{22} + a_{13} b_{32} + a_{14} b_{42} + a_{15} b_{52}
= \sum_{j=1,k=1}^{5}a_{1j}b_{j2}
\\
AB_{ik} = a_{ij} b_{jk} = \sum_{j=1}^{5}a_{ij}b_{jk}
$

# Matrix
> ## $ \text{such that }A \in \mathbb{R}^{m \times n} \\
f: \mathbb{R}^{n} \mapsto \mathbb{R}^{m} $
> ### sm.Matrix ( [  [ ], [ ], [ ]  ] )
>> ### $T = $ sm.Matrix( [ [1, 2],[3, 4] ] )
>> ### $ T_{i} = T.row(i) \to $ row vector,
>> ### T.row_join(Matrix) = T.col_insert(col_num_to_add, Matrix)
>>> ### T.col.del(num)
>> ### $ T^{i} = T.col(i) \to $ column vector
>> ### T.col_join(Matrix) = T.row_insert(row_num_to_add, Matrix)
>>> ### T.row_del(num)
>> ### v.to_matrix(N_system)
>> ### sm.vector.matrix_to_vector(T_matrix ,N_system)

# Matrix
> ### $
A = \begin{bmatrix}
a_{11} & a_{12} \\
a_{21} & a_{22} \\
\end{bmatrix}
B = \begin{bmatrix}
b_{11} & b_{12} \\
b_{21} & b_{22} \\
\end{bmatrix}
\\
AB = \begin{bmatrix}
a_{11} b_{11} + a_{12} b_{21}, & a_{11}b_{12} + a_{12} b{22} \\
a_{21} b_{11} + a_{22} b_{21}, & a_{21}b_{12} + a_{22} b{22} \\
\end{bmatrix}
= \begin{bmatrix}
b_{11} \begin{bmatrix}a_{11} \\ a_{21}\end{bmatrix}
+ b_{21}\begin{bmatrix}a_{12} \\ a_{22} \end{bmatrix}, &
b_{12} \begin{bmatrix}a_{11} \\ a_{21}\end{bmatrix}
+ b_{22}\begin{bmatrix}a_{12} \\ a_{22} \end{bmatrix}
\end{bmatrix}
\\ (AB) = \sum_{kj}^{2} b_{kj}(\sum_{i}^{2}a_{ik}G_{ij})
\\ (AB)^{j} = \sum_{k} b_{kj}(\sum_{i}a_{ik}G_{ij})
\\ (AB)^{j} = \sum_{k} b_{kj}(A^{k})
\\ \therefore
(AB) =
\begin{bmatrix} b_{11}A^1 + b_{21}A^2 & b_{12}A^1 + b_{22}A^2 \end{bmatrix}
\\
\Rightarrow (AB)^{k} = \sum_{j} b_{jk}A^j = b_{1k}A^1 + b_{2k}A^2+...+ b_{nk}A^n
$

> ### $
= \begin{bmatrix}
a_{11} \begin{bmatrix}b_{11} & b_{12}\end{bmatrix}
+
a_{12} \begin{bmatrix}b_{21} & b_{22} \end{bmatrix}
\\
a_{21} \begin{bmatrix}b_{11} & b_{12}\end{bmatrix}
+
a_{22}\begin{bmatrix}b_{21} & b_{22} \end{bmatrix}
\end{bmatrix}
$
>> ### B의 행에 맞추어  A의 열이 반복된다.
>>> ### $
b_{ij}\big(a_{ki}G_{kj}\big)
\Rightarrow
\sum_{ij}b_{ij}\Big(\sum_{k}a_{ki}G_{kj}\Big)
\\
= (AB)^{k} = B_{jk}(A_{ij}G_{ik})
= (AB)^{k} = B_{jk}A^{i}
$
> ### $
(AB)_{i} = A_{ij}(B_{jk}G_{ik})
(AB)_{i} = A_{ik}B_k
$

# AB
> ### $
AB = \sum_{ij}a_{ij}G_{ij} \sum_{lm}b_{lm}G_{lm} \\
= \sum_{ilm}a_{il}b_{lm}G_{im} \iff \sum_{ijk}a_{ij}b_{jk}G_{ik}
\\
(AB)^{1} = a_11 b_11 + a_12 b_21 + a_1n b_n1 \\  a_21b_21 + a_22 b_22 + a_2n b_2n
(AB)_{1} = a_11 b_11 + a_12 b_21 + a_1n b_n1 \\  a_21b_11
(AB)^{2} = a_11 b_12 + a_12 b_22 + a_1n b_n2 \\  a_21b_12
(AB)^{3} = a_11 b_13 + a_12 b_23 + a_1n b_n2 \\  a_21b_13
(AB)^{k} = a_11 b_1k + a_12 b_2k + a_1n b_nk \\  a_21b_1k
(AB)^{k} = a_ij b_jk + a_ij b_jk + a_1n b_nk \\  a_ijb_jk
$

```python
import sympy as sm
i,j,k,l,m,n = sm.symbols('i j k l m n', positive=True)
A = sm.MatrixSymbol('A',3,3)
B = sm.MatrixSymbol('B',m,n)
C = sm.MatrixSymbol('C',m,m)
D = sm.MatrixSymbol('D',m,m)
sm.Matrix(A)
sm.Matrix(B.subs({m:3,n:2}))
sm.Sum(C[i,j]*D[j,k],(j,0,m-1))
sm.MatrixExpr.from_index_summation(sm.Sum(C[i,j]*D[j,k],(j,0,m-1)))
3*(C + D) == 3*C + 3*D
C.inv()
D.T == D.transpose()
sm.Trace(sm.eye(3,3)).doit()

# [row_start_index : row_end_index), [col_start_index ~ col_end_index)
T = sm.Matrix([[1,2,3],[4,5,6],[7,8,9]])
T[:,:]
M11 = T[1:,1:]
T.adjoint()
T.adjugate()
```

$\displaystyle \left[\begin{matrix}-3 & 6 & -3\\6 & -12 & 6\\-3 & 6 & -3\end{matrix}\right]$

```python
sm.Sum(C[j,i]*D[j,k],(j,0,m-1))
sm.MatrixExpr.from_index_summation(sm.Sum(C[j,i]*D[j,k],(j,0,m-1)))
sm.combinatorics.Permutation(1,2,0).doit()
sm.Matrix([[25,15,-5],[15,18,0],[-5,0,11]]).det()
sm.Matrix([[25,15,-5],[15,18,0],[-5,0,11]]).LDLdecomposition()
```

    (Matrix([
     [   1,   0, 0],
     [ 3/5,   1, 0],
     [-1/5, 1/3, 1]]),
     Matrix([
     [25, 0, 0],
     [ 0, 9, 0],
     [ 0, 0, 9]]))

# dummy index
> ## $$ A_{ik} = \sum_{j\to 1}^{m} a_{ij} \: b_{ji}$$
$$
\begin{bmatrix}
A_{11}  & \dots  & A_{1k} & \dots  & A_{1n} \\
\vdots  & \ddots & \vdots & \ddots & \vdots \\
A_{i1}  & \dots  & A_{ik} & \dots  & A_{in} \\
\vdots  & \ddots & \vdots & \ddots & \vdots \\
A_{l1}  & \dots  & A_{lk} & \dots  & A_{ln} \\
\end{bmatrix}
\:=\:
\begin{bmatrix}
a_{11} & \dots  &  a_{1k} & \dots  &  a_{1m} \\
\vdots & \ddots &  \vdots & \ddots &  \vdots \\
a_{i1} & \dots  &  a_{ik} & \dots  &  a_{im} \\
\vdots & \ddots &  \vdots & \ddots &  \vdots \\
a_{l1} & \dots  &  a_{lk} & \dots  &  a_{lm} \\
\end{bmatrix}
\:
\begin{bmatrix}
b_{11} & \dots  & b_{1k} & \dots  & b_{1n} \\
\vdots & \ddots & \vdots & \ddots & \vdots \\
b_{i1} & \dots  & b_{ik} & \dots  & b_{in} \\
\vdots & \ddots & \vdots & \ddots & \vdots \\
b_{m1} & \dots  & b_{mk} & \dots  & b_{mn} \\
\end{bmatrix}
$$

```python

```

# Diagnolly Dominant Matrices
> ### if for each row,
>> ### the magnitude of the diagonal element
>>> ### is greater than or equl
>>> ### to the sum of the magnitudes of all other elements in that row.
> ## $$
\big|a_{ii}| \geq \sum_{j\neq i} \big|a_{ij}\big|
$$

```python
# diagonally dominant matrices
# Not diagonally dominant ( 1 less than 2
A = sm.Matrix([[1,2],[3,4]])

# Ok diagonally dominant Matrix
B = sm.Matrix([[3,1,1],[0,1,7],[2,1,4]])

a11, a12, a13 = sm.symbols('a_11 a_12 a_13')
a21, a22, a23 = sm.symbols('a_21 a_22 a_23')
a31, a32, a33 = sm.symbols('a_31 a_32 a_33')
x1,x2,x3 = sm.symbols('x1:4')
b1,b2,b3 = sm.symbols('b1:4')
eq1 = sm.Eq(a11*x1 + a12*x2 + a13*x3,b1)
eq2 = sm.Eq(a21*x1 + a22*x2 + a23*x3,b2)
eq3 = sm.Eq(a31*x1 + a32*x2 + a33*x3,b3)
sm.solve(eq1,x1)
```

$\displaystyle a_{11} x_{1} + a_{12} x_{2} + a_{13} x_{3} = b_{1}$

```python
import sympy.vector
import numpy as np
import matplotlib.pyplot as plt
%matplotlib widget

B = sm.vector.CoordSys3D('B')
C = B.create_new('C','cylindrical')
S = B.create_new('S','spherical')
t,q0,q1,q2,q3 = sm.symbols('t q0:4')
Q = B.orient_new_quaternion('Q',q0,q1,q2,q3)

## \vec{x'(t)} = A * \vec{x(t)}
# https://www.youtube.com/watch?v=8wAgRAWwE3M
# x' = \lambda * x
# first order system of linear differencital equations
# \vec{x'(t)} = [ -c2e^t sin(t) +2c3e^t sin(t) + c1e^t cos(t) ...]

A = sm.Matrix([[1,-1,2],[-1,1,0],[-1,0,1]])
# A.eigenvals().values()
r1,r2,r3 = A.eigenvals().keys()
lamb1, mu1 = sm.re(r1), sm.im(r1)
lamb2, mu2 = sm.re(r2), sm.im(r2)
lamb3, mu3 = sm.re(r3), sm.im(r3)

v1, v2, v3 = A.eigenvects()[0][2][0], A.eigenvects()[1][2][0], A.eigenvects()[2][2][0],

x_sup1 = sm.lambdify(t,v1*(sm.exp(lamb1*t) * sm.cos(mu1*t) + 1*sm.I*sm.sin(mu1*t)))
x_sup2 = sm.lambdify(t,v2*(sm.exp(lamb2*t) * sm.cos(mu2*t) + 1*sm.I*sm.sin(mu2*t)))
u = sm.lambdify(t,v2*(sm.exp(lamb2*t) * sm.cos(mu2*t) + 1*sm.I*sm.sin(mu2*t)))
v = sm.lambdify(t,v2*(sm.exp(lamb2*t) * sm.cos(mu2*t) + 1*sm.I*sm.sin(mu2*t)))

c1,c2,c3 = 1,1,1
x = lambda t: c1*x_sup2(t) + c2*u(t) + c3*v(t)
x(0)
```

    array([[0.+3.j],
           [3.+0.j],
           [3.+0.j]])

# Functional
>valuesf(x) \iff f(g(u)) \iff  f(g(h(t)))$
>> $ \begin{cases}
x \iff g(u) \iff {u}^2+ 3{u} + c\\
u \iff a{t}^2 + b{t} + c \\
\end{cases}$

> $f(x,y) \iff  f(g(u,v)) \iff f(g(h(t,s))$
>> $\begin{cases}
x \iff g(u,v) \iff {u}^2+ 3{v} + c\\
u \iff h(t,s) \iff a{t}^2 + b{s} + c\\
\end{cases}\\
\begin{cases}
y \iff g(u,v) \iff {v}^2+ 2{u} + d\\
v \iff h(t,s) \iff a{s}^2 + b{s} + c\\
\end{cases}$

> $f(x,y,z) \iff f(g(u_0,u_1,u_2)) \iff f(g(h(s_0,s_1,s_2)) $
>> $\begin{cases}
x \iff g(u_0,u_1,u_2) \iff {u_2}^2+ 3{u_0} + c\\
u_0 \iff h(s_0,s_1,s_2) \iff a{s_2}^2 + b{s_1} + c\\
\end{cases}\\
\begin{cases}
y \iff g(u_0,u_1,u_2) \iff {u_2}^2+ 2{u_1} + d\\
u_1 \iff h(s_0,s_1,s_2) \iff a{s_0}^2 + b{s_1} + c\\
\end{cases}\\
\begin{cases}
z \iff g(u_0,u_1,u_2) \iff {u_1}^2+ 2{u_2} + d\\
u_2 \iff h(s_0,s_1,s_2) \iff a{s_2}^2 + b{s_1} + c\\
\end{cases}$

> $f(x_0,x_1,x_2,x_3) \iff f(g(u_0,u_1,u_2,u_3)) \iff f(g(h(s_0,s_1,s_2,s_3))$
>> $\begin{cases}
x_0 \iff g(u_0,u_1,u_2,u_3) \iff {u_2}^2+ 3{u_0} + c\\
u_0 \iff h(s_0,s_1,s_2,s_3) \iff a{s_2}^2 + b{s_1} + c\\
\end{cases}\\
\begin{cases}
x_1 \iff g(u_0,u_1,u_2,u_3) \iff {u_2}^2+ 2{u_1} + d\\
u_1 \iff h(s_0,s_1,s_2,s_3) \iff a{s_0}^2 + b{s_1} + c\\
\end{cases}\\
\begin{cases}
x_2 \iff g(u_0,u_1,u_2,u_3) \iff {u_1}^2+ 2{u_2} + d\\
u_2 \iff h(s_0,s_1,s_2,s_3) \iff a{s_2}^2 + b{s_1} + c\\
\end{cases}\\
\begin{cases}
x_3 \iff g(u_0,u_1,u_2,u_3) \iff {u_1}^2+ 2{u_2} + d\\
u_3 \iff h(s_0,s_1,s_2,s_3) \iff a{s_2}^2 + b{s_1} + c\\
\end{cases}$

---
# single vaiable single function
> $ f(x) = x^2 \iff f \big[x \big] =  \big[x^2 \big]$
> $ \frac{df}{dx} = 2x \iff  \big [\frac{df}{dx}\big] =  \big[ 2x \big]$
# multivariable single function
> $ f(x_1, x_2) = 2x_1x_2 + 3x_1 + 2x_2
\iff f(x_1, x_2)
=
\begin{bmatrix} 2x_1x_2 + 3x_1 + 2x_2 \end{bmatrix}$

> $ \begin{bmatrix}
\frac{\partial f}{\partial x_1} \\ \frac{\partial f}{\partial x_2}
\end{bmatrix}
=
\begin{bmatrix}2x_2^2 + 3 \\ 4x_1x_2 + 2
\end{bmatrix}
$

# multivariable mutiple function
> $
f\begin{bmatrix}
x_1 \\
x_2
\end{bmatrix}
=
\begin{bmatrix}
f_1(x_1, x_2) \\
f_2(x_1,x_2)
\end{bmatrix}
=
\begin{bmatrix} 3x_1^2 x_2 + x_2 \\
2x_1x_2^3 - 2x_1
\end{bmatrix}
$

> $
\begin{bmatrix}
\frac{\partial f_1}{\partial x_1} & \frac{\partial f_1}{\partial x_2} \\
\frac{\partial f_2}{\partial x_1} & \frac{\partial f_2}{\partial x_2} \\
\end{bmatrix}
=
\begin{bmatrix}
9x_1^2x_2 & 3x_1 + 1\\
2x_2^3 -2 & 6x_1x_2^2
\end{bmatrix}
$

---
# Jacobian
> ## Notation
>> $J_f(x_1,x_2)$
>>> $ f.jacobian([x_1,x_2])$

```python
x,y, x1,x2 = sm.symbols('x y x1:3')
f = sm.Matrix([[3*x1**2 + x2],
              [2*x1*x2 - 2*x2]])

# J_{f} (x_{1}, x_{2})
J = sm.Matrix([[6*x1, 1],[2*x2, 2*x1 - 2]])
Jd = J.det()
fJ = f.jacobian(sm.Matrix([x1,x2]))
a1, a2 = [sm.diff(i,x1) for i in f]
b1, b2 = [sm.diff(i,x2) for i in f]
fcross = a1 * b2 - a2 * b1
fdot = a1 * b1 + a2 * b2

B = sm.vector.CoordSys3D('')
BJ = sm.Matrix([[6*B.x, 1],[2*B.y, 2*B.x - 2]]).subs({B.x:x,B.y:y})
```

```python
sm.plotting.plot3d(BJ[0])
```

        Figure

    <sympy.plotting.plot.Plot at 0x7f8897730100>

# Jacobian can relate teh joint velocities to catesian velocities in a robot manipulator
> ## notation
>> ### $
{}^{3}J,{}^{2}J, {}^{1}J \\
{}^{\circ}J = {}^0_{3}R {}^{3}J
$
>> ## heading angle v
>> ## $
\vec{v} \text{ 로봇이 향하는 방향의 속도 벡터 }
\begin{cases}
v_x = v cos\theta \\
v_y = v sin\theta
\end{cases}\\
\overset{\cdot}{\vec{v}}
\begin{cases}
\overset{\cdot}{v_x} = \frac{d}{dt}v\: cos(\theta) \\
\overset{\cdot}{v_y} = \frac{d}{dt}v\: sin(\theta)
\end{cases}
$
> ## position of robot
>> ## $X_g$ global coordinate
>>> ## $
x = \begin{bmatrix}x \\ y \\ \theta \end{bmatrix}
=
\begin{bmatrix}v\:cos\theta \\ v\:sin\theta \\ \theta \end{bmatrix}
\Rightarrow
x_g = \begin{bmatrix}x_g \\ y_g \\ \theta_g \end{bmatrix}\\
\overset{\cdot}{x}
=
\begin{bmatrix}
v cos{\theta} \\ vsin(\theta) \\ \omega
\end{bmatrix}
=
\begin{bmatrix}cos(\theta) & 0 \\ sin(\theta) & 0 \\ 0 & 1 \end{bmatrix}
\begin{bmatrix} v \\ \omega \end{bmatrix}
$

# [eigenvalue of the matrix is equal the root of polimonia](https://www.youtube.com/watch?v=VKUU3bZbPVM&list=PLIxff5DJJR7oBEy0Kdg12WWSlS6XFtr6r&index=31): time(11:31)
> ## Frobenius Matrix
>> ### $
\begin{bmatrix}
-a_{n-1} & -a_{n-2} & \cdots & -a_{1} & -a_{0} \\
1 & 0 & \cdots & 0 & 0 \\
\vdots  & \ddots & \ddots & \ddots & \vdots \\
0  & 0 & \cdots & 1 & 0 \\
\end{bmatrix}^{=A}\vec{v} = \lambda \vec{v} \\
\iff
\begin{vmatrix}
\lambda -a_{n-1} & -a_{n-2} & \cdots & -a_{1} & -a_{0} \\
1 & -\lambda & \cdots & 0 & 0 \\
\vdots  & \ddots & \ddots & \ddots & \vdots \\
0  & 0 & \cdots & 1 & -\lambda \\
\end{vmatrix}\\
\iff \text{det}(A - \lambda I) = (-1)^n \Big(\lambda^n + a_{n-1}\lambda^{n-1}+ a_{n-2}\lambda^{n-2}+ ... + a_{2}\lambda^{2} + a_{1}\lambda + a_{0}\Big)
$

---
# inverse 의미
> ### $ A\vec{x} = \vec{b} \iff \big(A^{-1}A \big)\vec{x} = A^{-1}\vec{b} \\
\therefore I\;\vec{x} = A^{-1}\;\vec{b}  \iff \vec{x} = A^{-1}\vec{x}$
> ### eigen value, eigen vector
>> ### $ A\vec{x} = \lambda \vec{x} \iff A\vec{x} - \lambda \vec{x} I = 0 \\
\therefore \Big( A\vec{x} - \lambda I \Big) \vec{x} = 0$

> ### $
\begin{bmatrix}
3 & 1 \\ 1 & 2
\end{bmatrix}
\begin{bmatrix}
x_0 \\  x_2
\end{bmatrix}
 =
\begin{bmatrix}
3  \\  2
\end{bmatrix}
$

```python
#import sympy.physics.vector
#R = sm.physics.vector.ReferenceFrame('R')
#v = R.x + R.y + R.z
#v.to_matrix(R)

import sympy as sm
import sympy.vector
x,y = sm.symbols('x y')
N = sm.vector.CoordSys3D('N')
T = sm.vector.CoordSys3D('T')

# A * \vec{x} = \vec{b}
# A = [3 & 2 \\ 1 & 2] \iff N.i=(3,1), N.j=(2,2)
# x = [x_0 \\  x_1]
# B = [3 \\ 2]
# [3 & 2 // 1 & 2] * [ x_0 \\ x_1 ] = [3 \\ 2 ]

T.i = 3*N.i + N.j
T.j = 2*N.i + 2*N.j
T.k = N.k
h = 3*N.i + 2*N.j
H = b.to_matrix(N)
T.i.to_matrix(N)
A = T.i.to_matrix(N).row_join(T.j.to_matrix(N)).row_join(T.k.to_matrix(N))

## B_i \cdot A^j = \delta_{i}^{j}
B = A.inv()

B*H
# 3*(1/2) + (3/4)  =  2.25
# 2*(1/2) + 2*(3/4) = 2.25
sm.solve([3*N.x + 2*N.y - 3, N.x + 2*N.y-2])
sm.solve([3*x + 2*y - 3, x + 2*y-2])
C = sm.Matrix([x,y,0])

# A^1(3,1) 과 수직이고 A^2(2,2)와 내적은 1인  basis 구하기
## (-1/4, 3/4)
eq1 = 3*x + y
eq2 = 2*x + 2*y - 1
sm.solve([eq1,eq2])
sy = (-1/4)*x + (3/4)*y
# A^2(2,2) 과 수직이고 A^1(3,1)와 내적은 1인  basis 구하기
## (1/2, -1/2)
A.col(1).T*C
eq3 = 2*x + 2*y
eq4 = 3*x + 1*y - 1
sm.solve([eq3,eq4])
sx = 1/2*x - 1/2*y
sm.Matrix([sx.subs({x:3,y:2}), sy.subs({x:3,y:2})]) == B*sm.Matrix([3,2,0])

```

    False

# determinant (행열식) 의미
> ### each basis 들의 성분이 만들어 내는 부피(면적)이다, 즉 기저 단위 부피이다.
# inverse (역행열의 의미)
> ### 변환된 시스템과 원래 시스템간의 단위 부피의 비율이 바로 행열식(determinant)이다 . 즉
> ### 기존 좌표값을 새로운 기저의 단위 부피로 나누어서 기본 단위를 바꾸기 위해서 이다.
> ### 따라서 기존 좌표값을 역행열로 변환 시키면 새로운 기저를 기준으로 하는 좌표값이 나온다.
# $ \text{변환행열의 열벡터(기저벡터)} \iff \text{ 역행열의 행벡터(기저벡터)}$
# 행열 A의 역행열을 구한다는것은
> ### 행열 A의 열벡터 $\{A^1,...,A^n\}$에
> ### 수직인 행벡터 $\{B_1,...,B_m\}$ 을 구한다는 것이다.
> ### 따라서 여러 수직선 그리고 상대 좌표 찾으려는 것이다.
> ### $ B_{i} \cdot A^{j} = \delta_{i}^{j}$

```python
import sympy as sm
import matplotlib.pyplot as plt
import numpy as np
fig = plt.figure(figsize=(8,8))
ax = fig.add_subplot()
ax.set_aspect('equal')
ax.set_xlim(xmin=-2,xmax=4)
ax.set_ylim(ymin=-2,ymax=4)
ax.spines[['left','bottom']].set_position('zero')

# A = [ 3 & 1 \\ 2 & 2 ]
ax.quiver(0,0,3,1,scale=1,units='xy')
ax.quiver(0,0,2,2,scale=1,units='xy')

# b = [ 3 \\ 2 ]
ax.quiver(0,0,3,2,color='r',scale=1,units='xy')

# projection 2 axis of orthogonal two axis A = A^{-1}
######
# A^{-1} = [1/2 & -1/2 \\ -1/4 & 3/4 ] \iff (1/2,-1/2),(-1/4, 3/4) :: 행으로 좌표를 읽음
######
ax.quiver(0,0,1/2,-1/2,color='r',scale=1,units='xy')
ax.quiver(0,0,-1/4,3/4,color='r',scale=1,units='xy')

#### solution ####
# A^{-1}*b = [1/2, 3/4] => A^1 을 1/2등분, A^2 을 3/4 등분이 해가 된다.
# 즉 현재 좌표계로 (3,2) 가 변환 A 를 거치면 변환의 basis인
#  A^1(1열벡터) basis, A^2(2 column vector) basis 기준으로는 어떤 좌표값이 되는가? => 해이다.
ax.plot([-0.3,3],[0.9,2],'magenta')
ax.plot([1/2,3],[-1/2,2],'magenta')

ax.plot(6/4,6/4,'co')
ax.plot(3/2,1/2,'co')
A.inv()*B
(A.inv())*sm.Matrix([3,2,0])
A*sm.Matrix([1/2,3/4,0])
```

$\displaystyle \left[\begin{matrix}3.0\\2.0\\0\end{matrix}\right]$

        Figure

# Ax = b
> ### $
\begin{bmatrix}
3 & 2 \\ 1 &2
\end{bmatrix}
\begin{bmatrix}
x_1 \\ x_2
\end{bmatrix}
=
\begin{bmatrix}
3 \\ 2
\end{bmatrix}
$

```python
import sympy as sm
A = sm.Matrix([[3,2],[1,2]])
A.inv()
```

$\displaystyle \left[\begin{matrix}\frac{1}{2} & - \frac{1}{2}\\- \frac{1}{4} & \frac{3}{4}\end{matrix}\right]$

```python
A.inv()*sm.Matrix([3,2])
```

$\displaystyle \left[\begin{matrix}\frac{1}{2}\\\frac{3}{4}\end{matrix}\right]$

```python

```


# Section 0 问题描述与完成项目流程

## 1. 问题描述

</img>

在该项目中，你将使用强化学习算法，实现一个自动走迷宫机器人。

1. 如上图所示，智能机器人显示在右上角。在我们的迷宫中，有陷阱（红色炸弹）及终点（蓝色的目标点）两种情景。机器人要尽量避开陷阱、尽快到达目的地。
2. 小车可执行的动作包括：向上走 `u`、向右走 `r`、向下走 `d`、向左走 `l`。
3. 执行不同的动作后，根据不同的情况会获得不同的奖励，具体而言，有以下几种情况。
    - 撞到墙壁：-10
    - 走到终点：50
    - 走到陷阱：-30
    - 其余情况：-0.1
4. 我们需要通过修改 `robot.py` 中的代码，来实现一个 Q Learning 机器人，实现上述的目标。

## 2. 完成项目流程

1. 配置环境，使用 `envirnment.yml` 文件配置名为 `robot-env` 的 conda 环境，具体而言，你只需转到当前的目录，在命令行/终端中运行如下代码，稍作等待即可。
```
conda env create -f envirnment.yml
```
安装完毕后，在命令行/终端中运行 `source activate robot-env`（Mac/Linux 系统）或 `activate robot-env`（Windows 系统）激活该环境。

2. 阅读 `main.ipynb` 中的指导完成项目，并根据指导修改对应的代码，生成、观察结果。
3. 导出代码与报告，上传文件，提交审阅并优化。
---

---

# Section 1 算法理解

## 1. 1 强化学习总览

强化学习作为机器学习算法的一种，其模式也是让智能体在“训练”中学到“经验”，以实现给定的任务。但不同于监督学习与非监督学习，在强化学习的框架中，我们更侧重通过智能体与环境的**交互**来学习。通常在监督学习和非监督学习任务中，智能体往往需要通过给定的训练集，辅之以既定的训练目标（如最小化损失函数），通过给定的学习算法来实现这一目标。然而在强化学习中，智能体则是通过其与环境交互得到的奖励进行学习。这个环境可以是虚拟的（如虚拟的迷宫），也可以是真实的（自动驾驶汽车在真实道路上收集数据）。

在强化学习中有五个核心组成部分，它们分别是：**环境（Environment）**、**智能体（Agent）**、**状态（State）**、**动作（Action）**和**奖励（Reward）**。在某一时间节点 $t$：

- 智能体在从环境中感知其所处的状态 $s_t$
- 智能体根据某些准则选择动作 $a_t$
- 环境根据智能体选择的动作，向智能体反馈奖励 $r_{t+1}$

通过合理的学习算法，智能体将在这样的问题设置下，成功学到一个在状态 $s_t$ 选择动作 $a_t$ 的策略 $\pi (s_t) = a_t$。

---

**问题 1**：请参照如上的定义，描述出 “机器人走迷宫这个问题” 中强化学习五个组成部分对应的实际对象：

- **环境** : 迷宫，包括里的墙壁，陷阱，终点，以及正常的通道路径。
- **状态** : 小车在迷宫里所处的坐标位置

- **动作** : 小车可执行的动作，包括向上走、向右走、向下走、向左走
- **奖励** : 小车在执行动作后，收到环境的反馈：撞到墙壁（-10）、走到终点（50）、走到陷阱（-30）、其他情况（-0.1）

$$T(s^{'}, a, s) = P(s^{'}|a,s)$$

---

## 1.2 计算 Q 值

在我们的项目中，我们要实现基于 Q-Learning 的强化学习算法。Q-Learning 是一个值迭代（Value Iteration）算法。与策略迭代（Policy Iteration）算法不同，值迭代算法会计算每个”状态“或是”状态-动作“的值（Value）或是效用（Utility），然后在执行动作的时候，会设法最大化这个值。因此，对每个状态值的准确估计，是我们值迭代算法的核心。通常我们会考虑**最大化动作的长期奖励**，即不仅考虑当前动作带来的奖励，还会考虑动作长远的奖励。

在 Q-Learning 算法中，我们把这个长期奖励记为 Q 值，我们会考虑每个 ”状态-动作“ 的 Q 值，具体而言，它的计算公式为：

$$
q(s_{t},a) = R_{t+1} + \gamma \times\max_a q(a,s_{t+1})
$$

也就是对于当前的“状态-动作” $(s_{t},a)$，我们考虑执行动作 $a$ 后环境给我们的奖励 $R_{t+1}$，以及执行动作 $a$ 到达 $s_{t+1}$后，执行任意动作能够获得的最大的Q值 $\max_a q(a,s_{t+1})$，$\gamma$ 为折扣因子。

不过一般地，我们使用更为保守地更新 Q 表的方法，即引入松弛变量 $alpha$，按如下的公式进行更新，使得 Q 表的迭代变化更为平缓。

$$
q(s_{t},a) = (1-\alpha) \times q(s_{t},a) + \alpha \times(R_{t+1} + \gamma \times\max_a q(a,s_{t+1}))
$$

---
</img>

**问题 2**：根据已知条件求 $q(s_{t},a)$，在如下模板代码中的空格填入对应的数字即可。

已知：如上图，机器人位于 $s_1$，行动为 `u`，行动获得的奖励与题目的默认设置相同。在 $s_2$ 中执行各动作的 Q 值为：`u`: -24，`r`: -13，`d`: -0.29、`l`: +40，$\gamma$ 取0.9。

$$
\begin{align}
q(s_{t},a) & = R_{t+1} + \gamma \times\max_a q(a,s_{t+1}) \\
 & =(-0.1) + (0.9)*(40) \\
 & =(35.9)
\end{align}
$$

---

## 1.3 如何选择动作

在强化学习中，「探索-利用」问题是非常重要的问题。具体来说，根据上面的定义，我们会尽可能地让机器人在每次选择最优的决策，来最大化长期奖励。但是这样做有如下的弊端：
1. 在初步的学习中，我们的 Q 值会不准确，如果在这个时候都按照 Q 值来选择，那么会造成错误。
2. 学习一段时间后，机器人的路线会相对固定，则机器人无法对环境进行有效的探索。

因此我们需要一种办法，来解决如上的问题，增加机器人的探索。由此我们考虑使用 epsilon-greedy 算法，即在小车选择动作的时候，以一部分的概率随机选择动作，以一部分的概率按照最优的 Q 值选择动作。同时，这个选择随机动作的概率应当随着训练的过程逐步减小。

---

**问题 3**：在如下的代码块中，实现 epsilon-greedy 算法的逻辑，并运行测试代码。

```python
import random
import numpy as np

actions = ['u','r','d','l']
qline = {'u':1.2, 'r':-2.1, 'd':-24.5, 'l':27}
epsilon = 0.3 # 以0.3的概率进行随机选择

def choose_action(epsilon):

    action = None
    if random.random() <= epsilon: # 以某一概率
        action = np.random.choice(actions, 1)[0]
    else:
        action = max(qline, key=qline.get)
    return action
```

```python
choose_action(epsilon)
```

    'l'

---

---

# Section 2 代码实现

## 2.1. `Maze` 类理解

我们首先引入了迷宫类 `Maze`，这是一个非常强大的函数，它能够根据你的要求随机创建一个迷宫，或者根据指定的文件，读入一个迷宫地图信息。

1. 使用 `Maze("file_name")` 根据指定文件创建迷宫，或者使用 `Maze(maze_size=(height,width))` 来随机生成一个迷宫。
2. 使用 `trap_number` 参数，在创建迷宫的时候，设定迷宫中陷阱的数量。
3. 直接键入迷宫变量的名字按回车，展示迷宫图像（如 `g=Maze("xx.txt")`，那么直接输入 `g` 即可。
4. 建议生成的迷宫尺寸，长在 6~12 之间，宽在 10～12 之间。

---

**问题 4**：在如下的代码块中，创建你的迷宫并展示。

```python
from Maze import Maze
%matplotlib inline
%config InlineBackend.figure_format = 'retina'

## todo: 创建迷宫并展示
maze = Maze(maze_size = (10, 10), trap_number = 3)
maze
```

---

你可能已经注意到，在迷宫中我们已经默认放置了一个机器人。实际上，我们为迷宫配置了相应的 API，来帮助机器人的移动与感知。其中你随后会使用的两个 API 为 `maze.sense_robot()` 及 `maze.move_robot()`。

1. `maze.sense_robot()` 为一个无参数的函数，输出机器人在迷宫中目前的位置。
2. `maze.move_robot(direction)` 对输入的移动方向，移动机器人，并返回对应动作的奖励值。

---

**问题 5**：随机移动机器人，并记录下获得的奖励，展示出机器人最后的位置。

```python
rewards = []

## 循环、随机移动机器人10次，记录下奖励
for i in range(10):
    maze.move_robot(np.random.choice(actions, 1)[0])

## 输出机器人最后的位置
print(maze.sense_robot())

## 打印迷宫，观察机器人位置
maze
```

## 2.2. `Robot` 类实现

`Robot` 类是我们需要重点实现的部分。在这个类中，我们需要实现诸多功能，以使得我们成功实现一个强化学习智能体。总体来说，之前我们是人为地在环境中移动了机器人，但是现在通过实现 `Robot` 这个类，机器人将会自己移动。通过实现学习函数，`Robot` 类将会学习到如何选择最优的动作，并且更新强化学习中对应的参数。

首先 `Robot` 有多个输入，其中 `alpha=0.5, gamma=0.9, epsilon0=0.5` 表征强化学习相关的各个参数的默认值，这些在之前你已经了解到，`Maze` 应为机器人所在迷宫对象。

随后观察 `Robot.update` 函数，它指明了在每次执行动作时，`Robot` 需要执行的程序。按照这些程序，各个函数的功能也就明了了。

最后你需要实现 `Robot.py` 代码中的8段代码，他们都在代码中以 `#TODO` 进行标注，你能轻松地找到他们。

---

**问题 6**：实现 `Robot.py` 中的8段代码，并运行如下代码检查效果（记得将 `maze` 变量修改为你创建迷宫的变量名）。

```python
from Robot import Robot
robot = Robot(maze) # 记得将 maze 变量修改为你创建迷宫的变量名
robot.set_status(learning=True,testing=False)
print(robot.update())

maze
```

---

## 2.3 用 `Runner` 类训练 Robot

在实现了上述内容之后，我们就可以开始对我们 `Robot` 进行训练并调参了。我们为你准备了又一个非常棒的类 `Runner`，来实现整个训练过程及可视化。使用如下的代码，你可以成功对机器人进行训练。并且你会在当前文件夹中生成一个名为 `filename` 的视频，记录了整个训练的过程。通过观察该视频，你能够发现训练过程中的问题，并且优化你的代码及参数。

---

**问题 7**：尝试利用下列代码训练机器人，并进行调参。可选的参数包括：

- 训练参数
    - 训练次数 `epoch`
- 机器人参数：
    - `epsilon0` (epsilon 初值)
    - `epsilon`衰减（可以是线性、指数衰减，可以调整衰减的速度），你需要在 Robot.py 中调整
    - `alpha`
    - `gamma`
- 迷宫参数:
    - 迷宫大小
    - 迷宫中陷阱的数量

```python
## 可选的参数：
epoch = 20

epsilon0 = 0.4
alpha = 0.6
gamma = 0.9

maze_size = (8,8)
trap_number = 2
```

```python
from Runner import Runner

g = Maze(maze_size=maze_size,trap_number=trap_number)
r = Robot(g,alpha=alpha, epsilon0=epsilon0, gamma=gamma)
r.set_status(learning=True)

runner = Runner(r, g)
runner.run_training(epoch, display_direction=True)
# runner.generate_movie(filename = "final1.mp4") # 你可以注释该行代码，加快运行速度，不过你就无法观察到视频了。
```

---

使用 `runner.plot_results()` 函数，能够打印机器人在训练过程中的一些参数信息。

- Success Times 代表机器人在训练过程中成功的累计次数，这应当是一个累积递增的图像。
- Accumulated Rewards 代表机器人在每次训练 epoch 中，获得的累积奖励的值，这应当是一个逐步递增的图像。
- Running Times per Epoch 代表在每次训练 epoch 中，小车训练的次数（到达终点就会停止该 epoch 转入下次训练），这应当是一个逐步递减的图像。

---

**问题 8**：使用 `runner.plot_results()` 输出训练结果，根据该结果对你的机器人进行分析。

- 指出你选用的参数如何，选用参数的原因。
- 建议你比较不同参数下机器人的训练的情况。
- 训练的结果是否满意，有何改进的计划。

```python
runner.plot_results()
```

```python
epsilon0_2 = 0.1
alpha_2 = 0.7
gamma_2 = 0.9

g_2 = Maze(maze_size=maze_size,trap_number=trap_number)
r_2 = Robot(g_2,alpha=alpha_2, epsilon0=epsilon0_2, gamma=gamma_2)
r_2.set_status(learning=True)

runner_2 = Runner(r_2, g_2)
runner_2.run_training(epoch, display_direction=True)
runner_2.plot_results()
```

```python
#low gamma, 奖励的影响扩散弱化，模型收敛变慢了

epsilon0_3 = 0.4
alpha_3 = 0.6
gamma_3 = 0.1

g_3 = Maze(maze_size=maze_size,trap_number=trap_number)
r_3 = Robot(g_3,alpha=alpha_3, epsilon0=epsilon0_3, gamma=gamma_3)
r_3.set_status(learning=True)

runner_3 = Runner(r_3, g_3)
runner_3.run_training(epoch, display_direction=True)
runner_3.plot_results()
```

```python
#alpha过低，机器人每次都主要受之前的策略知识影响，很少会受到新的知识影响，导致模型难收敛.

epsilon0_4 = 0.4
alpha_4 = 0.1
gamma_4 = 0.9

g_4 = Maze(maze_size=maze_size,trap_number=trap_number)
r_4 = Robot(g_4,alpha=alpha_4, epsilon0=epsilon0_4, gamma=gamma_4)
r_4.set_status(learning=True)

runner_4 = Runner(r_4, g_4)
runner_4.run_training(epoch, display_direction=True)
runner_4.plot_results()
```

(回答区)
1. epsilon0不选择太小，让小车一开始有一定概率进行探索，所以epsilon0初始值选择0.4；而alpha代表Q值的更新有多大比例是受到当前Q值的影响，如果alpha太大，对当前值依赖太大，无法根据学到的新值进行更新reward，因此设置值为0.6；最后，gamma的值，影响的是每次未来奖励的效果，如果gamma太小，会导致最终的终点和陷阱的反馈效果太弱，小车无法快速学习到最优策略。
2. 尝试过不同的epsilon0和alpha以及gamma的值，如果epsilon0选择太小，模型前期的探索不够，小车在前期无法充分利用探索的知识形成优化策略，导致accumulated reward在开始不久一段时间会出现上下波动，而不是稳定逐步上升；而如果alpha选择过小，小车的Q值更新不怎么依赖当前值，过多的依赖小车学习的策略更新值，但在开始时，小车并不能判断出最优策略，会发现累积奖励在前期也会出现上下波动的情况，同时running time也出现很大的波动；最后对于gamma值，如果设置的过小，小车对未来奖励的感知会很小，导致无法学习速度很慢，发现running time上下波动严重，小车并不能在20个epoch内学习到最优策略。
3. 对模型结果满意。

---

**问题 9：** 请将如下的文件打包，提交文件给审阅者。

- `robot.py`
- `robot_maze.ipynb`
- 由 `robot_maze.ipynb` 导出的 `robot_maze.html`


# Week 2: Linear Algebra II

# Tutorial 1

# [insert your name]

**Important reminders**: Before starting, click "File -> Save a copy in Drive". Produce a pdf for submission by "File -> Print" and then choose "Save to PDF".

To complete this tutorial, you should have watched Videos 2.1 through 2.6.

**Credits:**
The videos you watched for this week were from 3Blue1Brown. Some elements of this problem set are from or inspired by https://openedx.seas.gwu.edu/courses/course-v1:GW+EngComp4+2019/about. In particular, we are using their `plot_linear_transformation` and `plot_linear_transformations` functions, and the demonstration of the additional transformation of a matrix inverse (end of Exercise 2)

```python
# Imports
import numpy as np
import matplotlib.pyplot as plt
import matplotlib
import scipy.linalg

# Plotting parameters
matplotlib.rcParams.update({'font.size': 22})
```

```python
# @title Plotting functions
import numpy
from numpy.linalg import inv, eig
from math import ceil
from matplotlib import pyplot, ticker, get_backend, rc
from mpl_toolkits.mplot3d import Axes3D
from itertools import cycle

_int_backends = ['GTK3Agg', 'GTK3Cairo', 'MacOSX', 'nbAgg',
                 'Qt4Agg', 'Qt4Cairo', 'Qt5Agg', 'Qt5Cairo',
                 'TkAgg', 'TkCairo', 'WebAgg', 'WX', 'WXAgg', 'WXCairo']
_backend = get_backend()   # get current backend name

# shrink figsize and fontsize when using %matplotlib notebook
if _backend in _int_backends:
    fontsize = 4
    fig_scale = 0.75
else:
    fontsize = 5
    fig_scale = 1

grey = '#808080'
gold = '#cab18c'   # x-axis grid
lightblue = '#0096d6'  # y-axis grid
green = '#008367'  # x-axis basis vector
red = '#E31937'    # y-axis basis vector
darkblue = '#004065'

pink, yellow, orange, purple, brown = '#ef7b9d', '#fbd349', '#ffa500', '#a35cff', '#731d1d'

quiver_params = {'angles': 'xy',
                 'scale_units': 'xy',
                 'scale': 1,
                 'width': 0.012}

grid_params = {'linewidth': 0.5,
               'alpha': 0.8}

def set_rc(func):
    def wrapper(*args, **kwargs):
        rc('font', family='serif', size=fontsize)
        rc('figure', dpi=200)
        rc('axes', axisbelow=True, titlesize=5)
        rc('lines', linewidth=1)
        func(*args, **kwargs)
    return wrapper

@set_rc
def plot_vector(vectors, tails=None):
    ''' Draw 2d vectors based on the values of the vectors and the position of their tails.

    Parameters
    ----------
    vectors : list.
        List of 2-element array-like structures, each represents a 2d vector.

    tails : list, optional.
        List of 2-element array-like structures, each represents the coordinates of the tail
        of the corresponding vector in vectors. If None (default), all tails are set at the
        origin (0,0). If len(tails) is 1, all tails are set at the same position. Otherwise,
        vectors and tails must have the same length.

    Examples
    --------
    >>> v = [(1, 3), (3, 3), (4, 6)]
    >>> plot_vector(v)      # draw 3 vectors with their tails at origin
    >>> t = [numpy.array((2, 2))]
    >>> plot_vector(v, t)   # draw 3 vectors with their tails at (2,2)
    >>> t = [[3, 2], [-1, -2], [3, 5]]
    >>> plot_vector(v, t)   # draw 3 vectors with 3 different tails

    '''
    vectors = numpy.array(vectors)
    assert vectors.shape[1] == 2, "Each vector should have 2 elements."
    if tails is not None:
        tails = numpy.array(tails)
        assert tails.shape[1] == 2, "Each tail should have 2 elements."
    else:
        tails = numpy.zeros_like(vectors)

    # tile vectors or tails array if needed
    nvectors = vectors.shape[0]
    ntails = tails.shape[0]
    if nvectors == 1 and ntails > 1:
        vectors = numpy.tile(vectors, (ntails, 1))
    elif ntails == 1 and nvectors > 1:
        tails = numpy.tile(tails, (nvectors, 1))
    else:
        assert tails.shape == vectors.shape, "vectors and tail must have a same shape"

    # calculate xlimit & ylimit
    heads = tails + vectors
    limit = numpy.max(numpy.abs(numpy.hstack((tails, heads))))
    limit = numpy.ceil(limit * 1.2)   # add some margins

    figsize = numpy.array([2,2]) * fig_scale
    figure, axis = pyplot.subplots(figsize=figsize)
    axis.quiver(tails[:,0], tails[:,1], vectors[:,0], vectors[:,1], color=darkblue,
                  angles='xy', scale_units='xy', scale=1)
    axis.set_xlim([-limit, limit])
    axis.set_ylim([-limit, limit])
    axis.set_aspect('equal')

    # if xticks and yticks of grid do not match, choose the finer one
    xticks = axis.get_xticks()
    yticks = axis.get_yticks()
    dx = xticks[1] - xticks[0]
    dy = yticks[1] - yticks[0]
    base = max(int(min(dx, dy)), 1)   # grid interval is always an integer
    loc = ticker.MultipleLocator(base=base)
    axis.xaxis.set_major_locator(loc)
    axis.yaxis.set_major_locator(loc)
    axis.grid(True, **grid_params)

    # show x-y axis in the center, hide frames
    axis.spines['left'].set_position('center')
    axis.spines['bottom'].set_position('center')
    axis.spines['right'].set_color('none')
    axis.spines['top'].set_color('none')

@set_rc
def plot_transformation_helper(axis, matrix, *vectors, unit_vector=True, unit_circle=False, title=None):
    """ A helper function to plot the linear transformation defined by a 2x2 matrix.

    Parameters
    ----------
    axis : class matplotlib.axes.Axes.
        The axes to plot on.

    matrix : class numpy.ndarray.
        The 2x2 matrix to visualize.

    *vectors : class numpy.ndarray.
        The vector(s) to plot along with the linear transformation. Each array denotes a vector's
        coordinates before the transformation and must have a shape of (2,). Accept any number of vectors.

    unit_vector : bool, optional.
        Whether to plot unit vectors of the standard basis, default to True.

    unit_circle: bool, optional.
        Whether to plot unit circle, default to False.

    title: str, optional.
        Title of the plot.

    """
    assert matrix.shape == (2,2), "the input matrix must have a shape of (2,2)"
    grid_range = 20
    x = numpy.arange(-grid_range, grid_range+1)
    X_, Y_ = numpy.meshgrid(x,x)
    I = matrix[:,0]
    J = matrix[:,1]
    X = I[0]*X_ + J[0]*Y_
    Y = I[1]*X_ + J[1]*Y_
    origin = numpy.zeros(1)

    # draw grid lines
    for i in range(x.size):
        axis.plot(X[i,:], Y[i,:], c=gold, **grid_params)
        axis.plot(X[:,i], Y[:,i], c=lightblue, **grid_params)

    # draw (transformed) unit vectors
    if unit_vector:
        axis.quiver(origin, origin, [I[0]], [I[1]], color=green, **quiver_params)
        axis.quiver(origin, origin, [J[0]], [J[1]], color=red, **quiver_params)

    # draw optional vectors
    color_cycle = cycle([pink, darkblue, orange, purple, brown])
    if vectors:
        for vector in vectors:
            color = next(color_cycle)
            vector_ = matrix @ vector.reshape(-1,1)
            axis.quiver(origin, origin, [vector_[0]], [vector_[1]], color=color, **quiver_params)

    # draw optional unit circle
    if unit_circle:
        alpha =  numpy.linspace(0, 2*numpy.pi, 41)
        circle = numpy.vstack((numpy.cos(alpha), numpy.sin(alpha)))
        circle_trans = matrix @ circle
        axis.plot(circle_trans[0], circle_trans[1], color=red, lw=0.8)

    # hide frames, set xlimit & ylimit, set title
    limit = 4
    axis.spines['left'].set_position('center')
    axis.spines['bottom'].set_position('center')
    axis.spines['left'].set_linewidth(0.3)
    axis.spines['bottom'].set_linewidth(0.3)
    axis.spines['right'].set_color('none')
    axis.spines['top'].set_color('none')
    axis.set_xlim([-limit, limit])
    axis.set_ylim([-limit, limit])
    if title is not None:
        axis.set_title(title)

@set_rc
def plot_linear_transformation(matrix, *vectors, unit_vector=True, unit_circle=False):
    """ Plot the linear transformation defined by a 2x2 matrix using the helper
    function plot_transformation_helper(). It will create 2 subplots to visualize some
    vectors before and after the transformation.

    Parameters
    ----------
    matrix : class numpy.ndarray.
        The 2x2 matrix to visualize.

    *vectors : class numpy.ndarray.
        The vector(s) to plot along with the linear transformation. Each array denotes a vector's
        coordinates before the transformation and must have a shape of (2,). Accept any number of vectors.

    unit_vector : bool, optional.
        Whether to plot unit vectors of the standard basis, default to True.

    unit_circle: bool, optional.
        Whether to plot unit circle, default to False.

    """
    figsize = numpy.array([4,2]) * fig_scale
    figure, (axis1, axis2) = pyplot.subplots(1, 2, figsize=figsize)
    plot_transformation_helper(axis1, numpy.identity(2), *vectors, unit_vector=unit_vector, unit_circle=unit_circle, title='Before transformation')
    plot_transformation_helper(axis2, matrix, *vectors, unit_vector=unit_vector, unit_circle=unit_circle, title='After transformation')

@set_rc
def plot_linear_transformations(*matrices, unit_vector=True, unit_circle=False):
    """ Plot the linear transformation defined by a sequence of n 2x2 matrices using the helper
    function plot_transformation_helper(). It will create n+1 subplots to visualize some
    vectors before and after each transformation.

    Parameters
    ----------
    *matrices : class numpy.ndarray.
        The 2x2 matrices to visualize. Accept any number of matrices.

    unit_vector : bool, optional.
        Whether to plot unit vectors of the standard basis, default to True.

    unit_circle: bool, optional.
        Whether to plot unit circle, default to False.

    """
    nplots = len(matrices) + 1
    nx = 2
    ny = ceil(nplots/nx)
    figsize = numpy.array([2*nx, 2*ny]) * fig_scale
    figure, axes = pyplot.subplots(nx, ny, figsize=figsize)

    for i in range(nplots):  # fig_idx
        if i == 0:
            matrix_trans = numpy.identity(2)
            title = 'Before transformation'
        else:
            matrix_trans = matrices[i-1] @ matrix_trans
            if i == 1:
                title = 'After {} transformation'.format(i)
            else:
                title = 'After {} transformations'.format(i)
        plot_transformation_helper(axes[i//nx, i%nx], matrix_trans, unit_vector=unit_vector, unit_circle=unit_circle, title=title)
    # hide axes of the extra subplot (only when nplots is an odd number)
    if nx*ny > nplots:
        axes[-1,-1].axis('off')

```

# Key concept review & coding tips

## Linear transformations and matrices

*  A matrix is basically a table of numbers.
      *   We can represent matrices with numpy arrays, which we create as a list of rows: \begin{bmatrix}
4 & 1 &  2\\
3 & 2 & 0\
\end{bmatrix} would be `np.array([[4, 1, 2], [3, 2, 0]])`
*   Linear transformations take in an input vector and outputs a transformed vector. Under a linear transformation, all grid line remain parallel and evenly spaced and the origin remains fixed (it must preserve vector addition and scalar multiplication).
*  Matrices represent linear transformations: each column corresponds to where the corresponding standard basis vector ends up after the transformation
*  We can think of the matrix vector multiplication $A\bar{x}=\bar{b}$ as a linear transformation where $A$ acts on $\bar{x}$ to produce $\bar{b}$.  An alternate view is to think of it as solving a system of linear equations.
 *   `np.linalg.solve` solves matrix vector equations like the above

   *    As an example, solving $A\bar{x}=\bar{b}$ is equivalent to solving the system of linear equations:

   $$ \begin{align} 2x_1 + 3x_2 &= 6 \\ x_1 + 4x_2 &= 1 \end{align}$$

$$\text{if } A = \begin{bmatrix}
2 & 3 \\
1 & 4\
\end{bmatrix}, \bar{b} =\begin{bmatrix}
6 \\
1\
\end{bmatrix}$$

## Matrix multiplication

*   We can envision matrix multiplication as the composition of transformations. If C = AB, element $c_{ij}$  (the element of C in the ith row and jth column) equals the dot product of the ith row of A and the jth column of B.
      *  There are several ways to do matrix multiplication in Python: we can use `np.dot(A, B)`, `np.matmul(A, B)` or use a special operator @ so `A @ B`

## Determinants
*    The determinant of a matrix (det A) is a scalar value that can be viewed as describing the area changes induced by the corresponding linear transformation. It is negative if the linear transformation reverses the orientation of the space.
      *  `np.linalg.det(A)` computes the determinant

## Inverse matrices, column space, and null space

*    We can sometimes take the inverse of a matrix so that $A^{-1}A = I$ where $I$ is the identity matrix (all zeros except for ones on the diagonal).
        *    We can use `np.linalg.inv(A)` to compute $A^{-1}$ when it exists
        * `np.eye(d)` gives us the identity matrix of dimension d
*   The column space of a matrix is the span of the columns of the matrix. This is equivalent to the range of the linear transformation where, in informal language, the range is everywhere that can be "gotten to" by the transformation. In other words, the range is the set of all vectors that the linear transformation maps to.
*  The rank of a matrix is the dimension of the column space.
     *  `np.linalg.matrix_rank(A)` computes the rank
*  The null space of a matrix is the set of all vectors that land on the origin after the resulting transformation. In other words, it is the set of all solutions of $A\bar{x} = \bar{0}$.
    *  You can use `scipy.linalg.null_space` to find a basis for the null space of a matrix.
* If the matrix A is $m$ x $n$, the null space must be a subspace of $R^n$ and the column space must be a subspace of $R^m$.

# Exercise 1: Computation corner

For each computation below, please calculate it 1) by-hand and 2) using code. Check that the answers match! For by-hand calculation, please show some work when possible. For example, for matrix multiplication, write out the computation of each element in the resulting matrix so it looks something like this:

$$A = \begin{bmatrix}
5*2+4*1 &  3*5+1*2  \\
0*1+1*2 & 3*2+4*5 \\
\end{bmatrix} $$

Note that these are completely made up numbers for demonstration purposes - the above numbers don't make sense for a matrix multiplication.

## A) Matrix multiplication

Please compute C = AB where $$A = \begin{bmatrix}
5 &  3  \\
0 & 2 \\
\end{bmatrix}, B = \begin{bmatrix}
1 &  5  \\
4 & 3 \\
\end{bmatrix} $$

**Your math answer**

$$C = \begin{bmatrix}
1 * 5 + 4 * 3 &  5*5 + 3 * 3  \\
1 * 0 + 4 * 2 &  5*0 + 3 * 2 \\
\end{bmatrix}$$

```python
# Your code answer
A = np.array([[5, 3],
            [0, 2]])
B = np.array([[1, 5],
            [4, 3]])

np.matmul(A,B)
```

## B) Matrix multiplication

Please compute Z = XY where $$X = \begin{bmatrix}
3 &  2 & 1 \\
1 & 2 & 7 \\
\end{bmatrix}, Y = \begin{bmatrix}
0 &  1  \\
2 & 4 \\
5 & 1 \\
\end{bmatrix} $$

Before computing, figure out what the dimensions of Z will be (no need to explicitly answer this)

**Your math answer**

The matrix will have row,col of X,Y, so 2 row by 2 col

$$X = \begin{bmatrix}
3 * 0 + 2 * 2 + 1 * 5 & 3 * 1 + 2 * 4 + 1 * 1 \\
1 * 0 + 2 * 2 + 5 * 7 & 1 * 1 + 2 * 4 + 1 * 7\\
\end{bmatrix}$$

```python
A = np.array([[3, 2, 1],
            [1, 2, 7]])
B = np.array([[0, 1],
            [2, 4],
             [5,1]])
np.matmul(A,B)
```

## C) (Optional) Transpose

**Please come back to this problem if you complete the rest of the tutorial during class time.**

The **tranpose** of a matrix flips a matrix over its diagonal, changing the rows to columns and the columns to rows. We denote the transpose of matrix X with $X^T$. In numpy, we can get the transpose of an array X with `X.T`.

First, write out the transpose of X from part B yourself and then produce it using code.

**Your math answer**

It's like computing Z = XY where $$X = \begin{bmatrix}
9 & 12 \\
39 & 16 \\
\end{bmatrix}, Y = \begin{bmatrix}
0 &  1  \\
2 & 4 \\
5 & 1 \\
\end{bmatrix} $$

```python
# Your code answer
np.matmul(A,B).T
```

```python
np.matmul(np.matmul(A,B) , np.array([[0, -1], [1,0]]))
```

You could not compute $X^TY$ - why not?

**Your text answer**

The shapes don't match along dimension 1

# Exercise 2: Thinking about transformations

In the video *Linear transformations and matrices*, you learned that a matrix corresponding to a rotation by 90 degrees is $$X = \begin{bmatrix}
0 &  -1  \\
1 & 0 \\
\end{bmatrix}$$ You also saw that one matrix for which the transformation is horizontal shear is $$X = \begin{bmatrix}
1 &  1  \\
0 & 1 \\
\end{bmatrix}$$

In this exercise, we will think about some other types of transformations. We will use  `plot_linear_transformation(X)` to see the grid before and after the transformation corresponding to matrix $X$.

**Remember to think about where your basis vectors should end up! Then your matrix consists of the transformed basis vectors. Drawing out what you want to happen can help**

## A) Reflection across x2 axis

Come up with a matrix $A$ for which the corresponding linear transformation is reflection through the $x_2$ axis (flipping across the $x_2$ axis). For example, $\bar{x} = \begin{bmatrix}
1 \\
5  \\
\end{bmatrix}$ should become $\bar{b} = \begin{bmatrix}
-1 \\
5  \\
\end{bmatrix}$ when multiplied with $A$.

```python
A = np.array([[1, 0],
              [0, -1]])
plot_linear_transformation(A)
```

Would you expect the determinant of A to be positive or negative? Why? Would you expect the absolute value of the determinant to be greater than 1, less than 1, or equal to 1. Why?

**Your text answer**

## B) Projection onto x1

Come up with a matrix $A$ for which the corresponding linear transformation is projecting onto the $x_1$ axis. For example, $\bar{x} = \begin{bmatrix}
1 \\
5  \\
\end{bmatrix}$ should become $\bar{b} = \begin{bmatrix}
1 \\
0  \\
\end{bmatrix}$ when multiplied with $A$.

```python
A = ...
plot_linear_transformation(A)
```

Would you expect the determinant of A to be positive or negative? Why? Would you expect the absolute value of the determinant to be greater than 1, less than 1, or equal to 1. Why?

**Your text answer**

## C) (Optional) Figuring out the transformation from a matrix

**Please come back to this problem if you complete the rest of the tutorial during class time.**

$$A = \begin{bmatrix}
3 &  1  \\
0 & 3 \\
\end{bmatrix}$$

Try to answer the below questions without looking at the plot of the transformation, but feel free to do so if you get stuck

i) This matrix is a composition of two basic transformations, where possible basic transformations are reflection, contraction, expansion, horizontal shear, vertical shear, and projection.  What are the two basic transformations it is a composition of? (Hint: does this matrix look at all like either of the two in the description below Exercise 2?)

ii) Would you expect the determinant of A to be positive or negative? Why? Would you expect the absolute value of the determinant to be greater than 1, less than 1, or equal to 1. Why?

iii) Rewrite A as a matrix multiplication of two matrices where each matrix corresponds to one of the basic transformations.

**Your text answer**

```python
A = np.array([[3, 1], [0, 3]])
#plot_linear_transformation(A)
```

## Extra info: Matrix inverse transformation

We know that the inverse of a matrix essentially "undoes" the transformation of the matrix. Let's see this in action. We will plot the transformation of A then the additional transformation of $A^{-1}$ - the resulting plot should look like the original.

```python
A = np.array([[1,2], [2,1]])
A_inv = np.linalg.inv(M)
plot_linear_transformations(A, A_inv)
```

# Exercise 3: Encoding model matrices

Let's say we have a population of 3 visual neurons that respond to 3 pixel images. Each neural response is a weighted sum of the pixel image: we used this type of model in Week 1 Part 1 Exercise 2. We will now allow the pixels to have negative values.

We will look at two different populations of 3 neurons with different weights from the pixels: population f and population g. Below, we have the system of linear equations that dictates the neuron models for each population. $x_1$, $x_2$, and $x_3$ correspond to the pixel values. $r_{f1}$, $r_{f2}$, and $r_{f3}$ correspond to the responses of neurons 1, 2, and 3 in population f. $r_{g1}$, $r_{g2}$, and $r_{g3}$ correspond to the responses of neurons 1, 2, and 3 in population g.

Population f:
$$\begin{align}
x_1 + 3x_2 + 4x_3 &= r_{f1} \\
2x_1 + x_2 + 4x_3 &= r_{f2} \\
x_1 + 5x_2 + 6x_3 &= r_{f3} \\
\end{align}$$

Population g:
$$\begin{align}
x_2 + x_3 &= r_{g1} \\
6x_1 + 10x_2 &= r_{g2} \\
3x_1 + 6x_2 + x_3 &= r_{g3} \\
\end{align}$$

## A) Rewriting linear systems of equations to matrix equation

We want to rewrite the above system of equations for each population in the matrix equation $F\bar{x} = \bar{r}_f$ where $\bar{x}$ is the image and $\bar{r}_f$ is the vector of neural responses in population f. What is F?

We will do the same for population g: $G\bar{x} = \bar{r}_g$ where $\bar{r}_g$ is the vector of neural responses in population g. What is G?

**Your math answer**

F is the matrix of the coefficients such that
$$F = \begin{bmatrix}
1 & 3 & 4  \\
2 & 1 & 4 \\
1 & 5 & 6
\end{bmatrix}$$

We started with the linear system of equations view but, as always, we can think about this matrix equation in terms of a linear transformation. In particular matrices F and G are transforming vectors from a "pixel basis", where each element of a vector represents one pixel to a "neural basis" where each element represents the response of one neuron.

## B) Solving a matrix equation

We will now try to solve the matrix equation to find $\bar{x}$ for a given $\bar{r}_f$. What does this correspond to in the neuroscience setting (aka what is $\bar{x}$ here)?

**Your text answer**

As stated above, $\bar{x}$ is the image, so here it would be the afferent weights of retinal ganglion cells

Find $\bar{x}$ if $$\bar{r}_f = \begin{bmatrix}
1 \\
1 \\
2 \\
\end{bmatrix}$$

We will use two different coding methods: you will first use `np.linalg.inv`, and then `np.linalg.solve`.

```python
np.linalg.solve?
```

```python
# Define F
F = np.array([[1, 3, 4],
             [2, 1, 4],
             [1, 5, 6]])

# Define r_f
r_f = np.array([1,1,2])

# Find x using np.linalg.inv
x_using_inv = np.linalg.inv(F)

# Find x using np.linalg.solve
x_using_solve = np.linalg.solve(F, r_f)

# Check each method resulted in the same x
if np.all(np.isclose(x_using_inv, x_using_solve)):
  print('Solutions match')
else:
  print('PROBLEM: Solutions do not match!')

```

    PROBLEM: Solutions do not match!

```python
x_using_solve
```

    array([-2. , -1. ,  1.5])

```python
x_using_inv
```

    array([[ 7. , -1. , -4. ],
           [ 4. , -1. , -2. ],
           [-4.5,  1. ,  2.5]])

## C) Solving another matrix equation

Try to repeat the steps in B for population g where
$$\bar{r}_g = \begin{bmatrix}
1 \\
1 \\
2 \\
\end{bmatrix}$$

What problem do you run into?

```python
# Define G
G = ...

# Define r_g
r_g = ...

# Find x using np.linalg.inv
x_using_inv = ...

# Find x using np.linalg.solve
x_using_solve = ...

# Check each method resulted in the same x
if np.all(np.isclose(x_using_inv, x_using_solve)):
  print('Solutions match')
else:
  print('PROBLEM: Solutions do not match!')

```

## D) Calculate the rank of F/G

First think: from the video *Inverse Matrices, column space, and null space*, we know that if a n x n matrix is invertible, the matrix is not "squishing" space: all of $R^n$ can be reached via the transformation. Based on this, what do you expect the ranks of F and G to be based on parts B/C? (no need to explicitly answer, just discuss)

Now compute the rank of each below and see if you were right.

```python
rank_F = ...
rank_G = ...

print('The rank of F is '+str(rank_F))
print('The rank of G is '+str(rank_G))

```

## E) Linearly independent or dependent columns

Are the columns of F linearly dependent of independent? How do you know? How about the columns of G? How do you know? (Hint: use the words rank and column space in your answer)

**Your text answer**

## F) Finding the null space

Use `scipy.linalg.null_space` to find the basis of the null spaces for F and G.

```python
F_null_space = ...
G_null_space = ...
```

From the above computation, what is the dimension of the null space for F? For G? What does the null space correspond to in our neuroscience setting?

**Your text answer**

## G) Describing the populations of neurons

So what does all this matrix examination tell us about the neural processing in populations f and g? Obviously this is a toy system but let's think about it.

i) What is the dimensionality of the population of neural responses in population f? How about in g?

ii) If we wanted to decode images from the corresponding neural responses, would we always be able to completely recover the image when looking at population f? How about population g? What does this tell us about the information loss of the neural processing?

iii) If the columns of a matrix are linearly dependent, then the rows also are. What does this mean for the neural weights in population g?

**Your text answer**

## Extra info: Invertible matrix theorem

You may have noticed that F and G have a lot of differences in terms of invertibility, the rank, the dimension of the null space, the linear dependence of columns, etc.  There is a theorem that sums up a lot of these concepts: the **invertible matrix theorem**. This theorem essentially sorts square matrices into two types - invertible and not-invertible - and notes a whole bunch of qualities of each type.

Take a look at the below theorem. If you have time and really want to consolidate your knowledge, think through for each statement why they're either all true or all false. A lot of the theorem stems from what you already know as it is based on definitions or basic concepts.  You do not need to memorize this theorem but it is helpful to remember that these two types of matrices exist.

The below is an informal, incomplete encapsulation of the invertible matrix theorem (aka I'm not including every single statement):

### **Invertible matrix theorem**

Let $A$ be a square n x n matrix. The following statements are either all true or all false for this matrix:

a) $A$ is an invertible matrix

b) The equation $A\bar{x} = \bar{b}$ has only the trivial solution (aka only true if $\bar{x} = \bar{0}$).

c) The columns of $A$ form a linearly independent set.

d) The equation $A\bar{x} = \bar{b}$ has at least one solution for each $\bar{b}$ in $R^n$.

e) The columns of A span $R^n$

f) $A^T$ is an invertible matrix

g) The columns of A form a basis of $R^n$

h) Col A (the column space of A) = $R^n$

i) rank A = n

j) Nul A (the null space of A) = {$\bar{0}$}

k) dim Nul A (the dimension of the null space) = 0

l) The determinant of A is not 0


# Tutorial 4 - Pressure vs Temperature

Surfinpy has the functionality to generate phase diagrams as a function of pressure vs temperature based upon the methodology used in Molinari et al. (J. Phys. Chem. C  116, 12, 7073-7082) according to

\begin{align}
\gamma_{adsorbed, T, P} & = \gamma_{bare} + ( C ( E_{ads, T} - RTln(\frac{p}{p^o})
\end{align}

where $\gamma_{adsorbed, T, p}$ is the surface energy of the surface with adsorbed species at temperature (T) and pressure (P), $\gamma_{bare}$ is the suface energy of the bare surface, C is the coverage of adsorbed species, $E_{ads}$ is the adsorption energy,

\begin{align}
E_{ads, T} & =  E_{slab, adsorbant} - (E_{slab, bare} + n_{H_2O} E_{H_2O, T}) / n_{H_2O}
\end{align}

where $E_{slab, adsorbant}$ is the energy of the surface and the adsorbed species, $n_{H_2O}$ is he number of adsorbed species,

\begin{align}
E_{H_2O, (T)} & = E_{H_2O, (g)} - TS_{(T)}
\end{align}

where $S_{(T)}$ is the experimental entropy of gaseous water in the standard state.

So let's look at this in action. The module used is called p_vs_t

```python
import matplotlib.pyplot as plt
from surfinpy import utils as ut
from surfinpy import p_vs_t
from surfinpy import data
```

There are a number of user inputs that are required, the inputs are similiar to mu_vs_mu but with some small differences. First we need the energy from DFT of the adsorbant (eV), the energy of the surface ($j_m^2$) and the data for each surface. For clarity, in the `surfinpy.data.DataSet` objects for each surface it is always assumed that the adsorbing species is 'Y'.

```python
adsorbant = -14.00
```

```python
SE = 1.40
```

```python
stoich = data.DataSet(cation = 24, x = 48, y = 0, area = 60.22, energy = -575.00, label = 'Bare')
H2O =    data.DataSet(cation = 24, x = 48, y = 2, area = 60.22, energy = -605.00, label = '1 Water')
H2O_2 =  data.DataSet(cation = 24, x = 48, y = 8, area = 60.22, energy = -695.00, label = '2 Water')
```

```python
data = [H2O, H2O_2]
```

The coverage of the adsorbing species (in this case water) is also needed. surfinpy has a built in function within the utils module to do this for you, it takes the number of adsorbing species and the surface area and calculates the coverage for you.

```python
cov = ut.calculate_coverage(data)
```

We need the thermochemical data for the adsorbed species in order to make the DFT energy of our adsorbing species a temperature dependent term. This data has been taken directly from the NIST Janaf database and has been cut to just include the temperature and S values within the temperature range that interests us (In this case 0 - 1000 K).

```python
thermochem = ut.read_nist("H2O.txt")
```

Now we can generate our phase diagram .

```python
system = p_vs_t.calculate(stoich, data, SE, adsorbant, thermochem, coverage=cov)
ax = system.plot( colourmap='RdBu', figsize=(6, 4), ylabel='$log_{10} (P_{CO_2})$')
plt.savefig("../../../docs/source/Figures/Surfaces_6.png", dpi=600)
plt.show()
```

```python

```