## Lecture 9 Notebook: Descriptives

In this notebook, we will revisit the replication data from the paper "MPs for Sale? Returns to Office in Postwar British Politics" by Eggers and Hainmuller.

Let's do some more basic descriptive analysis.

Like before, we start by importing some libraries.

```python
# Importing libraries for tables and plots
from datascience import Table
%matplotlib inline
import matplotlib.pyplot as plt
plt.style.use('seaborn-whitegrid')
import numpy as np
from scipy import stats
```

The datascience library we used before is a bit limited in plotting capabilities, so this week we will us the "pandas" library which calls tables dataframes.

```python
# Importing the data into a table called mps
mps = Table.read_table("MPs.csv")
mps = mps.to_df()
mps
```

One way that we can refer to columns in a data frame is with the format dataframe['column']. So, to pull the 'party' variable from the mps dataframe we type mps['party']. Think of this as a "shortcut" for the mps.column('party') syntax we used before.

```python
mps["party"]
```

# Categorical Variables

How should we summarize this variable? For categorical and ordinal variables, a good place to start is with a frequency table. We can get that with the `.value_counts()` function.

```python
mps["party"].value_counts()
```

Pretty straightforward: our data frame has 223 Tory and 204 Labour candidates. Often it is nice to convert this to percentages, which we do by adding 'normalize=TRUE' to the function:

```python
mps["party"].value_counts(normalize=True)
```

Another categorical variable is the region:

```python
mps["region"].value_counts()
```

```python
mps["region"].value_counts(normalize=True)
```

To summarize categorical variables visually, we can use bar charts. The functionality builds nicely on what we did before: we first make a table with the value counts, then add `.plot(kind='bar')` at the end.

```python
mps["party"].value_counts().plot(kind='bar')
```

```python
mps["region"].value_counts().plot(kind='bar')
```

We can also display this with a pie chart:

```python
mps["region"].value_counts().plot(kind='pie')
```

What is a "typical" value for a categorical variable? If there isn't any "order" to the values, the best way to think about a typical value is the most common one, or the *mode*. We can compute this with the `stats.mode()` function.

```python
stats.mode(mps['party'])
```

```python
stats.mode(mps['region'])
```

# Ordinal Variables
There aren't any ordinal variables in the dataset, so here we will make one up. A good ordinal variable for this context is how highly ranked the MP was at the peak of their career. In the UK parliament, a natural ranking is "backbencher" (think regular MP), "cabinet member", and "Prime Minister". And we can actually figure out from the data whether they won or lost using the 'margin' variable.

So, we are going to create a rank variable that is 0 for losers (from the real data), and for winners we will randomly assign them a 1 for backbencher, 2, for cabinet member, and 3 for PM. So, higher rank means "more success".

First, lets create the winner variable

```python
mps['winner'] = 1*(mps['margin'] > 0)
```

```python
mps['winner'].value_counts()
```

In the following line of code we will do our random draw for winners, don't worry about the details here.

```python
mps['rank']=mps['winner'] * np.random.choice([1,2,3], 427, p=[.75, .2, .05])
```

Like with categorical variables, we can summarize these with `.value_counts()`

```python
mps['rank'].value_counts()
```

And a bar chart

```python
mps['rank'].value_counts().plot(kind='bar')
```

Also as with categorical variables, the mode will tell us the most frequent value:

```python
stats.mode(mps['rank'])
```

This tells us the most common outcome is to remain a 0, or a backbencher.

A second way we can define a typical value is the median. One way to think of the median is that we sort into increasing order, and then pick the number in the middle.

To pick some shorter examples, suppose we have five data points: 2, 6, 5, 4, 1. These sort to 1,2,4,5,6. And, hopefully this makes sense visually, the middle is the third point, in the sense that two are smaller than the median and two are larger. So, here the median is 3.

If we have 7 data points the median is the 4th largest, if we have 101 data points it is the 51st largest, etc. In general, if we have an **odd** number of data points n, the median is the (n+1)/2 largest.

Things are a bit tricker if we have an even number of data points. Say we add 100 to our initial list of 5, giving 1,2,4,5,6,100. In a sense the "middle" is now "between 4 and 5". So for this case we define the median as halfway between these two points.

Checking this:

```python
np.median([2,6,5,4,1])
```

```python
np.median([2,6,5,4,1,100])
```

Lets see the median of our rank variable:

```python
np.median(mps['rank'])
```

This tells us that the "typical" person in our data set doesn't even get into parliament!

[Optional: suppose we dropped 200 losers from the data set. What would the median be now?]

Since our data are numbers, we can also compute the mean or average. The mean is the sum of all of the numbers, divided by the number of observations. The formula for this is that for an array of numbers $X_1,X_2,...,X_n$, the mean is:
\begin{align}
    \bar{X} = \frac {\sum_{i=1}^n X_i}{n}
\end{align}

So in our 5 data point example it would be (2 + 6 + 5 + 4  +1)/5. Checking this.

```python
(2 + 6 + 5 + 4 + 1)/5
```

```python
np.mean([2,6,5,4,1])
```

Lets apply it to our rank variable

```python
np.mean(mps['rank'])
```

Does this really mean anything? Arguably not. .45 is not a valid rank, at it doesn't really make sense to describe someone as 45% between a loser and a backbencher.

For some ordinal variables means are informative. For example, if we gave them an ideological score like in Problem set 2, where -2 is extreme left, -1 is moderate left, 0 is centrist, 1 is moderate right, and 2 is extreme right, then the average of those would give us a number between -2 and 2 which would give a sense of the "average" ideology of the candidates.

# Numeric variables
There are a few numeric variables here, include the vote margin one. For numeric variables it usually doesn't make sense to construct a table, because often each value shows up only once. A nice analog to a frequency table for numeric variables is a *histogram*.

```python
mps.hist('margin')
```

... and losers

Histograms break numeric variables into "bins" and then plot the frequency of those bins. Here the bins have a width of about .05. So, the highest bin is telling us that about 115 candidates in the data with a margin of about -.04 to .01. (If you wanted to break these into more natural cutoffs like 0 to .05, that can be done, but let's just take the default).

This gives us a general shape of the data. The margin variable is typically close to zero, with fewer and fewer cases as the margin gets very big or small.

Remember last week we also looked at the net wealth at death, and we had to do a bit of mathematical trickery to determine this from the "natural logarithm of net wealth at death" variable.

```python
mps['net'] = np.exp(mps['ln.net'])
mps.hist('net')
```

Here wealth is being plotted in "tens of millions of pounds" (the 1e7 part). So .2 means 2 million, .4 means 4 million, etc. This is telling us that the vast majority of candidates have a net wealth below 1 million at death.

Now let's compute typical values. Again we can look at the mean, median and mode:

```python
print("Mean: ", np.mean(mps['margin']))
print("Median: ", np.median(mps['margin']))
print("Mode: ", stats.mode(mps['margin']))
```

For the margin variable, the mean and median are vary similar. The mode is way off though: someone who lost by 48%! In general we won't pay attention to the mode of numeric varaiables. This is because, as with many numeric variables, all of the margin amounts are unique. So the frequency of all of them is 1! It appears by default the `stats.mode` function picks the lowest value.

Now let's do net wealth:

```python
print("Mean: ", np.mean(mps['net']))
print("Median: ", np.median(mps['net']))
```

Here we get much different results when looking at the mean vs the mode. This is because the mean is very sensitive to "extreme observations".

Here is a simple way to see that. Remember when we started with a data array of [2,6,5,4,1] and then added 100, the median went up just a bit, from 4 to 4.5. Now let's do the same but for the mean

```python
np.mean([2,6,5,4,1])
```

```python
np.mean([2,6,5,4,1,100])
```

Adding this one new obseration makes the mean go up by 15! In some sense this gives a weird answer for a "typical value", as no value in this data array is near 20.

A similar thing could happen in our real data. Suppose we add a (deceased) Jeff Bezos to our data. His current net worth in pounds is about 135 billion. How would adding him change the mean and median?

```python
print("Mean with Bezos: ", np.mean(np.append(mps['net'], [135000000000])))
print("Median with Bezos: ", np.median(np.append(mps['net'], [135000000000])))
```

So, which is a better measure of a typical value? It depends! The median is less sensitive to outliers, describing a typical member independent of how extreme the non-typical units are. And if we want to answer questions like "how does winning office affect wealth", we don't want our answer to be entirely determined by whether Bezos was a winner or a loser when he ran (particularly if we know that his wealth was independent of his hypothetical run for office).

However, averages are often meaningful too. For example, if we did know that a handful of MPs took advantage of their office to become billionaires, then this would be politically meaningful, even if the typical MP got little out of holding office.

# Measuring Spread
When dealing with numeric variables, we also care a lot about the *spread* of the values. One way to think about this is "how far is a typical value from the average"? A first way to define this is the variance. If we have an array with n numeric observations,  $X_1,X_2,...X_n$ with mean $\bar{X}$, the variance is given by:
\begin{align}
    \frac {\sum_{i=1}^n (X_i - \bar{X})^2} {n-1}
\end{align}

This is close to saying "what is the average squared distance from the mean", though instead of dividing by n we divide by n-1. The reason for this difference is a bit subtle and has no noticeable impact once our $n$ is above 5 or so, as will almost always be the case.

Here is how we compute the variance.

```python
np.var(mps['margin'])
```

```python
np.var(mps['net'])
```

Note that in the margin case we got a very small number, and in the net worth we got an extremely large number. That is because our margin variable is typically around .1, and squaring this give .01. On the other hand, with the wealth variable, we are taking values in the hundred thousands and squaring them. One way to think abotu this problem is that the "unit" of the variance is the "unit squared" of the original variable. In order to get something easier to interpret, we often focus on the *standard deviation* of the variable, which is given by the square root of the variance.

```python
np.std(mps['margin'])
```

```python
np.std(mps['net'])
```

We can also compute variances and standard deviations for ordinal variables, though like with the mean whether or not this has much meaning depends on what we are doing.

```python
np.var(mps['rank'])
```

```python
np.std(mps['rank'])
```

```python

```


# Intro a Computação Quântica e Portas Lógicas Quânticas

Esta palestra é introdutória ao tema de Computação Quântica (originalmente dada para membros do grupo de pesquisa QPower), portanto considera um público com noções consideráveis de matemática, física e programação (você deve saber pelo menos multiplicar matrizes para entender o que vai acontecer aqui). Por essa razão, a abordagem aqui será mais _hands on_, e a linguagem utilizada será o Python.

A discussão não cobrirá o tema de _quantum annealing_, sendo limitada somente a portas lógicas quânticas.
It will just cover gate quantum computation model - there will be no annealing discussed here.

Desse modo, ao cobrir apenas conceitos gerais e portas lógicas quânticas, será mais fácil de estabelecer a distinção e a relação entre a computação clássica e a computação quântica. A palestra utilizará usando Qiskit (pronuncia-se _quískit_), uma biblioteca em _Python_ para computação quântica.

Ainda, para os que não possuem um sólido conhecimento em Python, os auxílios visuais utilizados aqui servirão de suporte para a compreensão dos conceitos abordados.

Meu email: ricardoprins@gmail.com

```
# Instalação do Qiskit
!pip install qiskit
```

    Collecting qiskit
      Downloading qiskit-0.29.1.tar.gz (12 kB)
    Collecting qiskit-terra==0.18.2
      Downloading qiskit_terra-0.18.2-cp37-cp37m-manylinux2010_x86_64.whl (6.1 MB)
         |████████████████████████████████| 6.1 MB 2.0 MB/s
    Collecting qiskit-aer==0.8.2
      Downloading qiskit_aer-0.8.2-cp37-cp37m-manylinux2010_x86_64.whl (18.0 MB)
         |████████████████████████████████| 18.0 MB 130 kB/s
    Collecting qiskit-ibmq-provider==0.16.0
      Downloading qiskit_ibmq_provider-0.16.0-py3-none-any.whl (235 kB)
         |████████████████████████████████| 235 kB 46.1 MB/s
    Collecting qiskit-ignis==0.6.0
      Downloading qiskit_ignis-0.6.0-py3-none-any.whl (207 kB)
         |████████████████████████████████| 207 kB 56.7 MB/s
    Collecting qiskit-aqua==0.9.5
      Downloading qiskit_aqua-0.9.5-py3-none-any.whl (2.1 MB)
         |████████████████████████████████| 2.1 MB 44.3 MB/s
    Requirement already satisfied: scipy>=1.0 in /usr/local/lib/python3.7/dist-packages (from qiskit-aer==0.8.2->qiskit) (1.4.1)
    Collecting pybind11>=2.6
      Downloading pybind11-2.7.1-py2.py3-none-any.whl (200 kB)
         |████████████████████████████████| 200 kB 72.2 MB/s
    Requirement already satisfied: numpy>=1.16.3 in /usr/local/lib/python3.7/dist-packages (from qiskit-aer==0.8.2->qiskit) (1.19.5)
    Collecting docplex>=2.21.207
      Downloading docplex-2.21.207.tar.gz (635 kB)
         |████████████████████████████████| 635 kB 71.7 MB/s
    Requirement already satisfied: h5py<3.3.0 in /usr/local/lib/python3.7/dist-packages (from qiskit-aqua==0.9.5->qiskit) (3.1.0)
    Collecting yfinance>=0.1.62
      Downloading yfinance-0.1.63.tar.gz (26 kB)
    Requirement already satisfied: fastdtw<=0.3.4 in /usr/local/lib/python3.7/dist-packages (from qiskit-aqua==0.9.5->qiskit) (0.3.4)
    Collecting retworkx>=0.8.0
      Downloading retworkx-0.10.2-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (1.4 MB)
         |████████████████████████████████| 1.4 MB 38.5 MB/s
    Requirement already satisfied: psutil>=5 in /usr/local/lib/python3.7/dist-packages (from qiskit-aqua==0.9.5->qiskit) (5.4.8)
    Requirement already satisfied: sympy>=1.3 in /usr/local/lib/python3.7/dist-packages (from qiskit-aqua==0.9.5->qiskit) (1.7.1)
    Collecting quandl
      Downloading Quandl-3.6.1-py2.py3-none-any.whl (26 kB)
    Requirement already satisfied: pandas in /usr/local/lib/python3.7/dist-packages (from qiskit-aqua==0.9.5->qiskit) (1.1.5)
    Collecting dlx<=1.0.4
      Downloading dlx-1.0.4.tar.gz (5.5 kB)
    Requirement already satisfied: scikit-learn>=0.20.0 in /usr/local/lib/python3.7/dist-packages (from qiskit-aqua==0.9.5->qiskit) (0.22.2.post1)
    Requirement already satisfied: setuptools>=40.1.0 in /usr/local/lib/python3.7/dist-packages (from qiskit-aqua==0.9.5->qiskit) (57.4.0)
    Collecting websocket-client>=1.0.1
      Downloading websocket_client-1.2.1-py2.py3-none-any.whl (52 kB)
         |████████████████████████████████| 52 kB 1.8 MB/s
    Requirement already satisfied: requests>=2.19 in /usr/local/lib/python3.7/dist-packages (from qiskit-ibmq-provider==0.16.0->qiskit) (2.23.0)
    Requirement already satisfied: python-dateutil>=2.8.0 in /usr/local/lib/python3.7/dist-packages (from qiskit-ibmq-provider==0.16.0->qiskit) (2.8.2)
    Collecting requests-ntlm>=1.1.0
      Downloading requests_ntlm-1.1.0-py2.py3-none-any.whl (5.7 kB)
    Requirement already satisfied: urllib3>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from qiskit-ibmq-provider==0.16.0->qiskit) (1.24.3)
    Collecting python-constraint>=1.4
      Downloading python-constraint-1.4.0.tar.bz2 (18 kB)
    Collecting fastjsonschema>=2.10
      Downloading fastjsonschema-2.15.1-py3-none-any.whl (21 kB)
    Requirement already satisfied: dill>=0.3 in /usr/local/lib/python3.7/dist-packages (from qiskit-terra==0.18.2->qiskit) (0.3.4)
    Collecting tweedledum<2.0,>=1.1
      Downloading tweedledum-1.1.1-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (943 kB)
         |████████████████████████████████| 943 kB 64.1 MB/s
    Collecting ply>=3.10
      Downloading ply-3.11-py2.py3-none-any.whl (49 kB)
         |████████████████████████████████| 49 kB 6.3 MB/s
    Collecting symengine>0.7
      Downloading symengine-0.8.1-cp37-cp37m-manylinux2010_x86_64.whl (38.2 MB)
         |████████████████████████████████| 38.2 MB 20 kB/s
    Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.7/dist-packages (from qiskit-terra==0.18.2->qiskit) (2.6.0)
    Requirement already satisfied: six in /usr/local/lib/python3.7/dist-packages (from docplex>=2.21.207->qiskit-aqua==0.9.5->qiskit) (1.15.0)
    Requirement already satisfied: cached-property in /usr/local/lib/python3.7/dist-packages (from h5py<3.3.0->qiskit-aqua==0.9.5->qiskit) (1.5.2)
    Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests>=2.19->qiskit-ibmq-provider==0.16.0->qiskit) (3.0.4)
    Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests>=2.19->qiskit-ibmq-provider==0.16.0->qiskit) (2021.5.30)
    Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests>=2.19->qiskit-ibmq-provider==0.16.0->qiskit) (2.10)
    Collecting ntlm-auth>=1.0.2
      Downloading ntlm_auth-1.5.0-py2.py3-none-any.whl (29 kB)
    Collecting cryptography>=1.3
      Downloading cryptography-3.4.8-cp36-abi3-manylinux_2_24_x86_64.whl (3.0 MB)
         |████████████████████████████████| 3.0 MB 47.7 MB/s
    Requirement already satisfied: cffi>=1.12 in /usr/local/lib/python3.7/dist-packages (from cryptography>=1.3->requests-ntlm>=1.1.0->qiskit-ibmq-provider==0.16.0->qiskit) (1.14.6)
    Requirement already satisfied: pycparser in /usr/local/lib/python3.7/dist-packages (from cffi>=1.12->cryptography>=1.3->requests-ntlm>=1.1.0->qiskit-ibmq-provider==0.16.0->qiskit) (2.20)
    Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.7/dist-packages (from scikit-learn>=0.20.0->qiskit-aqua==0.9.5->qiskit) (1.0.1)
    Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.7/dist-packages (from sympy>=1.3->qiskit-aqua==0.9.5->qiskit) (1.2.1)
    Requirement already satisfied: multitasking>=0.0.7 in /usr/local/lib/python3.7/dist-packages (from yfinance>=0.1.62->qiskit-aqua==0.9.5->qiskit) (0.0.9)
    Collecting lxml>=4.5.1
      Downloading lxml-4.6.3-cp37-cp37m-manylinux2014_x86_64.whl (6.3 MB)
         |████████████████████████████████| 6.3 MB 25.2 MB/s
    Requirement already satisfied: pytz>=2017.2 in /usr/local/lib/python3.7/dist-packages (from pandas->qiskit-aqua==0.9.5->qiskit) (2018.9)
    Requirement already satisfied: more-itertools in /usr/local/lib/python3.7/dist-packages (from quandl->qiskit-aqua==0.9.5->qiskit) (8.8.0)
    Collecting inflection>=0.3.1
      Downloading inflection-0.5.1-py2.py3-none-any.whl (9.5 kB)
    Building wheels for collected packages: qiskit, dlx, docplex, python-constraint, yfinance
      Building wheel for qiskit (setup.py) ... done
      Created wheel for qiskit: filename=qiskit-0.29.1-py3-none-any.whl size=11239 sha256=560b5df8dfb9db7b4a4f8cea4f3f7d0ff3796317657794ea9c91cb4007caa70a
      Stored in directory: /root/.cache/pip/wheels/7a/b8/82/921a13d73dac0322d4eea1eceb7a5419ac8f92cc0f54197456
      Building wheel for dlx (setup.py) ... done
      Created wheel for dlx: filename=dlx-1.0.4-py3-none-any.whl size=5719 sha256=94dc04f74b2ab61418e81f123073e727556408c8e6da66a49aa290f09198c786
      Stored in directory: /root/.cache/pip/wheels/78/55/c8/dc61e772445a566b7608a476d151e9dcaf4e092b01b0c4bc3c
      Building wheel for docplex (setup.py) ... done
      Created wheel for docplex: filename=docplex-2.21.207-py3-none-any.whl size=700544 sha256=f42221fcd76fd5186d44ac5a38efc670fdafd65e07b5b31370ef9f73424e4adb
      Stored in directory: /root/.cache/pip/wheels/d8/4e/62/e43a45757e70549e6aa4712ccfcf67440a203c278ecb68de49
      Building wheel for python-constraint (setup.py) ... done
      Created wheel for python-constraint: filename=python_constraint-1.4.0-py2.py3-none-any.whl size=24081 sha256=17eadd2a02707d47c4d742c41d8b219f865371e0df1c63dc19c87177750c7cdf
      Stored in directory: /root/.cache/pip/wheels/07/27/db/1222c80eb1e431f3d2199c12569cb1cac60f562a451fe30479
      Building wheel for yfinance (setup.py) ... done
      Created wheel for yfinance: filename=yfinance-0.1.63-py2.py3-none-any.whl size=23918 sha256=a7617ea60aa76ec188e8e7d9322fe5a2c3784613bebe9961c1ac913c688afff0
      Stored in directory: /root/.cache/pip/wheels/fe/87/8b/7ec24486e001d3926537f5f7801f57a74d181be25b11157983
    Successfully built qiskit dlx docplex python-constraint yfinance
    Installing collected packages: tweedledum, symengine, retworkx, python-constraint, ply, fastjsonschema, qiskit-terra, ntlm-auth, lxml, inflection, cryptography, yfinance, websocket-client, requests-ntlm, quandl, qiskit-ignis, pybind11, docplex, dlx, qiskit-ibmq-provider, qiskit-aqua, qiskit-aer, qiskit
      Attempting uninstall: lxml
        Found existing installation: lxml 4.2.6
        Uninstalling lxml-4.2.6:
          Successfully uninstalled lxml-4.2.6
    Successfully installed cryptography-3.4.8 dlx-1.0.4 docplex-2.21.207 fastjsonschema-2.15.1 inflection-0.5.1 lxml-4.6.3 ntlm-auth-1.5.0 ply-3.11 pybind11-2.7.1 python-constraint-1.4.0 qiskit-0.29.1 qiskit-aer-0.8.2 qiskit-aqua-0.9.5 qiskit-ibmq-provider-0.16.0 qiskit-ignis-0.6.0 qiskit-terra-0.18.2 quandl-3.6.1 requests-ntlm-1.1.0 retworkx-0.10.2 symengine-0.8.1 tweedledum-1.1.1 websocket-client-1.2.1 yfinance-0.1.63

```
# Inicializações iniciais
import numpy as np
import matplotlib.pyplot as plt
from math import sqrt, pi
from qiskit import *
from qiskit.extensions import Initialize
```

## Computação e Informação

Toda a teoria por trás de computadores está relacionada à Teoria da Informação. Ambos os campos caminham juntos, unidos e interrelacionados em seu progresso.

Desse modo, ao falarmos de computação clássica e computação quântica, é preciso começar de um ponto fundamental: ambas tratam de maneiras distintas de *processamento* de informação. Assim, é preciso definir o conceito de _processar_:

$$\text{Entrada } \Longrightarrow \text{ Saída}$$

Quando nós manipulamos qualquer quantidade de informação, e posteriormente a processamos, obtendo informação como resultado desta operação, estamos falando em _processamento_ de informação, ou ainda em _computação_ de informação. Para simplificar, não serão considerados transmissores, receptores e possíveis fontes de ruído: a ideia aqui é somente a de entender o sentido básico de computar ou processar informação em geral.

Essa informação nos ajuda a entender as diferenças fundamentais entre a computação clássica e a computação quântica.

A primeira distinção é na unidade fundamental de informação. Sem perda de generalidade, podemos evitar uma discussão profunda em Teoria de Informação [(clique aqui para saber mais)](http://people.math.harvard.edu/~ctm/home/text/others/shannon/entropy/entropy.pdf), e falar apenas nas unidades fundamentais: _bits_ e _qubits_.

## Bits e Qubits

Um _bit_ é a unidade fundamental de informação da computação clássica. Ela representa uma entidade capaz de armazenar duas posições *estáveis*, como um relé (interruptor). Ligado ou desligado, 0 ou 1, corrente ou ausência de corrente.

Vamos recorrer aqui a uma notação vetorial:

$$1 = \begin{pmatrix} 0\\1 \end{pmatrix}\space e\space\space 0 = \begin{pmatrix} 1\\0 \end{pmatrix} $$

```
zero = np.matrix('1;0')
um = np.matrix('0;1')
```

### Quatro operações básicas com bits

Com _bits_,  há quatro operações básicas possíveis:

$$\text{Identidade:}\space\space {f}(x) = x$$

$$\text{Negação:}\space\space {f}(x) = \neg x$$

$$\text{Constante-0:}\space\space {f}(x) = 0$$

$$\text{Constante-1:}\space\space {f}(x) = 1$$

Usando notação matricial:

```
# Identidade:
identidade = np.matrix('1 0; 0 1')
print('A matriz identidade:')
print(identidade,'\n')

# Zero multiplicado pela matriz identidade retorna zero,
result = identidade @ zero
print(result, '\n')

# e ainda: um, multiplicado pela matriz identidade retorna um, como esperado.
result = identidade @ um
print(result)
```

    A matriz identidade:
    [[1 0]
     [0 1]]

    [[1]
     [0]]

    [[0]
     [1]]

```
# Negação:
neg = np.matrix('0 1; 1 0')
print('A matriz negação:')
print(neg, '\n')

# Zero multiplicado pela matriz negação retorna um,
result = neg @ zero
print(result, '\n')

# e um multiplicado pela matriz negação retorna zero, como esperado.
result = neg @ um
print(result)
```

    A matriz negação:
    [[0 1]
     [1 0]]

    [[0]
     [1]]

    [[1]
     [0]]

```
# Constante-0:
c_zero = np.matrix('1 1; 0 0')
print('A matriz Constante-0')
print(c_zero, '\n')

# Multiplicando zero pela matriz Constante-0 retornará zero,
result = c_zero @ zero
print(result, '\n')

# e multiplicando um pela matriz Constante-0 retornará zero, como esperado.
result = c_zero @ um
print(result)
```

    A matriz Constante-0
    [[1 1]
     [0 0]]

    [[1]
     [0]]

    [[1]
     [0]]

```
# Constante-1:
c_um = np.matrix('0 0; 1 1')
print('A matriz Constante-1:')
print(c_um, '\n')

# Multiplicando zero pela matriz Constante-1 retornará um,
result = c_um @ zero
print(result, '\n')

# e multiplicando um pela matriz Constante-1 retornará um, como esperado.
result = c_um @ um
print(result)
```

    A matriz Constante-1:
    [[0 0]
     [1 1]]

    [[0]
     [1]]

    [[0]
     [1]]

### Operações reversíveis com bits

Para nosso propósito aqui, é conveniente definir _operações reversíveis_ como **quaisquer processos** (cf. a definição supracitada de processamento) **que permitam que a entrada seja identificada ao se conhecer a saída e da operação utilizada.**

Nas operações acima, quais são reversíveis e quais não são?

É possível na computação clássica ocorrer operações não-reversíveis, e isso não é um problema.

### Produto Kronecker

Para a compreensão plena dos próximos conceitos, é necessário explicar uma operação que nem sempre é ensinada nas aulas de Álgebra Linear: o produto Kronecker. [(para maiores explicações, clique aqui)](https://www.grc.nasa.gov/www/k-12/Numbers/Math/documents/Tensors_TM2002211716.pdf).

Também é conveniente estudar sobre produtos tensoriais, para maior compreensão, uma vez que produtos Kronecker são um caso particular de produtos tensoriais.

Esses exemplos são satisfatórios para o fim de explicar a operação:

$$\begin{pmatrix} x_{0} \\ x_{1} \end{pmatrix} \otimes \begin{pmatrix} y_{0} \\ y_{1} \end{pmatrix} = \begin{pmatrix} x_{0}y_{0} \\ x_{0}y_{1} \\ x_{1}y_{0} \\ x_{1}y_{1} \end{pmatrix} $$

Outro exemplo:

$$\begin{pmatrix} x_{0} \\ x_{1} \end{pmatrix} \otimes \begin{pmatrix} y_{0} \\ y_{1} \end{pmatrix} \otimes \begin{pmatrix} z_{0} \\ z_{1} \end{pmatrix} = \begin{pmatrix} x_{0}y_{0}z_{0} \\ x_{0}y_{0}z_{1} \\ x_{0}y_{1}z_{0} \\ x_{0}y_{1}z_{1} \\ x_{1}y_{0}z_{0} \\ x_{1}y_{0}z_{1} \\ x_{1}y_{1}z_{0} \\ x_{1}y_{1}z_{1} \end{pmatrix} $$

### A representação de múltiplos bits

Para representar os estados possíveis de múltiplos bits, utiliza-se o produto Kronecker de vetores de um bit. Novamente é preciso enfatizar que essa é uma explicação simplificada, porém útil o suficiente para a compreensão das operações com múltiplos bits, e suficientemente satisfatória para mostrar o poder da computação quântica.

A representação tensorial de múltiplos bits é chamada de _estado produto_. Essa representação é utilíssima, pois permite a fatoração do _estado produto_ em seus respectivos estados individuais.

Vamos usar como exemplo os estados produto para dois bits:

$$\text{Temos quatro estados possíveis para dois bits:}$$

$$00, 01, 10\space ou\space 11$$:

$$\begin{pmatrix} 1 \\ 0 \end{pmatrix} \otimes \begin{pmatrix} 1 \\ 0 \end{pmatrix} = \begin{pmatrix} 1 \\ 0 \\ 0 \\ 0 \end{pmatrix} \space\space\text{0 com 0,}$$

$$\begin{pmatrix} 1 \\ 0 \end{pmatrix} \otimes \begin{pmatrix} 0 \\ 1 \end{pmatrix} = \begin{pmatrix} 0 \\ 1 \\ 0 \\ 0 \end{pmatrix} \space\space\text{0 com 1,}$$

$$\begin{pmatrix} 0 \\ 1 \end{pmatrix} \otimes \begin{pmatrix} 1 \\ 0 \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \\ 1 \\ 0 \end{pmatrix} \space\space\text{1 com 0,}$$

$$\begin{pmatrix} 0 \\ 1 \end{pmatrix} \otimes \begin{pmatrix} 0 \\ 1 \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \\ 0 \\ 1 \end{pmatrix} \space\space\text{e 1 com 1}$$

```
# usando NumPy:

zero_zero = np.kron(zero, zero)
zero_um = np.kron(zero, um)
um_zero = np.kron(um, zero)
um_um = np.kron(um, um)
print(zero_zero,'\n\n', zero_um,'\n\n', um_zero,'\n\n', um_um)
```

    [[1]
     [0]
     [0]
     [0]]

     [[0]
     [1]
     [0]
     [0]]

     [[0]
     [0]
     [1]
     [0]]

     [[0]
     [0]
     [0]
     [1]]

Assim, como se pode observar, há quatro _estados produto_ possíveis para um par de bits.

Conclui-se, portanto, que o número de _estados produto_ possíveis em _n_ bits é igual a $ 2^{n}$.

### CNOT: Uma operação com múltiplos bits

Um dos blocos fundamentais para a computação reversível é a operação CNOT.

A operação CNOT funciona em pares de _bits_; um é designado como _bit controle_ e o outro como _bit alvo_.

O mecanismo de funcionamento dessa _porta_ (daqui em diante, essas operações serão chamadas de _portas_) é o seguinte:

- Se o bit controle for 1, o bit alvo é invertido
- Se o bit controle for 0, o bit alvo segue inalterado
- O bit controle nunca é alterado
- O bit mais significativo (aquele à esquerda) é o bit controle, e o outro, o bit alvo

Exemplo:

```
cnot = np.matrix('1 0 0 0; 0 1 0 0; 0 0 0 1; 0 0 1 0')
print(cnot)
```

    [[1 0 0 0]
     [0 1 0 0]
     [0 0 0 1]
     [0 0 1 0]]

```
# CNOT on 00:
result = cnot @ zero_zero
print('Como o bit controle é zero, o bit alvo segue sem alteração,\n')
print(result, '\n')

# CNOT on 01
result = cnot @ zero_um
print('o que da mesma maneira acontece aqui.\n')
print(result)
```

    Como o bit controle é zero, o bit alvo segue sem alteração,

    [[1]
     [0]
     [0]
     [0]]

    o que da mesma maneira acontece aqui.

    [[0]
     [1]
     [0]
     [0]]

```
# CNOT on 10:
result = cnot @ um_zero
print('No entanto, ele é invertido neste exemplo,\n')
print(result, '\n')

# CNOT on 11
result = cnot @ um_um
print('tal como no seguinte:\n')
print(result)
```

    No entanto, ele é invertido neste exemplo,

    [[0]
     [0]
     [0]
     [1]]

    tal como no seguinte:

    [[0]
     [0]
     [1]
     [0]]

### _Qubits_ - as unidades fundamentais de informação quântica

O primeiro ponto a ser mencionado aqui é que todas as operações com _qubits_ DEVEM ser reversíveis (e o são).

Disso decorre que elas são seu próprio inverso - ao aplicar a mesma operação duas vezes em sequência, o valor inicial necessariamente será obtido.

Para os fins deste treinamento, a afirmação acima será tratada de forma axiomática, bem como a seguinte: a mecânica quântica é, além de reversível, unitária. A demonstração disso decorre da equação de Schröddinger.

A única exceção aqui é para as operações/portas de medição, uma vez que estas não são, de fato, operações ou portas.

Ainda que todas as operações acima feitas tenham considerado _bits_, todas elas são *igualmente válidas para _qubits_*. É possível afirmar que os _bits_ vistos em sua forma vetorial são casos particulares de vetores _qubits_.

 **Definição:**

 Um qubit pode ser representado por um vetor $\begin{pmatrix} a \\ b \end{pmatrix} : \space a,b \in \mathbb{C}\space\space, \lVert a\rVert^{2} + \lVert b\rVert^{2} = 1$

Exemplos: $\begin{pmatrix} -1 \\ 0 \end{pmatrix}, \begin{pmatrix} \frac12 \\ \frac{\sqrt3}{2} \end{pmatrix}, \begin{pmatrix} \frac{1}{\sqrt2} \\ \frac{1}{\sqrt2} \end{pmatrix}, \begin{pmatrix} \frac{1}{\sqrt2} \\ \frac{-1}{\sqrt2} \end{pmatrix}$

Analogamente aos _bits_, é plausível buscar uma interpretação prática para o comportamento de um _qubit_: ele, tal como seu par da computação clássica, pode representar 0 ou 1 (um par de possibilidades, presença de corrente ou ausência, etc) ou ambos os valores possíveis ao mesmo tempo!

### Superposição quântica - uma explicação matemática

Agora é o momento de encarar as definições físicas mais complicadas relacionadas aos qubits.

Vamos, por um momento, deixar de lado o fato de que um qubit pode assumir ao mesmo tempo os valores 0 e 1, e considerar apenas as questões matemáticas decorrentes dessa característica.

Algo importante também verificável na mecânica quântica é que no momento em que se mede um qubit, ele instantaneamente entrará em colapso para um dos valores, seja 0 ou 1.

Com isso, de maneira igualmente axiomática, afirma-se que a probabilidade que um qubit $\begin{pmatrix} a \\ b  \end{pmatrix}$ tem de _colapsar_ para um valor 0 é  $\lVert a\rVert^{2}$ e para 1, $\lVert b\rVert^{2}$.

Esse axioma facilita a compreensão do comportamento de um sistema com múltiplos qubits.

### Múltiplos qubits

A representação de um sistema com mais de um qubit é análoga à que foi feita para bits clássicos - utiliza-se o produto Kronecker das matrizes de qubits individuais.

```
## Exemplo com NumPy:

# Partindo do qubit (1/sqrt(2), 1/sqrt(2)):
qubit = np.array([[1/sqrt(2)],[1/sqrt(2)]])

# O estado produto desse qubit é o produto Kronecker dele consigo mesmo:
product_state = np.kron(qubit, qubit)
print(product_state)
# Note que o vetor resultante para o estado produto também é unitário.
```

    [[0.5]
     [0.5]
     [0.5]
     [0.5]]

É possível observar no estado produto acima, que o sistema possui uma probabilidade de
$\lVert\frac12\rVert^{2}= \frac14 $ (vamos chamar de amplitude) de colapsar nos estados 00, 01, 10 ou 11 após a medição.

## Portas Quânticas (ou ainda, operações com qubits)

Todas as operações descritas acima são aplicáveis a qubits, com a seguinte ressalva: elas devem ser reversíveis.

### Hadamard

_Desse ponto em diante, iniciaremos o uso da biblioteca Qiskit para visualizar também a notação circuito._

A porta Hadamard (H-gate) leva um qubit 0 ou 1 para um estado de superposição _exactly equal_:

$$ H = \begin{pmatrix} \frac1{\sqrt2} \ \frac1{\sqrt2} \\ \frac1{\sqrt2} \ \frac{-1}{\sqrt2} \end{pmatrix}$$

```
hadamard = np.array([[1/sqrt(2),1/sqrt(2)],[1/sqrt(2),-1/sqrt(2)]])

# Applying Hadamard to qubit 0,
result = hadamard @ zero
print(result, '\n')

# and to qubit 1:
result = hadamard @ um
print(result)
```

    [[0.70710678]
     [0.70710678]]

    [[ 0.70710678]
     [-0.70710678]]

O resultado acima mostra vetores qubit em superposição, com a mesma probabilidade de colapsarem em ambos estados 0 ou 1.

**Pergunta**
O que acontece se aplicarmos a H-gate a um desses qubits resultantes da superposição acima?

### A notação circuito

Uma das notações utilizadas para representar uma sequência de operações com qubits é a notação circuito.

Uma das vantagens da biblioteca Qiskit é a de permitir a visualização das sucessivas operações utilizando essa operação, conforme exemplo abaixo:

```
def challengeTwo(qc, qubit):
  qc.x(qubit)
  qc.h(qubit)
  qc.x(qubit)
  qc.h(qubit)
  qc.x(qubit)
  return qc

quantum_circuit = QuantumCircuit(1)
initial_state = [1,0]
initializer = Initialize(initial_state)
initializer.label = "init"
challengeTwo(quantum_circuit, 0)
quantum_circuit.draw()

```

     ┌───┐┌───┐┌───┐┌───┐┌───┐
q_0: ┤ X ├┤ H ├┤ X ├┤ H ├┤ X ├
     └───┘└───┘└───┘└───┘└───┘

O circuito acima exibe uma sequência de operações realizadas com um qubit em um circuito (duh!), de onde deriva-se o nome _notação circuito_.

Para este caso, partindo de um qubit 0, as operações a seguir são realizadas:

1.   Uma porta negação (chamada de Pauli-X)
2.   Uma porta Hadamard
3.   Outra Pauli-X
4.   Outra Hadamard
5.   Mais uma Pauli-X

**Pergunta** Você consegue determinar o resultado da medição do qubit após essas cinco operações?

### Pauli

Para os que possuem familiaridade com álgebra linear, as matrizes Pauli não serão novidade: um conjunto de três matrizes 2x2 que são hermitianas e unitárias.

Começaremos pela Pauli X - carinhosamente chamada de "porta X"

####Pauli X ou porta NOT

$$ X = \begin{pmatrix} 0 \ 1 \\ 1 \ 0 \end{pmatrix}$$

Essa é análoga à matriz negação que utilizamos para operações com bits. Observemos seu comportamento com qubits:

```
from qiskit.visualization import plot_bloch_multivector

# Vamos desenhar um circuito com um qubit inicializado no estado 0
# e aplicar uma porta X a ele:

qc = QuantumCircuit(1)
qc.x(0)

# A representação abaixo é chamada de Esfera de Bloch, e é usada
# para representar qubits e seus estados. Como podemos ver abaixo,
# esse é o estado em que o qubit "repousa"
# após sua passagem pelo circuito.

def displayBloch():
  backend = Aer.get_backend('statevector_simulator')
  output = execute(qc, backend).result().get_statevector()
  return plot_bloch_multivector(output)

displayBloch()
```

Como esperado, o estado resultante é 1:
uma rotação de $\pi$ radianos sobre o eixo _x_ da esfera.

Você também verá essa porta sendo chamada de porta _NOT_.

#### Pauli Y

Análoga à porta X, esta rotaciona um qubit
$\pi$ radianos sobre o eixo _y_ da esfera:

$$ Y = \begin{pmatrix} 0 \ -i \\ i \ 0 \end{pmatrix}$$

```
# Inicializaremos um qubit no estado 1,
qc = QuantumCircuit(1)
initial_state = [0,1]
qc.initialize(initial_state,0)
qc.draw()
```

     ┌─────────────────┐
q_0: ┤ Initialize(0,1) ├
     └─────────────────┘

```
# essa é sua exibição na esfera de Bloch
displayBloch()
```

```
# e, a rotação resultante da Y-gate:
qc.y(0)
displayBloch()
```

#### Pauli Z

Rotação de $\pi$ radianos sobre o eixo _z_:

$$ Z = \begin{pmatrix} 1 \ 0 \\ 0 \ {-1} \end{pmatrix}$$

```
# Inicializamos um qubit cujo ângulo em relação ao eixo z
# possa ser facilmente observável.
initial_state = [sqrt(3)/2,1/2]
qc.initialize(initial_state,0)
displayBloch()
```

```
# aplicamos a Z-gate sobre esse qubit, verificando a rotação
qc.z(0)
displayBloch()
```

### $R_\phi$ (ou $R_z$)

A porta $R_\phi$ é uma porta parametrizada que executa uma rotação de $\phi$ graus sobre o eixo Z:

_Disso pode-se deduzir que a porta Pauli-Z é um caso particular da porta $R_\phi$ com $\phi=\pi$._

$$ R_\phi = \begin{pmatrix} 1 \ 0 \\ 0 \ e^{i\phi} \end{pmatrix} , com \space\space \phi \in \mathbb R $$

### Identidade

A porta I (Identidade) faz exatamente o que seu nome indica: nada.

No entanto, há algumas aplicações práticas para seu uso, as quais fogem ao escopo desta palestra.

### T

A porta T é outro caso particular da porta $R_\phi$, com $\phi = \frac{\pi}{4}$:

$$ T = \begin{pmatrix} 1 \ 0 \\ 0 \ e^{i\frac{\pi}{4}} \end{pmatrix} \text{e } T\dagger = \begin{pmatrix} 1 \ 0 \\ 0 \ e^{-i\frac{\pi}{4}} \end{pmatrix}$$

### S

A porta S é ainda outro caso particular da porta $R_\phi$, com $\phi = \frac{\pi}{2}$:

$$ S = \begin{pmatrix} 1 \ 0 \\ 0 \ e^{i\frac{\pi}{2}} \end{pmatrix} \text{and } S\dagger = \begin{pmatrix} 1 \ 0 \\ 0 \ e^{-i\frac{\pi}{2}} \end{pmatrix}$$

É conveniente notar que aplicar duas portas S é o mesmo que aplicar uma porta Z, por razões óbvias.

### $U_3$

Todas as portas acima podem ser consideradas como casos particulares da porta $U_3$.

Devido a sua evidente complexidade de exibição, dificilmente ela é utilizada em diagramas.

Uma observação importante: a escolha de rotações nas portas acima sobre o eixo z é apenas uma convenção computacional para facilitar a vida dos programadores e evitar que estes sejam institucionalizados em manicômios.

$$ U_3(\theta,\phi,\lambda) = \begin{pmatrix} \cos(\frac\theta2) \ -e^{i\lambda}\sin(\frac\theta2) \\ e^{i\phi}\sin(\frac\theta2) \ e^{i\phi+i\theta}\cos(\frac\theta2) \end{pmatrix}$$

# OBRIGADO A TODOS!


# Derivative of CTC Loss

关于CTC的介绍已经有很多不错的教程了，但是完整的描述CTCLoss的前向和反向过程的很少，而且有些公式推导省略和错误。本文主要关注CTC Loss的梯度是如何计算的，关于CTC的介绍这里不做过多赘述，具体参看文末参考。

CTC主要应用于语音和OCR中，已语音[Deepspeech2](https://arxiv.org/abs/1512.02595)模型为例，CTC的网络一般如下图所示，包含softmax和CTCLoss两部分。反向传播需要求得loss L相对于logits $u^i$的梯度。下面先介绍CTCLoss的前向计算。

> 图片来源于文末参考

## CTC Loss 的计算

CTC中path的定义与概率的计算如下：

path 是 $ L'^T$的元素，用 $ \pi $表示。 $ \textbf{x} $ 是输入特征，$\textbf{y}$ 是输出label， 都是序列。 $ L $ 是输出的 vocab, L‘ 是 $ L \cup {blank}$。 $y_{\pi_{t}}^t$ 表示在t时刻，$\pi_{t}$ label时的观察概率。其中$\pi_{t}$ 表示 $\pi$ path在t时刻的label。$\pi$ 是 $\textbf{y}$ 与 $ \textbf{x}$ 的一个alignment，长度是$T$，取值空间为$L'$。path也称为alignment。

公式（2）解释了给定输入 $\textbf{x}$ ，输出 $ \pi $  path 的概率，即从时间t=1到T每个时间点的概率 $y_{\pi_{t}}^t$ 相乘。

求出单条path后，就可以计算$p(l \mid x)$ 的概率，计算如下：

这里边 $\mathcal{B}$ 就是映射， 即所有多对一的映射（many-to-one mapping )的集合。 这样就算出来对应一个真正的 label $\textbf{l}$ 的概率了，这里是求和。 求和的原因就是 aab 和 abb 都是对应成ab, 所以 aab 的概率 + abb 的概率才是生成ab的概率。

公式（3）解释了给定输入 $\mathbf{x}$ ，求输出$\mathbf{l}$ 的概率， 即所有集合 $\mathcal{B}^{-1} (\mathbf{l})$ 中 path的概率和。

### CTC forward-backward 算法

CTC的优化采用算最大似然估计[MLE (maximum likelihood estimation)](https://en.wikipedia.org/wiki/Maximum_likelihood_estimation), 这个和神经网络本身的训练过程是一致的。

这个CTC 计算过程类似HMM的 [forward-backward algorithm](https://en.wikipedia.org/wiki/Forward%E2%80%93backward_algorithm)，下面就是这个算法的推导过程：

上图中的定义很清楚， 但是$ \alpha_{t-1}(s) $ and $ \alpha_{t-1}(s-1)$ 和 $\alpha_t(s)$ 的关系也不那么好看出来，下图给出了具体的关于 $\alpha_t(s)$ 的推导过程：

这里的公式比较适合用下面的图来理解，$\alpha_1(1)$ 其实对应的就是下图中左上角白色的圆圈。 就是上来第一个是blank 的概率， 而 $\alpha_1(2)$是label l 的第一个字母。 这里边我们假设每个字母之间都插入了空白，即label l扩展成l'，例如，l=[a, b, b, c]， l'=[-, a, -, b, -, b, -, c, -]。  然后对于其他圆点，在时间是1 的情况下概率都是 0. Figure 3中横轴是时间 t，从左到右是1到T；纵轴是s（sequence），从上到下是 1 到 $\mathbf{\mid l' \mid}$.

接下来我们分析递归公式 (resursion)，更多介绍可以参看 [2]. 公式6分情况考虑:

* 第一种情况就是当前的label是blank， 或者 $\mathbf{l'}_{s}= \mathbf{l'}_{s-2}$(相邻是重复字符)：

  这个时候他的概率来自于过去t-1的两个label 概率， 也就是 $a_{t-1} (s)$ 和 $a_{t-1} (s-1)$ 。

  $ a_{t-1} (s)$ 就是说当前的 sequence 已经是s 了，figure 3中表现为横跳， blank -->blank（例如t=3, s=3）；

  而 $a_{t-1} (s-1) $是说明当前的字符还不够， 需要再加一个， 所以在figure 3中就是斜跳，从黑色圆圈到白色圆圈（例如，t=3, s=5）。

  仔细观察figure 3， 除了第一排的白色圆圈， 其他白色圆圈都有两个输入， 就是上述的两种情况。  当然判断blank 的方法也可以是判断$I'_{s-2} = I'_{s}$. 这种情况也是说明$I'_{s}$ 是blank, 因为每一个字符必须用 blank 隔开， 即使是相同字符。

* 第二章情况 也可以用类似逻辑得出， 只不过当前的状态s 是黑色圆圈， 有三种情况输入。

最终的概率就如公式8 所示， 这个计算过程就是 CTC forward algroirthm， 基于 Fig. 3 的左边的初始条件。

基于Fig. 3 右边的初始条件，我们还是可以计算出一个概率， 那个就是 **CTC backward**. 这里我就不详细介绍了， 直接截图。

这样一直做乘法， 数字值越来越小，很快就会underflow。 这个时候就需要做 scaling.

算出了forward probability 和 backward probability 有什么用呢， 解释如下图。

上图是说 forward probability and backward probability 的乘积， 代表了这个 sequence $\mathbf{l}$ t时刻，是s label 的 所有paths 的概率。 这样的话 我们就计算了 Fig. 3 中的每个圆圈的概率。为什么$\alpha_t(s)\beta_t(s)$ 中多出一个 $y^t_{\mathbf{l'_s}}$ ，这是因为它在 $\alpha$  和 $\beta$ 中都包含该项，合并公式后就多出一项。

$p(\mathbf{l}|\mathbf{x})$  可以通过任意时刻 t 的所有 s 的 foward-backward 概率计算得来。取负对数后就是单个样本的NLL（Negative Log Likelihood）。

### 总结

总结一下，根据前向概率计算CTCLoss函数，可以得出如下结论：

1. 对于时序长度为T的输入序列x和输出序列z，前向概率：
   $$
   \begin{split}
   \alpha_t(s) &= \sum_{ \underset{\pi_t=l'_s}{\pi \in \mathcal{B}^{-1}(z)} } p(\pi_{1:t}|x) \newline
   \alpha_1(1) &= y_{-}^1 ; \quad \alpha_1(2)=y^1_{l'_2}, \quad \alpha_1(s)=0, \forall s > 2  \newline
   \alpha_t(s) &= 0, \quad \forall s < |l'| - 2(T-t) - 1 ,\quad \text{or} \quad \forall s < 1 \newline
   \alpha_t(s) &=
    \begin{cases}
      (\alpha_{t-1}(s) + \alpha_{t-1}(s-1) ) y^t_{l'_s} & \text{if $l'_s=b$ or $l'_{s-2} = l'_s$}  \newline
      (\alpha_{t-1}(s) + \alpha_{t-1}(s-1) + \alpha_{t-1}(s-2))y^t_{l'_s} & \text{otherwise}\newline
    \end{cases}
   \end{split}
   $$

2. 利用 $\alpha_t(s)$ 计算CTCLoss：
   $$
   -ln(p(l \mid x)) = -ln(\alpha_{T}(|l'|)+\alpha_{T}(|l'|-1))
   $$

根据后向概率计算CTCLoss函数，可以得出如下结论：

1. 对于时序长度为T的输入序列x和输出序列z，后向概率：
   $$
   \begin{split}
   \beta_t(s) &= \sum_{ \underset{\pi_t=l'_s}{\pi \in \mathcal{B}^{-1}(z)} } p(\pi_{t:T}|x) \newline
   \beta_T(|l'|) &= y_{-}^T ; \quad \beta_T(|l'|-1)=y^T_{l'_{|l'|-1}}, \quad \beta_T(s)=0, \forall s < |l'| - 1  \newline
   \beta_t(s) &= 0, \text{$\forall s > 2t$ or $\forall s < |l'|$} \newline
   \beta_t(s) &=
        \begin{cases}
          (\beta_{t+1}(s) + \beta_{t+1}(s+1) ) y^t_{l'_s} & \text{if $l'_s=b$ or $l'_{s+2} = l'_s$}  \newline
          (\beta_{t+1}(s) + \beta_{t+1}(s+1) + \beta_{t+1}(s+2))y^t_{l'_s} & \text{otherwise}\newline
        \end{cases}
   \end{split}
   $$

 2. 利用 $\beta_t(s)$计算CTCLoss：

$$
-ln(p(l \mid x)) = -ln(\beta_{1}(1)+\beta_{1}(2)) \newline
$$

根据任意时刻的前向概率和后向概率计算CTC Loss函数，得到如下结论：

1. 对于任意时刻t，利用前向概率和后向概率计算CTCLoss：

$$
p(l \mid x) = \sum_{s=1}^{|l'|} \frac{\alpha_t(s)\beta_t(s)}{y_{l'_s}^t}  \newline
-ln(p(l \mid x)) = -ln( \sum_{s=1}^{|l'|} \frac{\alpha_t(s) \beta_t(s)}{y_{l'_s}^t} )
$$
我们已经得到CTCLoss的计算方法，接下来对其进行求导。

## CTC梯度计算

### 微分公式

在计算梯度前，我们先回顾下基本的微分公式：
$$
C' = 0 \\
x'  = 1 \newline
x^n  =  n \cdot x^{n-1} \newline
(e^x)' =  e^x \newline
log(x)' =  \frac{1}{x} \newline
(u + v)' = u' + v' \newline
(\frac{u}{v})' =  \frac{u'v-uv'}{v^2} \newline
\frac{\mathrm{d}f(g(x))}{\mathrm{d}x} = \frac{\mathrm{d}f(g(x))}{\mathrm{d}g(x)} \cdot \frac{\mathrm{d}g(x)}{\mathrm{d}x}
$$

### CTC梯度

最大似然估计训练就是最大化训练集中每一个分类的对数概率，即最小化Eq. 12。

最后就是算微分了， 整个推导过程就是加法和乘法， 都可以微分。 $\mathit{O}^{ML}$关于神经网络的输出 $y^t_k$的梯度见Eq. 13。因为训练样本是相互独立的，所以可以单独考虑每个样本，公式如Eq.13。

下面是CTCLoss的梯度计算：

### CTC梯度推导

回顾下之前的公式，便于理解后续推导过程。

$$
p(l \mid x) = \sum_{s=1}^{|l'|} \frac{\alpha_t(s)\beta_t(s)}{y_{l'_s}^t}  \\
\begin{equation}
\alpha_t(s) \beta_t(s) = \sum_{ \underset{\pi_t=l'_s}{\pi \in \mathcal{B}^{-1}(l):} }  y^t_{l'_s}  \prod_{t=1}^T y^t_{\pi_t}
\end{equation}
$$

其中Eq. 15的计算过程如下：

$$
\begin{align*}
\frac{\partial p(
l \mid x)}{\partial y_k^t}
    & = \sum_{s \in lab(z,k)} \frac{ \partial \frac{ \alpha_t(s) \beta_t(s)}{y_{k}^t}}{\partial y_k^t}
    \newline
    & = \sum_{s \in lab(z,k)} \frac{(\alpha_t(s)\beta_t(s))’y_k^t - \alpha_t(s)\beta_t(s){y_k^t}'}{{y_k^t}^2}
    \newline
    &= \sum_{s \in lab(z,k)} \frac{(  \prod_{t'=1}^{t-1} y^{t'}_{\pi_{t'}} \cdot y_k^t \cdot y_k^t \cdot   \prod_{t'=t+1}^{T} y^{t'}_{\pi_{t'}}   )’ y_k^t - \alpha_t(s)\beta_t(s){y_k^t}'}{{y_k^t}^2}
    \newline
     &= \sum_{s \in lab(z,k)} \frac{2\alpha_t(s)\beta_t(s) - \alpha_t(s)\beta_t(s)}{{y_k^t}^2}
    \newline
      &= \sum_{s \in lab(z,k)} \frac{\alpha_t(s)\beta_t(s)}{{y_k^t}^2}
      \newline
      &= \frac{1}{{y_k^t}^2}  \sum_{s \in lab(z,k)} \alpha_t(s)\beta_t(s)  \tag{1} \newline
\end{align*}
$$

NLL的公式推导如下：
$$
\begin{split}
\frac{\partial {ln(p(l \mid x))} }{ \partial y^t_k }
  &= \frac{1}{p(l \mid x)} \frac{ \partial{p(l \mid x)} }{ \partial y_k^t } \newline
  &= \frac{1}{p(l \mid x) {y^t_k}^2 } \sum_{s \in lab(z,k)} \alpha_t(s)\beta_t(s)
\end{split}
\tag{2}
$$

已经算出了CTCLoss对于 $y_k^t$ 的梯度，接下来我们需要计算 CTCLoss对于$u^t_k$（logits）的梯度。套用链式法则，并替换$y^t_k$ 为 $y^t_{k'}$，结果如下图。图中 $k'$ 表示vocab中的某一个token，$K$ 是vocab的大小。

图中公式4根据链式法则得到：
$$
- \frac{ \partial ln(p(l \mid x)) }{ \partial u^t_k }
  = - \sum_{k'=1}^{K} \frac{ \partial ln(p(l \mid x)) }{ \partial y^t_{k'} } \frac{ \partial y^t_{k'} }{ \partial u^t_k }  \tag{4}
$$
图中公式3是softmax的梯度，参考 [4]，计算过程如下：
$$
softmax(j) = S_j  = \frac{ e^{a_j} }{ \sum_{k=1}^K e^{a_k} }, \enspace \forall j \in 1 \dots K
$$

$$
\begin{split}
\frac{ \partial S_i }{ \partial a_j}
  &= \frac{ \partial (\frac{ e^{ a_i } }{ \sum_k e^{ a_k } }) } { \partial a_j }
  \newline
  &=
  \begin{cases}
  	\frac{ e^a_i \sum - e^a_j e^a_i }{ \sum^2 }
  	&= \frac{ e^a_i }{ \sum } \frac{ \sum - e^a_j }{ \sum } \newline
    &= S_i(1-S_j)   & \text{i = j, $\sum$ stands for $\sum_{k=1}^K e^a_k$}
  	\newline
  	\frac{ 0 - e^a_j e^a_i }{ \sum^2 }
  	&= - \frac{ e^a_j }{ \sum } \frac{ e^a_i }{ \sum }  \newline
    &= -S_j S_i  & \text{i $\neq$ j, $\sum$ stands for $\sum_{k=1}^K e^a_k$}
  \end{cases}
  \newline
  &=
  \begin{cases}
  S_i(1 - S_j) & \text{$i = j$}
  \newline
  -S_j S_i = S_i (0 - S_j) & \text{$i \neq j$}
  \end{cases}
  \newline
  &= S_i (\delta_{ij} - S_j )
\end{split}
\tag{3}
$$
$$
\delta_{ij} =
 \begin{cases}
 1 & \text{if i = j} \newline
 0 & \text{otherwise}
 \end{cases}
$$

下图中黄色框中的部分表示公式（1），即遍历所有的vocab中的token，其结果是$p(l \mid x)$。这是因为label $l$ 中的token一定在vocab中，且 $s \in lab(l, k')$ 可以是空集。当 $k'$ 在 l 中，s 则为label中token是$k'$的概率；当$k'$不在l中，s为空，概率为0。

公式（2），（3）带入（4），并结合公式（1）的结果如上图右边，即：
$$
\begin{split}
- \frac{ \partial ln(p(l \mid x)) }{ \partial u^t_k } &=
	- \sum_{k'=1}^K \frac{ \partial ln(p(l \mid x)) }{ \partial y^t_{k'} }  \frac{ \partial y^t_{k'}}{ \partial u^t_k } \newline
	&= - \sum_{k'=1}^K \frac{  y^t_{k'}( \delta_{kk'} -  y^t_k )  }{ p(l \mid x) {y^t_{k'}}^2 } \sum_{s \in lab(l, k') } \alpha_t(s) \beta_t(s) \newline
	&= - \sum_{k'=1}^K \frac{  \delta_{kk'} -  y^t_k  }{ p(l \mid x) y^t_{k'} } \sum_{s \in lab(l, k') } \alpha_t(s) \beta_t(s) \newline
	&=  \sum_{k'=1}^K \frac{ y^t_k  - \delta_{kk'} }{ p(l \mid x) y^t_{k'} } \sum_{s \in lab(l, k') } \alpha_t(s) \beta_t(s)   \newline
	&= \sum_{k'=1}^K \frac{ y^t }{ p(l \mid x) y^t_{k'} }  \sum_{s \in lab(l, k') } \alpha_t(s) \beta_t(s) - \sum_{k'=1}^K \frac{ \delta_{kk'} }{ p(l \mid x) y^t_{k'} }  \sum_{s \in lab(l, k') } \alpha_t(s) \beta_t(s) \newline
	&=  \frac{ y^t_k }{ p(l \mid x) } ( \sum_{k'=1}^K \frac{1}{y^t_{k'}} \sum_{s \in lab(l, k') } \alpha_t(s) \beta_t(s) ) - \sum_{k'=1}^K \frac{ \delta_{kk'} }{ p(l \mid x) y^t_{k'} }  \sum_{s \in lab(l, k') } \alpha_t(s) \beta_t(s) \newline
	&=  \frac{ y^t_k }{ p(l \mid x) } p(l \mid x)   - \sum_{k'=1}^K \frac{ \delta_{kk'} }{ p(l \mid x) y^t_{k'} }  \sum_{s \in lab(l, k') } \alpha_t(s) \beta_t(s) \newline
	&= y^t_k - \frac{ 1 }{ p(l \mid x) y^t_k } \sum_{s \in lab(l, k)} \alpha_t(s) \beta_t(s)  \newline
\end{split}
$$
最终，为了通过softmax层传播CTCLoss的梯度，需要计算目标函数与 logits $u^t_k$ 的偏微分，即Eq. 16:
    $$
    \begin{align*}
    \hat{\alpha}_t(s) & \overset{def}{=} \frac{ \alpha_t(s) }{ C_t } ,\enspace C_t \overset{def}{=} \sum_s \alpha_t(s)
    \newline
    \hat{\beta}_t(s) & \overset{def}{=} \frac{ \beta_t(s) }{ D_t } ,\enspace D_t \overset{def}{=} \sum_s \beta_t(s)
    \newline
    - \frac{ \partial ln(p(l \mid x)) }{ \partial u^t_k } &= y^t_k - \frac{1}{y^t_k \sum_{s=1}^{\mid l' \mid} \frac{ \hat{\alpha}_t(s) \hat{\beta}_t(s) }{ y^t_{l'_s} } } \sum_{s \in lab(l, k)} \hat{\alpha}_t(s) \hat{\beta}_t(s) \tag{16}
    \newline
    \end{align*}
    $$

### 总结

* 通过动态规划算法计算$\alpha_t(s)$ 和 $\beta_t(s)$

* 通过$\alpha_t(s)$ 计算 $p(l \mid x)=\alpha_T(\mid l' \mid) + \alpha_T(\mid l' \mid -1)$

* 通过$\alpha_t(s)$ 和 $\beta_t(s)$

* 计算CTcLoss函数的导数:
    $$
    \begin{split}
    - \frac{ \partial ln(p(l \mid x)) }{ \partial u^t_k }
    &= y^t_k - \frac{ 1 }{ p(l \mid x) y^t_k } \sum_{s \in lab(l, k)} \alpha_t(s) \beta_t(s)
    \newline
    &= y^t_k - \frac{1}{y^t_k \sum_{s=1}^{\mid l' \mid} \frac{ \hat{\alpha}_t(s) \hat{\beta}_t(s) }{ y^t_{l'_s} } } \sum_{s \in lab(l, k)} \hat{\alpha}_t(s) \hat{\beta}_t(s)
    \newline
    \end{split}
    \tag{16}
    $$

## Source Code
本人在 [warp-ctc](https://github.com/zh794390558/warp-ctc) 上加了注释，并调整 index 的索引方式，便于理解代码。
对比上面的公式推导和lattice图可以快速理解 ctc 实现。

## Reference

[[1] A. Graves, S. Fernandez, F. Gomez, J. Schmidhuber. Connectionist Temporal lassification: Labeling Unsegmented Sequence Data  with Recurrent Neural Networks. ICML 2006, Pittsburgh, USA,  pp. 369-376.](http://www.cs.toronto.edu/~graves/icml_2006.pdf)

[[2] Sequence ModelingWith CTC](https://distill.pub/2017/ctc/)

[[3] NLP 之 CTC Loss 的工作原理](https://www.jianshu.com/p/e073c9d91b20)

[[4] The Softmax function and its derivative](https://eli.thegreenplace.net/2016/the-softmax-function-and-its-derivative/)

[[5] CTC Algorithm Explained Part 1：Training the Network（CTC算法详解之训练篇）](https://xiaodu.io/ctc-explained/)

```python

```


```python
import numpy as np
import pandas as pd
import respy as rp
from python.auxiliary_setup import load_model_specs
from python.auxiliary_plots import plot_chatter_numagents_sim
from python.auxiliary_plots import plot_chatter_numagents_both
from python.auxiliary_plots import plot_criterion_params
from python.auxiliary_plots import plot_moments_choices
from python.auxiliary_plots import plot_moments_wage
from python.auxiliary_plots import plot_chatter
```

## Method of Simulated Moments Estimation (MSM)

This lecture heavily relies on [respy](https://respy.readthedocs.io/en/latest/index.html)'s method of simulated moments (MSM) interface. A guide to using the interface can be found [here](https://respy.readthedocs.io/en/latest/how_to_guides/msm.html).

Table of Contents

1  Introduction

2  Observed Data

3  Data Moments

4  Weighting Matrix

5  Criterion Function

6  Estimation Exercise

6.1  Estimation for one parameter

6.1.1  Simulated Moments

6.1.2  Optimization Procedure

6.1.2.1  Upper and Lower Bounds

6.1.2.2  Constraints

6.1.2.3  Optimize

6.2  Chatter in the Criterion Function

6.2.1  Changing the Simulation Seed

6.2.2  Multiple Simulation Seeds

6.2.3  Changing the Simulation Seed for increasing Sample Size of Simulated Agents

6.2.4  Changing the Simulation Seed for increasing Sample Size of Simulated and Observed Agents

6.3  Moving away from the Optimum

6.3.1  Retrieving the true Parameter Vector

6.4  Derivative-Based Optimization Algorithm

7  References

### Introduction

The method of simulated moments approach to estimating model parameters is to minimize a certain distance between observed moments and simulated moments with respect to the parameters that generate the simulated model.

Denote $X$ our observed data and $m(X)$ the vector of observed moments. To construct the criterion function, we use the parameter vector $\theta$ to simulate data from the  model $\hat{X}$. We can then calculate the simulated moments $m(\hat{X}| \theta)$.

The criterion function is then given by

\begin{equation}
\psi(\theta) = (m(X) - m(\hat{X}| \theta))'\Omega(m(X) - m(\hat{X}| \theta))
\end{equation}

where the difference between observed and simulated moments $(m(X) - m(\hat{X}| \theta))$ constitutes a vector of the dimension $1\times M$ with $1,..,M$ denoting the number of moments. The $M\times M$ weighting matrix is given by $\Omega$.

The MSM estimator is defined as the solution to

\begin{equation}
\hat{\theta}(\Omega) = \underset{\theta}{\operatorname{argmin}} \psi(\theta).
\end{equation}

The criterion function is thus a strictly positive scalar and the estimator depends on the choice of moments $m$ and the weighting matrix $\Omega$. The weighting matrix applies some scaling for discrepancies between the observed and simulated moments. If we use the identity matrix, each moment is given equal weight and the criterion function reduces to the sum of squared moment deviations.

Aside from the choice of moments and weighting matrix, some other important choices that influence the the estimation are the simulator itself and the algorithm and specifications for the optimization procedure. Many explanations for simulated method of moments estimation also feature the number of simulations as a factor that is to be determined for estimation. We can ignore this factor for now since we are working with a large simulated dataset.

In the following we will set up the data, moments and weighting matrix needed to construct the criterion function and subsequently try to estimating the parameters of the model using this MSM setup.

###  Observed Data

We generate our model and data using [respy](https://respy.readthedocs.io/en/latest/) and a simple Robinson Crusoe model. In this model, the agent, Robinson Crusoe, in each time period decides between two choice options: working (i.e. going fishing) or spending time in the hammock.

We can use respy to simulate the data for this exercise.

```python
params_true, options = load_model_specs()
```

```python
# Generate observed data from model.
simulate = rp.get_simulate_func(params_true, options)
data_obs = simulate(params_true)
```

Let's take a look at the model specifications.

```python
params_true
```

      value

      category
      name

      delta
      delta
      0.950

      wage_fishing
      exp_fishing
      0.070

      nonpec_fishing
      constant
      -0.100

      nonpec_hammock
      constant
      1.046

      shocks_sdcorr
      sd_fishing
      0.010

      sd_hammock
      0.010

      corr_hammock_fishing
      0.000

```python
options
```

    {'estimation_draws': 100,
     'estimation_seed': 100,
     'estimation_tau': 0.001,
     'interpolation_points': -1,
     'n_periods': 5,
     'simulation_agents': 1000,
     'simulation_seed': 132,
     'solution_draws': 100,
     'solution_seed': 456,
     'covariates': {'constant': '1'}}

```python
data_obs.head(10)
```

      Experience_Fishing
      Shock_Reward_Fishing
      Meas_Error_Wage_Fishing
      Shock_Reward_Hammock
      Meas_Error_Wage_Hammock
      Dense_Key
      Core_Index
      Choice
      Wage
      Discount_Rate
      ...
      Nonpecuniary_Reward_Fishing
      Wage_Fishing
      Flow_Utility_Fishing
      Value_Function_Fishing
      Continuation_Value_Fishing
      Nonpecuniary_Reward_Hammock
      Wage_Hammock
      Flow_Utility_Hammock
      Value_Function_Hammock
      Continuation_Value_Hammock

      Identifier
      Period

      0
      0
      0
      0.999650
      1
      0.000410
      1
      0
      0
      fishing
      0.999650
      0.95
      ...
      -0.1
      0.999650
      0.899650
      4.739374
      4.041815
      1.046
      NaN
      1.046410
      4.732282
      3.879866

      1
      1
      1.000743
      1
      0.015065
      1
      1
      1
      fishing
      1.073305
      0.95
      ...
      -0.1
      1.073305
      0.973305
      4.042189
      3.230405
      1.046
      NaN
      1.061065
      3.905840
      2.994500

      2
      2
      0.996461
      1
      0.011853
      1
      2
      2
      fishing
      1.146203
      0.95
      ...
      -0.1
      1.146203
      1.046203
      3.224430
      2.292871
      1.046
      NaN
      1.057853
      3.076295
      2.124675

      3
      3
      0.998907
      1
      -0.007859
      1
      3
      3
      fishing
      1.232329
      0.95
      ...
      -0.1
      1.232329
      1.132329
      2.292998
      1.221756
      1.046
      NaN
      1.038141
      2.113919
      1.132398

      4
      4
      0.989419
      1
      0.012452
      1
      4
      4
      fishing
      1.309130
      0.95
      ...
      -0.1
      1.309130
      1.209130
      1.209130
      0.000000
      1.046
      NaN
      1.058452
      1.058452
      0.000000

      1
      0
      0
      0.992894
      1
      0.014190
      1
      0
      0
      hammock
      NaN
      0.95
      ...
      -0.1
      0.992894
      0.892894
      4.732618
      4.041815
      1.046
      NaN
      1.060190
      4.746062
      3.879866

      1
      0
      0.995369
      1
      -0.003848
      1
      1
      0
      hammock
      NaN
      0.95
      ...
      -0.1
      0.995369
      0.895369
      3.740144
      2.994500
      1.046
      NaN
      1.042152
      3.876343
      2.983359

      2
      0
      1.000668
      1
      -0.008441
      1
      2
      0
      hammock
      NaN
      0.95
      ...
      -0.1
      1.000668
      0.900668
      2.837176
      2.038430
      1.046
      NaN
      1.037559
      2.974068
      2.038430

      3
      0
      0.992542
      1
      -0.013488
      1
      3
      0
      hammock
      NaN
      0.95
      ...
      -0.1
      0.992542
      0.892542
      1.885496
      1.045215
      1.046
      NaN
      1.032512
      2.025466
      1.045215

      4
      0
      0.998695
      1
      -0.005397
      1
      4
      0
      hammock
      NaN
      0.95
      ...
      -0.1
      0.998695
      0.898695
      0.898695
      0.000000
      1.046
      NaN
      1.040603
      1.040603
      0.000000

10 rows × 21 columns

### Data Moments

For the setup of the estimation we first have to choose a set of moments that we will use to match the observed data and the simulated model. For this model we include two sets of moments:

1. The first set are Robinson's **choice frequencies** (choice frequencies here refers to the share of agents that have chosen a specific option) for each period.
2. The second set are moments that characterize the **wage distribution** for each period, i.e. the mean of the wage of all agents that have chosen fishing in a given period and the standard deviation of the wages.

We need a functions that compute the set of moments on the observed and simulated data.

```python
def calc_choice_frequencies(df):
    """Calculate choice frequencies"""
    return df.groupby("Period").Choice.value_counts(normalize=True).unstack()
```

```python
def calc_wage_distribution(df):
    """Calculate wage distribution."""
    return df.groupby(["Period"])["Wage"].describe()[["mean", "std"]]
```

```python
# Save functions in a dictionary to pass to the criterion function later on.
calc_moments = {
    "Choice Frequencies": calc_choice_frequencies,
    "Wage Distribution": calc_wage_distribution,
}
```

Respy's MSM interface additionally requires us to specify, how missings in the computed moments should be handled. Missing moments for instance can arise, when certain choice options aren't picked at all for some periods. We just specify a function that replaces missings with 0.

```python
def replace_nans(df):
    """Replace missing values in data."""
    return df.fillna(0)
```

Now we are ready to calculate the moments.

```python
moments_obs = {
    "Choice Frequencies": replace_nans(calc_moments["Choice Frequencies"](data_obs)),
    "Wage Distribution": replace_nans(calc_moments["Wage Distribution"](data_obs)),
}
```

```python
print("Choice Frequencies")
print(moments_obs["Choice Frequencies"])
print("\n Wage Distribution")
print(moments_obs["Wage Distribution"])
```

    Choice Frequencies
            fishing  hammock
    Period
    0         0.698    0.302
    1         0.698    0.302
    2         0.698    0.302
    3         0.698    0.302
    4         0.698    0.302

     Wage Distribution
                mean       std
    Period
    0       1.003189  0.008640
    1       1.072815  0.010737
    2       1.150106  0.012316
    3       1.234041  0.012406
    4       1.322711  0.013473

### Weighting Matrix

Next we specify a weighting matrix. It needs to be a square matrix with the same number of diagonal elements as there are moments. One option would be to use the identity matrix, but we use a weighting matrix that adjusts for the variance of each moment to improve the estimation process. The variances for the moments are constructed using a bootstrapping procedure.

```python
def get_weighting_matrix(data, calc_moments, num_boots, num_agents_msm):
    """ Compute weighting matrix for estimation with MSM."""
    # Seed for reproducibility.
    np.random.seed(123)

    index_base = data.index.get_level_values("Identifier").unique()

    # Create bootstrapped moments.
    moments_sample = list()
    for _ in range(num_boots):
        ids_boot = np.random.choice(index_base, num_agents_msm, replace=False)
        moments_boot = [calc_moments[key](data.loc[ids_boot, :]) for key in calc_moments.keys()]

        moments_boot = rp.get_flat_moments(moments_boot)

        moments_sample.append(moments_boot)

    # Compute variance for each moment and construct diagonal weighting matrix.
    moments_var = np.array(moments_sample).var(axis=0)
    weighting_matrix = np.diag(moments_var ** (-1))

    return np.nan_to_num(weighting_matrix)
```

```python
W = get_weighting_matrix(data_obs, calc_moments, 300, 500)
```

```python
pd.DataFrame(W)
```

      0
      1
      2
      3
      4
      5
      6
      7
      8
      9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19

      0
      4795.115376
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00

      1
      0.000000
      4795.115376
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00

      2
      0.000000
      0.000000
      4795.115376
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00

      3
      0.000000
      0.000000
      0.000000
      4795.115376
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00

      4
      0.000000
      0.000000
      0.000000
      0.000000
      4795.115376
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00

      5
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      4795.115376
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00

      6
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      4795.115376
      0.000000
      0.000000
      0.000000
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00

      7
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      4795.115376
      0.000000
      0.000000
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00

      8
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      4795.115376
      0.000000
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00

      9
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      4795.115376
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00

      10
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      9.466832e+06
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00

      11
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000e+00
      6.036166e+06
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00

      12
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000e+00
      0.000000e+00
      4.698312e+06
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00

      13
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000e+00
      0.000000e+00
      0.000000e+00
      4.248616e+06
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00

      14
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      3.566318e+06
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00

      15
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      1.779242e+07
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00

      16
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      1.258966e+07
      0.000000e+00
      0.000000e+00
      0.000000e+00

      17
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      9.317930e+06
      0.000000e+00
      0.000000e+00

      18
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      8.904094e+06
      0.000000e+00

      19
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      0.000000e+00
      1.067252e+07

### Criterion Function

We have collected the observed data for our model, chosen the set of moments we want to use for estimation and defined a weighting matrix based on these moments. We can now set up the criterion function to use for estimation.

As already discussed above, the criterion function is given by the weighted square product of the difference between observed moments $m(X)$ and simulated moments $m(\hat{X}| \theta)$. Trivially, if we have that $m(X) = m(\hat{X}| \theta)$, the criterion function returns a value of 0. Thus, the closer $\theta$ is to the real parameter vector, the smaller should be the value for the criterion function.

```python
criterion_msm = rp.get_moment_errors_func(
    params_true, options, calc_moments, replace_nans, moments_obs, W
)
```

Criterion function at the true parameter vector:

```python
fval = criterion_msm(params_true)
fval
```

    0.0

We can plot the criterion function to examine its behavior around the minimum in more detail. The plots below show the criterion function at varying values of all parameters in the the paramter vector.

```python
plot_criterion_params(params_true, list(params_true.index), criterion_msm, 0.05)
```

This depiction for most parameters conceals the fact that the criterion function is not a smooth function of our parameter values. We can reveal this property if we 'zoom in' on the function far enough. The plots below show the criterion function for varying values of *delta* around the true minimum value of 0.95. We can see that the function exhibits small plateaus and is thus not completely smooth.

```python
for radius in [0.05, 0.01, 0.001, 0.0001]:
    plot_criterion_params(params_true, list(params_true.index)[0:1], criterion_msm, radius)
```

###  Estimation Exercise

In the following we will conduct a simulation exercise to estimate the parameter vector using our criterion function and weighting matrix. We will begin by simulating data using the new parameter vector and examine how the simulated moments differ from the observed ones. We will then use an optimizer to minimize the criterion function in order to retrieve the true parameter vector. Additionally, we will explore how the criterion function behaves if we change the simulation seed, move away from the true values and test the optimization for a derivative-based optimization algorithm.

#### Estimation for one parameter

For now, our candidate parameter vector will just differ in *delta* from the true parameters.

```python
params_cand = params_true.copy()
params_cand.loc["delta", "value"] = 0.93
```

##### Simulated Moments

We can now use our model to simulate data using the candidate parameter vector. We can see that the choice frequencies and wage distribution differ from the moments of the observed dataset.

```python
params = params_cand.copy()
simulate = rp.get_simulate_func(params, options)
df_sim = simulate(params)
```

```python
moments_sim = {
    "Choice Frequencies": replace_nans(calc_moments["Choice Frequencies"](df_sim)),
    "Wage Distribution": replace_nans(calc_moments["Wage Distribution"](df_sim)),
}
```

We can plot the moments to compare the choice frequencies for each period.

```python
plot_moments_choices(moments_obs, moments_sim)
```

The plots below show the mean and the standard deviation in the wages for each period.

```python
plot_moments_wage(moments_obs, moments_sim)
```

The criterion function value for the candidate parameter vector is not zero.

```python
fval = criterion_msm(params_cand)
fval
```

    7838.131228444257

##### Optimization Procedure

We will now use an optimization procedure to retrieve the true parameter vector. For the optimization we can use [estimagic](https://estimagic.readthedocs.io/en/latest/index.html). In order to minimize the criterion function we need estimagic's `minimize` function.

```python
from estimagic import minimize  # noqa: E402
```

We have verified above that the criterion function gives a value of 0 for the true parameter vector. Before we try different parameter specifications, we can check whether an optimizer recognizes the true vector as the minimum of our criterion function.

As the code below shows, the optimization algorithm recognizes the true parameter vector as the minimum of the criterion function as it returns a function value of 0 and the true parameter values.

```python
rslt = minimize(
    criterion=criterion_msm,
    params=params_true,
    algorithm="nag_pybobyqa",
)
rslt["solution_criterion"]
```

    0.0

##### Upper and Lower Bounds

We can help the optimizer by specifying bounds for the parameters. Since we know the true parameters in the case of this model, we can just pick upper and lower bounds that are fairly close to the true values of the parameters to aid the optimizer in the search for the optimum. By default, the upper and lower bounds are set to $\infty$ and $-\infty$, so specifying upper and lower bounds substantially reduces the range of parameter values that the optimizer can potentially cover.

For optimization with estimagic, we can specify bounds by adding the columns *'lower'* and *'upper'* to the dataframe that contains the parameter values.

```python
params_cand["lower_bound"] = [0.69, 0.066, -0.2, 1, 0, 0, 0]
params_cand["upper_bound"] = [1, 0.078, -0.09, 1.1, 0.5, 0.5, 0.5]
params_cand
```

      value
      lower_bound
      upper_bound

      category
      name

      delta
      delta
      0.930
      0.690
      1.000

      wage_fishing
      exp_fishing
      0.070
      0.066
      0.078

      nonpec_fishing
      constant
      -0.100
      -0.200
      -0.090

      nonpec_hammock
      constant
      1.046
      1.000
      1.100

      shocks_sdcorr
      sd_fishing
      0.010
      0.000
      0.500

      sd_hammock
      0.010
      0.000
      0.500

      corr_hammock_fishing
      0.000
      0.000
      0.500

##### Constraints

Additionally we hold all other parameters fixed for now to aid the optimizer in finding the optimal value for *delta*.

```python
# Define base constraints to use for the rest of the notebook.
constr_base = [
    {"loc": "shocks_sdcorr", "type": "sdcorr"},
    {"loc": "delta", "type": "fixed"},
    {"loc": "wage_fishing", "type": "fixed"},
    {"loc": "nonpec_fishing", "type": "fixed"},
    {"loc": "nonpec_hammock", "type": "fixed"},
    {"loc": "shocks_sdcorr", "type": "fixed"},
]
```

```python
# Remove constraint for delta.
constr = constr_base.copy()
constr.remove({"loc": "delta", "type": "fixed"})
```

##### Optimize

We can now optimize the criterion function with respect to the parameter vector. The optimizer manages to reach a function value of 0 and finds an approximately correct *delta* for our model.

This exercise again reveals that we are dealing with a non-smooth criterion function. The optimizer does not return the exact value of 0.95 for *delta* because of the little plateaus we saw when zooming into the criterion function. As the plot shows, there is a small area around the true value for *delta* that also returns a function value of 0 and might thus be picked by the optimizer.

```python
rslt = minimize(
    criterion=criterion_msm,
    params=params_cand,
    algorithm="nag_pybobyqa",
    constraints=constr,
)
rslt["solution_criterion"]
```

    0.5248952065747418

#### Chatter in the Criterion Function

In this exercise we explore the sensitivity of the criterion function to the choice of simulation seed and number of agents.

##### Changing the Simulation Seed

As shown above, the optimizer manages to find a function value of exactly 0 for the true parameter vector. This is the case because respy controls the realization of random elements in the simulation process via a simulation seed. The model thus always simulates the exact same dataset for a given parameter vector. Our criterion function becomes exactly 0 at the true parameter vector because for this vector, the observed and simulated data are identical.

Changing the simulation seed results in simulated moments that, even for the true parameter vector, are never completely equal to the observed moments.

Let's estimate the model with a different simulation seed.

```python
options_new_seed = options.copy()
options_new_seed["simulation_seed"] = 400
```

We can see that the criterion function is not 0 anymore for the true parameter vector.

```python
criterion_msm = rp.get_moment_errors_func(
    params_true, options_new_seed, calc_moments, replace_nans, moments_obs, W
)
criterion_msm(params_true)
```

    38.97135021944551

Our optimizer thus also does not return a function value of 0 for the true parameter vector.

```python
rslt_new_seed = minimize(
    criterion=criterion_msm,
    params=params_true,
    algorithm="nag_pybobyqa",
    constraints=constr,
)
rslt_new_seed["solution_criterion"]
```

    17.369549302805428

Since the optimizer does not even recognize the true parameter vector, it is also not able to reach a criterion function  value of 0 for a different parameter vector.

```python
rslt_new_seed = minimize(
    criterion=criterion_msm,
    params=params_cand,
    algorithm="nag_pybobyqa",
    constraints=constr,
)

rslt_new_seed["solution_criterion"]
```

    17.369549302805428

##### Multiple Simulation Seeds

The section above shows what happens to the criterion function if we change the seed for simulating the data. In the sections below we will repeat this exercise for multiple seeds and explore what happens if we increase the number of simulated as well as observed agents in our model.

```python
# List of seeds, includes true simulation seed of 132.
seeds = list(range(120, 140, 1))
```

```python
# Define other inputs for criterion function
criterion_kwargs = dict()
criterion_kwargs["params"] = params_true.copy()
criterion_kwargs["options"] = options
criterion_kwargs["calc_moments"] = calc_moments
criterion_kwargs["replace_nans"] = replace_nans
criterion_kwargs["moments_obs"] = moments_obs
criterion_kwargs["weighting_matrix"] = W
```

The plot below shows the criterion function for different simulation seeds, including the true one. We can seed that different seeds lead to different fits between the simulated and observed data, with some seeds performing worse than others. Only the true seed leads to a function value of 0.

```python
plot_chatter(seeds, criterion_kwargs)
```

##### Changing the Simulation Seed for increasing Sample Size of Simulated Agents

We now repeat this exercise for an increasing number of simulated agents. As the plot below shows, increasing the number of simulated agents reduces the chatter considerably but the criterion function remains consistently above zero. Only the simulated sample of 1000 agents at the true simulation seed reaches a function of zero, since, as discussed before, the simulated sample is identical to the observed one for this calibration.

```python
# Changing the number of agents.
num_agents = [500, 1000, 5000, 10000, 50000]
```

```python
plot_chatter_numagents_sim(seeds, num_agents, criterion_kwargs)
```

##### Changing the Simulation Seed for increasing Sample Size of Simulated *and* Observed Agents

We now increase not only the number of simulated agents but also the number of observed agents in our sample. Doing so decreases the chatter in the criterion function as before but also levels out the function around 0 for all simulation seeds. For large enough samples of observed and simulated agents the choice of simulation seed is thus irrelevant.

```python
plot_chatter_numagents_both(seeds, num_agents, calc_moments, replace_nans, criterion_kwargs)
```

*Note*: In contrast to the previous plot, we can see that the function reaches a value of 0 at the true simulation seed for all quantities of agents. This is the case because we now simultaneously increase the number of both simulated and observed agents while the observed sample remained untouched in the previous plot. Increasing the number of agents simultaneously for both groups creates identical samples for the true simulation seed at each quantity of agents.

#### Moving away from the Optimum

For the next exercise we explore what happens to the criterion function if we move away from the optimum. We do this in a controlled fashion by changing the parameter values for *delta* and *wage_fishing* and fixing the latter in the constraints. Doing so should lead *delta* to move away from its optimal value of 0.95 during optimization.

```python
params_cand.loc["wage_fishing", "value"] = 0.072
params_cand
```

      value
      lower_bound
      upper_bound

      category
      name

      delta
      delta
      0.930
      0.690
      1.000

      wage_fishing
      exp_fishing
      0.072
      0.066
      0.078

      nonpec_fishing
      constant
      -0.100
      -0.200
      -0.090

      nonpec_hammock
      constant
      1.046
      1.000
      1.100

      shocks_sdcorr
      sd_fishing
      0.010
      0.000
      0.500

      sd_hammock
      0.010
      0.000
      0.500

      corr_hammock_fishing
      0.000
      0.000
      0.500

The optimizer cannot retrieve the true parameter and does not reach a value of 0.

```python
criterion_msm = rp.get_moment_errors_func(
    params_true, options_new_seed, calc_moments, replace_nans, moments_obs, W
)
rslt_wrong_fix = minimize(
    criterion=criterion_msm,
    params=params_cand,
    algorithm="nag_pybobyqa",
    constraints=constr,
)
rslt_wrong_fix["solution_criterion"]
```

    813.7988993578208

##### Retrieving the true Parameter Vector

We now repeat the estimation with the new parameter vector and free up *wage_fishing* to retrieve the optimal values for both parameters.

The parameter for *wage_fishing* is still 0.072 since we fixed it for the prior estimation:

```python
params_cand.loc[:, "value"] = rslt_wrong_fix["solution_params"][["value"]]
params_cand
```

      value
      lower_bound
      upper_bound

      category
      name

      delta
      delta
      0.923022
      0.690
      1.000

      wage_fishing
      exp_fishing
      0.072000
      0.066
      0.078

      nonpec_fishing
      constant
      -0.100000
      -0.200
      -0.090

      nonpec_hammock
      constant
      1.046000
      1.000
      1.100

      shocks_sdcorr
      sd_fishing
      0.010000
      0.000
      0.500

      sd_hammock
      0.010000
      0.000
      0.500

      corr_hammock_fishing
      0.000000
      0.000
      0.500

We now free up *wage_fishing* in the constraints in addition to *delta*.

```python
# Adjust constraints to free up both delta and wage_fishing.
constr_u = constr_base.copy()
constr_u.remove({"loc": "delta", "type": "fixed"})
constr_u.remove({"loc": "wage_fishing", "type": "fixed"})
```

Freeing up the non-optimal *wage_fishing* improves the estimates. The criterion function value is much closer to 0 and the optimizer manages to retrieve the true parameter values quite closely.

```python
rslt_unfix = minimize(
    criterion=criterion_msm,
    params=params_cand[["value"]],
    algorithm="nag_pybobyqa",
    constraints=constr_u,
)
rslt_unfix["solution_criterion"]
```

    710.9165575145821

For easier comparison, we can compute the difference between the true and estimated value:

```python
deviation = params_true["value"] - rslt_unfix["solution_params"]["value"]
deviation
```

    category        name
    delta           delta                   0.026703
    wage_fishing    exp_fishing            -0.001791
    nonpec_fishing  constant                0.000000
    nonpec_hammock  constant                0.000000
    shocks_sdcorr   sd_fishing              0.000000
                    sd_hammock              0.000000
                    corr_hammock_fishing    0.000000
    Name: value, dtype: float64

#### Derivative-Based Optimization Algorithm

So far we have used only one optimization algorithm to estimate the parameter vector. The algorithm we used, BOBYQA (Bound Optimization by Quadratic Approximation), is a derivative-free optimization algorithm which works fairly well on our criterion function. As discussed above, our criterion function is a step function that contains plateaus for certain ranges of parameter values. An important implication of this property is that we cannot calculate proper derivatives and thus derivative-based optimization algorithms will not work to estimate the parameters.

To demonstrate this problem, we will now try to estimate the parameters using an optimization algorithm that does use derivatives during optimization.

```python
# Define candidate parameter vector and constraints.
params_cand = params_true.copy()
params_cand.loc["delta", "value"] = 0.93

params_cand["lower_bound"] = [0.79, 0.066, -0.11, 1.04, 0, 0, 0]
params_cand["upper_bound"] = [1, 0.072, -0.095, 1.055, 0.5, 0.5, 0.5]

constr = constr_base.copy()
constr.remove({"loc": "delta", "type": "fixed"})
```

We try the L-BFGS-B (Limited-Memory-Broyden-Fletcher-Goldfarb-Shanno with box constraints) algorithm. As shown below, L-BFGS-B fails and the optimizer returns the same parameter vector that we used as an input.

```python
criterion_msm = rp.get_moment_errors_func(
    params_true, options_new_seed, calc_moments, replace_nans, moments_obs, W
)
rslt = minimize(
    criterion=criterion_msm,
    params=params_cand,
    algorithm="scipy_lbfgsb",
    constraints=constr,
)
rslt
```

    {'solution_x': array([0.93]),
     'solution_criterion': 7204.902739425344,
     'solution_derivative': array([0.]),
     'solution_hessian': None,
     'n_criterion_evaluations': 1,
     'n_derivative_evaluations': None,
     'n_iterations': 0,
     'success': True,
     'reached_convergence_criterion': None,
     'message': 'CONVERGENCE: NORM_OF_PROJECTED_GRADIENT_<=_PGTOL',
     'solution_params':                                      lower_bound  upper_bound  value
     category       name
     delta          delta                       0.790        1.000  0.930
     wage_fishing   exp_fishing                 0.066        0.072  0.070
     nonpec_fishing constant                   -0.110       -0.095 -0.100
     nonpec_hammock constant                    1.040        1.055  1.046
     shocks_sdcorr  sd_fishing                  0.000        0.500  0.010
                    sd_hammock                  0.000        0.500  0.010
                    corr_hammock_fishing        0.000        0.500  0.000}

### References

* Adda, J., & Cooper, R. W. (2003). *Dynamic Economics: Quantitative Methods and Applications*. MIT press.

* Adda, J., Dustmann, C., & Stevens, K. (2017). The Career Costs of Children. *Journal of Political Economy*, 125(2), 293-337.

* Andrews, I., Gentzkow, M., & Shapiro, J. M. (2017). Measuring the Sensitivity of Parameter Estimates to Estimation Moments. *The Quarterly Journal of Economics*, 132(4), 1553-1592.

* Bruins, M., Duffy, J. A., Keane, M. P., & Smith Jr, A. A. (2018). Generalized Indirect Inference for Discrete Choice Models. *Journal of econometrics*, 205(1), 177-203.

* Davidson, R., & MacKinnon, J. G. (2004). *Econometric Theory and Methods (Vol. 5)*. New York: Oxford University Press.

* Evans, R. W. (2018, July 5). Simulated Method of Moments (SMM) Estimation. Retrieved November 30, 2019, from https://notes.quantecon.org/submission/5b3db2ceb9eab00015b89f93.

* Frazier, D. T., Oka, T., & Zhu, D. (2019). Indirect Inference with a Non-Smooth Criterion Function. *Journal of Econometrics*, 212(2), 623-645.

* Gourieroux, M., & Monfort, D. A. (1996). *Simulation-based econometric methods*. Oxford university press.

* McFadden, D. (1989). A Method of Simulated Moments for Estimation of Discrete Response Models without Numerical Integration. *Econometrica: Journal of the Econometric Society*, 995-1026.