### Instructions

1. Make sure you are using a version of notebook greater than v.3. If you installed Anaconda with python 3 - this is likely to be fine. The next piece of code will check if you have the right version.
2. The notebook has both some open test cases that you can use to test the functionality of your code - however it will be run on another set of test cases that you can't from which marks will be awarded. So passing all the tests in this notebook is not a guarantee that you have done things correctly - though its highly probable.
3. Also make sure you submit a notebook that doesn't return any errors. One way to ensure this is to run all the cells before you submit the notebook.
4. When you are done create a zip file of your notebook and upload that
5. For each cell where you see "YOUR CODE HERE" delete the return notImplemented statement when you write your code there - don't leave it in the notebook.
6. Once you are done, you are done.

```python

```

# DSA 2018 Nyeri Preparatory Notebook
By Ciira Maina

In preparation for DSA 2018 Nyeri, we would like potential participants to complete a number of exercises in probability, machine learning and programming to ensure that they have the necessary prerequisite knowledge to attend the summer school. You will be required to submit notebooks with solutions to these exercises during the application process.

In this first exercise we will require you to download a dataset and perform computations on the data. These data are from a paper in 1966 by Cox and Lewis and report the time difference between nerve pulses on a nerve fibre. 799 observations are reported. These data are used for some examples in the text ["All of Statistics"](http://www.stat.cmu.edu/~larry/all-of-statistics/) by Larry Wasserman.

The data are available [here](http://www.stat.cmu.edu/~larry/all-of-statistics/=data/nerve.dat)

```python
import matplotlib.pyplot as plt
%matplotlib inline
import urllib.request
import numpy as np
```

## Obtain the data

Write code to obtain the data from the website above and store it in a one dimensional array of floating point numbers.

```python
nerve_data_url='http://www.stat.cmu.edu/~larry/all-of-statistics/=data/nerve.dat'
def read_data(url):
    # Read in data from url and return 1-D array
    fromUrl = urllib.request.urlopen(url).read().decode('utf-8')
    fromUrl = fromUrl.split('\r\n')

    newlist=[]
    for x in fromUrl:
        newlist.append(x.split('\t'))

    finalist=[]
    for i in newlist:
        for x in i:
            if x != '':
                finalist.append(float(x))

    return np.array(finalist)
```

```python
nerve_data = read_data(nerve_data_url)
assert len(nerve_data) == 799
```

## Preliminary Visualisation

Plot a histogram of the data. Ensure you label your axes.

```python
plt.hist(nerve_data)
plt.xlabel('Nerve Pulses')
plt.ylabel('Frequency of occurence')
plt.show()

```

## Preliminary analysis

The cumulative distribution function of a random variable $\mathbf{X}$ is given by

$
\begin{equation}
F_X(x)=P(\mathbf{X}\leq x)
\end{equation}$

If we obtain $n$ observations $X_1,\ldots,X_n$ from this distribution, the empirical distibution function is given by

$
\begin{equation}
\hat{F}_n(x)=\frac{\sum_{i=1}^n\mathbf{I}(X_i\leq x)}{n}
\end{equation}$

where
$
 \begin{equation}
       \mathbf{I}(X_i\leq x) =
        \begin{cases}
            1 & \text{if $X_i\leq x $} \\
            0 & \text{if $X_i> x$}
        \end{cases}
    \end{equation}
$
* Plot the empirical distribution function of the nerve pulse data
* Estimate the probability that the wait time between nerve pulses is less than $0.3$ - We will call this P1
* Estimate the probability that the wait time between nerve pulses is between $0.1$ and $0.3$ - We will call this P2

Given a random variable X obtain the empirical distribution of a given set of data

```python
def cdf(X, data):
    # Return a vector the size of X representing the CDF
    # YOUR CODE HERE

    raise NotImplementedError()
```

Plot the empirical distribution function of the nerve pulse data

```python
X=np.linspace(0,np.max(nerve_data),100)
# YOUR CODE HERE
raise NotImplementedError()
```

Estimate the probability that the wait time between nerve pulses is less than  0.3. Hint: refer to the previous fomula for the cummulative distribution

```python
def prob_x(x, data):
    # YOUR CODE HERE
    raise NotImplementedError()
```

```python
P1 = prob_x(0.3, nerve_data)
assert abs(P1-0.760951188986) < 1e-6
```

Estimate the probability that the wait time between nerve pulses is between  0.1 and  0.3

```python
def prob_xy(x,y,data):
    # Return probability of wait time between x, and y
    # YOUR CODE HERE
    raise NotImplementedError()
```

```python
P2 = prob_xy(0.1,0.3,nerve_data)
assert abs(P2-0.377972465582) < 1e-6
```

## Estimating properties of the distribution

We can estimate properties of the true distribution of the data $F_X(x)$ using the empirical distribution function $\hat{F}_n(x)$. To do this we can use "plug in" estimators. Here we will estimate the mean, variance and skewness. The expressions for the "plug in" estimators for these quantities are
* Mean: $\hat{\mu}=\frac{1}{n}\sum_{i=1}^nX_i$
* Variance: $\hat{\sigma}^2=\frac{1}{n}\sum_{i=1}^n(X_i-\hat{\mu})^2$
* Skewness: $\hat{\kappa}=\frac{\frac{1}{n}\sum_{i=1}^n(X_i-\hat{\mu})^3}{\hat{\sigma}^3}$

Compute the plug in estimators of the mean, variance and skewness for the nerve pulse wait time data.

```python
def dist_properties(data):
    # Return the mean, variance, skewness of the distribution
    # YOUR CODE HERE
    raise NotImplementedError()
```

```python
mu, var, kappa = dist_properties(nerve_data)
assert np.round(mu,3) == 0.219
assert np.round(var,3) == 0.044
assert np.round(kappa,3) == 1.761
```

Machine Learning Basics/DSA Preparatory Notebook 1.ipynb


# 2. Linear algebra

This chapter is devoted to linear algebra and its translation to Dirac's notation. The importance of linear algebra for quantum computing comes from the fact that quantum mechanics, the theory which describes the physical world, can be formulated in matrix representations. This means that the state of a physical systems can be represented as a vector in a particular vector space (a Hilbert space) and to each property of the system in the physical world there correspond a matrix (operators) in the vector space. The time evolution of a state (vector), or its properties, are then extracted by acting with the corresponding operator (matrix) on the state. Thus, it is fundamental to have a basic understanding of linear algebra and its rules.
To simplify the vector and matrix notations typical of linear algebra, we introduce Dirac's notation. This allows to write the state of a system as a vector in the vector space in a compact way. The basics of linear algebra for Hilbert spaces in terms of both Dirac's notation and matrix representation are then reviewed. Finally, operators are discussed in detail.

## 2.1 States

A physical system is a part of the physical world which we wish to investigate. Thus, a physical system can be seen as a specific physical object, like an atom or a cat, or a more complicated entity, such as the atmosphere of the earth, separated from the rest of the natural world. Certain properties can be chosen to describe the state of a physical system at any given time. The values of these properties determine the specific state of the system. If these values change, the state of the system will also change.

### Vector space $\mathcal{H}$

In quantum mechanics, a physical system has a mathematical representation in terms of a complex vector space, also called a Hilbert space $\mathcal{H}$. The characteristics of this vector space, like its dimensions, are determined by the properties chosen for the description of the state of the physical system. If one selects $n$ independent properties to describe a system, the vector space representing this system will have dimension $n$. A specific state of the system, or a specific set of values of the chosen properties of the system, is represented by a vector in the Hilbert space. We adopt Dirac's notation to denote vectors, thus a vector $\vec{A}$ will be written as a "ket" vector $\lvert A \rangle$. Multiples of a vector describing a certain state, like $5\lvert A \rangle$ or $25i\lvert A \rangle$, will still describe the same state of the system. Thus, each state has an infinite number of ket vectors associated to it.

$$\text{1. Pictorial representation of a $n$ dimensional Hilbert space. The $n$ basis vectors $\lvert e_i \rangle$  are all orthogonal to each other. The state of a system $\lvert \psi \rangle$ is a vector in the Hilbert space.}$$

In linear algebra, a vector in an $n$ dimensional space can be represented as an array with $n$ entries in a column

$$
 \lvert A \rangle =
        \begin{pmatrix}
        a_1 \\
        a_2 \\
        . \\
        . \\
        . \\
        a_n
        \end{pmatrix}
$$

### Adjoint space $\mathcal{H}^{\ast}$

We can imagine a vector space, the adjoint space $\mathcal{H}^{\ast}$, whose elements $\langle A \rvert$, called bra vectors, are in one-to-one correspondance with the vector space $\mathcal{H}$. For each element $\lvert A \rangle$ of the vector space, there is a corresponding element $\langle A \rvert$ in the adjoint space.

In matrix representation, to find the adjoint ($\dagger$) of a matrix we take the complex conjugate ($\ast$) of the entries and then transpose ($T$) the matrix

$$
\begin{pmatrix}
        a_{11} & a_{12} & ... & a_{1n} \\
        a_{21} & a_{22} & ... & a_{2n} \\
        . & . & ... & . \\
        . & . & ... & . \\
        . & . & ... & . \\
        a_{n1} & a_{n2} & ... & a_{nn}
        \end{pmatrix}^{\dagger} =
        \begin{pmatrix}
        a_{11}^{*} & a_{12}^{*} & ... & a_{1n}^{*} \\
        a_{21}^{*} & a_{22}^{*} & ... & a_{2n}^{*} \\
        . & . & ... & . \\
        . & . & ... & . \\
        . & . & ... & . \\
        a_{n1}^{*} & a_{n2}^{*} & ... & a_{nn}^{*}
        \end{pmatrix}^{T} =
        \begin{pmatrix}
        a_{11}^{*} & a_{21}^{*} & ... & a_{n1}^{*} \\
        a_{12}^{*} & a_{22}^{*} & ... & a_{n2}^{*} \\
        . & . & ... & . \\
        . & . & ... & . \\
        . & . & ... & . \\
        a_{1n}^{*} & a_{2n}^{*} & ... & a_{nn}^{*}
        \end{pmatrix}
$$

For vectors we have

$$
\begin{pmatrix}
        a_{1} \\
        a_{2}  \\
        .  \\
        .\\
        . \\
        a_{n}
        \end{pmatrix}^{\dagger} =
        \begin{pmatrix}
        a_{1}^{*} & a_{2}^{*} & ... & a_{n}^{*}
        \end{pmatrix}
$$

In Dirac's notation, the adjoint of a ket vector  $\lvert  A  \rangle $ is the corresponding bra vector $\langle A \rvert$: $\langle A \rvert = \lvert  A  \rangle^{\dagger} $.
Thus bra vector correspond to row vectors in matrix representation.

### Scalar product

Given vectors and their adjoint, one can define a scalar product on the Hilbert space.
The scalar product is a complex number associated with a bra-ket pair. The scalar product of the vector $\left\vert A\right\rangle \in \mathcal{H}$ with the vector $\left\vert
B\right\rangle \in \mathcal{H}$ is defined as

\begin{equation}
\left\langle A\right\vert B\rangle = c
\end{equation}

where $c$ a complex number. In the matrix representation, the scalar product is calculated by summing the products of corresponding entries

\begin{eqnarray}
\left\langle A\right\vert B\rangle &=&\left(
\begin{array}{c}
a_{1} \\
a_{2} \\
. \\
. \\
. \\
a_{n-1} \\
a_{n}%
\end{array}
\right) \cdot \left(
\begin{array}{c}
b_{1} \\
b_{2} \\
. \\
. \\
. \\
b_{n-1} \\
b_{n}%
\end{array}
\right)
=\left(
\begin{array}{ccccccc}
a_{1}^{\ast } & a_{2}^{\ast } & . & . & . & a_{n-1}^{\ast } & a_{n}^{\ast }%
\end{array}%
\right) \left(
\begin{array}{c}
b_{1} \\
b_{2} \\
. \\
. \\
. \\
b_{n-1} \\
b_{n}%
\end{array}%
\right) =\sum_{i=1}^{n}a_{i}^{\ast }b_{i}
\end{eqnarray}

We say that $\left\vert A\right\rangle $ and $\left\vert B\right\rangle $
are orthogonal if

\begin{equation}
\left\langle A\right\vert B\rangle =0.
\end{equation}

$\left\vert A\right\rangle $ is normalized if

\begin{equation}
\left\langle A\right\vert A\rangle =1.
\end{equation}

Putting these two properties together, we say that two states $\left\vert A\right\rangle $ and $\left\vert B\right\rangle $ are orthonormal if

\begin{equation}
\left\langle A\right\vert B\rangle =\delta _{AB},
\end{equation}

where $\delta _{AB}$ is the Kronecker symbol, which is equal to one if $A=B$ and zero otherwise.

### Basis

Let's say that a set of $n$ vectors $\left\{ \left\vert
e_{n}\right\rangle \right\} \subseteq \mathcal{H}$ forms an orthonormal
basis for $\mathcal{H}$. Thus, $\left\langle e_{n}\right\vert e_{m}\rangle
=\delta _{nm}$. Every vector $\left\vert B\right\rangle $ from the Hilbert space $\left\vert B\right\rangle \in \mathcal{H}$ may be expanded in terms of this
orthonormal basis as

\begin{equation}
\left\vert B\right\rangle =\sum_{i=1}^{n}b_{i}\left\vert e_{i}\right\rangle .
\end{equation}

The coefficients $b_{i}=\left\langle B\right\vert e_{i}\rangle $ are the projections of $\left\vert B\right\rangle $ with respect to the basis $\left\{ \left\vert e_{n}\right\rangle
\right\}$.

If $\left\{ \left\vert e_{n}\right\rangle \right\} \subseteq \mathcal{H}$ form an orthonormal
basis for $\mathcal{H}$ then $\left\{ \left\langle e_{n}\right\vert \right\}
$ is an orthonormal basis for the adjoint space $\mathcal{H}^{\ast }$.

### Tensor product

One can consider the tensor product between two Hilbert spaces $\mathcal{H}_A\otimes \mathcal{H}_B$. The basis for the new vector space obtained in this way is given by the tensor product of the bases of the two spaces.

In general, the tensor product of any two vectors can be written in matrix form as

\begin{equation*}
\left(
\begin{array}{c}
a_0 \\
a_1 \\
. \\
. \\
. \\
a_n
\end{array}
\right) \otimes \left(
\begin{array}{c}
b_0 \\
b_1\\
. \\
. \\
. \\
b_n
\end{array}
\right) = \left(
\begin{array}{c}
a_0b_0 \\
a_0b_1 \\
. \\
. \\
. \\
a_0b_n \\
. \\
. \\
. \\
a_nb_0 \\
a_nb_1 \\
. \\
. \\
. \\
a_nb_n \\
\end{array}
\right)
\end{equation*}

In Dirac's notation, the tensor product between two vectors is simply written as

$$ \lvert A \rangle \otimes \lvert B \rangle = \lvert A \rangle \lvert B \rangle = \lvert A , B \rangle = \lvert A B \rangle $$

## 2.2 Operators

Operators are associated to physical properties of the system. More specifically, to find out the value of a property of the system, we "act" with the operator on the state of the system. The result of the operation encodes the value of the physical property associated to the operator for the system in a certain state.

$$\text{2. In general, the effect of an operator $\hat{O}$ acting on a state $\lvert \psi \rangle$ is to rotate the vector corresponding to the state. Therefore, giving a new state $\lvert \psi' \rangle$. }$$

Operators can be written in Dirac's notation as ket-bra pairs $\lvert A\rangle \langle A\rvert$, this is called an outer product. By writing an operator in this way, one can immediately see that when an operator acts on a state, this extracts the contribution of the state to the operator: $\lvert A\rangle \langle A\rvert \lvert B \rangle = \langle A \lvert B \rangle \lvert A\rangle $.

In general, operators are denoted by a hat, therefore the operator associated with the outer product $\lvert A\rangle \langle A\rvert$ of the system is written as $\hat{A}$. A generic operator $\hat{O}$ can always be written in terms of a combination of outer products of the basis vectors of the Hilbert space

\begin{equation}
\hat{O}=\sum_{i,j=1}^{n}O_{ij}\lvert
b_{i}\rangle \langle b_{j}\rvert .
\end{equation}

where $O_{ij}=\langle b_{i}\rvert \hat{O} \lvert b_{j} \rangle $ are the matrix element and $\left\{
\lvert b_{n}\rangle \right\} $ is an orthonormal basis for the $n$ dimensional
vector space $\mathcal{H}$. In other words, any linear operator can be
written as combination of all the possible outer products of the basis vectors $\lvert b_{i}\rangle \langle b_{j}\rvert $.

We have seen how operators can be written in terms of outer products of ket-bra vectors, in linear algebra terms an operator is a linear transformation on the Hilbert space. Recall that a linear transformation maps vectors in $\mathcal{H}$ to vectors in $\mathcal{H}$ and is written a sa matrix. To see this, let us see the matrix form of an operator

\begin{equation*}
\hat{O} = \left(
\begin{array}{cccccc}
O_{11} & O_{12} & . & . & O_{1,N-1} & O_{1N} \\
O_{21} & O_{22} & . & . & . & . \\
. & . & . & . & . & . \\
. & . & . & . & . & . \\
O_{N1} & O_{N2} & . & . & O_{N,N-1} & O_{NN} \\
&  &  &  &  &
\end{array}
\right)
\end{equation*}

In the matrix representation of the operator $\hat{O}$, $O_{ij}$ is the matrix element in the $i^{th}$ row and $j^{th}$ column.
The action of the operator $\hat{O}$ on a vector $\lvert
A \rangle $ is then written as

\begin{equation}
\hat{O}\lvert A\rangle =\sum_{i,j=1}^{n} O_{ij}\lvert b_{i} \rangle \langle b_{j}\rvert
A\rangle =\sum_{i,j=1}^{n} O_{ij} \langle b_{j} \rvert A \rangle \vert b_{i} \rangle = \sum_{i,j = 1}^{n} O_{ij} A_j  \lvert b_i  \rangle .
\end{equation}

where $A_j$ is defined as $\langle b_j  \lvert  A  \rangle $, the component of $ \lvert  A  \rangle $ along  $\lvert  b_j  \rangle $ .

### Properties

#### Linear

Operators are linear:

\begin{equation}
\hat{O}(c_{1}\lvert A\rangle +c_{2}\lvert
B\rangle )=c_{1}\hat{O}\lvert A\rangle
+c_{2}\hat{O}\lvert B\rangle .
\end{equation}

#### Hermitean

Observables quantities in quantum mechanics are represented by Hermitean operators. Suppose that $\hat{O}$ is an operator on $\mathcal{H}$. The adjoint operator of $\hat{O}$ is denoted by $\hat{O}^{\dagger }$ and it is defined as a linear operator on $\mathcal{H}^{\ast }$ that satisfies the following condition:

\begin{equation}
\left( \langle A \rvert \hat{O}^{\dagger} \lvert B \rangle \right) ^{\dagger} = \langle B \rvert \hat{O} \lvert A \rangle
\end{equation}

That is, Hermitean operators satisfy: $\hat{O}=\hat{O}^{\dagger}$.

#### Unitary

An operator $\hat{U}^{-1}$ is the inverse of the operator $\hat{U}$ if $\hat{U}^{-1}\hat{U}=\hat{I},$ where $\hat{I}$
is the identity operator. Also, an operator $\hat{U}$ is unitary if $\hat{U}^{\dagger}=\hat{U}^{-1}$ so that $\hat{U}^{\dagger}\hat{U} = \hat{I}$ .
Note that unitary operators can be formed by exponentiating Hermitean operators $\hat{U} = e^{i\hat{O}}$, so that

$$ \hat{U}^{\dagger} \hat{U}= e^{-i\hat{O}^{\dagger}} e^{i\hat{O}} = e^{-i\left( \hat{O}^{\dagger}-\hat{O}\right)} =  \hat{I} $$

since $\hat{O}^{\dagger}-\hat{O}=0$ if $\hat{O}$ is Hermitean.

Time evolution operators in quantum mechanics are unitary.

### Expectation value

The value of the physical quantity corresponding to the operator $\hat{O}$ for the system in a state $\lvert A\rangle $ is called the expectation value $\langle \hat{O} \rangle$ of $\hat{O}$. This is calculated as

\begin{equation}
\langle \hat{O} \rangle =\langle
A\rvert \hat{O} \lvert A \rangle = c \; \text{ (a
number)}.
\end{equation}

The expectation value of an Hermitian operator is real:

\begin{equation}
\langle \hat{O}\rangle ^{\ast }=\left( \langle A\rvert \hat{O}\lvert A\rangle
\right) ^{\ast }=\left( \langle A\rvert \hat{O}^{\dagger }\lvert A\rangle \right) =\langle A\rvert
\hat{O}\lvert A\rangle =\langle \hat{O}\rangle
\end{equation}

### Eigenstate and eigenvalues

A vector $\lvert A \rangle $ is an eigenvector of an operator $\hat{O}$ if

\begin{equation}
\hat{O} \lvert A \rangle =a \lvert A\rangle
\end{equation}

where $a$ is a constant and is called the eigenvalue of $\hat{O}$
corresponding to eigenvector $\lvert A\rangle$. It is easy to
prove that the eigenvalues of an Hermitean operator are real.

\begin{eqnarray}
\hat{O}\lvert A\rangle &=&a\lvert A\rangle .   \\
\hat{O^{\dagger }}\lvert A\rangle &=& a^{\ast}\lvert A\rangle
\end{eqnarray}

Therefore, we have that $\left(\hat{O}-\hat{O}^{\dagger }\right) \lvert A\rangle =(a-a^{\ast })\lvert
A\rangle =0$. thus, $a=a^{\ast }$.

Also, one can prove that eigenvectors of the same Hermitean operator having
different eigenvalues are orthogonal.

### Tensor product

Tensor products of operators in matrix representation are calculated as

\begin{eqnarray}
\hat{A} \otimes \hat{B} & = &\left(
\begin{array}{cccccccc}
A_{11}B_{11} & ... & A_{11}B_{1q} & ... & ... & A_{1n}B_{11} & ... & A_{1n}B_{1q}
&  \\
. & . & . & . & . & . & . & . &  \\
. & . & . & . & . & . & . & . &  \\
A_{11}B_{p1} & ... & A_{11}B_{pq} & ... & ... & A_{1n}B_{p1} & ... & A_{1n}B_{pq}
&  \\
. & . & . & . & . & . & . & . &  \\
. & . & . & . & . & . & . & . & \\
A_{m1}B_{11} & ... & A_{m1}B_{1q} & ... & ... & A_{mn}B_{11} & ... & A_{mn}B_{1q}
&  \\
. & . & . & . & . & . & . & . & \\
. & . & . & . & . & . & . & . &  \\
A_{m1}B_{p1} & ... & A_{m1}B_{pq} & ... & ... & A_{mn}B_{p1} & ... & A_{mn}B_{pq}
&  \\
&  &  &  &  &  &  &
\end{array}
\right)
\end{eqnarray}

## Exercises

Find the bra (adjoint vector) corresponding to the following kets. Write it both in Dirac's notation and in matrix form

$ \lvert A \rangle =
        \begin{pmatrix}
        \frac{i}{\sqrt{2}} \\
        0 \\
        0 \\
        \frac{1}{\sqrt{2}}
        \end{pmatrix} $

$ \lvert B \rangle =
        \begin{pmatrix}
        \frac{i}{\sqrt{3}} \\
        0 \\
        \frac{1}{\sqrt{3}}  \\
        \frac{1}{\sqrt{3}}
        \end{pmatrix} $

$ \lvert C \rangle =
        \begin{pmatrix}
        \frac{1-i\sqrt{2}}{\sqrt{6}} \\
        \frac{\sqrt{3}}{\sqrt{6}} \\
        \frac{1+i\sqrt{2}}{\sqrt{6}}  \\
        0
        \end{pmatrix} $

Calculate the following scalar products using the kets given above

$ \langle A \rvert B \rangle $

$ \langle B \rvert C \rangle $

$ \langle A \rvert C \rangle $

Consider a two-dimensional Hilbert space $\mathcal{H}_2$ with basis vectors:

$ \lvert 0 \rangle = \begin{pmatrix}
        1 \\
        0
        \end{pmatrix} $,

$ \lvert 1 \rangle = \begin{pmatrix}
        0 \\
        1
        \end{pmatrix} $

Find a basis for the space $\mathcal{H}_2 \otimes \mathcal{H}_2$ from the basis for $\mathcal{H}_2$

Consider the Pauli $\hat{Z}$ operator

$$ \hat{Z} = \lvert 0 \rangle \langle 0 \rvert - \lvert 1 \rangle \langle 1 \rvert =  \begin{pmatrix}
        1 & 0\\
        0 & -1
        \end{pmatrix} $$

Calculate the expectation value of $\hat{Z}$ for a system in the state $\lvert 0 \rangle $ and $\lvert 1 \rangle $

 Check if $\hat{Z}$ is Hermitean

Find the eigenvalues and the eigenstates of $\hat{Z}$

Write the expression of the operator $\hat{Z} \otimes \hat{Z}$ in matrix form

## References

[1] P. A. M. Dirac, "A new notation for quantum mechanics". Mathematical Proceedings of the Cambridge Philosophical Society. 35 (3): 416–418 (1939).

[2] P. A. M. Dirac, The Principles of Quantum Mechanics (1947 Clarendon Press, Oxford).

[3] R. P. Feynman, R. B. Leighton, M. Sands, "The Feynman Lectures on Physics III". Reading, MA: Addison-Wesley (1965).


## Slanted Gratings
A slanted grating is kind of weird because you cannot do a Fourier decomposition along the x (or transverse) direction. The grating vector is now in some linear combination of x and z., which means something has to change, but not sure what.

Preliminarily, we can consider the simpliest slanted granting:
\begin{equation}
    \epsilon(x,z) = \epsilon_0+\Delta\epsilon \cos(K(x \cos(\phi) + z\sin(\phi)))
\end{equation}
which would require no Fourier decomposition(at least along the grating vector direction).

The question then is how do we consider the plane waves. We could still write incident waves in the Cartesian basis, but what happens when they hit the grating region?

One thought is, in the grating region:
\begin{equation}
    E = \sum_{i}U_xi + \exp[j(k_{xi}x+ k_{zi}z + iK_y y]
\end{equation}
Ordinarily, the decomposition would just contain the x and z components (the transverse components), but now we need to include the K_y component of the grating vector? This thought appears to be supported by the Moharam paper.

```python
import numpy as np
import matplotlib.pyplot as plt
```

```python
x = np.linspace(-1,1,1000)
z = np.linspace(-0.2, 0.2, 1000)
K =20
phi = 45*np.pi/180;
print(np.sin(phi))
[X,Z] = np.meshgrid(x,z);
eps = 2+1*np.cos(K*(X*np.cos(phi)+Z*np.sin(phi)))
```

    0.7071067811865475

```python
plt.figure();
plt.pcolormesh(X,Z,eps);
plt.clim(0,2.2)
plt.colorbar();
```

```python

```

```python

```

notebooks/.ipynb_checkpoints/Explorations of Slanted Gratings-checkpoint.ipynb


# Bayesian Credible Intervals

Exploring whether and how we can estimate Bayesian credible intervals for posterior probability distributions from our MCMC simulations. In particular, it is interesting to see if it will be possible to estimate credible intervals for probability distributions that are constrained by a left or right wall that is artificially imposed by our simulations (grid boundaries).

```python
%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np
```

```python
left  = np.genfromtxt('/Users/grefe950/Software/StarBay/interbay/chains/run08/GJ896A_W0300_N0200_B0000.dat')
right = np.genfromtxt('/Users/grefe950/Software/StarBay/interbay/chains/run08/GJ15A_W0300_N0200_B0000.dat')
mid   = np.genfromtxt('/Users/grefe950/Software/StarBay/interbay/chains/run08/GJ105A_W0300_N0200_B0000.dat')
```

Remold the flattened MCMC chains into individual chains.

```python
left_chains  = left.reshape(300, -1, 9)
right_chains = right.reshape(300, -1, 9)
mid_chains   = mid.reshape(300, -1, 9)
```

We can now also extract a new flattened chain using the last 50 iterations from the sample.

```python
left_samp  = left_chains[:, -50:, :].reshape(-1, 9)
right_samp = right_chains[:, -50:, :].reshape(-1, 9)
mid_samp   = mid_chains[:, -50:, :].reshape(-1, 9)
```

It will be good to first evaluate the shape of the three posterior probability distributions for the mixing length, which poses particular problems.

```python
fig, ax = plt.subplots(1, 3, figsize=(12, 4))

tmp = ax[0].hist(left_samp[:, 5], bins=25, alpha=0.5)
tmp = ax[1].hist(right_samp[:, 5], bins=25, alpha=0.5)
tmp = ax[2].hist(mid_samp[:, 5], bins=25, alpha=0.5)
```

As one can see, there we have adequately selected simulations whose posterior distribution functions exhibit different behaviors with respect to the simulation boundaries. Our `mid` simulation is not ideal, but does at least demonstrate a clear peak in the posterior distribution, whereas the other simulations run into a wall at either side of the artificial boundaries.

We can also compute kernel density estimates to estimate the probability density.

```python
from scipy.stats import gaussian_kde
```

```python
alphas = np.arange(0.5, 3.0, 0.005)

left_kde  = gaussian_kde(left_samp[:, 5],  bw_method='silverman')
right_kde = gaussian_kde(right_samp[:, 5], bw_method='silverman')
mid_kde   = gaussian_kde(mid_samp[:, 5],   bw_method='silverman')
```

Which when visualized look like

```python
fig, ax = plt.subplots(1, 1, figsize=(4, 4))

ax.plot(alphas, left_kde(alphas),  c='#b22222')
ax.plot(alphas, right_kde(alphas), c='#555555')
ax.plot(alphas, mid_kde(alphas),   c='#1e90ff')
```

The kernel density estimate for our "normal" distribution (light blue) provides an adequate representation of the probability distribution that permits identification of a modal value. However, the two other kernel density estimates do not provide an adequate constraint on the probability distribution. They fail at the edges and therefore any confidence interval defined using the modal value as a "best estimate" is bound to provide a very questionable constraint on the true value.

So, how can we estimate Bayesian credible intervals, knowing that the true values for two of the simulations show above exist outside of the simulation boundaries?

One thought is to model the posterior probabilities as Gaussian to find a credible interval that is consistent with the observed distributions. That is, we assume the final probabilty distribution should be Gaussian such that

\begin{eqnarray}
    p(x | \theta) = \left\{
    \begin{array}{l l l}
        \left(2\pi\sigma^2\right)^{-1/2} \exp\left[ -\frac{(x - \theta)^2}{2 \sigma^2} \right] & & x > 0 \\
        0 & & x \le 0
    \end{array}
    \right.
\end{eqnarray}

We can then estimate the (log) probability for a give set of data $p({\bf X} | \theta)$ as

\begin{eqnarray}
    \log p({\bf X} | \theta) = \left\{
    \begin{array}{l l l}
        N \left(2\pi\sigma^2\right)^{-1/2} - \sum_{i = 1}^{N}\left(\frac{x_i - \theta}{2 \sigma}\right)^2 & & x > 0 \\
        0 & & x \le 0
    \end{array}
    \right.
\end{eqnarray}

To derive Bayesian credible intervals, we must integrate over the probability distribution between two values of the central value, $\theta_1$ and $\theta_2$, and set it equal to the desired credible interval, $f$.

\begin{equation}
    N\left(2\pi\sigma^2\right)^{-1/2} \int_{\theta_1}^{\theta_2} \exp\left[
            - \sum_i (x_i - \theta)^2/\sigma^2 \right] d\theta = f
\end{equation}

This has a solution involving the error function.

\begin{equation}
    N\left(2\pi\right)^{-1/2} \sum_i \left[ \pi^{1/2} {\rm erf}\left( \frac{x_i - \theta_1}{2\sigma} \right)
        - {\rm erf}\left( \frac{x_i - \theta_2}{2\sigma} \right)\right] = f
\end{equation}

We can attempt to derive a credible interval under the assumption that the Gaussian probability is symmetric about the most likely value.

```python
from scipy.special import erf

N = left_samp.shape[0]
sigma = 0.1

values = 0.
for theta in np.arange(0.01, 1.0, 0.01):
    for theta_2 in (theta + np.arange(sigma, 5.*sigma, 0.05)):
        f = np.sum(np.pi**0.5*erf((left_samp[:, 5] - theta)/(2.*sigma)) - erf((left_samp[:, 5] - theta_2)/(2.*sigma)))
        #f = f*(2.*np.pi)**(-N/2)
        try:
            values = np.vstack((values, np.array([theta, theta_2, f])))
        except:
            values = np.array([theta, theta_2, f])
np.savetxt('data/credible_intervals.txt', values, fmt='%12.6e')
```

Explore how values of $f$ look as a funciton of $\theta$ and $\theta_2$.

```python
fig, ax = plt.subplots(1, 2, figsize=(8., 4.))

ax[0].plot(values[:, 0], values[:, 2]/30000., 'o', markersize=2.0, c='#555555', alpha=0.5)
ax[1].plot(values[:, 1], values[:, 2]/30000., 'o', markersize=2.0, c='#555555', alpha=0.5)
```

Let's instead aim for a 3D visualization.

```python
from mpl_toolkits.mplot3d import Axes3D
```

```python
fig = plt.figure()
ax = Axes3D(fig)

ax.plot_trisurf(values[:, 0], values[:, 1], values[:, 2])
```

---

## Alternative Approach: Sampling the KDE

Start with a simple random sampling from the sample used to compute KDEs. However, let us weight each point by a probability.

```python
Fbol_obs = np.log10(17.18340) - 8.0
Fbol_sig = np.abs(Fbol_obs - (np.log10(17.18340 + 0.10591) - 8.0))
Theta_obs = 1.030
Theta_sig = 0.007
FeH_obs = -0.27
FeH_sig = 0.1

PFbol = -0.5*np.log(2.0*np.pi*Fbol_sig**2) - 0.5*((mid_samp[:, 7] - Fbol_obs)/Fbol_sig)**2
PTheta = -0.5*np.log(2.0*np.pi*Theta_sig**2) - 0.5*((mid_samp[:, 8] - Theta_obs)/Theta_sig)**2
PFeH = -0.5*np.log(2.0*np.pi*FeH_sig**2) - 0.5*((mid_samp[:, 1] - FeH_obs)/FeH_sig)**2

logProb = PFbol + PTheta + PFeH # log(prob) that a given point matches observations

logProb = np.log(np.exp(logProb)/np.sum(np.exp(logProb)))
```

```python
np.random.seed()
rand_samp = np.random.choice(mid_samp[:, 5], size=10000, p=np.exp(logProb)) # weight by prob.
```

Look at the resulting distribution.

```python
fig, ax = plt.subplots(1, 1, figsize=(4, 4))

tmp = ax.hist(rand_samp, bins=25, alpha=0.5)
```

This provides a rough outline of the original sample. However, values that lead to improbable fits to stellar parameters are suppressed, leading to a more definite peak in the $\alpha$ distribution. While potentially obvious to many others, this clearly demonstrates, to me, the power of computing probabilities and then drawing random samples to form a new probability distribution. One can then apply a Gaussian kernel density estimate on the resulting distribution.

```python
kde = gaussian_kde(rand_samp, bw_method='silverman')
alpha_prob = kde(alphas)/np.sum(kde(alphas))
```

```python
fig, ax = plt.subplots(1, 1, figsize=(4, 4))

ax.plot(alphas, alpha_prob, '-', lw=3, alpha=0.5)
```

```python
alphas[np.argmax(alpha_prob)]
```

    0.92000000000000037

```python
from scipy.stats import bayes_mvs

bayes_mvs(rand_samp, alpha=0.5)   # credible intervals on the mean, variance, and standard deviation
```

    ((0.90385677011299992, (0.90332541764265173, 0.90438812258334811)),
     (0.0062060418422675585, (0.0061468441125431539, 0.006265239571991963)),
     (0.078778435134670949, (0.078402712199687502, 0.079154158069654396)))

```python

```


# 0. Outline

1. RNN

2. BP for RNN

3. LSTM

4. Applications

# 1. RNN

### 1. Recall: Feedforward Neural Network

$$
\begin{align}
z &= \sum_{i=1}^{n}w_{i}x_{i}+b \\
a &= g(z)
\end{align}
$$

### 2. Recall: Language Modelling

A language model estimates probabilities of sequence of words:

- For the sequence $\color{MediumOrchid}{w_{1}, w_{2}, \cdots, w_{n}}$，using the chain rule, we have:

$$
\color{MediumOrchid}{P\left(w_{1}, \ldots, w_{n}\right)=P\left(w_{n} \mid w_{1}, \ldots, w_{n-1}\right) P\left(w_{n-1} \mid w_{1}, \ldots, w_{n-2}\right) \ldots P\left(w_{2} \mid w_{1}\right) P\left(w_{1}\right)}
$$

- N-Gram Approximation: $\color{MediumOrchid}{P\left(w_{1}, \ldots, w_{n}\right)=\prod_{i=1}^{n} P\left(w_{i} \mid w_{i-N+1}, \ldots, w_{i-1}\right)}$

- Applications:
    - Machine Translation: $\color{MediumOrchid}{P(\text { the cat is small })>P(\text { small is the cat })}$

    - Grammar Checking: $\color{MediumOrchid}{P(\text { He graduated from SJTU. })>P(\text { He graduated on SJTU.) }}$

Context is import in modelling context,But:

- N-Gram models use only limited context;

- They failed to handling long(>N) dependencies which is common seen in NL.

RNNs Could address the above issues:

- It includes the previous hidden layer as the input;

- It could model dependencies of arbitrary length.

### 3. Why RNN？

**通用近似定理(Universal Approximation Theorem)**

一个包含足够多隐含层神经元的多层前馈网络，能以任意精度逼近任意预定的连续函数。

### 4. Simple RNN Architecture

$$
\begin{align}
h^{(t)} &= \sigma\left(Wx^{(t)} + Uh^{(t-1)} \right) \\
y^{(t)} &= Softmax  \left(Vh^{(t)} \right)
\end{align}
$$

语言模型的评价指标：

1. 困惑度（Perplexity，ppl）

2. 词错误率（Word Error Rate, WER）

### 5. Unfolding RNNs

A Full RNN use all previous time steps:

#### 5.1 Backprop of $V$

设Loss function:

$$L=\sum_{t=1}^{T} \ell\left(y^{(t)}, \hat{y}^{(t)}\right)$$

令 $\color{MediumOrchid}{z = Vh}$，则损失函数对 $\color{MediumOrchid}{V}$的偏导数可表示为:

$$
\color{MediumOrchid}{\frac{\partial \ell^{(t)}}{\partial V}=\frac{\partial \ell^{(t)}}{\partial y^{(t)}} \cdot \frac{\partial y^{(t)}}{\partial z^{(t)}} \cdot \frac{\partial z^{(t)}}{\partial V}}
$$

#### 5.2 Backprop through Time（BPTT）

$$
\begin{align}
\frac{\partial \ell^{(t)}}{\partial W} &= \frac{\partial \ell^{(t)}}{\partial y^{(t)}} \cdot \frac{\partial y^{(t)}}{\partial h^{(t)}} \cdot \frac{\partial h^{(t)}}{\partial W}\\
&+ \frac{\partial \ell^{(t)}}{\partial y^{(t)}} \cdot \frac{\partial y^{(t)}}{\partial h^{(t)}} \cdot \frac{\partial h^{(t)}}{\partial h^{(t-1)}} \cdot \frac{\partial h^{(t-1)}}{\partial W} \\
&+ \frac{\partial \ell^{(t)}}{\partial y^{(t)}} \cdot \frac{\partial y^{(t)}}{\partial h^{(t)}} \cdot \frac{\partial h^{(t)}}{\partial h^{(t-1)}} \cdot \frac{\partial h^{(t-1)}}{\partial h^{(t-2)}} \cdot \frac{\partial h^{(t-2)}}{\partial W} \\
&+  \cdots
\end{align}
$$

In sum:

$$
\frac{\partial \ell^{(t)}}{\partial W} = \sum_{k=0}^{t} \frac{\partial \ell^{(t)}}{\partial y^{(t)}} \cdot \frac{\partial y^{(t)}}{\partial h^{(t)}} \left(\prod_{j=k+1}^{t}\frac{ \partial h^{(j)}}{\partial h^{(j-1)}} \right) \cdot \frac{\partial h^{(k)}}{\partial W}
$$

#### 5.3 Real-Time Recurrent Learning（RTRL）

$$
\frac{\partial \ell^{(t)}}{\partial U} = \frac{\partial \ell^{(t)}}{\partial h^{(t)}} \cdot \frac{\partial h^{(t)}}{\partial U}
$$

令

$$
\begin{align}
h^{(t)} &= f\left(z^{(t)}\right) \\
&= f\left(Uh^{(t-1)} + Wx^{(t)}\right)
\end{align}
$$

则

$$
\begin{align}
\frac{\partial h^{(t)}}{\partial U} &= \frac{\partial h^{(t)}}{\partial z^{(t)}} \left(\frac{\partial z^{(t)}}{\partial U} + U\frac{\partial h^{(t-1)}}{\partial U} \right) \\
&= f^{\prime} \left( z^{(t)} \right) \odot \left(h^{(t-1)} + U\frac{\partial h^{(t-1)}}{\partial U} \right)
\end{align}
$$

### 6. Vanishing Gradients

$$
\frac{\partial \ell^{(t)}}{\partial W} = \sum_{k=0}^{t} \frac{\partial \ell^{(t)}}{\partial y^{(t)}} \cdot \frac{\partial y^{(t)}}{\partial h^{(t)}} \left( \color{MediumOrchid}{\prod_{j=k+1}^{t}\frac{ \partial h^{(j)}}{\partial h^{(j-1)}}} \right) \cdot \frac{\partial h^{(k)}}{\partial W}
$$

# 3. LSTM

### 3.1 原始LSTM

#### 遗忘门

$$
f_{t} = \sigma\left( W_{f} \cdot \left[h_{t-1}, x_{t}\right] + b_{f} \right)
$$

#### 输入门

$$
\begin{align}
i_{t} &= \sigma \left(W_{i} \cdot \left[h_{t-1}, x_{t}\right] + b_{i}\right) \\
\tilde{C}_{t} &= \tanh \left(W_{C} \cdot \left[h_{t-1}, x_{t}\right] + b_{C}\right) \\
C_{t} &= f_{t} * C_{t-1} + i_{t} * \tilde{C}_{t}
\end{align}
$$

#### 输出门

$$
\begin{align}
o_{t} &= \sigma \left(W_{o} \cdot \left[h_{t-1}, x_{t}\right] + b_{o}\right) \\
h_{t} &= o_{t} * \tanh \left(C_{t}\right)
\end{align}
$$

### 3.2 变种1: adding peephold

$$
\begin{aligned}
f_{t} &=\sigma\left(W_{f} \cdot\left[\color{red}{{C}_{t-1}}, h_{t-1}, x_{t}\right]+b_{f}\right) \\
i_{t} &=\sigma\left(W_{i} \cdot\left[\color{red}{{C}_{t-1}}, h_{t-1}, x_{t}\right]+b_{i}\right) \\
o_{t} &=\sigma\left(W_{o} \cdot\left[\color{red}{{C}_{t}}, h_{t-1}, x_{t}\right]+b_{o}\right)
\end{aligned}
$$

### 3.3 变种2：GRU

$$
\begin{align}
z_{t} &= \sigma\left(W_{z} \cdot\left[h_{t-1}, x_{t}\right]\right) \\
r_{t} &= \sigma\left(W_{r} \cdot\left[h_{t-1}, x_{t}\right]\right) \\
\tilde{h}_{t} &= \tanh \left(W \cdot\left[r_{t} * h_{t-1}, x_{t}\right]\right) \\
h_{t} &= \left(1-z_{t}\right) * h_{t-1}+z_{t} * \tilde{h}_{t}
\end{align}
$$

# 4. Application

### 4.1 RNNs for Classification

### 4.2 POS Tagging

### 4.3 NER

### 4.3 Machine Translation-Encoder_Decoder Architecture

```python

```


```python
from sympy import init_printing; init_printing();
from IPython.core.interactiveshell import InteractiveShell
InteractiveShell.ast_node_interactivity = "all"
```

```python
from silkpy import ParametricSurface
from sympy import symbols, sin, cos, pi, cot, Array, refine, Q
from silkpy.sympy_utility import dot
u, v = symbols('u, v', real=True)
```

```python
surf_choice = 'torus'
if surf_choice=='cylindrical':
    R = symbols('R', positive=True)
    s = ParametricSurface([u, v], [R*cos(u), R*sin(u), v])
elif surf_choice=='cone':
    w = symbols('omega', real=True)
    s = ParametricSurface([u, v], [v*cos(u), v*sin(u), v*cot(w)])
elif surf_choice=='Mobius':
    theta = symbols('theta', real=True)
    s = ParametricSurface([theta, v],
        Array([cos(theta),                sin(theta),                0                ]) +
        Array([sin(theta/2) * cos(theta), sin(theta/2) * sin(theta), cos(theta/2)]) * v)
elif surf_choice=='torus':
    from sympy import Q, ask
    from sympy.assumptions import global_assumptions
    a, r = symbols('a, r', real=True, positive=True)
#     global_assumptions.add(Q.positive(a - r))
    global_assumptions.add(Q.positive(a + r*cos(u)))
    s = ParametricSurface([u, v], [ (a+r*cos(u)) * cos(v), (a+r*cos(u)) * sin(v), r*sin(u)])
```

```python
s.christoffel_symbol.tensor()
```

```python

```

```python
s.metric_tensor.tensor()
s.metric_tensor.change_config('uu').tensor()
s.christoffel_symbol.tensor()
r_u, r_v = s.expr().diff(u), s.expr().diff(v); r_u, r_v
a_, b_ = r_u, r_v
s.weingarten_matrix
```

```python
Wa = s.weingarten_transform(a_)
Wb = s.weingarten_transform(b_)
dot(Wa, b_), dot(a_, Wb)
s.K_H
s.prin_curvature_and_vector
from silkpy.sympy_utility import dot
(_, vec1), (_, vec2) = s.prin_curvature_and_vector
dot(vec1, vec2) # The two principal curvature vectors are perpendicular to each other.
```

```python
InteractiveShell.ast_node_interactivity = "last"
```

```python
from sympy import sin, cos, pi
from silkpy.numeric.surface.geodesic import geodesic_ncurve
theta = pi / 24 # symbols('theta', real=True)
t_arr, (u_arr, v_arr) = geodesic_ncurve(
    s.subs({a:5, r:2}), [pi/4, pi/4], [cos(theta), sin(theta)])
```

```python
from sympy import sin, cos, pi
from silkpy.numeric.surface.geodesic import geodesic_polar_ncoordinate
rho_arr, theta_arr, u_grid, v_grid = geodesic_polar_ncoordinate(
    s.subs({a:5, r:2}), [pi/4, pi/4], rho1=2.4, nrho=12, ntheta=48)

from silkpy.symbolic.geometry_map import lambdify
x_grid, y_grid, z_grid = lambdify(s.subs({a:5, r:2}))(u_grid, v_grid)
```

```python
from silkpy.symbolic.surface.draw import draw_surface_plotly
import plotly.graph_objects as go

if surf_choice=='cylindrical':
    R = 1.0
    s = ParametricSurface([u, v], [R*cos(u), R*sin(u), v])
elif surf_choice=='cone':
    w = float(pi) / 4
    s = ParametricSurface([u, v], [v*cos(u), v*sin(u), v*cot(w)] )
    fig = draw_surface_plotly(s, domain=[(-2*float(pi), 2*float(pi)), (4, 6)])
elif surf_choice=='torus':
    fig = draw_surface_plotly(s.subs({a: 5, r:2}), domain=[(-float(pi), float(pi)), (-float(pi), float(pi))])

# fig.add_trace(go.Scatter3d(
#     x=x_arr, y=y_arr, z=z_arr,
#     mode='lines',
#     line=dict(color=t_arr, width=2)
# ))

import numpy as np
for i in range(len(theta_arr)):
    fig.add_trace(go.Scatter3d(
        x=x_grid[:, i],
        y=y_grid[:, i],
        z=z_grid[:, i],
        mode='lines',
        line=dict(color=rho_arr, width=2)
    ))

for i in range(len(rho_arr)):
    fig.add_trace(go.Scatter3d(
        x=np.r_[x_grid[i,:], x_grid[i,:]],
        y=np.r_[y_grid[i,:], y_grid[i,:]],
        z=np.r_[z_grid[i,:], z_grid[i,:]],
        mode='lines',
        line=dict(color=rho_arr[i], width=2)
    ))
fig.show()
```

## Not yet done

```python
from sympy import series, Eq
t0 = symbols('t_0', real=True)
```

```python
t0 = 0
exprs[0].subs(t, t0) + (t-t0) * exprs[0].diff(t, 1).subs(t, t0)
exprs[1].subs(t, t0) + (t-t0) * exprs[1].diff(t, 1).subs(t, t0)
```

```python
exprs[0].evalf(subs={t:0}) + exprs[0].diff(t, 1).evalf(subs={t:0})
```

```python
from sympy import Eq
import sympy.solvers.ode as ode
ode.systems.dsolve_system([
    Eq(linearized_exprs[0], 0),
    Eq(linearized_exprs[1], 0)], funcs=[u1, u2])
```

```python

```

```python
def curvature_curve(surface):
    from sympy import Matrix, Array, Eq
    from sympy import Function, symbols
    import sympy.solvers.ode as ode
    t = symbols('t', real=True)
    # u1, u2 = symbols('u1, u2', real=True, cls=Function)
    u1 = Function(surface.sym(0), real=True)(t)
    u2 = Function(surface.sym(1), real=True)(t)
    curvature_curve_mat = Matrix([
        [u1.diff(t)**2, -u1.diff(t) * u2.diff(t), u2.diff(t)**2],
        Array(surface.E_F_G).subs(surface.sym(0), u1),
        Array(surface.L_M_N).subs(surface.sym(1), u2)])
    # typically there would be two solutions
    sol_with_u1_equal_t = ode.systems.dsolve_system(
        [Eq(curvature_curve_mat.det(), 0 ), Eq(u1.diff(t), 1)])[0]
    sol_with_u2_equal_t = ode.systems.dsolve_system(
        [Eq(curvature_curve_mat.det(), 0 ), Eq(u2.diff(t), 1)])[0]
    return [sol_with_u1_equal_t, sol_with_u2_equal_t]
```

```python
curvature_curve(s)
```

```python

```