# Learning in LTN

This tutorial explains the concept of learning in Logic Tensor Networks.

In particular, it explains how to learn some language symbols (predicates, functions, constants) using the satisfaction
of a knowledge base as an objective.

For those unfamiliar with logic, a knowledge base is a container of logical formulas. Satisfying a knowledge base
as an objective means find a solution which maximize the satisfaction of all the formulas contained in the knowledge base.
In other words, we would find a representation for predicates, functions, and constants which increase the truth degree
of the formulas in the knowledge base.

This tutorial expects basic familiarity of the first two tutorials on LTN (grounding of symbols and connectives).

```python
import torch
import numpy as np
import ltn
import matplotlib.pyplot as plt
```

## Classification with Nearest Neighbour

We use the following simple example to illustrate learning in LTN.

The domain is the square $[0,4] \times [0,4]$. We have some points in this domain, and we need to infer their classes.
In particular, we know the classes of only two examples. One example is of class $A$ and the other of class $B$.
The rest of the points are not labelled, but there are two assumptions:
- $A$ and $B$ are mutually exclusive,
- any two close points should share the same label.

In the following, we plot our dataset made of 19 points. We distinguish between classified and not classified examples.

```python
points = np.array(
        [[0.4,0.3], [1.2,0.3], [2.2,1.3], [1.7,1.0], [0.5,0.5], [0.3, 1.5], [1.3, 1.1], [0.9, 1.7],
        [3.4,3.3], [3.2,3.3], [3.2,2.3], [2.7,2.0], [3.5,3.5], [3.3, 2.5], [3.3, 1.1], [1.9, 3.7], [1.3, 3.5],
        [3.3, 1.1],[3.9, 3.7]])
point_a = [3.3, 2.5]
point_b = [1.3, 1.1]
fig, ax = plt.subplots()
ax.set_xlim(0, 4)
ax.set_ylim(0, 4)
ax.scatter(points[:,0], points[:,1], color="black", label="unknown")
ax.scatter(point_a[0], point_a[1], color="blue", label="a")
ax.scatter(point_b[0], point_b[1], color="red", label="b")
ax.set_title("Dataset of individuals")
plt.legend();
```

## Definition of knowledge base

First, we need to define a membership predicate $C(x,l)$, where $x$ is an individual (point) and $l$ is a one-hot label
denoting one of the two classes (10 for class $A$ and 01 for class $B$).

$C$ is approximated by a simple MLP (Multi-Layer Perceptron). The last layer, that computes probabilities per class, uses a `softmax` activation, ensuring that the classes are mutually-exclusive.

We define the knowledge base $\mathcal{K}$ with the following rules:
\begin{align}
& C(a,l_a)\\
& C(b,l_b)\\
\forall x_1,x_2,l\ \big(\mathrm{Sim}(x_1,x_2) & \rightarrow \big(C(x_1,l)\leftrightarrow C(x_2,l)\big)\big)
\end{align}

where:
- $a$ and $b$ the two individuals already classified;
- $x_1$ and $x_2$ are variables ranging over all individuals;
- $l_a$ and $l_b$ are the one-hot labels for $A$ and $B$;
- $l$ is a variable ranging over the labels;
- $\mathrm{Sim}$ is a predicate measuring similarity between two points and defined as $\mathcal{G}(\mathrm{Sim}):\vec{u},\vec{v}\mapsto \exp(-\|\vec{u}-\vec{v} \|^2)$.

The objective is to learn the predicate $C$ in such a way to maximize the satisfaction of $\mathcal{K}$.

If $\theta$ denotes the set of trainable parameters, the task is:
\begin{align}
\theta^\ast = \mathrm{argmax}_{\theta\in\Theta}\ \mathrm{SatAgg}_{\phi\in\mathcal{K}} \text{ } \mathcal{G}_{\theta}(\phi)
\end{align}

where $\mathrm{SatAgg}$ is an operator that aggregates the truth values of the formulas in $\mathcal{K}$
(if there are more than one formula).

In order to define the knowledge base in LTN, we need to define our predicates, variables, and constants first.

In the
following, the names of predicates, variables, and constants are the same used in the problem formulation above.

```python
# Predicate C
class ModelC(torch.nn.Module):
    def __init__(self):
        super(ModelC, self).__init__()
        self.elu = torch.nn.ELU()
        self.softmax = torch.nn.Softmax(dim=1)
        self.dense1 = torch.nn.Linear(2, 5)
        self.dense2 = torch.nn.Linear(5, 5)
        self.dense3 = torch.nn.Linear(5, 2)

    def forward(self, x, l):
        """x: point, l: one-hot label"""
        x = self.elu(self.dense1(x))
        x = self.elu(self.dense2(x))
        prob = self.softmax(self.dense3(x))
        return torch.sum(prob * l, dim=1)

C = ltn.Predicate(ModelC())

# Predicate Sim
Sim = ltn.Predicate(func=lambda u, v: torch.exp(-1. * torch.sqrt(torch.sum(torch.square(u - v), dim=1))))
```

```python
# variables and constants
x1 = ltn.Variable("x1", torch.tensor(points))
x2 = ltn.Variable("x2", torch.tensor(points))
a = ltn.Constant(torch.tensor([3.3, 2.5]))
b = ltn.Constant(torch.tensor([1.3, 1.1]))
l_a = ltn.Constant(torch.tensor([1, 0]))
l_b = ltn.Constant(torch.tensor([0, 1]))
l = ltn.Variable("l", torch.tensor([[1, 0], [0, 1]]))
```

In the following, we plot the similarity of each point in the dataset with the point $a$. Lower the similarity, thinner the line
which connect the points.

```python
similarities_to_a = Sim(x1, a).value
fig, ax = plt.subplots()
ax.set_xlim(0, 4)
ax.set_ylim(0, 4)
ax.scatter(points[:,0], points[:,1], color="black")
ax.scatter(a.value[0], a.value[1], color="blue")
ax.set_title("Illustrating the similarities of each point to a")
for i, sim_to_a in enumerate(similarities_to_a):
    plt.plot([points[i,0], a.value[0]], [points[i,1],a.value[1]], alpha=float(sim_to_a.numpy()), color="blue")
```

For the connectives, we use the stable "product configuration" explained in the complementary notebook (2b-grounding-connectives.ipynb).

In particular, the operator for equivalence ($p \leftrightarrow q$) is implemented in LTN as $(p \rightarrow q)\land(q \rightarrow p)$, using
one operator for conjunction and one operator for implication. Such operator is available at `ltn.fuzzy_ops.Equiv`.

```python
Not = ltn.Connective(ltn.fuzzy_ops.NotStandard())
And = ltn.Connective(ltn.fuzzy_ops.AndProd())
Or = ltn.Connective(ltn.fuzzy_ops.OrProbSum())
Implies = ltn.Connective(ltn.fuzzy_ops.ImpliesReichenbach())
Equiv = ltn.Connective(ltn.fuzzy_ops.Equiv(ltn.fuzzy_ops.AndProd(), ltn.fuzzy_ops.ImpliesReichenbach()))
Forall = ltn.Quantifier(ltn.fuzzy_ops.AggregPMeanError(p=2), quantifier="f")
Exists = ltn.Quantifier(ltn.fuzzy_ops.AggregPMean(p=6), quantifier="e")
```

Now that we have defined our predicates, variables, and constants, we are ready for defining the knowledge base.

If there are several closed formulas in $\mathcal{K}$, their truth values need to be aggregated, and this is what the $\mathrm{SatAgg}$
operator does. Currently, $\mathrm{SatAgg}$ supports only closed formulas.

To be clear, a closed formula is a formula which does not have free variables appearing in it,
namely all the variables are quantified.

In LTN, the $\mathrm{SatAgg}$ operator is available at `ltn.fuzzy_ops.SatAgg`. The constructor `SatAgg()` takes as input
an aggregation operator, which is then used to aggregate the inputs when the operator is invoked. Specifically, the operator takes as
input a list of truth values of closed formulas and returns the aggregation of these values using the selected aggregator.

As `SatAgg` aggregator, we recommend using the generalized mean inspired operator `pMeanError`, already used to implement $\forall$. For this reason, the constructor
`SatAgg()` defines the operator using the `pMeanError`.
The hyper-parameter $p$ again allows flexibility in how strict the formula aggregation is ($p = 1$ corresponds to `mean`; $p \to +\inf$ corresponds to `min`).

In the following, we define the `SatAgg` operator and a training loop to learn our LTN model.

As shown, the `SatAgg` operator takes as input the formulas in our knowledge base and returns a truth value which is
interpreted as the satisfaction level of the entire knowledge base. Since we want to maximize this number, we need to
minimize $1 - SatAgg$, using gradient descent.

Notice that it is not needed to access the `value` attribute before giving the formulas to the `SatAgg` operator. This is because
the operator accepts `LTNObject` instances in input.

In the forward phase of the LTN, the truth values of the three formulas are computed, while in the backward phase the weights
of predicate $C$ are changed in such a way to minimize the loss function.

```python
# by default, SatAgg uses the pMeanError
sat_agg = ltn.fuzzy_ops.SatAgg()

# we need to learn the parameters of the predicate C
optimizer = torch.optim.Adam(C.parameters(), lr=0.001)

for epoch in range(2000):
    optimizer.zero_grad()
    loss = 1. - sat_agg(
        C(a, l_a),
        C(b, l_b),
        Forall([x1, x2, l], Implies(Sim(x1, x2), Equiv(C(x1, l), C(x2, l))))
    )
    loss.backward()
    optimizer.step()
    if epoch%200 == 0:
        print("Epoch %d: Sat Level %.3f "%(epoch, 1 - loss.item()))
print("Training finished at Epoch %d with Sat Level %.3f" %(epoch, 1 - loss.item()))
```

    Epoch 0: Sat Level 0.604
    Epoch 200: Sat Level 0.787
    Epoch 400: Sat Level 0.942
    Epoch 600: Sat Level 0.956
    Epoch 800: Sat Level 0.958
    Epoch 1000: Sat Level 0.959
    Epoch 1200: Sat Level 0.959
    Epoch 1400: Sat Level 0.959
    Epoch 1600: Sat Level 0.959
    Epoch 1800: Sat Level 0.959
    Training finished at Epoch 1999 with Sat Level 0.959

After a few epochs, the system has learned to identify samples close to the point $a$ (resp. $b$) as belonging to class
$A$ (resp. $B$) based on the rules of the knowledge base.

In the following, we draw a plot showing that our LTN has been able to learn how to classify
the data points using the satisfaction of a knowledge base as an objective. The colour is brighter when predicate
$C$ is more satisfied. On the left, we see that $C$ correctly classifies points of class $A$, while on
the right we see it correctly classifies points of class $B$.

```python
fig = plt.figure(figsize=(10, 3))
fig.add_subplot(1, 2, 1)
plt.scatter(x1.value[:, 0], x1.value[:, 1], c=C(x1, l_a).value.detach().numpy(), vmin=0, vmax=1)
plt.title("C(x,l_a)")
plt.colorbar()
fig.add_subplot(1, 2, 2)
plt.scatter(x1.value[:, 0], x1.value[:, 1], c=C(x1, l_b).value.detach().numpy(), vmin=0, vmax=1)
plt.title("C(x,l_b)")
plt.colorbar()
plt.show();
```

## Special Cases

### Variables grounded by batch

Usually, in the majority of learning tasks, we work with batches of data to make learning efficient.

In LTN, working with batches of data is straightforward. It is enough to ground the variables with different values
at each training step. In PyTorch, these values are usually returned by a `DataLoader`.

Let's see the same example with a bigger dataset (10000 points) which requires subdividing it in mini-batches.

In the following, we plot the dataset. The dataset has been randomly generated. Points $a$ and $b$ have been selected
in such a way they are enough distant between each other.

```python
r1 = 0
r2 = 4
points = (r1 - r2) * torch.rand((10000, 2)) + r2
points[-1] = torch.tensor([3., 3.])
points[-2] = torch.tensor([1., 1.])
points_a = torch.tensor([3., 3.])
points_b = torch.tensor([1., 1.])
a = ltn.Constant(torch.tensor([3., 3.]))
b = ltn.Constant(torch.tensor([1., 1.]))

fig, ax = plt.subplots()
ax.set_xlim(0, 4)
ax.set_ylim(0, 4)
ax.scatter(points[:,0], points[:,1], color="black", label="unknown")
ax.scatter(point_a[0], point_a[1], color="blue", label="a")
ax.scatter(point_b[0], point_b[1], color="red", label="b")
ax.set_title("Dataset of individuals")
plt.legend();
```

In the following, we define a data loader which takes as input the entire dataset and returns batches of data points
fetched from the dataset. It is possible to decide a batch size and whether to shuffle data.

Then, as anticipated before, it is enough to add some lines of code that iterate over the batches and ground the variables
with the new data points contained in the batches.

```python
# we define C again to re-initialize its weights
C = ltn.Predicate(ModelC())

# data loader which creates the batches
class DataLoader:
    def __init__(self,
                 dataset,
                 batch_size=1,
                 shuffle=True):
        self.data = dataset
        self.batch_size = batch_size
        self.shuffle = shuffle

    def __len__(self):
        return int(np.ceil(self.data.shape[0] / self.batch_size))

    def __iter__(self):
        n = self.data.shape[0]
        idxlist = list(range(n))
        if self.shuffle:
            np.random.shuffle(idxlist)

        for _, start_idx in enumerate(range(0, n, self.batch_size)):
            end_idx = min(start_idx + self.batch_size, n)
            batch_points = self.data[idxlist[start_idx:end_idx]]

            yield batch_points

train_loader = DataLoader(points, 512)

# by default, SatAgg uses the pMeanError
sat_agg = ltn.fuzzy_ops.SatAgg()

# we need to learn the parameters of the predicate C
optimizer = torch.optim.Adam(C.parameters(), lr=0.001)

for epoch in range(100):
    for (batch_idx, (batch_points)) in enumerate(train_loader):
        x1 = ltn.Variable("x1", batch_points)
        x2 = ltn.Variable("x2", batch_points)
        optimizer.zero_grad()
        loss = 1. - sat_agg(
            C(a, l_a),
            C(b, l_b),
            Forall([x1, x2, l], Implies(Sim(x1, x2), Equiv(C(x1, l), C(x2, l))))
        )
        loss.backward()
        optimizer.step()
    if epoch%10 == 0:
        print("Epoch %d: Sat Level %.3f "%(epoch, 1 - loss.item()))

print("Training finished at Epoch %d with Sat Level %.3f" %(epoch, 1 - loss.item()))
```

    Epoch 0: Sat Level 0.580
    Epoch 10: Sat Level 0.890
    Epoch 20: Sat Level 0.948
    Epoch 30: Sat Level 0.953
    Epoch 40: Sat Level 0.950
    Epoch 50: Sat Level 0.945
    Epoch 60: Sat Level 0.943
    Epoch 70: Sat Level 0.949
    Epoch 80: Sat Level 0.947
    Epoch 90: Sat Level 0.947
    Training finished at Epoch 99 with Sat Level 0.945

It is possible to observe that just after 20 epochs, LTN has learned to correctly classify the examples.

Let's see some plot as we did for the previous example.

As shown, LTN has learned how to correctly classify the data points. Also, it is possible to observe that there are some points
in the middle which are more difficult to classify. This is intuitive since their distance from points $a$ and $b$ is similar, so LTN
does not know which is the correct class to assign to them.

```python
x1 = ltn.Variable("x1", points)
x2 = ltn.Variable("x2", points)
fig = plt.figure(figsize=(10, 3))
fig.add_subplot(1, 2, 1)
plt.scatter(x1.value[:, 0], x1.value[:, 1], c=C(x1, l_a).value.detach().numpy(), vmin=0, vmax=1)
plt.title("C(x,l_a)")
plt.colorbar()
fig.add_subplot(1, 2, 2)
plt.scatter(x1.value[:, 0], x1.value[:, 1], c=C(x1, l_b).value.detach().numpy(), vmin=0, vmax=1)
plt.title("C(x,l_b)")
plt.colorbar()
plt.show();
```


# Inferential Statistics Ia - Frequentism

## Learning objectives

Welcome to the first Frequentist inference mini-project! Over the course of working on this mini-project and the next frequentist mini-project, you'll learn the fundamental concepts associated with frequentist inference. The following list includes the topics you will become familiar with as you work through these two mini-projects:
* the _z_-statistic
* the _t_-statistic
* the difference and relationship between the two
* the Central Limit Theorem, including its assumptions and consequences
* how to estimate the population mean and standard deviation from a sample
* the concept of a sampling distribution of a test statistic, particularly for the mean
* how to combine these concepts to calculate a confidence interval

## Prerequisites

For working through this notebook, you are expected to have a very basic understanding of:
* what a random variable is
* what a probability density function (pdf) is
* what the cumulative density function is
* a high-level sense of what the Normal distribution

If these concepts are new to you, please take a few moments to Google these topics in order to get a sense of what they are and how you might use them.

While it's great if you have previous knowledge about sampling distributions, this assignment will introduce the concept and set you up to practice working using sampling distributions. This notebook was designed to bridge the gap between having a basic understanding of probability and random variables and being able to apply these concepts in Python. The second frequentist inference mini-project focuses on a real-world application of this type of inference to give you further practice using these concepts.

For this notebook, we will use data sampled from a known normal distribution. This allows us to compare our results with theoretical expectations.

## I An introduction to sampling from the Normal distribution

First, let's explore the ways we can generate the Normal distribution. While there's a fair amount of interest in [sklearn](https://scikit-learn.org/stable/) within the machine learning community, you're likely to have heard of [scipy](https://docs.scipy.org/doc/scipy-0.15.1/reference/index.html) if you're coming from the sciences. For this assignment, you'll use [scipy.stats](https://docs.scipy.org/doc/scipy-0.15.1/reference/tutorial/stats.html) to complete your work.

```python
from scipy.stats import norm
from scipy.stats import t
import numpy as np
import pandas as pd
from numpy.random import seed
import matplotlib.pyplot as plt
```

__Q:__ Call up the documentation for the `norm` function imported above. What is the second listed method?

```python

```

__A:__ Probability Density Function

__Q:__ Use the method that generates random variates to draw five samples from the standard normal distribution.

__A:__

```python
seed(47)
# draw five samples here
sample= norm.rvs(size=5)
sample
```

    array([-0.84800948,  1.30590636,  0.92420797,  0.6404118 , -1.05473698])

__Q:__ What is the mean of this sample? Is it exactly equal to the value you expected? Hint: the sample was drawn from the standard normal distribution.

__A:__

```python
# Calculate and print the mean here, hint: use np.mean()
print(sample.mean())
```

    0.19355593334131074

__Q:__ What is the standard deviation of these numbers? Calculate this manually here as $\sqrt{\frac{\sum_i(x_i - \bar{x})^2}{n}}$. Hint: np.sqrt() and np.sum() will be useful here and remember that numpy supports [broadcasting](https://docs.scipy.org/doc/numpy/user/basics.broadcasting.html).

__A:__

```python
np.sqrt(((sample-sample.mean())**2).sum()/len(sample))
```

    0.9606195639478641

Here we have calculated the actual standard deviation of a small (size 5) data set. But in this case, this small data set is actually a sample from our larger (infinite) population. In this case, the population is infinite because we could keep drawing our normal random variates until our computers die. In general, the sample mean we calculate will not be equal to the population mean (as we saw above). A consequence of this is that the sum of squares of the deviations from the _population_ mean will be bigger than the sum of squares of the deviations from the _sample_ mean. In other words, the sum of squares of the deviations from the _sample_ mean is too small to give an unbiased estimate of the _population_ variance. An example of this effect is given [here](https://en.wikipedia.org/wiki/Bessel%27s_correction#Source_of_bias). Scaling our estimate of the variance by the factor $n/(n-1)$ gives an unbiased estimator of the population variance. This factor is known as [Bessel's correction](https://en.wikipedia.org/wiki/Bessel%27s_correction). The consequence of this is that the $n$ in the denominator is replaced by $n-1$.

__Q:__ If all we had to go on was our five samples, what would be our best estimate of the population standard deviation? Use Bessel's correction ($n-1$ in the denominator), thus $\sqrt{\frac{\sum_i(x_i - \bar{x})^2}{n-1}}$.

__A:__

```python
np.sqrt(((sample-sample.mean())**2).sum()/(len(sample)-1))
```

    1.0740053227518152

__Q:__ Now use numpy's std function to calculate the standard deviation of our random samples. Which of the above standard deviations did it return?

__A:__

```python
sample.std()
```

    0.9606195639478641

__Q:__ Consult the documentation for np.std() to see how to apply the correction for estimating the population parameter and verify this produces the expected result.

__A:__

```python
sample.std(ddof=1)
```

    1.0740053227518152

```python
np.sqrt(((sample-sample.mean())**2).sum()/(len(sample)-1)) == sample.std(ddof=1)
```

    True

### Summary of section

In this section, you've been introduced to the scipy.stats package and used it to draw a small sample from the standard normal distribution. You've calculated the average (the mean) of this sample and seen that this is not exactly equal to the expected population parameter (which we know because we're generating the random variates from a specific, known distribution). You've been introduced to two ways of calculating the standard deviation; one uses $n$ in the denominator and the other uses $n-1$ (Bessel's correction). You've also seen which of these calculations np.std() performs by default and how to get it to generate the other.

You use $n$ as the denominator if you want to calculate the standard deviation of a sequence of numbers. You use $n-1$ if you are using this sequence of numbers to estimate the population parameter. This brings us to some terminology that can be a little confusing.

The population parameter is traditionally written as $\sigma$ and the sample statistic as $s$. Rather unhelpfully, $s$ is also called the sample standard deviation (using $n-1$) whereas the standard deviation of the sample uses $n$. That's right, we have the sample standard deviation and the standard deviation of the sample and they're not the same thing!

The sample standard deviation
\begin{equation}
s = \sqrt{\frac{\sum_i(x_i - \bar{x})^2}{n-1}} \approx \sigma,
\end{equation}
is our best (unbiased) estimate of the population parameter ($\sigma$).

If your data set _is_ your entire population, you simply want to calculate the population parameter, $\sigma$, via
\begin{equation}
\sigma = \sqrt{\frac{\sum_i(x_i - \bar{x})^2}{n}}
\end{equation}
as you have complete, full knowledge of your population. In other words, your sample _is_ your population. It's worth noting at this point if your sample is your population then you know absolutely everything about your population, there are no probabilities really to calculate and no inference to be done.

If, however, you have sampled _from_ your population, you only have partial knowledge of the state of your population and the standard deviation of your sample is not an unbiased estimate of the standard deviation of the population, in which case you seek to estimate that population parameter via the sample standard deviation, which uses the $n-1$ denominator.

You're now firmly in frequentist theory territory. Great work so far! Now let's dive deeper.

## II Sampling distributions

So far we've been dealing with the concept of taking a sample from a population to infer the population parameters. One statistic we calculated for a sample was the mean. As our samples will be expected to vary from one draw to another, so will our sample statistics. If we were to perform repeat draws of size $n$ and calculate the mean of each, we would expect to obtain a distribution of values. This is the sampling distribution of the mean. The Central Limit Theorem (CLT) tells us that such a distribution will approach a normal distribution as $n$ increases. For the sampling distribution of the mean, the standard deviation of this distribution is given by

\begin{equation}
\sigma_{mean} = \frac{\sigma}{\sqrt n}
\end{equation}

where $\sigma_{mean}$ is the standard deviation of the sampling distribution of the mean and $\sigma$ is the standard deviation of the population (the population parameter).

This is important because typically we are dealing with samples from populations and all we know about the population is what we see in the sample. From this sample, we want to make inferences about the population. We may do this, for example, by looking at the histogram of the values and by calculating the mean and standard deviation (as estimates of the population parameters), and so we are intrinsically interested in how these quantities vary across samples. In other words, now that we've taken one sample of size $n$ and made some claims about the general population, what if we were to take another sample of size $n$? Would we get the same result? Would we make the same claims about the general population? This brings us to a fundamental question: _when we make some inference about a population based on our sample, how confident can we be that we've got it 'right'?_

Let's give our normal distribution a little flavor. Also, for didactic purposes, the standard normal distribution, with its variance equal to its standard deviation of one, would not be a great illustration of a key point. Let us imagine we live in a town of 50000 people and we know the height of everyone in this town. We will have 50000 numbers that tell us everything about our population. We'll simulate these numbers now and put ourselves in one particular town, called 'town 47', where the population mean height is 172 cm and population standard deviation is 5 cm.

```python
seed(47)
pop_heights = norm.rvs(172, 5, size=50000)
```

```python
_ = plt.hist(pop_heights, bins=30)
_ = plt.xlabel('height (cm)')
_ = plt.ylabel('number of people')
_ = plt.title('Distribution of heights in entire town population')
_ = plt.axvline(172, color='r')
_ = plt.axvline(172+5, color='r', linestyle='--')
_ = plt.axvline(172-5, color='r', linestyle='--')
_ = plt.axvline(172+10, color='r', linestyle='-.')
_ = plt.axvline(172-10, color='r', linestyle='-.')
```

Now, 50000 people is rather a lot to chase after with a tape measure. If all you want to know is the average height of the townsfolk, then can you just go out and measure a sample to get a pretty good estimate of the average height?

```python
def townsfolk_sampler(n):
    return np.random.choice(pop_heights, n)
```

Let's say you go out one day and randomly sample 10 people to measure.

```python
seed(47)
daily_sample1 = townsfolk_sampler(10)
```

```python
_ = plt.hist(daily_sample1, bins=10)
_ = plt.xlabel('height (cm)')
_ = plt.ylabel('number of people')
_ = plt.title('Distribution of heights in sample size 10')
```

The sample distribution doesn't look much like what we know (but wouldn't know in real-life) the population distribution looks like. What do we get for the mean?

```python

```

```python
np.mean(daily_sample1)
```

    173.47911444163503

And if we went out and repeated this experiment?

```python
daily_sample2 = townsfolk_sampler(10)
```

```python
np.mean(daily_sample2)
```

    173.7317666636263

__Q:__ Simulate performing this random trial every day for a year, calculating the mean of each daily sample of 10, and plot the resultant sampling distribution of the mean.

__A:__

```python

```

```python
seed(47)
# take your samples here
daily_mean = np.array([np.mean(townsfolk_sampler(10)) for i in range(365)])
```

```python
_ = plt.hist(daily_mean, bins= 10)
_ = plt.xlabel('Height (cm)')
_ = plt.ylabel('Number of people')
_ = plt.title('Distribution of mean heights(size 10)')
```

The above is the distribution of the means of samples of size 10 taken from our population. The Central Limit Theorem tells us the expected mean of this distribution will be equal to the population mean, and standard deviation will be $\sigma / \sqrt n$, which, in this case, should be approximately 1.58.

__Q:__ Verify the above results from the CLT.

__A:__

```python

```

```python

daily_mean.std(ddof=1)
```

    1.5778333114768472

Remember, in this instance, we knew our population parameters, that the average height really is 172 cm and the standard deviation is 5 cm, and we see some of our daily estimates of the population mean were as low as around 168 and some as high as 176.

__Q:__ Repeat the above year's worth of samples but for a sample size of 50 (perhaps you had a bigger budget for conducting surveys that year!) Would you expect your distribution of sample means to be wider (more variable) or narrower (more consistent)? Compare your resultant summary statistics to those predicted by the CLT.

__A:__

```python
seed(47)
# calculate daily means from the larger sample size here
daily_mean = np.array([np.mean(townsfolk_sampler(50)) for i in range(365)])

```

```python
_ = plt.hist(daily_mean, bins=10)
_ = plt.xlabel('Height (cm)')
_ = plt.ylabel('Number of people')
_ = plt.title('Distribution of mean heights of samples with size 10')
```

What we've seen so far, then, is that we can estimate population parameters from a sample from the population, and that samples have their own distributions. Furthermore, the larger the sample size, the narrower are those sampling distributions.

### III Normally testing times!

All of the above is well and good. We've been sampling from a population we know is normally distributed, we've come to understand when to use $n$ and when to use $n-1$ in the denominator to calculate the spread of a distribution, and we've  seen the Central Limit Theorem in action for a sampling distribution. All seems very well behaved in Frequentist land. But, well, why should we really care?

Remember, we rarely (if ever) actually know our population parameters but you still have to estimate them somehow. If we want to make inferences such as "is this observation unusual?" or "has my population mean changed?" then you need to have some idea of what the underlying distribution is so you can calculate relevant probabilities. In frequentist inference, you use the formulas above to deduce these population parameters. Take a moment in the next part of this assignment to refresh your understanding of how these probabilities work.

Recall some basic properties of the standard Normal distribution, such as about 68% of observations being within plus or minus 1 standard deviation of the mean.

__Q:__ Using this fact, calculate the probability of observing the value 1 or less in a single observation from the standard normal distribution. Hint: you may find it helpful to sketch the standard normal distribution (the familiar bell shape) and mark the number of standard deviations from the mean on the x-axis and shade the regions of the curve that contain certain percentages of the population.

__A:__ 1 - (1 - 0.68) / 2

Calculating this probability involved calculating the area under the pdf from the value of 1 and below. To put it another way, we need to integrate the pdf. We could just add together the known areas of chunks (from -Inf to 0 and then 0 to $+\sigma$ in the example above. One way to do this is using look up tables (literally). Fortunately, scipy has this functionality built in with the cdf() function.

__Q:__ Use the cdf() function to answer the question above again and verify you get the same answer.

__A:__

```python
norm.cdf(1)
```

    0.8413447460685429

__Q:__ Using our knowledge of the population parameters for our townsfolk's heights, what is the probability of selecting one person at random and their height being 177 cm or less? Calculate this using both of the approaches given above.

__A:__

```python
1 - (1 - 0.68) / 2
```

    0.8400000000000001

```python
norm(172,5).cdf(177)
```

    0.8413447460685429

__Q:__ Turning this question around. Let's say we randomly pick one person and measure their height and find they are 2.00 m tall? How surprised should we be at this result, given what we know about the population distribution? In other words, how likely would it be to obtain a value at least as extreme as this? Express this as a probability.

__A:__

```python
1 - norm(172, 5).cdf(200)
```

    1.0717590259723409e-08

We could calculate this probability by virtue of knowing the population parameters. We were then able to use the known properties of the relevant normal distribution to calculate the probability of observing a value at least as extreme as our test value. We have essentially just performed a z-test (albeit without having prespecified a threshold for our "level of surprise")!

We're about to come to a pinch, though here. We've said a couple of times that we rarely, if ever, know the true population parameters; we have to estimate them from our sample and we cannot even begin to estimate the standard deviation from a single observation. This is very true and usually we have sample sizes larger than one. This means we can calculate the mean of the sample as our best estimate of the population mean and the standard deviation as our best estimate of the population standard deviation. In other words, we are now coming to deal with the sampling distributions we mentioned above as we are generally concerned with the properties of the sample means we obtain.

Above, we highlighted one result from the CLT, whereby the sampling distribution (of the mean) becomes narrower and narrower with the square root of the sample size. We remind ourselves that another result from the CLT is that _even if the underlying population distribution is not normal, the sampling distribution will tend to become normal with sufficiently large sample size_. This is the key driver for us 'requiring' a certain sample size, for example you may frequently see a minimum sample size of 30 stated in many places. In reality this is simply a rule of thumb; if the underlying distribution is approximately normal then your sampling distribution will already be pretty normal, but if the underlying distribution is heavily skewed then you'd want to increase your sample size.

__Q:__ Let's now start from the position of knowing nothing about the heights of people in our town.
* Use our favorite random seed of 47, to randomly sample the heights of 50 townsfolk
* Estimate the population mean using np.mean
* Estimate the population standard deviation using np.std (remember which denominator to use!)
* Calculate the (95%) [margin of error](https://www.statisticshowto.datasciencecentral.com/probability-and-statistics/hypothesis-testing/margin-of-error/#WhatMofE) (use the exact critial z value to 2 decimal places - [look this up](https://www.statisticshowto.datasciencecentral.com/probability-and-statistics/find-critical-values/) or use norm.ppf())
* Calculate the 95% Confidence Interval of the mean
* Does this interval include the true population mean?

__A:__

```python
seed(47)
# take your sample now
sample = townsfolk_sampler(50)
```

```python
mean_calc = sample.mean()
print(mean_calc)
```

    172.7815108576788

```python
std_calc = sample.std(ddof=1)
print(std_calc)
```

    4.195424364433547

```python
norm(mean_calc, std_calc).ppf([0.025,0.975])
```

    array([164.5586302 , 181.00439151])

```python
n=50
norm(mean_calc, std_calc/np.sqrt(n)).ppf([0.025,0.975])
```

    array([171.61861992, 173.94440179])

__Q:__ Above we calculated the confidence interval using the critical z value. What is the problem with this? What requirement, or requirements, are we (strictly) failing?

__A:__  As we are not having proper knowledgr of the statistics data of the population, We are relying on methods like taking the sample from the population as of now to estimate the parameters.

__Q:__ Calculate the 95% confidence interval for the mean using the _t_ distribution. Is this wider or narrower than that based on the normal distribution above? If you're unsure, you may find this [resource](https://www.statisticshowto.datasciencecentral.com/probability-and-statistics/confidence-interval/) useful. For calculating the critical value, remember how you could calculate this for the normal distribution using norm.ppf().

__A:__

```python
df = 49
critical_val= t(df).ppf([0.025, 0.975])
critical_val* std_calc/np.sqrt(n) + mean_calc
```

    array([171.58918445, 173.97383727])

```python

```

```python

```

This is slightly wider than the previous confidence interval. This reflects the greater uncertainty given that we are estimating population parameters from a sample.

## Learning outcomes

Having completed this project notebook, you now have hands-on experience:
* sampling and calculating probabilities from a normal distribution
* the correct way to estimate the standard deviation of a population (the population parameter) from a sample
* what a sampling distribution is and how the Central Limit Theorem applies
* how to calculate critical values and confidence intervals


# Logistic map explorations (Taylor and beyond)

Adapted by Dick Furnstahl from the Ipython Cookbook by Cyrille Rossant.
Lyapunov plot modified on 29-Jan-2019 after discussion with Michael Heinz.

Here we consider the *logistic map*, which illustrates how chaos can arise from a simple nonlinear equation.  The logistic map models the evolution of a population including reproduction and mortality (see Taylor section 12.9).

The defining function for the logistic map is

$ f_r(x) = r x (1-x) \;, $

where $r$ is the control parameter analogous to $\gamma$ for the damped, driven pendulum.  To study the pendulum, we looked at $\phi(t)$ at $t_0$, $t_0 + \tau$, $t_0 + 2\tau$, and so on, where $\tau$ is the driving frequency and we found $\phi(t_0 + \tau)$ from $\phi(t_0)$ by solving the differential equation for the pendulum.  After transients had died off, we characterized the value of $\gamma$ being considered by the periodicity of the $\phi(t_n)$ values: period one (all the same), period two (alternating), chaotic (no repeats) and so on.

Here instead of a differential equation telling us how to generate a trajectory, we use $f_r(x)$:

$\begin{align}
  x_1 = f_r(x_0) \ \rightarrow\ x_2 = f_r(x_1) \ \rightarrow\  x_3 = f_r(x_2) \ \rightarrow\  \ldots
\end{align}$

There will be transients at the beginning, but this sequence of $x_i$ values may reach a point $x = x^*$ such that $f(x^*) = x^*$. We call this a *fixed point*.  If instead it ends up bouncing between two values of $x$, it is period two and we call it a two-cycle.  We can have a cascade of period doubling, as found for the damped, driven pendulum, leading to chaos, which is characterized by the mapping never repeating.  We can make a corresponding bifurcation diagram and identify Lyapunov exponents for each initial condition and $r$.

To adapt this notebook to a different map (such as the sine map), you will need to:
* Change the map function and its derivative.
* Change the range of the $r$ parameter to an appropriate range for the new map.
* Possibly change the initial $x$ value ($x_0$).
* Modify the limits of the plots appropriately.

```python
%matplotlib inline

# standard numpy and matplotlib imports
import numpy as np
import matplotlib.pyplot as plt

```

```python
def logistic(r, x):
    """Logistic map function: f(x) = r x(1-x)
    """
    return r * x * (1.-x)

def logistic_deriv(r, x):
    """Logistic map derivative: f'(x) = r(1-2x)
    """
    return r * (1. - 2.*x)
```

## Explore the logistic map and its fixed points

Start with a simple plot of the logistic function.  **What does changing $r$ do?**

```python
x, step = np.linspace(0,1, num=101, endpoint=True, retstep=True)
fig = plt.figure()
ax = fig.add_subplot(1,1,1)

r_param = 2
ax.plot(x, logistic(r_param, x), 'k-')
ax.set_xlabel('x')
ax.set_ylabel('f(x)')

fig.tight_layout()
```

Make plots showing the approach to fixed points.

**Play with the number of steps taken and the values of $r$.**
1. **Increase the number of steps in the plots to more cleanly identify the long-time trend for that value of $r$.**
2. **Try smaller values of $r$.  Are there always non-zero fixed points for small $r$?**

```python
def plot_system(r, x0, n, ax=None):
    """Plot the function and the y=x diagonal line."""
    t = np.linspace(0,1, num=101)
    ax.plot(t, logistic(r,t), 'k', lw=2)  # black, linewidth 2
    ax.plot([0,1], [0,1], 'k', lw=2)  # x is an array of 0 and 1,
                                      # y is the same array, so this plots
                                      #  a straight line from (0,0) to (1,1).

    # Recursively apply y=f(x) and plot two additional straight lines:
    #   line from (x, x) to (x, y)
    #   line from (x, y) to (y, y)
    x = x0
    for i in range(n):   # do n iterations, i = 0, 1, ..., n-1
        y = logistic(r, x)
        # Plot the two lines
        ax.plot([x,x], [x,y], color='blue', lw=1)
        ax.plot([x,y], [y,y], color='blue', lw=1)
        # Plot the positions with increasing opacity of the circles
        ax.plot([x], [y], 'or', ms=10, alpha=(i+1)/n)
        x = y  # recursive: reset x to y for the next iteration
    ax.set_xlim(0,1)
    ax.set_ylim(0,1)
    ax.set_title(f'$r={r:.1f}$, $x_0={x0:.1f}$')

fig = plt.figure(figsize=(12,12))

ax1 = fig.add_subplot(2,2,1)
# start at 0.1, r parameter is 2.5, take n steps
plot_system(r=2.5, x0=0.1, n=10, ax=ax1)

ax2 = fig.add_subplot(2,2,2)
# start at 0.1, r parameter is 3.2, take n steps
plot_system(r=3.2, x0=0.1, n=10, ax=ax2)

ax3 = fig.add_subplot(2,2,3)
# start at 0.1, r parameter is 2.6, take n steps
plot_system(r=3.5, x0=0.1, n=10, ax=ax3)

ax4 = fig.add_subplot(2,2,4)
# start at 0.1, r parameter is 3.7, take n steps
plot_system(r=3.7, x0=0.1, n=10, ax=ax4)
```

**What periods do these exhibit?  Are any chaotic?**

## Find the unstable point numerically

To find the value of $r$ at which the second fixed point becomes unstable, we must solve simultaneously the equations (see Taylor 12.9 in the "A Test for Stability" subsection):

$\begin{align}
  &f_r(x^*) = x^* \\
  &f_r'(x^*) = -1
\end{align}$

where $f'$ means the derivative with respect to $x$, or

$\begin{align}
  &f_r(x^*) - x^* = 0 \\
  &f_r'(x^*) + 1 = 0
\end{align}$

It is the latter form of the equations that we will pass to the
scipy function `fsolve` to find the values of $r$ and $x*$.

```python
from scipy.optimize import fsolve  # Google this to learn more!

def equations(passed_args):
    """Return the two expressions that must equal to zero."""
    r, x = passed_args
    return (logistic(r, x) - x, logistic_deriv(r, x) + 1)

# Call fsolve with initial values for r and x
r, x = fsolve(equations, (0.5, 0.5))

print_string = f'x* = {x:.10f},  r = {r:.10f}'
print(print_string)
```

**Verify analytically that these are the correct values for the logistic map.**  We use this same procedure for other maps where only numerical solutions are known.

## Now make the bifurcation diagram and calculate Lyapunov exponents

To make a bifurcation diagram, we:
* define an array of closely spaced $r$ values;
* for each $r$, iterate the map $x_{i+1} = f_r(x_i)$ a large number of times (e.g., 1000) to ensure convergence;
* plot the last 100 or so iterations as points at each $r$.

To think about how to calculate a Lyapunov exponent, consider two iterations of the map starting from two values of $x$ that are close together.  Call these initial values $x_0$ and $x_0 + \delta x_0$.  These are mapped by $f_r$ to $x_1$ and  $x_1 + \delta x_1$, then $x_2$ and $x_2 + \delta x_2$, up to $x_n + \delta x_n$.

How are the $\delta x_i$ related?  Each $\delta x_i$ is small, so use a Taylor expansion!
Expanding $f(x)$ about $x_n$:

$\begin{align}
  f(x_i &+ \delta x_i) = &f(x_i) + \delta x_i f'(x_i) + \frac{1}{2}(\delta x_i)^2 f''(x_i) + \cdots \\
\end{align}$

so, neglecting terms of order $(\delta x_i)^2$ and higher,

$\begin{align}
    x_{i+1} + \delta x_{i+1} \approx x_{i+1} + \delta x_i f'(x_i)
    \quad\mbox{or}\quad  \delta x_{i+1} \approx f'(x_i)\, \delta x_i \;.
\end{align}$

Iterating this result we get

$\begin{align}
  \delta x_1 &= f'(x_0)\, \delta x_0  \\
  \delta x_2 &= f'(x_1)\, \delta x_1 = f'(x_1)\times f'(x_0)\, \delta x_0  \\
  & \qquad\vdots  \\
  |\delta x_n| &= \left(\prod_{i=0}^{n-1} |f'(x_i)|\right) |\delta x_0| \;,
\end{align}$

where the last equation gives us the separation of two trajectories after $n$ steps $\delta x_n$ in terms of their
initial separation, $\delta x_0$.  (Why can we just take the absolute value of all the terms?)

We expect that this will vary exponentially at large $n$ like

$\begin{align}
  \left| \frac{\delta x_n}{\delta x_0} \right| = e^{\lambda n}
\end{align}$

and so we define the Lypunov exponent $\lambda$ by

$\begin{align}
  \lambda = \lim_{n\rightarrow\infty} \frac{1}{n} \sum_{i=1}^{n} \ln |f'(x_i)| \;,
\end{align}$

which in practice is well approximated by the sum for large $n$.

If $\lambda > 0$, then nearby trajectories diverge from each other exponentially at large $n$, which corresponds to chaos. However, if the
trajectories converge to a fixed point or a limit cycle, they will get closer together with increasing $n$, which corresponds to $\lambda<0$.

Ok, let's do it!

```python
n = 10000
# Here we'll use n values of r linearly spaced between 2.8 and 4.0
###  You may want to change the range of r for other maps
r = np.linspace(2.8, 4.0, n)

iterations = 1000   # iterations of logistic map; keep last iterations
last = 100          #  where results should have stabilized

x = 0.1 * np.ones(n)  # x_0 initial condition
                      ### (you may want to change for other maps)

lyapunov = np.zeros(n) # initialize vector used for the Lyapunov sums
```

```python
fig = plt.figure(figsize=(8,9))
ax1 = fig.add_subplot(2,1,1)   # bifurcation diagram
ax2 = fig.add_subplot(2,1,2)   # Lyapunov exponent

# Display the bifurcation diagram with one pixel per point x_n^(r) for last iterations
for i in range(iterations):
    x = logistic(r,x)  # just iterate: x_{i+1} = f_r(x_i)

    # Compute the partial sum of the Lyapunov exponent, which is the sum
    #  of derivatives of the logistic function (absolute value)
    lyapunov += np.log(abs(logistic_deriv(r,x)))
    # Display the bifurcation diagram.
    if i >= (iterations-last):  # only plot the last iterations
        ax1.plot(r, x, ',k', alpha=0.25)
ax1.set_xlim(2.8, 4)
ax1.set_xlabel(r'$r$')
ax1.set_title("Bifurcation")

# Display the Lyapunov exponent

# Negative Lyapunov exponent
ax2.plot(r[lyapunov < 0],
         lyapunov[lyapunov < 0] / iterations, 'o',
         color='black', alpha=0.5, ms=0.5)
# Positive Lyapunov exponent
ax2.plot(r[lyapunov >= 0],
         lyapunov[lyapunov >= 0] / iterations, 'o',
         color='red', alpha=0.5, ms=0.5)

# Add a zero line (lightened with alpha=0.5)
ax2.axhline(0, color='k', lw=0.5, alpha=0.5)

ax2.set_xlim(2.8, 4)
ax2.set_ylim(-2, 1)
ax2.set_xlabel(r'$r$')
ax2.set_title("Lyapunov exponent")

plt.tight_layout()
```

We see there is a fixed point for $r < 3$, then two and four periods and a chaotic behavior when $r$ belongs to certain areas of the parameter space.  **Do the values of $r$ where these behaviors occur agree with what you found in the plots with particular $r$ values?**

As for the pendulum, the Lyapunov exponent is positive when the system is chaotic (in red).  **Do these regions look consistent with the characteristics of chaos?**

```python

```


```python
import numpy as np
import pandas as pd
import sympy as sym
```

```python
def Motor_Failure_Risk_Calc(MotorStatus, Motors_Configuration, Lamdba, time):

    #%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
    # Program Name : Markov-based Drone's Reliability and MTTF Estimator      %
    # Author       : Koorosh Aslansefat                                       %
    # Version      : 1.0.2                                                    %
    # Description  : A Markov Process-Based Approach for Reliability          %
    #                Evaluation of the Propulsion System for Drones           %
    #%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

    ## Cite As: Aslansefat, Koorosh, Marques, F., Mendonça, R., & Barata, J.
    # (2019, May). A Markov Process-Based Approach for Reliability Evaluation
    #  of the Propulsion System in Multi-rotor Drones. In Doctoral Conference
    #  on Computing, Electrical and Industrial Systems (pp. 91-98). Springer

    ##

    # Motors Status 0 for failed and 1 for operational
    # For Example for Hexacopter MotorStatus can be:
    # MotorStatus = [1,1,1,1,1,1] means all motors are operational and MotorStatus = [0,1,1,1,1,1] means motor a has failed.
    # Lambda = Failure Rate of Propulsion System including rotors, motors' drivers and propellers.
    # Motors_Configuration: It can be 'PNPN' for quadcopters, 'PNPNPN' and 'PPNNPN' for hexacopters and 'PPNNPPNN' for octacopter.

    ##
    import numpy as np # linear algebra
    import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)
    import sympy as sym

    L = sym.Symbol('L')
    t = sym.Symbol('t')

    SysState = sum(MotorStatus)

    if  Motors_Configuration == 'PNPN':
        if len(MotorStatus) > 4:
               disp('The MotorStatus vector should match Motors_Configuration vector')
        else:
            if SysState < 4:
                P_Fail = 1
                MTTF = 0
            else:
                P_Fail = 1 - sym.exp(-4*L*t)
                MTTF = 1/(4*L)

    elif Motors_Configuration == 'PNPNPN':

        if SysState == 6:
            P0 = sym.Matrix([[1],[0],[0],[0],[0],[0]])
            Sflag = 4
        elif SysState == 5:
            P0 = sym.Matrix([[0],[1],[0],[0],[0],[0]])
            Sflag = 3
        elif SysState == 4:
            if (MotorStatus[0]+MotorStatus[3]) == 0 or (MotorStatus[1]+MotorStatus[4]) == 0 or (MotorStatus[2]+MotorStatus[5]) == 0:
                P0 = sym.Matrix([[0],[0],[1],[0],[0],[0]])
                Sflag = 2
            else:
                P0 = sym.Matrix([[0],[0],[0],[1],[0],[0]])
                Sflag = 1

        elif SysState == 3:
            if (MotorStatus[0]+MotorStatus[3]) == 0 or (MotorStatus[1]+MotorStatus[4]) == 0 or (MotorStatus[2]+MotorStatus[5]) == 0:
                P_Fail = 1
                MTTF = 0;
            else:
                P0 = sym.Matrix([[0],[0],[0],[0],[1],[0]])
                Sflag = 0
        else:
            P_Fail = 1
            MTTF = 0

        M = sym.Matrix([[-6*L,     0,    0,    0,    0,    0],
                            [ 6*L,  -5*L,    0,    0,    0,    0],
                            [   0,     L, -4*L,    0,    0,    0],
                            [   0,   2*L,    0, -4*L,    0,    0],
                            [   0,     0,    0,    L, -3*L,    0],
                            [   0,   2*L,  4*L,  3*L,  3*L,    0]])

        P = sym.exp(M*t)*P0
        P_Fail = P[-1]

        N = sym.Matrix([[-6*L,     0,    0,    0,    0],
                            [ 6*L,  -5*L,    0,    0,    0],
                            [   0,     L, -4*L,    0,    0],
                            [   0,   2*L,    0, -4*L,    0],
                            [   0,     0,    0,    L, -3*L]])
        tt = -1*N.inv()
        MTTF = sum(tt[Sflag,:])

    elif  Motors_Configuration == 'PPNNPN':

        if SysState == 6:
            P02 = sym.Matrix([[1],[0],[0],[0]])
            Sflag = 2
        elif SysState == 5:
            P02 = sym.Matrix([[0],[1],[0],[0]])
            Sflag = 1
        elif SysState == 4:
            P02 = sym.Matrix([[0],[0],[1],[0]])
            Sflag = 0
        else:
            P_Fail = 1
            MTTF = 0

        M2 = sym.Matrix([[-6*L,     0,    0,    0],
                         [  4*L,  -5*L,    0,    0],
                         [    0,   2*L, -4*L,    0],
                         [  2*L,   3*L,  4*L,    0]])
        P2 = sym.exp(M2*t)*P02
        P_Fail = P2[-1]

        N2 = sym.Matrix([[-6*L,     0,    0],
                          [4*L,  -5*L,    0],
                          [  0,   2*L, -4*L]])

        tt = -1*N2.inv()
        MTTF = sum(tt[Sflag,:])

    elif Motors_Configuration == 'PPNNPPNN':

        if SysState == 8:
            P03 = sym.Matrix([[1],[0],[0],[0],[0],[0]])
            Sflag = 4
        elif SysState == 7:
            P03 = sym.Matrix([[0],[1],[0],[0],[0],[0]])
            Sflag = 3
        elif SysState == 6:
            P03 = sym.Matrix([[0],[0],[1],[0],[0],[0]])
            Sflag = 2
        elif SysState == 5:
            P03 = sym.Matrix([[0],[0],[0],[1],[0],[0]])
            Sflag = 1
        elif SysState == 4:
            P03 = sym.Matrix([[0],[0],[0],[0],[1],[0]])
            Sflag = 0
        else:
            P_Fail = 1
            MTTF = 0

        M3 = sym.Matrix([[-8*L,     0,    0,    0,    0,    0],
                         [ 8*L,  -7*L,    0,    0,    0,    0],
                         [   0,   6*L, -6*L,    0,    0,    0],
                         [   0,     0,  4*L, -5*L,    0,    0],
                         [   0,     0,    0,  2*L, -4*L,    0],
                         [   0,   1*L,  2*L,  3*L,  4*L,    0]])

        P3 = sym.exp(M3*t)*P03
        P_Fail = P3[-1]

        N3 = sym.Matrix([[-8*L,     0,    0,    0,    0],
                         [ 8*L,  -7*L,    0,    0,    0],
                         [   0,   6*L, -6*L,    0,    0],
                         [   0,     0,  4*L, -5*L,    0],
                         [   0,     0,    0,  2*L, -4*L]])

        tt = -1*N3.inv()
        MTTF = sum(tt[Sflag,:])

    else:
        print('The current Motors_Configuration is not defined in this version, please check the updated versions')

    return P_Fail.evalf(subs={L: Lamdba, t: time}), MTTF.evalf(subs={L: Lamdba, t: time})
```

```python
P_Fail, MTTF = Motor_Failure_Risk_Calc([1,1,1,1,1,1], 'PNPNPN', 0.001, 100)

print(P_Fail)

print(MTTF)
```

    0.0489641066173940
    483.333333333333

```python
P_Fail, MTTF = Motor_Failure_Risk_Calc([1,1,1,1,1,1], 'PPNNPN', 0.001, 100)

print(P_Fail)

print(MTTF)
```

    0.196030818613950
    416.666666666667

```python
P_Fail, MTTF = Motor_Failure_Risk_Calc([1,1,1,1,1,1], 'PPNNPPNN', 0.001, 100)

print(P_Fail)

print(MTTF)
```

    0.196030818613950
    452.380952380952


# Chris Daigle
## Convex Optimization w/ Python
## Exam

```python
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
from sympy import *
from cvxpy import *
```

```python
df = pd.read_csv(
    '/Users/daiglechris/Git/ConvexOptimizationWithPython/Data/Stocks2-closeP.csv',
    sep = ",")
```

```python
df.head()
```

      timestamp
      F_close
      BA_close
      RTN_close
      VLO_close
      AAL_close
      CI_close
      IBM_close
      PG_close

      0
      2018-12-03
      9.60
      359.96
      172.87
      81.30
      39.65
      224.84
      125.31
      93.32

      1
      2018-11-30
      9.41
      346.76
      175.34
      79.90
      40.16
      223.38
      124.27
      94.51

      2
      2018-11-29
      9.37
      342.56
      173.58
      79.47
      38.42
      222.52
      121.48
      92.82

      3
      2018-11-28
      9.41
      333.50
      173.61
      79.42
      38.94
      221.72
      123.00
      93.01

      4
      2018-11-27
      9.28
      318.03
      171.67
      77.43
      38.29
      216.80
      120.03
      92.72

### Question 1:
(a) Compute the one day returns $\left (\dfrac{p_{t+1}-p_{t}}{p_{t}} \right )$ of each stock and find the mean and the variance of each stock's returns.

```python
oneDayReturns = df.iloc[:,1:].apply(lambda x: ((x - (x.shift(1))) / (x.shift(1))))
names = list(oneDayReturns.columns)
meansOfReturns = pd.DataFrame(oneDayReturns.describe().iloc[1].abs(),
                              index = names)
meansOfReturns1 = abs(oneDayReturns.describe().iloc[1].values)
varianceOfReturns = oneDayReturns.var()
table = pd.concat([meansOfReturns, varianceOfReturns],
                  axis = 1)
table.columns = ['Mean of the Returns', 'Variance of the Returns']
table
```

      Mean of the Returns
      Variance of the Returns

      F_close
      0.001533
      0.000352

      BA_close
      0.000098
      0.000331

      RTN_close
      0.001566
      0.000214

      VLO_close
      0.003054
      0.000501

      AAL_close
      0.000345
      0.000649

      CI_close
      0.002542
      0.000194

      IBM_close
      0.001668
      0.000267

      PG_close
      0.001568
      0.000145

|**Name of Stock**| **Mean of the Returns** | **Variance of the Returns**    |
|------------|:--------------------------------------------:|--------------:|
| Ford       | 0.001533| 0.000352  |
| Boeing     | 0.000098| 0.000331 |
| Raytheon       | 0.001566| 0.000214 |
| Valero Energy     | 0.003054 | 0.000501 |
| American Airlines       | 0.000345 | 0.000649  |
| Cigna     |0.002542| 0.000194|
| IBM       | 0.001668| 0.000267 |
| Procter & Gamble     | 0.001568|0.000145|

(b) Plot the returns of the stocks with the two highest variance and the stocks with the two highest mean. Save a pdf copy of you graphs with “plt.savefig(ʼsample.pdfʼ). Send the pdf file to olivier.morand@uconn.edu with “graph-your name” in the title of your email.

```python
highestVar = table.sort_values(by = 'Variance of the Returns',
                               axis = 0,
                               ascending=False).head(2)
highestMean = table.sort_values(by = 'Mean of the Returns',
                                axis = 0,
                                ascending=False).head(2)
dfHighest = pd.concat([highestMean, highestVar])
dfReturnHighest = oneDayReturns[dfHighest.T.columns]
```

```python
dfReturnHighest.plot(grid = True,
                     figsize = (12,8)).axhline(y = 0,
                                               color = "black",
                                               lw = 2)
plt.title('Returns of the stocks with the two highest variance and the stocks with the two highest mean')
plt.savefig('graph-Daigle.pdf')
plt.show()
```

### Question 2:
Two clients, John and Jane, have asked you for help to build a portfolio in which to invest their income. You seek a combination of the 8 stocks that will maximize each clientʼs utility. A portfolio is a vector $X = (x_{1}, ..., x_{8})$ with $\forall i = 1, ... 8$, $0 \le x_{i} \le 1$ and $\sum_{i=1}^{8} x_{i} = 1$ where $x_{i}$ is the fraction of income to be spend on asset $i$. Both clients have utility:
$$X.\overline{R}^{T} - \frac{1}{2}AX\Sigma X^{T}$$
where $\overline{R}$ is the $1 \times 8$ vector of means calculated in 1, and $\Sigma$ is the $8 \times 8$ covariance matrix of returns. John is risk averse and his risk coefficient is $A = 2$ while Jane's coefficient is $A = 10$.

(a) Explain below why the problem is convex.

The problem is convex because the Hessian Matrix of $X\Sigma X^{T}$ is positive semidefinite (PSD), $\frac{1}{2} A$ only scales the value of $X\Sigma X^{T}$, and the first portion $X.\overline{R}^{T}$ is just a vector of scalars, singleton or otherwise. So, we only need to evaluate the Hessian of $X\Sigma X^{T}$.

```python
x1, x2, x3, x4, x5, x6, x7, x8 = symbols('ford, Boeing, Raytheon, Valero, AmericanAir, Cigna, IBM, PG')
X = np.array([x1, x2, x3, x4, x5, x6, x7, x8])
R = meansOfReturns
Sigma = oneDayReturns.cov()
Z = np.array(X*Sigma*X.T)
```

```python
def hessian(x):
    """
    Calculate the hessian matrix with finite differences
    Parameters:
       - x : ndarray
    Returns:
       an array of shape (x.dim, x.ndim) + x.shape
       where the array[i, j, ...] corresponds to the second derivative x_ij
    """
    x_grad = np.gradient(x)
    hessian = np.empty((x.ndim, x.ndim) + x.shape, dtype=x.dtype)
    for k, grad_k in enumerate(x_grad):
        # iterate over dimensions
        # apply gradient again to every component of the first derivative.
        tmp_grad = np.gradient(grad_k)
        for l, grad_kl in enumerate(tmp_grad):
            hessian[k, l, :, :] = grad_kl
    return hessian
```

```python
H = hessian(np.array(Sigma))
np.linalg.det(H)
```

    array([[ 2.20799925e-63, -6.76560985e-48],
           [-7.95272270e-48, -3.57667386e-65]])

This works out to a series of zeros in the matrix. I'm not sure why there are 4 values, but regardless, zero times any real value is zero and thus, the matrix is PSD and the problem is thus convex.

(b) Solve for Jane's and Johns' optimal portfolio. Write your answer below

```python
mu = meansOfReturns1

w = Variable(8) # Decision variable

gamma = 2 # Risk Parameter in the utility function
ret = mu*w
risk = quad_form(w, Sigma)

obj = Maximize(ret - gamma*risk)
constraints = [sum(w) == 1 , w >=0]

# Form the problem
prob = Problem(obj, constraints)
```

```python
port_data = []
ret_data = []
risk_data = []
prob_data = []

gamma_vals = [2,10]
for i in range(2):
    gamma = gamma_vals[i]
    prob.solve() # Solve the problem for a specific value of gamma
    risk_data.append(sqrt(risk).value) # optimal value of the risk (standard deviation)
    ret_data.append(ret.value) # optimal value of the return
    prob_data.append(prob.value) # maximum utility
    port_data.append(w.value)
```

```python
print(np.sum(port_data[0]))
print(np.sum(port_data[1]))
```

    1.0000000000000788
    1.0000000000000788

Checking that the proportions add to 1. Because of floating point arithmetic, the value is not exactly 1, but the results "check out"

```python
results = pd.DataFrame(
    {"John's Optimal Portfolio":port_data[0],
     "Jane's Optimal Portfolio":port_data[1]},
    index= ['Ford', 'Boeing', 'Raytheon',
                  'Valero', 'American Airlines',
                  'Cigna', 'IBM', ' PG'])
results
```

      John's Optimal Portfolio
      Jane's Optimal Portfolio

      Ford
      7.262696e-08
      7.262696e-08

      Boeing
      2.046956e-08
      2.046956e-08

      Raytheon
      6.250528e-08
      6.250528e-08

      Valero
      4.409146e-01
      4.409146e-01

      American Airlines
      2.174878e-08
      2.174878e-08

      Cigna
      5.590849e-01
      5.590849e-01

      IBM
      7.084433e-08
      7.084433e-08

      PG
      2.445910e-07
      2.445910e-07

```python

```

```python

```


# Ritz method for a beam

**November, 2018**

We want to find a Ritz approximation of vibration frequencies/modes of a cantilever beam.
This is described by the following eigenvalue problem.

$$
\frac{\mathrm{d}^4w}{\mathrm{d}x^4} + \beta^4 w\, ,\quad
0 < x < L,\quad EI>0\, ,
$$

with

$$
w(0) = w'(0) = 0,\quad
\left(\frac{\mathrm{d}^2w}{\mathrm{d}x^2}\right)_{x=L} = 0,\quad
\left(\frac{\mathrm{d}^3 w}{\mathrm{d}x^3}\right)_{x=L} = 0\, ,
$$

and

$$\beta \equiv \left(\frac{\mu \omega^2}{EI}\right)^{1/4}\, .$$

```python
import numpy as np
import matplotlib.pyplot as plt
from sympy import *
```

```python
%matplotlib notebook
init_printing()

# Graphics setup
gray = '#757575'
plt.rcParams["mathtext.fontset"] = "cm"
plt.rcParams["text.color"] = gray
plt.rcParams["font.size"] = 12
plt.rcParams["xtick.color"] = gray
plt.rcParams["ytick.color"] = gray
plt.rcParams["axes.labelcolor"] = gray
plt.rcParams["axes.edgecolor"] = gray
plt.rcParams["axes.spines.right"] = False
plt.rcParams["axes.spines.top"] = False
plt.rcParams["figure.figsize"] = 4, 3
```

The exact solution for this problem is

$$w_n(x) = A_1\left[(\cosh\beta_n x - \cos\beta_n x) +
\frac{\cos\beta_n L + \cosh\beta_n L}{\sin\beta_n L + \sinh\beta_n L}(\sin\beta_n x - \sinh\beta_n x)\right]\, ,$$

where $\beta_n$ is the $n$th root of $\cosh(\beta_n L)\cos(\beta_n L)  + 1$.

```python
x = symbols('x')
beta, L = symbols("beta L")
val1 = 1.8750909912
val2 = 4.6941049111
```

```python
def plot_expr(expr, x, rango=(0, 1), ax=None, linestyle="solid"):
    """Plot SymPy expressions of a single variable"""
    expr_num = lambdify(x, expr, "numpy")
    x0 = rango[0]
    x1 = rango[1]
    x_num = np.linspace(0, 1, 101)
    if ax is None:
        plt.figure()
        ax = plt.gca()
    ax.plot(x_num, expr_num(x_num), linestyle=linestyle)
```

## Conventional formulation

The quadratic functional for this problem is

$$J[u] = \int\limits_0^L \left[\left(\frac{\mathrm{d}^2 w}{\mathrm{d}x^2}\right)^2 + \beta^4 w^2\right]\mathrm{d}x\, ,$$

and the weak problem $B(v, w) = \beta^2 A(v, w)$, with

$$
B(v, u) = \int\limits_0^L \frac{\mathrm{d}^2 v}{\mathrm{d}x^2}\frac{\mathrm{d}^2 u}{\mathrm{d}x^2}\mathrm{d}x\, ,\quad
A(v, u) = \int\limits_0^L  vw\mathrm{d}x\, .
$$

```python
def quad_fun(x, w, L):
    F1 = diff(w, x, 2)**2
    F2 = w**2
    U = integrate(F1, (x, 0, L))
    T = integrate(F2, (x, 0, L))
    return U, T

```

```python
def ritz_conventional(x, L, nterms):
    a = symbols("a0:%i"%(nterms))
    w = sum(a[k]*x**(k + 2) for k in range(nterms))
    U, T = quad_fun(x, w, L)
    K = Matrix(nterms, nterms, lambda i, j: U.diff(a[i], a[j]))
    M = Matrix(nterms, nterms, lambda i, j: T.diff(a[i], a[j]))
    return K, M
```

```python
K, M = ritz_conventional(x, 1, 2)

```

```python
Kaux = M.inv() * K
vals = list(Kaux.eigenvals())
vals
```

```python
nvals = [N(val**0.25) for val in vals]
nvals
```

## Lagrange multiplier formulation

We can write the problem as minimizing the functional

$$J(\psi, w) = \int\limits_0^L\left[\left(\frac{\mathrm{d} \psi}{\mathrm{d}x}\right)^2 + \beta^4
w^2\right]\mathrm{d}x\, ,$$

subject to

$$G(\psi, w) \equiv \psi + \frac{\mathrm{d}w}{\mathrm{d}x} = 0\, .$$

The Lagrangian is given by

$$L(\psi, w, \lambda) = \int\limits_0^L\left[\left(\frac{\mathrm{d} \psi}{\mathrm{d}x}\right)^2 + \beta^4 w^2\right]\mathrm{d}x + \int\limits_0^L \lambda\left(\psi + \frac{\mathrm{d}w}{\mathrm{d}x}\right)\mathrm{d}x\, , $$

where $\lambda$ is the Lagrange multiplier, which in this case represents the shear force.

```python
def lagran(x, w, psi, lamda, L):
    F1 = diff(psi, x)**2 + lamda*(psi + diff(w, x))
    F2 = w**2
    U = integrate(F1, (x, 0, L))
    T = integrate(F2, (x, 0, L))
    return U, T

```

```python
def ritz_multiplier(x, L, nterms):
    a = symbols("a0:%i"%(nterms))
    b = symbols("b0:%i"%(nterms))
    c = symbols("c0:%i"%(nterms))
    var = a + b + c
    w = sum(a[k]*x**(k + 1) for k in range(nterms))
    psi = sum(b[k]*x**(k + 1) for k in range(nterms))
    lamda = sum(c[k]*x**k for k in range(nterms))
    U, T = lagran(x, w, psi, lamda, L)
    K = Matrix(3*nterms, 3*nterms, lambda i, j: U.diff(var[i], var[j]))
    M = Matrix(3*nterms, 3*nterms, lambda i, j: T.diff(var[i], var[j]))
    return K, M
```

```python
K, M = ritz_multiplier(x, 1, 2)
```

```python
K
```

$$\left[\begin{matrix}0 & 0 & 0 & 0 & 1 & \frac{1}{2}\\0 & 0 & 0 & 0 & 1 & \frac{2}{3}\\0 & 0 & 2 & 2 & \frac{1}{2} & \frac{1}{3}\\0 & 0 & 2 & \frac{8}{3} & \frac{1}{3} & \frac{1}{4}\\1 & 1 & \frac{1}{2} & \frac{1}{3} & 0 & 0\\\frac{1}{2} & \frac{2}{3} & \frac{1}{3} & \frac{1}{4} & 0 & 0\end{matrix}\right]$$

```python
M
```

$$\left[\begin{matrix}\frac{2}{3} & \frac{1}{2} & 0 & 0 & 0 & 0\\\frac{1}{2} & \frac{2}{5} & 0 & 0 & 0 & 0\\0 & 0 & 0 & 0 & 0 & 0\\0 & 0 & 0 & 0 & 0 & 0\\0 & 0 & 0 & 0 & 0 & 0\\0 & 0 & 0 & 0 & 0 & 0\end{matrix}\right]$$

```python
Maux = K.inv() * M
vals = list(Maux.eigenvals())
nvals = [N(1/val**0.25) for val in vals if val != 0]
nvals
```

# The penalty function formulation

The augmented functional for this formulation is given by

$$P_K (\psi, w) = J(\psi, w) + \frac{K}{2}\int\limits_0^L \left(\psi + \frac{\mathrm{d}w}{\mathrm{d}x}\right)^2\mathrm{d}x\, ,$$

where $K$ is the penalty parameter.

```python
def augmented(x, w, psi, K, L):
    F1 = diff(psi, x)**2 + S(K)/2*(psi + diff(w, x))**2
    F2 = w**2
    U = integrate(F1, (x, 0, L))
    T = integrate(F2, (x, 0, L))
    return U, T
```

```python
def ritz_penalty(x, K, L, nterms):
    a = symbols("a0:%i"%(nterms))
    b = symbols("b0:%i"%(nterms))
    var = a + b
    w = sum(a[k]*x**(k + 1) for k in range(nterms))
    psi = sum(b[k]*x**(k + 1) for k in range(nterms))
    U, T = augmented(x, w, psi, K, L)
    K = Matrix(2*nterms, 2*nterms, lambda i, j: U.diff(var[i], var[j]))
    M = Matrix(2*nterms, 2*nterms, lambda i, j: T.diff(var[i], var[j]))
    return K, M
```

```python
K, M = ritz_penalty(x, 100, 1, 2)
```

```python
K
```

$$\left[\begin{matrix}100 & 100 & 50 & \frac{100}{3}\\100 & \frac{400}{3} & \frac{200}{3} & 50\\50 & \frac{200}{3} & \frac{106}{3} & 27\\\frac{100}{3} & 50 & 27 & \frac{68}{3}\end{matrix}\right]$$

```python
M
```

$$\left[\begin{matrix}\frac{2}{3} & \frac{1}{2} & 0 & 0\\\frac{1}{2} & \frac{2}{5} & 0 & 0\\0 & 0 & 0 & 0\\0 & 0 & 0 & 0\end{matrix}\right]$$

```python
Maux = K.inv() * M
vals = list(Maux.eigenvals())
nvals = [re(N(1/val**0.25)) for val in vals if val != 0]
nvals
```

## Mixed formulation

The mixed formulation involves rewriting a given higher order equation as a pair of lower
order equations by introducing secondary dependent variables. The original equation can be
decomposed into

$$
M(x)= \frac{\mathrm{d}^2 w}{\mathrm{d}x^2}\, ,\quad
\frac{\mathrm{d}^2M(x)}{\mathrm{d}x^2} = -\beta^4 w\, ,\quad 0<x<L\, .
$$

The functional in this case is

$$
I(w, M) = \int\limits_0^L\left(\frac{\mathrm{d}w}{\mathrm{d}x}\frac{\mathrm{d}M}{\mathrm{d}x}
+ \frac{M^2}{2} - \beta^4 w^2\right)\mathrm{d}x
$$

```python
def mixed_fun(x, w, M, L):
    F1 = diff(w, x)*diff(M, x) + M**2/2
    F2 = -w**2
    U = integrate(F1, (x, 0, L))
    T = integrate(F2, (x, 0, L))
    return U, T

```

```python
def ritz_mixed(x, L, nterms):
    a = symbols("a0:%i"%(nterms))
    b = symbols("b0:%i"%(nterms))
    var = a + b
    w = sum(a[k]*x**(k + 1) for k in range(nterms))
    M = sum(b[k]*(x - L)**(k + 1) for k in range(nterms))
    display(w, M)
    U, T = mixed_fun(x, w, M, L)
    Kmat = Matrix(2*nterms, 2*nterms, lambda i, j: U.diff(var[i], var[j]))
    Mmat = Matrix(2*nterms, 2*nterms, lambda i, j: T.diff(var[i], var[j]))
    return Kmat, Mmat
```

```python
K, M = ritz_mixed(x, 1, 2)
```

```python
K
```

$$\left[\begin{matrix}0 & 0 & 1 & -1\\0 & 0 & 1 & - \frac{2}{3}\\1 & 1 & \frac{1}{3} & - \frac{1}{4}\\-1 & - \frac{2}{3} & - \frac{1}{4} & \frac{1}{5}\end{matrix}\right]$$

```python
M
```

$$\left[\begin{matrix}- \frac{2}{3} & - \frac{1}{2} & 0 & 0\\- \frac{1}{2} & - \frac{2}{5} & 0 & 0\\0 & 0 & 0 & 0\\0 & 0 & 0 & 0\end{matrix}\right]$$

```python
Maux = K.inv() * M
vals = list(Maux.eigenvals())
nvals = [re(N(1/val**0.25)) for val in vals if val != 0]
nvals
```

```python

```


# Automatic derivation of CCSD theory

This notebook serves as an example of interactive usage of drudge for complex symbolic manipulations in Jupyter notebooks.  Here we can see how the classical CCSD theory can be derived automatically.

## Preparatory work

First, we need to set up the Spark environment.  Here we just use parallelization on the local machine.

```python
from pyspark import SparkContext
ctx = SparkContext('local[*]', 'ccsd')
```

Or we can also use the dummy spark to emulate the Spark environment in a purely serial way.  Note that we need just **one** Spark context.  These two cells should not be both evaluated.

```python
from dummy_spark import SparkContext
ctx = SparkContext()
```

With the Spark context, we can construct the drudge specific for this problem.  Then we can define some names that is going to be used frequently.

```python
from sympy import *
from drudge import *

dr = PartHoleDrudge(ctx)
dr.full_simplify = False
p = dr.names

c_ = p.c_
c_dag = p.c_dag
a, b = p.V_dumms[:2]
i, j = p.O_dumms[:2]
```

## Cluster excitation operator

Here, we by using the Einstein summation convention tensor creator, we can just define the cluster operator in a way very similar to how we would writen them down on paper.

```python
t = IndexedBase('t')

clusters = dr.einst(
    t[a, i] * c_dag[a] * c_[i] +
    t[a, b, i, j] * c_dag[a] * c_dag[b] * c_[j] * c_[i] / 4
)
```

We can have a peek at the cluster operator.

```python
clusters.display()
```

$$\sum_{i \in O} \sum_{a \in V} t_{a,i}    c^{\dagger}_{a} c^{}_{i}  + \sum_{i \in O} \sum_{j \in O} \sum_{a \in V} \sum_{b \in V} \frac{1}{4}    t_{a,b,i,j}    c^{\dagger}_{a} c^{\dagger}_{b} c^{}_{j} c^{}_{i}$$

Now we need tell the system about the symmetry on $t^2$, so that it can be used in simplification.

```python
dr.set_dbbar_base(t, 2)
```

## Similarity transform of the Hamiltonian

Here we can use a loop to nest the commutation conveniently.  And IPython magic can be used to time the operation.  Note that after the simplification, we explicitly redistribute the terms in the transformed Hamiltonian for better parallel performance in later operations.  Note that `drudge` does not automatically cache the result of tensor computations.  The `cache` method should be called explicitly when a tensor is going to be used multiple times.

```python
%%time

curr = dr.ham
h_bar = dr.ham
for order in range(0, 4):
    curr = (curr | clusters).simplify() / (order + 1)
    curr.cache()
    h_bar += curr
h_bar.repartition(cache=True)
```

    CPU times: user 348 ms, sys: 50.3 ms, total: 398 ms
    Wall time: 30.4 s

The transformed Hamiltonian can be very complex.  Instead of reading its terms, we can just have a peek by get a count of the number of terms it contains.

```python
h_bar.n_terms
```

    183

## Working equation derivation

With the similarity transformed Hamiltonian, we are now ready to derive the actual working equations.  First, the energy equation can be derived by taking the vacuum expectation value of the transformed Hamiltonian.

```python
en_eqn = h_bar.eval_fermi_vev().simplify()
```

We can have a look at its contents to see if it is what we would expect.

```python
en_eqn.display()
```

$$\sum_{i \in O} \sum_{a \in V} f_{i,a}  t_{a,i}   + \sum_{i \in O} \sum_{j \in O} \sum_{a \in V} \sum_{b \in V} \frac{1}{4}    t_{a,b,i,j}  u_{i,j,a,b}   + \sum_{i \in O} \sum_{j \in O} \sum_{a \in V} \sum_{b \in V} \frac{1}{2}    t_{a,i}  t_{b,j}  u_{i,j,a,b} $$

Next, we can create a projector to derive the working equation for the singles amplitude.

```python
proj = c_dag[i] * c_[a]
t1_eqn = (proj * h_bar).eval_fermi_vev().simplify()
```

In the same way, we can display its content.

```python
t1_eqn.display()
```

$$\sum_{j \in O} \sum_{b \in V} \sum_{c \in V} \frac{1}{2}    t_{b,c,i,j}  u_{a,j,b,c}   + \sum_{j \in O} \sum_{b \in V} \sum_{c \in V} t_{b,i}  t_{c,j}  u_{a,j,b,c}   + \sum_{j \in O} \sum_{b \in V} f_{j,b}  t_{a,b,i,j}   + \sum_{j \in O} \sum_{b \in V} t_{b,j}  u_{a,j,i,b}  - \sum_{j \in O} \sum_{b \in V} f_{j,b}  t_{a,j}  t_{b,i}  - \sum_{j \in O} f_{j,i}  t_{a,j}   + \sum_{j \in O} \sum_{k \in O} \sum_{b \in V} \sum_{c \in V} t_{c,k}  t_{a,b,i,j}  u_{j,k,b,c}   + \sum_{j \in O} \sum_{k \in O} \sum_{b \in V} \sum_{c \in V} \frac{1}{2}    t_{a,k}  t_{b,c,i,j}  u_{j,k,b,c}   + \sum_{j \in O} \sum_{k \in O} \sum_{b \in V} \sum_{c \in V} \frac{1}{2}    t_{c,i}  t_{a,b,j,k}  u_{j,k,b,c}  - \sum_{j \in O} \sum_{k \in O} \sum_{b \in V} \sum_{c \in V} t_{a,j}  t_{b,i}  t_{c,k}  u_{j,k,b,c}   + \sum_{b \in V} f_{a,b}  t_{b,i}   + f_{a,i}  - \sum_{j \in O} \sum_{k \in O} \sum_{b \in V}  \frac{1}{2}    t_{a,b,j,k}  u_{j,k,i,b}  - \sum_{j \in O} \sum_{k \in O} \sum_{b \in V} t_{a,j}  t_{b,k}  u_{j,k,i,b} $$

The working equation for the doubles amplitude can be done in the same way, just it can be slower.

```python
%%time

proj = c_dag[i] * c_dag[j] * c_[b] * c_[a]
t2_eqn = (proj * h_bar).eval_fermi_vev().simplify()
```

    CPU times: user 103 ms, sys: 13.6 ms, total: 116 ms
    Wall time: 8.45 s

Since the equation can be slightly complex, we can vaguely sort the terms in increasing complexity before display them.

```python
t2_eqn = t2_eqn.sort()
t2_eqn.display()
```

$$u_{a,b,i,j}   + \sum_{k \in O} t_{a,k}  u_{b,k,i,j}  - \sum_{k \in O} f_{k,i}  t_{a,b,k,j}  - \sum_{k \in O} f_{k,j}  t_{a,b,i,k}  - \sum_{k \in O} t_{b,k}  u_{a,k,i,j}   + \sum_{c \in V} f_{b,c}  t_{a,c,i,j}   + \sum_{c \in V} t_{c,i}  u_{a,b,c,j}   + \sum_{c \in V} t_{c,j}  u_{a,b,i,c}  - \sum_{c \in V} f_{a,c}  t_{b,c,i,j}   + \sum_{k \in O} \sum_{l \in O} \frac{1}{2}    t_{a,b,k,l}  u_{k,l,i,j}   + \sum_{k \in O} \sum_{l \in O} t_{a,k}  t_{b,l}  u_{k,l,i,j}   + \sum_{k \in O} \sum_{c \in V} t_{a,c,k,j}  u_{b,k,i,c}   + \sum_{k \in O} \sum_{c \in V} t_{b,c,i,k}  u_{a,k,c,j}  - \sum_{k \in O} \sum_{c \in V} t_{a,c,i,k}  u_{b,k,c,j}  - \sum_{k \in O} \sum_{c \in V} t_{b,c,k,j}  u_{a,k,i,c}   + \sum_{k \in O} \sum_{c \in V} f_{k,c}  t_{a,k}  t_{b,c,i,j}   + \sum_{k \in O} \sum_{c \in V} t_{a,k}  t_{c,i}  u_{b,k,c,j}   + \sum_{k \in O} \sum_{c \in V} t_{a,k}  t_{c,j}  u_{b,k,i,c}  - \sum_{k \in O} \sum_{c \in V} f_{k,c}  t_{c,j}  t_{a,b,i,k}  - \sum_{k \in O} \sum_{c \in V} f_{k,c}  t_{c,i}  t_{a,b,k,j}  - \sum_{k \in O} \sum_{c \in V} f_{k,c}  t_{b,k}  t_{a,c,i,j}  - \sum_{k \in O} \sum_{c \in V} t_{b,k}  t_{c,i}  u_{a,k,c,j}  - \sum_{k \in O} \sum_{c \in V} t_{b,k}  t_{c,j}  u_{a,k,i,c}   + \sum_{c \in V} \sum_{d \in V} \frac{1}{2}    t_{c,d,i,j}  u_{a,b,c,d}   + \sum_{c \in V} \sum_{d \in V} t_{c,i}  t_{d,j}  u_{a,b,c,d}   + \sum_{k \in O} \sum_{l \in O} \sum_{c \in V} t_{c,l}  t_{a,b,i,k}  u_{k,l,c,j}   + \sum_{k \in O} \sum_{l \in O} \sum_{c \in V} t_{b,l}  t_{a,c,k,j}  u_{k,l,i,c}   + \sum_{k \in O} \sum_{l \in O} \sum_{c \in V} t_{a,l}  t_{b,c,i,k}  u_{k,l,c,j}  - \sum_{k \in O} \sum_{l \in O} \sum_{c \in V} t_{c,l}  t_{a,b,k,j}  u_{k,l,i,c}   + \sum_{k \in O} \sum_{l \in O} \sum_{c \in V} \frac{1}{2}    t_{c,i}  t_{a,b,k,l}  u_{k,l,c,j}   + \sum_{k \in O} \sum_{l \in O} \sum_{c \in V} \frac{1}{2}    t_{c,j}  t_{a,b,k,l}  u_{k,l,i,c}  - \sum_{k \in O} \sum_{l \in O} \sum_{c \in V} t_{b,l}  t_{a,c,i,k}  u_{k,l,c,j}  - \sum_{k \in O} \sum_{l \in O} \sum_{c \in V} t_{a,l}  t_{b,c,k,j}  u_{k,l,i,c}   + \sum_{k \in O} \sum_{l \in O} \sum_{c \in V} t_{a,k}  t_{b,l}  t_{c,i}  u_{k,l,c,j}   + \sum_{k \in O} \sum_{l \in O} \sum_{c \in V} t_{a,k}  t_{b,l}  t_{c,j}  u_{k,l,i,c}   + \sum_{k \in O} \sum_{c \in V} \sum_{d \in V} t_{d,k}  t_{a,c,i,j}  u_{b,k,c,d}   + \sum_{k \in O} \sum_{c \in V} \sum_{d \in V} t_{d,j}  t_{b,c,i,k}  u_{a,k,c,d}   + \sum_{k \in O} \sum_{c \in V} \sum_{d \in V} t_{d,i}  t_{b,c,k,j}  u_{a,k,c,d}  - \sum_{k \in O} \sum_{c \in V} \sum_{d \in V} t_{d,j}  t_{a,c,i,k}  u_{b,k,c,d}  - \sum_{k \in O} \sum_{c \in V} \sum_{d \in V} t_{d,i}  t_{a,c,k,j}  u_{b,k,c,d}   + \sum_{k \in O} \sum_{c \in V} \sum_{d \in V} \frac{1}{2}    t_{a,k}  t_{c,d,i,j}  u_{b,k,c,d}  - \sum_{k \in O} \sum_{c \in V} \sum_{d \in V} t_{d,k}  t_{b,c,i,j}  u_{a,k,c,d}   + \sum_{k \in O} \sum_{c \in V} \sum_{d \in V} t_{a,k}  t_{c,i}  t_{d,j}  u_{b,k,c,d}  - \sum_{k \in O} \sum_{c \in V} \sum_{d \in V}  \frac{1}{2}    t_{b,k}  t_{c,d,i,j}  u_{a,k,c,d}  - \sum_{k \in O} \sum_{c \in V} \sum_{d \in V} t_{b,k}  t_{c,i}  t_{d,j}  u_{a,k,c,d}   + \sum_{k \in O} \sum_{l \in O} \sum_{c \in V} \sum_{d \in V} t_{a,c,k,j}  t_{b,d,i,l}  u_{k,l,c,d}   + \sum_{k \in O} \sum_{l \in O} \sum_{c \in V} \sum_{d \in V} \frac{1}{4}    t_{a,b,k,l}  t_{c,d,i,j}  u_{k,l,c,d}   + \sum_{k \in O} \sum_{l \in O} \sum_{c \in V} \sum_{d \in V} \frac{1}{2}    t_{a,b,l,j}  t_{c,d,i,k}  u_{k,l,c,d}  - \sum_{k \in O} \sum_{l \in O} \sum_{c \in V} \sum_{d \in V} t_{a,c,i,k}  t_{b,d,l,j}  u_{k,l,c,d}   + \sum_{k \in O} \sum_{l \in O} \sum_{c \in V} \sum_{d \in V} \frac{1}{2}    t_{a,d,i,j}  t_{b,c,k,l}  u_{k,l,c,d}   + \sum_{k \in O} \sum_{l \in O} \sum_{c \in V} \sum_{d \in V} t_{c,l}  t_{d,j}  t_{a,b,i,k}  u_{k,l,c,d}   + \sum_{k \in O} \sum_{l \in O} \sum_{c \in V} \sum_{d \in V} t_{a,k}  t_{d,l}  t_{b,c,i,j}  u_{k,l,c,d}   + \sum_{k \in O} \sum_{l \in O} \sum_{c \in V} \sum_{d \in V} t_{a,l}  t_{d,j}  t_{b,c,i,k}  u_{k,l,c,d}   + \sum_{k \in O} \sum_{l \in O} \sum_{c \in V} \sum_{d \in V} t_{a,l}  t_{d,i}  t_{b,c,k,j}  u_{k,l,c,d}  - \sum_{k \in O} \sum_{l \in O} \sum_{c \in V} \sum_{d \in V}  \frac{1}{2}    t_{a,b,i,l}  t_{c,d,k,j}  u_{k,l,c,d}  - \sum_{k \in O} \sum_{l \in O} \sum_{c \in V} \sum_{d \in V}  \frac{1}{2}    t_{a,c,k,l}  t_{b,d,i,j}  u_{k,l,c,d}  - \sum_{k \in O} \sum_{l \in O} \sum_{c \in V} \sum_{d \in V} t_{c,i}  t_{d,l}  t_{a,b,k,j}  u_{k,l,c,d}   + \sum_{k \in O} \sum_{l \in O} \sum_{c \in V} \sum_{d \in V} \frac{1}{2}    t_{c,i}  t_{d,j}  t_{a,b,k,l}  u_{k,l,c,d}  - \sum_{k \in O} \sum_{l \in O} \sum_{c \in V} \sum_{d \in V} t_{b,k}  t_{d,l}  t_{a,c,i,j}  u_{k,l,c,d}  - \sum_{k \in O} \sum_{l \in O} \sum_{c \in V} \sum_{d \in V} t_{b,l}  t_{d,j}  t_{a,c,i,k}  u_{k,l,c,d}  - \sum_{k \in O} \sum_{l \in O} \sum_{c \in V} \sum_{d \in V} t_{b,l}  t_{d,i}  t_{a,c,k,j}  u_{k,l,c,d}   + \sum_{k \in O} \sum_{l \in O} \sum_{c \in V} \sum_{d \in V} \frac{1}{2}    t_{a,k}  t_{b,l}  t_{c,d,i,j}  u_{k,l,c,d}   + \sum_{k \in O} \sum_{l \in O} \sum_{c \in V} \sum_{d \in V} t_{a,k}  t_{b,l}  t_{c,i}  t_{d,j}  u_{k,l,c,d} $$

## Working equation optimization

Evaluating the working equation takes a lot of effort.  Outside ``drudge``, a sister package named ``gristmill`` is available for the optimization and automatic code generation for tensor contractions.  To start with, we need to put the working equations into a tensor definitions with external indices and import the ``gristmill`` package.

```python
from gristmill import *

working_eqn = [
    dr.define(Symbol('e'), en_eqn),
    dr.define(t[a, i], t1_eqn),
    dr.define(t[a, b, i, j], t2_eqn)
]
```

We can have an estimation of the FLOP cost without any optimization.

```python
orig_cost = get_flop_cost(working_eqn, leading=True)
init_printing()
orig_cost
```

Since normally we have far more virtual orbitals than occupied orbitals, we have make the optimization based on this.

```python
%%time

eval_seq = optimize(
    working_eqn, substs={p.nv: 5000, p.no: 1000},
    contr_strat=ContrStrat.EXHAUST
)
```

    CPU times: user 7.31 s, sys: 239 ms, total: 7.54 s
    Wall time: 21.2 s

Now we can have some inspection of the evaluation sequence.

```python
len(eval_seq)
```

```python
opt_cost = get_flop_cost(eval_seq, leading=True)
opt_cost
```

Significant optimization can be seen.  Finally we can verify the correctness of the evaluation sequence.  This step can be very slow.  But it is adviced for mission-critical tasks.

```python
verify_eval_seq(eval_seq, working_eqn, simplify=True)
```

    True

Finally, we have have a peek at the details of the intermediates.

```python
for eqn in eval_seq:
    eqn.display(False)
```

$$\tau^{0}_{i,j,a,b} = t_{b,a,j,i}   + 2    t_{a,i}  t_{b,j} $$

$$\tau^{1}_{i,j,k,a} = \sum_{b \in V} t_{b,i}  u_{k,j,b,a} $$

$$\tau^{2}_{i,j,k,a} = u_{k,j,i,a}  - \tau^{1}_{i,k,j,a} $$

$$\tau^{3}_{i,a} = f_{i,a}   + \sum_{j \in O} \sum_{b \in V} t_{b,j}  u_{j,i,b,a} $$

$$\tau^{4}_{i,j} = \sum_{a \in V} f_{i,a}  t_{a,j} $$

$$\tau^{5}_{i,j} = - \sum_{k \in O} \sum_{a \in V} t_{a,k}  u_{k,i,j,a} $$

$$\tau^{6}_{i,j} = \sum_{k \in O} \sum_{a \in V} \sum_{b \in V} \tau^{0}_{k,j,a,b}  u_{k,i,a,b} $$

$$\tau^{7}_{i,j} = 2    f_{i,j}   + 2    \tau^{4}_{i,j}   + 2    \tau^{5}_{i,j}   + \tau^{6}_{i,j} $$

$$\tau^{8}_{i,j,a,b} = - \sum_{k \in O} t_{a,k}  u_{k,i,j,b} $$

$$\tau^{9}_{i,j,a,b} = - \sum_{c \in V} t_{c,i}  u_{j,a,c,b} $$

$$\tau^{10}_{i,j,a,b} = t_{b,a,j,i}  - t_{a,j}  t_{b,i} $$

$$\tau^{11}_{i,j,a,b} = \sum_{k \in O} \sum_{c \in V} \tau^{10}_{k,i,c,a}  u_{k,j,c,b} $$

$$\tau^{12}_{i,j,a,b} = - u_{j,a,i,b}   + \tau^{8}_{j,i,a,b}   + \tau^{9}_{i,j,a,b}   + \tau^{11}_{i,j,a,b} $$

$$\tau^{13}_{i,j,a,b} = \sum_{k \in O} \sum_{c \in V} t_{c,b,k,j}  \tau^{12}_{i,k,a,c} $$

$$\tau^{14}_{i,a,b,c} = - u_{i,a,c,b}   + \sum_{j \in O} t_{a,j}  u_{j,i,b,c} $$

$$\tau^{15}_{a,b,c,d} = u_{b,a,d,c}  - \sum_{i \in O} t_{a,i}  u_{i,b,c,d}  - \sum_{i \in O} t_{b,i}  \tau^{14}_{i,a,d,c} $$

$$\tau^{16}_{i,j,a,b} = \sum_{k \in O} t_{a,k}  \tau^{2}_{i,k,j,b} $$

$$\tau^{17}_{i,j,a,b} = u_{j,a,i,b}  - \tau^{9}_{i,j,a,b}  - \tau^{16}_{i,j,a,b} $$

$$\tau^{18}_{i,j,k,l} = - \sum_{a \in V} t_{a,i}  u_{k,j,l,a} $$

$$\tau^{19}_{i,j,a,b} = - t_{b,a,j,i}   + 2    t_{a,j}  t_{b,i} $$

$$\tau^{20}_{i,j,k,l} = 2    u_{j,i,l,k}   + 2    \tau^{18}_{k,j,i,l}  - 2    \tau^{18}_{l,j,i,k}  - \sum_{a \in V} \sum_{b \in V} \tau^{19}_{l,k,a,b}  u_{j,i,a,b} $$

$$\tau^{21}_{a,b} = \sum_{i \in O} f_{i,a}  t_{b,i} $$

$$\tau^{22}_{a,b} = - \sum_{i \in O} \sum_{c \in V} t_{c,i}  u_{i,a,c,b} $$

$$\tau^{23}_{a,b} = \sum_{i \in O} \sum_{j \in O} \sum_{c \in V} \tau^{0}_{i,j,c,b}  u_{i,j,c,a} $$

$$\tau^{24}_{a,b} = - 2    f_{a,b}   + 2    \tau^{21}_{b,a}   + 2    \tau^{22}_{a,b}   + \tau^{23}_{b,a} $$

$$\tau^{25}_{i,j,a,b} = \sum_{c \in V} \tau^{24}_{a,c}  t_{c,b,i,j} $$

$$\tau^{26}_{i,j,k,a} = \sum_{b \in V} t_{b,i}  u_{j,a,k,b} $$

$$\tau^{27}_{i,j,a,b} = u_{i,a,j,b}  - \tau^{9}_{j,i,a,b} $$

$$\tau^{28}_{i,j,k,l} = u_{j,i,l,k}  - \tau^{18}_{l,j,i,k} $$

$$\tau^{29}_{i,j,k,a} = - u_{i,a,k,j}  - \tau^{26}_{j,i,k,a}   + \sum_{b \in V} t_{b,k}  \tau^{27}_{i,j,a,b}  - \sum_{l \in O} t_{a,l}  \tau^{28}_{l,i,k,j} $$

$$\tau^{30}_{i,a,b,c} = - u_{b,a,i,c}   + \sum_{d \in V} t_{d,i}  u_{a,b,d,c} $$

$$\tau^{31}_{i,j,a,b} = \sum_{k \in O} \tau^{7}_{k,j}  t_{a,b,k,i} $$

$$\tau^{32}_{i,j,k,a} = u_{j,a,k,i}   + \tau^{26}_{i,j,k,a} $$

$$e = \sum_{i \in O} \sum_{a \in V} f_{i,a}  t_{a,i}   + \sum_{i \in O} \sum_{j \in O} \sum_{a \in V} \sum_{b \in V} \frac{1}{4}    \tau^{0}_{i,j,a,b}  u_{i,j,a,b} $$

$$t_{a,i} = - \sum_{j \in O} \sum_{b \in V} t_{b,j}  u_{j,a,i,b}   + \sum_{b \in V} f_{a,b}  t_{b,i}   + f_{a,i}  - \sum_{j \in O} \sum_{k \in O} \sum_{b \in V}  \frac{1}{2}    t_{b,a,j,k}  \tau^{2}_{i,j,k,b}   + \sum_{j \in O} \sum_{b \in V} \sum_{c \in V} \frac{1}{2}    \tau^{0}_{j,i,b,c}  u_{j,a,b,c}   + \sum_{j \in O} \sum_{b \in V} \tau^{3}_{j,b}  t_{b,a,j,i}  - \sum_{j \in O}  \frac{1}{2}    t_{a,j}  \tau^{7}_{j,i} $$

$$t_{a,b,i,j} = u_{b,a,j,i}  - \sum_{c \in V} t_{c,i}  u_{a,b,j,c}   + \tau^{13}_{i,j,a,b}  - \tau^{13}_{j,i,a,b}   + \sum_{c \in V} \sum_{d \in V} \frac{1}{2}    t_{c,d,j,i}  \tau^{15}_{b,a,c,d}  - \sum_{k \in O} \sum_{c \in V} \tau^{10}_{k,i,c,a}  \tau^{17}_{j,k,b,c}   + \sum_{k \in O} \sum_{c \in V} t_{c,a,k,j}  \tau^{17}_{i,k,b,c}   + \sum_{k \in O} \sum_{l \in O} \frac{1}{4}    t_{b,a,k,l}  \tau^{20}_{k,l,j,i}  -  \frac{1}{2}    \tau^{25}_{j,i,b,a}   + \frac{1}{2}    \tau^{25}_{j,i,a,b}  - \sum_{k \in O} t_{b,k}  \tau^{29}_{k,j,i,a}  - \sum_{c \in V} t_{c,j}  \tau^{30}_{i,b,a,c}   + \frac{1}{2}    \tau^{31}_{j,i,b,a}  -  \frac{1}{2}    \tau^{31}_{i,j,b,a}  - \sum_{k \in O} t_{a,k}  \tau^{32}_{j,k,i,b} $$


# Universidade Federal do Amazonas
# Departamento de Engenharia de Produção

## - Projeto Super 2021 - Curso Python 2.0

### Aula 3 - Biblioteca Numpy

**  Prof. William Costa **

## Aula 3 - Biblioteca Numpy

# - Finalizando o contéudo da aula passada (derivada, integral, solver)
### Biblioteca sympy - matemática simbólica

```python
# Importando a biblioteca sympy
import sympy as sp
sp.init_printing()
```

```python
# Criando um funçao sin(x)**2
x = sp.symbols('x')
fx = sp.sin(x)**2
fx
```

```python
# DERIVADA
# sp.diff() - calcula a derivada de uma função
C = sp.symbols('C')
dfdx = sp.diff(fx,x)
dfdx += C
dfdx
```

```python
# Derivando a expressão em funçao de x
x,y = sp.symbols(['x','y'])
sp.init_printing()

```

```python
# Derivando uma expressão duas ou mais vezes - derivada de 2a ordem , 3a, ...
#expres
expres = sp.exp(2*x) + sin(x)
expres
```

```python
# Calcular a derivada
dfdx = sp.diff(expres,x)
dfdx
```

```python
# Calcular a  1a derivada de fx
df2dx = sp.diff(expres,x,x)
df2dx
```

```python
# Exemplo - fx = x**3 + 5* x**2 + x
# Atividade - escrever a expressão acima usando a matemática simbólica
# calcular a primeira derivada da funçao  fx

```

```python
##### COMO ESCREVER/EXPRESSAR NA FORMA df/dx
y = x**2 + x -1
y
sp.Derivative(y,x)
```

```python
# Escrever de forma simbólica a derivada :
# f = sp.exp(2*x) + sp.sin(x)
```

## INTEGRAL -

```python
#  definida e indefinida
# sp.integrate()
import sympy as sp
sp.init_printing()

# criando as variaveis simbólicas x , y e z
x, y, z= sp.symbols(['x','y','z'])
f = sp.cos(2*x) * sp.sin(3*x)
f
sp.integrate(f,x)
```

```python
#  definida e indefinida
# sp.integrate()
import sympy as sp
sp.init_printing()

# criando as variaveis simbólicas x , y e z
x, y, z= sp.symbols(['x','y','z'])
f = sp.cos(2*x) * sp.sin(3*x)

sp.integrate(f,x) + C
```

```python
# Integrando a função
# .integrate(funcao,arg)
# Integral indefinida
# definindo uma função

```

```python
# Exemplo - f(x) = x**3
# Escrever a expressão com a simbologia matemática
# depois calcular integral indefinida da função f(x)
```

```python
# Integral Definida
# sp.integrate(funcao,(variavel,L-inf,L-sup))
# Exemplo anterior funcao f , limite inferior = 0 e limite superior=pi
f = x**2 + 1
f
sp.integrate(f,(x,0,5))
```

```python
# Integral dupla (multipla vezes)
# definindo a função
w = 4 - y**2
w
sp.integrate(w,(y,0,2),(x,0,3))
```

```python
# Escrevendo a integral literalmente
# Integral(expresao)
import sympy as sp
sp.init_printing()
x = sp.symbols('x')
f = 1/x
f
Integral(f,(x,1,5))
```

```python
# Exemplo -Escreva a integral da função =x**2 + 4*x + 5

```

# - SOLVERS
** solveset(eq, arg) ou solve **

```python
# Declarando um expressão
import sympy as sp
sp.init_printing()

# Declarando uma expressão
x = sp.symbols('x')
fx = x**2 + 6 *x +9
fx
```

```python
# Encontrando a raiz da expressão - SOLVER
# sp.solveset()
sp.solveset(fx,x)
```

```python
# método sp.solve()
sp.solve(fx,x)
```

```python
# Exemplo: f(x) = x**3 + 5 *x**2 + x
# Escrever a funçao acima usando a matemática simbólica,depois
# Aplique o método .solve( ) ou solveset() e encontrar a raiz ou raízes da expressão

```

```python
# Aplicando o SOLVER sistemas lineares 3 x 3
x1,x2,x3 = sp.symbols(['x1','x2','x3'])
coef = sp.Matrix([[3,2,4],[1,1,2],[4,3,-2]])
X = sp.Matrix([[x1,x2,x3]]).T
R = sp.Matrix([[1,2,3]]).T

# coef * X = R
#coef*X-R=0
sistema = coef*X-R
resposta = sp.solve(sistema,(x1,x2,x3))
resposta
```

```python
X
```

$\displaystyle \left[\begin{matrix}x_{1}\\x_{2}\\x_{3}\end{matrix}\right]$

```python

```


# Symbolic Hermitian Anticommutator

```python
import sympy
from sympy.codegen.ast import Assignment
```

Define symbols for the minimum number of real variables required to store the 3x3 Hermitian matrices we need to calculate:

$\dfrac{\partial f}{\partial t} = \imath \left[ H, f \right]$

Matrix entries for $f$

```python
fee_R, fuu_R, ftt_R = sympy.symbols('fee_R fuu_R ftt_R', real=True)
feu_R, fet_R, fut_R = sympy.symbols('feu_R fet_R fut_R', real=True)
feu_I, fet_I, fut_I = sympy.symbols('feu_I fet_I fut_I', real=True)
```

Matrix entries for $\dfrac{\partial f}{\partial t}$

```python
Dfee_R, Dfuu_R, Dftt_R = sympy.symbols('Dfee_R Dfuu_R Dftt_R', real=True)
Dfeu_R, Dfet_R, Dfut_R = sympy.symbols('Dfeu_R Dfet_R Dfut_R', real=True)
Dfeu_I, Dfet_I, Dfut_I = sympy.symbols('Dfeu_I Dfet_I Dfut_I', real=True)
```

Matrix entries for $H$

```python
Hee_R, Huu_R, Htt_R = sympy.symbols('Hee_R Huu_R Htt_R', real=True)
Heu_R, Het_R, Hut_R = sympy.symbols('Heu_R Het_R Hut_R', real=True)
Heu_I, Het_I, Hut_I = sympy.symbols('Heu_I Het_I Hut_I', real=True)
```

Define matrices $f$ and $H$ to be Hermitian by construction:

```python
F = sympy.Matrix([[fee_R, feu_R + sympy.I * feu_I, fet_R + sympy.I * fet_I],
                  [feu_R - sympy.I * feu_I, fuu_R, fut_R + sympy.I * fut_I],
                  [fet_R - sympy.I * fet_I, fut_R - sympy.I * fut_I, ftt_R]])
```

```python
H = sympy.Matrix([[Hee_R, Heu_R + sympy.I * Heu_I, Het_R + sympy.I * Het_I],
                  [Heu_R - sympy.I * Heu_I, Huu_R, Hut_R + sympy.I * Hut_I],
                  [Het_R - sympy.I * Het_I, Hut_R - sympy.I * Hut_I, Htt_R]])
```

Calculate anticommutator $[H,f] = H \cdot f - f \cdot H$

```python
AC = H*F - F*H
```

Calculate $\dfrac{\partial f}{\partial t} = \imath \left[ H, f \right]$

```python
dFdt = sympy.I * AC
```

Assign elements of the $\dfrac{\partial f}{\partial t}$ matrix to the real values we need to store the matrix

```python
iE = 0
iU = 1
iT = 2

dfee_R_dt = Assignment(Dfee_R, sympy.re(dFdt[iE,iE]))
dfuu_R_dt = Assignment(Dfuu_R, sympy.re(dFdt[iU,iU]))
dftt_R_dt = Assignment(Dftt_R, sympy.re(dFdt[iT,iT]))

dfeu_R_dt = Assignment(Dfeu_R, sympy.re(dFdt[iE,iU]))
dfet_R_dt = Assignment(Dfet_R, sympy.re(dFdt[iE,iT]))
dfut_R_dt = Assignment(Dfut_R, sympy.re(dFdt[iU,iT]))

dfeu_I_dt = Assignment(Dfeu_I, sympy.im(dFdt[iE,iU]))
dfet_I_dt = Assignment(Dfet_I, sympy.im(dFdt[iE,iT]))
dfut_I_dt = Assignment(Dfut_I, sympy.im(dFdt[iU,iT]))
```

Define a function to return a code string for calculating the independent variables we need to store $\dfrac{\partial f}{\partial t}$

```python
def get_rhs_code():
    lines = []

    lines.append(sympy.ccode(dfee_R_dt))
    lines.append(sympy.ccode(dfuu_R_dt))
    lines.append(sympy.ccode(dftt_R_dt))

    lines.append(sympy.ccode(dfeu_R_dt))
    lines.append(sympy.ccode(dfet_R_dt))
    lines.append(sympy.ccode(dfut_R_dt))

    lines.append(sympy.ccode(dfeu_I_dt))
    lines.append(sympy.ccode(dfet_I_dt))
    lines.append(sympy.ccode(dfut_I_dt))

    for l in lines:
        print(l + "\n")

    return "\n".join(lines)
```

```python
code_string = get_rhs_code()
```

    Dfee_R = -2*Het_I*fet_R + 2*Het_R*fet_I - 2*Heu_I*feu_R + 2*Heu_R*feu_I;

    Dfuu_R = 2*Heu_I*feu_R - 2*Heu_R*feu_I - 2*Hut_I*fut_R + 2*Hut_R*fut_I;

    Dftt_R = 2*Het_I*fet_R - 2*Het_R*fet_I + 2*Hut_I*fut_R - 2*Hut_R*fut_I;

    Dfeu_R = -Hee_R*feu_I - Het_I*fut_R + Het_R*fut_I + Heu_I*fee_R - Heu_I*fuu_R - Hut_I*fet_R + Hut_R*fet_I + Huu_R*feu_I;

    Dfet_R = -Hee_R*fet_I + Het_I*fee_R - Het_I*ftt_R - Heu_I*fut_R - Heu_R*fut_I + Htt_R*fet_I + Hut_I*feu_R + Hut_R*feu_I;

    Dfut_R = Het_I*feu_R - Het_R*feu_I + Heu_I*fet_R - Heu_R*fet_I + Htt_R*fut_I - Hut_I*ftt_R + Hut_I*fuu_R - Huu_R*fut_I;

    Dfeu_I = Hee_R*feu_R + Het_I*fut_I + Het_R*fut_R - Heu_R*fee_R + Heu_R*fuu_R - Hut_I*fet_I - Hut_R*fet_R - Huu_R*feu_R;

    Dfet_I = Hee_R*fet_R - Het_R*fee_R + Het_R*ftt_R - Heu_I*fut_I + Heu_R*fut_R - Htt_R*fet_R + Hut_I*feu_I - Hut_R*feu_R;

    Dfut_I = -Het_I*feu_I - Het_R*feu_R + Heu_I*fet_I + Heu_R*fet_R - Htt_R*fut_R + Hut_R*ftt_R - Hut_R*fuu_R + Huu_R*fut_R;

```python

```