## General Mixture Models

author: Jacob Schreiber <br>
contact: jmschreiber91@gmail.com

It is frequently the case that the data you have is not explained by a single underlying distribution. Typically this is because there are multiple phenomena occuring in the data set, each with their own underlying distribution.  If we want to try to recover the underlying distributions, we need to have a model which has multiple components. An example could be sensor readings where the majority of the time a sensor shows no signal, but sometimes it detects some phenomena. Modeling both phenomena as a single distribution would be silly because the readings would come from two distinct phenomena.

A solution to the problem of having more than one single underlying distribution is to use a mixture of distributions instead of a single distribution, commonly called a mixture model. This type of compositional model builds a more complex probability distribution from a set of simpler ones. A common type, called a Gaussian Mixture Model, is composed of Gaussian distributions, but mathematically there is no need for these distributions to all be Gaussian. In fact, there is no need for these distributions to be simple probability distributions.

In this tutorial we'll explore how to do mixture modeling in pomegranate, compare against scikit-learn's implementation of Gaussian mixture models, and explore more complex types of mixture modeling that one can do with probabilistic modeling.

```python
%matplotlib inline
import matplotlib.pyplot as plt
import seaborn; seaborn.set_style('whitegrid')
import numpy

from pomegranate import *

numpy.random.seed(0)
numpy.set_printoptions(suppress=True)

%load_ext watermark
%watermark -m -n -p numpy,scipy,pomegranate
```

    Sat Oct 06 2018

    numpy 1.14.2
    scipy 1.0.0
    pomegranate 0.10.0

    compiler   : GCC 7.2.0
    system     : Linux
    release    : 4.15.0-34-generic
    machine    : x86_64
    processor  : x86_64
    CPU cores  : 4
    interpreter: 64bit

### A simple example: Gaussian mixture models

Let's start off with a simple example. Perhaps we have a data set like the one below, which is made up of not a single blob, but many blobs. It doesn't seem like any of the simple distributions that pomegranate has implemented can fully capture what's going on in the data.

```python
X = numpy.concatenate([numpy.random.normal((7, 2), 1, size=(100, 2)),
                       numpy.random.normal((2, 3), 1, size=(150, 2)),
                       numpy.random.normal((7, 7), 1, size=(100, 2))])

plt.figure(figsize=(8, 6))
plt.scatter(X[:,0], X[:,1])
plt.show()
```

It seems clear to us that this data is composed of three blobs. Accordingly, rather than trying to find some complex single distribution that can describe this data, we can describe it as a mixture of three Gaussian distributions. In the same way that we could initialize a basic distribution using the `from_samples` method, we can initialize a mixture model using it, additionally passing in the type(s) of distribution(s) to use and the number of components.

```python
model = GeneralMixtureModel.from_samples(MultivariateGaussianDistribution, 3, X)
```

Now we can look at the probability densities if we had used a single Gaussian distribution versus using this mixture of three Gaussian models.

```python
x = numpy.arange(-1, 10.1, .1)
y = numpy.arange(-1, 10.1, .1)

xx, yy = numpy.meshgrid(x, y)
x_ = numpy.array(list(zip(xx.flatten(), yy.flatten())))

p1 = MultivariateGaussianDistribution.from_samples(X).probability(x_).reshape(len(x), len(y))
p2 = model.probability(x_).reshape(len(x), len(y))

plt.figure(figsize=(14, 6))
plt.subplot(121)
plt.contourf(xx, yy, p1, cmap='Blues', alpha=0.8)
plt.scatter(X[:,0], X[:,1])

plt.subplot(122)
plt.contourf(xx, yy, p2, cmap='Blues', alpha=0.8)
plt.scatter(X[:,0], X[:,1])
plt.show()
```

It looks like, unsurprisingly, the mixture model is able to better capture the structure of the data. The single model is so bad that the region of highest density (the darkest blue ellipse) has very few real points in it. In contrast, the darkest regions in the mixture densities correspond to where there are the most points in the clusters.

### A more complex example: independent component mixture models

Most data is more complex than the Gaussian case above. A common case in the domain of signal processing is analyzing segments of steady signal based on the mean, noise, and duration of each segment. Let's say that we have a signal like the following which is made up of long low-noise signals and shorter high-noise signals.

```python
mu = numpy.random.normal(7, 1, size=250)

std = numpy.random.lognormal(-2.0, 0.4, size=250)
std[::2] += 0.3

dur = numpy.random.exponential(250, size=250)
dur[::2] -= 140
dur = numpy.abs(dur)

data = numpy.concatenate([numpy.random.normal(mu_, std_, int(t)) for mu_, std_, t in zip(mu, std, dur)])

plt.figure(figsize=(14, 4))
plt.title("Randomly Generated Signal", fontsize=16)
plt.plot(data)
plt.xlabel("Time", fontsize=14)
plt.ylabel("Signal", fontsize=14)
plt.xlim(0, 10000)
plt.show()
```

We can show the distribution of segment properties to highlight the differences.

```python
plt.figure(figsize=(14, 4))
plt.subplot(131)
plt.title("Segment Means", fontsize=14)
plt.hist(mu[::2], bins=20, alpha=0.7)
plt.hist(mu[1::2], bins=20, alpha=0.7)

plt.subplot(132)
plt.title("Segment STDs", fontsize=14)
plt.hist(std[::2], bins=20, alpha=0.7)
plt.hist(std[1::2], bins=20, alpha=0.7)

plt.subplot(133)
plt.title("Segment Durations", fontsize=14)
plt.hist(dur[::2], bins=numpy.arange(0, 1000, 25), alpha=0.7)
plt.hist(dur[1::2], bins=numpy.arange(0, 1000, 25), alpha=0.7)
plt.show()
```

In this situation, it looks like the segment noise is going to be the feature with the most difference between the learned distributions. Let's use pomegranate to learn this mixture, modeling each feature with an appropriate distribution.

```python
X = numpy.array([mu, std, dur]).T.copy()

model = GeneralMixtureModel.from_samples([NormalDistribution, LogNormalDistribution, ExponentialDistribution], 2, X)
model
```

    {
        "weights" : [
            0.5126040711398554,
            0.4873959288601447
        ],
        "distributions" : [
            {
                "frozen" : false,
                "class" : "Distribution",
                "parameters" : [
                    [
                        {
                            "frozen" : false,
                            "class" : "Distribution",
                            "parameters" : [
                                6.987501980099923,
                                0.9214383552866655
                            ],
                            "name" : "NormalDistribution"
                        },
                        {
                            "frozen" : false,
                            "class" : "Distribution",
                            "parameters" : [
                                -0.8201943718176836,
                                0.13668606405233233
                            ],
                            "name" : "LogNormalDistribution"
                        },
                        {
                            "frozen" : false,
                            "class" : "Distribution",
                            "parameters" : [
                                0.0055513741389845135
                            ],
                            "name" : "ExponentialDistribution"
                        }
                    ],
                    [
                        1.0,
                        1.0,
                        1.0
                    ]
                ],
                "name" : "IndependentComponentsDistribution"
            },
            {
                "frozen" : false,
                "class" : "Distribution",
                "parameters" : [
                    [
                        {
                            "frozen" : false,
                            "class" : "Distribution",
                            "parameters" : [
                                7.01255052332367,
                                0.9030007437921286
                            ],
                            "name" : "NormalDistribution"
                        },
                        {
                            "frozen" : false,
                            "class" : "Distribution",
                            "parameters" : [
                                -1.9984343269872586,
                                0.37492907121195773
                            ],
                            "name" : "LogNormalDistribution"
                        },
                        {
                            "frozen" : false,
                            "class" : "Distribution",
                            "parameters" : [
                                0.003718687576817513
                            ],
                            "name" : "ExponentialDistribution"
                        }
                    ],
                    [
                        1.0,
                        1.0,
                        1.0
                    ]
                ],
                "name" : "IndependentComponentsDistribution"
            }
        ],
        "class" : "GeneralMixtureModel"
    }

It looks like the model is easily able to fit the differences that we observed in the data. The durations of the two components are similar, the means are almost identical to each other, but it's the noise that's the most different between the two.

## The API

### Initialization

The API for general mixture models is similar to that of the rest of pomegranate. The model can be initialized either through passing in some pre-initialized distributions or by calling `from_samples` on a data set, specifying the types of distributions you'd like.

You can simply pass in some distributions to initialize the mixture.

```python
model = GeneralMixtureModel([NormalDistribution(4, 1), NormalDistribution(7, 1)])
```

There is no reason these distributions have to be the same type. You can create a non-homogenous mixture by passing in different types of distributions.

```python
model2 = GeneralMixtureModel([NormalDistribution(5, 1), ExponentialDistribution(0.3)])
```

They will produce very different probability distributions, but both work as a mixture.

```python
x = numpy.arange(0, 10.01, 0.05)

plt.figure(figsize=(14, 3))
plt.subplot(121)
plt.title("~Norm(4, 1) + ~Norm(7, 1)", fontsize=14)
plt.ylabel("Probability Density", fontsize=14)
plt.fill_between(x, 0, model.probability(x))
plt.ylim(0, 0.25)

plt.subplot(122)
plt.title("~Norm(5, 1) + ~Exp(0.3)", fontsize=14)
plt.ylabel("Probability Density", fontsize=14)
plt.fill_between(x, 0, model2.probability(x))
plt.ylim(0, 0.25)
plt.show()
```

pomegranate offers a lot of flexibility when it comes to making mixtures directly from data. The normal option is just to specify the type of distribution, the number of components, and then pass in the data. Make sure that if you're making 1 dimensional data that you're still passing in a matrix whose second dimension is just set to 1.

```python
X = numpy.random.normal(3, 1, size=(200, 1))
X[::2] += 3

model = GeneralMixtureModel.from_samples(NormalDistribution, 2, X)
```

We can take a quick look at what it looks like:

```python
plt.figure(figsize=(6, 3))
plt.title("Learned Model", fontsize=14)
plt.ylabel("Probability Density", fontsize=14)
plt.fill_between(x, 0, model.probability(x))
plt.ylim(0, 0.25)
plt.show()
```

However, we can also fit a non-homogenous mixture by passing in a list of univariate distributions to be fit to univariate data. Let's try to fit to data of which some of it is normally distributed and some is exponentially distributed.

```python
X = numpy.concatenate([numpy.random.normal(5, 1, size=(200, 1)), numpy.random.exponential(1, size=(50, 1))])

model = GeneralMixtureModel.from_samples([NormalDistribution, ExponentialDistribution], 2, X)
```

```python
x = numpy.arange(0, 12.01, 0.01)

plt.figure(figsize=(6, 3))
plt.title("Learned Model", fontsize=14)
plt.ylabel("Probability Density", fontsize=14)
plt.fill_between(x, 0, model.probability(x))
plt.ylim(0, 0.5)
plt.show()
```

It looks like the mixture is capturing the distribution fairly well.

Next, if we want to make a mixture model that describes each feature using a different distribution, we can pass in a list of distributions---one or each feature---and it'll use that distribution to model the corresponding feature. For example, in the example below, the first feature will be modeled using a normal distribution, the second feature will be modeled using a log normal distribution, and the last feature will be modeled using an exponential distribution.

```python
X = numpy.array([mu, std, dur]).T.copy()

model = GeneralMixtureModel.from_samples([NormalDistribution, LogNormalDistribution, ExponentialDistribution], 2, X)
```

This is the same command we used in the second example. It creates two components, each of which model the three features respectively with those three distributions. The difference between this and the previous example is that the data here is multivariate and so univariate distributions are assumed to model different distributions.

### Prediction

Now that we have a mixture model, we can make predictions as to which component it is most likely to fall under. This is similar to the situation in which you predict which cluster a point falls under in k-means clustering. However, one of the benefits of using probabilistic models is that we can get a softer assignment of points based on probability, rather than a hard assignment to clusters for each point.

```python
X = numpy.random.normal(5, 1, size=(500, 1))
X[::2] += 3

x = numpy.arange(0, 10, .01)

model = GeneralMixtureModel.from_samples(NormalDistribution, 2, X)

plt.figure(figsize=(8, 4))
plt.hist(X, bins=50, normed=True)
plt.plot(x, model.distributions[0].probability(x), label="Distribution 1")
plt.plot(x, model.distributions[1].probability(x), label="Distribution 2")
plt.plot(x, model.probability(x), label="Mixture")
plt.legend(fontsize=14, loc=2)
plt.show()
```

The prediction task is identifying, for each point, whether it falls under the orange distribution or the green distribution. This is done using Bayes' rule, where the probability of each sample under each component is divided by the probability of the sample under all components of the mixture. The posterior probability of a sample belonging to component $i$ out of $k$ components is

\begin{equation}
P(M_{i}|D) = \frac{P(D|M_{i})P(M_{i})}{\sum\limits_{i=1}^{k} P(D|M_{i})P(M_{i})}
\end{equation}

where $D$ is the data, $M_{i}$ is the component of the model (such as the green or orange distributions), $P(D|M_{i}$ is the likelihood calculated by the `probability` function, $P(M_{i})$ is the prior probability of each component, stored under the `weights` attribute of the model, and $P(M_{i}|D)$ is the posterior probability that sums to 1 over all components.

We can calculate these posterior probabilites using the `predict_proba` method.

```python
X = numpy.arange(4, 9, 0.5).reshape(10, 1)

model.predict_proba(X)
```

    array([[0.99969547, 0.00030453],
           [0.99849679, 0.00150321],
           [0.99284427, 0.00715573],
           [0.96765651, 0.03234349],
           [0.86944497, 0.13055503],
           [0.60478603, 0.39521397],
           [0.26632011, 0.73367989],
           [0.08163   , 0.91837   ],
           [0.02197475, 0.97802525],
           [0.00582886, 0.99417114]])

As we scan from left to right, we can see that the probability of the first component (in orange) decreases, while the probability of the second component increases. This makes sense, because the green component is centered at a higher value. The posterior probability is not the probability of the point given the distribution, but rather the probability of the distribution given the point, i.e., which component is more likely to have generated the distribution.


Copyright (c) 2020 Juergen Koefinger, Max Planck Institute of Biophysics, Frankfurt am Main, Germany

Released under the MIT Licence, see the file LICENSE.txt.

# Descripton

**Perform the $h$, $h^\pm$, $(\chi^2,h)$, and $(\chi^2,h^\pm)$ statistical tests.**

The input are the **normalized residuals**, which are read from a file.

If no input is provided, you can perfrom the statistical tests for the true mode (i.e., without noise, all residuals are zero) as a check for consistency.

# Initialization

```python
# Plotting
import matplotlib
import matplotlib.pylab as plt
%matplotlib inline
matplotlib.rc('xtick', labelsize=16)
matplotlib.rc('ytick', labelsize=16)
matplotlib.rc('axes', labelsize=16)
# numpy and scipy
import numpy as np
import scipy
from scipy.stats import chi2
# hplusminus
import sys
sys.path.append("../")
import hplusminus as hpm
```

```python
# Parameters for gamma distribution used to calculate p-values
gamma_param=hpm.init()
```

# Read in normalized residuals (residuals divided by standard errors of the mean)

For uncorrelated Gaussian errors, the **residuals $r_i$** are distributed accorrding to $\propto \exp{-r_i^2/(2 \sigma_i^2)}$. The  residuals $r_i$ are given by the difference between model value $f_i$ and the data value $d_i$, i.e., $r_i=f_i-d_i$. The normalized residuals are given by
\begin{equation}
z_i = \frac{r_i}{\sigma_i}
\end{equation}
We consider ordered data, with the index $i=1, \ldots, N$ determining the order.

```python
i_path="../examples/"
```

```python
# Name of file containing normalized residuals. REPLACE WITH YOUR OWN!
file_name=i_path+"./true_model_normalized_residuals.txt"
file_name=i_path+"./alternative_model_normalized_residuals.txt"
#Set file_name="" and set number of data points to use true model.
#file_name=""
#number_data_points=500
```

```python
if file_name=="":
    # Generate normalized residuals by drawing residuals from a normal distribution for TESTING
    try:
        print("Generating residuals for true model")
        print("number_data_points = %d" % number_data_points)
    except:
        print("Error: Set positive integer value for number_data_points")
    normalized_residuals = np.random.normal(0, 1, number_data_points)
    np.savetxt(io_path+"normalized_residuals.txt", normalized_residuals)
else:
    # Loading normalized residuals (residuals divided by standard errors of the mean) from file
    print("Loading residuals from \"%s\"" % file_name)
    normalized_residuals=np.loadtxt(file_name)
    number_data_points=len(normalized_residuals)
```

## Plotting the normalized residuals

```python
plt.plot(normalized_residuals)
plt.grid()
plt.xlabel("index $i$")
plt.ylabel("normalized residuals $r_i/\sigma_i$")
```

# Evaluate statistical tests

```python
signs=np.sign(normalized_residuals)
chi_square=(normalized_residuals**2).sum()
```

## Plotting the signs of the residuals

```python
plt.plot(signs)
plt.grid()
plt.xlabel("index $i$")
plt.ylabel("signs $s_i$")
```

```python
# Calculate run-length histograms
num, blockLen, histo, edges = hpm.get_run_length_distributions(signs)
```

```python
# Dictionary for Shannon information for various tests
I={}
# Dictionary for p-values for various tests
p_value={}
```

## $\chi^2$-test

```python
I['chi2']=hpm.SI_chi2(chi_square, number_data_points)
```

```python
print("Shannon information = %3.2f" % I['chi2'])
```

```python
p_value['chi2']=hpm.get_p_value(I['chi2'], number_data_points, 'chi2', gamma_param)
```

```python
print("p-value = %3.2e" % p_value['chi2'])
```

## $h^\pm$-test

```python
I['hpm']=hpm.SI_hpm(number_data_points, num[1], histo['plus'], histo['minus'])
```

```python
print("Shannon information = %3.2f" % I['hpm'])
```

```python
p_value['hpm']=hpm.get_p_value(I['hpm'], number_data_points, 'hpm', gamma_param)
```

```python
print("p-value = %3.2e" % p_value['hpm'])
```

## $h$-test

```python
I['h']=hpm.SI_h(number_data_points, histo['all'])
```

```python
print("Shannon information = %3.2f" % I['h'])
```

```python
p_value['h']=hpm.get_p_value(I['h'], number_data_points, 'h', gamma_param)
```

```python
print("p-value = %3.2e" % p_value['h'])
```

## $(\chi^2, h)$-test

```python
I['chi2_h'] = I['h'] + I['chi2']
```

```python
print("Shannon information = %3.2f" % I['chi2_h'])
```

```python
p_value['chi2_h']=hpm.get_p_value(I['chi2_h'], number_data_points, 'chi2_h', gamma_param)
```

```python
print("p-value = %3.2e" % p_value['chi2_h'])
```

## $(\chi^2, h^\pm)$-test

```python
I['chi2_hpm'] = I['hpm'] + I['chi2']
```

```python
print("Shannon information = %3.2f" % I['chi2_hpm'])
```

```python
p_value['chi2_hpm']=hpm.get_p_value(I['chi2_hpm'], number_data_points, 'chi2_hpm', gamma_param)
```

```python
print("p-value = %3.2e" % p_value['chi2_hpm'] )
```

# Plotting the p-values

```python
fig = plt.figure()
ax=plt.gca()
ax.minorticks_on()
ax.tick_params(axis='y', which='minor', right=False)
ax.tick_params(axis='x', which='minor', bottom=False)
ax.bar([r"$\chi^2$", r"$h$", r"$h^\pm$", r"$(\chi^2, h)$", r"$(\chi^2, h^\pm)$", ],
       [p_value['chi2'],  p_value['h'] ,  p_value['hpm'] ,p_value['chi2_h'] , p_value['chi2_hpm'] ],
       0.5, zorder=100)
ax.set_yscale('log')
ax.set_ylabel("p-value")
ax.yaxis.set_label_coords(-0.2, 0.5)
ax.set_yticks([1, 1e-1, 1e-2, 1e-3, 1e-4, 1e-5, 1e-6])
ax.yaxis.grid()
```


**Notebook Outline:**

- [Setup with libraries](#Set-up-Cells)
- [Fundamental equations for Poisson MGWR](#Fundamental-equations-for-Poisson-MGWR)
- [Example Dataset](#Example-Dataset)
- [Helper functions](#Helper-functions)
- [Univariate example](#Univariate-example)
    - [Parameter check](#Parameter-check)
    - [Bandwidths check](#Bandwidths-check)

### Set up Cells

```python
import sys
sys.path.append("C:/Users/msachde1/Downloads/Research/Development/mgwr")
```

```python
import warnings
warnings.filterwarnings("ignore")
import pandas as pd
import numpy as np

from mgwr.gwr import GWR
from spglm.family import Gaussian, Binomial, Poisson
from mgwr.gwr import MGWR
from mgwr.sel_bw import Sel_BW
import multiprocessing as mp
pool = mp.Pool()
from scipy import linalg
import numpy.linalg as la
from scipy import sparse as sp
from scipy.sparse import linalg as spla
from spreg.utils import spdot, spmultiply
from scipy import special
import libpysal as ps
import seaborn as sns
import matplotlib.pyplot as plt
from copy import deepcopy
import copy
from collections import namedtuple
import spglm
```

### Fundamental equations for Poisson MGWR

\begin{align}
y = Poisson[E_i exp ({\sum} {\beta} & _k x _{k,i})]   - (1) \\
E_i = Offset   - (2) \\
\hat{y} = predicted(y)   - (3) \\
z = ({\sum} {\beta} & _k x _{k,i}) + ((y - \hat{y})/\hat{y})   - (4) \\
\end{align}

### Example Dataset

```python
data_p = ps.io.open(ps.examples.get_path('Tokyomortality.csv'))
coords = list(zip(data_p.by_col('X_CENTROID'),data_p.by_col('Y_CENTROID')))
off = np.array(data_p.by_col('eb2564')).reshape((-1,1))
y = np.array(data_p.by_col('db2564')).reshape((-1,1))
occ = np.array(data_p.by_col('OCC_TEC')).reshape((-1,1))
own = np.array(data_p.by_col('OWNH')).reshape((-1,1))
pop = np.array(data_p.by_col('POP65')).reshape((-1,1))
unemp = np.array(data_p.by_col('UNEMP')).reshape((-1,1))
X = np.hstack([occ,own,pop,unemp])
x = unemp

X_std = (X-X.mean(axis=0))/X.std(axis=0)
x_std = (x-x.mean(axis=0))/x.std(axis=0)
y_std = (y-y.mean(axis=0))/y.std(axis=0)
```

### Helper functions
Hardcoded here for simplicity in the notebook workflow

Please note: A separate bw_func_p will not be required when changes will be made in the repository

```python
kernel='bisquare'
fixed=False
spherical=False
search_method='golden_section'
criterion='AICc'
interval=None
tol=1e-06
max_iter=200
X_glob=[]
```

```python
def gwr_func(y, X, bw,family=Gaussian(),offset=None):
    return GWR(coords, y, X, bw, family,offset,kernel=kernel,
               fixed=fixed, constant=False,
               spherical=spherical, hat_matrix=False).fit(
                   lite=True, pool=pool)

def bw_func_p(coords,y, X):
    selector = Sel_BW(coords,y, X,family=Poisson(),offset=off, X_glob=[],
                      kernel=kernel, fixed=fixed,
                      constant=False, spherical=spherical)
    return selector

def bw_func(coords,y,X):
    selector = Sel_BW(coords,y,X,X_glob=[],
                      kernel=kernel, fixed=fixed,
                      constant=False, spherical=spherical)
    return selector

def sel_func(bw_func, bw_min=None, bw_max=None):
    return bw_func.search(
        search_method=search_method, criterion=criterion,
        bw_min=bw_min, bw_max=bw_max, interval=interval, tol=tol,
        max_iter=max_iter, pool=pool, verbose=False)
```

### Univariate example
#### GWPR model with independent variable, x = unemployment

```python
bw_gwpr=Sel_BW(coords,y_std,x_std,family=Poisson(),offset=off,constant=False).search()
```

```python
gwpr_model=GWR(coords,y_std,x_std,bw=bw_gwpr,family=Poisson(),offset=off,constant=False).fit()
```

#### MGWR Poisson loop with one independent variable, x = unemployment

##### Edited multi_bw function - original function in https://github.com/pysal/mgwr/blob/master/mgwr/search.py#L167

```python
def multi_bw(init,coords,y, X, n, k, family=Gaussian(),offset=None, tol=1e-06, max_iter=200, multi_bw_min=[None], multi_bw_max=[None],rss_score=False,bws_same_times=3,
                     verbose=False):

    if isinstance(family,spglm.family.Poisson):
        bw = sel_func(bw_func_p(coords,y,X))
        optim_model=gwr_func(y,X,bw,family=Poisson(),offset=offset)
        err = optim_model.resid_response.reshape((-1, 1))
        param = optim_model.params
        #This change for the Poisson model follows from equation (1) above
        XB = offset*np.exp(np.multiply(param, X))

    else:
        bw=sel_func(bw_func(coords,y,X))
        optim_model=gwr_func(y,X,bw)
        err = optim_model.resid_response.reshape((-1, 1))
        param = optim_model.params
        XB = np.multiply(param, X)

    bw_gwr = bw
    XB=XB

    if rss_score:
        rss = np.sum((err)**2)
    iters = 0
    scores = []
    delta = 1e6
    BWs = []
    bw_stable_counter = np.ones(k)
    bws = np.empty(k)

    try:
        from tqdm.auto import tqdm  #if they have it, let users have a progress bar
    except ImportError:

        def tqdm(x, desc=''):  #otherwise, just passthrough the range
            return x

    for iters in tqdm(range(1, max_iter + 1), desc='Backfitting'):
        new_XB = np.zeros_like(X)
        params = np.zeros_like(X)

        for j in range(k):
            temp_y = XB[:, j].reshape((-1, 1))
            temp_y = temp_y + err
            temp_X = X[:, j].reshape((-1, 1))

            #The step below will not be necessary once the bw_func is changed in the repo to accept family and offset as attributes
            if isinstance(family,spglm.family.Poisson):

                bw_class = bw_func_p(coords,temp_y, temp_X)

            else:
                bw_class = bw_func(coords,temp_y, temp_X)

            if np.all(bw_stable_counter == bws_same_times):
                #If in backfitting, all bws not changing in bws_same_times (default 3) iterations
                bw = bws[j]
            else:
                bw = sel_func(bw_class, multi_bw_min[j], multi_bw_max[j])
                if bw == bws[j]:
                    bw_stable_counter[j] += 1
                else:
                    bw_stable_counter = np.ones(k)

            #Changed gwr_func to accept family and offset as attributes
            optim_model = gwr_func(temp_y, temp_X, bw,family,offset)
            err = optim_model.resid_response.reshape((-1, 1))
            param = optim_model.params.reshape((-1, ))
            new_XB[:, j] = optim_model.predy.reshape(-1)
            params[:, j] = param
            bws[j] = bw

        num = np.sum((new_XB - XB)**2) / n
        den = np.sum(np.sum(new_XB, axis=1)**2)
        score = (num / den)**0.5
        XB = new_XB

        if rss_score:
            predy = np.sum(np.multiply(params, X), axis=1).reshape((-1, 1))
            new_rss = np.sum((y - predy)**2)
            score = np.abs((new_rss - rss) / new_rss)
            rss = new_rss
        scores.append(deepcopy(score))
        delta = score
        BWs.append(deepcopy(bws))

        if verbose:
            print("Current iteration:", iters, ",SOC:", np.round(score, 7))
            print("Bandwidths:", ', '.join([str(bw) for bw in bws]))

        if delta < tol:
            break

    print("iters = "+str(iters))
    opt_bws = BWs[-1]
    print("opt_bws = "+str(opt_bws))
    return (opt_bws, np.array(BWs), np.array(scores), params, err, bw_gwr)
```

##### Running the function with family = Poisson() and offset

```python
bw_mgwpr = multi_bw(init=None,coords=coords,y=y_std, X=x_std, n=262, k=x.shape[1], family=Poisson(),offset=off)
```

    iters = 2
    opt_bws = [178.]

##### Running without family and offset attributes runs the normal MGWR loop

```python
bw_mgwr = multi_bw(init=None,coords=coords,y=y_std, X=x_std, n=262, k=x.shape[1])
```

    iters = 1
    opt_bws = [73.]

### Parameter check

#### Difference in parameters from the GWPR model and MGWPR model

```python
max(bw_mgwpr[3]-gwpr_model.params)
```

    array([1.89357983e-05])

The parameters are not identical but the maximum difference in the parameters is to the order of 1e-05

### Bandwidths check

```python
bw_gwpr
```

    178.0

```python
bw_mgwpr[0]
```

    array([178.])

Bandwidths from both models is the same


# Simple Parametric-Interactive Plots

In this tutorial we are going to see how to create simple parametric-interactive plots, that is, plots where we can move sliders to change parameters. The word _simple_ refer to the fact that:

* the function expects one-output-value widgets in order to work properly, such as sliders, spinners, etc.
* only one plot is generated.
* we don't have many options regarding the overall layout.

The function that allows to do that is `iplot`, which stands for _interactive plot_. Keep in mind that it only works inside Jupyter Notebook, since it is based on [holoviz's panel](https://panel.holoviz.org/).

```python
from sympy import *
from spb.interactive import iplot
from spb.backends.k3d import KB
from spb.backends.matplotlib import MB
from spb.backends.plotly import PB
from spb.backends.bokeh import BB
```

```python
print(help(iplot))
```

## Example 1

In this example we are going to create a parameterized surface plot with one parameter, the damping coefficient. Note: the first time loading a 3D-interactive plot may takes a few seconds:

```python
x, y, d = symbols("x, y, d")
r = sqrt(x**2 + y**2)
expr = 10 * cos(r) * exp(-r * d)

iplot(
    (expr, (x, -10, 10), (y, -10, 10)),
    params = { d: (0.15, 0, 1) },
    title = "My Title",
    xlabel = "x axis",
    ylabel = "y axis",
    zlabel = "z axis",
    n = 100,
    threed = True,
    use_latex = True,
    backend = PB
)
```

* By moving the slider, the update only happens when we release the click from it. This behaviour improves user experience: it should minimize the lags while moving the sliders waiting for the previous evaluation to complete. Remember: in the background the symbolic expression is being numerically evaluated over the discretized domain.
* By default, ``iplot`` uses the backends we set in configuration settings in [Tutorial 3](tutorial-3.set-default-backends.ipynb). Alternatively, we can use a different backend by setting the ``backend`` keyword argument.
* ``(expr, (x, -10, 10), (y, -10, 10))``: with ``iplot``, we must always provide all the necessary ranges: it's required by internal algorithm in order to automatically detect the kind of expression we are trying to plot.
* ``params = { d: (0.15, 0, 1) }``: we specified the dictionary of parameters. In this case there is only one parameter:
  * the key represents the symbol
  * the value is a tuple of the form (default, min, max, N [optional], label [optional], spacing [optional]). In the above case, only the first three entries were provided. This tuple will be converted to a ``param.Number``, which represent a float number and will be rendered as a slider. Alternatively, we could have used [holoviz's param library](https://panel.holoviz.org/user_guide/Param.html):
  ```python
  import param
  parameters = {
      d: param.Number(0.15, softbounds=(0, 1), label="", step=0.025)
  }
  ```
  We can use any parameter that, once rendered in a GUI, returns a single numeric value. For example ``param.Parameter, param.Integer, param.Number, param.ObjectSelector, param.Boolean``.
* Next, we specifies the usual keyword argument to customize the plot. Note that ``n = 100`` sets the number of discretization points in both directions (x and y). Alternatively, we could set them separately with ``n1`` and ``n2``. Remember that ``iplot`` doesn't support adaptive algorithms.
* Since ``iplot`` is a very general function, there is a risk of ambiguity: the above expression can be plotted with contours or with a 3D surface. With ``threed = True`` we ask for a 3D surface plot. In the above example, set ``threed = False`` to get a contour plot.
* ``use_latex = True``: by default, the label of the slider will use the Latex code of the parameter-symbol to provide a legible experience (especially when dealing with symbols with subscripts, superscripts, ...); that's the theory. In practice though, at the time writing this tutorial it is a _hit or miss_ experience: most of the time this functionality doesn't work. If that is the case, then it is better to set ``use_latex = False``, which will display the string representation of the symbols.
* `iplot` returns two very different objects depending on the value of the keyword argument `show`:
  * `show=True` (default) returns a `panel` object that will be rendered in the output cell. We can prevent the rendering from happening by capturing this object in a variable, for example `p = iplot(...`. Then we can render it on a different cell by simply typing `p`.
  * `show=False` returns an instance of `InteractivePlot`, which can be used for debugging purposes.

## Example 2

Let's now plot three expressions having quite a lot of parameters. The only things the Reader needs to be aware of are:
* $z$ represents the position along the anular channel. It is the discretized domain;
* $r_{i}$ represents the inner radius of the channel;
* $r_{o}$ represents the outer radius of the channel;
* $r_{i} < r_{o}$, which is a physical condition.

```python
from assets import build_example
ro, ri, mdot, cp, hc, alpha, k, L, z, Tin, Pave, Twi, Two, Tp, _ = build_example()
display(Twi, Two, Tp)
```

As you can see, it is quite difficult to get a physical understanding of such expressions just by looking at their analytical form.

Also note that `Twi, Two, Tp` use a different number of parameters.

```python
Twi.free_symbols, Two.free_symbols, Tp.free_symbols
```

```python
%matplotlib widget

iplot(
    (Twi, (z, 0, 100), "Twi"),
    (Two, (z, 0, 100), "Two"),
    (Tp, (z, 0, 100), "Tp"),
    params = {
        ri: (0.2, 0.04, 0.5),
        ro: (0.4, 0.2, 1.6),
        L: (100, 25, 250),
        Pave: (1000, 400, 4000),
        Tin: (300, 100, 500),
        hc: (1, 0.4, 15),
        alpha: (0.031, 0.016, 0.031),
        mdot: (1, 0.5, 5),
        k: (0.2, 0.1, 2),
        cp: (15, 5, 25)
    },
    title = "Temperature distribution",
    xlabel = "Position [cm]",
    ylabel = "T [K]",
    ylim = (0, 3000),
    xlim = (0, 100),
    backend = MB
)
```

* Whenever we use it ``iplot`` with ``MatplotlibBackend``, we must use the magic line `%matplotlib widget`, otherwise the figure won't update. Note that with `MatplotlibBackend` the widgets will always be rendered below the figure.
* Independently of the number of parameters, ``iplot`` arranges the sliders in two columns. We can change the number of columns by setting ``ncols`` to some integer. We can also chose where to place the controls with the ``layout`` keyword argument, but this won't work with `MatplotlibBackend`. Read the documentation to find out the available options.
* Note that we set the sliders:
  * ``0.04 <= ri <= 0.5``
  * ``0.4 <= ro <= 1.6``

  Therefore, it is very well possible to break the physical condition $r_{i} < r_{o}$ (for example, $r_{i} = 0.5$ and $r_{o} = 0.4$), which would produce unphysical results. The selection of the bounds and the values of the sliders is critical, and we are responsible for it. Currently it is impossible to set relationships between parameters!

We can also change backend. Let's say we would like to generate a plot with Bokeh:

```python
iplot(
    (Twi, (z, 0, 100), "Twi"),
    (Two, (z, 0, 100), "Two"),
    (Tp, (z, 0, 100), "Tp"),
    params = {
        ri: (0.2, 0.04, 0.5),
        ro: (0.4, 0.2, 1.6),
        L: (100, 25, 250),
        Pave: (1000, 400, 4000),
        Tin: (300, 100, 500),
        hc: (1, 0.4, 15),
        alpha: (0.031, 0.016, 0.031),
        mdot: (1, 0.5, 5),
        k: (0.2, 0.1, 2),
        cp: (15, 5, 25)
    },
    title = "Temperature distribution",
    xlabel = "Position [cm]",
    ylabel = "T [K]",
    ylim = (0, 3000),
    xlim = (0, 100),
    backend = BB
)
```

As we can see, there are quite a few widgets in this plot. Maybe we are working with a small screen device, or maybe the width of Jupyter Notebook's main content area is limiting us. In such cases we can launch the plot on a different browser window to use all its available width. This only works with ``BokehBackend`` and ``PlotlyBackend``. Let's give it a try:

```python
t = iplot(
    (Twi, (z, 0, 100), "Twi"),
    (Two, (z, 0, 100), "Two"),
    (Tp, (z, 0, 100), "Tp"),
    params = {
        ri: (0.2, 0.04, 0.5),
        ro: (0.4, 0.2, 1.6),
        L: (100, 25, 250),
        Pave: (1000, 400, 4000),
        Tin: (300, 100, 500),
        hc: (1, 0.4, 15),
        alpha: (0.031, 0.016, 0.031),
        mdot: (1, 0.5, 5),
        k: (0.2, 0.1, 2),
        cp: (15, 5, 25)
    },
    title = "Temperature distribution",
    xlabel = "Position [cm]",
    ylabel = "T [K]",
    ylim = (0, 3000),
    xlim = (0, 100),
    backend = BB,
    layout = "sbl",
    ncols = 1,
    size = (800, 600),
    show = True
)
t.show()
```

Here, the ``panel`` object created by ``iplot`` has been "captured" into the variable ``t``. With ``t.show()`` we are launching a new server process that will server the interactive plot on a new browser window. Note that we layed out the widgets differently and we also increased the size of the plot.

## Example 3 - Using the param library

In this example we are going to illustrate the use of [holoviz's param library](https://panel.holoviz.org/user_guide/Param.html).

Let's say we would like to visualize the Fourier Series approximation of a [sawtooth wave](https://mathworld.wolfram.com/SawtoothWave.html), defined as:

$$
S(x) = \operatorname{frac}{\left(\frac{x}{T}\right)}
$$

where $T$ is the period. Its [Fourier Series](https://mathworld.wolfram.com/FourierSeriesSawtoothWave.html) is:
$$
\frac{1}{2} - \frac{1}{\pi} \sum_{n=1}^{\infty} \frac{1}{n} \sin{\left(\frac{2 \pi n x}{T} \right)}
$$

```python
x, T, n, m = symbols("x, T, n, m")
sawtooth = frac(x / T)
# Fourier Series of a sawtooth wave
# https://mathworld.wolfram.com/FourierSeriesSawtoothWave.html
fs = S(1) / 2 - (1 / pi) * Sum(sin(2 * n * pi * x / T) / n, (n, 1, m))
display(sawtooth, fs)
```

Note that we stopped the Fourier series at $m$ rathen than $\infty$, because $m$ represents the upper bound of the approximation.

In the above expressions:
* $T$ is a float number, therefore we can use the tuple-sintax used before.
* $n$ is an integer number, therefore we must specify an integer parameter with `param.Integer`.

```python
import param

iplot(
    (sawtooth, (x, 0, 10), "f"),
    (fs, (x, 0, 10), "approx"),
    params = {
        T: (2, 0, 10),
        m: param.Integer(3, bounds=(1, None), label="Sum up to n ")
    },
    xlabel = "x",
    ylabel = "y",
    backend = BB
)
```

```python

```


# Time dependent tensile response

```python
%matplotlib widget
import matplotlib.pylab as plt
from bmcs_beam.tension.time_dependent_cracking import TimeDependentCracking
```

```python
import sympy as sp
sp.init_printing()
import numpy as np
```

# Single material point

## Time dependent function

```python
TimeDependentCracking(T_prime_0 = 100).interact()
```

    VBox(children=(Output(), Tab(children=(VBox(children=(GridBox(children=(FloatSlider(value=45.0, description='\…

### Time-dependent temperature evolution function

Find a suitable continuous function that can represent the temperature evolution during the hydration. Currently the a function of a Weibull type has been chosen and transformed such that the peak value and the corresponding time can be specified as a parameter.

```python
t = sp.symbols('t', nonnegative=True)
```

```python
T_m = sp.Symbol("T_m", positive = True)
T_s = sp.Symbol("T_s", positive = True)
```

```python
omega_fn = 1 - sp.exp(-(t/T_s)**T_m)
```

```python
T_prime_0 = sp.Symbol("T_prime_0", positive = True)
```

```python
T_t = (1 - omega_fn) * T_prime_0 * t
```

**Shape functions for temperature evolution**

```python
T_t
```

```python
T_prime_t = sp.simplify(T_t.diff(t))
T_prime_t
```

**Transform the shape function**
to be able to explicitly specify the maximum temperature and corresponding time

```python
t_argmax_T = sp.Symbol("t_argmax_T")
T_s_sol = sp.solve( sp.Eq( sp.solve(T_prime_t,t)[0], t_argmax_T ), T_s)[0]
```

```python
T_max = sp.Symbol("T_max", positive=True)
T_prime_0_sol = sp.solve(sp.Eq(T_t.subs(T_s, T_s_sol).subs(t, t_argmax_T), T_max),
                         T_prime_0)[0]
```

```python
T_max_t = sp.simplify( T_t.subs({T_s: T_s_sol, T_prime_0: T_prime_0_sol}) )
T_max_t
```

```python
get_T_t = sp.lambdify((t, T_prime_0, T_m, T_s), T_t)
get_T_max_t = sp.lambdify((t, T_max, t_argmax_T, T_m), T_max_t)
data = dict(T_prime_0=100, T_m=1, T_s=1)
```

```python
_, ax = plt.subplots(1,1)
t_range = np.linspace(0,10,100)
plt.plot(t_range, get_T_t(t_range, **data));
plt.plot(t_range, get_T_max_t(t_range, 37, 1., 2));
```

    Canvas(toolbar=Toolbar(toolitems=[('Home', 'Reset original view', 'home', 'home'), ('Back', 'Back to previous …

### Time dependent compressive strength

**From Eurocode 2:**
$s$ captures the effect of cement type on the time evolution of the compressive strength
it ranges from $s = 0.2$ for class R (rapid), $s = 0.25$ for class N (normal), and $s = 0.38$ for class S (slow).

```python
s = sp.Symbol("s", positive=True)
```

```python
beta_cc = sp.exp( s * (1 - sp.sqrt(28/t)))
beta_cc
```

```python
get_beta_cc = sp.lambdify((t, s), beta_cc )
```

```python
_, ax = plt.subplots(1,1)
plt.plot(t_range, get_beta_cc(t_range, 0.2))
```

    Canvas(toolbar=Toolbar(toolitems=[('Home', 'Reset original view', 'home', 'home'), ('Back', 'Back to previous …

    <lambdifygenerated-17>:2: RuntimeWarning: divide by zero encountered in true_divide
      return (exp(s*(1 - 2*sqrt(7)/sqrt(t))))

    [<matplotlib.lines.Line2D at 0x7f3119d6e490>]

### Compressive strength

```python
f_cm_28 = sp.Symbol("f_cm28", positive=True)
f_cm_28
```

```python
f_cm_t = beta_cc * f_cm_28
f_cm_t
```

```python
get_f_cm_t = sp.lambdify((t, f_cm_28, s), f_cm_t)
```

### Tensile strength

```python
f_ctm = sp.Symbol("f_ctm", positive=True)
alpha_f = sp.Symbol("alpha_f", positive=True)
```

```python
f_ctm_t = beta_cc * f_ctm
f_ctm_t
```

```python
get_f_ctm_t = sp.lambdify((t, f_ctm, s), f_ctm_t)
```

### Elastic modulus

```python
E_cm_28 = sp.Symbol("E_cm28", positive=True)
```

```python
E_cm_t = (f_cm_t / f_cm_28)**0.3 * E_cm_28
E_cm_t
```

```python
get_E_cm_t = sp.lambdify((t, E_cm_28, s), E_cm_t)
```

## Uncracked state

- Specimen is clamped at both sides. Then $\varepsilon_\mathrm{app} = 0, \forall x \in \Omega$

- Then the matrix stress is given as
\begin{align}
 \sigma^\mathrm{m}(x,t) = - E^\mathrm{m}(t)
 \cdot \alpha \int_0^t T^\prime(x,\theta)\, \mathrm{d}\theta
\end{align}

```python
alpha = sp.Symbol("alpha", positive=True )
```

```python
eps_eff = alpha * T_max_t
```

```python
dot_T_max_t = sp.simplify(T_max_t.diff(t))
```

```python
dot_eps_eff = alpha * dot_T_max_t
dot_E_cm_t = E_cm_t.diff(t)
```

```python
sig_t = E_cm_t * eps_eff
```

```python
dot_sig_t = E_cm_t * dot_eps_eff + dot_E_cm_t * eps_eff
```

```python
sp.simplify(dot_sig_t)
```

Integral cannot be resolved algebraically - numerical integration is used

```python
#sig2_t = sp.integrate(dot_sig_t, (t,0,t))
```

# Single crack state

## Time-dependent debonding process

### Fibers

- If there is a crack at $x_I$, then there can be non-zero apparent strains within the debonded zone - measurable using local strain sensors, i.e.
\begin{align}
\exists x \in (L_I^{(-)},L_I^{(+)}), \; \varepsilon_\mathrm{app}^\mathrm{f}(x,t) \neq 0.
\end{align}
- However, the integral of apparent strain in the fibers must disappear within the debonded zone, i.e.
\begin{align}
\int_{L^{(-)}}^{L^{(+)}}\varepsilon^\mathrm{f}_\mathrm{app}(x,t)\, \mathrm{d}x = 0
\end{align}

- Crack bridging fiber stress is given as
\begin{align}
  \sigma^{\mathrm{f}}(x=0, t) = E^{\mathrm{f}} \varepsilon^{\mathrm{f}}_\mathrm{eff}(x=0, t)
\end{align}

### Matrix

 - The integrated apparent strain in the matrix must be equal to crack opening $w_I$, i.e.
\begin{align}
\int_{L_I^{(-)}}^{L_I^{(+)}}\varepsilon^\mathrm{m}_\mathrm{app}(x,t)\, \mathrm{d}x + w_I = 0
\end{align}

- Considering symmetry, we can write
\begin{align}
\int_{0}^{L_I^{(+)}}\varepsilon^\mathrm{m}_\mathrm{app}(x,t)\, \mathrm{d}x
+ \frac{1}{2} w_I(t) = 0
\end{align}

This relation holds for a homogeneous strain distribution along the bar specimen.
Considering a non reinforced concrete bar, it is possible to detect the time of
a crack occurrence by requiring setting:

\begin{align}
 f_\mathrm{ct}(t) = \sigma_\mathrm{c}(t)
\end{align}

# Multiple cracks

The temperature development during the hydration process follows the relation
\begin{align}
 T(t,x)
\end{align}

At the same time, the material parameters of the concrete matrix and of bond are
defined as time functions
\begin{align}
 E(t), f_\mathrm{ct}(t), \tau(t)
\end{align}

Temperature-induced concrete strain in a point $x$ at time $t$ is expressed as
\begin{align}
\bar{\varepsilon}_{T}(t,x) = \alpha \int_0^t \frac{\mathrm{d} T(t,x)}{\mathrm{d} t} {\mathrm{d} t}
\end{align}

\begin{align}
\bar{\varepsilon}_\mathrm{app} = \bar{\varepsilon}_\mathrm{eff} + \bar{\varepsilon}_\mathrm{\Delta T}
\end{align}

If the apparent strain is suppressed, i.e. $\bar{\varepsilon}_\mathrm{app} = 0$, the effective stress is given as
\begin{align}
 0 = \bar{\varepsilon}_\mathrm{eff} +
 \bar{\varepsilon}_{\Delta T} \implies
 \bar{\varepsilon}_\mathrm{eff} = - \alpha \Delta T
\end{align}

More precisely, this equation reads
\begin{align}
\bar{\varepsilon}_\mathrm{eff}(t) = - \alpha \, \int_0^t \frac{\mathrm{d}T}{ \mathrm{d}t}  \, \mathrm{d} t
\end{align}

Current force at the boundary of the specimen is then given as
\begin{align}
\sigma = E(t) \, \varepsilon_{\mathrm{eff}}(t)
\end{align}

\begin{align}
\sigma = E(t) \left(\varepsilon_{\mathrm{app}}(x,t) - \alpha \int_0^t T^\prime(x,\theta) \, \mathrm{d}\theta \right)
\end{align}

**Salient features of the algorithm**

Non-linearity included by cracking stress

- find the time and location of the next crack occurrence
- provide a local, crack-centered solution of the cracking problem


Trusted Notebook" width="500 px" align="left">

## _*The Simon Algorithm*_

The latest version of this notebook is available on https://github.com/qiskit/qiskit-tutorial.

The Simon algorithm is an example that shows a quantum algorithm can solve a problem exponentially efficient than any classical algorithms. Like the Grover search, it depends on the existence of a blackbox (or, oracle) function that returns a predefined output over specific input or query. In the query-complexity setting, one cares only about how many queries are required to solve a specific problem, but does not care how the blackbox is realized. However, in this tutorial we have to implement it using the unit gates available in QISKit, just like we have done with the Grover search.

We first describe the problem addressed by the Simon algorithm, show the steps of the algorithm and the construction of the blackbox function, and present the experimental results on simulators and real devices.

***
### Contributors
Rudy Raymond

## The Problem

The Simon algorithm deals with finding a hidden integer $s \in \{0,1\}^n$ from an oracle $f_s$ that satisfies $f_s(x) = f_s(y)$ if and only if $y = x \oplus s$ for all $x \in \{0,1\}^n$.  Here, the $\oplus$ is the bitwise XOR operation. Thus, if $s = 0\ldots 0$, i.e., the all-zero bitstring, then $f_s$ is a 1-to-1 (or, permutation) function. Otherwise, if $s \neq 0\ldots 0$, then $f_s$ is a 2-to-1 function.

The Simon algorithm can find the hidden integer using only $O(n)$ queries to the blackbox function, while any classical algorithms require $\Omega(\sqrt{2^n})$ queries.

## The Algorithm to Find the Hidden Integer

The Simon algorithm finds the hidden integer by combining quantum algorithm with postprocessing on classical computers as below.

1. Prepare two quantum registers each of length $n$ that are initialized to all-zero bitstring: the first one as input and the second one as output of the blackbox function.
$$
|0\rangle |0\rangle
$$

2. Apply Hadamard gates to the first register to create superposition of all possible inputs.
$$
H^{\otimes n} |0\rangle |0\rangle = \frac{1}{\sqrt{2^n}} \sum_{x=0}^{2^n-1} |x\rangle |0\rangle
$$

3. Query the blackbox function to obtain the answer to queries on the second register.
$$
\frac{1}{\sqrt{2^n}} \sum_{x=0}^{2^n-1} U_{f_s}|x\rangle |0\rangle = \frac{1}{\sqrt{2^n}} \sum_{x=0}^{2^n-1} |x\rangle |f_s(x)\rangle
$$

4. Apply Hadamard gates to the first register.
$$
\frac{1}{\sqrt{2^n}} \sum_{x=0}^{2^n-1} H^{\otimes n}|x\rangle |f_s(x)\rangle = \frac{1}{2^n} \sum_{y=0}^{2^n-1}\sum_{x=0}^{2^n-1} (-1)^{x \cdot y}|y\rangle |f_s(x)\rangle = \frac{1}{2^n} \sum_{y=0}^{2^n-1} |y\rangle \sum_{x=0}^{2^n-1} ( (-1)^{x \cdot y} + (-1)^{(x\oplus s) \cdot y} )  |f_s(x)\rangle
$$

    Notice that at the right-hand side of the above equation, because $(-1)^{(x\oplus s) \cdot y} = (-1)^{x\cdot y +  s \cdot y}$ we can conclude that the probability amplitude of the basis state $|y\rangle |f_s(x)\rangle$ is $(-1)^{x\cdot y} (1 + (-1)^{s \cdot y}  )$, which is zero if and only if $s \cdot y = 1$. Thus, measuring the first register will always give $y$ such that $s \cdot y = 0$. Moreover, we can obtain many different $y$'s by repeating Step 1 to 4.

5. Repeat Step 1 to 4 for $m$ times to obtain $y_1, y_2, \ldots, y_m$.

6. **(Classical post-processing)** Let $\mathbf{Y}$ be an $m\times n$ matrix whose $i$-th row is $y_i$ in Step 5, and $\vec{s}$ be the column vector whose $j$-th element is the $j$-th bit of $s$. Solve the following system of linear equations to obtain $s$.
$$
\mathbf{Y} \vec{s} = 0
$$

## The Circuit

We now implement the Simon algorithm with Qiskit by first preparing the environment.

```python
#initialization
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline

# importing Qiskit
from qiskit import BasicAer, IBMQ
from qiskit import QuantumCircuit, ClassicalRegister, QuantumRegister, execute
from qiskit.tools.monitor import job_monitor

# import basic plot tools
from qiskit.tools.visualization import plot_histogram
```

```python
# Load the saved IBMQ accounts
IBMQ.load_accounts()
```

We then set the hidden bitstring $s$ that will be used to construct the circuit of the blackbox function (whose details will be given later). The number of qubits used in the experiment is twice the length of the bitstring $s$.

```python
s = "010101" # the hidden bitstring

assert 1 < len(s) < 20, "The length of s must be between 2 and 19"

for c in s:
    assert c == "0" or c == "1", "s must be a bitstring of '0' and '1'"

n = len(s) #the length of the bitstring
```

We then use Qiskit to create the circuit of the Simon algorithm prior the querying the blackbox function.

```python
# Step 1
# Creating registers
# qubits for querying the oracle and recording its output
qr = QuantumRegister(2*n)
# for recording the measurement on the first register of qr
cr = ClassicalRegister(n)

circuitName = "Simon"
simonCircuit = QuantumCircuit(qr, cr)

# Step 2
# Apply Hadamard gates before querying the oracle
for i in range(n):
    simonCircuit.h(qr[i])

# Apply barrier to mark the beginning of the blackbox function
simonCircuit.barrier()
```

    <qiskit.extensions.standard.barrier.Barrier at 0xa17f90f28>

### Constructing a Circuit for the Blackbox Function

We now details the construction of 1-to-1 and 2-to-1 permutation circuit of the blackbox function. Let us assume the blackbox function receive $|x\rangle|0\rangle$ as input. With regards to a predetermined $s$, the blackbox function writes its output to the second register so that it transforms the input to $|x\rangle|f_s(x)\rangle$ such that $f(x) = f(x\oplus s)$ for all $x \in \{0,1\}^n$.

Such a blackbox function can be realized by the following procedures.

-  Copy the content of the first register to the second register.
$$
|x\rangle|0\rangle \rightarrow |x\rangle|x\rangle
$$

-  **(Creating 1-to-1 or 2-to-1 mapping)** If $s$ is not all-zero, then there is the least index $j$ so that $s_j = 1$. If $x_j = 0$, then XOR the second register with $s$. Otherwise, do not change the second register.
$$
|x\rangle|x\rangle \rightarrow |x\rangle|x \oplus s\rangle~\mbox{if}~x_j = 0~\mbox{for the least index j}
$$

-  **(Creating random permutation)** Randomly permute and flip the qubits of the second register.
$$
|x\rangle|y\rangle \rightarrow |x\rangle|f_s(y)\rangle
$$

Below is the circuit of the blacbox function based on the above procedures.

```python
# Step 3 query the blackbox function
# copy the content of the first register to the second register
for i in range(n):
    simonCircuit.cx(qr[i], qr[n+i])

# get the least index j such that s_j is "1"
j = -1
for i, c in enumerate(s):
    if c == "1":
        j = i
        break

# Creating 1-to-1 or 2-to-1 mapping with the j-th qubit of x as control to XOR the second register with s
for i, c in enumerate(s):
    if c == "1" and j >= 0:
        simonCircuit.cx(qr[j], qr[n+i]) #the i-th qubit is flipped if s_i is 1

# get random permutation of n qubits
perm = list(np.random.permutation(n))

#initial position
init = list(range(n))
i = 0
while i < n:
    if init[i] != perm[i]:
        k = perm.index(init[i])
        simonCircuit.swap(qr[n+i], qr[n+k]) #swap qubits
        init[i], init[k] = init[k], init[i] #marked swapped qubits
    else:
        i += 1

# randomly flip the qubit
for i in range(n):
    if np.random.random() > 0.5:
        simonCircuit.x(qr[n+i])

# Apply the barrier to mark the end of the blackbox function
simonCircuit.barrier()
```

    <qiskit.extensions.standard.barrier.Barrier at 0xa17faf390>

Now we can continue with the steps of the Simon algorithm: applying the Hadamard gates to the first register and measure.

```python
# Step 4 apply Hadamard gates to the first register
for i in range(n):
    simonCircuit.h(qr[i])

# Step 5 perform measurement on the first register
for i in range(n):
    simonCircuit.measure(qr[i], cr[i])

#draw the circuit
simonCircuit.draw(output='mpl')
```

## Experimenting with Simulators

We show the experiments of finding the hidden integer with simulators.

```python
# use local simulator
backend = BasicAer.get_backend("qasm_simulator")
# the number of shots is twice the length of the bitstring
shots = 2*n
job = execute(simonCircuit, backend=backend, shots=shots)
answer = job.result().get_counts()

plot_histogram(answer)
```

We can see that the results of the measurements are the basis whose inner product with the hidden string $s$ are zero.

*(Notice that the basis on the label of the x-axis in the above plot are numbered from right to left instead of from left to right that we used for $s$)*

Gathering the measurement results, we proceed to post-processing with computations that can be done on classical computers.

### Post Processing with Gaussian Elimination

The post processing is done with Gaussian elimination to solve the system of linear equations to determine $s$.

```python
# Post-processing step
# Constructing the system of linear equations Y s = 0
# By k[::-1], we reverse the order of the bitstring
lAnswer = [ (k[::-1],v) for k,v in answer.items() if k != "0"*n  ] #excluding the trivial all-zero
#Sort the basis by their probabilities
lAnswer.sort(key = lambda x: x[1], reverse=True)

Y = []
for k, v in lAnswer:
    Y.append( [ int(c) for c in k ] )

#import tools from sympy
from sympy import Matrix, pprint, MatrixSymbol, expand, mod_inverse

Y = Matrix(Y)

#pprint(Y)

#Perform Gaussian elimination on Y
Y_transformed = Y.rref(iszerofunc=lambda x: x % 2==0) # linear algebra on GF(2)

#to convert rational and negatives in rref of linear algebra on GF(2)
def mod(x,modulus):
    numer, denom = x.as_numer_denom()
    return numer*mod_inverse(denom,modulus) % modulus

Y_new = Y_transformed[0].applyfunc(lambda x: mod(x,2)) #must takecare of negatives and fractional values
#pprint(Y_new)

print("The hidden bistring s[ 0 ], s[ 1 ]....s[",n-1,"] is the one satisfying the following system of linear equations:")
rows, cols = Y_new.shape
for r in range(rows):
    Yr = [ "s[ "+str(i)+" ]" for i, v in enumerate(list(Y_new[r,:])) if v == 1 ]
    if len(Yr) > 0:
        tStr = " + ".join(Yr)
        print(tStr, "= 0")

```

    The hidden bistring s[ 0 ], s[ 1 ]....s[ 5 ] is the one satisfying the following system of linear equations:
    s[ 0 ] = 0
    s[ 1 ] + s[ 5 ] = 0
    s[ 2 ] = 0
    s[ 3 ] + s[ 5 ] = 0
    s[ 4 ] = 0

As seen above, the system of linear equations is satisfied by the hidden integer $s$. Notice that there can be more than one solutions to the system. In fact, all-zero bitstring is a trivial solution to the system of linear equations. But by having more samples one can narrow down the candidates of the solution, and then test the solution by querying the blackbock in the classical manner.

## Experimenting with Real Devices

We see how one can still find out the hidden integer by running the Simon algorithm on real devices. Due to imperfect quantum computers, obtaining the conclusion is not as easy as done with the simulator of perfect quantum computers.

We first register the token and query the available backends.

```python
BasicAer.backends()
```

    [<QasmSimulatorPy('qasm_simulator') from BasicAer()>,
     <StatevectorSimulatorPy('statevector_simulator') from BasicAer()>,
     <UnitarySimulatorPy('unitary_simulator') from BasicAer()>]

```python
IBMQ.backends()
```

    [<IBMQBackend('ibmqx4') from IBMQ()>,
     <IBMQBackend('ibmq_16_melbourne') from IBMQ()>,
     <IBMQBackend('ibmq_qasm_simulator') from IBMQ()>,
     <IBMQBackend('ibmq_20_tokyo') from IBMQ(ibm-q-melbourne, internal, default)>,
     <IBMQBackend('ibmq_qasm_simulator') from IBMQ(ibm-q-melbourne, internal, default)>,
     <IBMQBackend('ibmq_20_tokyo') from IBMQ(ibm-q-internal, research, melbourne)>,
     <IBMQBackend('ibmq_qasm_simulator') from IBMQ(ibm-q-internal, research, melbourne)>]

```python
#Use one of the available backends
backend = IBMQ.get_backend("ibmq_16_melbourne")

# show the status of the backend
print("Status of", backend, "is", backend.status())

shots = 10*n #run more experiments to be certain
max_credits = 3          # Maximum number of credits to spend on executions.

job_exp = execute(simonCircuit, backend=backend, shots=shots, max_credits=max_credits)
job_monitor(job_exp)
```

    Status of ibmq_16_melbourne is BackendStatus(backend_name='ibmq_16_melbourne', backend_version='1.0.0', operational=True, pending_jobs=0, status_msg='active')

    HTML(value="
Job Status: job is being initialized
")

```python
results = job_exp.result()
answer = results.get_counts(simonCircuit)

plot_histogram(answer)
```

```python
# Post-processing step
# Constructing the system of linear equations Y s = 0
# By k[::-1], we reverse the order of the bitstring
lAnswer = [ (k[::-1][:n],v) for k,v in answer.items() ] #excluding the qubits that are not part of the inputs
#Sort the basis by their probabilities
lAnswer.sort(key = lambda x: x[1], reverse=True)

Y = []
for k, v in lAnswer:
    Y.append( [ int(c) for c in k ] )

Y = Matrix(Y)

#Perform Gaussian elimination on Y
Y_transformed = Y.rref(iszerofunc=lambda x: x % 2==0) # linear algebra on GF(2)
Y_new = Y_transformed[0].applyfunc(lambda x: mod(x,2)) #must takecare of negatives and fractional values
#pprint(Y_new)

print("The hidden bistring s[ 0 ], s[ 1 ]....s[",n-1,"] is the one satisfying the following system of linear equations:")
rows, cols = Y_new.shape
for r in range(rows):
    Yr = [ "s[ "+str(i)+" ]" for i, v in enumerate(list(Y_new[r,:])) if v == 1 ]
    if len(Yr) > 0:
        tStr = " + ".join(Yr)
        print(tStr, "= 0")
```

    The hidden bistring s[ 0 ], s[ 1 ]....s[ 5 ] is the one satisfying the following system of linear equations:
    s[ 0 ] = 0
    s[ 1 ] = 0
    s[ 2 ] = 0
    s[ 3 ] = 0
    s[ 4 ] = 0
    s[ 5 ] = 0

## References

[1] "[On the power of quantum computation](https://epubs.siam.org/doi/abs/10.1137/S0097539796298637)", Daniel R. Simon, SIAM J. Comput., 26(5), 1474–1483 (1997)


# **8.1 Nonlinear moment-curvature relationship**

        Starting point

With the coverage of bond and cracking behavior, we can now approach the simulation of a composite as a whole. The described material models can be found and used in general finite tools. It is possible to simulate the material response using 3D solid finite element models. However, in engineering practice, more specialized methods of structural analysis and design are desired which allow us to get the answers to repeating questions. A frequent application of composite materials is provided in form of a beam exposed to bending. Let us show how to utilize the models and theory explained during the previous Tours and show how to apply them to provide an efficient and general method for an assessment of a beam.

        Where are we heading

One fundamental concept of engineering design is to decompose a complex problem into smaller subproblems that can be described managed in a more transparent way. This concept is relevant also for the description of structural behavior. The present notebook shows such a decomposition for a nonlinear structural analysis of an arbitrarily reinforced beam. It can consider steel, carbon, glass, basalt reinforcement with correspondingly defined constitutive laws. At the same time, various cross sectional shapes can be included as well.

Using the analysis method explained in the present notebook, is is possible  to explain the fundamental differences between the design of traditional steel-reinforced concrete structures, and the innovative TRC and carbon concrete structures. Understanding of these differences is crucial for achieving more economic, durable and, thus, sustainable design of structures in the future.

# **Quick structural analysis and assessment of arbitrary beam design**

For motivation, consider some examples of concrete girders reinforced with CFRP rebars, carbon textile fabrics and AR-glass fabrics.

# **Beam deflection calculated by integrating curvature**

Recalling the concepts of beam kinematics and equilibrium, we can sketch the computational scheme using the following diagram.

In case of a simply supported beam, the profile of bending moment is known a-priori as it does not depend on the material behavior. In other words, there are no redistribution effects along the beam. Therefore, if we know the relation between the bending moment $M$ and the local curvature $\kappa$ as depicted in the left figure above, we can calculate the deflection of the beam by integrating the curvature to obtain the cross sectional rotation, i.e.
$$
 \varphi(x) = \int_0^L \kappa(x) \, \mathrm{d}x + C
$$
and the deflection as
$$
 w(x) = \int_0^L \varphi(x) \, \mathrm{d}x + Cx + D
$$
The integration constants must be solved for particular boundary conditions.

Given this evaluation, we can directly construct the load-deflection curve depicted in the right diagram of the above Figure. The appealing feature of this evaluation scheme is the efficiency in comparison to a nonlinear finite-element model.

# **Cross section based nonlinear constitutive law**

To accomplish this kind of evaluation, however, we need to derive the $M-\kappa$ relation. This relation is affected by the stress redistribution  occurring within a cross section during the cracking. The crack propagation demonstrated in the notebook [7.1](../tour7_cracking/7_1_bending3pt_2d.ipynb#top) has already demonstrated the propagation of the stress peak through the cross section. While the example shown there considered a plain concrete without reinforcement, in the present notebook, we shall consider also the effect of the reinforcement.

The computational procedure is schematically shown in the Figure below. The key assumption
saying that the cross sections remain planar and normal to the deflection curve is used to
establish the relation between the curvature and strain profile along the beam height. Then,
the curvature can be defined as
\begin{align}
\kappa = \frac{\partial \varepsilon_{xx}}{\partial z}
\end{align}
Demoting the strains at the top and bottom of the cross section as $\varepsilon_\mathrm{top}$ and $\varepsilon_\mathrm{bot}$,
we can directly evaluate the derivative of strain over the height as
\begin{align}
\kappa = \frac{\varepsilon_\mathrm{top} - \varepsilon_\mathrm{bot}}{h}
\end{align}

We want to establish the relation between $\kappa$ and the corresponding bending moment $M$. Therefore, the evaluation starts by prescribing the
values of $\varepsilon_\mathrm{top}$ in a range given by the failure strain of the matrix or of the reinforcement. However, the curvature is given by two values, i.e. $\varepsilon_\mathrm{bot}$ and $\varepsilon_\mathrm{top}$. What is the value of $\varepsilon_\mathrm{bot}$ corresponding to the prescribed value of $\varepsilon_\mathrm{top}$. The answer can be found by applying the equilibrium condition requiring that the normal force within the cross section.
The force in concrete $F_\mathrm{c}$ is obtained by integrating the stress over the cross sectional height and the force in the reinforcement is obtained by summing up the crack bridging actions into F_\mathrm{r}. The equilibrium condition then reads
$$
  F_\mathrm{c}(\varepsilon_\mathrm{bot}) + F_\mathrm{r}(\varepsilon_\mathrm{bot}) = 0
$$

Since the strain at each vertical position $z$ is known, we can use the constitutive law of the concrete matrix and of the reinforcement to evaluate the stress profile in concrete $\sigma_\mathrm{c}(z)$ over the height, so that the concrete force is given as
$$
F_\mathrm{c} = \int_0^h \sigma_\mathrm{c}(z) \cdot b \, \mathrm{d}z
$$
and the individual crack bridging force contributions of the reinforcement layers $i$ as
$$
F_\mathrm{r} = \sum_i A_{\mathrm{r},i} \sigma_{\mathrm{r},i}
$$

Finally, the $M-\kappa$ relation can be evaluated as an expression
\begin{align}
M(\kappa) = \sum_i F_{\mathrm{r},i}(\kappa) z_i + \int_z \sigma_\mathrm{c} (\kappa) z_i \, \mathrm{d} z
\end{align}

# **Model implementation**

The described scheme represents a commonly used concept of interative scheme to identify the $M-\kappa$ relation. The implementation provided here exploits the flexibility of the Python ecosystem of packages for scientific computing, in particular, the `scipy` solvers of nonlinear equations. Moreover, by introducing the integration over the cross section with a variable width, various cross sectional shapes and nonlinear constitutive laws are covered.

The above shown model has been implemented in the model component `MKappa`.  covered range of configurations with the specification of inputs and outputs can be summarized as follows

```python
%matplotlib widget
from bmcs_cross_section.api import MKappa
from bmcs_cross_section.api import ReinfLayer, BarLayer, FabricLayer
from bmcs_cross_section.cs_design.cs_shape import TShape
import matplotlib.pyplot as plt
```

# **Concrete beam with short fibers**

To relate described model to the previously shown description of the bended beam in the notebook [7.2](../tour7_cracking/7_1_bending3pt_2d.ipynb#top), let us first simulate a beam without reinforcement. The model is constructed as follows

```python
mk_plain = MKappa()
```

By rendering the interactive interface, we can inspect the cross sectional design `cs_design`. The attribute `matrix` specifies the material law of the matrix. The `cross_section_layout` is an empty list, because no reinforcement has been specified. Finally,

```python
mk_plain.interact()
```

    VBox(children=(HBox(children=(VBox(children=(Tree(layout=Layout(align_items='stretch', border='solid 1px black…

# **Steel reinforced beam**

```python
mk_steel = MKappa()
bl1 = BarLayer(name='steel 1', z=30, ds=16, count=2 )
csl = mk_steel.cross_section_layout
csl.add_layer(bl1)
```

```python
mk_steel.interact()
```

    VBox(children=(HBox(children=(VBox(children=(Tree(layout=Layout(align_items='stretch', border='solid 1px black…

## Validation of the model using experimental results

Yang et al. (2010), http://dx.doi.org/10.1016/j.engstruct.2010.07.017, Steel reinforcement

# **Carbon reinforced beam**

```python
mk_carbon = MKappa(high_kappa=0.0002)
mk_carbon.cross_section_shape_.H=30
csl = mk_carbon.cross_section_layout
bl1 = FabricLayer(name='carbon 1', z=5 )
csl.add_layer(bl1)
```

```python
mk_carbon.interact()
```

    VBox(children=(HBox(children=(VBox(children=(Tree(layout=Layout(align_items='stretch', border='solid 1px black…

# **Identify the failure mechanism**

The nonlinear moment curvature curve captures the interaction between the tensile and compressive zone of a beam. Depending on the cross-sectional design, the peak load marks either the tensile or compressive failure of the cross section. The `MKappa` model component can be used to verify the cross sectional capacity calculated using the assessment rules in the design codes.

As an example, let us construct a steel reinforced cross section and configure the cross section in such a way either the compressive yielding of concrete or tensile yielding of steel will occur.

```python
mk_mechanism = MKappa(high_kappa=0.0002)
bl1 = BarLayer(name='rebar 1', z=20, ds=24, count=2, matmod='steel')  # material model
bl1.matmod_.trait_set(E=200000, f_t=500)
mk_mechanism.cross_section_layout.add_layer(bl1)
mk_mechanism.cross_section_shape_.trait_set(H=100, B=300)
mk_mechanism.interact()
```

    VBox(children=(HBox(children=(VBox(children=(Tree(layout=Layout(align_items='stretch', border='solid 1px black…

Exercises

- compare with the design code concept in EC2 evaluating the moment capacity of a reinforced cross section with the obtained peak load

Points to be added

 - Reference to the iterative procedures in EC2, explain the difference and
   that the present method is more consistent and not more complicated to implement.
 - Highlight the model components and mechanisms explained in Tours before, i.e.
   crack-bridging action and to multiple cracking
 - Monotonic versus cyclic
 - Preconfigure the webapp for commercially available reinforcement materials

      7.2 Fracture energy identification and size effect

    8.2 Reinforced beam, ultimate and serviceability limit states

```python

```


# BASIC CONTROLLERS

This notebook describes the proportional, integral, and differential controllers.

# Preliminaries

```python
!pip install -q control
!pip install -q tellurium
!pip install -q controlSBML

import control
import controlSBML as ctl
from IPython.display import HTML, Math
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import sympy
import tellurium as te
```

```python
TIMES = ctl.makeSimulationTimes(0, 5, 500)
```

# Antimony Model

```python
# Constants
CONSTANT_DCT = {"k1": 1, "k2": 2, "k3": 3, "k4": 4}
s = sympy.Symbol("s")
REF = 10
```

```python
MODEL = """
$S1 -> S2; k1*$S1
J1: S2 -> S3; k2*S2
J2: S3 -> S2; k3*S3
J3: S2 -> ; k4*S2

k1 = 1
k2 = 2
k3 = 3
k4 = 4
$S1 = 10
S2 = 0
S3 = 0
S4 = 0
"""
RR = te.loada(MODEL)
RR.simulate()
RR.plot()
```

# PID Controllers

The controllers considered here are systems that input the control error and produce a control signal used
to regulate the system under control.
The input signal is $e(t)$ and the output signal is $u(t)$.

A **proportional controller** has the parameter $k_P$.
This controller outputs a signal that is proportional to the control error.
That is, $u(t) = k_P e(t)$.
The transfer function for this controller is $H_{P} (s) = k_p$.

An **integral controller** has the parameter $k_I$.
This controller outputs a signal that is proportional to the *integral* of the control error.
That is, $u(t) = k_I \int_0^{t} e(\tau) d \tau$.
The transfer function for this controller is $H_{I} (s) = \frac{k_I}{s}$.

An **differential controller** has the parameter $k_D$.
This controller outputs a signal that is proportional to the derivative of the control error.
That is, $u(t) = k_D \frac{de(t)}{d t}$.
The transfer function for this controller is $H_{D} (s) = s k_D$.

These controllers can be used in combination.
For example, a PI controller produces
$u(t) = k_P e(t) +  k_I \int_0^{t} e(\tau) d \tau + k_D \frac{d e(t)}{d t}$,
and its transfer function is $H_{CPI}(s) = k_P + \frac{k_I}{s} + s k_D$.

# Analysis in Discrete Time

To improve our understanding of the different types of control, we'll do an analysis in discrete time.
Let $n$ index the instances.
Consider an input signal $e_n$ and an output signal $u_n$. Here's how the controllers work:
* **Proportional control**. $u_n = k_P  e_n$
* **Integral control**. $u_n = k_I  \sum_{i=0}^n e_i$
* **Differential control**. $u_n = k_D (e_n - e_{n-1})$

Now let's create an envaluation environment.

## Analysis Codes

```python
def evaluateController(e_vec, kp=0, ki=0, kd=0, is_plot=True, **kwargs):
    """
    Plots the output of the controller for the signal.

    Parameters
    ----------
    e_vec: list-float
    kp: float
    ki: float
    kd: float
    is_plot: bool
    kwargs: dict
        plot options

    Returns
    -------
    np.array
    """
    u_vec = []
    e_last = e_vec[0]
    e_sum = e_vec[0]
    for e_val in e_vec[1:]:
        e_sum  += e_val
        #
        u_val = kp*e_val
        u_val += ki*e_sum
        u_val += kd*(e_val - e_last)
        u_vec.append(u_val)
        #
        e_last = e_val
    #
    if is_plot:
        times = range(len(u_vec))
        y_vec = e_vec[0:len(times)]
        df = pd.DataFrame({
            "time": times,
            "e(t)": e_vec[0:len(times)],
            "u(t)": u_vec
        })
        ts = ctl.Timeseries(df)
        title = "kp=%2.2f ki=%2.2f kd=%2.2f" % (kp, ki, kd)
        kwargs["title"] = title
        ctl.plotOneTS(ts, **kwargs)
        if False:
            if ax is None:
                _, ax = plt.subplots(1)
            ax.plot(range(len(u_vec)), u_vec, color="blue")

            ax.plot(times, y_vec, linestyle="--", color="black")
            ax.legend(["u(t)", "e(t)"])
            ax.set_xlabel("time")
            ax.set_ylim([-1.5, 1.5])
            title = "kp=%2.2f ki=%2.2f kd=%2.2f" % (kp, ki, kd)
            ax.set_title(title)
    return np.array(u_vec)

# Tests
e_vec = np.repeat(1, 10)
u_vec = evaluateController(e_vec, kp=0, ki=0, kd=1, is_plot=False)
assert(u_vec[0] == 0)
print("OK!")
```

    OK!

## Analysis

What signals should we consider for $e(t)$?

```python
step_input = np.repeat(1, 10)
step_input = np.sin(0.1*np.array(range(100)))
_ = evaluateController(step_input, kp=2, ylim=[-2,2], figsize=(5,5))
_ = evaluateController(step_input, ki=1, ylim=[-2,2], figsize=(5,5))
_ = evaluateController(step_input, kd=1, ylim=[-2,2], figsize=(5,5))
```

# Analysis in Continuous Time

We can analyze controllers by looking at their transfer functions by considering poles, DC gain, and step respoinse.

$H_{PID} (s) = H_P(s) + H_I(s) + H_D(s) =  k_P + \frac{k_I}{s} + s k_D$

## Analysis Codes

```python
def plotPIDStepResponse(kp=1, ki=0, kd=0, is_plot=True):
    """
    Plots the step response of the PID controller.

    Parameters
    ----------
    kp: float
    ki: float
    kd: float
    is_plot: bool

    Returns
    -------
    control.TransferFunction
    """
    tf = control.TransferFunction([kp], [1])  \
          + control.TransferFunction([ki], [1, 0])  \
          + control.TransferFunction([kd, 0], [1])
    if is_plot:
        if len(tf.num[0][0]) > len(tf.den[0][0]):
            print("Improper transfer function. Cannot simulate.")
        else:
            result = control.step_response(tf)
            plt.plot(result.t.flatten(), result.y.flatten())
    return tf

# Tests
tf = plotPIDStepResponse(kp=1, ki=0, kd=0, is_plot=False)
assert(tf.dcgain() == 1)
print("OK!")
```

    OK!

```python
# Find the DC gains for different variations of controllers
print("kp=1: %2.2f" % plotPIDStepResponse(kp=1, ki=0, kd=0, is_plot=False).dcgain())
print("ki=1: %2.2f" % plotPIDStepResponse(kp=0, ki=1, kd=0, is_plot=False).dcgain())
```

    kp=1: 1.00
    ki=1: inf

## Controller in Closed Loop

### Extending ``plotTF`` to PID Controllers

```python
def plotTFs(Gs, kp=0, ki=0, kd=0, times=TIMES, ylim=None, title=None, is_plot=True):
    """
    Constructs the transfer functions for the proportional controller, and filter.
    Calculates the transfer functions HRYs, HREs, HNYs, HDYs and plots them.

    Parameters
    ----------
    Gs: control.TransferFunction
    kp: float
    ki: float
    kd: float
    times: list-float
    ylim: (float, float)
        limits of y-values
    title: str

    Returns
    -------
    dct
        key: name of transfer function
        value: control.TransferFunction
    """
    Cs = control.TransferFunction([kp], [1]) + control.TransferFunction([ki], [1, 0])  \
        + control.TransferFunction([kd, 0], [1])
    Fs = 1
    denom = 1 + Cs*Gs*Fs
    # Construct the transfer functions
    tf_dct = {
        "HRYs": Cs*Gs/denom,
        "HREs": 1/denom,
        "HNYs": -Fs/denom,
        "HDYs": Cs/denom,
    }
    # Construct the plots
    _, ax = plt.subplots(1)
    for tf in tf_dct.values():
        result = control.forced_response(tf, T=times, U=1)
        plt.plot(result.t.flatten(), result.y.flatten())
    # Refine plots
    plt.legend(list(tf_dct.keys()))
    xmax = max(result.t.flatten())
    plt.plot([0, xmax], [0, 0], linestyle="--", color="black")
    plt.plot([0, xmax], [1, 1], linestyle="--", color="grey")
    plt.ylim([-5, 5])
    title = "kp=%2.2f ki=%2.2f kd=%2.2f" % (kp, ki, kd)
    plt.title(title)
    if not is_plot:
        plt.close()
    return tf_dct

# Tests
Gs = control.TransferFunction([2], [1, 3])
dct = plotTFs(Gs, kp=10, ylim=[0, 3], title="Example", is_plot=False)
assert(len(dct) == 4)
assert("TransferFunction" in str(type(dct["HRYs"])))
print("OK!")
```

    OK!

```python
dct["HRYs"]
```

$$\frac{20 s + 60}{s^2 + 26 s + 69}$$

### Plots

```python
for kp in [1, 5, 10, 20]:
    for ki in [1, 5, 10]:
        title = "kp: %2.1f ki: %2.1f" % (kp, ki)
        _ = plotTFs(Gs, kp=kp, ki=ki, title=title, ylim=[-1, 3])
```

# Testbed

## Controller Factory

**We need to extend ``makeController`` to PID**

```python
# Extend to PID
def makeController(name, kp=0, ki=0, kd=0):
    """
    Creates a proportional controller as a NonlinearIOSystem with input "in" and output "out".

    Parameters
    ----------
    name: str
        Name of the system
    kp: float
    ki: float
    kd: float

    Returns
    -------
    control.NonlinearIOSystem
    """
    def updfcn(time, x_vec, u_vec, params):
        """
        Calculates the derivative of state.

        Parameters
        ----------
        x_vec: array of dimension 2
            0: last time
            1: last input
            2: sum of values
        u_vec: error signal input

        Returns
        -------
        dtime: float
            derivative of time
        dinput: float
            derivative of input
        dsum: float
            derivative of sum
        """
        try:
            u_val = u_vec[0]
        except:
            u_val = u_vec
        #
        dtime = time - x_vec[0]
        if np.isclose(dtime, 0):
            dinput = 0
            dsum = 0
        else:
            dinput = (u_val - x_vec[1])/dtime
            dsum = u_val/dtime
        return dtime, dinput, dsum

    def outfcn(_, x_vec, u_val, ___):
        # u: float (error signal)
        try:
            u_val = u_vec[0]
        except:
            u_val = u_vec
        return kp*(u_val) + ki*x_vec[2] + kd*(u_val - x_vec[1])
    #
    return control.NonlinearIOSystem(
        updfcn, outfcn, inputs=['in'], outputs=['out'], states=["dtime", "dinput", "dsum"],
        name=name)

# Tests
controller = makeController("controller", kp=1, ki=1, kd=1)
times = ctl.makeSimulationTimes()
U = np.repeat(1, len(times))
U[0] = 1
result = control.input_output_response(controller, T=times, U=U)
#trues = [r == kp*( t) for t, r in zip(result.t, result.outputs)]
#assert(all(trues))
print("OK!")
```

    OK!

## Closed Loop System

```python
# Elements of the system

factory = ctl.IOSystemFactory()
kp = 100
ki = 0
kd = 0
# Create the elements of the feedback loop
noise = factory.makeSinusoid("noise", 0, 20)
disturbance = factory.makeSinusoid("disturbance", 0, 2)

ctlsb = ctl.ControlSBML(MODEL, input_names=["S2"], output_names=["S3"])
system = ctlsb.makeNonlinearIOSystem("system")
controller = factory.makePIDController("controller", kp=kp, ki=ki, kd=kd)
fltr = factory.makePassthru("fltr")

sum_Y_N = factory.makeAdder("sum_Y_N")
sum_U_D = factory.makeAdder("sum_U_D")
sum_R_F = factory.makeAdder("sum_R_F")
```

```python
# Create the closed loop system
closed_loop = control.interconnect(
  [noise, disturbance, sum_Y_N, sum_R_F, sum_U_D, system, fltr, controller ],
  connections=[
    ['controller.in', 'sum_R_F.out'],    # e(t)
    ['sum_U_D.in1', 'controller.out'],   # u(t)
    ['sum_U_D.in2', 'disturbance.out'],  # d(t)
    ['system.S2',   'sum_U_D.out'],
    ['sum_Y_N.in1', 'system.S3'],        # y(t)
    ['sum_Y_N.in2', 'noise.out'],        # n(t)
    ['fltr.in',     'sum_Y_N.out'],
    ['sum_R_F.in1', '-fltr.out'],
  ],
  inplist=["sum_R_F.in2"],
  outlist=["sum_R_F.in2", "sum_Y_N.out", 'system.S2', 'system.S3'],
)
```

## Plots

We can use the transfer functions to guide our choice of parameters.
Note that we it may be necessary to divide by $s$ to calculate the DC gain.

```python
times = ctl.makeSimulationTimes(0, 50, 100)
result = control.input_output_response(closed_loop, T=times, U=10)
plt.plot(result.t, result.outputs[0].flatten())
plt.plot(result.t, result.outputs[1].flatten())
#plt.plot(result.t, result.outputs[2].flatten())
#plt.plot(result.t, result.outputs[3].flatten())
plt.ylim([5, 15])
legends = ["input", "output"]
plt.legend(legends)
```

## Transfer Function Analysis

```python
# Linearized analysis
print(kp, ki, kd)
Gs = ctlsb.makeTransferFunction()
dct = plotTFs(Gs, kp=kp, ki=ki, kd=kd, title=title, ylim=[-1, 3], is_plot=False)
dct["HRYs"]
```

    100 0 0

$$\frac{200 s + 600}{s^2 + 206 s + 609}$$

## Evaluation Environment

We want to repeatedly evaluate different controller parameters.
This is cumbersome to do if we have to rerun multiple cells and reset various parameters.
It's much more efficient to create an evaluation function.

```python
def runTestbed(model=MODEL, input_name="S2", output_name="S3", kp=0, ki=0, kd=0, noise_amp=0, disturbance_amp=0):
    """
    Run the testbed and plot the results.

    Parameters
    ----------
    model: str
        System under control
    input_name: str
    output_name: str
    kp: float
    ki: float
    kd: float
    noise_amp: float
    disturbance_amp: float

    Results
    -------
    control.InterconnectedSystem
    """
```

```python
def makeHRY(model=MODEL, input_name="S2", output_name="S3", time=0, kp=0, ki=0, kd=0):
    """
    Calculates the transfer function from the reference input to the output.

    Parameters
    ----------
    model: str
    input_name: str
    output_name: str
    time: float
    kp: float
    ki: float
    kd: float

    Returns
    -------
    control.TransferFunction
    """
    ctlsb = ctl.ControlSBML(MODEL, input_names=[input_name], output_names=[output_name])
    Gs = ctlsb.makeTransferFunction(time=time)
    dct = plotTFs(Gs, kp=kp, ki=ki, kd=kd, is_plot=False)
    return dct["HRYs"]

# TESTS
tf = makeHRY()
assert(tf.dcgain() == 0)
```

# More On Filters

How do filters help with sinusoidal noise and disturbances?

```python
# Create a testsbed
factory = ctl.IOSystemFactory()
fltr = factory.makeFilter("fltr", -50)
noise = factory.makeSinusoid("noise", 1, 20)
```

```python
# Create the testbed
test_bed = control.interconnect(
  [noise, fltr ],
  connections=[
    ['fltr.in', 'noise.out'],    # e(t)

  ],
  outlist=["fltr.out"],
)
```

```python
# Simulate it
times = ctl.makeSimulationTimes(0, 20, 500)
result = control.input_output_response(test_bed, T=times)
plt.plot(result.t, result.outputs.flatten())
plt.ylim([-1, 1])
```


```python
from sympy import *
from sympy.solvers import ode
init_session()
```

    IPython console for SymPy 1.7.1 (Python 3.7.12-64-bit) (ground types: python)

    These commands were executed:
    >>> from __future__ import division
    >>> from sympy import *
    >>> x, y, z, t = symbols('x y z t')
    >>> k, m, n = symbols('k m n', integer=True)
    >>> f, g, h = symbols('f g h', cls=Function)
    >>> init_printing()

    Documentation can be found at https://docs.sympy.org/1.7.1/

```python
an = Sum(117*k-3*k*(k-1), (k,3,n))
an.simplify

```

    <bound method Basic.simplify of Sum(-3*k*(k - 1) + 117*k, (k, 3, n))>

```python

summation(an.formula, (m,1,n))
```

```python
Sum(n**x, (n, 1, k))
```

```python
sequence(117*m-3*m*(m-1),(m,1,k))
```

```python
s = summation(1/(9*k**2),(k,1,n))
s
```

```python
limit(s, n, oo)
```

```python
integrate(x**(-2), x)

```

```python
N
```

    <function sympy.core.evalf.N>

```python
integrate(1/(x*sqrt((x**3 + 1))), x)
```

```python
limit(summation(1/(3*k+1) - 1/(3*k+2), (k,0,n)),n,oo).doit()
```

```python
float((sqrt(3)*pi)/9)
f(x).diff(x)
```

```python
eq = f(x) - 2*x*f(x).diff(x) - f(x).diff(x)**2
eq
```

    ('factorable', 'lie_group')

# 新しいセクション

```python

```

$\sqrt{3x-1}+(1+x)^2.$

```python

```


```python
import numpy as np
import numba
import matplotlib.pyplot as plt
import sympy as sym
plt.style.use('presentation')
%matplotlib notebook

def d2np(d):

    names = []
    numbers = ()
    dtypes = []
    for item in d:
        names += item
        if type(d[item]) == float:
            numbers += (d[item],)
            dtypes += [(item,float)]
        if type(d[item]) == np.ndarray:
            numbers += (d[item],)
            dtypes += [(item,np.float64,d[item].shape)]
    return np.array([numbers],dtype=dtypes)
```

```python
psi_ds,psi_qs,psi_dr,psi_qr = sym.symbols('psi_ds,psi_qs,psi_dr,psi_qr')
i_ds,i_qs,i_dr,i_qr = sym.symbols('i_ds,i_qs,i_dr,i_qr')
di_ds,di_qs,di_dr,di_qr = sym.symbols('di_ds,di_qs,di_dr,di_qr')
L_s,L_r,L_m = sym.symbols('L_s,L_r,L_m')
R_s,R_r = sym.symbols('R_s,R_r')
omega_s,omega_r,sigma = sym.symbols('omega_s,omega_r,sigma')
v_ds,v_qs,v_dr,v_qr = sym.symbols('v_ds,v_qs,v_dr,v_qr')

dpsi_ds = (L_s+L_m)*di_ds + L_m*di_dr
dpsi_qs = (L_s+L_m)*di_qs + L_m*di_qr
dpsi_dr = (L_r+L_m)*di_dr + L_m*di_ds
dpsi_qr = (L_r+L_m)*di_qr + L_m*di_qs

eq_ds = - dpsi_ds + v_ds + R_s*i_ds + omega_s*psi_qs
eq_qs = - dpsi_qs + v_qs + R_s*i_qs - omega_s*psi_ds
eq_dr = - dpsi_dr + v_dr + R_r*i_dr + sigma*omega_s*psi_qr
eq_qr = - dpsi_qr + v_qr + R_r*i_qr - sigma*omega_s*psi_dr

s=sym.solve([eq_ds,eq_qs,eq_dr,eq_qr],[di_ds,di_qs,di_dr,di_qr])

for item in s:
    print(item, '=', s[item])
```

    di_qs = (-L_m*R_r*i_qr + L_m*R_s*i_qs + L_m*omega_s*psi_dr*sigma - L_m*omega_s*psi_ds - L_m*v_qr + L_m*v_qs + L_r*R_s*i_qs - L_r*omega_s*psi_ds + L_r*v_qs)/(L_m*L_r + L_m*L_s + L_r*L_s)
    di_dr = (L_m*R_r*i_dr - L_m*R_s*i_ds + L_m*omega_s*psi_qr*sigma - L_m*omega_s*psi_qs + L_m*v_dr - L_m*v_ds + L_s*R_r*i_dr + L_s*omega_s*psi_qr*sigma + L_s*v_dr)/(L_m*L_r + L_m*L_s + L_r*L_s)
    di_ds = (-L_m*R_r*i_dr + L_m*R_s*i_ds - L_m*omega_s*psi_qr*sigma + L_m*omega_s*psi_qs - L_m*v_dr + L_m*v_ds + L_r*R_s*i_ds + L_r*omega_s*psi_qs + L_r*v_ds)/(L_m*L_r + L_m*L_s + L_r*L_s)
    di_qr = (L_m*R_r*i_qr - L_m*R_s*i_qs - L_m*omega_s*psi_dr*sigma + L_m*omega_s*psi_ds + L_m*v_qr - L_m*v_qs + L_s*R_r*i_qr - L_s*omega_s*psi_dr*sigma + L_s*v_qr)/(L_m*L_r + L_m*L_s + L_r*L_s)

```python
10*3*7/10/10

```

    2.1

```python
# [1] T. Demiray, F. Milano, and G. Andersson,
# “Dynamic phasor modeling of the doubly-fed induction generator under unbalanced conditions,” 2007 IEEE Lausanne POWERTECH, Proc., no. 2, pp. 1049–1054, 2007.

@numba.jit(nopython=True, cache=True)
def dfim(struct,i):

    L_m = struct[i]['L_m']
    L_r = struct[i]['L_r']
    L_s = struct[i]['L_s']
    R_r = struct[i]['R_r']
    R_s = struct[i]['R_s']

    Dt  = struct[i]['Dt']

    i_ds = struct[i]['x'][0,0]
    i_qs = struct[i]['x'][1,0]
    i_dr = struct[i]['x'][2,0]
    i_qr = struct[i]['x'][3,0]

    v_ds = struct[i]['v_ds']
    v_qs = struct[i]['v_qs']
    v_dr = struct[i]['v_dr']
    v_qr = struct[i]['v_qr']

    omega_r = struct[i]['omega_r']
    omega_s = struct[i]['omega_s']

    sigma = (omega_s - omega_r)/omega_s

    psi_ds = (L_s+L_m)*i_ds + L_m*i_dr
    psi_qs = (L_s+L_m)*i_qs + L_m*i_qr
    psi_dr = (L_r+L_m)*i_dr + L_m*i_ds
    psi_qr = (L_r+L_m)*i_qr + L_m*i_qs

    di_qs = (-L_m*R_r*i_qr + L_m*R_s*i_qs + L_m*omega_s*psi_dr*sigma - L_m*omega_s*psi_ds - L_m*v_qr + L_m*v_qs + L_r*R_s*i_qs - L_r*omega_s*psi_ds + L_r*v_qs)/(L_m*L_r + L_m*L_s + L_r*L_s)
    di_dr = (L_m*R_r*i_dr - L_m*R_s*i_ds + L_m*omega_s*psi_qr*sigma - L_m*omega_s*psi_qs + L_m*v_dr - L_m*v_ds + L_s*R_r*i_dr + L_s*omega_s*psi_qr*sigma + L_s*v_dr)/(L_m*L_r + L_m*L_s + L_r*L_s)
    di_ds = (-L_m*R_r*i_dr + L_m*R_s*i_ds - L_m*omega_s*psi_qr*sigma + L_m*omega_s*psi_qs - L_m*v_dr + L_m*v_ds + L_r*R_s*i_ds + L_r*omega_s*psi_qs + L_r*v_ds)/(L_m*L_r + L_m*L_s + L_r*L_s)
    di_qr = (L_m*R_r*i_qr - L_m*R_s*i_qs - L_m*omega_s*psi_dr*sigma + L_m*omega_s*psi_ds + L_m*v_qr - L_m*v_qs + L_s*R_r*i_qr - L_s*omega_s*psi_dr*sigma + L_s*v_qr)/(L_m*L_r + L_m*L_s + L_r*L_s)

    tau_e  = psi_qr*i_dr - psi_dr*i_qr

    struct[i]['f'][0,0] = di_ds
    struct[i]['f'][1,0] = di_qs
    struct[i]['f'][2,0] = di_dr
    struct[i]['f'][3,0] = di_qr

    struct[i]['i_ds'] = i_ds
    struct[i]['i_qs'] = i_qs
    struct[i]['i_dr'] = i_dr
    struct[i]['i_qr'] = i_qr

    struct[i]['psi_ds'] = psi_ds
    struct[i]['psi_qs'] = psi_qs
    struct[i]['psi_dr'] = psi_dr
    struct[i]['psi_qr'] = psi_qr

    struct[i]['tau_e'] = tau_e
    struct[i]['sigma'] = sigma

    return tau_e

```

```python
Omega_b = 2.0*np.pi*50.0
S_b = 1.0e6
U_b = 690.0
Z_b = U_b**2/S_b
Z_b = 1.0
Omega_b = 1.0
d =dict(R_r = 0.1*Z_b,
        R_s = 0.1*Z_b,
        L_r = 0.08*Z_b/Omega_b,
        L_s = 0.1*Z_b/Omega_b,
        L_m = 3.0*Z_b/Omega_b,
        psi_ds = 0.0,
        psi_qs = 0.0,
        psi_dr = 0.0,
        psi_qr = 0.0,
        i_ds   = 0.0,
        i_qs   = 0.0,
        i_dr   = 0.0,
        i_qr   = 0.0,
        v_ds   = 0.0,
        v_qs   = 0.0,
        v_dr   = 0.0,
        v_qr   = 0.0,
        omega_r   = Omega_b*0.99,
        omega_s   = Omega_b,
        sigma   = 0.0,
        tau_e   = 0.0,
        x = np.zeros((4,1)),
        f = np.zeros((4,1)),
        Dt = 0.0
        )

struct = d2np(d)
dfim(struct,0)
struct['L_s']/struct['R_s']
```

    array([ 1.])

```python
struct = d2np(d)
struct['v_ds'] = 1.0
struct['v_qs'] = 0.0
struct['omega_r'] = 0.9
Dt = 1.0e-6
struct[0]['x']= np.zeros((4,1))
Tau_e = []
T =[]
for it in range(100000):
    t = Dt*it
    x1 = np.copy(struct[0]['x'])
    dfim(struct,0)
    f1 = np.copy(struct[0]['f'])

    struct[0]['x'] = x1 + Dt*f1
    dfim(struct,0)
    struct[0]['x'] = x1 + 0.5*Dt*(f1 + struct[0]['f'])
    Tau_e += [float(struct['tau_e'])]
    T +=[t]

```

```python
fig, axes = plt.subplots(nrows=1, ncols=1, figsize=(8, 5), sharex = True)
axes.plot(T,Tau_e)
fig.savefig('dfim_tau_e.svg', bbox_inches='tight')

```

    <IPython.core.display.Javascript object>

```python
struct = d2np(d)
plt.tight_layout()
fig, axes = plt.subplots(nrows=1, ncols=1, figsize=(8, 5), sharex = True)
struct['beta'] = 0.0
nu_ws = np.linspace(5,10,10)
omegas = np.linspace(0.1,3.5,100)
for nu_w in nu_ws:
    P_w = []
    Omega = []
    for omega in omegas:
        struct['nu_w'] = nu_w
        struct['omega_t'] = omega
        wind_turbine_fit(struct,0)
        P_w += [float(struct['p_w'])]
        Omega += [float(struct['omega_t'])]
    axes.plot(Omega,np.array(P_w)/1e6)

omega = 1
Nu = np.linspace(2,12,50)
omegas_opt = []
p_ref_opt = []
for nu in Nu:
    struct['nu_w'] = nu
    for it in range(500):
        struct['omega_t'] = omega
        mppt_fit(struct,0)
        wind_turbine_fit(struct,0)
        p_ref = float(struct['p_mppt'])
        p_t = float(struct['p_w'])
        omega += 0.0000001*(-p_ref + p_t)/omega
        if abs(-p_ref + p_t)<0.01:
            break

    omegas_opt += [float(struct['omega_t'])]
    p_ref_opt += [float(struct['p_w'])]

axes.set_ylabel('$\sf P_t$')
axes.set_xlabel('$\sf \omega_t$')
axes.set_ylim([0,2.5])
axes.set_xlim([0,3.0])

fig.savefig('w_p_t.svg', bbox_inches='tight')
axes.plot(np.array(omegas_opt),np.array(p_ref_opt)/1e6)

fig.savefig('w_p_t_mppt.svg', bbox_inches='tight')

```

## without MPPT

```python
struct = d2np(d)
struct['nu_w'] = 5.0
for it in range(10000):
    wecs_mech_1(struct,0)
    struct[0]['x'] += 100.0e-3*struct[0]['f'][0,0]
print(struct['omega_t'])
print(struct['p_w']/1e6)
```

## with MPPT

```python
struct['omega_t'] = 1.0
struct['omega_r'] = 1.0*20
struct['nu_w'] = 8.5
t = 0.0
Omega_t = []
P_w = []
T = []
Dt = 100.0e-3
for it in range(500):
    if t>10.0:
        struct['nu_w'] = 6.0

    wecs_mech_1(struct,0)
    mppt_fit(struct,0)
    struct['tau_t'] = struct['p_w']/struct['omega_t']
    struct['tau_r'] = struct['p_mppt']/struct['omega_r']
    wind_turbine_fit(struct,0)
    struct[0]['x'] += Dt*struct[0]['f'][0,0]
    struct['omega_t'] = float(struct[0]['x'])
    Omega_t += [float(struct['omega_t'])]
    P_w += [float(struct['p_w'])]
    T += [t]
    t+= Dt
P_w = np.array(P_w)
print(struct['omega_t'])
print(struct['p_w']/1e6)
```

```python
plt.tight_layout()
fig, axes = plt.subplots(nrows=1, ncols=1, figsize=(8, 5), sharex = True)

axes.plot(T,Omega_t)

```

```python
plt.tight_layout()
fig, axes = plt.subplots(nrows=1, ncols=1, figsize=(8, 5), sharex = True)

axes.plot(T,P_w/1e6)
```

```python
struct['beta_ref'] = 0.0
struct['beta'] = 0.0
for it in range(1000):
    t = it*struct['Dt']
    if t>0.1:
        struct['beta_ref'] = 5.0

    pitch(struct,0)
    print(t, struct['beta'])
```

```python

```

```python

```