```python
import numpy as np
```

# Planetary Orbits

We have the followings relations:

\begin{equation}
l_2 v_2 = l_1 v_1 \qquad \text{Simplification of Kepler's Law} \\
v_2^2 - \frac{2GM}{v_1 l_1}v_2 - \left[ v_1^2 - \frac{2GM}{l_1}\right] = 0 \qquad \text{Second order Polynomial}
\end{equation}
where $m$ is the mass of the planet, $M = 1.9891 \times 10^{30}\,\text{Kg}$ is the mass of the Sun, and $G = 6.6738 \times 10^{-11}\,\text{m}^3 \text{Kg}^{-1}s^{-2}$ is the universal constant of gravitation.

---
The values $v_1$, $l_1$ and $v_2$, $l_2$ need to be the extreme values of the elliptical orbit.
$l_1$ is the closest point to the sun, it's the perihelion.
$l_2$ is the farthest point to the sun, it's the aphelion.

To solve this problem we will use Classes to better structure this problem.
To find the value of $v_2$ we only need to solve a second order polynomial.

```python
class planetSolarSystem:
    # Global Constants, we need to declare them global so we can access
    # them inside the methods of the class.
    global G, M
    G = 6.6738e-11
    M = 1.9891e30

    def __init__(self, v1, l1):
        """Creates the planet values.

        :param m: float mass of the planet.
        :param v1: float velocity of the closest point to the sun.
        :param l1: float lowest radius of the orbit.

        l1 must be the perihelion and l2 will be the aphelion.
        """
        self.v1 = v1
        self.l1 = l1
        self.v2 = self.calculate_v2()
        self.l2 = self.v1 * self.l1 / self.v2
        self.a = self.semi_major_axis()
        self.b = self.semi_minor_axis()
        self.T = self.orbital_period()
        self.e = self.orbital_eccentricity()

    def calculate_v2(self):
        """Calculates v2

        V2 is the velocity in the point of the biggest radius of the orbit.
        """
        a = 1
        b = -2 * G * M / (self.v1 * self.l1)
        c = -(self.v1**2 - 2 * G * M / self.l1)
        aux_1 = -b/2
        aux_2 = np.sqrt(b**2/4 - c)
        solution = np.array([aux_1 - aux_2, aux_1 + aux_2])
        return np.amin(solution)

    def semi_major_axis(self):
        """Returns the value of the major axis.
        """
        return (self.l1 + self.l2)/2

    def semi_minor_axis(self):
        """Returns the value of the minor axis.
        """
        return np.sqrt(self.l1 * self.l2)

    def orbital_period(self):
        """Returns the value of the orbital period.
        """
        return 2 * np.pi * self.a * self.b / (self.l1 * self.v1)

    def orbital_eccentricity(self):
        """Returns the value of the orbital eccentricity.
        """
        return (self.l2 - self.l1)/(self.l2 + self.l1)

    def print_relevant_info(self):
        """Used to show relevant information about the celestial body.
        """
        print("Semi-major axis: {:.3f}".format(self.a))
        print("Semi-minor axis: {:.3f}".format(self.b))
        print("Orbital period: {:.3f}".format(self.T))
        print("Orbital eccentricity: {:.3f}\n".format(self.e))
        pass
```

```python
v1earth = 3.0287e4
l1earth = 1.4710e11
v1halley = 5.4529e4
l1halley = 8.7830e10
earth = planetSolarSystem(v1earth, l1earth)
halley = planetSolarSystem(v1halley, l1halley)

# print the information asked by the exercise
earth.print_relevant_info()
halley.print_relevant_info()
earthT = earth.T
halleyT = halley.T

# convert to years
earthT = earthT / 60 / 60 / 24 / 365.25
halleyT = halleyT / 60 / 60 / 24 / 365.25
print("{:.1f} years".format(earthT))
print("{:.1f} years".format(halleyT))
```

    Semi-major axis: 149563598604.330
    Semi-minor axis: 149543307136.742
    Orbital period: 31543060.208
    Orbital eccentricity: 0.016

    Semi-major axis: 2685022330438.221
    Semi-minor axis: 681129146098.431
    Orbital period: 2399312511.845
    Orbital eccentricity: 0.967

    1.0 years
    76.0 years


# Lake model

see also [this general introduction to the workbench](https://waterprogramming.wordpress.com/2017/11/01/using-the-exploratory-modelling-workbench/) as a source of inspiration for completing the assignment below

## the lake model
The exploratory modeling workbench includes an example folder. This folder contains a variety of examples that demonstrate the functionality of the workbench. Many of these examples have been drawn from published cases. Here, we use the Lake Problem as an example for demonstrating some of the key functionality of the workbench.

We demonstrate some of the key capabilities of the exploratory modeling workbench using the Lake problem. The lake problem is a stylized and hypothetical decision problem where the population of a city has to decide on the amount of annual pollution it will put into a lake. It the pollution in the lake passes a threshold, it will suffer irreversible eutrophication.

\begin{equation}
    X_{(t+1)}=X_t+a_t+\frac{(X_t^q)}{(1+X_t^q )}- bX_t+\epsilon_t
\end{equation}

where $X_t$ is the pollution at time $t$, $a_t$ is the rate of anthropogenic pollution at time $t$, $b$ is the lake’s natural removal rate, $q$ is the lake's natural recycling rate, $\epsilon_t$ is the rate of natural pollution at time $t$. The rate of anthropogenic pollution $a_t$ is the decision variable and is somewhere between 0, and 0.1. So $a_t \in [0,0.1]$. The natural pollution $\epsilon_t$ is modeled, following Singh et al. (2015), as a log normal distribution with mean $\mu$ and standard deviation $\sigma$.
There are four outcomes of interest. The first is the average concentration of phosphor in the lake.

\begin{equation}
    f_{phosphorus}=  \frac{1}{\left\vert{T}\right\vert} \sum\limits_{t\in{T}} X_t
\end{equation}

where $\left\vert{T}\right\vert$ is the cardinality of the set of points in time.
The second objective is the economic benefit derived from polluting the lake. Following Singh et al. (2015), this is defined as the discounted benefit of pollution mines the costs of having a polluted lake

\begin{equation}
    f_{economic} = \sum\limits_{t \in {T}}\alpha a_t \delta^t
\end{equation}

where $\alpha$ is the utility derived from polluting and $\delta$ is the discount rate. By default, $\alpha$ is 0.04.
The third objective is related to the year over year change in the anthropogenic pollution rate.

\begin{equation}
    f_{inertia} =\frac{1}{\left\vert{T}\right\vert-1} \sum\limits_{t=1}^{\left\vert{T}\right\vert} I(|a_{t}-a_{t-1} |>\tau)
\end{equation}

where $I$ is an indicator function that is 0 if the statement is false, and 1 if the statement is true, $\tau$ is the threshold that is deemed undesirable, and is for illustrative purposes et to 0.2. Effectively, f_{inertia} is the fraction of years where the absolute value of the change in anthropogenic pollution is larger then $\tau$.
The fourth objective is the fraction of years where the pollution in the lake is below the critical threshold.

\begin{equation}
    f_{reliability} =  \frac{1}{\left\vert{T}\right\vert} \sum\limits_{t \in T}I(X_{t}<X_{crit} )
\end{equation}

where $I$ is an indicator function that is 0 if the statement is false, and 1 if the statement is true, $X_{crit}$ is the critical threshold of pollution and is a function of both $b$ and $q$.

The lake problem is characterized by both stochastic uncertainty and deep uncertainty. The stochastic uncertainty arises from the natural inflow. To reduce this stochastic uncertainty, multiple replications are performed and the average over the replication is taken. Deep uncertainty is presented by uncertainty about the mean $\mu$ and standard deviation $sigma$ of the lognormal distribution characterizing the natural inflow, the natural removal rate of the lake $\beta$, the natural recycling rate of the lake $q$, and the discount rate $\delta$. The table below specifies the ranges for the deeply uncertain factors, as well as their best estimate or default values.

## Assignment
1. Given the Python implementation of the lake problem in lakemodel_function.py, adapt this code and connect it to the workbench

for the uncertainties, use the following table

|Parameter	|Range	        |Default value|
|-----------|--------------:|------------:|
|$\mu$    	|0.01 – 0.05	|0.02         |
|$\sigma$	|0.001 – 0.005 	|0.0017       |
|$b$      	|0.1 – 0.45	    |0.42         |
|$q$	    |2 – 4.5	    |2            |
|$\delta$	|0.93 – 0.99	|0.98         |

For now, assume that for each year a release decision is made. The release is between 0 and 0.1. Carefully look at line 23 in lake_model.py to identify the name to use for each lever.

```python
from lakemodel_function import lake_problem

from ema_workbench import (Model, RealParameter, ScalarOutcome)

#instantiate the model
lake_model = Model('lakeproblem', function=lake_problem)
lake_model.time_horizon = 100 # used to specify the number of timesteps

#specify uncertainties
lake_model.uncertainties = [RealParameter('mean', 0.01, 0.05),
                            RealParameter('stdev', 0.001, 0.005),
                            RealParameter('b', 0.1, 0.45),
                            RealParameter('q', 2.0, 4.5),
                            RealParameter('delta', 0.93, 0.99)]

# set levers, one for each time step
lake_model.levers = [RealParameter(str(i), 0, 0.1) for i in
                     range(lake_model.time_horizon)] # we use time_horizon here

#specify outcomes
lake_model.outcomes = [ScalarOutcome('max_P'),
                       ScalarOutcome('utility'),
                       ScalarOutcome('inertia'),
                       ScalarOutcome('reliability')]
```

2. Explore the behavior of the system in the absence of any release using 1000 scenarios, and the default sampling approach.
    * visualize the outcomes of interest, are there any apparent trade-offs?
    * can you visually identify the uncertainties that drive system behavior?

```python
from ema_workbench import Policy, perform_experiments
from ema_workbench import ema_logging, SequentialEvaluator, MultiprocessingEvaluator

ema_logging.log_to_stderr(ema_logging.INFO)

policy = Policy("no release", **{l.name:0 for l in lake_model.levers})

n_scenarios = 1000
# results = perform_experiments(lake_model, n_scenarios, policy)

with MultiprocessingEvaluator(lake_model) as evaluator:
    results = evaluator.perform_experiments(n_scenarios, policy)

```

    [MainProcess/INFO] pool started
    [MainProcess/INFO] performing 1000 scenarios * 1 policies * 1 model(s) = 1000 experiments
    [MainProcess/INFO] 100 cases completed
    [MainProcess/INFO] 200 cases completed
    [MainProcess/INFO] 300 cases completed
    [MainProcess/INFO] 400 cases completed
    [MainProcess/INFO] 500 cases completed
    [MainProcess/INFO] 600 cases completed
    [MainProcess/INFO] 700 cases completed
    [MainProcess/INFO] 800 cases completed
    [MainProcess/INFO] 900 cases completed
    [MainProcess/INFO] 1000 cases completed
    [MainProcess/INFO] experiments finished
    [MainProcess/INFO] terminating pool

```python
experiments, outcomes = results
```

```python
experiments.head()
```

      b
      delta
      mean
      q
      stdev
      0
      1
      2
      3
      4
      ...
      93
      94
      95
      96
      97
      98
      99
      scenario
      policy
      model

      0
      0.210097
      0.971498
      0.018212
      4.188416
      0.002809
      0.0
      0.0
      0.0
      0.0
      0.0
      ...
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      1000
      no release
      lakeproblem

      1
      0.428540
      0.985574
      0.033526
      4.071724
      0.004715
      0.0
      0.0
      0.0
      0.0
      0.0
      ...
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      1001
      no release
      lakeproblem

      2
      0.189250
      0.959839
      0.023842
      3.540094
      0.002684
      0.0
      0.0
      0.0
      0.0
      0.0
      ...
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      1002
      no release
      lakeproblem

      3
      0.336643
      0.964847
      0.038453
      2.589152
      0.001026
      0.0
      0.0
      0.0
      0.0
      0.0
      ...
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      1003
      no release
      lakeproblem

      4
      0.132578
      0.948842
      0.034115
      2.369858
      0.004670
      0.0
      0.0
      0.0
      0.0
      0.0
      ...
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      0.0
      1004
      no release
      lakeproblem

5 rows × 108 columns

```python
for k,v in outcomes.items():
    print(k, v.shape)
```

    max_P (1000,)
    utility (1000,)
    inertia (1000,)
    reliability (1000,)

```python
outcomes['max_P'][0]
```

    0.08769934862388151

```python
from ema_workbench import save_results

save_results((experiments, outcomes), './blaat.tar.gz')
```

    Signature: save_results(results, file_name)
    Docstring:
    save the results to the specified tar.gz file. The results are
    stored as csv files. There is an x.csv, and a csv for each
    outcome. In addition, there is a metadata csv which contains
    the datatype information for each of the columns in the x array.

    Parameters
    ----------
    results : tuple
              the return of perform_experiments
    file_name : str
                the path of the file

    Raises
    ------
    IOError if file not found
    File:      ~/Documents/workspace/EMAworkbench/ema_workbench/util/utilities.py
    Type:      function

```python
experiments, outcomes = results

sns.pairplot(pd.DataFrame.from_dict(outcomes))
plt.show()
```

```python
from ema_workbench.analysis import pairs_plotting

pairs_plotting.pairs_scatter(experiments, outcomes)

fig = plt.gcf()
fig.set_size_inches(8,8)

plt.show()
```

3. Explore the behavior of the system over 1000 scenarios for 4 randomly sampled candidate strategies.
    * visualize the outcomes of interest
    * what can you say about how the release decision influences the system?

```python
n_scenarios = 1000
n_policies = 4
results = perform_experiments(lake_model, n_scenarios, n_policies)
```

    [MainProcess/INFO] performing 1000 scenarios * 4 policies * 1 model(s) = 4000 experiments
    [MainProcess/INFO] performing experiments sequentially
    [MainProcess/INFO] 400 cases completed
    [MainProcess/INFO] 800 cases completed
    [MainProcess/INFO] 1200 cases completed
    [MainProcess/INFO] 1600 cases completed
    [MainProcess/INFO] 2000 cases completed
    [MainProcess/INFO] 2400 cases completed
    [MainProcess/INFO] 2800 cases completed
    [MainProcess/INFO] 3200 cases completed
    [MainProcess/INFO] 3600 cases completed
    [MainProcess/INFO] 4000 cases completed
    [MainProcess/INFO] experiments finished

```python
experiments, outcomes = results
policies = experiments['policy']

data = pd.DataFrame.from_dict(outcomes)
data['policy'] = policies

sns.pairplot(data, hue='policy',  vars=outcomes.keys(), )
plt.show()
```

4. If you have not used parallelization in the foregoing, try to adapt your code to use parallelization. The workbench comes with two evaluators for parallelization. The `MultiProcessingingEvaluator` and the `IpyparallelEvaluator`. When can you use each?
    * for windows users, carefully read [the python documentation](https://docs.python.org/3.6/library/multiprocessing.html). Can you use multiprocessing from within the notebook if the lake_model function is defined within the notebook?

```python
from ema_workbench import MultiprocessingEvaluator

n_scenarios = 1000
n_policies = 4

with MultiprocessingEvaluator(lake_model) as evaluator:
    results = evaluator.perform_experiments(n_scenarios, n_policies)
```

    [MainProcess/INFO] pool started
    [MainProcess/INFO] performing 1000 scenarios * 4 policies * 1 model(s) = 4000 experiments
    [MainProcess/INFO] 400 cases completed
    [MainProcess/INFO] 800 cases completed
    [MainProcess/INFO] 1200 cases completed
    [MainProcess/INFO] 1600 cases completed
    [MainProcess/INFO] 2000 cases completed
    [MainProcess/INFO] 2400 cases completed
    [MainProcess/INFO] 2800 cases completed
    [MainProcess/INFO] 3200 cases completed
    [MainProcess/INFO] 3600 cases completed
    [MainProcess/INFO] 4000 cases completed
    [MainProcess/INFO] experiments finished
    [MainProcess/INFO] terminating pool

The ipyparallel Evaluator uses the ipyparallel library to suport multiprocessing. Working with this is a bit more involved than using multiprocessing. You first need to start ipcluster in cmd or a terminal:

    ipcluster start

next, you need a client to interact with this cluster.

```python
import ipyparallel

client = ipyparallel.Client()
client.ids
```

    [0, 1, 2, 3]

Next, we can define the lake model function. The %%px -local is known as a cell magic. This line means that the entire code block needs to be executed on all ipyarallel ipython engines, as well as in the ipython engine associated with the notebook.

```python
%%px --local

import math
import numpy as np

from scipy.optimize import brentq

def lake_problem(
         b = 0.42,          # decay rate for P in lake (0.42 = irreversible)
         q = 2.0,           # recycling exponent
         mean = 0.02,       # mean of natural inflows
         stdev = 0.0017,     # future utility discount rate
         delta = 0.98,      # standard deviation of natural inflows
         alpha = 0.4,       # utility from pollution
         nsamples = 100,    # Monte Carlo sampling of natural inflows
         **kwargs):
    decisions = [kwargs[str(i)] for i in range(100)]

    Pcrit = brentq(lambda x: x**q/(1+x**q) - b*x, 0.01, 1.5)
    nvars = len(decisions)
    X = np.zeros((nvars,))
    average_daily_P = np.zeros((nvars,))
    decisions = np.array(decisions)
    reliability = 0.0

    for _ in range(nsamples):
        X[0] = 0.0

        natural_inflows = np.random.lognormal(
                math.log(mean**2 / math.sqrt(stdev**2 + mean**2)),
                math.sqrt(math.log(1.0 + stdev**2 / mean**2)),
                size = nvars)

        for t in range(1,nvars):
            X[t] = (1-b)*X[t-1] + X[t-1]**q/(1+X[t-1]**q) + decisions[t-1] +\
                    natural_inflows[t-1]
            average_daily_P[t] += X[t]/float(nsamples)

        reliability += np.sum(X < Pcrit)/float(nsamples*nvars)

    max_P = np.max(average_daily_P)
    utility = np.sum(alpha*decisions*np.power(delta,np.arange(nvars)))
    inertia = np.sum(np.abs(np.diff(decisions)) > 0.02)/float(nvars-1)

    return max_P, utility, inertia, reliability
```

Now we can define our model as we did before. If you remove --local the code below will fail, can you explain why?

```python
#instantiate the model
lake_model = Model('lakeproblem', function=lake_problem)
lake_model.time_horizon = 100 # used to specify the number of timesteps

#specify uncertainties
lake_model.uncertainties = [RealParameter('mean', 0.01, 0.05),
                            RealParameter('stdev', 0.001, 0.005),
                            RealParameter('b', 0.1, 0.45),
                            RealParameter('q', 2.0, 4.5),
                            RealParameter('delta', 0.93, 0.99)]

# set levers, one for each time step
lake_model.levers = [RealParameter(str(i), 0, 0.1) for i in
                     range(lake_model.time_horizon)] # we use time_horizon here

#specify outcomes
lake_model.outcomes = [ScalarOutcome('max_P'),
                       ScalarOutcome('utility'),
                       ScalarOutcome('inertia'),
                       ScalarOutcome('reliability')]
```

And now we are ready to execute our code, note that IpyparallelEvaluator requires the client as interface to the cluster.

```python
from ema_workbench import  IpyparallelEvaluator

n_scenarios = 1000
n_policies = 4

with IpyparallelEvaluator(lake_model, client=client) as evaluator:
    results = evaluator.perform_experiments(n_scenarios, n_policies)
```

    [MainProcess/INFO] performing experiments using ipyparallel
    [MainProcess/INFO] performing 1000 scenarios * 4 policies * 1 model(s) = 4000 experiments
    [MainProcess/INFO] 400 cases completed
    [MainProcess/INFO] 800 cases completed
    [MainProcess/INFO] 1200 cases completed
    [MainProcess/INFO] 1600 cases completed
    [MainProcess/INFO] 2000 cases completed
    [MainProcess/INFO] 2400 cases completed
    [MainProcess/INFO] 2800 cases completed
    [MainProcess/INFO] 3200 cases completed
    [MainProcess/INFO] 3600 cases completed
    [MainProcess/INFO] 4000 cases completed
    [MainProcess/INFO] experiments finished

```python

```


```
import sympy as sm
# make the math look pretty
sm.init_printing()

```

```
# define all the symbols we will be using
#env parameters
g = sm.Symbol("g")
rho = sm.Symbol("rho")

#wind parameters
#thermals modelled with vertical airflows only

# vehicle parameters
e = sm.Symbol("e")
alph0 = sm.Symbol("alpha_0")
AR = sm.Symbol("AR")
m = sm.Symbol("m")
l = sm.Symbol("l")
Vh = sm.Symbol("V_H")
Vv = sm.Symbol("V_V")
Rf = sm.Symbol("V_F")
CLmin = sm.Symbol("C_{L_{min}}")
Cd0 = sm.Symbol("C_{D_0}")
Cdf = sm.Symbol("C_{D_f}")
CdL = sm.Symbol("C_{d_L}")
Cde = sm.Symbol("C_{d_e}")
Cdt = sm.Symbol("C_{d_t}")
a0 = sm.Symbol("a_0")

#variables
alpha = sm.Symbol("alpha")
alphaw = sm.Symbol("alpha_w")
beta = sm.Symbol("beta")
betaw = sm.Symbol("beta_w")
x = sm.Symbol("x")
y = sm.Symbol("y")
h = sm.Symbol("h")
chi = sm.Symbol("chi")
gamma = sm.Symbol("gamma")
sigma = sm.Symbol("sigma")
chiw = sm.Symbol("chi_w")
gammaw = sm.Symbol("gamma_w")
sigmaw = sm.Symbol("sigma_w")
V = sm.Symbol("V")
Vw = sm.Symbol("V_w")
D = sm.Symbol("D")
L = sm.Symbol("L")
C = sm.Symbol("C")

```

```
# relations
q = 1./2.*rho*(V**2)
qw = 1./2.*rho*(Vw**2)
S = (l**2)/AR
cbar = 1.03*l/AR
lt = 0.28*l
lf = 0.5*l
S = l**2/AR
St = Vh*cbar*S/lt
Sv = Vv*l*S/lt
Sf = 2.0*sm.pi*Rf*l*lf
ARv = 0.5*AR

Rvi = sm.Matrix([[sm.cos(chi),-sm.sin(chi),0.],[sm.sin(chi),sm.cos(chi),0.],[0.,0.,1.]])*sm.Matrix([[sm.cos(gamma),0.,sm.sin(gamma)],[0.,1.,0.],[-sm.sin(gamma),0.,sm.cos(gamma)]])*sm.Matrix([[1.,0.,0.],[0.,sm.cos(sigma),-sm.sin(sigma)],[0.,sm.sin(sigma),sm.cos(sigma)]])
Rbv = sm.Matrix([[sm.cos(alpha),0.,sm.sin(alpha)],[0.,1.,0.],[-sm.sin(alpha),0.,sm.cos(alpha)]])*sm.Matrix([[sm.cos(beta),sm.sin(beta),0.],[-sm.sin(beta),sm.cos(beta),0.],[0.,0.,1.]])

Tiv = Rvi

Rwi = Rvi.subs(chi,chiw).subs(gamma,gammaw).subs(sigma,sigmaw)
Rbw = Rbv.subs(alpha,alphaw).subs(beta,betaw)

#aerodynamics
#lift
CLa = a0 / (1.0+a0/(sm.pi*e*AR))

#drag
CD0 = Cdf*Sf/S + Cdt*(St + Sv)/S + Cde + Cd0
Rwi
```

```
# forces
#lift
CL = CLa*(alphaw - alph0)
Lw = qw*S*CL

#side
CC = CLa*Sv/S*betaw
Cw = qw*S*CC

#drag
CD = CD0 + CdL*((CL - CLmin)**2) + (CL**2)/(sm.pi*e*AR) + (Cw**2)/(sm.pi*e*AR)*(S/Sv)
Dw = qw*S*CD
```

```
# Dynamics
z = -h
dh = V*sm.sin(gamma)
dz = -dh
dx = V*sm.cos(chi)*sm.cos(gamma)
dy = V*sm.sin(chi)*sm.cos(gamma)
dV = -D/m-g*sm.sin(gamma)
dgamma = 1.0/(m*V)*(L*sm.cos(sigma) + C*sm.sin(sigma))-g/V*sm.cos(gamma)
dchi = 1.0/(m*V*sm.cos(gamma))*(L*sm.sin(sigma) - C*sm.cos(sigma))
f = sm.Matrix([[dh],[dx],[dy],[dV],[dgamma],[dchi]])

Vvec = sm.Matrix([[dx],[dy],[dz]])

f
```

```
sm.Matrix([f.diff(chi).transpose(), f.diff(gamma).transpose()]).transpose()
```

```

```


# Multi-Tissue Constrained Spherical Deconvolution

Multi-Tissue CSD (MT-CSD), or what *Jeurissen et al. (2014)* refers to as Multi-Shell Multi-Tissue CSD (MSMT-CSD), is in extension of regular CSD algorithm. In MT-CSD, Aside from the standard white matter convolution kernel, also a CSF and grey matter kernel are estimated from the data, and fitted simultaneously fitted for better FOD estimation.

$$
\begin{equation}
 E_{\textrm{MT-CSD}}= \underbrace{f_{\textrm{CSF}}\overbrace{E_{\textrm{iso}}(\cdot)}^{\textrm{Isotropic Response}}}_{\textrm{CSF}}+ \underbrace{f_{\textrm{GM}}\overbrace{E_{\textrm{iso}}(\cdot)}^{\textrm{Isotropic Response}}}_{\textrm{Grey Matter}} + \underbrace{f_r\left[\overbrace{\operatorname{FOD}(\operatorname{SH}|l_{\textrm{max}})}^{\textrm{Fiber Distribution}}\,*_{\mathbb{S}^2}\,\overbrace{E_{\textrm{WM}}(\cdot)}^{\textrm{Anisotropic Response}}\right]}_{\textrm{White Matter}}
\end{equation}
$$

The original algorithm uses registered T1 image segmentations of white/grey matter and CSF to obtain these separate kernels, but this has shown to be a difficult and expensive process. Instead, we implement unsupervised approach by *Dhollander et al. (2016)* that directly estimates the three tissue kernels from the dMRI data itself.

Once the kernels are estimated, they can be used exactly as any other Dmipy compartment model. This means they can be used in any modeling framework together with other models as usual.

As you will see, MT-CSD results in very well-defined FOD profiles and hardly any noisy voxels around CSF or Grey matter areas.

## Using Dmipy to set up the MT-CSD Model¶

Setting up the MT-CSD model in Dmipy is very easy. The only thing that needs to be done is to generate the WM/GM/CSF tissue response models from the data using the dhollander16 algorithm. Then, the rest of the modeling can be done as usual.

```python
from dmipy.data import saved_data
scheme_hcp, data_hcp = saved_data.wu_minn_hcp_coronal_slice()
```

    This data slice originates from Subject 100307 of the Human Connectome Project, WU-Minn Consortium (Principal Investigators: David Van Essen and Kamil Ugurbil; 1U54MH091657) funded by the 16 NIH Institutes and Centers that support the NIH Blueprint for Neuroscience Research; and by the McDonnell Center for Systems Neuroscience at Washington University.

### Unsupervised Three-Tissue Response Function Estimation

The dhollander16 algorithm, which heuristically estimates the white/grey matter and CSF response functions from the dMRI data (instead of using a T1 segmentation), can be directly called with the acquisition scheme and the data. The generated tissue response models are always generated in the same order.

To estimate the white matter response function, we can modularly choose any available algorithm, currently:
- the classic 'tournier07' algorithm based on FA only,
- the iterative 'tournier13' algorithm, which refines the response function based on the ratio between the first and secondarly peak of the FODs (currently used in MRtrix).

In the following commands we generate the three response functions, and change some of the default dhollander16 parameters to account for that we're only fitting a single slice instead of a whole brain:

```python
from dmipy.tissue_response.three_tissue_response import three_tissue_response_dhollander16
S0_tissue_responses, tissue_response_models, selection_map = three_tissue_response_dhollander16(
    scheme_hcp, data_hcp, wm_algorithm='tournier13',
    wm_N_candidate_voxels=150, gm_perc=0.2, csf_perc=0.4)
TR2_wm, TR1_gm, TR1_csf = tissue_response_models
S0_wm, S0_gm, S0_csf = S0_tissue_responses
```

    Tournier13 white matter response iteration 1
    Setup Tournier07 FOD optimizer in 0.0113952159882 seconds
    Fitting of 4530 voxels complete in 78.2730610371 seconds.
    Average of 0.017278821421 seconds per voxel.
    66.0 percent candidate voxel overlap.
    Tournier13 white matter response iteration 2
    Setup Tournier07 FOD optimizer in 0.0115969181061 seconds
    Fitting of 4530 voxels complete in 74.3183488846 seconds.
    Average of 0.0164058165308 seconds per voxel.
    96.7 percent candidate voxel overlap.
    Tournier13 white matter response iteration 3
    Setup Tournier07 FOD optimizer in 0.0115501880646 seconds
    Fitting of 4530 voxels complete in 110.235831976 seconds.
    Average of 0.0243346207452 seconds per voxel.
    99.3 percent candidate voxel overlap.
    Tournier13 white matter response iteration 4
    Setup Tournier07 FOD optimizer in 0.011011838913 seconds
    Fitting of 4530 voxels complete in 78.9287230968 seconds.
    Average of 0.0174235591825 seconds per voxel.
    100.0 percent candidate voxel overlap.
    White matter response converged

Notice that the white matter response is refined over several iterations (at each time taking the most source voxels that best fit a single-bundle profile). The overlap percentage indicates the convergence rate.

We can visualize which voxels were selected for which tissue responses. Notice that WM (red) was selected from coherent axon bundles, GM (green) from grey matter areas, and CSF (blue) from areas between sulci and ventricles.

```python
import matplotlib.pyplot as plt
%matplotlib inline
plt.figure(figsize=[6, 6])
plt.imshow(data_hcp[:, 0, :, 0].T, origin=True)
plt.imshow(selection_map.squeeze().transpose((1, 0, 2)), origin=True, alpha=0.8)
plt.axis('off')
plt.title('Selected tissue response voxels', fontsize=15);
```

We can visualize the tissue signal reponses of the three estimated kernels. Notice that these plots look extremely similar to those shown in *Jeurissen et al. (2014)*.

```python
fig, ax = plt.subplots()
for S0_response, TR_mod, name, c in zip(
        S0_tissue_responses[::-1], tissue_response_models[::-1],
        ['wm', 'gm', 'csf'][::-1], ['r', 'g', 'b'][::-1]):
    ax.plot(scheme_hcp.shell_bvalues, S0_response * TR_mod.spherical_mean(scheme_hcp) / 1e4,
            label=name, c=c, marker='.', ms=10)

plt.xlabel('b-value [s/m$^2$]', fontsize=14)
plt.ylabel('S0 Intensity', fontsize=14)
plt.title(r'Tissue Responses S$_0\times C^{SM}$(b)', fontsize=15)
ax.set_aspect(.25e10)
plt.legend(fontsize=15)
```

Now that the tissue response kernels are known, we can give them to a `MultiCompartmentSphericalHarmonicsModel` and fit them to the data as usual.

Note that tissue reponse models can only produce signals at the b-values (or other acquisition parameters) of the acquisition scheme shells they were estimated from. This means that you cannot use the tissue response kernels of one dataset to fit to another with a different acquisition scheme.

```python
from dmipy.core.modeling_framework import MultiCompartmentSphericalHarmonicsModel
from IPython.display import Image

mt_csd_mod = MultiCompartmentSphericalHarmonicsModel(
    models=tissue_response_models,
    S0_tissue_responses=S0_tissue_responses)
mt_csd_mod.visualize_model_setup(view=False, cleanup=False)
Image('Model Setup.png')
```

## Fitting MT-CSD to the Signal Attenuation or the Raw Signal

Tissue response models differ from parametric models in that they have a data-dependent, non-unity S0 tissue response. Including the S0-response in the fitting of MT-CSD affects both the estimation of the volume fractions and the shape of the FOD, as we will show below.

We can choose to include the S0 response in the fitting by setting `fit_S0_response=True`. This means the models are being fitted directly to the diffusion signal, instead of normalized models being fitted to the normalized signal attenuation.

```python
fit_args = {'acquisition_scheme': scheme_hcp, 'data': data_hcp, 'mask': data_hcp[..., 0]>0}

mt_csd_fits = []
for fit_S0_response in [True, False]:
    mt_csd_fits.append(mt_csd_mod.fit(fit_S0_response=fit_S0_response, **fit_args))
```

    Setup CVXPY FOD optimizer in 0.00970983505249 seconds
    Using parallel processing with 8 workers.
    Fitting of 8181 voxels complete in 353.459289074 seconds.
    Average of 0.0432049002657 seconds per voxel.
    Setup CVXPY FOD optimizer in 0.014527797699 seconds
    Using parallel processing with 8 workers.
    Fitting of 8181 voxels complete in 586.057314873 seconds.
    Average of 0.0716363910124 seconds per voxel.

Now that the fractions are fitted and there are 3, we can also visualize them as an RGB image. Notice that the WM/GM/CSF areas are reasonably nicely segmented by the estimation itself.

```python
import numpy as np
txts = ['Fit S0-response and Shape to Signal', 'Fit Only Shape to Signal Attenuation']
vfs_all = []

fig, axs = plt.subplots(2, 2, figsize=[17, 15])
axs = axs.ravel()
fig.suptitle('MT-CSD RGB WM/GM/CSF Volume Fraction Estimation', fontsize=20)
for i, fit in enumerate(mt_csd_fits):
    vfs = []
    names = ['partial_volume_0', 'partial_volume_1', 'partial_volume_2']
    for name in names:
        vfs.append(fit.fitted_parameters[name])
    vfs = np.transpose(np.array(vfs), (3, 2, 1, 0))
    vfs_all.append(vfs)
    vfs_sum = np.sum(vfs, axis=-1)
    vfs[vfs_sum > 0] = vfs[vfs_sum > 0] / vfs_sum[vfs_sum > 0][..., None]
    vfs_im = np.squeeze(np.clip(vfs, 0, 1))

    axs[i].imshow(vfs_im[15:-23, 10:-10], origin=True, interpolation='nearest')
    axs[i].axis('off');
    axs[i].set_title(txts[i], fontsize=18)

    im = axs[i + 2].imshow(np.squeeze(vfs_sum)[15:-23, 10:-10], origin=True, interpolation='nearest')
    axs[i + 2].axis('off');
    axs[i + 2].set_title('Estimated volume fraction sum', fontsize=18)
    fig.colorbar(im, ax=axs[i + 2], shrink=0.8)
```

Notice that the estimated volume fractions differ when including S0 response (left) compared to fitting the normalized signal attenuation.
- CSF is much more tightly delineated, which makes sense because CSF has a much higher S0-reponse than WM/GM.
- When fitting the S0-response we do not impose any unity constraint on the volume fractions. This is important we otherwise the fitting will be biased when the S0 response of the tissue differs from the response, but overall you can see the fractions don't vary much from unity.

### Visualizing Fiber Orientation Distributions

As before, we can visualize the FOD estimated using MT-CSD. We do both with and without including the S0 response.

```python
from dipy.data import get_sphere
from dipy.viz import window, actor
import numpy as np
import matplotlib.image as mpimg

sphere = get_sphere(name='symmetric362')
affine = np.eye(4)

txts = ['Fit S0-response and Shape to Signal', 'Fit Only Shape to Signal Attenuation']

fig, axs = plt.subplots(1, 2, figsize=[20, 10])
axs = axs.ravel()
fig.suptitle('MT-CSD FODs with WM/GM/CSF Background', fontsize=20)

for i, (txt, vfs, fit) in enumerate(zip(txts, vfs_all, mt_csd_fits)):
    fods = fit.fod(sphere.vertices)[70:90, :, 70:90]

    vfs2 = np.transpose(vfs, (2, 1, 0, 3))
    ai_im = actor.slicer(vfs2[70:90,0, 70:90, None, :],
        interpolation='nearest', affine=affine, opacity=0.7)

    ren = window.Renderer()
    fod_spheres = actor.odf_slicer(
        fods, sphere=sphere, scale=0.9, norm=False)
    fod_spheres.display_extent(
        0, fods.shape[0] - 1,
        0, fods.shape[1] - 1,
        0, fods.shape[2] - 1)
    fod_spheres.RotateX(90)
    fod_spheres.RotateZ(180)
    fod_spheres.RotateY(180)
    ren.add(fod_spheres)
    ren.add(ai_im)
    window.record(ren, size=[700, 700])

    img = mpimg.imread('fury.png')

    axs[i].imshow(img[100:-97, 100:-85])
    axs[i].set_title(txt, fontsize=18)
    axs[i].axis('off');
```

We show the RGB volume fractions as the background of the FODs. Red is WM, green is GM and blue is CSF. Notice that in WM the FODs are very sharp and noisefree, and the FODs have shrunken to be basically invisible in areas with higher (estimated) GM/CSF constributions. In other words, the signal contributions that would mess up the FOD in regular CSD have now instead been captured by the isotropic representations, resulting in cleaner FODs.

That being said, there are some observations to be made w.r.t. including the S0 response or not.
- The CSF area is smaller when including the S0 response, i.e. the overestimation of the CSF fraction has been reduced by including the tissue responses,
- The FOD profiles in the crossing area near the centrum semiovale (right side) are a bit more blobby when including the S0 response for some reason.

It is likely that these results will improve further when estimating the response functions from an entire brain (instead of just this example slice).

## References
- Tournier, J-Donald, Fernando Calamante, and Alan Connelly. "Robust determination of the fibre orientation distribution in diffusion MRI: non-negativity constrained super-resolved spherical deconvolution." Neuroimage 35.4 (2007): 1459-1472.
- Dhollander, T.; Raffelt, D. & Connelly, A. Unsupervised 3-tissue response function estimation from single-shell or multi-shell diffusion MR data without a co-registered T1 image. ISMRM Workshop on Breaking the Barriers of Diffusion MRI, 2016, 5
- Jeurissen, Ben, et al. "Multi-tissue constrained spherical deconvolution for improved analysis of multi-shell diffusion MRI data." NeuroImage 103 (2014): 411-426.


Dynamical system plotting using the following resources:
http://be150.caltech.edu/2017/handouts/dynamical_systems_approaches.html
http://systems-sciences.uni-graz.at/etextbook/sw2/phpl_python.html
https://tonysyu.github.io/animating-particles-in-a-flow.html
https://www.allendowney.com/blog/2019/07/25/matplotlib-animation-in-jupyter/
https://riptutorial.com/matplotlib/example/23558/basic-animation-with-funcanimation

Animations do not work in Jupyter lab, open with notebook!

```python
import numpy as np
import scipy.integrate
import scipy.optimize

# Plotting modules
import matplotlib.pyplot as plt
from matplotlib.animation import FuncAnimation
from IPython.display import HTML,clear_output
from time import sleep
```

```python
import sympy as sm
# define the system in this way (asuming a predator-prey-system with no negative values)
# to avoid interference x = r (for resource) and y = c (for consumer)
u, v = sm.symbols('u, v', negative=False)
###Equations of motion
a,b=2,2
U = sm.lambdify([u,v], v+u-u**3/3,["numpy"])
V = sm.lambdify([u,v], -(u-a+b*v),["numpy"])

# use sympy's way of setting equations to zero
UEqual = sm.Eq(U(u,v), 0)
VEqual = sm.Eq(V(u,v), 0)

# compute fixed points
fp=sm.solve( (UEqual, VEqual), u, v )
unc=sm.solve( UEqual, u, v )
vnc=sm.solve( VEqual, u, v )

uncfunc = sm.lambdify([u,v], unc[0],["numpy"])
vncfunc = sm.lambdify([u,v], vnc[0],["numpy"])

def Sys(X, t=0):
    # here X[0] = x and x[1] = y
    return np.array([ U(*X) , V(*X) ])
```

```python
###changes axes ranges and particle initial position
urange=[-3,5]
vrange=[-4,4]
particle_start=[.4,.5]
#plot
fig2 = plt.figure(figsize=(8,6))
ax4 = fig2.add_subplot(1,1,1)

x = np.linspace(*urange)
y = np.linspace(*vrange)

# plot nullclines
ax4.plot(x,np.array(uncfunc(x,y))[1], 'r-',  label='x-nullcline')
ax4.plot( vncfunc(x,y)[0],y, 'b-',  label='y-nullcline')

# plot fixed points
for point in fp:
    try:
        ax4.plot(point[0],point[1],"red", marker = "o", markersize = 10.0)
    except TypeError:
        pass
ax4.set_title("Quiverplot with nullclines")
ax4.legend(loc='best')

# quiverplot
# define a grid and compute direction at each point
# x = np.linspace(0, 2, 20)
# y = np.linspace(0, 2, 20)

X1 , Y1  = np.meshgrid(x, y)                    # create a grid
DX1, DY1 = Sys([X1, Y1])                        # compute growth rate on the grid
# M = (np.hypot(DX1, DY1))                        # norm growth rate
# M[ M == 0] = 1.                                 # avoid zero division errors
# DX1 /= M                                        # normalize each arrows
# DY1 /= M

ax4.streamplot(X1, Y1, DX1, DY1)
ax4.legend()
ax4.set_xlim(urange)
ax4.set_ylim(vrange)
ax4.grid()

###Add particle
point, = ax4.plot([], [], "blue", marker = "o", markersize = 10.0)
def init():
    point.set_data(*particle_start)
    return point,

# animation function.  This is called sequentially
def animate(i):
    curr = point.get_data()
    vel = np.array([U(*curr),V(*curr)])/100
    point.set_data(*(curr+vel))
    return point,

anim = FuncAnimation(fig2, animate, init_func=init,
                               frames=200, interval=1, blit=True,save_count=10)
plt.show()
```

    <IPython.core.display.Javascript object>

```python

```


## Implementation of a quantum algorithm for Discrete Log Problem over $\mathbb{Z}^*_N$ using IBM QisKit
Here, we present an implementation of a quantum algorithm for the discrete log problem on the multiplicative group. The algorithm is inspired from the explanation in the book **An Introduction to Quantum Computing**  by **P. Kaye, R. Laflamme, M. Mosca**. ([Link](http://mmrc.amss.cas.cn/tlb/201702/W020170224608149125645.pdf)). Refer section 7.4 of the book.

Here is the statement of discrete log problem for a group $G$.
Let $a, b \in G$. Let $b = a^t$. $t$ is called a discrete logarithm of $b$ to the base $a$.  The discrete log problem asks: Given $a, b \in G$, find a discrete logarithm of $b$ to the base $a$ or report that none exists.

There are two main parts of the algorithm for $G=\mathbb{Z}^*_N$ where $N \in \mathbb{Z}^{\geq 2}$:
The first part is the implementation of the transformation for modular multiplication.  For $a \in \mathbb{Z}^*_N$
$$U_a |x\rangle = |ax \text{ mod }N \rangle$$ This implementation is carried out in stages using the following functions: $\texttt{get_angles()}$, $\texttt{phi_add()}$, $\texttt{phi_add_mod_N()}$, $\texttt{multiply_mod_N()}$ and $\texttt{power_mod_N()}$.

The implementation of the first four functions is based on the [this](https://arxiv.org/abs/quant-ph/0205095) paper by Stephane Beauregard and is heavily inspired by [this](https://qiskit.org/documentation/_modules/qiskit/aqua/algorithms/single_sample/shor/shor.html) Qiskit implementation of Shor's algorithm.

The second part is the implementation of the phase estimation algorithm.

Using Shor's algorithm we can find $r$, the order of $a$ modulo $N$. WLOG, let $t \in \mathbb{Z}_r$. Now, on measuring the first two registers, we can find $(\frac{k}{r}, \frac{kt \text{ mod }r}{r} )$ for some $k \in \mathbb{Z}_r$. Since we know $r$, we get the pair $(k, kt \text{ mod }r)$. We run the circuit multiple times to find two pairs $(k_1, k_1t \text{ mod }r)$ and $(k_2, k_2t \text{ mod }r)$ such that $k_1$ and $k_2$ are relatively prime. Let $v_1 = k_1t \text{ mod }r$ and $v_2 = k_2t \text{ mod }r$. Now there exist $\lambda_1, \lambda_2 \in \mathbb{Z}$ such that $$\lambda_1 k_1 + \lambda_2 k_2 = 1$$ So, $t =  \lambda_1 k_1 t + \lambda_2 k_2 t$. Since $t = t \text{ mod }r$, $$t = (\lambda_1 v_1 + \lambda_2 v_2) \text{ mod }r$$

```python
!pip install qiskit==0.12.0
```

    Collecting qiskit==0.12.0
      Downloading https://files.pythonhosted.org/packages/5a/d6/bbb178455616d8aabd4d9df5110681d847da8303d3bd81d856fc14f929eb/qiskit-0.12.0.tar.gz
    Collecting qiskit_terra==0.9.0
      Downloading https://files.pythonhosted.org/packages/02/af/d41b625d00621e346844503cb1c27e3b0f9e5469e702276e5c1dfb236d03/qiskit_terra-0.9.0-cp36-cp36m-manylinux1_x86_64.whl (1.6MB)
         |████████████████████████████████| 1.6MB 2.7MB/s
    Collecting qiskit-aer==0.3.0
      Downloading https://files.pythonhosted.org/packages/6f/0e/1d7668ffafb85b373bc8681f3f6adffacd957a258a4a7211178cfc7e8559/qiskit_aer-0.3.0-cp36-cp36m-manylinux1_x86_64.whl (21.6MB)
         |████████████████████████████████| 21.6MB 72.9MB/s
    Collecting qiskit-ibmq-provider==0.3.2
      Downloading https://files.pythonhosted.org/packages/bb/eb/9c3e007c625b136c66a3f6da73f682f6ec32f6ee8274d998d84ec51cf3be/qiskit_ibmq_provider-0.3.2-py3-none-any.whl (85kB)
         |████████████████████████████████| 92kB 9.9MB/s
    Collecting qiskit-ignis==0.2.0
      Downloading https://files.pythonhosted.org/packages/78/f2/f4f89c65d2c13f473092f7e800519836611ddeeaab8d4591a70555b26a37/qiskit_ignis-0.2.0-py3-none-any.whl (100kB)
         |████████████████████████████████| 102kB 11.4MB/s
    Collecting qiskit-aqua==0.6.0
      Downloading https://files.pythonhosted.org/packages/5f/c0/894d00f156458edc5fd0dd559c07889c4bb3c8ca9ea17c921737bfcad6d9/qiskit_aqua-0.6.0-py3-none-any.whl (979kB)
         |████████████████████████████████| 983kB 45.8MB/s
    Collecting sympy>=1.3
      Downloading https://files.pythonhosted.org/packages/ce/5b/acc12e3c0d0be685601fc2b2d20ed18dc0bf461380e763afc9d0a548deb0/sympy-1.5.1-py2.py3-none-any.whl (5.6MB)
         |████████████████████████████████| 5.6MB 42.1MB/s
    Collecting marshmallow<3,>=2.17.0
      Downloading https://files.pythonhosted.org/packages/8d/5a/c8288b3fa34bfc5afeee56454db7c239b9d2f492c36172dafabf95c780af/marshmallow-2.21.0-py2.py3-none-any.whl (50kB)
         |████████████████████████████████| 51kB 6.9MB/s
    Requirement already satisfied: numpy>=1.13 in /usr/local/lib/python3.6/dist-packages (from qiskit_terra==0.9.0->qiskit==0.12.0) (1.18.3)
    Collecting marshmallow-polyfield<4,>=3.2
      Downloading https://files.pythonhosted.org/packages/fa/ba/6406b8cce3e42b2838d234310c3605627083bbb8f71dcc312c616480fe92/marshmallow_polyfield-3.2-py2.py3-none-any.whl
    Requirement already satisfied: psutil>=5 in /usr/local/lib/python3.6/dist-packages (from qiskit_terra==0.9.0->qiskit==0.12.0) (5.4.8)
    Requirement already satisfied: jsonschema<2.7,>=2.6 in /usr/local/lib/python3.6/dist-packages (from qiskit_terra==0.9.0->qiskit==0.12.0) (2.6.0)
    Collecting ply>=3.10
      Downloading https://files.pythonhosted.org/packages/a3/58/35da89ee790598a0700ea49b2a66594140f44dec458c07e8e3d4979137fc/ply-3.11-py2.py3-none-any.whl (49kB)
         |████████████████████████████████| 51kB 6.6MB/s
    Requirement already satisfied: scipy>=1.0 in /usr/local/lib/python3.6/dist-packages (from qiskit_terra==0.9.0->qiskit==0.12.0) (1.4.1)
    Requirement already satisfied: networkx>=2.2 in /usr/local/lib/python3.6/dist-packages (from qiskit_terra==0.9.0->qiskit==0.12.0) (2.4)
    Collecting nest-asyncio==1.0.0
      Downloading https://files.pythonhosted.org/packages/62/18/4c6a895c925e8822e72535d774861e9ca07089c4b938ed20002b1aaab66a/nest_asyncio-1.0.0-py3-none-any.whl
    Collecting requests-ntlm>=1.1.0
      Downloading https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
    Requirement already satisfied: requests>=2.19 in /usr/local/lib/python3.6/dist-packages (from qiskit-ibmq-provider==0.3.2->qiskit==0.12.0) (2.23.0)
    Collecting websockets<8,>=7
      Downloading https://files.pythonhosted.org/packages/43/71/8bfa882b9c502c36e5c9ef6732969533670d2b039cbf95a82ced8f762b80/websockets-7.0-cp36-cp36m-manylinux1_x86_64.whl (63kB)
         |████████████████████████████████| 71kB 9.8MB/s
    Requirement already satisfied: setuptools>=40.1.0 in /usr/local/lib/python3.6/dist-packages (from qiskit-ignis==0.2.0->qiskit==0.12.0) (46.1.3)
    Collecting dlx
      Downloading https://files.pythonhosted.org/packages/54/c0/b8fb5bb727e983b6f5251433ef941b48f38c65bb0bd6ec509e9185bcd406/dlx-1.0.4.tar.gz
    Collecting quandl
      Downloading https://files.pythonhosted.org/packages/07/ab/8cd479fba8a9b197a43a0d55dd534b066fb8e5a0a04b5c0384cbc5d663aa/Quandl-3.5.0-py2.py3-none-any.whl
    Requirement already satisfied: h5py in /usr/local/lib/python3.6/dist-packages (from qiskit-aqua==0.6.0->qiskit==0.12.0) (2.10.0)
    Requirement already satisfied: cvxopt in /usr/local/lib/python3.6/dist-packages (from qiskit-aqua==0.6.0->qiskit==0.12.0) (1.2.5)
    Collecting docplex
      Downloading https://files.pythonhosted.org/packages/03/a8/5dc675dbe179a2bd4d1d40522f009b4517ad32890d55ae49449709128d0a/docplex-2.13.184.tar.gz (571kB)
         |████████████████████████████████| 573kB 49.7MB/s
    Collecting pyscf; sys_platform != "win32"
      Downloading https://files.pythonhosted.org/packages/7a/ea/8ebe88368935dcc9a1a309764846e0bf5d04352d44067741aab3626ceb7f/pyscf-1.7.1-cp36-cp36m-manylinux1_x86_64.whl (24.3MB)
         |████████████████████████████████| 24.3MB 1.8MB/s
    Requirement already satisfied: fastdtw in /usr/local/lib/python3.6/dist-packages (from qiskit-aqua==0.6.0->qiskit==0.12.0) (0.3.4)
    Requirement already satisfied: scikit-learn>=0.20.0 in /usr/local/lib/python3.6/dist-packages (from qiskit-aqua==0.6.0->qiskit==0.12.0) (0.22.2.post1)
    Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.6/dist-packages (from sympy>=1.3->qiskit_terra==0.9.0->qiskit==0.12.0) (1.1.0)
    Requirement already satisfied: decorator>=4.3.0 in /usr/local/lib/python3.6/dist-packages (from networkx>=2.2->qiskit_terra==0.9.0->qiskit==0.12.0) (4.4.2)
    Collecting cryptography>=1.3
      Downloading https://files.pythonhosted.org/packages/3c/04/686efee2dcdd25aecf357992e7d9362f443eb182ecd623f882bc9f7a6bba/cryptography-2.9.2-cp35-abi3-manylinux2010_x86_64.whl (2.7MB)
         |████████████████████████████████| 2.7MB 42.9MB/s
    Collecting ntlm-auth>=1.0.2
      Downloading https://files.pythonhosted.org/packages/50/09/5e397eb18685b14fd8b209e26cdb4fa6451c82c1bcc651fef05fa73e7b27/ntlm_auth-1.4.0-py2.py3-none-any.whl
    Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.6/dist-packages (from requests>=2.19->qiskit-ibmq-provider==0.3.2->qiskit==0.12.0) (2.9)
    Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.6/dist-packages (from requests>=2.19->qiskit-ibmq-provider==0.3.2->qiskit==0.12.0) (1.24.3)
    Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.6/dist-packages (from requests>=2.19->qiskit-ibmq-provider==0.3.2->qiskit==0.12.0) (2020.4.5.1)
    Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.6/dist-packages (from requests>=2.19->qiskit-ibmq-provider==0.3.2->qiskit==0.12.0) (3.0.4)
    Requirement already satisfied: more-itertools in /usr/local/lib/python3.6/dist-packages (from quandl->qiskit-aqua==0.6.0->qiskit==0.12.0) (8.2.0)
    Requirement already satisfied: python-dateutil in /usr/local/lib/python3.6/dist-packages (from quandl->qiskit-aqua==0.6.0->qiskit==0.12.0) (2.8.1)
    Requirement already satisfied: pandas>=0.14 in /usr/local/lib/python3.6/dist-packages (from quandl->qiskit-aqua==0.6.0->qiskit==0.12.0) (1.0.3)
    Requirement already satisfied: six in /usr/local/lib/python3.6/dist-packages (from quandl->qiskit-aqua==0.6.0->qiskit==0.12.0) (1.12.0)
    Collecting inflection>=0.3.1
      Downloading https://files.pythonhosted.org/packages/52/c1/36be286d85dbd76527fb613527222a795d7c071da195fa916e7bf3cb03cb/inflection-0.4.0-py2.py3-none-any.whl
    Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.6/dist-packages (from scikit-learn>=0.20.0->qiskit-aqua==0.6.0->qiskit==0.12.0) (0.14.1)
    Requirement already satisfied: cffi!=1.11.3,>=1.8 in /usr/local/lib/python3.6/dist-packages (from cryptography>=1.3->requests-ntlm>=1.1.0->qiskit-ibmq-provider==0.3.2->qiskit==0.12.0) (1.14.0)
    Requirement already satisfied: pytz>=2017.2 in /usr/local/lib/python3.6/dist-packages (from pandas>=0.14->quandl->qiskit-aqua==0.6.0->qiskit==0.12.0) (2018.9)
    Requirement already satisfied: pycparser in /usr/local/lib/python3.6/dist-packages (from cffi!=1.11.3,>=1.8->cryptography>=1.3->requests-ntlm>=1.1.0->qiskit-ibmq-provider==0.3.2->qiskit==0.12.0) (2.20)
    Building wheels for collected packages: qiskit, dlx, docplex
      Building wheel for qiskit (setup.py) ... done
      Created wheel for qiskit: filename=qiskit-0.12.0-cp36-none-any.whl size=1397 sha256=4bf047879260b3bdac57129bbfb3990b236b6fd100b05b5ea31313ba9e50d6ab
      Stored in directory: /root/.cache/pip/wheels/de/fc/68/2318ad43f26583ec1ac8e30bedd43e71211c9256429b7f829c
      Building wheel for dlx (setup.py) ... done
      Created wheel for dlx: filename=dlx-1.0.4-cp36-none-any.whl size=5712 sha256=735c539e94e890f3f2537238b234a367f1087ba8b8efc1db400011dd4824182e
      Stored in directory: /root/.cache/pip/wheels/bb/ba/15/fdd0deb104df3254912998150ba9245668db06b00af5912d1a
      Building wheel for docplex (setup.py) ... done
      Created wheel for docplex: filename=docplex-2.13.184-cp36-none-any.whl size=633100 sha256=1a7aa1a2df940cc09a33bfcd2d24d4454250b4292cd14f4fc8d1f093ca4f0638
      Stored in directory: /root/.cache/pip/wheels/35/67/46/e9d3c1cb652addc314816a29396c1c427c2735e8f6c656aab3
    Successfully built qiskit dlx docplex
    Installing collected packages: sympy, marshmallow, marshmallow-polyfield, ply, qiskit-terra, qiskit-aer, nest-asyncio, cryptography, ntlm-auth, requests-ntlm, websockets, qiskit-ibmq-provider, qiskit-ignis, dlx, inflection, quandl, docplex, pyscf, qiskit-aqua, qiskit
      Found existing installation: sympy 1.1.1
        Uninstalling sympy-1.1.1:
          Successfully uninstalled sympy-1.1.1
    Successfully installed cryptography-2.9.2 dlx-1.0.4 docplex-2.13.184 inflection-0.4.0 marshmallow-2.21.0 marshmallow-polyfield-3.2 nest-asyncio-1.0.0 ntlm-auth-1.4.0 ply-3.11 pyscf-1.7.1 qiskit-0.12.0 qiskit-aer-0.3.0 qiskit-aqua-0.6.0 qiskit-ibmq-provider-0.3.2 qiskit-ignis-0.2.0 qiskit-terra-0.9.0 quandl-3.5.0 requests-ntlm-1.1.0 sympy-1.5.1 websockets-7.0

```python
import numpy as np
import math
from time import time
from math import gcd
from qiskit import(QuantumCircuit, QuantumRegister, ClassicalRegister, execute, Aer)
from qiskit.aqua.circuits import FourierTransformCircuits as ftc
from qiskit.aqua.circuits.gates import mcu1
from qiskit.visualization import plot_histogram
from qiskit.quantum_info.operators import Operator
from qiskit.aqua.utils import summarize_circuits
```

$\texttt{mod-exp(a, b, N)}$ computes $a^b\text{ mod }N$ where $a \in \mathbb{Z}$, $b \in \mathbb{Z}^{\geq0}$ and $N \in \mathbb{Z}^+$.

```python
def mod_exp(a, b, N):
    X = int(a)
    E = int(b)
    m = int(N)
    if(E<0 or m<1):
        return None
    X = X % m
    Y = 1
    while E > 0:
        if E % 2 == 0:
            X = (X * X) % m
            E = E/2
        else:
            Y = (X * Y) % m
            E = E - 1
    return Y
```

$\texttt{egcd(a, b)}$, where  $a, b \in \mathbb{Z}$,  returns a 3-tuple $(g, x, y)$ where  $a, b, g \in \mathbb{Z}$ such that $|g| = \text{gcd}(a, b)$ and $ax + by = g$. If $a=b=0$, the $g=0$ is returned.

```python
def egcd(a, b):
    if a == 0:
        return (b, 0, 1)
    else:
        g, y, x = egcd(b % a, a)
        return (g, x - (b // a) * y, y)
```

$\texttt{modinv(a, n)}$, where $a, n \in \mathbb{Z}$ such that $n \geq 2$, returns $a^{-1} \text{ mod } n$ if $a$ and $n$ are relatively prime and returns $\texttt{None}$ otherwise.

```python
def modinv(a, n):
    if(n<2):
        return None
    g, _, x = egcd(n, a)
    if(g!=1):
        return None
    return x % n
```

$\texttt{order(a, N)}$, where $a \in \mathbb{Z}$ and $N \in \mathbb{Z}^{\geq 2}$ such that $a$ and $N$ are relatively prime, returns the smallest positive integer $r$ such that $a^r \equiv 1 \text{ mod } N$.  One may replace it with Shor's order finding algorithm.

```python
def order(a, N):
    N = int(N)
    a = int(a) % N
    if(N<2 or gcd(a, N)!=1):
        return None
    r = 1
    p = a
    while(p != 1):
        p = (p*a) % N
        r = r + 1
    return r
```

$\texttt{get_noise(p_meas, p_gate)}$ sets the noise parameters for gates and measurements. For full explanation, see [this link](https://qiskit.org/textbook/ch-quantum-hardware/error-correction-repetition-code.html#Correcting-errors-in-qubits)

```python
def get_noise(p_meas, p_gate):
    error_meas = pauli_error([('X', p_meas), ('I', 1 - p_meas)])
    error_gate1 = depolarizing_error(p_gate, 1)
    error_gate2 = error_gate1.tensor(error_gate1)

    noise_model = NoiseModel()
    noise_model.add_all_qubit_quantum_error(error_meas, "measure")
    noise_model.add_all_qubit_quantum_error(error_gate1, ["x", "u1", "u2"])
    noise_model.add_all_qubit_quantum_error(error_gate2, ["cx"])

    return noise_model
```

$\texttt{get_angles(a, n)}$ computes the rotation angle to be used in Fourier transform-like circuit for addition with $a$. $n$ denotes the number of bits used for representation. $a$ must be known classically.

```python
def get_angles(a, n):
    bit = bin(abs(int(a)))[2:].zfill(n)
    theta = np.zeros(n)
    for i in range(n):
        for j in range(n-1-i, n):
            if bit[j] == '1':
                theta[i] += 2**(n-i-j-1)
    theta = theta * np.pi
    return theta
```

$\texttt{phi_add(circuit, q, a, n, c=[], inverse = False)}$ creates circuit to add(subtract, if $\texttt{inverse==True}$) $a$ to(from) $q$ modulo $2^n$ in Fourier Space. $n$ is the number of bits used for representation. $c$ is a (possibly empty) list of control bits. $a$ must be known classically. $q$ is a quantum register of $n$ bits.

```python
def phi_add(circuit, q, a, n, c=[], inverse = False):
    theta = get_angles(a, n)
    if(inverse):
        theta = -theta
    if not c:
        for i in range(n):
            circuit.u1(theta[i], q[i])
    else:
        for i in range(n):
            circuit.mcu1(theta[i], c, q[i])
```

$\texttt{phi_add_mod_N(circuit, q, a, n, N, c, aux, inverse = False)}$ creates circuit to add(subtract, if $\texttt{inverse==True}$) $a \in \mathbb{Z}^+$ to(from) $q$ mod $N$ in Fourier Space. $n$ is the number of bits used for representation. $c$ is a non-empty list of control bits. $a$ must be known classically. $q$ is a quantum register of $n+1$ bits. $q$ must be a non-negative integer less than $N$. $N$ must be a positive integer less than $2^n$. $aux$ is a single auxiliary qubit.

```python
def phi_add_mod_N(circuit, q, a, n, N, c, aux, inverse = False):
    if not c :
        print('There should be at least one control qubit')
        return None
    if N <= 0 or N >= 2**n :
        print('Choose appropriate N')
        return None
    if(inverse):
        a = (-a) % N
    else:
        a = a % N

    phi_add(circuit, q, a, n+1, c)
    phi_add(circuit, q, N, n+1, inverse=True)
    ftc.construct_circuit(circuit=circuit, qubits=[q[i] for i in reversed(range(n+1))], do_swaps=False, inverse=True)
    circuit.cx(q[n], aux)
    ftc.construct_circuit(circuit=circuit, qubits=[q[i] for i in reversed(range(n+1))], do_swaps=False)
    phi_add(circuit, q, N, n+1, [aux])
    phi_add(circuit, q, a, n+1, c, inverse=True)
    ftc.construct_circuit(circuit=circuit, qubits=[q[i] for i in reversed(range(n+1))], do_swaps=False, inverse=True)
    circuit.x(q[n])
    circuit.cx(q[n], aux)
    circuit.x(q[n])
    ftc.construct_circuit(circuit=circuit, qubits=[q[i] for i in reversed(range(n+1))], do_swaps=False)
    phi_add(circuit, q, a, n+1, c)
```

$\texttt{multiply_mod_N(circuit, q, a, n, N, c, aux)}$ creates circuit to multiply $a$ to $q$ mod $N$ in usual bit representation. $n$ is the number of bits used for representation. $a \in \mathbb{Z}^+$ must be known classically. $q$ is a quantum register of $n$ bits. $q$ must be a non-negative integer less than $N$ if we want the output to be $q$ mod $N$ (instead of $q$) when $c$ is $0$. $N$ must be a positive integer less than $2^n$. $a$ and $N$ must be relatively prime. $c$ is the only control bit. $aux$ is an auxiliary quantum register of $n+2$ bits.

```python
def multiply_mod_N(circuit, q, a, n, N, c, aux):
    if N <= 0 or N >= 2**n :
        print('Choose appropriate N')
        return None
    a = a % N
    a_inv = modinv(a, N)
    if(not a_inv):
        print('Inverse of a does not exist')
        return None
    pow_2_mod = np.zeros(n)
    pow_2_mod[0] = 1
    for i in range(1, n):
        pow_2_mod[i] = (2*pow_2_mod[i-1]) % N

    ftc.construct_circuit(circuit=circuit, qubits=[aux[i] for i in reversed(range(n+1))], do_swaps=False)
    for i in range(n):
        phi_add_mod_N(circuit, aux, (pow_2_mod[i]*a) % N, n, N, [c, q[i]], aux[n+1])
    ftc.construct_circuit(circuit=circuit, qubits=[aux[i] for i in reversed(range(n+1))], do_swaps=False, inverse=True)

    for i in range(n):
        circuit.cswap(c, q[i], aux[i])

    ftc.construct_circuit(circuit=circuit, qubits=[aux[i] for i in reversed(range(n+1))], do_swaps=False)
    for i in reversed(range(n)):
        phi_add_mod_N(circuit, aux, (pow_2_mod[i]*a_inv) % N, n, N, [c, q[i]], aux[n+1], inverse=True)
    ftc.construct_circuit(circuit=circuit, qubits=[aux[i] for i in reversed(range(n+1))], do_swaps=False, inverse=True)
```

$\texttt{power_mod_N(circuit, q, r, a, m, n, N, aux)}$ creates circuit to compute $(r \cdot a^q )\text{ mod } N$ in the usual bit representation. $n$ is the number of bits used in the representation. $a$ must be known classically. $q$ and $r$ are quantum registers of $m$ and $n$ bits respectively. $N$ must be a positive integer less than $2^n$. $a$ and $N$ must be relatively prime. $aux$ is an auxiliary quantum register of $n+2$ bits. $r$ must be a non-negative integer less than $N$. However, if we know that $q$ is always positive, $r$ can be any non-negative integer less that $2^n$.

```python
def power_mod_N(circuit, q, r, a, m, n, N, aux):
    if N <= 0 or N >= 2**n :
        print('Choose appropriate N')
        return None
    temp = a % N
    for i in range(m):
        multiply_mod_N(circuit, r, temp, n, N, q[i], aux)
        temp = (temp*temp) % N
```

$\texttt{discrete_log(a, b, N, n_shots=64)}$ returns  $t$ such that $a^t \equiv b \text{ mod } N$, with high probability, if such a $t$ exists. In this case, $t \in \mathbb{Z}_r$ where $r$ is the order of $a$ modulo $N$. Otherwise $t = \texttt{None}$.  $a, b \in \mathbb{Z}$ and $N \in \mathbb{Z}^{\geq 2}$ such that  $a$ and $N$ must be relatively prime.  This is the only function that uses the $\texttt{order()}$ function. One may replace its definition by Shor's order finding algorithm which works in polynomial time. This is not done here so as to focus only on the discrete log quantum algorithm.  $n\_shots$ is the maximum number of times the algorithm should be run to determine the answer.

```python
def discrete_log(a, b, N, noise_model=None, n_shots=32):
    N = int(N)
    a = int(a) % N
    b = int(b) % N
    n_shots = int(n_shots)

    if(N<2 or gcd(a, N) != 1 or n_shots<1):
        print('Input error')
        return None
    if(gcd(b, N) != 1):
        return None

    r = int(order(a, N))
    if(r==1):
        if(b==1):
            return 0
        else:
            return None
    m = int(np.ceil(np.log2(N+1)))
    n = int(np.ceil(np.log2(r+1))) + 1
    q = QuantumRegister(2*(m+n+1), 'q')
    c = ClassicalRegister(2*n, 'c')
    circuit = QuantumCircuit(q, c)

    for i in range(2*n):
        circuit.h(q[i])
    circuit.x(q[2*n])
    power_mod_N(circuit, q[:n], q[2*n:2*n+m], a, n, m, N, q[2*n+m:2*n+2*m+2])
    power_mod_N(circuit, q[n:2*n], q[2*n:2*n+m], b, n, m, N, q[2*n+m:2*n+2*m+2])
    ftc.construct_circuit(circuit=circuit, qubits=[q[i] for i in reversed(range(n))], inverse=True)
    ftc.construct_circuit(circuit=circuit, qubits=[q[i] for i in reversed(range(n, 2*n))], inverse=True)
    circuit.measure(q[:2*n], c[:2*n])

    simulator = Aer.get_backend('qasm_simulator')
    job = execute(circuit, simulator, noise_model=noise_model, shots=n_shots)
    result = job.result()
    counts = result.get_counts(circuit)
    M = len(counts)

    k = []
    kt_mod_r = []
    for key in counts:
        x = int(key[-n:], 2)
        y = int(key[-2*n:-n], 2)
        k.append(round((x*r)/(2**n)))
        kt_mod_r.append(round((y*r)/(2**n)))

    for j in range(M):
        for i in range(j+1):
            if(gcd(k[i], k[j]) == 1):
                _, v1, v2 = egcd(k[i], k[j])
                t = (kt_mod_r[i]*v1 + kt_mod_r[j]*v2)%r
                if(mod_exp(a, t, N) == b):
                    return t
    return None
```

Change the values of $a, b$ and $N$ in the following cell. Also, select the noise model by setting the arguments to the $\texttt{get_noise()}$ function. See the cell for $\texttt{get_noise()}$ for more details on noise parameters.

```python
a = 6
b = 30
N = 31
n_shots = 32
#noise_model = get_noise(0.01, 0.01)
noise_model = None
s_time = time()
print('t =', discrete_log(a, b, N, noise_model, n_shots))
print('Time:', round(time() - s_time), 's')
```

    t = 3
    Time: 197 s


Non Linear Regression Analysis

If the data shows a curvy trend, then linear regression will not produce very accurate results when compared to a non-linear regression because, as the name implies, linear regression presumes that the data is linear.
Let's learn about non linear regressions and apply an example on python. In this notebook, we fit a non-linear model to the datapoints corrensponding to China's GDP from 1960 to 2014.

Importing required libraries

```python
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
```

Though Linear regression is very good to solve many problems, it cannot be used for all datasets. First recall how linear regression, could model a dataset. It models a linear relation between a dependent variable y and independent variable x. It had a simple equation, of degree 1, for example y = $2x$ + 3.

```python
x = np.arange(-5.0, 5.0, 0.1)

##You can adjust the slope and intercept to verify the changes in the graph
y = 2*(x) + 3
y_noise = 2 * np.random.normal(size=x.size)
ydata = y + y_noise
#plt.figure(figsize=(8,6))
plt.plot(x, ydata,  'bo')
plt.plot(x,y, 'r')
plt.ylabel('Dependent Variable')
plt.xlabel('Indepdendent Variable')
plt.show()
```

Non-linear regressions are a relationship between independent variables $x$ and a dependent variable $y$ which result in a non-linear function modeled data. Essentially any relationship that is not linear can be termed as non-linear, and is usually represented by the polynomial of $k$ degrees (maximum power of $x$).

$$ \ y = a x^3 + b x^2 + c x + d \ $$

Non-linear functions can have elements like exponentials, logarithms, fractions, and others. For example: $$ y = \log(x)$$

Or even, more complicated such as :
$$ y = \log(a x^3 + b x^2 + c x + d)$$

Let's take a look at a cubic function's graph.

```python
x = np.arange(-5.0, 5.0, 0.1)

##You can adjust the slope and intercept to verify the changes in the graph
y = 1*(x**3) + 1*(x**2) + 1*x + 3
y_noise = 20 * np.random.normal(size=x.size)
ydata = y + y_noise
plt.plot(x, ydata,  'bo')
plt.plot(x,y, 'r')
plt.ylabel('Dependent Variable')
plt.xlabel('Indepdendent Variable')
plt.show()
```

As you can see, this function has $x^3$ and $x^2$ as independent variables. Also, the graphic of this function is not a straight line over the 2D plane. So this is a non-linear function.

Some other types of non-linear functions are:

### Quadratic

$$ Y = X^2 $$

```python
x = np.arange(-5.0, 5.0, 0.1)

##You can adjust the slope and intercept to verify the changes in the graph

y = np.power(x,2)
y_noise = 2 * np.random.normal(size=x.size)
ydata = y + y_noise
plt.plot(x, ydata,  'bo')
plt.plot(x,y, 'r')
plt.ylabel('Dependent Variable')
plt.xlabel('Indepdendent Variable')
plt.show()
```

### Exponential

An exponential function with base c is defined by $$ Y = a + b c^X$$ where b ≠0, c > 0 , c ≠1, and x is any real number. The base, c, is constant and the exponent, x, is a variable.

```python
X = np.arange(-5.0, 5.0, 0.1)

##You can adjust the slope and intercept to verify the changes in the graph

Y= np.exp(X)

plt.plot(X,Y)
plt.ylabel('Dependent Variable')
plt.xlabel('Indepdendent Variable')
plt.show()
```

### Logarithmic

The response $y$ is a results of applying logarithmic map from input $x$'s to output variable $y$. It is one of the simplest form of __log()__: i.e. $$ y = \log(x)$$

Please consider that instead of $x$, we can use $X$, which can be polynomial representation of the $x$'s. In general form it would be written as
\begin{equation}
y = \log(X)
\end{equation}

```python
X = np.arange(-5.0, 5.0, 0.1)

Y = np.log(X)

plt.plot(X,Y)
plt.ylabel('Dependent Variable')
plt.xlabel('Indepdendent Variable')
plt.show()
```

### Sigmoidal/Logistic

$$ Y = a + \frac{b}{1+ c^{(X-d)}}$$

```python
X = np.arange(-5.0, 5.0, 0.1)

Y = 1-4/(1+np.power(3, X-2))

plt.plot(X,Y)
plt.ylabel('Dependent Variable')
plt.xlabel('Indepdendent Variable')
plt.show()
```

# Non-Linear Regression example

For an example, we're going to try and fit a non-linear model to the datapoints corresponding to China's GDP from 1960 to 2014. We download a dataset with two columns, the first, a year between 1960 and 2014, the second, China's corresponding annual gross domestic income in US dollars for that year.

```python
import numpy as np
import pandas as pd

#downloading dataset
!wget -nv -O china_gdp.csv https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/china_gdp.csv

df = pd.read_csv("china_gdp.csv")
df.head(10)
```

    2019-07-30 05:45:53 URL:https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/china_gdp.csv [1218/1218] -> "china_gdp.csv" [1]

      Year
      Value

      0
      1960
      5.918412e+10

      1
      1961
      4.955705e+10

      2
      1962
      4.668518e+10

      3
      1963
      5.009730e+10

      4
      1964
      5.906225e+10

      5
      1965
      6.970915e+10

      6
      1966
      7.587943e+10

      7
      1967
      7.205703e+10

      8
      1968
      6.999350e+10

      9
      1969
      7.871882e+10

__Did you know?__ When it comes to Machine Learning, you will likely be working with large datasets. As a business, where can you host your data? IBM is offering a unique opportunity for businesses, with 10 Tb of IBM Cloud Object Storage: [Sign up now for free](http://cocl.us/ML0101EN-IBM-Offer-CC)

### Plotting the Dataset ###
This is what the datapoints look like. It kind of looks like an either logistic or exponential function. The growth starts off slow, then from 2005 on forward, the growth is very significant. And finally, it decelerate slightly in the 2010s.

```python
plt.figure(figsize=(8,5))
x_data, y_data = (df["Year"].values, df["Value"].values)
plt.plot(x_data, y_data, 'ro')
plt.ylabel('GDP')
plt.xlabel('Year')
plt.show()
```

### Choosing a model ###

From an initial look at the plot, we determine that the logistic function could be a good approximation,
since it has the property of starting with a slow growth, increasing growth in the middle, and then decreasing again at the end; as illustrated below:

```python
X = np.arange(-5.0, 5.0, 0.1)
Y = 1.0 / (1.0 + np.exp(-X))

plt.plot(X,Y)
plt.ylabel('Dependent Variable')
plt.xlabel('Indepdendent Variable')
plt.show()
```

The formula for the logistic function is the following:

$$ \hat{Y} = \frac1{1+e^{\beta_1(X-\beta_2)}}$$

$\beta_1$: Controls the curve's steepness,

$\beta_2$: Slides the curve on the x-axis.

### Building The Model ###
Now, let's build our regression model and initialize its parameters.

```python
def sigmoid(x, Beta_1, Beta_2):
     y = 1 / (1 + np.exp(-Beta_1*(x-Beta_2)))
     return y
```

Lets look at a sample sigmoid line that might fit with the data:

```python
beta_1 = 0.10
beta_2 = 1990.0

#logistic function
Y_pred = sigmoid(x_data, beta_1 , beta_2)

#plot initial prediction against datapoints
plt.plot(x_data, Y_pred*15000000000000.)
plt.plot(x_data, y_data, 'ro')
```

Our task here is to find the best parameters for our model. Lets first normalize our x and y:

```python
# Lets normalize our data
xdata =x_data/max(x_data)
ydata =y_data/max(y_data)
```

#### How we find the best parameters for our fit line?
we can use __curve_fit__ which uses non-linear least squares to fit our sigmoid function, to data. Optimal values for the parameters so that the sum of the squared residuals of sigmoid(xdata, *popt) - ydata is minimized.

popt are our optimized parameters.

```python
from scipy.optimize import curve_fit
popt, pcov = curve_fit(sigmoid, xdata, ydata)
#print the final parameters
print(" beta_1 = %f, beta_2 = %f" % (popt[0], popt[1]))
print(pcov)
```

     beta_1 = 690.447527, beta_2 = 0.997207
    [[ 1.52263822e+03 -2.88128654e-04]
     [-2.88128654e-04  7.25977514e-09]]

```python
help(curve_fit)
```

    Help on function curve_fit in module scipy.optimize.minpack:

    curve_fit(f, xdata, ydata, p0=None, sigma=None, absolute_sigma=False, check_finite=True, bounds=(-inf, inf), method=None, jac=None, **kwargs)
        Use non-linear least squares to fit a function, f, to data.

        Assumes ``ydata = f(xdata, *params) + eps``

        Parameters
        ----------
        f : callable
            The model function, f(x, ...).  It must take the independent
            variable as the first argument and the parameters to fit as
            separate remaining arguments.
        xdata : An M-length sequence or an (k,M)-shaped array for functions with k predictors
            The independent variable where the data is measured.
        ydata : M-length sequence
            The dependent data --- nominally f(xdata, ...)
        p0 : None, scalar, or N-length sequence, optional
            Initial guess for the parameters.  If None, then the initial
            values will all be 1 (if the number of parameters for the function
            can be determined using introspection, otherwise a ValueError
            is raised).
        sigma : None or M-length sequence or MxM array, optional
            Determines the uncertainty in `ydata`. If we define residuals as
            ``r = ydata - f(xdata, *popt)``, then the interpretation of `sigma`
            depends on its number of dimensions:

                - A 1-d `sigma` should contain values of standard deviations of
                  errors in `ydata`. In this case, the optimized function is
                  ``chisq = sum((r / sigma) ** 2)``.

                - A 2-d `sigma` should contain the covariance matrix of
                  errors in `ydata`. In this case, the optimized function is
                  ``chisq = r.T @ inv(sigma) @ r``.

                  .. versionadded:: 0.19

            None (default) is equivalent of 1-d `sigma` filled with ones.
        absolute_sigma : bool, optional
            If True, `sigma` is used in an absolute sense and the estimated parameter
            covariance `pcov` reflects these absolute values.

            If False, only the relative magnitudes of the `sigma` values matter.
            The returned parameter covariance matrix `pcov` is based on scaling
            `sigma` by a constant factor. This constant is set by demanding that the
            reduced `chisq` for the optimal parameters `popt` when using the
            *scaled* `sigma` equals unity. In other words, `sigma` is scaled to
            match the sample variance of the residuals after the fit.
            Mathematically,
            ``pcov(absolute_sigma=False) = pcov(absolute_sigma=True) * chisq(popt)/(M-N)``
        check_finite : bool, optional
            If True, check that the input arrays do not contain nans of infs,
            and raise a ValueError if they do. Setting this parameter to
            False may silently produce nonsensical results if the input arrays
            do contain nans. Default is True.
        bounds : 2-tuple of array_like, optional
            Lower and upper bounds on parameters. Defaults to no bounds.
            Each element of the tuple must be either an array with the length equal
            to the number of parameters, or a scalar (in which case the bound is
            taken to be the same for all parameters.) Use ``np.inf`` with an
            appropriate sign to disable bounds on all or some parameters.

            .. versionadded:: 0.17
        method : {'lm', 'trf', 'dogbox'}, optional
            Method to use for optimization.  See `least_squares` for more details.
            Default is 'lm' for unconstrained problems and 'trf' if `bounds` are
            provided. The method 'lm' won't work when the number of observations
            is less than the number of variables, use 'trf' or 'dogbox' in this
            case.

            .. versionadded:: 0.17
        jac : callable, string or None, optional
            Function with signature ``jac(x, ...)`` which computes the Jacobian
            matrix of the model function with respect to parameters as a dense
            array_like structure. It will be scaled according to provided `sigma`.
            If None (default), the Jacobian will be estimated numerically.
            String keywords for 'trf' and 'dogbox' methods can be used to select
            a finite difference scheme, see `least_squares`.

            .. versionadded:: 0.18
        kwargs
            Keyword arguments passed to `leastsq` for ``method='lm'`` or
            `least_squares` otherwise.

        Returns
        -------
        popt : array
            Optimal values for the parameters so that the sum of the squared
            residuals of ``f(xdata, *popt) - ydata`` is minimized
        pcov : 2d array
            The estimated covariance of popt. The diagonals provide the variance
            of the parameter estimate. To compute one standard deviation errors
            on the parameters use ``perr = np.sqrt(np.diag(pcov))``.

            How the `sigma` parameter affects the estimated covariance
            depends on `absolute_sigma` argument, as described above.

            If the Jacobian matrix at the solution doesn't have a full rank, then
            'lm' method returns a matrix filled with ``np.inf``, on the other hand
            'trf'  and 'dogbox' methods use Moore-Penrose pseudoinverse to compute
            the covariance matrix.

        Raises
        ------
        ValueError
            if either `ydata` or `xdata` contain NaNs, or if incompatible options
            are used.

        RuntimeError
            if the least-squares minimization fails.

        OptimizeWarning
            if covariance of the parameters can not be estimated.

        See Also
        --------
        least_squares : Minimize the sum of squares of nonlinear functions.
        scipy.stats.linregress : Calculate a linear least squares regression for
                                 two sets of measurements.

        Notes
        -----
        With ``method='lm'``, the algorithm uses the Levenberg-Marquardt algorithm
        through `leastsq`. Note that this algorithm can only deal with
        unconstrained problems.

        Box constraints can be handled by methods 'trf' and 'dogbox'. Refer to
        the docstring of `least_squares` for more information.

        Examples
        --------
        >>> import numpy as np
        >>> import matplotlib.pyplot as plt
        >>> from scipy.optimize import curve_fit

        >>> def func(x, a, b, c):
        ...     return a * np.exp(-b * x) + c

        Define the data to be fit with some noise:

        >>> xdata = np.linspace(0, 4, 50)
        >>> y = func(xdata, 2.5, 1.3, 0.5)
        >>> np.random.seed(1729)
        >>> y_noise = 0.2 * np.random.normal(size=xdata.size)
        >>> ydata = y + y_noise
        >>> plt.plot(xdata, ydata, 'b-', label='data')

        Fit for the parameters a, b, c of the function `func`:

        >>> popt, pcov = curve_fit(func, xdata, ydata)
        >>> popt
        array([ 2.55423706,  1.35190947,  0.47450618])
        >>> plt.plot(xdata, func(xdata, *popt), 'r-',
        ...          label='fit: a=%5.3f, b=%5.3f, c=%5.3f' % tuple(popt))

        Constrain the optimization to the region of ``0 <= a <= 3``,
        ``0 <= b <= 1`` and ``0 <= c <= 0.5``:

        >>> popt, pcov = curve_fit(func, xdata, ydata, bounds=(0, [3., 1., 0.5]))
        >>> popt
        array([ 2.43708906,  1.        ,  0.35015434])
        >>> plt.plot(xdata, func(xdata, *popt), 'g--',
        ...          label='fit: a=%5.3f, b=%5.3f, c=%5.3f' % tuple(popt))

        >>> plt.xlabel('x')
        >>> plt.ylabel('y')
        >>> plt.legend()
        >>> plt.show()

Now we plot our resulting regression model.

```python
x = np.linspace(1960, 2015, 55)
x = x/max(x)
plt.figure(figsize=(8,5))
y = sigmoid(x, *popt)
plt.plot(xdata, ydata, 'ro', label='data')
plt.plot(x,y, linewidth=3.0, label='fit')
plt.legend(loc='best')
plt.ylabel('GDP')
plt.xlabel('Year')
plt.show()
```

## Practice
Can you calculate what is the accuracy of our model?

```python
from sklearn.metrics import r2_score
score = r2_score(y, ydata)
score

```

    0.9718674410023772

Double-click __here__ for the solution.

```python
from sklearn.model_selection import train_test_split as tts
train_X, val_X, train_y, val_y = tts(xdata, ydata,
                                     random_state=1, test_size=0.2)
popt, pconv = curve_fit(sigmoid, train_X, train_y)
predictions = sigmoid(val_X, *popt)

# Validation
from sklearn.metrics import mean_absolute_error as mae
from sklearn.metrics import mean_squared_error as mse
from sklearn.metrics import r2_score

print('MAE: %.6f' % (mae(predictions, val_y)))
print('MSE: %.6f' % (mse(predictions, val_y)))
print('R-squared: %.6f' % (r2_score(predictions, val_y)))
```

    MAE: 0.039280
    MSE: 0.002044
    R-squared: 0.970681

Want to learn more?

IBM SPSS Modeler is a comprehensive analytics platform that has many machine learning algorithms. It has been designed to bring predictive intelligence to decisions made by individuals, by groups, by systems – by your enterprise as a whole. A free trial is available through this course, available here: SPSS Modeler

Also, you can use Watson Studio to run these notebooks faster with bigger datasets. Watson Studio is IBM's leading cloud solution for data scientists, built by data scientists. With Jupyter notebooks, RStudio, Apache Spark and popular libraries pre-packaged in the cloud, Watson Studio enables data scientists to collaborate on their projects without having to install anything. Join the fast-growing community of Watson Studio users today with a free account at Watson Studio

Thanks for completing this lesson!

Author:  Saeed Aghabozorgi

Saeed Aghabozorgi, PhD is a Data Scientist in IBM with a track record of developing enterprise level applications that substantially increases clients’ ability to turn data into actionable knowledge. He is a researcher in data mining field and expert in developing advanced analytic methods like machine learning and statistical modelling on large datasets.

Copyright © 2018 Cognitive Class. This notebook and its source code are released under the terms of the MIT License.


# Dense Coding

## Imports

```python
%matplotlib inline
```

```python
from IPython.display import display
```

```python
from sympy import init_printing
init_printing(use_latex=True)
```

```python
from sympy import sqrt, symbols, Rational
from sympy import expand, Eq, Symbol, simplify, exp, sin
from sympy.physics.quantum import *
from sympy.physics.quantum.qubit import *
from sympy.physics.quantum.gate import *
from sympy.physics.quantum.grover import *
from sympy.physics.quantum.qft import QFT, IQFT, Fourier
from sympy.physics.quantum.circuitplot import circuit_plot
```

## Dense coding

```python
psi = Qubit('00')/sqrt(2) + Qubit('11')/sqrt(2); psi
```

```python
circuits = [H(1)*CNOT(1,0), H(1)*CNOT(1,0)*X(1), H(1)*CNOT(1,0)*Z(1), H(1)*CNOT(1,0)*Z(1)*X(1)]
```

```python
for circuit in circuits:
    circuit_plot(circuit, nqubits=2)
    display(Eq(circuit*psi,qapply(circuit*psi)))
```


# 14 Linear Algebra: String Problem
## Motivating problem: Two masses on three strings
Two masses $M_1$ and $M_2$ are hung from a horizontal rod with length $L$ in such a way that a rope of length $L_1$ connects the left end of the rod to $M_1$, a rope of length $L_2$ connects $M_1$ and $M_2$, and a rope of length $L_3$ connects $M_2$ to the right end of the rod. The system is at rest (in equilibrium under gravity).

Find the angles that the ropes make with the rod and the tension forces in the ropes.

## Theoretical background
Treat $\sin\theta_i$ and $\cos\theta_i$ together with $T_i$, $1\leq i \leq 3$, as unknowns that have to simultaneously fulfill the nine equations
\begin{align}
-T_1 \cos\theta_1 + T_2\cos\theta_2 &= 0\\
 T_1 \sin\theta_1 - T_2\sin\theta_2 - W_1 &= 0\\
                   -T_2\cos\theta_2 + T_3\cos\theta_3 &= 0\\
                    T_2\sin\theta_2 + T_3\sin\theta_3 - W_2 &= 0\\
 L_1\cos\theta_1 + L_2\cos\theta_2 + L_3\cos\theta_3 - L &= 0\\
-L_1\sin\theta_1 -L_2\sin\theta_2 + L_3\sin\theta_3  &= 0\\
\sin^2\theta_1 + \cos^2\theta_1 - 1 &= 0\\
\sin^2\theta_2 + \cos^2\theta_2 - 1 &= 0\\
\sin^2\theta_3 + \cos^2\theta_3 - 1 &= 0
\end{align}

Consider the nine equations a vector function $\mathbf{f}$ that takes a 9-vector $\mathbf{x}$ of the unknowns as argument:
\begin{align}
\mathbf{f}(\mathbf{x}) &= 0\\
\mathbf{x} &= \left(\begin{array}{c}
x_0 \\ x_1 \\ x_2 \\
x_3 \\ x_4 \\ x_5 \\
x_6 \\ x_7 \\ x_8
\end{array}\right) =
\left(\begin{array}{c}
\sin\theta_1 \\ \sin\theta_2 \\ \sin\theta_3 \\
\cos\theta_1 \\ \cos\theta_2 \\ \cos\theta_3 \\
T_1 \\ T_2 \\ T_3
\end{array}\right) \\
\mathbf{L} &= \left(\begin{array}{c}
L \\ L_1 \\ L_2 \\ L_3
\end{array}\right), \quad
\mathbf{W} = \left(\begin{array}{c}
W_1 \\ W_2
\end{array}\right)
\end{align}

Using the unknowns from above, our system of 9 coupled equations is:

\begin{align}
-x_6 x_3 + x_7 x_4 &= 0\\
 x_6 x_0 - x_7 x_1 - W_1 &= 0\\
-x_7x_4 + x_8 x_5 &= 0\\
 x_7x_1 + x_8 x_2 - W_2 &= 0\\
 L_1x_3 + L_2 x_4 + L_3 x_5 - L &= 0\\
-L_1x_0 - L_2 x_1 + L_3 x_2  &= 0\\
x_{0}^{2} + x_{3}^{2} - 1 &= 0\\
x_{1}^{2} + x_{4}^{2} - 1 &= 0\\
x_{2}^{2} + x_{5}^{2} - 1 &= 0
\end{align}

Solve the root-finding problem $\mathbf{f}(\mathbf{x}) = 0$ with the **generalized Newton-Raphson** algorithm:

$$
\mathsf{J}(\mathbf{x}) \Delta\mathbf{x} = -\mathbf{f}(\mathbf{x})
$$

and

$$
\mathbf{x} \leftarrow \mathbf{x} + \Delta\mathbf{x}.
$$

## Problem setup
Set the problem parameters and the objective function $\mathbf{f}(\mathbf{x})$

```python
import numpy as np

# problem parameters
W = np.array([10, 20])
L = np.array([8, 3, 4, 4])

def f_2masses(x, L, W):
    return np.array([
            -x[6]*x[3] + x[7]*x[4],
             x[6]*x[0] - x[7]*x[1] - W[0],
            -x[7]*x[4] + x[8]*x[5],
             x[7]*x[1] + x[8]*x[2] - W[1],
             L[1]*x[3] + L[2]*x[4] + L[3]*x[5] - L[0],
            -L[1]*x[0] - L[2]*x[1] + L[3]*x[2],
            x[0]**2 + x[3]**2 - 1,
            x[1]**2 + x[4]**2 - 1,
            x[2]**2 + x[5]**2 - 1,
        ])

def fLW(x, L=L, W=W):
    return f_2masses(x, L, W)
```

### Initial values
Guess some initial values (they don't have to fullfil the equations!):

```python
# initial parameters
#theta0 = np.deg2rad([45, 45, 90])
#T0 = np.array([1, 1, 2])
#x0 = np.concatenate([np.sin(theta0), np.cos(theta0), T0])

x0 = np.array([1.5, 0.5, 0.5, 0.5, 0.5, 0.5, 1, 1, 1])
```

```python
x0
```

    array([1.5, 0.5, 0.5, 0.5, 0.5, 0.5, 1. , 1. , 1. ])

```python
f_2masses(x0, L, W)
```

    array([  0. ,  -9. ,   0. , -19. ,  -2.5,  -4.5,   1.5,  -0.5,  -0.5])

### Visualization
Plot the positions of the 2 masses and the 3 strings for any solution vector $\mathbf{x}$:

```python
import matplotlib
import matplotlib.pyplot as plt
%matplotlib inline
```

```python
def plot_2masses(x, L, W, **kwargs):
    """Plot 2 mass/3 string problem for parameter vector x and parameters L and W"""

    kwargs.setdefault('linestyle', '-')
    kwargs.setdefault('marker', 'o')
    kwargs.setdefault('linewidth', 1)

    ax = kwargs.pop('ax', None)
    if ax is None:
        ax = plt.subplot(111)

    r0 = np.array([0, 0])
    r1 = r0 + np.array([L[0], 0])
    rod = np.transpose([r0, r1])

    L1 = r0 + np.array([L[1]*x[3], -L[1]*x[0]])
    L2 = L1 + np.array([L[2]*x[4], -L[2]*x[1]])
    L3 = L2 + np.array([L[3]*x[5], L[3]*x[2]])
    strings = np.transpose([r0, L1, L2, L3])

    ax.plot(rod[0], rod[1], color="black", marker="d", linewidth=4)
    ax.plot(strings[0], strings[1], **kwargs)
    ax.set_aspect(1)
    return ax
```

What does the initial guess look like?

```python
plot_2masses(x0, L, W)
```

## Jacobian
Write a function `Jacobian(f, x, h=1e-5)` that computes the Jacobian matrix numerically (use the central difference algorithm).

\begin{align}
\mathbf{J} &= \frac{\partial \mathbf{f}}{\partial\mathbf{x}} \\
J_{ij} &= \frac{\partial f_i(x_1, \dots, x_j, \dots)}{\partial x_j} \\
       &\approx \frac{f_i(x_1, \dots, x_j + \frac{h}{2}, \dots) - f_i(x_1, \dots, x_j - \frac{h}{2}, \dots)}{h}
\end{align}

```python
def Jacobian(f, x, h=1e-5):
    """df_i/dx_j with central difference (fi(xj+h/2)-fi(xj-h/2))/h"""
    J = np.zeros((len(f(x)), len(x)), dtype=np.float64)
    hvec = np.zeros_like(x, dtype=np.float64)
    for j in range(len(x)):
        hvec *= 0
        hvec[j] = 0.5*h
        J[:, j] = (f(x + hvec) - f(x - hvec))/h
    return J
```

Test `Jacobian()` on
$$
\mathbf{g}(\mathbf{x}) = \begin{pmatrix}
  x_0^2 - x_1 \\ x_0
  \end{pmatrix}
$$
with analytical result

$$
\mathsf{J} = \left[\frac{\partial g_i}{\partial x_j}\right] =
\begin{pmatrix}
  \frac{\partial g_0}{\partial x_0} & \frac{\partial g_0}{\partial x_1}\\
  \frac{\partial g_1}{\partial x_0} & \frac{\partial g_1}{\partial x_1}
\end{pmatrix}
=
\begin{pmatrix}
  2 x_0 & -1\\
  1 & 0
\end{pmatrix}
$$

Given a test vector $\mathbf{x}_\text{test} = (1, 0)$, what is the numerical answer for $\mathsf{J}(\mathbf{x}_\text{test})$?

$$
\mathsf{J}(\mathbf{x}_\text{test}) = \begin{pmatrix}
  2 & -1\\
  1 & 0
\end{pmatrix}
$$

Test your `Jacobian()` function with $\mathbf{x}_\text{test}$ and check that you get the same answer:

```python
def g(x):
    return np.array([
            x[0]**2 - x[1],
            x[0]
        ])
x_test = np.array([1, 0])
J = Jacobian(g, x_test)
print(J)
```

    [[ 2. -1.]
     [ 1.  0.]]

Test that it also works for our starting vector:

```python
J0 = Jacobian(fLW, x0)
J0
```

    array([[ 0. ,  0. ,  0. , -1. ,  1. ,  0. , -0.5,  0.5,  0. ],
           [ 1. , -1. ,  0. ,  0. ,  0. ,  0. ,  1.5, -0.5,  0. ],
           [ 0. ,  0. ,  0. ,  0. , -1. ,  1. ,  0. , -0.5,  0.5],
           [ 0. ,  1. ,  1. ,  0. ,  0. ,  0. ,  0. ,  0.5,  0.5],
           [ 0. ,  0. ,  0. ,  3. ,  4. ,  4. ,  0. ,  0. ,  0. ],
           [-3. , -4. ,  4. ,  0. ,  0. ,  0. ,  0. ,  0. ,  0. ],
           [ 3. ,  0. ,  0. ,  1. ,  0. ,  0. ,  0. ,  0. ,  0. ],
           [ 0. ,  1. ,  0. ,  0. ,  1. ,  0. ,  0. ,  0. ,  0. ],
           [ 0. ,  0. ,  1. ,  0. ,  0. ,  1. ,  0. ,  0. ,  0. ]])

```python
J0.shape
```

    (9, 9)

## n-D Newton-Raphson Root Finding
Write a function `newton_raphson(f, x, Nmax=100, tol=1e-8, h=1e-5)` to find a root for a vector function `f(x)=0`. (See also [13 Root-finding by trial-and-error](https://asu-compmethodsphysics-phy494.github.io/ASU-PHY494/2020/03/26/13_Root_finding/) and the _1D Newton-Raphson algorithm_ in [13-Root-finding.ipynb](https://github.com/ASU-CompMethodsPhysics-PHY494/PHY494-resources/blob/master/13_root_finding/13-Root-finding.ipynb).) As a convergence criterion we demand that the length of the vector `f(x)` (the norm --- see `np.linalg.norm`) be less than the tolerance.

```python
def newton_raphson(f, x, Nmax=100, tol=1e-8, h=1e-5):
    """n-D Newton-Raphson: solves f(x) = 0.

    Iterate until |f(x)| < tol or nmax steps.
    """
    x = x.copy()
    for istep in range(Nmax):
        fx = f(x)
        if np.linalg.norm(fx) < tol:
            break
        J = Jacobian(f, x, h=h)
        Delta_x = np.linalg.solve(J, -fx)
        x += Delta_x
    else:
        print("Newton-Raphson: no root found after {0} iterations (eps={1}); "
            "best guess is {2} with error {3}".format(Nmax, tol, x, fx))
    return x
```

### Solve 2 masses/3 strings string problem

#### Solution

```python
x = newton_raphson(fLW, x0)
print(x0)
print(x)
```

    [1.5 0.5 0.5 0.5 0.5 0.5 1.  1.  1. ]
    [ 0.76100269  0.26495381  0.83570583  0.64874872  0.9642611   0.54917735
     17.16020978 11.54527968 20.27152804]

Plot the starting configuration and the solution:

```python
ax = plot_2masses(x0, L, W)
ax = plot_2masses(x, L, W, ax=ax)
```

Pretty-print the solution (angles in degrees):

```python
def pretty_print(x):
    theta = np.rad2deg(np.arcsin(x[0:3]))
    tensions = x[6:]
    print("theta1 = {0[0]:.1f} \t theta2 = {0[1]:.1f} \t theta3 = {0[2]:.1f}".format(theta))
    print("T1     = {0[0]:.1f} \t T2     = {0[1]:.1f} \t T3     = {0[2]:.1f}".format(tensions))
```

```python
print("Starting values")
pretty_print(x0)
print()
print("Solution")
pretty_print(x)
```

    Starting values
    theta1 = nan 	 theta2 = 30.0 	 theta3 = 30.0
    T1     = 1.0 	 T2     = 1.0 	 T3     = 1.0

    Solution
    theta1 = 49.6 	 theta2 = 15.4 	 theta3 = 56.7
    T1     = 17.2 	 T2     = 11.5 	 T3     = 20.3

    /Users/oliver/anaconda3/envs/phy494/lib/python3.6/site-packages/ipykernel_launcher.py:2: RuntimeWarning: invalid value encountered in arcsin

#### Show intermediate steps
Create a new function `newton_raphson_intermediates()` based on `newton_raphson()` that returns *all* trial `x` values including the last one.

```python
def newton_raphson_intermediates(f, x, Nmax=100, tol=1e-8, h=1e-5):
    """n-D Newton-Raphson: solves f(x) = 0.

    Iterate until |f(x)| < tol or nmax steps.

    Returns all intermediates.
    """
    intermediates = []
    x = x.copy()
    for istep in range(Nmax):
        fx = f(x)
        if np.linalg.norm(fx) < tol:
            break
        J = Jacobian(f, x, h=h)
        Delta_x = np.linalg.solve(J, -fx)
        intermediates.append(x.copy())
        x += Delta_x
    else:
        print("Newton-Raphson: no root found after {0} iterations (eps={1}); "
            "best guess is {2} with error {3}".format(Nmax, tol, x, fx))
    return np.array(intermediates)
```

Visualize the intermediate configurations:

```python
x_series = newton_raphson_intermediates(fLW, x0)
```

```python
ax = plt.subplot(111)
ax.set_prop_cycle("color", [plt.cm.viridis_r(i) for i in np.linspace(0, 1, len(x_series))])
for iteration, x in enumerate(x_series):
    plot_2masses(x, L, W, label=str(iteration), ax=ax)
ax.legend(loc="best");
```

It's convenient to turn the above plotting code into a function that we can reuse:

```python
def plot_series(x_series, L, W):
    """Plot all N masses/strings solution vectors in x_series (N, 9) array"""
    ax = plt.subplot(111)
    ax.set_prop_cycle("color", [plt.cm.viridis_r(i) for i in np.linspace(0, 1, len(x_series))])
    for iteration, x in enumerate(x_series):
        plot_2masses(x, L, W, label=str(iteration), ax=ax)
    ax.legend(loc="best")
    return ax
```

## Additional work
Try different masses, e.g. M1 = M2 = 10, or M1= 0 , M2 = 10.

Use nicer starting parameters that already fulfill the angle equations (7) - (9) (but it works with pretty much any guess):

```python
# initial parameters
theta0 = np.deg2rad([45, 45, 90])
T0 = np.array([1, 1, 2])
x0 = np.concatenate([np.sin(theta0), np.cos(theta0), T0])
```

### M1 = M2 = 10

```python
W_2 = np.array([10, 10])
def fLW_2(x, L=L, W=W_2):
    return f_2masses(x, L, W)
```

```python
x_series_2 = newton_raphson_intermediates(fLW_2, x0)
pretty_print(x_series_2[-1])
```

    theta1 = 57.9 	 theta2 = 8.8 	 theta3 = 52.1
    T1     = 13.1 	 T2     = 7.0 	 T3     = 11.3

```python
plot_series(x_series_2, L, W_2)
```

### M1 = 0, M2 = 10

```python
W_3 = np.array([0, 10])
def fLW_3(x):
    return f_2masses(x, L=L, W=W_3)
```

```python
x_series_3 = newton_raphson_intermediates(fLW_3, x0)
pretty_print(x_series_3[-1])
```

    theta1 = 30.0 	 theta2 = 30.0 	 theta3 = 61.0
    T1     = 4.8 	 T2     = 4.8 	 T3     = 8.7

```python
plot_series(x_series_3, L, W_3)
```