# Coordinate systems

## Introduction

This notebooks provides a tutorial about (curvilinear) coordinate systems. We use Mayavi to do the visualization of some of the surface for constant coordinate values.

### Use of Mayavi in this notebook
Mayavi can display either images or X3D elements on the notebook. The images are static and one cannot interact with them. The X3D output produces a fully interactive 3D scene. For information on how to interact with the scene, see here: http://www.x3dom.org/documentation/interaction/

Mayavi ships with some javascript files that can be installed as:

    $ jupyter nbextension install --py mayavi --user

This will install the x3dom Javascript and CSS files locally. Note that you do not need to “enable” the extension or anything after you run the above. For more instructions and options see the Installation of Jupyter Extensions. Doing this allows one to view X3D files without a network connection.

To view Mayavi visualizations on the notebook one should first do:

    from mayavi import mlab
    mlab.init_notebook()

    Subequently, one may simply do:

    s = mlab.test_plot3d()
    s

When the init_notebook method is called it configures the Mayavi objects so they can be rendered on the Jupyter notebook.

**More information:** http://docs.enthought.com/mayavi/mayavi/tips.html

```
from mayavi import mlab
import numpy as np
```

```
mlab.init_notebook()
```

    Notebook initialized with ipy backend.

```
red = (0.9, 0.1, 0.1)
blue = (0.2, 0.5, 0.7)
green = (0.3, 0.7, 0.3)
```

## Cylindrical coordinates

The [ISO standard 80000-2](https://en.wikipedia.org/wiki/ISO_80000-2) recommends the use of $(ρ, φ, z)$, where $ρ$ is the radial coordinate, $\varphi$ the azimuth, and $z$ the height.

For the conversion between cylindrical and Cartesian coordinates, it is convenient to assume that the reference plane of the former is the Cartesian $xy$-plane (with equation $z=0$, and the cylindrical axis is the Cartesian $z$-axis. Then the $z$-coordinate is the same in both systems, and the correspondence between cylindrical $(\rho, \varphi)$ and Cartesian $(x, y)$ are the same as for polar coordinates, namely

\begin{align}
x &= \rho \cos \varphi \\
y &= \rho \sin \varphi
\end{align}

in one direction, and

\begin{align}
    \rho &= \sqrt{x^2+y^2} \\
    \varphi &= \begin{cases}
        0 & \mbox{if } x = 0 \mbox{ and } y = 0\\
        \arcsin\left(\frac{y}{\rho}\right) & \mbox{if } x \geq 0 \\
        \arctan\left(\frac{y}{x}\right) & \mbox{if } x > 0 \\
        -\arcsin\left(\frac{y}{\rho}\right) + \pi & \mbox{if } x < 0
    \end{cases}
\end{align}

in the other. It is also common to use [$\varphi = atan2(y, x)$](https://en.wikipedia.org/wiki/Atan2).

```
cyl = mlab.figure(bgcolor=(1.0, 1.0, 1.0))
mlab.clf()
```

```
# Cylinder
phi, z = np.mgrid[0:2*np.pi:31j, -2:2*np.pi:31j]
x = 2*np.cos(phi)
y = 2*np.sin(phi)
z = z
cylinder = mlab.mesh(x, y, z, color=red)
```

```
# Vertical plane
rho, z = np.mgrid[0:3:31j, -2:2*np.pi:31j]
x = rho*np.cos(np.pi/4)
y = rho*np.sin(np.pi/4)
z = z
plane = mlab.mesh(x, y, z, color=blue, opacity=0.6)
```

```
# Horizontal plane
rho, phi = np.mgrid[0:3:31j, 0:2*np.pi:31j]
x = rho*np.cos(phi)
y = rho*np.sin(phi)
z = np.ones_like(x)
plane = mlab.mesh(x, y, z, color=green, opacity=0.6)
```

```
cyl
```

    Image(value=b'\x89PNG\r\n\x1a\n\x00\x00\x00\rIHDR\x00\x00\x01\x90\x00\x00\x01^\x08\x02\x00\x00\x00$?\xde_\x00\…

## Spherical coordinates

The use of $(r, θ, φ)$ to denote radial distance, inclination (or elevation), and azimuth, respectively, is common practice in physics, and is specified by [ISO standard 80000-2](https://en.wikipedia.org/wiki/ISO_80000-2).

The spherical coordinates of a point can be obtained from its [Cartesian coordinate system](https://en.wikipedia.org/wiki/ISO_80000-2) $(x, y, z)$

\begin{align}
r&=\sqrt{x^2 + y^2 + z^2} \\
\theta &= \arccos\frac{z}{\sqrt{x^2 + y^2 + z^2}} = \arccos\frac{z}{r} \\
\varphi &= \arctan \frac{y}{x}
\end{align}

The inverse tangent denoted in $\varphi = \arctan\left(\frac{y}{x}\right)$ must be suitably defined , taking into account the correct quadrant of $(x, y)$ (using the function ``atan2``).

Conversely, the Cartesian coordinates may be retrieved from the spherical coordinates (_radius_ $r$, _inclination_ $\theta$, _azimuth_ $\varphi$), where $r \in [0, \infty)$, $\theta \in [0, \pi]$ and $\varphi \in [0, 2\pi)$, by:

\begin{align}
x&=r \, \sin\theta \, \cos\varphi \\
y&=r \, \sin\theta \, \sin\varphi \\
z&=r \, \cos\theta
\end{align}

```
sph = mlab.figure(bgcolor=(1.0, 1.0, 1.0))
mlab.clf()
theta, phi = np.mgrid[0:np.pi:21j, 0:np.pi:21j]
```

```
# Sphere
x = np.sin(phi) * np.cos(theta)
y = np.sin(phi) * np.sin(theta)
z = np.cos(phi)
sphere = mlab.mesh(x, y, z, color=red)
spher2e = mlab.mesh(-x, -y, z,representation='wireframe', color=red)
```

```
# Cone
x = theta/3 * np.cos(phi)
y = theta/3 * np.sin(phi)
z = theta/3
cone = mlab.mesh(x, y, z, color=blue, opacity=0.6)
cone2 = mlab.mesh(-x, -y, z, representation='wireframe', color=blue)
```

```
# Plane
x = theta/np.pi
y = theta/np.pi
z = phi - np.pi/2
plane = mlab.mesh(x, y, z, color=green)
```

```
sph
```

    Image(value=b'\x89PNG\r\n\x1a\n\x00\x00\x00\rIHDR\x00\x00\x01\x90\x00\x00\x01^\x08\x02\x00\x00\x00$?\xde_\x00\…

## Ellipsoidal coordinates

```
ellip = mlab.figure(bgcolor=(1.0, 1.0, 1.0))
mlab.clf()
v, theta = np.mgrid[0:np.pi/2:21j, 0:np.pi:21j]
a = 3
b = 2
c = 1
```

```
# Ellipsoid
lam = 3
x =  np.sqrt(a**2  + lam) * np.cos(v) * np.cos(theta)
y = np.sqrt(b**2  + lam)* np.cos(v) * np.sin(theta)
z = np.sqrt(c**2 + lam) * np.sin(v)
ellipsoid = mlab.mesh(x, y, z, color=red)
ellipsoid2 = mlab.mesh(x, y, -z, color=red)
ellipsoid3 = mlab.mesh(x, -y, z, representation='wireframe', color=red)
ellipsoid4 = mlab.mesh(x, -y, -z, representation='wireframe', color=red)
```

```
# Hyperboloid of one sheet
mu = 2
x = np.sqrt(a**2 + mu) * np.cosh(v) * np.cos(theta)
y = np.sqrt(b**2 + mu) * np.cosh(v) * np.sin(theta)
z = np.sqrt(c**2 + mu) * np.sinh(v)
z = np.sqrt(c**2 + mu) * np.sinh(v)
hyper = mlab.mesh(x, y, z, color=blue)
hyper2 = mlab.mesh(x, y, -z, color=blue)
hyper3 = mlab.mesh(x, -y, z, representation='wireframe', color=blue)
hyper4 = mlab.mesh(x, -y, -z, representation='wireframe', color=blue)
```

```
# Hyperboloid of two sheets
nu = 1
x = np.sqrt(a**2 + nu) * np.cosh(v)
y = np.sqrt(c**2 + nu) * np.sinh(v) * np.sin(theta)
z = np.sqrt(b**2 + nu) * np.sinh(v) * np.cos(theta)
hyper_up = mlab.mesh(x, y, z, color=green)
hyper_down = mlab.mesh(-x, y, z, color=green)
hyper_up2 = mlab.mesh(x, -y, z, representation='wireframe', color=green)
hyper_down2 = mlab.mesh(-x, -y, z, representation='wireframe', color=green)
```

```
ellip
```

## References

1. Wikipedia contributors. ["Cylindrical coordinate system."](https://en.wikipedia.org/wiki/Cylindrical_coordinate_system) Wikipedia, The Free Encyclopedia. Wikipedia, The Free Encyclopedia, 12 Dec. 2016. Web. 9 Feb. 2017.

2. Wikipedia contributors. ["Spherical coordinate system."](https://en.wikipedia.org/wiki/Spherical_coordinate_system) Wikipedia, The Free Encyclopedia. Wikipedia, The Free Encyclopedia, 29 Jan. 2017. Web. 9 Feb. 2017.

3. Enthought Inc. [Mayavi: Tips and Trick](http://docs.enthought.com/mayavi/mayavi/tips.html). Web. 9 Feb. 2017.

```

```

```
from IPython.core.display import HTML
def css_styling():
    styles = open('./styles/custom_barba.css', 'r').read()
    return HTML(styles)
css_styling()
```

```

```


#
Vectores

```python
from sympy import *
init_printing(use_latex='mathjax')
x, y, z = symbols('x, y, z')
```

En Sympy un vector es simplemente una matriz fila y por ello para crear vectores debemos emplear "el constructor" **Matrix** (obligatorio con mayúscula). El vector es una lista de Python, y como tal sus coordenadas deben ir entre corchetes y separadas por comas. En realidad también se pueden usar los paréntesis, pero por motivos que no vienen al caso es mejor utilizar corchetes.

Sympy puede manejar vectores de cualquier dimensión, pero aquí trabajaremos en dimensión 3.

###
Dados los vectores $u=(4,3,-1)$ y $v=(2,9,1)$ realiza combinaciones lineales con ellos.

```python
v= Matrix([5,-3,6])
```

```python
u.dot(u)
```

$$101$$

Para calcular el producto escalar se utiliza el método **dot** y para el producto vectorial **cross**. Para el módulo (o norma) de un empleamos el método **norm**.

###
Calcula el producto escalar y vectorial de los vectores anteriores.

```python

```

```python

```

###
Comprueba que el producto escalar de un vector consigo mismo es el cuadrado del módulo.

```python

```

```python

```

###
Comprueba que el producto vectorial es antisimétrico y que es perpendicular a cada uno de los factores.

```python

```

El ángulo entre vectores se obtiene despejando $alpha$ en la fórmula:

$$\cos(\alpha)=\frac{u \cdot v}{|u| \cdot|v|}$$

###
Calcula el ángulo que forman $u$ y $v$.

```python
acos(u.dot(v)/(u.norm()*v.norm())).evalf()
```

$$1.02008837575294$$

```python

```

###
Dado el vector $w=(7,5,2)$ comprueba que el producto vectorial no es asociativo.

```python
plot()
```

```python
u.
```

```python

```


## Milestone 2

## Introduction

Differentiation, or the process of finding a derivative, is an extremely important mathematical operation with a wide range of applications. The discovery of extrema or zeros of functions is essential in any optimization problem, and the solving of differential equations is fundamental to modern science and engineering. Differentiation is essential in nearly all quantitative disciplines: physicists may take the derivative of the displacement of a moving object with respect to time in order to find the velocity of that object, and data scientists may use derivatives when optimizing weights in a neural network.

Naturally, we would like to compute the derivative as accurately and efficiently as possible. Two classical methods of calculating the derivative have clear shortcomings. Symbolic differentiation (finding the derivative of a given formula with respect to a specified variable, producing a new formula as its output) will be accurate, but can be quite expensive computationally. The finite difference method ($\frac{\partial f}{\partial x} = \frac{f(x+\epsilon)-f(x)}{\epsilon}$ for some small $\epsilon$) does not have this issue, but will be less precise as different values of epsilon will give different results. This brings us to automatic differentiation, a less costly and more precise approach.

## Background

Automatic differentiation breaks down the main function into elementary functions, evaluated upon one another. It then uses the chain rule to update the derivative at each step and ends in the derivative of the entire function.

To better understand this process, let's look at an example. Consider the example function

\begin{equation}
f(x) = x + 4(sin(\frac{x}{4}))
\end{equation}

We would like to compute the derivative of this function at a particular value of x. Let's say that in this case, we are interested in evaluating the derivative at $x=\pi$. In other words, we want to find $f'(\pi)$ where $f'(x) = \frac{\partial f}{\partial x}$

We know how to solve this _symbolically_ using methods that we learned in calculus, but remember, we want to compute this answer as accurately and efficiently as possible, which is why we want to solve it using automatic differentiation.

To solve this using automatic differentiation, we need to find the decomposition of the differentials provied by the chain rule. Remember, the chain rule is a formula for computing the derivative of the composition of two or more functions. So if we have a function $h\left(u\left(t\right)\right)$ and we want the derivative of $h$ with respect to $t$, then we know by the chain rule that the derivative is $\dfrac{\partial h}{\partial t} = \dfrac{\partial h}{\partial u}\dfrac{\partial u}{\partial t}.$ The chain rule can also be expanded to deal with multiple arguments and vector inputs (in which case we would be calculating the _gradient)_.

Our function $f(x)$ is composed of elemental functions for which we know the derivatives. We will separate out each of these elemental functions, evaluating the derivative at each step using the chain rule.

The evaluation trace for this problem looks like:

| Trace    | Elementary Operation    | Derivative     | $\left(f\left(a\right),  f^{\prime}\left(a\right)\right)$                                          |
| :------: | :----------------------:               | :------------------------------: | :------------------------------: |
| $x_{1}$  | $\pi$                      | $1$                | $\left(\pi, 1\right)$ |
| $x_{2}$  | $\frac{x_{1}}{4}$                               | $\frac{\dot{x}_{1}}{4}$                 | $\left(\dfrac{\pi}{4}, \frac{1}{4}\right)$ |
| $x_{3}$  | $\sin\left(x_{2}\right)$               | $\cos\left(x_{2}\right)\dot{x}_{2}$            | $\left(\dfrac{\sqrt{2}}{2}, \frac{\sqrt{2}}{8}\right)$ |
| $x_{4}$  | $4x_{3}$                               | $4\dot{x}_{3}$                  | $\left(2\sqrt{2}, \frac{\sqrt{2}}{2}\right)$ |
| $x_{5}$  | $x_{1} + x_{4}$                        | $\dot{x}_{1} + \dot{x}_{4}$ | $\left(\pi + 2\sqrt{2}, 1 + \frac{\sqrt{2}}{2}\right)$ |

This evaluation trace provides some intuition for how automatic differentiation (in the forward mode) is used to calculate the derivative of a function evaluated at a certain value ($f'(\pi) = 1 + \frac{\sqrt{2}}{2}$).

You may notice that when we computed the derivative above, we "seeded" the derivative with a value of 1. This seed vector doesn't have to be 1, but the utility of using a unit vector becomes apparent when we consider a problem involving directional derivatives.

The definition of the directional derivative (where p is the seed vector)
$$D_{p}x_{3} = \sum_{j=1}^{2}{\dfrac{\partial x_{3}}{\partial x_{j}}p_{j}}$$

can be expanded to

\begin{align}
  D_{p}x_{3} &= \dfrac{\partial x_{3}}{\partial x_{1}}p_{1} + \dfrac{\partial x_{3}}{\partial x_{2}}p_{2} \\
             &= x_{2}p_{1} + x_{1}p_{2}
\end{align}

If we choose p to be a the unit vector, we can see how this is beneficial:

$p = \left(1,0\right)$ gives $\dfrac{\partial f}{\partial x}$

$p = \left(0,1\right)$ gives $\dfrac{\partial f}{\partial y}$

So to summarize, the forward mode of automatic differentiation is really computing the _**product of the gradient of our function with the seed vector:**_

$$D_{p}x_{3} = \nabla x_{3}\cdot p.$$

If our function is a vector, then the forward mode actually computes $Jp$ where $J = \dfrac{\partial f_{i}}{\partial x_{j}}, \quad i = 1,\ldots, n, \quad j = 1,\ldots,m$ is the Jacobian matrix. Often we will really only want the "action" of the Jacobian on a vector, so we will just want to compute the matrix-vector product $Jp$ for some vector $p$.

Automatic differentiation can be used to compute derivatives to machine precision of functions $f:\mathbb{R}^{m} \to \mathbb{R}^{n}$

The forward mode is more efficient when $n\gg m$.
   - This corresponds to the case where the number of functions to evaluate is much greater than the number of inputs.
   - Actually computes the Jacobian-vector product $Jp$.

The reverse mode is more efficient when $n\ll m$.
   - This corresponds to the case where the number of inputs is much greater than the number of functions.
   - Actually computes the Jacobian-transpose-vector product $J^{T}p$.

## How to Use autodiff32
### Installation

**1) Create a virtual environment (optional)**

From the terminal, create a virtual environment:

_(The command below will create the virtual environment in your present working directory, so consider moving to a project folder or a known location before creating the environment)_

```virtualenv env```

activate the virtual environment:

```source env/bin/activate```

if you plan to launch a jupyter notebook using this virtual environment, run the following to install and set up jupyter in your virtual environment:

```python -m pip install jupyter```

```python -m ipykernel install --user --name=env```

**3) Install the autodiff32 package**

In the terminal, type:

```pip install autodiff32```

Package dependencies will be taken care of automatically!

_(Alternatively, it is also possible to install the autodiff32 package by downloading this GitHub repository. If you choose that method, use the requirements.txt file to ensure you have installed all necessary dependencies.)_

### How to use

It is easy to use the autodiff32 package in a Jupyter notebook, as we will demonstrate here:

_(Alternatively, you can start a Python interpreter by typing ```Python``` into the terminal, or work from your favorite Python IDE.)_

_(Remember, if you are using a virtual environment, follow steps 1 through 3 above and then type ```jupyter notebook``` into your terminal to launch a notebook. Inside the notebook, switch the kernel to that of your virtual environment.)_

```python
import autodiff32 as ad
import math # only necessary for this particular example

"""
Initialize an AutoDiff object
with the number you would like to pass into your function
"""
X = ad.AutoDiff(math.pi)

"""
Define your function of interest
"""
func = X + 4*ad.sin(X/4)
"""
Look at the derivative of your function
evaluated at the number you gave above
"""
print("derivative:",func.der)

"""
Look at the value of your function
evaluated at the number you gave:
"""
print("value:", func.val)
```

    derivative: 1.7071067811865475
    value: 5.970019778335983

Notice that this is the same equation used in our example above: $f(x) = x + 4(sin(\frac{x}{4}))$. Just for fun, let's see if the derivative that we calculated in the evolution trace is the same as the result using autodiff32:

```python
print("autodiff32 derivative:", func.der)
print("evolution trace derivative:", 1+math.sqrt(2)/2)
```

    autodiff32 derivative: 1.7071067811865475
    evolution trace derivative: 1.7071067811865475

We can see that the derivative calculated using autodiff32 is the same as the derivative calulated by walking manually through the evolution trace!

## Software Organization
### Directory Structure
Our structure is as follows:

    /cs207-FinalProject
        README.md
        LICENSE
        .gitignore
        .travis.yml
        setup.py
        requirements.txt
        docs/
            milestone1.ipynb
            milestone2.ipynb
        autodif32/
            __init__.py
            AutoDiffObj.py
            Elementary.py
            JacobianVectorFunc.py
            MultivariateVarCreator.py
        test/
            __init__.py
            autodiffobj_test.py
            elementary_test.py

### Basic Modules

The ```AutoDiffObj``` module creates an AutoDiff object based on the scalar value you would like to evaluate a function and its derivative at. It overloads the basic operations including multiply, add, negative, subtract, powers, division, and equality. It also includes a Jacobian method, which returns the Jacobian of the function. If the function is univariate, then the Jacobian is just the derivative. If the function is multivariate (not yet implemented), then the Jacobian will be an array.

The ```Elementary``` module implements some of the elementary functions, including exp, log, sqrt, sin, cos, tan, asin, acos, atan, sinh, cosh and tanh.

The ```JacobianVectorFunc``` module implements the Jacobian method for vector inputs.

The ```MultivariateVarCreator``` module takes in the values of each variable in a user defined multivariate function, and returns len(kwargs) number of AutoDiff class variables with derivative (seed) as an np.array. The MultivariateVarCreator class acts as a helper function for user to create multiple AutoDiff Objects conveniently (instead of manually create many AutoDiff objects) to use in the evaluation of the multivariate function. Please note that the implementation of multivariate functions is still in progress.

### Testing Suite
Our testing files live in the `test/` directory. The tests are run using pytest.

## Implementation details

The current implementation of AutoDiff32 allows for scalar univariate inputs to functions. Core classes include the AutoDiff class, . AutoDiff32 is externally dependent on numpy, and this dependency has been automatically taken care of in the released version of the package (as well as in the requirements.txt file if the user chooses to manually download the package). AutoDiff32 has implemented a number of elementary functions, as listed above in the description of the basic modules.

We plan to continue development of the AutoDiff32 package to allow for vector inputs as well as multivariate inputs. This will require robust Jacobian matrix functionality.

In addition to the currently implemented forward mode, we plan to build out the reverse mode of auto differentiation as an advanced feature.

DETAILS:

The core data structure (and also external dependencies) for our implementation will be numpy arrays, and the core classes we have implemented are described below:

1) An ***AutoDiff class*** which stores the current value and derivative for the current node. The class method will conatin overloading operators such as plus, minus, multiply, etc.

```python
"""
Initialize an Automatic Differentiation object which stores its current value and derivative

Note that the derivative needs to not be a scalar value

For multivariable differentiation problems of a scalar function, the derivative will be a vector
"""

def __init__(self,value,der=1) :
  self.val = val
  self.der = der

def check_dimension(self):
  """
  This method will be integrated in other class methods as well to check the dimensions when performing vector computations and catch appropriate errors
  """
  """
  This method has not been implemented yet
  """

#overloading operators to enable basic operations between classes and numbers (not exhaustive):

"""
These methods will differentiate cases in which other is a scalar, a vector, a class, or any child class

All of these operators will return AutoDiff classes
"""
def __mult__(self,other):
def __rmult__(self,other):
def __radd__(self,other):
def __add__(self,other):

# compute Jacobian for the multivariate function
def Jacobian(self):
  return self.der

#For univairate functions
X = AutoDiff(3)
func = 3*X + 2
print(func.val)
print（func.der)

```

Multivariate functions (both scalar and vector) can be also evaluated using the AutoDiff class as below, and the resulting Jacobian will be a vector array:

```python
X = AutoDiff(1,np.array([1,0]))
Y =  AutoDiff(1,np.array([0,1]))

func = X + 2*Y
```

While this way of defining and evaluating multivariate functions is feasible, it is very inconvenient for the users to have to keep track of the dimensionality of the derivatives. For,example, the above func definition will raise an error if Y's derivative is defined as np.array([0,0,1]). This potential problem in dimensionality will also be a cause difficulties in the error handling process in the code.

The way we tackle this problem is to create a helper class called **Multi_AutoDiff_Creator** as described below:

2) A ***Multi_AutoDiff_Creator class*** which helps the user create variable objects from a multivariable function

```python

"""
This class helps users initialize different variables from a multivariable function without explicitly specifying them using separate AutoDiff classes

It will need to import AutoDiff Class
"""
	def __init__(self,*args,**kwargs):
		deri = np.identity(len(kwargs))
		i = 0
		self.Vars =[]
		for key, value in kwargs.items():
			self.List.append(AutoDiff(value,deri[i]))
			i+=1

#Demo and comparison

'''initiate a multivariable function using Multi_AutoDiff_Creator class'''
X,Y= Multi_AutoDiff_Creator(X = 1., Y=3.).Vars
func = X + 2*Y*X
```

Notice that this class only serves as a helper class to ensure that every variable created has the correct format of dimensionality. The class itself has no influence on the forward mode differentation process.

For better calculation of the derivatives of our elementary functions, we also introduce our elementary function methods.

3)An ***Elementary function*** file which calculate derivatives for elementary functions as described previously
```python
    #Elementary Function Derivative (not exhaustive):
    def exp(self,ADobj):
    def tan(self,ADobj):
    def sin(self,ADobj):
    def cos(self.ADobj):
# ...

```

4) A ***Jacobian*** class which helps the user compute the Jacobian matrix for vector functions
```python
class Jacobian:
	def __init__(self,vector_func):
		self.function = vector_func
		self.l = len(self.function)

	def value(self):
		Jacob=[]
		for i in range(self.l):
			Jacob.append(self.function[i].der)
		return np.array(Jacob)

    x, y = ad.Multi_AutoDiff_Creator(x=2, y=3).Vars

    #Deifne a vector function
    func = np.array([x+y, 2*x*y])
    Jacob = ad.Jacobian(func)
    print(Jacob.value()) # this class method output the full jacobian matrix as an np array
```

5) An ***Advanced Feature Class*** which implements more advanced derivatives (such as the Hessian/Backward propagation) which inherits from the AutoDiff class.

6) A ***Test*** class which tests existing module functionalities, which we have already implemented.

**Aspects yet to be implemented:**

1) A dimensionality check method in both the AutoDiff and the Jacobian class to help check the dimensionality of users' operations

2) Error checking for all the trig functions such as sec, tanh, cosh, etc.

3)The advanced feature of back propagation

4) Other error handling issues that might occur

## Future Features

We plan to implement the reverse mode automatic differentiation algorithm for our future advanced feature. The directory structure and basic modules used will remain the same. Additional data structures might involve trees or linked lists if necessary for our implementation of the reverse mode of automatic differentation.

Since reverse mode automatic differentiation will require the storage of all relative derivatives for the final derivative propagation (we will need to keep intermediate data in the memory during the
forward pass for used in the backpropagation), the primary challenge for implementing this new feature is to understand how to store all of the derivatives (in particular what data structure we should use to store them) with respect to their related values and how to recursively calculate the final gradient.

The implementation can be a tree like structure in which, when we are overloading operators, we find a way to store the related values and derivatives as children for the previous nodes and when all the nodes (in this case autodiff objects) are traversed, we use recursion to back propagate the "tree".

Optimally, we would like to implement a very simple neural network finally using gradient descent (or back propagation algorithm) using our self implemented reverse mode with proper layers, nodes, loss and activation function.


# Super simple estimation of available solar energy
> How much energy is actually renewable and humanity's cap in consumption

- toc: true
- badges: true
- comments: true
- tags: [energy]

# Solar energy

## Stefan boltzmann's law

$ \text{Surface energy} = \sigma T^4$

For the sun, $T = \text{5,778 }K$

$\sigma = 5.67 \times 10 ^{-8} W.m^{-2}.K^{-4}$

```python
from sympy.physics.units import K, W, m, giga

sigma = 5.67 * 10**(-8) * W *m**(-2) * K**(-4)
T = 5778 * K
surface_energy = sigma * T**4
print(surface_energy)
```

    63196526.5460292*watt/meter**2

##  Total emitted solar energy

$ Radiation = \text{Surface of the sun} \times \text{Surface energy} $

$ Radiation = 4 \pi r^2 \times \text{Surface energy} $

```python
from sympy import *

r_sun = 696_340 * 1000 *m
surface_of_sun = 4 * pi * r_sun ** 2
radiation = surface_of_sun * surface_energy

print(radiation)
```

    1.22573302243694e+26*pi*watt

## Energy received at earth average distance

$ \text{Radiation received} = \frac{\text{Total sun radiation}}{ \text{sphere at earth's distance}}$

$ \text{Radiation received} = \frac{Radiation}{ 4 \pi D_{earth-sun}^2} $

```python
R_earth = 6_371 * 1000 * m
D_earth_sun = 148.88 * 10**6 * 1000 * m
earth_perp_surface = pi * R_earth **2
sphere = 4 * pi * D_earth_sun **2

radiation_received = radiation / sphere
print(radiation_received)
```

    1382.49374484614*watt/meter**2

## Energy received by the earth surface (before atmosphere)

$ \text{Energy received} = \text{radiation received} \times \frac{ \text{visible surface}}{ \text{earth's surface}} $

```python
power_received = radiation_received * pi * R_earth **2
surface_power_received = power_received / (4 * pi * R_earth **2)

print(surface_power_received)
print(power_received.n())
```

    345.623436211536*watt/meter**2
    1.76290235470883e+17*watt

> RADIATION RECEIVED BY SYSTEM EARTH  = $345 W.m^{-2}$
>
> MAXIMUM POWER WITH EARTH "DYSON SPHERE": $176 PW$

# Human consumption

13 511 MTep [Source International Energy agency](https://www.iea.org/data-and-statistics?country=WORLD&fuel=Energy%20supply&indicator=Total%20primary%20energy%20supply%20%28TPES%29%20by%20source)

```python
from sympy.physics.units import J, s, W
from sympy.physics.units.util import convert_to

million = 10 **6
kilo = 10**3
giga = 10 ** 9
toe = 41.868 * giga * J
ktoe = kilo * toe
Mtoe = million * toe

hour = 60 * 60 * s
year = 24 * h * 365.25

base = sum([3852538,2949909,670298,335519,204190,1286064,4329220])
Humanity_total_annual_consumption = base * ktoe

humanity_power_consumption = Humanity_total_annual_consumption / year
print(convert_to(humanity_power_consumption.n(), [W]).n())
```

    18080149776408.9*watt

```python
print(convert_to(humanity_power_consumption / power_received * 100, [J, s]).n())
```

    0.0102558997258785

We are currently consuming **0.01% of the maximum capacity of the earth covered by a Dyson sphere of solar panels**.

### A bit more realistic approach

After the atmosphere only $168 W.m^{-2}$ hit the surface. It's quite complicated to infer it depends on the wavelength of the incoming light, clouds, composition of the atmosphere and so on, so we just take the value from [here](https://fr.wikipedia.org/wiki/Bilan_radiatif_de_la_Terre).

Then we only have 29% of the earth surface that is landmass (where we can reasonably put solar panels in large quantity)

Of that 31% is covered in forest which are already some natural solar panels we don't  want to remove (for other obvious reasons) [source](http://www.earth-policy.org/indicators/C56/forests_2012)
And 38.4% is covered of agricultural land [source](https://en.wikipedia.org/wiki/Agricultural_land).

Then solar panels are not 100% efficient. They are roughly only 20% efficient with current technology at a reasonable cost.

```python
earth_power_received = 168 * W * m **(-2)

available_surface = 4 * pi * R_earth **2 * 0.29 * (1 -.31 - .384)

max_power = earth_power_received * available_surface * 0.2

print(max_power.n())
print(convert_to(humanity_power_consumption / max_power *100, [J, s]).n())
```

    1.52084087357243e+15*watt
    1.18882587196246

# Conclusion

In the end we are currently consuming **1.2% of the realistic available solar power energy**. That's would require posing solar panels everywhere on the planet that is not a forest or agricultural land. And we don't account yet for Energy return on energy invested (EROEI) which is likely to increase that percentage.

NB: This is a very superficial attempt to evaluate these numbers, however the result should be correct within an order of magnitude.

```python

```


### 微積分の計算について  No.2  問題と解答

##### 学籍番号[＿＿＿＿＿＿＿＿＿]クラス[＿＿＿＿＿]　クラス番号[＿＿＿＿＿] 名前[＿＿＿＿＿＿＿＿＿＿＿＿＿＿＿]

```python
from sympy import *
x, n , y = symbols('x n y')
init_printing()
```

```python
expr = x**5-2*x**3+x+3
deriv = Derivative(expr,x,1)
print( 'No.1')
deriv
```

```python
deriv.doit()
```

```python
expr = (x**2-3*x+1)*(x**2+1)
deriv = Derivative(expr,x,1)
print( 'No.2')
deriv
```

```python
deriv.doit()
```

```python
expr = 1/(x**2+1)
deriv = Derivative(expr,x,1)
print( 'No.3')
deriv
```

```python
deriv.doit()
```

```python
expr = (x**2+x+1)**4
deriv = Derivative(expr,x,1)
print( 'No.4')
deriv
```

```python
deriv.doit()
```

```python
expr = cbrt(x**2-x+1)
deriv = Derivative(expr,x,1)
print( 'No.5')
deriv
```

```python
simplify(deriv.doit())
```

```python
expr = sin(2*x+2)
deriv = Derivative(expr,x,1)
print( 'No.6')
deriv
```

```python
simplify(deriv.doit())
```

```python
expr = sin(x)*cos(2*x)
deriv = Derivative(expr,x,1)
print( 'No.7')
deriv
```

```python
deriv.doit()
```

```python
expr = 2**(3*x)
deriv = Derivative(expr,x,1)
print( 'No.8')
deriv
```

```python
deriv.doit()
```

```python
expr = exp(x)*sin(x)
deriv = Derivative(expr,x,1)
print( 'No.9')
deriv
```

```python
deriv.doit()
```

```python
expr = log(sqrt(x**2+1))
deriv = Derivative(expr,x,1)
print( 'No.10')
deriv
```

```python
deriv.doit()
```

```python
expr =sqrt((x**2-1)/(x**2+1))
deriv = Derivative(expr,x,1)
print( 'No.11')
deriv
```

```python
simplify(deriv.doit())
```

```python
expr = x**x
deriv = Derivative(expr,x,1)
print( 'No.12')
deriv
```

```python
deriv.doit()
```

```python
expr = asin(x)
deriv = Derivative(expr,x,1)
print( 'No.13')
deriv
```

$$ \frac{d}{dx}\sin^{-1}(x) $$

```python
deriv.doit()
```

```python
expr = atan(x)
deriv = Derivative(expr,x,1)
print( 'No.14')
deriv
```

$$ \frac{d}{dx}\tan^{-1}(x) $$

```python
deriv.doit()
```

```python

```


**Nguyễn Tiến Dũng**

KSTN Toán Tin - K62

*20170062*

Đại học Bách khoa Hà Nội

```python
import processviz as pvz
import sympy as sy

sy.init_printing()
```

**Câu 1:**

*a.* Các trạng thái của con chuột $I = \{sleep, eat, exercise\}$.

Phân phối xác suất tại $n=3$

```python
G1 = pvz.MarkovChain()
G1.from_file('./ass4/input_1.csv')
G1._get_state_vector(3)
G1.state_vector
```

    array([0.884, 0.044, 0.072])

*b.* Như chứng minh trong [assignment 2](https://github.com/jurgendn/processviz/blob/connect_state/Assignment2.ipynb) thì ma trận chính quy là ma trận thỏa mãn `irreducible` và `aperiodic`. Dễ thấy ma trận $P$ thỏa mãn cả 2 tính chất này.

Do đó, nếu $P$ có phân phối dừng thì phân phối đó chính là phân phối giới hạn của xích `Markov`

```python
G1.get_steady_state()
```

    array([0.8839779, 0.0441989, 0.0718232])

**Câu 2:**
*a.* $(X_n)$ là xích Markov, do việc trạng thái chuyển tiếp sang trái hoặc sang phải chỉ quyết định bởi vị trí hiện tại của nó.

*b.* Tìm ma trận chuyển trạng thái

Ma trận chuyển trạng thái được biểu diễn như sau:

```python
G2 = pvz.MarkovChain()
G2.from_file('./ass4/input_2.csv')
sy.Matrix(G2.data)
```

Ta thấy chu kì của mỗi trạng thái là 2 nên ma trận trên không là ma trận chính quy, do đó không tồn tại phân phối giới hạn.

**Câu 3:**

*a.* Ta chứng minh $(Y_n)$ là xích Markov. Ta có:

$$
P(Y_{n+1} = y_{n+1}|Y_{n} = y_n,...,Y_0 = y_0) = P[(X_{n+1}, X_{n+2}) = (i_{n+1}, i_{n+2})|(X_{n}, X_{n+1}) = (i_{n}, i_{n+1}),...,(X_0, X_1) = (i_0, i_1)]
= P[X_{n+1} = i_{n+1}, X_{n+2} = i_{n+2}|X_{n+1} = i_{n+1}, X_{n} = i_n,...,X_0 = i_0] = P[X_{n+2}= i_{n+2}|X_{n+1} = i_{n+1}]
$$

Tương tự, ta có:

$$
P(Y_{n+1} = y_{n+1}|Y_{n} = y_n) = P[(X_{n+1}, X_{n+2}) = (i_{n+1}, i_{n+2})|(X_{n}, X_{n+1}) = (i_{n}, i_{n+1})] = P[X_{n+2}= i_{n+2}|X_{n+1} = i_{n+1}]
$$

Từ 2 điều trên ta có $(Y_n)$ là xích Markov

**Câu 4:** Về ma trận chính quy thì em đã trình bày trong `câu 5` ở [assignment 2](https://github.com/jurgendn/processviz/blob/connect_state/Assignment2.ipynb) rồi nên ở đây em chỉ nêu lại cách làm chính để kiểm tra ma trận chính quy.

1. *Nếu P không tối giản thì P không phải là ma trận chính quy*
2. *Nếu P có chu kì thì P không phải là ma trận chính quy*

*P chính quy $\leftrightarrow$ `irreducible` + `aperiodic`*

---

Ví dụ, với ma trận $P = [[0,0.07,0,0,0.93],[0.93,0,0.07,0,0],[0,0.93,0,0.07,0],[0,0,0.93,0,0.07],[0.07,0,0,0.93,0]]$, ta có

```python
G41 = pvz.MarkovChain()
G41.from_file('./ass4/input_2.csv')
G41.is_regular()
```

    False

Ma trận trên không chính quy do các trạng thái có chu kì 2

---

Một ví dụ khác, với $P = [[0.9,0.05,0.05],[0.7,0,0.3],[0.8,0,0.2]]$

```python
G42 = pvz.MarkovChain()
G42.from_file('./ass4/input_1.csv')
G42.is_regular()
```

    True

**Câu 5:**

Phân phối dừng của ma trận chính quy là phân phối `Ergodic`. Do đó để tìm phân phối Ergodic, ta cần kiểm tra ma trận chính quy, rồi sau đó tìm phân phối dừng của ma trận.

Tính chính quy của ma trận có thể được tìm với method `is_regular`:

```python
G.is_regular()
```

`is_regular` gồm có:

1. Kiểm tra tính tối giản qua kiểm tra số thành phần liên thông bằng 1.
`get_connected_component()`
2. Nếu ma trận là tối giản, kiểm tra chu kì
`get_period`

Khi cả 2 điều kiện trên thỏa mãn, kết luận xích có phân phối Ergodic, và là phân phối dừng

```python
G5 = pvz.MarkovChain()
G5.from_file('./ass4/input_1.csv')
if G5.is_regular():
    print(G5.get_steady_state())
else:
    print('Invalid')
```

    [0.8839779 0.0441989 0.0718232]


```python
import sympy
from sympy import *
init_printing()
from IPython.display import display
```

# Beam Segment Equations

Loading along a beam is not always mathematically continuous.  Concentrated forces and moments, as well as starting and ending points of distributed loads interupt the mathematical continuity of the beam.  For this reason, it is convenient to segment the beam between discontinuities to form a series of mathematically continuous segments. This allows for the direct calculation of maximum/minimum forces, rotations, and deflections in each segment.

## Linear Distributed Load Equation

The equation for a linear distributed load is expressed as follows:

```python
w, w1, w2, x, L = symbols('omega, omega_1, omega_2, x, L')
w = w1+(w2-w1)/L*x
display(Eq(symbols('\\omega(x)'), w))
```

Where:

$\omega_1=$ magnitude of distributed load at starting point

$\omega_2=$ magnitude of distributed laod at ending point

$x_1=$ distributed load starting point

$x_2=$ distributed load ending point

## Shear Force Equation
The change in the shear force is equal to the area under (or integral of) the load diagram. If the shear force at the start of the segment, $V_1$, is known, the shear force at any point, $x$, relative to the start of the segment can be expressed as:

```python
V1 = symbols('V_1')
V = V1 + integrate(w, (x, 0, x))
display(Eq(symbols('V(x)'), V1 + Integral(symbols('\\omega(x)'), (x, 0, x))))
display(Eq(symbols('V(x)'), V1 + Integral(w, (x, 0, x))))
display(Eq(symbols('V(x)'), V))
print(V)
```

## Moment Equation

Similarly, the change in the moment diagram is equal to the area under the shear diagram. If the moment at the start of the segment, $M_1$, is known, the moment at any point, $x$, relative to the start of the segment can be expressed as:

```python
M1 = symbols('M_1')
M = M1 - integrate(V, (x, 0, x))
display(Eq(symbols('M(x)'), M1 - Integral(symbols('V(x)'), (x, 0, x))))
display(Eq(symbols('M(x)'), M1 - Integral(V, (x, 0, x))))
display(Eq(symbols('M(x)'), M))
print(M)
```

## Slope Equation
The slope of the elastic curve, $\theta$, can be obtained by integrating the moment diagram and dividing by the flexural rigidity, $EI$.

$\theta_1=$ slope (or rotation) at the start of the segment

```python
theta1, E, I = symbols('theta_1, E, I')
theta = theta1 - 1/(E*I)*integrate(M, (x, 0, x))
display(Eq(symbols('\\theta(x)'), theta1 + Integral(symbols('M(x)'), (x, 0, x))))
display(Eq(symbols('\\theta(x)'), theta1 + Integral(M, (x, 0, x))/(E*I)))
display(Eq(symbols('\\theta(x)'), theta))
print(theta)
```

## Deflection Equation
Integrating one more time gives us the equation for the deflection along the segment.

$\Delta_1=$ Deflection at the start of the segment

```python
Delta1 = symbols('Delta_1')
Delta = Delta1 + integrate(theta, (x, 0, x))
display(Eq(symbols('\\Delta(x)'), Delta1 + Integral(symbols('\\theta(x)'), (x, 0, x))))
display(Eq(symbols('\\Delta(x)'), Delta1 + Integral(theta, (x, 0, x))))
display(Eq(symbols('\\Delta(x)'), Delta))
print(Delta)
```

## Using the Equations
Each of the equations derived above requires knowledge of the initial conditions at the start of the segment ($V_1$, $M_1$, $\theta_1$, or $\Delta_1$).  An analysis will need to be performed to determine these values at the start of the beam.

The initial conditions at the start of the beam are the same as those at the start of the first segment.  If the initial conditions at the start of the first segment of the beam are known, the conditions at the start of the second segment can be found by calculating the conditions at the end of the first segment (from the equations with $x=L$) and adding any concentrated loads that may be at the junction between segments.  Likewise, the initial conditions for the third segment can be derived from the second segment, and so on until the last segment.  Any concentrated loads at the end of the last segment (including reactions) will need to be added to get shear and moment diagrams to close.

## Calculating $\theta_1$

For finite element solutions, the rotation $\theta_1$ at the start of the beam might be known. However, if end releases have been applied to the member (e.g. members with hinges at one or both ends), then either $\theta_1$ or $\theta_2$ or both will not be known. These terms will be lost when the stiffness matrix is condensed to simulate the end releases. In this situation the slope-deflection equation may be used to solve for $\theta_1$ as follows (since $M_1$, $M_2$, $\Delta_1$, $\Delta_2$, $FEM_1$, & $FEM_2$ will be known). The basic slope-deflection equation is:

$M_{nf}=\frac{2EI}{L}(2\theta_n+\theta_f-3\psi)+FEM_{nf}$

Substituting in appropriate values for each end of the segment yields the following equations:

```python
M1, M2, E, I, L, theta1, theta2, Delta1, Delta2 = symbols('M_1, M_2, E, I, L, theta_1, theta_2, Delta_1, Delta_2')
FEM1, FEM2 = symbols('FEM_1, FEM_2')
M1 = 2*E*I/L*(2*theta1 + theta2 - 3*(Delta2-Delta1)/L) + FEM1
M2 = 2*E*I/L*(2*theta2 + theta1 - 3*(Delta2-Delta1)/L) + FEM2
display(Eq(symbols('M_1'), M1))
display(Eq(symbols('M_2'), M2))
```

Solving the equation $M_2$ slope-deflection equation for $\theta_2$ gives:

```python
theta2 = solve(M2 - symbols('M_2'), theta2)[0]
display(Eq(symbols('\\theta_2'), expand(theta2)))
```

Substituting $\theta_2$ back into the slope-deflection equation for $M_1$ gives:

```python
M1 = M1.subs('theta_2', theta2)
display(Eq(symbols('M_1'), expand(M1)))
```

Solving this new equation for $\theta_1$ gives:

```python
theta1 = solve(M1 - symbols('M_1'), theta1)[0]
display(Eq(symbols('\\theta_1'), expand(theta1)))
print(expand(theta1))
```

## Finding Max and Min Values

The points of max/min shear can be found by setting the derivative of the equation for shear equal to zero. Ther derivative of the shear force is:

```python
display(Eq(Derivative(symbols('V'), x), diff(V, x)))
display(Eq(symbols('x'), solve(diff(V, x), symbols('x'))[0]))
print(solve(diff(V, x), symbols('x'))[0])
```

The points of max/min moment can be found by setting the derivative of the equation for moment equal to zero. The derivative of the moment is:

```python
display(Eq(Derivative(symbols('M'), x), diff(M, x)))
display(Eq(symbols('x_1'), solve(diff(M, x), symbols('x'))[0]))
display(Eq(symbols('x_2'), solve(diff(M, x), symbols('x'))[1]))
print(solve(diff(M, x), symbols('x'))[0])
print(solve(diff(M, x), symbols('x'))[1])
```

```python

```

Derivations/Beam Segment Equations (z-Axis Bending).ipynb


# 7.2 Softening function and damage function

```python
%matplotlib widget
import sympy as sp
import numpy as np
import matplotlib.pylab as plt
sp.init_printing()
```

Let us consider a softening function of the form
\begin{align}
f(w) = c_1 exp( - c_2 w )
\end{align}
This is function should describe the decay of stress starting from the material tensile strength and continuously deminishing to zero.
The variable $w$ represents the crack opening and the parameters $c_1$ and $c_2$ are the material parameters.

```python
c_1, c_2 = sp.symbols('c_1, c_2', positive=True)
w = sp.symbols(r'w')
```

```python
f = c_1 * sp.exp(-c_2*w)
```

Let us plot the function to verify its shape for the material parameters set to the value 1

```python
f_w = f.subs({c_1:1, c_2:1})
f_w
```

```python
w_arr = np.linspace(0.01,10,100)
fig.canvas.header_visible=False
fig, ax = plt.subplots(1,1,figsize=(5,3), tight_layout=True)
ax.plot(w_arr, sp.lambdify(w, f_w)(w_arr));
```

    Canvas(toolbar=Toolbar(toolitems=[('Home', 'Reset original view', 'home', 'home'), ('Back', 'Back to previous …

The function can be already used in this form. The question is however, how to set the material parameters $c_1$ and $c_2$. They can be directly associated to a particular type of material parameters - namely, to the tensile strength $f_\mathrm{t}$ and fracture energy $G_\mathrm{f}$

```python
f_t, G_f = sp.symbols('f_t, G_f', positive=True)
```

Softening starts at the level of the material strength so that we can set $f(w = 0) = f_\mathrm{t}$  to obtain the equation

```python
Eq1 = sp.Eq(f.subs({'w':0}), f_t)
Eq1
```

By solving for the $c_1$ we obtain the first substitution for our softening function.

```python
c_1_subs = sp.solve({Eq1}, c_1)
c_1_subs
```

Thus, if $w_0 = 0$, $c_1$ is equivalent to the tensile strength $f_\mathrm{t}$

The second possible mechanical interpretation is provided by the statement that softening directly represents the energy dissipation of a unit crack area. Thus, for large $w \rightarrow \infty$ it is equivalent to the energy producing a stress-free crack. This is the meaning of fracture energy.

We can thus obtain the fracture energy represented by the softening function by evaluating its integral in the range $w \in (0, \infty)$.

```python
int_f_w = sp.integrate(f.subs(c_1_subs), w)
int_f_w
```

As $c_2 > 0$, only the second term matters.
The determinate integral
\begin{align}
\left[ - \frac{f_\mathrm{t}}{c_2}
\exp(-c_2 w) \right]_0^{\infty}
\end{align}
is zero for $w = \infty$, so that the value in $w = 0$ delivers the result of the integral
\begin{align}
\frac{f_\mathrm{t}}{c_2}
\end{align}

This integral is equal to the fracture energy $G_\mathrm{f}$.

```python
E_0 = sp.symbols('E_0', positive=True )
```

```python
Eq2 = sp.Eq(-int_f_w.subs({'w':0}), G_f)
Eq2
```

and the value of $c_2$ delivers the second substitution for the softening function

```python
c_2_subs = sp.solve({Eq2}, c_2)
c_2_subs
```

The softening function with strength and fracture energy as a parameter now obtains the form

```python
f_w = f.subs(c_1_subs).subs(c_2_subs)
sp.simplify(f_w)
```

Verify that the fracture energy is recovered at $w$ in infinity

```python
sp.integrate(f_w, (w,0,sp.oo))
```

## What if there is a finite elastic stiffness

```python
E_0, w_0 = sp.symbols(r'E_b, w_0', positive=True)
```

```python
f = c_1 * sp.exp(-c_2*w)
```

```python
f_w_0 = sp.Piecewise(
    (1, w < w_0),
    (f.subs(w, w-w_0), True)
)
f_w_0
```

```python
sig_w = E_0 * w * f_w_0
sig_w
```

```python
subs_c_1 = sp.solve( {sp.Eq( sig_w.subs(w, w_0), E_b * w_0 )}, {c_1} )
subs_c_1
```

```python
G_f_ = sp.simplify( sp.integrate(sig_w.subs(subs_c_1), (w,0,sp.oo)) ).factor()
G_f_
```

```python
_, subs_c_2 = sp.solve( sp.Eq( G_f, G_f_ ), {c_2} )
sp.simplify(subs_c_2)
```

```python
g1_w_ = sp.simplify( f_w_0.subs(c_2, subs_c_2).subs(c_1,1) )
g1_w_
```

\begin{align}
g(\tilde{s}) = \displaystyle{\frac{w_0}{\tilde{w}}}
\exp\left(\frac{2 E_\mathrm{b} w_0 ( \tilde{w} - w_0) }
{E_\mathrm{b} w^2_0 - 2 G_\mathrm{f}}
 \right)
\end{align}

```python
g2_w_ = sp.Piecewise(
    (1, w < w_0),
    ( w_0 / w * sp.exp(
    (2*E_b*w_0*(w-w_0))/(E_b*w_0**2-2*G_f)
    ), True)
)
g2_w_
```

```python
get_g1_w_ = sp.lambdify( (w, w_0, E_b, G_f), g1_w_ )
get_g2_w_ = sp.lambdify( (w, w_0, E_b, G_f), g2_w_ )
```

```python
get_g1_w_(0.1, 0.1, 10, 0.1)
```

    array(1.)

```python
fig, ax = plt.subplots(1,1)
ax2 = ax.twinx()
w_range = np.linspace(0.0001,1,100)
ax.plot(w_range, get_g1_w_(w_range, 0.1, 10, 0.1), color='red')
ax.plot(w_range, get_g2_w_(w_range, 0.1, 10, 0.1), color='blue')
ax2.plot()
```

```python
#f_t = E_b * s_0
#g30_s_ =  f_t / E_b / s_equiv * sp.exp( - f_t / G_f_0 * (s_equiv - s_0) )
#g3_s_ = sp.simplify(g30_s_.subs(G_f_0, G_f - E_b*s_0**2/2))
```

## How to apply the softening function to a zone of length $L_s$?

If we wish to embed the softening behavior into a finite element simulation we encounter the problem that the deformation is not described by the crack opening $w$. The finite element discretization assumes by definition a smooth stran field $verepilon$ and there is no notion of discontinuity.

As a consequence, we need to account for a length of the softening zone  $L_s$ which is actually equivalent to the size of the finite element.

```python
L_s, epsilon = sp.symbols('L_s, varepsilon')
```

The crack opening $w$ is simply substituted by
\begin{align}
 w = \varepsilon L_\mathrm{s}
\end{align}
to obtain the softening function in terms of strains

```python
f_epsilon = f_w.subs({'w' : epsilon * L_s})
f_epsilon
```

Note the important fact that the integral of the softening function over the strains scales the dissipated energy by the term $1/L_\mathrm{s}$.

```python
sp.integrate(f_epsilon.subs(L_s,1), (epsilon,0,sp.oo))
```

As a consequence, by changing the size of the softening zone we also change the total amount of the energy dissipated!!! This feature of the softening function will be exploited in the implementation of the finite elements.

## How to convert a softening function to damage function?

The shape of the softening function describes the process of deterioration - starting from an undamaged state and ending in a fully damage state. Let us establish an equivalence between softening and damage evolution by requiring that they describe the same kind of stress decay.

Considering the softening zone of the length $L_s$ let us describe the damage within this zone using the state variable $omega$ and the elastic modulus $E_c$

```python
omega, E_c = sp.symbols('omega, E_c')
```

Then, the constitute law to be used in a finite element of the zone is given as

```python
sigma = (1 - omega) * E_c * epsilon
```

The stress decay described by the damage law and by the softening law should be the same. Let them set equal and solve for the damage variable $omega$. Using the sympy solver we obtain the algebraic solution as

```python
omega_Gf = sp.solve(sigma - f_epsilon, omega)[0]
omega_Gf
```

```python
sigma_v = (1- omega_Gf) * E_c * epsilon
sigma_v
```

This new damage function is defined using material parameters with a clear mechanical interpretation. Such kind of material law is attractive because it makes it possible to design tests that focus on an isolated phenomenon, i.e. the determination of the material strength, E modulus or fracture energy.

Let us now visually verify the shape of the damage function.
The set of parameters is assembled in a dictionary and then
they are all substituted into the damage function.

```python
data_f = dict(E_c = 28000, f_t = 3, L_s = 1, G_f = 0.01)
omega_Gf_epsilon = omega_Gf.subs(data_f)
omega_Gf_epsilon
```

The damage function is only valid in an inelastic regime. Therefore, we have to quantify the onset of inelasticity first as
\begin{align}
\varepsilon_0 = \frac{f_t}{E_c}
\end{align}

```python
epsilon_0 = (f_t / E_c).subs(data_f)
epsilon_0
```

Then the damage function can be plotted as

```python
splot(omega_Gf_epsilon, (epsilon,epsilon_0,epsilon_0*100))
```

The corresponding stress strain curve has then the form

```python
sigma = sp.Piecewise( (E_c * epsilon, epsilon < epsilon_0),
    (( 1 - omega_Gf ) * E_c * epsilon, epsilon >= epsilon_0))
sigma
```

```python
splot(sigma.subs(data_f), (epsilon,0,epsilon_0*100))
```

Note how the stress strain function scales with the change of the fracture energy and of the zone length.

Larger fracture energy makes the stress-strain response more ductile, while smaller makes it brittle.

On the other hand, larger size of the softening zone makes the softening behavior more brittle and smaller size of the zone makes it more ductile. Why?

```python
g_f = sp.integrate(sigma, epsilon)
```

```python
g_f
```

```python

```


# Lecture 30: Chi-Square, Student's t, Multivariate Normal

## Stat 110, Prof. Joe Blitzstein, Harvard University

----

## $\chi^2$ Distribution

The Chi-square Distribution is denoted as $\chi^2(n)$ or sometimes $\chi_{n}^2$, where $n$ indicates the _degrees of freedom_. It used everywhere (I think you used it before in feature analysis). It is related to Normal distribution.

Let $V = Z_1^2 + Z_2^3 + \dots + Z_n^2$, where the $Z_j$ are i.i.d. $\mathcal{N}(0,1)$. Then by definition, $V \sim \chi^2(n)$.

You will find that in a lot of things involving statistics, the sum of squares of $\mathcal{N}(0,1)$ often pops up.

```python
import matplotlib
import numpy as np
import matplotlib.pyplot as plt

from matplotlib.ticker import (MultipleLocator, FormatStrFormatter,
                               AutoMinorLocator)
from scipy.stats import chi2

%matplotlib inline

plt.xkcd()

dof_values = [1,2,3,4,5,6,7,8]
x = np.linspace(0, 10, 1000)

# plot the distributions
_, ax = plt.subplots(figsize=(12,8))

for d in dof_values:
    ax.plot(x, chi2.pdf(x, d), lw=3.2, alpha=0.6, label='df={}'.format(d))

# legend styling
legend = ax.legend()
for label in legend.get_texts():
    label.set_fontsize('large')
for label in legend.get_lines():
    label.set_linewidth(1.5)

# y-axis
ax.set_ylim([0.0, 0.5])
ax.set_ylabel(r'$f(x)$')

# x-axis
ax.set_xlim([0, 10.0])
ax.set_xlabel(r'$x$')

# x-axis tick formatting
majorLocator = MultipleLocator(2.0)
majorFormatter = FormatStrFormatter('%0.1f')
minorLocator = MultipleLocator(1.0)
ax.xaxis.set_major_locator(majorLocator)
ax.xaxis.set_major_formatter(majorFormatter)
ax.xaxis.set_minor_locator(minorLocator)

ax.grid(color='grey', linestyle='-', linewidth=0.3)

plt.suptitle(r'Examples of $\chi^2_n$ with varying degrees of freedom')

plt.show()
```

### Fact: $\chi^2(1)$ is $\operatorname{Gamma}(\frac{1}{2}, \frac{1}{2})$

#### Proof

Let $Y = Z^2$ where $Z \sim \mathcal{N}(0,1)$ and $y \gt 0$.

\begin{align}
    P(Y \le y) &= P(Z^2 \le y) \\
    &= P( -y^{\frac{1}{2}} \le y \le y^{\frac{1}{2}}) \\
    &= \Phi(y^{\frac{1}{2}}) - \Phi(y^{-\frac{1}{2}}) \\
    &= \Phi(y^{\frac{1}{2}}) - \left( 1 - \Phi(y^{\frac{1}{2}}) \right) \\
    &= 2 \, \Phi(y^{\frac{1}{2}}) - 1 \\
    \\
    \Rightarrow f_{Y}(y) &= y^{-\frac{1}{2}} \, \phi(y^{\frac{1}{2}}) \\
    &= \frac{1}{\sqrt{2\pi}} \, y^{-\frac{1}{2}} \, e^{-\frac{y}{2}} \\
    \\
    \operatorname{Gamma}\left(\frac{1}{2}, \frac{1}{2}\right) &= \frac{1}{\Gamma(\frac{1}{2})} \, \left(\frac{y}{2}\right)^{\frac{1}{2}} \, e^{-\frac{y}{2}} \, \frac{1}{y} \\
    &= \frac{1}{\sqrt{\pi}} \, \sqrt{\frac{y}{2}} \, e^{-\frac{y}{2}} \, \frac{1}{y} \\
    &= \frac{1}{\sqrt{2\pi}} \, y^{-\frac{1}{2}} \, e^{-\frac{y}{2}} &\blacksquare \\
\end{align}

Here's a quick graph to illustrate.

```python
from scipy.stats import gamma

_, (ax1, ax2) = plt.subplots(1, 2, sharey=True, figsize=(14,6))

x = np.linspace(0, 20, 1000)

ax1.plot(x, chi2.pdf(x, 1), lw=3.2, alpha=0.6, color='#33AAFF', label='df=1')
ax1.set_title('$\chi^2_1$', y=1.02)
ax1.set_xlim((0,20.0))
ax1.set_ylim((0,0.5))
ax1.legend()
ax1.grid(color='grey', linestyle='-', linewidth=0.3)

# gamma.pdf API: scale = 1 / beta
l = 0.5
ax2.plot(x, gamma.pdf(x, 0.5, scale=1/l), lw=3.2, alpha=0.6, color='#FF9933', label=r'$\alpha$=1/2, $\lambda$=1/2')
ax2.set_title(r'$Gamma(\frac{1}{2}, \frac{1}{2})$', y=1.02)
ax2.set_xlim((0,20.0))
ax2.set_ylim((0,0.5))
ax2.legend()
ax2.grid(color='grey', linestyle='-', linewidth=0.3)
None
```

### Fact:  $\chi^2(n)$ is $\operatorname{Gamma}\left( \frac{n}{2}, \frac{1}{2} \right)$

It follows then that $\chi^2(n) = \operatorname{Gamma}\left( \frac{n}{2}, \frac{1}{2} \right)$

```python
_, (ax1, ax2) = plt.subplots(1, 2, sharey=True, figsize=(14,6))

x = np.linspace(0, 20, 1000)

dof_values = [1, 2, 5, 10, 20]
col_alph_values = [0.8, 0.6, 0.5, 0.4, 0.3]

for df,c_alph in zip(dof_values, col_alph_values):
    ax1.plot(x, chi2.pdf(x, df), color='#33AAFF', lw=3.2, alpha=c_alph, label='df={}'.format(df))
ax1.set_title('$\chi^2_n$ for varying degrees of freedom', y=1.02)
ax1.set_xlim((0,20.0))
ax1.set_ylim((0,0.5))
ax1.legend()
ax1.grid(color='grey', linestyle='-', linewidth=0.3)

# gamma.pdf API: scale = 1 / lambda
l = 0.5
for alph,c_alph in zip(dof_values, col_alph_values):
    ax2.plot(x, gamma.pdf(x, alph/2, scale=1/l), lw=3.2, alpha=c_alph, color='#FF9933', label=r'$\alpha$={}/2, $\lambda$=1/2'.format(alph))
ax2.set_title(r'$Gamma(\frac{n}{2}, \frac{1}{2})$ for varying n', y=1.02)
ax2.set_xlim((0,20.0))
ax2.set_ylim((0,0.5))
ax2.legend()
ax2.grid(color='grey', linestyle='-', linewidth=0.3)
None
```

----

## Student's $t$-Distribution

The Student's $t$-distribution can be described in terms of the standard normal $Z \sim N(0,1)$ and $X^2$  $V(n)$ distributions, so that means it can be entirely described in terms of the standard normal distribution.

Let $T = \frac{Z}{\sqrt{V/n}}$, with $Z \sim \mathcal{N}(0,1)$ and $V \sim \chi^2(n)$, where $Z, V$ are independent.

Then we can write $T \sim t_n$, where $n$ is the degrees of freedom.

$t_1$ does not have a $1^{st}$ moment; $t_2$ does not have a $2^{nd}$ moment; $t_3$ does not have a $3^{rd}$ moment; and so on. Odd moments, if they exist, are 0

```python
from scipy.stats import t

dof_values   = [1,2,5,10,30,1E10]
col_alph_values = [0.2, 0.3, 0.4, 0.5, 0.6, 0.8]
x = np.linspace(-5, 5, 1000)

# plot the distributions
fig, ax = plt.subplots(figsize=(8, 6))

for df,c_alph in zip(dof_values, col_alph_values):
    if df > 30:
        dl = r'$+\infty$'
    else:
        dl = df
    ax.plot(x, t.pdf(x, df), lw=3.2, color='#A93226', alpha=c_alph, label=r'df={}'.format(dl))

# legend styling
legend = ax.legend()
for label in legend.get_texts():
    label.set_fontsize('large')
for label in legend.get_lines():
    label.set_linewidth(1.5)

# y-axis
ax.set_ylim([0, .43])
ax.set_ylabel(r'$P(x)$')

# x-axis
ax.set_xlim([-3.0, 3.0])
ax.set_xlabel(r'$x$')

ax.grid(color='grey', linestyle='-', linewidth=0.3)

plt.title(r'Examples of $t_n$ with varying degrees of freedom', y=1.02)
plt.text(x=3.5, y=0.22, s=r'Fatter tails with fewer degrees of freedom')
plt.text(x=3.5, y=0.19, s=r'Approaches $\mathbb{N}(0,1)$ as $df \rightarrow +\infty$')
plt.show()
```

### Properties

1. symmetric, i.e., $-T \sim t_n$

1. $n=1 \, \Rightarrow $ Cauchy, so $t_1$ does not have a mean

1. $n \ge 2 \Rightarrow \mathbb{E}(T) = \mathbb{E}(Z)\,\mathbb{E}\left(\frac{1}{\sqrt{V/n}}\right) = 0$

1. heavier-tailed than Normal

1. for $n$ large, $t_n$ looks very much like Normal

### Brief interlude: even moments of $Z$ and the Gamma distribution

It was proved earlier that for $Z \sim \mathcal{N}(0,1)$, the *even* moments are such that

\begin{align}
  \mathbb{E}(Z^2) &= 1 \\
  \mathbb{E}(Z^4) &= 1 \times 3 = 3 \\
  \mathbb{E}(Z^6) &= 1 \times 3 \times 5 = 15 &\quad \text{ skip factorial} \\
\end{align}

Now, this was proven using moment-generating functions, but we can also relate this to the Gamma distribution.

\begin{align}
  \mathbb{E}(Z^{2n}) &= \mathbb{E}\left( (Z^2)^n \right) \\
  &= \mathbb{E}\left( \chi^2_1 )^n \right) &\text{ but by definition } Z^2 \text{ is } \chi^2_1 \\
  &= \mathbb{E}\left( \operatorname{Gamma}(\frac{1}{2}, \frac{1}{2})^n \right)
\end{align}

... and after this point, we can use our knowledge of the Gamma distribution and LOTUS.

#### Finding $\mathbb{E}(Z^8)$ with $\operatorname{Gamma}(\frac{1}{2}, \frac{1}{2})$

\begin{align}
  \mathbb{E}(Z^8) &= \mathbb{E}\left( (Z^2)^4 \right) \\
  &= \mathbb{E}\left( \operatorname{Gamma}(\frac{1}{2}, \frac{1}{2})^4 \right) \\
  &= \frac{\Gamma(\alpha + 4)}{\Gamma(\alpha) \, \lambda^4} \\
  &= \frac{(\alpha + 3) \, \Gamma(\alpha + 3)}{\Gamma(\alpha) \, \lambda^4} \\
  &= \frac{(\alpha + 3) \, (\alpha + 2) \, \Gamma(\alpha + 2)}{\Gamma(\alpha) \, \lambda^4} \\
  &= \frac{(\alpha + 3) \, (\alpha + 2) \, (\alpha + 1) \, \Gamma(\alpha + 1)}{\Gamma(\alpha) \, \lambda^4} \\
  &= \frac{(\alpha + 3) \, (\alpha + 2) \, (\alpha + 1) \, \alpha \, \Gamma(\alpha)}{\Gamma(\alpha) \, \lambda^4} \\
  &= \frac{\frac{7}{2} \, \frac{5}{2} \, \frac{3}{2} \, \frac{1}{2}}{\frac{1}{2^4}} \\
  &= 1 \times 3 \times 5 \times 7 \\
  &= 105
\end{align}

You can [double-check this answer on WolframAlpha](http://www.wolframalpha.com/input/?i=expected+value+of+Z%5E8).

### Proof of property 5: Student's $t$ when $n$ get large

Let's prove property 5 above, but use the Law of Large Numbers (c.f. Lesson 29).

* Let $T_n = \frac{Z}{\sqrt{V/n}}$ with $Z_1, Z_2, \dots , Z_n$ i.i.d. $\mathcal{N}(0,1)$
* $V = Z_1^2 + Z_2^2 + \dots + Z_n^2$
* $Z$ is independent of the $Z_j^2$

Now we can choose any distribution for this case as long as it is $\mathcal{N}(0,1)$, and so there is nothing wrong in choosing the same distribution $Z$ for the numerator and all elements of the denominator as well.

Then $\frac{V_n}{n} \rightarrow 1$ with probability 1 by the Law of Large Numbers, since the average $\frac{V_n}{n}$ will approach the true average $Z_1^2$ as $n$ gets large. We know that $Z_1^2 = 1$.

Now, the Law of Large Numbers is with regards to point-wise estimates, so we can further state that $\sqrt{\frac{V_n}{n}} \rightarrow 1$ with probability 1.

So $T_n \rightarrow Z$ with probability 1, since the denominator goes to 1 when you have a large number of degrees of freedom; only the $Z$ in the numerator will be of importance.

\begin{align}
\end{align}

----

## Multivariate Normal

### Definition

Random vector $(X_1, X_2, \cdots , X_k) = \vec{X}$ is Multivariate Normal if every linear combination $t_1 X_1 + t_2 X_2 + \cdots + t_k X_k$ is Normal.

### An example that is multivariate Normal

Let $Z, W$ be i.i.d. $\mathcal{N}(0,1)$. Then $( Z + 2W, 3 \, Z + 5W)$ is multivariate Normal (MVN).

Given constants $s,t$

\begin{align}
  s (Z + 2 \, W) + t (3 \, Z + 5 \, W) &= (s + 3t) Z + (2s + 5t) W \\
\end{align}

But since this is $(s + 3t)$ and $(2s + 5t)$ are just scaling independent Normal random variables $Z$ and $W$ respectively; and since we know that the sum of Normal random variables is also Normal, we know that $(s + 3t) Z + (2s + 5t) W$ is also necessarily a Normal r.v.

### A non-example (NOT multivariate Normal)

Let $Z \sim \mathcal{N}(0,1)$, and let $S$ be a random sign that is independent of $Z$.

Then $Z,SZ$ are marginally $\mathcal{N}(0,1)$ (consider both individually on their own).

But $(Z, SZ)$ is _not_ multivariate normal! Just test this by considering $(Z + SZ)$.

$Z + SZ$ cannot be Normal, since:
* half the time, the sum of $Z$ and $SZ$ will be zero when $S$ is negative
* the other times, the sum will be $2Z$
* this is some mixture of discrete and continuous

## MGF of $\vec{X}$

Now with $X \sim \mathcal{N}(\mu, \sigma^2)$, the moment generating function $M(X)$ is

\begin{align}
  M(X) &= \mathbb{E}(e^{tX}) \\
  &= e^{t\mu + \frac{1}{2} \, t^2 \sigma^2} \\\\
\end{align}

Extending this one-dimensional case to the multidimensional $\vec{X}$:

\begin{align}
  M(\vec{X}) &= \mathbb{E}(e^{\vec{t} \, \vec{X}}) \\
  &= \mathbb{E}(e^{t_1 \, X_1 + t_2 \, X_2 + \cdots + t_n \, X_n}) \\
  &= \mathbb{E}(e^{t_1 \, \mu_1 + t_2 \, \mu_2 + \cdots + t_n \, \mu_n + \frac{1}{2} Var(t_1 \, X_1 + t_2 \, X_2 + \cdots + t_n \, X_n)}
\end{align}

## Theorem: Within an MVN, uncorrelated implies independent

Recall that in general, independence implies uncorrelation, but the vice versa is not always true. In the case of an MVN, however, it **is** true.

In other words, consider vector

\begin{align}
\vec{X} &= \begin{bmatrix}
              \vec{X_1} \\
              \vec{X_2} \\
            \end{bmatrix}
\end{align}

If every component of $\vec{X_1}$ is uncorrelated with every component of $\vec{X_2}$, then $\vec{X_1}$ is independent of $\vec{X_2}$.

### Example

Let $X,Y$ be i.i.d. $\mathcal{N}(0,1)$. Then $(X+Y, X-Y)$ is MVN (_bivariate Normal_ to be precise).

It is easy enough to show that $X+Y$ and $X-Y$ are uncorrelated:

\begin{align}
  \operatorname{Cov}(X+Y, X-Y) &= \operatorname{Var}(X) + \operatorname{Cov}(X,Y) - \operatorname{Cov}(X,Y) - \operatorname{Var}(Y) \\
  &= \operatorname{Var}(X) - \operatorname{Var}(Y) \\
  &= 1 - 1 \\
  &= 0
\end{align}

But can we show that $X+Y$ and $X-Y$ are _independent_?

### Proof

Let's try for something a bit more abstract.

We suppose that $X,Y$ are _independent_, zero-mean normal random variables with variances $\sigma_U, \sigma_V$.

Let $U = aX + bY$, and $V = cX + dY$ so that $U,V$ are jointly normal; this is a more general represention of the above example, where $a = 1, b=1, c=1, d=-1$.

Say we have some scalars $t_1, t_2$, and let $Z = t_{1}U + t_{2}V$. Then

\begin{align}
  M_{U,V}(t_1, t_2) &= \mathbb{E}(e^{t_1 \, U + t_2 \, V}) \\
  &= \mathbb{E}(e^{Z}) \\
  &= \mathbb{E}(e^{t_1 \, \mu_U + t_2 \, \mu_V + \frac{1}{2} Var(t_1 \, U + t_2 \, V)} \\
  &= \mathbb{E}(e^{\frac{1}{2} Var(t_1 \, U + t_2 \, V}) \\
  &= \mathbb{E}(e^{\frac{t_U^2 \, \sigma_U^2 + t_V^2 \, \sigma_V^2}{2}})
\end{align}

Now let $U\prime, V\prime$ be independent zero-mean normal random variables with the same variances $\sigma_U, \sigma_V$. Since $U\prime, V\prime$ are independent, they are also _uncorrelated_, and so the moment generating function of their bivariate normal distribution is given by $M_{U\prime,V\prime}(t_1, t_2) = \mathbb{E}(e^{\frac{t_U^2 \, \sigma_U^2 + t_V^2 \, \sigma_V^2}{2}}) $.

Since both $U,V$ and $U\prime,V\prime$ have the same moment generating function, they are boht associated with the same bivariate Normal distribution (they share the same joint PDF).

Thereforce, since $U\prime,V\prime$ are _independent_, we conclude that $U,V$ are also independent. QED.

----

View [Lecture 30: Chi-Square, Student-t, Multivariate Normal | Statistics 110](http://bit.ly/2Qfy1vJ) on YouTube.


```python
import math
import matplotlib.pyplot as plt
import numpy as np
```

# Sinus Kvadraticus

## Använda kod-biblotek
**Numpy** - utökad matematik
**Matplotlib** - grafritning

## Hur beräknas funktionsvärden t.ex sink 10° och cosk 175°?

### Sink(v)

\begin{cases}
-45^\circ \lt v \le 45^\circ \rightarrow sink(v) = tan(v) \\
45^\circ \lt v \le 135^\circ \rightarrow sink(v) = 1 \\
135^\circ \lt v \le 225^\circ \rightarrow sink(v) = -tan(v) \\
225^\circ \lt v \le 315^\circ \rightarrow sink(v) = -1
\end{cases}

```python
def sink(v):
    # Perioder
    p = 0 if v <= 2*np.pi else math.floor(v / (2*np.pi))
    v = v - 2 * np.pi * p

    if v >= 7*np.pi/4 or v < np.pi/4:
        return np.tan(v)
    elif v >= np.pi/4 and v < 3*np.pi/4:
        return 1
    elif v >= 3*np.pi/4 and v < 5*np.pi/4:
        return -np.tan(v)
    else:
        return -1
```

```python
sink(np.deg2rad(10))
```

    0.17632698070846498

### Cosk(v)

\begin{cases}
-45^\circ \lt v \le 45^\circ \rightarrow cosk(v) = 1\\
45^\circ \lt v \le 135^\circ \rightarrow cosk(v) = \frac{1}{tan(v)} \\
135^\circ \lt v \le 225^\circ \rightarrow cosk(v) = -1 \\
225^\circ \lt v \le 315^\circ \rightarrow cosk(v) = -\frac{1}{tan(v)}
\end{cases}

```python
def cosk(v):
    p = 0 if v <= 2*np.pi else math.floor(v / (2*np.pi))
    v = v - 2 * np.pi * p

    if v >= 7*np.pi/4 or v < np.pi/4:
        return 1
    elif v >= np.pi/4 and v < 3*np.pi/4:
        return 1/np.tan(v)
    elif v >= 3*np.pi/4 and v < 5*np.pi/4:
        return -1
    else:
        return -(1/np.tan(v))
```

```python
cosk(np.deg2rad(175))
```

    -1

## Hur ser graferna ut?

```python
x = np.arange(4*np.pi, step=0.001)
```

### Sink(v)

```python
y = [sink(v) for v in x]
plt.plot(x,y)
```

### Cosk(v)

```python
y = [cosk(v) for v in x]
plt.plot(x,y)
```

## Är funktionerna periodiska och isåfall vilken period?
Båda funktionerna har en period på 360 grader (eller två pi).

## Hur löser man ekvationer av typen sink v = 1 och cosk v = 0.8?

\begin{cases}
45^\circ \lt v \le 135^\circ \rightarrow sink(v) = 1 \\
225^\circ \lt v \le 315^\circ \rightarrow sink(v) = -1
\end{cases}

```python
def arcsink(l):
    if l == 1 or l == -1:
        raise BaseException("Function arcsink(l) is not defined for l == 1 || l == -1")
    else:
        return np.arctan(l)
```

\begin{cases}
-45^\circ \lt v \le 45^\circ \rightarrow cosk(v) = 1\\
135^\circ \lt v \le 225^\circ \rightarrow cosk(v) = -1 \\
\end{cases}

```python
def arccosk(l):
    if l == 1 or l == -1:
        raise BaseException("Function arccosk(l) is not defined for l == 1 || l == -1")
    else:
        return np.arctan(1 / l)
```

```python
np.rad2deg(arccosk(0.8))
```

    51.34019174590991

## Har till exempel trigonometriska ettan och formlerna för dubbla vinkeln någon motsvarighet för de nya funktionerna?

\begin{align}
sink(v)^2 + cosk(v)^2 = 1 \\
sink(2v) = 2 * sink(v) * cosk(v) \\
cosk(2v) = cosk(v)^2 - sink(v)^2 \\
sink(v) - cosk(v + 90^\circ) = 0
\end{align}

```python
x = np.arange(2*np.pi, step=0.001)
y = [cosk(v) ** 2 + sink(v) ** 2 for v in x]

plt.plot(x, y, label='cosk² v + sink² v')
plt.legend()
```

"Radien" eller "Hypotenusan" är inte konstant i enhetskvadraten: Pytagoras sats gäller inte.

```python
x = np.arange(2*np.pi, step=0.001)

y1 = [sink(2 * v) for v in x]
y2 = [2 * sink(v) * cosk(v) for v in x]

plt.plot(x, y1, label='sink 2v')
plt.plot(x, y2, label='2 * sink v * cosk v')
plt.legend()
```

```python
x = np.arange(2*np.pi, step=0.001)

y1 = [cosk(2*v) for v in x]
y2 = [cosk(v) ** 2 - sink(v) ** 2 for v in x]

plt.plot(x, y1, label='cosk 2v')
plt.plot(x, y2, label='cosk² v - sink² v')
plt.legend()
```

När sink är konstant beter sig cosk som en tan funktion och vice versa.

```python
x = np.arange(2*np.pi, step=0.001)

y = [sink(v) + cosk(v + np.pi/2) for v in x]

plt.axis([0, 6, 1, -1])
plt.plot(x, y, label='sink v + cosk(v + 0.5*π)')
plt.legend()
```

## Går det att göra ett program som ger funktions värdet då vinkeln är given och omvänt ger vinkeln då funktionsvärdet är givet?

```python
def sinkd(d):
    return arcsink(d) if d < 1 and d > -1 else sink(d)

```

```python
def coskd(d):
    return arccosk(d) if d < 1 and d > -1 else cosk(d)
```

Vinklar mellan -1 och 1 kan inte särskiljas mellan funktionsvärden


# Problemas

---
Defina una función que tome como argumentos los parametros de Denavit - Hartenberg, y cree una matriz de transformación homogénea.

```python
def DH_simbolico(a, d, α, θ):
    from sympy import Matrix, sin, cos
    # YOUR CODE HERE
    raise NotImplementedError()
```

```python
from sympy import Matrix, sin, cos, pi
from nose.tools import assert_equal
assert_equal(DH_simbolico(0,0,0,pi/2), Matrix([[0,-1,0,0],[1,0,0,0], [0,0,1,0],[0,0,0,1]]))
assert_equal(DH_simbolico(0,0,pi/2,0), Matrix([[1,0,0,0],[0,0,-1,0], [0,1,0,0],[0,0,0,1]]))
assert_equal(DH_simbolico(0,1,0,0), Matrix([[1,0,0,0],[0,1,0,0], [0,0,1,1],[0,0,0,1]]))
assert_equal(DH_simbolico(1,0,0,0), Matrix([[1,0,0,1],[0,1,0,0], [0,0,1,0],[0,0,0,1]]))
```

---
Cree una función que tome como argumentos los parametros de los grados de libertad de un manipulador tipo PUMA y devuelva las matrices de transformación homogénea asociadas a cada eslabon.

```python
def cinematica_PUMA(q1, q2, q3):
    from sympy import pi, var
    var("l1:4")
    # YOUR CODE HERE
    raise NotImplementedError()
    return A1, A2, A3
```

```python
from nose.tools import assert_equal
from sympy import pi, var, Matrix
var("l1:4")
A1, A2, A3 = cinematica_PUMA(0, 0, 0)
assert_equal(A1*A2*A3, Matrix([[1,0,0,l2+l3], [0,0,-1,0], [0,1,0,l1], [0,0,0,1]]))
A1, A2, A3 = cinematica_PUMA(pi/2, 0, 0)
assert_equal(A1*A2*A3, Matrix([[0,0,1,0], [1,0,0,l2+l3], [0,1,0,l1], [0,0,0,1]]))
A1, A2, A3 = cinematica_PUMA(0, pi/2, 0)
assert_equal(A1*A2*A3, Matrix([[0,-1,0,0], [0,0,-1,0], [1,0,0,l1+l2+l3], [0,0,0,1]]))
A1, A2, A3 = cinematica_PUMA(0, 0, pi/2)
assert_equal(A1*A2*A3, Matrix([[0,-1,0,l2], [0,0,-1,0], [1,0,0,l1+l3], [0,0,0,1]]))
```

---
Cree una función que dados los angulos del manipulador devuelva la transformación total del manipulador (ayudese de la función creada en el segundo problema).

```python
def transformacion_PUMA(q1, q2, q3):
    from sympy import pi, var
    var("l1:4")
    # YOUR CODE HERE
    raise NotImplementedError()
```

```python
from nose.tools import assert_equal
from sympy import pi, var, Matrix
var("l1:4")
assert_equal(transformacion_PUMA(0, 0, 0), Matrix([[1,0,0,l2+l3], [0,0,-1,0], [0,1,0,l1], [0,0,0,1]]))
assert_equal(transformacion_PUMA(pi/2, 0, 0), Matrix([[0,0,1,0], [1,0,0,l2+l3], [0,1,0,l1], [0,0,0,1]]))
assert_equal(transformacion_PUMA(0, pi/2, 0), Matrix([[0,-1,0,0], [0,0,-1,0], [1,0,0,l1+l2+l3], [0,0,0,1]]))
assert_equal(transformacion_PUMA(0, 0, pi/2), Matrix([[0,-1,0,l2], [0,0,-1,0], [1,0,0,l1+l3], [0,0,0,1]]))
```

---
Cree una función que dados los angulos del manipulador, grafique las posiciones de los eslabones del manipulador del primer punto (ayudese de las funciones creadas en el primer y segundo problemas, modificadas ligeramente para aceptar matrices numéricas, así como la función creada en la práctica anterior para la graficación de un sistema robótico).

```python
def DH_numerico(a, d, α, θ):
    # YOUR CODE HERE
    raise NotImplementedError()

def cinematica_PUMA(q1, q2, q3):
    # Considere que las longitudes son todas iguales a 1
    l1, l2, l3 = 1, 1, 1
    from numpy import pi
    # YOUR CODE HERE
    raise NotImplementedError()
    return A1, A2, A3

def grafica_PUMA(q1, q2, q3):
    from numpy import matrix
    # YOUR CODE HERE
    raise NotImplementedError()
    fig = figure(figsize=(8, 8))
    ax = fig.add_subplot(111, projection='3d')

    ax.plot(xs, ys, zs, "-o")

    ax.set_xlim(-1.1, 1.1)
    ax.set_ylim(-1.1, 1.1)
    ax.set_zlim(-0.1, 2.1)
    return ax
```

```python
%matplotlib inline
from matplotlib.pyplot import figure, plot, style
from mpl_toolkits.mplot3d import Axes3D
style.use("ggplot")
from numpy.testing import assert_allclose
from numpy import array
ax = grafica_PUMA(0,0.5,0.5)
ls = ax.get_lines()
assert_allclose(ls[0].get_xdata(), array([0, 0, 0.8775, 1.417885]), rtol=1e-01, atol=1e-01)
assert_allclose(ls[0].get_ydata(), array([-0.0384900179, 0, 0.00915,  0.03809]), rtol=1e-01, atol=1e-01)
```

---
Utilice la función ```interact``` para manipular la posición del manipulador, de tal manera que su posición sea aproximadamente $q_1=0.6rad$, $q_2=0.2rad$ y $q_3 = -0.8rad$

```python
%matplotlib inline
from matplotlib.pyplot import figure, plot, style
from mpl_toolkits.mplot3d import Axes3D
style.use("ggplot")
from ipywidgets import interact
from numpy import pi
τ = 2*pi
# YOUR CODE HERE
raise NotImplementedError()
```

```python
from nose.tools import assert_almost_equal
from numpy import pi
τ = 2*pi
```


## ADALINE
パーセプトロンとの違いは、重みの更新方法
* パーセプトロン：単位ステップ関数
* ADALINE：線形活性化関数 \\(\phi(z)\\)

目的関数(Objective function) ... 学習過程で最適化される関数。多くの場合は ** コスト関数 ** (cost function)

## コスト関数
誤差平方和(Sum of Squared Error：SSE)
$$ J(w) = \frac{1}{2}\sum_i(y^{(i)}-\phi(z^{(i)}))^2 $$

利点
* 微分可能である
* 凸関数であるため勾配降下法(gradient descent)を用いてコスト関数を最小化する重みを見つけ出すことができる。

### 勾配降下法を使った重み更新
コスト関数\\( J(w) \\)の勾配\\( \nabla J(w) \\)に沿って1ステップ進む
$$ w := w + \Delta w $$
重みの変化である\\( \Delta w \\)は、負の勾配に学習率\\( \eta \\)を掛けたもの
$$ \Delta w = -\eta\nabla J(w) $$

### 勾配計算(偏微分係数)
$$ \begin{align} \frac{\delta J}{\delta w_j} = \frac{\delta}{\delta w_j}\frac{1}{2}\sum_i(y^{(i)}-\phi(z^{(i)}))^2 \\
= \frac{1}{2}\frac{\delta}{\delta w_j}\sum_i(y^{(i)}-\phi(z^{(i)}))^2 \\
= \frac{1}{2}\sum_i2(y^{(i)}-\phi(z^{(i)}))\frac{\delta}{\delta w_j}(y^{(i)}-\phi(z^{(i)})) \\
= \sum_i(y^{(i)}-\phi(z^{(i)}))\frac{\delta}{\delta w_j}\Bigl( y^{(i)}-\sum_k(w_kx_k^{(i)})\Bigr) \\
= \sum_i(y^{(i)}-\phi(z^{(i)}))(-x_j^{(i)}) \\
= -\sum_i(y^{(i)}-\phi(z^{(i)}))x_j^{(i)} \\
\end{align} $$

```python
class AdalineGD(object):
    """ADAptive LInear NEuron classifier.

    Parameters
    ------------
    eta : float
        Learning rate (between 0.0 and 1.0)
    n_iter : int
        Passes over the training dataset.

    Attributes
    -----------
    w_ : 1d-array
        Weights after fitting.
    cost_ : list
        Sum-of-squares cost function value in each epoch.

    """
    def __init__(self, eta=0.01, n_iter=50):
        self.eta = eta
        self.n_iter = n_iter

    def fit(self, X, y):
        """ Fit training data.

        Parameters
        ----------
        X : {array-like}, shape = [n_samples, n_features]
            Training vectors, where n_samples is the number of samples and
            n_features is the number of features.
        y : array-like, shape = [n_samples]
            Target values.

        Returns
        -------
        self : object

        """
        self.w_ = np.zeros(1 + X.shape[1])
        self.cost_ = []

        for i in range(self.n_iter):
            net_input = self.net_input(X)
            # Please note that the "activation" method has no effect
            # in the code since it is simply an identity function. We
            # could write `output = self.net_input(X)` directly instead.
            # The purpose of the activation is more conceptual, i.e.,
            # in the case of logistic regression, we could change it to
            # a sigmoid function to implement a logistic regression classifier.
            output = self.activation(X)
            errors = (y - output)
            self.w_[1:] += self.eta * X.T.dot(errors)
            self.w_[0] += self.eta * errors.sum()
            cost = (errors**2).sum() / 2.0
            self.cost_.append(cost)
        return self

    def net_input(self, X):
        """Calculate net input"""
        return np.dot(X, self.w_[1:]) + self.w_[0]

    def activation(self, X):
        """Compute linear activation"""
        return self.net_input(X)

    def predict(self, X):
        """Return class label after unit step"""
        return np.where(self.activation(X) >= 0.0, 1, -1)
```

```python
import pandas as pd

df = pd.read_csv('https://archive.ics.uci.edu/ml/'
        'machine-learning-databases/iris/iris.data', header=None)
import matplotlib.pyplot as plt
import numpy as np

# select setosa and versicolor
y = df.iloc[0:100, 4].values
y = np.where(y == 'Iris-setosa', -1, 1)

# extract sepal length and petal length
X = df.iloc[0:100, [0, 2]].values

fig, ax = plt.subplots(nrows=1, ncols=2, figsize=(8, 4))

ada1 = AdalineGD(n_iter=10, eta=0.01).fit(X, y)
ax[0].plot(range(1, len(ada1.cost_) + 1), np.log10(ada1.cost_), marker='o')
ax[0].set_xlabel('Epochs')
ax[0].set_ylabel('log(Sum-squared-error)')
ax[0].set_title('Adaline - Learning rate 0.01')

ada2 = AdalineGD(n_iter=10, eta=0.0001).fit(X, y)
ax[1].plot(range(1, len(ada2.cost_) + 1), ada2.cost_, marker='o')
ax[1].set_xlabel('Epochs')
ax[1].set_ylabel('Sum-squared-error')
ax[1].set_title('Adaline - Learning rate 0.0001')

plt.tight_layout()
# plt.savefig('./adaline_1.png', dpi=300)
plt.show()
```

```python
from matplotlib.colors import ListedColormap

def plot_decision_regions(X, y, classifier, resolution=0.02):

    # setup marker generator and color map
    markers = ('s', 'x', 'o', '^', 'v')
    colors = ('red', 'blue', 'lightgreen', 'gray', 'cyan')
    cmap = ListedColormap(colors[:len(np.unique(y))])

    # plot the decision surface
    x1_min, x1_max = X[:, 0].min() - 1, X[:, 0].max() + 1
    x2_min, x2_max = X[:, 1].min() - 1, X[:, 1].max() + 1
    xx1, xx2 = np.meshgrid(np.arange(x1_min, x1_max, resolution),
                           np.arange(x2_min, x2_max, resolution))
    Z = classifier.predict(np.array([xx1.ravel(), xx2.ravel()]).T)
    Z = Z.reshape(xx1.shape)
    plt.contourf(xx1, xx2, Z, alpha=0.4, cmap=cmap)
    plt.xlim(xx1.min(), xx1.max())
    plt.ylim(xx2.min(), xx2.max())

    # plot class samples
    for idx, cl in enumerate(np.unique(y)):
        plt.scatter(x=X[y == cl, 0], y=X[y == cl, 1],
                    alpha=0.8, c=cmap(idx),
                    edgecolor='black',
                    marker=markers[idx],
                    label=cl)
```

```python
# standardize features
X_std = np.copy(X)
X_std[:, 0] = (X[:, 0] - X[:, 0].mean()) / X[:, 0].std()
X_std[:, 1] = (X[:, 1] - X[:, 1].mean()) / X[:, 1].std()

ada = AdalineGD(n_iter=15, eta=0.01)
ada.fit(X_std, y)

plot_decision_regions(X_std, y, classifier=ada)
plt.title('Adaline - Gradient Descent')
plt.xlabel('sepal length [standardized]')
plt.ylabel('petal length [standardized]')
plt.legend(loc='upper left')
plt.tight_layout()
# plt.savefig('./adaline_2.png', dpi=300)
plt.show()

plt.plot(range(1, len(ada.cost_) + 1), ada.cost_, marker='o')
plt.xlabel('Epochs')
plt.ylabel('Sum-squared-error')

plt.tight_layout()
# plt.savefig('./adaline_3.png', dpi=300)
plt.show()
```

```python

```