# NRPy+'s Reference Metric Interface

## Author: Zach Etienne
### Formatting improvements courtesy Brandon Clark

### NRPy+ Source Code for this module: [reference_metric.py](../edit/reference_metric.py)

## Introduction:
### Why use a reference metric? Benefits of choosing the best coordinate system for the problem

When solving a partial differential equation on the computer, it is useful to first pick a coordinate system well-suited to the geometry of the problem. For example, if we are modeling a spherically-symmetric star, it would be hugely wasteful to model the star in 3-dimensional Cartesian coordinates ($x$,$y$,$z$). This is because in Cartesian coordinates, we would need to choose high sampling in all three Cartesian directions. If instead we chose to model the star in spherical coordinates  ($r$,$\theta$,$\phi$), so long as the star is centered at $r=0$, we would not need to model the star with more than one point in the $\theta$ and $\phi$ directions!

A similar argument holds for stars that are *nearly* spherically symmetric. Such stars may exhibit density distributions that vary slowly in $\theta$ and $\phi$ directions (e.g., isolated neutron stars or black holes). In these cases the number of points needed to sample the angular directions will still be much smaller than in the radial direction.

Thus choice of an appropriate reference metric may directly mitigate the [Curse of Dimensionality](https://en.wikipedia.org/wiki/Curse_of_dimensionality).

# Table of Contents
$$\label{toc}$$

This notebook is organized as follow

1. [Step 1](#define_ref_metric): Defining a reference metric, [`reference_metric.py`](../edit/reference_metric.py)
1. [Step 2](#define_geometric): Defining geometric quantities, **`ref_metric__hatted_quantities()`**
1. [Step 3](#prescribed_ref_metric): Prescribed reference metrics in [`reference_metric.py`](../edit/reference_metric.py)
    1. [Step 3.a](#sphericallike): Spherical-like coordinate systems
        1. [Step 3.a.i](#spherical): **`reference_metric::CoordSystem = "Spherical"`**
        1. [Step 3.a.ii](#sinhspherical): **`reference_metric::CoordSystem = "SinhSpherical"`**
        1. [Step 3.a.iii](#sinhsphericalv2): **`reference_metric::CoordSystem = "SinhSphericalv2"`**
    1. [Step 3.b](#cylindricallike): Cylindrical-like coordinate systems
        1. [Step 3.b.i](#cylindrical): **`reference_metric::CoordSystem = "Cylindrical"`**
        1. [Step 3.b.ii](#sinhcylindrical): **`reference_metric::CoordSystem = "SinhCylindrical"`**
        1. [Step 3.b.iii](#sinhcylindricalv2): **`reference_metric::CoordSystem = "SinhCylindricalv2"`**
    1. [Step 3.c](#cartesianlike): Cartesian-like coordinate systems
        1. [Step 3.c.i](#cartesian): **`reference_metric::CoordSystem = "Cartesian"`**
    1. [Step 3.d](#prolatespheroidal): Prolate spheroidal coordinates
        1. [Step 3.d.i](#symtp): **`reference_metric::CoordSystem = "SymTP"`**
        1. [Step 3.d.ii](#sinhsymtp): **`reference_metric::CoordSystem = "SinhSymTP"`**
1. [Step 4](#latex_pdf_output): Output this notebook to $\LaTeX$-formatted PDF file

# Step 1: Defining a reference metric, [`reference_metric.py`](../edit/reference_metric.py) \[Back to [top](#toc)\]
$$\label{define_ref_metric}$$

***Note that currently only orthogonal reference metrics of dimension 3 or fewer are supported. This can be extended if desired.***

NRPy+ assumes all curvilinear coordinate systems map directly from a uniform, Cartesian numerical grid with coordinates $(x,y,z)$=(`xx[0]`,`xx[1]`,`xx[2]`). Thus when defining reference metrics, all defined coordinate quantities must be in terms of the `xx[]` array. As we will see, this adds a great deal of flexibility

For example,  [**reference_metric.py**](../edit/reference_metric.py) requires that the *orthogonal coordinate scale factors* be defined. As described [here](https://en.wikipedia.org/wiki/Curvilinear_coordinates), the $i$th scale factor is the positive root of the metric element $g_{ii}$. In ordinary spherical coordinates $(r,\theta,\phi)$, with line element $ds^2 = g_{ij} dx^i dx^j = dr^2+ r^2 d \theta^2 + r^2 \sin^2\theta \ d\phi^2$, we would first define
* $r = xx_0$
* $\theta = xx_1$
* $\phi = xx_2$,

so that the scale factors are defined as
* `scalefactor_orthog[0]` = $1$
* `scalefactor_orthog[1]` = $r$
* `scalefactor_orthog[2]` = $r \sin \theta$

Here is the corresponding code:

```python
import sympy as sp               # SymPy: The Python computer algebra package upon which NRPy+ depends
import NRPy_param_funcs as par   # NRPy+: parameter interface
import reference_metric as rfm   # NRPy+: Reference metric support

r = rfm.xx[0]
th = rfm.xx[1]
ph = rfm.xx[2]

rfm.scalefactor_orthog[0] = 1
rfm.scalefactor_orthog[1] = r
rfm.scalefactor_orthog[2] = r*sp.sin(th)

# Notice that the scale factor will be given
#    in terms of the fundamental Cartesian
#    grid variables, and not {r,th,ph}:
print("r*sin(th) = "+str(rfm.scalefactor_orthog[2]))
```

    r*sin(th) = xx0*sin(xx1)

Next suppose we wish to modify our radial coordinate $r(xx_0)$ to be an exponentially increasing function, so that our numerical grid $(xx_0,xx_1,xx_2)$ will map to a spherical grid with radial grid spacing ($\Delta r$) that *increases* with $r$. Generally we will find it useful to define $r(xx_0)$ to be an odd function, so let's choose

$$r(xx_0) = a \sinh(xx_0/s),$$

where $a$ is an overall radial scaling factor, and $s$ denotes the scale (in units of $xx_0$) over which exponential growth will take place. In our implementation below, note that we use the relation

$$\sinh(x) = \frac{e^x - e^{-x}}{2},$$

as SymPy finds it easier to evaluate exponentials than hyperbolic trigonometric functions.

```python
a,s = sp.symbols('a s',positive=True)
xx0_rescaled = rfm.xx[0] / s
r = a*(sp.exp(xx0_rescaled) - sp.exp(-xx0_rescaled))/2

# Must redefine the scalefactors since 'r' has been updated!
rfm.scalefactor_orthog[0] = 1
rfm.scalefactor_orthog[1] = r
rfm.scalefactor_orthog[2] = r*sp.sin(th)

print(rfm.scalefactor_orthog[2])
```

    a*(exp(xx0/s) - exp(-xx0/s))*sin(xx1)/2

Often we will find it useful to also define the appropriate mappings from (`xx[0]`,`xx[1]`,`xx[2]`) to Cartesian coordinates (for plotting purposes) and ordinary spherical coordinates (e.g., in case initial data when solving a PDE are naturally written in spherical coordinates). For this purpose, reference_metric.py also declares lists **`xxCart[]`** and **`xxSph[]`**, which in this case are defined as

```python
rfm.xxSph[0] = r
rfm.xxSph[1] = th
rfm.xxSph[2] = ph

rfm.xxCart[0] = r*sp.sin(th)*sp.cos(ph)
rfm.xxCart[1] = r*sp.sin(th)*sp.sin(ph)
rfm.xxCart[2] = r*sp.cos(th)

# Here we show off SymPy's pretty_print()
#   and simplify() functions. Nice, no?
sp.pretty_print(sp.simplify(rfm.xxCart[0]))
```

                            ⎛xx₀⎞
    a⋅sin(xx₁)⋅cos(xx₂)⋅sinh⎜───⎟
                            ⎝ s ⎠

# Step 2: Define geometric quantities, `ref_metric__hatted_quantities()` \[Back to [top](#toc)\]
$$\label{define_geometric}$$

Once `scalefactor_orthog[]` has been defined, the function **`ref_metric__hatted_quantities()`** within [reference_metric.py](../edit/reference_metric.py) can be called to define a number of geometric quantities useful for solving PDEs in curvilinear coordinate systems.

Adopting the notation of [Baumgarte, Montero, Cordero-Carrión, and Müller, PRD 87, 044026 (2012)](https://arxiv.org/abs/1211.6632), geometric quantities related to the reference metric are named "hatted" quantities, . For example, the reference metric is defined as $\hat{g}_{ij}$=`ghatDD[i][j]`:

```python
rfm.ref_metric__hatted_quantities()

sp.pretty_print(sp.Matrix(rfm.ghatDD))
```

    ⎡1           0                         0              ⎤
    ⎢                                                     ⎥
    ⎢                     2                               ⎥
    ⎢      ⎛ xx₀    -xx₀ ⎞                                ⎥
    ⎢      ⎜ ───    ─────⎟                                ⎥
    ⎢    2 ⎜  s       s  ⎟                                ⎥
    ⎢   a ⋅⎝ℯ    - ℯ     ⎠                                ⎥
    ⎢0  ───────────────────                0              ⎥
    ⎢            4                                        ⎥
    ⎢                                                     ⎥
    ⎢                                          2          ⎥
    ⎢                           ⎛ xx₀    -xx₀ ⎞           ⎥
    ⎢                           ⎜ ───    ─────⎟           ⎥
    ⎢                         2 ⎜  s       s  ⎟     2     ⎥
    ⎢                        a ⋅⎝ℯ    - ℯ     ⎠ ⋅sin (xx₁)⎥
    ⎢0           0           ─────────────────────────────⎥
    ⎣                                      4              ⎦

In addition to $\hat{g}_{ij}$, **`ref_metric__hatted_quantities()`** also provides:
* The rescaling "matrix" `ReDD[i][j]`, used for separating singular (due to chosen coordinate system) pieces of smooth rank-2 tensor components from the smooth parts, so that the smooth parts can be used within temporal and spatial differential operators.
* Inverse reference metric: $\hat{g}^{ij}$=`ghatUU[i][j]`.
* Reference metric determinant: $\det\left(\hat{g}_{ij}\right)$=`detgammahat`.
* First and second derivatives of the reference metric: $\hat{g}_{ij,k}$=`ghatDD_dD[i][j][k]`; $\hat{g}_{ij,kl}$=`ghatDD_dDD[i][j][k][l]`
* Christoffel symbols associated with the reference metric, $\hat{\Gamma}^i_{jk}$ = `GammahatUDD[i][j][k]` and their first derivatives $\hat{\Gamma}^i_{jk,l}$ = `GammahatUDD_dD[i][j][k][l]`

For example, the Christoffel symbol $\hat{\Gamma}^{xx_1}_{xx_2 xx_2}=\hat{\Gamma}^1_{22}$ is given by `GammahatUDD[1][2][2]`:

```python
sp.pretty_print(sp.simplify(rfm.GammahatUDD[1][2][2]))
```

    -sin(2⋅xx₁)
    ────────────
         2

Given the trigonometric identity $2\sin(x)\cos(x) = \sin(2x)$, notice that the above expression is equivalent to Eq. 18 of [Baumgarte, Montero, Cordero-Carrión, and Müller, PRD 87, 044026 (2012)](https://arxiv.org/abs/1211.6632). This is expected since the sinh-radial spherical coordinate system is equivalent to ordinary spherical coordinates in the angular components.

# Step 3: Prescribed reference metrics in [`reference_metric.py`](../edit/reference_metric.py) \[Back to [top](#toc)\]
$$\label{prescribed_ref_metric}$$

One need not manually define scale factors or other quantities for reference metrics, as a number of prescribed reference metrics are already defined in [reference_metric.py](../edit/reference_metric.py). These can be accessed by first setting the parameter **reference_metric::CoordSystem** to one of the following, and then calling the function **`rfm.reference_metric()`**.

```python
import indexedexp as ixp    # NRPy+: Symbolic indexed expression (e.g., tensors, vectors, etc.) support
import grid as gri          # NRPy+: Functions having to do with numerical grids

# Step 0a: Initialize parameters
thismodule = __name__
par.initialize_param(par.glb_param("char", thismodule, "CoordSystem", "Spherical"))

# Step 0b: Declare global variables
xx = gri.xx
xxCart = ixp.zerorank1(DIM=4) # Must be set in terms of xx[]s
Cart_to_xx = ixp.zerorank1(DIM=4) # Must be set in terms of xx[]s
Cartx,Carty,Cartz = sp.symbols("Cartx Carty Cartz", real=True)
Cart = [Cartx,Carty,Cartz]
xxSph  = ixp.zerorank1(DIM=4) # Must be set in terms of xx[]s
scalefactor_orthog = ixp.zerorank1(DIM=4) # Must be set in terms of xx[]s
have_already_called_reference_metric_function = False

CoordSystem = par.parval_from_str("reference_metric::CoordSystem")
M_PI,M_SQRT1_2 = par.Cparameters("#define",thismodule,["M_PI","M_SQRT1_2"],"")

global xxmin
global xxmax

global UnitVectors
UnitVectors = ixp.zerorank2(DIM=3)
```

We will find the following plotting function useful for analyzing coordinate systems in which the radial coordinate is rescaled.

```python
def create_r_of_xx0_plots(CoordSystem, r_of_xx0,rprime_of_xx0):
    import matplotlib.pyplot as plt  # matplotlib: Python module specializing in plotting capabilities
    plt.clf()
    Nr = 20
    dxx0 = 1.0 / float(Nr)
    xx0s    = []
    rs      = []
    deltars = []
    rprimes = []
    for i in range(Nr):
        xx0 = (float(i) + 0.5)*dxx0
        xx0s.append(xx0)
        rs.append(     sp.sympify(str(r_of_xx0     ).replace("xx0",str(xx0))))
        rprimes.append(sp.sympify(str(rprime_of_xx0).replace("xx0",str(xx0))))
        if i>0:
            deltars.append(sp.log(rs[i]-rs[i-1],10))
        else:
            deltars.append(sp.log(2*rs[0],10))

    # fig, ax = plt.subplots()
    fig = plt.figure(figsize=(12,12)) # 8 in x 8 in

    ax = fig.add_subplot(221)
    ax.set_title('$r(xx_0)$ for '+CoordSystem,fontsize='x-large')
    ax.set_xlabel('$xx_0$',fontsize='x-large')
    ax.set_ylabel('$r(xx_0)$',fontsize='x-large')
    ax.plot(xx0s, rs, 'k.', label='Spacing between\nadjacent gridpoints')
    # legend = ax.legend(loc='lower right', shadow=True, fontsize='x-large')
    # legend.get_frame().set_facecolor('C1')

    ax = fig.add_subplot(222)
    ax.set_title('Grid spacing for '+CoordSystem,fontsize='x-large')
    ax.set_xlabel('$xx_0$',fontsize='x-large')
    ax.set_ylabel('$\log_{10}(\Delta r)$',fontsize='x-large')
    ax.plot(xx0s, deltars, 'k.', label='Spacing between\nadjacent gridpoints\nin $r(xx_0)$ plot')
    legend = ax.legend(loc='lower right', shadow=True, fontsize='x-large')
    legend.get_frame().set_facecolor('C1')

    ax = fig.add_subplot(223)
    ax.set_title('$r\'(xx_0)$ for '+CoordSystem,fontsize='x-large')
    ax.set_xlabel('$xx_0$',fontsize='x-large')
    ax.set_ylabel('$r\'(xx_0)$',fontsize='x-large')
    ax.plot(xx0s, rprimes, 'k.', label='Nr=96')
    # legend = ax.legend(loc='upper left', shadow=True, fontsize='x-large')
    # legend.get_frame().set_facecolor('C1')

    plt.tight_layout(pad=2)
    plt.show()
```

## Step 3.a: Spherical-like coordinate systems \[Back to [top](#toc)\]
$$\label{sphericallike}$$

### Step 3.a.i: **`reference_metric::CoordSystem = "Spherical"`** \[Back to [top](#toc)\]
$$\label{spherical}$$

Standard spherical coordinates, with $(r,\theta,\phi)=(xx_0,xx_1,xx_2)$

```python
if CoordSystem == "Spherical":
    # Adding assumption real=True can help simplify expressions involving xx[0] & xx[1] below.
    xx[0] = sp.symbols("xx0", real=True)
    xx[1] = sp.symbols("xx1", real=True)

    RMAX = par.Cparameters("REAL", thismodule, ["RMAX"],10.0)
    xxmin = [sp.sympify(0), sp.sympify(0), -M_PI]
    xxmax = [         RMAX,          M_PI,  M_PI]

    r  = xx[0]
    th = xx[1]
    ph = xx[2]

    Cart_to_xx[0] = sp.sqrt(Cartx ** 2 + Carty ** 2 + Cartz ** 2)
    Cart_to_xx[1] = sp.acos(Cartz / Cart_to_xx[0])
    Cart_to_xx[2] = sp.atan2(Carty, Cartx)

    xxSph[0] = r
    xxSph[1] = th
    xxSph[2] = ph

    # Now define xCart, yCart, and zCart in terms of x0,xx[1],xx[2].
    # Note that the relation between r and x0 is not necessarily trivial in SinhSpherical coordinates. See above.
    xxCart[0] = xxSph[0]*sp.sin(xxSph[1])*sp.cos(xxSph[2])
    xxCart[1] = xxSph[0]*sp.sin(xxSph[1])*sp.sin(xxSph[2])
    xxCart[2] = xxSph[0]*sp.cos(xxSph[1])

    scalefactor_orthog[0] = sp.diff(xxSph[0],xx[0])
    scalefactor_orthog[1] = xxSph[0]
    scalefactor_orthog[2] = xxSph[0]*sp.sin(xxSph[1])

    # Set the unit vectors
    UnitVectors = [[ sp.sin(xxSph[1])*sp.cos(xxSph[2]), sp.sin(xxSph[1])*sp.sin(xxSph[2]),  sp.cos(xxSph[1])],
                   [ sp.cos(xxSph[1])*sp.cos(xxSph[2]), sp.cos(xxSph[1])*sp.sin(xxSph[2]), -sp.sin(xxSph[1])],
                   [                 -sp.sin(xxSph[2]),                  sp.cos(xxSph[2]),  sp.sympify(0)   ]]
```

Now let's analyze $r(xx_0)$ for **"Spherical"** coordinates.

```python
%matplotlib inline

CoordSystem = "Spherical"
par.set_parval_from_str("reference_metric::CoordSystem",CoordSystem)
rfm.reference_metric()

RMAX     = 10.0
r_of_xx0      = sp.sympify(str(rfm.xxSph[0]                   ).replace("RMAX",str(RMAX)))
rprime_of_xx0 = sp.sympify(str(sp.diff(rfm.xxSph[0],rfm.xx[0])).replace("RMAX",str(RMAX)))

create_r_of_xx0_plots(CoordSystem, r_of_xx0,rprime_of_xx0)
```

### Step 3.a.ii: **`reference_metric::CoordSystem = "SinhSpherical"`** \[Back to [top](#toc)\]
$$\label{sinhspherical}$$

Spherical coordinates, but with $$r(xx_0) = \text{AMPL} \frac{\sinh\left(\frac{xx_0}{\text{SINHW}}\right)}{\sinh\left(\frac{1}{\text{SINHW}}\right)}.$$

SinhSpherical uses two parameters: `AMPL` and `SINHW`. `AMPL` sets the outer boundary distance; and `SINHW` sets the focusing of the coordinate points near $r=0$, where a small `SINHW` ($\sim 0.125$) will greatly focus the points near $r=0$ and a large `SINHW` will look more like an ordinary spherical polar coordinate system.

```python
if CoordSystem == "SinhSpherical":
    xxmin = [sp.sympify(0), sp.sympify(0), -M_PI]
    xxmax = [sp.sympify(1),          M_PI,  M_PI]

    AMPL, SINHW = par.Cparameters("REAL",thismodule,["AMPL","SINHW"],[10.0,0.2])
    # Set SinhSpherical radial coordinate by default; overwrite later if CoordSystem == "SinhSphericalv2".
    r = AMPL * (sp.exp(xx[0] / SINHW) - sp.exp(-xx[0] / SINHW)) / \
               (sp.exp(1 / SINHW) - sp.exp(-1 / SINHW))
    th = xx[1]
    ph = xx[2]

    Cart_to_xx[0] = SINHW*sp.asinh(sp.sqrt(Cartx ** 2 + Carty ** 2 + Cartz ** 2)*sp.sinh(1/SINHW)/AMPL)
    Cart_to_xx[1] = sp.acos(Cartz / sp.sqrt(Cartx ** 2 + Carty ** 2 + Cartz ** 2))
    Cart_to_xx[2] = sp.atan2(Carty, Cartx)

    xxSph[0] = r
    xxSph[1] = th
    xxSph[2] = ph

    # Now define xCart, yCart, and zCart in terms of x0,xx[1],xx[2].
    # Note that the relation between r and x0 is not necessarily trivial in SinhSpherical coordinates. See above.
    xxCart[0] = xxSph[0]*sp.sin(xxSph[1])*sp.cos(xxSph[2])
    xxCart[1] = xxSph[0]*sp.sin(xxSph[1])*sp.sin(xxSph[2])
    xxCart[2] = xxSph[0]*sp.cos(xxSph[1])

    scalefactor_orthog[0] = sp.diff(xxSph[0],xx[0])
    scalefactor_orthog[1] = xxSph[0]
    scalefactor_orthog[2] = xxSph[0]*sp.sin(xxSph[1])

    # Set the unit vectors
    UnitVectors = [[ sp.sin(xxSph[1])*sp.cos(xxSph[2]), sp.sin(xxSph[1])*sp.sin(xxSph[2]),  sp.cos(xxSph[1])],
                   [ sp.cos(xxSph[1])*sp.cos(xxSph[2]), sp.cos(xxSph[1])*sp.sin(xxSph[2]), -sp.sin(xxSph[1])],
                   [                 -sp.sin(xxSph[2]),                  sp.cos(xxSph[2]),  sp.sympify(0)   ]]
```

Now we explore $r(xx_0)$ for `SinhSpherical` assuming `AMPL=10.0` and `SINHW=0.2`:

```python
%matplotlib inline

CoordSystem = "SinhSpherical"
par.set_parval_from_str("reference_metric::CoordSystem",CoordSystem)
rfm.reference_metric()

AMPL     = 10.0
SINHW    = 0.2
r_of_xx0      = sp.sympify(str(rfm.xxSph[0]                   ).replace("AMPL",str(AMPL)).replace("SINHW",str(SINHW)))
rprime_of_xx0 = sp.sympify(str(sp.diff(rfm.xxSph[0],rfm.xx[0])).replace("AMPL",str(AMPL)).replace("SINHW",str(SINHW)))

create_r_of_xx0_plots(CoordSystem, r_of_xx0,rprime_of_xx0)
```

### Step 3.a.iii: **`reference_metric::CoordSystem = "SinhSphericalv2"`** \[Back to [top](#toc)\]
$$\label{sinhsphericalv2}$$

The same as SinhSpherical coordinates, but with an additional `AMPL*const_dr*xx_0` term:
$$r(xx_0) = \text{AMPL} \left[\text{const_dr}\ xx_0 + \frac{\sinh\left(\frac{xx_0}{\text{SINHW}}\right)}{\sinh\left(\frac{1}{\text{SINHW}}\right)}\right].$$

```python
if CoordSystem == "SinhSphericalv2":
    # SinhSphericalv2 adds the parameter "const_dr", which allows for a region near xx[0]=0 to have
    # constant radial resolution of const_dr, provided the sinh() term does not dominate near xx[0]=0.
    xxmin = [sp.sympify(0), sp.sympify(0), -M_PI]
    xxmax = [sp.sympify(1),          M_PI,  M_PI]

    AMPL, SINHW = par.Cparameters("REAL",thismodule,["AMPL","SINHW"],[10.0,0.2])
    const_dr = par.Cparameters("REAL",thismodule,["const_dr"],0.0625)

    r = AMPL*( const_dr*xx[0] + (sp.exp(xx[0] / SINHW) - sp.exp(-xx[0] / SINHW)) /
               (sp.exp(1 / SINHW) - sp.exp(-1 / SINHW)) )
    th = xx[1]
    ph = xx[2]

    # NO CLOSED-FORM EXPRESSION FOR RADIAL INVERSION.
    # Cart_to_xx[0] = "NewtonRaphson"
    # Cart_to_xx[1] = sp.acos(Cartz / sp.sqrt(Cartx ** 2 + Carty ** 2 + Cartz ** 2))
    # Cart_to_xx[2] = sp.atan2(Carty, Cartx)

    xxSph[0] = r
    xxSph[1] = th
    xxSph[2] = ph

    # Now define xCart, yCart, and zCart in terms of x0,xx[1],xx[2].
    # Note that the relation between r and x0 is not necessarily trivial in SinhSpherical coordinates. See above.
    xxCart[0] = xxSph[0]*sp.sin(xxSph[1])*sp.cos(xxSph[2])
    xxCart[1] = xxSph[0]*sp.sin(xxSph[1])*sp.sin(xxSph[2])
    xxCart[2] = xxSph[0]*sp.cos(xxSph[1])

    scalefactor_orthog[0] = sp.diff(xxSph[0],xx[0])
    scalefactor_orthog[1] = xxSph[0]
    scalefactor_orthog[2] = xxSph[0]*sp.sin(xxSph[1])

    # Set the unit vectors
    UnitVectors = [[ sp.sin(xxSph[1])*sp.cos(xxSph[2]), sp.sin(xxSph[1])*sp.sin(xxSph[2]),  sp.cos(xxSph[1])],
                   [ sp.cos(xxSph[1])*sp.cos(xxSph[2]), sp.cos(xxSph[1])*sp.sin(xxSph[2]), -sp.sin(xxSph[1])],
                   [                 -sp.sin(xxSph[2]),                  sp.cos(xxSph[2]),  sp.sympify(0)   ]]
```

Now we explore $r(xx_0)$ for `SinhSphericalv2` assuming `AMPL=10.0`, `SINHW=0.2`, and `const_dr=0.05`. Notice that the `const_dr` term significantly increases the grid spacing near $xx_0=0$ relative to `SinhSpherical` coordinates.

```python
%matplotlib inline

CoordSystem = "SinhSphericalv2"
par.set_parval_from_str("reference_metric::CoordSystem",CoordSystem)
rfm.reference_metric()

AMPL     = 10.0
SINHW    = 0.2
const_dr = 0.05
r_of_xx0      = sp.sympify(str(rfm.xxSph[0]                   ).replace("AMPL",str(AMPL)).replace("SINHW",str(SINHW)).replace("const_dr",str(const_dr)))
rprime_of_xx0 = sp.sympify(str(sp.diff(rfm.xxSph[0],rfm.xx[0])).replace("AMPL",str(AMPL)).replace("SINHW",str(SINHW)).replace("const_dr",str(const_dr)))

create_r_of_xx0_plots(CoordSystem, r_of_xx0,rprime_of_xx0)
```

## Step 3.b: Cylindrical-like coordinate systems \[Back to [top](#toc)\]
$$\label{cylindricallike}$$

### Step 3.b.i: **`reference_metric::CoordSystem = "Cylindrical"`** \[Back to [top](#toc)\]
$$\label{cylindrical}$$

Standard cylindrical coordinates, with $(\rho,\phi,z)=(xx_0,xx_1,xx_2)$

```python
if CoordSystem == "Cylindrical":
    # Assuming the cylindrical radial coordinate
    #   is positive makes nice simplifications of
    #   unit vectors possible.
    xx[0] = sp.symbols("xx0", real=True)

    RHOMAX,ZMIN,ZMAX = par.Cparameters("REAL",thismodule,["RHOMAX","ZMIN","ZMAX"],[10.0,-10.0,10.0])
    xxmin = [sp.sympify(0), -M_PI, ZMIN]
    xxmax = [       RHOMAX,  M_PI, ZMAX]

    RHOCYL = xx[0]
    PHICYL = xx[1]
    ZCYL   = xx[2]

    Cart_to_xx[0] = sp.sqrt(Cartx ** 2 + Carty ** 2)
    Cart_to_xx[1] = sp.atan2(Carty, Cartx)
    Cart_to_xx[2] = Cartz

    xxCart[0] = RHOCYL*sp.cos(PHICYL)
    xxCart[1] = RHOCYL*sp.sin(PHICYL)
    xxCart[2] = ZCYL

    xxSph[0] = sp.sqrt(RHOCYL**2 + ZCYL**2)
    xxSph[1] = sp.acos(ZCYL / xxSph[0])
    xxSph[2] = PHICYL

    scalefactor_orthog[0] = sp.diff(RHOCYL,xx[0])
    scalefactor_orthog[1] = RHOCYL
    scalefactor_orthog[2] = sp.diff(ZCYL,xx[2])

    # Set the unit vectors
    UnitVectors = [[ sp.cos(PHICYL), sp.sin(PHICYL), sp.sympify(0)],
                   [-sp.sin(PHICYL), sp.cos(PHICYL), sp.sympify(0)],
                   [ sp.sympify(0),  sp.sympify(0),  sp.sympify(1)]]
```

Next let's plot **"Cylindrical"** coordinates.

```python
%matplotlib inline

import numpy as np                  # NumPy: A numerical methods module for Python
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D

R = np.linspace(0, 2, 24)
h = 2
u = np.linspace(0,  2*np.pi, 24)

x = np.outer(R, np.cos(u))
y = np.outer(R, np.sin(u))
z = h * np.outer(np.ones(np.size(u)), np.ones(np.size(u)))

r = np.arange(0,2,0.25)
theta = 2*np.pi*r*0

fig = plt.figure(figsize=(12,12)) # 8 in x 8 in
fig, (ax1, ax2) = plt.subplots(1, 2)

ax1 = plt.axes(projection='polar')

ax1.set_rmax(2)

ax1.set_rgrids(r,labels=[])

thetas = np.linspace(0,360,24, endpoint=True)
ax1.set_thetagrids(thetas,labels=[])

# ax.grid(True)
ax1.grid(True,linewidth='1.0')
ax1.set_title("Top Down View")
plt.show()

ax2 = plt.axes(projection='3d', xticklabels=[], yticklabels=[], zticklabels=[])
#ax2.plot_surface(x,y,z, alpha=.75, cmap = 'viridis') # z in case of disk which is parallel to XY plane is constant and you can directly use h

x=np.linspace(-2, 2, 100)
z=np.linspace(-2, 2, 100)
Xc, Zc=np.meshgrid(x, z)
Yc = np.sqrt(4-Xc**2)

rstride = 10
cstride = 10
ax2.plot_surface(Xc, Yc, Zc, alpha=1.0, rstride=rstride, cstride=cstride, cmap = 'viridis')
ax2.plot_surface(Xc, -Yc, Zc, alpha=1.0, rstride=rstride, cstride=cstride, cmap = 'viridis')
ax2.set_title("Standard Cylindrical Grid in 3D")
ax2.grid(False)
plt.axis('off')

plt.show()
```

### Step 3.b.ii" **`reference_metric::CoordSystem = "SinhCylindrical"`** \[Back to [top](#toc)\]
$$\label{sinhcylindrical}$$

Cylindrical coordinates, but with
$$\rho(xx_0) = \text{AMPLRHO} \frac{\sinh\left(\frac{xx_0}{\text{SINHWRHO}}\right)}{\sinh\left(\frac{1}{\text{SINHWRHO}}\right)}$$
and
$$z(xx_2) = \text{AMPLZ} \frac{\sinh\left(\frac{xx_2}{\text{SINHWZ}}\right)}{\sinh\left(\frac{1}{\text{SINHWZ}}\right)}$$

```python
if CoordSystem == "SinhCylindrical":
    # Assuming the cylindrical radial coordinate
    #   is positive makes nice simplifications of
    #   unit vectors possible.
    xx[0] = sp.symbols("xx0", real=True)

    xxmin = [sp.sympify(0), -M_PI, sp.sympify(-1)]
    xxmax = [sp.sympify(1),  M_PI, sp.sympify(+1)]

    AMPLRHO, SINHWRHO, AMPLZ, SINHWZ = par.Cparameters("REAL",thismodule,
                                                       ["AMPLRHO","SINHWRHO","AMPLZ","SINHWZ"],
                                                       [     10.0,       0.2,   10.0,    0.2])

    # Set SinhCylindrical radial & z coordinates by default; overwrite later if CoordSystem == "SinhCylindricalv2".
    RHOCYL = AMPLRHO * (sp.exp(xx[0] / SINHWRHO) - sp.exp(-xx[0] / SINHWRHO)) / (sp.exp(1 / SINHWRHO) - sp.exp(-1 / SINHWRHO))
    # phi coordinate remains unchanged.
    PHICYL = xx[1]
    ZCYL   = AMPLZ   * (sp.exp(xx[2] / SINHWZ)   - sp.exp(-xx[2] / SINHWZ))   / (sp.exp(1 / SINHWZ)   - sp.exp(-1 / SINHWZ))
    Cart_to_xx[0] = SINHWRHO*sp.asinh(sp.sqrt(Cartx ** 2 + Carty ** 2)*sp.sinh(1/SINHWRHO)/AMPLRHO)
    Cart_to_xx[1] = sp.atan2(Carty, Cartx)
    Cart_to_xx[2] = SINHWZ*sp.asinh(Cartz*sp.sinh(1/SINHWZ)/AMPLZ)

    xxCart[0] = RHOCYL*sp.cos(PHICYL)
    xxCart[1] = RHOCYL*sp.sin(PHICYL)
    xxCart[2] = ZCYL

    xxSph[0] = sp.sqrt(RHOCYL**2 + ZCYL**2)
    xxSph[1] = sp.acos(ZCYL / xxSph[0])
    xxSph[2] = PHICYL

    scalefactor_orthog[0] = sp.diff(RHOCYL,xx[0])
    scalefactor_orthog[1] = RHOCYL
    scalefactor_orthog[2] = sp.diff(ZCYL,xx[2])

    # Set the unit vectors
    UnitVectors = [[ sp.cos(PHICYL), sp.sin(PHICYL), sp.sympify(0)],
                   [-sp.sin(PHICYL), sp.cos(PHICYL), sp.sympify(0)],
                   [ sp.sympify(0),  sp.sympify(0),  sp.sympify(1)]]
```

Next let's plot **"SinhCylindrical"** coordinates.

```python
fig=plt.figure()

plt.clf()

fig = plt.figure()

ax = plt.subplot(1,1,1, projection='polar')

ax.set_rmax(2)

Nr = 20
xx0s = np.linspace(0,2,Nr, endpoint=True) + 1.0/(2.0*Nr)

rs = []
AMPLRHO = 1.0
SINHW = 0.4
for i in range(Nr):
    rs.append(AMPLRHO * (np.exp(xx0s[i] / SINHW) - np.exp(-xx0s[i] / SINHW)) / \
                        (np.exp(1.0 / SINHW) - np.exp(-1.0 / SINHW)))

ax.set_rgrids(rs,labels=[])

thetas = np.linspace(0,360,25, endpoint=True)
ax.set_thetagrids(thetas,labels=[])

# ax.grid(True)
ax.grid(True,linewidth='1.0')

plt.show()

```

### Step 3.b.iii: **`reference_metric::CoordSystem = "SinhCylindricalv2"`** \[Back to [top](#toc)\]
$$\label{sinhcylindricalv2}$$

Cylindrical coordinates, but with
$$\rho(xx_0) = \text{AMPLRHO} \left[\text{const_drho}\ xx_0 + \frac{\sinh\left(\frac{xx_0}{\text{SINHWRHO}}\right)}{\sinh\left(\frac{1}{\text{SINHWRHO}}\right)}\right]$$
and
$$z(xx_2) = \text{AMPLZ} \left[\text{const_dz}\ xx_2 + \frac{\sinh\left(\frac{xx_2}{\text{SINHWZ}}\right)}{\sinh\left(\frac{1}{\text{SINHWZ}}\right)}\right]$$

```python
if CoordSystem == "SinhCylindricalv2":
    # Assuming the cylindrical radial coordinate
    #   is positive makes nice simplifications of
    #   unit vectors possible.
    xx[0] = sp.symbols("xx0", real=True)

    # SinhCylindricalv2 adds the parameters "const_drho", "const_dz", which allows for regions near xx[0]=0
    # and xx[2]=0 to have constant rho and z resolution of const_drho and const_dz, provided the sinh() terms
    # do not dominate near xx[0]=0 and xx[2]=0.
    xxmin = [sp.sympify(0), -M_PI, sp.sympify(-1)]
    xxmax = [sp.sympify(1),  M_PI, sp.sympify(+1)]
    AMPLRHO, SINHWRHO, AMPLZ, SINHWZ = par.Cparameters("REAL",thismodule,
                                                       ["AMPLRHO","SINHWRHO","AMPLZ","SINHWZ"],
                                                       [     10.0,       0.2,   10.0,    0.2])
    const_drho, const_dz = par.Cparameters("REAL",thismodule,["const_drho","const_dz"],[0.0625,0.0625])

    RHOCYL = AMPLRHO * ( const_drho*xx[0] + (sp.exp(xx[0] / SINHWRHO) - sp.exp(-xx[0] / SINHWRHO)) / (sp.exp(1 / SINHWRHO) - sp.exp(-1 / SINHWRHO)) )
    PHICYL = xx[1]
    ZCYL   = AMPLZ   * ( const_dz  *xx[2] + (sp.exp(xx[2] / SINHWZ  ) - sp.exp(-xx[2] / SINHWZ  )) / (sp.exp(1 / SINHWZ  ) - sp.exp(-1 / SINHWZ  )) )

    # NO CLOSED-FORM EXPRESSION FOR RADIAL OR Z INVERSION.
    # Cart_to_xx[0] = "NewtonRaphson"
    # Cart_to_xx[1] = sp.atan2(Carty, Cartx)
    # Cart_to_xx[2] = "NewtonRaphson"

    xxCart[0] = RHOCYL*sp.cos(PHICYL)
    xxCart[1] = RHOCYL*sp.sin(PHICYL)
    xxCart[2] = ZCYL

    xxSph[0] = sp.sqrt(RHOCYL**2 + ZCYL**2)
    xxSph[1] = sp.acos(ZCYL / xxSph[0])
    xxSph[2] = PHICYL

    scalefactor_orthog[0] = sp.diff(RHOCYL,xx[0])
    scalefactor_orthog[1] = RHOCYL
    scalefactor_orthog[2] = sp.diff(ZCYL,xx[2])

    # Set the unit vectors
    UnitVectors = [[ sp.cos(PHICYL), sp.sin(PHICYL), sp.sympify(0)],
                   [-sp.sin(PHICYL), sp.cos(PHICYL), sp.sympify(0)],
                   [ sp.sympify(0),  sp.sympify(0),  sp.sympify(1)]]
```

For example, let's set up **`SinhCylindricalv2`** coordinates and output the Christoffel symbol $\hat{\Gamma}^{xx_2}_{xx_2 xx_2}$, or more simply $\hat{\Gamma}^2_{22}$:

```python
par.set_parval_from_str("reference_metric::CoordSystem","SinhCylindricalv2")

rfm.reference_metric()

sp.pretty_print(sp.simplify(rfm.GammahatUDD[2][2][2]))
```

                             ⎛ 2⋅xx₂     ⎞    1
                             ⎜ ──────    ⎟  ──────
                             ⎜ SINHWZ    ⎟  SINHWZ
                            -⎝ℯ       - 1⎠⋅ℯ
    ────────────────────────────────────────────────────────────────────────
           ⎛                  ⎛   2       ⎞   xx₂     ⎛ 2⋅xx₂     ⎞    1   ⎞
           ⎜                  ⎜ ──────    ⎟  ──────   ⎜ ──────    ⎟  ──────⎟
           ⎜                  ⎜ SINHWZ    ⎟  SINHWZ   ⎜ SINHWZ    ⎟  SINHWZ⎟
    SINHWZ⋅⎝- SINHWZ⋅const_dz⋅⎝ℯ       - 1⎠⋅ℯ       - ⎝ℯ       + 1⎠⋅ℯ      ⎠

As we will soon see, defining these "hatted" quantities will be quite useful when expressing hyperbolic ([wave-equation](https://en.wikipedia.org/wiki/Wave_equation)-like) PDEs in non-Cartesian coordinate systems.

## Step 3.c: Cartesian-like coordinate systems \[Back to [top](#toc)\]
$$\label{cartesianlike}$$

### Step 3.c.i: **`reference_metric::CoordSystem = "Cartesian"`** \[Back to [top](#toc)\]
$$\label{cartesian}$$

Standard Cartesian coordinates, with $(x,y,z)=$ `(xx0,xx1,xx2)`

```python
if CoordSystem == "Cartesian":
    xmin, xmax, ymin, ymax, zmin, zmax = par.Cparameters("REAL",thismodule,
                                                         ["xmin","xmax","ymin","ymax","zmin","zmax"],
                                                         [ -10.0,  10.0, -10.0,  10.0, -10.0,  10.0])
    xxmin = ["xmin", "ymin", "zmin"]
    xxmax = ["xmax", "ymax", "zmax"]

    xxCart[0] = xx[0]
    xxCart[1] = xx[1]
    xxCart[2] = xx[2]

    xxSph[0] = sp.sqrt(xx[0] ** 2 + xx[1] ** 2 + xx[2] ** 2)
    xxSph[1] = sp.acos(xx[2] / xxSph[0])
    xxSph[2] = sp.atan2(xx[1], xx[0])

    Cart_to_xx[0] = Cartx
    Cart_to_xx[1] = Carty
    Cart_to_xx[2] = Cartz

    scalefactor_orthog[0] = sp.sympify(1)
    scalefactor_orthog[1] = sp.sympify(1)
    scalefactor_orthog[2] = sp.sympify(1)

    # Set the transpose of the matrix of unit vectors
    UnitVectors = [[sp.sympify(1), sp.sympify(0), sp.sympify(0)],
                   [sp.sympify(0), sp.sympify(1), sp.sympify(0)],
                   [sp.sympify(0), sp.sympify(0), sp.sympify(1)]]
```

```python
%matplotlib inline

import numpy as np               # NumPy: A numerical methods module for Python
import matplotlib.pyplot as plt  # matplotlib: Python module specializing in plotting capabilities

plt.clf()

fig = plt.figure()
ax = fig.gca()
Nx = 16
ax.set_xticks(np.arange(0, 1., 1./Nx))
ax.set_yticks(np.arange(0, 1., 1./Nx))

for tick in ax.get_xticklabels():
    tick.set_rotation(60)

# plt.scatter(x, y)
ax.set_aspect('equal')

plt.grid()

# plt.savefig("Cartgrid.png",dpi=300)
plt.show()
# plt.close(fig)
```

### Step 3.c.ii: **`reference_metric::CoordSystem = "SinhCartesian"`** \[Back to [top](#toc)\]
$$\label{cartesian}$$

In this coordinate system, all three coordinates behave like the $z$-coordinate in SinhCylindrical coordinates, i.e.

$$
\begin{align}
x(xx_0) &= \text{AMPLX} \left[\frac{\sinh\left(\frac{xx_0}{\text{SINHWZ}}\right)}{\sinh\left(\frac{1}{\text{SINHWX}}\right)}\right]\ ,\\
y(xx_1) &= \text{AMPLY} \left[\frac{\sinh\left(\frac{xx_1}{\text{SINHWZ}}\right)}{\sinh\left(\frac{1}{\text{SINHWY}}\right)}\right]\ ,\\
z(xx_2) &= \text{AMPLZ} \left[\frac{\sinh\left(\frac{xx_2}{\text{SINHWZ}}\right)}{\sinh\left(\frac{1}{\text{SINHWZ}}\right)}\right]\ .
\end{align}
$$

```python
if CoordSystem == "SinhCartesian":

    # SinhCartesian coordinates allows us to push the outer boundary of the
    # computational domain a lot further away, while keeping reasonably high
    # resolution towards the center of the computational grid.

    # Set default values for min and max (x,y,z)
    xxmin = [sp.sympify(-1), sp.sympify(-1), sp.sympify(-1)]
    xxmax = [sp.sympify(+1), sp.sympify(+1), sp.sympify(+1)]

    # Declare basic parameters of the coordinate system and their default values
    AMPLX,SINHWX,AMPLY,SINHWY,AMPLZ,SINHWZ = par.Cparameters("REAL",thismodule,
                                                            ["AMPLX","SINHWX","AMPLY","SINHWY","AMPLZ","SINHWZ"],
                                                            [   10.0,     0.2,   10.0,     0.2,   10.0,     0.2])

    # Compute (xxCart0,xxCart1,xxCart2) from (xx0,xx1,xx2)
    xxCart[0] = AMPLX*(sp.exp(xx[0]/SINHWX) - sp.exp(-xx[0]/SINHWX))/(sp.exp(1/SINHWX) - sp.exp(-1/SINHWX))
    xxCart[1] = AMPLY*(sp.exp(xx[1]/SINHWY) - sp.exp(-xx[1]/SINHWY))/(sp.exp(1/SINHWY) - sp.exp(-1/SINHWY))
    xxCart[2] = AMPLZ*(sp.exp(xx[2]/SINHWZ) - sp.exp(-xx[2]/SINHWZ))/(sp.exp(1/SINHWZ) - sp.exp(-1/SINHWZ))

    # Compute (r,th,ph) from (xxCart2,xxCart1,xxCart2)
    xxSph[0] = sp.sqrt(xxCart[0] ** 2 + xxCart[1] ** 2 + xxCart[2] ** 2)
    xxSph[1] = sp.acos(xxCart[2] / xxSph[0])
    xxSph[2] = sp.atan2(xxCart[1], xxCart[0])

    # Compute (xx0,xx1,xx2) from (Cartx,Carty,Cartz)
    Cart_to_xx[0] = SINHWX*sp.asinh(AMPLX*Cartx*(sp.exp(1/SINHWX) - sp.exp(-1/SINHWX))/2)
    Cart_to_xx[1] = SINHWY*sp.asinh(AMPLY*Carty*(sp.exp(1/SINHWY) - sp.exp(-1/SINHWY))/2)
    Cart_to_xx[2] = SINHWZ*sp.asinh(AMPLZ*Cartz*(sp.exp(1/SINHWZ) - sp.exp(-1/SINHWZ))/2)

    # Compute scale factors
    scalefactor_orthog[0] = sp.diff(xxCart[0],xx[0])
    scalefactor_orthog[1] = sp.diff(xxCart[1],xx[1])
    scalefactor_orthog[2] = sp.diff(xxCart[2],xx[2])

    # Set the transpose of the matrix of unit vectors
    UnitVectors = [[sp.sympify(1), sp.sympify(0), sp.sympify(0)],
                   [sp.sympify(0), sp.sympify(1), sp.sympify(0)],
                   [sp.sympify(0), sp.sympify(0), sp.sympify(1)]]
```

```python
%matplotlib inline

import numpy as np               # NumPy: A numerical methods module for Python
import matplotlib.pyplot as plt  # matplotlib: Python module specializing in plotting capabilities

plt.clf()

fig = plt.figure()
ax = fig.gca()

# Set plot title
ax.set_title(r"$z=0$ slice of the 3D grid")

# Set SINH parameters. Here we assume:
#
# AMPLX  = AMPLY  = SINHA
# SINHWX = SINHWY = SINHW
SINHA = 10.0
SINHW = 0.3

# Set number of points. We assume the same point
# distribution along the (x,y)-directions
Nxxs = 20
xxis = np.linspace(-1,1,Nxxs, endpoint=True)

# Compute axis ticks by evaluating x and y using SinhCartesian coordinates
axis_ticks = []
for i in range(Nxxs):
    axis_ticks.append(SINHA * (np.exp(xxis[i] / SINHW) - np.exp(-xxis[i] / SINHW)) / \
                        (np.exp(1.0 / SINHW) - np.exp(-1.0 / SINHW)))

# Set the axis ticks
ax.set_xticks(axis_ticks)
ax.set_yticks(axis_ticks)

# Set x and y labels. Initialize array with empty strings
labelsx = ["" for i in range(Nxxs)]
labelsy = ["" for i in range(Nxxs)]

# Set x_min and x_max tick label
labelsx[0] = r"-AMPLX"
labelsx[-1] = r"AMPLX"

# Set y_min and y_max tick label
labelsy[0] = r"-AMPLY"
labelsy[-1] = r"AMPLY"

# Set tick labels
ax.set_xticklabels(labelsx)
ax.set_yticklabels(labelsy)

# Rotate x labels by 60 degrees
for tick in ax.get_xticklabels():
    tick.set_rotation(60)

# Draw the x=0 and y=0 ticklabel
ax.text(0,-11,"0",ha="center",va="center")
ax.text(-11,0,"0",ha="center",va="center")

# plt.scatter(x, y)
ax.set_aspect('equal')

plt.grid()

# plt.savefig("Cartgrid.png",dpi=300)
plt.show()
# plt.close(fig)
```

## Step 3.d: [Prolate spheroidal](https://en.wikipedia.org/wiki/Prolate_spheroidal_coordinates)-like coordinate systems \[Back to [top](#toc)\]
$$\label{prolatespheroidal}$$

### Step 3.d.i: **`reference_metric::CoordSystem = "SymTP"`** \[Back to [top](#toc)\]
$$\label{symtp}$$

Symmetric TwoPuncture coordinates, with $(\rho,\phi,z)=(xx_0\sin(xx_1), xx_2, \sqrt{xx_0^2 + \text{bScale}^2}\cos(xx_1))$

```python
if CoordSystem == "SymTP":

    var1, var2= sp.symbols('var1 var2',real=True)
    bScale, AW, AMAX, RHOMAX, ZMIN, ZMAX = par.Cparameters("REAL",thismodule,
                                                           ["bScale","AW","AMAX","RHOMAX","ZMIN","ZMAX"],
                                                           [0.5,     0.2,   10.0,    10.0, -10.0,  10.0])

    # Assuming xx0, xx1, and bScale
    #   are positive makes nice simplifications of
    #   unit vectors possible.
    xx[0],xx[1] = sp.symbols("xx0 xx1", real=True)

    xxmin = [sp.sympify(0), sp.sympify(0),-M_PI]
    xxmax = [         AMAX,          M_PI, M_PI]

    AA = xx[0]

    if CoordSystem == "SinhSymTP":
        AA = (sp.exp(xx[0]/AW)-sp.exp(-xx[0]/AW))/2

    var1 = sp.sqrt(AA**2 + (bScale * sp.sin(xx[1]))**2)
    var2 = sp.sqrt(AA**2 + bScale**2)

    RHOSYMTP = AA*sp.sin(xx[1])
    PHSYMTP = xx[2]
    ZSYMTP = var2*sp.cos(xx[1])

    xxCart[0] = AA  *sp.sin(xx[1])*sp.cos(xx[2])
    xxCart[1] = AA  *sp.sin(xx[1])*sp.sin(xx[2])
    xxCart[2] = ZSYMTP

    xxSph[0] = sp.sqrt(RHOSYMTP**2 + ZSYMTP**2)
    xxSph[1] = sp.acos(ZSYMTP / xxSph[0])
    xxSph[2] = PHSYMTP

    rSph  = sp.sqrt(Cartx ** 2 + Carty ** 2 + Cartz ** 2)
    thSph = sp.acos(Cartz / rSph)
    phSph = sp.atan2(Carty, Cartx)

    # Mathematica script to compute Cart_to_xx[]
    #             AA = x1;
    #             var2 = Sqrt[AA^2 + bScale^2];
    #             RHOSYMTP = AA*Sin[x2];
    #             ZSYMTP = var2*Cos[x2];
    #             Solve[{rSph == Sqrt[RHOSYMTP^2 + ZSYMTP^2],
    #                    thSph == ArcCos[ZSYMTP/Sqrt[RHOSYMTP^2 + ZSYMTP^2]],
    #                    phSph == x3},
    #                   {x1, x2, x3}]
    Cart_to_xx[0] = sp.sqrt(-bScale**2 + rSph**2 +
                            sp.sqrt(bScale**4 + 2*bScale**2*rSph**2 + rSph**4 -
                                    4*bScale**2*rSph**2*sp.cos(thSph)**2))*M_SQRT1_2 # M_SQRT1_2 = 1/sqrt(2); define this way for UnitTesting

    # The sign() function in the following expression ensures the correct root is taken.
    Cart_to_xx[1] = sp.acos(sp.sign(Cartz)*(
                              sp.sqrt(1 + rSph**2/bScale**2 -
                                      sp.sqrt(bScale**4 + 2*bScale**2*rSph**2 + rSph**4 -
                                              4*bScale**2*rSph**2*sp.cos(thSph)**2)/bScale**2)*M_SQRT1_2)) # M_SQRT1_2 = 1/sqrt(2); define this way for UnitTesting

    Cart_to_xx[2] = phSph
```

### Step 3.d.ii: **`reference_metric::CoordSystem = "SinhSymTP"`** \[Back to [top](#toc)\]
$$\label{sinhsymtp}$$

Symmetric TwoPuncture coordinates, but with $$xx_0 \to \sinh(xx_0/\text{AW})$$

```python
if CoordSystem == "SinhSymTP":

    var1, var2= sp.symbols('var1 var2',real=True)
    bScale, AW, AMAX, RHOMAX, ZMIN, ZMAX = par.Cparameters("REAL",thismodule,
                                                           ["bScale","AW","AMAX","RHOMAX","ZMIN","ZMAX"],
                                                           [0.5,     0.2,   10.0,    10.0, -10.0,  10.0])

    # Assuming xx0, xx1, and bScale
    #   are positive makes nice simplifications of
    #   unit vectors possible.
    xx[0],xx[1] = sp.symbols("xx0 xx1", real=True)

    xxmin = [sp.sympify(0), sp.sympify(0),-M_PI]
    xxmax = [         AMAX,          M_PI, M_PI]

    AA = xx[0]

    if CoordSystem == "SinhSymTP":
        # With xxmax[0] == AMAX, sinh(xx0/AMAX) will evaluate to a number between 0 and 1.
        #   Similarly, sinh(xx0/(AMAX*SINHWAA)) / sinh(1/SINHWAA) will also evaluate to a number between 0 and 1.
        #   Then AA = AMAX*sinh(xx0/(AMAX*SINHWAA)) / sinh(1/SINHWAA) will evaluate to a number between 0 and AMAX.
        AA = AMAX * (sp.exp(xx[0] / (AMAX*SINHWAA)) - sp.exp(-xx[0] / (AMAX*SINHWAA))) / (sp.exp(1 / SINHWAA) - sp.exp(-1 / AMAX))

    var1 = sp.sqrt(AA**2 + (bScale * sp.sin(xx[1]))**2)
    var2 = sp.sqrt(AA**2 + bScale**2)

    RHOSYMTP = AA*sp.sin(xx[1])
    PHSYMTP = xx[2]
    ZSYMTP = var2*sp.cos(xx[1])

    xxCart[0] = AA  *sp.sin(xx[1])*sp.cos(xx[2])
    xxCart[1] = AA  *sp.sin(xx[1])*sp.sin(xx[2])
    xxCart[2] = ZSYMTP

    xxSph[0] = sp.sqrt(RHOSYMTP**2 + ZSYMTP**2)
    xxSph[1] = sp.acos(ZSYMTP / xxSph[0])
    xxSph[2] = PHSYMTP

    scalefactor_orthog[0] = sp.diff(AA,xx[0]) * var1 / var2
    scalefactor_orthog[1] = var1
    scalefactor_orthog[2] = AA * sp.sin(xx[1])

    # Set the transpose of the matrix of unit vectors
    UnitVectors = [[sp.sin(xx[1]) * sp.cos(xx[2]) * var2 / var1,
                    sp.sin(xx[1]) * sp.sin(xx[2]) * var2 / var1,
                    AA * sp.cos(xx[1]) / var1],
                   [AA * sp.cos(xx[1]) * sp.cos(xx[2]) / var1,
                    AA * sp.cos(xx[1]) * sp.sin(xx[2]) / var1,
                        -sp.sin(xx[1]) * var2 / var1],
                   [-sp.sin(xx[2]), sp.cos(xx[2]), sp.sympify(0)]]
```

# Step 4: Output this notebook to $\LaTeX$-formatted PDF file \[Back to [top](#toc)\]
$$\label{latex_pdf_output}$$

The following code cell converts this Jupyter notebook into a proper, clickable $\LaTeX$-formatted PDF file. After the cell is successfully run, the generated PDF may be found in the root NRPy+ tutorial directory, with filename
[Tutorial-Reference_Metric.pdf](Tutorial-Reference_Metric.pdf) (Note that clicking on this link may not work; you may need to open the PDF file through another means.)

```python
import cmdline_helper as cmd    # NRPy+: Multi-platform Python command-line interface
cmd.output_Jupyter_notebook_to_LaTeXed_PDF("Tutorial-Reference_Metric")
```

    Created Tutorial-Reference_Metric.tex, and compiled LaTeX file to PDF file
        Tutorial-Reference_Metric.pdf


# Selección óptima de portafolios II

Entonces, tenemos que:
- La LAC describe las posibles selecciones de riesgo-rendimiento entre un activo libre de riesgo y un activo riesgoso.
- Su pendiente es igual al radio de Sharpe del activo riesgoso.
- La asignación óptima de capital para cualquier inversionista es el punto tangente de la curva de indiferencia del inversionista con la LAC.

Para todo lo anterior, supusimos que ya teníamos el portafolio óptimo (activo riesgoso).

En la clase pasada aprendimos a hallar este portafolio óptimo si el conjunto de activos riesgosos estaba conformado únicamente por dos activos:

$$w_{1,EMV}=\frac{(E[r_1]-r_f)\sigma_2^2-(E[r_2]-r_f)\sigma_{12}}{(E[r_2]-r_f)\sigma_1^2+(E[r_1]-r_f)\sigma_2^2-((E[r_1]-r_f)+(E[r_2]-r_f))\sigma_{12}}.$$

- Sin embargo, la complejidad del problema crece considerablemente con el número de variables, y la solución analítica deja de ser viable cuando mencionamos que un portafolio bien diversificado consta aproximadamente de 50-60 activos.
- En esos casos, este problema se soluciona con rutinas numéricas que hagan la optimización por nosotros, porque son una solución viable y escalable a más variables.

**Objetivos:**
- ¿Cuál es el portafolio óptimo de activos riesgosos cuando tenemos más de dos activos?
 - ¿Cómo construir la frontera de mínima varianza cuando tenemos más de dos activos?

*Referencia:*
- Notas del curso "Portfolio Selection and Risk Management", Rice University, disponible en Coursera.
___

## 1. Maximizando el radio de Sharpe

### ¿Qué pasa si tenemos más de dos activos riesgosos?

En realidad es algo muy similar a lo que teníamos con dos activos.
- Para dos activos, construir la frontera de mínima varianza es trivial: todas las posibles combinaciones.
- Con más de dos activos, recordar la definición: la frontera de mínima varianza es el lugar geométrico de los portafolios que proveen el mínimo riesgo para un nivel de rendimiento dado.

 Ver en el tablero.

Analíticamente:
- $N$ activos,
- caracterizados por $(\sigma_i,E[r_i])$,
- cada uno con peso $w_i$, con $i=1,2,\dots,N$.

Entonces, buscamos los pesos tales que

\begin{align}
\min_{w_1,\dots,w_n} & \quad \sum_{i=1}^{N}w_i^2\sigma_i^2+\sum_{i=1}^{N}\sum_{j=1,j\neq i}^{N}w_iw_j\sigma_{ij}\\
\text{s.a.}          & \quad \sum_{i=1}^{N}w_i=1, w_i\geq0\\
                     & \quad \sum_{i=1}^{N}w_iE[r_i]=\bar{\mu},
\end{align}

donde $\bar{\mu}$ corresponde a un nivel de rendimiento objetivo.

**Obviamente, tendríamos que resolver este problema para muchos niveles de rendimiento objetivo.**

-  Explicar relación con gráfica.

-  Recordar clase 10.

Lo anterior se puede escribir vectorialmente como:

\begin{align}
\min_{\boldsymbol{w}} & \quad \boldsymbol{w}^T\Sigma\boldsymbol{w}\\
\text{s.a.}          & \quad \boldsymbol{1}^T\boldsymbol{w}=1, \boldsymbol{w}\geq0\\
                     & \quad E[\boldsymbol{r}^T]\boldsymbol{w}=\bar{\mu},
\end{align}

donde:
- $\boldsymbol{w}=\left[w_1,\dots,w_N\right]^T$ es el vector de pesos,
- $\boldsymbol{1}=\left[1,\dots,1\right]^T$ es un vector de unos,
- $E[\boldsymbol{r}]=\left[E[r_1],\dots,E[r_N]\right]^T$ es el vector de rendimientos esperados, y
- $\Sigma=\left[\begin{array}{cccc}\sigma_{1}^2 & \sigma_{12}  & \dots  & \sigma_{1N} \\
                                   \sigma_{21}  & \sigma_{2}^2 & \dots  & \sigma_{2N} \\
                                   \vdots       & \vdots       & \ddots & \vdots      \\
                                   \sigma_{N1}  & \sigma_{N2}  & \dots  & \sigma_{N}^2\end{array}\right]$ es la matriz de varianza-covarianza.

**Esta última forma es la que comúnmente usamos al programar, por ser eficiente y escalable a problemas de N variables.**

### Entonces, ¿para cuántos niveles de rendimiento objetivo tendríamos que resolver el anterior problema con el fin de graficar la frontera de mínima varianza?
- Observar que el problema puede volverse muy pesado a medida que incrementamos el número de activos en nuestro portafolio...
- Una tarea bastante compleja.

### Sucede que, en realidad, sólo necesitamos conocer dos portafolios que estén sobre la *frontera de mínima varianza*.
- Si logramos encontrar dos portafolios sobre la frontera, entonces podemos a la vez encontrar todas las posibles combinaciones de estos dos portafolios para trazar la frontera de mínima varianza.
- Ver el caso de dos activos.

### ¿Qué portafolios usar?
Hasta ahora, hemos estudiando profundamente como hallar dos portafolios muy importantes que de hecho yacen sobre la frontera de mínima varianza:
1. Portafolio de EMV: máximo SR.
2. Portafolio de mínima varianza: básicamente, el mismo problema anterior, sin la restricción de rendimiento objetivo.

Luego, tomar todas las posibles combinaciones de dichos portafolios usando las fórmulas para dos activos de medias y varianzas:
- w: peso para el portafolio EMV,
- 1-w: peso para le portafolio de mínima varianza.

## 2. Ejemplo ilustrativo.

Retomamos el ejemplo de mercados de acciones en los países integrantes del $G5$: EU, RU, Francia, Alemania y Japón.

```python
# Importamos pandas y numpy
import pandas as pd
import numpy as np
```

```python
# Resumen en base anual de rendimientos esperados y volatilidades
annual_ret_summ = pd.DataFrame(columns=['EU', 'RU', 'Francia', 'Alemania', 'Japon'], index=['Media', 'Volatilidad'])
annual_ret_summ.loc['Media'] = np.array([0.1355, 0.1589, 0.1519, 0.1435, 0.1497])
annual_ret_summ.loc['Volatilidad'] = np.array([0.1535, 0.2430, 0.2324, 0.2038, 0.2298])

annual_ret_summ.round(4)
```

      EU
      RU
      Francia
      Alemania
      Japon

      Media
      0.1355
      0.1589
      0.1519
      0.1435
      0.1497

      Volatilidad
      0.1535
      0.243
      0.2324
      0.2038
      0.2298

```python
# Matriz de correlación
corr = pd.DataFrame(data= np.array([[1.0000, 0.5003, 0.4398, 0.3681, 0.2663],
                                    [0.5003, 1.0000, 0.5420, 0.4265, 0.3581],
                                    [0.4398, 0.5420, 1.0000, 0.6032, 0.3923],
                                    [0.3681, 0.4265, 0.6032, 1.0000, 0.3663],
                                    [0.2663, 0.3581, 0.3923, 0.3663, 1.0000]]),
                    columns=annual_ret_summ.columns, index=annual_ret_summ.columns)
corr.round(4)
```

      EU
      RU
      Francia
      Alemania
      Japon

      EU
      1.0000
      0.5003
      0.4398
      0.3681
      0.2663

      RU
      0.5003
      1.0000
      0.5420
      0.4265
      0.3581

      Francia
      0.4398
      0.5420
      1.0000
      0.6032
      0.3923

      Alemania
      0.3681
      0.4265
      0.6032
      1.0000
      0.3663

      Japon
      0.2663
      0.3581
      0.3923
      0.3663
      1.0000

```python
# Tasa libre de riesgo
rf = 0.05
```

Esta vez, supondremos que tenemos disponibles todos los mercados de acciones y el activo libre de riesgo.

#### 1. Construir la frontera de mínima varianza

##### 1.1. Encontrar portafolio de mínima varianza

```python
# Importamos funcion minimize del modulo optimize de scipy
from scipy.optimize import minimize
```

```python
## Construcción de parámetros
# 1. Sigma: matriz de varianza-covarianza
S = np.diag(annual_ret_summ.loc['Volatilidad'])
Sigma = S.dot(corr).dot(S)
# 2. Eind: rendimientos esperados activos individuales
Eind = annual_ret_summ.loc['Media'].values
```

```python
# Función objetivo
def var(w, Sigma):
    return w.T.dot(Sigma).dot(w)
```

```python
# Dato inicial
n = len(Eind) # 5
w0 = np.ones((n,)) / n
# Cotas de las variables
bnds = ((0, 1),) * n
# Restricciones
cons = {'type': 'eq', 'fun': lambda w: w.sum() - 1}
```

```python
# Portafolio de mínima varianza
minvar = minimize(fun=var,
                  x0=w0,
                  args=(Sigma,),
                  bounds=bnds,
                  constraints=cons
                 )
minvar
```

         fun: 0.01861776391061502
         jac: array([0.03718246, 0.03881475, 0.03859101, 0.03755156, 0.0370423 ])
     message: 'Optimization terminated successfully.'
        nfev: 49
         nit: 7
        njev: 7
      status: 0
     success: True
           x: array([6.17797049e-01, 3.46944695e-18, 0.00000000e+00, 2.09394358e-01,
           1.72808594e-01])

```python
# Pesos, rendimiento y riesgo del portafolio de mínima varianza
w_minvar = minvar.x
E_minvar = Eind.dot(w_minvar)
s_minvar = (w_minvar.dot(Sigma).dot(w_minvar))**0.5
RS_minvar = (E_minvar - rf) / s_minvar
w_minvar, E_minvar, s_minvar, RS_minvar
```

    (array([6.17797049e-01, 3.46944695e-18, 0.00000000e+00, 2.09394358e-01,
            1.72808594e-01]),
     0.13962903688859557,
     0.13644692708381168,
     0.6568783834431207)

##### 1.2. Encontrar portafolio EMV

```python
# Función objetivo
def menos_RS(w, Sigma, rf, Eind):
    Ep = Eind.dot(w)
    sp = w.dot(Sigma).dot(w)**0.5
    RS = (Ep - rf) / sp
    return -RS
```

```python
# Dato inicial
n = len(Eind) # 5
w0 = np.ones((n,)) / n
# Cotas de las variables
bnds = ((0, 1),) * n
# Restricciones
cons = {'type': 'eq', 'fun': lambda w: w.sum() - 1}
```

```python
# Portafolio EMV
emv = minimize(fun=menos_RS,
               x0=w0,
               args=(Sigma, rf, Eind),
               bounds=bnds,
               constraints=cons
              )
emv
```

         fun: -0.6644372965632436
         jac: array([-0.36088951, -0.36076408, -0.36036385, -0.36108153, -0.36062376])
     message: 'Optimization terminated successfully.'
        nfev: 35
         nit: 5
        njev: 5
      status: 0
     success: True
           x: array([0.50714174, 0.07470888, 0.02471533, 0.18943972, 0.20399434])

```python
# Pesos, rendimiento y riesgo del portafolio EMV
w_emv = emv.x
E_emv = Eind.dot(w_emv)
s_emv = w_emv.dot(Sigma).dot(w_emv)**0.5
RS_emv = (E_emv - rf) / s_emv
w_emv, E_emv, s_emv, RS_emv
```

    (array([0.50714174, 0.07470888, 0.02471533, 0.18943972, 0.20399434]),
     0.1420657564717856,
     0.13856199365687238,
     0.6644372965632436)

```python
w_emv, w_minvar
```

    (array([0.50714174, 0.07470888, 0.02471533, 0.18943972, 0.20399434]),
     array([6.17797049e-01, 3.46944695e-18, 0.00000000e+00, 2.09394358e-01,
            1.72808594e-01]))

##### 1.3. Construir frontera de mínima varianza

También debemos encontrar la covarianza (o correlación) entre estos dos portafolios:

```python
# Covarianza entre los portafolios
s_minvar_emv = w_minvar.dot(Sigma).dot(w_emv)
```

```python
# Correlación entre los portafolios
r_minvar_emv = s_minvar_emv / (s_minvar * s_emv)
r_minvar_emv
```

    0.9885708894197612

```python
# Vector de w
w = np.linspace(0, 8, 100)
```

```python
# DataFrame de portafolios:
# 1. Índice: i
# 2. Columnas 1-2: w, 1-w
# 3. Columnas 3-4: E[r], sigma
# 4. Columna 5: Sharpe ratio
frontera = pd.DataFrame(index=range(1,101),
                        data={'w_emv': w,
                              'w_minvar': 1 - w,
                              'Media': w * E_emv + (1 - w) * E_minvar,
                              'Vol': ((w * s_emv)**2 + ((1 - w) * s_minvar)**2 + 2 * w * (1 - w) * s_minvar_emv)**0.5
                             }
                       )
frontera['RS'] = (frontera['Media'] - rf) / frontera['Vol']
frontera.head(10)
```

      w_emv
      w_minvar
      Media
      Vol
      RS

      1
      0.000000
      1.000000
      0.139629
      0.136447
      0.656878

      2
      0.080808
      0.919192
      0.139826
      0.136500
      0.658064

      3
      0.161616
      0.838384
      0.140023
      0.136575
      0.659148

      4
      0.242424
      0.757576
      0.140220
      0.136670
      0.660130

      5
      0.323232
      0.676768
      0.140417
      0.136785
      0.661011

      6
      0.404040
      0.595960
      0.140614
      0.136922
      0.661790

      7
      0.484848
      0.515152
      0.140810
      0.137079
      0.662467

      8
      0.565657
      0.434343
      0.141007
      0.137257
      0.663043

      9
      0.646465
      0.353535
      0.141204
      0.137455
      0.663519

      10
      0.727273
      0.272727
      0.141401
      0.137674
      0.663895

```python
# Importar librerías de gráficos
import matplotlib.pyplot as plt
%matplotlib inline
```

```python
# Gráfica de dispersión de puntos coloreando
# de acuerdo a SR, los activos individuales
# y los portafolios hallados
plt.figure(figsize=(6, 4))
# Frontera de minima varianza
plt.scatter(frontera['Vol'],
            frontera['Media'],
            c=frontera['RS'],
            cmap='twilight',
            label='Front. Min. Var.'
           )
# Portafolio de minima varianza
plt.plot(s_minvar, E_minvar, 'or', ms=5, label='Port. Min. Var.')
# Portafolio EMV
plt.plot(s_emv, E_emv, 'ob', ms=5, label='Port. EMV')
# Indices individuales
plt.plot(annual_ret_summ.loc['Volatilidad'],
         annual_ret_summ.loc['Media'],
         'ko',
         ms=5,
         label='Indices'
        )
plt.colorbar()
plt.xlabel('Volatilidad $\sigma$')
plt.ylabel('Rendimiento esperado $E[r]$')
plt.grid()
plt.legend(loc='best')
```

**A partir de lo anterior, solo restaría construir la LAC y elegir la distribución de capital de acuerdo a las preferencias (aversión al riesgo).**
___

## 3. Comentarios finales

### 3.1. Restricciones adicionales

Los inversionistas pueden tener restricciones adicionales:
1. Restricciones en posiciones cortas.
2. Pueden requerir un rendimiento mínimo.
3. Inversión socialmente responsable: prescinden de inversiones en negocios o paises considerados éticamente o políticamente indeseables.

Todo lo anterior se puede incluir como restricciones en el problema de optimización, y puede ser llevado a cabo a costa de un cociente de Sharpe menor.

### 3.2. Críticas a la optimización media varianza
1. Solo importan medias y varianzas: recordar que la varianza subestima el riesgo en algunos casos.
2. Preferencias media-varianza tratan las ganancias y pérdidas simétricamente: el sentimiento de insatisfacción de una perdida es mayor al sentimiento de satisfacción de una ganancia (aversión a pérdidas).
3. La aversión al riesgo es constante: la actitud frente al riesgo puede cambiar, por ejemplo con el estado de la economía.
4. Horizonte corto (un periodo).
5. Basura entra - basura sale: la optimización media varianza es supremamente sensible a las entradas: estimaciones de rendimientos esperados y varianzas.
___

# Anuncios parroquiales

## 1. Revisar archivo Tarea 6. Definir fecha.
## 2. Fin de módulo. Abrir Clase0 y revisar objetivos del módulo.
## 3. [Nota interesante](http://yetanothermathprogrammingconsultant.blogspot.com/2016/08/portfolio-optimization-maximize-sharpe.html)

Created with Jupyter by Esteban Jiménez Rodríguez.


## [Problem 41](https://projecteuler.net/problem=41)
Pandigital prime

```python
import itertools
from sympy import isprime
seq = [7,6,5,4,3,2,1]
```

```python
def list2num(permutated_list):
    num = 0
    for i in range(7):
        num += permutated_list[i]*10**(7-i)
```

```python
list7 = list(itertools.permutations(seq))
int7 = [int(''.join([str(_) for _ in __])) for __ in list7]
```

```python
for i in int7:
    if isprime(i):
        print(i)
        break
```

    7652413

## [Problem 42](https://projecteuler.net/problem=42)
Coded triangle numbers

```python
import csv,string

f = open('./data/p042.txt', "r")
file = list(csv.reader(f))[0]
```

```python
def word2num(word):
    counter = 0
    for alphabet in word:
        counter += string.ascii_uppercase.index(alphabet)+1
    return counter
```

```python
num_list = [word2num(word) for word in file]
```

```python
triangle_list = [int(n*(n+1)/2) for n in range(1,20)]
```

```python
counter = 0
for num in num_list:
    if [num] == list(set([num]) & set(triangle_list)):
        counter += 1
counter
```

    162

## [Problem 43](https://projecteuler.net/problem=43)
Sub-string divisibility

```python
import copy
ten_str={str(i) for i in range(10)}
```

```python
pandigital_numbers=[]
tmps=[]
for i in range(1,59):
    num=str(i*17).zfill(3)
    if len(set(num))==3:
        pandigital_numbers.append(num)

for x in [13,11,7,5,3,2]:
    tmps=[]
    for num in pandigital_numbers:
        for i in ten_str-set(num):
            tmp=str(i)+num
            if int(tmp[:3])%x==0:
                tmps.append(tmp)
    pandigital_numbers=copy.copy(tmps)

pandigital_numbers=[str(list(ten_str-set(num))[0])+num for num in pandigital_numbers]
```

```python
sum_pandigital=0
for num in pandigital_numbers:
    sum_pandigital+=int(num)
print(sum_pandigital)
```

    16695334890

## [Problem 44](https://projecteuler.net/problem=44)
Pentagon numbers

```python
import numpy as np
from tqdm import tqdm
```

```python
def p(n):
    return int(n*(3*n-1)/2)

def factorize(n):
    answer=[]
    max_factor=int(np.sqrt(n))
    for i in range(1,max_factor+1):
        if n%i==0:
            answer.append([i,n//i])
    return answer

def check_squred(n):
    m=int(np.sqrt(n))
    if m-n/m==0.0:
        return True
    else:
        return False

def jk_list_from_m(m):
    jk_list=[]
    n=3*m*m-m
    factors=factorize(n)
    for factor in factors:
        q,p=factor
        j,k=int(((p+1)/3+q)/2),int(((p+1)/3-q)/2)
        if 3*(j*j-k*k)-j+k==n and k>0:
            jk_list.append([j,k])
    return jk_list

def check_valid_jk(jk):
    j,k=jk
    n=3*(j*j+k*k)-j-k
    l=int((1+np.sqrt(1+12*n))/6)
    if 3*l*l-l-n==0:
        return True
    else:
        return False
```

```python
max_m=5000
for m in tqdm(range(1,max_m)):
    jk_list=jk_list_from_m(m)
    for jk in jk_list:
        if check_valid_jk(jk):
            print(m,p(m))
```

     46%|████▌     | 2287/4999 [00:00<00:00, 3221.80it/s]

    1912 5482660

    100%|██████████| 4999/4999 [00:02<00:00, 2133.62it/s]

## [Problem 45](https://projecteuler.net/problem=45)
Triangular, pentagonal, and hexagonal

```python
import numpy as np
from tqdm import tqdm
```

```python
def check_valid_n(n):
    n=4*n*n-2*n
    m=int((1+np.sqrt(1+12*n))/6)
    if 3*m*m-m-n==0:
        l=int((-1+np.sqrt(1+4*n))/2)
        if l*l+l-n==0:
            return True
        else:
            return False
    else:
        return False
```

```python
max_n=100000
for n in tqdm(range(1,max_n)):
    if check_valid_n(n):
        print(n,2*n*n-n)
```

     37%|███▋      | 36701/99999 [00:00<00:00, 180175.55it/s]

    1 1
    143 40755
    27693 1533776805

    100%|██████████| 99999/99999 [00:00<00:00, 199493.64it/s]

## [Problem 46](https://projecteuler.net/problem=46)
Goldbach's other conjecture

```python
from sympy import sieve
import numpy as np
from itertools import product
from tqdm import tqdm
```

```python
max_num=10**5
goldbach_conjecture_list=[i for i in tqdm(range(2,max_num)) if i not in sieve and i%2==1]
```

    100%|██████████| 99998/99998 [00:06<00:00, 15029.80it/s]

```python
for goldbach_conjecture in tqdm(goldbach_conjecture_list):
    max_square_root=int(np.sqrt(goldbach_conjecture/2))
    check_prime_list=[goldbach_conjecture-2*i*i for i in range(1,max_square_root+1)]
    check_list=[num in sieve for num in check_prime_list]
    check={False}==set(check_list)
    if check:
        print(goldbach_conjecture)
        break
```

      5%|▌         | 2104/40408 [00:02<01:44, 365.06it/s]

    5777

## [Problem 47](https://projecteuler.net/problem=47)
Distinct primes factors

```python
from sympy import factorint
```

```python
def len_factor(n):
    return len(list(factorint(n).keys()))
```

```python
n=210
flag=True
while flag:
    four_nums=[n,n+1,n+2,n+3]
    prime_nums=[len_factor(n) for n in four_nums]
    flag=False
    for i in range(4):
        if prime_nums[i]!=4:
            flag=True
            n=four_nums[i]+1
            continue
```

```python
n,prime_nums
```

    (134043, [4, 4, 4, 4])

## [Problem 48](https://projecteuler.net/problem=48)
Self powers

```python
from scipy.special import comb
```

```python
answer=0
for i in range(1,11):
    answer+=i**i
for i in range(11,1001):
    x=i-10
    for k in range(10):
        answer+=(10**k)*(x**(i-k))*comb(i,k,True)
```

```python
answer%10**10
```

    9110846700

## [Problem 49](https://projecteuler.net/problem=49)
Prime permutations

```python
from sympy import sieve
from itertools import permutations
from itertools import combinations
from tqdm import tqdm
import numpy as np
from copy import copy
```

```python
def permutate_num(n):
    nums=[]
    for xs in permutations(str(n)):
        num=''
        for x in xs:
            num+=x
        nums.append(int(num))
    return nums

def increases(num_list):
    n=len(num_list)
    for i,j,k in combinations(range(n),3):
        if num_list[k]-num_list[j]==num_list[j]-num_list[i]:
            print(str(num_list[i])+str(num_list[j])+str(num_list[k]))
```

```python
prime_list=list(sieve.primerange(10**3,10**4))
prime_set=set(prime_list)
```

```python
possibly_permutation=[]
for prime in tqdm(prime_list):
    permutate_list=permutate_num(prime)
    permutate_prime_list=list(sorted(list(set([x for x in permutate_list if x in set(prime_list)]))))
    if len(permutate_prime_list)>2:
        possibly_permutation.append(permutate_prime_list)
```

    100%|██████████| 1061/1061 [00:00<00:00, 1143.42it/s]

```python
permutation_list=[]
for nums in possibly_permutation:
    if nums not in permutation_list:
        permutation_list.append(nums)
```

```python
for permutation in tqdm(permutation_list):
    increases(permutation)
```

    100%|██████████| 174/174 [00:00<00:00, 20055.20it/s]

    148748178147
    296962999629

## [Problem 50](https://projecteuler.net/problem=50)
Consecutive prime sum

```python
from sympy import sieve
import numpy as np
from tqdm import tqdm
```

```python
prime_list=[i for i in sieve.primerange(1,10**6)]
dp=[set(prime_list)]
```

```python
newArray=prime_list.copy()
for i in tqdm(range(600)):
    newArray=np.array(prime_list[i+1:])+np.array(newArray[:-1])
    dp.append(set(newArray))
```

    100%|██████████| 600/600 [00:08<00:00, 69.36it/s]

```python
for i in range(600):
    intersection=dp[600-i-1] & dp[0]
    if len(intersection)>0:
        print(intersection)
        break
```

    {997651}

```python

```


# Magnetotellurics

In this section we discuss how to solve the PDE for 2D magnetotellurics (MT) due to a variation of
the electric conductivity $\sigma$ in the subsurface using `esys.escript`. It is assumed that you have
worked through the [introduction section on `esys.escript`](escriptBasics.ipynb).
First we will provide the basic theory:

## References

- [Chave, A. D., & Jones, A. G. (Eds.) (2012). The magnetotelluric method: Theory and practice. New York: Cambridge University Press. ](https://doi.org/10.1017/CBO9781139020138)
- [R. Schaa, L. Gross, J. du Plessis: “PDE-based geophysical modelling using Finite Elements: examples from 3-D resistivity and 2-D magnetotellurics”, 2016, Journal of Geophysics and Engineering, 13, S59-73](https://doi.org/10.1088/1742-2132/13/2/S59)
- [Niwas, S., Gupta, P.K. & Gaur, V.K. J Earth Syst Sci (2005) 114: 523. https://doi.org/10.1007/BF02702028](https://doi.org/10.1007/BF02702028)

## Maxwell's Equations in 2D

In this prac we are investigating the magnetotellurics (MT) method. The MT
is targeting to recover the distribution of the electric conductivity $\sigma$ (in [S/m]) in the subsurface.
It is an electro-magnetic method (EM) using natural sources of electro-magnetic fields a very low
frequency which allows to target very deep targets ($>50km$) but at low resolution.
Here we will focus on the case of $H$−polarisation (also known as TM mode).
In this case it is assumed that the incoming magnetic field is polarized in direction $x$ perpendicular to the
$yz$ plane of investigation. There is also the case of $E$ polarization (also known as TE mode)
where the electric field polarized in the direction $x$.

For the TM mode the modeling equations are given as the a 2D simplification of the Maxwell's equations.
For the given transversal magnetic field $H_x$ the vertical and
horizontal components $E_z$ and $E_y$ of the electric field are given as
\begin{equation}\label{eqTMEz}
\frac{\partial H_x}{\partial y }= - \sigma E_z
\end{equation}
\begin{equation}\label{eqTMEy}
\frac{\partial H_x}{\partial z }=  \sigma E_y
\end{equation}
The horizontal components then need to fulful the equation:
\begin{equation}\label{eqMaxwell}
\frac{\partial E_z}{\partial y }  - \frac{\partial E_y}{ \partial z } = - \mathbf{j} \omega \mu_0 H_x
\end{equation}
where $\mathbf{j}$ is the complex unity, $\omega$ is the angular frequency ([Hz])
and $\mu_0= 4 \pi \cdot 10^{-7} \frac{N}{A^2}$ is the magnetic constant.

## Impedance

Field observations will deliver impedance as function of
angular frequency $\omega$. In the TM mode the impedance is given as
\begin{equation}\label{eqImp}
Z_{yx}(\omega)=\frac{E_y}{H_x}
\end{equation}
This is a complex number. For interpretation an alternative representation
based on apparent resistivity $\rho_a$ and phase $\phi$ is commonly used.
These quantities are derived from the impedance $Z_{yx}$ as
\begin{equation}\label{eqRhoa}
\rho_a(\omega) =\frac{1}{\omega \mu_0} |Z_{yx}(\omega)|^2
\end{equation}
and
\begin{equation}\label{eqPhase}
\phi((\omega)= \arctan( \frac{Z_{yx}(\omega).imag}{Z_{yx}(\omega).real})
\end{equation}
For constant conductivity $\sigma$.
the apparent resistivity and the phase are frequency independent
and calculate as
\begin{equation}\label{eqRhoaHH}
\rho_a(\omega) =\frac{1}{\sigma}
\end{equation}
and
\begin{equation}\label{eqPhaseHH}
\phi((\omega)= 45^o
\end{equation}

##  TM-mode in `esys-escript`

Now we need to define the PDE using the  `LinearSinglePDE` PDE template PDE. We work in the $x_0=y$
and $x_1=z$ coordinate system.

Here we restate the template: When $u$ is the unknown
we define the so-called flux vector $\mathbf{F}$ which is in essence the negative gradient of the solution
times some matrix $\mathbf{A}$:
\begin{equation} \label{EQESYSFLUX}
\mathbf{F} = - \mathbf{A} \mathbf{\nabla} u + \mathbf{X}
\end{equation}
with matrix $\mathbf{A}$ and vector  $\mathbf{X}$
The flux vector $\mathbf{F}$ needs to fulfill the conservation equation :
\begin{equation}\label{EQESYSCONSERVATION}
\mathbf{\nabla}^t \; \mathbf{F} + D \; u = Y
\end{equation}
where $D$ is a scalar and $Y$ is the right hand side.

The Maxwell equation \eqref{eqMaxwell} in the $x_0,x_1$-coordinates reads as
\begin{equation}\label{eqMaxwell2}
\frac{\partial E_z}{\partial x_0}  - \frac{\partial E_y}{ \partial x_1 } + \mathbf{j} \omega \mu_0 H_x =0
\end{equation}
from which we can identfiy the $D=\mathbf{j} \omega \mu_0$ and $Y=0$ and the flux
\begin{equation}\label{EQFLUX1}
\mathbf{F} =
\begin{bmatrix}
 F_0 \\
 F_1
\end{bmatrix}
= \begin{bmatrix}
 E_z \\
 -E_y
\end{bmatrix}
\end{equation}
The unknown of the problem becomes $u=H_x$ and
using \eqref{eqTMEz} and \eqref{eqTMEy}
hence the flux is given as
\begin{equation}\label{EQFLUX2}
\mathbf{F} =
\begin{bmatrix}
 - \frac{1}{\sigma} \frac{\partial H_x}{\partial y } \\
 - \frac{1}{\sigma} \frac{\partial H_x}{\partial z }
\end{bmatrix}
= -
\begin{bmatrix}
\frac{1}{\sigma} \frac{d u}{d x_0} & + & 0 \frac{d u}{d x_1}\\
0 \frac{d u}{d x_0} & + & \frac{1}{\sigma} \frac{d u}{d x_1}
\end{bmatrix}
\end{equation}
This gives the matrix $\mathbf{A}$
\begin{equation}
\begin{bmatrix}
A_{00} & A_{01} \\
A_{10}  & A_{11}
\end{bmatrix}=
\begin{bmatrix}
\frac{1}{\sigma} & 0  \\
0  & \frac{1}{\sigma}
\end{bmatrix}
\end{equation}
and  $\mathbf{X}=0$.

## Example Problem: Transects over a conductive Step

We consider a step of a conductive media ($\sigma_1=0.1$) within a resistive media ($\sigma_0=0.001$)
and want to calculate the distribution of the impedance along the surface. The step size is $3km$
and the top of the conductive layer is located at depth $h_1=5km$.
Use a domain of horizontal extend of $40km$ and depth of $80km$ with a grid spacing of $200m$.
Plot the profile of the apparent resistivity and the phase for frequencies $f=0.01, 0.3, 10 Hz$.

```python
NEy=200
NEz=400
dx=200   # [m] grid spacing
Ly=dx*NEy
Lz=dx*NEz
print("Domain dimension = %d x %d m"%(Ly, Lz))

sigma1=0.001
sigma2=0.1
h1=5000.
h2=10000.

import numpy as np
Mu0=4*np.pi*1e-7

#%matplotlib notebook
```

    Domain dimension = 40000 x 80000 m

As before the first step is to set up the domain:

```python
from esys.escript import *
from esys.finley import Rectangle

domain=Rectangle(n0=NEy, n1=NEz, l0=Ly, l1=Lz)
```

Then we initiate the PDE we need to solve for $u=H_x$. We need to tell the `LinearSinglePDE` that the
PDE has coefficients with complex values:

```python
from esys.escript.linearPDEs import LinearSinglePDE
pde=LinearSinglePDE(domain, isComplex=True)
```

In general `esys.escript` tries to use iterative solvers for the discrete version of the PDE as this
less memory demanding and in many cases very fast.
In this particular application case we want make sure that a direct solver is used as due to the complex $D$ the
PDE is not necessarily suitable for an iterative solver:

```python
from esys.escript.linearPDEs import SolverOptions
pde.getSolverOptions().setSolverMethod(SolverOptions.DIRECT)
```

The magnetic field is set to one at the top of the domain. As before we use
`q` to describe the location at which the $u=H_x$ is prescribed namely at $x_1=z=Lz$.
To set the actual value we use `r`. So far we always set the solution to zero and - as this is the default assumption - we did not to set a value (potentially a `Data` object) for `r`:

```python
x=domain.getX()
pde.setValue(r=1., q=whereZero(x[1]-Lz))
```

The PDE coefficient `D` is obvious from \eqref{eqMaxwell2}. We pick a frequency of $f=1Hz$.

```python
f=1. # Hz
pde.setValue(D=1j*2*np.pi*f*Mu0)
```

Setting the conductivity distribution needs a bit of thinking. We first introduce
to masks; `m1` which is one for locations below $L_z-h_1$ for the step of
the left side and `m2` which is one for locations below $L_z-h_2$ for the step of
the right side:

```python
X=Function(domain).getX()
m1=whereNegative(X[1]-(Lz-h1))
m2=whereNegative(X[1]-(Lz-h2))
```

As said `m1` is used on the left part of the domain where $x_0<\frac{Ly}{2}$ while on
the other side `m2` is used. We use an additional mask `mH` to separate the two sides:

```python
mH=whereNegative(X[0]-Ly/2)
m=m1*mH+m2*(1-mH)
```

Now we have a mask `m` which is has value one for the step basement. This allows us to set the
conductivity `SigmaS`:

```python
sigmaS=sigma1*(1-m)+sigma2*m
```

Let's take a look at `sigmaS`. To reduce the calculation time for the rendering  `sigmaS` is interpolated
from integration points to element centers prior to visualization.

```python
sigmaS_np=convertToNumpy(interpolate(sigmaS, ReducedFunction(domain)))
x_np=convertToNumpy(ReducedFunction(domain).getX())

import matplotlib.pyplot as plt
plt.figure(figsize=(5,5))
plt.tricontourf(x_np[0], x_np[1], sigmaS_np[0], 15)
plt.xlabel('y [m]')
plt.ylabel('z [m]')
plt.title("Conductivity distribution")
plt.colorbar()
```

    <matplotlib.colorbar.Colorbar at 0x7fe36e08dc50>

Ready to set the `A` coefficient:

```python
pde.setValue(A=1./sigmaS*kronecker(domain))
```

Now we can get the solution $u=H_x$:

```python
Hx=pde.getSolution()
print(abs(Hx))
```

    Summary: inf=1.55785e-20 sup=1 data points=80601

Impedance is obtained from the definition. We look at the value at element centers.

```python
Ey=1./sigmaS*grad(Hx, ReducedFunction(domain))[1]
Zyx=Ey/Hx

rho_a=1./(2*np.pi*f*Mu0)*abs(Zyx)**2
phi=phase(Zyx)
```

Let's plot the distribution of the apparent resistivity:

```python
rho_a_np=convertToNumpy(rho_a)
x_np=convertToNumpy(rho_a.getFunctionSpace().getX())
print(rho_a_np)

import matplotlib.pyplot as plt
plt.figure(figsize=(5,5))
plt.tricontourf(x_np[0], x_np[1], log(rho_a_np[0]), 15)
plt.xlabel('y [m]')
plt.ylabel('z [m]')
plt.title("log of apparent resitivity")
plt.colorbar()
```

Field data are collected along transects at the surface. Lets plot the apparent resistivity and
phase along the surface.
We first define a `Locator` for the points at/near the surface:

```python
from esys.escript.pdetools import Locator
locations_in_transect=[]
for k in range(NEy):
    x_t=dx*k+dx/2.
    locations_in_transect.append( [x_t, Lz ] )
locator_transect=Locator(where=ReducedFunction(domain), x=locations_in_transect)
```

Get the horizontal locations along the transect (just in case they have changed):

```python
transect_x=[ x[0] for x in locator_transect.getX()]
```

Grab the values for apparent resistivity and phase and then plot

```python
rho_a_x=locator_transect(rho_a)
phi_x=locator_transect(phi*180/np.pi)
```

```python
plt.figure(figsize=(10,5))
plt.clf()
plt.plot(transect_x,rho_a_x)
plt.xlabel('y [m]')
plt.ylabel('resistivity [m]')
plt.title("Resistivity vs offset")
```

```python
plt.figure(figsize=(10,5))
plt.clf()
plt.plot(transect_x,phi_x)
plt.xlabel('y [m]')
plt.ylabel('phase [deg]')
plt.title("Phase vs offset")
```


# Regressão Linear

Dada uma função *target* desconhecida $f:\mathbb{R}^{d} \rightarrow \mathbb{R}$ e uma hipótese $h_{\mathbf{w}}:\mathbb{R}^{d} \rightarrow \mathbb{R}$ o erro de $h_{\mathbf{w}}$ é definido por:

\begin{equation}
E_{out}(h_{\mathbf{w}}) = \mathbb{E}_{\mathbf{x}\sim p_{data}}L(h(\mathbf{x}; \mathbf{w}), \; f(\mathbf{x}))
\end{equation}

em que $p_{data}$ é a distribuição geradora dos dados, $L$  é alguma função de perda (por exemplo, o quadrado da diferença) e $h(\mathbf{x}; \mathbf{w}) = h_{\mathbf{w}}(\mathbf{x})$. Se tivéssemos acesso a $p_{data}$ poderíamos calcular a função acima para qualquer $h_{\mathbf{w}}$ e escolher aquele com erro mínimo.

Como não temos acesso a $p_{data}$, define-se

\begin{equation}
E_{in}(h_{\mathbf{w}}) = J(\mathbf{w}) = \frac{1}{N}\sum_{i=1}^{N} L(h(\mathbf{x}_{i}; \mathbf{w}), \; y_{i})
\end{equation}

em que $N$ é o tamanho do dataset de treinamento e $y_{i} = f(\mathbf{x}_{i})$.

Queremos encontrar a função $h_{\mathbf{w}}$ mais próxima de $f$. Isto corresponde a determinar o ponto mínimo da função $J(\mathbf{w})$.

## Funções auxiliares

Importamos os pacotes e definimos as funções que usaremos ao longo do exercício.

```python
from IPython.core.pylabtools import figsize, getfigs
from IPython.display import display
from IPython.display import HTML
import matplotlib.animation as animation
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
from sklearn.datasets import load_boston

def simple_step_plot(ylist, yname, title, figsize=(4, 4), labels=None):
    y0 = ylist[0]
    x = np.arange(1, len(y0) + 1, 1)
    fig, ax = plt.subplots(1, 1, figsize=figsize)
    for y in ylist:
        ax.plot(x, y)
    plt.xlabel('step')
    plt.ylabel(yname)
    plt.title(title,
              fontsize=14,
              fontweight='bold')
    plt.grid(True)
    if labels is not None:
        plt.legend(labels,
           loc='upper right')
    plt.show()
```

## O dataset

Vamos usar o dataset *Boston Housing Prices*, organizado pelo grupo de *Machine Learning* da *University of California Irvine* e disponível para baixar diretamente através do pacote `sklearn`. Para facilitar a visualização dos resultados, vamos escolher apenas uma variável de entrada e prever o valor mediano da casa (em $1000s).

```python
data = load_boston()
df = pd.DataFrame(data.data, columns=data.feature_names)

X = df.LSTAT.values.reshape((-1,1))
X = np.concatenate([X, np.ones((X.shape[0],1))], axis=1) # adiciona dimensão extra (bias)
y = data.target.reshape((-1,1))

plt.scatter(X[:,0], y, alpha=0.4)
plt.xlabel('LSTAT')
plt.ylabel('median value in $1000')
```

```python
print(data['target'])
```

    [24.  21.6 34.7 33.4 36.2 28.7 22.9 27.1 16.5 18.9 15.  18.9 21.7 20.4
     18.2 19.9 23.1 17.5 20.2 18.2 13.6 19.6 15.2 14.5 15.6 13.9 16.6 14.8
     18.4 21.  12.7 14.5 13.2 13.1 13.5 18.9 20.  21.  24.7 30.8 34.9 26.6
     25.3 24.7 21.2 19.3 20.  16.6 14.4 19.4 19.7 20.5 25.  23.4 18.9 35.4
     24.7 31.6 23.3 19.6 18.7 16.  22.2 25.  33.  23.5 19.4 22.  17.4 20.9
     24.2 21.7 22.8 23.4 24.1 21.4 20.  20.8 21.2 20.3 28.  23.9 24.8 22.9
     23.9 26.6 22.5 22.2 23.6 28.7 22.6 22.  22.9 25.  20.6 28.4 21.4 38.7
     43.8 33.2 27.5 26.5 18.6 19.3 20.1 19.5 19.5 20.4 19.8 19.4 21.7 22.8
     18.8 18.7 18.5 18.3 21.2 19.2 20.4 19.3 22.  20.3 20.5 17.3 18.8 21.4
     15.7 16.2 18.  14.3 19.2 19.6 23.  18.4 15.6 18.1 17.4 17.1 13.3 17.8
     14.  14.4 13.4 15.6 11.8 13.8 15.6 14.6 17.8 15.4 21.5 19.6 15.3 19.4
     17.  15.6 13.1 41.3 24.3 23.3 27.  50.  50.  50.  22.7 25.  50.  23.8
     23.8 22.3 17.4 19.1 23.1 23.6 22.6 29.4 23.2 24.6 29.9 37.2 39.8 36.2
     37.9 32.5 26.4 29.6 50.  32.  29.8 34.9 37.  30.5 36.4 31.1 29.1 50.
     33.3 30.3 34.6 34.9 32.9 24.1 42.3 48.5 50.  22.6 24.4 22.5 24.4 20.
     21.7 19.3 22.4 28.1 23.7 25.  23.3 28.7 21.5 23.  26.7 21.7 27.5 30.1
     44.8 50.  37.6 31.6 46.7 31.5 24.3 31.7 41.7 48.3 29.  24.  25.1 31.5
     23.7 23.3 22.  20.1 22.2 23.7 17.6 18.5 24.3 20.5 24.5 26.2 24.4 24.8
     29.6 42.8 21.9 20.9 44.  50.  36.  30.1 33.8 43.1 48.8 31.  36.5 22.8
     30.7 50.  43.5 20.7 21.1 25.2 24.4 35.2 32.4 32.  33.2 33.1 29.1 35.1
     45.4 35.4 46.  50.  32.2 22.  20.1 23.2 22.3 24.8 28.5 37.3 27.9 23.9
     21.7 28.6 27.1 20.3 22.5 29.  24.8 22.  26.4 33.1 36.1 28.4 33.4 28.2
     22.8 20.3 16.1 22.1 19.4 21.6 23.8 16.2 17.8 19.8 23.1 21.  23.8 23.1
     20.4 18.5 25.  24.6 23.  22.2 19.3 22.6 19.8 17.1 19.4 22.2 20.7 21.1
     19.5 18.5 20.6 19.  18.7 32.7 16.5 23.9 31.2 17.5 17.2 23.1 24.5 26.6
     22.9 24.1 18.6 30.1 18.2 20.6 17.8 21.7 22.7 22.6 25.  19.9 20.8 16.8
     21.9 27.5 21.9 23.1 50.  50.  50.  50.  50.  13.8 13.8 15.  13.9 13.3
     13.1 10.2 10.4 10.9 11.3 12.3  8.8  7.2 10.5  7.4 10.2 11.5 15.1 23.2
      9.7 13.8 12.7 13.1 12.5  8.5  5.   6.3  5.6  7.2 12.1  8.3  8.5  5.
     11.9 27.9 17.2 27.5 15.  17.2 17.9 16.3  7.   7.2  7.5 10.4  8.8  8.4
     16.7 14.2 20.8 13.4 11.7  8.3 10.2 10.9 11.   9.5 14.5 14.1 16.1 14.3
     11.7 13.4  9.6  8.7  8.4 12.8 10.5 17.1 18.4 15.4 10.8 11.8 14.9 12.6
     14.1 13.  13.4 15.2 16.1 17.8 14.9 14.1 12.7 13.5 14.9 20.  16.4 17.7
     19.5 20.2 21.4 19.9 19.  19.1 19.1 20.1 19.9 19.6 23.2 29.8 13.8 13.3
     16.7 12.  14.6 21.4 23.  23.7 25.  21.8 20.6 21.2 19.1 20.6 15.2  7.
      8.1 13.6 20.1 21.8 24.5 23.1 19.7 18.3 21.2 17.5 16.8 22.4 20.6 23.9
     22.  11.9]

## Gradiente descendente

Dados $\mathbf{w}, \mathbf{u} \in \mathbb{R}^{d}$ e $J:\mathbb{R}^{d} \rightarrow \mathbb{R}$ tal que $||\mathbf{u}||_{2} = 1$ a taxa de variação de $J$ no ponto $\mathbf{w}$ em direção a $\mathbf{u}$ é chamada de **derivada direcional**, $D_{\mathbf{u}}J(\mathbf{w})$. Podemos minimizar $J$ alterando $\mathbf{w}$ na direção de $-\nabla_{\mathbf{w}}J(\mathbf{w})$.

- $\mathbf{w}(0) = \mathbf{w}$
- for $t = 0, 1, 2, \dots$ do
    * $\mathbf{w}(t+1) = \mathbf{w}(t) - \eta \nabla_{\mathbf{w}(t)}J(\mathbf{w}(t))$

O parâmetro $\eta \in \mathbb{R}_{\geq}$ é chamado de **taxa de aprendizado** (*learning rate*); ele pondera o tamanho de cada atualização.

### Computando a função de custo

Usando o erro quadrárico médio, a função de custo $J(\mathbf{w})$ para a tarefa de regressão linear pode ser escrita dessa forma:

\begin{equation}
J(\mathbf{w}) = \frac{1}{N}\sum_{i=1}^{N}(\hat{y}_{i} - y_{i})^{2}
\end{equation}

```python
#TODO: implementar
def compute_cost(X, y, w):
    #TODO
    J = 999

    return J
```

### Computando os gradientes

A derivada parcial de $J(\mathbf{w})$ com relação a cada entrada $j$ de $\mathbf{w}$ é dada por

\begin{equation}
\frac{\partial J(\mathbf{w})}{\partial \mathbf{w}_{j}} = \frac{2}{N}\sum_{i=1}^{N} (\hat{y}_i - y_i) \mathbf{x}_{ij}
\end{equation}

Lembre que o gradiente de $J(\mathbf{w})$ com relação a $\mathbf{w}$ é:

\begin{equation}
\nabla_{\mathbf{w}}J(\mathbf{w}) = \begin{bmatrix}\frac{\partial J(\mathbf{w})}{\partial \mathbf{w}_{1}} \dots \frac{\partial J(\mathbf{w})}{\partial \mathbf{w}_{m}} \end{bmatrix}
\end{equation}

```python
def compute_wgrad(X, y, w):
    """
    Calculates gradient of J(w) with respect to w.

    :param X: design matrix
    :type X: np.ndarray(shape=(N, d))
    :param y: regression targets
    :type y: np.ndarray(shape=(N, 1))
    :param w: weights
    :type w: np.array(shape=(d, 1))
    :return: gradient
    :rtype: np.array(shape=(d, 1))
    """
    N = X.shape[0]
    diff = np.dot(X,w) - y
    grad = 2/N * np.dot(np.transpose(X), diff)

    return grad
```

### Computando as atualizações dos pesos

A versão mais simples do algoritmo *gradient descent* faz uso de todas as observações do dataset de treinamento (esse algoritmo também é conhecido como *batch gradient descent* ou *vanilla gradient descent*).

**Batch gradient descent**

- $\mathbf{w}(0) = \mathbf{w}$
- for $t = 0, 1, 2, \dots$ do
    * Compute the gradient $\nabla_{\mathbf{w}(t)}J(\mathbf{w}(t))$
    * Apply update : $\mathbf{w}(t+1) = \mathbf{w}(t) - \eta \nabla_{\mathbf{w}(t)}J(\mathbf{w}(t))$

```python
def gradient_descent(X, y, w, learning_rate, num_iters):

    weights_history = [w.flatten()]
    cost_history = [compute_cost(X, y, w)]

    w_current = w
    for t in range(0, num_iters-1):
        grad = compute_wgrad(X, y, w_current)
        w_current = w_current - learning_rate * grad
        weights_history.append(w_current.flatten())
        cost = compute_cost(X, y, w_current)
        cost_history.append(cost)
        if cost == 0:
            break

    return w, np.array(weights_history), cost_history
```

```python
X = np.zeros((10,2))
print(X)
```

    [[0. 0.]
     [0. 0.]
     [0. 0.]
     [0. 0.]
     [0. 0.]
     [0. 0.]
     [0. 0.]
     [0. 0.]
     [0. 0.]
     [0. 0.]]

## Treinando o modelo

Experimente variar o *learning rate* e os pesos iniciais.

```python
learning_rate = 0.002
initial_w = np.array([[100], [100]])
iterations = 20

w, weights_history, cost_history = gradient_descent(X, y, initial_w, learning_rate, iterations)
title = "Optimization landscape\nlearning rate = {} | iterations = {}".format(learning_rate, iterations)

simple_step_plot(
    [cost_history], 'loss',
    'Training loss\nlearning rate = {} | iterations = {}'.format(learning_rate, iterations))

plt.scatter(X[:,0], y, alpha=0.4)
x_reg = np.linspace(0, 40, 100)
y_reg = weights_history[-1,0]*x_reg + weights_history[-1,1]
plt.plot(x_reg, y_reg, c='r')
```

```python

```