# Intro to Network Science

**Author: [Erika Fille Legara](http://erikalegara.site)**

[](https://github.com/eflegara/Network-Science-Lectures/blob/master/LICENSE.md)

For context, this notebook is accompanied by a two-session lecture (sessions 5-6) of the Network Science course under [AIM's MSc in Data Science program](https://aim.edu/programs/degree-programs/master-science-data-science). The first two course sessions are focused on the introduction to Complex Systems; and 3-4 focus on the intro to the science of network (conceptual and high-level approach with most discussions on use cases). This notebook is for the first set of hands-on sessions.

The main reference for this lecture/notebook is [A-L Barabasi's Network Science](http://networksciencebook.comhttp://networksciencebook.com).

---

    Run in Google Colab

    View source on GitHub

```python
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
import warnings

warnings.filterwarnings("ignore")
%matplotlib inline
```

## 1. Graphs and DiGraphs

### 1.1. Adjacency Matrix

 The convention we use is if node $i \rightarrow j$, then $A_{ij}$; similary, if node $j \rightarrow i$, we are looking at $A_{ji}$.

For example, for the **DiGraph** `adj_DiG`, node $1 \rightarrow 4$, but not the other way around. Thus,

\begin{equation}
     A_{ij}=A_{14} = 1
\end{equation}
\begin{equation}
     A_{ji} = A_{41} = 0
\end{equation}

for $i=1$ and $j=4$.

In matrix form,

\begin{equation}
A_{ij}=
  \left( {\begin{array}{cccc}
  A_{11} & A_{12} & A_{13} & A_{14}\\
  A_{21} & A_{22} & A_{23} & A_{24}\\
  A_{31} & A_{32} & A_{33} & A_{34}\\
  A_{41} & A_{42} & A_{43} & A_{44}\\
  \end{array} } \right).
\end{equation}

Therefore, for the two network representations above, Graph (`adj_G`) and DiGraph (`adj_DiG`), the matrix representation are the following.

Important: The indices used in the matrices start at 1, while indexing in Python starts at 0. Do remember this to avoid indexing mistakes.

```python
adj_G = np.matrix([[0,1,0,1],
                   [1,0,0,1],
                   [0,0,0,1],
                   [1,1,1,0]])

adj_DiG = np.matrix([[0,1,0,1],
                     [0,0,0,0],
                     [0,0,0,0],
                     [0,1,1,0]])
```

### 1.2. Node Degree

The degree of a node is the total number of links connected to it. Let's first look at the *unidrected* graph `adj_G` where $A_{ij}=A_{ji}$. **Node 4** has degree $k_4 = 3$.

From the adjacency matrix, we can obtain this number by summing across any one of the axes. That is,

```python
adj_G.sum(axis=0).item(0,3)
```

    3

or

```python
adj_G.sum(axis=1).item(3,0)
```

    3

However, for directed graphs, extra care must be taken as switching axes may produce erroneous results. In the DiGraph, for **Node 4**, $k_4^{\text{in}}=1$ while $k_4^{\text{out}}=2$.

From the numpy matrix, these can be obtained using the following scripts.

```python
# in-degree
adj_DiG.sum(axis=0).item(0,3)
```

    1

```python
# out-degree
adj_DiG.sum(axis=1).item(3,0)
```

    2

Important: The order and axes depend on the way you define the adjacency matrix; that is, what you define as $i$ and $j$, how you represent the rows and columns, and how you represent $i \rightarrow j$ and $i \leftarrow j$ in the matrix.

## 2. Complete Graph

A complete graph is a graph where all nodes are connected to all other nodes in the graph. That is,

\begin{equation}
A_{ij}=
  \left( {\begin{array}{cccc}
  0 & 1 & 1 & 1\\
  1 & 0 & 1 & 1\\
  1 & 1 & 0 & 1\\
  1 & 1 & 1 & 0\\
  \end{array} } \right)
\end{equation}

```python
complete_G = np.matrix([[0,1,1,1],
                        [1,0,1,1],
                        [1,1,0,1],
                        [1,1,1,0]])
```

In a complete graph, the total number of links is $L = L_{\text{max}}$, and

\begin{equation}
L_{\text{max}}=\frac{N(N-1)}{2}.
\end{equation}

```python
lmax = complete_G.sum(axis=0).sum()/2
print(lmax)
```

    6.0

We divide the sum by 2, to avoid double counting. That is, once the link between $1,2$ is considered ($A_{12}=1$) we don't any more include $2,1$ ($A_{21}=1$).

The metric $L_{\text{max}}$ is relevant to appreciate how sparse most real-world networks are. That is,

| network | $N$  | $L$  | $L_{\text{max}}$ | $\left<k \right>$ |
|---|---|---|---|---|
| WWW | $325,729$ | $1.4 \times 10^6$  | $10^{12}$ |  4.51  |
|Protein | $1,870$   | $4,470$ | $10^7$ | $2.39$ |
|Co-authorship | $70,975$  | $2 \times 10^5$ | $3 \times 10^{10}$ | $3.9$ |
|Movie Actors | $212,250$ | $6 \times 10^6$ | $1.8 \times 10^{13}$ | $28.78$ |

## 3. NetworkX Graphs

From this point forward, we use the `NetworkX` package.

### 3.1 Basic Graph

In this part of the notebook, we will be using the [`networkx` package](https://networkx.org) for "the creation, manipulation, and study of the structure, dynamics, and functions of complex networks."

```python
import networkx as nx
```

Let's create an empty networkx graph `G`.

```python
G = nx.Graph()
```

```python
type(G)
```

    networkx.classes.graph.Graph

In the next cell, we build a graph by adding edges and nodes.

```python
G.add_edge(1,2)
G.add_edge(1,4)
G.add_edge(2,4)
G.add_edge(4,3)
```

The `pos` variable below sets the position/layout of the nodes in the visualization. More often than not, this is only for aesthetics purposes. To see other layouts, check [this out](https://networkx.github.io/documentation/networkx-1.10/reference/drawing.html#layout).

```python
pos = nx.spring_layout(G);
nx.draw_networkx_labels(G, pos, font_color ='white');
nx.draw(G, pos = pos, node_size=500);
```

```python
G.add_node(5)

pos = nx.spring_layout(G);
nx.draw_networkx_labels(G, pos, font_color ='white');
nx.draw(G, pos = pos, node_size=500);
```

In the previous cell, we added a 5th node, but didn't specify the link(s) connected to it. Thus, we can observe that the node is disconnected.

In the next cell, let's add a link between the 4th and 5th nodes.

```python
G.add_edge(4,5)

nx.draw_networkx_labels(G, pos, font_color ='white');
nx.draw(G, pos = pos, node_size=500);
```

```python
pos = nx.spring_layout(G);
nx.draw_networkx_labels(G, pos, font_color ='white');
nx.draw(G, pos = pos, node_size=500);
```

```python
G = nx.Graph()

G.add_edge('Erika', 'Chris')
G.add_edge('Erika', 'Gino')
G.add_edge('Chris', 'Gino')
G.add_edge('Gino', 'Christian')

pos = nx.spring_layout(G);
nx.draw_networkx_labels(G, pos, font_color ='k');
nx.draw(G, pos = pos, node_size=500);
```

```python
G.nodes()
```

    NodeView(('Erika', 'Chris', 'Gino', 'Christian'))

```python
G.add_node('Arielle')
G.nodes()
```

    NodeView(('Erika', 'Chris', 'Gino', 'Christian', 'Arielle'))

Let's try removing nodes. Can we use an index in removing the nodes? Say, index 1?

```python
try:
    G.remove_node(1)
except Exception as e:
    print(e)

G.nodes()
```

    The node 1 is not in the graph.

    NodeView(('Erika', 'Chris', 'Gino', 'Christian', 'Arielle'))

To remove a node, we have to actually identify the node by its label/name. If this has not initially been introduced, the equivalent would be the numerical name.

```python
G.remove_node('Arielle')
G.nodes()
```

    NodeView(('Erika', 'Chris', 'Gino', 'Christian'))

It's easy to check the number of nodes and edges in the network.

```python
print ('Number of nodes: ', G.number_of_nodes())
print ('Number of edges: ', G.number_of_edges())

pos = nx.spring_layout(G);
nx.draw_networkx_labels(G, pos, font_color ='k');
nx.draw(G, pos = pos, node_size=500);
```

```python
list(G.neighbors('Gino'))
```

    ['Erika', 'Chris', 'Christian']

```python
G.add_nodes_from(['Ann','Faye','Mary'])

pos = nx.spring_layout(G);
nx.draw_networkx_labels(G, pos, font_color ='k');
nx.draw(G, pos = pos, node_size=500);
```

```python
G.add_edges_from([('Ann', 'Erika'),
                  ('Ann', 'Faye'),
                  ('Faye', 'Mary'),
                  ('Erika', 'Mary'),
                  ('Erika', 'Faye')])

pos = nx.spring_layout(G);
nx.draw_networkx_labels(G, pos, font_color ='k');
nx.draw(G, pos = pos, node_size=500);
```

```python
G.degree('Erika')
```

    5

```python
list(G.neighbors('Erika'))
```

    ['Chris', 'Gino', 'Ann', 'Mary', 'Faye']

### 3.2 Weighted Graphs and Other Edge Attributes

More often than not, in the real-world, links have weights. For example, there are certain friendships that are stronger than others. Another example is quantifying how often two individuals exchange emails at work. These "strengths" can be quantified by adding a weight attribute. In addition to the weight, we can also add other edge attributes as shown below where we added the attribute `relationship_type`, which could be "friends", "family", "professional", for example.

```python
G['Ann']['Faye']['weight'] = 8
G['Faye']['Mary']['weight'] = 10
G['Faye']['Mary']['relatioinship_type'] = 'friends'
```

```python
G.edges(data=True)
```

    EdgeDataView([('Erika', 'Chris', {}), ('Erika', 'Gino', {}), ('Erika', 'Ann', {}), ('Erika', 'Mary', {}), ('Erika', 'Faye', {}), ('Chris', 'Gino', {}), ('Gino', 'Christian', {}), ('Ann', 'Faye', {'weight': 8}), ('Faye', 'Mary', {'weight': 10, 'relatioinship_type': 'friends'})])

```python
try:
    G['Ann']['Mary']['weight'] = 5
except Exception as e:
    print (e)
```

    'Mary'

```python
G.add_edge('Ann', 'Mary', weight = 30)
```

```python
G.add_edge('Ann', 'Mary', relatioinship_type = 'siblings')
```

```python
G.edges(data=True)
```

    EdgeDataView([('Erika', 'Chris', {}), ('Erika', 'Gino', {}), ('Erika', 'Ann', {}), ('Erika', 'Mary', {}), ('Erika', 'Faye', {}), ('Chris', 'Gino', {}), ('Gino', 'Christian', {}), ('Ann', 'Faye', {'weight': 8}), ('Ann', 'Mary', {'weight': 30, 'relatioinship_type': 'siblings'}), ('Faye', 'Mary', {'weight': 10, 'relatioinship_type': 'friends'})])

```python
G.add_edge('Ann', 'Mary', friendship = 2)
G['Ann']['Faye']['friendship'] = 4
G.edges(data=True)
```

    EdgeDataView([('Erika', 'Chris', {}), ('Erika', 'Gino', {}), ('Erika', 'Ann', {}), ('Erika', 'Mary', {}), ('Erika', 'Faye', {}), ('Chris', 'Gino', {}), ('Gino', 'Christian', {}), ('Ann', 'Faye', {'weight': 8, 'friendship': 4}), ('Ann', 'Mary', {'weight': 30, 'relatioinship_type': 'siblings', 'friendship': 2}), ('Faye', 'Mary', {'weight': 10, 'relatioinship_type': 'friends'})])

```python
nx.write_edgelist(G, 'myG')
```

### 3.3 Directed Graphs

Many networks in the real-world are also directed. That is, the relationship between two entities has a particular direction. This directionality could represent, for example, who emailed whom, or who responded to whom on Twitter, or who follows whom on Twitter or TikTok, etc.

```python
dG = nx.DiGraph()
```

```python
dG.add_edges_from([('Ann', 'Mary'), ('Mary', 'Ann')])
```

```python
dG.add_weighted_edges_from([('Ann', 'Mary', 3),
                            ('Mary', 'Ann', 2)])
```

```python
dG.edges(data=True)
```

    OutEdgeDataView([('Ann', 'Mary', {'weight': 3}), ('Mary', 'Ann', {'weight': 2})])

```python
dG = nx.to_directed(G)
```

```python
dG.edges(data=True)
```

    OutEdgeDataView([('Erika', 'Chris', {}), ('Erika', 'Gino', {}), ('Erika', 'Ann', {}), ('Erika', 'Mary', {}), ('Erika', 'Faye', {}), ('Chris', 'Erika', {}), ('Chris', 'Gino', {}), ('Gino', 'Erika', {}), ('Gino', 'Chris', {}), ('Gino', 'Christian', {}), ('Christian', 'Gino', {}), ('Ann', 'Erika', {}), ('Ann', 'Faye', {'weight': 8, 'friendship': 4}), ('Ann', 'Mary', {'weight': 30, 'relatioinship_type': 'siblings', 'friendship': 2}), ('Faye', 'Ann', {'weight': 8, 'friendship': 4}), ('Faye', 'Mary', {'weight': 10, 'relatioinship_type': 'friends'}), ('Faye', 'Erika', {}), ('Mary', 'Faye', {'weight': 10, 'relatioinship_type': 'friends'}), ('Mary', 'Erika', {}), ('Mary', 'Ann', {'weight': 30, 'relatioinship_type': 'siblings', 'friendship': 2})])

### 3.4 Adjacency Matrix to NetworkX Graph

Important: Did you know that you can create a networkx graph from a numpy (adjacency) matrix?

```python
nxG = nx.from_numpy_matrix(adj_G)
nxG.degree
```

    DegreeView({0: 2, 1: 2, 2: 1, 3: 3})

In this new scheme, the node labels start at 0 such that in the drawing where Node 4 has degree 3, here in the networkx graph it is **Node 4 is Node 3** and it has degree 3.

## 4. Graph Properties

Now that we have constructed our graphs, how then do we describe them? This section provides some of the metrics to characterize network structures.

```python
pos = nx.spring_layout(G);
nx.draw_networkx_labels(G, pos, font_color ='k');
nx.draw(G, pos = pos, node_size=500);
```

```python
pos
```

    {'Erika': array([0.02706149, 0.10393789]),
     'Chris': array([ 0.28659159, -0.29094789]),
     'Gino': array([-0.03128256, -0.49965561]),
     'Christian': array([-0.16533555, -1.        ]),
     'Ann': array([0.04453496, 0.53095473]),
     'Faye': array([-0.14542026,  0.52833546]),
     'Mary': array([-0.01614966,  0.62737541])}

### 4.1 Network Statistics

1. Size - total number of edges/links
2. Order - total number of vertices/nodes
3. Average degree

```python
N = G.order()
L = G.size()

avg_deg = float(L) / N

print ('Nodes: ', N)
print ('Edges: ', L)
print ('Average degree: ', avg_deg)
```

    Nodes:  7
    Edges:  10
    Average degree:  1.4285714285714286

### 4.2 Degree Distribution
The degree distribution, as the term implies, is the **probability distribution** of all node degrees over the entire network.

```python
nx.degree(G)
```

    DegreeView({'Erika': 5, 'Chris': 2, 'Gino': 3, 'Christian': 1, 'Ann': 3, 'Faye': 3, 'Mary': 3})

```python
degrees = [k for node, k in nx.degree(G)]
print(degrees)
```

    [5, 2, 3, 1, 3, 3, 3]

```python
k_min = np.min(degrees)
k_max = np.max(degrees)
k_avg = np.mean(degrees)

print ('Minimum degree: ', k_min)
print ('Maximum degree: ', k_max)
print ('Average degree: ', k_avg)
```

    Minimum degree:  1
    Maximum degree:  5
    Average degree:  2.857142857142857

```python
p, k = np.histogram([k for n,k in nx.degree(G)], bins = 6);
plt.bar(k[1:],p);
```

### 4.3 Shortest Paths

```python
nx.average_shortest_path_length(G)
```

    1.6666666666666667

```python
nx.shortest_path(G, 'Erika', 'Mary')
```

    ['Erika', 'Mary']

```python
nx.shortest_path(G, 'Mary', 'Gino')
```

    ['Mary', 'Erika', 'Gino']

```python
nx.shortest_path_length(G, 'Mary', 'Gino')
```

    2

```python
nx.shortest_path_length(G, 'Mary', 'Christian')
```

    3

```python
list(nx.all_pairs_shortest_path_length(G))
```

    [('Erika',
      {'Erika': 0,
       'Faye': 1,
       'Mary': 1,
       'Ann': 1,
       'Chris': 1,
       'Gino': 1,
       'Christian': 2}),
     ('Chris',
      {'Chris': 0,
       'Gino': 1,
       'Erika': 1,
       'Faye': 2,
       'Mary': 2,
       'Ann': 2,
       'Christian': 2}),
     ('Gino',
      {'Gino': 0,
       'Christian': 1,
       'Chris': 1,
       'Erika': 1,
       'Faye': 2,
       'Mary': 2,
       'Ann': 2}),
     ('Christian',
      {'Christian': 0,
       'Gino': 1,
       'Chris': 2,
       'Erika': 2,
       'Faye': 3,
       'Mary': 3,
       'Ann': 3}),
     ('Ann',
      {'Ann': 0,
       'Faye': 1,
       'Mary': 1,
       'Erika': 1,
       'Chris': 2,
       'Gino': 2,
       'Christian': 3}),
     ('Faye',
      {'Faye': 0,
       'Ann': 1,
       'Mary': 1,
       'Erika': 1,
       'Chris': 2,
       'Gino': 2,
       'Christian': 3}),
     ('Mary',
      {'Mary': 0,
       'Faye': 1,
       'Ann': 1,
       'Erika': 1,
       'Chris': 2,
       'Gino': 2,
       'Christian': 3})]

### 4.4 Clustering Coefficient

The clustering coefficient essentially quantifies how ones "friends" are "friends" with each other.

```python
nx.clustering(G, 'Erika')
```

    0.4

```python
nx.clustering(G, 'Faye')
```

    1.0

```python
cc = nx.clustering(G)
avg_clust = sum(cc.values()) / len(cc)
print(avg_clust)
```

    0.6761904761904762

## 5. Exercises: Real-World Network

### 5.1 Zachary's Karate Club

Zachary, W. W. (1977), ‘An Information Flow Model for Conflict and Fission in Small Groups’, Journal of Anthropological Research 33, 452-473.

**Description**

This is the well-known and much-used Zachary karate club network. The data was collected from the members of a university karate club by Wayne Zachary in 1977. Each node represents a member of the club, and each edge represents a tie between two members of the club.

```python
K = nx.karate_club_graph()
```

```python
pos = nx.fruchterman_reingold_layout(K);

plt.figure(figsize=(12,10));
plt.axis("off");
nx.draw_networkx_nodes(K, pos, node_size=300, node_color="black");
nx.draw_networkx_edges(K, pos, alpha=0.500);
nx.draw_networkx_labels(K, pos, font_color="white");
plt.show();
```

#### 5.1.1 Network Structure Properties

```python

```

#### 5.1.2 Network Insights

### 5.2 Davis Southern women social network

Vladimir Batagelj and Andrej Mrvar (2006): Pajek datasets
http://vlado.fmf.uni-lj.si/pub/networks/data/.

**Description**

This is a data set of 18 women observed over a nine-month period. During that period, various subsets of these women met in a series of 14 informal social events. The data recored which women met for which events.

```python
DS = nx.davis_southern_women_graph()
```

```python
pos = nx.fruchterman_reingold_layout(DS);

plt.figure(figsize=(12,10));
plt.axis("off");
nx.draw_networkx_nodes(DS, pos, node_size=300, node_color="lightblue");
nx.draw_networkx_edges(DS, pos, alpha=0.500);
nx.draw_networkx_labels(DS, pos, font_color='black');
plt.show();

```

#### 5.2.1 Network Structure Properties

```python

```

#### 5.2.2 Network Insights

### 5.3 Florentine families

- Breiger R. and Pattison P. (1986). Cumulated social roles: The duality of persons and their algebras. Social Networks, 8, 215-256.
- Padgett, John F. 1994. Marriage and Elite Structure in Renaissance Florence, 1282-1500. Paper delivered to the Social Science History Association.

**Description**

This is a data set of marriage and business ties among Renaissance Florentine families.

```python
florentine = nx.florentine_families_graph()
```

```python
pos = nx.fruchterman_reingold_layout(florentine);

plt.figure(figsize=(12,10));
plt.axis("off");
nx.draw_networkx_nodes(florentine, pos, node_size=300, node_color="lightgreen");
nx.draw_networkx_edges(florentine, pos, alpha=0.500);
nx.draw_networkx_labels(florentine, pos, font_color='black');
plt.show();

```

#### 5.3.1 Network Structure Properties

```python

```

#### 5.3.2 Network Insights

### 5.4 MSDS 2020 Hangout Network
The MSDS 2020 Hangout Network was generated in class on 06 February 2020. The instruction given to the students was:

*"Who among your classmates do you most frequently hangout with [not necessarily for classwork] (choose 3 to 8)? Alternatively, who are those you enjoy hanging out with the most?"*

The dataset is anonymized.

#### 5.4.1 Network Structure Properties

```python

```

#### 5.3.2 Network Insights

```python

```

.ipynb_checkpoints/Intro to Network Analysis-checkpoint.ipynb


# Filtro de Kalman para estimar la inclinación del chasis

El objetivo es estimar lo mejor posible el ángulo de inclinación del chasis $\theta$, utilizando únicamente la información de tres sensores: los ejes $x$ y $z$ del acelerómetro, y el eje $y$ del giróscopo. En este caso, podríamos conseguir una estimación más precisa utilizando el modelo completo del sistema, ya que disponemos también de *encoders* en los motores, pero eso requeriría programar un observador de estado innecesariamente complejo para este proyecto. Si se considera el sensor inercial como un sistema aislado, se puede obtener un filtro de Kalman muy sencillo, que proporciona una estimación del ángulo perfectamente válida para controlar el robot. Al no tener en cuenta el movimiento del sistema completo, tendremos que suponer que el acelerómetro sólo mide la aceleración de la gravedad, tratando cualquier otra aceleración como ruido del sensor.

## Modelo del sistema

El filtro de Kalman que utilizaremos tiene un solo estado, que es el ángulo de inclinación $\theta$, con un *input*, que es la velocidad angular del chasis $\omega_y$, y dos sensores, que son las aceleraciones según los ejes $x$ y $z$ locales. La ecuación de propagación de estados en forma discreta, para un paso de tiempo constante $\Delta t$, tendrá entonces la siguiente forma, con $A = \left[1\right]$ y $B = \left[\Delta t\right]$:

$$
    \theta_{n+1} = A\theta_n + B\left(\omega_{yn} + v_n\right)
$$

El término $v$ es el ruido del giróscopo, que consideraremos ruido blanco Gaussiano de varianza $\sigma_v^2$. Al introducir el giróscopo como *input*, nos evitamos tener que duplicar estados: si el giróscopo se tratase como un sensor, el vector de estados tendría que contener también la velocidad angular $\dot\theta$, porque sería necesaria para la función de observación $\mathbf h$.

Si partimos del supuesto de que el acelerómetro sólo percibe la gravedad, podemos establecer la siguiente expresión para la función de observación, considerando aceleraciones medidas en $g$:

$$
    \mathbf{h}_n = \begin{bmatrix}\cos\theta_n \\ \sin\theta_n\end{bmatrix} + \mathbf w_n
$$

donde $\mathbf w$ es un vector que contiene el ruido de los dos acelerómetros. Aunque no sea estrictamente cierto, porque se están incluyendo en $\mathbf w$ las aceleraciones debidas al movimiento del sensor, asumiremos que ambas señales contienen ruido blanco Gaussiano de varianza $\sigma_a^2$. La función $\mathbf h$ es no lineal, así que tendremos que utilizar un EKF, donde la matriz Jacobiana de la función de observación será:

$$
    \mathbf{H}_n = \begin{bmatrix}-\sin\theta_n \\ \cos\theta_n\end{bmatrix}
$$

La covarianza del ruido del sistema, teniendo en cuenta que el ruido del giróscopo va afectado por una ganancia $\Delta t$ en la ecuación de estados, es:

$$
    Q = \Delta_t^2\sigma_v^2
$$

Y la matriz de covarianza del ruido de los acelerómetros, si suponemos que son independientes entre sí, es:

$$
    \mathbf R = \mathbf I_2\sigma_a^2
$$

donde $\mathbf I_2$ es una matriz identidad de 2 x 2.

```python
from sympy import symbols, Symbol, Matrix, sin, cos, solve, simplify, eye, sqrt

theta, ax, az, gy, P = symbols('theta a_x a_z omega_y P')
sv, sa = symbols('sigma_v sigma_a', real=True, positive=True)
dt = Symbol('\Delta t', real=True, positive=True)

A = Matrix([1])                          # Matriz de propagación de estados
B = Matrix([dt])                         # Matriz de entrada

z = Matrix([[ax], [az]])                 # Vector de sensores (ejes x y z del acelerómetro)
h = Matrix([[cos(theta)], [sin(theta)]]) # Función de sensores
H = h.diff(theta)                        # Matriz Jacobiana de la función de sensores

Q = Matrix([(dt*sv)**2])                 # Matriz de covarianza del ruido del sistema
R = eye(2)*sa**2                         # Matriz de covarianza del ruido de los sensores
```

## Filtro de Kalman Extendido (EKF)

Las ecuaciones del EKF serán, teniendo en cuenta que en nuestro caso $A$, $B$, $Q$ y $\mathbf R$ son constantes:

 * Estimación *a priori* del estado:

 $$
     \hat\theta_{n+1}^- = A\hat\theta_n + B\omega_{yn}
 $$

 * Valor *a priori* de la covarianza de la estimación:

 $$
    P_{n+1}^- = AP_nA^{\mathsf T} + Q
 $$

 * Ganancia del filtro:

 $$
    \mathbf K_{n+1} = P_{n+1}^-\mathbf H_{n+1}^{-\mathsf T}\left(\mathbf H_{n+1}^-P_{n+1}^-\mathbf H_{n+1}^{-\mathsf T} + \mathbf R\right)^{-1}
 $$

 * Corrección para obtener la estimación *a posteriori* del estado:

 $$
     \hat\theta_{n+1} = \hat\theta_{n+1}^- + \mathbf K_{n+1}\left(\mathbf z_{n+1} - \mathbf h_{n+1}^-\right)
 $$

 * Valor *a posteriori* de la covarianza de la estimación:

 $$
     P_{n+1} = \left(1 - \mathbf K_{n+1}\mathbf H_{n+1}^-\right)P_{n+1}^-
 $$

## Filtro simplificado aprovechando la convergencia de $P$

En este filtro, se puede comprobar que la covarianza de la estimación del estado $P$ converge a un valor constante al cabo de unas cuantas iteraciones. Este hecho nos va a resultar útil para simplificar el algoritmo y hacerlo más eficiente, como se verá a continuación. Lo primero que tenemos que hacer es expresar $P_{n+1}$ como función de $P_n$:

```python
# Estimación a priori
P1 = A*P*A.T + Q

# Ganancia del filtro
K = P1*H.T*(H*P1*H.T + R).inv()

# Estimación a posteriori
P2 = simplify((eye(1) - K*H)*P1)
```

$$
    P_{n+1} = \frac{\sigma_a^2\left(P_n + \Delta t^2\sigma_v^2\right)}{P_n + \Delta t^2\sigma_v^2 + \sigma_a^2}
$$

Supondremos que, cuando $n$ sea suficientemente grande, $P$ convergerá a un valor constante $P_\infty$. Eso quiere decir que tanto $P_n$ como $P_{n+1}$ alcanzarán dicho valor, así que podemos sustituirlos en la ecuación anterior, y resolverla después para obtener $P_\infty$:

```python
# Convergencia: P(n) = P(n + 1) si 'n' tiende a infinito
Pinf = solve(P - P2[0], P)[0]
```

La ecuación tiene solución real, de modo que, efectivamente, se cumple que $P$ converge a un valor constante $P_\infty$:

$$
    P_\infty = \frac{1}{2}\left(\sqrt{\Delta t^4\sigma_v^4 + 4\Delta t^2\sigma_v^2\sigma_a^2} - \Delta t^2\sigma_v^2\right)
$$

Ahora estudiamos la estimación *a posteriori* del estado, mirando qué forma toma la corrección:

```python
# Corrección del estado
dx = simplify(K*(z - h))
```

$$
    \mathbf K_{n+1}\left(\mathbf z_{n+1} - \mathbf h_{n+1}^-\right) = \frac{P_n + \Delta t^2\sigma_v^2}{P_n + \Delta t^2\sigma_v^2 + \sigma_a^2}\left(\bar a_{zn+1}\cos\hat\theta_{n+1}^- - \bar a_{xn+1}\sin\hat\theta_{n+1}^-\right)
$$

Cuando el filtro converge, $P_n$ alcanza el valor constante $P_\infty$, de modo que podemos expresar la corrección como:

$$
    \mathbf K_{n+1}\left(\mathbf z_{n+1} - \mathbf h_{n+1}^-\right) = C_\infty\left(\bar a_{zn+1}\cos\hat\theta_{n+1}^- - \bar a_{xn+1}\sin\hat\theta_{n+1}^-\right)
$$

donde la constante $C_\infty$ es una función de las varianzas y el paso de tiempo:

$$
    C_\infty = \frac{P_\infty + \Delta t^2\sigma_v^2}{P_\infty + \Delta t^2\sigma_v^2 + \sigma_a^2} = \frac{P_\infty}{\sigma_a^2}
$$

Esta expresión se puede linealizar para pasos de tiempo pequeños, resultando:

```python
Cinf = Pinf/sa**2
C0 = Cinf.subs(dt, 0) + Cinf.diff(dt).subs(dt, 0)*dt
```

$$
    C_\infty \approx \frac{\sigma_v}{\sigma_a}\Delta t
$$

Se puede comprobar que, cuando $\sigma_v$ es menor que $\sigma_a$, esta aproximación mantiene su validez para pasos de tiempo bastante elevados. Esto es precisamente lo que ocurre en nuestro sistema: $\sigma_v^2$ es la varianza del ruido del giróscopo, que debería ser pequeña, mientras que $\sigma_a^2$ incluye las aceleraciones no debidas a la gravedad, que serán de una magnitud mucho mayor que el ruido propio de los sensores.

## Implementación en el microcontrolador

En la práctica, no necesitaremos conocer las varianzas del ruido del giróscopo y los acelerómetros. Bastará con establecer que el parámetro $C_\infty$ es igual a una constante $K$ multiplicada por el paso de tiempo:

$$
    C_\infty \approx K\Delta t
$$

La constante $K$ es un parámetro que ajustaremos manualmente para conseguir el comportamiento deseado: un valor bajo de $K$ dará mayor peso al giróscopo, mientras que un valor elevado aplicará con mas intensidad la corrección de los acelerómetros.

En resumen, el filtro de Kalman que se utilizará para estimar el ángulo de inclinación del robot consistirá simplemente en dos operaciones:

$$
\begin{align}
    \hat\theta_{n+1}^- &= \hat\theta_n + \omega_{yn}\Delta t \\
    \hat\theta_{n+1} &= \hat\theta_{n+1}^- + K\Delta t\left(\bar a_{zn+1}\cos\hat\theta_{n+1}^- - \bar a_{xn+1}\sin\hat\theta_{n+1}^-\right)
\end{align}
$$

En cada vuelta del bucle de control, primero se integra un paso de tiempo utilizando la lectura del giróscopo, y luego se aplica la corrección usando las aceleraciones.


```python
import sys
sys.path.append("/miniconda3/lib/python3.7/site-packages")
```

```python
import numpy as np
from sympy import *
import matplotlib.pyplot as plt
```

```python
d,f,m,α,β=sympy.symbols('d,f,m,α,β')
λ=(d/m)*(sin(α)+sin(β))
D=1/(diff(λ,β))
D
```

$\displaystyle \frac{m}{d \cos{\left(β \right)}}$

```python
x=np.arange(0,3.14,.1)
y=np.cos(x)
plt.plot(x,y)
plt.xlabel('Diffraction angle β (rad)', fontsize=14)
plt.ylabel('Spectral resolution Δλ (arb)',fontsize=14)
plt.title('Conceptual relationship between β and Δλ',fontsize=14)
```

```python
x=np.arange(.1,2,.01)
y=(1/x)
plt.plot(x,y)
plt.xlabel('Focal length f (arb)',fontsize=14)
plt.ylabel('Spectral resolution Δλ (arb)',fontsize=14)
plt.title('Conceptual relationship between f and Δλ',fontsize=14)
```

```python

```


# Analyzing World Series Victories

## Learning more about programming in python

If you want to learn more about programming in python on your own, I encourage you to go through the first 10-11 lectures of the free online introductory class here:https://github.com/agdelma/IntroCompPhysics.

### Notebook Basics

We can type command directly into the execution cells.  This can act like a simple calculator, or we can add more complex code.  To execute a cell, press the ▶ button on the tool bar, or hit `shift+Enter`.

#### An Interactive Calculator

```python
1+1
```

#### Your First Program
Writing a `hello world` program is one line!

```python
print('Hello World')
```

### Assignment

The assignment operator in Python is =. Python is a dynamically typed language, so we do not need to specify the type of a variable when we create one.

```python
answer = input('Enter some exciting text: ')
print(f'Your exciting text was: {answer}')
```

#### Loops and Control

These are the building blocks of any program in python. We can make the computer repeat things or perform actions based on the values of variables

```python
num_repeat = input('How many times should I repeat your exciting text? ')

if num_repeat.isnumeric():
    for n in range(int(num_repeat)):
        print(answer)
else:
    print("You didn't enter a valid number!")
```

## We are ready to begin our analysis of World Series Data!

We need to import some libraries to perform our analysis.  Libraries are software written by someone else that you can build programs out of!

```python
import pandas as pd
from scipy.special import binom
import matplotlib.pyplot as plt
import numpy as np

red,lime,blue,green,purple = "#CF5970","#94B848","#878CC0","#5EBAA1","#BD5DC7"
%config InlineBackend.figure_formats = ['svg']
```

## Take a peak at the World Series Data

```python
ws = pd.read_csv('world_series_data.csv')
ws.tail(10)
```

## Distribution of wins by team: American League

```python
plt.figure(figsize=(10,5))
h = ws["American League Team"].value_counts().plot.bar(color=red)
h.set_ylabel('World Series Appearances')
h.set_title('American League')
```

## Distribution of wins by team: National League

```python
plt.figure(figsize=(10,5))
h = ws["National League Team"].value_counts().plot.bar(color=blue)
h.set_ylabel('World Series Appearances')
h.set_title('National League')
```

## Are teams from one league better than the other?

The *probability of winning* based on the available data is given by:

\begin{equation}
P(\text{League Win}) = \frac{\text{Num. of Wins}}{\text{Total Num. Games}}
\end{equation}

```python
total_number_games = ws["Total Games"].sum()
al_wins = ws["AL Wins"].sum()
nl_wins = ws["NL Wins"].sum()

print(f'AL Win Probability: {100*al_wins/total_number_games:.2f}%')
print(f'NL Win Probability: {100*nl_wins/total_number_games:.2f}%')
```

Remember, probability is the likliehood **on average** of a given outcome occuring.

### It seems that the American League is slightly better.  Any ideas why?

## What is the probability of each World Series outcome?

In a *best of seven* series there are 4 possible outcomes:

* 4-0 (sweep)
* 4-1
* 4-2
* 4-3

These correspond to 4, 5, 6, and 7 games respectively.

Each of these outcomes could happen in 2 distinct ways if we neglect ordering, *i.e.* either the American League or National Team could win the world series.

We can quantify this by counting how may times each of the oucomes appears and dividing by the total number of series. The result is called a **probability histogram**

```python
plt.figure(figsize=(10,5))
h = ws['Total Games'].hist(range=[4,7],bins=[4,5,6,7,8],align='left',rwidth=0.9, density=True, color=green)
plt.xticks(np.arange(4,8), ('4-0', '4-1', '4-2', '4-3') )

# add a title to the current figure, our histogram
h.set_title("Outcome of World Series")
h.set_xlabel("Series Score")
h.set_ylabel("Probability")

# record the data for future plotting
ws_actual,binEdges = np.histogram(ws['Total Games'],bins=4, density=True)
ws_actual_prob = pd.Series(ws_actual, index=['4-0','4-1','4-2','4-3'], name='World Series')
```

## Questions
 * Can we undertsand this distribution?
 * Why do most series go to the full 7 games?
 * Why is 4-0 (a sweep) the most unlikely outcome?

## Let's simplify this problem and analyze/collect our own data for a *model* system

### Activity: Odds-n-Evens

 1. Get into groups of 3 (2 active players 1 umpire)
 2. There are 3 possible *series* per group
 3. For each series, decide who is American League (**EVENS**) and who is National League (**ODDS**)
 3. Play **best 4 out of 7** games of O/E.
 5. The umpire needs to record (1) the **winner (A or N)** and (2) the **length** of the series (4, 5, 6, or 7 games).

```python
oe_game_lengths = [4]
oe_game_winners = ['E']

data = {'Series Length':oe_game_lengths, 'Winners':oe_game_winners}
odds_evens = pd.DataFrame(data)
```

```python
plt.figure(figsize=(10,5))
h = odds_evens['Series Length'].hist(range=[4,7],bins=[4,5,6,7,8],align='left',rwidth=0.9, density=True, color=green)
plt.xticks(np.arange(4,8), ('4-0', '4-1', '4-2', '4-3') )

h.set_title("Outcome of Odds/Evens Series")
h.set_xlabel("Series Score")
h.set_ylabel("Probability")

# Record the probabilities
oe_actual,binEdges = np.histogram(odds_evens['Series Length'],bins=4, density=True)
oe_actual_prob = pd.Series(oe_actual, index=['4-0','4-1','4-2','4-3'], name='Odds & Evens')
```

```python
print(oe_actual_prob,'\n')
print(ws_actual_prob)
```

### Is there anything special about the American vs. National League in our Data?

```python
h = odds_evens['Winners'].value_counts().plot.bar(color=blue,xlabel='League', ylabel='Number of Wins')
```

## Question: How can we compute the probability and see if it agrees with our experiment?

<!--
**Answer**: First enumerate the number of outcomes in each case:
  * 4-0: AAAA, NNNN
  * 4-1: NAAAA, ANAAA, AANAA, AAANA -- One party needs to win **at most** 1 out of the first 4 games
-->

## We can write code to enumerate these for us!

```python
import itertools as it
outcome,prob,combos = {},{},{}
```

### 4-0 Sweep

```python
outcome['4-0'] = ["AAAA","NNNN"]
```

### 4-1

```python
# AL wins 4-1
score = set(it.permutations("AAAN"))
outcome['4-1'] = [ "".join(c) + "A" for c in score]

# NL wins 4-1
score = set(it.permutations("NNNA"))
outcome['4-1'] += [ "".join(c) + "N" for c in score]
```

### 4-2

```python
# AL wins 4-2
score = set(it.permutations("AAANN"))
outcome['4-2'] = [ "".join(c) + "A" for c in score]

# NL wins 4-2
score = set(it.permutations("NNNAA"))
outcome['4-2'] += [ "".join(c) + "N" for c in score]
```

### 4-3

```python
# AL wins 4-3
score = set(it.permutations("AAANNN"))
outcome['4-3'] = [ "".join(c) + "A" for c in score]

# NL wins 4-3
score = set(it.permutations("NNNAAA"))
outcome['4-3'] += [ "".join(c) + "N" for c in score]
```

## Now let's determine the probabilities

 * We have to remember that the probability to win a single game is equal to 1/2
 * Therefore, the probability of given outcome is equal to the number of ways for that score to happen, divided by: $2^{\text{number of games}}$

### List all the posibile combinations

```python
for score,out in outcome.items():
    combos[score] = pd.DataFrame({score:out})
```

```python
combos['4-0']
```

```python
combos['4-1']
```

```python
combos['4-2']
```

```python
combos['4-3']
```

### To determine the probability:

$$ \text{Prob(outcome)} = \frac{\text{number outcomes}}{2^{\text{number games}}} $$

```python
for score,out in outcome.items():
    prob[score] = 1.0*len(out)/(2**(len(out[0])))
exact_prob = pd.Series(prob, name='Exact Prob.')
```

### Let's compare the exact probability we have measured, with the observed

```python
exact_prob
```

```python
ws_actual_prob
```

```python
oe_actual_prob
```

### These look pretty close, let's compare on a plot

```python
plt.figure(figsize=(10,5))
exact_prob.plot.bar(alpha=1, color=red, label='Exact', width=0.25, position=0.5)
ws_actual_prob.plot.bar(alpha=0.75, color=green, label='World Series', width=0.25, position=0)
oe_actual_prob.plot.bar(alpha=0.75, color=blue, label='Odds/Evens',width=0.25, position=1)

plt.title("Outcome of World Series")
plt.xlabel("Game Score")
plt.ylabel("Probability")
plt.legend(loc='upper left', fontsize=15)
```

# Success!

### It turns out, this is an example of the binomial probability distribution for the special case where each of the two outcomes is equally likely.

The binomial distribution is the discrete probability distribution of the number of successes in a sequence of n independent yes/no experiments, each of which yields success with probability p.

See more: http://en.wikipedia.org/wiki/Binomial_distribution

$$P_b(n,k) = \binom{n}{k} \frac{1}{2^n}$$

Let's try to reproduce our previous calculation.  In our case, we will use the binomial probability for the first $n-1$ games of a $n$ game series, then multiply by $1/2$ the probability of our team winning the final game and finally, multiply the whole thing by $2$ as we don't care whether the AL or NL wins.  Thus, our probability is:

$$P(a,0) = P_b(a,0) \times 2 $$
$$P(a,b>0) = P_b(a+b-1,3) \times \frac{1}{2} \times 2 $$

```python
def P(a,b):
    ''' The probability for a series outcome a-b. '''
    if b == 0:
        return (binom(a,b)/2**a) * 2
    else:
        return binom(a+b-1,3)/2**(a+b-1)
```

```python
exact_prob
```

```python
for b in range(4):
    print (f'P(4-{b}) = {P(4,b)}')
```

```python
import scipy,matplotlib
print(np.__version__)
print(scipy.__version__)
print(pd.__version__)
print(matplotlib.__version__)
```


# Non Linear Regression Analysis

If the data shows a curvy trend, then linear regression will not produce very accurate results when compared to a non-linear regression because, as the name implies, linear regression presumes that the data is linear.
Let's learn about non linear regressions and apply an example on python. In this notebook, we fit a non-linear model to the datapoints corrensponding to China's GDP from 1960 to 2014.

### Importing required libraries

```python
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
```

Though Linear regression is very good to solve many problems, it cannot be used for all datasets. First recall how linear regression, could model a dataset. It models a linear relation between a dependent variable y and independent variable x. It had a simple equation, of degree 1, for example y = 2*(x) + 3.

```python
x = np.arange(-5.0, 5.0, 0.1)

##You can adjust the slope and intercept to verify the changes in the graph
y = 2*(x) + 3
y_noise = 2 * np.random.normal(size=x.size)
ydata = y + y_noise
#plt.figure(figsize=(8,6))
plt.plot(x, ydata,  'bo')
plt.plot(x,y, 'r')
plt.ylabel('Dependent Variable')
plt.xlabel('Indepdendent Variable')
plt.show()
```

Non-linear regressions are a relationship between independent variables $x$ and a dependent variable $y$ which result in a non-linear function modeled data. Essentially any relationship that is not linear can be termed as non-linear, and is usually represented by the polynomial of $k$ degrees (maximum power of $x$).

$$ \ y = a x^3 + b x^2 + c x + d \ $$

Non-linear functions can have elements like exponentials, logarithms, fractions, and others. For example: $$ y = \log(x)$$

Or even, more complicated such as :
$$ y = \log(a x^3 + b x^2 + c x + d)$$

Let's take a look at a cubic function's graph.

```python
x = np.arange(-5.0, 5.0, 0.1)

##You can adjust the slope and intercept to verify the changes in the graph
y = 1*(x**3) + 1*(x**2) + 1*x + 3
y_noise = 20 * np.random.normal(size=x.size)
ydata = y + y_noise
plt.plot(x, ydata,  'bo')
plt.plot(x,y, 'r')
plt.ylabel('Dependent Variable')
plt.xlabel('Indepdendent Variable')
plt.show()
```

As you can see, this function has $x^3$ and $x^2$ as independent variables. Also, the graphic of this function is not a straight line over the 2D plane. So this is a non-linear function.

Some other types of non-linear functions are:

### Quadratic

$$ Y = X^2 $$

```python
x = np.arange(-5.0, 5.0, 0.1)

##You can adjust the slope and intercept to verify the changes in the graph

y = np.power(x,2)
y_noise = 2 * np.random.normal(size=x.size)
ydata = y + y_noise
plt.plot(x, ydata,  'bo')
plt.plot(x,y, 'r')
plt.ylabel('Dependent Variable')
plt.xlabel('Indepdendent Variable')
plt.show()
```

### Exponential

An exponential function with base c is defined by $$ Y = a + b c^X$$ where b ≠0, c > 0 , c ≠1, and x is any real number. The base, c, is constant and the exponent, x, is a variable.

```python
X = np.arange(-5.0, 5.0, 0.1)

##You can adjust the slope and intercept to verify the changes in the graph

Y= np.exp(X)

plt.plot(X,Y)
plt.ylabel('Dependent Variable')
plt.xlabel('Indepdendent Variable')
plt.show()
```

### Logarithmic

The response $y$ is a results of applying logarithmic map from input $x$'s to output variable $y$. It is one of the simplest form of __log()__: i.e. $$ y = \log(x)$$

Please consider that instead of $x$, we can use $X$, which can be polynomial representation of the $x$'s. In general form it would be written as
\begin{equation}
y = \log(X)
\end{equation}

```python
X = np.arange(-5.0, 5.0, 0.1)

Y = np.log(X)

plt.plot(X,Y)
plt.ylabel('Dependent Variable')
plt.xlabel('Indepdendent Variable')
plt.show()
```

### Sigmoidal/Logistic

$$ Y = a + \frac{b}{1+ c^{(X-d)}}$$

```python
X = np.arange(-5.0, 5.0, 0.1)

Y = 1-4/(1+np.power(3, X-2))

plt.plot(X,Y)
plt.ylabel('Dependent Variable')
plt.xlabel('Indepdendent Variable')
plt.show()
```

# Non-Linear Regression example

For an example, we're going to try and fit a non-linear model to the datapoints corrensponding to China's GDP from 1960 to 2014. We download a dataset with two columns, the first, a year between 1960 and 2014, the second, China's corresponding annual gross domestic income in US dollars for that year.

```python
import numpy as np
import pandas as pd

#downloading dataset
!wget -nv -O china_gdp.csv https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/china_gdp.csv

df = pd.read_csv("china_gdp.csv")
df.head(10)
```

    2020-06-16 03:12:27 URL:https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/china_gdp.csv [1218/1218] -> "china_gdp.csv" [1]

      Year
      Value

      0
      1960
      5.918412e+10

      1
      1961
      4.955705e+10

      2
      1962
      4.668518e+10

      3
      1963
      5.009730e+10

      4
      1964
      5.906225e+10

      5
      1965
      6.970915e+10

      6
      1966
      7.587943e+10

      7
      1967
      7.205703e+10

      8
      1968
      6.999350e+10

      9
      1969
      7.871882e+10

__Did you know?__ When it comes to Machine Learning, you will likely be working with large datasets. As a business, where can you host your data? IBM is offering a unique opportunity for businesses, with 10 Tb of IBM Cloud Object Storage: [Sign up now for free](http://cocl.us/ML0101EN-IBM-Offer-CC)

### Plotting the Dataset ###
This is what the datapoints look like. It kind of looks like an either logistic or exponential function. The growth starts off slow, then from 2005 on forward, the growth is very significant. And finally, it deaccelerates slightly in the 2010s.

```python
plt.figure(figsize=(8,5))
x_data, y_data = (df["Year"].values, df["Value"].values)
plt.plot(x_data, y_data, 'ro')
plt.ylabel('GDP')
plt.xlabel('Year')
plt.show()
```

### Choosing a model ###

From an initial look at the plot, we determine that the logistic function could be a good approximation,
since it has the property of starting with a slow growth, increasing growth in the middle, and then decreasing again at the end; as illustrated below:

```python
X = np.arange(-5.0, 5.0, 0.1)
Y = 1.0 / (1.0 + np.exp(-X))

plt.plot(X,Y)
plt.ylabel('Dependent Variable')
plt.xlabel('Indepdendent Variable')
plt.show()
```

The formula for the logistic function is the following:

$$ \hat{Y} = \frac1{1+e^{\beta_1(X-\beta_2)}}$$

$\beta_1$: Controls the curve's steepness,

$\beta_2$: Slides the curve on the x-axis.

### Building The Model ###
Now, let's build our regression model and initialize its parameters.

```python
def sigmoid(x, Beta_1, Beta_2):
     y = 1 / (1 + np.exp(-Beta_1*(x-Beta_2)))
     return y
```

Lets look at a sample sigmoid line that might fit with the data:

```python
beta_1 = 0.10
beta_2 = 1990.0

#logistic function
Y_pred = sigmoid(x_data, beta_1 , beta_2)

#plot initial prediction against datapoints
plt.plot(x_data, Y_pred*15000000000000.)
plt.plot(x_data, y_data, 'ro')
```

Our task here is to find the best parameters for our model. Lets first normalize our x and y:

```python
# Lets normalize our data
xdata =x_data/max(x_data)
ydata =y_data/max(y_data)
```

#### How we find the best parameters for our fit line?
we can use __curve_fit__ which uses non-linear least squares to fit our sigmoid function, to data. Optimal values for the parameters so that the sum of the squared residuals of sigmoid(xdata, *popt) - ydata is minimized.

popt are our optimized parameters.

```python
from scipy.optimize import curve_fit
popt, pcov = curve_fit(sigmoid, xdata, ydata)
#print the final parameters
print(" beta_1 = %f, beta_2 = %f" % (popt[0], popt[1]))
```

Now we plot our resulting regresssion model.

```python
x = np.linspace(1960, 2015, 55)
x = x/max(x)
plt.figure(figsize=(8,5))
y = sigmoid(x, *popt)
plt.plot(xdata, ydata, 'ro', label='data')
plt.plot(x,y, linewidth=3.0, label='fit')
plt.legend(loc='best')
plt.ylabel('GDP')
plt.xlabel('Year')
plt.show()
```

## Practice
Can you calculate what is the accuracy of our model?

```python
# write your code here

```

Double-click __here__ for the solution.

## Want to learn more?

IBM SPSS Modeler is a comprehensive analytics platform that has many machine learning algorithms. It has been designed to bring predictive intelligence to decisions made by individuals, by groups, by systems – by your enterprise as a whole. A free trial is available through this course, available here: [SPSS Modeler](http://cocl.us/ML0101EN-SPSSModeler).

Also, you can use Watson Studio to run these notebooks faster with bigger datasets. Watson Studio is IBM's leading cloud solution for data scientists, built by data scientists. With Jupyter notebooks, RStudio, Apache Spark and popular libraries pre-packaged in the cloud, Watson Studio enables data scientists to collaborate on their projects without having to install anything. Join the fast-growing community of Watson Studio users today with a free account at [Watson Studio](https://cocl.us/ML0101EN_DSX)

### Thanks for completing this lesson!

Notebook created by: Saeed Aghabozorgi

Copyright © 2018 [Cognitive Class](https://cocl.us/DX0108EN_CC). This notebook and its source code are released under the terms of the [MIT License](https://bigdatauniversity.com/mit-license/).


```python
import sys
sys.path.append("../src")
import pandas as pd
import pickle
from sympy.parsing.sympy_parser import parse_expr
from sympy.parsing.sympy_parser import standard_transformations, convert_xor,implicit_multiplication
from utils import solve_eq_string, is_number
from text_to_template import number_parsing

%load_ext autoreload
%autoreload 2
```

### Task details and expectations:

**input**  - math problem in free text
**output** - numerical solution
**helpers** -
#### text template generation module
takes specific problem and factors out the numbers. The new representation generalizes the original problem
#### symbolic solver module
gets symbolic equation and generates the numerical solution

# Load data

```python
math_train = pd.read_json(r"../data/dev_data.json")
math_train[['ans', 'ans_simple', 'equations','text']].head()
```

      ans
      ans_simple
      equations
      text

      0
      {40;29}
      [40, 29]
      [unkn: x,y, equ: x=y+11, equ: 3*x=4*y+4]
      one number is 11 more than another number. Fin...

      1
      6; 9
      [6, 9]
      [unkn: x,y, equ: x + 3 = y, equ: 2*y + 12 = 5*x]
      One number is 3 less than a second number. Twi...

      2
      {34; 28}
      [34, 28]
      [unkn: x,y, equ: x+y=62, equ: x-y=6]
      Find two numbers whose sum is 62 and whose dif...

      3
      {42;26}
      [42, 26]
      [unkn: x,y, equ: x+y=68, equ: x-y=16]
      the sum of two numbers is 68. their difference...

      4
      {77; 20}
      [77, 20]
      [unkn: x,y, equ: x+y=97, equ: x-y=57]
      the sum of two numbers is 97. the difference o...

Example for verbal math problem

```python
sample = math_train.iloc[1]
```

```python
sample['text']
```

    'One number is 3 less than a second number. Twice the second number is 12 less than 5 times the first. Find the two numbers.'

The Requested equation and its solution

```python
sample[['equations','ans']]
```

    equations    [unkn: x,y, equ: x + 3 = y, equ: 2*y + 12 = 5*x]
    ans                                                      6; 9
    Name: 1, dtype: object

# Mapping free text to template
You get the equations and the text and you suppose to (see code in `number parsing`):
+ Code in `list_number_mapper`
    + Extract the numbers in the equations
    + Create template equations from math equations
+ Code in `number_mapper`
    + Extract the numbers from text
    + Create text template - replaces numbers with symbolics
    + Create list of symbols that correspond to the extracted numbers
+ Code in `generate_new_equation`
    + Reformats the equations - Replaces the symbolics in the template equations with numbers

```python
equation_list_template, eq_num_list, text_template, var_list, text_num_list =\
                number_parsing(sample['equations'], sample['text'])
```

```python
print(f"** original text **:\t{sample['text']}\n"+
    f"\n** equation list:\t{sample['equations']}\n"+
    f"\n** numbers in text **:\t{text_num_list}\n\n"+
    f"{'_'*30}--OUTPUT---{'_'*30}\n"+
    f"\n** template text **:\t{text_template}\n"+
    f"\n** eq list template **:\t{equation_list_template}\n"+
    f"\n** numlist from eq **:\t{eq_num_list}\n"+
    f"\n** symbol list **:\t{var_list}")
```

    ** original text **:	One number is 3 less than a second number. Twice the second number is 12 less than 5 times the first. Find the two numbers.

    ** equation list:	['unkn: x,y', 'equ: x + 3 = y', 'equ: 2*y + 12 = 5*x']

    ** numbers in text **:	['3', '2', '12', '5']

    ______________________________--OUTPUT---______________________________

    ** template text **:	one number is $n0 less than a second number $n1 the second number is $n2 less than $n3 times the first find the two numbers

    ** eq list template **:	['unkn: x,y', 'equ: x + $n0 = y ', 'equ: $n1*y + $n2 = $n3*x ']

    ** numlist from eq **:	['3', '2', '12', '5']

    ** symbol list **:	['$n0', '$n1', '$n2', '$n3']

# Solving symbolic equations
`solve_eq_string` gets as input the equations in the same format as the raw data. Another flag is passed to constrain the problem for integer solution.
The returned value is the solution in `float` format.
The function uses `sympy` and `wolfram` engines

```python
sample["equations"]
```

    ['unkn: x,y', 'equ: x + 3 = y', 'equ: 2*y + 12 = 5*x']

```python
answer = sample['ans']
solution = solve_eq_string(sample["equations"], integer_flag= is_number(sample["text"]))
print(f"answer:\t {answer}\nsolution:\t {solution}")
```

    answer:	 6; 9
    solution:	 [6.00000000000000, 9.00000000000000]

# What do I need to do?
Implement a model ,which is a child of the `ExampleModel` class. Use the `score` function to score your model.
You can test your solution with the evaluation script. It should look something like this

```python
from models.models import ExampleModel

math_train = pd.read_json(r"../data/dev_data.json")
math_test = pd.read_json(r"../data/test_data.json")

## model
model = ExampleModel()
model.fit(math_train)

## print evaluation result
print(f'result score on train: {model.score(math_train,frac=0.1,verbose=False,use_ans=True)}')
print(f'result score on test: {model.score(math_test,frac=1,verbose=True,use_ans=True)}')
```

Results will be printed in the following manner

```
result score on train: 0.861315

0 1 0.14285714285714285
0 2 0.16666666666666666
0 3 0.15384615384615385
0 4 0.14285714285714285
0 5 0.13333333333333333
...
```

First line is the accuracy on the training data. From there we iterate over the rows of the test data and print the accuracy in the current iteration


# Numpy (Numeric python)
> - 패키지 이름과 같이 **수리적 파이썬 활용**을 위한 파이썬 패키지
> - **선형대수학 구현**과 **과학적 컴퓨팅 연산**을 위한 함수를 제공
> - (key) `nparray` 다차원 배열을 사용하여 **벡터의 산술 연산**이 가능
> - **브로드캐스팅**을 활용하여 shape(형태 혹은 모양)이 다른 데이터의 연산이 가능
>> - 기존 언어에서는 제공 X
>> - 굉장히 파워풀한 기능으로서 빅데이터 연산에 굉장히 효율이 좋음

## Numpy 설치 와 import
> - 선행 학습을 통해 클래스와 함수에서 클래스를 불러들여 사용할 수 있다고 배웠습니다.
- 다만 직접 작성한 클래스가 아닐경우, 그리고 현재 컴퓨터에 사용해야 할 패키지가 없을경우 간단한 명령어로 설치가능.
>> - `pip`, `conda` 명령어 : python 라이브러리 관리 프로그램으로 오픈소스라이브러리를 간편하게 설치 할 수 있도록 하는 명령어

> - 1. 콘솔창에서 실행 시
**`pip` `install` `[패키지명]`** 혹은
**`conda` `install` `[패키지명]`**
> - 2. 주피터 노트북으로 실행 시
**`!pip` `install` `[패키지명]`**
> - 아나콘다 환경으로 python 환경설정 시 기본적으로 Numpy 설치가 되어있음

```python
# 주피터 노트북에서 Numpy 설치
!pip install numpy
```

    Requirement already satisfied: numpy in c:\anaconda3\lib\site-packages (1.20.1)

```python
# Numpy 사용을 위해 패키지 불러들이기
import numpy as np
# 관례적으로 np라는 약자를 많이 사용하게 됩니다.
# 파이썬을 사용하는 대부분의 유저들이 사용하고 있는 닉네임이니 이건 꼭 지켜서 사용해주시는 것을 추천드립니다.
```

## 데이터분석을 위한 잠깐의 선형대수학
numpy는 기본적으로 수리적 컴퓨팅을 위한 패키지 입니다. 선형대수학을 약간만 이해한다면 데이터를 훨씬 더 깊이있게 다룰 수 있습니다.

출처 : https://art28.github.io/blog/linear-algebra-1/

## 데이터의 구분에 따른 표현 방법과 예시

#### 스칼라
    1, 3.14, 실수 혹은 정수

#### 벡터
    [1, 2, 3, 4], 문자열

#### 3 X 4 매트릭스
    [[1, 2, 3, 4],
     [5, 6, 7, 8],
     [9, 0, 11, 12]]

#### 2 X 3 X 4 텐서
    [[[1, 2, 3, 4],
     [5, 6, 7, 8],
     [9, 0, 11, 12]],
     [[1, 2, 3, 4],
     [5, 6, 7, 8],
     [9, 0, 11, 12]]]

### 데이터로 표현한 선형대수학

출처 : https://m.blog.naver.com/nabilera1/221978354680

### 데이터 형태에 따른 사칙연산

> 스칼라 +, -, *, / -> 결과도 스칼라
벡터 +, -, 내적 -> +, - 결과는 벡터, 내적 결과는 스칼라
매트릭스 +, -, *, /
텐서 +, -, *, /

### 데이터분석에 자주 사용하는 특수한 연산
벡터와 벡터의 내적

$$\begin{bmatrix}1 & 2 & 3 & 4 \end{bmatrix} \times \begin{bmatrix}1 \\ 2 \\ 3 \\ 4 \end{bmatrix} = 1 * 1 + \
2 * 2 + 3 * 3 + 4 * 4 = 30$$
# $$ A^TA $$

#### 벡터와 벡터의 내적이 이루어지려면

    1. 벡터가 마주하는 shape의 갯수(길이)가 같아야 합니다.
    2. 연산 앞에 위치한 벡터는 전치(transpose) 되어야 합니다.

출처 : https://ko.wikipedia.org/wiki/%EC%A0%84%EC%B9%98%ED%96%89%EB%A0%AC

#### 벡터 내적으로 방정식 구현

$$y = \begin{bmatrix}1 & 2 & 1 \end{bmatrix} \times \begin{bmatrix}x_1 \\ x_2 \\ x_3 \\ \end{bmatrix} = 1 * x_1 + \
2 * x_2 + 1 * x_3 = x_1 + 2x_2 + x_3$$

## 브로드캐스팅
> 파이썬 넘파이 연산은 브로드캐스팅을 지원합니다.
벡터연산 시 shape이 큰 벡터의 길이만큼 shape이 작은 벡터가 연장되어 연산됩니다.

출처 : http://www.astroml.org/book_figures/appendix/fig_broadcast_visual.html

## Numpy function(함수)
> `numpy`는 컴퓨팅연산을 위한 다양한 연산함수를 제공합니다.
>> 연산함수 기본구조
ex) **`np.sum`**(연산대상, axis=연산방향)
**`dtype()`**

### 수리연산
- **`prod()`**
- **`dot()`**
- **`sum()`**
- **`cumprod()`**
- **`cumsum()`**
- **`abs()`**
- **`sqaure()`**
- **`sqrt()`**
- **`exp()`**
- **`log()`**

### 통계연산
- **`mean()`**
- **`std()`**
- **`var()`**
- **`max()`**
- **`min()`**
- **`argmax()`**
- **`argmin()`**

### 로직연산
- **`arange()`**
- **`isnan()`**
- **`isinf()`**
- **`unique()`**

### 기하
- **`shape()`**
- **`reshape()`**
- **`ndim()`**
- **`transpose()`**

각종 연산 함수 참고: https://numpy.org/doc/stable/reference/routines.math.html

### numpy 함수 실습

```python
# 함수 예제를 위한 데이터셋
test_list = [1, 2, 3, 4]
test_list2 = [[1, 3], [5, 7]]
test_flist = [1, 3.14, -4.5]
test_list_2nd = [[1, 2, 3],
              [4, 5, 6],
              [7, 8, 9]]
test_list_3rd = [[[1, 2, 3, 4],
              [5, 6, 7, 8]],

              [[1, 2, 3, 4],
               [5, 6, 7, 8]],

              [[1, 2, 3, 4],
               [5, 6, 7, 8]]]
test_exp = [0, 1, 10]
test_nan = [0, np.nan, np.inf]
```

```python
# 곱연산
np.prod(test_list)
```

    24

```python
# 합연산
np.sum(test_list)
```

    10

```python
# 누적곱연산
np.cumprod(test_list)
```

    array([ 1,  2,  6, 24], dtype=int32)

```python
# 누적합연산
np.cumsum(test_list)
```

    array([ 1,  3,  6, 10], dtype=int32)

```python
test_flist
```

    [1, 3.14, -4.5]

```python
# 절대값
np.abs(test_flist)
```

    array([1.  , 3.14, 4.5 ])

```python
# 제곱
np.square(test_list)
```

    array([ 1,  4,  9, 16], dtype=int32)

```python
# 루트
np.sqrt(test_list)
```

    array([1.        , 1.41421356, 1.73205081, 2.        ])

```python
# exp

```

```python
# 로그

```

```python
# 평균
np.mean(test_list)
```

    2.5

```python
# 표준편차
np.std(test_list)
```

    1.118033988749895

```python
# 분산
np.var(test_list)
```

    1.25

```python
# 최대값

```

```python
# 최소값

```

```python
test_list_2nd
```

    [[1, 2, 3], [4, 5, 6], [7, 8, 9]]

```python
# 최대값 인덱스
# 최대값이 존재하는 인덱스 넘버를 리턴
# 출력값이 인덱스
np.argmax(test_list_2nd)
```

    8

```python
# 최소값 인덱스
np.argmin(test_list_2nd)
```

    0

```python
# 범위설정
# range() 함수와 동일하게 작동함
# for i in range(0, 100, 10):
#     print(i)
np.arange(10, 100, 10)
```

    array([10, 20, 30, 40, 50, 60, 70, 80, 90])

```python
# (시작포인트, 마지막포인트 + 1, 테이터개수)
np.linspace(0, 10, 200)
```

    array([ 0.        ,  0.05025126,  0.10050251,  0.15075377,  0.20100503,
            0.25125628,  0.30150754,  0.35175879,  0.40201005,  0.45226131,
            0.50251256,  0.55276382,  0.60301508,  0.65326633,  0.70351759,
            0.75376884,  0.8040201 ,  0.85427136,  0.90452261,  0.95477387,
            1.00502513,  1.05527638,  1.10552764,  1.15577889,  1.20603015,
            1.25628141,  1.30653266,  1.35678392,  1.40703518,  1.45728643,
            1.50753769,  1.55778894,  1.6080402 ,  1.65829146,  1.70854271,
            1.75879397,  1.80904523,  1.85929648,  1.90954774,  1.95979899,
            2.01005025,  2.06030151,  2.11055276,  2.16080402,  2.21105528,
            2.26130653,  2.31155779,  2.36180905,  2.4120603 ,  2.46231156,
            2.51256281,  2.56281407,  2.61306533,  2.66331658,  2.71356784,
            2.7638191 ,  2.81407035,  2.86432161,  2.91457286,  2.96482412,
            3.01507538,  3.06532663,  3.11557789,  3.16582915,  3.2160804 ,
            3.26633166,  3.31658291,  3.36683417,  3.41708543,  3.46733668,
            3.51758794,  3.5678392 ,  3.61809045,  3.66834171,  3.71859296,
            3.76884422,  3.81909548,  3.86934673,  3.91959799,  3.96984925,
            4.0201005 ,  4.07035176,  4.12060302,  4.17085427,  4.22110553,
            4.27135678,  4.32160804,  4.3718593 ,  4.42211055,  4.47236181,
            4.52261307,  4.57286432,  4.62311558,  4.67336683,  4.72361809,
            4.77386935,  4.8241206 ,  4.87437186,  4.92462312,  4.97487437,
            5.02512563,  5.07537688,  5.12562814,  5.1758794 ,  5.22613065,
            5.27638191,  5.32663317,  5.37688442,  5.42713568,  5.47738693,
            5.52763819,  5.57788945,  5.6281407 ,  5.67839196,  5.72864322,
            5.77889447,  5.82914573,  5.87939698,  5.92964824,  5.9798995 ,
            6.03015075,  6.08040201,  6.13065327,  6.18090452,  6.23115578,
            6.28140704,  6.33165829,  6.38190955,  6.4321608 ,  6.48241206,
            6.53266332,  6.58291457,  6.63316583,  6.68341709,  6.73366834,
            6.7839196 ,  6.83417085,  6.88442211,  6.93467337,  6.98492462,
            7.03517588,  7.08542714,  7.13567839,  7.18592965,  7.2361809 ,
            7.28643216,  7.33668342,  7.38693467,  7.43718593,  7.48743719,
            7.53768844,  7.5879397 ,  7.63819095,  7.68844221,  7.73869347,
            7.78894472,  7.83919598,  7.88944724,  7.93969849,  7.98994975,
            8.04020101,  8.09045226,  8.14070352,  8.19095477,  8.24120603,
            8.29145729,  8.34170854,  8.3919598 ,  8.44221106,  8.49246231,
            8.54271357,  8.59296482,  8.64321608,  8.69346734,  8.74371859,
            8.79396985,  8.84422111,  8.89447236,  8.94472362,  8.99497487,
            9.04522613,  9.09547739,  9.14572864,  9.1959799 ,  9.24623116,
            9.29648241,  9.34673367,  9.39698492,  9.44723618,  9.49748744,
            9.54773869,  9.59798995,  9.64824121,  9.69849246,  9.74874372,
            9.79899497,  9.84924623,  9.89949749,  9.94974874, 10.        ])

```python
test_nan
```

    [0, nan, inf]

```python
# 결측 확인
np.isnan(test_nan)
```

    array([False,  True, False])

```python
# 발산 확인
np.isinf(test_nan)
```

    array([False, False,  True])

```python
test_list_3rd
```

    [[[1, 2, 3, 4], [5, 6, 7, 8]],
     [[1, 2, 3, 4], [5, 6, 7, 8]],
     [[1, 2, 3, 4], [5, 6, 7, 8]]]

```python
# 고유값 확인
np.unique(test_list_3rd)
```

    array([1, 2, 3, 4, 5, 6, 7, 8])

```python
# 데이터 구조(모양)확인
np.shape(test_list_3rd)
```

    (3, 2, 4)

```python
# 데이터 shape 변경
# 어떤 조건에서 reshape가능한가? 데이터 내부에 존재하는 속성 갯수가 같아야 함.
np.reshape(test_list_3rd, (2, 2, 6))
```

    array([[[1, 2, 3, 4, 5, 6],
            [7, 8, 1, 2, 3, 4]],

           [[5, 6, 7, 8, 1, 2],
            [3, 4, 5, 6, 7, 8]]])

```python
test_list_3rd
```

    [[[1, 2, 3, 4], [5, 6, 7, 8]],
     [[1, 2, 3, 4], [5, 6, 7, 8]],
     [[1, 2, 3, 4], [5, 6, 7, 8]]]

```python
# 데이터 차원확인
# 데이터가 존재하는 출방향이 늘어나면 차원수도 늘어남
# 기하학적 데이터의 차원수를 이야기하고
# 데이터분석을 할 때는 열 기준으로 차원을 이야기 한다

np.ndim(test_list_2nd)
```

    2

```python
test_list_2nd
```

    [[1, 2, 3], [4, 5, 6], [7, 8, 9]]

```python
# 전치행렬

```

## Numpy array (배열, 행렬)
> - numpy 연산의 기본이 되는 데이터 구조입니다.
- 리스트보다 간편하게 만들 수 있으며 **연산이 빠른** 장점이 있습니다.
- **브로드캐스팅 연산을 지원**합니다.
- 단, **같은 type**의 데이터만 저장 가능합니다.
- array 또한 numpy의 기본 함수로서 생성 가능합니다.
>> array 함수 호출 기본구조
ex) **`np.array(배열변환 대상 데이터)`**
ex) **`np.arange(start, end, step_forward)`**

### numpy array 실습

```python
# 기존 데이터 구조를 array로 변환
test_array = np.array(test_list)
test_array2 = np.array(test_list2)
test_farray = np.array(test_flist)
test_array_2nd = np.array(test_list_2nd)
test_array_3rd = np.array(test_list_3rd)
```

```python
# array 생성 확인
test_array
```

    array([1, 2, 3, 4])

```python
array_list = [1, 2, '3', 4.5]
```

```python
# 같은 타입의 데이터만 들어가는지 확인
array_test = np.array(array_list)
```

```python
array_test
```

    array(['1', '2', '3', '4.5'], dtype='<U32')

```python
# 2차원 배열 확인
test_array_2nd
```

    array([[1, 2, 3],
           [4, 5, 6],
           [7, 8, 9]])

```python
# 3차원 배열 확인
test_array_3rd
```

    array([[[1, 2, 3, 4],
            [5, 6, 7, 8]],

           [[1, 2, 3, 4],
            [5, 6, 7, 8]],

           [[1, 2, 3, 4],
            [5, 6, 7, 8]]])

```python
# np.arange 함수로 생성
np.arange(24).reshape(2, 12)
```

    array([[ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11],
           [12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23]])

### 특수한 형태의 array를 함수로 생성
함수 호출의 기본구조
> ex) **`np.ones([자료구조 shape])`**
>> 자료구조 shape은 정수, **[ ]**리스트, **( )** 튜플 로만 입력가능합니다.
- ones()
- zeros()
- empty()
- eye()

```python
# 1로 초기화한 array 생성
np.ones([3, 3])
```

    array([[1., 1., 1.],
           [1., 1., 1.],
           [1., 1., 1.]])

```python
# 0으로 초기화
np.zeros((4, 4))
```

    array([[0., 0., 0., 0.],
           [0., 0., 0., 0.],
           [0., 0., 0., 0.],
           [0., 0., 0., 0.]])

```python
# 빈 값으로 초기화
np.empty([5, 5])
```

    array([[7.74860419e-304, 7.74860419e-304, 7.74860419e-304,
            7.74860419e-304, 7.74860419e-304],
           [7.74860419e-304, 7.74860419e-304, 7.74860419e-304,
            7.74860419e-304, 7.74860419e-304],
           [7.74860419e-304, 7.74860419e-304, 7.74860419e-304,
            7.74860419e-304, 7.74860419e-304],
           [7.74860419e-304, 7.74860419e-304, 7.74860419e-304,
            7.74860419e-304, 7.74860419e-304],
           [7.74860419e-304, 7.74860419e-304, 7.74860419e-304,
            7.74860419e-304, 7.74860419e-304]])

```python
# 항등행렬 초기화
np.eye(4, 4)

# shape이 안맞는 경우 연산이 가능하도록 할 때
```

    array([[1., 0., 0., 0.],
           [0., 1., 0., 0.],
           [0., 0., 1., 0.],
           [0., 0., 0., 1.]])

```python
np.arange(9).reshape(3, 3) @ np.eye(3, 3)
```

    array([[0., 1., 2.],
           [3., 4., 5.],
           [6., 7., 8.]])

### array 속성 및 내장함수
`np.array` 에는 유용한 수리, 통계 연산을 위한 함수가 갖추어져 있습니다. 다차원 기하학적 연산을 위한 함수도 함께 살펴보겠습니다.

> array 내장 속성 호출 기본구조
ex) **`test_array.ndim`**
자주 사용하는 속성 `shape`, `dtype`, `ndim`

> array 내장함수 호출 기본구조
ex) **`test_array.prod()`**

위에 학습한 np.sum() 과는 달리 array 변수의 인자를 받아 그대로 사용합니다.

#### array 속성

```python
# 데이터 타입확인
test_array_3rd.dtype
```

    dtype('int32')

```python
# 데이터구조 확인
# np.shape()
test_array_3rd.shape
```

    (3, 2, 4)

```python
# 데이터 차원 확인
test_array_3rd.ndim
```

    3

```python
test_array_3rd
```

    array([[[1, 2, 3, 4],
            [5, 6, 7, 8]],

           [[1, 2, 3, 4],
            [5, 6, 7, 8]],

           [[1, 2, 3, 4],
            [5, 6, 7, 8]]])

```python
# 아래 행렬에 대해서도 데이터의 shape과 차원을 확인해봅니다.

```

```python
# 전치행렬
test_array_3rd.T
```

    array([[[1, 1, 1],
            [5, 5, 5]],

           [[2, 2, 2],
            [6, 6, 6]],

           [[3, 3, 3],
            [7, 7, 7]],

           [[4, 4, 4],
            [8, 8, 8]]])

#### array 내장함수

```python
test_array
```

    array([1, 2, 3, 4])

```python
# 내장함수 호출
test_array.sum()
```

    10

```python
test_array.cumsum()
```

    array([ 1,  3,  6, 10], dtype=int32)

```python
# numpy 함수와 키워드가 같습니다.
test_array.prod()
test_array.sum(axis=)
...
...
```

### array 연산
컴퓨팅 연산을 위한 패키지인 만큼 편리한 배열 연산 기능을 지원합니다. 여러 array 연산을 통해 다른 자료구조와의 연산 차이를 알아봅시다.

```python
test_list = [1, 2, 3, 4, 5, 6]
test_list + 7
```

```python
# array 덧셈, 뺄셈, 곱셈, 나눗셈
test_array * 6
```

    array([ 6, 12, 18, 24])

```python

```

```python
# 실제 연산속도 차이를 확인하기 위한 큰 데이터 생성
big_list = [x for x in range(500000)]
big_array = np.array(big_list)
len(big_list), len(big_array)
```

    (500000, 500000)

```python
%%time
big_list = [x + 1 for x in big_list]
#for index, item in enumerate(big_list):
#    big_list[index] = item + 1
```

    Wall time: 59.5 ms

```python
%%time
big_array + 1
```

    Wall time: 1.99 ms

    array([     1,      2,      3, ..., 499998, 499999, 500000])

```python
# 행렬내적
first_array = np.arange(15).reshape(5, 3)
second_array = np.arange(15).reshape(3, 5)
```

```python
first_array.shape, second_array.shape
```

    ((5, 3), (3, 5))

```python
# 행렬내적 연산
first_array @ second_array
```

    array([[ 25,  28,  31,  34,  37],
           [ 70,  82,  94, 106, 118],
           [115, 136, 157, 178, 199],
           [160, 190, 220, 250, 280],
           [205, 244, 283, 322, 361]])

### 벡터 가중합
벡터의 내적은 가중합을 계산할 때 쓰일 수 있다. **가중합(weighted sum)**이란 복수의 데이터를 단순히 합하는 것이 아니라 각각의 수에 어떤 가중치 값을 곱한 후 이 곱셈 결과들을 다시 합한 것을 말한다.
만약 데이터 벡터가 $x=[x_1, \cdots, x_N]^T$이고 가중치 벡터가 $w=[w_1, \cdots, w_N]^T$이면 데이터 벡터의 가중합은 다음과 같다.

$$
\begin{align}
w_1 x_1 + \cdots + w_N x_N = \sum_{i=1}^N w_i x_i
\end{align}
$$

이 값을 벡터 $x$와 $w$의 곱으로 나타내면 $w^Tx$ 또는 $x^Tw$ 라는 간단한 수식으로 표시할 수 있다.

쇼핑을 할 때 각 물건의 가격은 데이터 벡터, 각 물건의 수량은 가중치로 생각하여 내적을 구하면 총금액을 계산할 수 있다.

```python
# 벡터의 가중합 연습문제
# 삼성전자, 셀트리온, 카카오로 포트폴리오를 구성하려한다.
# 각 종목의 가격은 80,000원, 270,000원, 160,000원이다.
# 삼성전자 100주, 셀트리온 30주, 카카오 50주로 구성하기 위한 매수금액을 구하시오
port = np.array([80000, 270000, 160000])
nums = np.array([100, 30, 50])

port, nums, port @ nums
```

    (array([ 80000, 270000, 160000]), array([100,  30,  50]), 24100000)

### array 인덱싱, 슬라이싱(매우중요)
> 기본적으로 자료구조란 데이터의 묶음, 그 묶음을 관리 할 수 있는 바구니를 이야기 합니다.
데이터 분석을 위해 자료구조를 사용하지만 자료구조안 내용에 접근을 해야 할 경우도 있습니다.
>> **인덱싱**이란?
데이터 바구니 안에 든 내용 하나에 접근하는 명령, 인덱스는 내용의 순번
>> **슬라이싱**이란?
데이터 바구니 안에 든 내용 여러가지에 접근 하는 명령

기본적으로 인덱싱과 슬라이싱의 색인은 리스트와 동일합니다.

#### 인덱싱, 슬라이싱 실습

```python
# 10부터 19까지 범위를 가지는 array생성
practice = np.arange(10, 20)
```

```python
# 0부터 3번 인덱스까지
practice[:4]
```

    array([10, 11, 12, 13])

```python
# 4번 인덱스부터 마지막 인덱스까지
practice[4:]
```

    array([14, 15, 16, 17, 18, 19])

```python
# 마지막 인덱스부터 뒤에서 3번째 인덱스까지
practice[:-4:-1]
```

    array([19, 18, 17])

```python
# 0부터 3씩 증가하는 인덱스
practice[::3]
```

    array([10, 13, 16, 19])

#### 여러가지 인덱싱 및 슬라이싱 방법을 시도해봅시다

```python
index_test2 = np.array(range(25)).reshape([5, 5])
index_test2
```

    array([[ 0,  1,  2,  3,  4],
           [ 5,  6,  7,  8,  9],
           [10, 11, 12, 13, 14],
           [15, 16, 17, 18, 19],
           [20, 21, 22, 23, 24]])

```python
index_test2[2:, 1:4]
```

    array([[11, 12, 13],
           [16, 17, 18],
           [21, 22, 23]])

```python

```

    array([[11, 12, 13],
           [16, 17, 18],
           [21, 22, 23]])

```python
index_test2[0:2, 2:]
```

    array([[2, 3, 4],
           [7, 8, 9]])

```python

```

    array([[2, 3, 4],
           [7, 8, 9]])

```python
index_test3 = np.arange(40).reshape(2, 5, 4)
index_test3
```

    array([[[ 0,  1,  2,  3],
            [ 4,  5,  6,  7],
            [ 8,  9, 10, 11],
            [12, 13, 14, 15],
            [16, 17, 18, 19]],

           [[20, 21, 22, 23],
            [24, 25, 26, 27],
            [28, 29, 30, 31],
            [32, 33, 34, 35],
            [36, 37, 38, 39]]])

```python
index_test3[0, 3:, 1:3]
```

    array([[13, 14],
           [17, 18]])

```python

```

    array([[13, 14],
           [17, 18]])

```python
index_test3[:, :, 1]
```

    array([[ 1,  5,  9, 13, 17],
           [21, 25, 29, 33, 37]])

```python

```

    array([[ 1,  5,  9, 13, 17],
           [21, 25, 29, 33, 37]])

## 팬시인덱싱
numpy에서 벡터연산을 통해 bool 형태의 벡터를 기준으로 데이터를 선별하는 방법

```python
pet = np.array(['개', '고양이', '고양이', '햄스터', '개', '햄스터'])

indexing_test = np.random.randn(6, 5)
```

```python
indexing_test
```

    array([[ 0.01758269,  1.17531911,  0.04272206,  1.64404978, -0.05464789],
           [ 0.13472248,  0.0282913 ,  0.17494501,  1.63486306, -0.5954133 ],
           [-0.22648549,  0.62630269,  0.12122429,  1.39567674,  1.44466132],
           [ 0.95904236, -0.8472618 ,  1.23439726, -0.15695985, -1.13903603],
           [ 0.6804963 , -0.26831841, -0.10783034,  1.2246725 , -0.21611025],
           [ 0.85479063,  0.15053482,  0.67559927,  0.13979231, -1.55436411]])

```python
pet == '개'
```

    array([ True, False, False, False,  True, False])

```python
indexing_test[pet != '개']
```

    array([[ 0.13472248,  0.0282913 ,  0.17494501,  1.63486306, -0.5954133 ],
           [-0.22648549,  0.62630269,  0.12122429,  1.39567674,  1.44466132],
           [ 0.95904236, -0.8472618 ,  1.23439726, -0.15695985, -1.13903603],
           [ 0.85479063,  0.15053482,  0.67559927,  0.13979231, -1.55436411]])

```python
(pet == '개') | (pet == '고양이')
```

    array([ True,  True,  True, False,  True, False])

```python
indexing_test[(pet == '개') | (pet == '고양이')]
```

    array([[ 0.01758269,  1.17531911,  0.04272206,  1.64404978, -0.05464789],
           [ 0.13472248,  0.0282913 ,  0.17494501,  1.63486306, -0.5954133 ],
           [-0.22648549,  0.62630269,  0.12122429,  1.39567674,  1.44466132],
           [ 0.6804963 , -0.26831841, -0.10783034,  1.2246725 , -0.21611025]])

```python
(pet == '개').sum()
```

    2

```python
(pet == '개').any()
```

    True

```python
(pet == '개').all()
```

    False

```python

```


```python
from IPython.core.display import HTML
css_file = '../style.css'
HTML(open(css_file, 'r').read())
```

<link href='http://fonts.googleapis.com/css?family=Alegreya+Sans:100,300,400,500,700,800,900,100italic,300italic,400italic,500italic,700italic,800italic,900italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Arvo:400,700,400italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=PT+Mono' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Shadows+Into+Light' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Philosopher:400,700,400italic,700italic' rel='stylesheet' type='text/css'>

<style>

@font-face {
    font-family: "Computer Modern";
    src: url('http://mirrors.ctan.org/fonts/cm-unicode/fonts/otf/cmunss.otf');
}

/* Formatting for header cells */
.text_cell_render h1 {
    font-family: 'Philosopher', sans-serif;
    font-weight: 400;
    font-size: 2.2em;
    line-height: 100%;
    color: rgb(0, 80, 120);
    margin-bottom: 0.1em;
    margin-top: 0.1em;
    display: block;
}
.text_cell_render h2 {
    font-family: 'Philosopher', serif;
    font-weight: 400;
    font-size: 1.9em;
    line-height: 100%;
    color: rgb(245,179,64);
    margin-bottom: 0.1em;
    margin-top: 0.1em;
    display: block;
}

.text_cell_render h3 {
    font-family: 'Philosopher', serif;
    margin-top:12px;
    margin-bottom: 3px;
    font-style: italic;
    color: rgb(94,127,192);
}

.text_cell_render h4 {
    font-family: 'Philosopher', serif;
}

.text_cell_render h5 {
    font-family: 'Alegreya Sans', sans-serif;
    font-weight: 300;
    font-size: 16pt;
    color: grey;
    font-style: italic;
    margin-bottom: .1em;
    margin-top: 0.1em;
    display: block;
}

.text_cell_render h6 {
    font-family: 'PT Mono', sans-serif;
    font-weight: 300;
    font-size: 10pt;
    color: grey;
    margin-bottom: 1px;
    margin-top: 1px;
}

.CodeMirror{
        font-family: "PT Mono";
        font-size: 100%;
}

</style>

# Fun with algebra

## Preamble

In this notebook, we will learn more about some keywords (function) in `sympy`, specific to common algebraic manipulations.  We begin with importing `sympy` and initializing the use of LaTex printing to the screen.

```python
from sympy import init_printing
init_printing()
```

## Simplifying algebraic expressions

The `sympy` package has a general function called `simplify()` which is great at simplifying algebraic expressions.

```python
from sympy import symbols, simplify
```

```python
x, y = symbols('x y')  # Creating more than one symbol
```

Let's simplify expression (1).

$$\frac{x^3 + x^2 - x + 1}{x^2 + 2x + 1} \tag{1}$$

```python
simplify((x**3 + x**2 - x - 1)/(x**2 + 2*x + 1))
```

We can factor both the numerator and the denominator using the `.factor()` method on each of the expresssions.

```python
(x**3 + x**2 - x - 1).factor()  # Factoring the numerator
```

```python
(x**2 + 2*x + 1).factor()  # Factoring the denominator
```

Now it's easy to see the result of the simplification as we ${\left( x+1 \right)}^2$ in both the numerator and the denominator.

Note that `simplify()` is a relatively slow function as it _tries_ many possible solutions.  When appropriate rather factor or expand.  We have already seen the power of the `factor()` and `expand()` functions and methods.

## Collect

The `collect()` function collects common powers.  Expression (2) has three mathematical variables with the $x$ variable being of varying degree.

$$x y + x - 3 + 2 x^2 - z x^2 + x^3 \tag{2}$$

We have to set `z` as a mathematical variable too.

```python
z = symbols('z')  # Setting z as a mathematical variable
```

The `expr` computer variable is overwritten to contain expression (2).

```python
expr = x * y - 2 + 2 * x**2 - z * x**2 + x**3
expr
```

The `collect()` function is imported from `sympy`.

```python
from sympy import collect
```

Now we use the `collect()` function with two arguments.  The first is the computer variable that contains expression (2) and the second is the mathematical variable of interest.

```python
collect(expr, x)
```

The results is an expression with descending powers of $x$.

## Cancel

The `cancel()` function puts a rational function in canonical form, i.e. $\frac{p}{q}$.  We will import the function and use it for expression (3).

```python
((x-3)*(x+2)*(x-4)).expand()
```

```python
((x-1)*(x-3)).expand()
```

$$\frac{x^3 - 5x^2 - 2x +24}{x^2 - 4x + 3} \tag{3}$$

```python
from sympy import cancel
```

```python
# Instead of first creating a computer variable we can pass the expression directly as an argument
cancel(((x - 3) * (x + 2) * (x - 4)) / ((x - 1)*(x - 3)))
```

The `.factor()` method will accomplish the same thing, other than also factoring the polynomials.

```python
(((x - 3) * (x + 2) * (x - 4)) / ((x - 1)*(x - 3))).factor()
```

## Apart

The `apart()` function is especially helpful.  It calculates partial fraction decomposition.  This can take a long time if done by hand.  Take a look at expression (4) and then its (one possible) partial fraction decomposition.

$$\frac{3x^4 + 5x^3 - 3x^2 + 3x}{x^5 + 3x^4 - 9x - 27} \tag{4}$$

```python
from sympy import apart
```

```python
apart((1 / (x + 3)) + (x / (x**2 - 3)) + ((x - 1) / (x**2 + 3)))
```

## Trigonometric functions

The `sympy` package can both simplify and expand trigonometric function.  One of the most common trigonometric identity is shown in equation (5).

$${\left( \sin{\theta} \right)}^2 + {\left( \cos{\theta} \right)}^2 = 1 \tag{5}$$

First we have to learn how to create the mathematical variable $\theta$.  This is done using the same `symbol()` function that we are now familiar with.  First we have to create a computer variable to hold the symbol, just as we did with the mathematical symbol $x$ being held in the computer variable `x`, i.e. `x = symbols('x')`.  It is common just to use the written word for the Greek symbols.

The `symbols()` function recognizes special characters.  The Greek alphabet is included and hence we use the string `'theta'` as argument.

```python
theta = symbols('theta')
theta
```

To show this identity in action, we use the `trigsimp()` function.

```python
from sympy import trigsimp, sin, cos
```

```python
# Expression
(sin(theta))**2 + (cos(theta))**2
```

```python
# Trigonometric simplification
trigsimp((sin(theta))**2 + (cos(theta))**2)
```

Trigonometric expansion is just as easy with `expand_trig()`.  We will do this for the trigonometric identity in (6).

$$\sin( \theta + \phi ) = \sin(\phi) \cos(\theta) + \sin(\theta) \cos(\phi) \tag{6}$$

```python
# Creating phi
phi = symbols('phi')
phi
```

```python
from sympy import expand_trig
```

```python
expand_trig(sin(theta + phi))
```

## Powers

The common properties of powers do not always hold.  In (7) below, are the common (non-exclusive) conditions for the properties of powers.

$$\begin{align}&x^a x^b = x^{a+b}, \forall a, b, x \in \mathbb{C} \\
&x^a y^a = \left( xy \right)^a, \forall x, y > 0, \forall a \in \mathbb{R} \\
&\left( x^a \right)^b = x^{ab}, \forall b \in \mathbb{Z} \end{align}  \tag{7}$$

The `sympy` package will only calculate powers when they are true in general.  By default all mathematical symbols in `sympy` are complex.  When mathematical symbols are declared with the `symbols()` function, we can specify their type, i.e. real or positive.

```python
# Specifying the type of values that a mathematical symbol can hold
x, y = symbols('x y',
              positive = True)
a, b = symbols('a b',
              real = True)
```

Now we can use these mathematical symbols with the `powersimp()` function.  Below we import the function and look at the power properties.

```python
from sympy import powsimp
```

```python
expr = x**a * x**b  # Overwriting the computer variable expr
expr
```

```python
powsimp(expr)
```

```python
expr = x**a * y**a  # Overwriting the computer variable expr
expr
```

```python
powsimp(expr)
```

```python
expr = (x**a)**b  # Overwriting the computer variable expr
expr  # Automatically simplified
```

```python
powsimp(expr)
```

When the the complex number values that variables can hold does not allow for the use of the properties of powers, the simplification is not performed.

```python
m, n, c = symbols(' m n c')
```

```python
expr = m**c * n**c  # Overwriting the computer variable expr
expr
```

```python
powsimp(expr)
```

An example of this might be $m=n=-1, a = \frac{1}{2}$.
$$\begin{align} \left( -1 \right)^{\frac{1}{2}}\left( -1 \right)^{\frac{1}{2}} &= \left( -1 \times -1 \right)^{\frac{1}{1}} \\ i \cdot i &= 1^{\frac{1}{2}} \\ -1 &\ne \pm 1 \end{align}$$

The argument `force = True` can be added to the `powsimp()` function irrespective of having defined the type of mathematical variable.  This is useful if you know that you want to make the simplification an understand the mathematical constraints.

```python
powsimp(expr,
       force = True)
```

Just as an aside, the square root function, `sqrt(x)` in `sympy` is actuall shorthand syntax for `x**Rational(1,2)`.

```python
from sympy import sqrt, Rational
```

```python
sqrt(x) == x**Rational(1, 2)
```

    True

The `expand_power_exp()` and the `expand_power_base()` functions expand power expressions.

```python
from sympy import expand_power_exp, expand_power_base
```

In expression (8) and (9) we expand using the properties of powers.  If these functions fail to expand the expression that you are working on, remember the mathematical constraints  or use the `force = True` argument.

$$x^{ a + b} \tag{8}$$

```python
expand_power_exp(x**(a + b))
```

$$ \left( xy \right)^{a} \tag{9}$$

```python
expand_power_base((x*y)**a)
```

## Logarithms

The `log()` function in `sympy` calculates the natural logarithm by default.  The latter uses Euler's number as base.  In general the properties shown in (10) below will hold for all positive real numbers and real exponents.

$$\begin{align} &\log{ x y } = \log{x} + \log{y} \\ &\log{x^n} = n \log{x}, \forall x, y \in \mathbb{R}, > 0 \end{align} \tag{10}$$

```python
# Ovewrtiting the computer variable that hold the mathematical variables
# Specifying the values that each can hold
x, y = symbols('x y',
               positive = True)
n = symbols('n',
           real = True)
```

The `expand_log()` will expand expression according to the properties of logarithms given in (10) above.

```python
from sympy import expand_log, log
```

```python
expand_log(log(x * y))
```

```python
expand_log(x**2)
```

Division of two positive real numbers, $\frac{x}{y}$, is simply a special case.

$$\begin{align} \log{\frac{x}{y}} &= \log{x \frac{1}{y}} \\ &= \log{x y^{-1}} \\ &= \log{x} + \log{y^{-1}} \\ &= \log{x} - \log{y} \end{align}$$

```python
expand_log(log(x / y))
```

The `logcombine()` function will simplify our expression according to the properties in (10).

```python
from sympy import logcombine
```

```python
logcombine(log(x) + log(y))
```

```python
logcombine(n * log(x))
```

## Binomial

The binomial coefficient function, `binomial()` calculates the number of ways to choose $k$ items from a set of $n$ distinct items, where order does not matter.

```python
from sympy import binomial
```

We choose natural numbers as the binomial function pertains to discrete objects.

```python
n, k = symbols('n k', positive = True, integer = True)
```

```python
binomial(n, k)
```

We can substitute values using the `.evalf()` method.

```python
binomial(n, k).evalf(subs = {n:4, k:2})
```

```python

```

```python

```

Python/3. Computational Sciences and Mathematics/Linear Algebra/0.0 Start Here/0.2 Algebraic Operations Activity.ipynb


```python
import matplotlib.pyplot as plt
import scipy.stats as st
import seaborn as sns
import pandas as pd
from scipy.stats import norm, uniform, expon, t, probplot
import scipy.stats as st
from scipy.integrate import quad
from sympy.solvers import solve
from sympy import Symbol
import numpy as np
from pandas import Series, DataFrame
```

```python
st.binom_test(x=11, n=13, p=0.5, alternative="two-sided")
```

    0.022460937500000003

```python
methodeA = Series([79.98, 80.04, 80.02, 80.04, 80.03, 80.03, 80.04, 79.97, 80.05, 80.03, 80.02, 80.00, 80.02])
```

```python
methodeA.mean()
```

    80.02076923076923

```python
methodeA.median()
```

    80.03

```python

```