# Fundamentos de Matemática Discreta com Python

## Matemática Discreta

- Área da Matemática que lida com objetos discretos, a saber, conjuntos, sequencias, listas, coleções ou quaisquer entidades *contáveis*.
- Exemplo, $\mathbb{R}$ é incontável, ou não enumerável
- Vários exemplos de contáveis:
    - O conjunto das vogais da língua portuguesa;
    - O conjunto dos times de futebol brasileiros da série A em 2020;
    - O conjunto de nomes das estações do ano;
    - O conjunto das personagens do quarteto do filme *Os Pinguins de Madagascar* e;
    - O conjunto dos números pares positivos menores ou iguais a dez.

- Conjuntos denotados por *extensão*: quando listamos seus elementos

- $\{ a, e, i, o, u \}$
- $\{ \text{Atlético-PR}, \ldots, \text{Bahia}, \text{Botafogo}, \ldots, \text{Coritiba}, \ldots, \text{Fortaleza}, \ldots, \text{Internacional}, \ldots, \text{São Paulo}, \text{Sport}, \text{Vasco} \}$
- $\{ \text{Primavera}, \text{Verão}, \text{Outono}, \text{Inverno}\}$
- $\{ \text{Capitão}, \text{Kowalski}, \text{Recruta}, \text{Rico}\}$
- $\{ 2, 4, 6, 8,10\}$

- Denotados por *compreensão*: quando usamos uma propriedade que distingue seus elementos.

- $\{ c \in \mathbb{A} \, ; \, c \text{ é vogal} \}$
- $\{ t \in \mathbb{T} \, ; \, t \text{ é da Série A} \}$
- $\{ x  \, ; \, x \text{ é uma estação do ano} \}$
- $\{ p  \, ; \, p \text{ é personagem do quarteto principal do filme Os Pinguins de Madagascar} \}$
- $\{ e  \, ; \, e \text{ é estação do ano} \}$
- $\{ n \in \mathbb{Z} \, | \, n = 2k \wedge 2 \leq n \leq 10 \wedge k \in \mathbb{Z} \}$

Por livre conveniência:

- $\mathbb{A}$ é o conjunto de todas as letras de nosso alfabeto
- $\mathbb{T}$ é o conjunto de todos os times de futebol do Brasil.

## Estruturas de dados para objetos discretos

As principais que aprenderemos:

- `list`: estrutura cujo conteúdo é modificável e o tamanho variável. Listas são caracterizadas por *mutabilidade* e *variabilidade*. Objetos `list` são definidos por um par de colchetes e vírgulas que separam seus elementos: `[., ., ... ,.]`.

- `tuple`: estrutura cujo conteúdo não é modificável e o tamanho fixado. Tuplas são caracterizadas por *imutabilidade* e *invariabilidade*. Objetos `tuple` são definidos por um par de colchetes e vírgulas que separam seus elementos: `(., ., ... ,.)`.

- `dict`: estruturas contendo uma coleção de pares do tipo *chave-valor*. Dicionários são caracterizados por *arrays associativos* (*tabelas hash*). Objetos `dict` são definidos por um par de chaves e agrupamentos do tipo  `'chave':valor` (*key:value*), separados por vírgula: `{'chave1':valor1, 'chave2':valor2, ... ,'chaven':valorn}`. As chaves (*keys*) podem ser do tipo `int`, `float`, `str`, ou `tuple` ao passo que os valores podem ser de tipos arbitrários.

- `set`: estruturas similares a `dict`, porém não possuem chaves e contêm objetos únicos. Conjuntos são caracterizadas por *unicidade* de elementos. Objetos `set` são definidos por um par de chaves e vírgulas que separam seus elementos: `{., ., ... ,.}`.

## Listas

Estruturas `list` formam uma coleção de objetos arbitrários e podem ser criadas de modo sequenciado com operadores de pertencimento ou por expressões geradoras, visto que são estruturas iteráveis.

```python
vogais = ['a','e','i','o','u'] # elementos são 'str'
vogais
```

    ['a', 'e', 'i', 'o', 'u']

```python
times = ['Bahia', 'Sport', 'Fortaleza', 'Flamengo']
times
```

    ['Bahia', 'Sport', 'Fortaleza', 'Flamengo']

```python
pares10 = [2,4,6,8,10]
pares10
```

    [2, 4, 6, 8, 10]

```python
mix = ['Bahia',24,6.54,[1,2]] # vários objetos na lista
mix
```

    ['Bahia', 24, 6.54, [1, 2]]

### Listas por geração

**Exemplo**: crie uma lista dos primeiros 100 inteiros não-negativos.

```python
os_100 = range(100) # range é uma função geradora
print(list(os_100)) # casting com 'list'
```

    [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99]

**Exemplo**: crie o conjunto $\{ x \in \mathbb{Z} \, ; \, -20 \leq x < 10 \}$

```python
print(list(range(-20,10))) # print é usado para imprimir column-wise
```

    [-20, -19, -18, -17, -16, -15, -14, -13, -12, -11, -10, -9, -8, -7, -6, -5, -4, -3, -2, -1, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

**Exemplo**: crie o conjunto $\{ x \in \mathbb{Z} \, ; \, -20 \leq x \leq 10 \}$

```python
print(list(range(-20,11))) # para incluir 10, 11 deve ser o limite. Por quê?
```

    [-20, -19, -18, -17, -16, -15, -14, -13, -12, -11, -10, -9, -8, -7, -6, -5, -4, -3, -2, -1, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

## Adicionando e removendo elementos

Há vários métodos aplicáveis para adicionar e remover elementos em listas.

### Adição por apensamento

Adiciona elementos por concatenação no final da lista.

```python
times.append('Botafogo')
times
```

    ['Bahia', 'Sport', 'Fortaleza', 'Flamengo', 'Botafogo']

```python
times.append('Fluminense')
times
```

    ['Bahia', 'Sport', 'Fortaleza', 'Flamengo', 'Botafogo', 'Fluminense']

### Adição por extensão

Para incluir elementos através de um objeto iterável, sequenciável, usamos `extend`.

```python
falta = ['Vasco', 'Atlético-MG']
times.extend(falta) # usa outra lista pra estender a lista
times
```

    ['Bahia',
     'Sport',
     'Fortaleza',
     'Flamengo',
     'Botafogo',
     'Fluminense',
     'Vasco',
     'Atlético-MG']

#### Iteração e indexação

- *Iterar* sobre uma lista é "passear" por seus elementos

- Em Python, a indexação de listas vai de `0` a `n - 1`, onde `n` é o tamanho da lista.

Por exemplo:

$\text{posição} : \{p=0, p=1, \ldots, p={n-1}\}$

$\text{elementos na lista} : [x_1, x_2, \ldots, x_{n}]$

- Mesma idéia aplicável a qualquer coleção, sequencia ou objeto iterável.

### Remoção por índice

Suponha que tivéssemos criado a lista:

```python
pares = [0,2,5,6] # 5 não é par
pares
```

    [0, 2, 5, 6]

Como 5 não é par, não deveria estar na lista. Para excluírmos um elemento em uma posição específica, usamos `pop` passando o *índice* onde o elemento está.

```python
pares.pop(2) # o ímpar 5 está na posição 2 e NÃO 3!
pares

```

    [0, 2, 6]

### Adição por índice

Nesta lista, podemos pensar em incluir 4 entre 2 e 6. Para isto, usamos `insert(posicao,valor)`, para `valor` na `posicao` desejada.

```python
pares.insert(2,4) # 4 é inserido na posição de 6, que é deslocado
pares
```

    [0, 2, 4, 6]

### Apagar conteúdo da lista

Podemos apagar o conteúdo inteiro da lista com `clear`.

```python
times.clear()
times # lista está vazia
```

    []

Podemos contar o número de elementos da lista com `len`.

```python
len(times) # verifica que a lista está vazia

```

    0

```python
type([]) # a lista é vazia, mas continua sendo lista
```

    list

### Outros métodos de lista

Conte repetições de elementos na lista com `count`.

```python
numeros = [1,1,2,3,1,2,4,5,6,3,4,4,5,5]
print( numeros.count(1), numeros.count(3), numeros.count(7) )
```

    3 2 0

Localize a posição de um elemento com `index`.

```python
numeros.index(5) # retorna a posição da primeira aparição
```

    7

Remova a primeira aparição do elemento com `remove`.

```python
numeros.remove(1) # perde apenas o primeiro
numeros
```

    [1, 2, 3, 1, 2, 4, 5, 6, 3, 4, 4, 5, 5]

Faça uma reflexão ("flip") *in-place* (sem criar nova lista) da lista com `reverse`.

```python
numeros.reverse()
numeros
```

    [5, 5, 4, 4, 3, 6, 5, 4, 2, 1, 3, 2, 1]

Ordene a lista de maneira *in-place* (sem criar nova lista) com `sort`.

```python
numeros.sort()
numeros
```

    [1, 1, 2, 2, 3, 3, 4, 4, 4, 5, 5, 5, 6]

## Concatenação de listas

Listas são concatenadas ("somadas") com `+`. Caso já possua listas definidas, use `extend`.

```python
['Flamengo', 'Botafogo'] + ['Fluminense']
```

    ['Flamengo', 'Botafogo', 'Fluminense']

```python
['Flamengo', 'Botafogo'] + 'Fluminense' # erro: 'Fluminense' não é list
```

```python
times_nordeste = ['Fortaleza','Sport']
times_sul = ['Coritiba','Atlético-PR']
times_nordeste + times_sul
```

    ['Fortaleza', 'Sport', 'Coritiba', 'Atlético-PR']

```python
times_nordeste.extend(times_sul) # mesma coisa
times_nordeste
```

    ['Fortaleza', 'Sport', 'Coritiba', 'Atlético-PR']

## Fatiamento de listas

O fatiamento ("slicing") permite que selecionemos partes da lista através do modelo `start:stop`, em que `start` é um índice incluído na iteração, e `stop` não.

```python
letras = ['a','b','c','d','e','f','g']
letras[0:2]
```

    ['a', 'b']

```python
letras[1:4]
```

    ['b', 'c', 'd']

```python
letras[5:6]
```

    ['f']

```python
letras[0:7] # toda a lista
```

    ['a', 'b', 'c', 'd', 'e', 'f', 'g']

### Omissão de `start` e `stop`

```python
letras[:3] # até 3, exclusive
```

    ['a', 'b', 'c']

```python
letras[:5] # até 5, exclusive
```

    ['a', 'b', 'c', 'd', 'e']

```python
letras[4:] # de 4 em diante
```

    ['e', 'f', 'g']

```python
letras[6:] # de 6 em diante
```

    ['g']

### Modo reverso

```python
letras[-1] # último índice
```

    'g'

```python
letras[-2:-1] # do penúltimo ao último, exclusive
```

    ['f']

```python
letras[-3:-1]
```

    ['e', 'f']

```python
letras[-4:-2]
```

    ['d', 'e']

```python
letras[-7:-1] # toda a lista
```

    ['a', 'b', 'c', 'd', 'e', 'f']

```python
letras[-5:]
```

    ['c', 'd', 'e', 'f', 'g']

```python
letras[:-3]
```

    ['a', 'b', 'c', 'd']

## Elementos alternados com `step`

Podemos usar um dois pontos duplo (`::`) para dar um "passo" de alternância.

```python
letras[::2] # salta 2-1 intermediários
```

    ['a', 'c', 'e', 'g']

```python
letras[::3] # salta 3-1 intermediários
```

    ['a', 'd', 'g']

```python
letras[::7] # salto de igual tamanho
```

    ['a']

```python
letras[::8] # salto além do tamanho
```

    ['a']

## Mutabilidade de listas

Podemos alterar o conteúdo de elementos diretamente por indexação.

```python
from sympy.abc import x,y

ops = [x+y,x-y,x*y,x/y]
ops2 = ops.copy() # cópia de ops
ops
```

    [x + y, x - y, x*y, x/y]

```python
ops[0] = x-y
ops
```

    [x - y, x - y, x*y, x/y]

```python
ops[2] = x/y
ops
```

    [x - y, x - y, x/y, x/y]

```python
ops[1], ops[3] = x + y, x*y # mutação por desempacotamento
ops
```

    [x - y, x + y, x/y, x*y]

```python
ops[1:3] = [False, False, True] # mutação por fatiamento
ops
```

    [x - y, False, False, True, x*y]

```python
ops = ops2 # recuperando ops
ops
```

    [x + y, x - y, x*y, x/y]

```python
ops2 is ops
```

    True

```python
ops3 = [] # lista vazia
ops3
```

    []

```python
ops2 = ops + ops3 # concatenação cria uma lista nova
ops2
```

    [x + y, x - y, x*y, x/y]

```python
ops2 is ops # agora, ops2 não é ops
```

    False

```python
print(id(ops), id(ops2)) # imprime local na memória de ambas
```

    140614135474440 140614130646920

```python
ops2 == ops # todos os elementos são iguais
```

    True

O teste de identidade é `False`, mas o teste de igualdade é `True`.

**Exemplo:** Escreva uma função que calcule a área, perímetro, comprimento da diagonal, raio, perímetro e área do círculo inscrito, e armazene os resultados em uma lista.

```python
# usaremos matemática simbólica
from sympy import symbols
from math import pi

# símbolos
B, H = symbols('B H',positive=True)

def propriedades_retangulo(B,H):
    '''
        A função assume que a base B
        é maior do que a altura H. Senão,
        as propriedades do círculo inscrito
        não serão determinadas.
    '''
    d = (B**2 + H**2)**(1/2) # comprimento da diagonal
    r = H/2 # raio do círculo inscrito
    return [B*H, 2*(B+H), d, r, 2*pi*r, pi*(r)**2]

# lista de objetos símbolos
propriedades_retangulo(B,H)
```

    [B*H,
     2*B + 2*H,
     (B**2 + H**2)**0.5,
     H/2,
     3.14159265358979*H,
     0.785398163397448*H**2]

```python
# substituindo valores
B, H = 4.0, 2.5

propriedades_retangulo(B,H)
```

    [10.0, 13.0, 4.716990566028302, 1.25, 7.853981633974483, 4.908738521234052]

### Formatação de strings

O *template* a seguir usa a função `format` para substituição de valores indexados.

```python
templ = '{0} {1} ... {n}'.format(arg0,arg1,...,argn)
```

**Nota:** Para ajuda plena sobre formatação, consultar:

```python
help('FORMATTING')
```

```python
# considere R: retângulo; C: círculo inscrito

res = propriedades_retangulo(B,H) # resultado

props = ['Área de R',
         'Perímetro de R',
         'Diagonal de R',
         'Raio de C',
         'Perímetro de C',
         'Área de C'
        ] # propriedades

# template
templ = '{0:s} = {1:.2f}\n\
{2:s} = {3:.3f}\n\
{4:s} = {5:.4f}\n\
{6:s} = {7:.5f}\n\
{8:s} = {9:.6f}\n\
{10:s} = {11:.7f}'.format(props[0],res[0],\
                          props[1],res[1],\
                          props[2],res[2],\
                          props[3],res[3],\
                          props[4],res[4],\
                          props[5],res[5])

# impressão formatada
print(templ)
```

    Área de R = 10.00
    Perímetro de R = 13.000
    Diagonal de R = 4.7170
    Raio de C = 1.25000
    Perímetro de C = 7.853982
    Área de C = 4.9087385

### Como interpretar o que fizemos?

- `{0:s}` formata o primeiro argumento de `format`, o qual é `props[0]`, como `str` (`s`).
- `{1:.2f}` formata o segundo argumento de `format`, o qual é `res[0]`, como `float` (`f`) com duas casas decimais (`.2`).
- `{3:.3f}` formata o quarto argumento de `format`, o qual é `res[1]`, como `float` (`f`) com três casas decimais (`.3`).

A partir daí, percebe-se que um template `{X:.Yf}` diz para formatar o argumento `X` como `float` com `Y` casas decimais, ao passo que o template `{X:s}` diz para formatar o argumento `X` como `str`.

Além disso, temos:

- `\n`, que significa "newline", isto é, uma quebra da linha.
- `\`, que é um *caracter de escape* para continuidade da instrução na linha seguinte. No exemplo em tela, o *template* criado é do tipo *multi-line*.

**Nota:** a contrabarra em `\n` também é um caracter de escape e não um caracter *literal*. Isto é, para imprimir uma contrabarra literalmente, é necessário fazer `\\`. Vejamos exemplos de literais a seguir.

#### Exemplos de impressão de caracteres literais

```python
print('\\') # imprime contrabarra literal
print('\\\\') # imprime duas contrabarras literais
print('\'') # imprime plica
print('\"') # imprime aspas
```

    \
    \\
    '
    "

#### f-strings

Temos uma maneira bastante interessante de criar templates usando f-strings, que foi introduzida a partir da versão Python 3.6. Com f-strings a substituição é imediata.

```python
print(f'{props[0]} = {res[0]}') # estilo f-string
```

    Área de R = 10.0

#### Estilos de formatação

Veja um comparativo de estilos:

```python
print('%s = %f ' % (props[0], res[0])) # Python 2
print('{} = {}'.format(props[0], res[0])) # Python 3
print('{0:s} = {1:.4f}'.format(props[0], res[0])) # Python 3 formatado
```

    Área de R = 10.000000
    Área de R = 10.0
    Área de R = 10.0000

**Exemplo:** Considere o conjunto: V = $\{ c \in \mathbb{A} \, ; \, c \text{ é vogal} \}.$ Crie a concatenação de todos os elementos com f-string.

```python
V = ['a','e','i','o','u']
V
```

    ['a', 'e', 'i', 'o', 'u']

```python
f'{V[0]}{V[1]}{V[2]}{V[3]}{V[4]}' # pouco Pythônico
```

    'aeiou'

Veremos à frente meios mais elegantes de fazer coisas similares.

## Controle de fluxo: laço `for`

Em Python, podemos realizar iterar por uma coleção ou iterador usando *laços*. Introduziremos aqui o laço `for`. Em Python, o bloco padrão para este laço é dado por:

```python
for valor in sequencia:
    # faça algo com valor
```

Acima, `valor` é um iterador.

```python
for v in vogais: # itera sobre lista inteira
    print(v)
```

    a
    e
    i
    o
    u

```python
for v in vogais[0:3]: # itera parcialmente
    print(v + 'a')
```

    aa
    ea
    ia

```python
for v in vogais[-2:]:
    print(f'{v*10}')
```

    oooooooooo
    uuuuuuuuuu

## Compreensão de lista

Usando `for`, a criação de listas torna-se bastante facilitada.

**Exemplo:** crie a lista dos primeiros 10 quadrados perfeitos.

```python
Q = [q*q for q in range(1,11)]
Q
```

    [1, 4, 9, 16, 25, 36, 49, 64, 81, 100]

A operação acima equivale a:

```python
Q2 = []
for q in range(1,11):
    Q2.append(q*q)
Q2
```

    [1, 4, 9, 16, 25, 36, 49, 64, 81, 100]

**Exemplo:** crie a PA: $a_n = 3 + 6(n-1), \, 1 \leq n \leq 10$

```python
PA = [3 + 6*(n-1) for n in range(1,11) ]
PA
```

    [3, 9, 15, 21, 27, 33, 39, 45, 51, 57]

**Exemplo:** se $X = \{1,2,3\}$ e $Y=\{4,5,6\}$, crie a "soma" $X + Y$ elemento a elemento.

```python
X = [1,2,3]
Y = [4,5,6]

XsY = [ X[i] + Y[i] for i in range(len(X)) ]
XsY
```

    [5, 7, 9]

**Exemplo:** se $X = \{1,2,3\}$ e $Y=\{4,5,6\}$, cria o "produto" $X * Y$ elemento a elemento.

```python
XpY = [ X[i]*Y[i] for i in range(len(X)) ]
XpY

from sympy import lambdify
from sympy.abc import x

for i,v in enumerate(XpY)
    lambdify(x,'x**2')
```

## Tuplas

Tuplas são são sequencias imutáveis de tamanho fixo. Em Matemática, uma tupla é uma sequência ordenada de elementos. Em geral, o termo $n-$upla ("ênupla") é usado para se referir a uma tupla com $n$ elementos.

```python
par = 1,2; par
```

    (1, 2)

```python
type(par)
```

    tuple

```python
trio = (1,2,3); trio
```

    (1, 2, 3)

```python
quad = (1,2,3,4); quad
```

    (1, 2, 3, 4)

```python
nome = 'Nome'; tuple(nome) # casting
```

    ('N', 'o', 'm', 'e')

Tuplas são acessíveis por indexação.

```python
quad[2]
```

    3

```python
quad[1:4]
```

    (2, 3, 4)

```python
quad[3] = 5 # tuplas não são mutáveis
```

Se na tupla houver uma lista, a lista é modificável.

```python
super_trio = tuple([1,[2,3],4]) # casting
super_trio
```

    (1, [2, 3], 4)

```python
super_trio[1].extend([4,5])
super_trio
```

    (1, [2, 3, 4, 5], 4)

Tuplas também são concatenáveis com `+`.

```python
(2,3) + (4,3)
```

    (2, 3, 4, 3)

```python
('a',[1,2],(1,1)) # repetição
```

    ('a', [1, 2], (1, 1))

### Desempacotamento de tuplas

```python
a,b,c,d = (1,2,3,4)
```

```python
for i in [a,b,c,d]:
    print(i) # valor das variáveis
```

    1
    2
    3
    4

```python
a,b = (1,2)
a,b = b,a # troca de valores
a,b
```

    (2, 1)

### `enumerate`

Podemos controlar índice e valor ao iterar em uma sequencia.

```python
X = [1,2,3] # lista / sequencia

for i,x in enumerate(X): # (i,x) é uma tupla (índice,valor)
    print(f'{i} : {x}')
```

    0 : 1
    1 : 2
    2 : 3

**Exemplo:** Construa o produto cartesiano

$$A \times B = \{(a,b) \in \mathbb{Z} \times \mathbb{Z} \, ; \, -4 \leq a \leq 4 \wedge 3 \leq b \leq 7\}$$

```python
AB = [(a,b) for a in range(-4,5) for b in range(3,8)]
print(AB)
```

    [(-4, 3), (-4, 4), (-4, 5), (-4, 6), (-4, 7), (-3, 3), (-3, 4), (-3, 5), (-3, 6), (-3, 7), (-2, 3), (-2, 4), (-2, 5), (-2, 6), (-2, 7), (-1, 3), (-1, 4), (-1, 5), (-1, 6), (-1, 7), (0, 3), (0, 4), (0, 5), (0, 6), (0, 7), (1, 3), (1, 4), (1, 5), (1, 6), (1, 7), (2, 3), (2, 4), (2, 5), (2, 6), (2, 7), (3, 3), (3, 4), (3, 5), (3, 6), (3, 7), (4, 3), (4, 4), (4, 5), (4, 6), (4, 7)]

## Dicionários

Dicionários, ou especificamente, objetos `dict`, possuem extrema versatilidade e são muito poderosos. Criamos um `dict` por diversas formas. A mais simples é usar chaves e pares explícitos.

```python
d = {} # dict vazio
d
```

    {}

```python
type(d)
```

    dict

Os pares chave-valor incorporam quaisquer tipos de dados.

```python
d = {'par': [0,2,4,6,8], 'ímpar': [1,3,5,7,9], 'nome':'Meu dict', 'teste': True}
d
```

    {'par': [0, 2, 4, 6, 8],
     'ímpar': [1, 3, 5, 7, 9],
     'nome': 'Meu dict',
     'teste': True}

### Acesso a conteúdo

Para acessar o conteúdo de uma chave, indexamos pelo seu nome.

```python
d['par']
```

    [0, 2, 4, 6, 8]

```python
d['nome']
```

    'Meu dict'

**Exemplo:** construindo soma e multiplicação especial.

```python
# dict
op = {'X' :[1,2,3], 'delta' : 0.1}

# função
def sp(op):
    s = [x + op['delta'] for x in op['X']]
    p = [x * op['delta'] for x in op['X']]

    return (s,p) # retorna tupla

soma, prod = sp(op) # desempacota

for i,s in enumerate(soma):
    print(f'pos({i}) | Soma = {s} | Prod = {prod[i]}')
```

    pos(0) | Soma = 1.1 | Prod = 0.1
    pos(1) | Soma = 2.1 | Prod = 0.2
    pos(2) | Soma = 3.1 | Prod = 0.30000000000000004

### Inserção de conteúdo

```python
# apensa variáveis
op[1] = 3
op['novo'] = (3,4,1)
op
```

    {'X': [1, 2, 3], 'delta': 0.1, 1: 3, 'novo': (3, 4, 1)}

### Alteração de conteúdo

```python
op['novo'] = [2,1,4] # sobrescreve
op
```

    {'X': [1, 2, 3], 'delta': 0.1, 1: 3, 'novo': [2, 1, 4]}

### Deleção de conteúdo com `del` e `pop`

```python
del op[1] # deleta chave
op
```

    {'X': [1, 2, 3], 'delta': 0.1, 'novo': [2, 1, 4]}

```python
novo = op.pop('novo') # retorna e simultaneamente deleta
novo
```

    [2, 1, 4]

```python
op
```

    {'X': [1, 2, 3], 'delta': 0.1}

### Listagem de chaves e valores

Usamos os métodos `keys()` e `values()` para listar chaves e valores.

```python
arit = {'soma': '+', 'subtr': '-', 'mult': '*', 'div': '/'} # dict
k = list(arit.keys())
print(k)
val = list(arit.values())
print(val)
for v in range(len(arit)):
    print(f'A operação \'{k[v]}\' de "arit" usa o símbolo \'{val[v]}\'.')
```

    ['soma', 'subtr', 'mult', 'div']
    ['+', '-', '*', '/']
    A operação 'soma' de "arit" usa o símbolo '+'.
    A operação 'subtr' de "arit" usa o símbolo '-'.
    A operação 'mult' de "arit" usa o símbolo '*'.
    A operação 'div' de "arit" usa o símbolo '/'.

### Combinando dicionários

Usamos `update` para combinar dicionários. Este método possui um resultado similar a `extend`, usado em listas.

```python
pot = {'pot': '**'}
arit.update(pot)
arit
```

    {'soma': '+', 'subtr': '-', 'mult': '*', 'div': '/', 'pot': '**'}

### Dicionários a partir de sequencias

Podemos criar dicionários a partir de sequencias existentes usando `zip`.

```python
arit = ['soma', 'subtr', 'mult', 'div', 'pot']
ops = ['+', '-', '*', '/', '**']

dict_novo = {}

for chave,valor in zip(arit,ops):
    dict_novo[chave] = valor

dict_novo
```

    {'soma': '+', 'subtr': '-', 'mult': '*', 'div': '/', 'pot': '**'}

Visto que um `dict` é composto de várias tuplas de 2, podemos criar um  de maneira ainda mais simples.

```python
dict_novo = dict(zip(arit,ops))
dict_novo
```

    {'soma': '+', 'subtr': '-', 'mult': '*', 'div': '/', 'pot': '**'}

### *Hashability*

Dissemos acima que os valores de um `dict` podem ser qualquer objeto Python. Porém, as chaves estão limitadas por uma propriedade chamada *hashability*. Um objeto *hashable* em geral é imutável. Para saber se um objeto pode ser usado como chave de um `dict`, use a função `hash`. Caso retorne erro, a possibilidade de *hashing* é descartada.

```python
# todos aqui são imutáveis, portanto hashable'
hash('s'), hash(2), hash(2.1), hash((1,2))
```

    (2979719690768129234, 2, 230584300921369602, 3713081631934410656)

```python
# não hashable
hash([1,2]), hash((1,2),[3,4])
```

Para usar `list` como chave, podemos convertê-las em `tuple`.

```python
d = {}; d[tuple([1,2])] = 'hasheando lista em tupla'; d
```

    {(1, 2): 'hasheando lista em tupla'}

## Compreensão de dicionário

Podemos usar `for` para criar dicionários de maneira esperta do mesmo modo que as compreensões de lista com a distinção de incluir pares chaves/valor.

```python
{chave:valor for chave,valor in enumerate(arit)} # chave:valor
```

    {0: 'soma', 1: 'subtr', 2: 'mult', 3: 'div', 4: 'pot'}

```python
{valor:chave for chave,valor in enumerate(arit)} # valor:chave
```

    {'soma': 0, 'subtr': 1, 'mult': 2, 'div': 3, 'pot': 4}


```python
import numpy as np
from sympy import Matrix
```

```python
#a = [[6,1],[4,-2]]
#b = [[-3,6]]

#a = [[2,-8],[3,1]]
#b = [[-4,7]]

a = [[25.38,-15.48],[-14.10,8.60]]
b = [[30.60,-17]]
```

```python
A = Matrix(a)
```

```python
A
```

$\displaystyle \left[\begin{matrix}25.38 & -15.48\\-14.1 & 8.6\end{matrix}\right]$

```python
B = Matrix(b)
```

```python
B = B.T
```

```python
B
```

$\displaystyle \left[\begin{matrix}30.6\\-17\end{matrix}\right]$

```python
A.gauss_jordan_solve(B)
```

    (Matrix([
     [-0.0141843971631205],
     [               -2.0]]),
     Matrix(0, 1, []))

```python

```

```python

```


## Chebyshev Polynomials

In the last section, we found that a set of nodes with points more densely spaced at the edges of the interval seemed to work much better for polynomial interpolation than equally spaced points.  It turns out, the points we choose were roots of a Chebyshev polynomial.  Chebyshev polynomials are a set of orthogonal polynomials on the interval $[-1,1]$ with respect to the weight function $1/\sqrt{1-x^2}$.  They can be defined via trignometric functions as

$$ T_k(x)=\cos(k \arccos x).$$ (chebdef)

Note that we will restrict $x$ to be in $[-1,1]$ here so $\arccos x$ remains well-defined.  It should be clear from the definition in terms of the cosine function that $T_k(x)$ satisfies

>
> $$\begin{equation} -1 \geq T_k(x) \geq 1 \quad \text{for}\quad -1\geq x \geq 1\end{equation}.$$ (prop1)
>

and

>
> $T_k(x)$ has roots at $k \arccos x = (odd integer) \frac{\pi}{2}$ or
>
> $$\begin{equation} x_j=\cos\left( \frac{(2j+1)\pi}{2k} \right),\qquad j=0,...,k-1 \end{equation},$$ (prop2)
>

and

>
> $T_k(x)$ oscillates between $-1$ and $1$ a total of $k+1$ times, hitting these extremes at
>
>  $$x=\cos 0, \cos(\pi/k), ...,\cos((k-1)\pi/k)$$ (prop3)
>

Note that these zeros correspond to the projection onto $x$ of points spaced at equally spaced angles on the unit circle.  What is perhaps less clear is that Eq.{eq}`chebdef` corresponds to a polynomial at all.
However, for $k=0$ and $1$ it is easy to see that

$$\begin{align}
T_0(x)&=\cos(0 \arccos x) = 1,\\
T_1(x)&=\cos(1 \arccos x) = x.
\end{align}
$$

To see that we get a polynomial for higher $k$ it is helpful to derive the recursion relation for the Chebyshev polynomials.  If we set $y=\arccos x$ we have

$$\begin{align}
T_{k+1}(x)&=\cos\left( (k+1)y \right)= \cos(ky)\cos(y)-\sin(ky)\sin(y),\\
T_{k-1}(x)&=\cos\left( (k-1)y \right)= \cos(ky)\cos(y)+\sin(ky)\sin(y),
\end{align}$$

where we have used the trig-identies for the sum/difference of two angles.  Adding these equations gives

$$T_{k+1}(x)+T_{k-1}(x)=2\cos(ky)\cos(y)=2T_k(x)T_1(x)=2 x\, T_k(x),$$

making use of our previous expression that $T_1(x)=x$.  This can then be easily rearranged to give the recursion relation

$$T_{k+1}(x)=2 x\, T_k(x) - T_{k-1}(x).$$(chebrecursion)

As implied by the name, we can use this to recursively derive higher order polynomials given the two lower order ones.  Starting with the known $T_0(x)$ and $T_1(x)$ given above, this gives the next few polynomials as

$$\begin{align}
T_2(x)&= 2x^2-1,\\
T_3(x)&= 4x^3-3x,\\
T_4(x)&= 8x^4-8x^2+1.
\end{align}$$

From the pattern of these first few polynomials, and the multiplication by $2$ in the recursion it should also be clear that the leading coefficient (coefficient of the highest power of $x$) for $T_k(x)$ is $2^{k-1}$.  This, and the fact that we know the roots (in {eq}`prop2`) implies that

$$ \frac{T_{n+1}(x)}{2^n}=(x-x_0)(x-x_1)...(x-x_n)=\prod_{j=0}^n (x-x_j),$$ (Tnplus1)

where the $x_j$ are the roots of $T_{n+1}(x)$ from Equation {eq}`prop2`.

This now brings us back to the Error formula for polynomial interpolation that we derived in the previous section:

$$\begin{equation} f(x)-p_n(x) = \frac{f^{n+1}(\xi)}{(n+1)!}\prod_{j=0}^n (x-x_j). \end{equation}$$

*If* we select our interpolation nodes as the roots of $T_{n+1}(x)$, namely

$$ x_i=\cos\left( \frac{(2i+1)\pi}{2n+2} \right), \qquad\qquad i=0, 1, ..., n. $$

then we see that we can substitute {eq}`Tnplus1` into the error formula to get

$$\begin{equation} f(x)-p_n(x) = \frac{f^{n+1}(\xi)}{(n+1)!}\frac{T_{n+1}}{2^n},\end{equation}$$

so that

$$\begin{align} |f(x)-p_n(x)| &= \left|\frac{f^{n+1}(\xi)}{(n+1)!}\frac{T_{n+1}}{2^n}\right|,\\ &\leq \frac{|f^{n+1}(\xi)|}{(n+1)!2^n}.\end{align} $$

In fact, we can also show that *this is the best* we can do.

Again, we show this by first assuming the contrary and then showing that this leads to a contradiction.  In order for this to be the best choice, we must have $T_{n+1}(x)/2^n$ to be the monic polynomial (a polynomial with leading coefficient 1) with the smallest absolute maximum.  Suppose this is nto the case.  Then there must be another monic polynomial, say $Q_{n+1}(x)$ of degree $n$ with an even smaller absolute max on $[-1,1]$.  i.e.

$$ |Q_{n+1}(x)| < \frac{1}{2^n}$$

for $x\in [-1,1]$.

Since $T_{n+1}(x)$ alternates between $-1$ and $1$ a total of $n+2$ times (as noted in {eq}`(prop3)`), at these $n+2$ points the polynomial

$$Q_{n+1}(x)-\frac{T_{n+1}(x)}{2^n}$$

is alternatively positive and negative (because $Q_{n+1}$ is always less than $1/2^n$) at these extreme points for $T_{n+1}(x)$.  The intermediate value theorem then implies that $Q_{n+1}-\frac{T_{n+1}}{2^n}$ must cross zero at least $n+1$ times so it has $n+1$ roots.  This is a contradition as both $Q_{n+1}$ and $\frac{T_{n+1}}{2^n}(x)$ are monic so the leading term cancels in subtraction and therefore $Q_{n+1}-\frac{T_{n+1}}{2^n}$ has degree less than or equal to $n$ and so has at most $n$ zeros.

So, in summary,

- If you want to use polynomial interpolation to approximate a function on $[-1,1]$, select Chebyshev points as your interpolation nodes.

- When you do this, you can also bound the error if you can also bound $|f^{n+1}(\xi)|$.

Chebyshev points are in $[-1,1]$ but you should keep in mind that any interval $x \in [a,b]$ can be easily mapped into $y\in [-1,1]$ via a linear transformation

$$ x = \frac{b-a}{2}y+\frac{b+a}{2}. $$

Constructing the polynomial interpolation for $y$ can then be done using Chebyshev points.

We illustrate these points with an example below.

**Example** Let's construct a fourth order polynomial intepolation ($n=4$) for $f(x)=e^{-x^2}$ on the interval $[0,1]$.

We first need to map this interval onto $[-1,1]$ using the change of variables

$$ x = \frac{1}{2}(y+1).$$

We then select our Chebyshev points as

$$y_0 = \cos\frac{\pi}{10},\,y_1=\cos \frac{3\pi}{10}\,y_1=\cos \frac{5\pi}{10}\,y_1=\cos \frac{7\pi}{10}\,y_1=\cos \frac{9\pi}{10}.$$

and construct $p_4(y)$ for $f(y)=e^{-(y+1)^2/4}$ using the standard barycentric Lagrange interpolation routine.

```python
import matplotlib.pyplot as plt
import numpy as np
import math
from scipy.interpolate import barycentric_interpolate
n = 5
x_observed = np.cos((2.*np.arange(0,n+1,1)+1.)*math.pi/(2.*n+2))
y_observed = np.exp(-(x_observed+1.)**2/4)
x = np.linspace(min(x_observed), max(x_observed), num=100)
y = barycentric_interpolate(x_observed, y_observed, x)
y_actual = np.exp(-(x+1.)**2/4)
plt.plot((x_observed+1)/2, y_observed, "o", label="observation")
plt.plot((x+1)/2, y, label="5-node interpolation")
plt.plot((x+1)/2, y_actual, label="f(x)",linestyle='dashed')
plt.legend()
plt.show()
```

We can also construct an error bound.  The fifth derivative with respect to $x$ is $d^5 f(x)/dx^5=(-120x+160x^3-32x^5)e^{-x^2}$.  A quick plot (not shown, you can do this yourself) of this on the interval $[0,1]$ easily shows that $|d^5 f(x)/dx^5| < 33$.  As we are doing the interpolation along the transformed variable $y$ rather than $x$, to use the error formula we actually also need

$$\frac{df}{dy} = \frac{df}{dx} \frac{dx}{dy} = \frac{1}{2} \frac{df}{dx}.$$

Putting this all together gives

$$|f(y)-p_n(y)| \leq \frac{|f^{n+1}(\xi)|}{(n+1)!2^n} \leq \frac{33/2^5}{5!2^4} = 5\times 10^{-5}.$$

In this case we can also work out the actual error:

```python
plt.plot((x+1)/2, np.abs(y-y_actual),label="Actual Error")
plt.legend()
plt.show()
```

The largest actual error is about 1/2 our error bound so the bound slightly overestimates the error (but that is generally what you expect for a *bound*).

```python

```


Non Linear Regression Analysis

If the data shows a curvy trend, then linear regression will not produce very accurate results when compared to a non-linear regression because, as the name implies, linear regression presumes that the data is linear.
Let's learn about non linear regressions and apply an example on python. In this notebook, we fit a non-linear model to the datapoints corrensponding to China's GDP from 1960 to 2014.

Importing required libraries

```python
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
```

Though Linear regression is very good to solve many problems, it cannot be used for all datasets. First recall how linear regression, could model a dataset. It models a linear relation between a dependent variable y and independent variable x. It had a simple equation, of degree 1, for example y = $2x$ + 3.

```python
##Create a sample x
##Create regression line with slope equation
##Add a random noise and plot y for value of x.

x = np.arange(-5.0, 5.0, 0.1) #create x from -5 to +5 with 0.1 interval

##You can adjust the slope and intercept to verify the changes in the graph
y = 2*(x) + 3
#insert a sample noise equal to size of x
y_noise = 2 * np.random.normal(size=x.size)
ydata = y + y_noise

#print(y_noise)
#plt.plot(x, y_noise,  'ro')

#plt.figure(figsize=(8,6)) #for a bigger figure output
plt.plot(x, ydata,  'bo') #blue o markers
plt.plot(x,y, 'r') #x and y of color red
plt.ylabel('Dependent Variable')
plt.xlabel('Indepdendent Variable')
plt.show()
```

Non-linear regressions are a relationship between independent variables $x$ and a dependent variable $y$ which result in a non-linear function modeled data. Essentially any relationship that is not linear can be termed as non-linear, and is usually represented by the polynomial of $k$ degrees (maximum power of $x$).

$$ \ y = a x^3 + b x^2 + c x + d \ $$

Non-linear functions can have elements like exponentials, logarithms, fractions, and others. For example: $$ y = \log(x)$$

Or even, more complicated such as :
$$ y = \log(a x^3 + b x^2 + c x + d)$$

Let's take a look at a cubic function's graph.

```python
##Non linear - cubic function 𝑦=𝑎𝑥3+𝑏𝑥2+𝑐𝑥+𝑑
x = np.arange(-5.0, 5.0, 0.1) #This will form 50+50 on each side = 100 points
print(x.size)

##You can adjust the slope and intercept to verify the changes in the graph
y = 1*(x**3) + 1*(x**2) + 1*x + 3
y_noise = 20 * np.random.normal(size=x.size)
#print(y_noise) not packed to cubic fuction
#perfect graph for cubic
#ydata = y
#With noise
ydata = y + y_noise

plt.plot(x, ydata,  'bo')
plt.plot(x,y, 'r')
plt.ylabel('Dependent Variable')
plt.xlabel('Indepdendent Variable')
plt.show()
```

As you can see, this function has $x^3$ and $x^2$ as independent variables. Also, the graphic of this function is not a straight line over the 2D plane. So this is a non-linear function.

Some other types of non-linear functions are:

### Quadratic

$$ Y = X^2 $$

```python
x = np.arange(-5.0, 5.0, 0.1)

##You can adjust the slope and intercept to verify the changes in the graph

y = np.power(x,2)
y_noise = 2 * np.random.normal(size=x.size)
ydata = y + y_noise

plt.plot(x, ydata,  'bo')
plt.plot(x,y, 'r')
plt.ylabel('Dependent Variable')
plt.xlabel('Indepdendent Variable')
plt.show()
```

### Exponential

An exponential function with base c is defined by $$ Y = a + b c^X$$ where b ≠0, c > 0 , c ≠1, and x is any real number. The base, c, is constant and the exponent, x, is a variable.

```python
X = np.arange(-5.0, 5.0, 0.1)

##You can adjust the slope and intercept to verify the changes in the graph

Y= np.exp(X)
y_noise = 9 * np.random.normal(size=X.size)
ydata = Y + y_noise

plt.plot(X,Y, 'r')
plt.plot(X,ydata, 'bo')
plt.ylabel('Dependent Variable')
plt.xlabel('Indepdendent Variable')
plt.show()
```

### Logarithmic

The response $y$ is a results of applying logarithmic map from input $x$'s to output variable $y$. It is one of the simplest form of __log()__: i.e. $$ y = \log(x)$$

Please consider that instead of $x$, we can use $X$, which can be polynomial representation of the $x$'s. In general form it would be written as
\begin{equation}
y = \log(X)
\end{equation}

```python
X = np.arange(-5.0, 5.0, 0.1)

Y = np.log(X)
y_noise = 0.25 * np.random.normal(size=X.size)
ydata = Y + y_noise

plt.plot(X,Y, 'r')
plt.plot(X,ydata, 'bo')
plt.ylabel('Dependent Variable')
plt.xlabel('Indepdendent Variable')
plt.show()
```

### Sigmoidal/Logistic

$$ Y = a + \frac{b}{1+ c^{(X-d)}}$$

```python
X = np.arange(-5.0, 7.0, 0.2)

Y = 1-4/(1+np.power(3, X-2))
y_noise = 0.25 * np.random.normal(size=X.size)
ydata = Y + y_noise

plt.plot(X,Y, 'r')
plt.plot(X,ydata, 'bo')
plt.ylabel('Dependent Variable')
plt.xlabel('Indepdendent Variable')
plt.show()
```

# Non-Linear Regression example

For an example, we're going to try and fit a non-linear model to the datapoints corresponding to China's GDP from 1960 to 2014. We download a dataset with two columns, the first, a year between 1960 and 2014, the second, China's corresponding annual gross domestic income in US dollars for that year.

```python
import numpy as np
import pandas as pd

#downloading dataset
!wget -nv -O china_gdp.csv https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/china_gdp.csv

df = pd.read_csv("china_gdp.csv")
df.head(10)
```

    2020-11-22 05:07:03 URL:https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/china_gdp.csv [1218/1218] -> "china_gdp.csv" [1]

      Year
      Value

      0
      1960
      5.918412e+10

      1
      1961
      4.955705e+10

      2
      1962
      4.668518e+10

      3
      1963
      5.009730e+10

      4
      1964
      5.906225e+10

      5
      1965
      6.970915e+10

      6
      1966
      7.587943e+10

      7
      1967
      7.205703e+10

      8
      1968
      6.999350e+10

      9
      1969
      7.871882e+10

__Did you know?__ When it comes to Machine Learning, you will likely be working with large datasets. As a business, where can you host your data? IBM is offering a unique opportunity for businesses, with 10 Tb of IBM Cloud Object Storage: [Sign up now for free](http://cocl.us/ML0101EN-IBM-Offer-CC)

### Plotting the Dataset ###
This is what the datapoints look like. It kind of looks like an either logistic or exponential function. The growth starts off slow, then from 2005 on forward, the growth is very significant. And finally, it decelerate slightly in the 2010s.

```python
plt.figure(figsize=(8,5))#define size
x_data, y_data = (df["Year"].values, df["Value"].values) #Get values from Panda df
plt.plot(x_data, y_data, 'ro')
plt.ylabel('GDP')
plt.xlabel('Year')
plt.show()
```

### Choosing a model ###

From an initial look at the plot, we determine that the logistic function could be a good approximation,
since it has the property of starting with a slow growth, increasing growth in the middle, and then decreasing again at the end; as illustrated below:

```python
X = np.arange(-5.0, 5.0, 0.1)
Y = 1.0 / (1.0 + np.exp(-X))

plt.plot(X,Y)
plt.ylabel('Dependent Variable')
plt.xlabel('Indepdendent Variable')
plt.show()
```

The formula for the logistic function is the following:

$$ \hat{Y} = \frac1{1+e^{\beta_1(X-\beta_2)}}$$

$\beta_1$: Controls the curve's steepness,

$\beta_2$: Slides the curve on the x-axis.

### Building The Model ###
Now, let's build our regression model and initialize its parameters.

```python
def sigmoid(x, Beta_1, Beta_2):
     y = 1 / (1 + np.exp(-Beta_1*(x-Beta_2)))
     return y
```

Lets look at a sample sigmoid line that might fit with the data:

```python
beta_1 = 0.10
beta_2 = 1990.0

#logistic function
Y_pred = sigmoid(x_data, beta_1 , beta_2)

#plot initial prediction against datapoints
plt.plot(x_data, Y_pred*15000000000000.)
plt.plot(x_data, y_data, 'ro')
```

Our task here is to find the best parameters for our model. Lets first normalize our x and y:

```python
# Lets normalize our data - Feature Scaling - Max Normalization
xdata =x_data/max(x_data)
ydata =y_data/max(y_data)
```

#### How we find the best parameters for our fit line?
we can use __curve_fit__ which uses non-linear least squares to fit our sigmoid function, to data. Optimal values for the parameters so that the sum of the squared residuals of sigmoid(xdata, *popt) - ydata is minimized.

popt are our optimized parameters.

```python
#calculate covarinace matrix and optimal parametrs of theta(We take only optimal params)
from scipy.optimize import curve_fit
popt, pcov = curve_fit(sigmoid, xdata, ydata)
#print the final parameters
print(" beta_1 = %f, beta_2 = %f" % (popt[0], popt[1]))
```

     beta_1 = 690.447527, beta_2 = 0.997207

Now we plot our resulting regression model.

```python
x = np.linspace(1960, 2015, 55)#linear spacing between given interval
x = x/max(x)
plt.figure(figsize=(8,5))
y = sigmoid(x, *popt)
plt.plot(xdata, ydata, 'ro', label='data')
plt.plot(x,y, linewidth=3.0, label='fit')
plt.legend(loc='best')
plt.ylabel('GDP')
plt.xlabel('Year')
plt.show()
```

## Practice
Can you calculate what is the accuracy of our model?

```python
# split data into train/test(this method we use without scikit learn)
msk = np.random.rand(len(df)) < 0.8
train_x = xdata[msk]
test_x = xdata[~msk]
train_y = ydata[msk]
test_y = ydata[~msk]

# build the model using train set
popt, pcov = curve_fit(sigmoid, train_x, train_y)

# predict using test set
y_hat = sigmoid(test_x, *popt)

# evaluation
print("Mean absolute error: %.2f" % np.mean(np.absolute(y_hat - test_y)))
print("Residual sum of squares (MSE): %.2f" % np.mean((y_hat - test_y) ** 2))
from sklearn.metrics import r2_score
print("R2-score: %.2f" % r2_score(y_hat , test_y) )

```

    Mean absolute error: 0.03
    Residual sum of squares (MSE): 0.00
    R2-score: 0.98

Double-click __here__ for the solution.

Want to learn more?

IBM SPSS Modeler is a comprehensive analytics platform that has many machine learning algorithms. It has been designed to bring predictive intelligence to decisions made by individuals, by groups, by systems – by your enterprise as a whole. A free trial is available through this course, available here: SPSS Modeler

Also, you can use Watson Studio to run these notebooks faster with bigger datasets. Watson Studio is IBM's leading cloud solution for data scientists, built by data scientists. With Jupyter notebooks, RStudio, Apache Spark and popular libraries pre-packaged in the cloud, Watson Studio enables data scientists to collaborate on their projects without having to install anything. Join the fast-growing community of Watson Studio users today with a free account at Watson Studio

Thanks for completing this lesson!

Author:  Saeed Aghabozorgi

Saeed Aghabozorgi, PhD is a Data Scientist in IBM with a track record of developing enterprise level applications that substantially increases clients’ ability to turn data into actionable knowledge. He is a researcher in data mining field and expert in developing advanced analytic methods like machine learning and statistical modelling on large datasets.

Copyright © 2018 Cognitive Class. This notebook and its source code are released under the terms of the MIT License.

```python

```


# DMI Types - Symbolically Generating the Finite Difference Form

In Fidimag, we have implemented the Dzyaloshinskii-Moriya interaction in such a way as to allow a user to specify any type of DMI they want.

Here we provide a symbolic algebra approach to the generation of such systems, which reduces the amount of tedious maths that one needs to do to use a new DMI type. All one needs to know is the combination of Lifshitz Invariants that make up the energy functional. This is trivial to extend to DMI types which have more than one DMI parameter.

It is worth noting that there is some discrepancy in the literature about the signs of the DMI parameter; it may be that to be consistent with the literature you must make the mapping $D$ to $-D$ for a particular energy class.

```python
import numpy as np
import sympy as sp
from sympy.abc import x, y, z, i, j, k
from sympy import init_printing
import sympy.vector
from sympy.vector import Dot
init_printing()
N = sp.vector.CoordSys3D('N')
```

```python
Mx, My, Mz = sp.symbols('M_x M_y M_z', cls=sp.Function)
D1, D2 = sp.symbols('D_1 D_2')
dl = sp.Symbol('\delta l')

def cross(a, b):
    return (a[1]*b[2] - a[2]*b[1],
            a[2]*b[0] - a[0]*b[2],
            a[0]*b[1] - a[1]*b[2])

def LI(i, j, k):
    """
    Symbolic representation of the Lifshitz Invariant

    $ L_{ij}^{(k)) $

    """
    Mi, Mj = sp.symbols(f'M_{str(i)} M_{str(j)}', cls=sp.Function)

    return Mi(x,y,z)*sp.diff(Mj(x, y, z), k) - Mj(x,y,z)*sp.diff(Mi(x,y,z), k)

def VarDeriv(w):
    Fx = sp.diff(w, Mx(x, y, z)) - sp.diff(sp.diff(w, sp.diff(Mx(x, y, z), x)), x) \
                                 - sp.diff(sp.diff(w, sp.diff(Mx(x, y, z), y)), y) \
                                 - sp.diff(sp.diff(w, sp.diff(Mx(x, y, z), z)), z)

    Fy = sp.diff(w, My(x, y, z)) - sp.diff(sp.diff(w, sp.diff(My(x, y, z), x)), x) \
                                 - sp.diff(sp.diff(w, sp.diff(My(x, y, z), y)), y) \
                                 - sp.diff(sp.diff(w, sp.diff(My(x, y, z), z)), z)

    Fz = sp.diff(w, Mz(x, y, z)) - sp.diff(sp.diff(w, sp.diff(Mz(x, y, z), x)), x) \
                                 - sp.diff(sp.diff(w, sp.diff(Mz(x, y, z), y)), y) \
                                 - sp.diff(sp.diff(w, sp.diff(Mz(x, y, z), z)), z)

    return sp.simplify(Fx)*N.x + sp.simplify(Fy)*N.y + sp.simplify(Fz)*N.z

def discretised_form(field):
    subs = {N.x:N.i, N.y:N.j, N.z:N.k}
    disc = sp.as_finite_diff(field, 2*dl).expand()
    xm =  sp.Add(*[argi for argi in disc.args if Mx(-dl + x, y, z) in argi.atoms(sp.Function)])
    xm += sp.Add(*[argi for argi in disc.args if My(-dl + x, y, z) in argi.atoms(sp.Function)])
    xm += sp.Add(*[argi for argi in disc.args if Mz(-dl + x, y, z) in argi.atoms(sp.Function)])
    xp =  sp.Add(*[argi for argi in disc.args if Mx(x + dl, y, z) in argi.atoms(sp.Function)])
    xp += sp.Add(*[argi for argi in disc.args if My(x + dl, y, z) in argi.atoms(sp.Function)])
    xp += sp.Add(*[argi for argi in disc.args if Mz(x + dl, y, z) in argi.atoms(sp.Function)])
    ym =  sp.Add(*[argi for argi in disc.args if Mx(x, y - dl, z) in argi.atoms(sp.Function)])
    ym += sp.Add(*[argi for argi in disc.args if My(x, y - dl, z) in argi.atoms(sp.Function)])
    ym += sp.Add(*[argi for argi in disc.args if Mz(x, y - dl, z) in argi.atoms(sp.Function)])
    yp =  sp.Add(*[argi for argi in disc.args if Mx(x, y + dl, z) in argi.atoms(sp.Function)])
    yp += sp.Add(*[argi for argi in disc.args if My(x, y + dl, z) in argi.atoms(sp.Function)])
    yp += sp.Add(*[argi for argi in disc.args if Mz(x, y + dl, z) in argi.atoms(sp.Function)])
    zm =  sp.Add(*[argi for argi in disc.args if Mx(x, y, z - dl) in argi.atoms(sp.Function)])
    zm += sp.Add(*[argi for argi in disc.args if My(x, y, z - dl) in argi.atoms(sp.Function)])
    zm += sp.Add(*[argi for argi in disc.args if Mz(x, y, z - dl) in argi.atoms(sp.Function)])
    zp =  sp.Add(*[argi for argi in disc.args if Mx(x, y, z + dl) in argi.atoms(sp.Function)])
    zp += sp.Add(*[argi for argi in disc.args if My(x, y, z + dl) in argi.atoms(sp.Function)])
    zp += sp.Add(*[argi for argi in disc.args if Mz(x, y, z + dl) in argi.atoms(sp.Function)])
    return xm.subs(subs), xp.subs(subs), ym.subs(subs), yp.subs(subs), zm.subs(subs), zp.subs(subs)

def discretised_form_to_vector_1D(dform):
    total_list = []

    for coord in dform:
        all_symbols = list(coord.free_symbols)
        if N.i not in all_symbols and \
           N.j not in all_symbols and \
           N.k not in all_symbols:
            total_list.append([0.0, 0, 0])

        elif N.i not in all_symbols:
            jterm = coord.coeff(N.k, 0)
            kterm = coord.coeff(N.j, 0)

            val = list(jterm.as_coefficients_dict().values())[0]
            if val == 1:
                total_list.append([1., 0, 0])
            else:
                total_list.append([-1., 0, 0])
        elif N.j not in list(coord.free_symbols):
            iterm = coord.coeff(N.k, 0)
            kterm = coord.coeff(N.i, 0)

            val = list(iterm.as_coefficients_dict().values())[0]
            if val == -1:
                total_list.append([0., 1, 0])
            else:
                total_list.append([0., -1, 0])
        else:
            iterm = coord.coeff(N.j, 0)
            jterm = coord.coeff(N.i, 0)
            val = list(iterm.as_coefficients_dict().values())[0]
            if val == 1:
                total_list.append([0., 0, 1])
            else:
                total_list.append([0., 0, -1])

    total_list = np.array(total_list).flatten()
    return total_list

def discretised_form_to_vector_2D(dform):
    total_list = []
    for D in [D1, D2]:
        for coord in dform:
            all_symbols = list(coord.coeff(D).free_symbols)
            if N.i not in all_symbols and \
               N.j not in all_symbols and \
               N.k not in all_symbols:
                total_list.append([0.0, 0, 0])

            elif N.i not in all_symbols:
                jterm = coord.coeff(N.k, 0)
                kterm = coord.coeff(N.j, 0)

                val = list(jterm.as_coefficients_dict().values())[0]
                if val == 1:
                    total_list.append([1., 0, 0])
                else:
                    total_list.append([-1., 0, 0])
            elif N.j not in list(coord.free_symbols):
                iterm = coord.coeff(N.k, 0)
                kterm = coord.coeff(N.i, 0)

                val = list(iterm.as_coefficients_dict().values())[0]
                if val == -1:
                    total_list.append([0., 1, 0])
                else:
                    total_list.append([0., -1, 0])
            else:
                iterm = coord.coeff(N.j, 0)
                jterm = coord.coeff(N.i, 0)
                val = list(iterm.as_coefficients_dict().values())[0]
                if val == 1:
                    total_list.append([0., 0, 1])
                else:
                    total_list.append([0., 0, -1])

    total_list = np.array(total_list).flatten()
    return total_list

```

# Crystal Class T

```python
T = D1 * (LI(x, y, z) + LI(y, z, x) + LI(z, x, y))
T
```

```python
F_T = VarDeriv(T)
F_T
```

```python
xm, xp, ym, yp, zm, zp = discretised_form(F_T)
dform = xm , xp, ym, yp, zm, zp
```

    /usr/local/lib/python3.7/site-packages/sympy/core/decorators.py:38: SymPyDeprecationWarning:

    _as_finite_diff has been deprecated since SymPy 1.1. Use
    Derivative.as_finite_difference instead. See
    https://github.com/sympy/sympy/issues/11410 for more info.

      _warn_deprecation(wrapped, 3)

```python
discretised_form_to_vector_1D(dform)
```

    array([-1.,  0.,  0.,  1.,  0.,  0.,  0., -1.,  0.,  0.,  1.,  0.,  0.,
            0., -1.,  0.,  0.,  1.])

# Crystal Class C_nv

```python
C_nv = D1 * (LI(x, z, x) + LI(y, z, y))
C_nv
```

```python
F_C_nv = VarDeriv(C_nv)
F_C_nv
```

```python
dform_C_nv = discretised_form(F_C_nv)
dform_C_nv
```

```python
discretised_form_to_vector_1D(dform_C_nv)
```

    array([ 0.,  1.,  0.,  0., -1.,  0., -1.,  0.,  0.,  1.,  0.,  0.,  0.,
            0.,  0.,  0.,  0.,  0.])

# Crystal Class D_2d

```python
D_2d = D1 * (LI(y, z, x) + LI(x, z, y))
D_2d
```

```python
F_D_2d = VarDeriv(D_2d)
F_D_2d
```

```python
dform_D_2d = discretised_form(F_D_2d)
dform
```

```python
discretised_form_to_vector_1D(dform_D_2d)
```

    array([-1.,  0.,  0.,  1.,  0.,  0.,  0.,  1.,  0.,  0., -1.,  0.,  0.,
            0.,  0.,  0.,  0.,  0.])

# D_n

See here for details:

https://arxiv.org/pdf/1406.2177.pdf

```python
D_n = D1 * (LI(x, z, y) + LI(y, z, x)) + D2*LI(x, y, z)
D_n
```

```python
F_D_n = VarDeriv(D_n)
F_D_n
```

```python
dform_D_n = discretised_form(F_D_n)
dform_D_n
```

```python
discretised_form_to_vector_2D(dform_D_n)
```

    array([-1.,  0.,  0.,  1.,  0.,  0.,  0.,  1.,  0.,  0., -1.,  0.,  0.,
            0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,
            0.,  0.,  0.,  0.,  0.,  0., -1.,  0.,  0.,  1.])

# C_n

```python
C_n = D1 * (LI(y, z, y) + LI(x, z, x)) + D2 * (LI(y, z, x) - LI(x, z, y))
C_n
```

```python
F_C_n = VarDeriv(C_n)
F_C_n
```

```python
dform_C_n = discretised_form(F_C_n)
discretised_form_to_vector_2D(dform_C_n)
```

    /usr/local/lib/python3.7/site-packages/sympy/core/decorators.py:38: SymPyDeprecationWarning:

    _as_finite_diff has been deprecated since SymPy 1.1. Use
    Derivative.as_finite_difference instead. See
    https://github.com/sympy/sympy/issues/11410 for more info.

      _warn_deprecation(wrapped, 3)

    array([ 0.,  0.,  1.,  0.,  0.,  1.,  1.,  0.,  0.,  1.,  0.,  0.,  0.,
            0.,  0.,  0.,  0.,  0., -1.,  0.,  0.,  1.,  0.,  0.,  0.,  0.,
            1.,  0.,  0.,  1.,  0.,  0.,  0.,  0.,  0.,  0.])

sandbox/more-dmi/Generating the Finite Difference Forms of Different types of DMI.ipynb


Created by Zsolt Elter (2019)

# Group cross sections

The objectives of this notebook are the following:

- Display a simplified model of the neutron spectrum
- Demonstrate how group cross sections can be created

Exercise at the end!

## A simplified LWR spectrum

As a simplification let us consider the neutron spectrum ad the product of a normalized energy distribution function and the flux, a dimensioned quantity.

The distribution function is split in three components.

__Thermal component__: described by the Maxwell-distribution

$$f_{th}(E)=\alpha_{th}E\exp\big(-\frac{E}{kT}\big)$$

where $\alpha_{th}$ is a normalization constant, $k$ is the Boltzmann-constant, $T$ is the thermodynamical temperature (in K) and $E$ is the energy (in eV).

__Epithermal compontent__: Described by a $1/E$ slope within a piecewise function to account for the thermal and fast limit values:

\begin{equation}
    f_{epi}(E) =
    \begin{cases}
      \alpha_{epi}\frac{E^2-E_0^2}{E(E_1^2-E_0^2)} & \text{if $E_0\leq E \leq E_1$} \\
      \alpha_{epi}\frac{1}{E} & \text{if $E_1\leq E \leq E_2$} \\
      \alpha_{epi}\frac{E^2-E_3^2}{E(E_2^2-E_3^2)} & \text{if $E_2\leq E \leq E_3$} \\
      0 & \text{else}
    \end{cases}
\end{equation}

Where $\alpha_{epi}$ is a normalization constant, and the energy limits are chosen to be $E_0=0 eV$, $E_1=0.2 eV$ $E_25=0.25 eV$ and $E_3=1 MeV$

__Fast component__: corresponds to the fisson-spectrum (also known as Watt-spectrum) described by the following empirical formula:

$$f_{fast}(E)=\alpha_{fast}\exp\big(-\frac{E}{a}\big)\sinh(\sqrt{bE})$$

where $\alpha_{fast}$ is a normalization constant and for U-235 the parameters are $a=9.65\cdot 10^5 eV$ and $b=2.29\cdot 10^{-6} eV^{-1}$. But in the example we will use slightly different values (since the fission does not happen only in U-235).

The normalization is done so that $\int_0^\infty f_{th}(E)dE=\int_0^\infty f_{epi}(E)dE=\int_0^\infty f_{fast}(E)dE=1$. In practice this means that the above defined distributions will be divided by their integral.

Finally the spectrum will be a sum of these components whit some appropriate flux values:

$$\Phi(E) = \Phi_{th}f_{th}(E) + \Phi_{epi}f_{epi}(E) + \Phi_{fast}f_{fast}(E)$$.

For the following let's consider the constants to be

- $\Phi_{th} = 3.2\cdot 10^{14}$ ncm$^{-2}$s$^{-1}$
- $\Phi_{epi} = 1.98\cdot 10^{14}$ ncm$^{-2}$s$^{-1}$
- $\Phi_{fast} = 0.98\cdot 10^{14}$ ncm$^{-2}$s$^{-1}$
- $T=325$ K
- $a=8.09\cdot 10^5$ eV
- $b=9.32\cdot 10^{-7}$ eV$^{-1}$

These values and the model are taken from NIMA Vol 593 pp510-518 (https://www.sciencedirect.com/science/article/pii/S016890020800795X), and describe the spectrum in the BR2 reactor in Mol, Belgium. Nevertheless the structure gives a fair representation of any reactor spectrum if appropriate flux values are provided. One can browse some reference spectra to see similar characteristics. The main lack is in this model the sudden decreases in the resonance region characteristic to certain reactor materials eg. due the elastic scattering of Na-23 (https://www.nuclear-power.net/wp-content/uploads/2016/01/thermal-vs-fast-reactor-neutron-spectrum-min.png).

Let us implement this spectrum as a function of energy. Don't be afraid to play around with the parameters to see how it changes the spectrum.

```python
import numpy as np
import matplotlib.pyplot as plt

def getSpectrum(energy):
    """Approximating the spectrum as in NIMA Vol 593 pp510-518 paper.
    Constants describing the flux are taken from the same paper"""
    #thermal
    fth=[]
    k=8.6173324e-5 #eV/K
    T=325 #K
    for E in energy:
        fth.append(E*np.exp(-E/(k*T)))   #who needs pythonic way?
    energy=np.array(energy)
    fth=np.array(fth)
    fth=fth/np.trapz(fth,energy) #normalize

    #epithermal
    fepi=[]
    E0=0
    E1=0.2
    E2=0.25e6
    E3=1e6
    for E in energy:
        if E0<=E and E<E1:
            fepi.append((E**2-E0**2)/(E*(E1**2-E0**2)))
        elif E1<=E and E<E2:
            fepi.append(1/E)
        elif E2<=E and E<E3:
            fepi.append((E**2-E3**2)/(E*(E2**2-E3**2)))
        else:
            fepi.append(0.0)
    fepi=np.array(fepi)
    fepi=fepi/np.trapz(fepi,energy)

    #fast
    ffast=[]
    a=8.09e5 #u235 9.65e5
    b=9.32e-7 #u235 2.29e-6
    for E in energy:
        ffast.append(np.exp(-E/a)*np.sinh(np.sqrt(b*E)))
    ffast=np.array(ffast)
    ffast=ffast/np.trapz(ffast,energy)

    #create real spectrum
    PhiTh=3.2e14
    PhiEpi=1.98e14
    PhiFast=0.98e14

    phi=PhiTh*fth+PhiEpi*fepi+PhiFast*ffast
    #let's return all the components separetaly for plotting. But in practice "return phi" is enough
    return phi, PhiTh*fth,PhiEpi*fepi,PhiFast*ffast
```

```python
energy=np.logspace(-6,7,1000) #in eV
spectrum,spectrumTh,spectrumEpi,spectrumFast=getSpectrum(energy)

plt.loglog(energy,spectrum,label='total')
plt.ylim([1e3,1e16])
plt.loglog(energy,spectrumTh,'--',label='thermal')
plt.loglog(energy,spectrumEpi,'-.',label='epithermal')
plt.loglog(energy,spectrumFast,':',label='fast')
plt.legend()
plt.xlabel('Energy (eV)')
plt.ylabel(r'Flux (ncm$^{-2}$s$^{-1}$)')
plt.show()
```

## Group cross sections

Microscopic cross sections are continous function of the incoming neutron energy E. However, in practice one often needs to discretize it into one or more groups. The group cross sections are the spectrum weighted average of the cross sections:

$$\sigma_g=\frac{\int_{E_{g}}^{E_{g-1}}\sigma(E)\Phi(E)dE}{\int_{E_{g}}^{E_{g-1}}\Phi(E)dE}$$

Where $g$ stands for the group. In case of a one-group approximation the integral is performed from $0$ to $\infty$.

Note that the energy dependent flux needs to be a priori known for calculating the group cross sections, and in fact usually we want to know the group cross sections to calculate the flux. This is the beauty of reactor physics. Back in the days group constants were often known from experiments. Today a common method is to generate group constants from time consuming Monte Carlo methods, and then use them in fast deterministic calculations.

### One group fission cross section of U-235 and U238

One can access microscopic cross section data from many sources (eg. http://atom.kaeri.re.kr/nuchart/).

Cross sections can be copied from there and put into python arrays or list, or into a file and then the file can be read.

Here one solution is provided which makes use of the urllib library and send a request to the server, which then sends back the HTML content of the webpage (as a browser would communicate to a server). Getting a response might take couple of seconds. Then a function getXS is provided which helps reading cross section data from this HTML content. There are other, probably better solutions to do this.

We will read the fission cross sections from the data library JEFF3.2.

```python
def getXS(content):
    """getting data from the html content. I tried to avoid Beautiful soup.
    the content has a shape as follows like:
    Energy(eV) XS(b)<br>
    1.00000E-05 3.07139<br>"""
    energy=[]
    xs=[]
    flag=False
    i=0
    for line in content:
        x=line.strip().split()
        if  x[0]==b'Energy(eV)':
            flag=True
            continue
        if x[0]==b'</span>':
            flag=False
        if flag:
            energy.append(float(x[0]))
            xs.append(float(x[1][:-4]))
    return np.array(energy), np.array(xs)
```

```python
import urllib.request

website235='http://atom.kaeri.re.kr/nuchart/getData.jsp?target=jeff3.2,92,235,9228,3,18'
website238='http://atom.kaeri.re.kr/nuchart/getData.jsp?target=jeff3.2,92,238,9237,3,18'

with urllib.request.urlopen(website235) as response:
    content235 = response.readlines()
energy235,xs235=getXS(content235)

with urllib.request.urlopen(website238) as response:
    content238 = response.readlines()
energy238,xs238=getXS(content238)

plt.figure()
plt.loglog(energy235,xs235, 'g', label='U235')
plt.loglog(energy238,xs238, 'r', label='U238')
plt.legend()
plt.xlabel('Energy (eV)')
plt.ylabel('XS (barn)')
plt.show()
```

Finally one can weight the cross section with the spectrum and calculate the one group cross section.

```python
spectrum235,spectrumTh235,spectrumEpi235,spectrumFast235=getSpectrum(energy235)
spectrum238,spectrumTh238,spectrumEpi238,spectrumFast238=getSpectrum(energy238)

sigma_fiss_U235=np.trapz(xs235*spectrum235,energy235)/np.trapz(spectrum235,energy235)
sigma_fiss_U238=np.trapz(xs238*spectrum238,energy238)/np.trapz(spectrum238,energy238)

plt.figure()
plt.loglog(energy235,xs235, 'g', label='U235')
plt.axhline(sigma_fiss_U235,c='g',ls='--',label='U235 one group')
plt.loglog(energy238,xs238, 'r', label='U238')
plt.axhline(sigma_fiss_U238,c='r',ls='--',label='U238 one group')
plt.legend()
plt.xlabel('Energy (eV)')
plt.ylabel('XS (barn)')
plt.show()

print(sigma_fiss_U235)
print(sigma_fiss_U238)
```

# Exercise

Calculate the group cross sections in the thermal and fast groups to compare $\sigma_{f}/\sigma_{tot}$ values for Pu-239, Pu-240, Pu-242. Discuss what might be an advantage of fast reactors?

hint:
- you can interpolate the cross sections to evaluate it in other energies.
- 'http://atom.kaeri.re.kr/nuchart/getData.jsp?target=jeff3.2,92,238,9237,3,18' the URL has some structure: 9237: MAT number, 18: MT number. One can write a generic string and put these numbers there for the several cases.
- Avoid repeating the same code snippet as in this notebook when sending the request. Rather place info and results in nested dictionaries (example for possible stucture below, though you probably will not create like this, but on the fly), and iterate through that.

```python
data={'U235fiss': {'Z':92, 'A':235,'MAT':9228,'MT':18,'energy':energy235,'xs':xs235,'sigma':sigma_fiss_U235},
     'U238fiss': {'Z':92, 'A':238,'MAT':9237,'MT':18,'energy':energy238,'xs':xs238,'sigma':sigma_fiss_U238}}
data['U235fiss']['xs']
```

    array([3.14119e+04, 2.80953e+04, 2.56471e+04, ..., 2.21460e+00,
           2.22251e+00, 2.22970e+00])

```python

```


```python
%matplotlib inline
import matplotlib.pyplot as plt
import sympy as sym
from sympy import oo
sym.init_printing()
# X = sym.laplace_transform(x, t, s)

def pole_zero_plot(poles, zeros, ax=None):
    mR = -2 #  valor inicial para minimo real
    MR = 2  #  valor inicial para maximo real
    mI = -2 #  valor inicial para minimo imaginario
    MI = 2  #  valor inicial para maximo imaginario
    for pole in poles:
        plt.plot(complex(pole).real, complex(pole).imag, 'rx', markersize=10)
        if complex(pole).real < mR:
            mR = complex(pole).real
        if complex(pole).real > MR:
            MR = complex(pole).real
        if complex(pole).imag < mI:
            mI = complex(pole).imag
        if complex(pole).imag > MI:
            MI = complex(pole).imag

    for zero in zeros:
        plt.plot(complex(zero).real, complex(zero).imag, 'bo', markersize=10)
        if complex(zero).real < mR:
            mR = complex(zero).real
        if complex(zero).real > MR:
            MR = complex(zero).real
        if complex(zero).imag < mI:
            mI = complex(zero).imag
        if complex(zero).imag > MI:
            MI = complex(zero).imag

    if ax is None:
        ax = plt.gca()
    ax.set_aspect(1)

    plt.axis([mR*1.1, MR*1.1, mI*1.1, MI*1.1])
    plt.xlabel(r'$\Re \{ s \}$')
    plt.ylabel(r'$\Im \{ s \}$')
    plt.grid()

```

# Transformada de Laplace

## Definición

La [transformada de Laplace](https://es.wikipedia.org/wiki/Transformada_de_Laplace) está definida por

\begin{equation}
X(s) = \int_{-\infty}^{\infty} x(t) \, e^{-s t} \; dt
\end{equation}

En la mayoría de aplicaciones en ingeniería, las señales se definene para $t>0$, lo que permite expresar la transformada como:

\begin{equation}
X(s) = \int_{0}^{\infty} x(t) \, e^{-s t} \; dt
\end{equation}

donde $X(s) = \mathcal{L} \{ x(t) \}$ se usa como notación de la Transformada de Laplace de la señal $x(t)$.

La transformada inversa de Laplace es:

\begin{equation}
x(t) = \frac{1}{2 \pi j} \int_{\sigma - j \infty}^{\sigma + j \infty} X(s) \, e^{s t} \; ds
\end{equation}

La variable $s = \sigma + j \omega$ es compleja, de manera que en el kernel de transformación $e^{-s t}$ aparezcan los comportamientos:

- oscilatorios con $e^{-j \omega t}$
- exponenciales crecientes o decreientes con $e^{\sigma t}$
- constantes con $e^{0 t}$
- combinaciones de los anteriores.

Lo anterior permite analizar sistemas LTI con la transformada de Laplace.

El conjunto de valores de $s$ que hacen que la integral de transformación converja se llama Región de convergencia (ROC).

Puede interpretarse como el "parecido" entre la señal $x(t)$ y  $e^{\sigma} \cdot cos(\omega t)$.

### Transformada del delta de Dirac

\begin{equation}
\mathcal{L} \{ \delta(t) \} = \int_{-\infty}^{\infty} \delta(t) \, e^{- s t} dt = e^{- s t} \Big\vert_{t=0}  =  1
\end{equation}
para $s \in \mathbb{C}$. La ROC corresponde a todo el plano $S$.

Como $\mathcal{L} \{ \delta(t) \} = 1$, todas las frecuencias están presentes en la señal con igual peso.

```python
t = sym.symbols('t', real=True)
s = sym.symbols('s', complex=True)

X = sym.integrate(sym.DiracDelta(t)*sym.exp(-s*t), (t, -sym.oo, sym.oo))
X
```

### Transformada de una Exponencial Compleja y causal

La transformada $X(s) = \mathcal{L} \{ x(t) \}$ de una señal exponencial compleja y causal

\begin{equation}
x(t) = \epsilon(t) \cdot e^{-s_0 t}
\end{equation}

con frecuencia compleja $s_0 \in \mathbb{C}$ se obtiene de la expresión integral de la transfomada de Laplace

\begin{equation}
\begin{split}
X(s) &= \int_{0}^{\infty} e^{-s_0 t} \cdot e^{-s t} \; dt \\
&= \int_{0}^{\infty} e^{-(s_0+s) t} \; dt \\
&= \frac{-e^{-(s+s_0)t}}{s+s_0} \, \bigg\vert_{0}^{\infty} \\
&= \frac{-1}{s+s_0} \left[ \lim_{t \to \infty} \left( e^{- (s+s_0) t} \right) -1 \right] \\
&= \frac{1}{s + s_0}
\end{split}
\end{equation}

Observe que el límite anterior converge solo si $e^{- (s+s_0) t}$ converge, es decir que $\Re \{ s + s_0 \} > 0$. Por lo cual la ROC es:

\begin{equation}
\Re \{ s \} > \Re \{ - s_0 \}
\end{equation}

Entonces, la transformada de la señal exponencial compleja y causal es
\begin{equation}
\mathcal{L} \{ \epsilon(t) \cdot e^{-s_0 t} \} = \frac{1}{s + s_0} \qquad \forall s : \Re \{ s \} > \Re \{ - s_0 \}
\end{equation}

```python
t = sym.symbols('t', real=True)
s = sym.symbols('s', complex=True)

s0 = 1j
x = sym.exp(-s0*t)*sym.Heaviside(t)
X, a, cond = sym.laplace_transform(x, t, s)
X, a, cond
```

$\displaystyle \left( \frac{1.0}{s + i}, \  0, \  \text{True}\right)$

```python
help(sym.laplace_transform)
```

    Help on function laplace_transform in module sympy.integrals.transforms:

    laplace_transform(f, t, s, **hints)
        Compute the Laplace Transform `F(s)` of `f(t)`,

        .. math :: F(s) = \int_0^\infty e^{-st} f(t) \mathrm{d}t.

        For all "sensible" functions, this converges absolutely in a
        half plane  `a < \operatorname{Re}(s)`.

        This function returns ``(F, a, cond)``
        where ``F`` is the Laplace transform of ``f``, `\operatorname{Re}(s) > a` is the half-plane
        of convergence, and ``cond`` are auxiliary convergence conditions.

        If the integral cannot be computed in closed form, this function returns
        an unevaluated :class:`LaplaceTransform` object.

        For a description of possible hints, refer to the docstring of
        :func:`sympy.integrals.transforms.IntegralTransform.doit`. If ``noconds=True``,
        only `F` will be returned (i.e. not ``cond``, and also not the plane ``a``).

        >>> from sympy.integrals import laplace_transform
        >>> from sympy.abc import t, s, a
        >>> laplace_transform(t**a, t, s)
        (s**(-a)*gamma(a + 1)/s, 0, re(a) > -1)

        See Also
        ========

        inverse_laplace_transform, mellin_transform, fourier_transform
        hankel_transform, inverse_hankel_transform

```python
X
```

SymPy devuelve tres resutados:

- Primero la transformada de Laplace $X(s)$
- Segundo un valor $a$ que indica qué parte del semiplano $\Re \{ s \} > 0$ tiene convergencia en que $\Re \{ s \} > a$
- Tercero otras condiciones de convergencia.

Inicialmente se analizará solamente la transformada.

Tenga en cuenta que $s$ es una variable compleja, de modo que el "espectro" no puede representarse como una curva.

```python
sigma, omega = sym.symbols('sigma omega')

X1 = X.subs({s: sigma+sym.I*omega})
print(X1)

plt.rcParams['figure.figsize'] = 6,6
sym.plotting.plot3d(abs(X1),
                    (sigma, -20, 20),
                    (omega, -20, 20),
                    xlabel=r'$\Re\{s\}$',
                    ylabel=r'$\Im\{s\}$',
                    title=r'$|X(s)|$');
```

Los diagramas de polos y ceros son versiones simplificadas de la gráfica anterior.

```python
X
```

```python
sym.numer(X) # El numerador de X(s)
```

```python
sym.roots(sym.numer(X), s) # Las raíces del numerador de X(s)
```

```python
sym.roots(sym.denom(X),s) # El numerador de X(s)
```

```python
plt.rcParams['figure.figsize'] = 4,4
pole_zero_plot(sym.roots(sym.denom(X), s), sym.roots(sym.numer(X), s))
```

## Propiedades

Dadas las propiedades presentadas para la transformada de Fourier y la estructura de las transformadas de Laplace y de Fourier, se obtienen las mismas propiedades.

### Invertible
\begin{equation}
x(t) = \mathcal{L}^{-1} \left\{ \mathcal{L} \{ x(t) \} \right\}
\end{equation}

### Linealidad

\begin{equation}
\mathcal{L} \{ A \cdot x_1(t) + B \cdot x_2(t) \} = A \cdot X_1(s) + B \cdot X_2(s)
\end{equation}

**Ejemplo - Transformada de Laplace de una señal senoidal causal**

\begin{equation}
x(t) = sin( \omega _0 t) \cdot \epsilon(t)
\end{equation}

con $\omega _0 \in \mathbb{R}^+$

De acuerdo con las expresiones de Euler:
\begin{align}
\cos(\omega_0 t) &= \frac{1}{2} \left( e^{j \omega_0 t} + e^{-j \omega_0 t} \right) \\
\sin(\omega_0 t) &= \frac{1}{2j} \left( e^{j \omega_0 t} - e^{-j \omega_0 t} \right)
\end{align}

Así, la transformada de $sin( \omega _0 t)$ debe ser igual a la de $\frac{1}{2j} ( e^{j \omega_0 t} - e^{-j \omega_0 t})$.

Dado que la transformada de Laplace de una exponencial compleja decreciente y causal es :

\begin{equation}
\mathcal{L} \{ \epsilon(t) \cdot e^{-s_0 t} \} = \frac{1}{s + s_0}
\end{equation}

Entonces:
\begin{equation}
\mathcal{L} \{ sin( \omega _0 t) \} = \frac{1}{2j} \left( \frac{1}{s - j \omega_0} - \frac{1}{s + j \omega_0} \right) = \frac{\omega_0}{s^2 + \omega_0^2}
\end{equation}

Aplicando el mismo razonamiento puede encontrarse la transformada del coseno.
$$cos( \omega _0 t) = \frac{1}{2} \left( e^{j \omega_0 t} + e^{-j \omega_0 t} \right) $$
\begin{equation}
\mathcal{L} \{ cos( \omega _0 t) \} = \frac{1}{2} \left( \frac{1}{s - j \omega_0} + \frac{1}{s + j \omega_0} \right) = \frac{?}{?}
\end{equation}

```python
t = sym.symbols('t', real=True)
s = sym.symbols('s', complex=True)

w0 = 5
x = sym.sin(w0* t)*sym.Heaviside(t)
x
```

```python
X, a, cond = sym.laplace_transform(x, t, s)
X
```

```python
sigma, omega = sym.symbols('sigma omega')

X1 = X.subs({s: sigma+sym.I*omega})

plt.rcParams['figure.figsize'] = 6,6
sym.plotting.plot3d(abs(X1),
                    (sigma, -10, 10),
                    (omega, -10, 10),
                    xlabel=r'$\Re\{s\}$',
                    ylabel=r'$\Im\{s\}$',
                    title=r'$|X(s)|$');
```

\begin{equation}
s^2+25=0
\end{equation}
\begin{equation}
s^2=-25
\end{equation}

\begin{equation}
s = +- j 5
\end{equation}

```python
sym.denom(X)
```

```python
sym.roots(sym.denom(X), s)
```

```python
plt.rcParams['figure.figsize'] = 4,4

pole_zero_plot(sym.roots(sym.denom(X), s), sym.roots(sym.numer(X), s))
```

**Ejercicio**

Encuentre la transformada de Laplace de la señal $cos(\omega_0 t)  \cdot \epsilon(t)$

Desarrolle su análisis aquí

```python
# Escriba su código

```

### Lista de propiedades

A continuación se listan propiedades y teoremas:

|   | $x(t)$ | $X(s) = \mathcal{L} \{ x(t) \}$ | Región de convergencia ROC |
|:---|:---:|:---:|:---:|
| Linealidad | $A \, x_1(t) + B \, x_2(t)$ | $A \, X_1(s) + B \, X_2(s)$ | $\supseteq ROC\{x_1\} \cap ROC\{x_2\}$ |
| Señales reales | $x(t) = x^*(t)$ | $X(s) = X^*(s^*)$ | |
| Escalamiento | $x(a t)$ | $\frac{1}{\lvert a \rvert} X\left( \frac{s}{a} \right)$ | $s: \frac{s}{a} \in \text{ROC}\{x(t)\}$ |
| Convolución | $x(t) * h(t)$ | $X(s) \cdot H(s)$ | $\supseteq \text{ROC}\{x(t)\} \cap \text{ROC}\{h(t)\}$ |
| Desplazamiento | $x(t - \tau)$ | $e^{-s \tau} \cdot X(s)$ | $\text{ROC}\{x(t)\}$ |
| Modulación | $e^{s_0 t}\cdot x(t)$ | $X(s - s_0)$ | $s: s - \Re \{s_0\} \in \text{ROC}\{x(t)\}$ |
| Diferenciación | $\frac{d}{dt} x(t)$ | $s \cdot X(s) - x(0+)$ | $\supseteq \text{ROC}\{x(t)\}$ |
| Integración | $\int_{-\infty}^{t} x(t) \; dt$ | $\frac{1}{s} \cdot X(s)$ | $\supseteq \text{ROC}\{x(t)\} \cap \{s: \Re \{s\} > 0 \}$ |

donde $A, B, s_0 \in \mathbb{C}$, $a \in \mathbb{R} \setminus \{0\}$ y $\tau \in \mathbb{R}$.

## Algunas transformadas de Laplace

A continuación se listan las transformadas de Laplace de funciones usadas a menudo:

| $x(t)$                             | $X(s) = \mathcal{L} \{ x(t) \}$ | Región de Convergencia ROC |
|:----------------------------------:|:-------------------------------:|:--------------------------:|
| $\delta(t)$                        | $1$                             | $\mathbb{C}$               |
| $\epsilon(t)$                      | $\frac{1}{s}$                   | $\Re \{s\} > 0$            |
| $t \epsilon(t)$                    | $\frac{1}{s^2}$                 | $\Re \{s\} > 0$            |
| $e^{- s_0 t} \epsilon(t)$          | $\frac{1}{s + s_0}$             | $\Re \{s\} > \text{Re}\{-s_0\}$ |
| $\sin(\omega_0 t) \epsilon(t)$     | $\frac{\omega_0}{s^2 + \omega_0^2}$ | $\Re \{s\} > 0$        |
| $\cos(\omega_0 t) \epsilon(t)$     | $\frac{s}{s^2 + \omega_0^2}$    | $\Re \{s\} > 0$            |
| $t^n e^{-s_0 t} \epsilon(t)$       | $\frac{n!}{(s+s_0)^{n+1}}$      | $\Re \{s\} > \text{Re}\{-s_0\}$ |
| $e^{-s_0 t} \cos(\omega_0 t) \epsilon(t)$ | $\frac{s + s_0}{(s+s_0)^2 + \omega_0^2}$ | $\Re \{s\} > \Re \{-s_0\}$ |
| $e^{-s_0 t} \sin(\omega_0 t) \epsilon(t)$ | $\frac{\omega_0}{(s+s_0)^2 + \omega_0^2}$ | $\Re \{s\} > \Re \{-s_0\}$ |
| $t \cos(\omega_0 t) \epsilon(t)$ | $\frac{s^2 - \omega_0^2}{(s^2 + \omega_0^2)^2}$ | $\Re \{s\} > 0$ |
| $t \sin(\omega_0 t) \epsilon(t)$ | $\frac{2 \omega_0 s}{(s^2 + \omega_0^2)^2}$ | $\Re \{s\} > 0$ |

Donde $s_0 \in \mathbb{C}$, $\omega_0 \in \mathbb{R}$ and $n \in \mathbb{N}$. Otras transformadas se pueden encontrar [en línea](https://en.wikipedia.org/wiki/List_of_Laplace_transforms).

##  Circuitos
### Impedancias complejas y equivalencias

Los circuitos eléctricos pueden analizarse en el dominio transformado de Laplace enfocándose primero en el comportamiento de cada elemento. La siguiente tabla resume el comportamiento de los elementos pasivos de los circuitos eléctricos el el dominio del tiempo (Ecuación diferencial) y de la frecuencia (Laplace). La impedancia compleja se muestra en la última columna.

| Elemento | Dominio del tiempo  (EDO)| Dominio de la frecuencias con Laplace | $Z(s) = \frac{U(s)}{I(s)}$ |
|:---:|:---:|:---:|:---:|
|  | $u(t) = R \cdot i(t)$ | $U(s) = R \cdot I(s)$ | $R$ |
|  | $\begin{matrix} u(t) = L \frac{d}{dt} i(t) \\ i(t) = \frac{1}{L} \int_{0}^{t} u(\tau) d\tau + i(0+) \epsilon(t) \end{matrix}$ | $\begin{matrix} U(s) = s L I(s) - L i(0+) \\ I(s) = \frac{1}{s L} U(s) + \frac{1}{s} i(0+) \end{matrix}$ | $s L$ |
|  | $\begin{matrix} u(t) = \frac{1}{C} \int_{0}^{t} i(\tau) d\tau + u(0+) \epsilon(t) \\ i(t) = C \frac{d}{dt} u(t) \end{matrix}$ | $\begin{matrix} U(s) = \frac{1}{s C} I(s) + \frac{1}{s} u(0+) \\ I(s) = s C U(s) - C u(0+) \end{matrix}$ | $\frac{1}{s C}$

Considere el siguiente circuito suponiendo valores iniciales iguales a cero.

Para ilustrar, considere que $L = 0.5$, $R = 1$, $C = 0.4$ .

\begin{equation}
v_L(t) + v_R(t) + v_C(t) = x(t)
\end{equation}

\begin{equation}
L \frac{di}{dt} + R i(t) + \frac{1}{C} \int_{0}^{t} i(\tau) d\tau = x(t)
\end{equation}

La salida es el voltaje del capacitor

\begin{equation}
y(t) = \frac{1}{C} \int_{0}^{t} i(\tau) d\tau
\end{equation}

De donde se obtiene la corriente

\begin{equation}
C \frac{dy}{dt} = i(t)
\end{equation}

Incorporando la salida en la ecuación de la malla se obtiene

\begin{equation}
C L \frac{d^2 y(t)}{dt^2} + C R \frac{d y(t)}{dt} + y(t) = x(t)
\end{equation}

La ecuación anterior puede transformarse al dominio de Laplace.
\begin{equation}
C L s^2Y(s)  + C R sY(s) + Y(s) = X(s)
\end{equation}

Puede encontrarse una expresión para la salida en el dominio de Laplace como

\begin{equation}
Y(s) = \frac{1}{LC s^2 + RC s + 1} \cdot X(s)
\end{equation}

Observe que el circuito puede transformarse al dominio de Laplace directamente.

**Nota** Las mismas aproximaciones pueden realizarse en otros campos de conocimiento en donde se modelen fenómenos con Ecuaciones Diferenciales.

\begin{equation}
Y(s) = X(s) \frac{\frac{1}{sC}}{R + sL + \frac{1}{sC}}
\end{equation}

\begin{equation}
Y(s) = X(s) \frac{\frac{sC}{sC}}{RCs + sLCs + \frac{sC}{sC}}
\end{equation}

\begin{equation}
Y(s) = X(s) \frac{1}{RCs + LCs^2 + 1}
\end{equation}

```python
%matplotlib inline
import sympy as sym
sym.init_printing()

s = sym.symbols('s', complex=True)
t, R, L, C = sym.symbols('t R L C', real=True)
X = sym.Function('X')(s)

F = 1/(L*C*s**2 + R*C*s + 1)
Y = F * X
Y
```

Introduzca los valores de los elementos del circuito en un diccionario para usarlo después.

```python
RLC = {R: 1,
       L: 0.5,
       C: .4}
RLC
```

### Respuesta al escalón

La respuesta al escalón en el dominio de Laplace se obtiene al considerar que $X(s) = \frac{1}{s}$

```python
F = F.subs(RLC)
F
```

```python
Y
```

```python

Y_escalon = Y.subs(RLC).subs(X, 1/s)
Y_escalon
```

La respuesta al escalón en el dominio del tiempo se obtiene al considerar operar la respuesta en el doninio de la frecuencia con la Transformada inversa de Laplace.

```python
y_escalon = sym.inverse_laplace_transform(Y_escalon,s,t)
y_escalon
```

Ahora puede graficarse la señal de salida en el dominio del tiempo y el mapa de polos y ceros asociado a dicha señal.

```python
plt.rcParams['figure.figsize'] = 7, 2
sym.plot(y_escalon, (t, -1, 10), xlabel='$t$', ylabel='$y(t)$');
plt.rcParams['figure.figsize'] = 3, 3
pole_zero_plot(sym.roots(sym.denom(Y_escalon), s), sym.roots(sym.numer(Y_escalon), s))
```

La ubicación de las raíces en el plano $s$ determinan la forma de las señales y pueden asociarse con propiedades de las señales o sistemas que modelan.

```python

```


# Example of PCA scree plot and how to determine the number of relevant principal components to keep.

Principal Component Analysis is a very popular method to reduce the dimensionality of data. This is particularly useful
when we want to classify data in large dimensions. It can also be used as a visualisation tool to project data in
a lower dimension space (data embedding).

## The PCA transform
This linear transform allows us to move from the natural/original space $\cal{X}$ into the component space $\cal{Z}$ and is defined as

  $\bf{z} = \bf{W}^{T}(\bf{x}-\bf{\mu})$

with

$\begin{align}
\bf{x} &= [x_{1} x_{2} \cdots  x_{N}]^\top \\
\bf{\mu} &= [\mu_{1} \mu_{2} \cdots \mu_{N}]^\top  \\
\bf{z} &= [z_{1} z_{2} \cdots z_{N}]^\top  \\
\end{align}$

where N is the dimension of space and $\bf{\mu}$ is the mean of the N-dimensional data X.

The $W$ matrix if made of the N eigenvectors $\bf{w}_{i}$ of the covariance matrix $\Sigma$. They are stacked together along columns

$\begin{align}
\bf{W} &= \begin{pmatrix} \bf{w}_{1} & \bf{w}_{2} & \dotsb & \bf{w}_{N}  \end{pmatrix} \\
&= \begin{pmatrix} w_{1,1} & w_{2,1} & \dotsb & w_{N,1} \\
w_{1,2} & w_{2,2} & \dotsb & w_{N,2} \\
\vdots &  \vdots & \dotsb & \vdots \\
w_{1,N} & w_{2,N} & \dotsb & w_{N,N} \end{pmatrix}
\end{align}$

As for the covariance matrix $\Sigma$, it is computed from the N-dimensional data distribution X

$
\begin{align}
\bf{\Sigma} &= E\{(\bf{x}-\bf{\mu})(\bf{x}-\bf{\mu})^{T} \} \\
&= \begin{pmatrix} \sigma_{1}^2 & \sigma_{1,2} & \cdots & \sigma_{1,N} \\
\sigma_{1,2} & \sigma_{2}^2 & \cdots & \sigma_{2,N} \\
\vdots & \vdots & \ddots & \vdots \\
\sigma_{1,N} & \sigma_{2,N} & \cdots & \sigma_{N}^2 \end{pmatrix}
\end{align}
$

The $\it{inverse}$ PCA transform allows us to move from the component space $\cal{Z}$ back to the natural/original space $\cal{X}$
and is defined as

  $\bf{x} = \bf{W}\bf{z} + \bf{\mu}$

The principal components $\bf{z}$ are sorted in decreasing order of variance $var(z_{i})=\lambda_{i}$ where the $\lambda_{i}$ are
the eigenvalues of the covariance matrix $\bf{\Sigma}$. The scree plot associated to PCA displays $\lambda$ $\it{versus}$
the component number as we will see below.

N.B. The principal components usually come in one of the two popular notations: $z_{i}$ or $PCA_{i}$.

## PCA as a tool for dimensionality reduction

The last equation shows that we can reconstruct exactly the vector $\bf{x}$. We usually drop the
least significant principal components (the last components in z and the last columns of $\bf{W}$)
as they contain the least information and are often corrupted with noise. This produces
an approximate reconstruction of $\bf{x}$

  $\bf{x} \approx \bf{\tilde{W}}\bf{\tilde{z}} + \bf{\mu}$

with the modified arrays

  $\bf{\tilde{z}} = \begin{pmatrix} z_{1} \ z_{2} \ \dotsb \ z_{M} \end{pmatrix}^{T} $

and

$
\begin{align}
\bf{\tilde{W}} &= \begin{pmatrix} \bf{w}_{1} & \bf{w}_{2} & \dotsb & \bf{w}_{M}  \end{pmatrix} \\
&= \begin{pmatrix} w_{1,1} & w_{2,1} & \dotsb & w_{M,1} \\
w_{1,2} & w_{2,2} & \dotsb & w_{M,2} \\
\vdots &  \vdots & \ddots & \vdots \\
w_{1,N} & w_{2,N} & \dotsb & w_{M,N} \end{pmatrix}
\end{align}
$

We will show in this script how the scree plot can be used as a tool for finding out the optimal number of
components (M<N) to keep. Our approach is based on the reconstruction errors introduced
when dimensionality reduction is applied to the original data X.

```python
print(__doc__)

# Author: Pierre Gravel <pierre.gravel@iid.ulaval.ca>
# License: BSD

import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA
from sklearn.datasets import fetch_olivetti_faces

import random
from numpy.random import RandomState
import seaborn as sns
sns.set(color_codes=True)

# Used for reproductibility of the results
np.random.seed(43)
```

    Automatically created module for IPython interactive environment

# Data preprocessing

### Load the Olivetti faces dataset.

The dataset contains 400 images of size 64 x 64 that are each stored into a column of 4096 elements. The original 400 x 4096 data
must be reshaped into 4096 x 400 data. Hence, each line will correspond to an image (an observation) and each
column will correspond to a feature.

```python
X, _ = fetch_olivetti_faces(return_X_y=True, shuffle=True, random_state=RandomState(0))

# Reshape the array
X = X.T

```

Let us show a few (unsmoothed) images in the dataset. They contain visible quantisation noise and
probably also gaussian noise from electronics and Poisson noise from photon statistics. The PCA analysis
will reveal the impact of noise in the images.

```python
fig, ax = plt.subplots(2,2, figsize=(10,10))
for i in range(4):
    plt.subplot(2,2, i + 1)
    plt.imshow(X[:,i].reshape(64, 64), cmap='gray', interpolation='nearest')
    plt.xticks(())
    plt.yticks(())
plt.subplots_adjust(0.01, 0.05, 0.99, 0.93, 0.04, 0.)

plt.savefig('13.2.1_Examples_from_Olivetti_face_dataset.png')
plt.savefig('13.2.1_Examples_from_Olivetti_face_dataset.pdf')
```

### Split the dataset into a training and a test datasets

```python
X_train, X_test = train_test_split(X, random_state=0,train_size=0.8)

```

### Normalise the data
The normalisation transform is defined with the training dataset. Both training and test datasets are then normalised with it.

```python
sc = StandardScaler().fit(X_train)

X_train_s = sc.transform(X_train)
X_test_s = sc.transform(X_test)
```

## PCA approximation and data reconstruction

In this section, we apply the PCA transform to the training dataset and then reconstruct it using only a
fraction of the principal components. The result is an approximation of the training dataset.

Next, we apply the same PCA transform, i.e. defined for the training dataset, to the test dataset and reconstruct it using
the same procedure as before. The result is also an approximation of the test dataset, but it is based on the
training principal components, not on the test principal components (which are not computed here).

Warning: the next cell takes a few seconds to a minute to compute. In a few years from now, this warning will be pointless
given the increasing speeds of hardware and software!

```python
(n_images,n_components) = X_train.shape

error_train = np.zeros((n_components,1))
error_test = np.zeros((n_components,1))

for i in range(1,n_components+1):
    # Compute the PCA transform using only the first i principal components of the training dataset
    pca = PCA(n_components=i)
    pca.fit(X_train_s)

    # Apply the PCA transform to the training dataset and extract the first i principal components
    Z = pca.transform(X_train_s)

    # Reconstruct the training dataset using this reduced set of principal components
    x_train = pca.inverse_transform(Z)

    # Remove the normalisation transform
    x_train = sc.inverse_transform(x_train)

    # Apply the same PCA transform to the test dataset and extract the first i principal components
    Z = pca.transform(X_test_s)

    # Reconstruct the test dataset using the reduced set of principal components
    x_test = pca.inverse_transform(Z)

    # Remove the normalisation transform
    x_test = sc.inverse_transform(x_test)

    # Compute the reconstruction error for each dataset
    error_train[i-1] = np.square(np.subtract(x_train, X_train)).mean()
    error_test[i-1] = np.square(np.subtract(x_test, X_test)).mean()

```

## Display the PCA scree plot and the reconstruction errors

The scree plot shows the eigenvalues in a decreasing order. The most relevant eigenvectors are on the left and
the least relevant ones on the right. The sharp bend is often called the 'knee' of the curve.

The second panel shows the reconstruction errors for both training and test datasets. There are slight differences
between the two overlapping curves.

```python
n_comp = np.arange(1,n_components+1)

fig, ax = plt.subplots(2,1, figsize=(10,10))
ax[0].plot(n_comp, pca.singular_values_)
ax[0].set_xlabel('Eigenvectors', fontsize=16)
ax[0].set_ylabel('Eigenvalues', fontsize=16)
ax[0].set_title('Scree plot', fontsize=16)

ax[1].plot(n_comp, error_train, label='Training')
ax[1].plot(n_comp, error_test, label='Test')
ax[1].set_xlabel('Eigenvectors', fontsize=16)
ax[1].set_ylabel('Error', fontsize=16)
ax[1].set_title('Reconstruction error', fontsize=16)
ax[1].legend()

fig.tight_layout()

plt.savefig('13.2.2_Scree_plot_and_reconstruction_errors.png')
plt.savefig('13.2.2_Scree_plot_and_reconstruction_errors.pdf')
```

Let us zoom on the first 100 components in the second panel and reframe it to make differences in reconstruction errors
more visible. After roughly 25 eigenvectors, the reconstruction errors for the test dataset become larger than those
for the training one; both error curves start diverging.

Hence, the next eigenvectors describe features in the training data that become less relevant to
the test data. Because the test errors continue to decrease nevertheless, those features do not contain noise only, otherwise
the reconstruction error would flatten out.

The next components become more affected by noise. If there were no noise, both reconstruction errors
would be similar and continue to decrease at the same rate because statistically speaking, the two datasets are
well balanced and should contain similar principal components. Up to a certain point of course; they are
similar but not identical.

The difference in reconstruction errors indicates that the noise becomes more important in the next eigenvectors.
If a choice had to be made between the eigenvectors to keep and those to drop, it would be there. The signal-to-noise
ratio for the principal components starts to decrease past that point.

```python
fig, ax = plt.subplots(figsize=(10,5))
ax.plot(n_comp, error_train, label='Training')
ax.plot(n_comp, error_test, label='Test')
ax.set_xlabel('Eigenvectors', fontsize=16)
ax.set_ylabel('Error', fontsize=16)
ax.set_title('Reconstruction error', fontsize=16)
ax.legend()
ax.set_ylim([0.001, 0.006])
ax.set_xlim([0, 100])

fig.tight_layout()

plt.savefig('13.2.3_Zoom_on_reconstruction_errors.png')
plt.savefig('13.2.3_Zoom_on_reconstruction_errors.pdf')
```

In the next figure, the star indicates where the noise starts to seriously affect the true information, i.e. at eigenvector #25.
The star coincides with the 'knee' of the scree plot!

This approach, based on reconstruction errors, works best when there is a 'knee' in the scree plot. With some datasets,
the scree plot shows a monotonously decreasing curve. The reconstruction error explanation is less convincing in these cases.

```python
fig, ax = plt.subplots(figsize=(10,5))

ax.plot(n_comp, pca.singular_values_)
ax.set_xlabel('Eigenvectors', fontsize=16)
ax.set_ylabel('Eigenvalues', fontsize=16)
ax.set_title('Scree plot', fontsize=16)
ax.plot(25., 75, '*', color='r', markersize=20)

fig.tight_layout()

plt.savefig('13.2.4_Optimal_number_of_components_to_keep.png')
plt.savefig('13.2.4_Optimal_number_of_components_to_keep.pdf')
```


# Cadenas de Markov

## Transiciones de Estado

La secuencia de variables aleatorias $x_0, x_1, x_2, \dots, x_t , \dots$ representa un **proceso estocástico**.

Cuando se indexan solamente los puntos en el tiempo en el que ocurren *cambios* significativos, se habla de **procesos estocásticos de tiempo discreto**.

Hablamos de **estado** como la condición o característica de un sistema en momento dado. Se asume que hay un número finito de estados numerados $1, \dots, N$ que pueden describir al sistema en un cualquier momento. Cuando el sistema cambia de un estado a otro se dice que ocurre una **transición**.

\begin{equation}
P (x_{t+1} = s_{t+1} | x_t = s_t , x_{t−1} = s_{t−1}, \dots , x_1 = s_1, x_0 = s_0)\\
= P (x_{t+1} = s_{t+1} | x_t = s_t )
\end{equation}

Se conoce como **propiedad de Markov** el hecho que la probabilidad del estado en $t+1$ dependa solamente del estado en $t$.

Y como cada $x_t$ depende tan solo de $x_{t-1}$ e influye solamente en $x_{t+1}$ se dice que el proceso es una **cadena de Markov**.

Si, además el número de estados es numerable y finito, se habla de un **cadena de Markov de estado finito**.

Una suposición adicional es que la probabilidad de transición de un estado $i$ a un estado $j$ es constante en el tiempo.

\begin{equation}
P (x_{t+1} = j | x_t = i) = p_{ij}
\end{equation}

es independiente del índice del tiempo $t$, esto se conoce como la **propiedad estacionaria**.

La probabilidad $p_{ij}$ se denomina **probabilidad de transición**, al definirse para todos los estados del sistema $i,j = 1, 2, \dots, N$, y se suelen escribir en una **matriz de probabilidades de transición**.

\begin{equation}
\mathbf{P} = \begin{bmatrix}
       p_{11} & p_{12} & \dots & p_{1N} \\
       p_{21} & p_{22} & \dots & p_{2N} \\
       \vdots & \vdots & \ddots & \vdots \\
       p_{N1} & p_{N2} & \dots & p_{NN} \\
     \end{bmatrix}
\end{equation}

Además, los elementos de cada fila forma una distribución de probabilidades discreta, por lo tanto:

\begin{equation}
    \displaystyle\sum_{j=1}^{N}p_{ij}=1
\end{equation}

Un proceso de Markov comienza en algún momento inicial $t = 0$. Si el estado $x_0$ no se conoce con certeza, entonces debemos especificar las probabilidades con las que el sistema se encuentra inicialmente en cada uno de los $N$ estados. Denotamos esto como $P(x_0 = i) = p_i(0)$ para cada estado $i$, y usamos el vector

\begin{equation}
    \mathbf{p}(0) = [p_1(0), p_2(0), \dots, p_N(0)]
\end{equation}

para describir la **distribución de probabilidad inicial** del sistema.

## Ejemplo

En un lugar se pueden describir los días como soleados, nublados, y lluviosos. Después de observar los patrones históricos, la condición del clima del siguiente día puede ser descrita de acuerdo a las siguientes probabilidades de transición:

\begin{equation}
\mathbf{P} = \begin{bmatrix}
       0.7 & 0.1 & 0.2 \\
       0.2 & 0.7 & 0.1 \\
       0.5 & 0.2 & 0.3 \\
     \end{bmatrix}
\end{equation}

Las probabilidades de un paso pueden ilustrarse mediante un **diagrama de transición de estados**, en el que los estados se representan por *nodos* y las posibles transiciones como *vértices*.

```python
from graphviz import Digraph
# Create Digraph object
g = Digraph()
g.attr(rankdir='RL', label='\nDiagrama de transición', fontsize='16', fontname='Lato')
g.attr('node', fontsize='10', fontname='Lato')
g.attr('edge', fontsize='10', fontname='Lato')

# Add nodes
g.node('1', 'Soleado')
g.node('2', 'Nublado')
g.node('3', 'Lluvioso')

# Add edges
g.edge('1', '1', '0.7')
g.edge('1', '2', '0.1')
g.edge('1', '3', '0.2')
g.edge('2', '1', '0.2')
g.edge('2', '2', '0.7')
g.edge('2', '3', '0.1')
g.edge('3', '1', '0.5')
g.edge('3', '2', '0.2')
g.edge('3', '3', '0.3')

# Visualize the graph
g
```

Si se desea conocer si una determinada condición del clima prevalece durante dos días, se puede utilizar un **árbol de transición** de dos pasos.

```python
g = Digraph()
g.attr(rankdir='LR', label='\nÁrbol de transición', fontsize='16', fontname='Lato')
g.attr('node', fontsize='10', fontname='Lato')
g.attr('edge', fontsize='10', fontname='Lato')

g.node('1a', 'Soleado')

g.node('1b', 'Soleado')
g.node('2b', 'Nublado')
g.node('3b', 'Lluvioso')

g.node('1c', 'Soleado')
g.node('2c', 'Nublado')
g.node('3c', 'Lluvioso')

g.node('1d', 'Soleado')
g.node('2d', 'Nublado')
g.node('3d', 'Lluvioso')

g.node('1e', 'Soleado')
g.node('2e', 'Nublado')
g.node('3e', 'Lluvioso')

g.edge('1a', '1b', '0.7')
g.edge('1a', '2b', '0.1')
g.edge('1a', '3b', '0.2')
g.edge('1b', '1c', '0.7')
g.edge('1b', '2c', '0.1')
g.edge('1b', '3c', '0.2')
g.edge('2b', '1d', '0.2')
g.edge('2b', '2d', '0.7')
g.edge('2b', '3d', '0.1')
g.edge('3b', '1e', '0.5')
g.edge('3b', '2e', '0.2')
g.edge('3b', '3e', '0.3')

g
```

## Probabilidades de Estado

Si se denota la probabilidad $p_{ij}^{(n)}$ como la probabilidad de transición del estado $i$ al estado $j$ en $n$ pasos.

El cálculo de $p_{11}^{(2)}$ sería

\begin{equation}
(0.7)(0.7)+(0.1)(0.2)+(0.2)(0.5)\\
=(p_{11})(p_{11})+(p_{12})(p_{21})+(p_{13})(p_{31})\\
= 0.61
\end{equation}

Es posible generalizar que para cualquier $i$ y $j$, $p_{ij}^{(2)}$ es el producto interno de la $i$-ésima fila de $\mathbf{P}$ con la $j$-ésima columna de $\mathbf{P}$.

\begin{equation}
    p_{ij}^{(2)} = \sum_{k=1}^{N}p_{ik}p_{kj}
\end{equation}

```python
import numpy as np
p = np.array([[0.7, 0.1, 0.2], [0.2, 0.7, 0.1], [0.5, 0.2, 0.3]])
p
```

    array([[0.7, 0.1, 0.2],
           [0.2, 0.7, 0.1],
           [0.5, 0.2, 0.3]])

```python
p2 = p @ p
p2
```

    array([[0.61, 0.18, 0.21],
           [0.33, 0.53, 0.14],
           [0.54, 0.25, 0.21]])

```python
from IPython.display import display, Markdown
display(Markdown(rf'$p^2_{{11}} =$ {p2[0, 0]:.2f}'))
```

$p^2_{11} =$ 0.61

Lo que implica que la probabilidad que sea soleado dentro de dos días dado que hoy es soleado es igual a $0.61$.

----

Siendo `@` en numpy el operador para la multiplicación matricial o la multiplicación punto, siendo `p @ p` equivalente a `p.dot(p)`.

Se puede generalizar aún más esta expresión, llegando a lo que se conoce como la **Ecuación de Chapman-Kolgomorov**:

\begin{equation}
    p_{ij}^{(n)} = p_{ij}^{n} = \sum_{k=1}^{N}p_{ik}^{n-m}p_{kj}^m
\end{equation}

## Clasificación de los estados en un proceso de Markov

Un estado $j$ es **alcanzable** desde el estado $i$ si hay una secuencia de transiciones que comienza en estado $i$ y termina en estado $j$. Esto es, $p^{(n)}_{ij} \gt 0$ para algún $n$.

Una cadena Markov **irreducible** es aquella en la que se puede llegar a todos los estados desde todos los demás estados. Es decir, en una cadena irreducible, no es posible que el proceso quede *atrapado* y de ahí en adelante solo pueda hacer transiciones dentro de algún subconjunto de los estados.

Se dice que un conjunto de estados está **cerrado** si no se puede llegar a ningún estado fuera del conjunto desde ningún estado dentro del conjunto. Esto significa que una vez que el sistema entra en cualquier estado en el conjunto, nunca saldrá de este conjunto. En una cadena irreducible, sus estados constituyen un conjunto cerrado y ningún subconjunto de estos estados está cerrado.

Un caso particularmente interesante se presenta si un conjunto cerrado contiene sólo un estado. Este estado $i$ se llama **estado absorbente**, y $p_{ii} = 1$. El sistema nunca deja un estado de absorción.

Un estado $i$ es un **estado transitorio** si hay una transición del estado $i$ a otro estado $j$ desde la cual no se puede volver a alcanzar el estado $i$. Por lo tanto, siempre que se sale de un estado transitorio, hay una probabilidad positiva de que no se vuelva a ocupar nunca más. Y por lo tanto, la probabilidad a largo plazo de que un sistema se encuentre en un estado transitorio es esencialmente nula porque, con el tiempo, se saldrá del estado y no volverá a entrar.

Un **estado recurrente** es cualquier estado que no sea transitorio. En una cadena de Markov irreducible de estados finitos, todos los estados son recurrentes. Un caso especial de un estado recurrente es un estado absorbente, desde el cual no se puede alcanzar ningún otro estado.

Se dice que un estado es **periódico** si está ocupado sólo en momentos que difieren entre sí por múltiplos de alguna constante mayor que $1$.