```python
"""
IPython Notebook v4.0 para python 2.7
Librerías adicionales: Ninguna.
Contenido bajo licencia CC-BY 4.0. Código bajo licencia MIT. (c) Sebastian Flores.
"""

# Configuracion para recargar módulos y librerías
%reload_ext autoreload
%autoreload 2

from IPython.core.display import HTML

HTML(open("style/iwi131.css", "r").read())
```

# IWI131
## Programación de Computadores

### Sebastián Flores

http://progra.usm.cl/

https://www.github.com/usantamaria/iwi131

## ¿Qué contenido aprenderemos?

* Listas
* Tuplas

## ¿Porqué aprenderemos ese contenido?

* Listas
* Tuplas

Porque utilizar colecciones de elementos resulta natural y permite simplificar operaciones.

## Motivación

Calcule el promedio y la desviación estándar de $N$ datos, $x_1$, ..., $x_n$, ingresados por el usuario:

$$ \begin{align}
mean &= \frac{1}{n} \sum_{i=1} x_i \\
(std)^2 &= \frac{1}{n} \sum_{i=1} (x_i- mean)^2
\end{align}
$$

```python
N = int(raw_input("Ingrese numero de datos:"))
# Calculo del promedio
print "Ingrese numeros (para calcular promedio)"
j = 1
mean = 0.0
while j<=N:
    x_i = float(raw_input("Ingrese numero: "))
    mean += x_i
    j += 1
mean = mean / N
# Calculo de desviacion estandar
print "Ingrese los MISMOS numeros (para calcular desviacion estandar)"
j = 1
var = 0.0
while j<=N:
    x_i = float(raw_input("Ingrese numero: "))
    var += (x_i-mean)**2
    j += 1
std = (var/N)**0.5
# Imprimir resultados
print "promedio", mean
print "desviacion estandar", std
```

## Análisis del problema anterior
El problema anterior resultaba engorroso para el usuario porque necesitabamos realizar 2 ciclos while en los datos:

* La primera vez para calcular el promedio.
* La segunda vez para calcular la desviación estándar.

Proceso sería más fácil si primero almacenáramos los datos, y luego realizáramos los cálculos necesarios sin molestar al usuario.

## Tuplas
 * Colecciones ***inmutables*** de datos.
 * Tienen un número reducido de operaciones.
 * Se utilizan cuando el número de elementos y su contenido no varía: por ejemplo, coordenadas de un punto
 ***(x,y,z)*** o número complejo ***(Real,Imaginario)***.

## Listas
 * Colecciones ***mutables*** de datos.
 * Tienen un número amplio de operaciones.
 * Se utilizan cuando el número de elementos es variable.

## Tuplas
Colecciones heterogéneas e ***inmutables*** de datos.

Se definen con paréntesis redondos y como colección pueden tener datos de distintos tipos:

```python
posicion_alfil = (7, 6)

alumno = ('Fulano', 'De Tal', '201199001-1')

carta = (5, 'corazones')

fecha = (2011, 4, 12)

triangulo = ((5, 1), (2, 4), (-2, 0))

personaje = ('Arturo Prat', (1848, 4.0, 3), (1879, 5, 21))
```

```python
print personaje[0][0:5]
```

#### Tuplas
## Desempaquetado de Tuplas

Si una tupla tiene $n$ datos, es posible "desempaquetar" en variables de manera directa:

```python
punto = (6.6, -2.4, 3.7)
x, y, z = punto
print x
```

```python
personaje = ("Bernardo O'Higgins", (1778, 8, 20), (1842, 10, 24))
nombre, nacimiento, _ = personaje
an, mn, dn = nacimiento
ad, md, dd = defuncion
print ad - an
print personaje
```

#### Tuplas
## ¿Cómo reconocer una tupla?

Utilice **type**

```python
a = (1,2)
b = (-1,-2)
c1 = a + b
print c1
c2 = (a[0] + b[0], a[1] + b[1])
print c2
print type(a[1])
```

#### Tuplas
## ¿Donde hemos utilizado ya tuplas?

En las funciones de múltiples retornos:

```python
def rectangulo(a,b):
    peri = 2*(a+b)
    area = a*b
    return peri, area

s = rectangulo(1,2)
print type(s)
print s

p, a = rectangulo(2,3)  # Esto de clase anterior era una desempaquetado!
print type(p)
print p
print type(s)
print s
```

#### Tuplas
## ¿Qué métodos puedo aplicar a tuplas?
* **len**: cuantos elementos hay en la tupla
* **sum**: sacar elementos de la tupla

```python
s_mixto = (1, 1.0, True, "1")
print len(s_mixto)
#print sum(s_mixto)
print sum(s_mixto[:3])

s = (1, 1.0, 2., 2, 3.)
print len(s)
print s[1:-1]
print sum(s)
print sum(s[1:-1])
```

#### Listas
## Otros métodos

Testear pertenencia en tupla

    dato in tupla

regresará siempre un booleano: True si dato está en la tupla, False si no está en la tupla.

```python
mi_tupla = ("sebastian", "carlos", "jose")

print "sebastian" in mi_tupla

print "seba" in mi_tupla

print "carlos" in mi_tupla

print "pepe" in mi_tupla
```

#### Tuplas
## Acceder valores vs Cambiar valores
* Los valores de una tupla pueden accederse mediante índices: **[i], [i:], [:j], [i:j]**.

        mi_tupla[mi_indice]

* Los valores de una tupla no pueden cambiarse. La tupla es completamente inmutable.

```python
# Posicion original
punto = (0, 0, 0)
print punto

# Acceso a valores
v = (1,2,3)
dt = 2.0
print punto[0] + v[0]*dt
print punto[1] + v[1]*dt
print punto[2] + v[2]*dt

# Actualización de valores
#punto[0] = 1

s = (1, 1.0, 2., 2, 3.)
print s[1:-1]
print len(s)
print sum(s)
print sum(s[1:-1])
```

#### Tuplas
## Ejemplo: Fibonacci con tuplas
Imprima los primeros $n$ numeros de fibonacci

```python
n = int(raw_input("Ingrese n: "))
j = 1
a, b = 0, 1
while j<=n:
    print b,
    a, b = b, a+b # Mira Ma, sin variables auxiliares
    j+= 1
```

## Listas
Colecciones heterogéneas y ***mutables*** de datos.

Se definen con paréntesis cuadrados: [ ]

```python
mi_lista_1 = [1, 1., "1", True]
mi_lista_2 = [1, 2, 4, 10, 55]

print mi_lista_1
print mi_lista_2

# Convertir de tupla a lista
mi_tupla = tuple(mi_lista_1)
print mi_tupla, type(mi_tupla)
mi_nueva_lista = list(mi_tupla)
print mi_nueva_lista, type(mi_nueva_lista)

```

#### Listas
## Creando listas
Las listas se dicen mutables porque se pueden crear dinámicamente utilizando el método ***append***.

El método append agrega un dato **al final** de la lista.

```python
valores = []
print valores

valores.append(5)
print valores

valores.append(1)
print valores

valores.append(6)
print valores

valores.append(-4)
print valores
```

#### Listas
## Creando listas con datos del usuario

```python
# Terminamos de pedir datos cuando se ingresa "fin".
mi_lista = [] # Lista vacia
while True:
    mi_dato = raw_input("Ingrese dato: ")
    if mi_dato=="fin":
        break
    else:
        if mi_dato in ("True", "False"):
            mi_lista.append(bool(mi_dato))
        else:
            mi_lista.append(mi_dato)

# Imprimamos la lista
print mi_lista

print bool("True")
print bool("False")
print bool("")
```

#### Listas
## Accesando listas

Los elementos de la lista se accesan utilizando **[i:j]** de la misma forma que strings y tuplas.

```python
a = [0, 10, 20, 30]
print a[0]
print a[-1]
print a

a.append(2)
a.append(-3)
print a[0]
print a[-1]
print a
```

#### Listas
## Otros métodos

* len(lista): regresa la cantidad de objetos de la lista
* sum(lista): regresa la suma de los elementos de la lista (si se puede).
* lista.pop(): regresa el ultimo elemento de la lista y lo saca de la lista.
* lista.sort(): ordena los elementos de la lista de menor a mayor.
* lista.reverse(): ordena los elementos de la lista en el orden reverso al original.

```python
x = [1,3,5,7,9,0,2,4,6,8]
print len(x)

print sum(x)

x.reverse() # Revierte orden de la lista (y no regresa nada)
print x

x.sort()  # Ordena la lista (y no regresa nada)
print x

x.reverse() # Revierte orden de la lista (y no regresa nada)
print x

xi = x.pop()
print xi
print x
```

#### Listas
## Otros métodos

Testear pertenencia en lista

    dato in lista

regresará siempre un booleano: True si dato está en la lista, False si no está en la lista.

```python
mi_lista = ["sebastian", "carlos", "jose"]

print "sebastian" in mi_lista

print "seba" in mi_lista

print "carlos" in mi_lista

print "pepe" in mi_lista
```

#### Listas
## Otros métodos: range

El método range es práctico para **generar listas de números enteros**:

* range(m) regresa la lista [0,1,2,..,m-1]
* range(n,m) regresa la lista [n,n+1,n+2,...,m-1]
* range(n,m,k) regresa la lista [n,n+k, n+2 k,...,x] donde x es el mayor numero n+j k < m

```python
print range(5)
print range(10)

print range(2, 5)
print range(2, 10)

print range(2, 5, 3)
print range(2, 11, 3)
```

#### Listas
## Ejemplo
Calcula la suma de los recíprocos de los primeros n numeros naturales:

$$ \frac{1}{1}+\frac{1}{2}+\frac{1}{3}+\frac{1}{4}+..\frac{1}{n}$$

```python
n = 10
l = []
j = 1
while j<=n:
    l.append(1./j)
    j+= 1
print sum(l)
```

#### Listas
## Copiando listas: CUIDADO

Cuando las listas se asignan, en realidad hacen referencia a una lista común. Esto se hace para ahorrar memoria RAM y está implementado así en python.

```python
a = [5, 1, 4]
b = list(a) #  a
b.append(10)
# Modifiquemos b
b[0] = 1000
print b
print a
# la lista a tambien se vio modificada
```

Para evitar este comportamiento, debemos utilizar

    b = list(a)

#### Listas
## Aplicación al ejemplo
Calcule el promedio y la desviación estándar de $N$ datos, $x_1$, ..., $x_n$, ingresados por el usuario:

$$ \begin{align}
mean &= \frac{1}{n} \sum_{i=1} x_i \\
(std)^2 &= \frac{1}{n} \sum_{i=1} (x_i- mean)^2
\end{align}
$$

```python
###################################################################
# Cambiar para utilizar listas
###################################################################
N = int(raw_input("Ingrese numero de datos:"))
# Calculo del promedio
print "Ingrese numeros (para calcular promedio)"
j = 1
mean = 0.0
while j<=N:
    x_i = float(raw_input("Ingrese numero: "))
    mean += x_i
    j += 1
mean = mean / N
# Calculo de desviacion estandar
print "Ingrese los MISMOS numeros (para calcular desviacion estandar)"
j = 1
var = 0.0
while j<=N:
    x_i = float(raw_input("Ingrese numero: "))
    var += (x_i-mean)**2
    j += 1
std = (var/N)**0.5
# Imprimir resultados
print "promedio", mean
print "desviacion estandar", std
```

#### Listas
## Solución v1
Calcule el promedio y la desviación estándar de $N$ datos, $x_1$, ..., $x_n$, ingresados por el usuario.

```python
N = int(raw_input("Ingrese numero de datos:"))
lista_datos = []
# Crear la lista
j = 1
while j<=N:
    x_i = float(raw_input("Ingrese numero: "))
    lista_datos.append(x_i)
    j += 1
# Calcular el promedio
i = 0
mean = 0.0
while i<N:
    x_i = lista_datos[i]
    mean += x_i
    i += 1
mean = mean / N
# Calculo de desviacion estandar
k = 0
var = 0.0
while k<N:
    x_k = lista_datos[k]
    var += (x_k-mean)**2
    k += 1
std = (var/N)**0.5
# Imprimir resultados
print "promedio", mean
print "desviacion estandar", std
```

#### Listas
## Solución v2
Calcule el promedio y la desviación estándar de $N$ datos, $x_1$, ..., $x_n$, ingresados por el usuario.

```python
N = int(raw_input("Ingrese numero de datos:"))
lista_datos = []
# Crear la lista
j = 1
while j<=N:
    x_i = float(raw_input("Ingrese numero: "))
    lista_datos.append(x_i)
    j += 1

# Calcular el promedio
mean = sum(lista_datos) / N

# Calculo de desviacion estandar
k = 0
var = 0.0
while k<N:
    x_k = lista_datos[k]
    var += (x_k-mean)**2
    k += 1
std = (var/N)**0.5

# Imprimir resultados
print "promedio", mean
print "desviacion estandar", std
```

## Ejercicio Tipo Certamen: C2 1S 2014

Cineton, una nueva cadena de cines creada por emprendedores de la USM, est ́a ingresando
al mercado cinematogr ́afico. Por eso necesita de su ayuda para implementar ciertas funciones
 ́ de la cartelera
en Phyton y con ellas manejar la cartelera. Para ello se cuenta con la informaci on
de cine en una lista de tuplas como cartelera. A modo de ejemplo, en cartelera la pel ́ıcula
’Gloria’ (Chilena), creada en 2013, se exhibir ́a el mes de ’enero’ en las ’sala1’ y ’sala2’.

Cada elemento de la lista tiene la siguiente estructura:

    (mes, pais, nombre_pelicula, año_filmacion, [sala1, sala2, ...])

```python
cartelera = [
('febrero', 'FRANCIA', 'El muelle', 1962, ['sala1', 'sala3']),
('febrero', 'FRANCIA', 'La dama de honor', 2004, ['sala1', 'sala4']),
('abril', 'RUSIA', 'Padre del soldado', 1964, ['sala3', 'sala2', 'sala4']),
('enero', 'CHILE', 'Gloria', 2013, ['sala1', 'sala2']),
('mayo', 'MEXICO', 'Cumbres', 2013, ['sala3', 'sala2']),
('julio', 'FRANCIA', 'Melo', 1986, ['sala3', 'sala1']),
('junio', 'BELGICA', 'Rondo', 2012, ['sala4', 'sala2']),
('marzo', 'ALEMANIA', 'Tiempo de Canibales', 2014, ['sala1', 'sala2']),
('marzo', 'ALEMANIA', 'Soul Kitchen', 2009, ['sala3', 'sala4']),
]
```

#### Ejercicio Tipo Certamen: C2 1S 2014
## Pregunta 1

Desarrolle la función pelicula_por_pais(cartelera, pais) que recibe la lista de la cartelera y el nombre de un país, y que retorne la lista con las películas realizadas en dicho país. Cada elemento de esta lista resultante es una tupla con el nombre de la película y el año de filmación.

    >>> pelicula_por_pais(cartelera, 'FRANCIA')
    [('El muelle', 1962), ('La dama de honor', 2004), ('Melo', 1986)]

```python
# Solucion estudiantes
```

#### Ejercicio Tipo Certamen: C2 1S 2014
## Solución pregunta 1

Desarrolle la función pelicula_por_pais(cartelera, pais) que recibe la lista de la cartelera y el nombre de un país, y que retorne la lista con las películas realizadas en dicho país. Cada elemento de esta lista resultante es una tupla con el nombre de la película y el año de filmación.

    >>> pelicula_por_pais(cartelera, 'FRANCIA')
    [('El muelle', 1962), ('La dama de honor', 2004), ('Melo', 1986)]

```python
# Solucion

def pelicula_por_pais(cartelera, pais):
    n = len(cartelera)
    j = 0
    peliculas_pais = []
    while j<n:
        mes_j, pais_j, nombre_pelicula_j, anno_j, salas_j = cartelera[j]
        if pais_j==pais:
            peliculas_pais.append( (nombre_pelicula_j, anno_j) )
        j += 1

    return peliculas_pais

print pelicula_por_pais(cartelera, 'FRANCIA')
```

## Minitareas

Próximo lunes Actividad 3: listas y tuplas. ¡Practiquen!

Enviar minitareas.


# Tutorial 3: "Why" models
**Week 1, Day 1: Model Types**

**By Neuromatch Academy**

__Content creators:__ Matt Laporte, Byron Galbraith, Konrad Kording

__Content reviewers:__ Dalin Guo, Aishwarya Balwani, Madineh Sarvestani, Maryam Vaziri-Pashkam, Michael Waskom, Ella Batty

We would like to acknowledge [Steinmetz _et al._ (2019)](https://www.nature.com/articles/s41586-019-1787-x) for sharing their data, a subset of which is used here.

**Our 2021 Sponsors, including Presenting Sponsor Facebook Reality Labs**

___
# Tutorial Objectives

*Estimated timing of tutorial: 45 minutes*

This is tutorial 3 of a 3-part series on different flavors of models used to understand neural data. In parts 1 and 2 we explored mechanisms that would produce the data. In this tutorial we will explore models and techniques that can potentially explain *why* the spiking data we have observed is produced the way it is.

To understand why different spiking behaviors may be beneficial, we will learn about the concept of entropy. Specifically, we will:

- Write code to compute formula for entropy, a measure of information
- Compute the entropy of a number of toy distributions
- Compute the entropy of spiking activity from the Steinmetz dataset

```python
# @title Tutorial slides

# @markdown These are the slides for the videos in all tutorials today
from IPython.display import IFrame
IFrame(src=f"https://mfr.ca-1.osf.io/render?url=https://osf.io/6dxwe/?direct%26mode=render%26action=download%26mode=render", width=854, height=480)
```

```python
# @title Video 1: “Why” models
from ipywidgets import widgets

out2 = widgets.Output()
with out2:
  from IPython.display import IFrame
  class BiliVideo(IFrame):
      def __init__(self, id, page=1, width=400, height=300, **kwargs):
          self.id=id
          src = 'https://player.bilibili.com/player.html?bvid={0}&page={1}'.format(id, page)
          super(BiliVideo, self).__init__(src, width, height, **kwargs)

  video = BiliVideo(id="BV16t4y1Q7DR", width=854, height=480, fs=1)
  print('Video available at https://www.bilibili.com/video/{0}'.format(video.id))
  display(video)

out1 = widgets.Output()
with out1:
  from IPython.display import YouTubeVideo
  video = YouTubeVideo(id="OOIDEr1e5Gg", width=854, height=480, fs=1, rel=0)
  print('Video available at https://youtube.com/watch?v=' + video.id)
  display(video)

out = widgets.Tab([out1, out2])
out.set_title(0, 'Youtube')
out.set_title(1, 'Bilibili')

display(out)
```

    Tab(children=(Output(), Output()), _titles={'0': 'Youtube', '1': 'Bilibili'})

# Setup

```python
# Imports

import numpy as np
import matplotlib.pyplot as plt
from scipy import stats
```

```python
#@title Figure Settings
import ipywidgets as widgets #interactive display

%matplotlib inline
%config InlineBackend.figure_format = 'retina'
plt.style.use("https://raw.githubusercontent.com/NeuromatchAcademy/course-content/master/nma.mplstyle")
```

```python
#@title Plotting Functions

def plot_pmf(pmf,isi_range):
  """Plot the probability mass function."""
  ymax = max(0.2, 1.05 * np.max(pmf))
  pmf_ = np.insert(pmf, 0, pmf[0])
  plt.plot(bins, pmf_, drawstyle="steps")
  plt.fill_between(bins, pmf_, step="pre", alpha=0.4)
  plt.title(f"Neuron {neuron_idx}")
  plt.xlabel("Inter-spike interval (s)")
  plt.ylabel("Probability mass")
  plt.xlim(isi_range);
  plt.ylim([0, ymax])
```

```python
#@title Download Data
import io
import requests
r = requests.get('https://osf.io/sy5xt/download')
if r.status_code != 200:
  print('Could not download data')
else:
  steinmetz_spikes = np.load(io.BytesIO(r.content), allow_pickle=True)['spike_times']
```

---
# Section 1: Optimization and Information

*Remember that the notation section is located after the Summary for quick reference!*

Neurons can only fire so often in a fixed period of time, as the act of emitting a spike consumes energy that is depleted and must eventually be replenished. To communicate effectively for downstream computation, the neuron would need to make good use of its limited spiking capability. This becomes an optimization problem:

What is the optimal way for a neuron to fire in order to maximize its ability to communicate information?

In order to explore this question, we first need to have a quantifiable measure for information. Shannon introduced the concept of entropy to do just that, and defined it as

\begin{align}
  H_b(X) = -\sum_{x\in X} p(x) \log_b p(x)
\end{align}

where $H$ is entropy measured in units of base $b$ and $p(x)$ is the probability of observing the event $x$ from the set of all possible events in $X$. See the Bonus Section 1 for a more detailed look at how this equation was derived.

The most common base of measuring entropy is $b=2$, so we often talk about *bits* of information, though other bases are used as well (e.g. when $b=e$ we call the units *nats*).

First, let's explore how entropy changes between some simple discrete probability distributions. In the rest of this tutorial we will refer to these as probability mass functions (PMF), where $p(x_i)$ equals the $i^{th}$ value in an array, and mass refers to how much of the distribution is contained at that value.

For our first PMF, we will choose one where all of the probability mass is located in the middle of the distribution.

```python
n_bins = 50  # number of points supporting the distribution
x_range = (0, 1)  # will be subdivided evenly into bins corresponding to points

bins = np.linspace(*x_range, n_bins + 1)  # bin edges

pmf = np.zeros(n_bins)
pmf[len(pmf) // 2] = 1.0  # middle point has all the mass

# Since we already have a PMF, rather than un-binned samples, `plt.hist` is not
# suitable. Instead, we directly plot the PMF as a step function to visualize
# the histogram:
pmf_ = np.insert(pmf, 0, pmf[0])  # this is necessary to align plot steps with bin edges
plt.plot(bins, pmf_, drawstyle="steps")
# `fill_between` provides area shading
plt.fill_between(bins, pmf_, step="pre", alpha=0.4)
plt.xlabel("x")
plt.ylabel("p(x)")
plt.xlim(x_range)
plt.ylim(0, 1);
```

If we were to draw a sample from this distribution, we know exactly what we would get every time. Distributions where all the mass is concentrated on a single event are known as *deterministic*.

How much entropy is contained in a deterministic distribution? We will compute this in the next exercise.

## Coding Exercise 1: Computing Entropy

Your first exercise is to implement a method that computes the entropy of a discrete probability distribution, given its mass function. Remember that we are interested in entropy in units of _bits_, so be sure to use the correct log function.

Recall that $\log(0)$ is undefined. When evaluated at $0$, NumPy log functions (such as `np.log2`) return `np.nan` ("Not a Number"). By convention, these undefined terms— which correspond to points in the distribution with zero mass—are excluded from the sum that computes the entropy.

```python
def entropy(pmf):
  """Given a discrete distribution, return the Shannon entropy in bits.

  This is a measure of information in the distribution. For a totally
  deterministic distribution, where samples are always found in the same bin,
  then samples from the distribution give no more information and the entropy
  is 0.

  For now this assumes `pmf` arrives as a well-formed distribution (that is,
  `np.sum(pmf)==1` and `not np.any(pmf < 0)`)

  Args:
    pmf (np.ndarray): The probability mass function for a discrete distribution
      represented as an array of probabilities.
  Returns:
    h (number): The entropy of the distribution in `pmf`.

  """
  ############################################################################
  # Exercise for students: compute the entropy of the provided PMF
  #   1. Exclude the points in the distribution with no mass (where `pmf==0`).
  #      Hint: this is equivalent to including only the points with `pmf>0`.
  #   2. Implement the equation for Shannon entropy (in bits).
  #  When ready to test, comment or remove the next line
  ############################################################################

  # reduce to non-zero entries to avoid an error from log2(0)
  pmf = pmf[pmf>0]

  # implement the equation for Shannon entropy (in bits)
  h = -np.sum(pmf*np.log2(pmf))

  # return the absolute value (avoids getting a -0 result)
  return np.abs(h)

# Call entropy function and print result
print(f"{entropy(pmf):.2f} bits")
```

    0.00 bits

```python
# to_remove solution
def entropy(pmf):
  """Given a discrete distribution, return the Shannon entropy in bits.

  This is a measure of information in the distribution. For a totally
  deterministic distribution, where samples are always found in the same bin,
  then samples from the distribution give no more information and the entropy
  is 0.

  For now this assumes `pmf` arrives as a well-formed distribution (that is,
  `np.sum(pmf)==1` and `not np.any(pmf < 0)`)

  Args:
    pmf (np.ndarray): The probability mass function for a discrete distribution
      represented as an array of probabilities.
  Returns:
    h (number): The entropy of the distribution in `pmf`.
  """
  # reduce to non-zero entries to avoid an error from log2(0)
  pmf = pmf[pmf > 0]

  # implement the equation for Shannon entropy (in bits)
  h = -np.sum(pmf * np.log2(pmf))

  # return the absolute value (avoids getting a -0 result)
  return np.abs(h)

# Call entropy function and print result
print(f"{entropy(pmf):.2f} bits")
```

    0.00 bits

We expect zero surprise from a deterministic distribution. If we had done this calculation by hand, it would simply be $-1\log_2 1 = -0=0$.

Note that changing the location of the peak (i.e. the point and bin on which all the mass rests) doesn't alter the entropy. The entropy is about how predictable a sample is with respect to a distribution. A single peak is deterministic regardless of which point it sits on - the following plot shows a PMF that would also have zero entropy.

```python
# @markdown Execute this cell to visualize another PMF with zero entropy
pmf = np.zeros(n_bins)
pmf[2] = 1.0  # arbitrary point has all the mass

pmf_ = np.insert(pmf, 0, pmf[0])
plt.plot(bins, pmf_, drawstyle="steps")
plt.fill_between(bins, pmf_, step="pre", alpha=0.4)
plt.xlabel("x")
plt.ylabel("p(x)")
plt.xlim(x_range)
plt.ylim(0, 1);
```

What about a distribution with mass split equally between two points?

```python
# @markdown Execute this cell to visualize a PMF with split mass

pmf = np.zeros(n_bins)
pmf[len(pmf) // 3] = 0.5
pmf[2 * len(pmf) // 3] = 0.5

pmf_ = np.insert(pmf, 0, pmf[0])
plt.plot(bins, pmf_, drawstyle="steps")
plt.fill_between(bins, pmf_, step="pre", alpha=0.4)
plt.xlabel("x")
plt.ylabel("p(x)")
plt.xlim(x_range)
plt.ylim(0, 1);
```

Here, the entropy calculation is: $-(0.5 \log_2 0.5 + 0.5\log_2 0.5)=1$

There is 1 bit of entropy. This means that before we take a random sample, there is 1 bit of uncertainty about which point in the distribution the sample will fall on: it will either be the first peak or the second one.

Likewise, if we make one of the peaks taller (i.e. its point holds more of the probability mass) and the other one shorter, the entropy will decrease because of the increased certainty that the sample will fall on one point and not the other: : $-(0.2 \log_2 0.2 + 0.8\log_2 0.8)\approx 0.72$

Try changing the definition of the number and weighting of peaks, and see how the entropy varies.

If we split the probability mass among even more points, the entropy continues to increase. Let's derive the general form for $N$ points of equal mass, where $p_i=p=1/N$:

\begin{align}
 -\sum_i p_i \log_b p_i&= -\sum_i^N \frac{1}{N} \log_b \frac{1}{N}\\
                       &= -\log_b \frac{1}{N} \\
                       &= \log_b N
\end{align}

If we have $N$ discrete points, the _uniform distribution_ (where all points have equal mass) is the distribution with the highest entropy: $\log_b N$. This upper bound on entropy is useful when considering binning strategies, as any estimate of entropy over $N$ discrete points (or bins) must be in the interval $[0, \log_b N]$.

```python
# @markdown Execute this cell to visualize a PMF of uniform distribution

pmf = np.ones(n_bins) / n_bins  # [1/N] * N

pmf_ = np.insert(pmf, 0, pmf[0])
plt.plot(bins, pmf_, drawstyle="steps")
plt.fill_between(bins, pmf_, step="pre", alpha=0.4)
plt.xlabel("x")
plt.ylabel("p(x)")
plt.xlim(x_range);
plt.ylim(0, 1);
```

Here, there are 50 points and the entropy of the uniform distribution is $\log_2 50\approx 5.64$. If we construct _any_ discrete distribution $X$ over 50 points (or bins) and calculate an entropy of $H_2(X)>\log_2 50$, something must be wrong with our implementation of the discrete entropy computation.

---
# Section 2: Information, neurons, and spikes

*Estimated timing to here from start of tutorial: 20 min*

```python
# @title Video 2: Entropy of different distributions
from ipywidgets import widgets

out2 = widgets.Output()
with out2:
  from IPython.display import IFrame
  class BiliVideo(IFrame):
      def __init__(self, id, page=1, width=400, height=300, **kwargs):
          self.id=id
          src = 'https://player.bilibili.com/player.html?bvid={0}&page={1}'.format(id, page)
          super(BiliVideo, self).__init__(src, width, height, **kwargs)

  video = BiliVideo(id="BV1df4y1976g", width=854, height=480, fs=1)
  print('Video available at https://www.bilibili.com/video/{0}'.format(video.id))
  display(video)

out1 = widgets.Output()
with out1:
  from IPython.display import YouTubeVideo
  video = YouTubeVideo(id="o6nyrx3KH20", width=854, height=480, fs=1, rel=0)
  print('Video available at https://youtube.com/watch?v=' + video.id)
  display(video)

out = widgets.Tab([out1, out2])
out.set_title(0, 'Youtube')
out.set_title(1, 'Bilibili')

display(out)
```

    Tab(children=(Output(), Output()), _titles={'0': 'Youtube', '1': 'Bilibili'})

Recall the discussion of spike times and inter-spike intervals (ISIs) from Tutorial 1. What does the information content (or distributional entropy) of these measures say about our theory of nervous systems?

We'll consider three hypothetical neurons that all have the same mean ISI, but with different distributions:

1. Deterministic
2. Uniform
3. Exponential

Fixing the mean of the ISI distribution is equivalent to fixing its inverse: the neuron's mean firing rate. If a neuron has a fixed energy budget and each of its spikes has the same energy cost, then by fixing the mean firing rate, we are normalizing for energy expenditure. This provides a basis for comparing the entropy of different ISI distributions. In other words: if our neuron has a fixed budget, what ISI distribution should it express (all else being equal) to maximize the information content of its outputs?

Let's construct our three distributions and see how their entropies differ.

```python
n_bins = 50
mean_isi = 0.025
isi_range = (0, 0.25)

bins = np.linspace(*isi_range, n_bins + 1)
mean_idx = np.searchsorted(bins, mean_isi)

# 1. all mass concentrated on the ISI mean
pmf_single = np.zeros(n_bins)
pmf_single[mean_idx] = 1.0

# 2. mass uniformly distributed about the ISI mean
pmf_uniform = np.zeros(n_bins)
pmf_uniform[0:2*mean_idx] = 1 / (2 * mean_idx)

# 3. mass exponentially distributed about the ISI mean
pmf_exp = stats.expon.pdf(bins[1:], scale=mean_isi)
pmf_exp /= np.sum(pmf_exp)
```

```python
#@title
#@markdown Run this cell to plot the three PMFs
fig, axes = plt.subplots(ncols=3, figsize=(18, 5))

dists =  [# (subplot title, pmf, ylim)
          ("(1) Deterministic", pmf_single, (0, 1.05)),
          ("(1) Uniform", pmf_uniform, (0, 1.05)),
          ("(1) Exponential", pmf_exp, (0, 1.05))]

for ax, (label, pmf_, ylim) in zip(axes, dists):
  pmf_ = np.insert(pmf_, 0, pmf_[0])
  ax.plot(bins, pmf_, drawstyle="steps")
  ax.fill_between(bins, pmf_, step="pre", alpha=0.4)
  ax.set_title(label)
  ax.set_xlabel("Inter-spike interval (s)")
  ax.set_ylabel("Probability mass")
  ax.set_xlim(isi_range);
  ax.set_ylim(ylim);
```

```python
print(
  f"Deterministic: {entropy(pmf_single):.2f} bits",
  f"Uniform: {entropy(pmf_uniform):.2f} bits",
  f"Exponential: {entropy(pmf_exp):.2f} bits",
  sep="\n",
)
```

    Deterministic: 0.00 bits
    Uniform: 3.32 bits
    Exponential: 3.77 bits

---
# Section 3: Calculate entropy of ISI distributions from data

*Estimated timing to here from start of tutorial: 25 min*

## Section 3.1: Computing probabilities from histogram

```python
# @title Video 3: Probabilities from histogram
from ipywidgets import widgets

out2 = widgets.Output()
with out2:
  from IPython.display import IFrame
  class BiliVideo(IFrame):
      def __init__(self, id, page=1, width=400, height=300, **kwargs):
          self.id=id
          src = 'https://player.bilibili.com/player.html?bvid={0}&page={1}'.format(id, page)
          super(BiliVideo, self).__init__(src, width, height, **kwargs)

  video = BiliVideo(id="BV1Jk4y1B7cz", width=854, height=480, fs=1)
  print('Video available at https://www.bilibili.com/video/{0}'.format(video.id))
  display(video)

out1 = widgets.Output()
with out1:
  from IPython.display import YouTubeVideo
  video = YouTubeVideo(id="e2U_-07O9jo", width=854, height=480, fs=1, rel=0)
  print('Video available at https://youtube.com/watch?v=' + video.id)
  display(video)

out = widgets.Tab([out1, out2])
out.set_title(0, 'Youtube')
out.set_title(1, 'Bilibili')

display(out)
```

    Tab(children=(Output(), Output()), _titles={'0': 'Youtube', '1': 'Bilibili'})

In the previous example we created the PMFs by hand to illustrate idealized scenarios. How would we compute them from data recorded from actual neurons?

One way is to convert the ISI histograms we've previously computed into discrete probability distributions using the following equation:

\begin{align}
p_i = \frac{n_i}{\sum\nolimits_{i}n_i}
\end{align}

where $p_i$ is the probability of an ISI falling within a particular interval $i$ and $n_i$ is the count of how many ISIs were observed in that interval.

### Coding Exercise 3.1: Probabilty Mass Function

Your second exercise is to implement a method that will produce a probability mass function from an array of ISI bin counts.

To verify your solution, we will compute the probability distribution of ISIs from real neural data taken from the Steinmetz dataset.

```python
def pmf_from_counts(counts):
  """Given counts, normalize by the total to estimate probabilities."""
  ###########################################################################
  # Exercise: Compute the PMF. Remove the next line to test your function
  ###########################################################################

  pmf = counts/np.sum(counts)

  return pmf

# Get neuron index
neuron_idx = 283

# Get counts of ISIs from Steinmetz data
isi = np.diff(steinmetz_spikes[neuron_idx])
bins = np.linspace(*isi_range, n_bins + 1)
counts, _ = np.histogram(isi, bins)

# Compute pmf
pmf = pmf_from_counts(counts)

# Visualize
plot_pmf(pmf,isi_range)
```

```python
# to_remove solution
def pmf_from_counts(counts):
  """Given counts, normalize by the total to estimate probabilities."""
  pmf = counts / np.sum(counts)
  return pmf

# Get neuron index
neuron_idx = 283

# Get counts of ISIs from Steinmetz data
isi = np.diff(steinmetz_spikes[neuron_idx])
bins = np.linspace(*isi_range, n_bins + 1)
counts, _ = np.histogram(isi, bins)

# Compute pmf
pmf = pmf_from_counts(counts)

# Visualize
with plt.xkcd():
  plot_pmf(pmf,isi_range)
```

## Section 3.2: Calculating entropy from pmf

```python
# @title Video 4: Calculating entropy from pmf
from ipywidgets import widgets

out2 = widgets.Output()
with out2:
  from IPython.display import IFrame
  class BiliVideo(IFrame):
      def __init__(self, id, page=1, width=400, height=300, **kwargs):
          self.id=id
          src = 'https://player.bilibili.com/player.html?bvid={0}&page={1}'.format(id, page)
          super(BiliVideo, self).__init__(src, width, height, **kwargs)

  video = BiliVideo(id="BV1vA411e7Cd", width=854, height=480, fs=1)
  print('Video available at https://www.bilibili.com/video/{0}'.format(video.id))
  display(video)

out1 = widgets.Output()
with out1:
  from IPython.display import YouTubeVideo
  video = YouTubeVideo(id="Xjy-jj-6Oz0", width=854, height=480, fs=1, rel=0)
  print('Video available at https://youtube.com/watch?v=' + video.id)
  display(video)

out = widgets.Tab([out1, out2])
out.set_title(0, 'Youtube')
out.set_title(1, 'Bilibili')

display(out)
```

    Tab(children=(Output(), Output()), _titles={'0': 'Youtube', '1': 'Bilibili'})

Now that we have the probability distribution for the actual neuron spiking activity, we can calculate its entropy.

```python
print(f"Entropy for Neuron {neuron_idx}: {entropy(pmf):.2f} bits")
```

    Entropy for Neuron 283: 3.36 bits

### Interactive Demo 3.2: Entropy of neurons

We can combine the above distribution plot and entropy calculation with an interactive widget to explore how the different neurons in the dataset vary in spiking activity and relative information. Note that the mean firing rate across neurons is not fixed, so some neurons with a uniform ISI distribution may have higher entropy than neurons with a more exponential distribution.

```python
#@title
#@markdown **Run the cell** to enable the sliders.

def _pmf_from_counts(counts):
  """Given counts, normalize by the total to estimate probabilities."""
  pmf = counts / np.sum(counts)
  return pmf

def _entropy(pmf):
  """Given a discrete distribution, return the Shannon entropy in bits."""
  # remove non-zero entries to avoid an error from log2(0)
  pmf = pmf[pmf > 0]
  h = -np.sum(pmf * np.log2(pmf))
  # absolute value applied to avoid getting a -0 result
  return np.abs(h)

@widgets.interact(neuron=widgets.IntSlider(0, min=0, max=(len(steinmetz_spikes)-1)))
def steinmetz_pmf(neuron):
  """ Given a neuron from the Steinmetz data, compute its PMF and entropy """
  isi = np.diff(steinmetz_spikes[neuron])
  bins = np.linspace(*isi_range, n_bins + 1)
  counts, _ = np.histogram(isi, bins)
  pmf = _pmf_from_counts(counts)

  plot_pmf(pmf,isi_range)
  plt.title(f"Neuron {neuron}: H = {_entropy(pmf):.2f} bits")
```

    interactive(children=(IntSlider(value=0, description='neuron', max=733), Output()), _dom_classes=('widget-inte…

---
# Section 4: Reflecting on why models

*Estimated timing to here from start of tutorial: 35 min*

## Think! 3: Reflecting on how models

Please discuss the following questions for around 10 minutes with your group:

- Have you seen why models before?
- Have you ever done one?
- Why are why models useful?
- When are they possible? Does your field have why models?
- What do we learn from constructing them?

---
# Summary

*Estimated timing of tutorial: 45 minutes*

```python
# @title Video 5: Summary of model types
from ipywidgets import widgets

out2 = widgets.Output()
with out2:
  from IPython.display import IFrame
  class BiliVideo(IFrame):
      def __init__(self, id, page=1, width=400, height=300, **kwargs):
          self.id=id
          src = 'https://player.bilibili.com/player.html?bvid={0}&page={1}'.format(id, page)
          super(BiliVideo, self).__init__(src, width, height, **kwargs)

  video = BiliVideo(id="BV1F5411e7ww", width=854, height=480, fs=1)
  print('Video available at https://www.bilibili.com/video/{0}'.format(video.id))
  display(video)

out1 = widgets.Output()
with out1:
  from IPython.display import YouTubeVideo
  video = YouTubeVideo(id="X4K2RR5qBK8", width=854, height=480, fs=1, rel=0)
  print('Video available at https://youtube.com/watch?v=' + video.id)
  display(video)

out = widgets.Tab([out1, out2])
out.set_title(0, 'Youtube')
out.set_title(1, 'Bilibili')

display(out)
```

    Tab(children=(Output(), Output()), _titles={'0': 'Youtube', '1': 'Bilibili'})

Congratulations! You've finished your first NMA tutorial. In this 3 part tutorial series, we used different types of models to understand the spiking behavior of neurons recorded in the Steinmetz data set.

 - We used "what" models to discover that the ISI distribution of real neurons is closest to an exponential distribution
 - We used "how" models to discover that balanced excitatory and inbhitiory inputs, coupled with a leaky membrane, can give rise to neuronal spiking with exhibiting such an exponential ISI distribution
 - We used "why" models to discover that exponential ISI distributions contain the most information when the mean spiking is constrained

---
# Notation

\begin{align}
H(X) &\quad \text{entropy of random variable X}\\
b &\quad \text{base, e.g. b=2 or b=e}\\
x &\quad \text{event x}\\
p(x) &\quad \text{probability of observing event x}\\
\text{ISI} &\quad \text{interspike interval}\\
n_i &\quad \text{count of observed ISIs in interval i}\\
p_i  &\quad \text{probability of of an ISI falling within a particular interval i}
\end{align}

---
# Bonus

---
## Bonus Section 1: The foundations for Entropy

In his foundational [1948 paper](https://en.wikipedia.org/wiki/A_Mathematical_Theory_of_Communication) on information theory, Claude Shannon began with three criteria for a function $H$ defining the entropy of a discrete distribution of probability masses $p_i\in p(X)$ over the points $x_i\in X$:
1. $H$ should be continuous in the $p_i$.
  - That is, $H$ should change smoothly in response to smooth changes to the mass $p_i$ on each point $x_i$.
2. If all the points have equal shares of the probability mass, $p_i=1/N$, $H$ should be a non-decreasing function of $N$.
  - That is, if $X_N$ is the support with $N$ discrete points and $p(x\in X_N)$ assigns constant mass to each point, then $H(X_1) < H(X_2) < H(X_3) < \dots$
3. $H$ should be preserved by (invariant to) the equivalent (de)composition of distributions.
  - For example (from Shannon's paper) if we have a discrete distribution over three points with masses $(\frac{1}{2},\frac{1}{3},\frac{1}{6})$, then their entropy can be represented in terms of a direct choice between the three and calculated $H(\frac{1}{2},\frac{1}{3},\frac{1}{6})$. However, it could also be represented in terms of a series of two choices:
    1. either we sample the point with mass $1/2$ or not (_not_ is the other $1/2$, whose subdivisions are not given in the first choice),
    2. if (with probability $1/2$) we _don't_ sample the first point, we sample one of the two remaining points, masses $1/3$ and $1/6$.

    Thus in this case we require that $H(\frac{1}{2},\frac{1}{3},\frac{1}{6})=H(\frac{1}{2},\frac{1}{2}) + \frac{1}{2}H(\frac{1}{3}, \frac{1}{6})$

There is a unique function (up to a linear scaling factor) which satisfies these 3 requirements:

\begin{align}
  H_b(X) = -\sum_{x\in X} p(x) \log_b p(x)
\end{align}

Where the base of the logarithm $b>1$ controls the units of entropy. The two most common cases are $b=2$ for units of _bits_, and $b=e$ for _nats_.

We can view this function as the expectation of the self-information over a distribution:

\begin{align}
H_b(X) = E_{x\in X} \left[I_b(x)\right]\\
I_b(x) =-\log_b p(x)
\end{align}

Self-information is just the negative logarithm of probability, and is a measure of how surprising an event sampled from the distribution would be. Events with $p(x)=1$ are certain to occur, and their self-information is zero (as is the entropy of the distribution they compose) meaning they are totally unsurprising. The smaller the probability of an event, the higher its self-information, and the more surprising the event would be to observe.


# Lagrangian mechanics in generalized coordinates

> Marcos Duarte
> [Laboratory of Biomechanics and Motor Control](http://pesquisa.ufabc.edu.br/bmclab)
> Federal University of ABC, Brazil

The Lagrangian mechanics can be formulated completely independent of the Newtonian mechanics and Cartesian coordinates; Lagrange developed this new formalism based on the [principle of least action](http://nbviewer.jupyter.org/github/BMClab/bmc/blob/master/notebooks/principle_of_least_action.ipynb).
In this notebook, we will take a less noble path, we will deduce the Lagrange's equation from Newtonian mechanics.

Contents

1  Review on Newton's laws of motion

1.1  Mechanical energy

2  Lagrange's equation in Cartesian Coordinates

3  Generalized coordinates

4  Lagrange's equation

4.1  Constraints

5  Further reading

6  Video lectures on the internet

7  References

## Review on Newton's laws of motion

The [Newton's laws of motion](https://en.wikipedia.org/wiki/Newton's_laws_of_motion) laid the foundation for classical mechanics. They describe the relationship between the motion of a body and the possible forces acting upon it.

Consider the motion of a particle in three-dimensional space, its position in time can be represented by the following vector:

\begin{equation}
\vec{r}(t) = x(t)\hat{i} + y(t)\hat{j} + z(t)\hat{k}
\label{}
\end{equation}

And given its position, the particle's velocity and acceleration are:

\begin{equation} \begin{array}{l}
\vec{v}(t) = \dfrac{\mathrm d \vec{r}(t)}{\mathrm d t} = \dfrac{d x(t)}{\mathrm d t}\hat{i} + \dfrac{d y(t)}{\mathrm d t}\hat{j} + \dfrac{d z(t)}{\mathrm d t}\hat{k} \\
\vec{a}(t) = \dfrac{\mathrm d \vec{v}(t)}{\mathrm d t} = \dfrac{\mathrm d^2 \vec{r}(t)}{\mathrm d t^2} = \dfrac{d^2 x(t)}{\mathrm d t^2}\hat{i} + \dfrac{d^2 y(t)}{\mathrm d t^2}\hat{j} + \dfrac{d^2 z(t)}{\mathrm d t^2}\hat{k}
\label{}
\end{array} \end{equation}

The particle's linear momentum is defined as:

\begin{equation}
\vec{p}(t) = m\vec{v}(t)
\label{}
\end{equation}

where $m$ and $\vec{v}$ are the mass and velocity of the body.

Newton's second law relates the resultant force applied on the particle to the rate of variation of its linear momentum, and if the mass is constant:

\begin{equation} \begin{array}{l}
\vec{F}(t) = \dfrac{\mathrm d \vec{p}(t)}{\mathrm d t} = \dfrac{\mathrm d (m\vec{v}(t))}{\mathrm d t} \\
\vec{F}(t) = m\vec{a}(t)
\label{}
\end{array} \end{equation}

From Newton's second law, if the position of the particle at any time is known, one can determine the resultant force acting on it. If the position is not known, but the resultant force is, the position of the particle can determined solving the following second order ordinary differential equation:

\begin{equation}
\frac{\mathrm d^2 \vec{r}(t)}{\mathrm d t^2} = \frac{\vec{F}(t)}{m}
\label{}
\end{equation}

The differential equation above is referred as the equation of motion (EOM) of the particle. For example, a system of $N$ particles will require $3N$ EOMs to describe their motion.
The EOM has the general solution:

\begin{equation}
\vec{r}(t) = \int\!\bigg(\int\frac{\vec{F}(t)}{m} \, \mathrm{d}t\bigg) \, \mathrm{d}t
\label{}
\end{equation}

which requires the determination of two constants, the initial position and velocity.

### Mechanical energy

A related physical quantity is the mechanical energy, which is the sum of kinetic and potential energies.
The kinetic energy, $T$ of a particle is given by:

\begin{equation}
T = \frac{1}{2}m v^2
\label{}
\end{equation}

Which can be expressed in terms of its linear momentum as:

\begin{equation}
T = \frac{1}{2m}  p^2
\label{}
\end{equation}

And for a given coordinate of the particle's motion, its linear momentum can be obtained from its kinetic energy by:

\begin{equation}
\vec{p} = \frac{\partial T}{\partial \vec{v}}
\label{eq11}
\end{equation}

The potential energy $V$ is the stored energy of a particle and its formulation is dependent on the force acting on the particle. For a conservative force dependent solely on the particle position, such as due to the gravitational field near the Earth surface or due to a linear spring, the force can be expressed in terms of the gradient of the potential energy:

\begin{equation}
\vec{F} = -\nabla V(\vec{r}) = -\frac{\partial V}{\partial x}\hat{i} - \frac{\partial V}{\partial y}\hat{j} - \frac{\partial V}{\partial z}\hat{k}
\label{eq12}
\end{equation}

## Lagrange's equation in Cartesian Coordinates

For simplicity, let's first deduce the Lagrange's equation for a particle in Cartesian Coordinates and from Newton's second law.

Because we want to deduce the laws of motion based on the mechanical energy of the particle, one can see that the time derivative of the expression for the linear momentum as a function of the kinetic energy, cf. Eq. (\ref{eq11}), is equal to the force acting on the particle and we can substitute the force in Newton's second law by this term:

\begin{equation}
\frac{\mathrm d }{\mathrm d t}\bigg(\frac{\partial T}{\partial \dot x}\bigg) = m\ddot x
\label{eq13}
\end{equation}

We saw that a conservative force can also be expressed in terms of the potential energy of the particle, cf. Eq. (\ref{eq12}); substituting the right side of the equation above by this expression, we have:

\begin{equation}
\frac{\mathrm d }{\mathrm d t}\bigg(\frac{\partial T}{\partial \dot x}\bigg) = -\frac{\partial V}{\partial x}
\label{eq14}
\end{equation}

Using the fact that:

\begin{equation}
\frac{\partial T}{\partial x} = 0 \quad and \quad \frac{\partial V}{\partial \dot x} = 0
\label{eq15}
\end{equation}

We can write:

\begin{equation}
\frac{\mathrm d }{\mathrm d t}\bigg(\frac{\partial (T-V)}{\partial \dot x}\bigg) - \frac{\partial (T-V)}{\partial x} = 0
\label{eq16}
\end{equation}

Defining the Lagrange or Lagrangian function, $\mathcal{L}$, as the difference between the kinetic and potential energy in the system:

\begin{equation}
\mathcal{L} = T - V
\label{eq17}
\end{equation}

We have the Lagrange's equation in Cartesian Coordinates for a conservative force acting on a particle:

\begin{equation}
\frac{\mathrm d }{\mathrm d t}\bigg(\frac{\partial \mathcal{L}}{\partial \dot x}\bigg) - \frac{\partial \mathcal{L}}{\partial x} = 0
\label{eq18}
\end{equation}

Once all derivatives of the Lagrangian function are calculated, this equation will be the equation of motion for the particle. If there are $N$ independent particles in a three-dimensional space, there will be $3N$ equations of motion for the system.
The set of equations above for a system are known as Euler–Lagrange's equations, or Lagrange's equations of the second kind.

## Generalized coordinates

The direct application of Newton's laws to mechanical systems results in a set of equations of motion in terms of Cartesian coordinates of each of the particles that make up the system. In many cases, this is not the most convenient coordinate system to solve the problem or describe the movement of the system. For example, for a serial chain of rigid links, such as a member of the human body or from a robot manipulator, it may be simpler to describe the positions of each link by the angles between links.

Coordinate systems such as angles of a chain of links are referred as [generalized coordinates](https://en.wikipedia.org/wiki/Generalized_coordinates). Generalized coordinates uniquely specify the positions of the particles in a system. Although there may be several generalized coordinates to describe a system, usually a judicious choice of generalized coordinates provides the minimum number of independent coordinates that define the configuration of a system (which is the number of degrees of freedom of the system), turning the problem simpler to solve. In this case, when the number of generalized coordinates equals the number of degrees of freedom, the system is referred as a holonomic system. In a non-holonomic system, the number of generalized coordinates necessary do describe the system depends on the path taken by the system.

Being a little more technical, according to [Wikipedia](https://en.wikipedia.org/wiki/Configuration_space_(physics)):
"In classical mechanics, the parameters that define the configuration of a system are called generalized coordinates, and the vector space defined by these coordinates is called the configuration space of the physical system. It is often the case that these parameters satisfy mathematical constraints, such that the set of actual configurations of the system is a manifold in the space of generalized coordinates. This manifold is called the configuration manifold of the system."

In problems where it is desired to use generalized coordinates, one can write Newton's equations of motion in terms of Cartesian coordinates and then transform them into generalized coordinates. However, it would be desirable and convenient to have a general method that would directly establish the equations of motion in terms of a set of convenient generalized coordinates. In addition, general methods for writing, and perhaps solving, the equations of motion in terms of any coordinate system would also be desirable. The [Lagrangian mechanics](https://en.wikipedia.org/wiki/Lagrangian_mechanics) is such a method.

When describing a system of particles using any set of generalized coordinates, $q_1,\dotsc,q_{3N}$, these are related to, for example, the Cartesian coordinates by:

\begin{equation} \begin{array}{rcl}
q_i =q_i (x_1,\dotsc,x_{3N} ) \quad i=1,\dotsc,3N \\
x_i =x_i (q_1,\dotsc,q_{3N} ) \quad i=1,\dotsc,3N
\label{qx}
\end{array} \end{equation}

The Cartesian components of velocity as a function of generalized coordinates are:

\begin{equation}
\dot{x}_i =\frac{\mathrm d x_i (q_1, q_2,\dotsc,q_{3N}
)}{\mathrm d t}=\sum\limits_{j=1}^{3N} {\frac{\partial x_i }{\partial q_j }}
\frac{\mathrm d q_j }{\mathrm d t}
\label{eq_xdotqdot}
\end{equation}

where for simplicity we omitted the explicit mention of the temporal dependence of each coordinate.

That is, any Cartesian component of the particle velocity as a function of generalized coordinates is a function of all the components of position and velocity in the generalized coordinates:

\begin{equation}
\dot{x}_i = \dot{x}_i (q_1,\dotsc,q_{3N} ,\dot{q}_1,\dotsc,\dot{q}_{3N} ) \quad i=1,\dotsc,3N
\label{eq27}
\end{equation}

## Lagrange's equation

In analogy to Newtonian mechanics, one can think that the equations of motion can be obtained by equating the generalized force, $F_i$, to the temporal rate of change of each generalized momentum, $p_i$:

\begin{equation}
F_i =\frac{\partial p_i }{\partial t}
\label{eq28}
\end{equation}

In the formula above, let's substitute the quantity $p_i$ by its definition in terms of the kinetic energy:

\begin{equation}
\frac{\partial p_i }{\partial t} =\frac{\partial }{\partial t}\left( {\frac{\partial T}{\partial
\dot{q}_i }} \right)=\frac{\partial }{\partial t}\left(
{\sum\limits_{j=1}^{3N} {m_j \dot{x}_j \frac{\partial \dot{x}_j
}{\partial \dot{q}_i }} } \right)
\label{eq29}
\end{equation}

where we used:

\begin{equation}
\frac{\partial T}{\partial \dot{q}_i }=\sum\limits_{j=1}^{3N}
{\frac{\partial T}{\partial \dot{x}_j }\frac{\partial \dot{x}_j
}{\partial \dot{q}_i }}
\label{eq30}
\end{equation}

Using the [product rule](https://en.wikipedia.org/wiki/Product_rule), the derivative of the product in Eq. (\ref{eq29}) is:

\begin{equation}
\frac{\partial p_i }{\partial t}=\sum\limits_{j=1}^{3N} {m_j
\ddot{x}_j \frac{\partial \dot{x}_j }{\partial \dot{q}_i }}
+\sum\limits_{j=1}^{3N} {m_j \dot{x}_j \frac{\mathrm d }{\mathrm d t}\left(
{\frac{\partial \dot{x}_j }{\partial \dot{q}_i }} \right)}
\label{eq31}
\end{equation}

But:

\begin{equation}
\frac{\partial \dot{x}_i }{\partial \dot{q}_j }=\frac{\partial x_i
}{\partial q_j } \quad because \quad \frac{\partial
\dot{x}_i }{\partial \dot{q}_j }=\frac{\partial x_i }{\partial
t}\frac{\partial t}{\partial q_j }=\frac{\partial x_i }{\partial q_j}
\label{eq32}
\end{equation}

Then:

\begin{equation}
\frac{\partial p_i }{\partial t}=\sum\limits_{j=1}^{3N} {m_j
\ddot{x}_j \frac{\partial x_j }{\partial q_i }}
+\sum\limits_{j=1}^{3N} {m_j \dot{x}_j \frac{\mathrm d }{\mathrm d t}\left(
{\frac{\partial x_j }{\partial q_i }} \right)}
\label{eq33}
\end{equation}

The first term on the right side of the equation above is proportional to $m_j
\ddot{x}_j$ and we will define as the generalized force, $Q_i$. But, different from Newtonian mechanics, the temporal variation of the generalized momentum is equal to the generalized force plus another term, which will investigate now. The last part of this second term can be derived as:

\begin{equation}
\frac{\mathrm d }{\mathrm d t}\left( {\frac{\partial x_j }{\partial q_i }} \right) =
\sum\limits_{k=1}^{3N} {\frac{\mathrm d }{\mathrm d q_k }\left( {\frac{\partial
x_j }{\partial q_i }} \right)\frac{\mathrm d q_k }{\mathrm d t}} =\sum\limits_{k=1}^{3N}
{\frac{\partial^2 x_j }{\partial q_k \partial q_i }\dot{q}_k }
\label{eq34}
\end{equation}

where we used the [chain rule](https://en.wikipedia.org/wiki/Chain_rule) for the differentiation:
\begin{equation}
\frac{\mathrm d }{\mathrm d t}\Big( {f\big({g(t)}\big)}\Big) = \frac{\partial f}{\partial g}\frac{\partial g}{\partial t}
\label{eq35}
\end{equation}

But if we look at Eq. (\ref{eq_xdotqdot}) we see that the term at the right side of the Eq. (\ref{eq34}) can be obtained by:

\begin{equation}
\frac{\partial \dot{x}_j }{\partial q_i } = \frac{\partial }{\partial q_i }\left(\sum\limits_{k=1}^{3N} \frac{\partial
x_j }{\partial q_i }\dot{q}_k \right) = \sum\limits_{k=1}^{3N}
{\frac{\partial^2 x_j }{\partial q_k \partial q_i }\dot{q}_k }
\label{eq36}
\end{equation}

Comparing the Eq. (\ref{eq34}) and Eq. (\ref{eq36}) we have:

\begin{equation}
\frac{\mathrm d }{\mathrm d t}\left( {\frac{\partial x_j }{\partial q_i }} \right) =
\frac{\mathrm d }{\mathrm d q_i}\left( {\frac{\partial x_j }{\partial t }} \right)
\label{eq_dotdxdq}
\end{equation}

On the other hand, it is possible to relate the term $\partial \dot{x}_j / \partial q_i$ to the derivative of kinetic energy with respect to the coordinate $q_i$:

\begin{equation}
\frac{\partial T}{\partial q_i }=\frac{\partial }{\partial q_i }\left(
{\sum\limits_{j=1}^{3N} {\frac{1}{2}m_j \dot{x}_j^2} }
\right)=\sum\limits_{j=1}^{3N} {m_j \dot{x}_j } \frac{\partial
\dot{x}_j }{\partial q_i }
\label{eq38}
\end{equation}

where once again we used the chain rule for the differentiation.

Using Eq. (\ref{eq_dotdxdq}), Eq. (\ref{eq38}) becomes

\begin{equation}
\frac{\partial T}{\partial q_i }=\sum\limits_{j=1}^{3N} {m_j
\dot{x}_j } \frac{\mathrm d }{\mathrm d t}\left( {\frac{\partial x_j }{\partial q_i }}
\right)
\label{eq39}
\end{equation}

Returning to Eq. (\ref{eq33}), it can be rewritten as:

\begin{equation}
\frac{\mathrm d }{\mathrm d t}\left( {\frac{\partial T}{\partial \dot{q}_i }} \right) = Q_i + \frac{\partial T}{\partial q_i }
\label{eq40}
\end{equation}

and

\begin{equation}
\frac{\mathrm d }{\mathrm d t}\left( {\frac{\partial T}{\partial \dot{q}_i }} \right) - \frac{\partial T}{\partial q_i } = Q_i
\label{eq41}
\end{equation}

Now let's look at $Q_i$, the generalized force. It can be decomposed into two terms:

The first term, composed of the conservative forces, i.e. forces that can be written as potential gradients:

\begin{equation}
Q_C =-\frac{\partial V}{\partial q_i } \quad , \quad V=V\left( {q_1,\dotsc,q_{3N} } \right)
\label{eq42}
\end{equation}

An example of conservative force is the gravitational force.

And the second term, encompassing all non-conservative forces, $Q_{NC}$.

Then:

\begin{equation} Q_i =-\frac{\partial V}{\partial q_i }+Q_{NCi} \quad , \quad V=V\left( {q_1,\dotsc,q_{3N} } \right) \end{equation}

The Eq. (\ref{eq41}) becomes

\begin{equation}
\frac{\mathrm d }{\mathrm d t}\left( {\frac{\partial T}{\partial \dot{q}_i }}
\right)-\frac{\partial T}{\partial q_i }=-\frac{\partial V}{\partial q_i} + Q_{NCi}
\label{eq43}
\end{equation}

Rearranging, we have:

\begin{equation} \frac{\mathrm d }{\mathrm d t}\left( {\frac{\partial \left( {T-V} \right)}{\partial
\dot{q}_i }} \right)-\frac{\partial \left( {T-V} \right)}{\partial q_i} = Q_{NCi}
\label{eq44}
\end{equation}

This is possible because:

\begin{equation}
\frac{\partial V}{\partial \dot{q}_i} = 0
\label{eq45}
\end{equation}

Defining:

\begin{equation}
\mathcal{L} \equiv \mathcal{L}(q_1,\dotsc,q_{3N} ,\dot{q}_1,\dotsc,\dot{q}_{3N} ) = T - V
\label{eq46}
\end{equation}

as the Lagrange or Lagrangian function, we have the Lagrange's equation:

\begin{equation}
\frac{\mathrm d }{\mathrm d t}\left( {\frac{\partial \mathcal{L}}{\partial \dot{q}_i }}
\right)-\frac{\partial \mathcal{L}}{\partial q_i } = Q_{NCi} \quad i=1,\dotsc,3N
\label{eq47}
\end{equation}

Once all derivatives of the Lagrangian function are calculated, this equation will be the equation of motion for each particle. If there are  $N$ independent particles in a three-dimensional space, there will be $3N$ equations for the system.

The set of equations above for a system are known as Euler–Lagrange equations, or Lagrange's equations of the second kind.

### Constraints

An important class of problems in mechanics, in which the Lagrangian equations are particularly useful, are composed of constrained systems. A constraint is a restriction on the freedom of movement of a particle or a system of particles (a constraint decreases the number of degrees of freedom of a system). A rigid body, or the movement of a pendulum, are examples of constrained systems. It can be shown, in a similar way, that the Lagrange equation, deduced here for a system of free particles, is also valid for a system of particles under the action of constraints.
The Euler-Lagrange equation, for a system of $3N$ particles and with $k$ constraints, is then defined as:

\begin{equation}
\frac{\mathrm d }{\mathrm d t}\left( {\frac{\partial \mathcal{L}}{\partial \dot{q}_i}} \right)-\frac{\partial \mathcal{L}}{\partial q_i } = Q_{NCi} \quad i=1,\dotsc,3N-k
\label{eq48}
\end{equation}

## Further reading

- [The Principle of Least Action in ](https://www.feynmanlectures.caltech.edu/II_19.html)
- Vandiver JK (MIT OpenCourseWare) [An Introduction to Lagrangian Mechanics](https://ocw.mit.edu/courses/mechanical-engineering/2-003sc-engineering-dynamics-fall-2011/lagrange-equations/MIT2_003SCF11_Lagrange.pdf)

## Video lectures on the internet

- iLectureOnline: [Lectures in Lagrangian Mechanics](http://www.ilectureonline.com/lectures/subject/PHYSICS/34/245)
- MIT OpenCourseWare: [Introduction to Lagrange With Examples](https://youtu.be/zhk9xLjrmi4)

## References

- Goldstein H (1980) [Classical Mechanics](https://books.google.com.br/books?id=tJCuQgAACAAJ), 3rd ed., Addison-Wesley.
- Marion JB (1970) [Classical Dynamics of particles and systems](https://books.google.com.br/books?id=Ss43BQAAQBAJ), 2nd ed., Academic Press.
- Synge JL (1949) [Principles of Mechanics](https://books.google.com.br/books?id=qsYfENCRG5QC), 2nd ed., McGraw-hill.
- Taylor J (2005) [Classical Mechanics](https://archive.org/details/JohnTaylorClassicalMechanics). University Science Books.


```python
%matplotlib inline
```

Writing mathematical expressions
================================

An introduction to writing mathematical expressions in Matplotlib.

You can use a subset TeX markup in any matplotlib text string by placing it
inside a pair of dollar signs ($).

Note that you do not need to have TeX installed, since Matplotlib ships
its own TeX expression parser, layout engine, and fonts.  The layout engine
is a fairly direct adaptation of the layout algorithms in Donald Knuth's
TeX, so the quality is quite good (matplotlib also provides a ``usetex``
option for those who do want to call out to TeX to generate their text (see
:doc:`/tutorials/text/usetex`).

Any text element can use math text.  You should use raw strings (precede the
quotes with an ``'r'``), and surround the math text with dollar signs ($), as
in TeX. Regular text and mathtext can be interleaved within the same string.
Mathtext can use DejaVu Sans (default), DejaVu Serif, the Computer Modern fonts
(from (La)TeX), `STIX <http://www.stixfonts.org/>`_ fonts (with are designed
to blend well with Times), or a Unicode font that you provide.  The mathtext
font can be selected with the customization variable ``mathtext.fontset`` (see
:doc:`/tutorials/introductory/customizing`)

Here is a simple example::

    # plain text
    plt.title('alpha > beta')

produces "alpha > beta".

Whereas this::

    # math text
    plt.title(r'$\alpha > \beta$')

produces ":mathmpl:`\alpha > \beta`".

Note

Mathtext should be placed between a pair of dollar signs ($). To make it
   easy to display monetary values, e.g., "$100.00", if a single dollar sign
   is present in the entire string, it will be displayed verbatim as a dollar
   sign.  This is a small change from regular TeX, where the dollar sign in
   non-math text would have to be escaped ('\\\$').

Note

While the syntax inside the pair of dollar signs ($) aims to be TeX-like,
   the text outside does not.  In particular, characters such as::

     # $ % & ~ _ ^ \ { } \( \) \[ \]

   have special meaning outside of math mode in TeX.  Therefore, these
   characters will behave differently depending on the rcParam ``text.usetex``
   flag.  See the :doc:`usetex tutorial </tutorials/text/usetex>` for more
   information.

Subscripts and superscripts
---------------------------

To make subscripts and superscripts, use the ``'_'`` and ``'^'`` symbols::

    r'$\alpha_i > \beta_i$'

\begin{align}\alpha_i > \beta_i\end{align}

Some symbols automatically put their sub/superscripts under and over the
operator.  For example, to write the sum of :mathmpl:`x_i` from :mathmpl:`0` to
:mathmpl:`\infty`, you could do::

    r'$\sum_{i=0}^\infty x_i$'

\begin{align}\sum_{i=0}^\infty x_i\end{align}

Fractions, binomials, and stacked numbers
-----------------------------------------

Fractions, binomials, and stacked numbers can be created with the
``\frac{}{}``, ``\binom{}{}`` and ``\genfrac{}{}{}{}{}{}`` commands,
respectively::

    r'$\frac{3}{4} \binom{3}{4} \genfrac{}{}{0}{}{3}{4}$'

produces

\begin{align}\frac{3}{4} \binom{3}{4} \stackrel{}{}{0}{}{3}{4}\end{align}

Fractions can be arbitrarily nested::

    r'$\frac{5 - \frac{1}{x}}{4}$'

produces

\begin{align}\frac{5 - \frac{1}{x}}{4}\end{align}

Note that special care needs to be taken to place parentheses and brackets
around fractions.  Doing things the obvious way produces brackets that are too
small::

    r'$(\frac{5 - \frac{1}{x}}{4})$'

.. math ::

    (\frac{5 - \frac{1}{x}}{4})

The solution is to precede the bracket with ``\left`` and ``\right`` to inform
the parser that those brackets encompass the entire object.::

    r'$\left(\frac{5 - \frac{1}{x}}{4}\right)$'

.. math ::

    \left(\frac{5 - \frac{1}{x}}{4}\right)

Radicals
--------

Radicals can be produced with the ``\sqrt[]{}`` command.  For example::

    r'$\sqrt{2}$'

.. math ::

    \sqrt{2}

Any base can (optionally) be provided inside square brackets.  Note that the
base must be a simple expression, and can not contain layout commands such as
fractions or sub/superscripts::

    r'$\sqrt[3]{x}$'

.. math ::

    \sqrt[3]{x}

Fonts
-----

The default font is *italics* for mathematical symbols.

Note

This default can be changed using the ``mathtext.default`` rcParam.  This is
   useful, for example, to use the same font as regular non-math text for math
   text, by setting it to ``regular``.

To change fonts, e.g., to write "sin" in a Roman font, enclose the text in a
font command::

    r'$s(t) = \mathcal{A}\mathrm{sin}(2 \omega t)$'

\begin{align}s(t) = \mathcal{A}\mathrm{sin}(2 \omega t)\end{align}

More conveniently, many commonly used function names that are typeset in
a Roman font have shortcuts.  So the expression above could be written as
follows::

    r'$s(t) = \mathcal{A}\sin(2 \omega t)$'

\begin{align}s(t) = \mathcal{A}\sin(2 \omega t)\end{align}

Here "s" and "t" are variable in italics font (default), "sin" is in Roman
font, and the amplitude "A" is in calligraphy font.  Note in the example above
the calligraphy ``A`` is squished into the ``sin``.  You can use a spacing
command to add a little whitespace between them::

    r's(t) = \mathcal{A}\/\sin(2 \omega t)'

.. Here we cheat a bit: for HTML math rendering, Sphinx relies on MathJax which
   doesn't actually support the italic correction (\/); instead, use a thin
   space (\,) which is supported.

\begin{align}s(t) = \mathcal{A}\,\sin(2 \omega t)\end{align}

The choices available with all fonts are:

    ========================= ================================
    Command                   Result
    ========================= ================================
    ``\mathrm{Roman}``        :mathmpl:`\mathrm{Roman}`
    ``\mathit{Italic}``       :mathmpl:`\mathit{Italic}`
    ``\mathtt{Typewriter}``   :mathmpl:`\mathtt{Typewriter}`
    ``\mathcal{CALLIGRAPHY}`` :mathmpl:`\mathcal{CALLIGRAPHY}`
    ========================= ================================

.. role:: math-stix(mathmpl)
   :fontset: stix

When using the `STIX <http://www.stixfonts.org/>`_ fonts, you also have the
choice of:

    ================================ =========================================
    Command                          Result
    ================================ =========================================
    ``\mathbb{blackboard}``          :math-stix:`\mathbb{blackboard}`
    ``\mathrm{\mathbb{blackboard}}`` :math-stix:`\mathrm{\mathbb{blackboard}}`
    ``\mathfrak{Fraktur}``           :math-stix:`\mathfrak{Fraktur}`
    ``\mathsf{sansserif}``           :math-stix:`\mathsf{sansserif}`
    ``\mathrm{\mathsf{sansserif}}``  :math-stix:`\mathrm{\mathsf{sansserif}}`
    ================================ =========================================

There are also three global "font sets" to choose from, which are
selected using the ``mathtext.fontset`` parameter in `matplotlibrc
<matplotlibrc-sample>`.

``cm``: **Computer Modern (TeX)**

``stix``: **STIX** (designed to blend well with Times)

``stixsans``: **STIX sans-serif**

Additionally, you can use ``\mathdefault{...}`` or its alias
``\mathregular{...}`` to use the font used for regular text outside of
mathtext.  There are a number of limitations to this approach, most notably
that far fewer symbols will be available, but it can be useful to make math
expressions blend well with other text in the plot.

Custom fonts
~~~~~~~~~~~~

mathtext also provides a way to use custom fonts for math.  This method is
fairly tricky to use, and should be considered an experimental feature for
patient users only.  By setting the rcParam ``mathtext.fontset`` to ``custom``,
you can then set the following parameters, which control which font file to use
for a particular set of math characters.

    ============================== =================================
    Parameter                      Corresponds to
    ============================== =================================
    ``mathtext.it``                ``\mathit{}`` or default italic
    ``mathtext.rm``                ``\mathrm{}`` Roman (upright)
    ``mathtext.tt``                ``\mathtt{}`` Typewriter (monospace)
    ``mathtext.bf``                ``\mathbf{}`` bold italic
    ``mathtext.cal``               ``\mathcal{}`` calligraphic
    ``mathtext.sf``                ``\mathsf{}`` sans-serif
    ============================== =================================

Each parameter should be set to a fontconfig font descriptor (as defined in the
yet-to-be-written font chapter).

.. TODO: Link to font chapter

The fonts used should have a Unicode mapping in order to find any
non-Latin characters, such as Greek.  If you want to use a math symbol
that is not contained in your custom fonts, you can set the rcParam
``mathtext.fallback_to_cm`` to ``True`` which will cause the mathtext system
to use characters from the default Computer Modern fonts whenever a particular
character can not be found in the custom font.

Note that the math glyphs specified in Unicode have evolved over time, and many
fonts may not have glyphs in the correct place for mathtext.

Accents
-------

An accent command may precede any symbol to add an accent above it.  There are
long and short forms for some of them.

    ============================== =================================
    Command                        Result
    ============================== =================================
    ``\acute a`` or ``\'a``        :mathmpl:`\acute a`
    ``\bar a``                     :mathmpl:`\bar a`
    ``\breve a``                   :mathmpl:`\breve a`
    ``\ddot a`` or ``\''a``        :mathmpl:`\ddot a`
    ``\dot a`` or ``\.a``          :mathmpl:`\dot a`
    ``\grave a`` or ``\`a``        :mathmpl:`\grave a`
    ``\hat a`` or ``\^a``          :mathmpl:`\hat a`
    ``\tilde a`` or ``\~a``        :mathmpl:`\tilde a`
    ``\vec a``                     :mathmpl:`\vec a`
    ``\overline{abc}``             :mathmpl:`\overline{abc}`
    ============================== =================================

In addition, there are two special accents that automatically adjust to the
width of the symbols below:

    ============================== =================================
    Command                        Result
    ============================== =================================
    ``\widehat{xyz}``              :mathmpl:`\widehat{xyz}`
    ``\widetilde{xyz}``            :mathmpl:`\widetilde{xyz}`
    ============================== =================================

Care should be taken when putting accents on lower-case i's and j's.  Note that
in the following ``\imath`` is used to avoid the extra dot over the i::

    r"$\hat i\ \ \hat \imath$"

\begin{align}\hat i\ \ \hat \imath\end{align}

Symbols
-------

You can also use a large number of the TeX symbols, as in ``\infty``,
``\leftarrow``, ``\sum``, ``\int``.

.. math_symbol_table::

If a particular symbol does not have a name (as is true of many of the more
obscure symbols in the STIX fonts), Unicode characters can also be used::

   ur'$\u23ce$'

Example
-------

Here is an example illustrating many of these features in context.

.. figure:: ../../gallery/pyplots/images/sphx_glr_pyplot_mathtext_001.png
   :target: ../../gallery/pyplots/pyplot_mathtext.html
   :align: center
   :scale: 50

   Pyplot Mathtext