# Tutorial 1: Neural Rate Models
**Week 2, Day 4: Dynamic Networks**

**By Neuromatch Academy**

__Content creators:__ Qinglong Gu, Songtin Li, Arvind Kumar, John Murray, Julijana Gjorgjieva

__Content reviewers:__  Maryam Vaziri-Pashkam, Ella Batty, Lorenzo Fontolan, Richard Gao, Spiros Chavlis, Michael Waskom, Siddharth Suresh

**Our 2021 Sponsors, including Presenting Sponsor Facebook Reality Labs**

---
# Tutorial Objectives

*Estimated timing of tutorial: 1 hour, 25 minutes*

The brain is a complex system, not because it is composed of a large number of diverse types of neurons, but mainly because of how neurons are connected to each other. The brain is indeed a network of highly specialized neuronal networks.

The activity of a neural network constantly evolves in time. For this reason, neurons can be modeled as dynamical systems. The dynamical system approach is only one of the many modeling approaches that computational neuroscientists have developed (other points of view include information processing,  statistical models, etc.).

How the dynamics of neuronal networks affect the representation and processing of information in the brain is an open question. However, signatures of altered brain dynamics present in many brain diseases (e.g., in epilepsy or Parkinson's disease) tell us that it is crucial to study network activity dynamics if we want to understand the brain.

In this tutorial, we will simulate and study one of the simplest models of biological neuronal networks. Instead of modeling and simulating individual excitatory neurons (e.g., LIF models that you implemented yesterday), we will treat them as a single homogeneous population and approximate their dynamics using a single one-dimensional equation describing the evolution of their average spiking rate in time.

In this tutorial, we will learn how to build a firing rate model of a single population of excitatory neurons.

**Steps:**
- Write the equation for the firing rate dynamics of a 1D excitatory population.
- Visualize the response of the population as a function of parameters such as threshold level and gain, using the frequency-current (F-I) curve.
- Numerically simulate the dynamics of the excitatory population and find the fixed points of the system.

```python
# @title Tutorial slides

# @markdown These are the slides for the videos in all tutorials today
from IPython.display import IFrame
IFrame(src=f"https://mfr.ca-1.osf.io/render?url=https://osf.io/nvuty/?direct%26mode=render%26action=download%26mode=render", width=854, height=480)
```

---
# Setup

```python
# Imports

import matplotlib.pyplot as plt
import numpy as np
import scipy.optimize as opt       # root-finding algorithm
```

```python
# @title Figure Settings
import ipywidgets as widgets       # interactive display
%config InlineBackend.figure_format = 'retina'
plt.style.use("https://raw.githubusercontent.com/NeuromatchAcademy/course-content/master/nma.mplstyle")
```

```python
# @title Plotting Functions

def plot_fI(x, f):
  plt.figure(figsize=(6, 4))  # plot the figure
  plt.plot(x, f, 'k')
  plt.xlabel('x (a.u.)', fontsize=14)
  plt.ylabel('F(x)', fontsize=14)
  plt.show()

def plot_dr_r(r, drdt, x_fps=None):
  plt.figure()
  plt.plot(r, drdt, 'k')
  plt.plot(r, 0. * r, 'k--')
  if x_fps is not None:
    plt.plot(x_fps, np.zeros_like(x_fps), "ko", ms=12)
  plt.xlabel(r'$r$')
  plt.ylabel(r'$\frac{dr}{dt}$', fontsize=20)
  plt.ylim(-0.1, 0.1)

def plot_dFdt(x, dFdt):
  plt.figure()
  plt.plot(x, dFdt, 'r')
  plt.xlabel('x (a.u.)', fontsize=14)
  plt.ylabel('dF(x)', fontsize=14)
  plt.show()
```

---
# Section 1: Neuronal network dynamics

```python
# @title Video 1: Dynamic networks
from ipywidgets import widgets

out2 = widgets.Output()
with out2:
  from IPython.display import IFrame
  class BiliVideo(IFrame):
      def __init__(self, id, page=1, width=400, height=300, **kwargs):
          self.id=id
          src = 'https://player.bilibili.com/player.html?bvid={0}&page={1}'.format(id, page)
          super(BiliVideo, self).__init__(src, width, height, **kwargs)

  video = BiliVideo(id="BV1dh411o7qJ", width=854, height=480, fs=1)
  print('Video available at https://www.bilibili.com/video/{0}'.format(video.id))
  display(video)

out1 = widgets.Output()
with out1:
  from IPython.display import YouTubeVideo
  video = YouTubeVideo(id="p848349hPyw", width=854, height=480, fs=1, rel=0)
  print('Video available at https://youtube.com/watch?v=' + video.id)
  display(video)

out = widgets.Tab([out1, out2])
out.set_title(0, 'Youtube')
out.set_title(1, 'Bilibili')

display(out)
```

    Tab(children=(Output(), Output()), _titles={'0': 'Youtube', '1': 'Bilibili'})

This video covers how to model a network with a single population of neurons and introduces neural rate-based models. It overviews feedforward networks and defines the F-I (firing rate vs. input) curve.

## Section 1.1: Dynamics of a single excitatory population

 Click here for text recap of relevant part of video

Individual neurons respond by spiking. When we average the spikes of neurons in a population, we can define the average firing activity of the population. In this model, we are interested in how the population-averaged firing varies as a function of time and network parameters. Mathematically, we can describe the firing rate dynamic of a feed-forward network as:

\begin{align}
\tau \frac{dr}{dt} &= -r + F(I_{\text{ext}})  \quad\qquad (1)
\end{align}

$r(t)$ represents the average firing rate of the excitatory population at time $t$, $\tau$ controls the timescale of the evolution of the average firing rate, $I_{\text{ext}}$ represents the external input, and the transfer function $F(\cdot)$ (which can be related to f-I curve of individual neurons described in the next sections) represents the population activation function in response to all received inputs.

To start building the model, please execute the cell below to initialize the simulation parameters.

```python
# @markdown *Execute this cell to set default parameters for a single excitatory population model*

def default_pars_single(**kwargs):
  pars = {}

  # Excitatory parameters
  pars['tau'] = 1.     # Timescale of the E population [ms]
  pars['a'] = 1.2      # Gain of the E population
  pars['theta'] = 2.8  # Threshold of the E population

  # Connection strength
  pars['w'] = 0.  # E to E, we first set it to 0

  # External input
  pars['I_ext'] = 0.

  # simulation parameters
  pars['T'] = 20.       # Total duration of simulation [ms]
  pars['dt'] = .1       # Simulation time step [ms]
  pars['r_init'] = 0.2  # Initial value of E

  # External parameters if any
  pars.update(kwargs)

  # Vector of discretized time points [ms]
  pars['range_t'] = np.arange(0, pars['T'], pars['dt'])

  return pars

pars = default_pars_single()
print(pars)
```

    {'tau': 1.0, 'a': 1.2, 'theta': 2.8, 'w': 0.0, 'I_ext': 0.0, 'T': 20.0, 'dt': 0.1, 'r_init': 0.2, 'range_t': array([ 0. ,  0.1,  0.2,  0.3,  0.4,  0.5,  0.6,  0.7,  0.8,  0.9,  1. ,
            1.1,  1.2,  1.3,  1.4,  1.5,  1.6,  1.7,  1.8,  1.9,  2. ,  2.1,
            2.2,  2.3,  2.4,  2.5,  2.6,  2.7,  2.8,  2.9,  3. ,  3.1,  3.2,
            3.3,  3.4,  3.5,  3.6,  3.7,  3.8,  3.9,  4. ,  4.1,  4.2,  4.3,
            4.4,  4.5,  4.6,  4.7,  4.8,  4.9,  5. ,  5.1,  5.2,  5.3,  5.4,
            5.5,  5.6,  5.7,  5.8,  5.9,  6. ,  6.1,  6.2,  6.3,  6.4,  6.5,
            6.6,  6.7,  6.8,  6.9,  7. ,  7.1,  7.2,  7.3,  7.4,  7.5,  7.6,
            7.7,  7.8,  7.9,  8. ,  8.1,  8.2,  8.3,  8.4,  8.5,  8.6,  8.7,
            8.8,  8.9,  9. ,  9.1,  9.2,  9.3,  9.4,  9.5,  9.6,  9.7,  9.8,
            9.9, 10. , 10.1, 10.2, 10.3, 10.4, 10.5, 10.6, 10.7, 10.8, 10.9,
           11. , 11.1, 11.2, 11.3, 11.4, 11.5, 11.6, 11.7, 11.8, 11.9, 12. ,
           12.1, 12.2, 12.3, 12.4, 12.5, 12.6, 12.7, 12.8, 12.9, 13. , 13.1,
           13.2, 13.3, 13.4, 13.5, 13.6, 13.7, 13.8, 13.9, 14. , 14.1, 14.2,
           14.3, 14.4, 14.5, 14.6, 14.7, 14.8, 14.9, 15. , 15.1, 15.2, 15.3,
           15.4, 15.5, 15.6, 15.7, 15.8, 15.9, 16. , 16.1, 16.2, 16.3, 16.4,
           16.5, 16.6, 16.7, 16.8, 16.9, 17. , 17.1, 17.2, 17.3, 17.4, 17.5,
           17.6, 17.7, 17.8, 17.9, 18. , 18.1, 18.2, 18.3, 18.4, 18.5, 18.6,
           18.7, 18.8, 18.9, 19. , 19.1, 19.2, 19.3, 19.4, 19.5, 19.6, 19.7,
           19.8, 19.9])}

You can now use:
- `pars = default_pars_single()` to get all the parameters.
- `pars = default_pars_single(T=T_sim, dt=time_step)` to set new simulation time and time step
- To update an existing parameter dictionary, use `pars['New_para'] = value`

Because `pars` is a dictionary, it can be passed to a function that requires individual parameters as arguments using `my_func(**pars)` syntax.

## Section 1.2: F-I curves

*Estimated timing to here from start of tutorial: 17 min*

 Click here for text recap of relevant part of video

In electrophysiology, a neuron is often characterized by its spike rate output in response to input currents. This is often called the **F-I** curve, denoting the output spike frequency (**F**) in response to different injected currents (**I**). We estimated this for an LIF neuron in yesterday's tutorial.

The transfer function $F(\cdot)$ in Equation $1$ represents the gain of the population as a function of the total input. The gain is often modeled as a sigmoidal function, i.e., more input drive leads to a nonlinear increase in the population firing rate. The output firing rate will eventually saturate for high input values.

A sigmoidal $F(\cdot)$ is parameterized by its gain $a$ and threshold $\theta$.

$$ F(x;a,\theta) = \frac{1}{1+\text{e}^{-a(x-\theta)}} - \frac{1}{1+\text{e}^{a\theta}}  \quad(2)$$

The argument $x$ represents the input to the population. Note that the second term is chosen so that $F(0;a,\theta)=0$.

Many other transfer functions (generally monotonic) can be also used. Examples are the rectified linear function $ReLU(x)$ or the hyperbolic tangent $tanh(x)$.

### Coding Exercise 1.2: Implement F-I curve

Let's first investigate the activation functions before simulating the dynamics of the entire population.

In this exercise, you will implement a sigmoidal **F-I** curve or transfer function $F(x)$, with gain $a$ and threshold level $\theta$ as parameters:

$$ F(x;a,\theta) = \frac{1}{1+\text{e}^{-a(x-\theta)}} - \frac{1}{1+\text{e}^{a\theta}}$$

```python
def F(x, a, theta):
  """
  Population activation function.

  Args:
    x (float): the population input
    a (float): the gain of the function
    theta (float): the threshold of the function

  Returns:
    float: the population activation response F(x) for input x
  """
  #################################################
  ## TODO for students: compute f = F(x) ##
  # Fill out function and remove
  #raise NotImplementedError("Student excercise: implement the f-I function")
  #################################################

  # Define the sigmoidal transfer function f = F(x)
  sigmoid = lambda x: 1 / (1 + np.exp( -a * (x - theta) ) )
  return sigmoid(x) - sigmoid(0)

# Set parameters
pars = default_pars_single()  # get default parameters
x = np.arange(0, 10, .1)      # set the range of input

# Compute transfer function
f = F(x, pars['a'], pars['theta'])

# Visualize
plot_fI(x, f)
```

```python
# to_remove solution

def F(x, a, theta):
  """
  Population activation function.

  Args:
    x (float): the population input
    a (float): the gain of the function
    theta (float): the threshold of the function

  Returns:
    float: the population activation response F(x) for input x
  """

  # Define the sigmoidal transfer function f = F(x)
  f = (1 + np.exp(-a * (x - theta)))**-1 - (1 + np.exp(a * theta))**-1

  return f

# Set parameters
pars = default_pars_single()  # get default parameters
x = np.arange(0, 10, .1)      # set the range of input

# Compute transfer function
f = F(x, pars['a'], pars['theta'])

# Visualize
with plt.xkcd():
  plot_fI(x, f)
```

### Interactive Demo 1.2 : Parameter exploration of F-I curve
Here's an interactive demo that shows how the F-I curve changes for different values of the gain and threshold parameters.

1. How does the gain parameter ($a$) affect the F-I curve?
1. How does the threshold parameter ($\theta$) affect the F-I curve?

```python
# @title

# @markdown Make sure you execute this cell to enable the widget!

def interactive_plot_FI(a, theta):
  """
  Population activation function.

  Expecxts:
    a     : the gain of the function
    theta : the threshold of the function

  Returns:
    plot the F-I curve with give parameters
  """

  # set the range of input
  x = np.arange(0, 10, .1)
  plt.figure()
  plt.plot(x, F(x, a, theta), 'k')
  plt.xlabel('x (a.u.)', fontsize=14)
  plt.ylabel('F(x)', fontsize=14)
  plt.show()

_ = widgets.interact(interactive_plot_FI, a=(0.3, 3, 0.3), theta=(2, 4, 0.2))
```

    interactive(children=(FloatSlider(value=1.5, description='a', max=3.0, min=0.3, step=0.3), FloatSlider(value=3…

```python
# to_remove explanation

"""

1)  a determines the slope (gain) of the rising phase of the F-I curve

2) theta determines the input at which the function F(x) reaches its mid-value (0.5).
That is, theta shifts the F-I curve along the horizontal axis.

For our neurons we are using in this tutorial:
- a controls the gain of the neuron population
- theta controls the threshold at which the neuron population starts to respond
""";
```

## Section 1.3: Simulation scheme of E dynamics

*Estimated timing to here from start of tutorial: 27 min*

Because $F(\cdot)$ is a nonlinear function, the exact solution of our differential equation of population activity can not be determined via analytical methods. As we have seen before, we can use numerical methods, specifically the Euler method, to find the solution (that is, simulate the population activity).

```python
# @markdown *Execute this cell to enable the single population rate model simulator: `simulate_single`*

def simulate_single(pars):
  """
  Simulate an excitatory population of neurons

  Args:
    pars   : Parameter dictionary

  Returns:
    rE     : Activity of excitatory population (array)

  Example:
    pars = default_pars_single()
    r = simulate_single(pars)
  """

  # Set parameters
  tau, a, theta = pars['tau'], pars['a'], pars['theta']
  w = pars['w']
  I_ext = pars['I_ext']
  r_init = pars['r_init']
  dt, range_t = pars['dt'], pars['range_t']
  Lt = range_t.size

  # Initialize activity
  r = np.zeros(Lt)
  r[0] = r_init
  I_ext = I_ext * np.ones(Lt)

  # Update the E activity
  for k in range(Lt - 1):
      dr = dt / tau * (-r[k] + F(w * r[k] + I_ext[k], a, theta))
      r[k+1] = r[k] + dr

  return r

help(simulate_single)
```

    Help on function simulate_single in module __main__:

    simulate_single(pars)
        Simulate an excitatory population of neurons

        Args:
          pars   : Parameter dictionary

        Returns:
          rE     : Activity of excitatory population (array)

        Example:
          pars = default_pars_single()
          r = simulate_single(pars)

### Interactive Demo 1.3: Parameter Exploration of single population dynamics

Explore these dynamics of the population activity in this interactive demo.

1.  How does $r_{\text{sim}}(t)$ change with different $I_{\text{ext}}$ values?
2.  How does it change with different $\tau$ values?

Note that, $r_{\rm ana}(t)$ denotes the analytical solution - you will learn how this is computed in the next section.

```python
# @title

# @markdown Make sure you execute this cell to enable the widget!

# get default parameters
pars = default_pars_single(T=20.)

def Myplot_E_diffI_difftau(I_ext, tau):
  # set external input and time constant
  pars['I_ext'] = I_ext
  pars['tau'] = tau

  # simulation
  r = simulate_single(pars)

  # Analytical Solution
  r_ana = (pars['r_init']
           + (F(I_ext, pars['a'], pars['theta'])
           - pars['r_init']) * (1. - np.exp(-pars['range_t'] / pars['tau'])))

  # plot
  plt.figure()
  plt.plot(pars['range_t'], r, 'b', label=r'$r_{\mathrm{sim}}$(t)', alpha=0.5,
           zorder=1)
  plt.plot(pars['range_t'], r_ana, 'b--', lw=5, dashes=(2, 2),
           label=r'$r_{\mathrm{ana}}$(t)', zorder=2)
  plt.plot(pars['range_t'],
           F(I_ext, pars['a'], pars['theta']) * np.ones(pars['range_t'].size),
           'k--', label=r'$F(I_{\mathrm{ext}})$')
  plt.xlabel('t (ms)', fontsize=16.)
  plt.ylabel('Activity r(t)', fontsize=16.)
  plt.legend(loc='best', fontsize=14.)
  plt.show()

_ = widgets.interact(Myplot_E_diffI_difftau, I_ext=(0.0, 10., 1.),
                     tau=(1., 5., 0.2))
```

    interactive(children=(FloatSlider(value=5.0, description='I_ext', max=10.0, step=1.0), FloatSlider(value=3.0, …

```python
# to_remove explanation

"""

1) Weak inputs to the neurons eventually result in the activity converging to zero.
 Strong inputs to the neurons eventually result in the activity converging to max value

2) The time constant tau, does not affect the final response reached but it determines
the time the neurons take to reach to their fixed point.
""";
```

## Think! 1.3: Finite activities

Above, we have numerically solved a system driven by a positive input. Yet, $r_E(t)$ either decays to zero or reaches a fixed non-zero value.

1. Why doesn't the solution of the system "explode" in a finite time? In other words, what guarantees that $r_E$(t) stays finite?
2. Which parameter would you change in order to increase the maximum value of the response?

```python
# to_remove explanation

"""

1) As the F-I curve is bounded between zero and one, the system doesn't explode.
The f-curve guarantees this property

2) One way to increase the maximum response is to change the f-I curve. For
example, the ReLU is an unbounded function, and thus will increase the overall maximal
response of the network.

""";
```

---
# Section 2: Fixed points of the single population system

*Estimated timing to here from start of tutorial: 45 min*

## Section 2.1: Finding fixed points

```python
# @title Video 2: Fixed point
from ipywidgets import widgets

out2 = widgets.Output()
with out2:
  from IPython.display import IFrame
  class BiliVideo(IFrame):
      def __init__(self, id, page=1, width=400, height=300, **kwargs):
          self.id=id
          src = 'https://player.bilibili.com/player.html?bvid={0}&page={1}'.format(id, page)
          super(BiliVideo, self).__init__(src, width, height, **kwargs)

  video = BiliVideo(id="BV1v54y1v7Gr", width=854, height=480, fs=1)
  print('Video available at https://www.bilibili.com/video/{0}'.format(video.id))
  display(video)

out1 = widgets.Output()
with out1:
  from IPython.display import YouTubeVideo
  video = YouTubeVideo(id="Ox3ELd1UFyo", width=854, height=480, fs=1, rel=0)
  print('Video available at https://youtube.com/watch?v=' + video.id)
  display(video)

out = widgets.Tab([out1, out2])
out.set_title(0, 'Youtube')
out.set_title(1, 'Bilibili')

display(out)
```

    Tab(children=(Output(), Output()), _titles={'0': 'Youtube', '1': 'Bilibili'})

This video introduces recurrent networks and how to derive their fixed points. It also introduces vector fields and phase planes in one dimension.

 Click here for text recap of video

We can now extend our feed-forward network to a recurrent network, governed by the equation:

\begin{align}
\tau \frac{dr}{dt} &= -r + F(w\cdot r + I_{\text{ext}})  \quad\qquad (3)
\end{align}
 where as before, $r(t)$ represents the average firing rate of the excitatory population at time $t$, $\tau$ controls the timescale of the evolution of the average firing rate, $I_{\text{ext}}$ represents the external input, and the transfer function $F(\cdot)$ (which can be related to f-I curve of individual neurons described in the next sections) represents the population activation function in response to all received inputs. Now we also have $w$ which denotes the strength (synaptic weight) of the recurrent input to the population.

As you varied the two parameters in the last Interactive Demo, you noticed that, while at first the system output quickly changes, with time, it reaches its maximum/minimum value and does not change anymore. The value eventually reached by the system is called the **steady state** of the system, or the **fixed point**. Essentially, in the steady states the derivative with respect to time of the activity ($r$) is zero, i.e. $\displaystyle \frac{dr}{dt}=0$.

We can find that the steady state of the Equation. (1) by setting $\displaystyle{\frac{dr}{dt}=0}$ and solve for $r$:

$$-r_{\text{steady}} + F(w\cdot r_{\text{steady}} + I_{\text{ext}};a,\theta) = 0, \qquad (4)$$

When it exists, the solution of Equation. (4) defines a **fixed point** of the dynamical system in Equation (3). Note that if $F(x)$ is nonlinear, it is not always possible to find an analytical solution, but the solution can be found via numerical simulations, as we will do later.

From the Interactive Demo, one could also notice that the value of $\tau$ influences how quickly the activity will converge to the steady state from its initial value.

In the specific case of $w=0$, we can also analytically compute  the solution of Equation (1) (i.e., the thick blue dashed line) and deduce the role of $\tau$ in determining the convergence to the fixed point:

$$\displaystyle{r(t) = \big{[}F(I_{\text{ext}};a,\theta) -r(t=0)\big{]} (1-\text{e}^{-\frac{t}{\tau}})} + r(t=0)$$ \\

We can now numerically calculate the fixed point with a root finding algorithm.

### Coding Exercise 2.1.1: Visualization of the fixed points

When it is not possible to find the solution for Equation (3) analytically, a graphical approach can be taken. To that end, it is useful to plot $\displaystyle{\frac{dr}{dt}}$ as a function of $r$. The values of $r$ for which the plotted function crosses zero on the y axis correspond to fixed points.

Here, let us, for example, set $w=5.0$ and $I^{\text{ext}}=0.5$. From Equation (3), you can obtain

$$\frac{dr}{dt} = [-r + F(w\cdot r + I^{\text{ext}})]\,/\,\tau $$

Then, plot the $dr/dt$ as a function of $r$, and check for the presence of fixed points.

```python
def compute_drdt(r, I_ext, w, a, theta, tau, **other_pars):
  """Given parameters, compute dr/dt as a function of r.

  Args:
    r (1D array) : Average firing rate of the excitatory population
    I_ext, w, a, theta, tau (numbers): Simulation parameters to use
    other_pars : Other simulation parameters are unused by this function

  Returns
    drdt function for each value of r
  """
  #########################################################################
  # TODO compute drdt and disable the error
  #raise NotImplementedError("Finish the compute_drdt function")
  #########################################################################

  # Calculate drdt
  drdt = ( -r + F(w*r + I_ext, a, theta) ) / tau

  return drdt

# Define a vector of r values and the simulation parameters
r = np.linspace(0, 1, 1000)
pars = default_pars_single(I_ext=0.5, w=5)

# Compute dr/dt
drdt = compute_drdt(r, **pars)

# Visualize
plot_dr_r(r, drdt)
```

```python
# to_remove solution

def compute_drdt(r, I_ext, w, a, theta, tau, **other_pars):
  """Given parameters, compute dr/dt as a function of r.

  Args:
    r (1D array) : Average firing rate of the excitatory population
    I_ext, w, a, theta, tau (numbers): Simulation parameters to use
    other_pars : Other simulation parameters are unused by this function

  Returns
    drdt function for each value of r
  """
  # Calculate drdt
  drdt = (-r + F(w * r + I_ext, a, theta)) / tau

  return drdt

# Define a vector of r values and the simulation parameters
r = np.linspace(0, 1, 1000)
pars = default_pars_single(I_ext=0.5, w=5)

# Compute dr/dt
drdt = compute_drdt(r, **pars)

# Visualize
with plt.xkcd():
  plot_dr_r(r, drdt)
```

### Coding Exercise 2.1.2: Numerical calculution of fixed points

We will now find the fixed points numerically. To do so, we need to specify initial values ($r_{\text{guess}}$) for the root-finding algorithm to start from. From the line $\displaystyle{\frac{dr}{dt}}$ plotted above in the last exercise, initial values can be chosen as a set of values close to where the line crosses zero on the y axis (real fixed point).

The next cell defines three helper functions that we will use:

- `my_fp_single(r_guess, **pars)` uses a root-finding algorithm to locate a fixed point near a given initial value
- `check_fp_single(x_fp, **pars)` verifies that the values of $r_{\rm fp}$ for which $\displaystyle{\frac{dr}{dt}} = 0$ are the true fixed points
- `my_fp_finder(r_guess_vector, **pars)` accepts an array of initial values and finds the same number of fixed points, using the above two functions

```python
# @markdown *Execute this cell to enable the fixed point functions*

def my_fp_single(r_guess, a, theta, w, I_ext, **other_pars):
  """
  Calculate the fixed point through drE/dt=0

  Args:
    r_guess  : Initial value used for scipy.optimize function
    a, theta, w, I_ext : simulation parameters

  Returns:
    x_fp    : value of fixed point
  """
  # define the right hand of E dynamics
  def my_WCr(x):
    r = x
    drdt = (-r + F(w * r + I_ext, a, theta))
    y = np.array(drdt)

    return y

  x0 = np.array(r_guess)
  x_fp = opt.root(my_WCr, x0).x.item()

  return x_fp

def check_fp_single(x_fp, a, theta, w, I_ext, mytol=1e-4, **other_pars):
  """
   Verify |dr/dt| < mytol

  Args:
    fp      : value of fixed point
    a, theta, w, I_ext: simulation parameters
    mytol   : tolerance, default as 10^{-4}

  Returns :
    Whether it is a correct fixed point: True/False
  """
  # calculate Equation(3)
  y = x_fp - F(w * x_fp + I_ext, a, theta)

  # Here we set tolerance as 10^{-4}
  return np.abs(y) < mytol

def my_fp_finder(pars, r_guess_vector, mytol=1e-4):
  """
  Calculate the fixed point(s) through drE/dt=0

  Args:
    pars    : Parameter dictionary
    r_guess_vector  : Initial values used for scipy.optimize function
    mytol   : tolerance for checking fixed point, default as 10^{-4}

  Returns:
    x_fps   : values of fixed points

  """
  x_fps = []
  correct_fps = []
  for r_guess in r_guess_vector:
    x_fp = my_fp_single(r_guess, **pars)
    if check_fp_single(x_fp, **pars, mytol=mytol):
      x_fps.append(x_fp)

  return x_fps

help(my_fp_finder)
```

    Help on function my_fp_finder in module __main__:

    my_fp_finder(pars, r_guess_vector, mytol=0.0001)
        Calculate the fixed point(s) through drE/dt=0

        Args:
          pars    : Parameter dictionary
          r_guess_vector  : Initial values used for scipy.optimize function
          mytol   : tolerance for checking fixed point, default as 10^{-4}

        Returns:
          x_fps   : values of fixed points

```python
# Set parameters
r = np.linspace(0, 1, 1000)
pars = default_pars_single(I_ext=0.5, w=5)

# Compute dr/dt
drdt = compute_drdt(r, **pars)

#############################################################################
# TODO for students:
# Define initial values close to the intersections of drdt and y=0
# (How many initial values? Hint: How many times do the two lines intersect?)
# Calculate the fixed point with these initial values and plot them
#raise NotImplementedError('student_exercise: find fixed points numerically')
#############################################################################

# Initial guesses for fixed points
r_guess_vector = [.0, .5, 1.]

# Find fixed point numerically
x_fps = my_fp_finder(pars, r_guess_vector)

# Visualize
plot_dr_r(r, drdt, x_fps)
```

```python
# to_remove solution

# Set parameters
r = np.linspace(0, 1, 1000)
pars = default_pars_single(I_ext=0.5, w=5)

# Compute dr/dt
drdt = compute_drdt(r, **pars)

# Initial guesses for fixed points
r_guess_vector = [0, .4, .9]

# Find fixed point numerically
x_fps = my_fp_finder(pars, r_guess_vector)

# Visualize
with plt.xkcd():
  plot_dr_r(r, drdt, x_fps)
```

### Interactive Demo 2.1: fixed points as a function of recurrent and external inputs.

You can now explore how the previous plot changes when the recurrent coupling $w$ and the external input $I_{\text{ext}}$ take different values. How does the number of fixed points change?

```python
# @title

# @markdown Make sure you execute this cell to enable the widget!

def plot_intersection_single(w, I_ext):
  # set your parameters
  pars = default_pars_single(w=w, I_ext=I_ext)

  # find fixed points
  r_init_vector = [0, .4, .9]
  x_fps = my_fp_finder(pars, r_init_vector)

  # plot
  r = np.linspace(0, 1., 1000)
  drdt = (-r + F(w * r + I_ext, pars['a'], pars['theta'])) / pars['tau']

  plot_dr_r(r, drdt, x_fps)

_ = widgets.interact(plot_intersection_single, w=(1, 7, 0.2),
                     I_ext=(0, 3, 0.1))
```

    interactive(children=(FloatSlider(value=4.0, description='w', max=7.0, min=1.0, step=0.2), FloatSlider(value=1…

```python
# to_remove explanation

"""

The fixed points of the single excitatory neuron population are determined by both
recurrent connections w and external input I_ext. In a previous interactive demo
we saw how the system showed two different steady-states when w = 0. But when w
doe not equal 0, for some range of w the system shows three fixed points and the
steady state depends on the initial conditions (i.e.
r at time zero.).

""";
```

## Section 2.2: Relationship between trajectories & fixed points

Let's examine the relationship between the population activity over time and the fixed points.

Here, let us first set $w=5.0$ and $I_{\text{ext}}=0.5$, and investigate the dynamics of $r(t)$ starting with different initial values $r(0) \equiv r_{\text{init}}$.

```python
# @markdown Execute to visualize dr/dt

def plot_intersection_single(w, I_ext):
  # set your parameters
  pars = default_pars_single(w=w, I_ext=I_ext)

  # find fixed points
  r_init_vector = [0, .4, .9]
  x_fps = my_fp_finder(pars, r_init_vector)

  # plot
  r = np.linspace(0, 1., 1000)
  drdt = (-r + F(w * r + I_ext, pars['a'], pars['theta'])) / pars['tau']

  plot_dr_r(r, drdt, x_fps)

plot_intersection_single(w = 5.0, I_ext = 0.5)
```

### Interactive Demo 2.2: dynamics as a function of the initial value

Let's now set $r_{\rm init}$ to a value of your choice in this demo. How does the solution change? What do you observe? How does that relate to the previous plot of $\frac{dr}{dt}$?

```python
# @markdown Make sure you execute this cell to enable the widget!

pars = default_pars_single(w=5.0, I_ext=0.5)

def plot_single_diffEinit(r_init):
  pars['r_init'] = r_init
  r = simulate_single(pars)

  plt.figure()
  plt.plot(pars['range_t'], r, 'b', zorder=1)
  plt.plot(0, r[0], 'bo', alpha=0.7, zorder=2)
  plt.xlabel('t (ms)', fontsize=16)
  plt.ylabel(r'$r(t)$', fontsize=16)
  plt.ylim(0, 1.0)
  plt.show()

_ = widgets.interact(plot_single_diffEinit, r_init=(0, 1, 0.02))
```

    interactive(children=(FloatSlider(value=0.0, description='r_init', max=1.0, step=0.02), Output()), _dom_classe…

```python
# to_remove explanation
"""

There are three fixed points of the system for these values of
w and I_ext. Choose the initial value in this demo and see in which direction
the system output moves. When r_init is in the vicinity of the leftmost fixed points
it moves towards the left most fixed point. When r_init is in the vicinity of the
rightmost fixed points it moves towards the rightmost fixed point.
""";
```

We will plot the trajectories of $r(t)$ with $r_{\text{init}} = 0.0, 0.1, 0.2,..., 0.9$.

```python
# @markdown Execute this cell to see the trajectories!

pars = default_pars_single()
pars['w'] = 5.0
pars['I_ext'] = 0.5

plt.figure(figsize=(8, 5))
for ie in range(10):
  pars['r_init'] = 0.1 * ie  # set the initial value
  r = simulate_single(pars)  # run the simulation

  # plot the activity with given initial
  plt.plot(pars['range_t'], r, 'b', alpha=0.1 + 0.1 * ie,
           label=r'r$_{\mathrm{init}}$=%.1f' % (0.1 * ie))

plt.xlabel('t (ms)')
plt.title('Two steady states?')
plt.ylabel(r'$r$(t)')
plt.legend(loc=[1.01, -0.06], fontsize=14)
plt.show()
```

We have three fixed points but only two steady states showing up - what's happening?

It turns out that the stability of the fixed points matters. If a fixed point is stable, a trajectory starting near that fixed point will stay close to that fixed point and converge to it (the steady state will equal the fixed point). If a fixed point is unstable, any trajectories starting close to it will diverge and go towards stable fixed points. In fact, the only way for a trajectory to reach a stable state at an unstable fixed point is if the initial value **exactly** equals the value of the fixed point.

### Think! 2.2: Stable vs unstable fixed points

Which of the fixed points for the model we've been examining in this section are stable vs unstable?

```python
# @markdown Execute to print fixed point values

# Initial guesses for fixed points
r_guess_vector = [0, .4, .9]

# Find fixed point numerically
x_fps = my_fp_finder(pars, r_guess_vector)

print(f'Our fixed points are {x_fps}')
```

    Our fixed points are [0.04153669901331739, 0.4471192240898344, 0.8997171538560865]

```python
# to_remove explanation

"""

The trajectory is converging to either the first fixed point or the third so those
ones are stable. The second fixed point is unstable as we do not see trajectories converging
to it.

"""
```

We can simulate the trajectory if we start at the unstable fixed point: you can see that it remains at that fixed point (the red line below).

```python
# @markdown Execute to visualize trajectory starting at unstable fixed point
pars = default_pars_single()
pars['w'] = 5.0
pars['I_ext'] = 0.5

plt.figure(figsize=(8, 5))
for ie in range(10):
  pars['r_init'] = 0.1 * ie  # set the initial value
  r = simulate_single(pars)  # run the simulation

  # plot the activity with given initial
  plt.plot(pars['range_t'], r, 'b', alpha=0.1 + 0.1 * ie,
           label=r'r$_{\mathrm{init}}$=%.1f' % (0.1 * ie))

pars['r_init'] = x_fps[1]  # set the initial value
r = simulate_single(pars)  # run the simulation

# plot the activity with given initial
plt.plot(pars['range_t'], r, 'r', alpha=0.1 + 0.1 * ie,
          label=r'r$_{\mathrm{init}}$=%.4f' % (x_fps[1]))

plt.xlabel('t (ms)')
plt.title('Two steady states?')
plt.ylabel(r'$r$(t)')
plt.legend(loc=[1.01, -0.06], fontsize=14)
plt.show()
```

See Bonus Section 1 to cover how to determine the stability of fixed points in a quantitative way.

## Think! 2: Inhibitory populations

Throughout the tutorial, we have assumed $w> 0 $, i.e., we considered a single population of **excitatory** neurons. What do you think will be the behavior of a population of inhibitory neurons, i.e., where $w> 0$ is replaced by $w< 0$?

```python
# to_remove explanation

"""

You can check this by going back the interactive demo 2.1 and set the
weight to w<0. You will notice that the system has only one fixed point and that
is at zero value. For this particular dynamics, the system will eventually converge
to zero. But try it out!
""";
```

---
# Summary

*Estimated timing of tutorial: 1 hour, 25 minutes*

In this tutorial, we have investigated the dynamics of a rate-based single population of neurons.

We learned about:
- The effect of the input parameters and the time constant of the network on the dynamics of the population.
- How to find the fixed point(s) of the system.

We build on these concepts in the bonus material - check it out if you have time. You will learn:

- How to determine the stability of a fixed point by linearizing the system.
- How to add realistic inputs to our model.

---
# Bonus

---
## Bonus Section 1: Stability analysis via linearization of the dynamics

```python
# @title Video 3: Stability of fixed points
from ipywidgets import widgets

out2 = widgets.Output()
with out2:
  from IPython.display import IFrame
  class BiliVideo(IFrame):
      def __init__(self, id, page=1, width=400, height=300, **kwargs):
          self.id=id
          src = 'https://player.bilibili.com/player.html?bvid={0}&page={1}'.format(id, page)
          super(BiliVideo, self).__init__(src, width, height, **kwargs)

  video = BiliVideo(id="BV1oA411e7eg", width=854, height=480, fs=1)
  print('Video available at https://www.bilibili.com/video/{0}'.format(video.id))
  display(video)

out1 = widgets.Output()
with out1:
  from IPython.display import YouTubeVideo
  video = YouTubeVideo(id="KKMlWWU83Jg", width=854, height=480, fs=1, rel=0)
  print('Video available at https://youtube.com/watch?v=' + video.id)
  display(video)

out = widgets.Tab([out1, out2])
out.set_title(0, 'Youtube')
out.set_title(1, 'Bilibili')

display(out)
```

    Tab(children=(Output(), Output()), _titles={'0': 'Youtube', '1': 'Bilibili'})

Here we will dive into the math of how to figure out the stability of a fixed point.

Just like in our equation for the feedforward network, a generic linear system
$$\frac{dx}{dt} = \lambda (x - b),$$
has a fixed point for $x=b$. The analytical solution of such a system can be found to be:
$$x(t) = b + \big{(} x(0) - b \big{)} \text{e}^{\lambda t}.$$
Now consider a small perturbation of the activity around the fixed point: $x(0) = b+ \epsilon$, where $|\epsilon| \ll 1$. Will the perturbation $\epsilon(t)$ grow with time or will it decay to the fixed point? The evolution of the perturbation with time can be written, using the analytical solution for $x(t)$, as:
 $$\epsilon (t) = x(t) - b = \epsilon \text{e}^{\lambda t}$$

- if $\lambda < 0$, $\epsilon(t)$ decays to zero, $x(t)$ will still converge to $b$ and the fixed point is "**stable**".

- if $\lambda > 0$, $\epsilon(t)$ grows with time, $x(t)$ will leave the fixed point $b$ exponentially, and the fixed point is, therefore, "**unstable**" .

Similar to what we did in the linear system above, in order to determine the stability of a fixed point $r^{*}$ of the excitatory population dynamics, we perturb Equation (1) around $r^{*}$ by $\epsilon$, i.e. $r = r^{*} + \epsilon$. We can plug in Equation (1) and obtain the equation determining the time evolution of the perturbation $\epsilon(t)$:

\begin{align}
\tau \frac{d\epsilon}{dt} \approx -\epsilon + w F'(w\cdot r^{*} + I_{\text{ext}};a,\theta) \epsilon
\end{align}

where $F'(\cdot)$ is the derivative of the transfer function $F(\cdot)$. We can rewrite the above equation as:

\begin{align}
\frac{d\epsilon}{dt} \approx \frac{\epsilon}{\tau }[-1 + w F'(w\cdot r^* + I_{\text{ext}};a,\theta)]
\end{align}

That is, as in the linear system above, the value of

$$\lambda = [-1+ wF'(w\cdot r^* + I_{\text{ext}};a,\theta)]/\tau \qquad (4)$$

determines whether the perturbation will grow or decay to zero, i.e., $\lambda$ defines the stability of the fixed point. This value is called the **eigenvalue** of the dynamical system.

The derivative of the sigmoid transfer function is:
\begin{align}
\frac{dF}{dx} & = \frac{d}{dx} (1+\exp\{-a(x-\theta)\})^{-1}  \\
& = a\exp\{-a(x-\theta)\} (1+\exp\{-a(x-\theta)\})^{-2}. \qquad (5)
\end{align}

We provide a helper function `dF` which computes this derivative.

```python
# @markdown Execute this cell to enable helper function `dF` and visualize derivative

def dF(x, a, theta):
  """
  Population activation function.

  Args:
    x     : the population input
    a     : the gain of the function
    theta : the threshold of the function

  Returns:
    dFdx  : the population activation response F(x) for input x
  """

  # Calculate the population activation
  dFdx = a * np.exp(-a * (x - theta)) * (1 + np.exp(-a * (x - theta)))**-2

  return dFdx

# Set parameters
pars = default_pars_single()  # get default parameters
x = np.arange(0, 10, .1)      # set the range of input

# Compute derivative of transfer function
df = dF(x, pars['a'], pars['theta'])

# Visualize
plot_dFdt(x, df)
```

### Bonus Coding Exercise 1: Compute eigenvalues

As discussed above, for the case with $w=5.0$ and $I_{\text{ext}}=0.5$, the system displays **three** fixed points. However, when we simulated the dynamics and varied the initial conditions $r_{\rm init}$, we could only obtain **two** steady states. In this exercise, we will now check the stability of each of the three fixed points by calculating the corresponding eigenvalues with the function `eig_single`. Check the sign of each eigenvalue (i.e., stability of each fixed point). How many of the fixed points are stable?

Note that the expression of the eigenvalue at fixed point $r^*$
$$\lambda = [-1+ wF'(w\cdot r^* + I_{\text{ext}};a,\theta)]/\tau$$

```python
def eig_single(fp, tau, a, theta, w, I_ext, **other_pars):
  """
  Args:
    fp   : fixed point r_fp
    tau, a, theta, w, I_ext : Simulation parameters

  Returns:
    eig : eigevalue of the linearized system
  """
  #####################################################################
  ## TODO for students: compute eigenvalue and disable the error
  #raise NotImplementedError("Student excercise: compute the eigenvalue")
  ######################################################################
  # Compute the eigenvalue
  eig = (1 / tau) * ( -1 + w * dF(w*fp + I_ext, a, theta) )

  return eig

# Find the eigenvalues for all fixed points
pars = default_pars_single(w=5, I_ext=.5)
r_guess_vector = [0, .4, .9]
x_fp = my_fp_finder(pars, r_guess_vector)

for fp in x_fp:
  eig_fp = eig_single(fp, **pars)
  print(f'Fixed point1 at {fp:.3f} with Eigenvalue={eig_fp:.3f}')
```

    Fixed point1 at 0.042 with Eigenvalue=-0.583
    Fixed point1 at 0.447 with Eigenvalue=0.498
    Fixed point1 at 0.900 with Eigenvalue=-0.626

```python
# to_remove solution
def eig_single(fp, tau, a, theta, w, I_ext, **other_pars):
  """
  Args:
    fp   : fixed point r_fp
    tau, a, theta, w, I_ext : Simulation parameters

  Returns:
    eig : eigevalue of the linearized system
  """
  # Compute the eigenvalue
  eig = (-1. + w * dF(w * fp + I_ext, a, theta)) / tau

  return eig

# Find the eigenvalues for all fixed points
pars = default_pars_single(w=5, I_ext=.5)
r_guess_vector = [0, .4, .9]
x_fp = my_fp_finder(pars, r_guess_vector)

for fp in x_fp:
  eig_fp = eig_single(fp, **pars)
  print(f'Fixed point1 at {fp:.3f} with Eigenvalue={eig_fp:.3f}')
```

    Fixed point1 at 0.042 with Eigenvalue=-0.583
    Fixed point1 at 0.447 with Eigenvalue=0.498
    Fixed point1 at 0.900 with Eigenvalue=-0.626

**SAMPLE OUTPUT**

```
Fixed point1 at 0.042 with Eigenvalue=-0.583
Fixed point2 at 0.447 with Eigenvalue=0.498
Fixed point3 at 0.900 with Eigenvalue=-0.626
```

We can see that the first and third fixed points are stable (negative eigenvalues) and the second is unstable (positive eigenvalue) - as we expected!

---
## Bonus Section 2: Noisy input drives the transition between two stable states

As discussed in several previous tutorials, the Ornstein-Uhlenbeck (OU) process is usually used to generate a noisy input into the neuron. The OU input $\eta(t)$ follows:

$$\tau_\eta \frac{d}{dt}\eta(t) = -\eta (t) + \sigma_\eta\sqrt{2\tau_\eta}\xi(t)$$

Execute the following function `my_OU(pars, sig, myseed=False)` to generate an OU process.

```python
# @markdown Execute to get helper function `my_OU` and visualize OU process

def my_OU(pars, sig, myseed=False):
  """
  A functions that generates Ornstein-Uhlenback process

  Args:
    pars       : parameter dictionary
    sig        : noise amplitute
    myseed     : random seed. int or boolean

  Returns:
    I          : Ornstein-Uhlenbeck input current
  """

  # Retrieve simulation parameters
  dt, range_t = pars['dt'], pars['range_t']
  Lt = range_t.size
  tau_ou = pars['tau_ou']  # [ms]

  # set random seed
  if myseed:
      np.random.seed(seed=myseed)
  else:
      np.random.seed()

  # Initialize
  noise = np.random.randn(Lt)
  I_ou = np.zeros(Lt)
  I_ou[0] = noise[0] * sig

  # generate OU
  for it in range(Lt - 1):
    I_ou[it + 1] = (I_ou[it]
                    + dt / tau_ou * (0. - I_ou[it])
                    + np.sqrt(2 * dt / tau_ou) * sig * noise[it + 1])

  return I_ou

pars = default_pars_single(T=100)
pars['tau_ou'] = 1.  # [ms]
sig_ou = 0.1
I_ou = my_OU(pars, sig=sig_ou, myseed=2020)
plt.figure(figsize=(10, 4))
plt.plot(pars['range_t'], I_ou, 'r')
plt.xlabel('t (ms)')
plt.ylabel(r'$I_{\mathrm{OU}}$')
plt.show()
```

In the presence of two or more fixed points, noisy inputs can drive a transition between the fixed points! Here, we stimulate an E population for 1,000 ms applying OU inputs.

```python
# @markdown Execute this cell to simulate E population with OU inputs

pars = default_pars_single(T=1000)
pars['w'] = 5.0
sig_ou = 0.7
pars['tau_ou'] = 1.  # [ms]
pars['I_ext'] = 0.56 + my_OU(pars, sig=sig_ou, myseed=2020)

r = simulate_single(pars)

plt.figure(figsize=(10, 4))
plt.plot(pars['range_t'], r, 'b', alpha=0.8)
plt.xlabel('t (ms)')
plt.ylabel(r'$r(t)$')
plt.show()
```

You can see that the population activity is changing between fixed points (it goes up and down)!

```python

```


# Praktikum Technische Informatik
---
# Versuch 7: Verlustbehaftete Audiocodierung
---

Inhaltsverzeichnis

1  Verlustbehaftete Audiocodierung

2  Darstellung von Audiosignalen

2.1  Diskrete Fourier-Transformation

2.2  Inverse Diskrete Fourier-Transformation

2.3  Kurzzeit-Fourier-Transformation

3  Eigenschaften des menschlichen Gehörs

3.1  Dezibel

3.2  Die Ruhehörschwelle

3.3  Verdeckung

3.3.1  Verdeckung im Frequenzbereich

3.3.2  Verdeckung im Zeitbereich

4  Das MP3-Kompressionsverfahren

4.1  Hybride Filterbänke und MDCT

4.2  Quantisierung

4.3  Das psychoakustische Modell

4.4  Huffman Codierung

4.5  MP3 Decodierer

## Verlustbehaftete Audiocodierung

Seit Jahren spalten sich die Musikliebhaber in zwei Lager: Die einen sehen in der verlustbehafteten Audiokompression einen Meilenstein der digitalen Audiosignalverarbeitung und -archivierung. Die anderen vertreten den Standpunkt, dass wahrer Musikgenuss nur bei unkomprimierten Signalquellen möglich sei.

Dieser Praktikumsversuch befasst sich mit den Ideen und Tricks der verlustbehafteten Audiokompression und stellt ihre praktische Umsetzung am Beispiel von MP3 vor.

**Wichtig: Lesen Sie zunächst dieses Skript und bearbeiten Sie danach die [Vorbereitungsaufgaben](Vorbereitungsaufgaben.ipynb)**

## Darstellung von Audiosignalen

Die einfachste digitale Darstellung von Audiosignalen ist die einer Sequenz $x[n] := x(nT)$ von Abtastwerten. Als Abtastfrequenz $f_s = 1/T$ wird in diesem Praktikumsversuch $44100 \text{ Hz}$ angenommen. Dies ist auch die Abtastfrequenz, die auf CDs verwendet wird. Der Wertebereich für jedes Sample beträgt $-1 \leq x[n] \leq +1$. Wird dieser Bereich überschritten, kommt es zu Verzerrungen (Clipping) bei der Signalausgabe.

Außer im Zeitbereich können Audiosignale auch im Frequenzbereich dargestellt und manipuliert werden. Hierzu dient die diskrete Fourier-Transformation.

### Diskrete Fourier-Transformation

Die diskrete Fourier-Transformation ist durch folgende Gleichung definiert:

\begin{equation}
        \displaystyle X[k]=\sum_{n=0}^{N-1} x[n]e^{-j2\pi \frac{nk}{N}} \qquad \text{ für } k=0,1,\dots,N-1
        \tag{7.1}\label{Gl.7.1}
\end{equation}

Die diskrete Fourier-Transformation berechnet aus einer endlichen Folge $x[n]$ mit $N$ Elementen die transformierte Folge $X[k]$, ebenfalls mit $N$ Elementen. Der Koeffizient $X[0]$ entspricht dabei dem Gleichanteil des Signals. Die Koeffizienten $X[i], i = 1,\dots,\frac{N}{2}$, geben die Amplitude und Phase der $i$-fachen Grundfrequenz an.

```python
import sys
sys.path.insert(0,'ient_nb')

import matplotlib.pyplot as plt
import matplotlib.pylab as pll
import numpy as np

import rwth_nb.plots.mpl_decorations as rwth_plt

from IPython.display import set_matplotlib_formats
set_matplotlib_formats('pdf', 'png')
plt.rcParams['savefig.dpi'] = 300

t = np.linspace(0,2/1000,801);
s1 = np.cos(2*np.pi*t*1000);

xn = s1[0:800:50];

xk =  np.fft.fft(xn,8);
xkr = xk.real

fig, ax = plt.subplots(1,2, figsize=(16, 3));

ax[0].xaxis.set_ticks_position('bottom')
ax[0].yaxis.set_ticks_position('left')

ax[1].xaxis.set_ticks_position('bottom')
ax[1].yaxis.set_ticks_position('left')

ax[0].plot( t*1000, s1);
markerline, stemlines, baseline = ax[1].stem(np.linspace(0,7,8),xk.real, use_line_collection=True)
plt.setp(baseline, 'color', 'none')

ax[0].set_xlabel(r'$\rightarrow$ t in s/1000');
ax[0].set_ylabel(r'$\uparrow$ s1');
rwth_plt.axis(ax[0]);

ax[1].set_xlabel(r'$\rightarrow$ k');
ax[1].set_ylabel(r'$\uparrow$ X[k]');
rwth_plt.axis(ax[1]);
plt.show()
```

>*Abbildung 7.1: Diskrete Fourier-Transformation*

Da der Wertebereich von Audiosignalen reell ist, bilden die Koeffizienten $X[1]$ und $X[N-1]$, $X[2]$ und $X[N-2]$ usw. konjugiert-komplexe Paare. Betrachtet man nur die Beträge der einzelnen Koeffizienten, so ist das Spektrum zur halben Abtastfrequenz symmetrisch. Welcher tatsächlichen Frequenz die einzelnen Koeffizienten in der Audiosignalverarbeitung entsprechen, hängt von der Abtastrate $f_s$ und der Transformationslänge $N$ ab. Die Grundfrequenz ist in Abhängigkeit dieser Größen in Gl. 7.2 definiert:

\begin{equation}
    \displaystyle f_\Delta = \frac{f_s}{N} \tag{7.2}\label{Gl.7.2}
\end{equation}

Wenn das Signal aus Abb. 7.1 mit einer Abtastrate von $f_s = 8000 \text{ Hz}$ abgetastet wird, entspricht der Koeffizient $X[1]$ bei einer Transformationslänge von $N = 8$ der Grundfrequenz $f_\Delta = 1000 \text{ Hz}$.

Die Berechnung der diskreten Fourier-Transformation nach Gl. 7.1 erfordert $N^2$  komplexe Multiplikationen. Durch die Anwendung der schnellen Fourier-Transformation (*Fast Fourier Transform*, FFT) lässt sich der Rechenaufwand durch eine geschickte Wiederverwertung von Zwischenergebnissen auf $N log_2 N$ komplexe Multiplikationen verringern. Die Routinen *fft* und *ifft*, die in vielen numerischen Programmen zu finden sind, benutzen die schnelle Fourier-Transformation, um eine DFT zu berechnen.

### Inverse Diskrete Fourier-Transformation

Ein im Frequenzbereich vorliegendes Signal kann mit der inversen Diskreten Fourier-Transformation (Gl.7.3) in den Zeitbereich rücktransformiert werden.

\begin{equation}
    \displaystyle x[n]= \frac{1}{N}\sum_{k=0}^{N-1}X[k]e^{j2\pi\frac{nk}{N}} \qquad \text{für } n=0,1,\dots,N-1 \tag{7.3}\label{Gl.7.3}
\end{equation}

Auch für die Berechnung der inversen DFT kann die FFT benutzt werden. Die Gleichungen 7.1 und 7.3 bilden ein Transformationspaar.

### Kurzzeit-Fourier-Transformation

Die diskrete Fourier-Transformation ist nur für Folgen mit endlicher Länge durchführbar. Dabei wird angenommen, dass die betrachtete Folge ein Ausschnitt aus einer unendlich periodisch fortgesetzten Funktion sei.

In vielen Anwendungen, so auch in der Audiokodierung, ist der Verlauf eines Audiosignals sowohl im Zeit-, als auch im Frequenzbereich von Interesse. Damit man eine Repräsentation in beiden Bereichen erhält, wird das Audiosignal zunächst in kleine Blöcke unterteilt. Ein einfaches Ausschneiden eines Blockes aus einer längeren Folge ist äquivalent zu einer Multiplikation des Signals mit einer Rechteckfunktion. Eine Multiplikation im Zeitbereich entspricht im Frequenzbereich einer Faltungsoperation. Aus der Multiplikation im Zeitbereich mit einer Rechteckfunktion wird somit im Frequenzbereich eine Faltung mit der si-Funktion. Dadurch wird das
Spektrum verschmiert (*Leakage-Effekt*).

Durch die Wahl einer geeigneten Fensterfunktion kann dieser Effekt verringert werden. Bei
der MP3-Kodierung wird das Sinus-Fenster benutzt:

\begin{equation}
   \displaystyle w[n] = \sin \left[\frac{\pi(n+0,5)}{N}\right] \qquad \text{für } 0\leq n < N \tag{7.4}\label{Gl.7.4}
\end{equation}

```python
import sys
sys.path.insert(0,'ient_nb')

import matplotlib.pyplot as plt
import matplotlib.pylab as pll

t1 = np.linspace(0,12*np.pi,100);
sig = np.sin(t1-0.1*np.pi);

t2 = np.linspace(0,99,100);
w = np.sin((np.pi*(t2+0.5))/100);

mul = np.multiply(sig,w);

fig,ax = plt.subplots(1,3, figsize=(8,2));

ax[0].tick_params(bottom=False, left=False, labelbottom=False, labelleft=False);
ax[1].tick_params(bottom=False, left=False, labelbottom=False, labelleft=False);
ax[1].set_ylabel("x",rotation=0, fontsize="xx-large", horizontalalignment = "right", verticalalignment="center");
ax[2].tick_params(bottom=False, left=False, labelbottom=False, labelleft=False);
ax[2].set_ylabel('=',rotation=0, fontsize="xx-large", horizontalalignment = "right", verticalalignment="center");

ax[0].plot(t1,sig);
ax[1].plot(t1,w);
ax[2].plot(t1,mul);

plt.show()
```

>*Abbildung 7.2: Multiplikation mit einer Fensterfunktion*

Abbildung 7.2 zeigt schematisch die Wirkungsweise einer Fensterfunktion. Das Signal auf der linken Seite - welches in der Praxis oft nicht ohne Sprungstellen periodisch fortsetzbar ist - wird mit der Fensterfunktion multipliziert. Das so gefensterte Signal wird an den Intervallgrenzen deutlich gedämpft wodurch der Leakage-Effekt stark verringert wird.

Offensichtlich gehen an den Intervallgrenzen durch die Fensterung Informationen verloren. Bei Anwendung der inversen Fourier-Transformation wird im Zeitbereich ebenfalls mit der Fensterfunktion multipliziert, so dass eine einfache Aneinanderreihung der rücktransformierten Blöcke zu starken Lautstärkeschwankungen führt. Diese Probleme kann man umgehen, indem man die Transformationsblöcke zu 50% überlappen läßt. Das gesamte Verfahren ist in der Literatur als *Kurzzeit-Fouriertransformation* (*Short-Time Fourier Transform*, STFT) bekannt.

Durch diese Überlappung verdoppelt sich allerdings die zu kodierende Datenmenge. Aus diesem Grund benutzt man in der Audiocodierung nicht die Kurzzeit-Fouriertransformation, sondern die *Modified Discrete Cosine Transform* (MDCT). Bei der MDCT reichen $N/2$ Koeffizienten zur Kodierung von $N$ Samples aus; allerdings tritt dadurch an den Blockrändern Aliasing auf. Nun ist die MDCT so strukturiert, dass das Aliasing von benachbarten (sich überlappenden) Blöcken sich jeweils aufhebt. Dadurch ist es möglich, den Vorteil des Überlapps (Vermeidung von Blockartefakten) zu bekommen, ohne die Datenmenge gegenüber einer nicht-überlappenden Transformation zu erhöhen.

## Eigenschaften des menschlichen Gehörs

Die verschiedenen Algorithmen der verlustbehafteten Audiokompression bauen letztlich alle auf der Kenntnis auf, dass das menschliche Gehör nicht alle Klänge wahrnehmen kann. Daher wird in diesem Abschnitt besprochen, welche Klänge das menschliche Ohr wahrnehmen kann und welche nicht. Vorher werden einige Grundlagen der Signalverarbeitung behandelt, die für das Verständnis des Versuchs notwendig sind.

### Dezibel

Das Dezibel stellt das Verhältnis zweier Werte an einer logarithmischen Skala dar. Es ist nicht für bestimmte physikalische Größen definiert, sondern liefert das logarithmierte Verhältnis zweier Größen. Vergleicht man Amplituden miteinander, so verwendet man Formel 7.5. Vergleicht man hingegen Leistungen, so benutzt man Formel 7.6. Der Grund hierfür liegt in der Tatsache, dass die Leistung eines Signals quadratisch mit seiner Amplitude zusammenhängt.

\begin{equation}
    \displaystyle [\text{dB}] = 20 \log_{10} \left(\frac{A}{A_0}\right) \tag{7.5}\label{Gl.7.5}
\end{equation}

\begin{equation}
    \displaystyle [\text{dB}] = 10 \log_{10} \left(\frac{P}{P_0}\right) \tag{7.6}\label{Gl.7.6}
\end{equation}

Um den Schalldruckpegel $L_p$ eines Signals in der logarithmischen Einheit Dezibel (dB) angeben zu können, wurde international eine Bezugsgröße $P_0 = 20\mu \text{Pa} = 10^{-12}W/m^2$ willkürlich festgelegt. Der Schalldruckpegel $L_p$ berechnet sich somit wie folgt:

\begin{equation}
    \displaystyle L_p = 20 \log_{10} \left(\frac{P}{10^{-12} \frac{W}{m^2}}\right) \tag{7.7}\label{Gl.7.7}
\end{equation}

### Die Ruhehörschwelle

Das menschliche Gehör hat ein begrenztes, frequenzabhängiges Auflösungsvermögen. Um einen Ton wahrzunehmen, muss der Ton über der so genannten *Ruhehörschwelle* liegen. Diese Ruhehörschwelle ist nicht nur von Mensch zu Mensch verschieden, sondern ändert (verschlechtert) sich auch im Leben eines Menschen gerade in den hohen Frequenzen. In Abb. 7.3 ist die Ruhehörschwelle $L$ einer normal hörenden Person in Abhängigkeit von der Frequenz $f$ skizziert. Sie
ist durch Mittelung über viele Versuchspersonen empirisch bestimmt worden.

```python
import sys
sys.path.insert(0,'ient_nb')

import matplotlib.pyplot as plt
import matplotlib.pylab as pll

from IPython.display import set_matplotlib_formats
set_matplotlib_formats('pdf', 'png')
plt.rcParams['savefig.dpi'] = 300

data = pll.loadtxt('img/source/hoerschwelle.dat');

fig, ax = plt.subplots()
ax.plot( data[:,0], data[:,1]);
rwth_plt.axis(ax);

plt.xscale('log',basex=2);
plt.xticks([0.02,0.05,0.1,0.2,0.5,1,2,5,10,20],[0.02,0.05,0.1,0.2,0.5,1,2,5,10,20]);
plt.grid()
ax.set_xlabel(r'$\rightarrow f $ in $[kHz]$');
ax.set_ylabel(r'$\uparrow$ Schalldruck in $[dB]$');

plt.show()
```

>*Abbildung 7.3: Ruhehörschwelle des Menschen*

### Verdeckung

Liegt ein Signal oberhalb der Ruhehörschwelle, so bedeutet es nicht automatisch, dass es vom menschlichen Gehör wahrgenommen werden kann. Ein Signal kann durch ein Störsignal (sog. *Maskierer*) verdeckt werden.

Man unterscheidet grundsätzlich zwei Arten der Verdeckung:

• Verdeckung im Frequenzbereich

• Verdeckung im Zeitbereich

#### Verdeckung im Frequenzbereich

Bestimmt man die Hörschwelle nicht bei Stille sondern bei Anwesenheit eines schmalbandigen Maskierers (z.B. Sinuston, Schmalbandrauschen) mit der Frequenz $f_0$ , so stellt man fest, dass sich die Hörschwelle gerade in der Umgebung von $f_0$ deutlich von der Ruhehörschwelle unterscheidet. Ein solcher Maskierer kann Signale in einem ähnlichen Frequenzbereich verdecken. Abb. 7.4 illustriert diesen Effekt.

```python
import sys
sys.path.insert(0,'ient_nb')

import matplotlib.pyplot as plt
import matplotlib.pylab as pll

from IPython.display import set_matplotlib_formats
set_matplotlib_formats('pdf', 'png')
plt.rcParams['savefig.dpi'] = 300

data = pll.loadtxt('img/source/hoerschwelle.dat');

fig, ax = plt.subplots(figsize=(12,5))
ax.plot( data[:,0], data[:,1]);
rwth_plt.axis(ax);

data = pll.loadtxt('img/source/verdeckung2.dat');
ax.plot( data[:,0], data[:,1], color='red');
rwth_plt.axis(ax);

lim = ax.get_ylim();

ax.bar(1, height=80, width=0.15, bottom=lim[0], color="orange")
ax.bar(3, height=46, width=0.45, bottom=lim[0], color="grey")

plt.xscale('log',basex=2);
plt.xticks([1,3,10],["$f_0$","$f_1$",10]);
plt.yticks([]);
ax.set_xlabel(r'$\rightarrow f $ in $[kHz]$');
ax.set_ylabel(r'$\uparrow$ Schalldruck in $[dB]$');

ax.annotate('absolute Hörschwelle',
            xy=(0.031, 57.6), xycoords='data',
            xytext=(50, 25), textcoords='offset points',
            arrowprops=dict(facecolor='black', shrink=0.1, width=1, headlength=5, headwidth=5),
            horizontalalignment='left', verticalalignment='bottom')

ax.annotate('Maskierungsschwelle',
            xy=(0.7, 29), xycoords='data',
            xytext=(-50, 0), textcoords='offset points',
            arrowprops=dict(facecolor='black', shrink=0.1, width=1, headlength=5, headwidth=5),
            horizontalalignment='right', verticalalignment='center')

ax.annotate('Maskierer',
            xy=(1, 70), xycoords='data',
            xytext=(50, 10), textcoords='offset points',
            arrowprops=dict(facecolor='black', shrink=0.15, width=1, headlength=5, headwidth=5),
            horizontalalignment='left', verticalalignment='bottom')

ax.annotate('maskiertes Signal',
            xy=(3.2, 10), xycoords='data',
            xytext=(50, 0), textcoords='offset points',
            arrowprops=dict(facecolor='black', shrink=0.1, width=1, headlength=5, headwidth=5),
            horizontalalignment='left', verticalalignment='center')

plt.show()
```

>*Abbildung 7.4: Verdeckung eines Signals durch einen sog. Maskierer im Frequenzbereich*

In Abb. 7.5 ist der Einfluss eines $70\text{dB }$ lauten Sinustons mit der Frequenz $1\text{kHz }$ auf die Hörschwelle skizziert. Als Verhältnis zwischen der maximalen Leistung eines sinusförmigen Testsignals und der Leistung des maskierenden Rauschsignals $P_N$ werden in der Literatur  Werte von $3 - 6 \text{ dB}$ angegeben. Die Art der Verdeckung wird auch als *noise masking tone (NMT)*, *pure tone masked by noise* oder *pure tone masked by pure tone* bezeichnet  .

```python
import sys
sys.path.insert(0,'ient_nb')

import matplotlib.pyplot as plt
import matplotlib.pylab as pll

from IPython.display import set_matplotlib_formats
set_matplotlib_formats('pdf', 'png')
plt.rcParams['savefig.dpi'] = 300

data = pll.loadtxt('img/source/hoerschwelle.dat');

fig, ax = plt.subplots()
ax.plot( data[:,0], data[:,1]);
rwth_plt.axis(ax);

data = pll.loadtxt('img/source/verdeckung2.dat');
ax.plot( data[:,0], data[:,1],color='red');
rwth_plt.axis(ax);

plt.xscale('log',basex=2);
plt.xticks([0.02,0.05,0.1,0.2,0.5,1,2,5,10,20],[0.02,0.05,0.1,0.2,0.5,1,2,5,10,20]);
plt.grid()
ax.set_xlabel(r'$\rightarrow f $ in $[kHz]$');
ax.set_ylabel(r'$\uparrow$ Schalldruck in $[dB]$');

plt.show()
```

>*Abbildung 7.5: Veränderung der Hörschwelle durch einen Sinuston mit der Frequenz $f = 1\text{kHz}$ und einer Leistung $L = 70\text{dB}$*

Auf andere Verdeckungseffekte im Frequenzbereich, wie *tone masking noise*, bei dem der Einfluss eines Sinustons auf die Wahrnehmbarkeit von breitbandigem Rauschen untersucht
wird, soll im Rahmen dieses Praktikumsversuches nicht weiter eingegangen werden.

#### Verdeckung im Zeitbereich

Bei einer starken Lautstärkeschwankung lässt sich ein weiterer Verdeckungseffekt beobachten: die *Verdeckung im Zeitbereich* . Sowohl bei einer Lautstärkeänderung von laut nach leise als auch umgekehrt ist dieses Phänomen zu beobachten.

Durch einen leistungsstarken Maskierer wird ein nachfolgendes, schwächeres Signal verdeckt. Erst nach etwa $200 \text{ms }$ erreicht die Empfindlichkeit des Ohrs wieder die Ruhempfindlichkeit (engl.: *Recovery Time*). Diese Art der Verdeckung wird als *Nachverdeckung* bezeichnet. Aber auch bei einem Lautstärkeübergang von leise nach laut ist im leisen Bereich eine Verdeckung zu beobachten (*Vorverdeckung*).

Beide Arten der zeitlichen Verdeckung sind in Abbildung 7.6 skizziert. Im Rahmen des Praktikums soll aus Zeitgründen auf die Verdeckung im Zeitbereich ebenfalls nicht eingegangen werden.

>*Abbildung 7.6: Verdeckung im Zeitbereich*

## Das MP3-Kompressionsverfahren

Mit dem Standard *MPEG-1* wurde neben der Videocodierung auch der Versuch unternommen, Audiosignale (verlustbehaftet) zu komprimieren. In der endgültigen Version beinhaltet der MPEG-1 Standard drei verlustbehaftete Codierverfahren für Audiosignale, die als *Layer 1*, *Layer 2* und *Layer 3* bezeichnet werden. Sie nehmen mit größer werdendem Index in ihrer Qualität aber auch in ihrer Komplexität zu und sind zueinander abwärtskompatibel.

In diesem Praktikumsversuch wird der komplexeste aber auch berühmteste der drei Audio-Codecs vorgestellt (Codec ist ein Kunstwort aus **Co**der und **Dec**oder). Der *MPEG-1-Layer 3*, kurz MP3 genannt, ist vor allem durch die Musikübertragung über das Internet bekannt geworden. Sein Nachfolger AAC wird u.a. in iTunes verwendet. AAC basiert auf dem gleichen Grundprinzip, nutzt jedoch eine reine MDCT-Filterbank und zusätzliche Tricks, um eine noch höhere Kompressionsrate zu erzielen.

>*Abbildung 7.7: Blockdiagramm eines ISO/MPEG/Audio Codierers, Layer III, Mono*

In Abb. 7.7 ist die Strukutur des MP3-Kompressionsverfahrens dargestellt. Im Folgenden
werden die einzelnen Schritte detaillierter beschrieben.

### Hybride Filterbänke und MDCT

Das digitale Audiosignal wird zunächst in 32 Teilbänder zerlegt. Die Breiten der einzelnen Bandpassfilter sind an das menschliche Gehör angepasst. Jedes dieser Teilbänder wird anschließend mit Hilfe einer MDCT in 18 Teilbereiche unterteilt, wodurch bis zu 576 (=32·18) Kanäle entstehen (vgl. Abb. 7.7). Der AAC-Codec verwendet eine reine MDCT als Filterbank.

### Quantisierung

Anschließend werden die einzelnen Frequenzwerte quantisiert. Dabei wird ein nicht linearer Quantisierer eingesetzt.

\begin{equation}
    \displaystyle x_q = \text{sgn}(x)\cdot\left[\left(\frac{|x|}{d}\right)^{\frac{3}{4}}-0.0946\right] \tag{7.8}\label{Gl.7.8}
\end{equation}
mit

\begin{align*}
    |\dots|&: \text{Betragsfunktion} \\
    d&: \text{aktuelle Stufenbreite des Quantisierers}\\
    [\dots]&: \text{Gaußklammer}\\
    x_q&: \text{quantisierter Wert von $x$}\\
    \text{sgn}(x)&: \text{Vorzeichenfunktion (1 für positive, -1 für negative Werte)}\\
\end{align*}

Die Genauigkeit der Quantisierung (die aktuelle Stufenbreite des Quantisierers $d$) richtet sich nach der momentanen Hörschwelle des Ohres in den einzelnen Frequenzbändern.

### Das psychoakustische Modell

Bei der Quantisierung der einzelnen Frequenzbänder wird danach gestrebt, den Quantisierungsfehler (Differenzsignal zwischen Originalsignal und rekonstruiertem Signal) unterhalb der Hörbarkeitsschwelle zu halten. Die aktuelle Hörbarkeitsschwelle wird durch das *psychoakustische Modell* (vgl. Abb. 7.7) geschätzt. Das psychoakustische Modell berücksichtigt hierbei die Verdeckungseffekte im Zeit- und Frequenzbereich.

Im Laufe der Zeit wurde dieses Modell durch Hörtests immer wieder verfeinert und verbessert. Geschulten Testhörern (sog. *Golden Ears*) wurden dabei veränderte/verbesserte Versionen des psychoakustischen Modells vorgestellt und so versucht, die hörbaren Verfälschungen zu minimieren oder gar ganz zu eliminieren. So wurde beispielsweise festgestellt, dass harmonische und unharmonische Signale unterschiedliche Verdeckungseigenschaften aufweisen.

Die Implementierung des psychoakustischen Modells ist der Hauptunterschied zwischen den unterschiedlichen MP3-Codierern und maßgeblich für die Qualität des Codierers verantwortlich. Da bei MP3 nur der Decodierer standardisiert ist, nicht jedoch der Codierer, wird den Entwicklern genug Spielraum gegeben, das akustische Modell zu optimieren.

### Huffman Codierung

Die Frequenzwerte werden nach ihrer Quantisierung Huffman-codiert. Dabei wird häufig auftretenden Werten ein kurzes Codewort (mit wenigen Bits) zugewiesen, während Werte, die weniger häufig vorkommen, ein längeres Codewort bekommen. Der genaue Algorithmus der Huffman-Codierung ist u. a. in (Lüke und Ohm, 2014),(Ohm, 2015) beschrieben.

### MP3 Decodierer

>*Abbildung 7.8: Blockdiagramm eines MP3 Decoders*

Die Decodierung eines MP3-Datenstroms ist im Vergleich zu der Codierung deutlich einfacher. Hauptaufgabe bei der Decodierung ist die Invertierung der einzelnen Codierschritte (vgl. Abb. 7.8). Eine Kenntnis des psychoakustischen Modells ist nicht notwendig. Im Gegensatz zum Codierer ist der Decodierer im Standard fest definiert. Verbesserungen des MP3-Verfahrens finden somit ausschließlich auf der Codiererseite statt.

Für an den technischen Details des MP3-Codecs, wie der *Bit-Reservoir Technik*, der *Struktur des Bit-Streams* oder der Codierung von *Stereo Signalen* interessierte Studenten sei auf (Brandenburg und Stoll, 1995) verwiesen.


# Functions

```python
def say_hello():
    # block belonging to the function
    print('hello world')
# End of function

say_hello()
say_hello()
```

    hello world
    hello world

**Function Parameters:**

```python
def print_max(a, b):
    if a > b:
        print(a, 'is maximum')
    elif a == b:
        print(a, 'is equal to', b)
    else:
        print(b, 'is maximum')

# directly pass literal values
print_max(3, 4)

x = 5
y = 7

# pass variables as arguments
print_max(x, y)
```

    4 is maximum
    7 is maximum

**The local variables**

```python
x = 50

def func(x):
    print('x is', x)
    x = 2
    print('Changed local x to', x)

func(x)
print('x is still', x)
```

    x is 50
    Changed local x to 2
    x is still 50

**The 'global' statement**

```python
x = 50

def func():
    global x

    print('x is', x)
    x = 2
    print('Changed global x to', x)

func()
print('Value of x is', x)
```

    x is 50
    Changed global x to 2
    Value of x is 2

**Default argument values**

```python
def say(message, times=1):
    print(message * times)

say('Hello')
say('World', 5)
```

    Hello
    WorldWorldWorldWorldWorld

**Keyword Arguments**

```python
def func(a, b=5, c=10):
    print('a is', a, 'and b is', b, 'and c is', c)

func(3, 7)
func(25, c=24)
func(c=50, a=100)
```

    a is 3 and b is 7 and c is 10
    a is 25 and b is 5 and c is 24
    a is 100 and b is 5 and c is 50

**VarArgs parameters**

```python
def total(a=5, *numbers, **phonebook):
    print('a', a)

    #iterate through all the items in tuple
    for single_item in numbers:
        print('single_item', single_item)

    #iterate through all the items in dictionary
    for first_part, second_part in phonebook.items():
        print(first_part,second_part)

total(10,1,2,3,Jack=1123,John=2231,Inge=1560)
```

    a 10
    single_item 1
    single_item 2
    single_item 3
    Jack 1123
    John 2231
    Inge 1560

**The return statement**

```python
def maximum(x, y):
    if x > y:
        return x
    elif x == y:
        return 'The numbers are equal'
    else:
        return y

print(maximum(2, 3))
```

    3

**DocStrings:**

```python
def print_max(x, y):
    '''Prints the maximum of two numbers.

The two values must be integers.'''
    # convert to integers, if possible
    x = int(x)
    y = int(y)

    if x > y:
        print(x, 'is maximum')
    else:
        print(y, 'is maximum')

print_max(3, 5)
print(print_max.__doc__)
```

    5 is maximum
    Prints the maximum of two numbers.

    The two values must be integers.

# Project

In this project, we plan to do a recursive implementation of calculating fubonacci numbers. Remember, fibonacci numbers can be calculated using the following equation:

\begin{equation}
f(x)=
\begin{cases}
    0,              & \text{if } x = 0\\
    1,              & \text{if } x = 1\\
    f(x-1)+f(x-2),  & \text{otherwise}
\end{cases}
\end{equation}

```python
# Your code goes here!

```

```python
for i in range(12):
    print(recursive_fibo(i), end=' ')
```

    0 1 1 2 3 5 8 13 21 34 55 89


# Session 3: Introduction to sequence modeling

In this session we are going to deal with sequences, see how we can model different types of problems as sequences and how to deal with them from a machine learning perspective.

## Sequencces in real life

## Application of sequences

Depending on the type of sequence problem we are treating, there are many ways in how we can model it in terms of inputs and outputs. as shown in the image below:

We can use neural networks also for this types of problems, but until now the tasks that we were doing doesn't have time dependence between them. Another point is that using standard neural networks for time series is that they may be too big for covering many time steps or we may fail in capturing time dependencies that are too far away from each other. There is one neural network model that broke through those barrries in sequence modeling and started to advance the state of the art in fields like Natural Language processing and that is Recurrent Neural Networks.

## Recurrent Neural Networks

Recurrent nerual networks are a family of neural network architectures that are used for sequential data and use the notion of recurrence. They can have many forms in terms of applications but the basic notion is that the parameters of the model are shared through the entire time history and it takes as input the state of the previous time-step.

LSTMs
_______________
Common Problem with RNNs: Vanishing and exploding gradients tend to appear when a gradient of an output is small or large, so that error is going to propagate through the whole sequence making the learning diverge or stop updating the weights. This vanishing/exploding gradient problem is usally present when dealing with problems with long range dependencies.

One way to deal with these problems and address long-range dependencies is by adding gates to the RNN cell. One of the most effectives architectures for this are Long-Short Term Memory networks or LSTMs.

\begin{equation}
\begin{array}{l}
i_{t}=\sigma\left(W_{i i} x_{t}+b_{i i}+W_{h i} h_{t-1}+b_{h i}\right) \\
f_{t}=\sigma\left(W_{i f} x_{t}+b_{i f}+W_{h f} h_{t-1}+b_{h f}\right) \\
g_{t}=\tanh \left(W_{i g} x_{t}+b_{i g}+W_{h g} h_{t-1}+b_{h g}\right) \\
o_{t}=\sigma\left(W_{i o} x_{t}+b_{i o}+W_{h o} h_{t-1}+b_{h o}\right) \\
c_{t}=f_{t} \odot c_{t-1}+i_{t} \odot g_{t} \\
h_{t}=o_{t} \odot \tanh \left(c_{t}\right)
\end{array}
\end{equation}

The LSTM has 4 gates that control the flow of information:

*   The forget gate $f_t$ that takes the previous hidden state and the current input to determine how much informaion of the hidden state to forget.
*   The input gate $i_t$ that controls how much of the new input to retain.
*   The external input gate $g_t$ that controls which parts of the hidden step to update.
*   The output gate $o_t$ that controls which information to pass to the next time step.

The LSTM introduces a cell state, that serves as a memory for the cell of the network.

#Generaring Names with a Character RNN

Tutorial from pytorch [docs](https://pytorch.org/tutorials/intermediate/char_rnn_generation_tutorial.html).

```python
!gdown --id 1ccN7lWQTyrH27NsB__7zNqqSVVFzOCe4
!unzip ./data.zip
```

## Helper functions

Here it is defined a set of functions that are going to be used to train the network. The network takes as input a category, the current letter and the current hidden state and it outputs the predicted letter and next hidden state. For training the network we select randomly a pair of category and word that then are one-hot encoded and fed into the network. The predictions are compared to a target letter for each time step in order to compute the loss function and then find the gradients used for updating the weights of the model.

```python
from __future__ import unicode_literals, print_function, division
from io import open
import glob
import os
import unicodedata
import string

all_letters = string.ascii_letters + " .,;'-"
n_letters = len(all_letters) + 1 # Plus EOS marker

def findFiles(path): return glob.glob(path)

# Turn a Unicode string to plain ASCII, thanks to https://stackoverflow.com/a/518232/2809427
def unicodeToAscii(s):
    return ''.join(
        c for c in unicodedata.normalize('NFD', s)
        if unicodedata.category(c) != 'Mn'
        and c in all_letters
    )

# Read a file and split into lines
def readLines(filename):
    lines = open(filename, encoding='utf-8').read().strip().split('\n')
    return [unicodeToAscii(line) for line in lines]

# Build the category_lines dictionary, a list of lines per category
category_lines = {}
all_categories = []
for filename in findFiles('data/names/*.txt'):
    category = os.path.splitext(os.path.basename(filename))[0]
    all_categories.append(category)
    lines = readLines(filename)
    category_lines[category] = lines

n_categories = len(all_categories)

if n_categories == 0:
    raise RuntimeError('Data not found. Make sure that you downloaded data '
        'from https://download.pytorch.org/tutorial/data.zip and extract it to '
        'the current directory.')

print('# categories:', n_categories, all_categories)
print(unicodeToAscii("O'Néàl"))

import random

# Random item from a list
def randomChoice(l):
    return l[random.randint(0, len(l) - 1)]

# Get a random category and random line from that category
def randomTrainingPair():
    category = randomChoice(all_categories)
    line = randomChoice(category_lines[category])
    return category, line

# One-hot vector for category
def categoryTensor(category):
    li = all_categories.index(category)
    tensor = torch.zeros(1, n_categories)
    tensor[0][li] = 1
    return tensor

# One-hot matrix of first to last letters (not including EOS) for input
def inputTensor(line):
    tensor = torch.zeros(len(line), 1, n_letters)
    for li in range(len(line)):
        letter = line[li]
        tensor[li][0][all_letters.find(letter)] = 1
    return tensor

# LongTensor of second letter to end (EOS) for target
def targetTensor(line):
    letter_indexes = [all_letters.find(line[li]) for li in range(1, len(line))]
    letter_indexes.append(n_letters - 1) # EOS
    return torch.LongTensor(letter_indexes)

# Make category, input, and target tensors from a random category, line pair
def randomTrainingExample():
    category, line = randomTrainingPair()
    category_tensor = categoryTensor(category)
    input_line_tensor = inputTensor(line)
    target_line_tensor = targetTensor(line)
    return category_tensor, input_line_tensor, target_line_tensor
```

## Model and Training loop

```python
import torch
import torch.nn as nn

class RNN(nn.Module):
    def __init__(self, input_size, hidden_size, output_size):
        super(RNN, self).__init__()
        self.hidden_size = hidden_size

        self.i2h = nn.Linear(n_categories + input_size + hidden_size, hidden_size)
        self.i2o = nn.Linear(n_categories + input_size + hidden_size, output_size)
        self.o2o = nn.Linear(hidden_size + output_size, output_size)
        self.dropout = nn.Dropout(0.1)
        self.softmax = nn.LogSoftmax(dim=1)

    def forward(self, category, input, hidden):
        input_combined = torch.cat((category, input, hidden), 1)
        hidden = self.i2h(input_combined)
        output = self.i2o(input_combined)
        output_combined = torch.cat((hidden, output), 1)
        output = self.o2o(output_combined)
        output = self.dropout(output)
        output = self.softmax(output)
        return output, hidden

    def initHidden(self):
        return torch.zeros(1, self.hidden_size)
```

```python
import torch
import torch.nn as nn

class LSTM_name(nn.Module):
    def __init__(self, input_size, hidden_size, output_size):
        super(LSTM_name, self).__init__()
        self.hidden_size = hidden_size
        self.i2o = nn.Linear(n_categories + input_size, input_size)
        self.lstm = nn.LSTM(input_size=input_size, hidden_size=hidden_size, batch_first=True)
        self.o2o = nn.Linear(hidden_size,output_size)
        self.dropout = nn.Dropout(0.1)
        self.softmax = nn.LogSoftmax(dim=1)

    def forward(self, category, input, hidden):
        input_combined = torch.cat((category.unsqueeze(1), input.unsqueeze(1)), -1)
        #hidden = self.i2h(input_combined)
        output = self.i2o(input_combined)
        #output_combined = torch.cat((hidden, output), 1)
        output, hidden = self.lstm(output, hidden)
        output = self.o2o(output.reshape(-1, self.hidden_size))
        output = self.dropout(output)
        output = self.softmax(output)
        return output, hidden

    def initHidden(self, batch_size):
        ''' Initializes hidden state '''
        h0 = torch.zeros((1,batch_size,self.hidden_size)).to(device)
        c0 = torch.zeros((1,batch_size,self.hidden_size)).to(device)
        hidden = (h0,c0)
        return hidden
```

```python
criterion = nn.NLLLoss()

def train(category_tensor, input_line_tensor, target_line_tensor, optimizer):
    target_line_tensor.unsqueeze_(-1).to(device)
    input_line_tensor =  input_line_tensor.to(device)
    target_line_tensor = target_line_tensor.to(device)
    hidden = rnn.initHidden()
    optimizer.zero_grad()

    loss = 0

    for i in range(input_line_tensor.size(0)):
        output, hidden = rnn(category_tensor.to(device), input_line_tensor[i], hidden)
        l = criterion(output, target_line_tensor[i])
        loss += l

    loss.backward()
    optimizer.step()

    return output, loss.item() / input_line_tensor.size(0)
```

```python
import time
import math

def timeSince(since):
    now = time.time()
    s = now - since
    m = math.floor(s / 60)
    s -= m * 60
    return '%dm %ds' % (m, s)
```

```python
device = "cpu"
```

```python
rnn = LSTM_name(n_letters, 128, n_letters).to(device)
optimizer = torch.optim.Adam(rnn.parameters(),lr=5e-4)

n_iters = 100000
print_every = 5000
plot_every = 500
all_losses = []
total_loss = 0 # Reset every plot_every iters

start = time.time()

for iter in range(1, n_iters + 1):
    output, loss = train(*randomTrainingExample(),optimizer=optimizer)
    total_loss += loss

    if iter % print_every == 0:
        print('%s (%d %d%%) %.4f' % (timeSince(start), iter, iter / n_iters * 100, loss))

    if iter % plot_every == 0:
        all_losses.append(total_loss / plot_every)
        total_loss = 0
```

```python
import matplotlib.pyplot as plt
import matplotlib.ticker as ticker

plt.figure()
plt.plot(all_losses)
```

```python
max_length = 20
rnn = rnn.cpu()
rnn.eval()
# Sample from a category and starting letter
def sample(category, start_letter='A'):
    with torch.no_grad():  # no need to track history in sampling
        category_tensor = categoryTensor(category)
        input = inputTensor(start_letter)
        hidden = rnn.initHidden()

        output_name = start_letter

        for i in range(max_length):
            output, hidden = rnn(category_tensor, input[0], hidden)
            topv, topi = output.topk(1)
            topi = topi[0][0]
            if topi == n_letters - 1:
                break
            else:
                letter = all_letters[topi]
                output_name += letter
            input = inputTensor(letter)

        return output_name

# Get multiple samples from one category and multiple starting letters
def samples(category, start_letters='ABC'):
    print(category)
    print('---')
    for start_letter in start_letters:
        print(sample(category, start_letter))
    print("   ")

samples('Russian', 'RUS')

samples('German', 'GER')

samples('Spanish', 'SPA')

samples('Chinese', 'CHI')
```

# Sentiment Analysis using a LSTM

Network Architecture
_______________________

Bidirectional LSTM

Multi-Layer LSTM

```python
import torch
import torch.nn as nn
import torch.nn.functional as F
from torchtext import datasets
import numpy as np
import matplotlib.pyplot as plt
from torch.utils.data import DataLoader
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
from torchtext.data.utils import get_tokenizer
from collections import Counter
from torchtext.vocab import Vocab
from torch.utils.data.dataset import random_split
from torch.nn.utils.rnn import pad_sequence
import os
```

```python
train_iter, test_iter = datasets.IMDB()
max_vocab_size = 25000
tokenizer = get_tokenizer('spacy')
'''for root, dir, files in os.walk('./'):
      if 'counter_imdb.py' in files:
         counter = torch.load(*files)'''
counter = torch.load('./counter_imdb.py')
if counter is None:
  counter = Counter()
  for (label, line) in train_iter:
      counter.update(tokenizer(line))
vocab = Vocab(counter, min_freq=1, max_size=max_vocab_size, vectors="glove.6B.100d",
              unk_init =torch.Tensor.normal_)

text_pipeline = lambda x: [vocab[token] for token in tokenizer(x)]
def label_pipeline(x):
  if x=='pos':
    return 1
  else:
    return 0

def collate_batch(batch):
    label_list, text_list, text_lengths = [], [], []
    for (_label, _text) in batch:
         label_list.append(label_pipeline(_label))
         processed_text = torch.tensor(text_pipeline(_text), dtype=torch.int64)
         text_list.append((processed_text))
         text_lengths.append(len(processed_text))
    label_list = torch.tensor(label_list, dtype=torch.int64)
    text_lengths = torch.tensor(text_lengths, dtype=torch.int64)
    text_list = pad_sequence(text_list, batch_first=True)
    return label_list.to(device), text_list.to(device), text_lengths.to(device)
```

```python
class RNN(nn.Module):
  def __init__(self,vocab_size,embed_dim,hidden_dim,num_layers, pad_idx):
    super(RNN,self).__init__()
    self.num_layers = num_layers
    self.hidden_dim = hidden_dim
    self.embed = nn.Embedding(vocab_size,embed_dim, padding_idx = pad_idx)

    self.rnn = nn.LSTM(input_size=embed_dim, hidden_size = hidden_dim,
                        num_layers = num_layers,bidirectional=True, batch_first=True)

    self.fc = nn.Sequential(nn.Dropout(0.3),nn.Linear(2*hidden_dim,1),nn.Sigmoid())

  def forward(self,x,h0, text_lengths):
    x = self.embed(x)
    x = nn.utils.rnn.pack_padded_sequence(x, text_lengths.to('cpu'), batch_first=True, enforce_sorted=False)
    x, (hidden, cell) = self.rnn(x,h0)
    x, output_lengths = nn.utils.rnn.pad_packed_sequence(x, batch_first=True)
    hidden = self.fc(torch.cat((hidden[-2,:,:], hidden[-1,:,:]), dim = 1))
    return hidden

  def init_hidden(self, batch_size):
        ''' Initializes hidden state '''
        h0 = torch.zeros((self.num_layers*2,batch_size,self.hidden_dim)).to(device)
        c0 = torch.zeros((self.num_layers*2,batch_size,self.hidden_dim)).to(device)
        hidden = (h0,c0)
        return hidden
```

```python
def binary_accuracy(preds, y):
    """
    Returns accuracy per batch, i.e. if you get 8/10 right, this returns 0.8, NOT 8
    """

    #round predictions to the closest integer
    rounded_preds = torch.round((preds))
    correct = (rounded_preds == y).float() #convert into float for division
    acc = correct.sum() / len(correct)
    return acc
```

```python
def train(epochs,model, train_dataloader, val_dataloader=None):
  model = model
  learn_rate = 1e-3
  n_epochs = epochs
  #Loss function and Optimizer
  criterion = nn.BCELoss()
  optimizer = torch.optim.Adam(model.parameters(),lr=learn_rate)

  #Training Loop
  train_loss = []
  val_loss = []
  metric = []
  for epoch in range(n_epochs):  # loop over the dataset multiple times
      epoch_loss = 0.0
      running_loss = 0.0
      for i, (label, text, text_lengths) in enumerate(train_dataloader):
          # zero the parameter gradients
          h = model.init_hidden(label.shape[0])
          optimizer.zero_grad()
          # forward + backward + optimize
          predicted_label = model(text,h,text_lengths)
          loss = criterion(predicted_label.squeeze(), label.float())
          loss.backward()
          optimizer.step()
          epoch_loss += loss.item()
          # print statistics
          running_loss += loss.item()

          if i % 10 == 9:    # print every 50 mini-batches
              print('[%d, %5d] loss: %.3f' %
                    (epoch + 1, i + 1, running_loss / 10))
              train_loss.append(running_loss/10)
              running_loss = 0.0
      #train_loss.append(epoch_loss/(i+1))
      #Evaluation of the trained model
      epoch_loss = 0.0
      running_loss = 0.0
      acc = 0.0
      if val_dataloader:
        print("validating...")
        with torch.no_grad():
            for i, (label, text, text_lengths) in enumerate(val_dataloader):
                val_h = model.init_hidden(label.shape[0])
                predicted_label = model(text, val_h,text_lengths)
                loss = criterion(predicted_label.squeeze(), label.float())
                acc += binary_accuracy(predicted_label.squeeze(), label.float()).item()
                running_loss += loss.item()
                if i % 10 == 9:    # print every 50 mini-batches
                  print('[%d, %5d] loss: %.3f' %
                        (epoch + 1, i + 1, running_loss / 10))
                  val_loss.append(running_loss/10)
                  running_loss = 0.0
        acc = acc/(i+1)
        print('Accuracy of the network on the test reviews: %d %%' % (acc*100))
        metric.append(acc)
        #val_loss.append(epoch_loss/(i+1))
  return train_loss, val_loss, metric
```

```python
vocab_size = len(vocab)
embed_dim = 100
hidden_dim = 32
num_layers = 2
pad_idx = vocab.stoi['pad']
#initialize RNN
model = RNN(vocab_size,embed_dim,hidden_dim,num_layers,pad_idx).to(device)
print(model)
def count_parameters(model):
    return sum(p.numel() for p in model.parameters() if p.requires_grad)

print(f'The model has {count_parameters(model):,} trainable parameters')
```

```python
pretrained_embeddings = vocab.vectors

print(pretrained_embeddings.shape)
```

```python
model.embed.weight.data.copy_(pretrained_embeddings)
```

```python
UNK_IDX = vocab.stoi['unk']

model.embed.weight.data[UNK_IDX] = torch.zeros(embed_dim)
model.embed.weight.data[pad_idx] = torch.zeros(embed_dim)

print(model.embed.weight.data)
```

```python
#Dataloaders
train_iter, test_iter = datasets.IMDB()
train_dataset = list(train_iter)
test_dataset = list(test_iter)
num_train = int(len(train_dataset) * 0.6)
batch_size = 128
split_train_, split_valid_ = \
    random_split(train_dataset, [num_train, len(train_dataset) - num_train])

train_dataloader = DataLoader(split_train_, batch_size=batch_size, collate_fn=collate_batch,
                              shuffle=True)
valid_dataloader = DataLoader(split_valid_, batch_size=batch_size, collate_fn=collate_batch,
                              shuffle=False)
test_dataloader = DataLoader(test_dataset, batch_size=batch_size, collate_fn=collate_batch,
                             shuffle=True)
```

```python
train_loss, val_loss, metric = train(epochs=1, model=model, train_dataloader=train_dataloader, val_dataloader=valid_dataloader)
```

```python
import spacy
nlp = spacy.load('en_core_web_sm')

def predict_sentiment(model, sentence):
    model.eval()
    tokenized = [tok.text for tok in nlp.tokenizer(sentence)]
    indexed = [vocab.stoi[t] for t in tokenized]
    length = [len(indexed)]
    tensor = torch.LongTensor(indexed).to(device)
    tensor = tensor.unsqueeze(0)
    length_tensor = torch.LongTensor(length)
    prediction = model(tensor,model.init_hidden(1), length_tensor)
    if torch.round(prediction) == 0:
      print('negative review')
    else:
      print('positive review')
    return prediction.item()
```

```python
predict_sentiment(model, "Should be fired and sued for destroying justice league movie.")
```

```python
predict_sentiment(model, "Great movie, i liked the acting and the music.")
```

#Exercises

## 1. Create a Name Generator using a LSTM

```python

```

## 2. Create a Multi-Class Text Classifier

For this task we will use the Yahoo Answers dataset, it has questions and 10 categories:

For this problem we will use a similar LSTM model as before but with some differences:

* We will use the CrossEntropyLoss.
* The output dimension of the linear layers will be 10.
* We need to remove the squeeze() in the prediced labels when calculating the loss.
* Remove the conversion t float() of the labels.
* Remove the activation of the final linear layer, CrossEntropyLoss does logSoftmax and Negative log-likelihood combined.

```python
import torch
import torch.nn as nn
import torch.nn.functional as F
from torchtext import datasets
import numpy as np
import matplotlib.pyplot as plt
from torch.utils.data import DataLoader
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
from torchtext.data.utils import get_tokenizer
from collections import Counter
from torchtext.vocab import Vocab
from torch.utils.data.dataset import random_split
from torch.nn.utils.rnn import pad_sequence
```

```python
train_iter, test_iter = datasets.YahooAnswers()
```

```python
max_vocab_size = 25000
#tokenizer = get_tokenizer('spacy')
counter = torch.load('./counter_yahoo.py')
tokenizer = get_tokenizer('spacy')
'''if counter is None:
  counter = Counter()
  for (label, line) in train_iter:
      counter.update(tokenizer(line))'''
vocab = Vocab(counter, min_freq=1, max_size=max_vocab_size, vectors="glove.6B.100d",
              unk_init =torch.Tensor.normal_)
```

```python
next(test_iter)
```

    (9,
     "What makes friendship click? How does the spark keep going? good communication is what does it.  Can you move beyond small talk and say what's really on your mind.  If you start doing this, my expereince is that potentially good friends will respond or shun you.  Then you know who the really good friends are.")

```python
text_pipeline = lambda x: [vocab[token] for token in tokenizer(x)]
label_pipeline = lambda x: int(x) - 1

def collate_batch(batch):
    label_list, text_list, text_lengths = [], [], []
    for (_label, _text) in batch:
         label_list.append(label_pipeline(_label))
         processed_text = torch.tensor(text_pipeline(_text), dtype=torch.int64)
         text_list.append((processed_text))
         text_lengths.append(len(processed_text))
    label_list = torch.tensor(label_list, dtype=torch.int64)
    text_lengths = torch.tensor(text_lengths, dtype=torch.int64)
    text_list = pad_sequence(text_list, batch_first=True)
    return label_list.to(device), text_list.to(device), text_lengths.to(device)
```

```python
[vocab[token] for token in ['here', 'is', 'an', 'example']]
```

    [193, 11, 59, 387]

```python
# Build your LSTM model

"To DO"
```

```python
def categorical_accuracy(preds, y):
    """
    Returns accuracy per batch, i.e. if you get 8/10 right, this returns 0.8, NOT 8
    """
    top_pred = preds.argmax(1, keepdim = True)
    correct = top_pred.eq(y.view_as(top_pred)).sum()
    acc = correct.float() / y.shape[0]
    return acc
```

```python
def train(epochs,model, train_dataloader, val_dataloader=None):
  model = model
  learn_rate = 1e-3
  n_epochs = epochs

  #Loss function and Optimizer
  criterion = #define loss function
  optimizer = torch.optim.Adam(model.parameters(),lr=learn_rate)

  #Training Loop
  train_loss = []
  val_loss = []
  metric = []
  for epoch in range(n_epochs):  # loop over the dataset multiple times
      epoch_loss = 0.0
      running_loss = 0.0
      for i, (label, text, text_lengths) in enumerate(train_dataloader):
          # zero the parameter gradients
          h = model.init_hidden(label.shape[0])
          optimizer.zero_grad()
          # forward + backward + optimize
          predicted_label = model(text,h,text_lengths)
          loss = criterion(predicted_label, label)
          loss.backward()
          optimizer.step()
          epoch_loss += loss.item()
          # print statistics
          running_loss += loss.item()

          if i % 10 == 9:    # print every 10 mini-batches
              print('[%d, %5d] loss: %.3f' %
                    (epoch + 1, i + 1, running_loss / 50))
              running_loss = 0.0
      train_loss.append(epoch_loss/(i+1))
      #Evaluation of the trained model
      epoch_loss = 0.0
      acc = 0.0
      if val_dataloader:
        print("validating...")
        with torch.no_grad():
            for i, (label, text, text_lengths) in enumerate(val_dataloader):
                val_h = model.init_hidden(label.shape[0])
                predicted_label = model(text, val_h,text_lengths)
                loss = criterion(predicted_label, label)
                acc += categorical_accuracy(predicted_label, label)
                epoch_loss += loss.item()
        acc = acc.item()/(i+1)
        print('Accuracy of the network on the test reviews: %d %%' % (acc*100))
        metric.append(acc)
        val_loss.append(epoch_loss/(i+1))
  return train_loss, val_loss, metric
```

```python
vocab_size = len(vocab)
embed_dim = 100
hidden_dim = 32
num_layers = 2
batch_size = 256
pad_idx = vocab.stoi['pad']
#initialize RNN
model = text_classificator(vocab_size,embed_dim,hidden_dim,num_layers,pad_idx).to(device)
print(model)
def count_parameters(model):
    return sum(p.numel() for p in model.parameters() if p.requires_grad)

print(f'The model has {count_parameters(model):,} trainable parameters')
```

```python
pretrained_embeddings = vocab.vectors

print(pretrained_embeddings.shape)
```

```python
model.embed.weight.data.copy_(pretrained_embeddings)
```

```python
UNK_IDX = vocab.stoi['unk']

model.embed.weight.data[UNK_IDX] = torch.zeros(embed_dim)
model.embed.weight.data[pad_idx] = torch.zeros(embed_dim)

print(model.embed.weight.data)
```

```python
#Dataloaders
train_iter, test_iter = datasets.YahooAnswers()
train_dataset = list(train_iter)
test_dataset = list(test_iter)
num_train = int(len(train_dataset) * 0.6)
split_train_, split_valid_ = \
    random_split(train_dataset, [num_train, len(train_dataset) - num_train])

train_dataloader = DataLoader(split_train_, batch_size=batch_size, collate_fn=collate_batch,
                              shuffle=True)
valid_dataloader = DataLoader(split_valid_, batch_size=batch_size, collate_fn=collate_batch,
                              shuffle=False)
test_dataloader = DataLoader(test_dataset, batch_size=batch_size, collate_fn=collate_batch,
                             shuffle=True)
```

```python
train_loss, val_loss, metric = train(epochs=1, model=model, train_dataloader=train_dataloader, val_dataloader=valid_dataloader)
```

```python
label, text = test_dataset[np.random.randint(len(test_dataset))]
```

```python
import spacy
nlp = spacy.load('en_core_web_sm')

def predict_sentiment(model, sentence):
    model.eval()
    tokenized = [tok.text for tok in nlp.tokenizer(sentence)]
    indexed = [vocab.stoi[t] for t in tokenized]
    length = [len(indexed)]
    tensor = torch.LongTensor(indexed).to(device)
    tensor = tensor.unsqueeze(0)
    length_tensor = torch.LongTensor(length)
    prediction = model(tensor,model.init_hidden(1), length_tensor)
    return prediction.argmax(1).item() + 1
```

```python
print(text)
print(predict_sentiment(model,text))
print(label)
```