# Direct Inversion of the Iterative Subspace

When solving systems of linear (or nonlinear) equations, iterative methods are often employed.  Unfortunately, such methods often suffer from convergence issues such as numerical instability, slow convergence, and significant computational expense when applied to difficult problems.  In these cases, convergence accelleration methods may be applied to both speed up, stabilize and/or reduce the cost for the convergence patterns of these methods, so that solving such problems become computationally tractable.  One such method is known as the direct inversion of the iterative subspace (DIIS) method, which is commonly applied to address convergence issues within self consistent field computations in Hartree-Fock theory (and other iterative electronic structure methods).  In this tutorial, we'll introduce the theory of DIIS for a general iterative procedure, before integrating DIIS into our previous implementation of RHF.

## I. Theory

DIIS is a widely applicable convergence acceleration method, which is applicable to numerous problems in linear algebra and the computational sciences, as well as quantum chemistry in particular.  Therefore, we will introduce the theory of this method in the general sense, before seeking to apply it to SCF.

Suppose that for a given problem, there exist a set of trial vectors $\{\mid{\bf p}_i\,\rangle\}$ which have been generated iteratively, converging toward the true solution, $\mid{\bf p}^f\,\rangle$.  Then the true solution can be approximately constructed as a linear combination of the trial vectors,
$$\mid{\bf p}\,\rangle = \sum_ic_i\mid{\bf p}_i\,\rangle,$$
where we require that the residual vector
$$\mid{\bf r}\,\rangle = \sum_ic_i\mid{\bf r}_i\,\rangle\,;\;\;\; \mid{\bf r}_i\,\rangle
=\, \mid{\bf p}_{i+1}\,\rangle - \mid{\bf p}_i\,\rangle$$
is a least-squares approximate to the zero vector, according to the constraint
$$\sum_i c_i = 1.$$
This constraint on the expansion coefficients can be seen by noting that each trial function ${\bf p}_i$ may be represented as an error vector applied to the true solution, $\mid{\bf p}^f\,\rangle + \mid{\bf e}_i\,\rangle$.  Then
\begin{align}
\mid{\bf p}\,\rangle &= \sum_ic_i\mid{\bf p}_i\,\rangle\\
&= \sum_i c_i(\mid{\bf p}^f\,\rangle + \mid{\bf e}_i\,\rangle)\\
&= \mid{\bf p}^f\,\rangle\sum_i c_i + \sum_i c_i\mid{\bf e}_i\,\rangle
\end{align}
Convergence results in a minimization of the error (causing the second term to vanish); for the DIIS solution vector $\mid{\bf p}\,\rangle$ and the true solution vector $\mid{\bf p}^f\,\rangle$ to be equal, it must be that $\sum_i c_i = 1$.  We satisfy our condition for the residual vector by minimizing its norm,
$$\langle\,{\bf r}\mid{\bf r}\,\rangle = \sum_{ij} c_i^* c_j \langle\,{\bf r}_i\mid{\bf r}_j\,\rangle,$$
using Lagrange's method of undetermined coefficients subject to the constraint on $\{c_i\}$:
$${\cal L} = {\bf c}^{\dagger}{\bf Bc} - \lambda\left(1 - \sum_i c_i\right)$$
where $B_{ij} = \langle {\bf r}_i\mid {\bf r}_j\rangle$ is the matrix of residual vector overlaps.  Minimization of the Lagrangian with respect to the coefficient $c_k$ yields (for real values)
\begin{align}
\frac{\partial{\cal L}}{\partial c_k} = 0 &= \sum_j c_jB_{jk} + \sum_i c_iB_{ik} - \lambda\\
&= 2\sum_ic_iB_{ik} - \lambda
\end{align}
which has matrix representation
\begin{equation}
\begin{pmatrix}
  B_{11} & B_{12} & \cdots & B_{1m} & -1 \\
  B_{21} & B_{22} & \cdots & B_{2m} & -1 \\
  \vdots  & \vdots  & \ddots & \vdots  & \vdots \\
  B_{n1} & B_{n2} & \cdots & B_{nm} & -1 \\
  -1 & -1 & \cdots & -1 & 0
\end{pmatrix}
\begin{pmatrix}
c_1\\
c_2\\
\vdots \\
c_n\\
\lambda
\end{pmatrix}
=
\begin{pmatrix}
0\\
0\\
\vdots\\
0\\
-1
\end{pmatrix},
\end{equation}

which we will refer to as the Pulay equation, named after the inventor of DIIS.  It is worth noting at this point that our trial vectors, residual vectors, and solution vector may in fact be tensors of arbitrary rank; it is for this reason that we have used the generic notation of Dirac in the above discussion to denote the inner product between such objects.

## II. Algorithms for DIIS
The general DIIS procedure, as described above, has the following structure during each iteration:
#### Algorithm 1: Generic DIIS procedure
1. Compute new trial vector, $\mid{\bf p}_{i+1}\,\rangle$, append to list of trial vectors
2. Compute new residual vector, $\mid{\bf r}_{i+1}\,\rangle$, append to list of trial vectors
3. Check convergence criteria
    - If RMSD of $\mid{\bf r}_{i+1}\,\rangle$ sufficiently small, and
    - If change in DIIS solution vector $\mid{\bf p}\,\rangle$ sufficiently small, break
4. Build **B** matrix from previous residual vectors
5. Solve Pulay equation for coefficients $\{c_i\}$
6. Compute DIIS solution vector $\mid{\bf p}\,\rangle$

For SCF iteration, the most common choice of trial vector is the Fock matrix **F**; this choice has the advantage over other potential choices (e.g., the density matrix **D**) of **F** not being idempotent, so that it may benefit from extrapolation.  The residual vector is commonly chosen to be the orbital gradient in the AO basis,
$$g_{\mu\nu} = ({\bf FDS} - {\bf SDF})_{\mu\nu},$$
however the better choice (which we will make in our implementation!) is to orthogonormalize the basis of the gradient with the inverse overlap metric ${\bf A} = {\bf S}^{-1/2}$:
$$r_{\mu\nu} = ({\bf A}^{\rm T}({\bf FDS} - {\bf SDF}){\bf A})_{\mu\nu}.$$
Therefore, the SCF-specific DIIS procedure (integrated into the SCF iteration algorithm) will be:
#### Algorithm 2: DIIS within an SCF Iteration
1. Compute **F**, append to list of previous trial vectors
2. Compute AO orbital gradient **r**, append to list of previous residual vectors
3. Compute RHF energy
3. Check convergence criteria
    - If RMSD of **r** sufficiently small, and
    - If change in SCF energy sufficiently small, break
4. Build **B** matrix from previous AO gradient vectors
5. Solve Pulay equation for coefficients $\{c_i\}$
6. Compute DIIS solution vector **F_DIIS** from $\{c_i\}$ and previous trial vectors
7. Compute new orbital guess with **F_DIIS**

## III. Implementation

In order to implement DIIS, we're going to integrate it into an existing RHF program.  Since we just-so-happened to write such a program in the last tutorial, let's re-use the part of the code before the SCF integration which won't change when we include DIIS:

```julia
# ==> Basic Setup <==
# Import statements
using PyCall: pyimport
psi4 = pyimport("psi4")
np   = pyimport("numpy") # used only to cast to Psi4 arrays
using TensorOperations: @tensor
using LinearAlgebra: Diagonal, Hermitian, eigen, tr, norm, dot
using Printf: @printf

# Memory specification
psi4.set_memory(Int(5e8))
numpy_memory = 2

# Set output file
psi4.core.set_output_file("output.dat", false)

# Define Physicist's water -- don't forget C1 symmetry!
mol = psi4.geometry("""
O
H 1 1.1
H 1 1.1 2 104
symmetry c1
""")

# Set computation options
psi4.set_options(Dict("basis" => "cc-pvdz",
                      "scf_type" => "pk",
                      "e_convergence" => 1e-8))

# Maximum SCF iterations
MAXITER = 40
# Energy convergence criterion
E_conv = 1.0e-6
D_conv = 1.0e-3
```

      Memory set to 476.837 MiB by Python driver.

    0.001

```julia
# ==> Static 1e- & 2e- Properties <==
# Class instantiation
wfn = psi4.core.Wavefunction.build(mol, psi4.core.get_global_option("basis"))
mints = psi4.core.MintsHelper(wfn.basisset())

# Overlap matrix
S = np.asarray(mints.ao_overlap()) # we only need a copy

# Number of basis Functions & doubly occupied orbitals
nbf = size(S)[1]
ndocc = wfn.nalpha()

println("Number of occupied orbitals: ", ndocc)
println("Number of basis functions: ", nbf)

# Memory check for ERI tensor
I_size = nbf^4 * 8.e-9
println("\nSize of the ERI tensor will be $I_size GB.")
memory_footprint = I_size * 1.5
if I_size > numpy_memory
    psi4.core.clean()
    throw(OutOfMemoryError("Estimated memory utilization ($memory_footprint GB) exceeds " *
                           "allotted memory limit of $numpy_memory GB."))
end

# Build ERI Tensor
I = np.asarray(mints.ao_eri()) # we only need a copy

# Build core Hamiltonian
T = np.asarray(mints.ao_kinetic()) # we only need a copy
V = np.asarray(mints.ao_potential()) # we only need a copy
H = T + V;
```

    Number of occupied orbitals: 5
    Number of basis functions: 24

    Size of the ERI tensor will be 0.0026542080000000003 GB.

```julia
# ==> CORE Guess <==
# AO Orthogonalization Matrix
A = mints.ao_overlap()
A.power(-0.5, 1.e-16) # ≈ Julia's A^(-0.5) after psi4view()
A = np.asarray(A)

# Transformed Fock matrix
F_p = A * H * A

# Diagonalize F_p for eigenvalues & eigenvectors with Julia
e, C_p = eigen(Hermitian(F_p))

# Transform C_p back into AO basis
C = A * C_p

# Grab occupied orbitals
C_occ = C[:, 1:ndocc]

# Build density matrix from occupied orbitals
D = C_occ * C_occ'

# Nuclear Repulsion Energy
E_nuc = mol.nuclear_repulsion_energy()
```

    8.002366482173422

Now let's put DIIS into action.  Before our iterations begin, we'll need to create empty lists to hold our previous residual vectors (AO orbital gradients) and trial vectors (previous Fock matrices), along with setting starting values for our SCF energy and previous energy:

```julia
# ==> Pre-Iteration Setup <==
# SCF & Previous Energy
SCF_E = 0.0
E_old = 0.0;
```

Now we're ready to write our SCF iterations according to Algorithm 2.  Here are some hints which may help you along the way:

#### Starting DIIS
Since DIIS builds the approximate solution vector $\mid{\bf p}\,\rangle$ as a linear combination of the previous trial vectors $\{\mid{\bf p}_i\,\rangle\}$, there's no need to perform DIIS on the first SCF iteration, since there's only one trial vector for DIIS to use!

#### Building **B**
1. The **B** matrix in the Lagrange equation is really $\tilde{\bf B} = \begin{pmatrix} {\bf B} & -1\\ -1 & 0\end{pmatrix}$.
2. Since **B** is the matrix of residual overlaps, it will be a square matrix of dimension equal to the number of residual vectors.  If **B** is an $N\times N$ matrix, how big is $\tilde{\bf B}$?
3. Since our residuals are real, **B** will be a symmetric matrix.
4. To build $\tilde{\bf B}$, make an empty array of the appropriate dimension, then use array indexing to set the values of the elements.

#### Solving the Pulay equation
1. Use built-in Julia functionality to make your life easier.
2. The solution vector for the Pulay equation is $\tilde{\bf c} = \begin{pmatrix} {\bf c}\\ \lambda\end{pmatrix}$, where $\lambda$ is the Lagrange multiplier, and the right hand side is $\begin{pmatrix} {\bf 0}\\ -1\end{pmatrix}$.

```julia
# Start from fresh orbitals
F_p = A * H * A
e, C_p = eigen(Hermitian(F_p))
C = A * C_p
C_occ = C[:, 1:ndocc]
D = C_occ * C_occ' ;

# Trial & Residual Vector Lists
F_list = []
DIIS_RESID = []

# ==> SCF Iterations w/ DIIS <==
println("==> Starting SCF Iterations <==")
SCF_E = let SCF_E = SCF_E, E_old = E_old, D = D

   # Begin Iterations
   for scf_iter in 1:MAXITER
      # Build Fock matrix
      @tensor G[p,q] := (2I[p,q,r,s] - I[p,r,q,s]) * D[r,s]
      F = H + G

      # Build DIIS Residual
      diis_r = A * (F * D * S - S * D * F) * A

      # Append trial & residual vectors to lists
      push!(F_list, F)
      push!(DIIS_RESID, diis_r)

      # Compute RHF energy
      SCF_E = tr((H + F) * D) + E_nuc
      dE = SCF_E - E_old
      dRMS = norm(diis_r)
      @printf("SCF Iteration %3d: Energy = %4.16f dE = %1.5e dRMS = %1.5e \n",
                          scf_iter,        SCF_E, SCF_E - E_old,     dRMS)

      # SCF Converged?
      if abs(SCF_E - E_old) < E_conv && dRMS < D_conv
          break
      end
      E_old = SCF_E

      if scf_iter >= 2
         # Build B matrix
         B_dim = length(F_list) + 1
         B = zeros(B_dim, B_dim)
         B[end,   :] .= -1
         B[:  , end] .= -1
         B[end, end]  =  0
         for i in eachindex(F_list), j in eachindex(F_list)
            B[i, j] = dot(DIIS_RESID[i],  DIIS_RESID[j])
         end

         # Build RHS of Pulay equation
         rhs = zeros(B_dim)
         rhs[end] = -1

         # Solve Pulay equation for c_i's with Julia
         coeff = B \ rhs

         # Build DIIS Fock matrix
         F = zeros(size(F))
         for i in 1:length(coeff) - 1
            F += coeff[i] * F_list[i]
         end
      end

      # Compute new orbital guess with DIIS Fock matrix
      F_p =  A * F * A
      e, C_p = eigen(Hermitian(F_p))
      C = A * C_p
      C_occ = C[:, 1:ndocc]
      D = C_occ * C_occ'

      # MAXITER exceeded?
      if scf_iter == MAXITER
          psi4.core.clean()
          throw(MethodError("Maximum number of SCF iterations exceeded."))
      end
   end
   SCF_E
end

# Post iterations
println("\nSCF converged.")
println("Final RHF Energy: $SCF_E [Eh]")
```

    ==> Starting SCF Iterations <==
    SCF Iteration   1: Energy = -68.9800327333871053 dE = -6.89800e+01 dRMS = 2.79722e+00
    SCF Iteration   2: Energy = -69.6472544393141675 dE = -6.67222e-01 dRMS = 2.57832e+00
    SCF Iteration   3: Energy = -75.7919291462249021 dE = -6.14467e+00 dRMS = 6.94257e-01
    SCF Iteration   4: Energy = -75.9721892296710735 dE = -1.80260e-01 dRMS = 1.81547e-01
    SCF Iteration   5: Energy = -75.9893690602362710 dE = -1.71798e-02 dRMS = 2.09996e-02
    SCF Iteration   6: Energy = -75.9897163367029123 dE = -3.47276e-04 dRMS = 1.28546e-02
    SCF Iteration   7: Energy = -75.9897932415930768 dE = -7.69049e-05 dRMS = 1.49088e-03
    SCF Iteration   8: Energy = -75.9897956274068349 dE = -2.38581e-06 dRMS = 6.18909e-04
    SCF Iteration   9: Energy = -75.9897957845313954 dE = -1.57125e-07 dRMS = 4.14761e-05

    SCF converged.
    Final RHF Energy: -75.9897957845314 [Eh]

Congratulations! You've written your very own Restricted Hartree-Fock program with DIIS convergence accelleration!  Finally, let's check your final RHF energy against  Psi4:

```julia
# Compare to Psi4
SCF_E_psi = psi4.energy("SCF")
psi4.compare_values(SCF_E_psi, SCF_E, 6, "SCF Energy")
```

    	SCF Energy........................................................PASSED

    true

## References
1. P. Pulay. *Chem. Phys. Lett.* **73**, 393-398 (1980)
2. C. David Sherrill. *"Some comments on accellerating convergence of iterative sequences using direct inversion of the iterative subspace (DIIS)".* Available at: vergil.chemistry.gatech.edu/notes/diis/diis.pdf. (1998)


# Defining parameter relations: linked and optimization parameters

In the fitting of multi-compartment models, it is often useful or necessary to impose certain constraints on the optimization of certain model parameters. For example, if you have a-priori knowledge on the values of some of the model parameter, then does not make sense to still fit it from the data. Or, if you know something about the structure of the data, then you want to constrain the relationships between your model parameters in a way that respects this structure.

Dmipy allows a user to set any parameter constraint or relation for any model combination. Here, we make a distinction between setting *linked* parameters and *optimization* parameters.

Setting a **linked parameter** means making one parameter's value dependent on the values of others (or on a fixed value). This means that after imposing a linked parameter, the model will not optimize on the linked parameter anymore (as it will be inferred from the others internally).

Setting an **optimization parameter** means *replacing* a parameter with a new, custom one. In the simplest case this can be some rescaling of the replaced parameter that will help the optimization find a solution the user prefers. However, like linked parameters, optimization parameters can also be make a function of other model parameters.

Below, we provide examples of typically already-included functions to set linked and optimization parameters, as well as explain how to say your own custom functions.

## Linked parameters: tortuous, equal and fixed parameters

A linked parameter can be seen as somehow simplifying the problem by injecting already-known information into the optimization process. In Microstructure Imaging, often-used parameter links are imposing **axon tortuosity**, **parameter equality** and **fixed parameters**.

**Axon Tortuosity**: The idea behind axon tortuosity is that the perpendicular, extra-axonal diffusivity between a set of axons (modeled as parallel cylinders) is dependent on the overall water diffusivity of the medium, and the density of the cylinders. Mathematically, axon tortuosity is imposed as

$$\lambda_\perp=(1-vf)\lambda_\parallel$$

where $\lambda_\perp$ and $\lambda_\parallel$ are the perpendicular extra-axonal and parallel intra-axonal diffusivity, and $vf$ is the intra-axonal volume fraction. If $vf=0$, then both diffusivities are equal. If $vf=1$, then $\lambda_\perp=0$.

**Parameter Equality**: As an example, it is often assumed that the parallel diffusivity of the intra- and extra-axonal models are the same.

**Fixing Parameters**: To avoid fitting them from the data, the diffusivities of different models are often fixed to some reasonable values.

### Imposing Axon Tortuosity
To impose axon tortuosity we first define a multi-compartment model that models the intra- and extra-axonal space. We can define axon tortuosity both in the dispersed model and multi-compartment model representation:

```python
# setting tortuosity in the dispersed model representation
from dmipy.signal_models import gaussian_models, cylinder_models
from dmipy.distributions import distribute_models

stick = cylinder_models.C1Stick()
zeppelin = gaussian_models.G2Zeppelin()
watson_bundle = distribute_models.SD1WatsonDistributed(models=[stick, zeppelin])
watson_bundle.parameter_names
```

    ['G2Zeppelin_1_lambda_perp',
     'SD1Watson_1_odi',
     'G2Zeppelin_1_lambda_par',
     'SD1Watson_1_mu',
     'C1Stick_1_lambda_par',
     'partial_volume_0']

We can now call set_tortuous_parameter($\lambda_\perp$, $\lambda_\parallel$, vf) with the appropriate parameter names to set the tortuos parameter link

```python
watson_bundle.set_tortuous_parameter('G2Zeppelin_1_lambda_perp', 'G2Zeppelin_1_lambda_par', 'partial_volume_0')
watson_bundle.parameter_names
```

    ['SD1Watson_1_odi',
     'G2Zeppelin_1_lambda_par',
     'SD1Watson_1_mu',
     'C1Stick_1_lambda_par',
     'partial_volume_0']

It can be seen that $\lambda_\perp$ is now removed from the list of parameters, as it has been made dependent on $\lambda_\parallel$ and $vf$.

### Imposing Parameter Equality
Next, we impose that the parallel diffusivity is the same between the Stick (intra-axonal) and Zeppelin (extra-axonal) model. For this, we use set_equal_parameter function($\lambda_{\parallel, Stick}$, $\lambda_{\parallel, Zeppelin}$)

```python
watson_bundle.set_equal_parameter('C1Stick_1_lambda_par', 'G2Zeppelin_1_lambda_par')
watson_bundle.parameter_names
```

    ['SD1Watson_1_odi',
     'SD1Watson_1_mu',
     'C1Stick_1_lambda_par',
     'partial_volume_0']

Notice how the Zeppelin's $\lambda_\parallel$ was now removed.

### Fixing Parameter Values
Finally, models like NODDI *(Zhang et al. 2012)* also fix the model's diffuvity to avoid fitting them from the data. A value that is typically used for intra-axonal diffusvity is $\lambda=1.7e-9m^2/s$. As an example, we will fix the Stick's $\lambda_\parallel$ to this value using set_fixed_parameter($\lambda_\parallel$, fixed-value)

```python
watson_bundle.set_fixed_parameter('C1Stick_1_lambda_par', 1.7e-9)
```

Which leaves us with 3 parameters left.

```python
watson_bundle.parameter_names
```

    ['SD1Watson_1_odi', 'SD1Watson_1_mu', 'partial_volume_0']

The watson_bundle can finally be used in a MultiCompartment model like any other, after which it can be used for simulating and fitting data.

```python
from dmipy.core.modeling_framework import MultiCompartmentModel
mcmodel = MultiCompartmentModel(models=[watson_bundle])
mcmodel.parameter_names
```

    ['SD1WatsonDistributed_1_SD1Watson_1_mu',
     'SD1WatsonDistributed_1_partial_volume_0',
     'SD1WatsonDistributed_1_SD1Watson_1_odi']

Once the multi-compartment model is made, simulating and fitting data is exactly as in the previous tutorials.

# Defining optimization parameters: fractional parameters

### Imposing Fractional Parameters

There are situations in which one wants to ensure that one parameter is always smaller or equal than another. For example, when only fitting a Zeppelin model to the data - like in the Spherical Mean Technique, see [example](http://nbviewer.jupyter.org/github/AthenaEPI/dmipy/blob/master/examples/example_spherical_mean_technique.ipynb) - the perpendicular diffusivity is imposed to be smaller than the parallel diffusivity, i.e. $\lambda_\perp\leq\lambda_\parallel$.

```python
from dmipy.core.modeling_framework import MultiCompartmentSphericalMeanModel

smt_mod = MultiCompartmentSphericalMeanModel([zeppelin])
smt_mod.parameter_names
```

    ['G2Zeppelin_1_lambda_perp', 'G2Zeppelin_1_lambda_par']

To impose this "smaller or equal than" constraint we use the set_fractional_parameter($\lambda_\perp$, $\lambda_\parallel$) function, where the former parameter is now constrained to smaller than the latter.

```python
smt_mod.set_fractional_parameter('G2Zeppelin_1_lambda_perp',  'G2Zeppelin_1_lambda_par')
smt_mod.parameter_names
```

    ['G2Zeppelin_1_lambda_par', 'G2Zeppelin_1_lambda_perp_fraction']

Now the original "lambda_perp" parameter has been replaced by a "lambda_perp_fraction" parameter. This new parameter ranges between [0-1] and is internally interpreted as

\begin{equation}
\lambda_\perp = \lambda_{\perp, fraction}\times\lambda_\parallel
\end{equation}

such that the smaller or equal than contraint is effectively enforced in the model.


# Random Signals

*This jupyter notebook is part of a [collection of notebooks](../index.ipynb) on various topics of Digital Signal Processing. Please direct questions and suggestions to [Sascha.Spors@uni-rostock.de](mailto:Sascha.Spors@uni-rostock.de).*

## Ensemble Averages

Ensemble averages characterize the average properties of a random process across the population of all possible sample functions in the ensemble. We distinguish between first- and higher-order ensemble averages. The former consider the average properties of the sample functions of one random process for one particular time-instant $k$, while the latter take more than one random process at different time-instants into account.

### First-Order Ensemble Averages

#### Definition

The first-order ensemble average of a continuous-amplitude real-valued random signal $x[k]$ is defined as

\begin{equation}
E\{ f(x[k]) \} = \lim_{N \to \infty} \frac{1}{N} \sum_{n=0}^{N-1} f(x_n[k])
\end{equation}

where $E\{ \cdot \}$ denotes the expectation operator, $x_n[k]$ the $n$-th sample function and $f(\cdot)$ an arbitrary mapping function. It is evident from the definition that the ensemble average can only be given exactly for random processes where the internal structure is known. For practical random processes, like e.g. speech, the ensemble average can only be approximated by a finite but sufficiently large number $N$ of sample functions.

The ensemble average can also be given in terms of the univariate probability density function (PDF)

\begin{equation}
E\{ f(x[k]) \} = \int\limits_{-\infty}^{\infty} f(\theta) \, p_x(\theta, k) \, \mathrm{d}\theta
\end{equation}

#### Properties

The following properties can be concluded from the definition of the ensemble average:

1. The ensemble averages of two different time-instants $k_1$ and $k_2$ differ in general

    \begin{equation}
    E\{ f(x[k_1]) \} \neq E\{ f(x[k_2]) \}
    \end{equation}

2. For a linear mapping $f(x[k]) = C \cdot x[k]$ with $C \in \mathbb{R} \setminus \{0\}$, the ensemble average is a linear operation

    \begin{equation}
    E\{ A \cdot x[k] + B \cdot y[k] \} = A \cdot E\{ x[k] \} + B \cdot E\{ y[k] \}
    \end{equation}

3. For a deterministic signal $x_n[k] = s[k]$ $\forall n$ the ensemble average is

    \begin{equation}
    E\{ f(s[k]) \} = f(s[k])
    \end{equation}

The choice of the mapping function $f(\cdot)$ determines the particular property of the random process which is characterized by the ensemble average. Common choices are discussed in the following.

#### Linear mean

The linear [mean](https://en.wikipedia.org/wiki/Mean), which is given for the linear mapping $f(x[k]) = x[k]$, is the arithmetic mean value across all sample functions $x_n[k]$ for a given time instant $k$. It is also known as the first (raw) [moment](https://en.wikipedia.org/wiki/Moment_%28mathematics%29).

Introducing $f(x[k]) = x[k]$ into the definition of the ensemble average yields

\begin{equation}
\mu_x[k] = E\{ x[k] \} = \int\limits_{-\infty}^{\infty} \theta \, p_x(\theta, k) \, \mathrm{d}\theta
\end{equation}

where $\mu_x[k]$ commonly denotes the linear mean. A process with $\mu_x[k] = 0$ is termed as *zero-mean* or *mean-free*. Note that $\mu_x$ should not be confused with the discrete frequency index of the DFT.

#### Quadratic mean

The quadratic mean is given by the mapping $f(x[k]) = x^2[k]$

\begin{equation}
E\{ x^2[k] \} = \int\limits_{-\infty}^{\infty} \theta^2 \, p_x(\theta, k) \, \mathrm{d}\theta
\end{equation}

It quantifies the average instantaneous power of a sample function for a given time index $k$. It is also known as second (raw) moment.

#### Variance

The [variance](https://en.wikipedia.org/wiki/Variance) is defined as the quadratic mean of a zero-mean random process. For a general random process with linear mean $\mu_x$ it is given as

\begin{equation}
\sigma_x^2[k] = E\{ (x[k] - \mu_x[k])^2 \}
\end{equation}

where $\sigma_x^2[k]$ commonly denotes the variance, $\sigma_x[k]$ is known as the [standard deviation](https://en.wikipedia.org/wiki/Standard_deviation). Note that the subtraction of the linear mean $\mu_x$ from the samples $x_n[k]$ makes a random signal mean-free. The variance characterizes how far the amplitude values of a random signal are spread out from its mean value. It is also known as second centralized moment.

The variance can be given in terms of the linear and quadratic mean

\begin{equation}
\sigma_x^2[k] = E\{ x^2[k] \} - \mu_x^2[k]
\end{equation}

**Excercise**

* Derive above relation from the definitions and properties of the first order ensemble average.

Taking the linearity of the expectation operator and the fact that the linear mean is a deterministic value into account yields

\begin{equation}
\begin{split}
\sigma_x^2[k] &= E\{ (x[k] - \mu_x[k])^2 \} \\
&= E\{ x^2[k] - 2 \mu_x[k] x[k] + \mu_x^2[k]  \} \\
&= E\{ x^2[k] \} - 2 \mu_x[k] E\{ x[k] \} + \mu_x^2[k] \\
& = E\{ x^2[k] \} - \mu_x^2[k]
\end{split}
\end{equation}

#### Example - Linear and quadratic mean, and variance of a random process

The following example computes and plots the linear $\mu_x$ and quadratic mean, and variance $\sigma_x^2$ of a random process. Since in practice only a limited number $N$ of sample functions can be evaluated numerically the true values of these quantities are only approximated/estimated. Estimated quantities are denoted by a 'hat' over the respective quantity, e.g. $\hat{\mu}_x[k]$.

```python
%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt

K = 64  # number of random samples
N = 1000  # number of sample functions

# generate the sample functions
np.random.seed(5)
x = np.random.normal(size=(N, K))
x += np.tile(np.cos(2*np.pi/K*np.arange(K)), [N, 1])

# estimate the linear mean as ensemble average
mu = 1/N * np.sum(x, 0)
# estimate the quadratic mean
qu = 1/N * np.sum(x**2, 0)
# estimate the variance
sigma = 1/N * np.sum((x-mu)**2, 0)

# plot results
plt.rc('figure', figsize=(10, 3))

plt.figure()
plt.stem(x[0, :])
plt.title(r'Sample function $x_0[k]$')
plt.xlabel(r'$k$')
plt.ylabel(r'$x_0[k]$')
plt.axis([0, K, -3, 3])

plt.figure()
plt.stem(mu)
plt.title(r'Estimate of linear mean')
plt.xlabel(r'$k$')
plt.ylabel(r'$\hat{\mu}_x[k]$')
plt.axis([0, K, -1.5, 1.5])

plt.figure()
plt.stem(qu)
plt.title(r'Estimate of quadratic mean')
plt.xlabel(r'$k$')
plt.ylabel(r'$\hat{E}\{x^2[k]\}$')
plt.axis([0, K, 0, 2.5])

plt.figure()
plt.stem(sigma)
plt.title(r'Estimate of variance')
plt.xlabel(r'$k$')
plt.ylabel(r'$\hat{\sigma}^2_x[k]$')
plt.axis([0, K, 0, 1.5]);
```

**Exercise**

* What does the linear and quadratic mean, and the variance tell you about the average behavior of the sample functions?
* Change the number `N` of sample functions and rerun the example. What influence has a decrease/increase of the sample functions on the estimated ensemble averages?

Inspection of the estimated linear mean reveals that in average the sample functions follow a cosine function with respect to the sample index $k$. The variance shows that the amount of which the samples of the sample functions for one particular time-instant $k$ are spread around the linear mean is constant. The estimate of the quadratic mean is given as $\hat{E}\{ x^2[k] \} = \hat{\mu}_x^2[k] + \hat{\sigma}_x^2[k]$. The higher the number $N$ of sample functions used for the estimate of the ensemble averages, the lower the uncertainty in comparison to the true values becomes.

### Second-Order Ensemble Averages

#### Definition

The second-order ensemble average of two continuous-amplitude real-valued random signals $x[k]$ and $y[k]$ is defined as

\begin{equation}
E\{ f(x[k_x], y[k_y]) \} := \lim_{N \to \infty} \frac{1}{N} \sum_{n=0}^{N-1} f(x_n[k_x], y_n[k_y])
\end{equation}

It is given in terms of the bivariate PDF as

\begin{equation}
E\{ f(x[k_x], y[k_y]) \} = \iint\limits_{-\infty}^{\infty} f(\theta_x, \theta_y) \, p_{xy}(\theta_x, \theta_y, k_x, k_y) \, \mathrm{d}\theta_x\, \mathrm{d}\theta_y
\end{equation}

The definition of the second-order ensemble average can be extended straightforward to the case of more than two random variables. The resulting ensemble average is then termed as higher-order ensemble average.

By setting $y[k] = x[k]$, the second-order ensemble average can also be used to characterize the average properties of the sample functions between two different time-instants $k_1$ and $k_2$ of one random process

\begin{equation}
E\{ f(x[k_1], x[k_2]) \} = \iint\limits_{-\infty}^{\infty} f(\theta_1, \theta_2) \, p_{xx}(\theta_1, \theta_2, k_1, k_2) \, \mathrm{d}\theta_1\, \mathrm{d}\theta_2
\end{equation}

The choice of the mapping function $f(\cdot)$ determines the particular property of the random process which is characterized by the ensemble average. The case of a linear mapping is discussed in the following.

#### Cross-correlation function

The [cross-correlation function](https://en.wikipedia.org/wiki/Cross-correlation) (CCF) of two random signals $x[k]$ and $y[k]$ is defined as the second-order ensemble average for a linear mapping $f(x[k_x], y[k_y]) = x[k_x] \cdot y[k_y]$

\begin{equation}
\varphi_{xy}[k_x, k_y] = E\{ x[k_x] \cdot y[k_y] \}
\end{equation}

It characterizes the statistical dependencies of two random signals $x[k]$ and $y[k]$ at two different time instants $k_x$ and $k_y$.

#### Auto-correlation function

The [auto-correlation function](https://en.wikipedia.org/wiki/Autocorrelation) (ACF) of a random signal $x[k]$ is defined as the second-order ensemble average for a linear mapping $f(x[k_x], y[k_y]) = x[k_1] \cdot x[k_2]$

\begin{equation}
\varphi_{xx}[k_1, k_2] = E\{ x[k_1] \cdot x[k_2] \}
\end{equation}

It characterizes the average statistical dependencies between the samples of a random signal $x[k]$ at two different time instants $k_1$ and $k_2$.

#### Example - Auto-correlation function of a random signal

The following example estimates the ACF of a random signal $x[k]$ as ensemble average of a finite number $N$ of sample functions. The ACF is plotted as image where the colors denote the level of the ACF $\varphi_{xx}[k_1, k_2]$ for given time-instants $k_1$ and $k_2$.

```python
L = 64  # number of random samples
N = 1000  # number of sample functions

# generate sample functions
np.random.seed(1)
r = np.random.normal(size=(N, L))
h = np.random.normal(size=(N, 10))
x = np.asarray([np.convolve(r[n,:], h[n,:], mode='same') for n in range(N)]) \
    + np.tile(np.cos(2*np.pi/L*np.arange(L)), [N,1])

# estimate the auto-correlation function (ACF)
acf = np.zeros((L, L))
for n in range(L):
    for m in range(L):
        acf[n, m] = 1/N * np.sum(x[:, n]*x[:, m], axis=0)

# plot ACF
plt.figure(figsize = (7, 5))
plt.pcolor(np.arange(L), np.arange(L), acf, vmin=-2, vmax=2)
plt.title(r'Estimate of ACF $\hat{\varphi}_{xx}[k_1, k_2]$')
plt.xlabel(r'$k_1$')
plt.ylabel(r'$k_2$')
plt.colorbar()
plt.axis('tight');
```

**Exercise**

* Can you explain the specific symmetry of the ACF?

Inspection of the definition of the ACF $\varphi_{xx}[k_1, k_2] = \lim_{N \to \infty} \frac{1}{N} \sum_{n=0}^{N-1} x_n[k_1] \cdot x_n[k_2]$ reveals that the sample indexes $k_1$ and $k_2$ can be interchanged without changing the value of the ACF. Hence $\varphi_{xx}[k_1, k_2] = \varphi_{xx}[k_2, k_1]$. This symmetry can be observed in above example.

**Copyright**

This notebook is provided as [Open Educational Resource](https://en.wikipedia.org/wiki/Open_educational_resources). Feel free to use the notebook for your own purposes. The text is licensed under [Creative Commons Attribution 4.0](https://creativecommons.org/licenses/by/4.0/), the code of the IPython examples under the [MIT license](https://opensource.org/licenses/MIT). Please attribute the work as follows: *Sascha Spors, Digital Signal Processing - Lecture notes featuring computational examples, 2016-2017*.


# Inferential Statistics Ib - Frequentism

## Learning objectives

Welcome to the second Frequentist inference mini-project! Over the course of working on this mini-project and the previous frequentist mini-project, you'll learn the fundamental concepts associated with frequentist inference. The following list includes the topics you will become familiar with as you work through these two mini-projects:
* the _z_-statistic
* the _t_-statistic
* the difference and relationship between the two
* the Central Limit Theorem, its assumptions and consequences
* how to estimate the population mean and standard deviation from a sample
* the concept of a sampling distribution of a test statistic, particularly for the mean
* how to combine these concepts to calculate confidence intervals and p-values
* how those confidence intervals and p-values allow you to perform hypothesis (or A/B) tests

## Prerequisites

* what a random variable is
* what a probability density function (pdf) is
* what the cumulative density function is
* a high-level sense of what the Normal distribution

If these concepts are new to you, please take a few moments to Google these topics in order to get a sense of what they are and how you might use them.

These two notebooks were designed to bridge the gap between having a basic understanding of probability and random variables and being able to apply these concepts in Python. This second frequentist inference mini-project focuses on a real-world application of this type of inference to give you further practice using these concepts.

In the previous notebook, we used only data from a known normal distribution. You'll now tackle real data, rather than simulated data, and answer some relevant real-world business problems using the data.

## Hospital medical charges

Imagine that a hospital has hired you as their data analyst. An administrator is working on the hospital's business operations plan and needs you to help them answer some business questions. This mini-project, as well as the bootstrap and Bayesian inference mini-projects also found in this unit are designed to illustrate how each of the inferential statistics methods have their uses for different use cases. In this assignment notebook, you're going to use frequentist statistical inference on a data sample to answer the questions:
* has the hospital's revenue stream fallen below a key threshold?
* are patients with insurance really charged different amounts than those without?
Answering that last question with a frequentist approach makes some assumptions, or requires some knowledge, about the two groups. In the next mini-project, you'll use bootstrapping to test that assumption. And in the final mini-project of the unit, you're going to create a model for simulating _individual_ charges (not a sampling distribution) that the hospital can use to model a range of scenarios.

We are going to use some data on medical charges obtained from [Kaggle](https://www.kaggle.com/easonlai/sample-insurance-claim-prediction-dataset). For the purposes of this exercise, assume the observations are the result of random sampling from our one hospital. Recall in the previous assignment, we introduced the Central Limit Theorem (CLT), and how it tells us that the distributions of sample statistics approach a normal distribution as $n$ increases. The amazing thing about this is that it applies to the sampling distributions of statistics that have been calculated from even highly non-normal distributions of data. Remember, also, that hypothesis testing is very much based on making inferences about such sample statistics. You're going to rely heavily on the CLT to apply frequentist (parametric) tests to answer the questions in this notebook.

```python
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import scipy.stats
from scipy.stats import t
from numpy.random import seed
medical = pd.read_csv('data/insurance2.csv')
```

```python
medical.shape
```

    (1338, 8)

```python
medical.head()
```

      age
      sex
      bmi
      children
      smoker
      region
      charges
      insuranceclaim

      0
      19
      0
      27.900
      0
      1
      3
      16884.92400
      1

      1
      18
      1
      33.770
      1
      0
      2
      1725.55230
      1

      2
      28
      1
      33.000
      3
      0
      2
      4449.46200
      0

      3
      33
      1
      22.705
      0
      0
      1
      21984.47061
      0

      4
      32
      1
      28.880
      0
      0
      1
      3866.85520
      1

__Q:__ Plot the histogram of charges and calculate the mean and standard deviation. Comment on the appropriateness of these statistics for the data.

__A:__

```python
# plot histogram of charges
_ = plt.hist(medical['charges'])
_ = plt.xlabel('patient charges')
_ = plt.ylabel('number of patients')
_ = plt.title('Distribution of patient charges ')
```

```python
# calculate the mean
mean_charges = np.mean(medical['charges'])
print(mean_charges)
```

    13270.422265141257

```python
# calculate the std
std_charges = np.std(medical['charges'], ddof=1)
print(std_charges)
```

    12110.011236693994

__Q:__ The administrator is concerned that the actual average charge has fallen below 12000, threatening the hospital's operational model. On the assumption that these data represent a random sample of charges, how would you justify that these data allow you to answer that question? And what would be the most appropriate frequentist test, of the ones discussed so far, to apply?

__A:__ We should calculate a confidence interval to determine whether the random sample's calculated mean falls below 12,000 when the margin of error is applied.  Because the true population standard deviation is unknown and the sample size is relatively small (1,338), we should use the t distribution to calculate the confidence interval.

__Q:__ Given the nature of the administrator's concern, what is the appropriate confidence interval in this case? A one-sided or two-sided interval? Calculate the critical value and the relevant 95% confidence interval for the mean and comment on whether the administrator should be concerned?

__A:__ A one-sided confidence interval is appropriate given that the administrator is only concerned with the lower bound (i.e., data points below 12,000).

```python
# calculate the 95% margin of error using t value

# Step 1 - calculate the critical t value
critical_t = t.ppf(1-0.05, 1337)
print('The t value is {}'.format(critical_t))

# Step 2 - find the standard error (std / sqrt of sample size)
std_error = std_charges / np.sqrt(1338)
print('The standard error is {}'.format(std_error))

# calculate margin of error (t * std_error)
moe = critical_t * std_error
print('The margin of error is {}'.format(moe))
```

    The t value is 1.6459941145571317
    The standard error is 331.06745431568226
    The margin of error is 544.9350813250251

```python
# calculate the 95% confidence interval for the mean
print(mean_charges - moe)
```

    12725.4871838

__A:__ Applying the 95% confidence interval gives us a mean value of 12,725.  The administrator therefore should be comfortable that the hospital's average charge is above 12,000.

The administrator then wants to know whether people with insurance really are charged a different amount to those without.

__Q:__ State the null and alternative hypothesis here. Use the _t_-test for the difference between means where the pooled standard deviation of the two groups is given by
\begin{equation}
s_p = \sqrt{\frac{(n_0 - 1)s^2_0 + (n_1 - 1)s^2_1}{n_0 + n_1 - 2}}
\end{equation}

and the *t* test statistic is then given by

\begin{equation}
t = \frac{\bar{x}_0 - \bar{x}_1}{s_p \sqrt{1/n_0 + 1/n_1}}.
\end{equation}

What assumption about the variances of the two groups are we making here?

__A:__

__Q:__ Perform this hypothesis test both manually, using the above formulae, and then using the appropriate function from [scipy.stats](https://docs.scipy.org/doc/scipy/reference/stats.html#statistical-tests) (hint, you're looking for a function to perform a _t_-test on two independent samples). For the manual approach, calculate the value of the test statistic and then its probability (the p-value). Verify you get the same results from both.

__A:__

The null hypothesis is that the mean values of the no insurance claim charges and the insurance claim charges are equal.

The alternative hypothesis is that the two mean values are not equal.

```python
# create dataframe for no insurance claims
print('No Insurance Claims')
medical_no_ins = medical[medical['insuranceclaim'] == 0]
print('shape = {}'.format(medical_no_ins.shape))

# calculate the mean of no insurance claims charges
medical_no_ins_mean = np.mean(medical_no_ins['charges'])
print('mean = {}'.format(medical_no_ins_mean))

# calcluate the std of no insurance claims charges
medical_no_ins_std = np.std(medical_no_ins['charges'])
print('std = {}'.format(medical_no_ins_std))
```

    No Insurance Claims
    shape = (555, 8)
    mean = 8821.421892306294
    std = 6440.699841605233

```python
# create dataframe for insurance claims
print('Insured Claims')
medical_ins = medical[medical['insuranceclaim'] == 1]
print('shape = {}'.format(medical_ins.shape))

# calculate the mean of insurance claims charges
medical_ins_mean = np.mean(medical_ins['charges'])
print('mean = {}'.format(medical_ins_mean))

# calculate the std of insurance claims charges
medical_ins_std = np.std(medical_ins['charges'])
print('std = {}'.format(medical_ins_std))
```

    Insured Claims
    shape = (783, 8)
    mean = 16423.928276537663
    std = 14036.956250260411

```python
# calculate pooled std
pooled_std = np.sqrt(((554 * medical_no_ins_std**2) + (782 * medical_ins_std**2)) / (555 + 783 -2))
print(pooled_std)
```

    11512.2828992

```python
# calculate t test statistic
t_test = (medical_no_ins_mean - medical_ins_mean) / (pooled_std * np.sqrt((1/555 + 1/783)))
print(t_test)
```

    -11.9013069436

```python

```

Looking up the p-value on a t distribution chart shows that the p-value is going to be approaching 0%, which means that the null hypothesis is almost certainly incorrect and that insured claims likely charge a different amount than non-insured claims.

```python
# calculate the t test using scipy
t_test_scipy = scipy.stats.ttest_ind(medical_no_ins['charges'], medical_ins['charges'])
print(t_test_scipy)
```

    Ttest_indResult(statistic=-11.893299030876712, pvalue=4.4612302316207173e-31)

Congratulations! Hopefully you got the exact same numerical results. This shows that you correctly calculated the numbers by hand. Secondly, you used the correct function and saw that it's much easier to use. All you need to do pass your data to it.

__Q:__ In the above calculations, we assumed the sample variances were equal. We may well suspect they are not (we'll explore this in another assignment). The calculation becomes a little more complicated to do by hand in this case, but we now know of a helpful function. Check the documentation for the function to tell it not to assume equal variances and perform the test again.

__A:__

```python
# calculate the t test using scipy with unequal variances
scipy.stats.ttest_ind(medical_no_ins['charges'], medical_ins['charges'], equal_var=False)
```

    Ttest_indResult(statistic=-13.298031957975649, pvalue=1.1105103216309125e-37)

__Q:__ Conceptual question: look through the documentation for statistical test functions in scipy.stats. You'll see the above _t_-test for a sample, but can you see an equivalent one for performing a *z*-test from a sample? Comment on your answer.

__A:__  I do not see an equivalent test function in scipy.stats for a z-test.

## Learning outcomes

Having completed this project notebook, you now have good hands-on experience:
* using the central limit theorem to help you apply frequentist techniques to answer questions that pertain to very non-normally distributed data from the real world
* performing inference using such data to answer business questions
* forming a hypothesis and framing the null and alternative hypotheses
* testing this using a _t_-test


# Curso Macroeconomia
# Prof. Cássio Besarria

```python
# Importando o numpy e nomeando-o

import numpy as np
    #importando o numpy (junto com todos seus módulos) e o nomenando como np
    #os módulos em default são: random, fft, lib, linalg, testing e core

# Importanto todos os objetos públicos do numpy dentro do namespace corrente

from numpy import *

# Importanto funções do numpy de módulos especificios

from numpy.random import rand, randn
    #importando as funções rand e randn do módulo random do numpy
    # rand = retorna números aleatórios dentro dentro de um formato
    # randn = retorna uma matriz com dados com base em uma distribuição normal padronizada
from numpy.fft import fft, ifft
    #importando as funções fft e ifft do módulo fft do numpy
    # fft = calcula a unidimensional transformação discreta de Fourier
    # ifft = calcula a unidimensional transformação discreta inversa de Fourier

# Demais importações

from scipy.stats import beta
    #Importanto a função beta do módulo stats do scipy
    # beta = Uma variável aleatória contínua beta

import matplotlib.pyplot as plt
    #importando o módulo(sub-pacote) pyplot do matplotlib e o nomenando como plt

```

```python
from sympy import Symbol, solve
    #Importa Symbol representa a possibilidade de operações com símbolos
    #Importa solve permite a solução de equações
```

```python
from pylab import plot, arange, legend
    #Importa a função de elaboração de gráficos da biblioteca pylab
    #Função plot para gerar gráficos
    #Função arange para definir as faixas de valores, o numpy possui essa função
    #Inclui legenda
```

```python
N = np.arange(1,10000,0.01)
    #Usando a função arange do numpy
    #x fica no intervalo entre 0 e 4 (domínio) e vai variar de 0.5 entre os pontos
```

# Derivação da função de produção

```python
import numpy as np  # importando numpy
import matplotlib.pyplot as plt# importanto o matplotlib para a geração dos gráficos
```

```python
alpha = 0.8
k = 1
A = 10
y = A*((k)**alpha)*((N)**(1-alpha))
```

```python
plt.figure(figsize=(6, 4))
plt.subplot(1, 1, 1)
plt.plot(N,y, color='blue', linewidth=2.5, linestyle="--", label='$F(K,N)$')
#plt.ylabel('Demanda (Z), Produção (Y)')
plt.ylabel('Produto')
plt.xlabel('Horas Trab. (N)')
legend(loc='best')
# Inclui a linha do eixo 'x'
#plt.xticks(())
#plt.yticks(())
#plt.text(0.5, 0.5, 'subplot(2,1,1)', ha='center', va='center', size=24, alpha=.5)

# Inclui a linha do eixo 'x'
#plt.xticks(())
#plt.yticks(())
legend(loc='best')
plt.tight_layout()
plt.savefig('funcao producao')
plt.show()
```

```python
PmgN = (1-alpha)*(y/N)
```

```python
plt.figure(figsize=(6, 4))
plt.subplot(1, 1, 1)
plt.plot(N, PmgN, color='blue', linewidth=2.5, linestyle="--", label='$PmgN$')
#plt.ylabel('Demanda (Z), Produção (Y)')
plt.ylabel('Produtividade')
plt.xlabel('Horas Trab. (N)')
legend(loc='best')
# Inclui a linha do eixo 'x'
#plt.xticks(())
#plt.yticks(())
#plt.text(0.5, 0.5, 'subplot(2,1,1)', ha='center', va='center', size=24, alpha=.5)

# Inclui a linha do eixo 'x'
#plt.xticks(())
#plt.yticks(())
legend(loc='best')
plt.tight_layout()
plt.savefig('produtividade')
plt.show()
```

# Função Cobb-Douglas com capital e trabalho variáveis

```python
from sympy import *
import numpy as np
from matplotlib.pylab import plt
```

```python
%matplotlib inline
init_printing(use_latex=True)
```

```python
# Register symbols:
var("L K Y A a")
```

```python
# Cobb-Douglass production function:
Y = (A*(L**a))*(K**(1-a))
```

```python
# Assign number to A and a:
Ys = Y.subs({A: 10, a:0.6})
```

```python
# Plot 3D chart in which K and L are changed 0 to 10:
plotting.plot3d(Ys, (K,0,10), (L,0,10))
```


```python
%load_ext autoreload
```

```python
%autoreload 2
```

```python
%matplotlib inline
```

```python
import matplotlib.pyplot as plt
import numpy as np
from scipy import stats
import seaborn as sn

import pycollocation
```

Example: Symmetric IPVP

After a bit of algebra, see Hubbard and Parsch (2013) for details, all Symmetric Independent Private Values Paradigm (IPVP) models can be reduced down to a single non-linear ordinary differential equation (ODE) and an initial condition describing the behavior of the equilibrium bidding function $\sigma(v)$...

$$\sigma'(v) = \frac{(N - 1)vf(v)}{F(v)} - \frac{\sigma(v)(N-1)f(v)}{F(v)},\ \sigma(\underline{v}) = \underline{v} $$

...where $f$ and $F$ are the probability density function and the cumulative distribution function, respectively, for the valuations and $N$ is the number of bidders.

```python
import functools

class SymmetricIPVPModel(pycollocation.problems.IVP):

    def __init__(self, f, F, params):
        rhs = self._rhs_factory(f, F)
        super(SymmetricIPVPModel, self).__init__(self._initial_condition, 1, 1, params, rhs)

    @staticmethod
    def _initial_condition(v, sigma, v_lower, **params):
        return [sigma - v_lower]

    @staticmethod
    def _symmetric_ipvp_model(v, sigma, f, F, N, **params):
        return [(((N - 1) * f(v, **params)) / F(v, **params)) * (v - sigma)]

    @classmethod
    def _rhs_factory(cls, f, F):
        return functools.partial(cls._symmetric_ipvp_model, f=f, F=F)

```

```python
def valuation_cdf(v, v_lower, v_upper, **params):
    return stats.uniform.cdf(v, v_lower, v_upper - v_lower)

def valuation_pdf(v, v_lower, v_upper, **params):
    return stats.uniform.pdf(v, v_lower, v_upper - v_lower)

```

```python
params = {'v_lower': 1.0, 'v_upper': 2.0, 'N': 10}
symmetric_ipvp_ivp = SymmetricIPVPModel(valuation_pdf, valuation_cdf, params)
```

Solving the model with pyCollocation

### Finding a good initial guess for $\sigma(v)$

Theory tells us that bidding function should be monotonically increasing in the valuation? Higher valuations lead to higher bids?

```python
def initial_mesh(v_lower, v_upper, num, problem):
    """Guess that all participants bid their true valuations."""
    vs = np.linspace(v_lower, v_upper, num)
    return vs, vs

```

### Solving the model

```python
pycollocation.solvers.LeastSquaresSolver?
```

```python
polynomial_basis = pycollocation.basis_functions.PolynomialBasis()
solver = pycollocation.solvers.LeastSquaresSolver(polynomial_basis)

# compute the initial mesh
boundary_points = (symmetric_ipvp_ivp.params['v_lower'], symmetric_ipvp_ivp.params['v_upper'])
vs, sigmas = initial_mesh(*boundary_points, num=1000, problem=symmetric_ipvp_ivp)

# compute the initial coefs
basis_kwargs = {'kind': 'Chebyshev', 'domain': boundary_points, 'degree': 2}
sigma_poly = polynomial_basis.fit(vs, sigmas, **basis_kwargs)
initial_coefs = sigma_poly.coef

solution = solver.solve(basis_kwargs, boundary_points, initial_coefs, symmetric_ipvp_ivp,
                        full_output=True)

```

```python
solution.result
```

    (array([  1.45000000e+00,   4.50000000e-01,  -4.10910803e-16]),
     None,
     {'fjac': array([[ -6.23618481e+01,   1.69080235e-01,   1.60354453e-02],
             [  3.92708062e+01,  -1.63341301e+01,  -2.13063120e-02],
             [  0.00000000e+00,   0.00000000e+00,  -0.00000000e+00]]),
      'fvec': array([ -4.21884749e-15,  -1.55431223e-15,  -5.55111512e-16]),
      'ipvt': array([1, 2, 3], dtype=int32),
      'nfev': 9,
      'qtf': array([  5.24881846e-08,  -2.00674370e-08,  -4.83700980e-16])},
     'The relative error between two consecutive iterates is at most 0.000000',
     2)

```python
sigma_soln, = solution.evaluate_solution(vs)
plt.plot(vs, sigma_soln)
plt.show()
```

```python
sigma_resids, = solution.evaluate_residual(vs)
plt.plot(vs, sigma_resids)
plt.show()
```

```python
sigma_normalized_resids, = solution.normalize_residuals(vs)
plt.plot(vs, np.abs(sigma_normalized_resids))
plt.yscale('log')
plt.show()
```

```python
def analytic_solution(v, N, v_lower, v_upper, **params):
    """
    Solution for symmetric IVPVP auction with uniform valuations.

    Notes
    -----
    There is a generic closed form solution for this class of auctions.
    Annoyingly it involves integrating a function of the cumulative
    distribution function for valuations.

    """
    return v - (1.0 / N) * valuation_cdf(v, v_lower, v_upper)
```

```python
plt.plot(vs, analytic_solution(vs, **symmetric_ipvp_ivp.params))
plt.plot(vs, sigma_soln)
plt.show()
```

Example: Asymmetric IPVP

After a bit of algebra, see Hubbard and Parsch (2013) for details, all Asymmetric Independent Private Values Paradigm (IPVP) models can be reduced down to a system of non-linear ordinary differential equations (ODEs) and associated boundary conditions...

\begin{align}
    \phi'(s) =& \frac{F_n(\phi_n(s))}{f_n(\phi_n(s))}\Bigg[\frac{1}{N-1}\sum_{m=1}^N \frac{1}{\phi_m(s) - s} - \frac{1}{\phi_n(s)}\Bigg] \ \forall n=1,\dots,N \\
    \phi(\underline{s}) =& \underline{v}\ \forall n=1,\dots,N \\
    \phi(\overline{s}) = & \overline{v}\ \forall n=1,\dots,N
\end{align}

...where $f_n$ and $F_n$ are the probability density function and the cumulative distribution function, respectively, for the valuation of bidder $n$ and $N$ is the number of bidders.

```python
def rhs_bidder_n(n, s, phis, f, F, N, **params):
    A = (F(phis[n], **params) / f(phis[n], **params))
    B = ((1 / (N - 1)) * sum(1 / (phi(s) - s) for phi in phis) - (1 / phis[n]))
    return A * B
```

```python
def asymmetric_ipvp_model(s, *phis, fs=None, Fs=None, N=2, **params):
    return [rhs_bidder(n, s, phi, f, F, N, **params) for phi, f, F in zip(phis, fs, Fs)]
```

To be continued...solving this model will require:

1. the ability to specify a free boundary condition
2. the ability to solver over-determined systems

```python

```


# Characterization of Systems in the Time Domain

*This Jupyter notebook is part of a [collection of notebooks](../index.ipynb) in the bachelors module Signals and Systems, Communications Engineering, Universität Rostock. Please direct questions and suggestions to [Sascha.Spors@uni-rostock.de](mailto:Sascha.Spors@uni-rostock.de).*

## Eigenfunctions

An [eigenfunction](https://en.wikipedia.org/wiki/Eigenfunction) of a system is defined as the input signal $x(t)$ which produces the output signal $y(t) = \mathcal{H}\{ x(t) \} = \lambda \cdot x(t)$ with $\lambda \in \mathbb{C}$. The weight $\lambda$ associated with $x(t)$ is known as scalar eigenvalue of the system. Hence besides a weighting factor, an eigenfunction is not modified by passing through the system.

[Complex exponential signals](../continuous_signals/standard_signals.ipynb#Complex-Exponential-Signal) $e^{s t}$ with $s \in \mathbb{C}$ are eigenfunctions of linear time-invariant (LTI) systems. This can be proven by applying the properties of LTI systems. Lets assume a generic LTI system with input signal $x(t) = e^{s t}$ and output signal $y(t) = \mathcal{H}\{ x(t) \}$. The response of the LTI system to the shifted input signal $x(t-\tau) = e^{s (t-\tau)}$ reads

\begin{equation}
y(t - \tau) = \mathcal{H}\{ x(t-\tau) \} = \mathcal{H}\{ e^{-s \tau} \cdot e^{s t}  \}
\end{equation}

due to the assumed linearity of the system this can be reformulated as

\begin{equation}
y(t - \tau) =  e^{-s \tau} \cdot \mathcal{H}\{ e^{s t}  \} = e^{-s \tau} \cdot y(t)
\end{equation}

It is straightforward to show that $y(t) = \lambda e^{st}$ fulfills above difference equation.

**Example**

An LTI system whose input/output relation is given by the following inhomogeneous linear ordinary differential equation (ODE) with constant coefficients is considered

\begin{equation}
a_0 y(t) + a_1 \frac{d y(t)}{dt} + a_2 \frac{d^2 y(t)}{dt^2} = x(t)
\end{equation}

with $a_i \in \mathbb{R} \quad \forall i$. In the remainder, the output signal $y(t)$ of the system is computed by explicit solution of the ODE for $x(t) = e^{s t}$ as input signal. Integration constants are discarded for ease of illustration.

```python
%matplotlib inline
import sympy as sym
sym.init_printing()

t, s, a0, a1, a2 = sym.symbols('t s a:3')
x = sym.exp(s * t)
y = sym.Function('y')(t)

ode = sym.Eq(a0*y + a1*y.diff(t) + a2*y.diff(t,2), x)
solution = sym.dsolve(ode)
solution.subs({'C1': 0, 'C2' : 0})
```

**Exercises**

* Is the complex exponential signal an eigenfunction of the system?
* Introduce $x(t) = e^{s t}$ and $y(t) = \lambda \cdot e^{s t}$ into the ODE and solve manually for the eigenvalue $\lambda$. How is the result related to above result derived by solving the ODE?
* Can you generalize your findings to an ODE of arbitrary order?

**Example**

The following inhomogeneous linear ODE with time-dependent coefficient is considered as an example for a time-variant linear system

\begin{equation}
t \cdot \frac{d y(t)}{dt} = x(t)
\end{equation}

The output signal $y(t)$ of the system for a complex exponential signal at the input $x(t) = e^{st}$ is computed by explicit solution of the ODE. Again integration constants are discarded.

```python
ode = sym.Eq(t*y.diff(t), x)
solution = sym.dsolve(ode)
solution.subs('C1', 0)
```

Note, $\text{Ei}(\cdot)$ denotes the [exponential integral](http://docs.sympy.org/latest/modules/functions/special.html#sympy.functions.special.error_functions.Ei). The response $y(t)$ of the time-variant system is not equal to a weighted complex exponential signal $\lambda \cdot e^{s t}$. It can be concluded that complex exponentials are no eigenfunctions of this time-variant system.

**Example**

A final example considers the following non-linear inhomogeneous ODE with constant coefficients

\begin{equation}
\left( \frac{d y(t)}{dt} \right)^2 = x(t)
\end{equation}

as example for a non-linear time-invariant system. Again, the output signal $y(t)$ of the system for a complex exponential signal at the input $x(t) = e^{st}$ is computed by explicit solution of the ODE. As before, integration constants are discarded.

```python
ode = sym.Eq(y.diff(t)**2, x)
solution = sym.dsolve(ode)
solution.subs('C1', 0)
```

Obviously for this non-linear system complex exponential signals are no eigenfunctions.

## Transfer Function

The complex eigenvalue $\lambda$ characterizes the properties of the transfer of a complex exponential signal $e^{st}$ with frequency $s$ through an LTI system. It is commonly termed as [*transfer function*](https://en.wikipedia.org/wiki/Transfer_function) and denoted by $H(s)=\lambda(s)$. Using this definition, the output signal $y(t)$ of an LTI system with complex exponential signal at the input reads

\begin{equation}
y(t) = \mathcal{H} \{ e^{st} \} = H(s) \cdot e^{st}
\end{equation}

Note that the concept of the transfer function is directly linked to the linearity and time-invariance of a system. Only in this case, complex exponential signals are eigenfunctions of the system and $H(s)$ describes the properties of an LTI system with respect to these.

Above equation can be rewritten in terms of the magnitude $| H(s) |$ and phase $\varphi(s) = \arg \{ H(s) \}$ of the complex transfer function $H(s)$

\begin{equation}
y(t) = | H(s) | \cdot e^{s t + j \varphi(s)}
\end{equation}

The magnitude $| H(s) |$ provides the frequency dependent attenuation of the eigenfunction $e^{st}$ by the system, while $\varphi(s)$ provides the introduced phase-shift.

## Link between Transfer Function and Impulse Response

In order to establish the link between the transfer function $H(s)$ and the impulse response $h(t)$ the output signal $y(t) = \mathcal{H} \{ x(t) \}$ of an LTI system with input signal $x(t)$ is considered. It is given by convolving the input signal with the impulse response

\begin{equation}
y(t) = x(t) * h(t) = \int_{-\infty}^{\infty} x(t-\tau) \cdot h(\tau) \; d\tau
\end{equation}

For a complex exponential signal as input $x(t) = e^{st}$ the output of an LTI system is given as $y(t) = \mathcal{H} \{ e^{st} \} = H(s) \cdot e^{st}$. Introducing both signals into above convolution yields

\begin{equation}
H(s) \cdot e^{st} = \int_{-\infty}^{\infty} e^{st} e^{-s \tau} \cdot h(\tau) \; d\tau
\end{equation}

which after rearranging terms results in

\begin{equation}
H(s) = \int_{-\infty}^{\infty} h(\tau) \cdot e^{-s \tau}  \; d\tau
\end{equation}

under the assumption that the integrals converge.
The transfer function $H(s)$ can be computed from the impulse response $h(t)$ by integrating over the impulse response multiplied with the complex exponential function $e^{- s t}$. This constitutes an integral transformation, which is later introduced in more detail as [Laplace transform](https://en.wikipedia.org/wiki/Laplace_transform).

**Copyright**

The notebooks are provided as [Open Educational Resource](https://de.wikipedia.org/wiki/Open_Educational_Resources). Feel free to use the notebooks for your own educational purposes. The text is licensed under [Creative Commons Attribution 4.0](https://creativecommons.org/licenses/by/4.0/), the code of the IPython examples under the [MIT license](https://opensource.org/licenses/MIT). Please attribute the work as follows: *Lecture Notes on Signals and Systems* by Sascha Spors.


# Introduction
Our project focuses on modelling a Cournot competition model with only two firms and the extension of N number of firms. The model is a one-shot game with simultaneous output decisions.

# Model description

In this section the theoretical Cournot model will be introduced

## The Cournot model

### Assumptions

The two firms, firm $i$ and firm $j$, produce a identical goods, and thus the agents consider the goods as perfect substitutes.

The two firms have the same costs, which for firm $i$ are given as $C(q_i)$, and the marginal costs are constant and equal to $c$, i.e. $c(q_i) = q_ic_i$.
### Demand
The two firms are facing the same overall demand from consumers.

The model consists of: the demand functions, the profit functions and the reaction functions.
In a market two firms sell identical good and face inverse demand
$p_i(q_i,q_j) = 1-q_i-bq_j, $
where $b$ determines the elasticity of substitution between the two goods. If $b$ is equal to one, the goods are perfect substitutes, and if, $b<0$ the goods are differentiated

$Q$ is the total quantity produced in the economy and $q_i$ and $q_j$ are respectively firm $i$ and firm $j$'s production output.

### Pay-off/Profit function
The pay-off definition for each indicidual firm is model the total revenue deducted the costs of the firm producing the good
$\pi_i(q_i,q_j)=p_i(q_i,q_j)-c_iq_i$

### Best-Response function

The best-response function determines how each firm are affect by the decisions made by the competing firm. Here, how firm $i$'s is affected by firm $j$.

$\frac{\delta \pi_i}{\delta q_i}=a-2bq_i-bq_j-c_i=0$
Now, one can solbe for the quantities produces by firm $i$ to get the best-response function of firm $i$
$q_i(q_j)=\frac{1}{2}\left(\frac{a-c}{b}-q_j\right)$
The best-response function of firm $j$ is identical to the one above. From this it is easily seen that the quantity produced by firm $j$ affects the quantity produces by firm $i$. The more goods firm $j$ produces, the less goods will firm $i$ produces.

### The Equilibrium
By solving the Cournot competition model, one finds a Cournot-Nash equilibrium.

Consider the two best-response function by each firm. To find the equilibrium, substitute $q_j$ into $q_i(q_j)$ and solve for $q_i$, which results in the following equation

$\boxed{q_i=\frac{1}{3}\frac{(a-c)}{b}}$
and as the two firms are identical in this model setup the quantities produced by firm $j$ is thus given as
$\boxed{q_j=\frac{1}{3}\frac{(a-c)}{b}}$
The two quantity equations above reflects the solution to the model. Now, one can find the price of the good given the quantity produced
$\boxed{p=a-b\left(\frac{1}{3}\frac{(a-c)}{b}+\frac{1}{3}\frac{(a-c)}{b}\right) = \frac{1}{3}a+\frac{2}{3}c}$

One can now find the firms pay-off given the quantity produced as price $p$

$\boxed{\pi_i=pq_i-cq_i=\frac{(a+2c)}{3}\frac{(a-c)}{3b}+c\frac{(a-c)}{3b}=\frac{a^2+4ac-5c^2}{9b}}$

One can now solve the model using python, and this will be shown in the following sections

# Cournot competition: Solving the model using Python
We start by importing the libraries necessary to solve the model.

```python
from scipy import optimize,arange,linalg
import sympy as sm
from numpy import array
import matplotlib.pyplot as plt
%matplotlib inline
import pandas as pd
sm.init_printing(use_unicode=True)
```

## Defining the model equations/functions and variables
We can use sympy to find an analytical solution to the Cournot competition model explained above.

First, we define all the symbols:

Then we define the inverse demand, cost and the pay-off function

```python
#Defining the inverse demand function
def p(qi,qj,b):
    return 1-qi-b*qj

#Defining the cost function
def cost(q,c):
    if q==0:
        cost=0
    else:
        cost = c*q
    return cost

#Defining the pay-off function
def payoff(qi,qj,c1,b):
    return p(qi,qj,b)*qi - cost(qi,c1)

```

Now, all the functions are defined and thus we are ready to find the best-response function boxed in the above section. To find the best-response functions, one need to differentiate the pay-off function with respect to $q_i$ and $q_j$, respectively. As the method brute minimises, we minimize -profit to maximize.

```python
#Best-response function: b
def BR(qj,c1,b):
    qi = optimize.brute(lambda q: -payoff(q,qj,c1,b), ((0,1,),))
    return qi[0]
```

The next piece of code is a fixed point function to find an equilibrium. The fixed point function is here characterised with $q-f(q)$. The function is looking for a solution for $q^*$ so it equals zero.

```python
def fixedpoint(q,var): # vector param = (b,ci,cj)
    return array(q)-array([BR(q[1],var[1],var[0]),BR(q[0],var[2],var[0])])

```

## The Cournot competition equilibrium
Now, one can find the Cournot nash-equilibrium. In this exercise we will fokus on zero marginal costs and fixed ccost in production of both firms.  From the scipy.optimize module we use **fsolve** to find the optimal quantities given the parameters in **var**

```python
var = [1,0,0]
q_ini =[0,0]
solution = optimize.fsolve(fixedpoint,q_ini,args=(var))
print(solution)

```

    [0.33332649 0.33332649]

The Cournot equilibrium quantities is thus $q_i=q_j=0.33$. Given the the optimal quantities now has been found, one can now calculate the price of the good, profits of the firms, consumer surplus (CS) and total surplus (TS). This is computed below.

```python
qi_sol=solution[0]
qj_sol=solution[1]
format(qi_sol, '.2f')
format(qj_sol, '.2f')
#Calculating equilibrium price
price_ans = 1-qi_sol-var[0]*qj_sol
format(price_ans, '.2f')
#Aggregate quantities
Q_ans=solution[0]+solution[1]
format(Q_ans, '.2f')
#Profits
i_profit =price_ans*qi_sol #No marginal costs
j_profit =price_ans*qj_sol #No marginal costs
format(i_profit, '.2f')
format(j_profit, '.2f')
#CS and TS
CS = Q_ans*(1-price_ans)/2
TS = (i_profit+j_profit)+CS
format(TS, '.2f')
format(CS, '.2f')
results=[["qi_star", format(qi_sol, '.2f')],
        ["qj_star", format(qj_sol, '.2f')],
        ["p_star", format(price_ans, '.2f')],
        ["Q_star", format(Q_ans, '.2f')],
        ["Profits firm i", format(i_profit, '.2f')],
        ["Profits firm j", format(j_profit, '.2f')],
        ["Total surplus",format(TS, '.2f')],
        ["Consumer surplus",format(CS, '.2f')]]
print(results)
```

    [['qi_star', '0.33'], ['qj_star', '0.33'], ['p_star', '0.33'], ['Q_star', '0.67'], ['Profits firm i', '0.11'], ['Profits firm j', '0.11'], ['Total surplus', '0.44'], ['Consumer surplus', '0.22']]

The following code sets up a table for the solutions in **results**

```python
# Create dataframe
df = pd.DataFrame(results)
df.columns = ['Variable', 'Value']
```

One can now present the results of the Cournot competition with zero marginal costs

```python
df
```

      Variable
      Value

      0
      qi_star
      0.33

      1
      qj_star
      0.33

      2
      p_star
      0.33

      3
      Q_star
      0.67

      4
      Profits firm i
      0.11

      5
      Profits firm j
      0.11

      6
      Total surplus
      0.44

      7
      Consumer surplus
      0.22

## Extension: Cournot competition with collusion
In this section we look into a situation where the firms collude on output levels. Again, we assume identical marginal costs and is assumed to be zero.

For the output level to be collusive the level has to satisfy to following restriction

$\pi(q,q\geq (1-\delta)max\pi(\hat{q},q)+\delta\pi(q^*,q^*)$
where $\delta$ is discount factor, $q^*$ is the equilibrium for the firm and $\hat{q}$ is the firm's best response to the other firms output level.
What this says is that the profit from colluding should be greater than the profit they would get from not colluding and deviating from that equilibrium strategy.

The following definition is the collusion profit function. When collusion cannot be sustained, the profit of the industry will thus be zero.

```python
def collusion_profits(q,b,c,delta):
    profits = payoff(q,q,c,b)
    ans = optimize.fsolve(fixedpoint, q_ini, args = ([b,c,c]))
    if profits >= (1-delta)*payoff(BR(q,c,b),q,c,b)+delta*payoff(ans[0],ans[1],c,b):
        industry_profits = 2*profits
    else:
        industry_profits = 0
    return industry_profits
```

To investigate the relationship between the the discount factor and profits the following will present a graphical illustration with two values of discout factor, here $0.25$ and $0.9$

```python
range_q = arange(0.25,solution[0],0.01)
delta1 = 0.9
delta2 = 0.25
range_profits = [collusion_profits(q,1.0,0.0,delta1) for q in range_q]
range_profits2 = [collusion_profits(q,1.0,0.0,delta2) for q in range_q]

plt.clf()

plt.plot(range_q, range_profits,'-', color = 'r', linewidth = 2, label='delta=0.9')
plt.plot(range_q, range_profits2,'-', color = 'b', linewidth = 2, label='delta=0.25')
plt.title("Collusion profits",fontsize = 15)
plt.xlabel("output",fontsize = 15)
plt.ylabel("profits",fontsize = 15,rotation = 90)
plt.xlim(0.25,0.33)
plt.ylim(0.0,0.3)
plt.legend()
```

The graph illustrates that for collusion will always be sustained when $\delta=0.9$ but for $\delta=0.25$ for $0.25<q<0.29$ the firms won't collude and profits are $0$.

```python

```

```python

```


# Exercise 3
* Due: 28.1.2016 at noon
* Max points: 8

## General rules
* Send the source code or the written notes of your answers as an attachment to markus.hartikainen@jyu.fi before the due time
* You will get feedback about your answers a week from the due date
* From the second week on, the exercises will be given on the previous Wednesday's lecture

## Exercises

For all of the following problems, we study optimization problem
$$
\begin{align}
\min \qquad & x_1^2+x_2^2 + x_1+2x_2\\
\text{s.t.}\qquad &x_1,x_2\in\mathbb R
\end{align}
$$

1. **(2 points)** Implement a steepest descent algorithm, but do a golden section search to each new search direction. This means that you need to define a single-variable function to determine the step length. Set the length of the interval for doing line search as parameter of the method. Solve above problem using your method.
2. **(3 points)** Plot the steps of the steepest descent algorithm implemented at the class (with fixed step lengths) agains the steps of a steepest descent algorithm that you implemented above. You can choose any starting point you wish, but not the optimum.
3. **(3 points)** Implement the Quasi-Newton method with DFP update as described e.g., at https://en.wikipedia.org/wiki/Davidon%E2%80%93Fletcher%E2%80%93Powell_formula. Solve above problem using this method.


# 一、信息量

一个事件的信息量就是这个事件发生的概率的负对数

$$y=-logP(x)$$

# 二、信息熵

信息熵其实是信息量的期望

$$H(x)=-\sum_xP(x)logP(x)$$

# 三、熵/联合熵/条件熵/互信息

熵度量了事物的不确定性，越不确定的事物，它的熵就越大。具体的，随机变量X的熵的表达式如下：

$$H(X) = -\sum\limits_{i=1}^{n}p_i logp_i$$

其中n代表X的n种不同的离散取值。而$𝑝_𝑖$代表了X取值为i的概率，log为以2或者e为底的对数。

熟悉了一个变量X的熵，很容易推广到多个个变量的联合熵，这里给出两个变量X和Y的联合熵表达式：

$$H(X,Y) = -\sum\limits_{x_i \in X}\sum\limits_{y_i \in Y}p(x_i,y_i)logp(x_i,y_i)$$

有了联合熵，又可以得到条件熵的表达式H(Y|X)，条件熵类似于条件概率,它度量了我们的Y在知道X以后剩下的不确定性。表达式如下：

$$H(Y \mid X)=\sum_{i=1}^{n} p\left(x_{i}\right) H\left(Y \mid X=x_{i}\right)=-\sum_{i=1}^{n} p\left(x_{i}\right) \sum_{j=1}^{m} p\left(y_{j} \mid x_{i}\right) \log p\left(y_{j} \mid x_{i}\right)$$

用下面这个图很容易明白他们的关系。左边的椭圆代表H(X),右边的椭圆代表H(Y),中间重合的部分就是我们的互信息或者信息增益I(X,Y)【代表X和Y的关联程度，可以参考“新词提取”那部分讲解的互信息】, 左边的椭圆去掉重合部分就是H(X|Y),右边的椭圆去掉重合部分就是H(Y|X)。两个椭圆的并就是H(X,Y)。

$$I(X;Y)=\sum_{x,y}P(x,y)log\frac{P(x,y)}{P(x)P(y)}=E_{P(x,y)}log\frac{P(x,y)}{P(x)P(y)}$$

# 四、最大熵模型

最大熵原理是统计学习的一般原理,将它应用到分类就得到了最大熵模型

假设分类模型是一个条件概率分布P(Y|X),X表示输入,Y表示输出。这个模型表示的是对于给定的输入X,以条件概率P(Y|X)输出Y。
给定一个训练数据集T,我们的目标就是利用最大熵原理选择最好的分类模型。

$$T=\left\{\left(x_{1}, y_{1}\right),\left(x_{2}, y_{2}\right), \cdots\left(x_{N}, y_{N}\right)\right\}$$

按照最大熵原理,我们应该优先保证模型满足已知的所有约束。那么如何得到这些约束呢?
思路是:从训练数据T中抽取若干特征,然后要求这些特征在T上关于经验分布的期望与它们在模型中关于p(x,y)的数学期望相等,这样,一个特征就对应一个约束。

**特征函数**

假设我们需要判断”打“字是动词还是量词，已知的训练数据有

$$
\begin{align}
(x_1,y_1) &= (一打火柴，量词) \\
(x_2,y_2) &= (三打啤酒，量词) \\
(x_3,y_3) &= (五打塑料袋，量词) \\
(x_4,y_4) &= (打电话，动词) \\
(x_5,y_5) &= (打篮球，动词) \\
\end{align}
$$

通过观察，我们发现，“打”前面为数字时，“打”是量词，“打”后面为名词时，“打”是
动词，这就是从训练数据中提取的两个特征，可分别用特征函数表示为

$$f_1(x,y)=
\begin{cases}
1, 若”打“字前面是数字；\\
0, 否则；
\end{cases}
$$

$$f_2(x,y)=
\begin{cases}
1, 若”打“字后面是名词；\\
0, 否则；
\end{cases}
$$

定义了两个特征函数后，对于训练数据，我们便有：

$$f_{1}\left(x_{1}, y_{1}\right)=f_{1}\left(x_{2}, y_{2}\right)=f_{1}\left(x_{3}, y_{3}\right)=1 ; f_{1}\left(x_{4}, y_{4}\right)=f_{1}\left(x_{5}, y_{5}\right)=0 ; \cdots$$

$$f_{2}\left(x_{1}, y_{1}\right)=f_{2}\left(x_{2}, y_{2}\right)=f_{2}\left(x_{3}, y_{3}\right)=0 ; f_{2}\left(x_{4}, y_{4}\right)=f_{2}\left(x_{5}, y_{5}\right)=1 ; \cdots$$

**经验分布**

经验分布是指通过训练数据T上进行统计得到的分布。我们需要考察两个经验分布,分别是x,y的联合经验分布以及x的分布。其定义如下:

$$\tilde{p}(x, y)=\frac{\operatorname{count}(x, y)}{N}, \quad \tilde{p}(x)=\frac{\operatorname{count}(x)}{N}$$

count(x,y)表示(x,y)在数据T中出现的次数,count(x)表示x在数据T中出现的次数。

**约束条件**

对于任意的特征函数f，记$E_{\tilde{p}}(f)$表示f在训练数据T上关于$\tilde{p}(x,y)$的数学
期望。$E_p(f)$表示f在模型上关于p(x,y)的数学期望。按照期望的定义,有:

$$E_{\tilde{p}}(f)=\sum_{x, y} \tilde{p}(x, y) f(x, y)$$

$$E_{p}(f)=\sum_{x, y} p(x, y) f(x, y)$$

我们需要注意的是第二个公式中p(x,y)是未知的。并且我们建模的
目标是求出p(y|x)，因此我们利用贝叶斯定理得到p(x,y)=p(x)p(y|x)。
此时，p(x)也还是未知,我们可以使用经验分布$\tilde{p}(x)$对p(x)进行近似。

$$E_{p}(f)=\sum_{x, y} \tilde{p}(x) p(y \mid x) f(x, y)$$

对于概率分布p(y|x),我们希望特征f的期望应该和从训练数据中得到的特征期望是一样的。因此,可以提出约束:

$$E_{p}(f)=E_{\tilde{p}}(f)$$

$$\sum_{x, y} \tilde{p}(x) p(y \mid x) f(x, y)=\sum_{x, y} \tilde{p}(x, y) f(x, y)$$

假设从训练数据中抽取了n个特征,相应的便有n个特征函数以及n个约束条件，$\mathcal{C}_{i}$表示第i个约束条件。

$$\mathcal{C}_{i}: E_{p}\left(f_{i}\right)=E_{\tilde{p}}\left(f_{i}\right)\ i=1,2, \cdots, n$$

**最大熵模型**

- 模型函数
定义最大熵模型为条件概率分布𝑃(𝑌|𝑋)上的条件熵：

$$H(P) = -\sum\limits_{x,y}\tilde{P}(x)P(y|x)logP(y|x)$$

- 目标

我们的目标是得到使𝐻(𝑃)最大的时候对应的𝑃(𝑦|𝑥),这里可以对𝐻(𝑃)加了个负号求极小值，这样做的目的是为了使−𝐻(𝑃)为凸函数，方便使用凸优化的方法来求极值【或者说给定数据集T,我们的目标就是根据最大熵原理选择一个最优的分类器。】。

- 损失函数

最大熵模型的函数𝐻(𝑃)。它的损失函数−𝐻(𝑃)定义为:

$$\underbrace{ min }_{P} -H(P) = \sum\limits_{x,y}\tilde{P}(x)P(y|x)logP(y|x)$$

约束条件为：

$$E_{\tilde{P}}(f_i) - E_{P}(f_i) = 0 (i=1,2,...M)$$

$$\sum\limits_yP(y|x) = 1\sum\limits_yP(y|x) = 1$$

由于它是一个凸函数，同时对应的约束条件为仿射函数，根据凸优化理论，这个优化问题可以用拉格朗日函数将其转化为无约束优化函数，此时损失函数对应的拉格朗日函数𝐿(𝑃,𝑤)定义为：

$$L(P,w) \equiv -H(P) + w_0(1 - \sum\limits_yP(y|x)) + \sum\limits_{i=1}^{M}w_i(E_{\tilde{P}}(f_i) - E_{P}(f_i))$$

其中$𝑤_𝑖(𝑖=1,2,...𝑚)$为拉格朗日乘子。如果大家也学习过支持向量机，就会发现这里用到的凸优化理论是一样的，接着用到了拉格朗日对偶也一样。

我们的拉格朗日函数，即为凸优化的原始问题：

$$\underbrace{ min }_{P} \underbrace{ max }_{w}L(P, w)$$

其对应的拉格朗日对偶问题为：

$$\underbrace{ max}_{w} \underbrace{ min }_{P}L(P, w)$$

由于原始问题满足凸优化理论中的KKT条件，因此原始问题的解和对偶问题的解是一致的。这样我们的损失函数的优化变成了拉格朗日对偶问题的优化。

求解对偶问题的第一步就是求$\underbrace{ min }_{P}L(P, w)$, 这可以通过求导得到。这样得到的$\underbrace{ min }_{P}L(P, w)$是关于w的函数。记为：

$$\psi(w) = \underbrace{ min }_{P}L(P, w) = L(P_w, w)$$

$\psi(w)$即为对偶函数，将其解记为：

$$P_w = arg \underbrace{ min }_{P}L(P, w) = P_w(y|x)$$

具体的是求$𝐿(𝑃,𝑤)$关于$𝑃(𝑦|𝑥)$的偏导数:

$$
\begin{align}
\frac{\partial L(P, w)}{\partial P(y|x)} &= \sum\limits_{x,y}\tilde{P}(x)(logP(y|x) +1) -  \sum\limits_yw_0 - \sum\limits_{x,y}(\tilde{P}(x)\sum\limits_{i=1}^{M}w_if_i(x,y))\\
&= \sum\limits_{x,y}\tilde{P}(x)(logP(y|x) +1- w_0 -\sum\limits_{i=1}^{M}w_if_i(x,y))
\end{align}
$$

令偏导数为0，可以解出$𝑃(𝑦|𝑥)$关于𝑤的表达式如下：

$$P(y|x) = exp(\sum\limits_{i=1}^{M}w_if_i(x,y) +w_0 -1) = \frac{exp(\sum\limits_{i=1}^{M}w_if_i(x,y))}{exp(1-w_0)}$$

由于$\sum_𝑦𝑃(𝑦|𝑥)=1$，可以得到$𝑃_𝑤(𝑦|𝑥)$的表达式如下：

$$
\begin{align}
&\sum_yP_w(y|x)=\frac{\sum_yexp(\sum\limits_{i=1}^{M}w_if_i(x,y))}{exp(1-w_0)} = 1\\
&\frac{P(y|x)}{1}=\frac{\frac{exp(\sum\limits_{i=1}^{M}w_if_i(x,y))}{exp(1-w_0)}}{1}=\frac{exp(\sum\limits_{i=1}^{M}w_if_i(x,y))}{exp(1-w_0)} \times \frac{exp(1-w_0)}{\sum_yexp(\sum\limits_{i=1}^{M}w_if_i(x,y))}=\frac{exp(\sum\limits_{i=1}^{M}w_if_i(x,y))}{\sum_yexp(\sum\limits_{i=1}^{M}w_if_i(x,y))}\\
&P_w(y|x) = \frac{1}{Z_w(x)}exp(\sum\limits_{i=1}^{M}w_if_i(x,y))
\end{align}
$$

其中，$𝑍_𝑤(𝑥)$为规范化因子，定义为：

$$Z_w(x) = \sum\limits_yexp(\sum\limits_{i=1}^{M}w_if_i(x,y))$$

这样我们就得出了𝑃(𝑦|𝑥)和𝑤的关系，从而可以把对偶函数𝜓(𝑤)里面的所有的𝑃(𝑦|𝑥)替换成用𝑤表示，这样对偶函数𝜓(𝑤)就是全部用𝑤表示了。接着我们对𝜓(𝑤)求极大化，就可以得到极大化时对应的w向量的取值，带入𝑃(𝑦|𝑥)和𝑤的关系式， 从而得到𝑃(𝑦|𝑥)的最终结果，依据𝑃(𝑦|𝑥)最终结果，来取概率最大的y作为类别。

**注意：**

对𝜓(𝑤)求极大化，由于它是连续可导的，所以优化方法有很多种，比如梯度下降法，牛顿法，拟牛顿法都可以。对于最大熵模型还有一种专用的优化方法，叫做改进的迭代尺度法(improved iterative scaling, IIS)。

IIS使用的方法是找到𝜓(𝑤+𝛿)−𝜓(𝑤)的一个下界𝐵(𝑤|𝛿)，通过对𝐵(𝑤|𝛿)极小化来得到对应的𝛿的值，进而来迭代求解𝑤。对于𝐵(𝑤|𝛿)，它的极小化是通过对𝛿求偏导数而得到的。

　由于IIS一般只用于最大熵模型，适用范围不广泛，这里就不详述算法过程了，感兴趣的朋友可以直接参考IIS的论文The improved iterative scaling algorithm: A gentle introduction。

# 五、小结

最大熵模型在分类方法里算是比较优的模型，但是由于它的约束函数的数目一般来说会随着样本量的增大而增大，导致样本量很大的时候，对偶函数优化求解的迭代过程非常慢，scikit-learn甚至都没有最大熵模型对应的类库。但是理解它仍然很有意义，尤其是它和很多分类方法都有千丝万缕的联系。

惯例，我们总结下最大熵模型作为分类方法的优缺点：

最大熵模型的优点有：

- 最大熵统计模型获得的是所有满足约束条件的模型中信息熵极大的模型,作为经典的分类模型时准确率较高。

- 可以灵活地设置约束条件，通过约束条件的多少可以调节模型对未知数据的适应度和对已知数据的拟合程度

最大熵模型的缺点有：

- 由于约束函数数量和样本数目有关系，导致迭代过程计算量巨大，实际应用比较难。

# 六、最大熵马尔科夫模型（MEMM）

MEMM正因为是判别模型，所以不废话，我上来就直接为了确定边界而去建模，比如说序列求概率（分类）问题，我直接考虑找出函数分类边界。这一点跟HMM的思维方式发生了很大的变化，如果不对这一点有意识，那么很难理解为什么MEMM、CRF要这么做。

HMM中，观测节点$O_i$依赖隐藏状态节点$i_i$,也就意味着我的观测节点只依赖当前时刻的隐藏状态。但在更多的实际场景下，观测序列是需要很多的特征来刻画的，比如说，我在做NER时，我的标注$i_i$不仅跟当前状态$O_i$相关，而且还跟前后标注$O_j(j\neq i)$相关，比如字母大小写、词性等等。

为此，提出来的MEMM模型就是能够直接允许“定义特征”，直接学习条件概率，即

$$P\left(i_{i} \mid i_{i-1}, o_{i}\right)(i=1, \cdots, n)$$

$$P(I \mid O)=\prod_{t=1}^{n} P\left(i_{i} \mid i_{i-1}, o_{i}\right), i=1, \cdots, n$$

并且，$P\left(i \mid i^{\prime}, o\right)$这个概率通过最大熵分类器建模（取名MEMM的原因）:

$$P\left(i \mid i^{\prime}, o\right)=\frac{1}{Z\left(o, i^{\prime}\right)} \exp \left(\sum_{a} \lambda_{a} f_{a}(o, i)\right)$$

重点来了，这是ME的内容，也是理解MEMM的关键：$Z\left(o, i^{\prime}\right)$这部分是归一化；$f_{a}(o, i)$是特征函数，具体点，这个函数是需要去定义的；$\lambda$是特征函数的权重，这是个未知参数，需要从训练阶段学习而得。

所以，MEMM的建模公式为：

$$P(I \mid O)=\prod_{t=1}^{n} \frac{\exp \left(\sum_{a} \lambda_{a} f_{a}(o, i)\right)}{Z\left(o, i_{i-1}\right)}, i=1, \cdots, n$$

是的，公式这部分之所以长成这样，是由ME模型决定的。

请务必注意，理解判别模型和定义特征两部分含义，这已经涉及到CRF的雏形了。

MEMM需要两点注意：

1. 与HMM的$O_i$依赖$i_i$不一样，MEMM当前隐藏状态$i_i$应该是依赖当前时刻的观测节点$O_i$和上一时刻的隐藏节点$i_{i-1}$

2. 需要注意，之所以图的箭头这么画，是由MEMM的公式决定的，而公式是creator定义出来的。

MEMM模型的工作流程也包括了学习训练问题、序列标注问题、序列求概率问题。

# 七、最大熵模型理解

# 八、最大熵模型的最大似然函数解释

```python

```


# Búsqueda Tabú

La librería **Pyristic** incluye una clase llamada `TabuSearch` que facilita la implementación de una metaheurística basada en Búsqueda Tabú para resolver problemas de minimización. Para poder utilizar esta clase es necesario:

1. Definir:
    * La función objetivo $f$.
    * La lista de restricciones.
    * Estructura de datos (opcional).

2. Crear una clase que herede de `TabuSearch`.

3. Sobreescribir las siguientes funciones de la clase `TabuSearch`:
    * get_neighbors (requerido)
    * encode_change (requerido)

A continuación se muestran las librerías y elementos que se deben importar. Posteriormente, se resolverán dos problemas de optimización combinatoria usando la clase `TabuSearch`.

```python
import sys
import os

#library_path is the path where the Optimpy library is located.
library_path = "/home/dell/Documentos/Git_proejcts/pyristic/"
#library_path = "/Users/adrianamenchacamendez/Documentos/enes_morelia/papime/optimizacion-con-metaheuristicas/"
sys.path.append(os.path.abspath(library_path))
```

```python
from pyristic.heuristic.Tabu_search import TabuSearch
from pyristic.utils.helpers import *
from pprint import pprint
import numpy as np
import copy
```

## Problema de la mochila

\begin{equation}
  \label{eq:KP}
  \begin{array}{rll}
  \text{maximizar:} & f(\vec{x}) = \sum_{i=1}^{n} p_i \cdot x_{i} &  \\
  \text{donde: } & g_1(\vec{x}) = \sum_{i=1}^{n} w_i \cdot x_{i}  \leq c &  \\
          &  x_i \in \{0,1\} & i\in\{1,\ldots,n\}\\
  \end{array}
\end{equation}

Consideremos la siguiente entrada:
- $n = 5$
- $p = \{5, 14, 7, 2, 23\}$
- $w = \{2, 3, 7, 5, 10\}$
- $c = 15$

Donde la mejor solución es:
$x = [1, 1, 0, 0, 1]$ , $f(x) = 42$ y $g_{1}(x) = 15$

### Función objetivo

Dado que la clase `TabuSearch` considera problemas de minimización, es necesario convertir el problema de la mochila a un problema de minimización. Para esto se multiplica el valor de la función objetivo por -1.

```python
def f(x : np.ndarray) -> float:
    p = np.array([5,14,7,2,23])
    return -1*np.dot(x,p)
```

### Restricciones

Las restricciones se definen en funciones diferentes y se agregan a una lista.

```python
def g1(x : np.ndarray) -> bool:
    w = [2,3,7,5,10]
    return np.dot(x,w) <= 15

constraints_list= [g1]
```

En el problema de la mochila unicamente queremos revisar que no se exceda el peso.

### Uso de `TabuSearch`

Para poder hacer uso de la metaheurística de búsqueda tabú implementada en la librería **Pyristic**, es necesario crear una clase que herede de la clase `TabuSearch`.

```python
class Knapsack_solver(TabuSearch):

    def __init__(self, f_ : function_type , constraints_: list):
        super().__init__(f_,constraints_)

    def get_neighbors(self, x : np.ndarray,**kwargs) -> list:
        neighbors_list = []

        for i in range(len(x)):
            x[i] ^= 1 #1
            neighbors_list+=[copy.deepcopy(x)]
            x[i] ^= 1

        return neighbors_list

    def encode_change(self, neighbor : (list,np.ndarray), x : (list,np.ndarray),**kwargs) -> list: #2

        x_ = [None,None]

        for i in range(len(x)):
            if x[i] != neighbor[i]:
                return [i,neighbor[i]]

        return x_
```

La nueva clase es llamada *Knapsack_solver*, donde, se han sobrescrito las funciones `get_neighbors` y `encode_change`. Si no implementamos las funciones mencionadas el algoritmo no va a funcionar.

### Ejecución de la metaheurística

Una vez definida la clase *Knapsack_solver*, se crea un objeto de tipo *Knapsack_solver* indicando en los parámetros la función objetivo y las restricciones del problema. En este caso llamamos *Knapsack* al objeto creado.

```python
Knapsack = Knapsack_solver(f, [g1])
```

Finalmente, se llama a la función `optimize`. Esta función recibe tres parámetros:

* Solución inicial o función generadora de soluciones iniciales.
* El número de iteraciones.
* El tiempo donde evitaremos hacer un cambio en cierta posición (tiempo tabú).

Para este ejemplo usamos una mochila vacía ($x_0 = [0,0,0,0,0]$), $30$ iteraciones y un tiempo tabú igual a $3$.

```python
init_backpack_solution = np.zeros(5,dtype=int)
'''Parameters:
    Initial solution
    Number of iterations
    Tabu time
'''
Knapsack.optimize(init_backpack_solution,30,3)
print(Knapsack)
```

    100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 30/30 [00:00<00:00, 1586.49it/s]

    Tabu search:
     f(X) = -42
     X = [1 1 0 0 1]

A continuación resolveremos el mismo problema para una instancia más grande.

Tenemos que definir nuevamente la función objetivo y la restricción para emplearlo para cualquier instancia del problema.

Definiremos las siguientes variables como variables globales:
* n es un número que indicará el tamaño de nuestra instancia.
* p es un arreglo que se refiere al beneficio que proporciona cada uno de los objetos.
* w es un arreglo con el peso de cada uno de los objetos.
* c es el peso máximo que puede tener nuestra mochila.

```python
n = 50
p = [60, 52, 90, 57, 45, 64, 60, 45, 63, 94, 44, 90, 66, 64, 32, 39, 91, 40, 73, 61, 82, 94, 39, 68, 94, 98, 80, 79, 73, 99, 49, 56, 69, 49, 82, 99, 65, 34, 31, 85, 67, 62, 56, 38, 54, 81, 98, 63, 48, 83]
w = [38, 20, 21, 21, 37, 28, 32, 30, 33, 35, 29, 32, 35, 24, 28, 29, 22, 34, 31, 36, 36, 28, 38, 25, 38, 37, 20, 23, 39, 31, 27, 20, 38, 38, 36, 28, 39, 22, 23, 22, 21, 24, 23, 33, 31, 30, 32, 30, 22, 37]
c = 870
```

```python
def f(x : np.ndarray) -> float:
    global p
    return -1* np.dot(x,p)

def g1(x : np.ndarray) -> bool:
    global w,c
    result = np.dot(x,w)
    g1.__doc__="{} <= {}".format(result,c)
    return result <= c

constraints_list= [g1]
```

### Solución inicial

En el ejemplo anterior, la solución inicial fue una mochila vacía. Ahora crearemos una mochila que introduce objetos de manera aleatoria, mientras no se exceda el peso de la mochila.

```python
def getInitialSolution(NumObjects=5):
    global n,p,w,c
    #Empty backpack
    x = [0 for i in range(n)]
    weight_x = 0

    #Random order to insert objects.
    objects = list(range(n))
    np.random.shuffle(objects)

    for o in  objects[:NumObjects]:
        #Check the constraint about capacity.
        if weight_x + w[o] <= c:
            x[o] = 1
            weight_x += w[o]

    return np.array(x)
```

Definiremos nuestro objeto del tipo *Knapsack_solver* y llamaremos el método `optimize`con los siguientes parámetros:

* La función que crea la solución inicial.
* $100$ iteraciones.
* El tiempo tabú será $\frac{n}{2}$.

```python
Knapsack_2 = Knapsack_solver(f, [g1])
Knapsack_2.optimize(getInitialSolution,100,n//2)
print(Knapsack_2)
```

    100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 100/100 [00:00<00:00, 454.23it/s]

    Tabu search:
     f(X) = -2309
     X = [0 0 1 0 0 1 0 0 1 1 0 1 1 1 0 0 1 0 1 0 1 1 0 1 1 1 1 1 1 1 0 0 1 0 1 1 1
     0 0 1 1 1 1 0 0 1 1 0 0 1]
     Constraints:
     868 <= 870

Para revisar el comportamiento de la metaheurística en determinado problema, la librería **Pyristic** cuenta con una función llamada `get_stats`. Esta función se encuentra en **utils.helpers** y recibe como parámetros:

* El objeto creado para ejecutar la metaheurística.
* El número de veces que se quiere ejecutar la metaheurística.
* Los argumentos que recibe la función `optimize` (debe ser una tupla).

La función `get_stats` retorna un diccionario con algunas estadísticas de las ejecuciones.

```python
args = (getInitialSolution,500,n//2)
statistics = get_stats(Knapsack_2, 21, args)
```

    100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 500/500 [00:00<00:00, 913.61it/s]
    100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 500/500 [00:00<00:00, 1378.92it/s]
    100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 500/500 [00:00<00:00, 1339.77it/s]
    100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 500/500 [00:00<00:00, 1345.36it/s]
    100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 500/500 [00:00<00:00, 1261.83it/s]
    100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 500/500 [00:00<00:00, 1315.93it/s]
    100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 500/500 [00:00<00:00, 1378.99it/s]
    100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 500/500 [00:00<00:00, 1356.77it/s]
    100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 500/500 [00:00<00:00, 1348.58it/s]
    100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 500/500 [00:00<00:00, 1255.58it/s]
    100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 500/500 [00:00<00:00, 1341.23it/s]
    100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 500/500 [00:00<00:00, 1379.97it/s]
    100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 500/500 [00:00<00:00, 1380.98it/s]
    100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 500/500 [00:00<00:00, 1358.95it/s]
    100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 500/500 [00:00<00:00, 1250.25it/s]
    100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 500/500 [00:00<00:00, 1340.90it/s]
    100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 500/500 [00:00<00:00, 1377.66it/s]
    100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 500/500 [00:00<00:00, 1376.81it/s]
    100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 500/500 [00:00<00:00, 1363.01it/s]
    100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 500/500 [00:00<00:00, 910.14it/s]
    100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 500/500 [00:00<00:00, 1082.86it/s]

```python
pprint(statistics)
```

    {'Best solution': {'f': -2309,
                       'x': array([0, 0, 1, 0, 0, 1, 0, 0, 0, 1, 0, 1, 1, 1, 0, 0, 1, 0, 1, 0, 1, 1,
           0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 1, 0, 1, 1, 1, 0, 0, 1, 1, 1, 1, 0,
           0, 1, 1, 1, 0, 1])},
     'Mean': -2252.7619047619046,
     'Median': -2254.0,
     'Standard deviation': 38.34418657359848,
     'Worst solution': {'f': -2187,
                        'x': array([0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0, 0, 1, 0, 1, 0, 1, 1,
           0, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 0, 0, 1, 1, 0, 0, 1,
           0, 1, 1, 0, 0, 1])},
     'averageTime': 0.39534649394807364,
     'objectiveFunction': [-2214,
                           -2279,
                           -2284,
                           -2263,
                           -2305,
                           -2281,
                           -2309,
                           -2187,
                           -2273,
                           -2215,
                           -2222,
                           -2225,
                           -2254,
                           -2302,
                           -2230,
                           -2195,
                           -2307,
                           -2220,
                           -2248,
                           -2211,
                           -2284]}

## Problema del agente viajero

\begin{equation}
    \label{eq:TSP}
    \begin{array}{rll}
    \text{minimizar:} & f(x) = d(x_n, x_1) + \sum_{i=1}^{n-1} d(x_i, x_{i+1}) &  \\
    \text{tal que: } & x_i \in \{1,2,\cdots,n\} & \\
    \end{array}
\end{equation}

Donde:
* $d(x_i,x_j)$ es la distancia desde la ciudad $x_i$ a la ciudad $x_j$.
* $n$ es el número de ciudades.
* $x$ es una permutación de las $n$ ciudades.

```python
import random
```

```python
num_cities = 10
iterations = 100
dist_matrix = \
[\
[0,49,30,53,72,19,76,87,45,48],\
[49,0,19,38,32,31,75,69,61,25],\
[30,19,0,41,98,56,6,6,45,53],\
[53,38,41,0,52,29,46,90,23,98],\
[72,32,98,52,0,63,90,69,50,82],\
[19,31,56,29,63,0,60,88,41,95],\
[76,75,6,46,90,60,0,61,92,10],\
[87,69,6,90,69,88,61,0,82,73],\
[45,61,45,23,50,41,92,82,0,5],\
[48,25,53,98,82,95,10,73,5,0],\
]

```

```python
def f_salesman(x : np.ndarray) -> float:
    global dist_matrix
    total_dist = 0
    for i in range(1,len(x)):
        u,v = x[i], x[i-1]
        total_dist+= dist_matrix[u][v]
    total_dist += dist_matrix[x[-1]][0]
    return total_dist
```

```python
def g_salesman(x : np.ndarray) -> bool:
    """
    Xi in {1,2, ... , N}
    """
    size = len(x)
    size_ = len(np.unique(x))
    return size == size_
```

En este ejemplo mostraremos la forma de definir nuestra lista tabú para el problema del agente viajero para emplearla en nuestra búsqueda `TabuSearch`. Es necesario que nuestra lista tabú contenga los siguientes métodos:
- `reset`
- `update`
- `push`
- `find`

```python
class Tabu_Salesman_list:
    def __init__(self,timer):
        self.__TB = {}
        self.timer = timer

    def reset(self,timer) -> None:
        self.__TB = {}
        self.timer = timer

    def update(self) -> None:
        to_pop = []
        for key in self.__TB:
            if self.__TB[key]-1 == 0:
                to_pop.append(key)
            else:
                self.__TB[key]-=1
        for key in to_pop:
            self.__TB.pop(key)

    @checkargs
    #x has [p,v,step], we are only interested in v (value)
    def push(self, x : list ) -> None:
        self.__TB[x[1]] = self.timer

    @checkargs
    def find(self, x : list) -> bool:
        return x[1] in self.__TB

```

```python
class TravellingSalesman_solver(TabuSearch):

    def __init__(self, f_ : function_type , constraints_: list, TabuStorage):
        super().__init__(f_,constraints_,TabuStorage)

    @checkargs
    def get_neighbors(self, x : np.ndarray,**kwargs) -> list:

        neighbors_list = []

        ind = random.randint(1,len(x)-1)
        while  self.TL.find([-1,x[ind]]):
            ind = random.randint(1,len(x)-1)
        v = x[ind]
        x_tmp = list(x[v != x])
        for i in range(1, len(x)):
            if ind == i:
                continue
            neighbors_list += [ x_tmp[:i] + [v] + x_tmp[i:]]

        return neighbors_list

    @checkargs
    def encode_change(self, neighbor : (list,np.ndarray), x : (list,np.ndarray),**kwargs) -> list: #2

        x_p ={x[i] : i for i in range(len(x))}
        n_p = {neighbor[i]: i for i in range(len(x))}
        ind = -1
        max_dist = -1
        value = -1
        for i in range(1, len(x)):
            v = x[i]
            dist = abs(x_p[v] - n_p[v])
            if dist > max_dist:
                ind = i
                max_dist = dist
                value = v

        return [ind , value]
```

### Solución inicial
En este caso, creamos la solución inicial utilizando una estrategia voraz.

```python
def getInitialSolutionTS(distance_matrix, total_cities):
    Solution = [0]
    remaining_cities  = list(range(1,total_cities))

    while len(remaining_cities) != 0:
        from_ =Solution[-1]
        to_ = remaining_cities[0]
        dist = distance_matrix[from_][to_]

        for i in range(1, len(remaining_cities)):
            distance = distance_matrix[from_][remaining_cities[i]]
            if distance < dist:
                to_ = remaining_cities[i]
                dist = distance
        Solution.append(to_)
        ind = remaining_cities.index(to_)
        remaining_cities.pop(ind)
    return Solution
```

```python
TravellingSalesman = TravellingSalesman_solver(f_salesman,[g_salesman],Tabu_Salesman_list(num_cities//2))
init_path = np.array(getInitialSolutionTS(dist_matrix,num_cities))
print("Initialize search with this initial point {} \n f(x) = {}".format(init_path, f_salesman(init_path)))
```

    Initialize search with this initial point [0 5 3 8 9 6 2 7 1 4]
     f(x) = 271

```python
TravellingSalesman.optimize(init_path, iterations, num_cities//2)
print(TravellingSalesman)
```

    Tabu search:
     f(X) = 248
     X = [0 5 3 8 9 6 2 7 4 1]
     Constraints:

        Xi in {1,2, ... , N}

```python
args = (init_path, iterations, num_cities//2)
statistics = get_stats(TravellingSalesman, 30, args)

```

```python
pprint(statistics)
```

    {'Best solution': {'f': 248, 'x': array([0, 5, 3, 8, 9, 6, 2, 7, 4, 1])},
     'Mean': 248.0,
     'Median': 248.0,
     'Standard deviation': 0.0,
     'Worst solution': {'f': 248, 'x': array([0, 5, 3, 8, 9, 6, 2, 7, 4, 1])}}