# Spectral Analysis of Deterministic Signals

*This jupyter notebook is part of a [collection of notebooks](../index.ipynb) on various topics of Digital Signal Processing. Please direct questions and suggestions to [Sascha.Spors@uni-rostock.de](mailto:Sascha.Spors@uni-rostock.de).*

## Introduction

The analysis of the spectral properties of a signal plays an important role in digital signal processing. Some application examples are the

* [Spectrum analyzer](https://en.wikipedia.org/wiki/Spectrum_analyzer)
* Detection of (harmonic) signals
* [Estimation of fundamental frequency and harmonics](https://en.wikipedia.org/wiki/Modal_analysis)
* Spectral suppression: acoustic echo suppression, noise reduction, ...

In the practical realization of spectral analysis techniques the [discrete Fourier transformation](https://en.wikipedia.org/wiki/Discrete_Fourier_transform) (DFT) is applied to discrete finite-length signals in order to gain insights into their spectral composition. A basic task in spectral analysis is to determine the amplitude (and phase) of dominant harmonic contributions in a signal mixture. The properties of the DFT with respect to the analysis of an harmonic exponential signal are discussed in the following.

## The Leakage Effect

[Spectral leakage](https://en.wikipedia.org/wiki/Spectral_leakage) is a fundamental effect of the DFT. It limits the ability to detect harmonic signals in signal mixtures and hence the performance of spectral analysis. In order to discuss this effect, first the DFT of a single harmonic exponential signal is regarded. Its spectrum is derived in four steps:

1. Fourier transform of an harmonic exponential signal,
2. discrete-time Fourier transform (DTFT) of a discrete harmonic exponential signal, and
3. DTFT of a finite-length discrete harmonic exponential signal
4. sampling of the DTFT

These steps are detailed in the remaining subsections.

### Fourier Transformation of an Exponential Signal

The harmonic exponential signal is defined as

\begin{equation}
x(t) = \mathrm{e}^{\,\mathrm{j}\, \omega_0 \, t}
\end{equation}

where $\omega_0 = 2 \pi f$ denotes the angular frequency of the signal. The Fourier transform of the exponential signal is

\begin{equation}
X(\mathrm{j}\, \omega) = \int\limits_{-\infty}^{\infty} x(t) \,\mathrm{e}^{\,- \mathrm{j}\, \omega \,t} \mathrm{d}t = 2\pi \; \delta(\omega - \omega_0)
\end{equation}

The spectrum consists of a single shifted Dirac impulse located at the angular frequency $\omega_0$ of the exponential signal. Hence the spectrum $X(\mathrm{j}\, \omega)$ consists of a clearly isolated and distinguishable event. In practice, it is not possible to compute the Fourier transformation of a continuous signal by means of digital signal processing.

### Discrete-Time Fourier Transformation of the Discrete Exponential Signal

Now lets consider sampled signals. The discrete exponential signal $x[k]$ is derived from its continuous counterpart $x(t)$ above by equidistant sampling $x[k] := x(k T)$ with the sampling interval $T$

\begin{equation}
x[k] = \mathrm{e}^{\,\mathrm{j}\, \Omega_0 \,k}
\end{equation}

where $\Omega_0 = \omega_0 T$ denotes the normalized angular frequency. The [discrete-time Fourier transform](https://en.wikipedia.org/wiki/Discrete-time_Fourier_transform) (DTFT) is the Fourier transformation of a sampled signal. For the exponential signal it is given as

\begin{equation}
X(\mathrm{e}^{\,\mathrm{j}\, \Omega}) = \sum_{k = -\infty}^{\infty} x[k]\, \mathrm{e}^{\,-\mathrm{j}\, \Omega \,k} = 2\pi \sum_{n = -\infty}^{\infty} \delta((\Omega-\Omega_0) - 2\,\pi\,n)
\end{equation}

The spectrum of the DTFT is $2\pi$-periodic due to sampling. As a consequence, the transformation of the discrete exponential signal consists of a series Dirac impulses. For the region of interest $-\pi < \Omega \leq \pi$ the spectrum consists of a clearly isolated and distinguishable event, as for the continuous case.

The DTFT cannot be realized in practice, since is requires the knowledge of the signal $x[k]$ for all time instants $k$. In general, a measured signal is only known within a finite time-interval. The DFT of a signal of finite length can be derived from the DTFT in two steps:

1. truncation (windowing) of the signal and
2. sampling of the DTFT spectrum of the windowed signal.

The consequences of these two steps are investigated in the following.

### Discrete-Time Fourier Transformation of a Truncated Discrete Exponential Signal

In general, truncation of a signal $x[k]$ to a length of $N$ samples is modeled by multiplying the signal with a window function $w[k]$ of length $N$

\begin{equation}
x_N[k] = x[k] \cdot w[k]
\end{equation}

where $x_N[k]$ denotes the truncated signal and $w[k] = 0$ for $\{k: k < 0 \wedge k \geq N \}$. The spectrum $X_N(\mathrm{e}^{\,\mathrm{j}\, \Omega})$ can be derived from the multiplication theorem of the DTFT as

\begin{equation}
X_N(\mathrm{e}^{\,\mathrm{j}\, \Omega}) = \frac{1}{2 \pi} X(\mathrm{e}^{\,\mathrm{j}\, \Omega}) \circledast_N W(\mathrm{e}^{\,\mathrm{j}\, \Omega})
\end{equation}

where $\circledast$ denotes the cyclic/[circular convolution](https://en.wikipedia.org/wiki/Circular_convolution) of length $N$. A hard truncation of the signal to $N$ samples is modeled by the rectangular signal

\begin{equation}
w[k] = \text{rect}_N[k] = \begin{cases}
1 & \mathrm{for} \; 0\leq k<N \\
0 & \mathrm{otherwise}
\end{cases}
\end{equation}

Its spectrum is given as

\begin{equation}
W(\mathrm{e}^{\,\mathrm{j}\, \Omega}) = \mathrm{e}^{\,-\mathrm{j} \, \Omega \,\frac{N-1}{2}} \cdot \frac{\sin(\frac{N \,\Omega}{2})}{\sin(\frac{\Omega}{2})}
\end{equation}

The DTFT $X_N(\mathrm{e}^{\,\mathrm{j}\, \Omega})$ of the truncated exponential signal is derived by introducing the DTFT of the exponential signal and the window function, exploiting the properties of the Dirac impulse and the cyclic convolution as

\begin{equation}
X_N(\mathrm{e}^{\,\mathrm{j}\, \Omega}) = \mathrm{e}^{\,-\mathrm{j}\, (\Omega-\Omega_0) \, \frac{N-1}{2}} \cdot \frac{\sin(\frac{N\, (\Omega-\Omega_0)}{2})}{\sin(\frac{(\Omega-\Omega_0)}{2})}
\end{equation}

Clearly the DTFT of the truncated harmonic exponential signal $x_N[k]$ is not given by a series of Dirac impulses. Above equation is evaluated numerically in order to illustrate the properties of $X_N(\mathrm{e}^{\,\mathrm{j}\, \Omega})$.

```python
%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt

Om0 = 1  # frequency of exponential signal
N = 32  # length of signal

# DTFT of finite length exponential signal (analytic)
Om = np.linspace(-np.pi, np.pi, num=1024)
XN = np.exp(-1j * (Om-Om0) * (N-1) / 2) * (np.sin(N * (Om-Om0) / 2)) / (np.sin((Om-Om0) / 2))

# plot spectrum
plt.figure(figsize = (10, 8))
plt.plot(Om, abs(XN), 'r')
plt.title(r'Absolute value of the DTFT of a truncated exponential signal $e^{j \Omega_0 k}$ with $\Omega_0=$%2.2f' %Om0)
plt.xlabel(r'$\Omega$')
plt.ylabel(r'$|X_N(e^{j \Omega})|$')
plt.axis([-np.pi, np.pi, -0.5, N+5])
plt.grid()
```

**Excercise**

* Change the frequency `Om0` of the signal and rerun the example. How does the magnitude spectrum change?
* Change the length `N` of the signal and rerun the example. How does the magnitude spectrum change?

The maximum of the absolute value of the spectrum is located at the frequency $\Omega_0$. It should become clear that truncation of the exponential signal leads to a broadening of the spectrum. The shorter the signal, the wider the mainlobe becomes.

### The Leakage Effect of the Discrete Fourier Transformation

The DFT is derived from the DTFT $X_N(\mathrm{e}^{\,\mathrm{j}\, \Omega})$ of the truncated signal $x_N[k]$ by sampling the DTFT equidistantly at $\Omega = \mu \frac{2 \pi}{N}$

\begin{equation}
X[\mu] = X_N(\mathrm{e}^{\,\mathrm{j}\, \Omega})\big\vert_{\Omega = \mu \frac{2 \pi}{N}}
\end{equation}

For the DFT of the exponential signal we finally get

\begin{equation}
X[\mu] = \mathrm{e}^{\,\mathrm{j}\, (\Omega_0 - \mu \frac{2 \pi}{N}) \frac{N-1}{2}} \cdot \frac{\sin(\frac{N \,(\Omega_0 - \mu \frac{2 \pi}{N})}{2})}{\sin(\frac{\Omega_0 - \mu \frac{2 \pi}{N}}{2})}
\end{equation}

The sampling of the DTFT is illustrated in the following example. Note that the normalized angular frequency $\Omega_0$ has been expressed in terms of the periodicity $P$ of the exponential signal $\Omega_0 = P \; \frac{2\pi}{N}$.

```python
N = 32  # length of the signal
P = 10.33  # periodicity of the exponential signal
Om0 = P * (2*np.pi/N)  # frequency of exponential signal

# truncated exponential signal
x = np.exp(1j*Om0*np.arange(N))

# DTFT of finite length exponential signal (analytic)
Om = np.linspace(0, 2*np.pi, num=1024)
Xw = np.exp(-1j*(Om-Om0)*(N-1)/2)*(np.sin(N*(Om-Om0)/2))/(np.sin((Om-Om0)/2))

# DFT of the exponential signal by FFT
X = np.fft.fft(x)
mu = np.arange(N) * 2*np.pi/N

# plot spectra
plt.figure(figsize = (10, 8))
plt.hold(True)

plt.plot(Om, abs(Xw), 'r', label=r'$X_N(e^{j \Omega})$')
plt.stem(mu, abs(X), label=r'$X_N[\mu]$', basefmt=' ')
plt.title(r'Absolute value of the DTFT/DFT of a truncated exponential signal $e^{j \Omega_0 k}$ with $\Omega_0=$%2.2f' %Om0)

plt.xlabel(r'$\Omega$')
plt.ylabel(r'$|X_N(e^{j \Omega})|$, $|X[\mu]|$')
plt.axis([0, 2*np.pi, -0.5, N+5]);
plt.legend()
plt.grid()

plt.show()
```

**Exercise**

* Change the periodicity `P` of the exponential signal and rerun the example. What happens if the periodicity is an integer? Why?
* Change the length `N` of the DFT? How does the spectrum change?
* What conclusions can be drawn for the analysis of a single exponential signal by the DFT?

You should have noticed that for an exponential signal whose periodicity is an integer $P \in \mathbb{N}$, the DFT consists of a discrete Dirac pulse $X[\mu] = \delta[\mu - P]$. In this case, the sampling points coincide with the maximum of the main lobe or the zeros of the DTFT. For non-integer $P$, hence non-periodic exponential signals with respect to the signal length $N$, the DFT has additional contributions. The shorter the length $N$, the wider these contributions are spread in the spectrum. This smearing effect is known as *leakage effect* of the DFT. It limits the achievable frequency resolution of the DFT when analyzing signal mixtures consisting of more than one exponential signal. This is illustrated by the following numerical examples.

### Analysis of Signal Mixtures by the Discrete Fourier Transformation

In order to discuss the implications of the leakage effect when analyzing signal mixtures, the superposition of two exponential signals with different amplitudes and frequencies is considered

\begin{equation}
x_N[k] = A_1 \cdot e^{\mathrm{j} \Omega_1 k} + A_2 \cdot e^{\mathrm{j} \Omega_2 k}
\end{equation}

For convenience, a function is defined that calculates and plots the magnitude spectrum of $x_N[k]$.

```python
def dft_signal_mixture(N, A1, P1, A2, P2):
    # N: length of signal/DFT
    # A1, P1, A2, P2: amplitude and periodicity of 1st/2nd complex exponential

    # generate the signal mixture
    Om0_1 = P1 * (2*np.pi/N)  # frequency of 1st exponential signal
    Om0_2 = P2 * (2*np.pi/N)  # frequency of 2nd exponential signal
    k = np.arange(N)
    x = A1 * np.exp(1j*Om0_1*k) + A2 * np.exp(1j*Om0_2*k)

    # DFT of the signal mixture
    mu = np.arange(N)
    X = np.fft.fft(x)

    # plot spectrum
    plt.figure(figsize = (10, 8))
    plt.stem(mu, abs(X), basefmt=' ')
    plt.title(r'Absolute value of the DFT of a signal mixture')
    plt.xlabel(r'$\mu$')
    plt.ylabel(r'$|X[\mu]|$')
    plt.axis([0, N, -0.5, N+5]);
    plt.grid()
```

Lets first consider the case that the frequencies of the two exponentials are rather far apart

```python
dft_signal_mixture(32, 1, 10.3, 1, 15.2)
```

Investigating the magnitude spectrum one could conclude that the signal consists of two major contributions at the frequencies $\mu_1 = 10$ and $\mu_2 = 15$. Now lets take a look at a situation where the frequencies are closer together

```python
dft_signal_mixture(32, 1, 10.3, 1, 10.9)
```

From visual inspection of the spectrum it is rather unclear if the mixture consists of one or two exponential signals. So far the levels of both signals where chosen equal.

Lets consider the case where the second signal has a much lower level that the first one. The frequencies have been chosen equal to the first example

```python
dft_signal_mixture(32, 1, 10.3, 0.1, 15.2)
```

Now the contribution of the second exponential is almost hidden in the spread spectrum of the first exponential. From these examples it should have become clear that the leakage effect limits the spectral resolution of the DFT.

**Copyright**

This notebook is provided as [Open Educational Resource](https://en.wikipedia.org/wiki/Open_educational_resources). Feel free to use the notebook for your own purposes. The text is licensed under [Creative Commons Attribution 4.0](https://creativecommons.org/licenses/by/4.0/), the code of the IPython examples under the [MIT license](https://opensource.org/licenses/MIT). Please attribute the work as follows: *Sascha Spors, Digital Signal Processing - Lecture notes featuring computational examples, 2016-2017*.


```
# default_exp core
```

# pySplitting

> This is a port of the Mathematica script to python3.
> First we define a number of helper functions. The important part is CreateConditions at the end.
> We have sympy as dependency and a suitably recent Python3.

```
#hide
from nbdev.showdoc import *
```

```
%nbdev_export

from sympy import *
from sympy import sympify
from itertools import *

# Mathematica counts from 1, python from zero!!!!!!
def findpos(lam):
    """This determines whether the index string is a strictly ascending sequence.

    Parameters
    ----------
    lam : an indexable object of things that are comparable. Usually an array of integers

    Returns
    -------
    bool : the position in the string where two ints are equal.
    """
    retval = -1
    i = 0
    while (i < len(lam)-1) and (retval == -1):
        if lam[i] == lam[i+1]:
            i = i + 1
        else:
            retval = i
    if retval == -1:
        retval = len(lam)-1
    return retval
```

```
%nbdev_export
def isstrictlyascending(lam):
    """This determines whether the index string is a strictly ascending sequence

    Parameters
    ----------
    lam : an indexable object of things that are comparable. Usually an array of integers

    Returns
    -------
    bool : true if the sequence is strictly ascending, else false.
    """
    retval = True
    for i in range(0, len(lam)-1):
#        print(lam[i],lam[i+1])
        retval = retval and lam[i] < lam[i+1]
#    i = 0
#    while (i < len(lam)-1) and retval == True:
#        if StrictLessThan(lam[i], lam[i+1]):
#        if sympify(lam[i] < lam[i+1]):
#            i = i+1
#        else:
#            retval = False
    return retval
```

```
%nbdev_export
def alpha(lam):
    """
    The alpha factor. This determines the real alpha coefficient as given by Thalhammer and elaborated by Blanes.

    Parameters
    ----------
    lam : A multiindex of integers.

    Returns
    -------
    Rational: alpha(lam)
    """
    retval = 1
    if (len(lam) > 1) and not isstrictlyascending(lam):
        pos = findpos(lam)
        retval = 1/factorial(pos+1)*alpha(lam[pos+1:])
    return retval
```

```
# Let's try a test. This should give 1/6
assert alpha([1,1,1,2])==Rational(1,6)
```

```
%nbdev_export
def CreateMuVectors(p,k):
    """This function creates the set of possible $$\mu$$ vectors.

    Parameters
    ----------
    p : the considered order
    k : 1 <=k<=p

    Returns
    -------
    array : A list of multiindices
    """
    absmu = p-k
    retval = []
    for el in product(range(0,absmu+1),repeat=k):
        if sum(el)<=absmu:
            retval.append(el)
    return retval
```

```
%nbdev_export
def CreateLyndonIndices(p,k):
    """This function creates the set of Lyndon indices.

    Parameters
    ----------
    p : the considered order
    k : 1 <=k<=p

    Returns
    -------
    array : A list of Lyndon multiindices
    """
    retval = []
    S = list(range(1,p+1))#create the alphabet
    w = [S[0] - 1]
    while len(w) > 0:
        w[-1]=w[-1]+1
        m = len(w)
        if m == k:
            str = []
            for it in w:
                str.append(S[it-1])
            retval.append(str)
        # repeat w to get a string of length n
        while len(w) < k:
            w.append(w[-m])
        # remove the last character as long as it is equal to the largest character in S
        while (len(w) > 0) and w[-1] == S[-1]:
            w.pop()

    retval2=[]
    for el in retval:
        if sum(el)<=p:
            retval2.append([num - 1 for num in el])# subtract 1 from the final result. FIXME: this could be optimized
    return retval2
```

```
%nbdev_export
def CreateEquation(mu, bvec, cvec):
    """This function gives the coefficient for a particular multiindex mu.

    It is the real coefficient of a Len(mu) long product of iterated commutators [A, B]_(mu_k).

    Parameters
    ----------
    mu : a multiindex
    bvec : a string of symbols.
    cvec : a string of symbols(usually the partial sums of the avec).

    Returns
    -------
    expr : a symbolic expression, a polynomial in terms of the contents of bvec and cvec.
"""
    k = len(mu)
    retval1=1
    for l in range(k):
        retval1=retval1*1/sympify(sum(mu[l:])+k-l)
    #print(retval1)
    retval2=0
    for it in combinations_with_replacement([*range(len(bvec),0,-1)],r=k):
        retvalprod = 1
        for i in range(k):
            retvalprod = retvalprod*bvec[it[i]-1]*cvec[it[i]-1]**mu[i]
        retval2=retval2 + alpha(it)*retvalprod

    return retval1-retval2
```

```
%nbdev_export

from itertools import accumulate

def CreateConditions(p,avec,bvec,indexgenerator=CreateLyndonIndices):
    """This creates the set of equations using by default the Lyndon Basis elements.

    Parameters
    ----------
    p : the considered order
    avec: The set of symbols to use for the first operator.
    bvec: The set of symbols to use for the second operator.
    indexgenerator: (optional) by default we use indexgenerator for the Lyndon indices. Using CreateMuVectors
                     the indices from the overcomplete Hall-Basis can be used.

    Returns
    -------
    array : An array of Equations that have to be satisfied to fulfill the requested order p.
    """
    cvec=[*accumulate(avec)]
    cvec[-1]=1
    retval = [Eq(sum(avec),1)]
    for k in range(1,p+1):
        vecs=indexgenerator(p,k)
        for mu in vecs:
            retval.append(Eq(CreateEquation(mu,bvec,cvec),0))
    return retval
```

Having all the functions available we can compare the reductions due to the Lyndon indices in comparison to the Hall Basis.

```
avec=[*symarray('a',3)]
bvec=[*symarray('b',3)]
print("Equations in the Lyndon Basis")
print(CreateConditions(3,avec,bvec))
print("=========================")
print("Equations in the Hall Basis")
print(CreateConditions(3,avec,bvec,CreateMuVectors))
```

    Equations in the Lyndon Basis
    [Eq(a_0 + a_1 + a_2, 1), Eq(-b_0 - b_1 - b_2 + 1, 0), Eq(-a_0*b_0 - b_1*(a_0 + a_1) - b_2 + 1/2, 0), Eq(-a_0**2*b_0 - b_1*(a_0 + a_1)**2 - b_2 + 1/3, 0), Eq(-a_0*b_0**2/2 - a_0*b_0*b_1 - a_0*b_0*b_2 - b_1**2*(a_0 + a_1)/2 - b_1*b_2*(a_0 + a_1) - b_2**2/2 + 1/6, 0)]
    =========================
    Equations in the Hall Basis
    [Eq(a_0 + a_1 + a_2, 1), Eq(-b_0 - b_1 - b_2 + 1, 0), Eq(-a_0*b_0 - b_1*(a_0 + a_1) - b_2 + 1/2, 0), Eq(-a_0**2*b_0 - b_1*(a_0 + a_1)**2 - b_2 + 1/3, 0), Eq(-b_0**2/2 - b_0*b_1 - b_0*b_2 - b_1**2/2 - b_1*b_2 - b_2**2/2 + 1/2, 0), Eq(-a_0*b_0**2/2 - a_0*b_0*b_1 - a_0*b_0*b_2 - b_1**2*(a_0 + a_1)/2 - b_1*b_2*(a_0 + a_1) - b_2**2/2 + 1/6, 0), Eq(-a_0*b_0**2/2 - b_0*b_1*(a_0 + a_1) - b_0*b_2 - b_1**2*(a_0 + a_1)/2 - b_1*b_2 - b_2**2/2 + 1/3, 0), Eq(-b_0**3/6 - b_0**2*b_1/2 - b_0**2*b_2/2 - b_0*b_1**2/2 - b_0*b_1*b_2 - b_0*b_2**2/2 - b_1**3/6 - b_1**2*b_2/2 - b_1*b_2**2/2 - b_2**3/6 + 1/6, 0)]


In the code segment below, we import the tools we are going to need and bring in our training data. After that on the tenth line of the code we remove any nan values in the data. Then in the next line we randomize the order of the data. The last line we print the shape of the array our data is stored in.

```python
import numpy as np
import keras
import pandas
from keras_tqdm import TQDMNotebookCallback
from sklearn.preprocessing import StandardScaler

data = np.array(pandas.read_csv("./training_noavg.csv", header=0))
## Have to drop all the rows that have nan values because they will not help with net
## clean out rows with nan values
data = data[~np.isnan(data).any(axis=1)]
np.random.shuffle(data)

print(data.shape)
```

    (1543, 9)

Next, we take and get a standardscaler. next we split the data from what location it is at and put it in x. after we split the data we use the scaler from earlier to transform the data. then we take the locations from earlier and put them in an array label then we use keras.utils.to_categorical to turn it into one hot encoding.

```python
from sympy import *
init_printing(use_latex=True)

import matplotlib.pyplot as plt
%matplotlib inline

## we will use scaled data
scaler = StandardScaler()
## when testing predicitions
## X = scaler.fit_transform( X )
## test = scaler.transform( test )

X = data[:,0:8]
X = scaler.fit_transform(X)
print(X.shape)
display(X)

labels = data[:,8]
print(labels.shape)
display(labels)

Y = keras.utils.to_categorical(labels, len(np.unique(labels)))
```

    (1543, 8)

    array([[-2.94439193, -0.76184435, -0.29408942, ...,  0.74520521,
             1.18529535,  0.0363514 ],
           [ 0.57228641,  0.20786143,  1.2706691 , ..., -1.64675956,
            -0.79404526,  1.11500938],
           [ 0.92395424, -1.73155012,  1.04713217, ..., -0.27992255,
             1.51518545,  0.252083  ],
           ...,
           [-0.48271709,  0.93514076, -1.41177407, ...,  0.74520521,
             0.19562504, -1.68950137],
           [ 0.57228641,  0.20786143,  1.04713217, ..., -1.64675956,
            -1.12393536,  1.33074098],
           [-0.13104926,  1.1775672 , -1.18823714, ...,  0.74520521,
             0.19562504, -1.47376977]])

    (1543,)

    array([2., 1., 2., ..., 0., 1., 0.])

```python
input_size = X.shape[1]
output_size = Y.shape[1]
display(X.shape[1])
```

The code below is the declaration of the network. we are using a multilayer net the hidden layers are declared on the third and fourth lines below. They are both given 64 hidden units and fed the input dimensions which is the number of columns in this case 8. In the first hidden layer we used the relu activation function and the second is using sigmoid. we are using these activation functions because they proved to work the best for our data set. and we have set the bis initializers to 0.01. For the output layer which is on line 6 of the code below we used the activation function SoftMax. And then we compile the model using the categorical crossentropy loss function and adam and the optimizer. the output below the code segment is the summary of the model.

```python
model = keras.models.Sequential()

model.add(keras.layers.Dense(64,input_dim=8,activation='relu', bias_initializer=keras.initializers.Constant(value=0.01)))
model.add(keras.layers.Dense(64,input_dim=8,activation='sigmoid', bias_initializer=keras.initializers.Constant(value=0.01)))

model.add(keras.layers.Dense(3,activation='softmax'))
#categorical_crossentropy
model.compile(loss='categorical_crossentropy',optimizer='adam',metrics=['accuracy'])

print(model.summary())
```

    _________________________________________________________________
    Layer (type)                 Output Shape              Param #
    =================================================================
    dense_4 (Dense)              (None, 64)                576
    _________________________________________________________________
    dense_5 (Dense)              (None, 64)                4160
    _________________________________________________________________
    dense_6 (Dense)              (None, 3)                 195
    =================================================================
    Total params: 4,931
    Trainable params: 4,931
    Non-trainable params: 0
    _________________________________________________________________
    None

Now we train the model with our training data. for our data set 100 epochs was enough, but a different data set may need more or less, epochs is the number of times the modal runs through the data set. we are using a validation split to check the accuracy of the modal on data it is not training on. a validation split takes part of the training data and keeps it to test with instead of training with it.

```python
history = model.fit(X, Y,
    batch_size=56,
    epochs=100,
    verbose=0,
    callbacks=[TQDMNotebookCallback()],
    validation_split = 0.2)
```

    HBox(children=(IntProgress(value=0, description='Training', style=ProgressStyle(description_width='initial')),…

    HBox(children=(IntProgress(value=0, description='Epoch 0', max=1234, style=ProgressStyle(description_width='in…

```python
plt.figure(1)

plt.subplot(211)
plt.plot(history.history['acc'])
plt.plot(history.history['val_acc'])
plt.title('model accuracy')
plt.ylabel('accuracy')
plt.xlabel('epoch')
plt.legend(['train', 'test'], loc='upper left')

plt.subplot(212)
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.title('model loss')
plt.ylabel('loss')
plt.xlabel('epoch')
plt.legend(['train', 'test'], loc='upper left')
plt.tight_layout()
plt.show()

score = model.evaluate(X, Y, verbose=1)
print('Test loss:', score[0])
print('Test accuracy:', score[1])
```

Above are graphs that show how the model did in its training as you can see the accuracy is basically 100%. and the loss is low that is what we are looking for. and right below we are saving the model and its weights to be used in the demo.

```python
model.save_weights('./Demo/MLN.weights')
model.save('./Demo/MLN.model')
```

```python

```


# Projector
$Def$

If a square matrix $P \in \mathbb{R}^{m\times m}$ satisfies $P^2 = P$, then we call $P$ a **projector**. **Range** of $A$, the set of images, $i.e.$, *the space spanned by the columnss of matrix $A$*. **Null** of $A$, all vectors whose images are zero, $i.e.$, all $\vec{x}$ satisfying $A\vec{x} = \vec{0}$.

$Theorem$ **1**

For ***any*** matrix $A\in \mathbb{R}^{m \times m}$, $\DeclareMathOperator*{\null}{null} \DeclareMathOperator*{\range}{range} \boxed{ \null\left(A\right) \subseteq \range\left(I-A\right) }$.

$Proof$

If $\vec{x} \in \null \left(A\right)$, then $A\vec{x} = \vec{0}$. Then $\vec{x} = \vec{x} - A\vec{x} = \left(I - A \right) \vec{x} \in \range\left(I-A\right)$

$Theorem$ **2**

Let $P \in \mathbb{R}^{m \times m}$. Then $\boxed{ P^2 = P}$ is *eqivalent* to $\boxed{ \null(P) = \range(I - P)}$

$Proof$

$\Rightarrow)$ Since we already have $\null (P) \subseteq \range(I-P)$ from *Theorem 1*, now we prove that $\null(P) \supseteq \range(I-P)$. So for $\vec{x} \in \range(I-P)$, there exists a $\vec{y}$ such that $\vec{x} = (I-P)\vec{y}$, so that $P\vec{x} = P(I-P)\vec{y} = (P - P^2)\vec{y} = \vec{0}$, so $\vec{x} \in \null(P)$. Half Done!

$\Leftarrow)$ Since now $\forall \vec{x} \in \mathbb{R}^{m}$, $(I-P)\vec{x} \in \range(I - P) = \null(P)$, so that $P\big( (I-P)\vec{x} \big) = (P - P^2)\vec{x} = \vec{0}$, that is to say that in terms of map, $P$ and $P^2$ are equivalent, $i.e.$, $P = P^2$. ALL DONE! $\square$

In general, any projector $P \in \mathbb{R}^{ m \times m}$ maps $\mathbb{R}^{ m}$ onto its **range**. For any vector $\vec{v}$ in the range of $P$, $P\vec{v} = \vec{v}$. For any vector $\vec{v}$ *not* in the **range** of $P$, the difference $\vec{v} − P\vec{v}$ is in the **null** of $P$.

$Def$

**orthogonal projector** and **oblique projector**: Let $P \in \mathbb{R}^{ m \times m}$ be a projector. If $\range(I-P) \perp \range(P)$, $i.e.$, $\null(P) \perp \range(P)$, then it is an orthogonal projector, otherwise oblique projector.

>**e.g.** $B = \left[ \begin{array}{ccc}
1 & 1\\
0 & 0
\end{array}\right]$
>
>Now $B^2 = \left[ \begin{array}{ccc}
1 & 1\\
0 & 0
\end{array}\right] = B$, so it is a projector. And its **range** is the space spanned by $\left[ \begin{array}{c}
1\\
0
\end{array}\right]$, and its **null** is spanned by $\left[ \begin{array}{c}
1\\
-1
\end{array}\right]$.
>
>And obviously $\range(B) \not \perp \null(B)$

How to determine whether it is oblique or orthogonal projector?

$Theorem$ **3**

Let $P \in \mathbb{R}^{ m \times m}$ be a projector. $P$ is an *orthogonal projector* $iff$ $\boxed{ P = P^{\mathrm{T}} }$.

$Proof$

$\Rightarrow)$ Denote the dimension of $\range(P)$ by $r$, so that the dimension of $\null(P)$ is $m-r$. Denote the basis of $\range(P)$ by $\vec{q}_{1}, \vec{q}_{2},\dots,\vec{q}_{r}$, and the basis of $\null(P)$ by $\vec{q}_{r+1}, \vec{q}_{r+2},\dots,\vec{q}_{m}$. Since $P$ is orthogonal projector, $\{ \vec{q}_{1}, \vec{q}_{2},\dots,\vec{q}_{r} \}$ are orthogonal to $\{ \vec{q}_{r+1}, \vec{q}_{r+2},\dots,\vec{q}_{m} \}$.

Let $Q = \left[ \begin{array}{cccc}
\vec{q}_1 & \vec{q}_2 & \cdots & \vec{q}_m
\end{array}\right]$, so that $Q$ is an orthogonal matrix, $Q^{\mathrm{T}}Q = I$. Then we have

$$\begin{align}
Q^{\mathrm{T}}PQ =& \left[ \begin{array}{cccc}
\vec{q}_1 & \vec{q}_2 & \cdots & \vec{q}_m
\end{array}\right]^{\mathrm{T}} P \left[ \begin{array}{cccc}
\vec{q}_1 & \vec{q}_2 & \cdots & \vec{q}_m
\end{array}\right] \\
=& \left[ \begin{array}{cccc}
\vec{q}_1 & \vec{q}_2 & \cdots & \vec{q}_m
\end{array}\right]^{\mathrm{T}} \left[ \begin{array}{cccc}
P\vec{q}_1 & P\vec{q}_2 & \cdots & P\vec{q}_m
\end{array}\right] \\
=& \left[ \begin{array}{ccccccc}
\vec{q}_1 & \vec{q}_2 & \cdots & \vec{q}_r & \vec{q}_{r+1} & \cdots & \vec{q}_m
\end{array}\right]^{\mathrm{T}} \left[ \begin{array}{ccccccc}
\vec{q}_1 & \vec{q}_2 & \cdots & \vec{q}_r & 0 & \cdots & 0
\end{array}\right] \\
=& I_r
\end{align}$$

Therefore

$$\begin{align}
P =& QI_rQ^{\mathrm{T}} \\
=& Q(I_rI_r)Q^{\mathrm{T}} = (QI_r)(I_rQ^{\mathrm{T}}) \\
=& \left[ \begin{array}{cccc}
\vec{q}_1 & \vec{q}_2 & \cdots & \vec{q}_r
\end{array}\right] \left[ \begin{array}{cccc}
\vec{q}_1^{\mathrm{T}} & \vec{q}_2^{\mathrm{T}} & \cdots & \vec{q}_r^{\mathrm{T}}
\end{array}\right]^{\mathrm{T}} \\
=& Q_r Q_r^{\mathrm{T}}
\end{align}$$

OK, now $P$ is symmetrix.

$Conclusion$

If $P$ is orthogonal projector, then $P$ can be written as $\sum \limits_{i = 1}^{r} \vec{q}_i \vec{q}_{i} ^{ \mathrm{T}}$, where $\vec{q}_1,\vec{q}_2,\dots,\vec{q}_r$ are set of orthonormal basis vectors of $\range(P)$. And on the other hand, if $Q_r = \left[ \begin{array}{cccc}
\vec{q}_1 & \vec{q}_2 & \cdots & \vec{q}_r
\end{array}\right]$ contains a set of orthonormal vectors in $\mathbb{R}^{m}$, then the orthogonal projector from $\mathbb{R}^{m}$ to the range of $Q_r$ is $P = Q_rQ_r^{\mathrm{T}} = \sum \limits_{i = 1}^{r} \vec{q}_i \vec{q}_{i} ^{ \mathrm{T}}$.

In general for any given vector $\vec{v} \in \mathbb{R}^{m}$, not necessarily a normalized vector, the orthogonal projector to the direction $\vec{v}$ is

$$P_{\vec{v}} = \bigg(\frac{\vec{v}} {\|\vec{v}\|}\bigg)\bigg( \frac{\vec{v}} {\|\vec{v}\|}\bigg)^{\mathrm{T}} = \boxed{ \frac{\vec{v}\vec{v}^{\mathrm{T}}} {\|\vec{v}\|^2}}$$

After that, given any vector $\vec{x} \in \mathbb{R}^{m}$, we have $P_{\vec{v}}\vec{x}$ is the orthogonal projection of $\vec{x}$ onto the direction $\vec{v}$ and the difference $\vec{x} − P_{\vec{v}}\vec{x}$ is perpendicular to $\vec{v}$.

And more generally, given a matrix $W \in \mathbb{R}^{m \times n}$, assuming that $m \geq n$ and the columns of $W$ are linearly independent. The orthogonal projector from $\mathbb{R}^{m}$ to the column space (the range) of $W$ can be determined as follows, denoted as $P$.
1. $\vec{v}$ be any vector in $\mathbb{R}^{m}$ and $\vec{y} = P\vec{v} \in \range(W)$ is the image of $\vec{v}$ under this orthogonal projector.
2. Since the projector is orthogonal, the difference $\vec{v} − \vec{y}$ is orthogonal to $\range(W)$, $i.e.$, we have $W^{\mathrm{T}}(\vec{v} − \vec{y}) = \vec{0}$.
3. We also have $\vec{y} = W\vec{x}$, for some $\vec{x} \in \mathbb{R}^{n}$, so that $W^{\mathrm{T}}(\vec{v} − W\vec{x}) = \vec{0}$, $i.e.$, $\vec{x} = \big( W^{\mathrm{T}}W \big)^{-1}W^{\mathrm{T}}\vec{v}$. Since $W$ is of full column rank as assumed ahead, $\big( W^{\mathrm{T}}W \big)^{-1}$ exists.
4. $\vec{y} = P\vec{v} = W\vec{x} = W\big( W^{\mathrm{T}}W \big)^{-1}W^{\mathrm{T}}\vec{v} \Rightarrow \boxed{ P = W\big( W^{\mathrm{T}}W \big)^{-1}W^{\mathrm{T}}}$.

$\dagger$ ***NO*** orthogonal matrix, except the identity matrix, is an orthogonal projector.$\ddagger$

# Householder reflector
To do Gaussian elimination. For any given vector $\vec{x}$, we want to find an operator (matrix) $F$ such that,

$$\vec{x} = \left[ \begin{array}{c}
x_1 \\
x_2 \\
\vdots \\
x_n
\end{array}\right] \overset{F}{\longrightarrow } F\vec{x} = \left[ \begin{array}{c}
\pm \left\| \vec{x}\right\|_2 \\
0 \\
\vdots \\
0
\end{array}\right] = \pm \left\| \vec{x}\right\|_2 \vec{e}_1$$

All are zeros below the first entry. Then how to determine matrix $F$? Assume that $\vec{x}$ is reflected to $F\vec{x} = \|\vec{x}\|_2 \vec{e}_1$, the positive direction of $x\text{-axis}$.
1. Define $\vec{v} = \vec{x} - F\vec{x} = \vec{x} - \|\vec{x}\|_2 \vec{e}_1$
2. Following above, the orthogonal projector onto the direction of $\vec{v}$ is $P_{\vec{v}} = \newcommand{\ffrac}{\displaystyle \frac} \ffrac{\vec{v}\vec{v}^{\mathrm{T}}} {\|\vec{v}\|_2^2}$, so that we can see that $\vec{v} = 2P_{\vec{v}}\vec{x}$
3. $F\vec{x} = \vec{x} - \vec{v} = (I - 2P_{\vec{v}})\vec{x}$, $i.e.$, $\boxed{ F = I - 2P_{\vec{v}} = I - 2 \displaystyle \frac{\vec{v}\vec{v}^{\mathrm{T}}} {\|\vec{v}\|_2^2}}$

$Proof$

Brief proof of the second point.

To prove $2P_{\vec{v}}\vec{x} = 2\cdot\ffrac{\vec{v}\vec{v}^{\mathrm{T}}\vec{x}} {\|\vec{v}\|_2^2}= \vec{v}$, it equals to prove that $2\vec{v}^{\mathrm{T}}\vec{x} = \|\vec{v}\|_2^2$.

$$2\vec{v}^{\mathrm{T}}\vec{x} = 2(\vec{x}^{\mathrm{T}} - \|\vec{x}\|_2 \vec{e}_1^{\mathrm{T}})\vec{x} = 2\|\vec{x}\|_2^2 - 2\|\vec{x}\|_2 \cdot x_1 \\
\|\vec{v}\|_2^2 = x_1^2 -2\|\vec{x}\|_{2}x_{1} + \|\vec{x}\|_2^2 + x_2^2 + \cdots + x_n^2 = 2\|\vec{x}\|_2^2 - 2\|\vec{x}\|_2 \cdot x_1$$

ALL DONE! $\square$

***
Now we have found what we want, which reflects the vector $\vec{x}$ to the direction of $\vec{e}_1$ by multiplying matrix (operator) $F$. Besides we have

$$F^{\mathrm{T}}F = (I - 2P_{\vec{v}})^{\mathrm{T}}(I - 2P_{\vec{v}}) = I^2 - 4P_{\vec{v}} + 4P_{\vec{v}}^2 = I - 4P_{\vec{v}} + 4P_{\vec{v}} = I$$

So actually this *Householder reflector* is an *orthogonal matrix*.

## Which reflector
For a given $\vec{x}$ we have

| $$F\vec{x} = +\sideset{}{^{2}}{\|\vec{x}\|} \sideset{}{_{1}}{\vec{e}}$$ | $$F\vec{x} = -\sideset{}{^{2}}{\|\vec{x}\|} \sideset{}{_{1}}{\vec{e}}$$ |
|:---------------------------------------------------------------------:|:---------------------------------------------------------------------:|
|                                         |                                         |

As mentioned before, subtracting two numbers which are *close* is an **ill-conditioned** problem. So that we prefer that $\vec{x}$ and $F\vec{x}$ are in opposite direction, $i.e.$

$$\DeclareMathOperator*{\sign}{sign}
F\vec{x} = \left[\begin{array}{c}
-\sign(x_1)\|\vec{x}\|_2 \\
0 \\
0 \\
\vdots \\
0
\end{array} \right]
= -\sign(x_1) \|\vec{x}\|_2\vec{e}_1 $$

And then $\vec{v} = \vec{x} - F\vec{x} = \vec{x} + \sign(x_1)\|\vec{x}\|_2\vec{e}_1$, while $F$ keeps the same expression.

# QR factorization by Householder reflectors
Now we can use the Householder reflectors to reduce a matrix $A$ to its upper trigangular form. Let $A$ be an $m\times n$ size matrix, assuming that $m \geq n$ and $\DeclareMathOperator*{\rank}{rank} \rank(A) = n$, a column full rank matrix.

We got our first operator $F_1$ that can reflect the first column of $A$ to its $\vec{e}_1$ direction.
1. Take $\vec{x} =
\left[\begin{array}{c}
a_{11} \\
a_{21} \\
\vdots \\
a_{m1}
\end{array} \right]
$, and the projection $F_1\vec{x} = -\sign(x_1)\|\vec{x}\|_2 \vec{e}_1$,then $\vec{v}_1 = \vec{x} - F_1 \vec{x} = \vec{x} + \sign(x_1) \|\vec{x}\|_2 \vec{e}_1$.
2. We can get the Householder reflector:
$$Q_1 = F_1 = I - 2 \frac{\vec{v}_1\vec{v}_1^{\mathrm{T}}} {\|\vec{v}_1\|_2^2}$$
3. Now we have
$$Q_1A = \left[\begin{array}{cccc}
a_{11}^{(1)} & a_{12} ^{(1)} & \cdots & a_{1n} ^{(1)} \\
0 & a_{22} ^{(1)} & \cdots & a_{2n} ^{(1)} \\
0 & a_{32} ^{(1)} & \cdots & a_{3n} ^{(1)} \\
\vdots & \vdots & \vdots & \vdots \\
0 & a_{m2} ^{(1)} & \cdots & a_{mn} ^{(1)} \\
\end{array} \right]$$
So now we take $\vec{x} =
\left[\begin{array}{c}
a_{22} ^{(1)} \\
a_{32} ^{(1)} \\
\vdots \\
a_{m2} ^{(1)}
\end{array} \right]
$, and the projection $F_2\vec{x} = -\sign(x_1)\|\vec{x}\|_2 \vec{e}_1$,then $\vec{v}_2 = \vec{x} - F_2 \vec{x} = \vec{x} + \sign(x_1) \|\vec{x}\|_2 \vec{e}_1$.
4. We can get the second Householder reflector:
$$F_2 = I - 2 \frac{\vec{v}_2\vec{v}_2^{\mathrm{T}}} {\|\vec{v}_2\|_2^2}$$
Notice that at this time $F_2$ is an $(m-1)\times (m-1)$ orthogonal matrix, so we define
$$Q_2 = \left[\begin{array}{cc}
1 & \vec{0}^{\mathrm{T}} \\
\vec{0} & F_2
\end{array} \right]_{m \times m}$$

So on and so forth, after $n$ times loop, we get the QR factorization as following:

$$Q_{n}Q_{n-1}\cdots Q_{2}Q_{1}A = \left[\begin{array}{cccc}
a_{11}^{(1)} & a_{12} ^{(1)} & \cdots & a_{1n} ^{(1)} \\
0 & a_{22} ^{(2)} & \cdots & a_{2n} ^{(2)} \\
0 & 0 & \ddots & \vdots \\
\vdots & \vdots & \ddots & a_{nn} ^{(n)} \\
0 & 0 & \cdots & 0 \\
\vdots & \vdots & & \vdots \\
0 & 0 & \cdots & 0
\end{array} \right]_{m \times n} = R$$

$R$ is upper triangular, and each $Q_i$ is an orthogonal matrix. So $Q_i^{\mathrm{T}} = Q_i^{-1}$. And seeing from how we create $Q_i$, we also have $Q_i = Q_i^{\mathrm{T}}$

$$A = Q_1Q_2Q_3\cdots Q_nR := Q_{m \times m}R_{m \times n}$$

$Algorithm$ **Householder Triangularization**

```
for k = 1 to n
    x = A_{k:m,k}
    v_k = x + sign(x_1) \|x\|_2 e_1
    v_k = v_k / \|v_k\|_2
    A_{k:m,k:n} = A_{k:m,k:n} − 2v_k ( v_k^T A_{k:m,k:n} )
end
```

The amount on floating point operations of the Householder Triangularization is approximately ():

$$\begin{align}
\sum_{k=1}^n \sum_{j=k}^{n} 4(m-k+1) =& \sum_{k=1}^n 4(m-k+1)(n-k+1) \\
=& \sum_{k=1}^n 4\big(mn+m+n+1+k^2+2k(m+1)(n+1)\big) \\
\approx & 2mn^2 - \frac{2} {3} n^3
\end{align}$$
***
After the above algorithm, the result $R$ is stored in the upper triangular part of $A$. As for $Q$, it's gone. But we can rebuild it using the saved vector $\vec{v}_k$.

Why $Q$ is not needed? Because we use QR factorization to solve the equation $A\vec{x} = \vec{b}$. Then we will have $Q^{\mathrm{T}}QR\vec{x} = Q^{\mathrm{T}}\vec{b} = R\vec{x}$. So actually what we need is the product of matrix $Q$ (or its transpose) and a given vector $\vec{b}$, which can be achieved with the stored $\vec{v}_k$.

$$\begin{align}
Q =& Q_1Q_2Q_3\cdots Q_n \\
=& \big( I_{m} - 2\vec{v}_1 \vec{v}_1^{\mathrm{T}} \big) \left( \begin{array}{cc}
1 & \vec{0}^{\mathrm{T}} \\
\vec{0} & I_{m-1} - 2\vec{v}_2 \vec{v}_2^{\mathrm{T}}
\end{array} \right) \cdots \left( \begin{array}{cc}
I_{n-1} & \mathbf{0} \\
\mathbf{0} & I_{m-n+1} - 2\vec{v}_n \vec{v}_n^{\mathrm{T}}
\end{array} \right) \\[1em]
Q^{\mathrm{T}} =& Q_n^{\mathrm{T}}Q_{n-1}^{\mathrm{T}}Q_{n-2}^{\mathrm{T}}\cdots Q_1^{\mathrm{T}} = Q_nQ_{n-1}Q_{n-2}\cdots Q_1 \\
=& \left( \begin{array}{cc}
I_{n-1} & \mathbf{0} \\
\mathbf{0} & I_{m-n+1} - 2\vec{v}_n \vec{v}_n^{\mathrm{T}}
\end{array} \right) \left( \begin{array}{cc}
I_{n-2} & \mathbf{0} \\
\mathbf{0} & I_{m-n+2} - 2\vec{v}_{n-1} \vec{v}_{n-1}^{\mathrm{T}}
\end{array} \right) \cdots \left( \begin{array}{cc}
1 & \vec{0}^{\mathrm{T}} \\
\vec{0} & I_{m-1} - 2\vec{v}_2 \vec{v}_2^{\mathrm{T}}
\end{array} \right)
\big( I_m - 2\vec{v}_1 \vec{v}_1^{\mathrm{T}} \big)
\end{align}$$

$Algorithm$ **Given $\vec{b}$, find $Q\vec{b}$**

```
x = b
for k = n : −1 : 1
    x_{k:m} = x_{k:m} − 2 v_k ( v_k^T x_{k:m} )
end
```

$Algorithm$ **Given $\vec{b}$, find $Q^{\mathrm{T}}\vec{b}$**

```
x = b
for k = 1 : n
    x_{k:m} = x_{k:m} − 2 v_k ( v_k^T x_{k:m} )
end
```

And if you still want the explicit $Q$, just using $Q\vec{e}_1, Q\vec{e}_2, \dots , Q\vec{e}_n$ to get that.

# QR factorization by Gram-Schmidt orthogonalization
Still we assume that $A$ be an $m\times n$ size matrix, and $m \geq n$ with $\rank(A) = n$, a column full rank matrix. Let $Q \in \mathbb{R}^{m \times m}$ and $R \in \mathbb{R}^{m \times n}$ be the result of QR factorization.

$$A = \left[\begin{array}{cccc}
\vec{a}_1^c & \vec{a}_2^c & \cdots \vec{a}_n^c
\end{array} \right] = \left[\begin{array}{cccc}
\vec{q}_1^c & \vec{q}_2^c & \cdots \vec{q}_m^c
\end{array} \right]\left[\begin{array}{c}
\begin{array}{cccc}
r_{11} & r_{12} & \cdots & r_{1n} \\
0 & r_{22} & \cdots & r_{2n} \\
\vdots & 0 & \ddots & \vdots \\
0 & 0 & \cdots & r_{nn}
\end{array} \\
\mathbf{0}
\end{array} \right]=QR$$

But actually we don't need a **full QR factorization**, here's the **reduced** one.

$$A = \left[\begin{array}{cccc}
\vec{a}_1^c & \vec{a}_2^c & \cdots \vec{a}_n^c
\end{array} \right] = \left[\begin{array}{cccc}
\vec{q}_1^c & \vec{q}_2^c & \cdots \vec{q}_n^c
\end{array} \right]\left[\begin{array}{cccc}
r_{11} & r_{12} & \cdots & r_{1n} \\
0 & r_{22} & \cdots & r_{2n} \\
\vdots & \ddots & \ddots & \vdots \\
0 & \cdots & 0 & r_{nn}
\end{array}\right]=\hat{Q}\hat{R}$$

And more easily we can find that

$\begin{align}
\vec{a}_1^c =& r_{11} \vec{q}_1^c \\
\vec{a}_2^c =& r_{12} \vec{q}_1^c  + r_{22} \vec{q}_2^c \\
\vdots & \\
\vec{a}_j^c =& r_{1j} \vec{q}_1^c  + r_{2j} \vec{q}_2^c + \cdots + r_{jj} \vec{q}_j^c\\
\vdots & \\
\vec{a}_n^c =& r_{1n} \vec{q}_1^c  + r_{2n} \vec{q}_2^c + \cdots + r_{nn} \vec{q}_j^c\\
\end{align}$

$Conclusion$

1. $\vec{a}_j^c \in \big< \vec{q}_1^c, \vec{q}_2^c, \dots, \vec{q}_j^c \big>$, $j = 1,2,\dots, n$
2. $r_{jj}\neq 0$. Firstly that $r_{11}$ can't, otherwise $\vec{a}_1^c = \vec{0}$, however $A$ is a column full rank matrix. And for $j = 2,3, \dots, n$, still $r_{jj}\neq 0$, otherwise $\vec{a}_1^c, \vec{a}_2^c, \dots, \vec{a}_j^c$ can be expressed by $\big< \vec{q}_1^c, \vec{q}_2^c, \dots, \vec{q}_{j-1}^c \big>$, which implies that these $j$ columns are linear dependent, can't be! (And we can get to this by seeing how we get the $R$.)
3. $\left\{\begin{align}
\vec{q}_1^c =& \ffrac{\vec{a}_1^c} {r_{11}} \\
\vec{q}_2^c =& \ffrac{\vec{a}_2^c - r_{12}\vec{q}_1^{c}} {r_{22}} \\
\vdots & \\
\vec{q}_j^c =& \ffrac{\vec{a}_j^c - r_{1j}\vec{q}_1^{c} - r_{2j}\vec{q}_2^{c} -\cdots -r_{j-1,j}\vec{q}_{j-1}^{c}} {r_{jj}}\\
\vdots & \\
\vec{q}_n^c =& \ffrac{\vec{a}_n^c - r_{1n}\vec{q}_1^{c} - r_{2n}\vec{q}_2^{c} -\cdots -r_{n-1,n}\vec{q}_{n-1}^{c}} {r_{nn}}\\
\end{align}\right.$
4. $\vec{q}_j^c \in \big< \vec{a}_1^c, \vec{a}_2^c, \dots, \vec{a}_j^c \big>$, $j = 1,2,\dots, n$
5. $\big< \vec{a}_1^c, \vec{a}_2^c, \dots, \vec{a}_j^c \big> = \big< \vec{q}_1^c, \vec{q}_2^c, \dots, \vec{q}_j^c \big>$
6. (From the process of getting $R$, we can see that each time $r_{jj} = \|\vec{x}\|$.) So that actually $\|\vec{q}_j^c\| = 1$, and $\vec{q}_1^c, \vec{q}_2^c, \dots, \vec{q}_j^c$ is actually set of orthonormal vectors.
7. So $r_{jj} = \|\vec{a}_j - r_{1j}\vec{q}_1 - r_{2j}\vec{q}_2 - \cdots - r_{j-1,j}\vec{q}_{j-1} \|_2$, and $\vec{q}_j^c = \ffrac{\vec{a}_j - r_{1j}\vec{q}_1 - r_{2j}\vec{q}_2 - \cdots - r_{j-1,j}\vec{q}_{j-1}} {r_{jj}}$

For the algorithms below, calculate $r_{1,1}$ and $\vec{q}_1^{c}$ first.

$Algorithm$ **QR factorization by classical Gram-Schmidt**

```
for j = 1 to n
    \vec{v} = \vec{a}_j
    for i = 1 to j−1
        r_{ij} = \vec{q}_i^T \vec{a}_j
        \vec{v} = \vec{v} − r_{ij} \vec{q}_i
    end
    r_{jj} = \| \vec{v} \|_2
    \vec{q}_j = \vec{v}/r_{jj}
end
```

With flops $\sum\limits_{j=1}^{n}\sum\limits_{i=1}^{j-1} 4m \approx 2mn^2$

$Algorithm$ **QR factorization by modified Gram-Schmidt**

```
for j = 1 to n
    \vec{v} = \vec{a}_j
    for i = 1 to j−1
        r_{ij} = \vec{q}_i^T \vec{v}
        \vec{v} = \vec{v} − r_{ij} \vec{q}_i
    end
    r_{jj} = \| \vec{v} \|_2
    \vec{q}_j = \vec{v}/r_{jj}
end
```

# Backward stability of QR factorization
$Theorem$

Let $A = QR$ by Householder triangularization and for the computed factors, we have $\tilde{Q}\tilde{R} = A + \delta A$, with $\ffrac{\|\delta A\|} {\| A \|} = O(\epsilon_{machine})$. But it is not backward stable if using the classical or modified Gram-Schmidt algorithm.

Take an example.

$$A = \left[\begin{array}{cc}
0.7 & 0.7 + 10^{-15} \\
0.7 + 10^{-15} & 0.7 + 10^{-15}
\end{array} \right]$$

Condition number of $A$ is about $10^{15}$. Denote the result from Gram-Schmidt algorithm in MATLAB as $Q_G, R_G$, respectively; and similar for $Q_H, R_H$ using Householder triangularization. The result are

$$Q_G Q_G^{\mathrm{T}} = \left[\begin{array}{cc}
0.890 & 0.012 \\
0.012 & 1.109
\end{array} \right],Q_H Q_H^{\mathrm{T}} = \left[\begin{array}{cc}
1 & 0 \\
0 & 1
\end{array} \right]$$

# Solution of system of linear equations through QR factorization
Given $A \in \mathbb{R}^{n \times n}$ and $\vec{b} \in \mathbb{R}^{n}$. As mentioned above, we have $R\vec{x} = Q^{\mathrm{T}}\vec{b}$ after the QR factorization, which can be solved by backward substitution or backslash function.

Comparing using the LU factorization, it's more stable but more expensive.

# Legendre polynomials 勒让德多项式
Consider the space of polynomials of degree less or equal than $n − 1$ on the interval $x \in [ − 1, 1]$. The monomials $1, x, x^2 , ... , x^{n − 1}$ form a basis of this space; $p(x) = a_1 + a_2 x + \cdots + a_n x^{n − 1}$.

$Def$ **inner product**

$$\big(p(x),q(x)\big) = \int _{-1} ^{1} p(x)q(x) \,\mathrm{d}x$$

Two polynomials are orthogonal if $\big(p(x),q(x)\big)=0$.

Then after that the fact is $1, x, x^2 , ... , x^{n − 1}$ is not an orthogonal basis. So using QR factorization to find it.

$$A = \left[\begin{array}{cccc}
1 & x & \cdots & x^{n-1}
\end{array} \right] = \left[\begin{array}{cccc}
P_0(x) & P_1(x) & \cdots & P_{n-1}(x)
\end{array} \right] \left[\begin{array}{cccc}
r_{11} & r_{12}&  \cdots & r_{1n} \\
0 & r_{22} & \cdots & r_{2n} \\
\vdots & \ddots & \ddots& \vdots \\
0 & \cdots & 0 & r_{nn}
\end{array} \right]$$

And the polynomials in $Q$ are the **Legendre polynomials**.

$$\displaystyle P_{n}(x)={1 \over 2^{n}n!}{\mathrm{d}^{n} \over \mathrm{d}x^{n}}\left[(x^{2}-1)^{n}\right]$$

Here each polynomial $P_i(x)$ is of degree $i$, and since $P_i(1) = 1$, so they form an orthogonal basis

Computational/Intro to Numerical Computing/Note_Chap06_Householder Triangularization, Gram-Schmidt Orthogonalization, and QR Factorization.ipynb


## Constraint Handling

### Inequality Constraints

**If somebody is interested in implementing or willing to make contribution(s) regarding constraint handling of inequality constraints please let us know.
The G problem is suite is already available to experiment with different algorithms. So far, mostly parameter-less constraint handling is used for our algorithms.**

### Equality Constraints

We got a couple of questions of how equality constraints should be handled in a genetic algorithm. In general, functions without any smoothness are challenging to handle for genetic algorithms. An equality constraint is basically an extreme case, where the constraint violation is 0 at exactly one point and otherwise 1.

Let us consider the following constraint $g(x)$ where $x$ represents a variable:

$g(x): x = 5$

An equality constraint can be expressed by an inequality constraint:

$g(x): |x - 5| \leq 0$

or

$g(x): (x-5)^2   \leq 0$

However, all of constraints above are very strict and make **most of the search space infeasible**. Without providing more information to the algorithm those constraint are very difficult to satisfy.

For this reason, the constraint can be smoothed by adding an epsilon to it and, therefore, having two inequality constraints:

$g'(x): 5 - \epsilon \leq x \leq 5 + \epsilon$

Also, it can be simply expressed in one inequality constraint by:

$g'(x): (x-5)^2  - \hat{\epsilon} \leq 0$

Depending on the $\epsilon$ the solutions will be more or less close to the desired value. However, the genetic algorithm does not know anything about the problem itself which makes it difficult to handle and focus the search in the infeasible space.

**Constraint Handling Through Repair**

A simple approach is to handle constraints through a repair function. This is only possible if the equation of the constraint is known. The repair makes sure every solution that is evaluated is in fact feasible. Let us consider the following example where
the equality constraints need to consider more than one variable:

\begin{align}
\begin{split}
\min \;\; & f_1(x) = (x_1^2 + x_2^2) \\
\max \;\; & f_2(x) = -(x_1-1)^2 - x_2^2 \\[1mm]
\text{s.t.} \;\; & g_1(x_1, x_3) : x_1 + x_3 = 2\\[2mm]
& -2 \leq x_1 \leq 2 \\
& -2 \leq x_2 \leq 2 \\
& -2 \leq x_3 \leq 2
\end{split}
\end{align}

We implement the problem using by squaring the term and using an $\epsilon$ as we have explained above. The source code for the problem looks as follows:

```python
import numpy as np

from pymoo.model.problem import Problem

class MyProblem(Problem):

    def __init__(self):
        super().__init__(n_var=3,
                         n_obj=2,
                         n_constr=1,
                         xl=np.array([-2, -2, -2]),
                         xu=np.array([2, 2, 2]))

    def _evaluate(self, x, out, *args, **kwargs):
        f1 = x[:, 0] ** 2 + x[:, 1] ** 2
        f2 = (x[:, 0] - 1) ** 2 + x[:, 1] ** 2
        g1 = (x[:, 0] + x[:, 2] - 2) ** 2 - 1e-5

        out["F"] = np.column_stack([f1, f2])
        out["G"] = g1

```

As you might have noticed the problem has similar characteristics to problem in our getting started.
Before a solution is evaluated a repair function is called. To make sure a solution is feasible, an approach would be to either set $x_3 = 2 - x_1$ or $x_1 = 2 - x_3$. Additionally, we need to consider that this repair might produce a variable to be out of bounds.

```python
from pymoo.model.repair import Repair

class MyRepair(Repair):

    def _do(self, problem, pop, **kwargs):
        for k in range(len(pop)):
            x = pop[k].X
            if np.random.random() < 0.5:

                x[2] = 2 - x[0]
                if x[2] > 2:
                    val = x[2] - 2
                    x[0] += val
                    x[2] -= val

            else:
                x[0] = 2 - x[2]
                if x[0] > 2:
                    val = x[0] - 2
                    x[2] += val
                    x[0] -= val

        return pop
```

Now the algorithm object needs to be initialized with the repair operator and then can be run to solve the problem:

```python
from pymoo.algorithms.nsga2 import NSGA2

algorithm = NSGA2(pop_size=100, repair=MyRepair(), eliminate_duplicates=True)

from pymoo.optimize import minimize
from pymoo.visualization.scatter import Scatter

res = minimize(MyProblem(),
               algorithm,
               ('n_gen', 20),
               seed=1,
               verbose=True)

plot = Scatter()
plot.add(res.F, color="red")
plot.show()
```

In our case it is easy to verify if the constraint is violated or not:

```python
print(res.X[:, 0] + res.X[:, 2])
```

    [2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2.
     2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2.
     2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2.
     2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2.
     2. 2. 2. 2.]

If you would like to compare the solution without a repair you will see how searching only in the feasible space helps:

```python
algorithm = NSGA2(pop_size=100, eliminate_duplicates=True)

res = minimize(MyProblem(),
               algorithm,
               ('n_gen', 20),
               seed=1,
               verbose=True)

plot = Scatter()
plot.add(res.F, color="red")
plot.show()

```

```python
print(res.X[:, 0] + res.X[:, 2])
```

    [1.99992699 1.99992699 1.9987293  1.99992699 2.00089231]

Here in fact the $\epsilon$ term is necessary to find any feasible solution at all.


# The Efficient Frontier of Optimal Portfolio Transactions

### Introduction

[Almgren and Chriss](https://cims.nyu.edu/~almgren/papers/optliq.pdf) showed that for each value of risk aversion there is a unique optimal execution strategy. The optimal strategy is obtained by minimizing the **Utility Function** $U(x)$:

\begin{equation}
U(x) = E(x) + \lambda V(x)
\end{equation}

where $E(x)$ is the **Expected Shortfall**, $V(x)$ is the **Variance of the Shortfall**, and $\lambda$ corresponds to the trader’s risk aversion. The expected shortfall and variance of the optimal trading strategy are given by:

In this notebook, we will learn how to visualize and interpret these equations.

# The Expected Shortfall

As we saw in the previous notebook, even if we use the same trading list, we are not guaranteed to always get the same implementation shortfall due to the random fluctuations in the stock price. This is why we had to reframe the problem of finding the optimal strategy in terms of the average implementation shortfall and the variance of the implementation shortfall. We call the average implementation shortfall, the expected shortfall $E(x)$, and the variance of the implementation shortfall $V(x)$. So, whenever we talk about the expected shortfall we are really talking about the average implementation shortfall. Therefore, we can think of the expected shortfall as follows. Given a single trading list, the expected shortfall will be the value of the average implementation shortfall if we were to implement this trade list in the stock market many times.

To see this, in the code below we implement the same trade list on 50,000 trading simulations. We call each trading simulation an episode. Each episode will consist of different random fluctuations in stock price. For each episode we will compute the corresponding implemented shortfall. After all the 50,000 trading simulations have been carried out we calculate the average implementation shortfall and the variance of the implemented shortfalls. We can then compare these values with the values given by the equations for $E(x)$ and $V(x)$ from the Almgren and Chriss model.

```python
%matplotlib inline

import matplotlib.pyplot as plt

import utils

# Set the default figure size
plt.rcParams['figure.figsize'] = [17.0, 7.0]

# Set the liquidation time
l_time = 60

# Set the number of trades
n_trades = 60

# Set trader's risk aversion
t_risk = 1e-6

# Set the number of episodes to run the simulation
episodes = 10

utils.get_av_std(lq_time = l_time, nm_trades = n_trades, tr_risk = t_risk, trs = episodes)

# Get the AC Optimal strategy for the given parameters
ac_strategy = utils.get_optimal_vals(lq_time = l_time, nm_trades = n_trades, tr_risk = t_risk)
ac_strategy
```

# Extreme Trading Strategies

Because some investors may be willing to take more risk than others, when looking for the optimal strategy we have to consider a wide range of risk values, ranging from those traders that want to take zero risk to those who want to take as much risk as possible. Let's take a look at these two extreme cases. We will define the **Minimum Variance** strategy as that one followed by a trader that wants to take zero risk and the **Minimum Impact** strategy at that one followed by a trader that wants to take as much risk as possible. Let's take a look at the values of $E(x)$ and $V(x)$ for these extreme trading strategies. The `utils.get_min_param()` uses the above equations for $E(x)$ and $V(x)$, along with the parameters from the trading environment to calculate the expected shortfall and standard deviation (the square root of the variance) for these strategies. We'll start by looking at the Minimum Impact strategy.

```python
import utils

# Get the minimum impact and minimum variance strategies
minimum_impact, minimum_variance = utils.get_min_param()
```

### Minimum Impact Strategy

This trading strategy will be taken by trader that has no regard for risk. In the Almgren and Chriss model this will correspond to having the trader's risk aversion set to $\lambda = 0$. In this case the trader will sell the shares at a constant rate over a long period of time. By doing so, he will minimize market impact, but will be at risk of losing a lot of money due to the large variance. Hence, this strategy will yield the lowest possible expected shortfall and the highest possible variance, for a given set of parameters. We can see that for the given parameters, this strategy yields an expected shortfall of \$197,000 dollars but has a very big standard deviation of over 3 million dollars.

```python
minimum_impact
```

AC Optimal Strategy for Minimum Impact

  Number of Days to Sell All the Shares:	     250	       Initial Portfolio Value:	         $50,000,000.00

  Half-Life of The Trade:	                1,284,394.9	   Expected Shortfall:	                $197,000.00

  Utility:	                               $197,000.00	   Standard Deviation of Shortfall:	  $3,453,707.55

### Minimum Variance Strategy

This trading strategy will be taken by trader that wants to take zero risk, regardless of transaction costs. In the Almgren and Chriss model this will correspond to having a variance of $V(x) = 0$. In this case, the trader would prefer to sell the all his shares immediately, causing a known price impact, rather than risk trading in small increments at successively adverse prices. This strategy will yield the smallest possible variance, $V(x) = 0$, and the highest possible expected shortfall, for a given set of parameters. We can see that for the given parameters, this strategy yields an expected shortfall of over 2.5 million dollars but has a standard deviation equal of zero.

```python
minimum_variance
```

AC Optimal Strategy for Minimum Variance

  Number of Days to Sell All the Shares:	       1	         Initial Portfolio Value:	         $50,000,000.00

  Half-Life of The Trade:	                     0.2	        Expected Shortfall:	               $2,562,500.00

  Utility:	                               $2,562,500.00	   Standard Deviation of Shortfall:	      $0.00

# The Efficient Frontier

The goal of Almgren and Chriss was to find the optimal strategies that lie between these two extremes. In their paper, they showed how to compute the trade list that minimizes the expected shortfall for a wide range of risk values. In their model, Almgren and Chriss used the parameter $\lambda$ to measure a trader's risk-aversion. The value of $\lambda$ tells us how much a trader is willing to penalize the variance of the shortfall, $V(X)$, relative to expected shortfall, $E(X)$. They showed that for each value of $\lambda$ there is a uniquely determined optimal execution strategy. We define the **Efficient Frontier** to be the set of all these optimal trading strategies. That is, the efficient frontier is the set that contains the optimal trading strategy for each value of $\lambda$.

The efficient frontier is often visualized by plotting $(x,y)$ pairs for a wide range of $\lambda$ values, where the $x$-coordinate is given by the equation of the expected shortfall, $E(X)$, and the $y$-coordinate is given by the equation of the variance of the shortfall, $V(X)$. Therefore, for a given a set of parameters, the curve defined by the efficient frontier represents the set of optimal trading strategies that give the lowest expected shortfall for a defined level of risk.

In the code below, we plot the efficient frontier for $\lambda$ values in the range $(10^{-7}, 10^{-4})$, using the default parameters in our trading environment. Each point of the frontier represents a distinct strategy for optimally liquidating the same number of stocks. A risk-averse trader, who wishes to sell quickly to reduce exposure to stock price volatility, despite the trading costs incurred in doing so, will likely choose a value of $\lambda = 10^{-4}$. On the other hand, a trader
who likes risk, who wishes to postpones selling, will likely choose a value of $\lambda = 10^{-7}$. In the code, you can choose a particular value of $\lambda$ to see the expected shortfall and level of variance corresponding to that particular value of trader's risk aversion.

```python
%matplotlib inline

import matplotlib.pyplot as plt

import utils

# Set the default figure size
plt.rcParams['figure.figsize'] = [17.0, 7.0]

# Plot the efficient frontier for the default values. The plot points out the expected shortfall and variance of the
# optimal strategy for the given the trader's risk aversion. Valid range for the trader's risk aversion (1e-7, 1e-4).
utils.plot_efficient_frontier(tr_risk = 1e-6)
```

```python

```


### Método Newton - Levenberg Marquard

Insalar las librerias para realizar calculos simbólicos.

```python
!pip install sympy
```

    Requirement already satisfied: sympy in c:\users\juanlombana\anaconda3\lib\site-packages (1.6.2)
    Requirement already satisfied: mpmath>=0.19 in c:\users\juanlombana\anaconda3\lib\site-packages (from sympy) (1.1.0)

```python
import numpy as np
import numpy.linalg as npl
from sympy import symbols, diff, pi, cos, exp, sqrt, Abs
import matplotlib.pyplot as plt
```

## Función de Ackley

### 2 dimensiones

```python
x1, x2= symbols('x1 x2') #variables simbolicas
x = np.array([[-0.5,0.5]])#Vector de pasos con el punto inicial
lamb = 100 #Valor lambda para el meta-hessiano
z = np.array([]) #Vector de resultados
error = 10**-60 #Error minimo, criterio de parada

a = 20
b = 0.2
c = 2*pi
d=2

f=-a*exp(-b*sqrt(1/d*(x1**2+x2**2)))-exp(1/d*(cos(c*x1)+cos(c*x2)))+a+exp(1)#función objetivo
#Derivadas
df_dx1=diff(f,x1)
df_dx2=diff(f,x2)
d2f_dx12=diff(df_dx1,x1)
d2f_dx22=diff(df_dx2,x2)
d2f_dx1dx2=diff(df_dx1,x2)
x_temp = [(x1,x[0,0]),(x2,x[0,1])]
grad = np.array([df_dx1.subs(x_temp).evalf(),df_dx2.subs(x_temp).evalf()])#Vector gradiente
H = np.array([[d2f_dx12.subs(x_temp).evalf(),d2f_dx1dx2.subs(x_temp).evalf()],[d2f_dx1dx2.subs(x_temp).evalf(),d2f_dx22.subs(x_temp).evalf()]])#Hessiano
last_z = f.subs(x_temp)#evaluar Z con el punto inicial
z = np.append(z,[last_z.evalf()]) # Evaluación del punto inicial
ide = np.identity(d) #matriz de identidad
i = 0
while i <= 100:
    h_new = npl.inv(((lamb*ide)+H.astype(float))) #Inverso del Hessiano Aproximado.
    new_x = (x[i].T-np.matmul(h_new,grad.T)).T #Siguiente paso
    x = np.append(x,new_x).reshape((i+2),d) #Agregar el paso al vector de pasos
    i += 1 #Aumentar el contador de iteraciones
    x_temp = [(x1,x[i,0]),(x2,x[i,1])]
    grad = np.array([df_dx1.subs(x_temp).evalf(),df_dx2.subs(x_temp).evalf()])#Vector gradiente
    H = np.array([[d2f_dx12.subs(x_temp).evalf(),d2f_dx1dx2.subs(x_temp).evalf()],[d2f_dx1dx2.subs(x_temp).evalf(),d2f_dx22.subs(x_temp).evalf()]])#Hessiano
    actual_z = f.subs(x_temp)
    z = np.append(z,[actual_z.evalf()]) # Evaluación del punto actual
    if Abs(actual_z-last_z).evalf()<=error:
        break
    else:
        last_z = actual_z
print("last iterarion: " + str(i))
print("final Z value: " + str(actual_z.evalf()))
print("last x values: "+ str(new_x))
```

    last iterarion: 101
    final Z value: 0.0349736563155063
    last x values: [-0.00791087437099679 0.00791087437099679]

```python
vals = np.arange(-1,1,0.1)
z_c = np.ones(len(vals)**2).reshape(len(vals),len(vals))
for i in range(0,len(vals)):
    for j in range(0,len(vals)):
        x_test = [(x1,vals[i]),(x2,vals[j])]
        z_c[i,j] = float(f.subs(x_test).evalf())
plt.contourf(vals,vals,z_c)
plt.plot(x[:,0],x[:,1],c='red')
plt.colorbar();
```

### 5 Dimensiones

```python
x1, x2, x3, x4, x5= symbols('x1 x2 x3 x4 x5') #variables simbolicas
x = np.array([[-0.5,0.5,0.5,0.5,0.5]])#Vector de pasos con el punto inicial
lamb = 100 #Valor lambda para el meta-hessiano
z = np.array([]) #Vector de resultados
ide = np.identity(d) #matriz de identidad
error = 10**-60 #Error minimo, criterio de parada

a = 20
b = 0.2
c = 2*pi
d = 5

f=-a*exp(-b*sqrt(1/d*(x1**2+x2**2+x3**2+x4**3+x5**2)))-exp(1/d*(cos(c*x1)+cos(c*x2)+cos(c*x3)+cos(c*x4)+cos(c*x5)))+a+exp(1)#función objetivo
#Derivadas
df_dx1=diff(f,x1)
df_dx2=diff(f,x2)
df_dx3=diff(f,x3)
df_dx4=diff(f,x4)
df_dx5=diff(f,x5)

d2f_dx12=diff(df_dx1,x1)
d2f_dx22=diff(df_dx2,x2)
d2f_dx32=diff(df_dx3,x3)
d2f_dx42=diff(df_dx4,x4)
d2f_dx52=diff(df_dx5,x5)

d2f_dx1dx2=diff(df_dx1,x2)
d2f_dx1dx3=diff(df_dx1,x3)
d2f_dx1dx4=diff(df_dx1,x4)
d2f_dx1dx5=diff(df_dx1,x5)
d2f_dx2dx3=diff(df_dx2,x3)
d2f_dx2dx4=diff(df_dx2,x4)
d2f_dx2dx5=diff(df_dx2,x5)
d2f_dx3dx4=diff(df_dx3,x4)
d2f_dx3dx5=diff(df_dx3,x5)
d2f_dx4dx5=diff(df_dx4,x5)

x_temp = [(x1,x[0,0]),
          (x2,x[0,1]),
          (x3,x[0,2]),
          (x4,x[0,3]),
          (x5,x[0,4])]
grad = np.array([df_dx1.subs(x_temp).evalf(),
                 df_dx2.subs(x_temp).evalf(),
                 df_dx3.subs(x_temp).evalf(),
                 df_dx4.subs(x_temp).evalf(),
                 df_dx5.subs(x_temp).evalf()])#Vector gradiente
H = np.array([
    [d2f_dx12.subs(x_temp).evalf(),d2f_dx1dx2.subs(x_temp).evalf(),d2f_dx1dx3.subs(x_temp).evalf(),d2f_dx1dx4.subs(x_temp).evalf(),d2f_dx1dx5.subs(x_temp).evalf()],
    [d2f_dx1dx2.subs(x_temp).evalf(),d2f_dx22.subs(x_temp).evalf(),d2f_dx2dx3.subs(x_temp).evalf(),d2f_dx2dx4.subs(x_temp).evalf(),d2f_dx2dx5.subs(x_temp).evalf()],
    [d2f_dx1dx3.subs(x_temp).evalf(),d2f_dx2dx3.subs(x_temp).evalf(),d2f_dx32.subs(x_temp).evalf(),d2f_dx3dx4.subs(x_temp).evalf(),d2f_dx3dx5.subs(x_temp).evalf()],
    [d2f_dx1dx4.subs(x_temp).evalf(),d2f_dx2dx4.subs(x_temp).evalf(),d2f_dx3dx4.subs(x_temp).evalf(),d2f_dx42.subs(x_temp).evalf(),d2f_dx4dx5.subs(x_temp).evalf()],
    [d2f_dx1dx5.subs(x_temp).evalf(),d2f_dx2dx5.subs(x_temp).evalf(),d2f_dx3dx5.subs(x_temp).evalf(),d2f_dx4dx5.subs(x_temp).evalf(),d2f_dx52.subs(x_temp).evalf()]])#Hessiano
last_z = f.subs(x_temp)#evaluar Z con el punto inicial
z = np.append(z,[last_z.evalf()]) # Evaluación del punto inicial
i = 0
while i <= 100:
    h_new = npl.inv(((lamb*ide)+H.astype(float))) #Inverso del Hessiano Aproximado.
    new_x = (x[i].T-np.matmul(h_new,grad.T)).T #Siguiente paso
    x = np.append(x,new_x).reshape((i+2),d) #Agregar el paso al vector de pasos
    i += 1 #Aumentar el contador de iteraciones
    x_temp = [(x1,x[i,0]),
              (x2,x[i,1]),
              (x3,x[i,2]),
              (x4,x[i,3]),
              (x5,x[i,4])]
    grad = np.array([df_dx1.subs(x_temp).evalf(),
                 df_dx2.subs(x_temp).evalf(),
                 df_dx3.subs(x_temp).evalf(),
                 df_dx4.subs(x_temp).evalf(),
                 df_dx5.subs(x_temp).evalf()])#Vector gradiente
    H = np.array([
        [d2f_dx12.subs(x_temp).evalf(),d2f_dx1dx2.subs(x_temp).evalf(),d2f_dx1dx3.subs(x_temp).evalf(),d2f_dx1dx4.subs(x_temp).evalf(),d2f_dx1dx5.subs(x_temp).evalf()],
        [d2f_dx1dx2.subs(x_temp).evalf(),d2f_dx22.subs(x_temp).evalf(),d2f_dx2dx3.subs(x_temp).evalf(),d2f_dx2dx4.subs(x_temp).evalf(),d2f_dx2dx5.subs(x_temp).evalf()],
        [d2f_dx1dx3.subs(x_temp).evalf(),d2f_dx2dx3.subs(x_temp).evalf(),d2f_dx32.subs(x_temp).evalf(),d2f_dx3dx4.subs(x_temp).evalf(),d2f_dx3dx5.subs(x_temp).evalf()],
        [d2f_dx1dx4.subs(x_temp).evalf(),d2f_dx2dx4.subs(x_temp).evalf(),d2f_dx3dx4.subs(x_temp).evalf(),d2f_dx42.subs(x_temp).evalf(),d2f_dx4dx5.subs(x_temp).evalf()],
        [d2f_dx1dx5.subs(x_temp).evalf(),d2f_dx2dx5.subs(x_temp).evalf(),d2f_dx3dx5.subs(x_temp).evalf(),d2f_dx4dx5.subs(x_temp).evalf(),d2f_dx52.subs(x_temp).evalf()]])#Hessiano
    actual_z = f.subs(x_temp)
    z = np.append(z,[actual_z.evalf()]) # Evaluación del punto actual
    if Abs(actual_z-last_z).evalf()<=error:
        break
    else:
        last_z = actual_z
print("last iterarion: " + str(i))
print("final Z value: " + str(actual_z.evalf()))
print("last x values: "+ str(new_x))
```

    last iterarion: 101
    final Z value: 0.0151586750491971
    last x values: [0.00404241467517202 -0.00404241467517206 -0.00404241467517194
     -7.34771631268092e-8 -0.00404241467517196]

### 10 Dimensiones

```python
x1,x2,x3,x4,x5,x6,x7,x8,x9,x10= symbols('x1 x2 x3 x4 x5 x6 x7 x8 x9 x10') #variables simbolicas
x = np.array([[-0.5,0.5,0.5,0.5,0.5,-0.5,0.5,0.5,0.5,0.5]])#Vector de pasos con el punto inicial
lamb = 1000 #Valor lambda para el meta-hessiano
z = np.array([]) #Vector de resultados
ide = np.identity(d) #matriz de identidad
error = 10**-60 #Error minimo, criterio de parada

a = 20
b = 0.2
c = 2*pi
d = 10

f=-a*exp(-b*sqrt(1/d*(x1**2+x2**2+x3**2+x4**3+x5**2+x6**2+x7**2+x8**2+x9**3+x10**2)))-exp(1/d*(cos(c*x1)+cos(c*x2)+cos(c*x3)+cos(c*x4)+cos(c*x5)+cos(c*x6)+cos(c*x7)+cos(c*x8)+cos(c*x9)+cos(c*x10)))+a+exp(1)#función objetivo
#Derivadas
df_dx1=diff(f,x1)
df_dx2=diff(f,x2)
df_dx3=diff(f,x3)
df_dx4=diff(f,x4)
df_dx5=diff(f,x5)
df_dx6=diff(f,x6)
df_dx7=diff(f,x7)
df_dx8=diff(f,x8)
df_dx9=diff(f,x9)
df_dx10=diff(f,x10)

d2f_dx12=diff(df_dx1,x1)
d2f_dx22=diff(df_dx2,x2)
d2f_dx32=diff(df_dx3,x3)
d2f_dx42=diff(df_dx4,x4)
d2f_dx52=diff(df_dx5,x5)
d2f_dx62=diff(df_dx6,x6)
d2f_dx72=diff(df_dx7,x7)
d2f_dx82=diff(df_dx8,x8)
d2f_dx92=diff(df_dx9,x9)
d2f_dx102=diff(df_dx10,x10)

d2f_dx1dx2=diff(df_dx1,x2)
d2f_dx1dx3=diff(df_dx1,x3)
d2f_dx1dx4=diff(df_dx1,x4)
d2f_dx1dx5=diff(df_dx1,x5)
d2f_dx1dx6=diff(df_dx1,x6)
d2f_dx1dx7=diff(df_dx1,x7)
d2f_dx1dx8=diff(df_dx1,x8)
d2f_dx1dx9=diff(df_dx1,x9)
d2f_dx1dx10=diff(df_dx1,x10)
d2f_dx2dx3=diff(df_dx2,x3)
d2f_dx2dx4=diff(df_dx2,x4)
d2f_dx2dx5=diff(df_dx2,x5)
d2f_dx2dx6=diff(df_dx2,x6)
d2f_dx2dx7=diff(df_dx2,x7)
d2f_dx2dx8=diff(df_dx2,x8)
d2f_dx2dx9=diff(df_dx2,x9)
d2f_dx2dx10=diff(df_dx2,x10)
d2f_dx3dx4=diff(df_dx3,x4)
d2f_dx3dx5=diff(df_dx3,x5)
d2f_dx3dx6=diff(df_dx3,x6)
d2f_dx3dx7=diff(df_dx3,x7)
d2f_dx3dx8=diff(df_dx3,x8)
d2f_dx3dx9=diff(df_dx3,x9)
d2f_dx3dx10=diff(df_dx3,x10)
d2f_dx4dx5=diff(df_dx4,x5)
d2f_dx4dx6=diff(df_dx4,x6)
d2f_dx4dx7=diff(df_dx4,x7)
d2f_dx4dx8=diff(df_dx4,x8)
d2f_dx4dx9=diff(df_dx4,x9)
d2f_dx4dx10=diff(df_dx4,x10)
d2f_dx5dx6=diff(df_dx5,x6)
d2f_dx5dx7=diff(df_dx5,x7)
d2f_dx5dx8=diff(df_dx5,x8)
d2f_dx5dx9=diff(df_dx5,x9)
d2f_dx5dx10=diff(df_dx5,x10)
d2f_dx6dx7=diff(df_dx6,x7)
d2f_dx6dx8=diff(df_dx6,x8)
d2f_dx6dx9=diff(df_dx6,x9)
d2f_dx6dx10=diff(df_dx6,x10)
d2f_dx7dx8=diff(df_dx7,x8)
d2f_dx7dx9=diff(df_dx7,x9)
d2f_dx7dx10=diff(df_dx7,x10)
d2f_dx8dx9=diff(df_dx8,x9)
d2f_dx8dx10=diff(df_dx8,x10)
d2f_dx9dx10=diff(df_dx9,x10)

x_temp = [(x1,x[0,0]),
          (x2,x[0,1]),
          (x3,x[0,2]),
          (x4,x[0,3]),
          (x5,x[0,4]),
          (x6,x[0,5]),
          (x7,x[0,6]),
          (x8,x[0,7]),
          (x9,x[0,8]),
          (x10,x[0,9])]
grad = np.array([df_dx1.subs(x_temp).evalf(),
                 df_dx2.subs(x_temp).evalf(),
                 df_dx3.subs(x_temp).evalf(),
                 df_dx4.subs(x_temp).evalf(),
                 df_dx5.subs(x_temp).evalf(),
                 df_dx6.subs(x_temp).evalf(),
                 df_dx7.subs(x_temp).evalf(),
                 df_dx8.subs(x_temp).evalf(),
                 df_dx9.subs(x_temp).evalf(),
                 df_dx10.subs(x_temp).evalf()])#Vector gradiente
H = np.array([
    [d2f_dx12.subs(x_temp).evalf(),d2f_dx1dx2.subs(x_temp).evalf(),d2f_dx1dx3.subs(x_temp).evalf(),d2f_dx1dx4.subs(x_temp).evalf(),d2f_dx1dx5.subs(x_temp).evalf(),
    d2f_dx1dx6.subs(x_temp).evalf(),d2f_dx1dx7.subs(x_temp).evalf(),d2f_dx1dx8.subs(x_temp).evalf(),d2f_dx1dx9.subs(x_temp).evalf(),d2f_dx1dx10.subs(x_temp).evalf()],

    [d2f_dx1dx2.subs(x_temp).evalf(),d2f_dx22.subs(x_temp).evalf(),d2f_dx2dx3.subs(x_temp).evalf(),d2f_dx2dx4.subs(x_temp).evalf(),d2f_dx2dx5.subs(x_temp).evalf(),
    d2f_dx2dx6.subs(x_temp).evalf(),d2f_dx2dx7.subs(x_temp).evalf(),d2f_dx2dx8.subs(x_temp).evalf(),d2f_dx2dx9.subs(x_temp).evalf(),d2f_dx2dx10.subs(x_temp).evalf()],

    [d2f_dx1dx3.subs(x_temp).evalf(),d2f_dx2dx3.subs(x_temp).evalf(),d2f_dx32.subs(x_temp).evalf(),d2f_dx3dx4.subs(x_temp).evalf(),d2f_dx3dx5.subs(x_temp).evalf(),
    d2f_dx3dx6.subs(x_temp).evalf(),d2f_dx3dx7.subs(x_temp).evalf(),d2f_dx3dx8.subs(x_temp).evalf(),d2f_dx3dx9.subs(x_temp).evalf(),d2f_dx3dx10.subs(x_temp).evalf()],

    [d2f_dx1dx4.subs(x_temp).evalf(),d2f_dx2dx4.subs(x_temp).evalf(),d2f_dx3dx4.subs(x_temp).evalf(),d2f_dx42.subs(x_temp).evalf(),d2f_dx4dx5.subs(x_temp).evalf(),
    d2f_dx4dx6.subs(x_temp).evalf(),d2f_dx4dx7.subs(x_temp).evalf(),d2f_dx4dx8.subs(x_temp).evalf(),d2f_dx4dx9.subs(x_temp).evalf(),d2f_dx4dx10.subs(x_temp).evalf()],

    [d2f_dx1dx5.subs(x_temp).evalf(),d2f_dx2dx5.subs(x_temp).evalf(),d2f_dx3dx5.subs(x_temp).evalf(),d2f_dx4dx5.subs(x_temp).evalf(),d2f_dx52.subs(x_temp).evalf(),
    d2f_dx5dx6.subs(x_temp).evalf(),d2f_dx5dx7.subs(x_temp).evalf(),d2f_dx5dx8.subs(x_temp).evalf(),d2f_dx5dx9.subs(x_temp).evalf(),d2f_dx5dx10.subs(x_temp).evalf()],

    [d2f_dx1dx6.subs(x_temp).evalf(),d2f_dx2dx6.subs(x_temp).evalf(),d2f_dx3dx6.subs(x_temp).evalf(),d2f_dx4dx6.subs(x_temp).evalf(),d2f_dx5dx6.subs(x_temp).evalf(),
    d2f_dx62.subs(x_temp).evalf(),d2f_dx6dx7.subs(x_temp).evalf(),d2f_dx6dx8.subs(x_temp).evalf(),d2f_dx6dx9.subs(x_temp).evalf(),d2f_dx6dx10.subs(x_temp).evalf()],

    [d2f_dx1dx7.subs(x_temp).evalf(),d2f_dx2dx7.subs(x_temp).evalf(),d2f_dx3dx7.subs(x_temp).evalf(),d2f_dx4dx7.subs(x_temp).evalf(),d2f_dx5dx7.subs(x_temp).evalf(),
    d2f_dx6dx7.subs(x_temp).evalf(),d2f_dx72.subs(x_temp).evalf(),d2f_dx7dx8.subs(x_temp).evalf(),d2f_dx7dx9.subs(x_temp).evalf(),d2f_dx7dx10.subs(x_temp).evalf()],

    [d2f_dx1dx8.subs(x_temp).evalf(),d2f_dx2dx8.subs(x_temp).evalf(),d2f_dx3dx8.subs(x_temp).evalf(),d2f_dx4dx8.subs(x_temp).evalf(),d2f_dx5dx8.subs(x_temp).evalf(),
    d2f_dx6dx8.subs(x_temp).evalf(),d2f_dx7dx8.subs(x_temp).evalf(),d2f_dx82.subs(x_temp).evalf(),d2f_dx8dx9.subs(x_temp).evalf(),d2f_dx8dx10.subs(x_temp).evalf()],

    [d2f_dx1dx9.subs(x_temp).evalf(),d2f_dx2dx9.subs(x_temp).evalf(),d2f_dx3dx9.subs(x_temp).evalf(),d2f_dx4dx9.subs(x_temp).evalf(),d2f_dx5dx9.subs(x_temp).evalf(),
    d2f_dx6dx9.subs(x_temp).evalf(),d2f_dx7dx9.subs(x_temp).evalf(),d2f_dx8dx9.subs(x_temp).evalf(),d2f_dx92.subs(x_temp).evalf(),d2f_dx9dx10.subs(x_temp).evalf()],

    [d2f_dx1dx10.subs(x_temp).evalf(),d2f_dx2dx10.subs(x_temp).evalf(),d2f_dx3dx10.subs(x_temp).evalf(),d2f_dx4dx10.subs(x_temp).evalf(),d2f_dx5dx10.subs(x_temp).evalf(),
    d2f_dx6dx10.subs(x_temp).evalf(),d2f_dx7dx10.subs(x_temp).evalf(),d2f_dx8dx10.subs(x_temp).evalf(),d2f_dx9dx10.subs(x_temp).evalf(),d2f_dx102.subs(x_temp).evalf()]])#Hessiano
last_z = f.subs(x_temp)#evaluar Z con el punto inicial
z = np.append(z,[last_z.evalf()]) # Evaluación del punto inicial
i = 0
while i <= 100:
    h_new = npl.inv(((lamb*ide)+H.astype(float))) #Inverso del Hessiano Aproximado.
    new_x = (x[i].T-np.matmul(h_new,grad.T)).T #Siguiente paso
    x = np.append(x,new_x).reshape((i+2),d) #Agregar el paso al vector de pasos
    i += 1 #Aumentar el contador de iteraciones

    x_temp = [(x1,x[0,0]),
              (x2,x[0,1]),
              (x3,x[0,2]),
              (x4,x[0,3]),
              (x5,x[0,4]),
              (x6,x[0,5]),
              (x7,x[0,6]),
              (x8,x[0,7]),
              (x9,x[0,8]),
              (x10,x[0,9])]
    grad = np.array([df_dx1.subs(x_temp).evalf(),
                     df_dx2.subs(x_temp).evalf(),
                     df_dx3.subs(x_temp).evalf(),
                     df_dx4.subs(x_temp).evalf(),
                     df_dx5.subs(x_temp).evalf(),
                     df_dx6.subs(x_temp).evalf(),
                     df_dx7.subs(x_temp).evalf(),
                     df_dx8.subs(x_temp).evalf(),
                     df_dx9.subs(x_temp).evalf(),
                     df_dx10.subs(x_temp).evalf()])#Vector gradiente
    H = np.array([ #Hessiano
        [d2f_dx12.subs(x_temp).evalf(),d2f_dx1dx2.subs(x_temp).evalf(),d2f_dx1dx3.subs(x_temp).evalf(),d2f_dx1dx4.subs(x_temp).evalf(),d2f_dx1dx5.subs(x_temp).evalf(),
        d2f_dx1dx6.subs(x_temp).evalf(),d2f_dx1dx7.subs(x_temp).evalf(),d2f_dx1dx8.subs(x_temp).evalf(),d2f_dx1dx9.subs(x_temp).evalf(),d2f_dx1dx10.subs(x_temp).evalf()],

        [d2f_dx1dx2.subs(x_temp).evalf(),d2f_dx22.subs(x_temp).evalf(),d2f_dx2dx3.subs(x_temp).evalf(),d2f_dx2dx4.subs(x_temp).evalf(),d2f_dx2dx5.subs(x_temp).evalf(),
        d2f_dx2dx6.subs(x_temp).evalf(),d2f_dx2dx7.subs(x_temp).evalf(),d2f_dx2dx8.subs(x_temp).evalf(),d2f_dx2dx9.subs(x_temp).evalf(),d2f_dx2dx10.subs(x_temp).evalf()],

        [d2f_dx1dx3.subs(x_temp).evalf(),d2f_dx2dx3.subs(x_temp).evalf(),d2f_dx32.subs(x_temp).evalf(),d2f_dx3dx4.subs(x_temp).evalf(),d2f_dx3dx5.subs(x_temp).evalf(),
        d2f_dx3dx6.subs(x_temp).evalf(),d2f_dx3dx7.subs(x_temp).evalf(),d2f_dx3dx8.subs(x_temp).evalf(),d2f_dx3dx9.subs(x_temp).evalf(),d2f_dx3dx10.subs(x_temp).evalf()],

        [d2f_dx1dx4.subs(x_temp).evalf(),d2f_dx2dx4.subs(x_temp).evalf(),d2f_dx3dx4.subs(x_temp).evalf(),d2f_dx42.subs(x_temp).evalf(),d2f_dx4dx5.subs(x_temp).evalf(),
        d2f_dx4dx6.subs(x_temp).evalf(),d2f_dx4dx7.subs(x_temp).evalf(),d2f_dx4dx8.subs(x_temp).evalf(),d2f_dx4dx9.subs(x_temp).evalf(),d2f_dx4dx10.subs(x_temp).evalf()],

        [d2f_dx1dx5.subs(x_temp).evalf(),d2f_dx2dx5.subs(x_temp).evalf(),d2f_dx3dx5.subs(x_temp).evalf(),d2f_dx4dx5.subs(x_temp).evalf(),d2f_dx52.subs(x_temp).evalf(),
        d2f_dx5dx6.subs(x_temp).evalf(),d2f_dx5dx7.subs(x_temp).evalf(),d2f_dx5dx8.subs(x_temp).evalf(),d2f_dx5dx9.subs(x_temp).evalf(),d2f_dx5dx10.subs(x_temp).evalf()],

        [d2f_dx1dx6.subs(x_temp).evalf(),d2f_dx2dx6.subs(x_temp).evalf(),d2f_dx3dx6.subs(x_temp).evalf(),d2f_dx4dx6.subs(x_temp).evalf(),d2f_dx5dx6.subs(x_temp).evalf(),
        d2f_dx62.subs(x_temp).evalf(),d2f_dx6dx7.subs(x_temp).evalf(),d2f_dx6dx8.subs(x_temp).evalf(),d2f_dx6dx9.subs(x_temp).evalf(),d2f_dx6dx10.subs(x_temp).evalf()],

        [d2f_dx1dx7.subs(x_temp).evalf(),d2f_dx2dx7.subs(x_temp).evalf(),d2f_dx3dx7.subs(x_temp).evalf(),d2f_dx4dx7.subs(x_temp).evalf(),d2f_dx5dx7.subs(x_temp).evalf(),
        d2f_dx6dx7.subs(x_temp).evalf(),d2f_dx72.subs(x_temp).evalf(),d2f_dx7dx8.subs(x_temp).evalf(),d2f_dx7dx9.subs(x_temp).evalf(),d2f_dx7dx10.subs(x_temp).evalf()],

        [d2f_dx1dx8.subs(x_temp).evalf(),d2f_dx2dx8.subs(x_temp).evalf(),d2f_dx3dx8.subs(x_temp).evalf(),d2f_dx4dx8.subs(x_temp).evalf(),d2f_dx5dx8.subs(x_temp).evalf(),
        d2f_dx6dx8.subs(x_temp).evalf(),d2f_dx7dx8.subs(x_temp).evalf(),d2f_dx82.subs(x_temp).evalf(),d2f_dx8dx9.subs(x_temp).evalf(),d2f_dx8dx10.subs(x_temp).evalf()],

        [d2f_dx1dx9.subs(x_temp).evalf(),d2f_dx2dx9.subs(x_temp).evalf(),d2f_dx3dx9.subs(x_temp).evalf(),d2f_dx4dx9.subs(x_temp).evalf(),d2f_dx5dx9.subs(x_temp).evalf(),
        d2f_dx6dx9.subs(x_temp).evalf(),d2f_dx7dx9.subs(x_temp).evalf(),d2f_dx8dx9.subs(x_temp).evalf(),d2f_dx92.subs(x_temp).evalf(),d2f_dx9dx10.subs(x_temp).evalf()],

        [d2f_dx1dx10.subs(x_temp).evalf(),d2f_dx2dx10.subs(x_temp).evalf(),d2f_dx3dx10.subs(x_temp).evalf(),d2f_dx4dx10.subs(x_temp).evalf(),d2f_dx5dx10.subs(x_temp).evalf(),
        d2f_dx6dx10.subs(x_temp).evalf(),d2f_dx7dx10.subs(x_temp).evalf(),d2f_dx8dx10.subs(x_temp).evalf(),d2f_dx9dx10.subs(x_temp).evalf(),d2f_dx102.subs(x_temp).evalf()]])

    actual_z = f.subs(x_temp)
    z = np.append(z,[actual_z.evalf()]) # Evaluación del punto actual
    if Abs(actual_z-last_z).evalf()<=error:
        break
    else:
        last_z = actual_z
print("last iterarion: " + str(i))
print("final Z value: " + str(actual_z.evalf()))
print("last x values: "+ str(new_x))
```

    last iterarion: 1
    final Z value: 4.16054879402645
    last x values: [-0.499615934016941 0.499615934016941 0.499615934016941 0.499712061006067
     0.499615934016941 -0.499615934016941 0.499615934016941 0.499615934016941
     0.499712061006067 0.499615934016941]

## Funcion de Rastrigin

### 2 Dimensiones

```python
x1, x2= symbols('x1 x2') #variables simbolicas
x = np.array([[-0.5,0.5]])#Vector de pasos con el punto inicial
lamb = 200 #Valor lambda para el meta-hessiano
z = np.array([]) #Vector de resultados
error = 10**-60 #Error minimo, criterio de parada

d=2

f=10*d+((x1**2-10*cos(2*pi*x1))+(x2**2-10*cos(2*pi*x2))) #función objetivo
#Derivadas
df_dx1=diff(f,x1)
df_dx2=diff(f,x2)
d2f_dx12=diff(df_dx1,x1)
d2f_dx22=diff(df_dx2,x2)
d2f_dx1dx2=diff(df_dx1,x2)
x_temp = [(x1,x[0,0]),(x2,x[0,1])]
grad = np.array([df_dx1.subs(x_temp).evalf(),df_dx2.subs(x_temp).evalf()])#Vector gradiente
H = np.array([[d2f_dx12.subs(x_temp).evalf(),d2f_dx1dx2.subs(x_temp).evalf()],[d2f_dx1dx2.subs(x_temp).evalf(),d2f_dx22.subs(x_temp).evalf()]])#Hessiano
last_z = f.subs(x_temp)#evaluar Z con el punto inicial
z = np.append(z,[last_z.evalf()]) # Evaluación del punto inicial
ide = np.identity(d) #matriz de identidad
i = 0
while i <= 100:
    h_new = npl.inv(((lamb*ide)+H.astype(float))) #Inverso del Hessiano Aproximado.
    new_x = (x[i].T-np.matmul(h_new,grad.T)).T #Siguiente paso
    x = np.append(x,new_x).reshape((i+2),d) #Agregar el paso al vector de pasos
    i += 1 #Aumentar el contador de iteraciones
    x_temp = [(x1,x[i,0]),(x2,x[i,1])]
    grad = np.array([df_dx1.subs(x_temp).evalf(),df_dx2.subs(x_temp).evalf()])#Vector gradiente
    H = np.array([[d2f_dx12.subs(x_temp).evalf(),d2f_dx1dx2.subs(x_temp).evalf()],[d2f_dx1dx2.subs(x_temp).evalf(),d2f_dx22.subs(x_temp).evalf()]])#Hessiano
    actual_z = f.subs(x_temp)
    z = np.append(z,[actual_z.evalf()]) # Evaluación del punto actual
    if Abs(actual_z-last_z).evalf()<=error:
        break
    else:
        last_z = actual_z
print("last iterarion: " + str(i))
print("final Z value: " + str(actual_z.evalf()))
print("last x values: "+ str(new_x))
```

    last iterarion: 101
    final Z value: 1.07220843741900e-36
    last x values: [5.21146399753934e-20 -5.21146399753934e-20]

```python
vals = np.arange(-1,1,0.1)
z_c = np.ones(len(vals)**2).reshape(len(vals),len(vals))
for i in range(0,len(vals)):
    for j in range(0,len(vals)):
        x_test = [(x1,vals[i]),(x2,vals[j])]
        z_c[i,j] = float(f.subs(x_test).evalf())
plt.contourf(vals,vals,z_c)
plt.plot(x[:,0],x[:,1],c='red')
plt.colorbar();
```

### 5 Dimensiones

```python
x1, x2, x3, x4, x5= symbols('x1 x2 x3 x4 x5') #variables simbolicas
x = np.array([[-0.5,0.5,0.5,0.5,0.5]])#Vector de pasos con el punto inicial
lamb = 200 #Valor lambda para el meta-hessiano
z = np.array([]) #Vector de resultados
error = 10**-60 #Error minimo, criterio de parada

d = 5

f=10*d+((x1**2-10*cos(2*pi*x1))+(x2**2-10*cos(2*pi*x2))+(x3**2-10*cos(2*pi*x3))+(x4**2-10*cos(2*pi*x4))+(x5**2-10*cos(2*pi*x5))) #función objetivo
#Derivadas
df_dx1=diff(f,x1)
df_dx2=diff(f,x2)
df_dx3=diff(f,x3)
df_dx4=diff(f,x4)
df_dx5=diff(f,x5)

d2f_dx12=diff(df_dx1,x1)
d2f_dx22=diff(df_dx2,x2)
d2f_dx32=diff(df_dx3,x3)
d2f_dx42=diff(df_dx4,x4)
d2f_dx52=diff(df_dx5,x5)

d2f_dx1dx2=diff(df_dx1,x2)
d2f_dx1dx3=diff(df_dx1,x3)
d2f_dx1dx4=diff(df_dx1,x4)
d2f_dx1dx5=diff(df_dx1,x5)
d2f_dx2dx3=diff(df_dx2,x3)
d2f_dx2dx4=diff(df_dx2,x4)
d2f_dx2dx5=diff(df_dx2,x5)
d2f_dx3dx4=diff(df_dx3,x4)
d2f_dx3dx5=diff(df_dx3,x5)
d2f_dx4dx5=diff(df_dx4,x5)

x_temp = [(x1,x[0,0]),
          (x2,x[0,1]),
          (x3,x[0,2]),
          (x4,x[0,3]),
          (x5,x[0,4])]
grad = np.array([df_dx1.subs(x_temp).evalf(),
                 df_dx2.subs(x_temp).evalf(),
                 df_dx3.subs(x_temp).evalf(),
                 df_dx4.subs(x_temp).evalf(),
                 df_dx5.subs(x_temp).evalf()])#Vector gradiente
H = np.array([
    [d2f_dx12.subs(x_temp).evalf(),d2f_dx1dx2.subs(x_temp).evalf(),d2f_dx1dx3.subs(x_temp).evalf(),d2f_dx1dx4.subs(x_temp).evalf(),d2f_dx1dx5.subs(x_temp).evalf()],
    [d2f_dx1dx2.subs(x_temp).evalf(),d2f_dx22.subs(x_temp).evalf(),d2f_dx2dx3.subs(x_temp).evalf(),d2f_dx2dx4.subs(x_temp).evalf(),d2f_dx2dx5.subs(x_temp).evalf()],
    [d2f_dx1dx3.subs(x_temp).evalf(),d2f_dx2dx3.subs(x_temp).evalf(),d2f_dx32.subs(x_temp).evalf(),d2f_dx3dx4.subs(x_temp).evalf(),d2f_dx3dx5.subs(x_temp).evalf()],
    [d2f_dx1dx4.subs(x_temp).evalf(),d2f_dx2dx4.subs(x_temp).evalf(),d2f_dx3dx4.subs(x_temp).evalf(),d2f_dx42.subs(x_temp).evalf(),d2f_dx4dx5.subs(x_temp).evalf()],
    [d2f_dx1dx5.subs(x_temp).evalf(),d2f_dx2dx5.subs(x_temp).evalf(),d2f_dx3dx5.subs(x_temp).evalf(),d2f_dx4dx5.subs(x_temp).evalf(),d2f_dx52.subs(x_temp).evalf()]])#Hessiano
last_z = f.subs(x_temp)#evaluar Z con el punto inicial
z = np.append(z,[last_z.evalf()]) # Evaluación del punto inicial
ide = np.identity(d) #matriz de identidad
i = 0
while i <= 100:
    h_new = npl.inv(((lamb*ide)+H.astype(float))) #Inverso del Hessiano Aproximado.
    new_x = (x[i].T-np.matmul(h_new,grad.T)).T #Siguiente paso
    x = np.append(x,new_x).reshape((i+2),d) #Agregar el paso al vector de pasos
    i += 1 #Aumentar el contador de iteraciones
    x_temp = [(x1,x[i,0]),
              (x2,x[i,1]),
              (x3,x[i,2]),
              (x4,x[i,3]),
              (x5,x[i,4])]
    grad = np.array([df_dx1.subs(x_temp).evalf(),
                 df_dx2.subs(x_temp).evalf(),
                 df_dx3.subs(x_temp).evalf(),
                 df_dx4.subs(x_temp).evalf(),
                 df_dx5.subs(x_temp).evalf()])#Vector gradiente
    H = np.array([
        [d2f_dx12.subs(x_temp).evalf(),d2f_dx1dx2.subs(x_temp).evalf(),d2f_dx1dx3.subs(x_temp).evalf(),d2f_dx1dx4.subs(x_temp).evalf(),d2f_dx1dx5.subs(x_temp).evalf()],
        [d2f_dx1dx2.subs(x_temp).evalf(),d2f_dx22.subs(x_temp).evalf(),d2f_dx2dx3.subs(x_temp).evalf(),d2f_dx2dx4.subs(x_temp).evalf(),d2f_dx2dx5.subs(x_temp).evalf()],
        [d2f_dx1dx3.subs(x_temp).evalf(),d2f_dx2dx3.subs(x_temp).evalf(),d2f_dx32.subs(x_temp).evalf(),d2f_dx3dx4.subs(x_temp).evalf(),d2f_dx3dx5.subs(x_temp).evalf()],
        [d2f_dx1dx4.subs(x_temp).evalf(),d2f_dx2dx4.subs(x_temp).evalf(),d2f_dx3dx4.subs(x_temp).evalf(),d2f_dx42.subs(x_temp).evalf(),d2f_dx4dx5.subs(x_temp).evalf()],
        [d2f_dx1dx5.subs(x_temp).evalf(),d2f_dx2dx5.subs(x_temp).evalf(),d2f_dx3dx5.subs(x_temp).evalf(),d2f_dx4dx5.subs(x_temp).evalf(),d2f_dx52.subs(x_temp).evalf()]])#Hessiano
    actual_z = f.subs(x_temp)
    z = np.append(z,[actual_z.evalf()]) # Evaluación del punto actual
    if Abs(actual_z-last_z).evalf()<=error:
        break
    else:
        last_z = actual_z
print("last iterarion: " + str(i))
print("final Z value: " + str(actual_z.evalf()))
print("last x values: "+ str(new_x))
```

    last iterarion: 101
    final Z value: 2.68052109354750e-36
    last x values: [5.21146399753934e-20 -5.21146399753934e-20 -5.21146399753934e-20
     -5.21146399753934e-20 -5.21146399753934e-20]

### 10 Dimensiones

```python
x1,x2,x3,x4,x5,x6,x7,x8,x9,x10= symbols('x1 x2 x3 x4 x5 x6 x7 x8 x9 x10') #variables simbolicas
x = np.array([[-0.5,0.5,0.5,0.5,0.5,-0.5,0.5,0.5,0.5,0.5]])#Vector de pasos con el punto inicial
lamb = 1000 #Valor lambda para el meta-hessiano
z = np.array([]) #Vector de resultados
error = 10**-60 #Error minimo, criterio de parada

d = 10

f=f=10*d+((x1**2-10*cos(2*pi*x1))+(x2**2-10*cos(2*pi*x2))+(x3**2-10*cos(2*pi*x3))+(x4**2-10*cos(2*pi*x4))+(x5**2-10*cos(2*pi*x5))+(x6**2-10*cos(2*pi*x6))+(x7**2-10*cos(2*pi*x7))+(x8**2-10*cos(2*pi*x8))+(x9**2-10*cos(2*pi*x9))+(x10**2-10*cos(2*pi*x10))) #function objetivo
#Derivadas
df_dx1=diff(f,x1)
df_dx2=diff(f,x2)
df_dx3=diff(f,x3)
df_dx4=diff(f,x4)
df_dx5=diff(f,x5)
df_dx6=diff(f,x6)
df_dx7=diff(f,x7)
df_dx8=diff(f,x8)
df_dx9=diff(f,x9)
df_dx10=diff(f,x10)

d2f_dx12=diff(df_dx1,x1)
d2f_dx22=diff(df_dx2,x2)
d2f_dx32=diff(df_dx3,x3)
d2f_dx42=diff(df_dx4,x4)
d2f_dx52=diff(df_dx5,x5)
d2f_dx62=diff(df_dx6,x6)
d2f_dx72=diff(df_dx7,x7)
d2f_dx82=diff(df_dx8,x8)
d2f_dx92=diff(df_dx9,x9)
d2f_dx102=diff(df_dx10,x10)

d2f_dx1dx2=diff(df_dx1,x2)
d2f_dx1dx3=diff(df_dx1,x3)
d2f_dx1dx4=diff(df_dx1,x4)
d2f_dx1dx5=diff(df_dx1,x5)
d2f_dx1dx6=diff(df_dx1,x6)
d2f_dx1dx7=diff(df_dx1,x7)
d2f_dx1dx8=diff(df_dx1,x8)
d2f_dx1dx9=diff(df_dx1,x9)
d2f_dx1dx10=diff(df_dx1,x10)
d2f_dx2dx3=diff(df_dx2,x3)
d2f_dx2dx4=diff(df_dx2,x4)
d2f_dx2dx5=diff(df_dx2,x5)
d2f_dx2dx6=diff(df_dx2,x6)
d2f_dx2dx7=diff(df_dx2,x7)
d2f_dx2dx8=diff(df_dx2,x8)
d2f_dx2dx9=diff(df_dx2,x9)
d2f_dx2dx10=diff(df_dx2,x10)
d2f_dx3dx4=diff(df_dx3,x4)
d2f_dx3dx5=diff(df_dx3,x5)
d2f_dx3dx6=diff(df_dx3,x6)
d2f_dx3dx7=diff(df_dx3,x7)
d2f_dx3dx8=diff(df_dx3,x8)
d2f_dx3dx9=diff(df_dx3,x9)
d2f_dx3dx10=diff(df_dx3,x10)
d2f_dx4dx5=diff(df_dx4,x5)
d2f_dx4dx6=diff(df_dx4,x6)
d2f_dx4dx7=diff(df_dx4,x7)
d2f_dx4dx8=diff(df_dx4,x8)
d2f_dx4dx9=diff(df_dx4,x9)
d2f_dx4dx10=diff(df_dx4,x10)
d2f_dx5dx6=diff(df_dx5,x6)
d2f_dx5dx7=diff(df_dx5,x7)
d2f_dx5dx8=diff(df_dx5,x8)
d2f_dx5dx9=diff(df_dx5,x9)
d2f_dx5dx10=diff(df_dx5,x10)
d2f_dx6dx7=diff(df_dx6,x7)
d2f_dx6dx8=diff(df_dx6,x8)
d2f_dx6dx9=diff(df_dx6,x9)
d2f_dx6dx10=diff(df_dx6,x10)
d2f_dx7dx8=diff(df_dx7,x8)
d2f_dx7dx9=diff(df_dx7,x9)
d2f_dx7dx10=diff(df_dx7,x10)
d2f_dx8dx9=diff(df_dx8,x9)
d2f_dx8dx10=diff(df_dx8,x10)
d2f_dx9dx10=diff(df_dx9,x10)

x_temp = [(x1,x[0,0]),
          (x2,x[0,1]),
          (x3,x[0,2]),
          (x4,x[0,3]),
          (x5,x[0,4]),
          (x6,x[0,5]),
          (x7,x[0,6]),
          (x8,x[0,7]),
          (x9,x[0,8]),
          (x10,x[0,9])]
grad = np.array([df_dx1.subs(x_temp).evalf(),
                 df_dx2.subs(x_temp).evalf(),
                 df_dx3.subs(x_temp).evalf(),
                 df_dx4.subs(x_temp).evalf(),
                 df_dx5.subs(x_temp).evalf(),
                 df_dx6.subs(x_temp).evalf(),
                 df_dx7.subs(x_temp).evalf(),
                 df_dx8.subs(x_temp).evalf(),
                 df_dx9.subs(x_temp).evalf(),
                 df_dx10.subs(x_temp).evalf()])#Vector gradiente
H = np.array([
    [d2f_dx12.subs(x_temp).evalf(),d2f_dx1dx2.subs(x_temp).evalf(),d2f_dx1dx3.subs(x_temp).evalf(),d2f_dx1dx4.subs(x_temp).evalf(),d2f_dx1dx5.subs(x_temp).evalf(),
    d2f_dx1dx6.subs(x_temp).evalf(),d2f_dx1dx7.subs(x_temp).evalf(),d2f_dx1dx8.subs(x_temp).evalf(),d2f_dx1dx9.subs(x_temp).evalf(),d2f_dx1dx10.subs(x_temp).evalf()],

    [d2f_dx1dx2.subs(x_temp).evalf(),d2f_dx22.subs(x_temp).evalf(),d2f_dx2dx3.subs(x_temp).evalf(),d2f_dx2dx4.subs(x_temp).evalf(),d2f_dx2dx5.subs(x_temp).evalf(),
    d2f_dx2dx6.subs(x_temp).evalf(),d2f_dx2dx7.subs(x_temp).evalf(),d2f_dx2dx8.subs(x_temp).evalf(),d2f_dx2dx9.subs(x_temp).evalf(),d2f_dx2dx10.subs(x_temp).evalf()],

    [d2f_dx1dx3.subs(x_temp).evalf(),d2f_dx2dx3.subs(x_temp).evalf(),d2f_dx32.subs(x_temp).evalf(),d2f_dx3dx4.subs(x_temp).evalf(),d2f_dx3dx5.subs(x_temp).evalf(),
    d2f_dx3dx6.subs(x_temp).evalf(),d2f_dx3dx7.subs(x_temp).evalf(),d2f_dx3dx8.subs(x_temp).evalf(),d2f_dx3dx9.subs(x_temp).evalf(),d2f_dx3dx10.subs(x_temp).evalf()],

    [d2f_dx1dx4.subs(x_temp).evalf(),d2f_dx2dx4.subs(x_temp).evalf(),d2f_dx3dx4.subs(x_temp).evalf(),d2f_dx42.subs(x_temp).evalf(),d2f_dx4dx5.subs(x_temp).evalf(),
    d2f_dx4dx6.subs(x_temp).evalf(),d2f_dx4dx7.subs(x_temp).evalf(),d2f_dx4dx8.subs(x_temp).evalf(),d2f_dx4dx9.subs(x_temp).evalf(),d2f_dx4dx10.subs(x_temp).evalf()],

    [d2f_dx1dx5.subs(x_temp).evalf(),d2f_dx2dx5.subs(x_temp).evalf(),d2f_dx3dx5.subs(x_temp).evalf(),d2f_dx4dx5.subs(x_temp).evalf(),d2f_dx52.subs(x_temp).evalf(),
    d2f_dx5dx6.subs(x_temp).evalf(),d2f_dx5dx7.subs(x_temp).evalf(),d2f_dx5dx8.subs(x_temp).evalf(),d2f_dx5dx9.subs(x_temp).evalf(),d2f_dx5dx10.subs(x_temp).evalf()],

    [d2f_dx1dx6.subs(x_temp).evalf(),d2f_dx2dx6.subs(x_temp).evalf(),d2f_dx3dx6.subs(x_temp).evalf(),d2f_dx4dx6.subs(x_temp).evalf(),d2f_dx5dx6.subs(x_temp).evalf(),
    d2f_dx62.subs(x_temp).evalf(),d2f_dx6dx7.subs(x_temp).evalf(),d2f_dx6dx8.subs(x_temp).evalf(),d2f_dx6dx9.subs(x_temp).evalf(),d2f_dx6dx10.subs(x_temp).evalf()],

    [d2f_dx1dx7.subs(x_temp).evalf(),d2f_dx2dx7.subs(x_temp).evalf(),d2f_dx3dx7.subs(x_temp).evalf(),d2f_dx4dx7.subs(x_temp).evalf(),d2f_dx5dx7.subs(x_temp).evalf(),
    d2f_dx6dx7.subs(x_temp).evalf(),d2f_dx72.subs(x_temp).evalf(),d2f_dx7dx8.subs(x_temp).evalf(),d2f_dx7dx9.subs(x_temp).evalf(),d2f_dx7dx10.subs(x_temp).evalf()],

    [d2f_dx1dx8.subs(x_temp).evalf(),d2f_dx2dx8.subs(x_temp).evalf(),d2f_dx3dx8.subs(x_temp).evalf(),d2f_dx4dx8.subs(x_temp).evalf(),d2f_dx5dx8.subs(x_temp).evalf(),
    d2f_dx6dx8.subs(x_temp).evalf(),d2f_dx7dx8.subs(x_temp).evalf(),d2f_dx82.subs(x_temp).evalf(),d2f_dx8dx9.subs(x_temp).evalf(),d2f_dx8dx10.subs(x_temp).evalf()],

    [d2f_dx1dx9.subs(x_temp).evalf(),d2f_dx2dx9.subs(x_temp).evalf(),d2f_dx3dx9.subs(x_temp).evalf(),d2f_dx4dx9.subs(x_temp).evalf(),d2f_dx5dx9.subs(x_temp).evalf(),
    d2f_dx6dx9.subs(x_temp).evalf(),d2f_dx7dx9.subs(x_temp).evalf(),d2f_dx8dx9.subs(x_temp).evalf(),d2f_dx92.subs(x_temp).evalf(),d2f_dx9dx10.subs(x_temp).evalf()],

    [d2f_dx1dx10.subs(x_temp).evalf(),d2f_dx2dx10.subs(x_temp).evalf(),d2f_dx3dx10.subs(x_temp).evalf(),d2f_dx4dx10.subs(x_temp).evalf(),d2f_dx5dx10.subs(x_temp).evalf(),
    d2f_dx6dx10.subs(x_temp).evalf(),d2f_dx7dx10.subs(x_temp).evalf(),d2f_dx8dx10.subs(x_temp).evalf(),d2f_dx9dx10.subs(x_temp).evalf(),d2f_dx102.subs(x_temp).evalf()]])#Hessiano
last_z = f.subs(x_temp)#evaluar Z con el punto inicial
z = np.append(z,[last_z.evalf()]) # Evaluación del punto inicial
ide = np.identity(d) #matriz de identidad
i = 0
while i <= 100:
    h_new = npl.inv(((lamb*ide)+H.astype(float))) #Inverso del Hessiano Aproximado.
    new_x = (x[i].T-np.matmul(h_new,grad.T)).T #Siguiente paso
    x = np.append(x,new_x).reshape((i+2),d) #Agregar el paso al vector de pasos
    i += 1 #Aumentar el contador de iteraciones

    x_temp = [(x1,x[0,0]),
              (x2,x[0,1]),
              (x3,x[0,2]),
              (x4,x[0,3]),
              (x5,x[0,4]),
              (x6,x[0,5]),
              (x7,x[0,6]),
              (x8,x[0,7]),
              (x9,x[0,8]),
              (x10,x[0,9])]
    grad = np.array([df_dx1.subs(x_temp).evalf(),
                     df_dx2.subs(x_temp).evalf(),
                     df_dx3.subs(x_temp).evalf(),
                     df_dx4.subs(x_temp).evalf(),
                     df_dx5.subs(x_temp).evalf(),
                     df_dx6.subs(x_temp).evalf(),
                     df_dx7.subs(x_temp).evalf(),
                     df_dx8.subs(x_temp).evalf(),
                     df_dx9.subs(x_temp).evalf(),
                     df_dx10.subs(x_temp).evalf()])#Vector gradiente
    H = np.array([ #Hessiano
        [d2f_dx12.subs(x_temp).evalf(),d2f_dx1dx2.subs(x_temp).evalf(),d2f_dx1dx3.subs(x_temp).evalf(),d2f_dx1dx4.subs(x_temp).evalf(),d2f_dx1dx5.subs(x_temp).evalf(),
        d2f_dx1dx6.subs(x_temp).evalf(),d2f_dx1dx7.subs(x_temp).evalf(),d2f_dx1dx8.subs(x_temp).evalf(),d2f_dx1dx9.subs(x_temp).evalf(),d2f_dx1dx10.subs(x_temp).evalf()],

        [d2f_dx1dx2.subs(x_temp).evalf(),d2f_dx22.subs(x_temp).evalf(),d2f_dx2dx3.subs(x_temp).evalf(),d2f_dx2dx4.subs(x_temp).evalf(),d2f_dx2dx5.subs(x_temp).evalf(),
        d2f_dx2dx6.subs(x_temp).evalf(),d2f_dx2dx7.subs(x_temp).evalf(),d2f_dx2dx8.subs(x_temp).evalf(),d2f_dx2dx9.subs(x_temp).evalf(),d2f_dx2dx10.subs(x_temp).evalf()],

        [d2f_dx1dx3.subs(x_temp).evalf(),d2f_dx2dx3.subs(x_temp).evalf(),d2f_dx32.subs(x_temp).evalf(),d2f_dx3dx4.subs(x_temp).evalf(),d2f_dx3dx5.subs(x_temp).evalf(),
        d2f_dx3dx6.subs(x_temp).evalf(),d2f_dx3dx7.subs(x_temp).evalf(),d2f_dx3dx8.subs(x_temp).evalf(),d2f_dx3dx9.subs(x_temp).evalf(),d2f_dx3dx10.subs(x_temp).evalf()],

        [d2f_dx1dx4.subs(x_temp).evalf(),d2f_dx2dx4.subs(x_temp).evalf(),d2f_dx3dx4.subs(x_temp).evalf(),d2f_dx42.subs(x_temp).evalf(),d2f_dx4dx5.subs(x_temp).evalf(),
        d2f_dx4dx6.subs(x_temp).evalf(),d2f_dx4dx7.subs(x_temp).evalf(),d2f_dx4dx8.subs(x_temp).evalf(),d2f_dx4dx9.subs(x_temp).evalf(),d2f_dx4dx10.subs(x_temp).evalf()],

        [d2f_dx1dx5.subs(x_temp).evalf(),d2f_dx2dx5.subs(x_temp).evalf(),d2f_dx3dx5.subs(x_temp).evalf(),d2f_dx4dx5.subs(x_temp).evalf(),d2f_dx52.subs(x_temp).evalf(),
        d2f_dx5dx6.subs(x_temp).evalf(),d2f_dx5dx7.subs(x_temp).evalf(),d2f_dx5dx8.subs(x_temp).evalf(),d2f_dx5dx9.subs(x_temp).evalf(),d2f_dx5dx10.subs(x_temp).evalf()],

        [d2f_dx1dx6.subs(x_temp).evalf(),d2f_dx2dx6.subs(x_temp).evalf(),d2f_dx3dx6.subs(x_temp).evalf(),d2f_dx4dx6.subs(x_temp).evalf(),d2f_dx5dx6.subs(x_temp).evalf(),
        d2f_dx62.subs(x_temp).evalf(),d2f_dx6dx7.subs(x_temp).evalf(),d2f_dx6dx8.subs(x_temp).evalf(),d2f_dx6dx9.subs(x_temp).evalf(),d2f_dx6dx10.subs(x_temp).evalf()],

        [d2f_dx1dx7.subs(x_temp).evalf(),d2f_dx2dx7.subs(x_temp).evalf(),d2f_dx3dx7.subs(x_temp).evalf(),d2f_dx4dx7.subs(x_temp).evalf(),d2f_dx5dx7.subs(x_temp).evalf(),
        d2f_dx6dx7.subs(x_temp).evalf(),d2f_dx72.subs(x_temp).evalf(),d2f_dx7dx8.subs(x_temp).evalf(),d2f_dx7dx9.subs(x_temp).evalf(),d2f_dx7dx10.subs(x_temp).evalf()],

        [d2f_dx1dx8.subs(x_temp).evalf(),d2f_dx2dx8.subs(x_temp).evalf(),d2f_dx3dx8.subs(x_temp).evalf(),d2f_dx4dx8.subs(x_temp).evalf(),d2f_dx5dx8.subs(x_temp).evalf(),
        d2f_dx6dx8.subs(x_temp).evalf(),d2f_dx7dx8.subs(x_temp).evalf(),d2f_dx82.subs(x_temp).evalf(),d2f_dx8dx9.subs(x_temp).evalf(),d2f_dx8dx10.subs(x_temp).evalf()],

        [d2f_dx1dx9.subs(x_temp).evalf(),d2f_dx2dx9.subs(x_temp).evalf(),d2f_dx3dx9.subs(x_temp).evalf(),d2f_dx4dx9.subs(x_temp).evalf(),d2f_dx5dx9.subs(x_temp).evalf(),
        d2f_dx6dx9.subs(x_temp).evalf(),d2f_dx7dx9.subs(x_temp).evalf(),d2f_dx8dx9.subs(x_temp).evalf(),d2f_dx92.subs(x_temp).evalf(),d2f_dx9dx10.subs(x_temp).evalf()],

        [d2f_dx1dx10.subs(x_temp).evalf(),d2f_dx2dx10.subs(x_temp).evalf(),d2f_dx3dx10.subs(x_temp).evalf(),d2f_dx4dx10.subs(x_temp).evalf(),d2f_dx5dx10.subs(x_temp).evalf(),
        d2f_dx6dx10.subs(x_temp).evalf(),d2f_dx7dx10.subs(x_temp).evalf(),d2f_dx8dx10.subs(x_temp).evalf(),d2f_dx9dx10.subs(x_temp).evalf(),d2f_dx102.subs(x_temp).evalf()]])

    actual_z = f.subs(x_temp)
    z = np.append(z,[actual_z.evalf()]) # Evaluación del punto actual
    if Abs(actual_z-last_z).evalf()<=error:
        break
    else:
        last_z = actual_z
print("last iterarion: " + str(i))
print("final Z value: " + str(actual_z.evalf()))
print("last x values: "+ str(new_x))
```

    last iterarion: 1
    final Z value: 202.500000000000
    last x values: [-0.498353139097258 0.498353139097258 0.498353139097258 0.498353139097258
     0.498353139097258 -0.498353139097258 0.498353139097258 0.498353139097258
     0.498353139097258 0.498353139097258]

## Función de Levy

## Función de Griewank

## Función de Rosenbrock Extendida

```python

```