# Actividad práctica: Predicción de series de tiempo

En esta tarea se pide entrenar y evaluar un predictor para la serie de tiempo Mackey-Glass. Esta serie de tiempo se obtiene de la solución de la siguiente ecuación diferencial

$$
\frac{dy}{dt} = 0.2 \frac{ y(t-\tau)}{1 + y(t-\tau)^{10}} - 0.1 y(t),
$$

donde el parámetro $\tau$ controla el comportamiento dinámico de la serie de tiempo

- Siga las instrucciones en este notebook para resolver el problema de predicción
- Conteste las preguntas que se encuentran en este enunciado
- Finalmente envíe su notebook con los resultados y respuestas a phuijse@inf.uach.cl
- No olvide cambiar el título para reflejar los integrantes de su grupo

```python
import numpy as np

%matplotlib notebook
from IPython.display import display, Audio, HTML
import matplotlib.pylab as plt
from ipywidgets import interact, interactive, fixed, interact_manual
import ipywidgets as widgets

import scipy.signal
%matplotlib notebook
from matplotlib import animation, patches
import soundfile as sf
from style import *

import pandas as pd
from pandas import *

NMSE = lambda y, yhat : np.sum((y - yhat)**2)/np.var(y)
```

***

Use el código que se muestra a continuación para generar la serie de tiempo Mackey Glass

- Considere una razón señal a ruido (SNR) de 2.
- Considere $\tau=17$ (comportamiento debilmente caótico)

Se generaran 1000 muestras de la serie de tiempo. Use los primeros 500 puntos para entrenar y los siguientes 500 puntos para hacer predicción

```python

# número de observaciones (no modificar)
N = 1000
# Razón señal a ruido (2., 0.5)
SNR = 2.
# constantes de la ecuación diferencial (no modificar)
a, b = 0.1, 0.2
# comportamiento dinámico de Mackey-Glass (17, 30)
tau = 17.
print(tau)
# paso de integración (no modificar)
dt = 0.05
# condición incial (no modificar)
y0 = 0.9
# largo temporal (no modificar)
tt = 5.
t = np.linspace(0, tt, num=N)

N_full, tau_full = int(N*tt/dt), int(tau/dt)
ymg = y0*np.ones(shape=(N_full, ))
# Runge-Kutta integration
for n in range(tau_full, N_full-1):
    byd = b*ymg[n-tau_full]/(1.0 + ymg[n-tau_full]**10.0)
    yk1 = dt*(-a*ymg[n] + byd)
    yk2 = dt*(-a*(ymg[n]+yk1/2) + byd)
    yk3 = dt*(-a*(ymg[n]+yk2/2) + byd)
    yk4 = dt*(-a*(ymg[n]+yk3) + byd)
    ymg[n+1] = ymg[n] + yk1/6 + yk2/3 +yk3/3 +yk4/6;
ymg = ymg[::int(tt/dt)]
#ymg = ymg - np.mean(ymg)
# Contaminación con ruido blanco aditivo
s_noise = np.sqrt(np.var(ymg)/SNR)
np.random.seed(0)
y_obs = ymg + s_noise*np.random.randn(len(ymg))

# Gráfico
fig, ax = plt.subplots(1, figsize=(9, 3), tight_layout=True)
ax.plot(t[:500], y_obs[:500])
ax.set_title('Serie de tiempo Mackey-Glass (entrenamiento)');

```

    17.0

    <IPython.core.display.Javascript object>

***

1. Describa en detalle el algoritmo LMS indicando sus semejanzas y diferencias con el filtro de Wiener
1. Partiendo del error instantaneo $J_n^s(\textbf{w}) = e_n^2$ derive la regla de actualización de pesos
1. La siguiente clase de *Python* predice y entrena un filtro LMS. Complete la línea que dice

` self.w = ? `

con el valor correcto de actualización de peso del filtro LMS

 1) EL algoritmo LMS es un tipo de filtro adaptativo, usa el descenso del gradiente para estimar una señal variable en el tiempo. Este algoritmo es uno de los más utilizados usado debido a su baja complejidad computacional y estabilidad.

   El filtro consiste en dos componentes, la primera es el calculo de la salida del filtro FIR convulucionado la entrada con los coeficientes. Ademas de la estimación del error.  y la segunda componente es el coeficiente de actualizacion de pesos.

Semejanzas:
            - LMS converge a la solucion de Wiener
            - LMS es derivado de Wiener
            - Son predictores/estimadores
            - Ambos puede ser usados para identificar la respuesta al impulso de sistema desconocido.
            - Se entrena de manera recursiva y online

Diferencias:
             - LMS: No se requiere conocimiento estadístico del proceso
             - LMS: No es necesario calcular e invertir la matriz de correlación
             - LMS se actualiza online y tiene costo 𝐿.
             - LMS: El aprendizaje es estadístico
             - Wiener: se entrena offline y tiene costo 𝐿2
             - Wiener: El aprendizaje es determinista.

2) Derivada $J_n^s(\textbf{w}) = e_n^2$

Sabiendo que $e_n = (d_n - y_n)$,  tambien $y_n = \sum_{k=0}^{L} w_{n, k} u_{n-k}$

$$
\begin{align}
J^s_n(\textbf{w}) &= e_n^2  \\
&= (d_n - y_n)^2 \\
&= (d_n - \sum_{k=0}^{L} w_{n, k} u_{n-k} )^2 \\
\end{align}
$$

$$
\begin{align}
\frac{dJ(w)}{dw} &=   2 (d_n - \sum_{k=0}^{L} w_{n, k} u_{n-k})  -u_{n-k} \\
\frac{dJ(w)}{dw} &=   -2 e_n u_{n-k} \\
\end{align}
$$

Usando la regla SGD llegamos a:
$$
\begin{align}
\textbf{w}_{n+1} = \textbf{w}_{n} + 2 \mu e_n \textbf{u}_{n}\nonumber \\
\end{align}
$$

```python
class LMS_filter(object):

    def __init__(self, L=1, mu=0.5, normalized=True):
        self.L = L
        self.mu = mu
        self.w = np.zeros(shape=(L, ))
        self.normalized = normalized

    def __len__(self):
        return self.L

    def predict(self, u):
        return np.dot(self.w, u)

    def update(self, u, d):
        d_pred = self.predict(u)
        norm = 1.
        if self.normalized:
            norm = np.sum(u**2) + 1e-6
        self.w = self.w + 2 * self.mu * (d - d_pred) * u / norm
```

***
## Predicción con algoritmo LMS

1. Entrene el predictor con el algoritmo normalized LMS usando el siguiente bloque de código
1. Construya una tabla con los NMSE de entrenamiento y prueba para distintos valores de $\mu$ y $L$
    - Se recomienda hacer un barrido logarítmico en $\mu$ (por ejemplo `mu=np.logspace(-2, 0, num=20)`)
    - Use al menos los siguientes valores de $L$: [5, 10, 20, 30]
1. Describa cada experimento analizando sus resultados de forma cuantitativa y cualitativa
    - ¿Se sobreajuste el filtro a los datos de entrenamiento?
    - ¿Se desestabiliza el filtro?
1. Indique que combinación obtiene menor MSE de prueba
***

1. Repita el experimento para $\tau = 30$ (comportamiento fuertemente caótico)
1. Compare los resultados obtenidos con cada serie de tiempo. ¿Qué casos son más sencillos y cuales más complicados?

3) Dada la cantidad de posibiladades que existen entre L y mu, revisaremos algunos casos mas relevantes:

Para 𝜏=17 (puede probar los valores, abajo se encuentra el gráfico con un slider)

a) L=10 y  mu=0.784760 Se puede apreciar en el grafico que esta sobreajustado, es decir, anda muy bien
  para el entrenamiento, pero en el test se ve terriblemente afectado. Se desastebiliza bastante.
  Siendo MSE entrenamiento 1032.4427, prueba 5186.2362

b) L=20 y mu=0.29 Este es uno de los mejores casos, no esta sobreajustado, para el entrenamiento como con el test anda bastante bien,
ya que el error cuadratico es bastante pequeño. El filtro no se desastebiliza en general.
Siendo MSE entrenamiento 303.7687, prueba 311.9917.

c) L=30 y mu=0.61 Este es un caso medio, tiene un sobreajuste medio, posee un error cuadratico moderado,
                 Siendo MSE entrenamiento 377.1947, prueba 502.9953

4)
 Valores Optimos L = 30, mu = 0.088587, test = 187.915610

Para 𝜏=30

a) L=10 y  mu=0.784760 Se puede ver en el gráfico que se encuentra sobreajustado, tiene un error cuadratico tremendo, por lo que se desastebiliza bastante.
Siendo MSE entrenamiento 1519.2776, prueba 7629.6103

b) L=20 y mu=0.29 En este caso, se puede apreciar que esta mediamante sobreajustado, el error cuadratico es medio. Se desastebiliza medianamente. Siendo MSE entrenamiento 413.9651, prueba 869.1495

c) L=30 Y mu=0.61 Esta bastante sobreajustado, el error cuadratico es muy grande. Se desastebiliza bastante. Siendo MSE entrenamiento 395.9658, prueba 2181.1657.

4) Valores Optimos L = 20, mu = 0.069519, test = 316.133279

----------------------------------------------------------------------------------
Compare los resultados obtenidos con cada serie de tiempo. ¿Qué casos son más sencillos y cuales más complicados?

Se puede apreciar que para el 𝜏=30, en los casos anteriormentes estudiados, el resultado es mucho más caotico, por ejemplo para 𝜏=17 L=20 y mu=0.29, resulta ser uno de lo mejores casos, pero para el 𝜏=30, para ser un caso sin ningún atractivo. Tambien sucede para la mayoria de los otros casos, en vez de mejorar se empeoran más.

```python
data_NMSE_train = []
data_NMSE_test = []
L = [5,10,20,30]
mu = np.logspace(-2,0,num=20)
```

```python
def lms_update(L,mu,print_plot):
    lms = LMS_filter(L, mu, normalized=True)

    # Entrenamiento
    y_pred = np.zeros(shape=(len(y_obs), ))
    for k in range(lms.__len__(), 500):
        y_window = y_obs[k-lms.__len__():k]
        y_pred[k] = lms.predict(y_window)
        lms.update(d=y_obs[k], u=y_window)
    # Prueba
    for k in range(500, len(y_obs)):
        y_window = y_obs[k-lms.__len__():k]
        y_pred[k] = lms.predict(y_window)

    data_NMSE_train.append((NMSE(ymg[lms.__len__():500], y_pred[lms.__len__():500])))
    data_NMSE_test.append(NMSE(ymg[500:], y_pred[500:]))

    if print_plot:
        print("MSE entrenamiento %0.4f, prueba %0.4f" %(NMSE(ymg[lms.__len__():500], y_pred[lms.__len__():500]),
                                                NMSE(ymg[500:], y_pred[500:])))

        fig, ax = plt.subplots(3, figsize=(9, 6), tight_layout=True)
        ax[0].plot(t, y_obs, 'k.', alpha=0.5, label='Observado'); ax[0].legend();
        ax[1].plot(t, ymg, 'g-', alpha=0.5, lw=2, label='Intrínseco');
        ax[1].plot(t[:500], y_pred[:500], alpha=0.75, lw=2, label='Predicho train');
        ax[1].plot(t[500:], y_pred[500:], alpha=0.75, lw=2, label='Predicho test'); ax[1].legend();

        ax[2].plot(t[:500], (ymg[:500] - y_pred[:500])**2, label='Error cuadrático train');
        ax[2].plot(t[500:], (ymg[500:] - y_pred[500:])**2, label='Error cuadrático test'); ax[2].legend();
```

```python
L = [5,10,20,30]
mu = np.logspace(-2,0,num=20)

l_opt = mu_opt = test_opt = 100000000

for l_val in L:
    print("L = %i"%l_val)
    data_NMSE_train = []
    data_NMSE_test = []
    for mu_val in mu:
        lms_update(l_val,mu_val,print_plot=False)
    df = DataFrame({"MU":mu, "MSE TRAIN": data_NMSE_train,"MSE TEST": data_NMSE_test})
    print(df)

    for opt in data_NMSE_test:
        if opt < test_opt:
            test_opt = opt
            l_opt = l_val
            mu_opt = mu[data_NMSE_test.index(opt)]

    if l_val < 30 : print("--------------------------------------------------------------------------")

print("\n Valores Optimos L = %i, mu = %f, test = %f"%(l_opt,mu_opt,test_opt))
```

    L = 5
              MU      MSE TRAIN      MSE TEST
    0   0.010000    1227.875391    772.859405
    1   0.012743    1136.920362    737.473806
    2   0.016238    1058.872798    699.637279
    3   0.020691     990.578039    660.837201
    4   0.026367     930.763455    622.758244
    5   0.033598     879.281332    587.180397
    6   0.042813     836.287966    555.906458
    7   0.054556     801.895461    530.890604
    8   0.069519     776.418603    514.488722
    9   0.088587     760.872241    509.466977
    10  0.112884     757.278881    518.471730
    11  0.143845     768.607048    543.273189
    12  0.183298     798.571314    584.660489
    13  0.233572     851.908173    645.379641
    14  0.297635     936.243278    748.348290
    15  0.379269    1068.216727   1006.644944
    16  0.483293    1291.635428   1772.736146
    17  0.615848    1738.864635   3777.327184
    18  0.784760    3023.395867   8874.572654
    19  1.000000  123371.808378  94442.718775
    --------------------------------------------------------------------------
    L = 10
              MU     MSE TRAIN       MSE TEST
    0   0.010000    912.099052     423.230278
    1   0.012743    804.230523     397.231547
    2   0.016238    713.672711     369.693701
    3   0.020691    635.917830     342.033763
    4   0.026367    568.649212     316.111994
    5   0.033598    511.086790     293.966839
    6   0.042813    463.096531     277.338406
    7   0.054556    424.522610     267.086546
    8   0.069519    394.954180     262.800512
    9   0.088587    373.835671     263.091263
    10  0.112884    360.748398     266.954069
    11  0.143845    355.712192     275.863157
    12  0.183298    359.354943     295.692839
    13  0.233572    372.954128     339.096950
    14  0.297635    398.729870     431.733114
    15  0.379269    441.246101     624.345174
    16  0.483293    512.334656    1007.395141
    17  0.615848    649.907140    1808.116782
    18  0.784760   1032.442654    5186.236229
    19  1.000000  46336.221329  249576.524976
    --------------------------------------------------------------------------
    L = 20
              MU    MSE TRAIN     MSE TEST
    0   0.010000   902.903762   384.048834
    1   0.012743   794.987922   356.407854
    2   0.016238   704.573384   326.186499
    3   0.020691   626.496842   294.540765
    4   0.026367   557.821017   263.240859
    5   0.033598   497.422853   234.607472
    6   0.042813   445.223056   211.297905
    7   0.054556   401.436546   195.900058
    8   0.069519   366.099767   190.286071
    9   0.088587   338.876363   194.811128
    10  0.112884   319.071715   207.820169
    11  0.143845   305.854010   226.244492
    12  0.183298   298.650713   247.712456
    13  0.233572   297.579452   273.612759
    14  0.297635   303.768731   311.991668
    15  0.379269   319.596041   380.104072
    16  0.483293   349.603436   506.636199
    17  0.615848   406.023102   735.343638
    18  0.784760   546.010514  1184.102736
    19  1.000000  6312.370314  3061.585437
    --------------------------------------------------------------------------
    L = 30
              MU    MSE TRAIN      MSE TEST
    0   0.010000   886.420516    393.917378
    1   0.012743   780.536946    367.709917
    2   0.016238   692.560547    338.827779
    3   0.020691   616.853192    308.246748
    4   0.026367   549.896538    277.455504
    5   0.033598   490.108472    248.366341
    6   0.042813   437.245307    223.113853
    7   0.054556   391.667762    203.768189
    8   0.069519   353.756862    191.895248
    9   0.088587   323.584299    187.915610
    10  0.112884   300.827216    190.610961
    11  0.143845   284.908128    197.549334
    12  0.183298   275.299752    206.794641
    13  0.233572   271.912414    219.072401
    14  0.297635   275.611815    239.266085
    15  0.379269   289.035002    277.780126
    16  0.483293   318.121735    353.581804
    17  0.615848   377.194723    502.995255
    18  0.784760   522.921199    807.070882
    19  1.000000  5885.702770  23877.500507

     Valores Optimos L = 30, mu = 0.088587, test = 187.915610

```python
def lms_slider(L,mu):
    lms_update(L,mu,print_plot=True)

print("Plot for TAU=%f"%tau)
interact(lms_slider, L=widgets.SelectionSlider(options=L, value=l_opt),mu=np.logspace(-2,0,num=20))
```

    Plot for TAU=17.000000

    interactive(children=(SelectionSlider(description='L', index=3, options=(5, 10, 20, 30), value=30), Dropdown(d…

    <function __main__.lms_slider(L, mu)>

```python

# número de observaciones (no modificar)
N = 1000
# Razón señal a ruido (2., 0.5)
SNR = 2.
# constantes de la ecuación diferencial (no modificar)
a, b = 0.1, 0.2
# comportamiento dinámico de Mackey-Glass (17, 30)
tau = 30.
print("SERIES RESTARTED WITH TAU=%f"%tau)
# paso de integración (no modificar)
dt = 0.05
# condición incial (no modificar)
y0 = 0.9
# largo temporal (no modificar)
tt = 5.
t = np.linspace(0, tt, num=N)

N_full, tau_full = int(N*tt/dt), int(tau/dt)
ymg = y0*np.ones(shape=(N_full, ))
# Runge-Kutta integration
for n in range(tau_full, N_full-1):
    byd = b*ymg[n-tau_full]/(1.0 + ymg[n-tau_full]**10.0)
    yk1 = dt*(-a*ymg[n] + byd)
    yk2 = dt*(-a*(ymg[n]+yk1/2) + byd)
    yk3 = dt*(-a*(ymg[n]+yk2/2) + byd)
    yk4 = dt*(-a*(ymg[n]+yk3) + byd)
    ymg[n+1] = ymg[n] + yk1/6 + yk2/3 +yk3/3 +yk4/6;
ymg = ymg[::int(tt/dt)]
#ymg = ymg - np.mean(ymg)
# Contaminación con ruido blanco aditivo
s_noise = np.sqrt(np.var(ymg)/SNR)
np.random.seed(0)
y_obs = ymg + s_noise*np.random.randn(len(ymg))
```

    SERIES RESTARTED WITH TAU=30.000000

```python
test_opt = 100000000

for l_val in L:
    print("L = %i"%l_val)
    data_NMSE_train = []
    data_NMSE_test = []
    for mu_val in mu:
        lms_update(l_val,mu_val,print_plot=False)
    df = DataFrame({"MU":mu, "MSE TRAIN": data_NMSE_train,"MSE TEST": data_NMSE_test})
    print(df)

    for opt in data_NMSE_test:
        if opt < test_opt:
            test_opt = opt
            l_opt = l_val
            mu_opt = mu[data_NMSE_test.index(opt)]

    if l_val < 30 : print("--------------------------------------------------------------------------")

print("\n Valores Optimos L = %i, mu = %f, test = %f"%(l_opt,mu_opt,test_opt))
```

    L = 5
              MU      MSE TRAIN       MSE TEST
    0   0.010000     757.438945     484.937709
    1   0.012743     703.711347     468.751682
    2   0.016238     657.686966     451.333702
    3   0.020691     617.535840     433.044089
    4   0.026367     582.719060     414.372074
    5   0.033598     553.516097     396.206732
    6   0.042813     530.495121     379.896389
    7   0.054556     514.218674     367.069998
    8   0.069519     505.222932     359.426844
    9   0.088587     504.122342     359.182371
    10  0.112884     511.802804     371.129315
    11  0.143845     529.944022     406.841344
    12  0.183298     562.114352     490.537855
    13  0.233572     615.426024     666.628826
    14  0.297635     703.386561    1012.345948
    15  0.379269     853.558416    1656.061172
    16  0.483293    1132.264637    2787.574026
    17  0.615848    1745.624303    4736.445823
    18  0.784760    3793.709929   11101.463509
    19  1.000000  196930.621407  446613.865841
    --------------------------------------------------------------------------
    L = 10
              MU     MSE TRAIN      MSE TEST
    0   0.010000    870.855171    575.736496
    1   0.012743    808.265638    548.918064
    2   0.016238    753.366145    520.862259
    3   0.020691    703.984646    492.929239
    4   0.026367    659.278339    466.717307
    5   0.033598    619.397295    444.115552
    6   0.042813    584.979005    427.119332
    7   0.054556    556.711426    417.442913
    8   0.069519    535.080861    416.090568
    9   0.088587    520.276711    423.329601
    10  0.112884    512.202985    439.826130
    11  0.143845    510.639431    470.153593
    12  0.183298    515.701366    530.389344
    13  0.233572    528.775209    659.526811
    14  0.297635    554.086084    928.553645
    15  0.379269    601.481782   1446.145974
    16  0.483293    693.857566   2388.376499
    17  0.615848    897.177145   4116.372447
    18  0.784760   1519.277605   7629.610298
    19  1.000000  35167.718365  44750.857322
    --------------------------------------------------------------------------
    L = 20
              MU    MSE TRAIN       MSE TEST
    0   0.010000   767.269409     442.658269
    1   0.012743   700.754176     422.047926
    2   0.016238   644.344434     400.353810
    3   0.020691   594.562475     378.766320
    4   0.026367   549.460171     358.657196
    5   0.033598   508.423825     341.346440
    6   0.042813   471.678562     327.883817
    7   0.054556   439.793039     319.112061
    8   0.069519   413.413944     316.133279
    9   0.088587   393.221692     321.088324
    10  0.112884   379.955301     338.283643
    11  0.143845   374.411559     376.205191
    12  0.183298   377.461349     451.581229
    13  0.233572   390.160137     596.680169
    14  0.297635   413.965147     869.149515
    15  0.379269   451.249338    1371.865598
    16  0.483293   507.396243    2352.298561
    17  0.615848   600.510505    4662.218734
    18  0.784760   821.753298   12121.763286
    19  1.000000  8394.325860  106860.462964
    --------------------------------------------------------------------------
    L = 30
              MU    MSE TRAIN     MSE TEST
    0   0.010000   756.820731   428.184950
    1   0.012743   692.238041   409.309774
    2   0.016238   637.893521   389.449866
    3   0.020691   590.017396   369.852373
    4   0.026367   546.233093   351.971336
    5   0.033598   505.500058   337.183478
    6   0.042813   467.726190   326.486667
    7   0.054556   433.268969   320.520586
    8   0.069519   402.565133   320.039277
    9   0.088587   375.975583   326.629639
    10  0.112884   353.791937   343.678673
    11  0.143845   336.326109   378.014412
    12  0.183298   324.040591   442.163684
    13  0.233572   317.701456   556.126241
    14  0.297635   318.552016   746.765402
    15  0.379269   328.586965  1045.304657
    16  0.483293   351.373656  1495.887432
    17  0.615848   395.965837  2181.165746
    18  0.784760   504.181616  3149.443421
    19  1.000000  2606.811031  7095.578743

     Valores Optimos L = 20, mu = 0.069519, test = 316.133279

```python
def lms_slider(L,mu):
    lms_update(L,mu,print_plot=True)

print("Plot for TAU=%f"%tau)
interact(lms_slider, L=widgets.SelectionSlider(options=L, value=l_opt),mu=np.logspace(-2,0,num=20))
```

    Plot for TAU=30.000000

    interactive(children=(SelectionSlider(description='L', index=2, options=(5, 10, 20, 30), value=20), Dropdown(d…

    <function __main__.lms_slider(L, mu)>

***
## Predicción con algoritmo RLS

1. Describa en detalle el algoritmo RLS indicando sus semejanzas y diferencias con el algoritmo LMS
1. Partiendo del error histórico $J_N(\textbf{w}) = \sum_{i=1}^N \beta^{N-i} e_i^2$ derive la regla recursiva de actualización de pesos
1. La siguiente clase de *Python* predice y entrena un filtro RLS. Complete las líneas que dice

` self.w = ? ` y `self.Phi_inv = ?`

con el valor correcto de actualización de peso del filtro RLS

1. El  algoritmo RLS provee una mayor velocidad de convergencia y menor error que el algoritmo LMS.
Este algoritmo es una versión muy parecida del algoritmo LMS, pero en el criterio de optimización,
es donde el algoritmo RLS, calcula una solución más exacta, es debido a esto que es más costoso (requiere muchas más operaciones).

Semejanzas:
            - Son estimadores/predictores
            - Ambos son secuenciales, metodos online que resulven el mismo tipo de problema
            - Ambos convergen a la "misma" solución

Diferencias:

             - RLS: Mayor velocidad de convergencia
             - RLS: Menos error
             - RLS: Mayor costo en operaciones
             - RLS: Capaz de adaptarse a cambios más bruscos.

             - LMS: Menos costoso
             - LMS: Proceso Markov

***
2. Partiendo de
$$
\begin{align}
    J_N(\textbf{w}) = \sum_{i=1}^n \beta^{n-i} e_i^2, \text{donde } 0<\beta<=1
\end{align}
$$

El coste de la funcion se minimiza tomando la derivada parcial de $k$ de los coeficientes de $\textbf{w}_n$ e igualando a cero.

$$
\begin{align}
    \frac{\partial{J(\textbf{w}_n)}}{\partial \textbf{w}_n(k)} = \sum_{i=0}^n 2 \beta^{n-i}e_i\frac{\partial e_i}{\partial \textbf{w}_n (k)} = -\sum_{i=0}^n 2 \beta^{n-i}e_i \textbf{u}(i-k)=0
\end{align}
$$

Reemplazando $e_i$ por la definicion de la señal de error obtenemos.

$$
\begin{align}
\sum_{i=0}^n \beta^{N-i} \Big[ d_i - \sum_{k=0}^L \textbf{w}_{n,k} \textbf{u}_{i-k} \Big] d_{i-k} = 0
\end{align}
$$

Reordenando las ecuaciones.

$$
\begin{align}
\sum_{l=0}^L \textbf{w}_{n,k} \Big[ \sum_{i=0}^n \beta^{n-i} \textbf{u}_{i-l} \textbf{u}_{i-k} \Big] = \sum_{i=0}^n\beta^{n-i} d_i \textbf{u}_{i-k}
\end{align}
$$

Esta forma puede ser representada en terminos de matrices.

$$
\begin{align}
\Phi_n \textbf{w}_n = \theta_n
\end{align}
$$

Donde
- Matriz de correlacion ponderada. $\Phi_n = \beta \Phi_{n-1} + \textbf{u}_n \textbf{u}_n^T$
- Vector de correlacion cruzada ponderada. $\theta_n = \theta_{n-1} + d_n \textbf{u}_n$

Asi, podemos encontrar los coeficientes que minimizan la funcion como:

$$
\begin{align}
\textbf{w}_n = \Phi_n^{-1} \theta_n
\end{align}
$$

Queremos derivar una solucion recursiva de la forma

$$
\begin{align}
\textbf{w}_n = \textbf{w}_{n-1} - \Delta \textbf{w}_{n-1}
\end{align}
$$

Estableciendo las condiciones iniciales ($\Phi_0 = \delta I, \theta_0 = 0 $) y usando la identidad de Woodbury: $(A + UCV)^{-1} = A^{-1} - A^{-1} U (C^{-1} + VA^{-1} U)^{-1} V A^{-1}$

Con: $A = \Phi_{n-1}^{-1}, U = \textbf{u}_n, C = 1, V = \textbf{u}_n^{-1}$.
$$
\begin{align}
\Phi^{-1} &= \Big[ \beta \Phi_{n-1} + \textbf{u}_n \textbf{u}_n^T\Big]^{-1} \\
&= \beta^{-1} \Phi_{n-1}^{-1} - \beta^{-1} \textbf{k}_n \textbf{u}_n^T \Phi_{n-1}^{-1}
\end{align}
$$

Donde $\textbf{k}_n = \Phi^{-1}_n \textbf{u}_n$ es la ganancia.

Asi
$$
\begin{align}
\textbf{w}_n &= \Phi^{-1}_n \theta_n \\
&= \beta \Phi^{-1}_n \theta_{n-1} + d_n \Phi^{-1}_n \textbf{u}_n \\
&=  \Phi_{n-1}^{-1} \theta_{n-1} - \textbf{k}_n \textbf{u}_n^T \Phi_{n-1}^{-1} \theta_{n-1} + \Phi_n^{-1} \textbf{u}_n d_n ; \text{  si  }   \textbf{w}_{n-1} =  \Phi_{n-1}^{-1} \theta_{n-1} \\
&= \textbf{w}_{n-1} + \textbf{k}_n ( d_n - \textbf{u}_n^T  \textbf{w}_{n-1} ) \\
&=  \textbf{w}_{n-1} + \textbf{k}_n e_n
\end{align}
$$

Donde $\textbf{k}_n e_n = \Delta \textbf{w}_{n-1}$ tambien llamado Factor de correccion.

```python
class RLS_filter(object):

    def __init__(self, L=1, beta=0.9, delta=10.):
        self.L = L
        self.beta = beta
        self.w = np.zeros(shape=(L, ))
        self.Phi_inv = delta*np.eye(L)

    def __len__(self):
        return self.L

    def predict(self, u):
        return np.dot(self.w, u)

    def update(self, u, d):
        invbeta = 1.0/self.beta
        d_pred = self.predict(u)
        e = d - d_pred
        r = 1. + invbeta*np.dot(np.dot(u, self.Phi_inv), u.T)
        k = invbeta*np.dot(self.Phi_inv, u)/r
        self.Phi_inv = invbeta* self.Phi_inv - invbeta * self.Phi_inv *np.sum(k*u)
        self.w = self.w + np.dot(k,e)

```

***

1. Entrene el predictor con el algoritmo RLS usando el siguiente bloque de código
1. Considere primero  $\tau=17$
1. Construya una tabla con los NMSE de entrenamiento y prueba para distintos valores de $\beta$ y $L$
    - Se recomienda hacer un barrido lineal en $\beta$ (por ejemplo `mu=np.linspace(0.8, 1.0, num=20)`)
    - Use al menos los siguientes valores de $L$: [5, 10, 20, 30]
1. Describa cada experimento analizando sus resultados de forma cuantitativa y cualitativa
    - ¿Cuánto demora el filtro en estabilizarse?
    - ¿Se sobreajuste el filtro a los datos de entrenamiento?
    - ¿Se desestabiliza el filtro?
1. Indique que combinación obtiene menor MSE de prueba
1. Repita el experimento para $\tau=30$
1. Compare con los resultados obtenidos con el algoritmo LMS ¿Qué algoritmo demora menos en converger?

```python
# número de observaciones (no modificar)
N = 1000
# Razón señal a ruido (2., 0.5)
SNR = 2.
# constantes de la ecuación diferencial (no modificar)
a, b = 0.1, 0.2
# comportamiento dinámico de Mackey-Glass (17, 30)
tau = 17.
print("SERIES RESTARTED WITH TAU=%f"%tau)
# paso de integración (no modificar)
dt = 0.05
# condición incial (no modificar)
y0 = 0.9
# largo temporal (no modificar)
tt = 5.
t = np.linspace(0, tt, num=N)

N_full, tau_full = int(N*tt/dt), int(tau/dt)
ymg = y0*np.ones(shape=(N_full, ))
# Runge-Kutta integration
for n in range(tau_full, N_full-1):
    byd = b*ymg[n-tau_full]/(1.0 + ymg[n-tau_full]**10.0)
    yk1 = dt*(-a*ymg[n] + byd)
    yk2 = dt*(-a*(ymg[n]+yk1/2) + byd)
    yk3 = dt*(-a*(ymg[n]+yk2/2) + byd)
    yk4 = dt*(-a*(ymg[n]+yk3) + byd)
    ymg[n+1] = ymg[n] + yk1/6 + yk2/3 +yk3/3 +yk4/6;
ymg = ymg[::int(tt/dt)]
#ymg = ymg - np.mean(ymg)
# Contaminación con ruido blanco aditivo
s_noise = np.sqrt(np.var(ymg)/SNR)
np.random.seed(0)
y_obs = ymg + s_noise*np.random.randn(len(ymg))
```

    SERIES RESTARTED WITH TAU=17.000000

```python
beta = np.linspace(0.8, 1.0, num=20)

data_NMSE_train = []
data_NMSE_test = []
```

```python
beta = np.linspace(0.8, 1.0, num=20)

def rls_update(L,beta,print_plot):
    NMSE = lambda y, yhat : np.sum((y - yhat)**2)/np.var(y)

    rls = RLS_filter(L, beta, delta=1.)
    # Entrenamiento
    y_pred = np.zeros(shape=(len(y_obs), ))
    for k in range(rls.__len__(), 500):
        y_window = y_obs[k-rls.__len__():k]
        rls.update(d=y_obs[k], u=y_window)
        y_pred[k] = rls.predict(y_window)
    # Prueba
    for k in range(500, len(y_obs)):
        y_window = y_obs[k-rls.__len__():k]
        y_pred[k] = rls.predict(y_window)

    data_NMSE_train.append(NMSE(ymg[rls.__len__():500], y_pred[rls.__len__():500]))
    data_NMSE_test.append(NMSE(ymg[500:], y_pred[500:]))

    if print_plot:
        print("MSE entrenamiento %0.6f, prueba %0.6f" %(NMSE(ymg[rls.__len__():500], y_pred[rls.__len__():500]),
                                                        NMSE(ymg[500:], y_pred[500:])))
        fig, ax = plt.subplots(3, figsize=(9, 6), tight_layout=True)
        ax[0].plot(t, y_obs, 'k.', alpha=0.5, label='Observado'); ax[0].legend();
        ax[1].plot(t, ymg, 'g-', alpha=0.5, lw=2, label='Intrínseco');
        ax[1].plot(t[:500], y_pred[:500], alpha=0.75, lw=2, label='Predicho train');
        ax[1].plot(t[500:], y_pred[500:], alpha=0.75, lw=2, label='Predicho test'); ax[1].legend();

        ax[2].plot(t[:500], (ymg[:500] - y_pred[:500])**2, label='Error cuadrático train');
        ax[2].plot(t[500:], (ymg[500:] - y_pred[500:])**2, label='Error cuadrático test'); ax[2].legend();

```

```python
l_opt = beta_opt = test_opt = 100000000

for l_val in L:
    print("L = %i"%l_val)
    data_NMSE_train = []
    data_NMSE_test = []
    for beta_val in beta:
        rls_update(l_val,beta_val,print_plot=False)
    df = DataFrame({"BETA":beta, "MSE TRAIN": data_NMSE_train,"MSE TEST": data_NMSE_test})
    print(df)

    for opt in data_NMSE_test:
        if opt < test_opt:
            test_opt = opt
            l_opt = l_val
            beta_opt = beta[data_NMSE_test.index(opt)]

    if l_val < 30 : print("--------------------------------------------------------------------------")

print("\n Valores Optimos L = %i, BETA = %f, test = %f"%(l_opt,beta_opt,test_opt))
```

    L = 5
            BETA   MSE TRAIN    MSE TEST
    0   0.800000  446.502647  482.575908
    1   0.810526  456.042680  481.412888
    2   0.821053  465.967688  480.929565
    3   0.831579  476.320564  481.214212
    4   0.842105  487.152050  482.365856
    5   0.852632  498.522510  484.496984
    6   0.863158  510.504207  487.737604
    7   0.873684  523.184194  492.241307
    8   0.884211  536.667989  498.194303
    9   0.894737  551.084194  505.828762
    10  0.905263  566.590199  515.442367
    11  0.915789  583.378993  527.426588
    12  0.926316  601.686791  542.306929
    13  0.936842  621.800329  560.798681
    14  0.947368  644.060541  583.880453
    15  0.957895  668.853052  612.880197
    16  0.968421  696.555507  649.536131
    17  0.978947  727.329723  695.841483
    18  0.989474  760.256048  752.456721
    19  1.000000  790.150974  808.209635
    --------------------------------------------------------------------------
    L = 10
            BETA   MSE TRAIN    MSE TEST
    0   0.800000  227.755735  271.361601
    1   0.810526  232.828595  270.703163
    2   0.821053  238.245190  270.101063
    3   0.831579  244.051924  269.582837
    4   0.842105  250.305245  269.195589
    5   0.852632  257.073988  269.012834
    6   0.863158  264.442192  269.143202
    7   0.873684  272.512485  269.741467
    8   0.884211  281.410023  271.022415
    9   0.894737  291.286978  273.278203
    10  0.905263  302.327378  276.899956
    11  0.915789  314.751878  282.404412
    12  0.926316  328.821516  290.466322
    13  0.936842  344.838446  301.956563
    14  0.947368  363.139324  317.983288
    15  0.957895  384.071644  339.924541
    16  0.968421  407.928256  369.409418
    17  0.978947  434.760680  408.070838
    18  0.989474  463.744259  456.074367
    19  1.000000  491.085495  504.133085
    --------------------------------------------------------------------------
    L = 20
            BETA   MSE TRAIN    MSE TEST
    0   0.800000  199.467265  204.890663
    1   0.810526  204.777660  202.010746
    2   0.821053  210.518378  199.393747
    3   0.831579  216.745867  197.142502
    4   0.842105  223.527184  195.385266
    5   0.852632  230.941991  194.281072
    6   0.863158  239.084834  194.026180
    7   0.873684  248.067685  194.861856
    8   0.884211  258.022663  197.083776
    9   0.894737  269.104780  201.053411
    10  0.905263  281.494358  207.211833
    11  0.915789  295.398521  216.096413
    12  0.926316  311.050651  228.360706
    13  0.936842  328.705781  244.797079
    14  0.947368  348.628090  266.358758
    15  0.957895  371.062413  294.168747
    16  0.968421  396.168482  329.471639
    17  0.978947  423.843344  373.352683
    18  0.989474  453.107656  425.245322
    19  1.000000  480.095300  475.273492
    --------------------------------------------------------------------------
    L = 30
            BETA   MSE TRAIN    MSE TEST
    0   0.800000  190.038808  188.372077
    1   0.810526  195.373823  188.061995
    2   0.821053  201.132218  188.056245
    3   0.831579  207.367142  188.442178
    4   0.842105  214.141399  189.327670
    5   0.852632  221.529166  190.845371
    6   0.863158  229.617918  193.157849
    7   0.873684  238.510518  196.463851
    8   0.884211  248.327354  201.006011
    9   0.894737  259.208326  207.080500
    10  0.905263  271.314312  215.049265
    11  0.915789  284.827466  225.355780
    12  0.926316  299.949279  238.545210
    13  0.936842  316.894588  255.289100
    14  0.947368  335.878481  276.411024
    15  0.957895  357.090146  302.898357
    16  0.968421  380.638862  335.852568
    17  0.978947  406.420602  376.213939
    18  0.989474  433.685080  423.413218
    19  1.000000  459.809888  469.209527

     Valores Optimos L = 30, BETA = 0.821053, test = 188.056245

```python
def rls_slider(L,beta):
    rls_update(L,beta,print_plot=True)

print("Plot for TAU=%f"%tau)
interact(rls_slider, L=widgets.SelectionSlider(options=L, value=l_opt),beta = np.linspace(0.8, 1.0, num=20))
```

    Plot for TAU=17.000000

    interactive(children=(SelectionSlider(description='L', index=3, options=(5, 10, 20, 30), value=30), Dropdown(d…

    <function __main__.rls_slider(L, beta)>

```python
# número de observaciones (no modificar)
N = 1000
# Razón señal a ruido (2., 0.5)
SNR = 2.
# constantes de la ecuación diferencial (no modificar)
a, b = 0.1, 0.2
# comportamiento dinámico de Mackey-Glass (17, 30)
tau = 30.
print("SERIES RESTARTED WITH TAU=%f"%tau)
# paso de integración (no modificar)
dt = 0.05
# condición incial (no modificar)
y0 = 0.9
# largo temporal (no modificar)
tt = 5.
t = np.linspace(0, tt, num=N)

N_full, tau_full = int(N*tt/dt), int(tau/dt)
ymg = y0*np.ones(shape=(N_full, ))
# Runge-Kutta integration
for n in range(tau_full, N_full-1):
    byd = b*ymg[n-tau_full]/(1.0 + ymg[n-tau_full]**10.0)
    yk1 = dt*(-a*ymg[n] + byd)
    yk2 = dt*(-a*(ymg[n]+yk1/2) + byd)
    yk3 = dt*(-a*(ymg[n]+yk2/2) + byd)
    yk4 = dt*(-a*(ymg[n]+yk3) + byd)
    ymg[n+1] = ymg[n] + yk1/6 + yk2/3 +yk3/3 +yk4/6;
ymg = ymg[::int(tt/dt)]
#ymg = ymg - np.mean(ymg)
# Contaminación con ruido blanco aditivo
s_noise = np.sqrt(np.var(ymg)/SNR)
np.random.seed(0)
y_obs = ymg + s_noise*np.random.randn(len(ymg))
```

    SERIES RESTARTED WITH TAU=30.000000

```python
l_opt = beta_opt = test_opt = 100000000

for l_val in L:
    print("L = %i"%l_val)
    data_NMSE_train = []
    data_NMSE_test = []
    for beta_val in beta:
        rls_update(l_val,beta_val,print_plot=False)
    df = DataFrame({"BETA":beta, "MSE TRAIN": data_NMSE_train,"MSE TEST": data_NMSE_test})
    print(df)

    for opt in data_NMSE_test:
        if opt < test_opt:
            test_opt = opt
            l_opt = l_val
            beta_opt = beta[data_NMSE_test.index(opt)]

    if l_val < 30 : print("--------------------------------------------------------------------------")

print("\n Valores Optimos L = %i, BETA = %f, test = %f"%(l_opt,beta_opt,test_opt))
```

    L = 5
            BETA   MSE TRAIN    MSE TEST
    0   0.800000  323.883654  330.354249
    1   0.810526  329.435006  328.526812
    2   0.821053  335.164801  327.099040
    3   0.831579  341.094889  326.083027
    4   0.842105  347.252543  325.496281
    5   0.852632  353.671997  325.363987
    6   0.863158  360.396429  325.722606
    7   0.873684  367.480558  326.625488
    8   0.884211  374.994008  328.151402
    9   0.894737  383.025627  330.417267
    10  0.905263  391.688874  333.596751
    11  0.915789  401.128242  337.946705
    12  0.926316  411.526297  343.843360
    13  0.936842  423.110186  351.828807
    14  0.947368  436.155132  362.664046
    15  0.957895  450.979602  377.374411
    16  0.968421  467.917542  397.252316
    17  0.978947  487.208574  423.731746
    18  0.989474  508.504464  457.638543
    19  1.000000  528.608815  493.020841
    --------------------------------------------------------------------------
    L = 10
            BETA   MSE TRAIN    MSE TEST
    0   0.800000  320.974865  368.283423
    1   0.810526  328.200927  363.625214
    2   0.821053  335.734106  359.277451
    3   0.831579  343.617987  355.293606
    4   0.842105  351.907292  351.746971
    5   0.852632  360.670403  348.734764
    6   0.863158  369.992436  346.383664
    7   0.873684  379.979007  344.857443
    8   0.884211  390.760806  344.367680
    9   0.894737  402.499169  345.189023
    10  0.905263  415.392759  347.681213
    11  0.915789  429.685500  352.321407
    12  0.926316  445.675779  359.752264
    13  0.936842  463.726653  370.854050
    14  0.947368  484.275970  386.851489
    15  0.957895  507.841676  409.463408
    16  0.968421  535.000707  441.076343
    17  0.978947  566.234541  484.779139
    18  0.989474  601.073570  543.023977
    19  1.000000  634.231056  605.150474
    --------------------------------------------------------------------------
    L = 20
            BETA   MSE TRAIN    MSE TEST
    0   0.800000  250.499459  329.028505
    1   0.810526  257.136845  326.523364
    2   0.821053  264.257646  324.578681
    3   0.831579  271.914644  323.215547
    4   0.842105  280.167926  322.463131
    5   0.852632  289.085908  322.361561
    6   0.863158  298.746468  322.966148
    7   0.873684  309.238173  324.353635
    8   0.884211  320.661527  326.631564
    9   0.894737  333.130134  329.952442
    10  0.905263  346.771540  334.535244
    11  0.915789  361.727318  340.697777
    12  0.926316  378.151590  348.904153
    13  0.936842  396.206484  359.830703
    14  0.947368  416.051509  374.447714
    15  0.957895  437.820031  394.095710
    16  0.968421  461.563994  420.486020
    17  0.978947  487.100184  455.413987
    18  0.989474  513.457410  499.325295
    19  1.000000  536.900355  543.841889
    --------------------------------------------------------------------------
    L = 30
            BETA   MSE TRAIN    MSE TEST
    0   0.800000  205.789486  330.035713
    1   0.810526  211.474582  326.260133
    2   0.821053  217.525083  322.847427
    3   0.831579  223.971397  319.792380
    4   0.842105  230.847480  317.091142
    5   0.852632  238.191234  314.742435
    6   0.863158  246.044916  312.749659
    7   0.873684  254.455540  311.124594
    8   0.884211  263.475207  309.893936
    9   0.894737  273.161286  309.110646
    10  0.905263  283.576292  308.873204
    11  0.915789  294.787187  309.356895
    12  0.926316  306.863609  310.861580
    13  0.936842  319.874003  313.877590
    14  0.947368  333.877436  319.160926
    15  0.957895  348.905708  327.783358
    16  0.968421  364.921279  341.074554
    17  0.978947  381.703134  360.289919
    18  0.989474  398.448283  385.559318
    19  1.000000  412.394538  411.386558

     Valores Optimos L = 30, BETA = 0.905263, test = 308.873204

```python
def rls_slider(L,beta):
    rls_update(L,beta,print_plot=True)

print("Plot for TAU=%f"%tau)
interact(rls_slider, L=widgets.SelectionSlider(options=L, value=l_opt),beta = np.linspace(0.8, 1.0, num=20))
```

    Plot for TAU=30.000000

    interactive(children=(SelectionSlider(description='L', index=3, options=(5, 10, 20, 30), value=30), Dropdown(d…

    <function __main__.rls_slider(L, beta)>

RLS
Para tau = 17

4.
- Observando los graficos comparativos del filtro RLS, se puede afirmar que la convergencia (entiendase decaimiento  y estabilizacion del error cuadratico) del filtro RLS es bastante rapida (t < 0.2) si miramos el error cuadratico de entrenamiento.
- Para ciertos valores de L y $\beta$ (por ejemplo, L=30 y $\beta$ > 0.9) se puede observar claramente el sobreajuste del filtro a los datos.
- Por lo observado en el grafico, el filtro no alcanza a desestabilizarse de forma notoria, independiente de los valores de L y $\beta$.
5.
A partir de los datos de la tabla para cada combinacion de L y $\beta$, el mejor MSE de prueba para tau=17. se obtiene con
$L = 30, \beta = 0.821, test = 188.056$

RLS
Para tau = 30.

4.
- Para este valor de tau, la convergencia del filtro (entiendase decaimiento y estabilizacion del error cuadratico) es mucho mas rapida, observandose valores por debajo de 0.15[seg] (para L = 30, $\beta$ = 0.905263), sin embargo, para valores de L = 5 y $\beta$ < 1 el valor del error cuadratico medio cambia de manera muy rapida y su valor es en general ,comparativamente para este tau y L, el mas alto.
- Se puede observar claramente un sobreajuste para valores de L=20 y $\beta$ > 0.9. Este sobreajuste es bastante pronunciado, ya que la diferencia entre la seccion del grafico de prueba y el grafico intrinseco es bastante marcada.
- El filtro no se desestabiliza de forma notoria para ningun valor de L y $\beta$.
5.
Analizando los datos de la tabla para tau=30, L=[5,10,20,30] y $0.8 <= \beta <= 1$; el mejor MSE de prueba se obtiene con $L = 30, \beta = 0.905263$ y su valor es MSE Test = $308.8732$

7. Comparando los resultados obtenidos con los diferentes valores de tau, tanto para el filtro LMS como para el filtro RLS, el algoritmo que tiende a converger de manera mas rapida es el algoritmo RLS, aunque se requiera un gasto computacional mayor.


# Quantization of Signals

*This jupyter notebook is part of a [collection of notebooks](../index.ipynb) on various topics of Digital Signal Processing. Please direct questions and suggestions to [Sascha.Spors@uni-rostock.de](mailto:Sascha.Spors@uni-rostock.de).*

## Characteristic of a Linear Uniform Quantizer

The characteristics of a quantizer depend on the mapping functions $f(\cdot)$, $g(\cdot)$ and the rounding operation $\lfloor \cdot \rfloor$ introduced in the [previous section](introduction.ipynb). A linear quantizer bases on linear mapping functions $f(\cdot)$ and $g(\cdot)$. A uniform quantizer splits the mapped input signal into quantization steps of equal size. Quantizers can be described by their nonlinear in-/output characteristic $x_Q[k] = \mathcal{Q} \{ x[k] \}$, where $\mathcal{Q} \{ \cdot \}$ denotes the quantization process. For linear uniform quantization it is common to differentiate between two characteristic curves, the so called mid-tread and mid-rise. Both are introduced in the following.

### Mid-Tread Characteristic Curve

The in-/output relation of the mid-tread quantizer is given as

\begin{equation}
x_Q[k] = Q \cdot \underbrace{\left\lfloor \frac{x[k]}{Q} + \frac{1}{2} \right\rfloor}_{index}
\end{equation}

where $Q$ denotes the constant quantization step size and $\lfloor \cdot \rfloor$ the [floor function](https://en.wikipedia.org/wiki/Floor_and_ceiling_functions) which maps a real number to the largest integer not greater than its argument. Without restricting $x[k]$ in amplitude, the resulting quantization indexes are [countable infinite](https://en.wikipedia.org/wiki/Countable_set). For a finite number of quantization indexes, the input signal has to be restricted to a minimal/maximal amplitude $x_\text{min} < x[k] < x_\text{max}$ before quantization. The resulting quantization characteristic of a linear uniform mid-tread quantizer is shown below

The term mid-tread is due to the fact that small values $|x[k]| < \frac{Q}{2}$ are mapped to zero.

#### Example - Mid-tread quantization of a sine signal

The quantization of one period of a sine signal $x[k] = A \cdot \sin[\Omega_0\,k]$ by a mid-tread quantizer is simulated. $A$ denotes the amplitude of the signal, $x_\text{min} = -1$ and $x_\text{max} = 1$ are the smallest and largest output values of the quantizer, respectively.

```python
%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt

A = 1.2  # amplitude of signal
Q = 1/10  # quantization stepsize
N = 2000  # number of samples

def uniform_midtread_quantizer(x, Q):
    # limiter
    x = np.copy(x)
    idx = np.where(np.abs(x) >= 1)
    x[idx] = np.sign(x[idx])
    # linear uniform quantization
    xQ = Q * np.floor(x/Q + 1/2)

    return xQ

def plot_signals(x, xQ):
    e = xQ - x
    plt.figure(figsize=(10,6))
    plt.plot(x, label=r'signal $x[k]$')
    plt.plot(xQ, label=r'quantized signal $x_Q[k]$')
    plt.plot(e, label=r'quantization error $e[k]$')
    plt.xlabel(r'$k$')
    plt.axis([0, N, -1.1*A, 1.1*A])
    plt.legend()
    plt.grid()

# generate signal
x = A * np.sin(2*np.pi/N * np.arange(N))
# quantize signal
xQ = uniform_midtread_quantizer(x, Q)
# plot signals
plot_signals(x, xQ)
```

**Exercise**

* Change the quantization stepsize `Q` and the amplitude `A` of the signal. Which effect does this have on the quantization error?

Solution: The smaller the quantization step size, the smaller the quantization error is for $|x[k]| < 1$. Note, the quantization error is not bounded for $|x[k]| > 1$ due to the clipping of the signal $x[k]$.

### Mid-Rise Characteristic Curve

The in-/output relation of the mid-rise quantizer is given as

\begin{equation}
x_Q[k] = Q \cdot \Big( \underbrace{\left\lfloor\frac{ x[k] }{Q}\right\rfloor}_{index}  + \frac{1}{2} \Big)
\end{equation}

where $\lfloor \cdot \rfloor$ denotes the floor function. The quantization characteristic of a linear uniform mid-rise quantizer is illustrated below

The term mid-rise copes for the fact that $x[k] = 0$ is not mapped to zero. Small positive/negative values around zero are mapped to $\pm \frac{Q}{2}$.

#### Example - Mid-rise quantization of a sine signal

The previous example is now reevaluated using the mid-rise characteristic

```python
A = 1.2  # amplitude of signal
Q = 1/10  # quantization stepsize
N = 2000  # number of samples

def uniform_midrise_quantizer(x, Q):
    # limiter
    x = np.copy(x)
    idx = np.where(np.abs(x) >= 1)
    x[idx] = np.sign(x[idx])
    # linear uniform quantization
    xQ = Q * (np.floor(x/Q) + .5)

    return xQ

# generate signal
x = A * np.sin(2*np.pi/N * np.arange(N))
# quantize signal
xQ = uniform_midrise_quantizer(x, Q)
# plot signals
plot_signals(x, xQ)
```

**Exercise**

* What are the differences between the mid-tread and the mid-rise characteristic curves for the given example?

Solution: The mid-tread and the mid-rise quantization of the sine signal differ for signal values smaller than half of the quantization interval. Mid-tread has a representation of $x[k] = 0$ while this is not the case for the mid-rise quantization.

**Copyright**

This notebook is provided as [Open Educational Resource](https://en.wikipedia.org/wiki/Open_educational_resources). Feel free to use the notebook for your own purposes. The text is licensed under [Creative Commons Attribution 4.0](https://creativecommons.org/licenses/by/4.0/), the code of the IPython examples under the [MIT license](https://opensource.org/licenses/MIT). Please attribute the work as follows: *Sascha Spors, Digital Signal Processing - Lecture notes featuring computational examples, 2016-2018*.


###### Content provided under a Creative Commons Attribution license, CC-BY 4.0; code under BSD 3-Clause license (c)2015 L.A. Barba, Pi-Yueh Chuang.

# Joukowski Transformation

This is your [_AeroPython_](https://github.com/barbagroup/AeroPython) assignment for the second course module, titled **"Potential vortices and lift."** The first course module, "Building blocks of potential flow" (first three IPython Notebooks of the series) ended with the representation of potential flow around a 2D cylinder by means of superposing a doublet singularity and a free stream. And in the second module, you learned that adding a vortex singularity, you can get lift around the cylinder. You may ask: *is all this useful?*

Here's how it all starts getting useful. Using some simple techniques from complex-variable calculus, we can generate the flow around some airfoils starting with the flow around 2D cylinders. The trick is to use a [*conformal map*](http://en.wikipedia.org/wiki/Conformal_map) (a complex function that preserves angles) to move from the cylinder plane to the airfoil plane.

Let's explore this classic method!

## Introduction

You learned how to construct potential flow over a cylinder via superposition of a free stream and a [doublet](03_Lesson03_doublet.ipynb) singularity. You also learned that adding a vortex can generate [lift on the cylinder](06_Lesson06_vortexLift.ipynb) But what's the big deal? Why is this important to us? What can we do with this potential cylinder flow?

Back in the years when computers were not available, fluid dynamicists and mathematicians used a powerful tool—complex analysis—to study potential flows without directly solving partial-differential governing equations. With the magic of complex analysis and the known solution of potential flow over a cylinder, they could easily obtain many kinds of external potential flows, including flows over several types of airfoils.

Nowadays, though, we no longer use these magical tools. But, not just out of nostalgia, it's still interesting to know the basic concept behind the magic: conformal mappings. In this assignment, we will guide you step by step to obtain potential flow over an airfoil starting with the flow over a cylinder and the famous conformal mapping called the **Joukowski transformation**. You will realize how important potential cylinder flow is in the history of aerodynamics!

Don't worry. We won't talk too much about the mathematics. And you don't have to calculate by hand as the pioneers of aerodynamics did. Just follow the steps and let Python do the heavy lifting.

---

## 1. Complex numbers in Python

Let's start with two complex planes, one defined by points $z = x + iy$, and the other defined by points $\xi = \xi_x+i\xi_y$. The Joukowski transformation takes a point in the $z$-plane and "maps" it to the $\xi$-plane by:

$$
\begin{equation}
\xi = z + \frac{c^2}{z}
\end{equation}
$$

where $c$ is a constant parameter. Before we discuss the Joukowski transformation, let's gain some practice with complex numbers in Python.

Using complex numbers, your function for the Joukowski transformation will look
very simple, and you won't have to calculate the real and imaginary parts separately.

Python (and therefore, NumPy) can deal with complex numbers right out of the box. But the imaginary number, $i=\sqrt{-1}$, is represented by `j`, *not* `i`, to avoid clashes with the common use of `i` in iterations.

If you haven't used them before, try some simple manipulations of complex numbers now. For example, enter the following in a cell block:

```Python
3 + 2j
```

Now try:

```Python
a = 3
b = 3
z = a + b * 1j
print('z = ', z)
print('The type of the variable is ', type(z))
```

### Exercise

Get familiar with complex-number operations in Python by answering the following:

1. $(2.75+3.69i)\times(8.55-6.13i)=$
2. $1.4\times e^{i5.32}=$
3. $\frac{7.51-9.15i}{4.43+9.64i}=$

Don't forget to enter your answers on [Open edX](https://openedx.seas.gwu.edu/courses/course-v1:MAE+MAE6226+2017/about).
(Notes: you will need to enter your results with a 2-digit precision after the decimal point in the auto-graded section.)

---

## 2. Shapes created by the Joukowski transformation

Start by writing a Python function that takes `z` and `c` as parameters, and returns  the Joukowski transformation of `z`.

We can generate several patterns using Joukowski transforms. Use your Python function to perform the calculations described below and answer the questions.

Just take $c=1$ here for simplicity.

1. On the $z$-plane, place a circle with radius $R$ larger than $c=1$, say $R=1.5$, centered at the origin. What does the shape look like when mapped on the $\xi$-plane?
    1. circle
    2. ellipse
    3. symmetric airfoil
    4. cambered airfoil
2. Now, place a circle on the $z$-plane with radius such that $c \lt R \lt 2c$ (e.g., $c=1$; $R=1.2$), centered at $(x_c,y_c)=(c-R, 0)$. What does the shape look like on the $\xi$-plane?
    1. circle
    2. ellipse
    3. symmetric airfoil
    4. cambered airfoil
3. Put a circle at $(x_c, y_c)=(-\Delta x, \Delta y)$, where $\Delta x$ and $\Delta y$ are small positive numbers, e.g. $\Delta x=0.1$ and $\Delta y=0.1$. The radius $R$ of the circle is $R = \sqrt{(c - x_c)^2 + y_c^2}$. What does the shape on $\xi$-plane look like?
    1. circle
    2. ellipse
    3. symmetric airfoil
    4. cambered airfoil
4. Consider the case when you obtain a symmetric airfoil. In polar coordinates, $(\theta, r=R)$, which point on the circle corresponds to the trailing edge of the airfoil?
    * $\theta=$?

Don't forget to enter your answers on [Open edX](https://openedx.seas.gwu.edu/courses/course-v1:MAE+MAE6226+2017/about).
(Notes: you will need to enter your results with a 2-digit precision after the decimal point in the auto-graded section.)

---

## 3. Grid points on the $z$-plane using polar coordinates

By the Joukowski transformation, a point on the $z$-plane corresponds to a point on the $\xi$-plane. As you saw in the previous section, this transformation sometimes gives a shape that looks quite a lot like an airfoil. _What's the use of this?_

It turns out that in complex analysis, when you have a solution to Laplace's equation on the complex plane and apply a [conformal mapping](http://en.wikipedia.org/wiki/Conformal_map), the transformed function is still a solution of the Laplace equation.

This means that we can map the potential function and the streamlines of flow over a cylinder on the $z$-plane to the $\xi$-plane, and obtain the streamlines of flow over an airfoil. The stream function around the airfoil will be given by:

$$
\begin{equation}
\psi(\xi_x, \xi_y) = \psi(\xi_x(x, y), \xi_y(x, y))
\end{equation}
$$

where the complex coordinates of $\xi$, $\xi_x$ and $\xi_y$, are obtained from the Joukowski transformation of $z=x+iy$.

In this exercise, you will obtain the flow over a symmetric Joukowski airfoil with zero and non-zero angle of attack. We obtain the shape of our target airfoil by placing a cylinder centered at $(x_c, y_c)=(-0.15, 0)$ with radius $R = 1.15$ and parameter $c=1$. You'll reach the goal step-by-step with problems (3) to (6).

First, build a set of grid points on the $z$-plane and see how these points look like on the $\xi$-plane. Use polar coordinates to build your grid on the $z$-plane. If you place grid points inside the cylinder, they will end up outside the airfoil after a Joukowski transformation (try it for yourself!). This is a problem. The streamlines inside the cylinder are not physical anyway, so we'll just ignore this region.

### Exercise

1. Build a grid in polar coordinates in the $z$-plane. Use $N_r = 100$ points in the radial direction, within the range $R \le r \le 5$, and $N_\theta = 145$ points in the axial direction, within the range $0 \le \theta \le 2\pi$.
2. Use the Joukowski transformation to map the grid into the $\xi$-plane.
3. Plot the grid points in both planes using the function `pyplot.scatter()`.

You should obtain figures that look like this:

---

## 4. Flow over a symmetric Joukowski airfoil with zero angle of attack

### Stream function and streamlines

Now, you will evaluate potential flow over the cylinder on the $z$-plane. As mentioned above, $\psi(\xi) = \psi(\xi(z))$. This means that, after evaluating the stream function at a certain point on the $z$-plane, the corresponding point on the $\xi$-plane has the same value of the stream function. We can plot the streamlines on both planes using the function `pyplot.contour()`, because the stream function is a scalar function.

Use $1$ as the free stream velocity, i.e., $U_{\infty}=1$. You have to calculate the strength of the doublet first in order to have a cylinder with radius $R=1.15$.

You should obtain streamline patterns similar to those shown in the following figures.

### Velocity vector and pressure coefficient

To get the pressure coefficients, we need to compute the velocity fields.
We can simply obtain the velocity field on the $z$-plane using the coordinates of the grid points on the $z$-plane.
But can we just say that the velocity at the corresponding points on the $\xi$-plane is the same as that on the $z$-plane, just like what we did for the stream function?
_The answer is no._

The values of the stream function remain the same on the original and mapped points because the stream function is a scalar solution of Laplace's equation!

However, the velocity is a vector and is not a solution of Laplace's equation.
When the coordinate system changes by the conformal map, the values of the vector in the new system are different.

Now, let's go back to our problem: the $z$-plane and the $\xi$-plane are two different coordinate systems.
The velocity at a specific point on the $z$-plane is not the same as that at the corresponding point on the $\xi$-plane.
Some manipulation must be done.

We recall that the velocity potential, $\phi$, is also a solution of Laplace's equation; therefore, the potential values remain the same at a point under the conformal transformation.
The same applies to the complex potential, $F(z) = F(\xi(z)) = \phi + i \psi$.

The complex velocity in the $z$-plane is defined by:

$$
\begin{equation}
W_z = \frac{dF}{dz} = u_z - i v_z
\end{equation}
$$

where $u_z = \frac{\partial \psi}{\partial y}$ and $v_z = -\frac{\partial \psi}{\partial x}$.

We can obtain the complex velocity in the $\xi$-plane, $W_\xi = u_\xi - i v_\xi$, by applying a chain rule:

$$
\begin{equation}
W_\xi = \frac{dF}{d\xi} = \frac{dF}{dz} \times \frac{dz}{d\xi} = \frac{dF}{dz} / \frac{d\xi}{dz} = W_z / \frac{d\xi}{dz}
\end{equation}
$$

where

$$
\begin{equation}
\frac{d\xi}{dz} = \frac{d\left(z + \frac{c^2}{z}\right)}{dz} = 1 - \left(\frac{c}{z}\right)^2
\end{equation}
$$

Once we have the velocity fields, we can get the pressure coefficient on each plane.

The velocity field on each plane should look like this (using the function `pyplot.quiver()`):

The pressure coefficient on each plane should look like this (using the function `pyplot.contourf()`):

### Exercise

* Write the Python code to obtain the figures shown above (streamlines, velocity field, and pressure coefficient in both planes).
* Answer the following questions:
    1. What is the strength of the doublet?
    2. What is the velocity at the $62^\text{nd}$ point on the airfoil surface? (We assume that the trailing edge is the $1^\text{st}$ point and that we index the points counterclockwise.)
    3. What is the minimum value of the pressure coefficient on the airfoil surface?

Don't forget to enter your answers on [Open edX](https://openedx.seas.gwu.edu/courses/course-v1:MAE+MAE6226+2017/about).
(Notes: you will need to enter your results with a 2-digit precision after the decimal point in the auto-graded section.)

---

## 5. Flow over a symmetric Joukowski airfoil with non-zero angle of attack, without circulation

Now, we want to place the airfoil at an angle of attack (AoA) with respect to the free stream.
Of course we can use flow over a cylinder and the Joukowski transformation to do this.
*But how?*
Can we superpose a free stream with non-zero inlet angle and a doublet to obtain what we want?
Actually, we can't.
If we do so, we won't be able to obtain a closed streamline, like we did in the previous case (in which a closed circular streamline can be taken as a cylinder surface).

A way to achieve a uniform flow with an inlet angle is by simply rotating the complex plane $z$ to get a new complex plane $z'$, where the $x'$-axis (i.e., real part of $z'$) is parallel to the free-stream direction, with its origin located at the center of the cylinder $\left(x_c, y_c\right)$, as shown in the figure below.

The relationship between the $z'$-plane and the $z$-plane is:

$$
\begin{equation}
z'=\left[ z-(x_c+iy_c) \right]e^{-i\times AoA}
\end{equation}
$$

i.e.,

$$
\begin{equation}
\left\{
\begin{array}{lcr}
x' & = & (x-x_c)\cos(AoA) + (y-y_c)\sin(AoA) \\
y' & = & - (x-x_c)\sin(AoA) + (y-y_c)\cos(AoA)
\end{array}
\right.
\end{equation}
$$

Now, we can obtain the flow over a cylinder on the new plane $z'$ by adding a free stream with **zero inlet angle** to a doublet **centered at the origin**.
Then, we can obtain the flow on the $z$-plane and, finally, on the $\xi$-plane.
Again, the stream function remains the same at the same point under the three coordinate systems ($z'$, $z$, and $\xi$).

You should get streamlines in the $z$-plane and in the $\xi$-plane that look like this:

The velocity vector need to be rotated back from the $z'$-plane to the $z$-plane.

$$
\begin{equation}
W_z = \frac{dF}{dz} = \frac{dF}{dz'} \times \frac{dz'}{dz} = W_{z'} e^{-i \times \text{AoA}}
\end{equation}
$$

Once you have the velocity in the $z$-plane, you can use what you learned in the previous exercise to get the velocity in the $\xi$-plane.

You should get the velocity and pressure coefficient (in the $z$ and $\xi$ planes) that look like this:

### Exercise

* Write the Python code to obtain the figures shown above (streamlines, velocity field, and pressure coefficient in the $z$ and $\xi$ planes). Set the angle of attack to $\text{AoA} = 20^\circ$.
* Answer the following questions:
    1. Do you think the flow over the airfoil looks reasonable? Explain why.
    2. What are the index of the stagnation points on the airfoil? (We assume that the trailing edge is the $1^\text{st}$ point and that we index the points counterclockwise.)
    3. How much is the lift? The lift, $L=-\oint p \times \sin{\theta} dA$, is perpendicular to the free-stream direction.
    4. How much is the drag? The drag, $D=-\oint p \times \cos{\theta} dA$, is in the direction of the free-stream.
    5. What is the velocity at the $50^\text{th}$ point on the airfoil?
    6. What is the pressure coefficient at the $75^\text{th}$ point on the airfoil?

Don't forget to enter your answers on [Open edX](https://openedx.seas.gwu.edu/courses/course-v1:MAE+MAE6226+2017/about).
(Notes: you will need to enter your results with a 2-digit precision after the decimal point in the auto-graded section.)

---

## 6. Flow over a symmetric Joukowski airfoil with non-zero angle of attack, with circulation

The flow obtained in the last exercise is not physical.
What we need is a **vortex**.
In [Lesson 6](06_Lesson06_vortexLift.ipynb), we saw that, by adding a vortex (i.e., circulation) to a potential flow over a cylinder, we can modify the position of the stagnation points and generate lift.

To make the flow more physical, we need to satisfy the [Kutta condition](http://en.wikipedia.org/wiki/Kutta_condition),
>"A body with a sharp trailing edge which is moving through a fluid will create about itself a circulation of sufficient strength to hold the rear stagnation point at the trailing edge."

This condition helps us in choosing the strength of the vortex to add.
The strength of the vortex should be calculated so that the rear-stagnation point on the cylinder moves from $\theta=\text{AoA}$ to $\theta=0^\circ$ in the $z$-plane.
You should be able to calculate the strength with what you learned in [Lesson 6](06_Lesson06_vortexLift.ipynb).

The streamlines, velocity, and pressure coefficient in the $z$-plane and in the $\xi$-plane should look like this:

### Exercise

* Write the Python code to obtain the figures shown above (streamlines, velocity field, and pressure coefficient in the $z$ and $\xi$ planes). Set the angle of attack to $\text{AoA} = 20^\circ$.
* Answer the following questions:
    1. What is the strength of the vortex?
    2. How much is the lift and does it agree with the Kutta-Joukowski theorem? The lift, $L=-\oint p \times \sin{\theta} dA$, is perpendicular to the free-stream direction.
    3. How much is the drag? The drag, $D=-\oint p \times \cos{\theta} dA$, is in the direction of the free-stream.
    4. What are the index of the stagnation points on the airfoil? (We assume that the trailing edge is the $1^\text{st}$ point and that we index the points counterclockwise.)
    5. What is the velocity at the $92^\text{nd}$ point on the airfoil?
    6. What is the pressure coefficient at the $111^\text{th}$ point on the airfoil?
    7. Explain what is going on with the pressure coefficient at the trailing edge of the airfoil.

Don't forget to enter your answers on [Open edX](https://openedx.seas.gwu.edu/courses/course-v1:MAE+MAE6226+2017/about).
(Notes: you will need to enter your results with a 2-digit precision after the decimal point in the auto-graded section.)

---

```python
from IPython.core.display import HTML
def css_styling(filepath):
    styles = open(filepath, 'r').read()
    return HTML(styles)
css_styling('../styles/custom.css')
```

<link href='http://fonts.googleapis.com/css?family=Fenix' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Alegreya+Sans:100,300,400,500,700,800,900,100italic,300italic,400italic,500italic,700italic,800italic,900italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Source+Code+Pro:300,400' rel='stylesheet' type='text/css'>


# Model Project
***

_In this model project we will present a simple Robinson Crusoe production economy. We will solve the model analytically using sympy, evaluate the markets in different parameterizations of price and wage and visualize one solution_

## The theoretical model:
Imagine that Crusoe is schizophenic and makes his decisions as a manager and consumer separately. His decsisions are however, guided by market prices - labor wage and consumption price. It is assumed that Crusoe is endowed with a total time endowment of 60 hours per week.

 **Producer problem:** When Crusoe acts as a manager he seeks to maximize his profit subject to the production function while taking the price and wage as given.

\\[ \max_{x,l} px-wl \\]

subject to

\\[ x=f(l)=Al^\beta \\]

 Where p is the market price of the good, w is the wage, x is the good and l is labor. A and $\beta$ reflect technology and returns to scale, respectively.

 **Consumer problem:** When acting as a consumer, Crusoe maximizes his utility of the consumption good x and leisure (the latter is defined as whats left of total time endowment when working l hours)

As the consumer and owner of the firm, Crusoe will receive profit from "selling"  his labor in the producer problem.

\\[ \max_{x,l} u(x,(L-l))=x^\alpha(L-l)^{(1-\alpha)} \\]

subject to

\\[ px=wl+\pi(w,p) \\]

 When solving the model, we need to derive demand and supply expressions for labor and the consumption good. When equalizing supply and demand in one of the markets and acquiring an equilibrium, it follows from Walras' law that the other market will also reach an equilibrium.

Hence, the model is solved by first optimizing in the markets separately, deriving the supply and demand expressions, then equalizing supply and demand across markets and solving for the consumption price and labor wage.

## The analytical solution:

Install packages.

```python
import numpy as np
import sympy as sm

%matplotlib inline
import matplotlib.pyplot as plt
```

In the follwoing, we will primarily analyze the model using sympy. However, due to computational problems when solving the model using sympy with algebraic expressions only (see https://docs.sympy.org/0.7.6/tutorial/solvers.html), we define some of the parameter values as the following:

```python
A = 13.15
beta = 0.5
alpha = 2/3
L = 60
I = 10
```

### Producer problem:
The producer problem is quite simple and the easiest way to solve the maximization problem is to substitute the constraint (production function) into the profit function, deriving the reduced form.

```python
# Profit function
def prof(x,l):
    return p*x-w*l

# Production function
def prod(l):
    return A*l**beta

# Reduced form - subsituting production function into profit function
def reduced(l):
    return prof(prod(l),l)
```

Substituting for x and maximizing w.r.t labor (l):

```python
# Optimization using sympy diff:
focProd = sm.diff(reduced(l),l)

# Isolating labor and thus, deriving labor demand:
laborDemand = sm.solve(focProd,l)

# Finding the supply of goods:
profSubs = prof(x,laborDemand[0])
goodSupply = sm.solve(profSubs,x)

# Printing labor demand and goods supply
print("Labor demand: lD=", laborDemand)
print("supply of goods: xS=", goodSupply)
```

    Labor demand: lD= [43.230625*p**2/w**2]
    supply of goods: xS= [43.230625*p/w]

Profit, as a function of w and p, is derived by inserting labor demand and goods supply into the profit function:

```python
Profit = (1-beta)*(A*p)**(1/(1-beta))*(beta/w)**(beta/(1-beta))
print(Profit)
```

    43.230625*p**2.0*(1/w)**1.0

### Consumer problem:
The consumer problem is not as simple as the producer problem and we solve this using sympy's version of lagrange.

```python
# The variables to maximize w.r.t
x, l = sm.var('x,l',real=True)

# The objective function
util = x**alpha*(L-l)**(1-alpha)

# Budget constraint
bud = p*x-w*l-Profit

# Specifying the shadow price
lam = sm.symbols('lambda',real = True)

# Setting up the Lagragian
L = util-lam*bud

```

```python
# Differentiating w.r.t x and l
gradL = [sm.diff(L,c) for c in [x,l]]

# The focs and the shadow price
KKT_eqs = gradL + [bud]
KKT_eqs
```

```python
# Showing the stationary points solving the constrained problem
stationary_points = sm.solve(KKT_eqs,[x,l,lam],dict=True)
stationary_points
```

We note that all proposed solutions of l and x are the same. The only variation between the three proposed solutions is the shadow price. Hence, we proceed with the solutions of l and x and derive the optimal wage. This is done by equalizing supply and demand of labor, hence, obtaining equilibrium in one market.

```python
equalLab = sm.Eq(-14.4102083333333*p**2/w**2+40,43.230625*p**2/w**2)
opt_wag = sm.solve(equalLab,w)

print("Optimal wage depending on price", opt_wag)
```

    Optimal wage depending on price [-1.20042527186549*p, 1.20042527186549*p]

Since the price cannot be negative the solution is 1.2*p. This means that any set of prices will imply an approximately 1.2 times higher wage with equilibrium in both markets. In the following, we evaluate the optimal wage expression and labor demand in different values of p and hence, w.

```python
# We convert the symbolic optimal wage expression a function depending of p
_opt_wag = sm.lambdify(p,opt_wag[1])

# We evaluate the wage in a price of 10 and 1, respectively.
p1_vec = np.array([10,1])
wages = _opt_wag(p1_vec)
print("Optimal wage, when price is 10 and 1, respectively: ",wages)

# We evaluate the labor demand in the prices and wages. First making the labor demand
# expression a function depending on p and w.
_lab_dem = sm.lambdify((p,w),laborDemand)
p1_vec = np.array([10,1])

# Labor demand evaluated in price and wages.
labor_dem1 = [_lab_dem(p1_vec[0],wages[0]), _lab_dem(p1_vec[1],wages[1])]
print("Labor demand evaluated in combination of wages and prices: ", labor_dem1)

# Labor supply from lagrange optimization problem
_lab_sup = [-14.4102083333333*p1_vec[0]**2/wages[0]**2+40,-14.4102083333333*p1_vec[1]**2/wages[1]**2+40]
print("Labor supply evaluated in combination of wages and prices: ", _lab_sup)

# Profit in different combination of wages and prices
Profit_eval = 43.230625*p1_vec**2.0*(1/wages)**1.0
print("Profit evaluated in combination of prices and wages:", [Profit_eval[0], Profit_eval[1]])

# Demand of consumption good
good_opt = prod(30)
print("Demand and supply of consumption good: ", good_opt)

# Utility in different combination of wages and prices
_utility = sm.lambdify((x,l),util)
print("Utility evaluated in combination of prices and wages:", _utility(good_opt,30))
```

    Optimal wage, when price is 10 and 1, respectively:  [12.00425272  1.20042527]
    Labor demand evaluated in combination of wages and prices:  [[29.999999999999954], [29.999999999999957]]
    Labor supply evaluated in combination of wages and prices:  [30.00000000000004, 30.000000000000036]
    Profit evaluated in combination of prices and wages: [360.12758155964644, 36.01275815596465]
    Demand and supply of consumption good:  72.02551631192935
    Utility evaluated in combination of prices and wages: 53.78956164406131

## Visualization:

We now visualize the solution when the price is 1 and the wage is 1.2.

```python
def prof_ny(x,l,profit):
    return 1*x-1.2*l+profit

def util_ny(l, profit):
    return (profit/((L-l)**(1-alpha)))**(1/alpha)

def budget_ny(x,l,profit):
    return x-1.2*l+profit

fig = plt.figure(figsize=(8,4),dpi=100)

labor_vec = np.linspace(0,59,500)
goods_vec = np.linspace(0,140,500)

ax_left = fig.add_subplot(1,2,1)
ax_left.plot(labor_vec, prod(labor_vec))
ax_left.plot(labor_vec, prof_ny(goods_vec,labor_vec,0))
ax_left.plot(labor_vec, prof_ny(goods_vec,labor_vec,Profit_eval[1]))
ax_left.set_title('Price-taking producer')
ax_left.set_xlabel('l')
ax_left.set_ylabel('x')

ax_right = fig.add_subplot(1,2,2)
ax_right.plot(labor_vec, util_ny(labor_vec,54))
ax_right.plot(labor_vec, budget_ny(goods_vec,labor_vec,Profit_eval[1]))
ax_right.set_title('Price-taking consumer')
ax_right.set_xlabel('l')
ax_right.set_ylabel('x')

```

The found solution of p=1 and w=1,2 is represented by the tangency of the isoprofit curve and the production plan in the left panel at the tangency of the budget constraint and indifference curve at the right panel.