```python
import numpy as np, pandas as pd, matplotlib as mpl, matplotlib.pyplot as plt#, seaborn as sns
%matplotlib inline
mpl.style.use('classic')
```

Symbolic math

Definitions

$Y=[α\cdot F^{-\rho}+(1-α)\cdot R^{-\rho} ]^{-1/\rho}$ Arrow 1961

MIT: $Y=[α\cdot F^{\rho}+(1-α)\cdot R^{\rho} ]^{1/\rho}$   http://www.sciencedirect.com/science/article/pii/S0140988306000624?np=y Eq. 1

$\sigma = \frac{1}{1+\rho}$

MIT: $\sigma = \frac{1}{1-\rho}$

```python
from sympy import *
init_printing()
```

```python
a = Symbol('alpha')
s = Symbol('sigma')
r = Symbol('rho')
var('Y F R C c_f c_r');
```

```python
eq=Eq(Y,(a*F**-r+(1-a)*R**-r)**(-1/r))
eq  #main CES equation
    #Y = value of function - analogous to energy demand?
    #R = renewables
    #F = fossils
    #alfa = fossil share
    #rho = factor related to elasticity of substituion
```

```python
R_sol=solve(eq,R)[0]
R_sol #expression of R from the main CES equation
```

```python
eq2=Eq(s,1/(1+r))
eq2 #definition of elasticity of substituion
    #rho = elasticity of substitution
```

```python
r_sol=solve(eq2,r)[0]
r_sol #exression of rho in terms of sigma
```

```python
[RR_sol,rr_sol]=solve([eq,eq2],[R,r])[0]
RR_sol #expression of R in terms of sigma
```

```python
Y_sol=solve(eq,Y)[0]
Y_sol #expression of Y in terms of sigma = main expression
```

```python
YR_sol=solve([eq,eq2],[Y,r])[0][0]
YR_sol #expression of Y in terms of sigma
```

Sample policy calculation replicated from page 60 of http://globalchange.mit.edu/files/document/MITJPSPGC_Rpt125.pdf (EPPA).

```python
s_fix=0.4 #fixed elasticity for demand, EPPA pag. 24
s_fix2=0.5 #fixed elasticity for production, EPPA pag. 17
```

```python
step=1 #step size for numerical calculation
years=np.arange(2015,2100+step,step)
years #years to complete transition (fossil phase-out)
```

    array([2015, 2016, 2017, 2018, 2019, 2020, 2021, 2022, 2023, 2024, 2025,
           2026, 2027, 2028, 2029, 2030, 2031, 2032, 2033, 2034, 2035, 2036,
           2037, 2038, 2039, 2040, 2041, 2042, 2043, 2044, 2045, 2046, 2047,
           2048, 2049, 2050, 2051, 2052, 2053, 2054, 2055, 2056, 2057, 2058,
           2059, 2060, 2061, 2062, 2063, 2064, 2065, 2066, 2067, 2068, 2069,
           2070, 2071, 2072, 2073, 2074, 2075, 2076, 2077, 2078, 2079, 2080,
           2081, 2082, 2083, 2084, 2085, 2086, 2087, 2088, 2089, 2090, 2091,
           2092, 2093, 2094, 2095, 2096, 2097, 2098, 2099, 2100])

```python
gamma=0.00 #demand increase rate PER YEAR
Y_zero=100 #initial demand
```

```python
init_F=0.95 #initial fossil share, estimation from EPPA pag. 62, fig. 20
F_fix=np.array([init_F*Y_zero*(1.0-1.0/(len(years)-1)*i) for i in range(len(years))])
F_fix #array of predefined fossil values (in order complete the transition as defined by years)
```

    array([ 95.        ,  93.88235294,  92.76470588,  91.64705882,
            90.52941176,  89.41176471,  88.29411765,  87.17647059,
            86.05882353,  84.94117647,  83.82352941,  82.70588235,
            81.58823529,  80.47058824,  79.35294118,  78.23529412,
            77.11764706,  76.        ,  74.88235294,  73.76470588,
            72.64705882,  71.52941176,  70.41176471,  69.29411765,
            68.17647059,  67.05882353,  65.94117647,  64.82352941,
            63.70588235,  62.58823529,  61.47058824,  60.35294118,
            59.23529412,  58.11764706,  57.        ,  55.88235294,
            54.76470588,  53.64705882,  52.52941176,  51.41176471,
            50.29411765,  49.17647059,  48.05882353,  46.94117647,
            45.82352941,  44.70588235,  43.58823529,  42.47058824,
            41.35294118,  40.23529412,  39.11764706,  38.        ,
            36.88235294,  35.76470588,  34.64705882,  33.52941176,
            32.41176471,  31.29411765,  30.17647059,  29.05882353,
            27.94117647,  26.82352941,  25.70588235,  24.58823529,
            23.47058824,  22.35294118,  21.23529412,  20.11764706,
            19.        ,  17.88235294,  16.76470588,  15.64705882,
            14.52941176,  13.41176471,  12.29411765,  11.17647059,
            10.05882353,   8.94117647,   7.82352941,   6.70588235,
             5.58823529,   4.47058824,   3.35294118,   2.23529412,
             1.11764706,   0.        ])

```python
#calculate Y_fix_zero through resubstituting initial values into CES
Y_fix_zero=YR_sol.subs([[R,Y_zero*(1-init_F)],[F,Y_zero*init_F],[a,init_F],[s,s_fix2]])
Y_fix=np.array([Y_fix_zero*((1+gamma*step)**i) for i in range(len(years))])
Y_fix #array of predefined demand values
```

    array([50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000], dtype=object)

```python
Y_sol, YR_sol, eq2, R_sol, RR_sol
```

```python
R_fix=[RR_sol.subs([[Y,Y_fix[k]],[F,F_fix[k]],[s,s_fix2],[a,init_F]]) for k in range(len(F_fix))]
plt.scatter(R_fix,F_fix)
plt.xlim(0,200);
```

Carbon tax

```python
eq3=Eq(c_r,c_f/((a/(1-a))*(F/R)**(-1-r)))
eq3 #cost of R compared to F
    #c_f fixed cost of F
```

```python
c_r_sol=solve(eq3,c_r)[0]
c_r_sol #cost of R expressed in terms of rho
```

```python
cR_r_sol=solve([eq3,eq2],[c_r,r])[0][0]
cR_r_sol #cost of R expressed in terms of sigma
```

```python
eq4=Eq(C,c_r*((a/(1-a))*(F/R)**(-1-r))-c_f)
eq4 #carbon tax: relative cost of F compared to R - fixed cost of F
```

```python
C_sol=solve(eq4,C)[0]
C_sol #expression of C in terms of sigma, F, R, a, c_f
```

```python
CR_sol=solve([eq4,eq2],[C,r])[0][0]
CR_sol #expression of C in terms of rho, F, R, a, c_f
```

```python
c_f_fix=1 #cost of fossil, taken 1 to normalize
zeta=0.00 #R cost decrease rate PER YEAR
k=0
c_r_fix_zero=cR_r_sol.subs([[R,R_fix[k]],[F,F_fix[k]],[s,s_fix2],[a,init_F],[c_f,c_f_fix]])
c_r_fix=np.array([c_r_fix_zero*((1-zeta*step)**i) for i in range(len(years))])
c_r_fix #array of predefined R cost values, based on cost decrease rate zeta
```

    array([19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000], dtype=object)

```python
C_sol
```

```python
C_fix=[CR_sol.subs([[R,R_fix[k]],[F,F_fix[k]],[s,s_fix2],[a,init_F],[c_f,c_f_fix],[c_r,c_r_fix[k]]])\
       for k in range(len(F_fix))]
plt.scatter(years[:-1],C_fix[:-1],color='r')
plt.ylim(0,1000);
```

```python
def ces(init_F=0.75,s_fix=0.5,zeta=0.0,gamma=0.0,step=1):
    years=np.arange(2015,2100+step,step)
    Y_zero=100 #initial demand
    F_fix=np.array([init_F*Y_zero*(1.0-1.0/(len(years)-1)*i) for i in range(len(years))])
    Y_fix_zero=YR_sol.subs([[R,Y_zero*(1-init_F)],[F,Y_zero*init_F],[a,init_F],[s,s_fix]])
    Y_fix=np.array([Y_fix_zero*((1+gamma*step)**i) for i in range(len(years))])
    R_fix=[RR_sol.subs([[Y,Y_fix[k]],[F,F_fix[k]],[s,s_fix],[a,init_F]]) for k in range(len(F_fix))]

    R_fix=[i for i in R_fix if np.complex(i).imag==0]
    R_fix=[i for i in R_fix if i>0]

    k=0
    c_f_fix=1 #cost of fossil, taken 1 to normalize
    c_r_fix_zero=cR_r_sol.subs([[R,R_fix[k]],[F,F_fix[k]],[s,s_fix],[a,init_F],[c_f,c_f_fix]])
    c_r_fix=np.array([c_r_fix_zero*((1-zeta*step)**i) for i in range(len(years))])
    C_fix=[CR_sol.subs([[R,R_fix[k]],[F,F_fix[k]],[s,s_fix],[a,init_F],[c_f,c_f_fix],[c_r,c_r_fix[k]]])\
       for k in range(len(R_fix))]

    return [R_fix[:-1],F_fix[:len(R_fix)-1],C_fix[:len(R_fix)-1]]
```

```python
fig = plt.subplots(1,3,figsize=(14,4))
lw=2

gs = mpl.gridspec.GridSpec(1, 4,width_ratios=[1.5,0.01,1,1])
bg=['#EEEEEE','none','#EEEEEE','#EEEEEE']
ax = [plt.subplot(gs[i],axisbg=bg[i],axisbelow=True) for i in range(4)]
plt.subplots_adjust(wspace = .1)
ax[1].spines['top'].set_color('none')
ax[1].spines['bottom'].set_color('none')
ax[1].spines['left'].set_color('none')
ax[1].spines['right'].set_color('none')

init_F0= [0.85,0.9,0.95]
s_fix0 = [0.5, 1.5, 2.5]
init_F= [init_F0[i] for i in range(len(init_F0)) for j in range(len(s_fix0))]
s_fix = [s_fix0[j] for i in range(len(init_F0)) for j in range(len(s_fix0))]
ls0=["--","-",":"]
ss0=["","",""]
ls = [ls0[j] for i in range(len(init_F0)) for j in range(len(s_fix0))]
ss = [ss0[j] for i in range(len(init_F0)) for j in range(len(s_fix0))]
colors0=["#dd1c77","#2c7fb8","#2ca25f"]
colors=[colors0[i] for i in range(len(init_F0)) for j in range(len(s_fix0))]
ax2=ax[2].twinx()
ax2.set_axisbelow(True)
ax[1].xaxis.set_visible(False)
ax[1].yaxis.set_visible(False)
ax[2].set_yticks([])
ax3=ax[3].twinx()
ax3.set_axisbelow(True)
ax[3].set_yticks([])
ax[0].set_xlim(0,200)
ax[2].set_xlim(2015,2100)
ax[2].set_xticks([2025,2050,2075,2100])
ax[3].set_xlim(2015,2100)
ax[3].set_xticks([2025,2050,2075,2100])
ax[0].set_ylim(0,100)
ax2.set_ylim(0,1000)
ax3.set_ylim(0,1000)
ax[0].grid(color='white', linestyle='solid')
ax[2].grid(color='white', linestyle='solid')
ax[3].grid(color='white', linestyle='solid')
ax2.grid(color='white', linestyle='solid')
ax3.grid(color='white', linestyle='solid')

for i in range(len(init_F)):

    [x,y,c]=ces(init_F[i],s_fix[i])
    [x2,y2,c2]=ces(init_F[i],s_fix[i],0.01)
    ax[0].plot(x,y,linewidth=lw,linestyle=ls[i],color=colors[i],marker=ss[i])

    #for legend
    if i in range(len(s_fix0)):
        ax[0].plot([-10],[-10],'#444444',label='$\\sigma='+str(s_fix[i])+'$',linestyle=ls[i],marker=ss[i],linewidth=lw)
        ax3.plot([-10],[-10],'#2ca25f',label='$\\sigma='+str(s_fix[i])+'$',linestyle=ls[i],marker=ss[i],linewidth=lw)
    if i in {2,5,7}:
        ax[0].plot([-10],[-10],label='$\\alpha='+str(init_F[i])+'$',color=colors[i],linewidth=lw)
        ax2.plot([-10],[-10],label='$\\alpha='+str(init_F[i])+'$',color=colors[i],linewidth=lw)
    if i in {1,4,7}:
        ax2.fill_between(np.array(years[:len(c)]).astype(float),np.array(c).astype(float),np.array(c2).astype(float),\
                         color=colors[i],alpha=0.5)#,zorder=10)
        ax2.plot(years[:len(c)],c2,linewidth=lw,linestyle=ls[i],color=colors[i],marker=ss[i])
    if i in {1,4,7}:
        ax2.plot(years[:len(c)],c,linewidth=lw,linestyle=ls[i],color=colors[i],marker=ss[i])
    if i in {6,7,8}:
        ax3.plot(years[:len(c)],c,linewidth=lw,linestyle=ls[i],color=colors[i],marker=ss[i],zorder=10)
    if i in {7}:
        ax2.annotate('$\\zeta = 0\%$', xy=(years[48], c[48]),  xycoords='data',
                xytext=(-50, 30), textcoords='offset points',
                arrowprops=dict(arrowstyle="->"))#,zorder=11)
        ax2.annotate('$\\zeta = 1\%$', xy=(years[43], c2[43]),  xycoords='data',
                xytext=(-50, 30), textcoords='offset points',
                arrowprops=dict(arrowstyle="->"))#,zorder=11)

ax[0].legend(loc=3,framealpha=0.5,fontsize=11)
ax2.legend(loc=2,framealpha=0.5,fontsize=11)
ax3.legend(loc=2,framealpha=0.5,fontsize=11)
ax[0].set_xlabel(u'Renewable energy $R$')
ax[2].set_xlabel(u'Year')
ax[3].set_xlabel(u'Year')
ax2.set_yticklabels(["","","","","",""])
ax[0].set_ylabel(u'Fossil energy $F$')
ax3.set_ylabel(u'Carbon tax $C$')
ax[0].set_title('$F-R$ curve\n $Y=[\\alpha\cdot \
                F^{-\\rho}+(1-\\alpha)\cdot R^{-\\alpha} ]^{-1/\\rho}, \\sigma = 1/(1+\\rho)$',size=12,y=1.02)
ax[2].set_title('Carbon tax $C$\n $C=c_R \cdot [\\alpha/(1-\\alpha)] \
                \cdot (F/R)^{-\\rho-1} - c_F,$  $\\zeta:$ $yearly$ $decrease$ $in$ $c_R$',size=12,y=1.02,x=1.1)

plt.suptitle(u'Constant Elasticity (CES) Energy Substitution Model at Constant Energy Demand $Y$',fontsize=13,y=1.12)
plt.savefig('ces2.png',bbox_inches = 'tight', pad_inches = 0.1, dpi=150)
plt.show()
```

```python
fig = plt.subplots(1,3,figsize=(14,4))
lw=2

gs = mpl.gridspec.GridSpec(1, 4,width_ratios=[1.5,0.01,1,1])
bg=['#EEEEEE','none','#EEEEEE','#EEEEEE']
ax = [plt.subplot(gs[i],axisbg=bg[i],axisbelow=True) for i in range(4)]
plt.subplots_adjust(wspace = .1)
ax[1].spines['top'].set_color('none')
ax[1].spines['bottom'].set_color('none')
ax[1].spines['left'].set_color('none')
ax[1].spines['right'].set_color('none')

init_F0= [0.85,0.9,0.95]
s_fix0 = [0.5, 1.5, 2.5]
init_F= [init_F0[i] for i in range(len(init_F0)) for j in range(len(s_fix0))]
s_fix = [s_fix0[j] for i in range(len(init_F0)) for j in range(len(s_fix0))]
ls0=["--","-",":"]
ss0=["","",""]
ls = [ls0[j] for i in range(len(init_F0)) for j in range(len(s_fix0))]
ss = [ss0[j] for i in range(len(init_F0)) for j in range(len(s_fix0))]
colors0=["#dd1c77","#2c7fb8","#2ca25f"]
colors=[colors0[i] for i in range(len(init_F0)) for j in range(len(s_fix0))]
ax2=ax[2].twinx()
ax2.set_axisbelow(True)
ax[1].xaxis.set_visible(False)
ax[1].yaxis.set_visible(False)
ax[2].set_yticks([])
ax3=ax[3].twinx()
ax3.set_axisbelow(True)
ax[3].set_yticks([])
ax[0].set_xlim(0,200)
ax[2].set_xlim(2015,2100)
ax[2].set_xticks([2025,2050,2075,2100])
ax[3].set_xlim(2015,2100)
ax[3].set_xticks([2025,2050,2075,2100])
ax[0].set_ylim(0,100)
ax2.set_ylim(0,1000)
ax3.set_ylim(0,1000)

for i in range(len(init_F)):

    [x,y,c]=ces(init_F[i],s_fix[i],0.0,0.03)
    [x2,y2,c2]=ces(init_F[i],s_fix[i],0.01,0.03)
    ax[0].plot(x,y,linewidth=lw,linestyle=ls[i],color=colors[i],marker=ss[i])

    #for legend
    if i in range(len(s_fix0)):
        ax[0].plot([-10],[-10],'#444444',label='$\\sigma='+str(s_fix[i])+'$',linestyle=ls[i],marker=ss[i],linewidth=lw)
        ax3.plot([-10],[-10],'#2ca25f',label='$\\sigma='+str(s_fix[i])+'$',linestyle=ls[i],marker=ss[i],linewidth=lw)
    if i in {2,5,7}:
        ax[0].plot([-10],[-10],label='$\\alpha='+str(init_F[i])+'$',color=colors[i],linewidth=lw)
        ax2.plot([-10],[-10],label='$\\alpha='+str(init_F[i])+'$',color=colors[i],linewidth=lw)
    if i in {1,4,7}:
        ax2.fill_between(np.array(years[:len(c)]).astype(float),np.array(c).astype(float),np.array(c2).astype(float),\
                         color=colors[i],alpha=0.5)#,zorder=10)
        ax2.plot(years[:len(c)],c2,linewidth=lw,linestyle=ls[i],color=colors[i],marker=ss[i])
    if i in {1,4,7}:
        ax2.plot(years[:len(c)],c,linewidth=lw,linestyle=ls[i],color=colors[i],marker=ss[i])
    if i in {6,7,8}:
        ax3.plot(years[:len(c)],c,linewidth=lw,linestyle=ls[i],color=colors[i],marker=ss[i],zorder=10)
    if i in {4}:
        ax2.annotate('$\\zeta = 0\%$', xy=(years[40], c[40]),  xycoords='data',
                xytext=(-50, 30), textcoords='offset points',
                arrowprops=dict(arrowstyle="->"))#,zorder=11)
        ax2.annotate('$\\zeta = 1\%$', xy=(years[35], c2[35]),  xycoords='data',
                xytext=(-50, 30), textcoords='offset points',
                arrowprops=dict(arrowstyle="->"))#,zorder=11)

ax[0].grid(color='white', linestyle='solid')
ax[2].grid(color='white', linestyle='solid')
ax[3].grid(color='white', linestyle='solid')
ax2.grid(color='white', linestyle='solid')
ax3.grid(color='white', linestyle='solid')
ax[0].legend(loc=3,framealpha=0.5,fontsize=11)
ax2.legend(loc=2,framealpha=0.5,fontsize=11)
ax3.legend(loc=1,framealpha=0.5,fontsize=11)
ax[0].set_xlabel(u'Renewable energy $R$')
ax[2].set_xlabel(u'Year')
ax[3].set_xlabel(u'Year')
ax2.set_yticklabels(["","","","","",""])
ax[0].set_ylabel(u'Fossil energy $F$')
ax3.set_ylabel(u'Carbon tax $C$')
ax[0].set_title('$F-R$ curve\n $Y=[\\alpha\cdot \
                F^{-\\rho}+(1-\\alpha)\cdot R^{-\\alpha} ]^{-1/\\rho}, \\sigma = 1/(1+\\rho)$',size=12,y=1.02)
ax[2].set_title('Carbon tax $C$\n $C=c_R \cdot [\\alpha/(1-\\alpha)] \
                \cdot (F/R)^{-\\rho-1} - c_F,$  $\\zeta:$ $yearly$ $decrease$ $in$ $c_R$',size=12,y=1.02,x=1.1)

plt.suptitle(u'Constant Elasticity (CES) Energy Substitution Model at Yearly Energy Demand Increase Rate $\\gamma=0.03$',fontsize=13,y=1.12)
plt.savefig('ces3.png',bbox_inches = 'tight', pad_inches = 0.1, dpi=150)
plt.show()
```

```python
def sces(init_F=0.75,s_fix=0.5,rf=-1,zeta=0.0,gamma=0.0,step=1):
    #dynamically adjust sigma and resolve equation
    years=np.arange(2015,2100+step,step)
    Y_zero=100 #initial demand
    F_fix=np.array([init_F*Y_zero*(1.0-1.0/(len(years)-1)*i) for i in range(len(years))])
    Y_fix_zero=YR_sol.subs([[R,Y_zero*(1-init_F)],[F,Y_zero*init_F],[a,init_F],[s,s_fix]])
    Y_fix=np.array([Y_fix_zero*((1+gamma*step)**i) for i in range(len(years))])
    #init R_fix, C_fix
    R_fix=[0 for k in range(len(F_fix))]
    C_fix=[0 for k in range(len(F_fix))]

    r_fix=solve(eq2,r)[0].subs(s,s_fix)
    r_span=-(rf-r_fix)
    s_fix_k=[]
    r_fix_k=[]

    for k in range(len(F_fix)):
        t=1
        s_fix_k.append(solve(eq2,s)[0].subs(r,r_fix)) #append to sigma array
        r_fix_k.append(r_fix) #append to rho array
        #update sigma every t years
        if (k%t==0):
            r_fix=r_fix-(r_span)/len(years) #update rho

    R_fix=[RR_sol.subs([[Y,Y_fix[k]],[F,F_fix[k]],[s,s_fix_k[k]],[a,init_F]]) for k in range(len(F_fix))]

    R_fix=[i for i in R_fix if np.complex(i).imag==0]
    R_fix=[i for i in R_fix if i>0]

    k=0
    c_f_fix=1 #cost of fossil, taken 1 to normalize
    c_r_fix_zero=cR_r_sol.subs([[R,R_fix[k]],[F,F_fix[k]],[s,s_fix_k[k]],[a,init_F],[c_f,c_f_fix]])
    c_r_fix=np.array([c_r_fix_zero*((1-zeta*step)**i) for i in range(len(years))])
    C_fix=[CR_sol.subs([[R,R_fix[k]],[F,F_fix[k]],[s,s_fix_k[k]],[a,init_F],[c_f,c_f_fix],[c_r,c_r_fix[k]]])\
       for k in range(len(R_fix))]

    return [R_fix[:-1],F_fix[:len(R_fix)-1],C_fix[:len(R_fix)-1],s_fix_k[:len(R_fix)-1],r_fix_k[:len(R_fix)-1]]
```

```python
[x,y,c,ys,yr]=sces(0.95,1.5,-1)
plt.plot(years[:len(c)],c)
[x,y,c,ys,yr]=sces(0.95,1.5,-1,0.01)
plt.plot(years[:len(c)],c,'--')
[x,y,c,ys,yr]=sces(0.85,1.5,-1)
plt.plot(years[:len(c)],c)
[x,y,c,ys,yr]=sces(0.85,1.5,-1,0.01)
plt.plot(years[:len(c)],c,'--')
[x,y,c,ys,yr]=sces(0.90,1.5,-1)
plt.plot(years[:len(c)],c)
[x,y,c,ys,yr]=sces(0.90,1.5,-1,0.01)
plt.plot(years[:len(c)],c,'--')
```

```python
def dces(init_F=0.75,s_fix=0.5,sf=0.01,zeta=0.0,gamma=0.0,step=1):
    #dynamically adjust sigma and resolve equation
    years=np.arange(2015,2100+step,step)
    Y_zero=100 #initial demand
    F_fix=np.array([init_F*Y_zero*(1.0-1.0/(len(years)-1)*i) for i in range(len(years))])
    Y_fix_zero=YR_sol.subs([[R,Y_zero*(1-init_F)],[F,Y_zero*init_F],[a,init_F],[s,s_fix]])
    Y_fix=np.array([Y_fix_zero*((1+gamma*step)**i) for i in range(len(years))])
    #init R_fix, C_fix
    R_fix=[0 for k in range(len(F_fix))]
    C_fix=[0 for k in range(len(F_fix))]

    s_fix_k=[]
    r_fix_k=[]

    for k in range(len(F_fix)):
        t=1
        s_fix_k.append(s_fix) #append to sigma array
        r_fix_k.append(solve(eq2,r)[0].subs(s,s_fix)) #append to rho array
        #update sigma every t years
        if (k%t==0):
            s_fix*=1+sf #update rho

    R_fix=[RR_sol.subs([[Y,Y_fix[k]],[F,F_fix[k]],[s,s_fix_k[k]],[a,init_F]]) for k in range(len(F_fix))]

    R_fix=[i for i in R_fix if np.complex(i).imag==0]
    R_fix=[i for i in R_fix if i>0]

    k=0
    c_f_fix=1 #cost of fossil, taken 1 to normalize
    c_r_fix_zero=cR_r_sol.subs([[R,R_fix[k]],[F,F_fix[k]],[s,s_fix_k[k]],[a,init_F],[c_f,c_f_fix]])
    c_r_fix=np.array([c_r_fix_zero*((1-zeta*step)**i) for i in range(len(years))])
    C_fix=[CR_sol.subs([[R,R_fix[k]],[F,F_fix[k]],[s,s_fix_k[k]],[a,init_F],[c_f,c_f_fix],[c_r,c_r_fix[k]]])\
       for k in range(len(R_fix))]

    return [R_fix[:-1],F_fix[:len(R_fix)-1],C_fix[:len(R_fix)-1],s_fix_k[:len(R_fix)-1],r_fix_k[:len(R_fix)-1]]
```

```python
[x,y,c,ys,yr]=dces(0.95,1.5,0.1)
plt.plot(years[:len(c)],c)
[x,y,c,ys,yr]=dces(0.95,1.5,0.1,0.01)
plt.plot(years[:len(c)],c,'--')
[x,y,c,ys,yr]=dces(0.85,1.5,0.1)
plt.plot(years[:len(c)],c)
[x,y,c,ys,yr]=dces(0.85,1.5,0.1,0.01)
plt.plot(years[:len(c)],c,'--')
[x,y,c,ys,yr]=dces(0.90,1.5,0.1)
plt.plot(years[:len(c)],c)
[x,y,c,ys,yr]=dces(0.90,1.5,0.1,0.01)
plt.plot(years[:len(c)],c,'--')
```

```python
fig = plt.subplots(2,2,figsize=(10,5))
lw=2

gs = mpl.gridspec.GridSpec(2, 2,height_ratios=[1,0.3])
bg=['#EEEEEE','#EEEEEE','#EEEEEE','#EEEEEE']
ax = [plt.subplot(gs[i],axisbg=bg[i],axisbelow=True) for i in range(4)]
plt.subplots_adjust(hspace = .001)

init_F= [0.85,0.9,0.95]
s_fix=1.5
rf=-1
sf=0.1

colors=["#dd1c77","#2c7fb8","#2ca25f"]

for i in range(len(init_F)):

    [x2,y2,c2,ys,yr]=sces(init_F[i],s_fix,rf,0.01)
    [x,y,c,ys,yr]=sces(init_F[i],s_fix,rf)

    ax[0].fill_between(np.array(years[:len(c)]).astype(float),np.array(c).astype(float),np.array(c2).astype(float),\
                         color=colors[i],alpha=0.5)
    ax[0].plot(years[:len(c)],c,linewidth=lw,color=colors[i])
    ax[0].plot(years[:len(c)],c2,linewidth=lw,color=colors[i],label='$\\alpha='+str(init_F[i])+'$')
    if i in {0}:
        ax[2].plot(years[:len(yr)],yr,linewidth=lw,color="#d95f0e",label='$\\rho(t)$')
        ax[2].tick_params(axis='y', colors='#993404')
        ax[2].plot([-10],[-10],linewidth=lw,color="#756bb1",label='$\\sigma(t)$')
        ax2=ax[2].twinx()
        ax2.plot(years[:len(ys)],ys,linewidth=lw,color="#756bb1")
        ax2.tick_params(axis='y', colors='#3f007d')
    if i in {2}:
        ax[0].annotate('$\\zeta = 0\%$', xy=(years[45], c[45]),  xycoords='data',
                xytext=(30, 30), textcoords='offset points',
                arrowprops=dict(arrowstyle="->"))
        ax[0].annotate('$\\zeta = 1\%$', xy=(years[48], c2[48]),  xycoords='data',
                xytext=(50, 36), textcoords='offset points',
                arrowprops=dict(arrowstyle="->"))

    [x2,y2,c2,ys,yr]=dces(init_F[i],s_fix,sf,0.01)
    [x,y,c,ys,yr]=dces(init_F[i],s_fix,sf)

    ax[1].fill_between(np.array(years[:len(c)]).astype(float),np.array(c).astype(float),np.array(c2).astype(float),\
                         color=colors[i],alpha=0.5)
    ax[1].plot(years[:len(c)],c,linewidth=lw,color=colors[i])
    ax[1].plot(years[:len(c)],c2,linewidth=lw,color=colors[i],label='$\\alpha='+str(init_F[i])+'$')
    if i in {0}:
        ax[3].plot(years[:len(yr)],yr,linewidth=lw,color="#d95f0e",label='$\\rho(t)$')
        ax[3].tick_params(axis='y', colors='#993404')
        ax[3].plot([-10],[-10],linewidth=lw,color="#756bb1",label='$\\sigma(t)$')
        ax3=ax[3].twinx()
        ax3.plot(years[:len(ys)],ys,linewidth=lw,color="#756bb1")
        ax3.tick_params(axis='y', colors='#3f007d')
    if i in {2}:
        ax[1].annotate('$\\zeta = 0\%$', xy=(years[64], c[64]),  xycoords='data',
                xytext=(-50, 30), textcoords='offset points',
                arrowprops=dict(arrowstyle="->"))
        ax[1].annotate('$\\zeta = 1\%$', xy=(years[33], c2[33]),  xycoords='data',
                xytext=(-50, 30), textcoords='offset points',
                arrowprops=dict(arrowstyle="->"))

ax[0].grid(color='white', linestyle='solid')
ax[1].grid(color='white', linestyle='solid')
ax[2].grid(color='white', linestyle='solid')
ax[3].grid(color='white', linestyle='solid')
ax[0].set_xlim(2015,2100)
ax[1].set_xlim(2015,2100)
ax[0].set_xticks([2015,2050,2075,2100])
ax[1].set_xticks([2015,2050,2075,2100])
ax[2].set_xlim(2015,2100)
ax[3].set_xlim(2015,2100)
ax[2].set_xticks([2015,2050,2075,2100])
ax[3].set_xticks([2015,2050,2075,2100])
ax[0].set_xticklabels([])
ax[1].set_xticklabels([])
ax[0].set_ylim(0,40)
ax[1].set_ylim(0,40)

ax[2].set_ylim(-1,-.4)
ax[3].set_ylim(-1,-.4)
ax[2].set_yticks([-.5,-.7,-.9])
ax[3].set_yticks([-.5,-.7,-.9])
ax2.set_ylim(0,60)
ax3.set_ylim(0,4500)
ax2.set_yticks([0,20,40,60])
ax3.set_yticks([0,1500,3000,4500])

ax[1].set_yticklabels([])
ax[2].legend(loc=3,framealpha=0.5,fontsize=11)
ax[3].legend(loc=3,framealpha=0.5,fontsize=11)
ax[0].legend(loc=2,framealpha=0.5,fontsize=11)
ax[1].legend(loc=2,framealpha=0.5,fontsize=11)
ax[2].set_xlabel(u'Year')
ax[3].set_xlabel(u'Year')
ax[0].set_ylabel(u'Carbon tax $C$')
ax[0].text(0.98,0.98,'linearly decreasing $\\rho$',
                horizontalalignment='right',
                verticalalignment='top',
                transform=ax[0].transAxes,fontsize=11)
ax[1].text(0.98,0.98,'exponentially decreasing $\\rho$',
                horizontalalignment='right',
                verticalalignment='top',
                transform=ax[1].transAxes,fontsize=11)

ax[0].set_title('Carbon tax $C$\n $C=c_R \cdot [\\alpha/(1-\\alpha)] \
                \cdot [F(\\rho(t))/R(\\rho(t))]^{-\\rho(t)-1} - c_F,$  $\\zeta:$ $yearly$ $decrease$ $in$ $c_R$',size=12,y=1.02,x=1.1)

plt.suptitle(u'Dynamic Elasticity (DES) Energy Substitution Model at Constant Energy Demand $Y$',fontsize=13,y=1.07)
plt.savefig('ces4.png',bbox_inches = 'tight', pad_inches = 0.1, dpi=150)
plt.show()
```

```python
fig = plt.subplots(2,2,figsize=(10,5))
lw=2

gs = mpl.gridspec.GridSpec(2, 2,height_ratios=[1,0.3])
bg=['#EEEEEE','#EEEEEE','#EEEEEE','#EEEEEE']
ax = [plt.subplot(gs[i],axisbg=bg[i],axisbelow=True) for i in range(4)]
plt.subplots_adjust(hspace = .001)

init_F= [0.85,0.9,0.95]
s_fix=1.5
rf=-1
sf=0.1

colors=["#dd1c77","#2c7fb8","#2ca25f"]

for i in range(len(init_F)):

    [x2,y2,c2,ys,yr]=sces(init_F[i],s_fix,rf,0.01,0.03)
    [x,y,c,ys,yr]=sces(init_F[i],s_fix,rf,0.0,0.03)

    ax[0].fill_between(np.array(years[:len(c)]).astype(float),np.array(c).astype(float),np.array(c2).astype(float),\
                         color=colors[i],alpha=0.5)
    ax[0].plot(years[:len(c)],c,linewidth=lw,color=colors[i])
    ax[0].plot(years[:len(c)],c2,linewidth=lw,color=colors[i],label='$\\alpha='+str(init_F[i])+'$')
    if i in {0}:
        ax[2].plot(years[:len(yr)],yr,linewidth=lw,color="#d95f0e",label='$\\rho(t)$')
        ax[2].tick_params(axis='y', colors='#993404')
        ax[2].plot([-10],[-10],linewidth=lw,color="#756bb1",label='$\\sigma(t)$')
        ax2=ax[2].twinx()
        ax2.plot(years[:len(ys)],ys,linewidth=lw,color="#756bb1")
        ax2.tick_params(axis='y', colors='#3f007d')
    if i in {2}:
        ax[0].annotate('$\\zeta = 0\%$', xy=(years[45], c[45]),  xycoords='data',
                xytext=(30, 30), textcoords='offset points',
                arrowprops=dict(arrowstyle="->"))
        ax[0].annotate('$\\zeta = 1\%$', xy=(years[48], c2[48]),  xycoords='data',
                xytext=(50, 36), textcoords='offset points',
                arrowprops=dict(arrowstyle="->"))

    [x2,y2,c2,ys,yr]=dces(init_F[i],s_fix,sf,0.01,0.03)
    [x,y,c,ys,yr]=dces(init_F[i],s_fix,sf,0.0,0.03)

    ax[1].fill_between(np.array(years[:len(c)]).astype(float),np.array(c).astype(float),np.array(c2).astype(float),\
                         color=colors[i],alpha=0.5)
    ax[1].plot(years[:len(c)],c,linewidth=lw,color=colors[i])
    ax[1].plot(years[:len(c)],c2,linewidth=lw,color=colors[i],label='$\\alpha='+str(init_F[i])+'$')
    if i in {0}:
        ax[3].plot(years[:len(yr)],yr,linewidth=lw,color="#d95f0e",label='$\\rho(t)$')
        ax[3].tick_params(axis='y', colors='#993404')
        ax[3].plot([-10],[-10],linewidth=lw,color="#756bb1",label='$\\sigma(t)$')
        ax3=ax[3].twinx()
        ax3.plot(years[:len(ys)],ys,linewidth=lw,color="#756bb1")
        ax3.tick_params(axis='y', colors='#3f007d')
    if i in {2}:
        ax[1].annotate('$\\zeta = 0\%$', xy=(years[64], c[64]),  xycoords='data',
                xytext=(-50, 30), textcoords='offset points',
                arrowprops=dict(arrowstyle="->"))
        ax[1].annotate('$\\zeta = 1\%$', xy=(years[33], c2[33]),  xycoords='data',
                xytext=(-50, 30), textcoords='offset points',
                arrowprops=dict(arrowstyle="->"))

ax[0].grid(color='white', linestyle='solid')
ax[1].grid(color='white', linestyle='solid')
ax[2].grid(color='white', linestyle='solid')
ax[3].grid(color='white', linestyle='solid')
ax[0].set_xlim(2015,2100)
ax[1].set_xlim(2015,2100)
ax[0].set_xticks([2015,2050,2075,2100])
ax[1].set_xticks([2015,2050,2075,2100])
ax[2].set_xlim(2015,2100)
ax[3].set_xlim(2015,2100)
ax[2].set_xticks([2015,2050,2075,2100])
ax[3].set_xticks([2015,2050,2075,2100])
ax[0].set_xticklabels([])
ax[1].set_xticklabels([])
ax[0].set_ylim(0,80)
ax[1].set_ylim(0,80)

ax[2].set_ylim(-1,-.4)
ax[3].set_ylim(-1,-.4)
ax[2].set_yticks([-.5,-.7,-.9])
ax[3].set_yticks([-.5,-.7,-.9])
ax2.set_ylim(0,60)
ax3.set_ylim(0,4500)
ax2.set_yticks([0,20,40,60])
ax3.set_yticks([0,1500,3000,4500])

ax[1].set_yticklabels([])
ax[2].legend(loc=3,framealpha=0.5,fontsize=11)
ax[3].legend(loc=3,framealpha=0.5,fontsize=11)
ax[0].legend(loc=2,framealpha=0.5,fontsize=11)
ax[1].legend(loc=2,framealpha=0.5,fontsize=11)
ax[2].set_xlabel(u'Year')
ax[3].set_xlabel(u'Year')
ax[0].set_ylabel(u'Carbon tax $C$')
ax[0].text(0.98,0.98,'linearly decreasing $\\rho$',
                horizontalalignment='right',
                verticalalignment='top',
                transform=ax[0].transAxes,fontsize=11)
ax[1].text(0.98,0.98,'exponentially decreasing $\\rho$',
                horizontalalignment='right',
                verticalalignment='top',
                transform=ax[1].transAxes,fontsize=11)

ax[0].set_title('Carbon tax $C$\n $C=c_R \cdot [\\alpha/(1-\\alpha)] \
                \cdot [F(\\rho(t))/R(\\rho(t))]^{-\\rho(t)-1} - c_F,$  $\\zeta:$ $yearly$ $decrease$ $in$ $c_R$',size=12,y=1.02,x=1.1)

plt.suptitle(u'Dynamic Elasticity (DES) Energy Substitution Model at Yearly Energy Demand Increase Rate $\\gamma=0.03$',fontsize=13,y=1.07)
plt.savefig('ces5.png',bbox_inches = 'tight', pad_inches = 0.1, dpi=150)
plt.show()
```


```
from sympy import symbols
from sympy.physics.mechanics import *
import sympy.physics.mechanics as me
```

```
q1, q2 = dynamicsymbols('q1 q2')
q1d, q2d = dynamicsymbols('q1 q2', 1)
u1, u2 = dynamicsymbols('u1 u2')
u1d, u2d = dynamicsymbols('u1 u2', 1)
l, m, g = symbols('l m g')
```

```
N = ReferenceFrame('N')
A = N.orientnew('A', 'Axis', [q1, N.z])
B = N.orientnew('B', 'Axis', [q2, N.z])
```

```
A.set_ang_vel(N, u1 * N.z)
B.set_ang_vel(N, u2 * N.z)
```

```
O = Point('O')
P = O.locatenew('P', l * A.x)
R = P.locatenew('R', l * B.x)
```

```
O.v2pt_theory?
```

```
O.set_vel(N, 0)
P.v2pt_theory(O, N, A)
R.v2pt_theory(P, N, B)
```

    l*u1*A.y + l*u2*B.y

```
ParP = Particle('ParP', P, m)
ParR = Particle('ParR', R, m)
```

```
kd = [q1d - u1, q2d - u2]
FL = [(P, m * g * N.x), (R, m * g * N.x)]
BL = [ParP, ParR]
```

```
KM = KanesMethod(N, q_ind=[q1, q2], u_ind=[u1, u2], kd_eqs=kd)
```

```
(fr, frstar) = KM.kanes_equations(FL, BL)
kdd = KM.kindiffdict()
mm = KM.mass_matrix_full
fo = KM.forcing_full
qudots = mm.inv() * fo
qudots = qudots.subs(kdd)
```

```
qudots.simplify()
```

```
qudots
```

```
angular_momentum?
```

```
me.rigidbody?
```

```
am = me.angular_momentum(O, N, ParP, ParR)
am
```

    (l**2*m*u1 + l*m*(l*(sin(q1)*sin(q2) + cos(q1)*cos(q2)) + l)*u1)*A.z + l*m*(l*(sin(q1)*sin(q2) + cos(q1)*cos(q2)) + l)*u2*B.z

```

```


```python
# Copyright (c) Thalesians Ltd, 2018-2019. All rights reserved
# Copyright (c) Paul Alexander Bilokon, 2018-2019. All rights reserved
# Author: Paul Alexander Bilokon <paul@thalesians.com>
# Version: 1.0 (2018.07.22)
# Email: education@thalesians.com
# Platform: Tested on Windows 10 with Python 3.6
```

```python
%matplotlib inline
```

# Set theory and proofs

Mathematics, as it is taught at school, is usually taught by example. Pupils are asked to learn the various mathematical "laws" by rote. For example, we all know from school that $$3 + 5 = 8$$ and $$\frac{d}{dx} x^2 = 2x.$$ We rarely pause and ask, why is this the case, and what are we really doing?

In fact, a lot is going on in these equations. For example, we would have to spend a lot of time explaining what $dx$ "really" is in $$\frac{d}{dx} x^2 = 2x.$$ We would either have to introduce the notion of an **infinitesimal** or resort to Silvanus Thompson's explanation from *Calculus Made Easy* (1910):

$dx$ means a little bit of $x$.

This explanation may be sufficient if we want to *do* mathematics rather than *understand* it. Thompson uses the Ancient Simian Proverb as an epigraph to his work:

What one fool can do, another can.

What are we doing, what is mathematics anyway? **"Mathematics"** is not a "-logy", unlike many of the sciences. The word itself comes from the Ancient Greek μαθηματικός (mathēmatikós, "fond of learning"). Thus mathematics is... a form of learning? This definition is as abstract as mathematics itself...

It was probably G. H. Hardy in *A Mathematician's Apology* (1940) who defined mathematics as the "study of patterns":

A mathematician, like a painter or a poet, is a maker of patterns. If his patterns are more permanent than theirs, it is because they are made with ideas.

Lynn Arthur Steen seconds him in *The Science of Patterns* (Science, 1988):

Mathematics is often defined as the science of space and number, as the discipline rooted in geometry and arithmetic. Although the diversity of modern mathematics has always exceeded this definition, it was not until the recent resonance of computers and mathematics that a more apt definition became fully evident. Mathematics is the science of patterns. The mathematician seeks patterns in number, in space, in science, in computers, and in imagination.

According to this definition, mathematics is a *study of patterns*. And patterns are more general than numbers. Therefore, a number cannot be the most basic "unit" of study in mathematics. What is this most basic "unit"? What abstraction could we introduce to study patterns in all their various forms, including numbers?

In 1874, Georg Cantor introduced just such an abstraction in *On a Property of the Collection of All Real Algebraic Numbers*: the *set*. It has proved to be a very fruitful abstraction, allowing us, among other things, to formalize the notion of *infinity*.

## Sets

A **set** is (arguably) the most fundamental object in mathematics. It is a collection of distinct objects. For example, we could talk about the set of numbers from one up to ten, inclusive. We could give this set a name, say, $S$, and write it like so: $$S = \{1, 2, 3, 4, 5, 6, 7, 8, 9, 10\}.$$ We refer to objects in a particular set as its **elements**. We have just *defined* a particular set by listing its elements. Thus 3 is an element of $S$, and we write $$3 \in S.$$ On the other hand, 11 is not an element of $S$, and we write $$11 \notin S.$$ Neither, in fact, is Joe: $$\text{Joe} \notin S.$$

## Finite and infinite sets

This particular set, $$S = \{1, 2, 3, 4, 5, 6, 7, 8, 9, 10\},$$ is **finite**. Consider the set of all **natural numbers**: $$\mathbb{N} = \{1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, ...\}.$$ There is no way to list all of its elements, of course, but at least, if we had an infinite amount of time, we could enumerate (list) them: "one", "two", "three", and so on. We call such sets **countably infinite** or **denumerable**.

## Equivalence of sets

Two sets, $A$ and $B$ are said to be **equivalent** or **equinumerous** (we write $A \sim B$) if a one-to-one correspondence can be set up between all their elements. For example, the sets $A = \{1, 2, 3\}$ and $B = \{a, b, c\}$ are equivalent:
$$
1 \mapsto a, \\
2 \mapsto b, \\
3 \mapsto c.$$
This is not the only such one-to-one correspondence; for example, we could use this one:
$$
1 \mapsto b, \\
2 \mapsto a, \\
3 \mapsto c.
$$

On the other hand, $A = \{1, 2\}$ and $B = \{a, b, c\}$ are not equivalent: we need a *one-to-one* correspondence between *all* elements of $A$ and all the elements of $B$, but $A$ has fewer elements than $B$.

Finite sets are equivalent if and only if (or, to use Paul Halmos's abbreviation, **iff**) they have the same number of elements. In fact, equivalence is a generalization of this notion (that the sets have the same number of elements) to sets with infinitely many elements.

Thus every countably infinite (denumerable) set is equivalent to the set of natural numbers, $\mathbb{N}$: enumerating a set or listing its elements is the same as finding a one-to-one correspondence between the elements of this set and natural numbers.

Are positive rational numbers (i.e. fractions $\frac{p}{q}$, $p, q \in \mathbb{N}$, $q \neq 0$) denumerable? What do you think?

First, note that positive rationals can be arranged in a table:

$p$:	     	 $1$	           	 $2$	           	 $3$	           	 $4$	           	 $5$	           	 $\ldots$

$q$

$1$	      	 $\frac{1}{1}$	 	 $\frac{2}{1}$	 	 $\frac{3}{1}$	 	 $\frac{4}{1}$	 	 $\frac{5}{1}$	 	 $\ldots$

$2$	      	 $\frac{1}{2}$	 	 $\frac{2}{2}$	 	 $\frac{3}{2}$	 	 $\frac{4}{2}$	 	 $\frac{5}{2}$	 	 $\ldots$

$3$	      	 $\frac{1}{3}$	 	 $\frac{2}{3}$	 	 $\frac{3}{3}$	 	 $\frac{4}{3}$	 	 $\frac{5}{3}$	 	 $\ldots$

$4$	      	 $\frac{1}{4}$	 	 $\frac{2}{4}$	 	 $\frac{3}{4}$	 	 $\frac{4}{4}$	 	 $\frac{5}{4}$	 	 $\ldots$

$5$	      	 $\frac{1}{5}$	 	 $\frac{2}{5}$	 	 $\frac{3}{5}$	 	 $\frac{4}{5}$	 	 $\frac{5}{5}$	 	 $\ldots$

$\vdots$	 	 $\vdots$	      	 $\vdots$	      	 $\vdots$	      	 $\vdots$	      	 $\vdots$	      	 $\ddots$

Not all entries in this table are distinct, for example, $\frac{1}{1} = \frac{2}{2} = \frac{3}{3} = \frac{4}{4} = \frac{5}{5}$, $\frac{2}{4} = \frac{1}{2}$, etc. Let us erase all rational numbers that have non-trivial common factors in the numerator and denominator, while keeping the first occurrence. All rational numbers in our table are now unique; if we continue the table indefinitely, it will include all positive rational numbers:

$p$:	     	 $1$	           	 $2$	           	 $3$	           	 $4$	           	 $5$	           	 $\ldots$

$q$

$1$	      	 $\frac{1}{1}$	 	 $\frac{2}{1}$	 	 $\frac{3}{1}$	 	 $\frac{4}{1}$	 	 $\frac{5}{1}$	 	 $\ldots$

$2$	      	 $\frac{1}{2}$	 	 	              	 $\frac{3}{2}$	 	 	              	 $\frac{5}{2}$	 	 $\ldots$

$3$	      	 $\frac{1}{3}$	 	 $\frac{2}{3}$	 	 	              	 $\frac{4}{3}$	 	 $\frac{5}{3}$	 	 $\ldots$

$4$	      	 $\frac{1}{4}$	 	 	              	 $\frac{3}{4}$	 	 	              	 $\frac{5}{4}$	 	 $\ldots$

$5$	      	 $\frac{1}{5}$	 	 $\frac{2}{5}$	 	 $\frac{3}{5}$	 	 $\frac{4}{5}$	 	 	              	 $\ldots$

$\vdots$	 	 $\vdots$	      	 $\vdots$	      	 $\vdots$	      	 $\vdots$	      	 $\vdots$	      	 $\ddots$

Finally, we associate these numbers with the natural numbers $1, 2, 3, \ldots$. We start in the top-left corner of the table and then follow the arrows:

$p$:	     	 $1$	           	           $2$	           	              $3$	           	           $4$	           	              $5$	           	 $\ldots$

$q$

$1$	      	 $\frac{1}{1}$	 	           $\frac{2}{1}$	 $\rightarrow$	 $\frac{3}{1}$	 	           $\frac{4}{1}$	 $\rightarrow$	 $\frac{5}{1}$	 	 $\ldots$

	         	 $\downarrow$	  $\nearrow$	 	              $\swarrow$	    	              $\nearrow$	 	              $\swarrow$

$2$	      	 $\frac{1}{2}$	 	           $\swarrow$	    	              $\frac{3}{2}$	 	           $\swarrow$	    	              $\frac{5}{2}$	 	 $\ldots$

	         	 	              $\swarrow$	 	              $\nearrow$	    	              $\swarrow$	 	              $\nearrow$

$3$	      	 $\frac{1}{3}$	 	           $\frac{2}{3}$	 	              $\swarrow$	    	           $\frac{4}{3}$	 	              $\frac{5}{3}$	 	 $\ldots$

	         	 $\downarrow$	  $\nearrow$	 	              $\swarrow$	    	              $\nearrow$	 	              $\swarrow$

$4$	      	 $\frac{1}{4}$	 	           $\swarrow$	    	              $\frac{3}{4}$	 	           $\swarrow$	    	              $\frac{5}{4}$	 	 $\ldots$

	         	 	              $\swarrow$	 	              $\nearrow$	    	              $\swarrow$	 	              $\nearrow$

$5$	      	 $\frac{1}{5}$	 	           $\frac{2}{5}$	 	              $\frac{3}{5}$	 	           $\frac{4}{5}$	 	              $\swarrow$	    	 $\ldots$

	         	 $\downarrow$	  $\nearrow$	 	              $\swarrow$	    	              $\nearrow$	 	              $\swarrow$

$\vdots$	 	 $\vdots$	      	           $\vdots$	      	              $\vdots$	      	           $\vdots$	      	              $\vdots$	      	 $\ddots$

It is easy to see that all *negative* rational numbers are also denumerable: we simply put the minus sign before the fractions in the tables above.

What about all **real numbers**, including positive and negative rationals, such as $\frac{1}{2}$ and $-\frac{1}{2}$, zero, and irrationals, such as $\sqrt{2}$, $\pi = 3.1415926535\ldots$ and $e = 2.7182818284\ldots$? Clearly, this set (denoted $\mathbb{R}$) is also infinite. Can we enumerate it?

We know that all real numbers can be written as decimal fractions, e.g. $\pi = 3.1415926535...$. Suppose we can enumerate all real numbers between 0 and 1, inclusive:
$$
a_1 = 0.a_{11}a_{12}a_{13}a_{14}a_{15}\ldots, \\
a_2 = 0.a_{21}a_{22}a_{23}a_{24}a_{25}\ldots, \\
a_3 = 0.a_{31}a_{32}a_{33}a_{34}a_{35}\ldots, \\
\vdots \\
a_k = 0.a_{k1}a_{k2}a_{k3}a_{k4}a_{k5}\ldots, \\
\vdots \\
$$
Here $a_{11}, a_{12}, a_{13}, a_{14}, a_{15}, \ldots, a_{21}, a_{22}, a_{23}, \ldots$ are all decimal digits, $0, 1, 2, 3, \ldots 9$.

If indeed it is possible to enumerate all real numbers between 0 and 1, then all of them appear on our list. But consider the number $b$ which differs from $a_1$ in the first digit (so that digit is anything but $a_{11}$), from $a_2$ in the second digit (so that digit is anything but $a_{22}$), from $a_3$ in the third digit (so that digit is anything but $a_{33}$), and so on. We have highlighted these digits below:
$$
a_1 = 0.\mathbf{a_{11}}a_{12}a_{13}a_{14}a_{15}\ldots, \\
a_2 = 0.a_{21}\mathbf{a_{22}}a_{23}a_{24}a_{25}\ldots, \\
a_3 = 0.a_{31}a_{32}\mathbf{a_{33}}a_{34}a_{35}\ldots, \\
\vdots \\
a_k = 0.a_{k1}a_{k2}a_{k3}a_{k4}a_{k5}\ldots \mathbf{a_{kk}}\ldots, \\
\vdots \\
$$

By construction, $b$ differs from *all* numbers on our list, therefore $b$ is *not* on our list. So our attempt to enumerate all numbers between 0 and 1 (let alone *all* real numbers!) has failed.

We have just shown that real numbers are not enumerable using the so-called **Cantor's diagonal slash argument**. It is a "diagonal slash" for obvious reasons, while it's "Cantor's" because the aforementioned Georg Cantor discovered it.

Incidentally, Cantor's set theory, which has become the foundation of modern mathematics, was initially rejected by many prominent mathematicians. Leopold Kronecker, for instance, said:

I don't know what predominates in Cantor's theory — philosophy or theology, but I am sure that there is no mathematics there.

Thus the set $\mathbb{R}$, just like the set $\mathbb{N}$, is infinite, but it is even "more" infinite that $\mathbb{N}$: it is **uncountable** or **uncountably infinite**: we can't even enumerate it! In set theory this idea of *different kinds of infinity* generates further to the notion of **cardinality**.

## Subsets and supersets

Notice that all elements of our example set $$S = \{1, 2, 3, 4, 5, 6, 7, 8, 9, 10\}$$ are also elements of $\mathbb{N}$. We say that $S$ is a **subset** of $\mathbb{N}$ and write $$S \subseteq \mathbb{N}.$$ Of course, $$\mathbb{N} \nsubseteq S.$$

We could, equivalently, say that $\mathbb{N}$ is a **superset** of $S$ and write $$\mathbb{N} \supseteq S.$$

Similarly, $$\mathbb{N} \subseteq \mathbb{R},$$ which is the same thing as $$\mathbb{R} \supseteq \mathbb{N},$$ and $$\mathbb{R} \nsubseteq \mathbb{N},$$ which is the same thing as $$\mathbb{N} \nsupseteq \mathbb{R}.$$

Since $S$ is a subset of $\mathbb{N}$, we could use the following "syntactic sugar" to define $S$:
$$S = \{x \, | \, x \in \mathbb{N}, x \leq 10\}.$$
We read "$|$" as "such that". Thus, instead of listing all elements of a set, we could define it by mentioning a particular **property** of its elements, such as $x \leq 10$.

We can write $$|S| = 10$$ to indicate that $S$ has exactly 10 elements.

## Equality of sets

Any two sets $A$ and $B$ are equal, $A = B$, iff (if and only if) $A \subseteq B$ *and* $B \subseteq A$.

We shall add that we consider only distinct objects when talking about elements of sets. Repeats are not allowed. Thus $\{2, 2\}$ is really the same set as $\{2\}$ and it is deemed to contain exactly one element.

Nor do we care about the order of elements in a set: $\{1, 2, 3\}$ and $\{3, 2, 1\}$, for instance, are deemed to be equal.

A set containing a single element, such as $\{5\}$, is called a **singleton** set. We distinguish sets from their elements. Thus $5 \in A$, whereas $\{5\} \notin A$. This is because $\{5\}$ is *not* the *number five*, it is a *set containing* the number five.

In the programming language Python there is a data structure called `set`, which works according to principles that mimic those of a mathematical set.

```python
A = set([1, 2, 3, 4, 5, 6, 7, 8, 9, 10])
```

```python
A
```

    {1, 2, 3, 4, 5, 6, 7, 8, 9, 10}

```python
set([2, 3, 4]).issubset(A)
```

    True

```python
A.issubset(set([2, 3, 4]))
```

    False

```python
set([2, 3, 4]) == A
```

    False

```python
set([1, 2, 3]) == set([3, 2, 1])
```

    True

```python
set([2, 2]) == set([2])
```

    True

```python
set([2, 2])
```

    {2}

```python
len(set([2, 2]))
```

    1

```python
B = set([1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 1, 1, 1])
```

```python
A.issubset(B)
```

    True

```python
B.issubset(A)
```

    True

```python
A == B
```

    True

```python
len(B)
```

    10

```python
set(['foo', 'bar', 'baz', 1, 2, 3, 4, 5, 1./7.])
```

    {0.14285714285714285, 'baz', 2, 1, 3, 4, 5, 'bar', 'foo'}

## Proof by contradiction

When we were asked whether the set of positive rational numbers was denumerable, we simply enumerated its elements. We **proved** the statement "the set of rational numbers is denumerable" by actually finding — constructing — the requisite enumeration. Such a proof is known as a **constructive proof**: the existence of a mathematical object (in this case, the one-to-one correspondence between the natural numbers and the positive rational numbers) is  demonstrated by creating or providing a method for creating the object.

**Cantor's diagonal slash argument** is an example of a different kind of proof — **proof by contradiction** or, in Latin, ***reductio ad absurdum***.

In *A Mathematician's Apology*, G. H. Hardy described proof by contradiction as "one of a mathematician's finest weapons", saying

It is a far finer gambit than any chess gambit: a chess player may offer the sacrifice of a pawn or even a piece, but a mathematician offers the game.

Here is another example. This one is due to Euclid (c. 300 BC).

Recall that a prime is a natural number greater than 1 that cannot be formed by multiplying two smaller natural numbers. The first few primes are 2, 3, 5, 7, 11, 13, 17, 19, 23. Indeed, we can quickly come up with a list of primes using Python list comprehensions (in practice there are *far* more efficient algorithms for finding primes):

```python
[x for x in range(2, 100) if all(x % y != 0 for y in range(2, x))]
```

    [2,
     3,
     5,
     7,
     11,
     13,
     17,
     19,
     23,
     29,
     31,
     37,
     41,
     43,
     47,
     53,
     59,
     61,
     67,
     71,
     73,
     79,
     83,
     89,
     97]

How do you prove that there are infinitely many primes, i.e. the set of all primes is infinite (obviously, countably infinite, since it is a subset of natural numbers, a countably infinite set)?

Since we are talking about proofs by contradiction, it may be sensible to assume that that's how we shall proceed. Assume *for a contradiction* that $P$, the set of *all* primes, is finite. Say, there exist exactly $n$ primes, $n \in \mathbb{N}$:
$$P = \{p_1, p_2, p_3, \ldots, p_n\}.$$

Multiplying the elements of $P$ together, we obtain another number. Let us add 1 to that number to obtain
$$a = p_1 p_2 p_3 \ldots p_n + 1.$$
Clearly this number is greater than any of the elements of $P$, so it is not in $P$. Since, by our assumption, $P$ contains all primes, $a$ is not a prime. Then there exists some prime in $P$, say, $p_k$, $1 \leq k \leq n$, that divides a. Then,
$$a = p_k \cdot b, \quad b \in \mathbb{N}.$$
But then
$$
\begin{align}
1 &= a - p_1 p_2 p_3 \ldots p_n \\
  &= p_k \cdot b - p_1 \ldots p_{k-1} p_k p_{k+1} \ldots p_n \\
  &= p_k (b - p_1 \ldots p_{k-1} p_{k+1} \ldots p_n).
\end{align}
$$
In other words, $p_k$ divides 1. Since no natural number other than 1 divides 1, we have a contradiction. Our assumption that $P$ is finite must have been wrong. There are therefore infinitely many primes. **Q.E.D.** (which signifies the end of the proof and is an initialism of the Latin phrase *quod erat demonstrandum*, "what was to be demonstrated").

Instead of Q.E.D., people sometimes put the **Halmos symbol**, □, at the end of the proof.

Here is yet another example: prove that $\sqrt{2}$ is not a rational number, i.e. that $\sqrt{2}$ is irrational.

*Assume for a contradiction* that $\sqrt{2}$ is rational. Then we can write it as $\sqrt{2} = \frac{p}{q}$, where $p$ and $q$ are natural numbers, $q \neq 0$. Further, we can assume that $\frac{p}{q}$ is a fraction in lowest terms, i.e. there is no prime that divides both $p$ and $q$. (Thus $\frac{3}{9}$ is not in lowest terms, since $3$ divides both $3$ and $9$, whereas $\frac{1}{3}$ is.)

Since $\sqrt{2} = \frac{p}{q}$, on squaring both sides, we obtain $2 = \frac{p^2}{q^2}$, hence $p^2 = 2q^2$. Since the right-hand side is even, the left-hand side must be even. Therefore $p$ is even, say $p = 2a$ for some $a \in \mathbb{N}$. But then $p^2 = 4a^2$, so $4a^2 = 2q^2$, whence $q^2 = 2a^2$, thus $q$ is even.

This contradicts our assumption that $\frac{p}{q}$ is a fraction in lowest terms. We have reached the contradiction. Therefore $\sqrt{2}$ cannot be written as a fraction in the form $\frac{p}{q}$. In other words, $\sqrt{2}$ is irrational. □

That $\sqrt{2}$ is irrational was discovered by Pythagoras and his followers, i.e. the Pythagoreans, in the 6th century BC. Prior to this discovery, people believed that all numbers were rational, i.e. could be expressed as simple fractions.

The discovery of irrational numbers is said to have been shocking to the Pythagoreans (as it violated their mystical worldview). They kept this discovery secret. Hippasus of Metapontum divulged this secret and is supposed to have drowned at sea, apparently as a punishment from the gods for divulging it.

## Russell's paradox and Zermelo-Fraenkel set theory

Our treatment of set theory is very informal. In practice it was built out of axioms. Refer to *Naïve set theory* by Paul Halmos for a more detailed overview.

Now, suppose that we have a set containing all sets that are not elements of themselves. Symbolically, let $$R = \{x \, | \, x \notin x\}.$$ Is $R$ an element of itself?

If it were, $R \in R$, then we could substitute it for $x$ in "$x \, | \, x \notin x$", and so $R \notin R$.

If it weren't, then $R \notin R$, and, by definition, since $R$ is not an element of itself, it is in $R$, $R \in R$.

Thus we have a paradox: $R \in R \Leftrightarrow R \notin R$.

This particular paradox was discovered by Bertrand Russell in 1901 and bears his name, so it is called **Russell's paradox**.

This paradox confounded the so-called **naïve set theorists**, who did not know how to deal with it. Eventually, in 1908, Ernst Zermelo proposed an axiomatization of set theory that avoided the paradoxes of naïve set theory, which was eventually elaborated by Abraham Fraenkel, Thoralf Skolem, and Zermelo himself. The result became known as the **Zermelo-Fraenkel set theory** or **ZFC**. It is ZFC that remains the canonical axiomatic set theory to this day.

We won't go into the details of how Russell's paradox is resolved in ZFC, but we shall prefer to talk about **collections** or **families** of sets, rather than sets of sets.

## Union and intersection

The **union** of two sets $A$ and $B$ is the set of elements which are in $A$, in $B$, or in both $A$ and $B$: $$A \cup B = \{x \, | \, x \in A \text{ or } x \in B\}.$$

We can check that, in Python,

```python
A = set([3, 5, 7, 9])
B = set([1, 2, 3])
A.union(B)
```

    {1, 2, 3, 5, 7, 9}

The **intersection** of two sets $A$ and $B$ is the set of elements which are in $A$ *and* in $B$: $$A \cap B = \{x \, | \, x \in A \text{ and } x \in B\}.$$

In Python,

```python
A = set([3, 5, 7, 9])
B = set([1, 2, 3])
A.intersection(B)
```

    {3}

**Venn diagrams** are helpful in visualising sets, including set unions and intersections (which are, of course, themselves sets).

The Python package `matplotlib_venn` is helpful in constructing them. It can be installed using `easy_install matplotlib-venn`.

```python
from matplotlib_venn import venn2
v = venn2(subsets = (2, 2, 1));
v.get_label_by_id('01').set_text('')
v.get_patch_by_id('01').set_linewidth(2)
v.get_patch_by_id('01').set_edgecolor('black')
v.get_patch_by_id('01').set_facecolor('white')

v.get_label_by_id('10').set_text('')
v.get_patch_by_id('10').set_linewidth(2)
v.get_patch_by_id('10').set_edgecolor('black')
v.get_patch_by_id('10').set_facecolor('white')

v.get_label_by_id('11').set_text('$A \cap B$')
v.get_patch_by_id('11').set_linewidth(2)
v.get_patch_by_id('11').set_edgecolor('black')
v.get_patch_by_id('11').set_facecolor('#ff0000')
```

```python
from matplotlib_venn import venn2
v = venn2(subsets = (2, 2, 1));
v.get_label_by_id('01').set_text('')
v.get_patch_by_id('01').set_linewidth(2)
v.get_patch_by_id('01').set_edgecolor('black')
v.get_patch_by_id('01').set_facecolor('#ff0000')

v.get_label_by_id('10').set_text('')
v.get_patch_by_id('10').set_linewidth(2)
v.get_patch_by_id('10').set_edgecolor('black')
v.get_patch_by_id('10').set_facecolor('#ff0000')

v.get_label_by_id('11').set_text('$A \cup B$')
v.get_patch_by_id('11').set_linewidth(2)
v.get_patch_by_id('11').set_edgecolor('black')
v.get_patch_by_id('11').set_facecolor('#ff0000')
```

John Venn (1834 - 1923) was an English logician and philosopher. He introduced the diagrams that would later bear his name in an 1880 paper entitled *On the Diagrammatic and Mechanical Representation of Propositions and Reasonings*.

There is a stained glass window at Gonville and Caius College, Cambridge, where Venn studied and worked, commemorating Venn and the Venn diagram.

The union and intersection may be extended to more than two sets. For example, let $G$ be the set of Greek uppercase letter glyphs,

$$G = \{A, B, \Gamma, \Delta, E, Z, H, \Theta, I, K, \Lambda, M, N, \Xi, O, \Pi, P, \Sigma, T, Y, \Phi, X, \Psi, \Omega\},$$

$E$ be the set of English uppercase letter glyphs,

$$E = \{A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U, V, W, X, Y, Z\},$$

and $R$ be the set of Russian uppercase letter glyphs,

$$R = \{\text{А, Б, В, Г, Д, Е, Ё, Ж, З, И, Й, К, Л, М, Н, О, П, Р, С, Т, У, Ф, Х, Ц, Ч, Ш, Щ, Ъ, Ы, Ь, Э, Ю, Я}\}$$.

Then their intersection is given by

$$G \cap E \cap R = \{A, B, E, H, K, M, O, P, T, X, Y\}.$$

We can also take unions and intersections of infinitely many sets. Let $N_2$ denote the set of positive multiples of 2,
$$N_2 = \{2, 4, 6, 8, 10, 12, \ldots\},$$
$N_3$ denote the set of positive multiples of 3,
$$N_3 = \{3, 6, 9, 12, 15, 18, \ldots\},$$
and so on. Then we can write
$$\mathbb{N} = \{1\} \cup \bigcup_{i=2}^{\infty} N_i.$$
Since here we have taken a union of countably many sets (we can enumerate $N_2, N_3, N_4, \ldots$), we refer to $\bigcup_{i=2}^{\infty} N_i$ as a **countable union**.

It is also possible to define uncountable, or **arbitrary**, unions and intersections.

Unions and intersections are connected by the following relations:
$$
(A \cup B) \cap C = (A \cap C) \cup (B \cap C), \\
(A \cap B) \cup C = (A \cup C) \cap (B \cup C).
$$

## Set difference, De Morgan's laws

Another important operation on sets is the **set difference**,
$$A \setminus B = \{x \, | \, x \in A \text{ and } x \notin B\}.$$

If we assume that all the sets that we are considering are subsets of some large  set $\Omega$, we may write $A^{\complement}$, $A'$, or $\overline{A}$ instead of $\Omega \setminus A$ and refer to $\Omega \setminus A$ as the **complement** of $A$ (in $\Omega$).

The following relations, known as **De Morgan's laws**, so named after the 19-th century British mathematician Augustus De Morgan, play an important part in **set theory**:
$$\overline{A \cup B} = \overline{A} \cap \overline{B},$$
and
$$\overline{A \cap B} = \overline{A} \cup \overline{B}.$$

They can be generalized to arbitrary unions and intersections of infinitely many sets.

Here is an exercise for you: *prove De Morgan's laws*.

Assume that an element belongs to the set on the left-hand side and show that it also belongs to the set on the right-hand side (so the set on the left-hand side is a subset of the set on the right-hand side). Then assume that an element belongs to the set on the right-hand side and show that it also belongs to the set on the left-hand side (so the set on the right-hand side is a subset of the set on the left-hand side). If two sets are subsets of each other, then they are equal.

## Cartesian products

The Cartesian product of two sets $A$ and $B$, written $A \times B$, is the set of all ordered pairs $(a, b)$ where $a \in A$ and $b \in B$.

For example, the Cartesian product of the sets $A = \{\text{foo}, \text{bar}, \text{baz}\}$ and $B = \{3, 12\}$ is the set
$$A \times B = \{(\text{foo}, 3), (\text{bar}, 3), (\text{baz}, 3), (\text{foo}, 12), (\text{bar}, 12), (\text{baz}, 12)\}.$$

As another example, consider the 2-dimensional plane, the set of pairs $(x, y)$ with $x \in \mathbb{R}$, $y \in \mathbb{R}$.

More generally, Cartesian products can be defined for $n \in \mathbb{N}$ sets. For example, the 3-dimensional space, the set of triples $(x, y, z)$, $x \in \mathbb{R}$, $y \in \mathbb{R}$, $z \in \mathbb{R}$, is a 3-fold Cartesian product $\mathbb{R}^3 = \mathbb{R} \times \mathbb{R} \times \mathbb{R}$.

## Functions

A **binary relation** *R* between a set $X$ (the **set of departure**) and a set $Y$ (the **set of destination** or **codomain**) is specified by its **graph**, $G$, which is a set of ordered pairs $(x, y)$, a subset of the cartesian product $X \times Y$. The binary relation is also known as a **mapping** or **correspondence**.

The statement $(x, y) \in G$ is read as **"$x$ is $R$-related to $y$"** and is denoted $xRy$ or $R(x, y)$. The order is important: $xRy$ does not necessarily imply $yRx$ for a particular binary relation $R$.

If $R$ is a binary relation between $X$ and $Y$, then the set $\{y \in Y \,|\, xRy \text{ for some } x \in X\}$ is called the **image** or **range**, of $R$. The set $\{x \in X \,|\, xRy \text{ for some } y \in Y\}$ is called the **domain** of $R$.

A **function** $f$ from a set $X$ to a set $Y$ (we sometimes write $f: X \rightarrow Y$) is a special case of a binary relation: it is defined by a set $G \subseteq X \times Y$ of ordered pairs $(x, y)$ such that, for *each* element $x \in X$ there corresponds *one, and only one,* pair $(x, y) \in G$, and we write $f(x) = y$ or $f: x \mapsto y$.

Suppose that $A = \{1, 2\}$, $B = \{a, b, c\}$.

Is the binary relation $f: A \rightarrow B$ defined by $G_f = \{(1, b), (2, a)\}$ a function?

*x*	*f(x)*

$1$	$b$

$2$	$a$

Indeed, it is a function: to *each* element of $x$, $f$ maps *exactly one* element of $B$.

Suppose that $A = \{1, 2\}$, $B = \{a, b, c\}$, as before.

Is the binary relation $g: A \rightarrow B$ defined by $G_g = \{(1, b), (1, a), (2, a)\}$ a function?

*x*	*g(x)*

$1$	$b$

$1$	$a$

$2$	$a$

No, it isn't: two distinct elements of $B$, $a, b \in B$, are mapped to $1 \in A$, so $g$ is not a function by definition.

Suppose that $A = \{1, 2\}$, $B = \{a, b, c\}$, as before.

Is the binary relation $h: A \rightarrow B$ defined by $G_h = \{(2, a)\}$ a function?

*x*	*h(x)*

$2$	$a$

No, it isn't: $h$ doesn't map any element of $B$ to $1 \in A$.

Suppose that $A = \{1, 2\}$, $B = \{a, b, c\}$, as before.

Is the binary relation $\alpha: A \rightarrow B$ defined by $G_{\alpha} = \{(1, a), (2, a)\}$ a function?

*x*	*$\alpha$(x)*

$1$	$a$

$2$	$a$

Indeed it is: to *each* element of $A$ there corresponds *exactly one* element of $B$.

Now, consider the binary relation $s: \mathbb{R} \rightarrow \mathbb{R}$ defined, for all $x \in \mathbb{R}$, by $s(x) = x^2$ or, in another notation, $s: x \mapsto x^2$:

```python
import numpy as np
import matplotlib.pyplot as plt
xs = np.linspace(-10., 10., 50)
plt.plot(xs, [x*x for x in xs]);
plt.xlabel('x')
plt.ylabel('s(x)');
```

Is this a function?

Indeed, $s$ is a function: to *each* $x \in \mathbb{R}$ there corresponds one, and only one (thus exactly one) $s(x) \in \mathbb{R}$.

What about the binary relation $r: \mathbb{R} \rightarrow \mathbb{R}$ defined, for all $x \in \mathbb{R}$, by $r(x) = \sqrt{x}$?

First of all, this definition doesn't quite make sense. $\sqrt{\cdot}$ is defined only for nonnegative real numbers.

Suppose we redefine $r: \mathbb{R} \rightarrow \mathbb{R}$: for all $x \geq 0$, $r(x) = \sqrt{x}$.

Is this now a function?

It's not a function $r: \mathbb{R} \rightarrow \mathbb{R}$, since it does not define a value $f(x)$ for *all* $x \in \mathbb{R}$, only for those $x$ that are nonnegative.

How could we fix the definition of $r$ so that it is a function?

One way to do it is to define it as $r: \mathbb{R}_{\geq 0} \rightarrow \mathbb{R}$, where $\mathbb{R}_{\geq 0}$ is the set of *nonnegative* real numbers.

```python
xs = np.linspace(-10., 10., 50)
xs = [x for x in xs if x >= 0]
plt.plot(xs, [np.sqrt(x) for x in xs]);
plt.xlabel('x')
plt.ylabel('s(x)');
```

## Image and inverse image

Let $f: X \rightarrow Y$ be a function and $A \subseteq X$. Then the **image** $f[A]$ of $A$ under $f$ is the set $$f[A] = \{y \in Y \,|\, y = f(x) \text{ for some } x \in A\}.$$

This is consistent with the definition of the **image** of a function given above: the image of the function $f$ is the image $f[X]$ of the entire set $X$.

Let $f: X \rightarrow Y$ be a function and $B \subseteq Y$. Then the **preimage** or **inverse image** of $B$ under $f$ is the set $$f^{-1}[B] = \{x \in X \,|\, f(x) \in B\}.$$

For example, for $f: \mathbb{R} \rightarrow \mathbb{R}$, $f: x \mapsto x^2$, the inverse image of the singleton set $\{4\}$ is the set $\{-2, 2\}$. The inverse image of the set $\{4, 9\}$ is the set $\{-3, -2, 2, 3\}$.

The inverse image of the union of two sets is equal to the union of their inverse images:
$$f^{-1}[A \cup B] = f^{-1}[A] \cup f^{-1}[B].$$

Exercise: How would you prove this?

The inverse image of the intersection of two sets is equal to the intersection of their inverse images:
$$f^{-1}[A \cap B] = f^{-1}[A] \cap f^{-1}[B].$$

The image of the union of two sets is equal to the union of their images:
$$f[A \cup B] = f[A] \cup f[B].$$

Is the image of the intersection of two sets equal to the intersection of their images:
$$f[A \cap B] \overset{?}{=} f[A] \cap f[B].$$

The image of the intersection of two sets is, in general, *not* equal to the intersection of their images:
$$f[A \cap B] \neq f[A] \cap f[B].$$

To see this, consider $f: \mathbb{R} \times \mathbb{R} \rightarrow \mathbb{R}$, defined by $f: (x, y) \mapsto x$, a **projection** on the $x$-plane.

Define $A = \{(x, 0) \,|\, 0 \leq x \leq 1\}$ and $A = \{(x, 1) \,|\, 0 \leq x \leq 1\}$.

The two sets do not intersect, or, in other words, their intersection is the so-called **empty set**: $A \cap B = \{\} = \emptyset$.

However, the images of the two sets coincide: $f(A) = f(B) = \{x \,|\, 0 \leq x \leq 1\}$.

We have just disproven
$$f[A \cap B] = f[A] \cap f[B],$$
equivalently, we have just proven
$$f[A \cap B] \neq f[A] \cap f[B].$$
by producing a particular **counterexample** — this is yet another proof method, very common in mathematics. □

## One-to-one, onto, bijections, and inverse functions

A function $f: X \rightarrow Y$ is **injective** or **one-to-one** if each possible element $y \in Y$ of its codomain $y$ is mapped to by at most one argument $x \in X$. We call **injective** functions **injections**.

It is **surjective** or **onto** if each possible element $y \in Y$ is mapped to by at least one argument $x \in X$. We call **surjective** functions **surjections**.

It is **bijective** if it is both injective and surjective (one-to-one and onto). We call such functions **bijections** or **one-to-one correspondences**.

A function $f: X \rightarrow Y$ is **invertible** if there exists a function $g: Y \rightarrow X$ such that, for all $y \in Y$, $g(f(x)) = x$. We call $g$ an inverse of $f$ and sometimes denote it by $f^{-1}$.

One can check that a function is invertible iff it is a bijection.

Suppose that $A = \{1, 2\}$, $B = \{a, b, c\}$.

Is the function $f: A \rightarrow B$ defined by $G_f = \{(1, b), (2, a)\}$ one-to-one, onto, a bijection, invertible?

*x*	*f(x)*

$1$	$b$

$2$	$a$

* It is one-to-one, since for each element it its image, $f[A]$, there corresponds a single element of $A$: to $a$, there corresponds 2, and to $b$, there corresponds 1.

* It is *not* onto, since no element of $A$ is mapped to $c \in B$.

* Since the function is *not* one-to-one *and* onto, but only one-to-one, it is not a bijection.

* Therefore it is not invertible. Indeed we could not define the inverse function $f^{-1}: B \rightarrow A$ since we wouldn't be able to map $c \in B$ to anything — it wouldn't be a function.

As before, $A = \{1, 2\}$, $B = \{a, b, c\}$.

Is the function $\alpha: A \rightarrow B$ defined by $G_{\alpha} = \{(1, a), (2, a)\}$ one-to-one, onto, a bijection, invertible?

*x*	*$\alpha$(x)*

$1$	$a$

$2$	$a$

* It is *not* one-to-one, since for $a \in f[A]$, there correspond two elements of $A$: 1 and 2: $f(1) = f(2) = a$.
* It is *not* onto, since no element of $A$ is mapped to $b \in B$, nor is there an element of $A$ mapped to $c \in B$.
* Since the function is *not* one-to-one *and* onto, in fact, it is neither, it is not a bijection.
* Therefore it is not invertible.

In fact, is there *any* bijection (and therefore *any* invertible function) between $A = \{1, 2\}$ and $B = \{a, b, c\}$?

Since $|A| \neq |B|$ there isn't!

But between $C = \{1, 2\}$ and $D = \{a, b\}$ there are two bijections. One is
$\beta: C \rightarrow D$ defined by $G_{\beta} = \{(1, a), (2, b)\}$:

*x*	*$\beta$(x)*

$1$	$a$

$2$	$b$

Its inverse is $\beta^{-1}: D \rightarrow C$ defined by $G_{\beta^{-1}} = \{(a, 1), (b, 2)\}$:

*y*	*$\beta^{-1}$(y)*

$a$	$1$

$b$	$2$

The other bijection between $C = \{1, 2\}$ and $D = \{a, b\}$ is $\gamma: C \rightarrow D$ defined by $G_{\gamma} = \{(1, b), (2, a)\}$:

*x*	*$\gamma$(x)*

$1$	$b$

$2$	$a$

Its inverse is $\gamma^{-1}: D \rightarrow C$ defined by $G_{\gamma^{-1}} = \{(a, 2), (b, 1)\}$:

*y*	*$\gamma^{-1}$(y)*

$a$	$2$

$b$	$1$

Since $\gamma$ is a bijection, so is $\gamma^{-1}$, and it is therefore invertible; the inverse of $\gamma^{-1}$ is $\gamma$.

## How natural numbers can be constructed from sets

We have mentioned that sets are the "most general" mathematical objects. But then we informally introduced other objects, such as ordered pairs. At first sight, ordered pairs are different from sets. Why do we then say that sets are the "most general" mathematical objects?

In fact, ordered pairs can be expressed as sets. There are several ways to do this, one of them the so-called **Kuratowski's definition** proposed in 1921 by Kazimierz Kuratowski:
$$(a, b) = \{\{a\}, \{a, b\}\}.$$

This definition can be used even when the two elements of the pair are identical:
$$(a, a) = \{\{a\}, \{a, a\}\} = \{\{a\}, \{a\}\} = \{\{a\}\}.$$

Triples can be defined as nested pairs:
$$(a, b, c) = (a, (b, c)),$$
and so on.

But what about *numbers*? Surely there is no way to use sets to define numbers?

In fact, we can define natural numbers and zero as follows:
$$
0 = \{\} = \emptyset, \\
1 = \{0\} = \{\emptyset\}, \\
2 = \{0, 1\} = \{\emptyset, \{\emptyset\}\}, \\
3 = \{0, 1, 2\} = \{\emptyset, \{\emptyset\}, \{\emptyset, \{\emptyset\}\}\}, \\
\vdots
$$

This definition is part of the Zermelo-Fraenkel (ZF) set theory. We can then use the **Dedekind-Peano axioms** to define arithmetic for natural numbers in terms of set theory.

Having constructed $\mathbb{N}$, we can then construct the rationals $\mathbb{Q}$, the reals $\mathbb{R}$, building on the foundation of set theory.

Proceeding onwards, we obtain... the rest of mathematics!


# Realization of Recursive Filters

*This jupyter notebook is part of a [collection of notebooks](../index.ipynb) on various topics of Digital Signal Processing. Please direct questions and suggestions to [Sascha.Spors@uni-rostock.de](mailto:Sascha.Spors@uni-rostock.de).*

## Introduction

Computing the output $y[k] = \mathcal{H} \{ x[k] \}$ of a [linear time-invariant](https://en.wikipedia.org/wiki/LTI_system_theory) (LTI) system is of central importance in digital signal processing. This is often referred to as [*filtering*](https://en.wikipedia.org/wiki/Digital_filter) of the input signal $x[k]$. We already have discussed the realization of [non-recursive filters](../nonrecursive_filters/introduction.ipynb). This section focuses on the realization of recursive filters.

### Recursive Filters

Linear difference equations with constant coefficients represent linear time-invariant (LTI) systems

\begin{equation}
\sum_{n=0}^{N} a_n \; y[k-n] = \sum_{m=0}^{M} b_m \; x[k-m]
\end{equation}

where $y[k] = \mathcal{H} \{ x[k] \}$ denotes the response of the system to the input signal $x[k]$, $N$ the order, $a_n$ and $b_m$ constant coefficients, respectively. Above equation can be rearranged with respect to the output signal $y[k]$ by extracting the first element ($n=0$) of the left-hand sum

\begin{equation}
y[k] = \frac{1}{a_0} \left( \sum_{m=0}^{M} b_m \; x[k-m] - \sum_{n=1}^{N} a_n \; y[k-n] \right)
\end{equation}

It is evident that the output signal $y[k]$ at time instant $k$ is given as a linear combination of past output samples $y[k-n]$ superimposed by a linear combination of the actual $x[k]$ and past $x[k-m]$ input samples. Hence, the actual output $y[k]$ is composed from the two contributions

1. a [non-recursive part](../nonrecursive_filters/introduction.ipynb#Non-Recursive-Filters), and
2. a recursive part where a linear combination of past output samples is fed back.

The impulse response of the system is given as the response of the system to a Dirac impulse at the input $h[k] = \mathcal{H} \{ \delta[k] \}$. Using above result and the properties of the discrete Dirac impulse we get

\begin{equation}
h[k] = \frac{1}{a_0} \left( b_k  - \sum_{n=1}^{N} a_n \; h[k-n] \right)
\end{equation}

Due to the feedback, the impulse response will in general be of infinite length. The impulse response is termed as [infinite impulse response](https://en.wikipedia.org/wiki/Infinite_impulse_response) (IIR) and the system as recursive system/filter.

### Transfer Function

Applying a $z$-transform to the left- and right-hand side of the difference equation and rearranging terms yields the transfer function $H(z)$ of the system

\begin{equation}
H(z) = \frac{Y(z)}{X(z)} = \frac{\sum_{m=0}^{M} b_m \; z^{-m}}{\sum_{n=0}^{N} a_n \; z^{-n}}
\end{equation}

The transfer function is given as a [rational function](https://en.wikipedia.org/wiki/Rational_function) in $z$. The polynominals of the numerator and denominator can be expressed alternatively by their roots as

\begin{equation}
H(z) = \frac{b_M}{a_N} \cdot \frac{\prod_{\mu=1}^{P} (z - z_{0\mu})^{m_\mu}}{\prod_{\nu=1}^{Q} (z - z_{\infty\nu})^{n_\nu}}
\end{equation}

where $z_{0\mu}$ and $z_{\infty\nu}$ denote the $\mu$-th zero and $\nu$-th pole of degree $m_\mu$ and $n_\nu$ of $H(z)$, respectively. The total number of zeros and poles is denoted by $P$ and $Q$. Due to the symmetries of the $z$-transform, the transfer function of a real-valued system $h[k] \in \mathbb{R}$ exhibits complex conjugate symmetry

\begin{equation}
H(z) = H^*(z^*)
\end{equation}

Poles and zeros are either real valued or complex conjugate pairs for real-valued systems ($b_m\in\mathbb{R}$, $a_n\in\mathbb{R}$). For the poles of a causal and stable system $H(z)$ the following condition has to hold

\begin{equation}
\max_{\nu} | z_{\infty\nu} | < 1
\end{equation}

Hence, all poles have to be located inside the unit circle $|z| = 1$. Amongst others, this implies that $M \leq N$.

### Example

The following example shows the pole/zero diagram, the magnitude and phase response, and impulse response of a recursive filter with so-called [Butterworth](https://en.wikipedia.org/wiki/Butterworth_filter) lowpass characteristic.

```python
import numpy as np
import matplotlib.pyplot as plt
from matplotlib.markers import MarkerStyle
from matplotlib.patches import Circle
import scipy.signal as sig

N = 5  # order of recursive filter
L = 128  # number of computed samples

def zplane(z, p, title='Poles and Zeros'):
    "Plots zero and pole locations in the complex z-plane"
    ax = plt.gca()

    ax.plot(np.real(z), np.imag(z), 'bo', fillstyle='none', ms=10)
    ax.plot(np.real(p), np.imag(p), 'rx', fillstyle='none', ms=10)
    unit_circle = Circle((0, 0), radius=1, fill=False,
                         color='black', ls='solid', alpha=0.9)
    ax.add_patch(unit_circle)
    ax.axvline(0, color='0.7')
    ax.axhline(0, color='0.7')

    plt.title(title)
    plt.xlabel(r'Re{$z$}')
    plt.ylabel(r'Im{$z$}')
    plt.axis('equal')
    plt.xlim((-2, 2))
    plt.ylim((-2, 2))
    plt.grid()

# compute coefficients of recursive filter
b, a = sig.butter(N, 0.2, 'low')
# compute transfer function
Om, H = sig.freqz(b, a)
# compute impulse response
k = np.arange(L)
x = np.where(k == 0, 1.0, 0)
h = sig.lfilter(b, a, x)

# plot pole/zero-diagram
plt.figure(figsize=(5, 5))
zplane(np.roots(b), np.roots(a))
# plot magnitude response
plt.figure(figsize=(10, 3))
plt.plot(Om, 20 * np.log10(abs(H)))
plt.xlabel(r'$\Omega$')
plt.ylabel(r'$|H(e^{j \Omega})|$ in dB')
plt.grid()
plt.title('Magnitude response')
# plot phase response
plt.figure(figsize=(10, 3))
plt.plot(Om, np.unwrap(np.angle(H)))
plt.xlabel(r'$\Omega$')
plt.ylabel(r'$\varphi (\Omega)$ in rad')
plt.grid()
plt.title('Phase response')
# plot impulse response (magnitude)
plt.figure(figsize=(10, 3))
plt.stem(20*np.log10(np.abs(np.squeeze(h))), use_line_collection=True)
plt.xlabel(r'$k$')
plt.ylabel(r'$|h[k]|$ in dB')
plt.grid()
plt.title('Impulse response (magnitude)')
```

    Text(0.5, 1.0, 'Impulse response (magnitude)')

**Exercise**

* Does the system have an IIR?
* What happens if you increase the order `N` of the filter?

Solution: It can be concluded from the last illustration, showing the magnitude of the impulse response $|h[k]|$ on a logarithmic scale, that the magnitude of the impulse response decays continuously for increasing $k$ but does not become zero at some point. This behavior continues with increasing $k$ as can be observed when increasing the number `L` of computed samples in above example. The magnitude response $|H(e^{j \Omega})|$ of the filter decays faster with increasing order `N` of the filter.

**Copyright**

This notebook is provided as [Open Educational Resource](https://en.wikipedia.org/wiki/Open_educational_resources). Feel free to use the notebook for your own purposes. The text is licensed under [Creative Commons Attribution 4.0](https://creativecommons.org/licenses/by/4.0/), the code of the IPython examples under the [MIT license](https://opensource.org/licenses/MIT). Please attribute the work as follows: *Sascha Spors, Digital Signal Processing - Lecture notes featuring computational examples*.


# Optimización de funciones escalares diferenciables con `SymPy`

> - Mediante optimización se obtienen soluciones elegantes tanto en teoría como en ciertas aplicaciones.
> - La teoría de optimización usa elementos comenzando con cálculo elemental y álgebra lineal básica, y luego se extiende con análisis funcional y convexo.
> - Las aplicaciones en optimización involucran ciencia, ingeniería, economía, finanzas e industria.
> - El amplio y creciente uso de la optimización lo hace escencial para estudiantes y profesionales de cualquier rama de la ciencia y la tecnología.

**Referencia:**
- http://www.math.uwaterloo.ca/~hwolkowi//henry/reports/talks.d/t06talks.d/06msribirs.d/optimportance.shtml

Algunas aplicaciones son:

1. Ingeniería
 - Encontrar la composición de equilibrio de una mezcla de diferentes átomos.
 - Planeación de ruta para un robot (o vehículo aéreo no tripulado).
 - Planeación de la mano de obra óptima en una construcción o planta de producción.
2. Distribución óptima de recursos.
 - Distribución de rutas de vuelo.
 - Encontrar una dieta óptima.
 - Planeación de ruta óptima.
3. Optimización financiera
 - Administración de riesgos.
 - Portafolios de inversión.

En esta clase veremos aspectos básicos de optimización. En específico, veremos cómo obtener máximos y mínimos de una función escalar de una variable (como en cálculo diferencial).

___
## 0. Librerías que usaremos

Como ya dijimos en la primer clase `python` es el lenguaje de programación (el cual es de alto nivel). Sin embargo, `python` solo tiene unos pocos comandos primitivos y para hacer más fácil su uso en nuestras actividades de simulación en ingeniería, otras personas ya han escrito ciertas librerías por nosotros.

### 0.1 `NumPy`

`NumPy` (Numerical Python) es la librería fundamental para computación científica (numérica) con `Python`. Contiene, entre otras cosas:
- un objeto tipo arreglo N-dimensional muy poderoso
- funciones sofisticadas
- funciones de álgebra lineal, transformada de Fourier y números aleatorios.

Por lo anterior, `NumPy` es de amplio uso entre la comunidad científica e ingenieril (por su manejo de cantidades vectoriales). De la misma manera, se usa para guardar datos. Para nuestros propósitos, se puede usar libremente.

**Referencia:**
- http://www.numpy.org/

`NumPy` ya viene incluido en la instalación estándar de Anaconda por defecto. Para comenzar a usarlo, solo debemos de importarlo:

```python
# importar la librería numpy

```

### 0.2 `SymPy`

`SymPy` (Symbolic Python) es una librería de `Python` para matemáticas simbólicas. Su objetivo es convertirse en un sistema de álgebra computacional con las mejores características, manteniendo el código lo más simple posible para que sea comprensible.

**Referencia:**
- http://www.sympy.org/en/index.html

`SymPy` ya viene incluido en la instalación estándar de Anaconda por defecto. Para comenzar a usarlo, solo debemos de importarlo:

```python
# importar la librería sympy

# imprimir en formato latex

```

La funcionalidad de imprimir en formato LaTex que nos da `SymPy` mediante el proyecto `mathjax` hace de `SymPy` una herramienta muy atractiva...

Notar que en `SymPy` y en `NumPy` existen funciones con el mismo nombre, pero reciben tipos de datos diferentes...

```python
# diferencias de funciones de sympy y numpy

```

Explicar el uso de la sintaxis `from numpy import *` y sus peligros (no recomendable).

```python
# importar con * y ver que pasa
# from numpy import *
# from sympy import *
# No recomendado
```

### 0.3 `PyPlot` de `matplotlib`

El módulo `PyPlot` de la librería `matplotlib` contiene funciones que nos permite generar una gran cantidad de gráficas rápidamente. Las funciones de este módulo están escritas con el mismo nombre que las funciones para graficar en `Matlab`.

**Referencia:**
- https://matplotlib.org/api/pyplot_summary.html

```python
# importar matplotlib.pyplot

# comando para que las gráficas salgan en la misma ventana

```

Ya que revisamos todas las librerías que usaremos, empecemos con la clase como tal...

___
Basamos todos los resultados en los siguientes teoremas:

## 1. Teorema de Fermat (análisis)

Si una función $f(x)$ alcanza un máximo o mínimo local en $x=c$, y si la derivada $f'(c)$ existe en el punto $c$, entonces $f'(c) = 0$.

### Ejemplo

Sabemos que la función $f(x)=x^2$ tiene un mínimo global en $x=0$, pues

$$f(x)=x^2\geq0,\qquad\text{y}\qquad f(x)=x^2=0 \qquad\text{si y solo si}\qquad x=0.$$

```python
# declarar la variable real x

```

```python
# declarar ahora f=x^2 y mostrar

```

```python
# derivar f respecto a x y mostrar

```

```python
# resolver f'(x)=0 y mostrar soluciones

```

Veamos la gráfica...

```python
# convertir f e una función que se pueda evaluar numéricamente (función lambdify de la librería sympy)

# graficar

```

Ver diferencias entre f y f_num

```python
# intentar evaluar f y f_num

```

**Otra manera de hacer lo anterior**

Concepto de función...

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

El converso del teorema anterior no es cierto.

### Actividad
Considere $g(x)=x^3$.
- Usando `sympy`, muestre que $g'(0)=0$.
- Sin embargo, descartar que $x=0$ es un extremo de $g(x)$ viendo su **gráfica**.

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

## 2. Criterio de la segunda derivada

Sea $f(x)$ una función tal que $f’(c)=0$ y cuya segunda derivada existe en un intervalo abierto que contiene a $c$.
- Si $f’’(c)>0$, entonces $f(c)$ es un mínimo relativo.
- Si $f’’(c)<0$, entonces $f(c)$ es un máximo relativo.
- Si $f’’(c)=0$, entonces el criterio no decide.

### Ejemplo

Mostrar, usando `sympy`, que la función $f(x)=x^2$ tiene un mínimo relativo en $x=0$.

Ya vimos que $f'(0)=0$. Notemos que:

```python

```

```python

```

```python

```

Por tanto, por el criterio de la segunda derivada, $f(0)=0$ es un mínimo relativo (en efecto, el mínimo global).

### Ejemplo
¿Qué pasa con $g(x)=x^3$ al intentar utilizar el criterio de la segunda derivada? (usar `sympy`).

```python

```

```python

```

```python

```

### Actividad

¿Qué pasa con $h(x)=x^4$ al intentar utilizar el criterio de la segunda derivada?.

```python
def h(x):
    return x**4
```

```python
dh = sym.diff(h(x), x, 1)
dh
```

$$4 x^{3}$$

```python
p_c_h = sym.solve(dh, x)
p_c_h
```

$$\left [ 0\right ]$$

```python
d2h = sym.diff(h(x), x, 2)
d2h
```

$$12 x^{2}$$

```python
d2h.subs(x, p_c_h[0])
```

$$0$$

Como la segunda derivada en el punto crítico es cero, el criterio de la segunda derivada no decide.

## 3. Método para determinar extremos absolutos de una función continua y=f(x) en [a,b]

- Determinar todos los valores críticos $c_1, c_2, c_3, \dots, c_n$ en $(a,b)$.
- Evaluar $f$ en todos los valores críticos y en los extremos $x=a$ y $x=b$.
- El más grande y el más pequeño de los valores de la lista $f(a), f(b), f(c_1), f(c_2), \dots, f(c_n)$ son el máximo absoluto y el mínimo absoluto, respectivamente, de f en el intervalo [a,b].

### Ejemplo

Determinar los extremos absolutos de $f(x)=x^2-6x$ en $\left[0,5\right]$.

Obtenemos los puntos críticos de $f$ en $\left[0,5\right]$:

```python
f = x**2-6*x
f
```

$$x^{2} - 6 x$$

```python
df = sym.diff(f, x)
df
```

$$2 x - 6$$

```python
x_c = sym.solve(df, x)
x_c
```

$$\left [ 3\right ]$$

Evaluamos $f$ en los extremos y en los puntos críticos:

```python
f.subs(x, 0), f.subs(x, 5), f.subs(x, x_c[0])
```

$$\left ( 0, \quad -5, \quad -9\right )$$

Concluimos que el máximo absoluto de $f$ en $\left[0,5\right]$ es $0$ y se alcanza en $x=0$, y que el mínimo absoluto es $-9$ y se alcanza en $x=3$.

```python
f_num = sym.lambdify([x], f, 'numpy')
x_vec = np.linspace(0, 5, 100)

plt.figure(figsize=(8,6))
plt.plot(x_vec, f_num(x_vec), 'k--', label = '$y=f(x)$')
plt.plot([0], [0], '*r', label = '$(0,0=\max_{0\leq x\leq 5} f(x))$')
plt.plot([3], [-9], '*b', label = '$(3,-9=\min_{0\leq x\leq 5} f(x))$')
plt.legend(loc='best')
plt.xlabel('x')
plt.show()
```

### Actividad
Determinar los valores extremos absolutos de $h(x)=x^3-3x$ en $\left[-2.2,1.8\right]$, usando `sympy`. Mostrar en una gráfica.

```python
sym.var('x', real = True)
```

$$x$$

```python
def h(x):
    return x**3-3*x
```

```python
dh = sym.diff(h(x), x)
dh
```

$$3 x^{2} - 3$$

```python
p_c = sym.solve(dh, x)
p_c
```

$$\left [ -1, \quad 1\right ]$$

```python
h(-1), h(1), h(-2.2), h(1.8)
```

$$\left ( 2, \quad -2, \quad -4.048000000000003, \quad 0.4320000000000004\right )$$

```python
x_vec = np.linspace(-2.2, 1.8, 50)

plt.figure(figsize=(7,5))
plt.plot(x_vec, h(x_vec), 'b:', label = '$y=x^3-3x$')
plt.plot([-2.2], [h(-2.2)], 'mo', label = '(-2.2,-4.048)')
plt.plot([-1], [h(-1)], 'yo', label = '(-1,2)')
plt.legend(loc='best')
plt.xlabel('$x$')
plt.ylabel('$y$')
plt.grid()
plt.show()
```

### En varias variables...

El procedimiento es análogo.

Si una función $f:\mathbb{R}^n\to\mathbb{R}$ alcanza un máximo o mínimo local en $\boldsymbol{x}=\boldsymbol{c}\in\mathbb{R}^n$, y $f$ es diferenciable en el punto $\boldsymbol{x}=\boldsymbol{c}$, entonces $\left.\frac{\partial f}{\partial \boldsymbol{x}}\right|_{\boldsymbol{x}=\boldsymbol{c}}=\boldsymbol{0}$ (todas las derivadas parciales en el punto $\boldsymbol{x}=\boldsymbol{c}$ son cero).

**Criterio de la segunda derivada:** para ver si es máximo o mínimo, se toma la segunda derivada (matriz jacobiana) y se verifica definición negativa o positiva, respectivamente.

Si se restringe a cierta región, hay ciertas técnicas. La más general, pero también la más compleja es la de **multiplicadores de Lagrange**.

```python
sym.var('x y')
x, y
```

$$\left ( x, \quad y\right )$$

```python
def f(x, y):
    return x**2 + y**2
```

```python
dfx = sym.diff(f(x,y), x)
dfy = sym.diff(f(x,y), y)
dfx, dfy
```

$$\left ( 2 x, \quad 2 y\right )$$

```python
xy_c = sym.solve([dfx, dfy], [x, y])
xy_c
```

$$\left \{ x : 0, \quad y : 0\right \}$$

```python
x_c, y_c = xy_c[x], xy_c[y]
x_c, y_c
```

$$\left ( 0, \quad 0\right )$$

```python
d2fx = sym.diff(f(x,y), x, 2)
d2fy = sym.diff(f(x,y), y, 2)
dfxy = sym.diff(f(x,y), x, y)

Jf = sym.Matrix([[d2fx, dfxy], [dfxy, d2fy]])
Jf.eigenvals()
```

$$\left \{ 2 : 2\right \}$$

```python
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
```

```python
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')

x = np.linspace(-2, 2, 100)
y = x
X, Y = np.meshgrid(x, y)

ax.plot_surface(X, Y, f(X, Y))
ax.plot([x_c], [y_c], [f(x_c,y_c)], '*r')
```

### Tareas.

1. Por parejas harán un proyecto colaborativo. En moodle subiré como hacerlo paso a paso y qué es lo que se debe entregar. Quien no tenga pareja, acercarse a mi para realizarlo (no se acepta como excusa que no tenía pareja).

2. Introducción a numpy. Obtener máximos y mínimos absolutos, usando `sympy`, de las funciones dadas en los intervalos dados, y graficar la función en dicho intervalo señalando los puntos máximo y mínimo absolutos.

**Definir fecha de las tareas**

**Curso moodle: Simulación matemática | Esteban Jiménez Rodríguez | P. 2018**
**Contraseña moodle: simmat2018**

**Recordar tarea para hoy y recuento de la clase**

<footer id="attribution" style="float:right; color:#808080; background:#fff;">
Created with Jupyter by Esteban Jiménez Rodríguez.
</footer>