# Algorithmic Control Design for Oscillator-Like Systems

In this tutorial, we will use ``kontrol.regulator.oscillator`` submodule to make control regulators for oscillator-like systems.

``kontrol.regulator.oscillator.pid_oscillator()`` is a function that makes PID-like controllers algorithmically based on simple critiera:

1. It returns a derivative controller that critically damps the osciallator.
2. Then it adds proportional/integral (or both) controller such that they match the first UGF of the derivative control. Alternatively, DC gain or integrator time constants can be specified to override the function.

In this tutorial, we will make PID, PD, D control regulators to control the plant

\begin{equation}
P(s) = G_1\frac{\omega_1^2}{s^2 + \omega_1/Q_1 s + \omega_1^2} + G_2\frac{\omega_2^2}{s^2 + \omega_2/Q_2 s + \omega_2^2}\,,
\end{equation}
where $G$s are DC gains, $\omega$s are resonance frequencies, and $Q$s are quality factors.

The regulator is defined as

\begin{equation}
K_\mathrm{PID}(s) = K_p + K_i/s + K_d s\,,
\end{equation}
where $K_p$, $K_i$, and $K_d$ are the proportional, integral, and derivative gains respectively.

```python
import numpy as np
import control
import matplotlib.pyplot as plt
import kontrol.regulator

## define the plant here
s = control.tf("s")
g1 = 100
w1 = 1
q1 = 100
g2 = 10
w2 = 3
q2 = 50

plant = g1*w1**2 / (s**2 + w1/q1*s + w1**2) + g2*w2**2 / (s**2 + w2/q2*s + w2**2)

f = np.logspace(-2, 1, 1000)
plt.figure(figsize=(6, 4))
plt.subplot(211, title="Bode plot")
plt.loglog(f, abs(plant(1j*2*np.pi*f)), label="Plant")
plt.ylabel("Amplitude")
plt.xlabel("Frequency (Hz)")
plt.legend(loc=0)
plt.grid(which="both")

plt.subplot(212)
plt.semilogx(f, np.angle(plant(1j*2*np.pi*f)), label="Plant")
plt.ylabel("Phase (rad)")
plt.xlabel("Frequency (Hz)")
plt.legend(loc=0)
plt.grid(which="both")
```

```python
regulator_pid = kontrol.regulator.oscillator.pid(plant, regulator_type="PID")
regulator_pd = kontrol.regulator.oscillator.pid(plant, regulator_type="PD")
# regulator_pi = kontrol.regulator.oscillator.pid(plant, regulator_type="PI")
# regulator_i = kontrol.regulator.oscillator.pid(plant, regulator_type="I")
regulator_d = kontrol.regulator.oscillator.pid(plant, regulator_type="D")

oltf_pid = regulator_pid * plant
oltf_pd = regulator_pd * plant
# oltf_pi = regulator_pi * plant
# oltf_i = regulator_i * plant
oltf_d = regulator_d * plant

plt.title("Open Loop Transfer Function")
plt.loglog(f, abs(oltf_pid(1j*2*np.pi*f)), label="PID control")
plt.loglog(f, abs(oltf_pd(1j*2*np.pi*f)), label="PD control")
# plt.loglog(f, abs(oltf_pi(1j*2*np.pi*f)), label="PI control")
# plt.loglog(f, abs(oltf_i(1j*2*np.pi*f)), label="I control")
plt.loglog(f, abs(oltf_d(1j*2*np.pi*f)), label="D control")
plt.hlines(1, min(f), max(f), color="k", ls="--", label="Unity Gain")
plt.legend(loc=0)
plt.grid(which="both")
plt.ylabel("Magnitude")
plt.xlabel("Frequency (Hz)")
```

```python
t = np.linspace(0, 64, 1024)

# u = np.zeros_like(t)
u = t > 32
u = u.astype(float)

cltf_pid = oltf_pid / (1+oltf_pid)
cltf_pd = oltf_pd / (1+oltf_pd)
# cltf_pi = oltf_pi / (1+oltf_pi)
# cltf_i = oltf_i / (1+oltf_i)
cltf_d = oltf_d / (1+oltf_d)

_, step_plant = control.forced_response(plant, U=u, T=t)
_, step_pid = control.forced_response(cltf_pid, U=u, T=t)
_, step_pd = control.forced_response(cltf_pd, U=u, T=t)
# _, step_pi = control.forced_response(cltf_pi, U=u, T=t)
# _, step_i = control.forced_response(cltf_i, U=u, T=t)
_, step_d = control.forced_response(cltf_d, U=u, T=t)
plt.title("Step response")

# plt.plot(t, step_plant, label="Plant")
plt.plot(t, step_pid, label="Closed loop, PID control")
plt.plot(t, step_pd, label="Closed loop, PD control")
# plt.plot(t, step_pi, label="Closed loop, PI control")
# plt.plot(t, step_i, label="Closed loop, I control")
plt.plot(t, step_d, label="Closed loop, D control")
plt.plot(t, u, "k--", label="Setpoint")

plt.legend(loc=0)
plt.xlabel("Time (s)")
plt.show()
```


# Deep probabilistic models with applications in astronomy

We will review the basics of inference with probabilistic models, the more recent "deep" probabilistic models and how to implement them using the [Pyro probabilistic programming library](http://docs.pyro.ai/en/stable/getting_started.html). After that we will have a hands-on experience training probabilistic models to analyze time series from astronomical survey projects.

- This notebook lives at: https://github.com/phuijse/EVIC2020
    - git clone https://github.com/phuijse/EVIC2020
    - conda env create -f environment.yml
- [Open in google colab](https://colab.research.google.com/drive/1-RfsaAUnQ6foX6yGHGbbp_e_P5zR1dVv?usp=sharing)
- [More material and code examples on this topic](https://github.com/magister-informatica-uach/INFO320)
- [Lecture slides on neural networks (in spanish)](https://docs.google.com/presentation/d/1IJ2n8X4w8pvzNLmpJB-ms6-GDHWthfsJTFuyUqHfXg8/edit?usp=sharing)

Author: Pablo Huijse, phuijse at inf dot uach dot cl

```python
%matplotlib notebook
import matplotlib.pyplot as plt
import numpy as np
from IPython.display import YouTubeVideo
from tqdm.notebook import tqdm
import corner

import torch
import pyro
print(f"Torch version: {torch.__version__}")
print(f"Pyro version: {pyro.__version__}")

%load_ext autoreload
%autoreload 2
from astro_utils import plot_params, plot_lc, plot_lc_folded, featurize_lc, plot_lc_features, make_train_plots
```

# Pyro basics

Example: Bayesian linear regression

$$
y_i = w x_i + b +  \epsilon_i
$$

with $N$ observations $(x_i, y_i)$, Gaussian noise and Gaussian priors for $w$ and  $b$

The generative process in this case is

- Sample $w \sim \mathcal{N}(\mu_w, \sigma_w^2)$
- Sample $b \sim \mathcal{N}(\mu_b, \sigma_b^2)$
- For $i=1,2,\ldots, N$
    - Sample $y_i \sim \mathcal{N}(w x_i + b, \sigma_\epsilon^2)$

where $\mu_w, \sigma_w, \mu_b, \sigma_b, \sigma_\epsilon$ are hyperparameters

## Writing a model

To write the model we use the submodules and primitives

- `pyro.distributions` to define prior/likelihoods
- `pyro.sample` to define random variables (RV): Expects name, distribution and optionally observations
- `pyro.plate` for conditionally independent RV: Expects name, and size

```python
from pyro.distributions import Normal, Uniform

def model(x, y=None):
    w = pyro.sample("w", Normal(0.0, 1.0))
    b = pyro.sample("b", Normal(0.0, 1.0))
    s = pyro.sample("s", Uniform(0.0, 1.0))
    with pyro.plate('dataset', size=len(x)):
        return pyro.sample("y", Normal(x*w + b, s), obs=y)
```

We can use

- `pyro.infer.Predictive`

to draw samples from the model

```python
predictive = pyro.infer.Predictive(model, num_samples=500)

hatx = np.linspace(-6, 6, num=100).astype('float32')
apriori_samples = predictive(torch.from_numpy(hatx))

# Plot samples from the priors
figure = corner.corner(np.stack([apriori_samples[var].detach().numpy()[:, 0] for var in ['b', 'w', 's']]).T,
                       smooth=1., labels=["bias", "weight", "noise_std"], bins=20,
                       quantiles=[0.16, 0.5, 0.84],
                       show_titles=True, title_kwargs={"fontsize": 12})

# Plot posterior predictive of y given x
y_trace = apriori_samples["y"].detach().numpy()
med = np.median(y_trace, axis=0)
qua = np.quantile(y_trace, (0.05, 0.95), axis=0)

fig, ax = plt.subplots(figsize=(7, 3), tight_layout=True)
ax.plot(hatx, y_trace.T);
```

## Inference

In the bayesian setting we want the posterior distribution

$$
p(\theta | \mathcal{D}) = \frac{p(\mathcal{D}|\theta) p(\theta)}{\int_\theta p(\mathcal{D}|\theta) p(\theta)}
$$

where $\mathcal{D}$ is our dataset and $\theta = (w, b)$

For complex models the posterior is intractable. So we either do

- MCMC: Train a Markov chain to generate samples as if they came from the actual posterior: Sampling based
- Variational Inference: Choose a more simple posterior that is similar to the actual posterior: Optimization based

### Variational Inference

Propose an approximate (simple) posterior $q_\phi(\theta)$, e.g. factorized Gaussian

Optimize $\phi$ so that $q_\phi$ approximates $p(\theta|\mathcal{D})$

This is typically done by maximizing a lower bound on the evidence

$$
\mathcal{ELBO}(\phi) = \mathbb{E}_{\theta \sim q_\phi}[ \log p(\mathcal{D}|\theta)] - \text{KL}[q_\phi(\theta)|p(\theta)]
$$

- Maximize the likelihood of the model
- Minimize the distance between the approximate posterior and the prior

Once $q$ has been trained we use it as a replacement for $p(\theta|\mathcal{D})$ to calculate the **posterior predictive distribution**

$$
p(\mathbf{y}|\mathbf{x}, \mathcal{D}) = \int p(\mathbf{y}|\mathbf{x}, \theta) p(\theta| \mathcal{D}) \,d\theta
$$

## VI with Pyro

We use `pyro.infer.SVI` to perform **Stochastic Variational Inference**, which expects

- A generative model
- An approximate posterior (guide)
- Cost function: Typically ELBO
- Optimizer: How to optimize the ELBO, typically gradient descent based

We can use the `pyro.infer.autoguide` to create approximate posteriors from predefined recipes, for example a factorized Gaussian posterior (`AutoDiagonalNormal`)

```python
pyro.enable_validation(True) # Useful for debugging
pyro.clear_param_store()

# Create a guide (approximate posterior)
from pyro.infer.autoguide import AutoDiagonalNormal as approx_posterior
guide = approx_posterior(model)

# Stochastic Variational Inference
svi = pyro.infer.SVI(model=model,
                     guide=guide,
                     loss=pyro.infer.Trace_ELBO(),
                     optim=pyro.optim.ClippedAdam({"lr": 1e-2}))
```

Lets consider a dataset with two observations $\mathcal{D} = \{ (-2, -2), (2, 2) \}$

`svi.step(x, y)` performs a gradient ascent step to maximize the ELBO

```python
fig, ax = plt.subplots(1, 3, figsize=(10, 2.5), dpi=80, tight_layout=True)

nepochs = 1000
loss = np.zeros(shape=(nepochs, ))
params = np.zeros(shape=(nepochs, 2, 3))

# Observed data
x = torch.tensor([-2., 2.])
y = torch.tensor([-2., 2.])

for k in tqdm(range(nepochs)):
    loss[k] = svi.step(x, y)

    phi = [param.detach().numpy() for param in guide.parameters()]
    params[k, 0, :] = phi[0] # Locations
    params[k, 1, :] = phi[1] # Scales
    if np.mod(k, 10) == 0:
        plot_params(ax, k+1, loss, params)
        fig.canvas.draw()
```

Again we can use `pyro.infer.Predictive` to draw samples from the model

This time we use the guide to sample $w$ and $b$

```python
predictive = pyro.infer.Predictive(model,
                                   guide=guide,
                                   num_samples=1000)

posterior_samples = predictive(torch.from_numpy(hatx))

# Plot posterior of w,  b and s
figure = corner.corner(np.stack([posterior_samples[var].detach().numpy()[:, 0] for var in ['b', 'w', 's']]).T,
                       smooth=1., labels=["bias", "weight", "noise_std"], bins=20,
                       quantiles=[0.16, 0.5, 0.84],
                       show_titles=True, title_kwargs={"fontsize": 12})

# Plot posterior predictive of y given x
y_trace = posterior_samples["y"].detach().numpy()
med = np.median(y_trace, axis=0)
qua = np.quantile(y_trace, (0.05, 0.95), axis=0)

fig, ax = plt.subplots(figsize=(7, 3), tight_layout=True)
ax.plot(hatx, med)
ax.fill_between(hatx, qua[0], qua[1], alpha=0.5);

ax.errorbar(x.numpy(), y.numpy(), yerr=2*posterior_samples['s'].median().item(),
           fmt='none', c='k', zorder=100);

```

# PyTorch basics

Artificial Neuron: Linear regressor plus activation function (e.g. sigmoid -> Logistic regression)

$$
\hat y = g\left(\sum_i w_i x_i + b\right)
$$

Multilayer perceptron: Several fully connected layers concatenated

$$
\begin{align}
f_i &=   b_i + \sum_{j=1}^H w_{ij} h_j  \nonumber \\
&=  b_i + \sum_{j=1}^H w_{ij} g \left( b_j + \sum_{d=1}^D w_{jd} x_d  \right) \nonumber
\end{align}
$$

Neural network model in pytorch

- We write a class that inherits from `torch.nn.Module`
- The constructor `__init__(self, args):` define the layers, e.g. fully-connected (`Linear`), convolutional (`Conv1D`, `Conv2D`)
- The function `forward(self, x):` defines how layers are connected

```python
class MultiLayerPerceptron(torch.nn.Module):

    def __init__(self, D, H, O):
        super(type(self), self).__init__()
        self.hidden_layer = torch.nn.Linear(D, H) # WX + B
        self.output_layer = torch.nn.Linear(H, O)
        self.activation = torch.nn.Sigmoid()

    def forward(self, x):
        z = self.activation(self.hidden_layer(x))
        return self.output_layer(z)
```

# Variational Autoencoder (VAE) [(Kingma et al., 2014)](https://arxiv.org/abs/1312.6114)

Non-linear latent variable model

- Latent variables: $z \sim \mathcal{N}(0, I)$
- Observed variable: $x|z \sim \mathcal{N}(\hat \mu, \hat \sigma^2)$

We use a neural network $f_\theta$ to model a non-linear function $\hat \mu (z)$

We want to infer the latent given the observed

$$
p(z|x) = \frac{p(x|z) p(z)}{\int p(x|z) p(z) dz}
$$

But this is not tractable so we use a variational approximation (factorized Gaussian)

$$
\begin{align}
q_\phi(z|x) &= \mathcal{N}(\mu(x), \sigma(x)^2) \nonumber \\
&= \mu(x) + \sigma(x) \epsilon, \quad \epsilon \sim \mathcal{N}(0, I) \nonumber
\end{align}
$$

where a neural network $g_\phi$ are used to model the non-linear functions $\mu(x)$ and $\sigma(x)$

We amortize the parameters of $q$ using the neural network, making inference very efficient

Note: The VAE is not a full bayesian neural network, the parameters of the neural net are point estimates

```python
from pyro.distributions import Normal
import torch.nn as nn

class Encoder(torch.nn.Module):
    def __init__(self, data_dim, latent_dim, hidden_dim):
        super(type(self), self).__init__()
        self.hidden = nn.Linear(data_dim, hidden_dim)
        self.z_loc = nn.Linear(hidden_dim, latent_dim)
        self.z_scale = nn.Linear(hidden_dim, latent_dim)
        self.activation = nn.Softplus()

    def forward(self, x):
        h = self.activation(self.hidden(x))
        return self.z_loc(h), self.activation(self.z_scale(h))

class Decoder(torch.nn.Module):
    def __init__(self, data_dim, latent_dim, hidden_dim):
        super(type(self), self).__init__()
        self.hidden = nn.Linear(latent_dim, hidden_dim)
        self.x_loc = nn.Linear(hidden_dim, data_dim)
        self.activation = nn.Softplus()

    def forward(self, z):
        h = self.activation(self.hidden(z))
        return self.x_loc(h)

class VariationalAutoEncoder(torch.nn.Module):

    def __init__(self, data_dim, latent_dim, hidden_dim, sigmax):
        super(VariationalAutoEncoder, self).__init__()
        self.encoder = Encoder(data_dim, latent_dim, hidden_dim)
        self.decoder = Decoder(data_dim, latent_dim, hidden_dim)
        self.latent_dim = latent_dim
        self.data_dim = data_dim
        self.sigmax = sigmax

    def model(self, x):
        pyro.module("decoder", self.decoder)
        with pyro.plate("data", size=x.shape[0]):
            # p(z)
            z_loc = torch.zeros(x.shape[0], self.latent_dim, device=x.device)
            z_scale = torch.ones(x.shape[0], self.latent_dim, device=x.device)
            z = pyro.sample("latent", Normal(z_loc, z_scale).to_event(1))
            # p(x|z)
            x_loc = self.decoder.forward(z)
            pyro.sample("observed", Normal(x_loc, self.sigmax).to_event(1), obs=x)
            return x_loc

    def guide(self, x):
        pyro.module("encoder", self.encoder)
        with pyro.plate("data", size=x.shape[0]):
            # q(z|x)
            z_loc, z_scale  = self.encoder.forward(x)
            pyro.sample("latent", Normal(z_loc, z_scale).to_event(1))
```

# Astronomical time series

- **Light curve:** Time series of a star's flux (brightness)
- We will consider two optical passbands
- The "apparent" brightness is estimated through **Photometry**
- Main tool for variable star studies

```python
import bz2
import pickle

with bz2.BZ2File("lcdata.pbz2", 'r') as f:
    lcs, periods, labels = pickle.load(f)

fig, ax = plt.subplots(figsize=(7, 3), tight_layout=True)
plot_lc(ax, lcs[0])
```

- Light curve "features": Irregular sampling, gaps in observations, [heteroscedastic](https://en.wikipedia.org/wiki/Heteroscedasticity) noise
- **Variable stars**: Brightness change in time either regularly or stochastically
- Some variable stars are radial pulsators. They expand/heat and contract/cool regularly. Examples: RR Lyrae and Cepheid

```python
YouTubeVideo('sXJBrRmHPj8')
```

If we know the pulsation period we can use the **epoch folding transformation** to obtain a phase diagram

```python
def fold(time, period):
    """
    returns phase = time/period - floor(time/period)
    """
    return np.mod(time, period)/period

fig, ax = plt.subplots(figsize=(7, 3), tight_layout=True)
plot_lc_folded(ax, lcs[0], periods[0])
```

# Training a VAE for periodic light curves

Very simple preprocessing and data preparation routine

```python
pha_interp = np.linspace(0, 1, num=40)
mag_interp = np.zeros(shape=(len(lcs), 2, len(pha_interp)))
err_interp = np.zeros(shape=(len(lcs), 2, len(pha_interp)))

for k, (lc, period) in enumerate(zip(lcs, periods)):
    mag_interp[k], err_interp[k], stats = featurize_lc(lc, period, pha_interp)

fig, ax = plt.subplots(figsize=(7, 3), tight_layout=True)
plot_lc_features(ax, pha_interp, mag_interp[0], err_interp[0])
```

```python
from sklearn.preprocessing import LabelEncoder
from torch.utils.data import TensorDataset, DataLoader, Subset

# reproducibility
torch.manual_seed(12345);
np.random.seed(12345)

fig, ax = plt.subplots(figsize=(7, 2), tight_layout=True)
ax.hist(labels, bins=np.arange(-0.5, 5.5, step=1), rwidth=0.8);

le = LabelEncoder()
labels_int = le.fit_transform(labels)
# Create light curve dataset from numpy arrays
lc_dataset = TensorDataset(torch.from_numpy(mag_interp.astype('float32')),
                           torch.from_numpy(err_interp.astype('float32'))
                           )

# Generate data loaders
idx = np.random.permutation(len(lcs))
train_loader = DataLoader(dataset=Subset(lc_dataset, idx[len(lcs)//5:]),
                          batch_size=32, shuffle=True)

valid_loader = DataLoader(dataset=Subset(lc_dataset, idx[:len(lcs)//5]), batch_size=512)
```

Define a VAE that receives two inputs (r and g) and generates two outputs

```python
from pyro.distributions import Normal
import torch.nn as nn

class VariationalAutoEncoder(torch.nn.Module):

    def __init__(self, data_dim, latent_dim, hidden_dim):
        super(VariationalAutoEncoder, self).__init__()
        self.encoder = Encoder(data_dim, latent_dim, hidden_dim)
        self.decoder = Decoder(data_dim, latent_dim, hidden_dim)
        self.latent_dim = latent_dim
        self.data_dim = data_dim

    def model(self, mags, errs):
        pyro.module("decoder", self.decoder)
        with pyro.plate("minibatch", size=mags.shape[0]):
            # p(z)
            z_loc = torch.zeros(mags.shape[0], self.latent_dim, device=mags.device)
            z_scale = torch.ones(mags.shape[0], self.latent_dim, device=mags.device)
            z = pyro.sample("latent", Normal(z_loc, z_scale).to_event(1))
            # p(x|z)
            x_loc1, x_loc2 = self.decoder.forward(z)
            pyro.sample("observed_g", Normal(x_loc1, errs[:, 0, :]).to_event(1), obs=mags[:, 0, :])
            pyro.sample("observed_r", Normal(x_loc2, errs[:, 1, :]).to_event(1), obs=mags[:, 1, :])
            return x_loc1, x_loc2

    def guide(self, mags, errs):
        pyro.module("encoder", self.encoder)
        with pyro.plate("minibatch", size=mags.shape[0]):
            # q(z|x)
            z_loc, z_scale  = self.encoder.forward(mags)
            pyro.sample("latent", Normal(z_loc, z_scale).to_event(1))
```

```python
class Encoder(torch.nn.Module):
    def __init__(self, data_dim, latent_dim, hidden_dim):
        super(type(self), self).__init__()
        self.hidden1_g = nn.Linear(data_dim, hidden_dim)
        self.hidden1_r = nn.Linear(data_dim, hidden_dim)
        self.hidden2 = nn.Linear(hidden_dim*2, hidden_dim)
        self.z_loc = nn.Linear(hidden_dim, latent_dim)
        self.z_scale = nn.Linear(hidden_dim, latent_dim)
        self.activation = nn.Softplus()

    def forward(self, x):
        hg = self.activation(self.hidden1_g(x[:, 0, :]))
        hr = self.activation(self.hidden1_r(x[:, 1, :]))
        h = torch.cat([hg, hr], dim=1)
        h = self.activation(self.hidden2(h))
        return self.z_loc(h), self.activation(self.z_scale(h))

class Decoder(torch.nn.Module):
    def __init__(self, data_dim, latent_dim, hidden_dim):
        super(type(self), self).__init__()
        self.hidden1 = nn.Linear(latent_dim, hidden_dim)
        self.hidden2 = nn.Linear(hidden_dim, hidden_dim)
        self.x_locg = nn.Linear(hidden_dim, data_dim)
        self.x_locr = nn.Linear(hidden_dim, data_dim)
        self.activation = nn.Softplus()

    def forward(self, z):
        h = self.activation(self.hidden1(z))
        h = self.activation(self.hidden2(h))
        return self.x_locg(h), self.x_locr(h)
```

Model and SVI object definitions and training loop

```python
pyro.enable_validation(True)
pyro.clear_param_store()

vae = VariationalAutoEncoder(data_dim=40, latent_dim=2, hidden_dim=20).cuda()

svi = pyro.infer.SVI(model=vae.model,
                     guide=vae.guide,
                     optim=pyro.optim.Adam({"lr": 1e-3}),
                     loss=pyro.infer.Trace_ELBO())

fig, ax = plt.subplots(1, 2, figsize=(8, 3), tight_layout=True)

nepochs = 100
epoch_loss = np.zeros(shape=(nepochs, 2))
for nepoch in tqdm(range(nepochs)):

    # Training loop
    for mags, errs in train_loader:
        epoch_loss[nepoch, 0] += svi.step(mags.cuda(), errs.cuda())
    epoch_loss[nepoch, 0] /= len(train_loader.dataset)

    # Validation loop
    for mags, errs in valid_loader:
        epoch_loss[nepoch, 1] += svi.evaluate_loss(mags.cuda(), errs.cuda())
    epoch_loss[nepoch, 1] /= len(valid_loader.dataset)

    # Plot latent space and learning curves
    Z = torch.cat((vae.encoder(lc_dataset.tensors[0].cuda())), dim=1)
    Z = Z.detach().cpu().numpy()
    make_train_plots(ax, nepoch, Z, labels_int, epoch_loss)
    fig.canvas.draw()
```

## Input and reconstructions

```python
vae = vae.cpu()

fig, ax = plt.subplots(2, 10, figsize=(8, 2), tight_layout=True)

mags, errs = next(iter(valid_loader))
z_loc, z_scale = vae.encoder.forward(mags)
mug, mur = vae.decoder(z_loc)
for k, (c, mu) in enumerate(zip(['g', 'r'], [mug, mur])):
    for i in range(10):
        ax[k, i].plot(mags.numpy()[i, k, :], c=c)
        ax[k, i].plot(mu[i].detach().numpy(), ls='--', c=c)
        ax[k, i].invert_yaxis();
        ax[k, i].axis('off')
```

## Latent space visualization

```python
fig, ax = plt.subplots(1, 1, figsize=(8, 6), tight_layout=True)

for k, name in zip(np.unique(labels_int), np.unique(labels)):
    mask = labels_int == k
    ax.errorbar(x=Z[mask, 0], y=Z[mask, 1],
                xerr=Z[mask, 2], yerr=Z[mask, 3],
                fmt='none', alpha=0.5, label=name)
ax.legend(loc=4)
ax.set_xlim([-5., 5.])
ax.set_ylim([-5., 5.]);
```

## Latent space interpolation

```python
M = 10
z_plot = np.linspace(-5., 5., num=M)

fig, ax = plt.subplots(M, M, figsize=(8, 6), tight_layout=True)
for i in range(M):
    for j in range(M):
        z = torch.tensor(np.array([z_plot[j], z_plot[M-1-i]]), dtype=torch.float32)
        mug, mur = vae.decoder.forward(z)
        mug = mug.detach().numpy()
        mur = mur.detach().numpy()
        ax[i, j].plot(mug, lw=2, c='g')
        ax[i, j].plot(mur, lw=2, c='r')
        ax[i, j].invert_yaxis();
        ax[i, j].axis('off')
```

# Interested in Astroinformatics?

```python

```


# Adams Bashforth
#### John S Butler
john.s.butler@tudublin.ie
[Course Notes](https://johnsbutler.netlify.com/files/Teaching/Numerical_Analysis_for_Differential_Equations.pdf)    [Github](https://github.com/john-s-butler-dit/Numerical-Analysis-Python)

The Adams Bashforth method is an explicit multistep method. This notebook illustrates the 2 step Adams Bashforth method for a linear initial value problem, given by

\begin{equation}y^{'}=t-y, \ \ (0 \leq t \leq 2) \end{equation}
with the initial condition
\begin{equation}y(0)=1.\end{equation}
The video below walks through the notebook.

```python
from IPython.display import HTML
HTML('')
```

    /Users/johnbutler/anaconda3/lib/python3.7/site-packages/IPython/core/display.py:717: UserWarning: Consider using IPython.display.IFrame instead
      warnings.warn("Consider using IPython.display.IFrame instead")

## Python Libraries

```python
import numpy as np
import math
import pandas as pd
%matplotlib inline
import matplotlib.pyplot as plt # side-stepping mpl backend
import matplotlib.gridspec as gridspec # subplots
import warnings

warnings.filterwarnings("ignore")

```

### Defining the function
\begin{equation} f(t,y)=t-y.\end{equation}

```python
def myfun_ty(t,y):
    return t-y
```

## Discrete Interval
Defining the step size  $h$  from the interval range  $a\leq t \leq b$  and number of steps  $N$
\begin{equation}h=\frac{b−a}{N}.\end{equation}

This gives the discrete time steps,
\begin{equation}t_i=t_0+ih,\end{equation}
where  $t0=a.$

Here the interval is  $0≤t≤2$  and number of step  4
\begin{equation}h=\frac{2−0}{4}=0.5.\end{equation}

This gives the discrete time steps,
\begin{equation}t_i=0+i0.5,\end{equation}
for  $i=0,1,⋯,4.$

```python
# Start and end of interval
b=2
a=0
# Step size
N=4
h=(b-a)/(N)
t=np.arange(a,b+h,h)
fig = plt.figure(figsize=(10,4))
plt.plot(t,0*t,'o:',color='red')
plt.xlim((0,2))
plt.title('Illustration of discrete time points for h=%s'%(h))
```

## Exact Solution
THe initial value problem has the exact solution
\begin{equation}y(t)=2e^{-t}+t-1.\end{equation}
The figure below plots the exact solution.

```python
IC=1 # Intial condtion
y=(IC+1)*np.exp(-t)+t-1
fig = plt.figure(figsize=(6,4))
plt.plot(t,y,'o-',color='black')
plt.title('Exact Solution ')
plt.xlabel('time')
```

## 2-step Adams Bashforth

The general 2-step Adams Bashforth difference equation is
\begin{equation}w_{i+1} = w_{i} + \frac{h}{2}(3f(t_i,w_i)-f(t_{i-1},w_{i-1})). \end{equation}
For the specific intial value problem the 2-step Adams Bashforth difference equation is
\begin{equation}w_{i+1} = w_{i} + \frac{h}{2}(3(t_i-w_i)-(t_{i-1}-w_{i-1})). \end{equation}

for $i=0$ the difference equation is:
\begin{equation}w_{1} = w_{0} + \frac{h}{2}(3(t_0-w_0)-(t_{-1}-w_{-1})), \end{equation}
this is not solvable as  $w_{-1}$ is unknown.
for $i=1$ the difference equation is:
\begin{equation}w_{2} = w_{1} + \frac{h}{2}(3(t_1-w_1)-(t_{0}-w_{0})), \end{equation}
this is not solvable as  $w_{1}$ is unknown, but it can be approximated using a one step method.
Here, as the exact solution is known,
\begin{equation}w_1=2e^{-t_1}+t_1-1.\end{equation}

```python
### Initial conditions
w=np.zeros(len(t))
w[0]=IC
w[1]=y[1] # NEED FOR THE METHOD
```

### Loop

```python
for k in range (1,N):
    w[k+1]=w[k]+h/2.0*(3*myfun_ty(t[k],w[k])-myfun_ty(t[k-1],w[k-1]))
```

### Plotting solution

```python
def plotting(t,w,y):
    fig = plt.figure(figsize=(10,4))
    plt.plot(t,y, 'o-',color='black',label='Exact')
    plt.plot(t,w,'s:',color='blue',label='Adams-Bashforth')
    plt.xlabel('time')
    plt.legend()
    plt.show
```

The plot below shows the exact solution (black) and the 2 step Adams-Bashforth approximation (red) of the intial value problem

```python
plotting(t,w,y)
```

## Local Error
The Error for the 2 step Adams Bashforth is:
\begin{equation}y_{n+1}=y_n+\frac{h}{2}[3f(t_{n},w_{n})-f(t_{n-1},w_{n-1})] +\frac{5h^3}{12}y'''(\eta),\end{equation}
where $\eta \in [t_{n-1},t_{n+1}]$.

Rearranging the equations gives
\begin{equation}\frac{y_{n+1}-y_n}{h}=\frac{1}{2}[3f(t_{n},w_{n})-f(t_{n-1},w_{n-1})] +\frac{5h^2}{12}y'''(\eta).\end{equation}

For our specific initial value problem the error is of the form:
\begin{equation}\frac{5h^2}{12}y'''(\eta)=\frac{5h^2}{12}2e^{-\eta} \leq\frac{5(0.5)^2}{12} 2\leq 0.208 \end{equation}

```python

d = {'time t_i': t, 'Adams Bashforth, w_i': w,'Exact':y,'Error |w-y|':np.round(np.abs(y-w),5),'LTE':round(2*0.5**2/12*5,5)}
df = pd.DataFrame(data=d)
df
```

      time t_i
      Adams Bashforth, w_i
      Exact
      Error |w-y|
      LTE

      0
      0.0
      1.000000
      1.000000
      0.00000
      0.20833

      1
      0.5
      0.713061
      0.713061
      0.00000
      0.20833

      2
      1.0
      0.803265
      0.735759
      0.06751
      0.20833

      3
      1.5
      1.004082
      0.946260
      0.05782
      0.20833

      4
      2.0
      1.326837
      1.270671
      0.05617
      0.20833

```python

```

Chapter 04 - Multistep Methods/401_Adams Bashforth Example.ipynb


# Thinking about Differential Equations, Autonomous first order ODE (solution annimations, equilibrium points, phase plot,..., just some notes)
### Joseph E. McEwen, (c) 2021, version 0.0.1

This notebook provides code to animate solutions to first order autonomous differential equaitons. All the cool plots are after the code at the bottom of the notebook.

**Required software**
* I recomend you install the [Anaconda distribution of python.](https://www.anaconda.com/products/individual)
* You will also need to install ffmpeg.

**References**
* Introduction to Differential Equations by Boyce.
* Elementary Differential Equations by Trench, it is free here, https://digitalcommons.trinity.edu/mono/8/.
* Nonlinear Dynamics and Chaos by Steven H. Strogatz (not the book for your first exposure to differential equaitons, but a lot of fun when you got at least one differential equations class under your belt).
* A website from UIUC, https://faculty.math.illinois.edu/~hirani/cbmg/diffeq.html.

**Why this notebook**

This notebook has a few purposes.
* One, these are my notes. If I want to learn something, I know that I need to take a deep dive into the subject. Coding is a good way to do this, by coding you have to learn all the details.
* Two, perhaps this notebook can aid students. In fact, this notebook was inspired by a tutoring session I held for a differential equations class. After the class I took some time to recap the ideas disscussed in the tutoring session. I thought writing up my ideas would help to organize my mind and I could share my thoughts in written form along with some cool animations.
* Plus, I just like to code. I wrote this notebook with two audiences in mind, me and students.

**Are animations useful**

I like animations. But, maybe they are not that great of a learning device. Perhaps it is just better to make hand drawn diagrams. This really forces one to use their imagination, which I think is a key ingredient to doing any type of math.

**If you don't know Python**

Perhaps you don't know python or don't know that much python. That is no problem and awesome. It is no problem because you can just ignore the code and pay attention to the animations and the text. It is awesome because this notebook might make you interested in learning python (and provide you with example code).

**Python versus Matlab**

You might say, "hey, Joe, you can do this in Matlab and it is much faster!" Yeah, I agree with that. But Python is free and Matlab is expensive. Expensive means that is is exclusive (not everyone can use it). You comprende now?

**This code is really limited**

Yes, this code is really limited. I may expand it, I may not. Expanding this notebook might depend on if students find it useful - we will see.

**What could be better**

* I would like if making animations in Python was faster.
* I would like if Python's streamplot function was nicer.
* I probably should add some functionality for non-autonomous ODEs.

```python
import numpy as np
%matplotlib inline
import matplotlib.pyplot as plt
import matplotlib
import sympy as sym
import scipy
from sympy import latex
from sympy import symbols, Function
from scipy.integrate import solve_ivp

from matplotlib import animation, rc
from IPython.display import HTML, Math

rc('animation', html='jshtml')
```

```python
# # get dependencies
# print(np.__version__)
# print(matplotlib.__version__)

# %reload_ext watermark
# print(" ")
# %watermark --iversions
```

    1.19.2
    3.3.4

    scipy      1.6.2
    matplotlib 3.3.4
    sympy      1.7.1
    numpy      1.19.2

We will input our differential equation as a function of $y$ and $t$. So far this code only accepts first order differential equations (i.e. only first derivatives). To input a differential equation we input the right hand side function of an equation
\begin{align}
\frac{dy}{dt} = f(y,t)~.
\end{align}

```python
# sympy symbols for t and y
t,y=symbols("t,y")
```

```python
class ODETOOLS:

    def __init__(self,f,ny,nt,ymin,ymax,tmin,tmax):

        '''
        Ordinary differential equation python tool kit.
        inputs:
        f is the right handside of the differential equation. f contains sympy symbols t and y and numerical values.
        ny is the number number of grid points for y between ymin and ymax.
        nt is the number of grid points for t between tmin and tmax.

        '''

        if t in f.free_symbols:
            self.de_type="non-autonomous"
        else:
            self.de_type="autonomous"

        self.f=f
        self.rhs=sym.lambdify((t,y),self.f)

        self.ymin=ymin
        self.ymax=ymax

        self.tmin=tmin
        self.tmax=tmax

        self.nt=nt

        # define Y as a function of t to display derivative
        Y=symbols("y",cls=Function)
        self.lhs=Y(t)

        self.eqn=sym.Eq(self.lhs.diff(t),f)

        print("Your differential equation is:")
        display(self.eqn)

        self.t=np.linspace(tmin,tmax,nt)
        self.y=np.linspace(ymin,ymax)

    def streamplot(self,ylabel=None,xlabel=None,density=None,minlength=.2,figsize=(6,6)):
        '''
        Function to make streamplot of differential equations.
        '''

        T,Y=np.meshgrid(self.t,self.y)
        dtdt=np.ones_like(T)
        dydt=self.rhs(T,Y)

        fig=plt.figure(figsize=figsize)
        ax=fig.add_subplot(111)

        if ylabel is not None:
            ax.set_ylabel(ylabel,size=20)

        if xlabel is not None:
            ax.set_xlabel(xlabel,size=20)

        ax.streamplot(T,Y,dtdt,dydt,color="k",minlength=minlength)
        ax.grid()

    def plot_fy(self,n=200):
        '''
        Plot the an autonomous differenetial equation.
        '''

        if self.de_type=="autonomous":

            y_range=np.linspace(self.ymin,self.ymax,n)
            fy=self.rhs(0,y_range)

            fig=plt.figure()
            ax=fig.add_subplot(111)
            ax.set_ylabel(r"$dy/dt$", size=20)
            ax.set_xlabel(r"$y$", size=20)

            ax.plot(y_range,fy)
            ax.grid()
        else:
            print("This is not an autonmous differential equation, can not plot in one-dimension.")

    def solve(self,y0, method="RK45"):
        '''
        Function to solve the differential equation numericaly, default method is RK45.
        '''

        n_y=y0.size
        sol_mat=np.zeros((n_y,self.nt))
        t_mat=np.zeros((n_y,self.nt))

        for i in range(n_y):

            sol=solve_ivp(self.rhs,(self.tmin,self.tmax),np.array([y0[i]]),t_eval=self.t, method=method)

            y_sol=sol["y"][0,:]

            # if the returned solution is not the same size as nt
            # then fill will values that go off the grid
            # this probably happens if the solution diverges
            if y_sol.size < self.nt:

                y_sol=np.r_[y_sol, np.ones(self.nt-y_sol.size)*self.ymin*1000]

            sol_mat[i,:]=y_sol
            t_mat[i,:]=self.t

        return t_mat, sol_mat

    def animate_de(self,y01,y02=None,ylabel=None,xlabel=None,minlength=.2,method="RK45"):
        '''
        Function to animate the evolution of a set of solutions and
        to animate the phase plot.

        y01 are a set of initial values.
        y02 are another (optional) set of initial values that you would like to distinguish from y01 (perhaps fixed points).
        '''

        N_frames=self.t.size

        # convert to numpy array if entered as a list
        y01=np.asarray(y01)
        if y02 is not None:
            y02=np.asarray(y02)

        T,Y=np.meshgrid(self.t,self.y)
        dtdt=np.ones_like(T)
        dydt=self.rhs(T,Y)

        t_mat1, sol1=self.solve(y01, method=method)

        if y02 is not None:
            t_mat2, sol2=self.solve(y02, method=method)

        fig=plt.figure(figsize=(14,8))
        ax=fig.add_subplot(121)
        ax.set_title("solution evolution animation",size=20)
        ax.grid()

        if ylabel is not None:
            ax.set_ylabel(ylabel,size=20)

        if xlabel is not None:
            ax.set_xlabel(xlabel,size=20)

        ax.set_xlim(self.tmin,self.tmax)
        ax.set_ylim(self.ymin,self.ymax)

        ax.streamplot(T,Y,dtdt,dydt,color="blue",minlength=minlength)

        pts1,=ax.plot([],[],'o',color="black",ms=10,alpha=.8)

        if y02 is not None:
            pts2,=ax.plot([],[],'o',color="red",ms=10)

        # second plot
        ax=fig.add_subplot(122)
        ax.set_title("phase plot", size=20)

        DT=self.tmax-self.tmin
        eps=DT/20

        ax.set_xlim(self.tmin-eps,self.tmin+eps)
        ax.set_ylim(self.ymin,self.ymax)
        ax.axes.get_xaxis().set_visible(False)

        ax.axvline(self.tmin,alpha=.7)

        ppts1,=ax.plot([],[],'o',color="black",ms=10,alpha=.8)
        if y02 is not None:
            ppts2,=ax.plot([],[],'o',color="red",ms=10,alpha=.8)

        plt.close()

        def init():
            pts1.set_data([], [])
            ppts1.set_data([], [])

            if y02 is not None:
                pts2.set_data([], [])
                ppts2.set_data([], [])
                return (pts1,ppts1,pts2,ppts2,)
            else:
                return (pts1,ppts1,)

        def animate(i):
            pts1.set_data(t_mat1[:,i], sol1[:,i])
            ppts1.set_data(t_mat1[:,0], sol1[:,i])

            if y02 is not None:

                pts2.set_data(t_mat2[:,i], sol2[:,i])
                ppts2.set_data(t_mat2[:,0], sol2[:,i])
                return (pts1,ppts1,pts2,ppts2,)

            else:
                return (pts1,ppts1,)

        anim = animation.FuncAnimation(fig, animate, init_func=init,
                               frames=N_frames, interval=100,
                               blit=True)

        return anim
```

### Example 1
We want to analyze the differential equation
\begin{align}
\frac{dy}{dt} = .5 y - 450~.
\end{align}
This ODE is taken from Boyce, chapter 1 equation 7, which models the growth of field mice.

Below is the code to input the example ODE.

```python
# right hand side function
f=.5*y-450

n=30      # number of y-points
nt=2*n    # number or t-points
ymin=850  # minimum y for plot
ymax=1000 # maximum y for plot
tmin=0    # minimum time
tmax=5    # maximum time

ode1=ODETOOLS(f,n,nt,ymin,ymax,tmin,tmax)

# display streamplot
ode1.streamplot(ylabel="dy/dt", xlabel="time")
```

```python
# check type, plot trajectories of various points, examine phase plot

print("Type of ODE:", ode1.de_type)
ode1.solve(np.array([900,920,890,870,930]))
y02=[900]
y01=[920,890,870,930]
anim=ode1.animate_de(y01,y02,ylabel="number of mice",xlabel="time")
HTML(anim.to_html5_video())
```

    Type of ODE: autonomous

```python
# save
# anim.save("mice_owl_solution.mp4")
```

### Example 2
A secnd example is [logistic equation](https://mathworld.wolfram.com/LogisticEquation.html) (see chapter 2.5 in Boyce)
\begin{align}
\frac{dy}{dt} = r(1-y/K) y~,
\end{align}
where $r$ and $K$ are parameters of the differential equation, the intrinsic growth rate and saturation level, respectivley.

```python
#parameters
r=1
K=1
f=r*(1-y/K)*y

n=30
nt=2*n
ymin=-1
ymax=2
tmin=0
tmax=5

ode2=ODETOOLS(f,n,nt,ymin,ymax,tmin,tmax)
ode2.plot_fy()
```

```python
ode2.streamplot()
```

```python
anim=ode2.animate_de([1/2,1.5, 1.1, .1, -.1],[0,1])
HTML(anim.to_html5_video())
```

In the above animation there are two types of fixed points (the red dots). One is stable fixed point, while the other is an unstable fixed point.

```python
# save animaion
# anim.save("logistic_solution.mp4")
```

```python

```

    2021-09-21T09:48:33-07:00

    CPython 3.8.5
    IPython 7.22.0

    compiler   : Clang 10.0.0
    system     : Darwin
    release    : 20.6.0
    machine    : x86_64
    processor  : i386
    CPU cores  : 4
    interpreter: 64bit

```python

```


```python
import numpy as np, pandas as pd, matplotlib as mpl, matplotlib.pyplot as plt#, seaborn as sns
%matplotlib inline
mpl.style.use('classic')
```

Symbolic math

Definitions

$Y=[α\cdot F^{-\rho}+(1-α)\cdot R^{-\rho} ]^{-1/\rho}$ Arrow 1961

MIT: $Y=[α\cdot F^{\rho}+(1-α)\cdot R^{\rho} ]^{1/\rho}$   http://www.sciencedirect.com/science/article/pii/S0140988306000624?np=y Eq. 1

$\sigma = \frac{1}{1+\rho}$

MIT: $\sigma = \frac{1}{1-\rho}$

```python
from sympy import *
init_printing()
```

```python
a = Symbol('alpha')
s = Symbol('sigma')
r = Symbol('rho')
var('Y F R C c_f c_r');
```

```python
eq=Eq(Y,(a*F**-r+(1-a)*R**-r)**(-1/r))
eq  #main CES equation
    #Y = value of function - analogous to energy demand?
    #R = renewables
    #F = fossils
    #alfa = fossil share
    #rho = factor related to elasticity of substituion
```

```python
R_sol=solve(eq,R)[0]
R_sol #expression of R from the main CES equation
```

```python
eq2=Eq(s,1/(1+r))
eq2 #definition of elasticity of substituion
    #rho = elasticity of substitution
```

```python
r_sol=solve(eq2,r)[0]
r_sol #exression of rho in terms of sigma
```

```python
[RR_sol,rr_sol]=solve([eq,eq2],[R,r])[0]
RR_sol #expression of R in terms of sigma
```

```python
Y_sol=solve(eq,Y)[0]
Y_sol #expression of Y in terms of sigma = main expression
```

```python
YR_sol=solve([eq,eq2],[Y,r])[0][0]
YR_sol #expression of Y in terms of sigma
```

Sample policy calculation replicated from page 60 of http://globalchange.mit.edu/files/document/MITJPSPGC_Rpt125.pdf (EPPA).

```python
s_fix=0.4 #fixed elasticity for demand, EPPA pag. 24
s_fix2=0.5 #fixed elasticity for production, EPPA pag. 17
```

```python
step=1 #step size for numerical calculation
years=np.arange(2015,2100+step,step)
years #years to complete transition (fossil phase-out)
```

    array([2015, 2016, 2017, 2018, 2019, 2020, 2021, 2022, 2023, 2024, 2025,
           2026, 2027, 2028, 2029, 2030, 2031, 2032, 2033, 2034, 2035, 2036,
           2037, 2038, 2039, 2040, 2041, 2042, 2043, 2044, 2045, 2046, 2047,
           2048, 2049, 2050, 2051, 2052, 2053, 2054, 2055, 2056, 2057, 2058,
           2059, 2060, 2061, 2062, 2063, 2064, 2065, 2066, 2067, 2068, 2069,
           2070, 2071, 2072, 2073, 2074, 2075, 2076, 2077, 2078, 2079, 2080,
           2081, 2082, 2083, 2084, 2085, 2086, 2087, 2088, 2089, 2090, 2091,
           2092, 2093, 2094, 2095, 2096, 2097, 2098, 2099, 2100])

```python
gamma=0.00 #demand increase rate PER YEAR
Y_zero=100 #initial demand
```

```python
init_F=0.95 #initial fossil share, estimation from EPPA pag. 62, fig. 20
F_fix=np.array([init_F*Y_zero*(1.0-1.0/(len(years)-1)*i) for i in range(len(years))])
F_fix #array of predefined fossil values (in order complete the transition as defined by years)
```

    array([ 95.        ,  93.88235294,  92.76470588,  91.64705882,
            90.52941176,  89.41176471,  88.29411765,  87.17647059,
            86.05882353,  84.94117647,  83.82352941,  82.70588235,
            81.58823529,  80.47058824,  79.35294118,  78.23529412,
            77.11764706,  76.        ,  74.88235294,  73.76470588,
            72.64705882,  71.52941176,  70.41176471,  69.29411765,
            68.17647059,  67.05882353,  65.94117647,  64.82352941,
            63.70588235,  62.58823529,  61.47058824,  60.35294118,
            59.23529412,  58.11764706,  57.        ,  55.88235294,
            54.76470588,  53.64705882,  52.52941176,  51.41176471,
            50.29411765,  49.17647059,  48.05882353,  46.94117647,
            45.82352941,  44.70588235,  43.58823529,  42.47058824,
            41.35294118,  40.23529412,  39.11764706,  38.        ,
            36.88235294,  35.76470588,  34.64705882,  33.52941176,
            32.41176471,  31.29411765,  30.17647059,  29.05882353,
            27.94117647,  26.82352941,  25.70588235,  24.58823529,
            23.47058824,  22.35294118,  21.23529412,  20.11764706,
            19.        ,  17.88235294,  16.76470588,  15.64705882,
            14.52941176,  13.41176471,  12.29411765,  11.17647059,
            10.05882353,   8.94117647,   7.82352941,   6.70588235,
             5.58823529,   4.47058824,   3.35294118,   2.23529412,
             1.11764706,   0.        ])

```python
#calculate Y_fix_zero through resubstituting initial values into CES
Y_fix_zero=YR_sol.subs([[R,Y_zero*(1-init_F)],[F,Y_zero*init_F],[a,init_F],[s,s_fix2]])
Y_fix=np.array([Y_fix_zero*((1+gamma*step)**i) for i in range(len(years))])
Y_fix #array of predefined demand values
```

    array([50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000, 50.0000000000000,
           50.0000000000000, 50.0000000000000], dtype=object)

```python
Y_sol, YR_sol, eq2, R_sol, RR_sol
```

```python
R_fix=[RR_sol.subs([[Y,Y_fix[k]],[F,F_fix[k]],[s,s_fix2],[a,init_F]]) for k in range(len(F_fix))]
plt.scatter(R_fix,F_fix)
plt.xlim(0,200);
```

Carbon tax

```python
eq3=Eq(c_r,c_f/((a/(1-a))*(F/R)**(-1-r)))
eq3 #cost of R compared to F
    #c_f fixed cost of F
```

```python
c_r_sol=solve(eq3,c_r)[0]
c_r_sol #cost of R expressed in terms of rho
```

```python
cR_r_sol=solve([eq3,eq2],[c_r,r])[0][0]
cR_r_sol #cost of R expressed in terms of sigma
```

```python
eq4=Eq(C,c_r*((a/(1-a))*(F/R)**(-1-r))-c_f)
eq4 #carbon tax: relative cost of F compared to R - fixed cost of F
```

```python
C_sol=solve(eq4,C)[0]
C_sol #expression of C in terms of sigma, F, R, a, c_f
```

```python
CR_sol=solve([eq4,eq2],[C,r])[0][0]
CR_sol #expression of C in terms of rho, F, R, a, c_f
```

```python
c_f_fix=1 #cost of fossil, taken 1 to normalize
zeta=0.00 #R cost decrease rate PER YEAR
k=0
c_r_fix_zero=cR_r_sol.subs([[R,R_fix[k]],[F,F_fix[k]],[s,s_fix2],[a,init_F],[c_f,c_f_fix]])
c_r_fix=np.array([c_r_fix_zero*((1-zeta*step)**i) for i in range(len(years))])
c_r_fix #array of predefined R cost values, based on cost decrease rate zeta
```

    array([19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000, 19.0000000000000,
           19.0000000000000, 19.0000000000000], dtype=object)

```python
C_sol
```

```python
C_fix=[CR_sol.subs([[R,R_fix[k]],[F,F_fix[k]],[s,s_fix2],[a,init_F],[c_f,c_f_fix],[c_r,c_r_fix[k]]])\
       for k in range(len(F_fix))]
plt.scatter(years[:-1],C_fix[:-1],color='r')
plt.ylim(0,1000);
```

```python
def ces(init_F=0.75,s_fix=0.5,zeta=0.0,gamma=0.0,step=1):
    years=np.arange(2015,2100+step,step)
    Y_zero=100 #initial demand
    F_fix=np.array([init_F*Y_zero*(1.0-1.0/(len(years)-1)*i) for i in range(len(years))])
    Y_fix_zero=YR_sol.subs([[R,Y_zero*(1-init_F)],[F,Y_zero*init_F],[a,init_F],[s,s_fix]])
    Y_fix=np.array([Y_fix_zero*((1+gamma*step)**i) for i in range(len(years))])
    R_fix=[RR_sol.subs([[Y,Y_fix[k]],[F,F_fix[k]],[s,s_fix],[a,init_F]]) for k in range(len(F_fix))]

    R_fix=[i for i in R_fix if np.complex(i).imag==0]
    R_fix=[i for i in R_fix if i>0]

    k=0
    c_f_fix=1 #cost of fossil, taken 1 to normalize
    c_r_fix_zero=cR_r_sol.subs([[R,R_fix[k]],[F,F_fix[k]],[s,s_fix],[a,init_F],[c_f,c_f_fix]])
    c_r_fix=np.array([c_r_fix_zero*((1-zeta*step)**i) for i in range(len(years))])
    C_fix=[CR_sol.subs([[R,R_fix[k]],[F,F_fix[k]],[s,s_fix],[a,init_F],[c_f,c_f_fix],[c_r,c_r_fix[k]]])\
       for k in range(len(R_fix))]

    return [R_fix[:-1],F_fix[:len(R_fix)-1],C_fix[:len(R_fix)-1]]
```

```python
mwd=[]

init_F0= [0.85,0.9,0.95]
s_fix0 = [0.5, 1.25, 2.0]
init_F= [init_F0[i] for i in range(len(init_F0)) for j in range(len(s_fix0))]
s_fix = [s_fix0[j] for i in range(len(init_F0)) for j in range(len(s_fix0))]

for i in range(len(init_F)):

    [x,y,c]=ces(init_F[i],s_fix[i])
    [x2,y2,c2]=ces(init_F[i],s_fix[i],0.01)

    mwd.append({})
    for e in range(len(x)):
        mwd[i][2015+e]={"y":y[e],"x":x[e],"c":c[e],"y2":y[e],"x2":x[e],"c2":c2[e],}
```

```python
fig = plt.subplots(1,3,figsize=(14,4))
lw=2

gs = mpl.gridspec.GridSpec(1, 4,width_ratios=[1.5,0.01,1,1])
bg=['#EEEEEE','none','#EEEEEE','#EEEEEE']
ax = [plt.subplot(gs[i],axisbg=bg[i],axisbelow=True) for i in range(4)]
plt.subplots_adjust(wspace = .1)
ax[1].spines['top'].set_color('none')
ax[1].spines['bottom'].set_color('none')
ax[1].spines['left'].set_color('none')
ax[1].spines['right'].set_color('none')

ls0=[":","--","-"]
ss0=["","",""]
ls = [ls0[j] for i in range(len(init_F0)) for j in range(len(s_fix0))]
ss = [ss0[j] for i in range(len(init_F0)) for j in range(len(s_fix0))]
colors0=["#dd1c77","#2c7fb8","#2ca25f"]
colors=[colors0[i] for i in range(len(init_F0)) for j in range(len(s_fix0))]
ax2=ax[2].twinx()
ax2.set_axisbelow(True)
ax[1].xaxis.set_visible(False)
ax[1].yaxis.set_visible(False)
ax[2].set_yticks([])
ax3=ax[3].twinx()
ax3.set_axisbelow(True)
ax[3].set_yticks([])
ax[2].set_xlim(2015,2100)
ax[2].set_xticks([2025,2050,2075,2100])
ax[3].set_xlim(2015,2100)
ax[3].set_xticks([2025,2050,2075,2100])
ax[0].set_ylim(0,100)
ax2.set_ylim(0,800)
ax3.set_ylim(0,800)
ax[0].grid(color='white', linestyle='solid')
ax[2].grid(color='white', linestyle='solid')
ax[3].grid(color='white', linestyle='solid')
ax2.grid(color='white', linestyle='solid')
ax3.grid(color='white', linestyle='solid')

for i in range(len(init_F)):

    #[x,y,c]=ces(init_F[i],s_fix[i])
    #[x2,y2,c2]=ces(init_F[i],s_fix[i],0.01)
    x=[]
    y=[]
    x2=[]
    y2=[]
    c=[]
    c2=[]
    for e in sorted(mwd[i].keys()):
        x.append(mwd[i][e]["x"])
        y.append(mwd[i][e]["y"])
        c.append(mwd[i][e]["c"])
        x2.append(mwd[i][e]["x2"])
        y2.append(mwd[i][e]["y2"])
        c2.append(mwd[i][e]["c2"])

    ax[0].plot(x,y,linewidth=lw,linestyle=ls[i],color=colors[i],marker=ss[i])

    #for legend
    if i in range(len(s_fix0)):
        ax[0].plot([-10],[-10],'#444444',label='$\\sigma='+str(s_fix[i])+'$',linestyle=ls[i],marker=ss[i],linewidth=lw)
        ax3.plot([-10],[-10],'#2ca25f',label='$\\sigma='+str(s_fix[i])+'$',linestyle=ls[i],marker=ss[i],linewidth=lw)
    if i in {2,5,7}:
        ax[0].plot([-10],[-10],label='$\\alpha='+str(init_F[i])+'$',color=colors[i],linewidth=lw)
        ax2.plot([-10],[-10],label='$\\alpha='+str(init_F[i])+'$',color=colors[i],linewidth=lw)
    if i in {2,5,8}:
        ax2.fill_between(np.array(years[:len(c)]).astype(float),np.array(c).astype(float),np.array(c2).astype(float),\
                         color=colors[i],alpha=0.5)#,zorder=10)
        ax2.plot(years[:len(c)],c2,linewidth=lw,linestyle=ls[i],color=colors[i],marker=ss[i])
        ax2.plot(years[:len(c)],c,linewidth=lw,linestyle=ls[i],color=colors[i],marker=ss[i])
    if i in {6,7,8}:
        ax3.plot(years[:len(c)],c,linewidth=lw,linestyle=ls[i],color=colors[i],marker=ss[i],zorder=10)
        if i in {7}:
            ax3.text(0.04,0.72,'$\\alpha='+str(init_F[i])+'$',
                    horizontalalignment='left',
                    verticalalignment='top',
                    transform=ax3.transAxes,fontsize=11)
    if i in {8}:
        ax2.annotate('$\\zeta = 0\%$', xy=(years[58], c[58]),  xycoords='data',
                xytext=(-50, 30), textcoords='offset points',
                arrowprops=dict(arrowstyle="->"))#,zorder=11)
        ax2.annotate('$\\zeta = 1\%$', xy=(years[78], c2[78]),  xycoords='data',
                xytext=(-50, 30), textcoords='offset points',
                arrowprops=dict(arrowstyle="->"))#,zorder=11)
        ax2.text(0.04,0.72,'$\\sigma='+str(s_fix[i])+'$',
                horizontalalignment='left',
                verticalalignment='top',
                transform=ax2.transAxes,fontsize=11)

ax[0].legend(loc=3,framealpha=0.5,fontsize=11)
ax2.legend(loc=2,framealpha=0.5,fontsize=11)
ax3.legend(loc=2,framealpha=0.5,fontsize=11)
ax[0].set_xlabel(u'Renewable energy $R$')
ax[2].set_xlabel(u'Year')
ax[3].set_xlabel(u'Year')
ax2.set_yticklabels(["","","","","",""])
ax[0].set_ylabel(u'Fossil energy $F$')
ax3.set_ylabel(u'Marginal abatement cost $MAC$')
ax[0].set_title('$F-R$ curve\n $Y=[\\alpha\cdot \
                F^{-\\rho}+(1-\\alpha)\cdot R^{-\\alpha} ]^{-1/\\rho}, \\sigma = 1/(1+\\rho)$',size=12,y=1.02)
ax[2].set_title('Marginal abatement cost $MAC$\n $MAC=c_R \cdot [\\alpha/(1-\\alpha)] \
                \cdot (F/R)^{-\\rho-1} - c_F,$  $\\zeta:$ $yearly$ $decrease$ $in$ $c_R$',size=12,y=1.02,x=1.1)

plt.suptitle(u'Constant Elasticity (CES) Energy Substitution Model at Constant Energy Demand $Y$',fontsize=13,y=1.12)
ax[0].set_xlim(0,2000)
#plt.savefig('ces2-large.png',bbox_inches = 'tight', pad_inches = 0.1, dpi=150)
ax[0].set_xlim(0,200)
#plt.savefig('ces2.png',bbox_inches = 'tight', pad_inches = 0.1, dpi=150)
plt.show()
```

```python
mwd2=[]

init_F0= [0.85,0.9,0.95]
s_fix0 = [0.5, 1.25, 2.0]
init_F= [init_F0[i] for i in range(len(init_F0)) for j in range(len(s_fix0))]
s_fix = [s_fix0[j] for i in range(len(init_F0)) for j in range(len(s_fix0))]

for i in range(len(init_F)):

    [x,y,c]=ces(init_F[i],s_fix[i],0.0,0.03)
    [x2,y2,c2]=ces(init_F[i],s_fix[i],0.01,0.03)

    mwd2.append({})
    for e in range(len(x)):
        mwd2[i][2015+e]={"y":y[e],"x":x[e],"c":c[e],"y2":y[e],"x2":x[e],"c2":c2[e],}
```

```python
fig = plt.subplots(1,3,figsize=(14,4))
lw=2

gs = mpl.gridspec.GridSpec(1, 4,width_ratios=[1.5,0.01,1,1])
bg=['#EEEEEE','none','#EEEEEE','#EEEEEE']
ax = [plt.subplot(gs[i],axisbg=bg[i],axisbelow=True) for i in range(4)]
plt.subplots_adjust(wspace = .1)
ax[1].spines['top'].set_color('none')
ax[1].spines['bottom'].set_color('none')
ax[1].spines['left'].set_color('none')
ax[1].spines['right'].set_color('none')

ax2=ax[2].twinx()
ax2.set_axisbelow(True)
ax[1].xaxis.set_visible(False)
ax[1].yaxis.set_visible(False)
ax[2].set_yticks([])
ax3=ax[3].twinx()
ax3.set_axisbelow(True)
ax[3].set_yticks([])
ax[2].set_xlim(2015,2100)
ax[2].set_xticks([2025,2050,2075,2100])
ax[3].set_xlim(2015,2100)
ax[3].set_xticks([2025,2050,2075,2100])
ax[0].set_ylim(0,100)
ax2.set_ylim(0,1000)
ax3.set_ylim(0,1000)

for i in range(len(init_F)):

    [x,y,c]=ces(init_F[i],s_fix[i],0.0,0.03)
    [x2,y2,c2]=ces(init_F[i],s_fix[i],0.01,0.03)
    x=[]
    y=[]
    x2=[]
    y2=[]
    c=[]
    c2=[]
    for e in sorted(mwd2[i].keys()):
        x.append(mwd2[i][e]["x"])
        y.append(mwd2[i][e]["y"])
        c.append(mwd2[i][e]["c"])
        x2.append(mwd2[i][e]["x2"])
        y2.append(mwd2[i][e]["y2"])
        c2.append(mwd2[i][e]["c2"])

    ax[0].plot(x,y,linewidth=lw,linestyle=ls[i],color=colors[i],marker=ss[i])

    #for legend
    if i in range(len(s_fix0)):
        ax[0].plot([-10],[-10],'#444444',label='$\\sigma='+str(s_fix[i])+'$',linestyle=ls[i],marker=ss[i],linewidth=lw)
        ax3.plot([-10],[-10],'#2ca25f',label='$\\sigma='+str(s_fix[i])+'$',linestyle=ls[i],marker=ss[i],linewidth=lw)
    if i in {2,5,7}:
        ax[0].plot([-10],[-10],label='$\\alpha='+str(init_F[i])+'$',color=colors[i],linewidth=lw)
        ax2.plot([-10],[-10],label='$\\alpha='+str(init_F[i])+'$',color=colors[i],linewidth=lw)
    if i in {2,5,8}:
        ax2.fill_between(np.array(years[:len(c)]).astype(float),np.array(c).astype(float),np.array(c2).astype(float),\
                         color=colors[i],alpha=0.5)#,zorder=10)
        ax2.plot(years[:len(c)],c2,linewidth=lw,linestyle=ls[i],color=colors[i],marker=ss[i])
        ax2.plot(years[:len(c)],c,linewidth=lw,linestyle=ls[i],color=colors[i],marker=ss[i])
    if i in {6,7,8}:
        ax3.plot(years[:len(c)],c,linewidth=lw,linestyle=ls[i],color=colors[i],marker=ss[i],zorder=10)
    if i in {8}:
        ax2.annotate('$\\zeta = 0\%$', xy=(years[48], c[48]),  xycoords='data',
                xytext=(-50, 30), textcoords='offset points',fontsize=12,
                arrowprops=dict(arrowstyle="->"))#,zorder=11)
        ax2.annotate('$\\zeta = 1\%$', xy=(years[68], c2[68]),  xycoords='data',
                xytext=(-50, 30), textcoords='offset points',fontsize=12,
                arrowprops=dict(arrowstyle="->"))#,zorder=11)

ax[0].grid(color='white', linestyle='solid')
ax[2].grid(color='white', linestyle='solid')
ax[3].grid(color='white', linestyle='solid')
ax2.grid(color='white', linestyle='solid')
ax3.grid(color='white', linestyle='solid')
ax[0].legend(loc=3,framealpha=0.5,fontsize=12)
ax2.legend(loc=2,framealpha=0.5,fontsize=12)
ax3.legend(loc=4,framealpha=0.5,fontsize=12)
ax[0].set_xlabel(u'Renewable energy $R$',fontsize=12)
ax[2].set_xlabel(u'Year',fontsize=12)
ax[3].set_xlabel(u'Year',fontsize=12)
ax2.set_yticklabels(["","","","","",""])
ax[0].set_ylabel(u'Fossil energy $F$',fontsize=12)
ax3.set_ylabel(u'Marginal abatement cost $MAC$',fontsize=12)
ax[0].set_title('$F-R$ curve\n $Y=[\\alpha\cdot \
                F^{-\\rho}+(1-\\alpha)\cdot R^{-\\alpha} ]^{-1/\\rho}, \\sigma = 1/(1+\\rho)$',size=16,y=1.03)
ax[2].set_title('Marginal abatement cost $MAC$\n $MAC=c_R \cdot [\\alpha/(1-\\alpha)] \
                \cdot (F/R)^{-\\rho-1} - c_F,$  $\\zeta:$ $yearly$ $decrease$ $in$ $c_R$',size=16,y=1.03,x=1.1)

plt.suptitle(u'Constant Elasticity (CES) Energy Substitution Model at Yearly Energy Demand Increase Rate $\\gamma=3\\%$',fontsize=16,y=1.16)
ax[0].set_xlim(0,25000)
plt.savefig('ces3-largex.png',bbox_inches = 'tight', pad_inches = 0.1, dpi=150)
#ax[0].set_xlim(0,200)
#plt.savefig('ces3x.png',bbox_inches = 'tight', pad_inches = 0.1, dpi=150)
plt.show()
```

```python
def sces(init_F=0.75,s_fix=0.5,rf=-1,zeta=0.0,gamma=0.0,step=1):
    #dynamically adjust sigma and resolve equation
    years=np.arange(2015,2100+step,step)
    Y_zero=100 #initial demand
    F_fix=np.array([init_F*Y_zero*(1.0-1.0/(len(years)-1)*i) for i in range(len(years))])
    Y_fix_zero=YR_sol.subs([[R,Y_zero*(1-init_F)],[F,Y_zero*init_F],[a,init_F],[s,s_fix]])
    Y_fix=np.array([Y_fix_zero*((1+gamma*step)**i) for i in range(len(years))])
    #init R_fix, C_fix
    R_fix=[0 for k in range(len(F_fix))]
    C_fix=[0 for k in range(len(F_fix))]

    r_fix=solve(eq2,r)[0].subs(s,s_fix)
    r_span=-(rf-r_fix)
    s_fix_k=[]
    r_fix_k=[]

    for k in range(len(F_fix)):
        t=1
        s_fix_k.append(solve(eq2,s)[0].subs(r,r_fix)) #append to sigma array
        r_fix_k.append(r_fix) #append to rho array
        #update sigma every t years
        if (k%t==0):
            r_fix=r_fix-(r_span)/len(years) #update rho

    R_fix=[RR_sol.subs([[Y,Y_fix[k]],[F,F_fix[k]],[s,s_fix_k[k]],[a,init_F]]) for k in range(len(F_fix))]

    R_fix=[i for i in R_fix if np.complex(i).imag==0]
    R_fix=[i for i in R_fix if i>0]

    k=0
    c_f_fix=1 #cost of fossil, taken 1 to normalize
    c_r_fix_zero=cR_r_sol.subs([[R,R_fix[k]],[F,F_fix[k]],[s,s_fix_k[k]],[a,init_F],[c_f,c_f_fix]])
    c_r_fix=np.array([c_r_fix_zero*((1-zeta*step)**i) for i in range(len(years))])
    C_fix=[CR_sol.subs([[R,R_fix[k]],[F,F_fix[k]],[s,s_fix_k[k]],[a,init_F],[c_f,c_f_fix],[c_r,c_r_fix[k]]])\
       for k in range(len(R_fix))]

    return [R_fix[:-1],F_fix[:len(R_fix)-1],C_fix[:len(R_fix)-1],s_fix_k[:len(R_fix)-1],r_fix_k[:len(R_fix)-1]]
```

```python
[x,y,c,ys,yr]=sces(0.95,1.5,-1)
plt.plot(years[:len(c)],c)
[x,y,c,ys,yr]=sces(0.95,1.5,-1,0.01)
plt.plot(years[:len(c)],c,'--')
[x,y,c,ys,yr]=sces(0.85,1.5,-1)
plt.plot(years[:len(c)],c)
[x,y,c,ys,yr]=sces(0.85,1.5,-1,0.01)
plt.plot(years[:len(c)],c,'--')
[x,y,c,ys,yr]=sces(0.90,1.5,-1)
plt.plot(years[:len(c)],c)
[x,y,c,ys,yr]=sces(0.90,1.5,-1,0.01)
plt.plot(years[:len(c)],c,'--')
```

```python
def dces(init_F=0.75,s_fix=0.5,sf=0.01,zeta=0.0,gamma=0.0,step=1):
    #dynamically adjust sigma and resolve equation
    years=np.arange(2015,2100+step,step)
    Y_zero=100 #initial demand
    F_fix=np.array([init_F*Y_zero*(1.0-1.0/(len(years)-1)*i) for i in range(len(years))])
    Y_fix_zero=YR_sol.subs([[R,Y_zero*(1-init_F)],[F,Y_zero*init_F],[a,init_F],[s,s_fix]])
    Y_fix=np.array([Y_fix_zero*((1+gamma*step)**i) for i in range(len(years))])
    #init R_fix, C_fix
    R_fix=[0 for k in range(len(F_fix))]
    C_fix=[0 for k in range(len(F_fix))]

    s_fix_k=[]
    r_fix_k=[]

    for k in range(len(F_fix)):
        t=1
        s_fix_k.append(s_fix) #append to sigma array
        r_fix_k.append(solve(eq2,r)[0].subs(s,s_fix)) #append to rho array
        #update sigma every t years
        if (k%t==0):
            s_fix*=1+sf #update rho

    R_fix=[RR_sol.subs([[Y,Y_fix[k]],[F,F_fix[k]],[s,s_fix_k[k]],[a,init_F]]) for k in range(len(F_fix))]

    R_fix=[i for i in R_fix if np.complex(i).imag==0]
    R_fix=[i for i in R_fix if i>0]

    k=0
    c_f_fix=1 #cost of fossil, taken 1 to normalize
    c_r_fix_zero=cR_r_sol.subs([[R,R_fix[k]],[F,F_fix[k]],[s,s_fix_k[k]],[a,init_F],[c_f,c_f_fix]])
    c_r_fix=np.array([c_r_fix_zero*((1-zeta*step)**i) for i in range(len(years))])
    C_fix=[CR_sol.subs([[R,R_fix[k]],[F,F_fix[k]],[s,s_fix_k[k]],[a,init_F],[c_f,c_f_fix],[c_r,c_r_fix[k]]])\
       for k in range(len(R_fix))]

    return [R_fix[:-1],F_fix[:len(R_fix)-1],C_fix[:len(R_fix)-1],s_fix_k[:len(R_fix)-1],r_fix_k[:len(R_fix)-1]]
```

```python
[x,y,c,ys,yr]=dces(0.95,1.5,0.1)
plt.plot(years[:len(c)],c)
[x,y,c,ys,yr]=dces(0.95,1.5,0.1,0.01)
plt.plot(years[:len(c)],c,'--')
[x,y,c,ys,yr]=dces(0.85,1.5,0.1)
plt.plot(years[:len(c)],c)
[x,y,c,ys,yr]=dces(0.85,1.5,0.1,0.01)
plt.plot(years[:len(c)],c,'--')
[x,y,c,ys,yr]=dces(0.90,1.5,0.1)
plt.plot(years[:len(c)],c)
[x,y,c,ys,yr]=dces(0.90,1.5,0.1,0.01)
plt.plot(years[:len(c)],c,'--')
```

```python
fig = plt.subplots(2,2,figsize=(10,5))
lw=2

gs = mpl.gridspec.GridSpec(2, 2,height_ratios=[1,0.3])
bg=['#EEEEEE','#EEEEEE','#EEEEEE','#EEEEEE']
ax = [plt.subplot(gs[i],axisbg=bg[i],axisbelow=True) for i in range(4)]
plt.subplots_adjust(hspace = .001)

init_F= [0.85,0.9,0.95]
s_fix=1.5
rf=-1
sf=0.1

colors=["#dd1c77","#2c7fb8","#2ca25f"]

for i in range(len(init_F)):

    [x2,y2,c2,ys,yr]=sces(init_F[i],s_fix,rf,0.01)
    [x,y,c,ys,yr]=sces(init_F[i],s_fix,rf)

    ax[0].fill_between(np.array(years[:len(c)]).astype(float),np.array(c).astype(float),np.array(c2).astype(float),\
                         color=colors[i],alpha=0.5)
    ax[0].plot(years[:len(c)],c,linewidth=lw,color=colors[i])
    ax[0].plot(years[:len(c)],c2,linewidth=lw,color=colors[i],label='$\\alpha='+str(init_F[i])+'$')
    if i in {0}:
        ax[2].plot(years[:len(yr)],yr,linewidth=lw,color="#d95f0e",label='$\\rho(t)$')
        ax[2].tick_params(axis='y', colors='#993404')
        ax[2].plot([-10],[-10],linewidth=lw,color="#756bb1",label='$\\sigma(t)$')
        ax2=ax[2].twinx()
        ax2.plot(years[:len(ys)],ys,linewidth=lw,color="#756bb1")
        ax2.tick_params(axis='y', colors='#3f007d')
    if i in {2}:
        ax[0].annotate('$\\zeta = 0\%$', xy=(years[45], c[45]),  xycoords='data',
                xytext=(30, 30), textcoords='offset points',
                arrowprops=dict(arrowstyle="->"))
        ax[0].annotate('$\\zeta = 1\%$', xy=(years[48], c2[48]),  xycoords='data',
                xytext=(50, 36), textcoords='offset points',
                arrowprops=dict(arrowstyle="->"))

    [x2,y2,c2,ys,yr]=dces(init_F[i],s_fix,sf,0.01)
    [x,y,c,ys,yr]=dces(init_F[i],s_fix,sf)

    ax[1].fill_between(np.array(years[:len(c)]).astype(float),np.array(c).astype(float),np.array(c2).astype(float),\
                         color=colors[i],alpha=0.5)
    ax[1].plot(years[:len(c)],c,linewidth=lw,color=colors[i])
    ax[1].plot(years[:len(c)],c2,linewidth=lw,color=colors[i],label='$\\alpha='+str(init_F[i])+'$')
    if i in {0}:
        ax[3].plot(years[:len(yr)],yr,linewidth=lw,color="#d95f0e",label='$\\rho(t)$')
        ax[3].tick_params(axis='y', colors='#993404')
        ax[3].plot([-10],[-10],linewidth=lw,color="#756bb1",label='$\\sigma(t)$')
        ax3=ax[3].twinx()
        ax3.plot(years[:len(ys)],ys,linewidth=lw,color="#756bb1")
        ax3.tick_params(axis='y', colors='#3f007d')
    if i in {2}:
        ax[1].annotate('$\\zeta = 0\%$', xy=(years[64], c[64]),  xycoords='data',
                xytext=(-50, 30), textcoords='offset points',
                arrowprops=dict(arrowstyle="->"))
        ax[1].annotate('$\\zeta = 1\%$', xy=(years[33], c2[33]),  xycoords='data',
                xytext=(-50, 30), textcoords='offset points',
                arrowprops=dict(arrowstyle="->"))

ax[0].grid(color='white', linestyle='solid')
ax[1].grid(color='white', linestyle='solid')
ax[2].grid(color='white', linestyle='solid')
ax[3].grid(color='white', linestyle='solid')
ax[0].set_xlim(2015,2100)
ax[1].set_xlim(2015,2100)
ax[0].set_xticks([2015,2050,2075,2100])
ax[1].set_xticks([2015,2050,2075,2100])
ax[2].set_xlim(2015,2100)
ax[3].set_xlim(2015,2100)
ax[2].set_xticks([2015,2050,2075,2100])
ax[3].set_xticks([2015,2050,2075,2100])
ax[0].set_xticklabels([])
ax[1].set_xticklabels([])
ax[0].set_ylim(0,40)
ax[1].set_ylim(0,40)

ax[2].set_ylim(-1,-.4)
ax[3].set_ylim(-1,-.4)
ax[2].set_yticks([-.5,-.7,-.9])
ax[3].set_yticks([-.5,-.7,-.9])
ax2.set_ylim(0,60)
ax3.set_ylim(0,4500)
ax2.set_yticks([0,20,40,60])
ax3.set_yticks([0,1500,3000,4500])

ax[1].set_yticklabels([])
ax[2].legend(loc=3,framealpha=0.5,fontsize=11)
ax[3].legend(loc=3,framealpha=0.5,fontsize=11)
ax[0].legend(loc=2,framealpha=0.5,fontsize=11)
ax[1].legend(loc=2,framealpha=0.5,fontsize=11)
ax[3].set_xlabel(u'Year\n$\\sigma(t)=\\sigma_0(1+\\sigma_r)^{t-t_0}$\n$\\sigma_0=1.5,$ $\\sigma_r=0.1,$ $t_0=2015$')
ax[2].set_xlabel(u'Year\n$\\rho(t) = \\rho_0+(\\rho_1-\\rho_0)(t-t_0)/(t_1-t_0)$\n$\\rho_0=-0.33,$ $\\rho_1=-1,$ $t_0=2015,$ $t_1=2100$')
ax[0].set_ylabel(u'Marginal abatement cost $MAC$')
#[s_fix*(1+sf)**(i-years[0]) for i in years]
ax[0].text(0.98,0.98,'linearly decreasing $\\rho$',
                horizontalalignment='right',
                verticalalignment='top',
                transform=ax[0].transAxes,fontsize=11)
ax[1].text(0.98,0.98,'exponentially decreasing $\\rho$',
                horizontalalignment='right',
                verticalalignment='top',
                transform=ax[1].transAxes,fontsize=11)

ax[0].set_title('Marginal abatement cost $MAC$\n $MAC=c_R \cdot [\\alpha/(1-\\alpha)] \
                \cdot [F(\\rho)/R(\\rho)]^{-\\rho-1} - c_F,$  $\\sigma = 1/(1+\\rho),$  $\\zeta:yearly$ $decrease$ $in$ $c_R(t)=c_{R0}(1-\\zeta)^{t-t_0}$',size=12,y=1.02,x=1.1)
plt.suptitle(u'Dynamic Elasticity (DES) Energy Substitution Model at Constant Energy Demand $Y$',fontsize=13,y=1.07)
plt.savefig('des1.png',bbox_inches = 'tight', pad_inches = 0.1, dpi=150)
plt.show()
```

```python
fig = plt.subplots(2,2,figsize=(10,6))
lw=2

gs = mpl.gridspec.GridSpec(2, 2,height_ratios=[1,0.3])
bg=['#EEEEEE','#EEEEEE','#EEEEEE','#EEEEEE']
ax = [plt.subplot(gs[i],axisbg=bg[i],axisbelow=True) for i in range(4)]
plt.subplots_adjust(hspace = .4)

init_F= [0.85,0.9,0.95]
s_fix=1.5
rf=-1
sf=0.1

colors=["#dd1c77","#2c7fb8","#2ca25f"]

for i in range(len(init_F)):

    #[x2,y2,c2,ys,yr]=sces(init_F[i],s_fix,rf,0.01)
    [x,y,c,ys,yr]=sces(init_F[i],s_fix,rf)

    ax[0].plot(x,y,linewidth=lw,color=colors[i],label='$\\alpha='+str(init_F[i])+'$')
    #ax[0].plot(x2,y2,linewidth=lw,color=colors[i])
    if i in {0}:
        ax[2].plot(years[:len(yr)],yr,linewidth=lw,color="#d95f0e",label='$\\rho(t)$')
        ax[2].tick_params(axis='y', colors='#993404')
        ax[2].plot([-10],[-10],linewidth=lw,color="#756bb1",label='$\\sigma(t)$')
        ax2=ax[2].twinx()
        ax2.plot(years[:len(ys)],ys,linewidth=lw,color="#756bb1")
        ax2.tick_params(axis='y', colors='#3f007d')
    if i in {2}:
        ax[0].annotate('$\\zeta = 0\%$', xy=(years[45], c[45]),  xycoords='data',
                xytext=(30, 30), textcoords='offset points',
                arrowprops=dict(arrowstyle="->"))
        ax[0].annotate('$\\zeta = 1\%$', xy=(years[48], c2[48]),  xycoords='data',
                xytext=(50, 36), textcoords='offset points',
                arrowprops=dict(arrowstyle="->"))

    #[x2,y2,c2,ys,yr]=dces(init_F[i],s_fix,sf,0.01)
    [x,y,c,ys,yr]=dces(init_F[i],s_fix,sf)

    ax[1].plot(x,y,linewidth=lw,color=colors[i],label='$\\alpha='+str(init_F[i])+'$')
    #ax[1].plot(x2,y2,linewidth=lw,color=colors[i],label='$\\alpha='+str(init_F[i])+'$')
    if i in {0}:
        ax[3].plot(years[:len(yr)],yr,linewidth=lw,color="#d95f0e",label='$\\rho(t)$')
        ax[3].tick_params(axis='y', colors='#993404')
        ax[3].plot([-10],[-10],linewidth=lw,color="#756bb1",label='$\\sigma(t)$')
        ax3=ax[3].twinx()
        ax3.plot(years[:len(ys)],ys,linewidth=lw,color="#756bb1")
        ax3.tick_params(axis='y', colors='#3f007d')
    if i in {2}:
        ax[1].annotate('$\\zeta = 0\%$', xy=(years[64], c[64]),  xycoords='data',
                xytext=(-50, 30), textcoords='offset points',
                arrowprops=dict(arrowstyle="->"))
        ax[1].annotate('$\\zeta = 1\%$', xy=(years[33], c2[33]),  xycoords='data',
                xytext=(-50, 30), textcoords='offset points',
                arrowprops=dict(arrowstyle="->"))

ax[0].grid(color='white', linestyle='solid')
ax[1].grid(color='white', linestyle='solid')
ax[2].grid(color='white', linestyle='solid')
ax[3].grid(color='white', linestyle='solid')
ax[0].set_xlim(0,2000)
ax[1].set_xlim(0,2000)
#ax[0].set_xticks([2015,2050,2075,2100])
#ax[1].set_xticks([2015,2050,2075,2100])
ax[2].set_xlim(2015,2100)
ax[3].set_xlim(2015,2100)
ax[2].set_xticks([2015,2050,2075,2100])
ax[3].set_xticks([2015,2050,2075,2100])
#ax[0].set_xticklabels([])
#ax[1].set_xticklabels([])
ax[0].set_ylim(0,100)
ax[1].set_ylim(0,100)
ax[0].set_xlabel(u'Renewable energy $R$')
ax[1].set_xlabel(u'Renewable energy $R$')
ax[0].set_ylabel(u'Fossil energy $F$')

ax[2].set_ylim(-1,-.4)
ax[3].set_ylim(-1,-.4)
ax[2].set_yticks([-.5,-.7,-.9])
ax[3].set_yticks([-.5,-.7,-.9])
ax2.set_ylim(0,60)
ax3.set_ylim(0,4500)
ax2.set_yticks([0,20,40,60])
ax3.set_yticks([0,1500,3000,4500])

ax[1].set_yticklabels([])
ax[2].legend(loc=3,framealpha=0.5,fontsize=11)
ax[3].legend(loc=3,framealpha=0.5,fontsize=11)
ax[0].legend(loc=2,framealpha=0.5,fontsize=11)
ax[1].legend(loc=2,framealpha=0.5,fontsize=11)
ax[3].set_xlabel(u'Year\n$\\sigma(t)=\\sigma_0(1+\\sigma_r)^{t-t_0}$\n$\\sigma_0=1.5,$ $\\sigma_r=0.1,$ $t_0=2015$')
ax[2].set_xlabel(u'Year\n$\\rho(t) = \\rho_0+(\\rho_1-\\rho_0)(t-t_0)/(t_1-t_0)$\n$\\rho_0=-0.33,$ $\\rho_1=-1,$ $t_0=2015,$ $t_1=2100$')
#[s_fix*(1+sf)**(i-years[0]) for i in years]
ax[0].text(0.98,0.98,'linearly decreasing $\\rho$',
                horizontalalignment='right',
                verticalalignment='top',
                transform=ax[0].transAxes,fontsize=11)
ax[1].text(0.98,0.98,'exponentially decreasing $\\rho$',
                horizontalalignment='right',
                verticalalignment='top',
                transform=ax[1].transAxes,fontsize=11)

ax[0].set_title('$F-R$ curve for $Y=[\\alpha\cdot \
                F^{-\\rho}+(1-\\alpha)\cdot R^{-\\alpha} ]^{-1/\\rho},$ $\\sigma = 1/(1+\\rho)$',size=12,y=1.02,x=1.1)
plt.suptitle(u'Dynamic Elasticity (DES) Energy Substitution Model at Constant Energy Demand $Y$',fontsize=13,y=1.01)
plt.savefig('des1b.png',bbox_inches = 'tight', pad_inches = 0.1, dpi=150)
plt.show()
```

```python
fig = plt.subplots(2,2,figsize=(10,5))
lw=2

gs = mpl.gridspec.GridSpec(2, 2,height_ratios=[1,0.3])
bg=['#EEEEEE','#EEEEEE','#EEEEEE','#EEEEEE']
ax = [plt.subplot(gs[i],axisbg=bg[i],axisbelow=True) for i in range(4)]
plt.subplots_adjust(hspace = .001)

init_F= [0.85,0.9,0.95]
s_fix=1.5
rf=-1
sf=0.1

colors=["#dd1c77","#2c7fb8","#2ca25f"]

for i in range(len(init_F)):

    [x2,y2,c2,ys,yr]=sces(init_F[i],s_fix,rf,0.01,0.03)
    [x,y,c,ys,yr]=sces(init_F[i],s_fix,rf,0.0,0.03)

    ax[0].fill_between(np.array(years[:len(c)]).astype(float),np.array(c).astype(float),np.array(c2).astype(float),\
                         color=colors[i],alpha=0.5)
    ax[0].plot(years[:len(c)],c,linewidth=lw,color=colors[i])
    ax[0].plot(years[:len(c)],c2,linewidth=lw,color=colors[i],label='$\\alpha='+str(init_F[i])+'$')
    if i in {0}:
        ax[2].plot(years[:len(yr)],yr,linewidth=lw,color="#d95f0e",label='$\\rho(t)$')
        ax[2].tick_params(axis='y', colors='#993404')
        ax[2].plot([-10],[-10],linewidth=lw,color="#756bb1",label='$\\sigma(t)$')
        ax2=ax[2].twinx()
        ax2.plot(years[:len(ys)],ys,linewidth=lw,color="#756bb1")
        ax2.tick_params(axis='y', colors='#3f007d')
    if i in {2}:
        ax[0].annotate('$\\zeta = 0\%$', xy=(years[45], c[45]),  xycoords='data',
                xytext=(30, 30), textcoords='offset points',
                arrowprops=dict(arrowstyle="->"))
        ax[0].annotate('$\\zeta = 1\%$', xy=(years[48], c2[48]),  xycoords='data',
                xytext=(50, 36), textcoords='offset points',
                arrowprops=dict(arrowstyle="->"))

    [x2,y2,c2,ys,yr]=dces(init_F[i],s_fix,sf,0.01,0.03)
    [x,y,c,ys,yr]=dces(init_F[i],s_fix,sf,0.0,0.03)

    ax[1].fill_between(np.array(years[:len(c)]).astype(float),np.array(c).astype(float),np.array(c2).astype(float),\
                         color=colors[i],alpha=0.5)
    ax[1].plot(years[:len(c)],c,linewidth=lw,color=colors[i])
    ax[1].plot(years[:len(c)],c2,linewidth=lw,color=colors[i],label='$\\alpha='+str(init_F[i])+'$')
    if i in {0}:
        ax[3].plot(years[:len(yr)],yr,linewidth=lw,color="#d95f0e",label='$\\rho(t)$')
        ax[3].tick_params(axis='y', colors='#993404')
        ax[3].plot([-10],[-10],linewidth=lw,color="#756bb1",label='$\\sigma(t)$')
        ax3=ax[3].twinx()
        ax3.plot(years[:len(ys)],ys,linewidth=lw,color="#756bb1")
        ax3.tick_params(axis='y', colors='#3f007d')
    if i in {2}:
        ax[1].annotate('$\\zeta = 0\%$', xy=(years[64], c[64]),  xycoords='data',
                xytext=(-50, 30), textcoords='offset points',
                arrowprops=dict(arrowstyle="->"))
        ax[1].annotate('$\\zeta = 1\%$', xy=(years[33], c2[33]),  xycoords='data',
                xytext=(-50, 30), textcoords='offset points',
                arrowprops=dict(arrowstyle="->"))

ax[0].grid(color='white', linestyle='solid')
ax[1].grid(color='white', linestyle='solid')
ax[2].grid(color='white', linestyle='solid')
ax[3].grid(color='white', linestyle='solid')
ax[0].set_xlim(2015,2100)
ax[1].set_xlim(2015,2100)
ax[0].set_xticks([2015,2050,2075,2100])
ax[1].set_xticks([2015,2050,2075,2100])
ax[2].set_xlim(2015,2100)
ax[3].set_xlim(2015,2100)
ax[2].set_xticks([2015,2050,2075,2100])
ax[3].set_xticks([2015,2050,2075,2100])
ax[0].set_xticklabels([])
ax[1].set_xticklabels([])
ax[0].set_ylim(0,80)
ax[1].set_ylim(0,80)

ax[2].set_ylim(-1,-.4)
ax[3].set_ylim(-1,-.4)
ax[2].set_yticks([-.5,-.7,-.9])
ax[3].set_yticks([-.5,-.7,-.9])
ax2.set_ylim(0,60)
ax3.set_ylim(0,4500)
ax2.set_yticks([0,20,40,60])
ax3.set_yticks([0,1500,3000,4500])

ax[1].set_yticklabels([])
ax[2].legend(loc=3,framealpha=0.5,fontsize=11)
ax[3].legend(loc=3,framealpha=0.5,fontsize=11)
ax[0].legend(loc=2,framealpha=0.5,fontsize=11)
ax[1].legend(loc=2,framealpha=0.5,fontsize=11)
ax[3].set_xlabel(u'Year\n$\\sigma(t)=\\sigma_0(1+\\sigma_r)^{t-t_0}$\n$\\sigma_0=1.5,$ $\\sigma_r=0.1,$ $t_0=2015$')
ax[2].set_xlabel(u'Year\n$\\rho(t) = \\rho_0+(\\rho_1-\\rho_0)(t-t_0)/(t_1-t_0)$\n$\\rho_0=-0.33,$ $\\rho_1=-1,$ $t_0=2015,$ $t_1=2100$')
ax[0].set_ylabel(u'Marginal abatement cost $MAC$')
#[s_fix*(1+sf)**(i-years[0]) for i in years]
ax[0].text(0.98,0.98,'linearly decreasing $\\rho$',
                horizontalalignment='right',
                verticalalignment='top',
                transform=ax[0].transAxes,fontsize=11)
ax[1].text(0.98,0.98,'exponentially decreasing $\\rho$',
                horizontalalignment='right',
                verticalalignment='top',
                transform=ax[1].transAxes,fontsize=11)

ax[0].set_title('Marginal abatement cost $MAC$\n $MAC=c_R \cdot [\\alpha/(1-\\alpha)] \
                \cdot [F(\\rho)/R(\\rho)]^{-\\rho-1} - c_F,$  $\\sigma = 1/(1+\\rho),$  $\\zeta:yearly$ $decrease$ $in$ $c_R(t)=c_{R0}(1-\\zeta)^{t-t_0}$',size=12,y=1.02,x=1.1)
plt.suptitle(u'Dynamic Elasticity (DES) Energy Substitution Model at Yearly Energy Demand Increase Rate $\\gamma=3\\%$',fontsize=13,y=1.07)
plt.savefig('des2x.png',bbox_inches = 'tight', pad_inches = 0.1, dpi=150)
plt.show()
```

```python
fig = plt.subplots(2,2,figsize=(10,6))
lw=2

gs = mpl.gridspec.GridSpec(2, 2,height_ratios=[1,0.3])
bg=['#EEEEEE','#EEEEEE','#EEEEEE','#EEEEEE']
ax = [plt.subplot(gs[i],axisbg=bg[i],axisbelow=True) for i in range(4)]
plt.subplots_adjust(hspace = .4)

init_F= [0.85,0.9,0.95]
s_fix=1.5
rf=-1
sf=0.1

colors=["#dd1c77","#2c7fb8","#2ca25f"]

for i in range(len(init_F)):

    #[x2,y2,c2,ys,yr]=sces(init_F[i],s_fix,rf,0.01)
    [x,y,c,ys,yr]=sces(init_F[i],s_fix,rf,0.0,0.03)

    ax[0].plot(x,y,linewidth=lw,color=colors[i],label='$\\alpha='+str(init_F[i])+'$')
    #ax[0].plot(x2,y2,linewidth=lw,color=colors[i])
    if i in {0}:
        ax[2].plot(years[:len(yr)],yr,linewidth=lw,color="#d95f0e",label='$\\rho(t)$')
        ax[2].tick_params(axis='y', colors='#993404')
        ax[2].plot([-10],[-10],linewidth=lw,color="#756bb1",label='$\\sigma(t)$')
        ax2=ax[2].twinx()
        ax2.plot(years[:len(ys)],ys,linewidth=lw,color="#756bb1")
        ax2.tick_params(axis='y', colors='#3f007d')

    #[x2,y2,c2,ys,yr]=dces(init_F[i],s_fix,sf,0.01)
    [x,y,c,ys,yr]=dces(init_F[i],s_fix,sf,0.0,0.03)

    ax[1].plot(x,y,linewidth=lw,color=colors[i],label='$\\alpha='+str(init_F[i])+'$')
    #ax[1].plot(x2,y2,linewidth=lw,color=colors[i],label='$\\alpha='+str(init_F[i])+'$')
    if i in {0}:
        ax[3].plot(years[:len(yr)],yr,linewidth=lw,color="#d95f0e",label='$\\rho(t)$')
        ax[3].tick_params(axis='y', colors='#993404')
        ax[3].plot([-10],[-10],linewidth=lw,color="#756bb1",label='$\\sigma(t)$')
        ax3=ax[3].twinx()
        ax3.plot(years[:len(ys)],ys,linewidth=lw,color="#756bb1")
        ax3.tick_params(axis='y', colors='#3f007d')

ax[0].grid(color='white', linestyle='solid')
ax[1].grid(color='white', linestyle='solid')
ax[2].grid(color='white', linestyle='solid')
ax[3].grid(color='white', linestyle='solid')
#ax[0].set_xlim(0,2000)
#ax[1].set_xlim(0,2000)
#ax[0].set_xticks([2015,2050,2075,2100])
#ax[1].set_xticks([2015,2050,2075,2100])
ax[2].set_xlim(2015,2100)
ax[3].set_xlim(2015,2100)
ax[2].set_xticks([2015,2050,2075,2100])
ax[3].set_xticks([2015,2050,2075,2100])
#ax[0].set_xticklabels([])
#ax[1].set_xticklabels([])
ax[0].set_ylim(0,100)
ax[1].set_ylim(0,100)
ax[0].set_xlabel(u'Renewable energy $R$')
ax[1].set_xlabel(u'Renewable energy $R$')
ax[0].set_ylabel(u'Fossil energy $F$')

ax[2].set_ylim(-1,-.4)
ax[3].set_ylim(-1,-.4)
ax[2].set_yticks([-.5,-.7,-.9])
ax[3].set_yticks([-.5,-.7,-.9])
ax2.set_ylim(0,60)
ax3.set_ylim(0,4500)
ax2.set_yticks([0,20,40,60])
ax3.set_yticks([0,1500,3000,4500])

ax[1].set_yticklabels([])
ax[2].legend(loc=3,framealpha=0.5,fontsize=11)
ax[3].legend(loc=3,framealpha=0.5,fontsize=11)
ax[0].legend(loc=2,framealpha=0.5,fontsize=11)
ax[1].legend(loc=2,framealpha=0.5,fontsize=11)
ax[3].set_xlabel(u'Year\n$\\sigma(t)=\\sigma_0(1+\\sigma_r)^{t-t_0}$\n$\\sigma_0=1.5,$ $\\sigma_r=0.1,$ $t_0=2015$')
ax[2].set_xlabel(u'Year\n$\\rho(t) = \\rho_0+(\\rho_1-\\rho_0)(t-t_0)/(t_1-t_0)$\n$\\rho_0=-0.33,$ $\\rho_1=-1,$ $t_0=2015,$ $t_1=2100$')
#[s_fix*(1+sf)**(i-years[0]) for i in years]
ax[0].text(0.98,0.98,'linearly decreasing $\\rho$',
                horizontalalignment='right',
                verticalalignment='top',
                transform=ax[0].transAxes,fontsize=11)
ax[1].text(0.98,0.98,'exponentially decreasing $\\rho$',
                horizontalalignment='right',
                verticalalignment='top',
                transform=ax[1].transAxes,fontsize=11)

ax[0].set_title('$F-R$ curve for $Y=[\\alpha\cdot \
                F^{-\\rho}+(1-\\alpha)\cdot R^{-\\alpha} ]^{-1/\\rho},$ $\\sigma = 1/(1+\\rho)$',size=12,y=1.02,x=1.1)
plt.suptitle(u'Dynamic Elasticity (DES) Energy Substitution Model at Yearly Energy Demand Increase Rate $\\gamma=3\\%$',fontsize=13,y=1.01)
plt.savefig('des2b.png',bbox_inches = 'tight', pad_inches = 0.1, dpi=150)
plt.show()
```