# Model

The generalized Roy model is characterized by the following set of equations.

**Potential Outcomes**

\begin{align}
Y_1 &= X\beta_1 + U_1 \\
Y_0 &= X\beta_0 + U_0
\end{align}

**Cost**

\begin{align}
C = Z\gamma + U_C
\end{align}

**Choice**

\begin{align}
S &= Y_1 - Y_0 - C\\
D &= I[S > 0]
\end{align}

Collecting unobservables:

\begin{align}
V &= U_C - (U_1 - U_0)
\end{align}

Rewriting the choice equation:

\begin{align}
P(X,Z) &= Pr(D = 1 | X,Z) = F_V(X(\beta_1 - \beta_0) - Z\gamma) \\
U_S & = F_V (V)\\
D & = \mathbb{1}\{P(X,Z) > U_S\}
\end{align}

**Observed Outcome**

\begin{align}
Y = D Y_1 + (1 - D)Y_0
\end{align}

$(Y_1, Y_0)$ are objective outcomes associated with each potential treatment state $D$ and realized after the treatment decision. $Y_1$ refers to the outcome in the treated state and $Y_0$ in the untreated state. $C$ denotes the subjective cost of treatment participation. Any subjective benefits,e.g. job amenities, are included (as a negative contribution) in the subjective cost of treatment. Agents take up treatment $D$ if they expect the objective benefit to outweigh the subjective cost. In that case, their subjective evaluation, i.e. the expected surplus from participation $S$, is positive. If agents take up treatment, then the observed outcome $Y$ corresponds to the outcome in the presence of treatment $Y_1$. Otherwise, $Y_0$ is observed. The unobserved potential outcome is referred to as the counterfactual outcome.

## Objects of Interest

**Individual-specific Treatment Effect**

\begin{align}
B = Y_1 - Y_0 = X(\beta_1 - \beta_0) + (U_1 - U_0)
\end{align}

* Heterogeneity
 * Observed
 * Unobserved

**Average Treatment Effect**

\begin{align}
ATE & = E\left[Y_1 - Y_0 \right]\\
TT  & = E\left[Y_1 - Y_0 \mid D = 1\right]\\
TUT & = E\left[Y_1 - Y_0 \mid D = 0\right]
\end{align}

**Marginal Treatment Effect**

The marginal effect parameters condition on the unobserved desire to select into treatment. The *Marginal Benefit*, *Marginal Cost*, and *Marginal Surplus* are defined as:

\begin{align}
B^{MTE}(x,u_S) & = E[Y_1 - Y_0|X = x,U_S = u_S]\\
C^{MTE}(z,u_S) & = E[C|Z = z,U_S = u_S]\\
S^{MTE}(x,z,u_S) & = E[S|X = x,Z = z, U_S = u_S]
\end{align}

**Distribution of Potential Outcomes**

\begin{align}
F_{Y_1,Y_0}
\end{align}

* Distribution of Benefits
 * Heterogeneity
 * Population Shares

## Additional References

Aakvik, A., Heckman, J. J., and Vytlacil, E. J. (2005). Treatment Effects for Discrete Outcomes When Responses to Treatment Vary Among Observationally Identical Persons: An Application to Norwegian Vocational Rehabilitation Programs. *Journal of Econometrics*, 125(1-2):15–51.

Abbring, J. and Heckman, J. J. (2007). Econometric Evaluation of Social Programs, Part III: Distributional Treatment Effects, Dynamic Treatment Effects, Dynamic Discrete Choice, and General Equilibrium Policy Evaluation. In Heckman, J. J. and Leamer, E. E., editors, *Handbook of Econometrics*, volume 6B, pages 5145–5303. Elsevier Science, Amsterdam, Netherlands

Browning, M., Heckman, J. J., and Hansen, L. P. (1999). Micro Data and General Equilibrium Models. In Taylor, J. B. and Woodford, M., editors, *Handbook of Macroeconomics*, volume 1A, pages 543–633. Elsevier Science, Amsterdam, Netherlands.

Carneiro, P., Hansen, K., and Heckman, J. J. (2003). Estimating Distributions of Treatment Effects with an Application to the Returns to Schooling and Measurement of the Effects of Uncertainty on College Choice. *International Economic Review*, 44(2):361–422.

Eisenhauer, P. (2012). Issues in the Economics and Econometrics
of Policy Evaluation: Explorations Using a Factor Structure Model. Unpublished Manuscript.

Heckman, J. J. (2001). Micro Data, Heterogeneity, and the Evaluation of Public Policy: Nobel Lecture. *Journal of Political Economy*, 109(4):673–748.

Heckman, J. J., Smith, J., and Clements, N. (1997). Making the Most Out of Programme Evaluations and Social Experiments: Accounting for Heterogeneity in Programme Impacts. *The Review of Economic Studies*, 64(4):487–535.

Heckman, J. J. and Vytlacil, E. J. (2007a). Econometric Evaluation of Social Programs, Part I: Causal Effects, Structural Models and Econometric Policy Evaluation. In Heckman, J. J. and Leamer, E. E., editors, *Handbook of Econometrics*, volume 6B, pages 4779–4874. Elsevier Science, Amsterdam, Netherlands.

Heckman, J. J. and Vytlacil, E. J. (2007b). Econometric Evaluation of Social Programs, Part II: Using the Marginal Treatment Effect to Organize Alternative Economic Estimators to Evaluate Social Programs and to Forecast their Effects in New Environments. In Heckman, J. J. and Leamer, E. E., editors, *Handbook of Econometrics*, volume 6B, pages 4875–5144. Elsevier Science, Amsterdam, Netherlands.

Quandt, R. E. (1958). The Estimation of the Parameters of a Linear Regression System Obeying two Separate Regimes. *Journal of the American Statistical Association*, 53(284):873–880.

Quandt, R. E. (1958). The Estimation of the Parameters of a Linear Regression System Obeying two Separate Regimes. *Journal of the American Statistical Association*, 53(284):873–880.

**Formatting**

```python
import urllib; from IPython.core.display import HTML
HTML(urllib.urlopen('http://bit.ly/1OKmNHN').read())
```

<link href='http://fonts.googleapis.com/css?family=Alegreya+Sans:100,300,400,500,700,800,900,100italic,300italic,400italic,500italic,700italic,800italic,900italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Arvo:400,700,400italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=PT+Mono' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Shadows+Into+Light' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Philosopher:400,700,400italic,700italic' rel='stylesheet' type='text/css'>

<style>

@font-face {
    font-family: "Computer Modern";
    src: url('http://mirrors.ctan.org/fonts/cm-unicode/fonts/otf/cmunss.otf');
}

#notebook_panel { /* main background */
    background: #888;
    color: #f6f6f6;
}

div.cell { /* set cell width to about 80 chars */
    width: 1000px;
}

div #notebook { /* centre the content */
    background: #fff; /* white background for content */
    width: 1200px;
    margin: auto;
    padding-left: 1em;
}

#notebook li { /* More space between bullet points */
margin-top:0.8em;
}

/* draw border around running cells */
div.cell.border-box-sizing.code_cell.running {
    border: 3px solid #111;
}

/* Put a solid color box around each cell and its output, visually linking them together */
div.cell.code_cell {
    background-color: rgba(171,165,131,0.3);
    border-radius: 10px; /* rounded borders */
    padding: 1em;
    margin-top: 1em;
}

div.text_cell_render{
    font-family: 'Arvo' sans-serif;
    line-height: 130%;
    font-size: 150%;
    width:900px;
    margin-left:auto;
    margin-right:auto;
}

/* Formatting for header cells */
.text_cell_render h1 {
    font-family: 'Philosopher', sans-serif;
    font-weight: 400;
    font-size: 32pt;
    line-height: 100%;
    color: rgb(12,85,97);
    margin-bottom: 0.1em;
    margin-top: 0.1em;
    display: block;
}

.text_cell_render h2 {
    font-family: 'Philosopher', serif;
    font-weight: 700;
    font-size: 24pt;
    line-height: 100%;
    color: rgb(171,165,131);
    margin-bottom: 0.1em;
    margin-top: 0.1em;
    display: block;
}

.text_cell_render h3 {
    font-family: 'Philosopher', serif;
    margin-top:12px;
    margin-bottom: 3px;
    font-style: italic;
    color: rgb(95,92,72);
}

.text_cell_render h4 {
    font-family: 'Philosopher', serif;
}

.text_cell_render h5 {
    font-family: 'Alegreya Sans', sans-serif;
    font-weight: 300;
    font-size: 16pt;
    color: grey;
    font-style: italic;
    margin-bottom: .1em;
    margin-top: 0.1em;
    display: block;
}

.text_cell_render h6 {
    font-family: 'PT Mono', sans-serif;
    font-weight: 300;
    font-size: 10pt;
    color: grey;
    margin-bottom: 1px;
    margin-top: 1px;
}

.CodeMirror{
        font-family: "PT Mono";
        font-size: 120%;
}

</style>


# Optimizer tweaks

```
%load_ext autoreload
%autoreload 2

%matplotlib inline
```

```
#export
from exp.nb_08 import *
```

## Imagenette data

We grab the data from the previous notebook.

```
path = datasets.untar_data(datasets.URLs.IMAGENETTE_160)
```

```
tfms = [make_rgb, ResizeFixed(128), to_byte_tensor, to_float_tensor]
il = ImageItemList.from_files(path, tfms=tfms)
sd = SplitData.split_by_func(il, partial(grandparent_splitter, valid_name='val'))
ll = label_by_func(sd, parent_labeler)
```

```
bs=64
```

```
train_dl,valid_dl = get_dls(ll.train,ll.valid,bs, num_workers=8)
```

```
x,y = next(iter(valid_dl))
```

```
show_image(x[0])
ll.train.y.processor.vocab[y[0]]
```

```
nfs = [32,64,128,256,512]
```

```
sched = combine_scheds([0.25, 0.75], [sched_cos(0.4/25, 0.4), sched_cos(0.4, 0.)])
```

```
cbfs = [partial(AvgStatsCallback,accuracy),
        CudaCallback,
        partial(BatchTransformXCallback, norm_imagenette),
        partial(ParamScheduler, 'lr', sched)]
```

```
data = DataBunch(train_dl, valid_dl, 3, 10)
```

This is the baseline of training with vanilla SGD.

```
learn,run = get_learn_run(nfs, data, 0.4, conv_layer, cbs=cbfs)
```

```
run.fit(3, learn)
```

    train: [1.9124067213917326, tensor(0.3359, device='cuda:0')]
    valid: [2.0151365966796875, tensor(0.3040, device='cuda:0')]
    train: [1.3411626471129983, tensor(0.5552, device='cuda:0')]
    valid: [1.173794921875, tensor(0.6240, device='cuda:0')]
    train: [0.9769951897442609, tensor(0.6884, device='cuda:0')]
    valid: [0.938296630859375, tensor(0.7280, device='cuda:0')]

## Weight decay

Weight decay comes from the idea of L2 regularization, which consists in adding to your loss function the sum of all the weights squared. Why do that? Because when we compute the gradients, it will add a contribution to them that will encourage the weights to be as small as possible.

Why would it prevent overfitting? The idea is that the larger the coefficient are, the more sharp canyons we will have in the loss function. If we take the basic example of parabola, `y = a * (x**2)`, the larger `a` is, the more *narrow* the parabola is.

```
x = torch.linspace(-2,2,100)
a_s = [1,2,5,10,50]
ys = [a * x**2 for a in a_s]
_,ax = plt.subplots()
for a,y in zip(a_s,ys): ax.plot(x,y, label=f'a={a}')
ax.set_ylim([0,5])
ax.legend();
```

So by letting our model learn high parameters, it might fit all the data points in the training set with an over-complex function that has very sharp changes, which will lead to overfitting.

Limiting our weights from growing to much is going to hinder the training of the model, but it will yield to a state where it generalizes better. Going back to the theory a little bit, weight decay (or just `wd`) is a parameter that controls that sum of squares we add to our loss:
``` python
loss_with_wd = loss + (wd / 2) * sum([(p ** 2).sum() for p in model_parameters])
```

In practice though, it would be very inefficient (and maybe numerically unstable) to compute that big sum and add it to the loss. If you remember a little bit of high schoool math, you should now that the derivative of `p ** 2` with respect to `p` is simple `2 * p`, so adding that big sum to our loss is exactly the same as doing
``` python
weight.grad += wd * weight
```

for every weight in our model, which is equivalent to (in the case of vanilla SGD) updating the parameters
with
``` python
new_weight = weight - lr * weight.grad - lr * wd * weight
```

This last formula explains why the name of this technique is weight decay, as each weight is decayed by a factor `lr * wd`.

This only works for standard SGD, as we have seen that with momentum, RMSProp or in Adam, the update has some additional formulas around the gradient. In those cases, the formula that comes from L2 regularization:
``` python
weight.grad += wd * weight
```
is different than weight decay
``` python
new_weight = weight - lr * weight.grad - lr * wd * weight
```

Most libraries use the first one, but as it was pointed out in [Decoupled Weight Regularization](https://arxiv.org/pdf/1711.05101.pdf) by Ilya Loshchilov and Frank Hutter, it is better to use the second one with the Adam optimizer, which is why fastai made it its default.

```
class Optimizer(optim.Optimizer):
    def __init__(self, params, steppers, **defaults):
        super().__init__(params, defaults)
        self.steppers = listify(steppers)

    def step(self):
        for pg in self.param_groups:
            for p in pg['params']:
                if p.grad is not None: compose(p, self.steppers, pg=pg)
```

Weight decay is substracting `lr x wd x weights` to the weights

```
class WeightDecay():
    _defaults = dict(wd=0.)
    def __call__(self,p,pg):
        p.data.mul_(1 - pg['lr'] * pg['wd'])
        return p
```

L2 regularization is adding `wd x weight` to the gradients.

```
class L2_Reg():
    _defaults = dict(wd=0.)
    def __call__(self,p,pg):
        p.grad.data.add_(pg['wd'], p.data)
        return p
```

And this is the classic SGD step.

```
def sgd_step(p, pg):
    p.data.add_(-pg['lr'], p.grad.data)
    return p
```

A stepper may introduce new hyperparameters so we associate a `_defaults` variable to it to make sure it's present in the param groups.

```
class Optimizer(optim.Optimizer):
    def __init__(self, params, steppers, **defaults):
        self.steppers = listify(steppers)
        stepper_defaults = {}
        for stepper in self.steppers: stepper_defaults.update(getattr(stepper,'_defaults',{}))
        super().__init__(params, {**stepper_defaults, **defaults})

    def step(self):
        for pg in self.param_groups:
            for p in pg['params']:
                if p.grad is not None: compose(p, self.steppers, pg=pg)
```

```
opt_func = partial(Optimizer, steppers=[WeightDecay(), sgd_step])
```

```
model = learn.model
```

```
opt = opt_func(model.parameters(), lr=0.1)
opt.param_groups[0]['wd'],opt.param_groups[0]['lr']
```

    (0.0, 0.1)

```
opt = opt_func(model.parameters(), lr=0.1, wd=1e-4)
opt.param_groups[0]['wd']
```

    0.0001

```
learn,run = get_learn_run(nfs, data, 0.4, conv_layer, cbs=cbfs, opt_func=partial(opt_func, wd=0.01))
```

```
run.fit(3, learn)
```

    train: [1.8981493936133085, tensor(0.3393, device='cuda:0')]
    valid: [1.958132568359375, tensor(0.3380, device='cuda:0')]
    train: [1.4495471488676903, tensor(0.5189, device='cuda:0')]
    valid: [1.43742529296875, tensor(0.5160, device='cuda:0')]
    train: [1.0363493467164961, tensor(0.6788, device='cuda:0')]
    valid: [0.894203369140625, tensor(0.7360, device='cuda:0')]

### With momentum

Momentum requires to add some state. We need to save the moving average of the gradients to be able to do the step and store this inside the optimizer state if we want it saved by PyTorch (when doing checkpointing).

```
class StatefulOptimizer(optim.Optimizer):
    def __init__(self, params, steppers, stats=None, **defaults):
        self.steppers,self.stats = listify(steppers),listify(stats)
        base_defaults = {}
        for stepper in self.steppers: base_defaults.update(getattr(stepper,'_defaults',{}))
        for stat in self.stats:       base_defaults.update(getattr(stat,'_defaults',{}))
        super().__init__(params, {**base_defaults, **defaults})

    def step(self):
        for pg in self.param_groups:
            for p in pg['params']:
                if p.grad is not None:
                    if p not in self.state:
                        init_state = {}
                        for stat in self.stats: init_state.update(stat.init_state(p))
                        self.state[p] = init_state
                    state = self.state[p]
                    for stat in self.stats: state = stat.update(p, pg, state)
                    compose(p, self.steppers, pg=pg, state=state)
                    self.state[p] = state
```

```
class Stat():
    _defaults = {}
    def init_state(self, p):        raise NotImplementedError
    def update(self, p, pg, state): raise NotImplementedError
```

```
class AverageGrad(Stat):
    _defaults = dict(mom=0.9)

    def init_state(self, p): return {'grad_avg': torch.zeros_like(p.grad.data)}
    def update(self, p, pg, state):
        state['grad_avg'].mul_(pg['mom']).add_(p.grad.data)
        return state
```

We update the previous classes/functions to take the new `state` argument.

```
class WeightDecay():
    _defaults = dict(wd=0.)
    def __call__(self,p,pg,state):
        p.data.mul_(1 - pg['lr'] * pg['wd'])
        return p
```

```
class L2_Reg():
    _defaults = dict(wd=0.)
    def __call__(self,p,pg,state):
        p.grad.data.add_(pg['wd'], p.data)
        return p
```

```
def sgd_step(p, pg,state):
    p.data.add_(-pg['lr'], p.grad.data)
    return p
```

Then we add the momentum step:

```
def momentum_step(p, pg, state):
    p.data.add_(-pg['lr'], state['grad_avg'])
    return p
```

```
sgd_mom = partial(StatefulOptimizer, steppers=momentum_step, stats=AverageGrad())
```

```
learn,run = get_learn_run(nfs, data, 0.4, conv_layer, cbs=cbfs, opt_func=sgd_mom)
```

```
run.fit(3, learn)
```

    train: [1.8154774022801303, tensor(0.3647, device='cuda:0')]
    valid: [1.7109781494140626, tensor(0.4300, device='cuda:0')]
    train: [1.3523889502869553, tensor(0.5452, device='cuda:0')]
    valid: [1.12460888671875, tensor(0.6160, device='cuda:0')]
    train: [1.00159814218338, tensor(0.6681, device='cuda:0')]
    valid: [0.8756005859375, tensor(0.7140, device='cuda:0')]

### Momentum experiments

What does momentum do to the gradients exactly? Let's do some plots to find out!

```
x = torch.linspace(-4, 4, 200)
y = torch.randn(200) + 0.3
betas = [0.5,0.7,0.9,0.99]
```

```
def plot_mom(f):
    _,axs = plt.subplots(2,2, figsize=(12,8))
    for beta,ax in zip(betas, axs.flatten()):
        ax.plot(y, linestyle='None', marker='.')
        avg,res = None,[]
        for i,yi in enumerate(y):
            avg,p = f(avg, beta, yi, i)
            res.append(p)
        ax.plot(res, color='red')
        ax.set_title(f'beta={beta}')
```

This is the regular momentum.

```
def mom1(avg, beta, yi, i):
    if avg is None: avg=yi
    res = beta * avg + yi
    return res,res
plot_mom(mom1)
```

As we can see, with a too high value, it may go way too high with no way to change its course.

Another way to smooth noisy data is to do an exponentially moving average.

```
def mom2(avg, beta, yi, i):
    if avg is None: avg=yi
    avg = beta * avg + (1-beta) * yi
    return avg, avg
plot_mom(mom2)
```

We can see it gets to a zero-constant when the data is purely random. If the data has a certain shape, it will get that shape (with some delay for high beta).

```
y = 1 - (x/3) ** 2 + torch.randn(200) * 0.1
```

```
y[0]=0.5
```

```
plot_mom(mom2)
```

Debiasing is here to correct the wrong information we may have in the very first batch.

```
def mom3(avg, beta, yi, i):
    if avg is None: avg=0
    avg = beta * avg + (1-beta) * yi
    return avg, avg/(1-beta**(i+1))
plot_mom(mom3)
```

### Adam and friends

In Adam, we use the gradient averages but with dampening, so let's add this to the `AverageGrad` class.

```
class AverageGrad(Stat):
    _defaults = dict(mom=0.9)

    def __init__(self, dampening:bool=False): self.dampening=dampening
    def init_state(self, p): return {'grad_avg': torch.zeros_like(p.grad.data)}
    def update(self, p, pg, state):
        pg['mom_damp'] = 1 - pg['mom'] if self.dampening else 1.
        state['grad_avg'].mul_(pg['mom']).add_(pg['mom_damp'], p.grad.data)
        return state
```

We also need to track the moving average of the gradients squared.

```
class AverageSqrGrad(Stat):
    _defaults = dict(sqr_mom=0.99)

    def __init__(self, dampening:bool=True): self.dampening=dampening
    def init_state(self, p): return {'sqr_avg': torch.zeros_like(p.grad.data)}
    def update(self, p, pg, state):
        pg['sqr_damp'] = 1 - pg['sqr_mom'] if self.dampening else 1.
        state['sqr_avg'].mul_(pg['sqr_mom']).addcmul_(pg['sqr_damp'],p.grad.data,p.grad.data)
        return state
```

We also need the number of steps done during training for the debiasing.

```
class StepCount(Stat):
    def init_state(self, p): return {'step': 0}
    def update(self, p, pg, state):
        state['step'] += 1
        return state
```

```
def debias(mom, damp, step): return damp * (1 - mom**step) / (1-mom)
```

Then the Adam step is just the following:

```
class AdamStep():
    _defaults = dict(eps=1e-5)
    def __call__(self, p, pg, state):
        debias1 = debias(pg['mom'],     pg['mom_damp'], state['step'])
        debias2 = debias(pg['sqr_mom'], pg['sqr_damp'], state['step'])
        p.data.addcdiv_(-pg['lr'] / debias1, state['grad_avg'], (state['sqr_avg']/debias2 + pg['eps']).sqrt())
        return p
```

```
adam = partial(StatefulOptimizer, steppers=AdamStep(), stats=[AverageGrad(dampening=True), AverageSqrGrad(), StepCount()])
```

```
learn,run = get_learn_run(nfs, data, 0.1, conv_layer, cbs=cbfs, opt_func=adam)
```

```
run.fit(3, learn)
```

    train: [2.089980835320692, tensor(0.2443, device='cuda:0')]
    valid: [2.606945068359375, tensor(0.2740, device='cuda:0')]
    train: [1.8419557727916085, tensor(0.3501, device='cuda:0')]
    valid: [1.6391795654296875, tensor(0.4460, device='cuda:0')]
    train: [1.4618398395086862, tensor(0.5004, device='cuda:0')]
    valid: [1.3107017822265625, tensor(0.5980, device='cuda:0')]

It's then super easy to implement a new optimizer. This is LAMB from a [very recent paper](https://arxiv.org/pdf/1904.00962.pdf):

$\begin{align}
g_{t}^{l} &= \nabla L(w_{t-1}^{l}, x_{t}) \\
m_{t}^{l} &= \beta_{1} m_{t-1}^{l} + (1-\beta_{1}) g_{t}^{l} \\
v_{t}^{l} &= \beta_{2} v_{t-1}^{l} + (1-\beta_{2}) g_{t}^{l} \odot g_{t}^{l} \\
m_{t}^{l} &= m_{t}^{l} / (1 - \beta_{1}^{t}) \\
v_{t}^{l} &= v_{t}^{l} / (1 - \beta_{2}^{t}) \\
r_{1} &= \|w_{t-1}^{l}\|_{2} \\
s_{t}^{l} &= \frac{m_{t}^{l}}{\sqrt{v_{t}^{l} + \epsilon}} + \lambda w_{t-1}^{l} \\
r_{2} &= \| s_{t}^{l} \|_{2} \\
\eta^{l} &= \eta * r_{1}/r_{2} \\
w_{t}^{l} &= w_{t}^{l-1} - \eta_{l} * s_{t}^{l} \\
\end{align}$

```
class LambStep():
    _defaults = dict(eps=1e-6, wd=0.)
    def __call__(self, p, pg, state):
        debias1 = debias(pg['mom'],     pg['mom_damp'], state['step'])
        debias2 = debias(pg['sqr_mom'], pg['sqr_damp'], state['step'])
        r1 = p.data.pow(2).mean().sqrt()
        step = (state['grad_avg']/ debias1) / (state['sqr_avg']/debias2 + pg['eps']).sqrt() + pg['wd'] * p.data
        r2 = step.pow(2).mean().sqrt()
        p.data.add_(-pg['lr'] * min(r1/r2,10), step)
        return p
```

```
lamb = partial(StatefulOptimizer, steppers=LambStep(), stats=[AverageGrad(dampening=True), AverageSqrGrad(), StepCount()])
```

```
learn,run = get_learn_run(nfs, data, 0.4, conv_layer, cbs=cbfs, opt_func=lamb)
```

```
run.fit(3, learn)
```

    train: [2.1830554060997365, tensor(0.2314, device='cuda:0')]
    valid: [1.9674951171875, tensor(0.3460, device='cuda:0')]
    train: [1.9502472685939196, tensor(0.3281, device='cuda:0')]
    valid: [1.61225732421875, tensor(0.4560, device='cuda:0')]
    train: [1.5729571106231581, tensor(0.4596, device='cuda:0')]
    valid: [1.42828564453125, tensor(0.5340, device='cuda:0')]

Other recent variants of optimizers:
- [Large Batch Training of Convolutional Networks](https://arxiv.org/abs/1708.03888) (LARS also uses weight statistics, not just gradient statistics. Can you add that to this class?)
- [Adafactor: Adaptive Learning Rates with Sublinear Memory Cost](https://arxiv.org/abs/1804.04235) (Adafactor combines stats over multiple sets of axes)
- [Adaptive Gradient Methods with Dynamic Bound of Learning Rate](https://arxiv.org/abs/1902.09843)

## Export

```
!python notebook2script.py 09_optimizers.ipynb
```

    Converted 09_optimizers.ipynb to nb_09.py


# Leistungsanpassung und Spannungsregelung

```python
# Bibliotheken laden
import numpy as np
import matplotlib.pyplot as plt
import cmath # für komplexe Zahlen
# Graphikparameter
plt.style.use('classic')
plt.rcParams.update({'font.size':24})
plt.rcParams.update({'figure.autolayout': True})
```

Eine Quelle hat eine bestimmte Kennlinie. Dies gilt auch bei Wechselspannungssystemen. Wie man die Quelle an den Bedarf der Last anpasst, beschreibt der Begriff der **Leistungsanpassung**. Außerdem hat die Blindleistung einer Last auch einen Effekt auf die Netzspannung. Dies wird zur **Spannungsregelung** herangezogen.

In diesem Notebook geht es um beide Effekte.

## Leistungsanpassung

Das folgende Schaltbild zeigt den prinzipiellen Aufbau einer Quelle mit Innenimpedanz $\underline{Z_i}=R_i+jX_i$, an der ein Verbraucher mit der Impedanz $\underline{Z_V}=R_V+jX_V$ angeschlossen ist:

Die Frage ist nun, wann die maximale Leistung aus der Quelle abgegeben werden kann.

### Fall 1: Verbraucher mit veränderbarem $R_V$ und veränderbarem $X_V$

```python
# Zeitvektor
t = np.linspace(0,60e-3,100000) # 0 bis 60 ms in 100.000 Schritten
# Spannungsquelle
Uq = 230
f = 50
omega = 2*np.pi*f
uq = Uq*np.sqrt(2)*np.cos(omega*t) # Phase 0 angenommen
# Innenwiderstand und Innenreaktanz
Ri = 3 # kann nur positiv sein! Anfangswert 3 Ohm
Xi = 2 # kann positiv oder negativ sein, Anfangswert 2 Ohm
# Verbraucherwiderstand und Verbraucherreaktanz
RV = 5 # kann nur positiv sein! Anfangswert 5 Ohm
XV = 8 # kann positiv oder negativ sein, Anfangswert 8 Ohm
# Strom berechnen
Ikomplex = Uq/complex(Ri+RV,Xi+XV)
# Ikomplex = Ikomplexbetrag * exp(j*phaseIkomplex)
i = np.sqrt(2)*np.abs(Ikomplex)*np.cos(omega*t + cmath.phase(Ikomplex))
# Spannung an der Last berechnen
UVkomplex = Ikomplex*complex(RV,XV)
uv = np.sqrt(2)*np.abs(UVkomplex)*np.cos(omega*t + cmath.phase(UVkomplex))
# Leistung berechnen
pq = uq*i # Augenblicksleistung der Quelle (Abgabe)
Pq = np.mean(pq) # Wirkleistung der Quelle (Abgabe)
pV = uv*i # Augenblicksleistung der Last (Aufnahme)
PV = np.mean(pV) # Wirkleistungsaufnahme der Last

# Diagramme von uq, uv, i, pq, pv
# - wir machen 5 Diagramme untereinander
# - jeweils blau ist der Zeitverlauf
# - und grün der Mittelwert
fig, ax = plt.subplots(5, 1, sharex=True, sharey=False, figsize=(8, 8))
# Quellspannung uq
ax[0].plot(1000*t,uq)
ax[0].set_ylabel(r'$u_q [V]$')
ax[0].grid()
# Strom i
ax[1].plot(1000*t,i)
ax[1].set_ylabel(r'$i [A]$')
ax[1].grid()
# Verbraucherspannung uv
ax[2].plot(1000*t,uv)
ax[2].set_ylabel(r'$u_v [V]$')
ax[2].grid()
# Leistungsabgabe der Quelle
ax[3].plot(1000*t,0.001*pq)
ax[3].plot(1000*t,0.001*Pq*np.ones(len(t)))
ax[3].set_ylabel(r'$p_q [kW]$')
ax[3].grid()
# Leistungsaufnahme der Last
ax[4].plot(1000*t,0.001*pV)
ax[4].plot(1000*t,0.001*PV*np.ones(len(t)))
ax[4].set_ylabel(r'$p_V [kW]$')
ax[4].grid()
ax[4].set_xlabel(r'$t [ms]$')
```

Wir geben nun auch die Wirkleistung, Blindleistung und Scheinleistung an Quelle und Last aus. Dafür beachten wird, dass die komplexe Scheinleistung der Quelle sich aus der Phase des Stroms ergibt:

\begin{equation}
\underline{S_q}=\underline{U_q}\cdot\underline{I^*}=U_q\cdot I\cdot e^{-j\varphi_i} = S\cdot e^{-j\varphi_i}
\end{equation}

Die Phase des Stroms ist:

```python
phii=cmath.phase(Ikomplex)
print('phii=', phii,' rad')
```

    phii= -0.8960553845713439  rad

**Leistungen an der Quelle**:

```python
Sq = Uq*np.abs(Ikomplex)*complex(np.cos(-phii),np.sin(-phii))
print('Sq=',0.001*Sq,'kVA')
```

    Sq= (2.580487804878049+3.225609756097561j) kVA

```python
Qq = Sq.imag
print('Qq=',0.001*Qq,'kvar')
```

    Qq= 3.225609756097561 kvar

```python
print('Pq=',0.001*Pq,'kW')
```

    Pq= 2.580513609756097 kW

Wir geben nun auch die Wirkleistung, Blindleistung und Scheinleistung an der Last an. Dafür beachten wir, dass die komplexe Scheinleistung an der Last sich aus der Phase des Stroms und der Spannung ergibt:

\begin{equation}
\underline{S_V}=\underline{U_V}\cdot\underline{I_V^*} = U_V\cdot e^{j\varphi_u}\cdot I\cdot e^{-j\varphi_i} = U_V \cdot I \cdot e^{j\varphi_u - \varphi_i}
\end{equation}

**Leistungen am Verbraucher**:

```python
# Phase der Spannung am Verbraucher
phiuv = cmath.phase(UVkomplex)
# Scheinleistung am Verbraucher
SV = np.abs(UVkomplex)*np.abs(Ikomplex)*complex(np.cos(phiuv-phii),np.sin(phiuv-phii))
print('SV=',0.001*SV,'kVA')
```

    SV= (1.612804878048781+2.580487804878049j) kVA

```python
QV = SV.imag
print('QV=',0.001*QV,'kvar')
```

    QV= 2.580487804878049 kvar

```python
print('PV=',0.001*PV,'kW')
```

    PV= 1.6128265132361688 kW

**Aufgabe: Verändern Sie zunächst `XV` so, dass die Leistung $P_V$ möglichst groß wird, während Sie `RV` konstant lassen. Welchen Wert muss `XV` dann haben und notieren Sie zudem nochmals den Wert von `Xi`.**

*Ergebnis:*
*`XV` muss den Wert ... haben, wenn `Xi` den Wert ... hat, damit die Leistung maximal wird.*

**Aufgabe: Behalten Sie den soeben ermittelten Wert von `XV` bei. Ändern Sie nun `RV` so, dass die Leistung $P_V$ möglichst groß wird, während Sie alle anderen Werte konstant lassen. Welchen Wert muss `RV` dann haben und notieren Sie zudem nochmals den Wert von `Ri`.**

*Ergebnis: RV muss den Wert ... haben, wenn Ri den Wert ... hat, damit die Leistung maximal wird.*

**Aufgabe: Ermitteln Sie die Phase der Verbraucherimpedanz und der Quellimpedanz und vergleichen Sie.**

*Ergebnis:*

### Fall 2: Verbraucher mit veränderbarem $R_V$ und $X_V=0$ (reiner Wirkwiderstand)

**Aufgabe: Setzen Sie nun in der obigen Berechnung für $X_V=0$ ein. Und ermitteln Sie wieder, bei welchem `RV` bezogen auf `Ri` sich die maximale Leistung einstellt.**

*Ergebnis: RV muss den Wert ... haben, wenn Ri den Wert ... hat, damit die Leistung maximal wird.*

### Fall 3: Verbraucher mit $R_V=0$ und veränderbarem $X_V$ (reiner Blindwiderstand)

**Aufgabe: Setzen Sie nun in der obigen Berechnung für $R_V=0$ ein. Und ermitteln Sie wieder, bei welchem `XV` bezogen auf `Xi` sich die maximale Leistung einstellt.**

*Ergebnis:*

## Spannungsregelung mit Blindleistung

Wir betrachten nun das elektrische Stromnetz als ideale Quelle und eine Leitung mit einer Induktivität (das einfachste Ersatzschaltbild für eine Leitung, siehe Skript).

Nun regeln wir am Verbraucher die Blindleistung so, dass die Spannung sich auf der Leitung auf der Verbraucherseite besser im Spannungsband hält. Das Spannungsband beträgt im Niederspannungsnetz $\pm 10\%$ der Nennspannung.

```python
# Leitungsreaktanz pro Leitungslänge
l = np.linspace(10e3,50e3,1000) # 10 bis 50 km in 1000 Schritten
xL = 50e-6 # 50 uOhm/km
XL = l*xL
# Verbraucherwiderstand und Verbraucherreaktanz
RV = 5 # kann nur positiv sein! Anfangswert 5 Ohm
XV = 8 # kann positiv oder negativ sein, Anfangswert 8 Ohm
# Strom berechnen
Ikompl = Uq/(RV+1j*(XL+XV))
# Spannung an der Last berechnen
UVkompl = Ikompl*(RV+1j*XV)
# Spannung an der Leitung berechnen
ULkompl = Ikompl*1j*XL
# Leistung berechnen
SVkompl = UVkompl*np.conj(Ikomplex)
SLkompl = ULkompl*np.conj(Ikomplex)
PVkompl = SVkompl.real
PLkompl = SLkompl.real
QVkompl = SVkompl.imag
QLkompl = SLkompl.imag

# Diagramme von uq, uv, i, pq, pv
# - wir machen 5 Diagramme untereinander
# - jeweils blau ist der Zeitverlauf
# - und grün der Mittelwert
fig, ax = plt.subplots(2, 1, sharex=True, sharey=False, figsize=(8, 5))
# Strombetrag
ax[0].plot(0.001*l,np.abs(Ikompl))
ax[0].set_ylabel(r'$I$ [A]')
ax[0].grid()
# Spannung am Leitungsende beim Verbraucher
ax[1].plot(0.001*l,np.abs(UVkompl),label=r'$U_V$ [V]')
ax[1].plot(0.001*l,0.9*Uq*np.ones(len(l)),label='untere Grenze')
ax[1].plot(0.001*l,1.1*Uq*np.ones(len(l)),label='obere Grenze')
ax[1].set_xlabel(r'Leitungslänge von der Quelle aus [km]')
ax[1].grid()
ax[1].legend()
```

Mit unseren Zahlenwerten sinkt die Spannung ab 25 km Leitungslänge unterhalb den erlaubten Spannungswert. Ob dies tatsächlich nach dieser Leitungslänge bereits der Fall ist, hängt davon ab, wie viele Lasten angeschlossen sind (die die Spannung verringeren) und wie viele Erzeuger angeschlossen sind (die die Spannung wiederum erhöhen). Genauer lernen Sie das im Modul elektrische Netze. Hier soll es nur darum gehen, wie man gegensteuern kann, wenn die Spannung zu hoch oder zu tief wird.

**Aufgabe: Verändern Sie die Reaktanz des Verbrauchers `XV` so, dass das Spannungsband wieder eingehalten wird bis zur Leitungslänge $l=50\text{ km}$.**

*Ergebnis: Die Reaktanz des Verbrauchers muss von 8 Ohm auf ... verändert werden, damit das Spannungsband wieder eingehalten werden kann. Sie darf in negativer Richtung auf ... verändert werden, da sonst das Spannungsband in der anderen Richtung verletzt wird.*

**Aufgabe: Welches Bauteil benötigen Sie, um die Reaktanz in der beschriebenen Weise zu verändern? Dimensionieren Sie dieses Bauteil.**

*Ergebnis: ich benötige eine Induktivität / eine Kapazität der Größe ... um die Spannungsgrenze nach unten einzuhalten und ich benötige eine Induktivität / eine Kapazität der Größe ... um die Spannungsgrenze nach oben einzuhalten.*

## Leistung bei Spannungserhöhung mit kapazitiver Blindleistung

Zunächst betrachten wir die Leitung mit der Induktivität $X_L$, vernachlässigbarem Ohmschem Widerstand $R_L=0$ und der Last mit $X_V=0$. Die Last $R_V$ wird nun variiert, um den Effekt der Last auf die Spannung am Verbraucheranschluss zu ermitteln.

```python
# Hier nochmal alle Größen neu definieren, der Übersicht halber
# Netzspannung
Uq = 230
# Ohmscher Widerstand der Leitung ist 0
RL = 0
# Leitungsreaktanz
XL = .1 # Anfangswert .1 Ohm
# Ohmscher Widerstand am Verbraucher
RV = np.append(0,np.geomspace(.0001,1e12,1000)) # von 0 bis fast unendlich in vielen Schritten
# Verbraucherreaktanz vernachlässigbar ist 0
XV = 0
# Strom ohne Kondensator
Iohne = Uq/(RL+RV+1j*(XL+XV)) # 1j ist die komplexe Einheit, complex geht nur für Skalare
# Spannung am Verbraucher ohne Kondensator
UVohne = Iohne*(RV+1j*XV)
# Leistung am Verbraucher ohne Kondensator
SVohne = UVohne*np.conj(Iohne)
PVohne = SVohne.real # Wirkleistung
# Kondensatorreaktanz (negativ!)
XC1 = -1
XC2 = -.5
# Strom mit Kondensator
Imit1 = Uq/(RL+ 1j*XL + (((RV + 1j*XV)*(1j*XC1))/(RV + 1j*(XC1+XV)))) # ZL + ZV || ZC
Imit2 = Uq/(RL+ 1j*XL + (((RV + 1j*XV)*(1j*XC2))/(RV + 1j*(XC2+XV))))
# Spannung am Verbraucher ohne Kondensator
UVmit1 = Imit1*(RV+1j*XV)
UVmit2 = Imit2*(RV+1j*XV)
# Leistung am Verbraucher mit Kondensator
SVmit1 = UVmit1*np.conj(Imit1)
SVmit2 = UVmit2*np.conj(Imit2)
PVmit1 = SVmit1.real
PVmit2 = SVmit2.real

# normierte Größen: Spannung
#uvohnew = UVohneW/Uq
uvohne = np.abs(UVohne)/Uq
uvmit1 = np.abs(UVmit1)/Uq
uvmit2 = np.abs(UVmit2)/Uq
# normierte Größen: Leistung
pvohne = PVohne/np.max(PVohne)
pvmit1 = PVmit1/np.max(PVohne)
pvmit2 = PVmit2/np.max(PVohne)

# Diagramm
plt.plot(pvohne,uvohne,'b-')
plt.plot(pvmit1,uvmit1,'b--')
plt.plot(pvmit2,uvmit2,'b:')
plt.xlabel('$P_V/P_N$')
plt.ylabel('$U_V/U_N$')
plt.axis([0,1.4*max(pvohne),0,1.4])
plt.plot([0,1.4],0.9*np.ones(2),'r-')
plt.plot([0,1.4],1.1*np.ones(2),'r-')
plt.grid()
```

**Aufgabe: Welche Schwierigkeit hat der Netzbetreiber bei einer zeitlich variablen Last, die zwischen 80 und 130% der Nennleistung variiert?**

*Antwort: Wenn die Last zunimmt von 0% bis 80% dann sinkt die Spannung bei 80% auf die untere Spannungsgrenze und kapazitive Blindleistung wird nötig (z. B. gestrichelte Linie). Dann nimmt die Last weiter zu auf 110% und die Spannung ist wieder so niedrig, dass ... . Ab einer gewissen Blindleistung (gepunktete Linie) ist der Zusammenhang zwischen Leistung und Spannung zweideutig: Eine Verringerung der Leistung kann sowohl zur Erhöhung der Spannung als auch zur Verringerung der Spannung führen. Es kann daher passieren, dass ...*