# Homework 4

This Notebook builds on the DCOPF model introduced in [Notebook 6](https://github.com/east-winds/power-systems-optimization/tree/master/Notebooks) and incorporates some elements of Economic Dispatch introduced in [Notebook 4](https://github.com/east-winds/power-systems-optimization/tree/master/Notebooks).

First, load (or install if necessary) a set of packages you'll need for this assignment...

```julia
using JuMP
using GLPK
using DataFrames
using CSV
using Plots; plotly();
```

## Question 1: Modifying IEEE-14

**A. Increased generation costs**

Copy the IEEE 14 bus system and DCOPF solver function from Notebook 6. In addition, add the following line to the return call of the function:
```julia
status = termination_status(DCOPF)
```
This tells you the solver termination status for the problem: e.g. was an optimal solution found, was the solution infeasible, was it unbounded, etc.

Make the following change to the system:

- Increase the variable cost of Generator 1 to \$30 / MWh

Run the DCOPF and output generation, flows, and prices.

```julia
datadir = joinpath("..","Notebooks","ieee_test_cases")
gens = CSV.read(joinpath(datadir,"Gen14.csv"), DataFrame);
lines = CSV.read(joinpath(datadir,"Tran14.csv"), DataFrame);
loads = CSV.read(joinpath(datadir,"Load14.csv"), DataFrame);

# Rename all columns to lowercase (by convention)
for f in [gens, lines, loads]
    rename!(f,lowercase.(names(f)))
end

# create generator ids
gens.id = 1:nrow(gens);

# create line ids
lines.id = 1:nrow(lines);
# add set of rows for reverse direction with same parameters
lines2 = copy(lines)
lines2.f = lines2.fromnode
lines2.fromnode = lines.tonode
lines2.tonode = lines2.f
lines2 = lines2[:,names(lines)]
append!(lines,lines2)

# calculate simple susceptance, ignoring resistance as earlier
lines.b = 1 ./ lines.reactance

# keep only a single time period
loads = loads[:,["connnode","interval-1_load"]]
rename!(loads,"interval-1_load" => "demand");
```

```julia

```

```julia

```

```julia

```

```julia

```

Regarding the above results, answer the following:

- How has generation changed compared to the default system?
- What explains the new prices?

```julia

```

**B. Constrained line**

Make the following changes to the system:

- Increase the variable cost of Generator 1 to \$30 / MWh
- Reduce flow limit on the line connecting 2 and 3 ($l_{23}$) to 70 MW

Run the DCOPF and output generation, flows, and prices.

```julia

```

```julia

```

```julia

```

Regarding the above results, answer the following:

- Which node has the highest price and why?
- What is the difference in prices across $l_{23}$, also known as the congestion rent? How do you interpret this value (what is it's practical meaning?)

```julia

```

**C. Demand increase**

Make the following changes to the system:

- Increase the variable cost of Generator 1 to \$30 / MWh
- Reduce flow limit on the line connecting 2 and 3 ($l_{23}$) to 70 MW
- Increase demands everywhere by 5\%.

```julia

```

Calculate the total available generating capacity:

```julia

```

Calculate the new total demand:

```julia

```

Run the DCOPF and show prices.

```julia

```

**What is happening in this system?**

```julia

```

## Question 2: Linear losses

Up until now, we have ignored transmission losses. A quadratic approximation of losses is given by:

\begin{align}
LOSS_{ij} &\approx \frac{G_{ij}}{BaseMVA} (\theta_i-\theta_j)^2 \\
 & \approx \frac{1}{BaseMVA} \frac{R_{ij}}{R_{ij}^2+X_{ij}^2}(\theta_i-\theta_j)^2
\end{align}

where $G$ is the line's conductance, $R$ is the line's resistance and $X$ is the line's reactance. See the `lines` data frame for these parameters.

For our purposes, we will approximate this quadratic via:

$$
LOSS_{ij} \geq \frac{R_{ij}}{BaseMVA} \times (MaxFlow_{ij})^2
\left(\frac{|FLOW_{ij}|}{MaxFlow_{ij}} - 0.165 \right)
$$

where $MaxFlow_{ij}=200 MW$ in this problem. Note the greater than equal sign, as we do not want to have negative losses.

This approximation is based on Fitiwi et al. (2016), "Finding a representative network losses model for large-scale transmission expansion planning with renewable energy sources," *Energy* 101: 343-358, https://doi.org/10.1016/j.energy.2016.02.015.

Note that this is a linear approximation of transmission losses, which are actually a quadratic function of power flows. Fitiwi et al. 2016 and other papers describe piece-wise or segment-wise linear approximations of the quadratic function which provide a tighter lower bound approximation of losses, but we'll use a single linear term for this assignment.

See Jenkins & Sepulveda et al. 2017, "Enhanced decision support for a changing electricity landscape: the GenX configurable electricity resource capacity expansion model", MIT Energy Initiative Working Paper 2017-10 http://bit.ly/GenXModel Section 5.8, for an example of a linear segment-wise approximation of quadratic transmission losses.

**A. Code linear losses**

Reload the original data from Notebook 6 and copy the IEEE 14 bus system and DCOPF solver function from Notebook 6 into a new function `dcopf_ieee_lossy`.

Make the following changes:
- Increase the variable cost of Generator 1 to \$30 / MWh
- Change all transmission line capacities to 200 MW

Implement losses into the supply/demand balance equations. A standard way to implement absolute values in linear programming is by introducing two non-negative auxiliary variables $x^+$, $x^-$ $\geq 0$:

$$
x = x^+ - x^-
$$

and the absolute value can be represented as:

$$
|x| = x^+ + x^-
$$

(You should satisfy yourself that this equality holds.)

It makes the formulation easier if losses are added to the supply/demand balance constraint in each node by splitting losses in half between the receiving and sending end.

Indicate which equations and variables you have added and explain your steps using inline code comments (e.g. `# Comment`).

Run the lossy DCOPF and output generation, flows, losses, and prices.

```julia

```

```julia

```

```julia

```

```julia

```

```julia

```

```julia

```

**B. Interpret results**

Run the same parameters in the lossless OPF from problem 1. How do prices and flows change? What is the largest magnitude difference in prices between the solution with losses and the lossless OPF solution?

```julia

```

## Question 3 - Security contingencies

Power system operators need to ensure that power is delivered reliably even in the event of unexpected outages (**contingencies**). One common contigency that must be planned for is the loss of a transmission line. The security-constrained OPF (SCOPF) run by operators solves for an optimal dispatch that is simultaneously robust (i.e., feasible) to each of the lines failing individually. This is what is known as **N-1 security**, because we assume that at most one component fails in any given scenario.

In this problem, we will not code a full SCOPF, but rather investigate what happens to the feasibility of our problem when we remove transmission lines.

**A. Setup data**

The following code loads the original dataset (with one row per line) and includes a function `format_lines` that converts this to a format that our solver function can use (duplicating rows for both directions, adding susceptance, etc.).

```julia
lines = CSV.read(joinpath(datadir,"Tran14.csv"), DataFrame);
rename!(lines,lowercase.(names(lines)))

function format_lines(lines)
    # create line ids
    lines.id = 1:nrow(lines);
    # add set of rows for reverse direction with same parameters
    lines2 = copy(lines)
    lines2.f = lines2.fromnode
    lines2.fromnode = lines.tonode
    lines2.tonode = lines2.f
    lines2 = lines2[:,names(lines)]
    append!(lines,lines2)

    # calculate simple susceptance, ignoring resistance as earlier
    lines.b = 1 ./ lines.reactance
    return(lines)
end
```

Next:

1. Set the capacity of all lines in the system at 100 MW, except for the line $l_{12}$, which you should set to 200 MW.

2. Create a load dataframe `loads_sens` that increases demands everywhere by 10\%

```julia

```

**B. Loop over line contingencies**

Create a dataframe `status` with the `fromnode` and `tonode` columns of `lines`.

Create a [for loop](https://docs.julialang.org/en/v1/manual/control-flow/#man-loops) that iterates over each line and:
- sets the reactance to be a very high value, 1e9 (i.e., no power will be transmitted)
- creates a version of the dataframe that our solver function can use via `format_lines`
- runs DCOPF
- stores the solution status in a `opf` column in the corresponding row of the `status` dataframe

Show the `status` results.

```julia

```

**3. Interpret results**

Are all of the cases feasible? If not, how many are infeasible?

```julia

```

Pick two cases where the solution gives a different status. (For our purposes, dual infeasible and primal infeasible are the same.) What is happening here?

Given this, do you conclude that the system with the assumed transmission line ratings is secure as-is, or do we need to add more redundancy to the system?

```julia

```


# About

Code snippet for finding closest image dimensions for which cuFFT uses faster Cooley-Tukey implementation (as opposed to Bluestein).
See https://docs.nvidia.com/cuda/cufft/index.html

Quote from there:
_Algorithms highly optimized for input sizes that can be written in the form 2 a × 3 b × 5 c × 7 d . In general the smaller the prime factor, the better the performance, i.e., powers of two are fastest._

Volker dot Hilsenstein at monash dot edu, April 2019

# Requirements

This notebook requires the following additional packages. Use `pip` or `conda` to install them.

* `sympy`
* `numpy`
* `tqdm`

#  Implementation

```python
from sympy.ntheory import factorint
import warnings
import numpy as np

def is_optimal_for_cuFFT(n: int, allowed_factors) -> bool:
    factorization = factorint(n)
    if len(factorization) == 0: # factorint(1) returns empyt dict
        return False
    factors = set(factorization.keys())
    return factors.issubset(set(allowed_factors))

def _closest_optimal(n: int, search_next_largest: bool, allowed_factors) -> int:
    while(not is_optimal_for_cuFFT(n, allowed_factors) and n>=1):
        if search_next_largest:
            n += 1
        else:
            n -= 1
    # edge case: decreasing search with start value smaller than allowed factor
    if n < min(allowed_factors):

        warnings.warn(f"{n}One provided dimension is smaller than smallest allowed factor and search direction is decreasing")
        return(min(allowed_factors))
    return n

def closest_optimal(n, search_next_largest: bool=True, allowed_factors=(2,3,5,7)):
    """ Finds closest optimal array dimensions for cuFFT

    Parameters
    ----------
    n : iterable of integers
        Input dimensions
    search_next_largest : bool
        if True (default) search closest optimal dimensions that are larger or equal to original
        otherwise look for smaller ones.
    allowed_factor: tuple of integers
        allowed factors in decomposition. Defaults to (2,3,5,7) which are the factors listed in
        the cuFFT documentation.

    Returns
    -------
    np.array of ints
        optimal dimensions for cuFFT

    See also
    --------
    https://docs.nvidia.com/cuda/cufft/index.html

    """
    n = np.asarray(n)
    scalar_input = False
    if n.ndim == 0:
        n = n[None]
        scalar_input = True
    ret = np.array([_closest_optimal(ni, search_next_largest, allowed_factors) for ni in n])
    if scalar_input:
        return ret[0]
    return ret
```

# Examples

```python
# Simple case, single number
closest_optimal(123)
```

    125

```python
# find a smaller optimal dimension
closest_optimal(123, search_next_largest=False)
```

    120

```python
# don't allow all factors
closest_optimal(123, search_next_largest=False, allowed_factors=(2,3))
```

    108

```python
# only allow a single factor
# use a comma to make it a tuple, otherwise it will throw an error!
closest_optimal(123, search_next_largest=False, allowed_factors=(2,))
```

    64

```python
# apply to multiple dimensions
closest_optimal((123, 23, 615))
```

    array([125,  24, 625])

```python
# edge case, one dimension smaller than smallest factor and decreasing search should generate a warning
closest_optimal((1, 23, 615), search_next_largest=False)
```

    c:\users\volker\anaconda3\envs\spimenv\lib\site-packages\ipykernel_launcher.py:21: UserWarning: 0One provided dimension is smaller than smallest allowed factor and search direction is decreasing

    array([  2,  21, 600])

```python
# one dimension smaller than smallest factor and increasing search should not generate a warning
closest_optimal((1, 23, 615))
```

    array([  2,  24, 625])

# Todo

* could allow `search_next_largest` to be an iterable of bools, to apply different strategies (rounding up/rounding down) according to dimension.
* could remove `sympy`-dependency by implementing recursive modulo tests as in `notGoodDimension` from https://github.com/dmilkie/cudaDecon/blob/master/RL-Biggs-Andrews.cpp. However, I find the explicit factorization more readable than the recursion.

# Rainbow table

Create a rainbow table with pre-computed good dimensions. Note that with this idiotic approach of building a rainbow table is very slow (building by multiplication is quicker than building by factorization). But we only have to do it once.

## idiotic approach (building by factorization)

Be lazy, recycle the code defined above and start testing numbers.
The tqdm progress bar indicates the problem.
Big-O for factorization is bad, hence used for encryption.

```python
import tqdm
def create_cuFFT_dim_rainbow_table(nr_of_els: int = 1000):
    good_dimensions = [0] * nr_of_els
    val = 1
    for i in tqdm.tqdm(range(nr_of_els)):
        good_dimensions[i] = closest_optimal(val)
        val = good_dimensions[i]+1
    return good_dimensions
```

```python
good_dimensions = create_cuFFT_dim_rainbow_table()
```

    100%|███████████████████████████████████████████████| 1000/1000 [00:06<00:00, 155.06it/s]

```python
print(repr(good_dimensions))
```

    [2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 14, 15, 16, 18, 20, 21, 24, 25, 27, 28, 30, 32, 35, 36, 40, 42, 45, 48, 49, 50, 54, 56, 60, 63, 64, 70, 72, 75, 80, 81, 84, 90, 96, 98, 100, 105, 108, 112, 120, 125, 126, 128, 135, 140, 144, 147, 150, 160, 162, 168, 175, 180, 189, 192, 196, 200, 210, 216, 224, 225, 240, 243, 245, 250, 252, 256, 270, 280, 288, 294, 300, 315, 320, 324, 336, 343, 350, 360, 375, 378, 384, 392, 400, 405, 420, 432, 441, 448, 450, 480, 486, 490, 500, 504, 512, 525, 540, 560, 567, 576, 588, 600, 625, 630, 640, 648, 672, 675, 686, 700, 720, 729, 735, 750, 756, 768, 784, 800, 810, 840, 864, 875, 882, 896, 900, 945, 960, 972, 980, 1000, 1008, 1024, 1029, 1050, 1080, 1120, 1125, 1134, 1152, 1176, 1200, 1215, 1225, 1250, 1260, 1280, 1296, 1323, 1344, 1350, 1372, 1400, 1440, 1458, 1470, 1500, 1512, 1536, 1568, 1575, 1600, 1620, 1680, 1701, 1715, 1728, 1750, 1764, 1792, 1800, 1875, 1890, 1920, 1944, 1960, 2000, 2016, 2025, 2048, 2058, 2100, 2160, 2187, 2205, 2240, 2250, 2268, 2304, 2352, 2400, 2401, 2430, 2450, 2500, 2520, 2560, 2592, 2625, 2646, 2688, 2700, 2744, 2800, 2835, 2880, 2916, 2940, 3000, 3024, 3072, 3087, 3125, 3136, 3150, 3200, 3240, 3360, 3375, 3402, 3430, 3456, 3500, 3528, 3584, 3600, 3645, 3675, 3750, 3780, 3840, 3888, 3920, 3969, 4000, 4032, 4050, 4096, 4116, 4200, 4320, 4374, 4375, 4410, 4480, 4500, 4536, 4608, 4704, 4725, 4800, 4802, 4860, 4900, 5000, 5040, 5103, 5120, 5145, 5184, 5250, 5292, 5376, 5400, 5488, 5600, 5625, 5670, 5760, 5832, 5880, 6000, 6048, 6075, 6125, 6144, 6174, 6250, 6272, 6300, 6400, 6480, 6561, 6615, 6720, 6750, 6804, 6860, 6912, 7000, 7056, 7168, 7200, 7203, 7290, 7350, 7500, 7560, 7680, 7776, 7840, 7875, 7938, 8000, 8064, 8100, 8192, 8232, 8400, 8505, 8575, 8640, 8748, 8750, 8820, 8960, 9000, 9072, 9216, 9261, 9375, 9408, 9450, 9600, 9604, 9720, 9800, 10000, 10080, 10125, 10206, 10240, 10290, 10368, 10500, 10584, 10752, 10800, 10935, 10976, 11025, 11200, 11250, 11340, 11520, 11664, 11760, 11907, 12000, 12005, 12096, 12150, 12250, 12288, 12348, 12500, 12544, 12600, 12800, 12960, 13122, 13125, 13230, 13440, 13500, 13608, 13720, 13824, 14000, 14112, 14175, 14336, 14400, 14406, 14580, 14700, 15000, 15120, 15309, 15360, 15435, 15552, 15625, 15680, 15750, 15876, 16000, 16128, 16200, 16384, 16464, 16800, 16807, 16875, 17010, 17150, 17280, 17496, 17500, 17640, 17920, 18000, 18144, 18225, 18375, 18432, 18522, 18750, 18816, 18900, 19200, 19208, 19440, 19600, 19683, 19845, 20000, 20160, 20250, 20412, 20480, 20580, 20736, 21000, 21168, 21504, 21600, 21609, 21870, 21875, 21952, 22050, 22400, 22500, 22680, 23040, 23328, 23520, 23625, 23814, 24000, 24010, 24192, 24300, 24500, 24576, 24696, 25000, 25088, 25200, 25515, 25600, 25725, 25920, 26244, 26250, 26460, 26880, 27000, 27216, 27440, 27648, 27783, 28000, 28125, 28224, 28350, 28672, 28800, 28812, 29160, 29400, 30000, 30240, 30375, 30618, 30625, 30720, 30870, 31104, 31250, 31360, 31500, 31752, 32000, 32256, 32400, 32768, 32805, 32928, 33075, 33600, 33614, 33750, 34020, 34300, 34560, 34992, 35000, 35280, 35721, 35840, 36000, 36015, 36288, 36450, 36750, 36864, 37044, 37500, 37632, 37800, 38400, 38416, 38880, 39200, 39366, 39375, 39690, 40000, 40320, 40500, 40824, 40960, 41160, 41472, 42000, 42336, 42525, 42875, 43008, 43200, 43218, 43740, 43750, 43904, 44100, 44800, 45000, 45360, 45927, 46080, 46305, 46656, 46875, 47040, 47250, 47628, 48000, 48020, 48384, 48600, 49000, 49152, 49392, 50000, 50176, 50400, 50421, 50625, 51030, 51200, 51450, 51840, 52488, 52500, 52920, 53760, 54000, 54432, 54675, 54880, 55125, 55296, 55566, 56000, 56250, 56448, 56700, 57344, 57600, 57624, 58320, 58800, 59049, 59535, 60000, 60025, 60480, 60750, 61236, 61250, 61440, 61740, 62208, 62500, 62720, 63000, 63504, 64000, 64512, 64800, 64827, 65536, 65610, 65625, 65856, 66150, 67200, 67228, 67500, 68040, 68600, 69120, 69984, 70000, 70560, 70875, 71442, 71680, 72000, 72030, 72576, 72900, 73500, 73728, 74088, 75000, 75264, 75600, 76545, 76800, 76832, 77175, 77760, 78125, 78400, 78732, 78750, 79380, 80000, 80640, 81000, 81648, 81920, 82320, 82944, 83349, 84000, 84035, 84375, 84672, 85050, 85750, 86016, 86400, 86436, 87480, 87500, 87808, 88200, 89600, 90000, 90720, 91125, 91854, 91875, 92160, 92610, 93312, 93750, 94080, 94500, 95256, 96000, 96040, 96768, 97200, 98000, 98304, 98415, 98784, 99225, 100000, 100352, 100800, 100842, 101250, 102060, 102400, 102900, 103680, 104976, 105000, 105840, 107163, 107520, 108000, 108045, 108864, 109350, 109375, 109760, 110250, 110592, 111132, 112000, 112500, 112896, 113400, 114688, 115200, 115248, 116640, 117600, 117649, 118098, 118125, 119070, 120000, 120050, 120960, 121500, 122472, 122500, 122880, 123480, 124416, 125000, 125440, 126000, 127008, 127575, 128000, 128625, 129024, 129600, 129654, 131072, 131220, 131250, 131712, 132300, 134400, 134456, 135000, 136080, 137200, 137781, 138240, 138915, 139968, 140000, 140625, 141120, 141750, 142884, 143360, 144000, 144060, 145152, 145800, 147000, 147456, 148176, 150000, 150528, 151200, 151263, 151875, 153090, 153125, 153600, 153664, 154350, 155520, 156250, 156800, 157464, 157500, 158760, 160000, 161280, 162000, 163296, 163840, 164025, 164640, 165375, 165888, 166698, 168000, 168070, 168750, 169344, 170100, 171500, 172032, 172800, 172872, 174960, 175000, 175616, 176400, 177147, 178605, 179200, 180000, 180075, 181440, 182250, 183708, 183750, 184320, 185220, 186624, 187500, 188160, 189000, 190512, 192000, 192080, 193536, 194400, 194481, 196000, 196608, 196830, 196875, 197568, 198450, 200000, 200704, 201600, 201684, 202500, 204120, 204800, 205800, 207360, 209952, 210000, 211680, 212625, 214326, 214375, 215040, 216000, 216090, 217728, 218700, 218750, 219520, 220500, 221184, 222264, 224000, 225000, 225792, 226800, 229376, 229635, 230400, 230496, 231525, 233280, 234375, 235200, 235298, 236196, 236250, 238140, 240000, 240100, 241920, 243000, 244944, 245000, 245760, 246960, 248832, 250000, 250047, 250880, 252000, 252105, 253125, 254016, 255150, 256000, 257250, 258048, 259200, 259308, 262144, 262440, 262500, 263424, 264600, 268800, 268912, 270000, 272160, 273375, 274400, 275562, 275625, 276480, 277830, 279936, 280000, 281250, 282240, 283500, 285768, 286720, 288000, 288120, 290304, 291600, 294000, 294912, 295245, 296352, 297675, 300000, 300125, 301056, 302400, 302526, 303750, 306180, 306250, 307200, 307328, 308700, 311040, 312500, 313600, 314928, 315000, 317520, 320000, 321489, 322560, 324000, 324135, 326592, 327680, 328050, 328125, 329280, 330750, 331776, 333396, 336000, 336140, 337500, 338688, 340200, 343000, 344064, 345600, 345744, 349920, 350000, 351232, 352800, 352947, 354294, 354375, 357210, 358400, 360000, 360150, 362880, 364500, 367416, 367500, 368640, 370440, 373248, 375000, 376320, 378000, 381024, 382725, 384000, 384160, 385875, 387072]

```python
good_dimensions_fac = good_dimensions # save for later
```

## sensible approach (building by multiplication)

```python
# increase these maximum exponents as desired
n7 = 36
n5 = 43
n3 = 63
n2 = 100
```

```python
max_to_consider = min(2**n2, 3**n3, 5**n5, 7**n7)
i = 0
good_dimensions = []
for pow7 in tqdm.tqdm(range(n7+1)):
    for pow5 in range(n5+1):
        for pow3 in range(n3+1):
            for pow2 in range(n2+1):
                prod = 2**pow2 * 3**pow3 * 5**pow5 * 7**pow7
                if prod < max_to_consider:
                    good_dimensions.append(prod)
print("sorting")
good_dimensions = sorted(good_dimensions)
# drop the 1
good_dimensions = good_dimensions[1:]
```

    100%|████████████████████████████████████████████████████| 37/37 [00:20<00:00,  1.73it/s]

    sorting

```python
# Check both the factorization and the multiplication approach give the same first 1000 elements
assert(good_dimensions_fac[:] == good_dimensions[:1000])
```

# Rainbow table lookup

```python
from bisect import bisect_left
lookup_larger = lambda x : good_dimensions[bisect_left(good_dimensions, x)]
lookup_smaller = lambda x : good_dimensions[bisect_left(good_dimensions, x)-1]
```

```python
testnr = 11223344

print("Rainbow lookup:")
print("next largest :", lookup_larger(testnr))
print("nex smallest :", lookup_smaller(testnr))

print("Factorization check:")
print("next largest :", closest_optimal(testnr))
print("nex smallest :", closest_optimal(testnr, search_next_largest=False ))
```

    Rainbow lookup:
    next largest : 11239424
    nex smallest : 11200000
    Factorization check:
    next largest : 11239424
    nex smallest : 11200000

# Timing

### factorization

```python
%%timeit
closest_optimal(testnr)
```

    939 ms ± 17.7 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

### rainbow table

```python
%%timeit
lookup_larger(testnr)
```

    617 ns ± 15.3 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

# Serialize rainbow table into usable python code

Generate a `.py` file with the first `n_elements` of the rainbow table.

Note that the lambda functions don't deal with any edge cases and work on scalars only.
They will throw an exception if the input value exceeds the largest value in the rainbow table.

```python
n_elements = 8000

py_code = """
from bisect import bisect_left

good_dimensions = {rainbowtable}

lookup_larger = lambda x : good_dimensions[bisect_left(good_dimensions, x)]
lookup_smaller = lambda x : good_dimensions[bisect_left(good_dimensions, x)-1]
""".format(rainbowtable=repr(good_dimensions[:n_elements]))

with open("rainbow_cufft_pad.py", "w") as pyfile:
    pyfile.write(py_code)
```


# Variational Principle  using Symbolic Mathematics in Python

## 1. Introduction

The variational principle tells us that we can use a trial wavefunction to solve the Schrodinger equation using the following theorem:

$${{\int {{\Psi ^*}\hat H{\rm{ }}\Psi } d\tau } \over {\int {{\Psi ^*}\Psi } d\tau }} \ge {E_0}$$

We will use Sympy to solve the particle in a box problem by guessing a trial wavefunction using variational principle

```python
import sympy as sym
```

This exercise is a bit more self-guided than the other notebooks we have done. One of the most useful things you can do is **open last week's notebook to remember the commands in sympy**. Also, remember that google is your friend:

1. [Sympy tutorial](https://docs.sympy.org/latest/tutorial/index.html)
2. [Stack Overflow](https://stackoverflow.com/search?q=sympy+)
3. [Stack Exchange](https://stackexchange.com/)

## 2. Particle in a box

The wave function that we pick for a particle in a box needs to have the following properties

1. single valued
1. normalizable
1. function and its first derivative are continuous
1. boundary condition that the wave function goes to zero at the ends of the box

Particle in a box: a is a classical particle, red is real part, blue is imaginary part.

This particle only expericnes kinetic energy between the box, so the Hamiltonian for this system is

$$\hat H = {{ - {\hbar ^2}} \over {2m}}{{{d^2}} \over {d{x^2}}} + \left\{ {\matrix{{V(x) = 0} & {0 < x < a}  \cr {V(x) = \infty } & {x < 0\text{ }{\rm{ or}}\;x > a}  \cr } } \right.$$

For our purposes, that means we can consider the Hamiltonian to be

$$\hat H = {{ - {\hbar ^2}} \over {2m}}{{{d^2}} \over {d{x^2}}}$$

as long as we keep the limits of integration to be $(0,a)$

### 2.1 Trial Wave function

Although the particle in box has a well known solution

[https://en.wikipedia.org/wiki/Particle_in_a_box](https://en.wikipedia.org/wiki/Particle_in_a_box)

(or check your favorite pchem book)

We are going to guess a trial wave function:

$$\Phi (x) = \left( {{x \over a} - {{{x^3}} \over a}} \right) + \alpha \left( {{{{x^5}} \over {{a^5}}} - {1 \over 2}\left( {{{{x^7}} \over {{a^7}}} + {{{x^7}} \over {{a^7}}}} \right)} \right)$$

### 2.2 Exercise: Variational Theorem

We are going to follow the following plan:

1. Solve for the energy of the trial wave function above

$${E_{trial}} = {{\int\limits_0^a {\Phi (x){{ - {\hbar ^2}} \over {2m}}{{{d^2}} \over {d{x^2}}}\Phi (x)dx} } \over {\int\limits_0^a {\Phi {{(x)}^2}dx} }}$$

Your answer will be a function of $ m,a,\text{and } \alpha$ We will use $\alpha$ as the parameter we vary to minimize the energy and make a new trial wave function.

2. Minimize the trial energy
We will use a first derivative of the trial energy $${d \over {d\alpha }}{E_{trial}}(\alpha )$$ to find the value of $\alpha$ that gives you the lowest energy

3. Plot your new wavefunction compared to the ground state particle in a box: $${\psi _{true}}(x) = {\left( {{2 \over a}} \right)^{1/2}}\sin {{n\pi x} \over a}$$ Plot as a function of $x/a$ from $0$ to $1$. Assuming this has $m=m_e$, and $a=a_0$ use atomic (theorist) units to plot the function.

4. Compare your trial energy to the actual energy (using atomic units)
$${E_{true}}(n = 1) = {{{\hbar ^2}{\pi ^2}} \over {2m{a^2}}}$$

```python
sym.init_printing()
x,a,m,alpha,hbar,n=sym.symbols('x,a,m,alpha,hbar,n')

twf=((x/a)-(x**3)/a)+alpha*((x**5)/(a**5)-0.5*((x**7)/(a**7)+(x**7)/(a**7)))
twf
```

```python
single=(sym.diff((twf),x))
double=(sym.diff((single),x))
double
```

```python
upper_i=twf*(-1*hbar)/(2*m)*double
upper_i_ans=(sym.integrate((upper_i),(x,0,a)))
upper_i_ans
```

```python
lower_i=twf**2
lower_i_ans=(sym.integrate((lower_i),(x,0,a)))
lower_i_ans
```

```python
Etrial=upper_i_ans/lower_i_ans
Etrial
```

```python
first_d=sym.diff((Etrial),alpha)
first_d
```

```python
twf_newalpha=((x/a)-(x**3)/a)+0*((x**5)/(a**5)-0.5*((x**7)/(a**7)+(x**7)/(a**7)))
single_newalpha=(sym.diff((twf_newalpha),x))
double_newalpha=(sym.diff((single_newalpha),x))
upper_i_newalpha=twf_newalpha*(-1*hbar)/(2*m)*double_newalpha
upper_i_ans_newalpha=(sym.integrate((upper_i_newalpha),(x,0,a)))
lower_i_newalpha=twf_newalpha**2
lower_i_ans_newalpha=(sym.integrate((lower_i_newalpha),(x,0,a)))
Etrial_newalpha=upper_i_ans_newalpha/lower_i_ans_newalpha
Etrial_newalpha
```

```python
import numpy as np
import matplotlib.pyplot as plt
pi = np.pi
z = np.linspace(0.00000000001,1,100)
y1 = np.sin(pi*z)
plt.plot(z,y1)
plt.xlabel('a/x', fontsize=24)
plt.show()
print('Please ignore y-axis as the function is multiplied by a constant.')
```

    <Figure size 640x480 with 1 Axes>

    Please ignore y-axis as the function is multiplied by a constant.

```python
print('Compare my wave function to the true one: DIFFERENT!')
```

    Compare my wave function to the true one: DIFFERENT!

### 2.3 Exercise: New trial wavefunction

Determine the minimum energy of the particle in a box using a new trial wavefunction $$x^\alpha(x-a)^\alpha$$

1. Find the minimum energy, $E_{trial}$
2. Plot the new trial wavefunction and compare it to the true solution and the wavefunction you found above
3. Compare you new energy to the trial energy you found above
4. Which wavefunction is better? How do you know?

```python
sym.init_printing()
x,a,m,alpha,hbar=sym.symbols('x,a,m,alpha,h_bar')

twf_2=x**alpha*(x-alpha)**alpha
twf_2
```

```python
single_2=(sym.diff((twf_2),x))
double_2=(sym.diff((single_2),x))
double_2
```

```python
upper_i_2=twf_2*(-1*hbar)/(2*m)*double_2
upper_i_ans_2=(sym.integrate((upper_i_2),(x,0,a)))
upper_i_ans_2
```

```python
lower_i_2=twf_2**2
lower_i_ans_2=(sym.integrate((lower_i_2),(x,0,a)))
lower_i_ans_2
```

```python
Etrial_2=upper_i_ans_2/lower_i_ans_2
Etrial_2
```

```python
print('The first E_trial is better because it looks closer to the true E')
```

### 2.4 Exercise: Design your own wavefunction!

**Now you get to make your own wavefunction!**

The only guidance I would give you is that it make sense mathematically and that it include $\alpha$ so that you can minimize the energy.

Remember that $a$ and $x$ are both length units, and that trigonometric, logarithmic, and exponential functions are all unitless

Using your new wavefunction:

1. Find the minimum energy, $E_{trial}$
2. Plot the new trial wavefunction and compare it to the true solution and the wavefunction you found above
3. Compare you new energy to the trial energy you found above
4. Which wavefunction is better? How do you know?

```python
sym.init_printing()
x,a,m,alpha,hbar=sym.symbols('x,a,m,alpha,h_bar')

twf_3=a**x*alpha
twf_3
```

```python
single_3=(sym.diff((twf_3),x))
double_3=(sym.diff((single_3),x))
double_3
```

```python
upper_i_3=twf_3*(-1*hbar)/(2*m)*double_3
upper_i_ans_3=(sym.integrate((upper_i_3),(x,0,a)))
upper_i_ans_3
```

```python
lower_i_3=twf_3**2
lower_i_ans_3=(sym.integrate((lower_i_3),(x,0,a)))
lower_i_ans_3
```

```python
Etrial_3=upper_i_ans_3/lower_i_ans_3
Etrial_3
```

```python
print('The first E_trial is still better because this try is far from the true E as well.')
```

# Reading Homework

Read the following sections in Kramer

- 4.2.3 Born-Oppenheimer approximation
- 4.3.2 Secular equation
- All of 4.5

For each subsection
- write down the subchapter name
- what was the most important idea
- draw an idea digram of the main idea

**Make sure to upload this to the assignment repository**

Example idea diagram:

```python

```


Text provided under a Creative Commons Attribution license, CC-BY.  All code is made available under the FSF-approved MIT license.  (c) Lorena A. Barba, Gilbert F. Forsyth 2015. Thanks to NSF for support via CAREER award #1149784.
[@LorenaABarba](https://twitter.com/LorenaABarba)

12 steps to Navier-Stokes
=====
***

We continue our journey to solve the Navier-Stokes equation with Step 4. But don't continue unless you have completed the previous steps! In fact, this next step will be a combination of the two previous ones. The wonders of *code reuse*!

Step 4: Burgers' Equation
----
***

You can read about Burgers' Equation on its [wikipedia page](http://en.wikipedia.org/wiki/Burgers'_equation).

Burgers' equation in one spatial dimension looks like this:

$$\frac{\partial u}{\partial t} + u \frac{\partial u}{\partial x} = \nu \frac{\partial ^2u}{\partial x^2}$$

As you can see, it is a combination of non-linear convection and diffusion. It is surprising how much you learn from this neat little equation!

We can discretize it using the methods we've already detailed in Steps [1](./01_Step_1.ipynb) to [3](./04_Step_3.ipynb).  Using forward difference for time, backward difference for space and our 2nd-order method for the second derivatives yields:

$$\frac{u_i^{n+1}-u_i^n}{\Delta t} + u_i^n \frac{u_i^n - u_{i-1}^n}{\Delta x} = \nu \frac{u_{i+1}^n - 2u_i^n + u_{i-1}^n}{\Delta x^2}$$

As before, once we have an initial condition, the only unknown is $u_i^{n+1}$. We will step in time as follows:

$$u_i^{n+1} = u_i^n - u_i^n \frac{\Delta t}{\Delta x} (u_i^n - u_{i-1}^n) + \nu \frac{\Delta t}{\Delta x^2}(u_{i+1}^n - 2u_i^n + u_{i-1}^n)$$

### Initial and Boundary Conditions

To examine some interesting properties of Burgers' equation, it is helpful to use different initial and boundary conditions than we've been using for previous steps.

Our initial condition for this problem is going to be:

\begin{eqnarray}
u &=& -\frac{2 \nu}{\phi} \frac{\partial \phi}{\partial x} + 4 \\\
\phi &=& \exp \bigg(\frac{-x^2}{4 \nu} \bigg) + \exp \bigg(\frac{-(x-2 \pi)^2}{4 \nu} \bigg)
\end{eqnarray}

This has an analytical solution, given by:

\begin{eqnarray}
u &=& -\frac{2 \nu}{\phi} \frac{\partial \phi}{\partial x} + 4 \\\
\phi &=& \exp \bigg(\frac{-(x-4t)^2}{4 \nu (t+1)} \bigg) + \exp \bigg(\frac{-(x-4t -2 \pi)^2}{4 \nu(t+1)} \bigg)
\end{eqnarray}

Our boundary condition will be:

$$u(0) = u(2\pi)$$

This is called a *periodic* boundary condition. Pay attention! This will cause you a bit of headache if you don't tread carefully.

### Saving Time with SymPy

The initial condition we're using for Burgers' Equation can be a bit of a pain to evaluate by hand.  The derivative $\frac{\partial \phi}{\partial x}$ isn't too terribly difficult, but it would be easy to drop a sign or forget a factor of $x$ somewhere, so we're going to use SymPy to help us out.

[SymPy](http://sympy.org/en/) is the symbolic math library for Python.  It has a lot of the same symbolic math functionality as Mathematica with the added benefit that we can easily translate its results back into our Python calculations (it is also free and open source).

Start by loading the SymPy library, together with our favorite library, NumPy.

```python
import numpy
import sympy
```

We're also going to tell SymPy that we want all of its output to be rendered using $\LaTeX$. This will make our Notebook beautiful!

```python
from sympy import init_printing
init_printing(use_latex=True)
```

Start by setting up symbolic variables for the three variables in our initial condition and then type out the full equation for $\phi$.  We should get a nicely rendered version of our $\phi$ equation.

```python
x, nu, t = sympy.symbols('x nu t')
phi = (sympy.exp(-(x - 4 * t)**2 / (4 * nu * (t + 1))) +
       sympy.exp(-(x - 4 * t - 2 * numpy.pi)**2 / (4 * nu * (t + 1))))
phi
```

It's maybe a little small, but that looks right.  Now to evaluate our partial derivative $\frac{\partial \phi}{\partial x}$ is a trivial task.

```python
phiprime = phi.diff(x)
phiprime
```

If you want to see the unrendered version, just use the Python print command.

```python
print(phiprime)
```

    -(-8*t + 2*x)*exp(-(-4*t + x)**2/(4*nu*(t + 1)))/(4*nu*(t + 1)) - (-8*t + 2*x - 12.5663706143592)*exp(-(-4*t + x - 6.28318530717959)**2/(4*nu*(t + 1)))/(4*nu*(t + 1))

### Now what?

Now that we have the Pythonic version of our derivative, we can finish writing out the full initial condition equation and then translate it into a usable Python expression.  For this, we'll use the *lambdify* function, which takes a SymPy symbolic equation and turns it into a callable function.

```python
from sympy.utilities.lambdify import lambdify

u = -2 * nu * (phiprime / phi) + 4
print(u)
```

    -2*nu*(-(-8*t + 2*x)*exp(-(-4*t + x)**2/(4*nu*(t + 1)))/(4*nu*(t + 1)) - (-8*t + 2*x - 12.5663706143592)*exp(-(-4*t + x - 6.28318530717959)**2/(4*nu*(t + 1)))/(4*nu*(t + 1)))/(exp(-(-4*t + x - 6.28318530717959)**2/(4*nu*(t + 1))) + exp(-(-4*t + x)**2/(4*nu*(t + 1)))) + 4

### Lambdify

To lambdify this expression into a useable function, we tell lambdify which variables to request and the function we want to plug them in to.

```python
ufunc = lambdify((t, x, nu), u)
print(ufunc(1, 4, 3))
```

    3.4917066420644494

### Back to Burgers' Equation

Now that we have the initial conditions set up, we can proceed and finish setting up the problem.  We can generate the plot of the initial condition using our lambdify-ed function.

```python
from matplotlib import pyplot
%matplotlib inline

###variable declarations
nx = 101
nt = 100
dx = 2 * numpy.pi / (nx - 1)
nu = .07
dt = dx * nu

x = numpy.linspace(0, 2 * numpy.pi, nx)
un = numpy.empty(nx)
t = 0

u = numpy.asarray([ufunc(t, x0, nu) for x0 in x])
u
```

    array([ 4.        ,  4.06283185,  4.12566371,  4.18849556,  4.25132741,
            4.31415927,  4.37699112,  4.43982297,  4.50265482,  4.56548668,
            4.62831853,  4.69115038,  4.75398224,  4.81681409,  4.87964594,
            4.9424778 ,  5.00530965,  5.0681415 ,  5.13097336,  5.19380521,
            5.25663706,  5.31946891,  5.38230077,  5.44513262,  5.50796447,
            5.57079633,  5.63362818,  5.69646003,  5.75929189,  5.82212374,
            5.88495559,  5.94778745,  6.0106193 ,  6.07345115,  6.136283  ,
            6.19911486,  6.26194671,  6.32477856,  6.38761042,  6.45044227,
            6.51327412,  6.57610598,  6.63893783,  6.70176967,  6.76460125,
            6.82742866,  6.89018589,  6.95176632,  6.99367964,  6.72527549,
            4.        ,  1.27472451,  1.00632036,  1.04823368,  1.10981411,
            1.17257134,  1.23539875,  1.29823033,  1.36106217,  1.42389402,
            1.48672588,  1.54955773,  1.61238958,  1.67522144,  1.73805329,
            1.80088514,  1.863717  ,  1.92654885,  1.9893807 ,  2.05221255,
            2.11504441,  2.17787626,  2.24070811,  2.30353997,  2.36637182,
            2.42920367,  2.49203553,  2.55486738,  2.61769923,  2.68053109,
            2.74336294,  2.80619479,  2.86902664,  2.9318585 ,  2.99469035,
            3.0575222 ,  3.12035406,  3.18318591,  3.24601776,  3.30884962,
            3.37168147,  3.43451332,  3.49734518,  3.56017703,  3.62300888,
            3.68584073,  3.74867259,  3.81150444,  3.87433629,  3.93716815,  4.        ])

```python
pyplot.figure(figsize=(11, 7), dpi=100)
pyplot.plot(x, u, marker='o', lw=2)
pyplot.xlim([0, 2 * numpy.pi])
pyplot.ylim([0, 10]);
```

This is definitely not the hat function we've been dealing with until now. We call it a "saw-tooth function".  Let's proceed forward and see what happens.

### Periodic Boundary Conditions

One of the big differences between Step 4 and the previous lessons is the use of *periodic* boundary conditions.  If you experiment with Steps 1 and 2 and make the simulation run longer (by increasing `nt`) you will notice that the wave will keep moving to the right until it no longer even shows up in the plot.

With periodic boundary conditions, when a point gets to the right-hand side of the frame, it *wraps around* back to the front of the frame.

Recall the discretization that we worked out at the beginning of this notebook:

$$u_i^{n+1} = u_i^n - u_i^n \frac{\Delta t}{\Delta x} (u_i^n - u_{i-1}^n) + \nu \frac{\Delta t}{\Delta x^2}(u_{i+1}^n - 2u_i^n + u_{i-1}^n)$$

What does $u_{i+1}^n$ *mean* when $i$ is already at the end of the frame?

Think about this for a minute before proceeding.

```python
for n in range(nt):
    un = u.copy()
    for i in range(1, nx-1):
        u[i] = un[i] - un[i] * dt / dx *(un[i] - un[i-1]) + nu * dt / dx**2 *\
                (un[i+1] - 2 * un[i] + un[i-1])
    u[0] = un[0] - un[0] * dt / dx * (un[0] - un[-2]) + nu * dt / dx**2 *\
                (un[1] - 2 * un[0] + un[-2])
    u[-1] = u[0]

u_analytical = numpy.asarray([ufunc(nt * dt, xi, nu) for xi in x])
```

```python
pyplot.figure(figsize=(11, 7), dpi=100)
pyplot.plot(x,u, marker='o', lw=2, label='Computational')
pyplot.plot(x, u_analytical, label='Analytical')
pyplot.xlim([0, 2 * numpy.pi])
pyplot.ylim([0, 10])
pyplot.legend();
```

***

What next?
----

The subsequent steps, from 5 to 12, will be in two dimensions. But it is easy to extend the 1D finite-difference formulas to the partial derivatives in 2D or 3D. Just apply the definition — a partial derivative with respect to $x$ is the variation in the $x$ direction *while keeping $y$ constant*.

Before moving on to [Step 5](./07_Step_5.ipynb), make sure you have completed your own code for steps 1 through 4 and you have experimented with the parameters and thought about what is happening. Also, we recommend that you take a slight break to learn about [array operations with NumPy](./06_Array_Operations_with_NumPy.ipynb).

```python
from IPython.core.display import HTML
def css_styling():
    styles = open("../styles/custom.css", "r").read()
    return HTML(styles)
css_styling()
```

<link href='http://fonts.googleapis.com/css?family=Fenix' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Alegreya+Sans:100,300,400,500,700,800,900,100italic,300italic,400italic,500italic,700italic,800italic,900italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Source+Code+Pro:300,400' rel='stylesheet' type='text/css'>
<style>
    @font-face {
        font-family: "Computer Modern";
        src: url('http://mirrors.ctan.org/fonts/cm-unicode/fonts/otf/cmunss.otf');
    }
    div.cell{
        width:800px;
        margin-left:16% !important;
        margin-right:auto;
    }
    h1 {
        font-family: 'Alegreya Sans', sans-serif;
    }
    h2 {
        font-family: 'Fenix', serif;
    }
    h3{
		font-family: 'Fenix', serif;
        margin-top:12px;
        margin-bottom: 3px;
       }
	h4{
		font-family: 'Fenix', serif;
       }
    h5 {
        font-family: 'Alegreya Sans', sans-serif;
    }
    div.text_cell_render{
        font-family: 'Alegreya Sans',Computer Modern, "Helvetica Neue", Arial, Helvetica, Geneva, sans-serif;
        line-height: 135%;
        font-size: 120%;
        width:600px;
        margin-left:auto;
        margin-right:auto;
    }
    .CodeMirror{
            font-family: "Source Code Pro";
			font-size: 90%;
    }
/*    .prompt{
        display: None;
    }*/
    .text_cell_render h1 {
        font-weight: 200;
        font-size: 50pt;
		line-height: 100%;
        color:#CD2305;
        margin-bottom: 0.5em;
        margin-top: 0.5em;
        display: block;
    }
    .text_cell_render h5 {
        font-weight: 300;
        font-size: 16pt;
        color: #CD2305;
        font-style: italic;
        margin-bottom: .5em;
        margin-top: 0.5em;
        display: block;
    }

    .warning{
        color: rgb( 240, 20, 20 )
        }
</style>


# Withdraw Liquidity
A position $\alpha$ with $i_\alpha = i$ and $\Delta s_\alpha$ is withdrawn/destroyed. Recall that by the sign conventions, $\Delta s_\alpha < 0$.

```python
import os
import sys
sys.path.insert(0, os.path.abspath(os.path.join(os.path.abspath(''), '..')))

import inspect
from model.amm.omnipool_amm import remove_risk_liquidity
```

### Constraints
Removing liquidity must leave prices $p_j^Q$ unchanged for all assets $j$. It must also leave $\frac{R_i}{S_i}$ unchanged. We furthermore require that the value of the assets withdrawn ($p_i^Q\Delta r_\alpha + \Delta q_\alpha$) is $\frac{2p_i^Q}{p_i^Q + p_\alpha}\sqrt{\frac{p_i^Q}{p_\alpha}}$ percent of the assets originally contributed to the pool.

Note that $\Delta s_\alpha$ corresponds to $-\frac{\Delta s_\alpha}{S_i}R_i$ currently. We can calculate the initial $i$ contributed by $\alpha$ as $R_i^\alpha = \sqrt{\frac{p_i^Q}{p_\alpha}} \frac{-\Delta s_\alpha}{S_i}R_i$. The value withdrawn by LP should thus be (denominated in $i$) $\frac{2p_i^Q}{p_i^Q + p_\alpha}\frac{p_i^Q}{p_\alpha} \frac{-\Delta s_\alpha}{S_i}p_\alpha R_i$.

### Requirements
$$
-\Delta s_\alpha \leq s_\alpha\\
\Delta s_\alpha < 0
$$

### Updating AMM state

$$
\begin{align}
\Delta B_i &= max\left(\frac{p_i^Q - p_\alpha}{p_i^Q + p_\alpha}\Delta s_\alpha, -B_i\right)\\
\Delta S_i &= \Delta s_\alpha + \Delta B_i\\
\Delta R_i &= \frac{R_i}{S_i} \Delta S_i \\
\Delta Q_i &= Q_i \frac{\Delta R_i}{R_i}
\end{align}
$$

### Updating agent state

$$
\begin{align}
\Delta r_\alpha &= - \Delta R_i\\
\Delta q_\alpha &= - p_i^Q\left(\frac{2p_i^Q}{p_i^Q + p_\alpha} \frac{\Delta s_\alpha}{S_i}R_i + \Delta r_\alpha\right)
\end{align}
$$

```python
print(inspect.getsource(remove_risk_liquidity))
```

    def remove_risk_liquidity(
            old_state: dict,
            old_agents: dict,
            LP_id: string,
            delta_S: float,
            i: int
    ) -> tuple:
        """Compute new state after liquidity removal"""
        assert delta_S <= 0, "delta_S cannot be positive: " + str(delta_S)
        assert i >= 0, "invalid value for i: " + str(i)

        new_state = copy.deepcopy(old_state)
        new_agents = copy.deepcopy(old_agents)

        piq = price_i(old_state, i)
        p0 = new_agents[LP_id]['p'][i]
        mult = 2 * piq / (piq + p0) * math.sqrt(piq / p0)

        # Share update
        delta_B = max((mult - 1) * delta_S, - old_state['B'][i])
        new_state['B'][i] += delta_B
        new_state['S'][i] += delta_S + delta_B
        new_agents[LP_id]['s'][i] += delta_S

        # Token amounts update
        delta_R = old_state['R'][i] * max((delta_S + delta_B) / old_state['S'][i], -1)
        new_state['R'][i] += delta_R
        new_agents[LP_id]['r'][i] -= delta_R
        if piq >= p0:  # prevents rounding errors
            new_agents[LP_id]['q'] -= price_i(old_state, i) * (
                    mult * delta_S / old_state['S'][i] * old_state['R'][i] - delta_R)

        # HDX burn
        delta_Q = old_state['P'][i] * delta_R
        new_state['Q'][i] += delta_Q

        return new_state, new_agents

```python

```


###### Content under Creative Commons Attribution license CC-BY 4.0, code under MIT license (c)2014 L.A. Barba, C.D. Cooper, G.F. Forsyth.

# Spreading out

You've reached the fourth module of the open course [**"Practical Numerical Methods with Python"**](http://openedx.seas.gwu.edu/courses/GW/MAE6286/2014_fall/about), titled *Spreading out: Parabolic PDEs*. We hope that you are enjoying the ride of [#numericalmooc](https://twitter.com/hashtag/numericalmooc) so far!

We introduced finite-difference methods for partial differential equations (PDEs) in the [second module](https://github.com/numerical-mooc/numerical-mooc/tree/master/lessons/02_spacetime), and looked at convection problems in more depth in [module 3](https://github.com/numerical-mooc/numerical-mooc/tree/master/lessons/03_wave). Now we'll look at solving problems dominated by diffusion.

Why do we separate the discussion of how to solve convection-dominated and diffusion-dominated problems, you might ask? It's all about the harmony between mathematical model and numerical method. Convection and diffusion are inherently different physical processes.

The titles of the course modules are meant to spark your imagination:

* _Riding the wave_—imagine a surfer on a tall wave, moving fast towards the beach ... convection implies transport, speed, direction. The physics has a directional bias, and we discovered that numerical methods should be compatible with that. That's why we use _upwind_ methods for convection, and we pay attention to problems where waves move in opposite directions, needing special schemes.

* _Spreading out_—now imagine a drop of food dye in a cup of water, slowly spreading in all directions until all the liquid takes a uniform color. [Diffusion](http://en.wikipedia.org/wiki/Diffusion) spreads the concentration of something around (atoms, people, ideas, dirt, anything!). Since it is not a directional process, we need numerical methods that are isotropic (like central differences).

```
from IPython.display import Image
Image(url='http://upload.wikimedia.org/wikipedia/commons/f/f9/Blausen_0315_Diffusion.png')
```

## Parabolic PDEs

You already met the simplest parabolic PDE—the [1-D diffusion equation](http://nbviewer.ipython.org/github/numerical-mooc/numerical-mooc/blob/master/lessons/02_spacetime/02_03_1DDiffusion.ipynb)—in module 2. Its main feature is that it has a second-order derivative in space. Here it is again:

$$
\frac{\partial u}{\partial t} = \alpha \frac{\partial^2 u}{\partial x^2}
$$

Check out the article on [parabolic PDEs](http://en.wikipedia.org/wiki/Parabolic_partial_differential_equation) in Wikipedia. Now compare with the diffusion equation above, with the two independent variables here being $x, t$. You'll see that with no mixed derivatives, and only one second-order derivative (in the spatial variable $x$), it satisfies the condition of a parabolic PDE. Work it out on a piece of paper if you need to.

In the previous module, discussing hyperbolic conservation laws, we learned that solutions have characteristics: information travels along certain paths on space-time phase space. In contrast, parabolic equations don't have characteristics, because any local change in the initial condition will eventually affect the entire domain, although its effect will be felt at smaller intensity with longer distances. This is typical of diffusion processes.

In this first lesson of the module, we first review the 1D diffusion equation and then take a deeper look into the issue of boundary conditions. In the next notebook, we'll introduce _implicit discretizations_ for the first time, which will take us to the land of linear solvers. In the third lesson we'll graduate to two dimensions—more boundary condition and stability issues will come up. We'll then study 2D implicit methods, and go into Crank-Nicolson method: perhaps the most popular of them all. _Enjoy!_

## Heat conduction

Heat conduction is a diffusive process. Let's remind ourselves of the heat equation in one spatial dimension:

\begin{equation}
\frac{\partial T}{\partial t} = \alpha \frac{\partial^2 T}{\partial x^2}
\end{equation}

Here, $\alpha$ is the thermal diffusivity, a property of the material, and $T$ is the temperature.

In the [third lesson of module 2](http://nbviewer.ipython.org/github/numerical-mooc/numerical-mooc/blob/master/lessons/02_spacetime/02_03_1DDiffusion.ipynb), we discretized the diffusion equation  with a forward-time, centered-space scheme, subject to the following stability constraint:

\begin{equation}
\alpha \frac{\Delta t}{(\Delta x)^2} \leq \frac{1}{2}
\end{equation}

Let's look into it more deeply now, using a 1D temperature-evolution problem.

### Problem set up

Say we have a graphite rod, with [thermal diffusivity](http://en.wikipedia.org/wiki/Thermal_diffusivity) of $\alpha=1.22\times10^{-3} {\rm m}^2/{\rm s}$, length $L=1{\rm m}$, and temperature $T=0{\rm C}$ everywhere. At time $t=0$, we raise the temperature on the left-side end, $x=0$, to $T=100{\rm C}$, and hold it there. *How will the temperature evolve in the rod?*

As usual, start by importing some libraries and setting up the discretization. We'll begin by using a spatial grid with 51 points and advance for 100 time steps, using a forward-time/centered scheme.

.
#### Figure 1. Graphite rod, temperature fixed on ends.

```
import numpy
import matplotlib.pyplot as plt
%matplotlib inline
from matplotlib import rcParams
rcParams['font.family'] = 'serif'
rcParams['font.size'] = 16
```

```
L = 1.
nt = 100
nx = 51.
alpha = 1.22e-3

dx = L/(nx-1)

Ti = numpy.zeros(nx)
Ti[0] = 100
```

Remember the forward-time, centered-space discretization? You should work it out on a piece of paper yourself (if you can't do it without looking it up, it means you need to do this more!).

\begin{equation}\frac{T_{i}^{n+1}-T_{i}^{n}}{\Delta t}=\alpha\frac{T_{i+1}^{n}-2T_{i}^{n}+T_{i-1}^{n}}{\Delta x^2}\end{equation}

To obtain the temperature at the next time step, $T^{n+1}_i$, from the known information at the current time step, we compute

\begin{equation}T_{i}^{n+1}=T_{i}^{n}+\frac{\alpha\Delta t}{\Delta x^2}(T_{i+1}^{n}-2T_{i}^{n}+T_{i-1}^{n})\end{equation}

Check the [third notebook of module 2](http://nbviewer.ipython.org/github/numerical-mooc/numerical-mooc/blob/master/lessons/02_spacetime/02_03_1DDiffusion.ipynb), if you need to refresh your memory!

The following function implements this numerical scheme:

```
def ftcs(T, nt, dt, dx, alpha):
    """Solves the diffusion equation with forward-time, centered scheme

    Parameters:
    ----------
    T: array of float
        Initial temperature profile
    nt: int
        Number of time steps
    dt: float
        Time step size
    dx: float
        Mesh size
    alpha: float
        Diffusion coefficient (thermal diffusivity)

    Returns:
    -------
    T: array of float
        Temperature profile after nt time steps with forward in time scheme

    """
    for n in range(nt):
        Tn = T.copy()
        T[1:-1] = Tn[1:-1] + alpha*dt/dx**2*(Tn[2:] -2*Tn[1:-1] + Tn[0:-2])

    return T
```

We are all set to run! First, let's use a time step `dt` that satisfies the stability constraint

```
sigma = 1/2.0
dt = sigma * dx*dx/alpha
T = Ti.copy()
T = ftcs(T, nt, dt, dx, alpha)
```

And plot the solution!

```
plt.plot(numpy.linspace(0,1,nx), T, color='#003366', ls='-', lw=3)
plt.ylim(0,100)
plt.xlabel('Length of Rod')
plt.ylabel('Temperature');
```

## Boundary Conditions

In the rod problem above, we stated that the left-hand side of the rod is held to a temperature $T = 100{\rm C}$.  This is an example of a *boundary condition* (BC): a rule that defines how the problem interacts with the borders of the domain.  In this case, the domain spans the length of the rod, from $0 \leq x \leq 1$.

There are many types of boundary conditions, and they have an important effect on the solution of the problem. For example, in [module 2, lesson 4](http://nbviewer.ipython.org/urls/raw.github.com/numerical-mooc/numerical-mooc/master/lessons/02_spacetime/02_04_1DBurgers.ipynb) on Burgers' Equation, we used a periodic boundary condition to "connect" the right-hand and left-hand borders of the domain.

We need to discuss boundary conditions in a little more detail, and this is good place to do it. Read on!

### Dirichlet boundary

The rod example above uses a *Dirichlet* BC on the left-hand side.  A Dirichlet boundary is one in which the border is held to a specific value of the solution variable.

What about the right-hand end of the rod $(x=1)$?  In the discretization we set up above, the problem hasn't evolved for long enough time for the heat to travel the full length of the rod.  Let's increase the number of timesteps, `nt`, and see what happens.

```
nt = 1000
T = Ti.copy()
T = ftcs(T, nt, dt, dx, alpha)
```

```
plt.plot(numpy.linspace(0,1,nx), T, color='#003366', ls='-', lw=3)
plt.ylim(0,100)
plt.xlabel('Length of Rod')
plt.ylabel('Temperature');
```

Hmmm ... it looks like we're pinning the right BC to the value $T=0{\rm C}$, you agree?
Now study the code for the `ftbs` function, above, and try to figure out what happens at the right-most point of the spatial grid.

Did you figure it out?

It never updates the value at `T[-1]`!  That value is set to zero in the initial condition and stays that way throughout the entire simulation.  We effectively have a Dirichlet boundary at *both* ends.  The left-hand side of the rod is held to a temperature of $T = 100{\rm C}$ and the right-hand side is held to $T = 0{\rm C}$  Because both end tempera- tures are "pinned" to set values, the temperature distribution within the rod will relax (after enough time) to a linear temperature gradient across the length of the rod.

Once the solution is relaxed, no number of extra timesteps will make it change.  This is an example of a *steady-state solution* and we'll learn more about those in Module 5 of the course.

Dirichlet BCs show up in many engineering applications, among them thermo- dynamics (e.g., setting a surface temperature) and fluid dynamics (e.g., the no-slip conditions at walls in viscous fluids).

### Neumann boundary

Another commonly used BC is the *Neumann* boundary: rather than specifying the value of the solution at the border, they specify the value of the *derivative* of the solution at the border.

In our example, if we apply a Neumann boundary to the right-hand end of the rod, we can represent that mathematically as

\begin{equation}
\left. \frac{\partial T}{\partial x} \right|_{x = 1} = q(t)
\end{equation}

How can we enforce that in code?  One easy way is using a finite-difference discretization of the derivative at the boundary.  At time step $n$, for $N$ points, that would be

\begin{equation}
\left. \frac{\partial T}{\partial x} \right|_{x=1} \approx \frac{T^n_N - T^n_{N-1}}{\Delta x} = q(t)
\end{equation}

In the context of heat conduction, the space derivative of temperature is the heat flux density $q$, the amount of heat per unit time, per unit area.

For example, if the rod at $x=1$ has some insulating material, no heat is going to be able to get out through that end, and the Neumann boundary condition is:

\begin{equation}
\left. \frac{\partial T}{\partial x} \right|_{x=1} = 0
\end{equation}

Think about what the equation is saying: the change in temperature when moving in the $x$ direction is zero at the right-most edge of the rod. That means that the temperature should be equal on the last two spatial grid points.

To enforce this Neumann boundary condition at the right-hand end of the rod, we add the following line of code:

```Python
T[-1] = T[-2]
```

That is, the temperature at the boundary (`T[-1]`) is equal to the temperature directly to its left (`T[-2]`).  The spatial gradient is zero and a Neumann condition is satisfied there.

The Dirichlet conditions in the rod example were enforced without any extra input from us, but the Neumann condition will require an update at each iteration of the loop.

Finally, Dirichlet and Neumann BCs can sometimes be mixed to better represent the physics of a problem.

Let's revisit the heated-rod problem.  This time, the temperature at $x = 0$ will remain fixed at $T = 100{\rm C}$—that's the Dirichlet condition— and a Neumann BC applies at $x = 1$.

```
def ftcs_mixed(T, nt, dt, dx, alpha):
    """Solves the diffusion equation with forward-time, centered scheme using
    Dirichlet b.c. at left boundary and Neumann b.c. at right boundary

    Parameters:
    ----------
    u: array of float
        Initial temperature profile
    nt: int
        Number of time steps
    dt: float
        Time step size
    dx: float
        Mesh size
    alpha: float
        Diffusion coefficient (thermal diffusivity)

    Returns:
    -------
    u: array of float
        Temperature profile after nt time steps with forward in time scheme

    """
    for n in range(nt):
        Tn = T.copy()
        T[1:-1] = Tn[1:-1] + alpha*dt/dx**2*(Tn[2:] -2*Tn[1:-1] + Tn[0:-2])
        T[-1] = T[-2]

    return T
```

```
nt = 1000
T = Ti.copy()
T = ftcs_mixed(T, nt, dt, dx, alpha)
```

```
plt.plot(numpy.linspace(0,1,nx), T, color='#003366', ls='-', lw=3)
plt.ylim(0,100)
plt.title('Rod with LHS Dirichlet B.C. and RHS Neumann B.C.\n')
plt.xlabel('Length of Rod')
plt.ylabel('Temperature');
```

Now heat is accumulating in the domain. Our insulator works!  If you increase the number of timesteps `nt` further, you will see that the outflow temperature at $x = 1$ continues to increase.

### Explicit schemes and BCs

The numerical schemes we've learned so far are called _explicit_, because we compute the updated solution at $t^{n+1}$ using only known information at time $t^n$. This is simple but has limitations, particularly with the small sizes of time step necessary to obtain stability.

Here's another issue with explicit schemes. Figure 2 shows the superposed stencil of several grid points over three time steps computed using the forward-time, centered scheme. You know that to calculate $T_i^{n+1}$ you use the information from the grid points $i-1, i, i+1$ at the previous time step. Think about what happens at the boundary: any change in the boundary condition will feed into the solution only at the next time step, not immediately. But this contradicts the physics of the problem, as any change on the boundary should be felt right away under the diffusion equation. To include boundary effects on the same time level, we can use an _implicit_ update (subject of the next lesson).

.
#### Figure 2. Stencil with explicit scheme over 3 time steps.

### Explicit schemes and time step

Let's replace the parameter $\sigma=\alpha \frac{\Delta t}{(\Delta x)^2}$ into Equation 4 and rearrange it a little bit, grouping together the terms at grid point $i$ on the right-hand side:

\begin{equation}
T_{i}^{n+1} = \sigma T_{i-1}^{n}+(1- 2 \sigma) T_{i}^{n} + \sigma T_{i+1}^{n}
\end{equation}

It's helpful to look at a sketch of the stencil for the heat equation with the weights added in for the contribution of each grid point. See Figure 3, which shows the weights for two values of $\sigma$: $\frac{1}{2}$ and $1$. Notice that with $\sigma=\frac{1}{2}$, the solution variable at the next time step is _independent_ from its own value at the present step, which seems a bit weird. And with $\sigma=1$, the current value affects negatively the future value—now _that's_ a red flag!

In fact, the solution will develop growing errors with $\sigma>\frac{1}{2}$, i.e., become unstable. This limits the time step that can be used quite significantly, because $\Delta t \propto (\Delta x)^2$. This is a serious cost of _explicit_ methods and an incentive to consider _implicit_ alternatives (next lesson).

.
#### Figure 3. Stencils for the heat equation with $\sigma= \frac{1}{2}, 1$.

---
###### The cell below loads the style of the notebook

```
from IPython.core.display import HTML
css_file = '../../styles/numericalmoocstyle.css'
HTML(open(css_file, "r").read())
```

<link href='http://fonts.googleapis.com/css?family=Alegreya+Sans:100,300,400,500,700,800,900,100italic,300italic,400italic,500italic,700italic,800italic,900italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Arvo:400,700,400italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=PT+Mono' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Shadows+Into+Light' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Nixie+One' rel='stylesheet' type='text/css'>
<style>

@font-face {
    font-family: "Computer Modern";
    src: url('http://mirrors.ctan.org/fonts/cm-unicode/fonts/otf/cmunss.otf');
}

#notebook_panel { /* main background */
    background: rgb(245,245,245);
}

div.cell { /* set cell width */
    width: 750px;
}

div #notebook { /* centre the content */
    background: #fff; /* white background for content */
    width: 1000px;
    margin: auto;
    padding-left: 0em;
}

#notebook li { /* More space between bullet points */
margin-top:0.8em;
}

/* draw border around running cells */
div.cell.border-box-sizing.code_cell.running {
    border: 1px solid #111;
}

/* Put a solid color box around each cell and its output, visually linking them*/
div.cell.code_cell {
    background-color: rgb(256,256,256);
    border-radius: 0px;
    padding: 0.5em;
    margin-left:1em;
    margin-top: 1em;
}

div.text_cell_render{
    font-family: 'Alegreya Sans' sans-serif;
    line-height: 140%;
    font-size: 125%;
    font-weight: 400;
    width:600px;
    margin-left:auto;
    margin-right:auto;
}

/* Formatting for header cells */
.text_cell_render h1 {
    font-family: 'Nixie One', serif;
    font-style:regular;
    font-weight: 400;
    font-size: 45pt;
    line-height: 100%;
    color: rgb(0,51,102);
    margin-bottom: 0.5em;
    margin-top: 0.5em;
    display: block;
}
.text_cell_render h2 {
    font-family: 'Nixie One', serif;
    font-weight: 400;
    font-size: 30pt;
    line-height: 100%;
    color: rgb(0,51,102);
    margin-bottom: 0.1em;
    margin-top: 0.3em;
    display: block;
}

.text_cell_render h3 {
    font-family: 'Nixie One', serif;
    margin-top:16px;
	font-size: 22pt;
    font-weight: 600;
    margin-bottom: 3px;
    font-style: regular;
    color: rgb(102,102,0);
}

.text_cell_render h4 {    /*Use this for captions*/
    font-family: 'Nixie One', serif;
    font-size: 14pt;
    text-align: center;
    margin-top: 0em;
    margin-bottom: 2em;
    font-style: regular;
}

.text_cell_render h5 {  /*Use this for small titles*/
    font-family: 'Nixie One', sans-serif;
    font-weight: 400;
    font-size: 16pt;
    color: rgb(163,0,0);
    font-style: italic;
    margin-bottom: .1em;
    margin-top: 0.8em;
    display: block;
}

.text_cell_render h6 { /*use this for copyright note*/
    font-family: 'PT Mono', sans-serif;
    font-weight: 300;
    font-size: 9pt;
    line-height: 100%;
    color: grey;
    margin-bottom: 1px;
    margin-top: 1px;
}

.CodeMirror{
        font-family: "PT Mono";
        font-size: 90%;
}

</style>


```python
import os
import numpy as np
import pandas as pd
import time

from regain.covariance import GraphicalLasso as rg_GL
from sklearn.covariance import GraphicalLasso as sk_GL

from regain_benchmark import regain_time
from sklearn_benchmark import sklearn_time
from gglasso_benchmark import gglasso_time

from utilita import network_generation, model_solution, benchmark_parameters
from utilita import sparsity_benchmark, dict_shape, calc_hamming_dict, benchmarks_dataframe

from plots import plot_accuracy, plot_scalability, plot_lambdas, plot_bm

from tqdm.contrib import tzip
from tqdm.contrib.itertools import product

pd.set_option('display.max_columns', None)
```

The notebook illustrates the comparison of methods solving [Graphical lasso](https://en.wikipedia.org/wiki/Graphical_lasso) problem implemented in [regain](https://github.com/fdtomasi/regain), [gglasso](https://github.com/fabian-sp/GGLasso/tree/o-benchmarking) and [sklearn](https://scikit-learn.org/stable/modules/generated/sklearn.covariance.graphical_lasso.html) libraries, respectivelly.

### Power networks

We generate a precision matrix following the approach of [Danaher et al.,
2014](https://arxiv.org/abs/1111.0324) where blockwise [subnetworks](https://en.wikipedia.org/wiki/Scale-free_network) are sampled from a power-law degree distribution.

```python
S_dict=dict()
X_dict=dict()
Theta_dict=dict()

#p_list=[100, 500, 1000, 2000]
#N_list=[110, 550, 1100, 2200]

p_list=[100, 200]
N_list=[110, 220]

print(" Power network generation ".center(40, '-'))

for p, N in tzip(p_list, N_list):
    start = time.perf_counter()
    S, X, Theta = network_generation(p, N, M=10)
    end = time.perf_counter()
    print("p: %5d, N : %5d, Time : %5.4f" % (p, N, end-start))

    S_dict[p, N] = S.copy()
    X_dict[p, N] = X.copy()
    Theta_dict[p, N] = Theta.copy()
```

```python
print("\n Shape of S_i:", dict_shape(S_dict))
print("\n Shape of X_i:", dict_shape(X_dict))
print("\n Shape of Theta_i:", dict_shape(Theta_dict))
```

### Hyperparameters

Since the solution of graphical lasso varies in the presented methods, we pick hyperparameters which allow us to have approximatelly similar solutions for all the solvers, so one can compare the methods.

We take the following steps to achieve comparable soltuions of graphical lasso:
- Solve graphical problem by one of the methods at the high level of precision ( $Z^{*}$)
- Solve graphical problem by each of the solvers at the lower level of precision ($Z_{i}$)
- Calculate the normalized Eucledean distance ($\text{accuracy}$) between $Z^{*}$ and solution $Z_{i}$:
$$
\begin{align}
\text{accuracy} = \frac{\lVert Z^{*} - Z_{i} \rVert}{\lVert Z^{*} \rVert}
\end{align}
$$

```python
lambda_list = [0.5, 0.1, 0.05]
sk_params, rg_params, gglasso_params, lambda_list = benchmark_parameters(lambda_list = lambda_list)
```

### Model solution

```python
model_time_dict = dict()
model_Z_dict = dict()
reference_solver = "regain"

print(f"Solving for a reference solution with solver {reference_solver}:")

for X, l1 in product(list(X_dict.values()), lambda_list):

    Z, Z_time, info = model_solution(solver=reference_solver, X=X, lambda1=l1)

    key = "p_" + str(X.shape[1]) + "_N_" + str(X.shape[0]) + "_l1_" + str(l1)
    model_time_dict.update({key: Z_time})
    model_Z_dict.update({key: Z})
```

```python
time_dict = dict()
accuracy_dict = dict()
Z_dict = dict()
iter_dict = dict()
```

```python
# how often to repeat each call (averaging runtime)
n_iter = 2
```

### GGLasso

```python
for X, S in tzip(list(X_dict.values()), list(S_dict.values())):
    Omega_0 = np.eye(len(S))
    gg_result = gglasso_time(S=S, X=X, Omega_0=Omega_0, Z=model_Z_dict, lambda_list=lambda_list,
                                              n_iter=n_iter, gglasso_params=gglasso_params, warm_start=False)

    time_dict.update(gg_result['time'])
    accuracy_dict.update(gg_result['accuracy'])
    Z_dict.update(gg_result['sol'])
    iter_dict.update(gg_result['iter'])
```

### Sklearn

```python
for X, S in tzip(list(X_dict.values()), list(S_dict.values())):
    sk_result = sklearn_time(X=X, Z=model_Z_dict, sk_params=sk_params, lambda_list=lambda_list, \
                                              n_iter=n_iter, max_iter=30)

    time_dict.update(sk_result['time'])
    accuracy_dict.update(sk_result['accuracy'])
    Z_dict.update(sk_result['sol'])
    iter_dict.update(sk_result['iter'])
```

### Regain

```python
for X, S in tzip(list(X_dict.values()), list(S_dict.values())):
    rg_result = regain_time(X=X, Z=model_Z_dict, rg_params=rg_params, lambda_list=lambda_list, \
                                             n_iter=n_iter, warm_start=False)

    time_dict.update(rg_result['time'])
    accuracy_dict.update(rg_result['accuracy'])
    Z_dict.update(rg_result['sol'])
    iter_dict.update(rg_result['iter'])
```

### Hamming distances

We calculate [Hamming](https://en.wikipedia.org/wiki/Hamming_distance) distance between model solution $Z^{*}$ and given solution $Z_{i}$ with a specified rounding threshold for non zero entries.

```python
hamming_dict = calc_hamming_dict(Theta_dict=Theta_dict, Z_dict=Z_dict, t_rounding=1e-8)
```

### Visualization

```python
df = benchmarks_dataframe(time_dict, accuracy_dict, hamming_dict, iter_dict)
df.head()

#df.to_csv('../data/synthetic/bm2000.csv')
```

```python
plot_bm(df, min_acc = 1e-3, lambda_list=lambda_list)
```

```python
fig = plot_lambdas(df, upper_bound=0.1, lower_bound=0.000001)
fig.show()
```

It is clear that solving graphical lasso on individual blocks is less time consuming then solving it on the entire graph.

So, one can advise to use a block version of ADMM implemented in gglasso in a high-dimensional setup.

```python
fig = plot_accuracy(df, upper_bound=0.1, lower_bound=0.0000001, lambda_filter=0.05, sortby=['p', 'time'])
fig.show()
```

```python
frames = sparsity_benchmark(df, upper_bound=0.01, lower_bound=0.0001, lambda_filter=0.1)
frames
```