```python
%matplotlib inline
```

Background on projectors and projections
========================================

This tutorial provides background information on projectors and Signal Space
Projection (SSP), and covers loading and saving projectors, adding and removing
projectors from Raw objects, the difference between "applied" and "unapplied"
projectors, and at what stages MNE-Python applies projectors automatically.
   :depth: 2

We'll start by importing the Python modules we need; we'll also define a short
function to make it easier to make several plots that look similar:

```python
import os
import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D  # noqa
from scipy.linalg import svd
import mne

def setup_3d_axes():
    ax = plt.axes(projection='3d')
    ax.view_init(azim=-105, elev=20)
    ax.set_xlabel('x')
    ax.set_ylabel('y')
    ax.set_zlabel('z')
    ax.set_xlim(-1, 5)
    ax.set_ylim(-1, 5)
    ax.set_zlim(0, 5)
    return ax
```

What is a projection?
^^^^^^^^^^^^^^^^^^^^^

In the most basic terms, a *projection* is an operation that converts one set
of points into another set of points, where repeating the projection
operation on the resulting points has no effect. To give a simple geometric
example, imagine the point $(3, 2, 5)$ in 3-dimensional space. A
projection of that point onto the $x, y$ plane looks a lot like a
shadow cast by that point if the sun were directly above it:

```python
ax = setup_3d_axes()

# plot the vector (3, 2, 5)
origin = np.zeros((3, 1))
point = np.array([[3, 2, 5]]).T
vector = np.hstack([origin, point])
ax.plot(*vector, color='k')
ax.plot(*point, color='k', marker='o')

# project the vector onto the x,y plane and plot it
xy_projection_matrix = np.array([[1, 0, 0], [0, 1, 0], [0, 0, 0]])
projected_point = xy_projection_matrix @ point
projected_vector = xy_projection_matrix @ vector
ax.plot(*projected_vector, color='C0')
ax.plot(*projected_point, color='C0', marker='o')

# add dashed arrow showing projection
arrow_coords = np.concatenate([point, projected_point - point]).flatten()
ax.quiver3D(*arrow_coords, length=0.96, arrow_length_ratio=0.1, color='C1',
            linewidth=1, linestyle='dashed')
```

Note

The ``@`` symbol indicates matrix multiplication on NumPy arrays, and was
    introduced in Python 3.5 / NumPy 1.10. The notation ``plot(*point)`` uses
    Python `argument expansion`_ to "unpack" the elements of ``point`` into
    separate positional arguments to the function. In other words,
    ``plot(*point)`` expands to ``plot(3, 2, 5)``.

Notice that we used matrix multiplication to compute the projection of our
point $(3, 2, 5)$onto the $x, y$ plane:

\begin{align}\left[
      \begin{matrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 0 \end{matrix}
    \right]
    \left[ \begin{matrix} 3 \\ 2 \\ 5 \end{matrix} \right] =
    \left[ \begin{matrix} 3 \\ 2 \\ 0 \end{matrix} \right]\end{align}

...and that applying the projection again to the result just gives back the
result again:

\begin{align}\left[
      \begin{matrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 0 \end{matrix}
    \right]
    \left[ \begin{matrix} 3 \\ 2 \\ 0 \end{matrix} \right] =
    \left[ \begin{matrix} 3 \\ 2 \\ 0 \end{matrix} \right]\end{align}

From an information perspective, this projection has taken the point
$x, y, z$ and removed the information about how far in the $z$
direction our point was located; all we know now is its position in the
$x, y$ plane. Moreover, applying our projection matrix to *any point*
in $x, y, z$ space will reduce it to a corresponding point on the
$x, y$ plane. The term for this is a *subspace*: the projection matrix
projects points in the original space into a *subspace* of lower dimension
than the original. The reason our subspace is the $x,y$ plane (instead
of, say, the $y,z$ plane) is a direct result of the particular values
in our projection matrix.

Example: projection as noise reduction
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Another way to describe this "loss of information" or "projection into a
subspace" is to say that projection reduces the rank (or "degrees of
freedom") of the measurement — here, from 3 dimensions down to 2. On the
other hand, if you know that measurement component in the $z$ direction
is just noise due to your measurement method, and all you care about are the
$x$ and $y$ components, then projecting your 3-dimensional
measurement into the $x, y$ plane could be seen as a form of noise
reduction.

Of course, it would be very lucky indeed if all the measurement noise were
concentrated in the $z$ direction; you could just discard the $z$
component without bothering to construct a projection matrix or do the matrix
multiplication. Suppose instead that in order to take that measurement you
had to pull a trigger on a measurement device, and the act of pulling the
trigger causes the device to move a little. If you measure how
trigger-pulling affects measurement device position, you could then "correct"
your real measurements to "project out" the effect of the trigger pulling.
Here we'll suppose that the average effect of the trigger is to move the
measurement device by $(3, -1, 1)$:

```python
trigger_effect = np.array([[3, -1, 1]]).T
```

Knowing that, we can compute a plane that is orthogonal to the effect of the
trigger (using the fact that a plane through the origin has equation
$Ax + By + Cz = 0$ given a normal vector $(A, B, C)$), and
project our real measurements onto that plane.

```python
# compute the plane orthogonal to trigger_effect
x, y = np.meshgrid(np.linspace(-1, 5, 61), np.linspace(-1, 5, 61))
A, B, C = trigger_effect
z = (-A * x - B * y) / C
# cut off the plane below z=0 (just to make the plot nicer)
mask = np.where(z >= 0)
x = x[mask]
y = y[mask]
z = z[mask]
```

Computing the projection matrix from the ``trigger_effect`` vector is done
using `singular value decomposition <svd_>`_ (SVD); interested readers may
consult the internet or a linear algebra textbook for details on this method.
With the projection matrix in place, we can project our original vector
$(3, 2, 5)$ to remove the effect of the trigger, and then plot it:

```python
# compute the projection matrix
U, S, V = svd(trigger_effect, full_matrices=False)
trigger_projection_matrix = np.eye(3) - U @ U.T

# project the vector onto the orthogonal plane
projected_point = trigger_projection_matrix @ point
projected_vector = trigger_projection_matrix @ vector

# plot the trigger effect and its orthogonal plane
ax = setup_3d_axes()
ax.plot_trisurf(x, y, z, color='C2', shade=False, alpha=0.25)
ax.quiver3D(*np.concatenate([origin, trigger_effect]).flatten(),
            arrow_length_ratio=0.1, color='C2', alpha=0.5)

# plot the original vector
ax.plot(*vector, color='k')
ax.plot(*point, color='k', marker='o')
offset = np.full((3, 1), 0.1)
ax.text(*(point + offset).flat, '({}, {}, {})'.format(*point.flat), color='k')

# plot the projected vector
ax.plot(*projected_vector, color='C0')
ax.plot(*projected_point, color='C0', marker='o')
offset = np.full((3, 1), -0.2)
ax.text(*(projected_point + offset).flat,
        '({}, {}, {})'.format(*np.round(projected_point.flat, 2)),
        color='C0', horizontalalignment='right')

# add dashed arrow showing projection
arrow_coords = np.concatenate([point, projected_point - point]).flatten()
ax.quiver3D(*arrow_coords, length=0.96, arrow_length_ratio=0.1,
            color='C1', linewidth=1, linestyle='dashed')
```

Just as before, the projection matrix will map *any point* in $x, y, z$
space onto that plane, and once a point has been projected onto that plane,
applying the projection again will have no effect. For that reason, it should
be clear that although the projected points vary in all three $x$,
$y$, and $z$ directions, the set of projected points have only
two *effective* dimensions (i.e., they are constrained to a plane).

.. sidebar:: Terminology

    In MNE-Python, the matrix used to project a raw signal into a subspace is
    usually called a :term:`projector <projector>` or a *projection
    operator* — these terms are interchangeable with the term *projection
    matrix* used above.

Projections of EEG or MEG signals work in very much the same way: the point
$x, y, z$ corresponds to the value of each sensor at a single time
point, and the projection matrix varies depending on what aspects of the
signal (i.e., what kind of noise) you are trying to project out. The only
real difference is that instead of a single 3-dimensional point $(x, y,
z)$ you're dealing with a time series of $N$-dimensional "points" (one
at each sampling time), where $N$ is usually in the tens or hundreds
(depending on how many sensors your EEG/MEG system has). Fortunately, because
projection is a matrix operation, it can be done very quickly even on signals
with hundreds of dimensions and tens of thousands of time points.

Signal-space projection (SSP)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

We mentioned above that the projection matrix will vary depending on what
kind of noise you are trying to project away. Signal-space projection (SSP)
[1]_ is a way of estimating what that projection matrix should be, by
comparing measurements with and without the signal of interest. For example,
you can take additional "empty room" measurements that record activity at the
sensors when no subject is present. By looking at the spatial pattern of
activity across MEG sensors in an empty room measurement, you can create one
or more $N$-dimensional vector(s) giving the "direction(s)" of
environmental noise in sensor space (analogous to the vector for "effect of
the trigger" in our example above). SSP is also often used for removing
heartbeat and eye movement artifacts — in those cases, instead of empty room
recordings the direction of the noise is estimated by detecting the
artifacts, extracting epochs around them, and averaging. See
`tut-artifact-ssp` for examples.

Once you know the noise vectors, you can create a hyperplane that is
orthogonal
to them, and construct a projection matrix to project your experimental
recordings onto that hyperplane. In that way, the component of your
measurements associated with environmental noise can be removed. Again, it
should be clear that the projection reduces the dimensionality of your data —
you'll still have the same number of sensor signals, but they won't all be
*linearly independent* — but typically there are tens or hundreds of sensors
and the noise subspace that you are eliminating has only 3-5 dimensions, so
the loss of degrees of freedom is usually not problematic.

Projectors in MNE-Python
^^^^^^^^^^^^^^^^^^^^^^^^

In our example data, `SSP <ssp-tutorial>` has already been performed
using empty room recordings, but the :term:`projectors <projector>` are
stored alongside the raw data and have not been *applied* yet (or,
synonymously, the projectors are not *active* yet). Here we'll load
the `sample data <sample-dataset>` and crop it to 60 seconds; you can
see the projectors in the output of :func:`~mne.io.read_raw_fif` below:

```python
sample_data_folder = mne.datasets.sample.data_path()
sample_data_raw_file = os.path.join(sample_data_folder, 'MEG', 'sample',
                                    'sample_audvis_raw.fif')
raw = mne.io.read_raw_fif(sample_data_raw_file)
raw.crop(tmax=60).load_data()
```

In MNE-Python, the environmental noise vectors are computed using `principal
component analysis <pca_>`_, usually abbreviated "PCA", which is why the SSP
projectors usually have names like "PCA-v1". (Incidentally, since the process
of performing PCA uses `singular value decomposition <svd_>`_ under the hood,
it is also common to see phrases like "projectors were computed using SVD" in
published papers.) The projectors are stored in the ``projs`` field of
``raw.info``:

```python
print(raw.info['projs'])
```

``raw.info['projs']`` is an ordinary Python :class:`list` of
:class:`~mne.Projection` objects, so you can access individual projectors by
indexing into it. The :class:`~mne.Projection` object itself is similar to a
Python :class:`dict`, so you can use its ``.keys()`` method to see what
fields it contains (normally you don't need to access its properties
directly, but you can if necessary):

```python
first_projector = raw.info['projs'][0]
print(first_projector)
print(first_projector.keys())
```

The :class:`~mne.io.Raw`, :class:`~mne.Epochs`, and :class:`~mne.Evoked`
objects all have a boolean :attr:`~mne.io.Raw.proj` attribute that indicates
whether there are any unapplied / inactive projectors stored in the object.
In other words, the :attr:`~mne.io.Raw.proj` attribute is ``True`` if at
least one :term:`projector` is present and all of them are active. In
addition, each individual projector also has a boolean ``active`` field:

```python
print(raw.proj)
print(first_projector['active'])
```

Computing projectors
~~~~~~~~~~~~~~~~~~~~

```python
# In MNE-Python, SSP vectors can be computed using general purpose functions
# :func:`mne.compute_proj_raw`, :func:`mne.compute_proj_epochs`, and
# :func:`mne.compute_proj_evoked`. The general assumption these functions make
# is that the data passed contains raw data, epochs or averages of the artifact
# you want to repair via projection. In practice this typically involves
# continuous raw data of empty room recordings or averaged ECG or EOG
# artifacts. A second set of high-level convenience functions is provided to
# compute projection vectors for typical use cases. This includes
# :func:`mne.preprocessing.compute_proj_ecg` and
# :func:`mne.preprocessing.compute_proj_eog` for computing the ECG and EOG
# related artifact components, respectively; see :ref:`tut-artifact-ssp` for
# examples of these uses. For computing the EEG reference signal as a
# projector, the function :func:`mne.set_eeg_reference` can be used; see
# :ref:`tut-set-eeg-ref` for more information.
#
# .. warning:: It is best to compute projectors only on channels that will be
#              used (e.g., excluding bad channels). This ensures that
#              projection vectors will remain ortho-normalized and that they
#              properly capture the activity of interest.
#
#
# Visualizing the effect of projectors
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#
# You can see the effect the projectors are having on the measured signal by
# comparing plots with and without the projectors applied. By default,
# ``raw.plot()`` will apply the projectors in the background before plotting
# (without modifying the :class:`~mne.io.Raw` object); you can control this
# with the boolean ``proj`` parameter as shown below, or you can turn them on
# and off interactively with the projectors interface, accessed via the
# :kbd:`Proj` button in the lower right corner of the plot window. Here we'll
# look at just the magnetometers, and a 2-second sample from the beginning of
# the file.

mags = raw.copy().crop(tmax=2).pick_types(meg='mag')
for proj in (False, True):
    fig = mags.plot(butterfly=True, proj=proj)
    fig.subplots_adjust(top=0.9)
    fig.suptitle('proj={}'.format(proj), size='xx-large', weight='bold')
```

Additional ways of visualizing projectors are covered in the tutorial
`tut-artifact-ssp`.

Loading and saving projectors
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

SSP can be used for other types of signal cleaning besides just reduction of
environmental noise. You probably noticed two large deflections in the
magnetometer signals in the previous plot that were not removed by the
empty-room projectors — those are artifacts of the subject's heartbeat. SSP
can be used to remove those artifacts as well. The sample data includes
projectors for heartbeat noise reduction that were saved in a separate file
from the raw data, which can be loaded with the :func:`mne.read_proj`
function:

```python
ecg_proj_file = os.path.join(sample_data_folder, 'MEG', 'sample',
                             'sample_audvis_ecg-proj.fif')
ecg_projs = mne.read_proj(ecg_proj_file)
print(ecg_projs)
```

There is a corresponding :func:`mne.write_proj` function that can be used to
save projectors to disk in ``.fif`` format:

.. code-block:: python3

    mne.write_proj('heartbeat-proj.fif', ecg_projs)

Note

By convention, MNE-Python expects projectors to be saved with a filename
    ending in ``-proj.fif`` (or ``-proj.fif.gz``), and will issue a warning
    if you forgo this recommendation.

Adding and removing projectors
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Above, when we printed the ``ecg_projs`` list that we loaded from a file, it
showed two projectors for gradiometers (the first two, marked "planar"), two
for magnetometers (the middle two, marked "axial"), and two for EEG sensors
(the last two, marked "eeg"). We can add them to the :class:`~mne.io.Raw`
object using the :meth:`~mne.io.Raw.add_proj` method:

```python
raw.add_proj(ecg_projs)
```

To remove projectors, there is a corresponding method
:meth:`~mne.io.Raw.del_proj` that will remove projectors based on their index
within the ``raw.info['projs']`` list. For the special case of replacing the
existing projectors with new ones, use
``raw.add_proj(ecg_projs, remove_existing=True)``.

To see how the ECG projectors affect the measured signal, we can once again
plot the data with and without the projectors applied (though remember that
the :meth:`~mne.io.Raw.plot` method only *temporarily* applies the projectors
for visualization, and does not permanently change the underlying data).
We'll compare the ``mags`` variable we created above, which had only the
empty room SSP projectors, to the data with both empty room and ECG
projectors:

```python
mags_ecg = raw.copy().crop(tmax=2).pick_types(meg='mag')
for data, title in zip([mags, mags_ecg], ['Without', 'With']):
    fig = data.plot(butterfly=True, proj=True)
    fig.subplots_adjust(top=0.9)
    fig.suptitle('{} ECG projector'.format(title), size='xx-large',
                 weight='bold')
```

When are projectors "applied"?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

By default, projectors are applied when creating :class:`epoched
<mne.Epochs>` data from :class:`~mne.io.Raw` data, though application of the
projectors can be *delayed* by passing ``proj=False`` to the
:class:`~mne.Epochs` constructor. However, even when projectors have not been
applied, the :meth:`mne.Epochs.get_data` method will return data *as if the
projectors had been applied* (though the :class:`~mne.Epochs` object will be
unchanged). Additionally, projectors cannot be applied if the data are not
`preloaded <memory>`. If the data are `memory-mapped`_ (i.e., not
preloaded), you can check the ``_projector`` attribute to see whether any
projectors will be applied once the data is loaded in memory.

Finally, when performing inverse imaging (i.e., with
:func:`mne.minimum_norm.apply_inverse`), the projectors will be
automatically applied. It is also possible to apply projectors manually when
working with :class:`~mne.io.Raw`, :class:`~mne.Epochs` or
:class:`~mne.Evoked` objects via the object's :meth:`~mne.io.Raw.apply_proj`
method. For all instance types, you can always copy the contents of
:samp:`{<instance>}.info['projs']` into a separate :class:`list` variable,
use :samp:`{<instance>}.del_proj({<index of proj(s) to remove>})` to remove
one or more projectors, and then add them back later with
:samp:`{<instance>}.add_proj({<list containing projs>})` if desired.

Warning

Remember that once a projector is applied, it can't be un-applied, so
    during interactive / exploratory analysis it's a good idea to use the
    object's :meth:`~mne.io.Raw.copy` method before applying projectors.

Best practices
~~~~~~~~~~~~~~

In general, it is recommended to apply projectors when creating
:class:`~mne.Epochs` from :class:`~mne.io.Raw` data. There are two reasons
for this recommendation:

1. It is computationally cheaper to apply projectors to data *after* the
   data have been reducted to just the segments of interest (the epochs)

2. If you are applying amplitude-based rejection criteria to epochs, it is
   preferable to reject based on the signal *after* projectors have been
   applied, because the projectors may reduce noise in some epochs to
   tolerable levels (thereby increasing the number of acceptable epochs and
   consequenty increasing statistical power in any later analyses).

References
^^^^^^^^^^

.. [1] Uusitalo MA and Ilmoniemi RJ. (1997). Signal-space projection method
       for separating MEG or EEG into components. *Med Biol Eng Comput*
       35(2), 135–140. doi:10.1007/BF02534144

.. LINKS

   https://docs.python.org/3/tutorial/controlflow.html#tut-unpacking-arguments


# Exercise 7) Learning and Planning

In this exercise, we will again investigate the inverted pendulum from the `gym` environment. We want to check, which benefits the implementation of planning offers.

Please note that the parameter $n$ has a different meaning in the context of planning (number of planning steps per actual step) than in the context of n-step learning.

```python
import numpy as np
import gym
gym.logger.set_level(40)
import random
import time
from tqdm.notebook import tqdm
import matplotlib.pyplot as plt
plt.style.use('seaborn-talk')
```

We will reuse the discretization routine from the previous exercise:

```python
d_T = 15
d_theta = 15
d_omega = 15

def discretize_state(states):
    limits = [1, 1, 8]
    nb_disc_intervals = [d_theta, d_theta, d_omega]

    # bring to value range [-1, 1]
    norm_states = [state / limit for state, limit in zip(states, limits)]
    interval_lengths = [2 / d for d in nb_disc_intervals]
    disc_state = [(norm_state + 1) // interval_length for norm_state, interval_length in zip(norm_states, interval_lengths)]
    disc_state = [(state - 1) if state == d else state for state, d in zip(disc_state, nb_disc_intervals)] # ensure that disc_state < d

    return np.array(disc_state)

def continualize_action(disc_action):
    limit = 2
    interval_length = 2 / (d_T-1)
    norm_action = disc_action * interval_length
    cont_action = (norm_action - 1) * limit
    return np.array(cont_action).flatten()
```

## 1) Dyna-Q

Write a Dyna-Q algorithm to solve the inverted pendulum. Check the quality of the result for different number of episodes, number of steps per episode and number of planning steps per interaction.

Make sure that the total number of learning steps stays the same for different n, such that comparisons are fair:

$\text{episodes} \cdot \text{steps} \cdot (1+n) = \text{const.}$

Interesting metrics for a comparison could be e.g. the execution time (the tqdm loading bar shows execution time of loops, alternatively you can use the time.time() command to get the momentary system time in seconds) and training stability.

## Solution 1)

YOUR ANSWER HERE

```python
def pendulumDynaQ(alpha, gamma, epsilon, n, nb_episodes, nb_steps):

    env = gym.make('Pendulum-v0')
    env = env.unwrapped

    action_values = np.zeros([d_theta, d_theta, d_omega, d_T])
    pi = np.zeros([d_theta, d_theta, d_omega])

    model = {} # dictionary

    cumulative_reward_history = [] # we can use this to figure out how well the learning worked

    for j in tqdm(range(nb_episodes), position=0, leave=True):

        # YOUR CODE HERE
        raise NotImplementedError()

    return cumulative_reward_history, pi

```

Function to evaluate and render the measurement using the gym environment:

```python
def experiment(pi,nb_steps = 300):
# Runs the inverted pendulum experiment using policy pi for nb_steps steps

    env = gym.make('Pendulum-v0')
    env = env.unwrapped

    state = env.reset() # initialize x_0
    disc_state = tuple(discretize_state(state).astype(int)) # only tuples of integers can be used as index
    disc_action = pi[disc_state].astype(int)

    for k in range(nb_steps):

        cont_action = continualize_action(disc_action)
        env.render() # comment out for faster execution
        state, reward, done, _ = env.step(cont_action)
        disc_state = tuple(discretize_state(state).astype(int))

        if done:
            break

        disc_action = pi[disc_state].astype(int) # exploitative action

    env.close()
```

YOUR ANSWER HERE

```python
### train
print("Run without planning")
no_planning_history, no_planning_pi = pendulumDynaQ(alpha = 0.1, gamma = 0.9, epsilon = 0.1, n = 0, nb_episodes = 5000, nb_steps = 500)
```

```python
### run and render the experiment
experiment(no_planning_pi,nb_steps = 300)
```

YOUR ANSWER HERE

```python
### train
print("Run with planning")
with_planning_history, with_planning_pi = pendulumDynaQ(alpha = 0.1, gamma = 0.9, epsilon = 0.1, n = 9, nb_episodes = 500, nb_steps = 500)
```

```python
### run and render the experiment
experiment(with_planning_pi,nb_steps = 300)
```

Now lets compare the cumulative rewards:

```python
plt.plot(no_planning_history)
plt.plot(with_planning_history)
plt.xlabel("episode")
plt.ylabel(r"$\sum R$")
plt.show()
```

YOUR ANSWER HERE

## 2) Simulation-based planning

Although it can be useful for small state spaces, building a system model by storing large amounts of state transitions like in task (1) is rarely feasible in engineering. As engineers, we are capable of a more efficient way of system modeling that we can utilize here: differential equations.

Using a state-space model allows to efficiently integrate existing pre-knowledge into the Dyna-Q algorithm we already used. To do so, write a class `pendulum_model` that implements a model of the pendulum. This class should work similar to `gym`: it should at least have a `step` and a `reset` method. In the step method, make use of forward Euler integration to simulate the system dynamics. In the reset method, allow to pass an optional initial state to the model, such that we can easily compare model and environment. If no initial state is passed to the `reset` function, a random initial state should be determined.

Integrate this model into a Dyna-Q algorithm.

Model of the pendulum in differential-equation form for change of the angular frequency $\omega$ and the angle $\theta$ depending on the torque $T_\mathrm{u}$:

\begin{align}
\dot{\omega} &= -\frac{3 g}{2 l} \text{sin}(\theta +\pi) + \frac{1}{J} T_\mathrm{u}
\\
\dot{\theta} &= \omega
\end{align}

Parameters (gravity constant $g$, mass $m$, length  $l$ and intertia $J$ of the pendulum):

\begin{align}
g&=10 \, \frac{\text{m}}{\text{s}^2}
&
m&=1 \, \text{kg}
&
l&=1 \, \text{m}
&
J&=\frac{1}{3} m l^2
\end{align}

Forward Euler integration:

\begin{align}
\dot{x}(k T_S) \approx \frac{x[k+1] - x[k]}{T_S}
\end{align}

with sampling time $T_S = 0.05 \, \text{s}$

Reward function:

\begin{align}
r_{k+1} = -(\theta^2[k] + 0.1 \, \text{s}^2 \cdot \omega^2[k] + 0.001 \frac{1}{(\text{N}\text{m})^2} \cdot T_\mathrm{u}^2[k])
\end{align}

Limitations of state and action space:
\begin{align}
\theta &\in [-\pi, \pi]
&
\omega &\in [-8  \, \frac{1}{\text{s}}, 8  \, \frac{1}{\text{s}}]
&
T_\mathrm{u} &\in [-2 \, \text{N}\text{m}, 2 \, \text{N}\text{m}]
\end{align}

And of course input and output space:
\begin{align}
\text{action}&=T_\mathrm{u}
&
\text{state}&=
\begin{bmatrix}
\text{cos}(\theta)\\
\text{sin}(\theta)\\
\omega
\end{bmatrix}
\end{align}

## Solution 2)

YOUR ANSWER HERE

```python

class pendulum_model:
    def __init__(self, dt=0.05, m=1, g=10, l=1):

        # YOUR CODE HERE
        raise NotImplementedError()

    def reset(self, state=None):

        # YOUR CODE HERE
        raise NotImplementedError()

        return state

    def step(self, T_u):

        # YOUR CODE HERE
        raise NotImplementedError()

        return state, reward

    def angle_normalize(self, theta):
        # usage of this helper function is optional
        return (((theta+np.pi) % (2*np.pi)) - np.pi)
```

The following cell is for debugging of the `pendulum_model` class.

```python
env = gym.make('Pendulum-v0')
env = env.unwrapped # removes a builtin time limit of k_T = 200, we want to determine the time limit ourselves

model = pendulum_model()

state = env.reset()
print(state)
m_state = model.reset(state) # model is set to state of env

for _ in range(10000):
    print(f"state: {state}")
    print(f"model: {m_state}")
    print()

    action = env.action_space.sample()

    state, reward, done, _ = env.step(action) # take action on env
    m_state, m_reward = model.step(action) # take the same action on model

    print(f"reward difference: {reward- m_reward}, env_reward:{reward}, model_reward:{m_reward}")

env.close()
```

Using $-\mathrm{sin}(\theta)$ instead of $\mathrm{sin}(\theta +\pi)$ makes no difference when assuming analytical precision, but due to numeric errors these formulations will still yield different results in numpy, mainly because $\pi$ is represented with finite (float) precision. In order to yield the same numbers as in `gym`, we will still make use of the (more cumbersome) $\mathrm{sin}(\theta +\pi)$.

Write a function for the Dyna-Q algorithm, which uses the model we defined above:

```python
def pendulumModelDynaQ(alpha, gamma, epsilon, n, nb_episodes, nb_steps):

    env = gym.make('Pendulum-v0')
    env = env.unwrapped
    model = pendulum_model()

    action_values = np.zeros([d_theta, d_theta, d_omega, d_T])
    pi = np.zeros([d_theta, d_theta, d_omega])

    cumulative_reward_history = [] # we can use this to figure out how well the learning worked

    for j in tqdm(range(nb_episodes), position=0, leave=True):

        # YOUR CODE HERE
        raise NotImplementedError()

    return cumulative_reward_history, pi
```

Use the following cell to compare the learing from experience from 1) to the learning using the defined model: (Beware, nb_steps = 10000 can take some time)

```python
### train both setups once
print("Run with planning from experience")
exp_planning_history, exp_planning_pi = pendulumDynaQ(alpha = 0.1, gamma = 0.9, epsilon = 0.1, n = 19, nb_episodes = 30, nb_steps = 10000)

print("Run with planning from model")
model_planning_history, model_planning_pi = pendulumModelDynaQ(alpha = 0.1, gamma = 0.9, epsilon = 0.1, n = 19, nb_episodes = 30, nb_steps = 10000)

plt.plot(exp_planning_history)
plt.plot(model_planning_history)
plt.xlabel("episode")
plt.ylabel(r"$\sum R$")
plt.show()

```

Use the following cell to execute the policy we got using the model:

```python
experiment(model_planning_pi,nb_steps = 300)
```

### Extra-task:
Change the model parameters (e.g. $g$, $m$, $l$) so that our model differs from the "real world" (we got from gym).
What do you observe

By changing the parameters our model differs from the "real world". Depending on the amount of difference, the learing curve looks worse than the one with the correct values.
The experiment result is also depending on the random starting position.
Depending on the parameter difference, the experiment can not be executed successfully any more.

Try to change the parameters on your own.

YOUR ANSWER HERE

```python
# YOUR CODE HERE
raise NotImplementedError()
```


```python
import numpy as np
from scipy import integrate
from scipy.special import legendre
```

```python
density_conversion = 37.96e-9  # 1 m_sun/kpc^3 = 37.96e-9 GeV/cm^3
r_GC = 8.33  # distance to Galactic Center, kpc
```

```python
def to_cartesian(r, th, ph):
    return np.array([r*np.sin(th)*np.cos(ph),
                     r*np.sin(th)*np.sin(ph),
                     r*np.cos(th)])
```

## Density profiles

```python
def rho_NFW(r, mx=None, sv=None):
    """NFW profile with annihilation plateau cutoff.

    Parameters
    ----------
    r : float
        Distance from GC (kpc).
    """
    r_s = 24.42  # kpc
    rho_s = 0.184  # GeV/cm^3
    t_bh = 1e10*365.25*24*60*60  # s

    def _rho_NFW(r):  # profile without cutoff
        return rho_s * r_s/r * (1. + r/r_s)**-2

    if mx is not None and r_s is not None:
        rho_core = mx / (sv * t_bh)  # annihilation plateau

        return min(_rho_NFW(r), rho_core)
    else:
        return _rho_NFW(r)
```

```python
def rho_bulge(x, y, z, coords="Cartesian"):
    if coords == "Spherical":
        x, y, z = to_cartesian(x, y, z)

    rho0b = density_conversion * 9.6e9
    r0 = 0.5
    r_max = 3.5

    # Ellipsoid orientation
    alpha = 25. * np.pi / 180.

    # Ellipsoid axis sizes
    xb = r_GC/8. * 0.899
    yb = r_GC/8. * 0.386
    zb = r_GC/8. * 0.25

    # Rotate into ellipsoid's coordinate system
    rot_mat = np.array([[np.cos(alpha), -np.sin(alpha), 0.],
                        [np.sin(alpha), np.cos(alpha), 0.],
                        [0., 0., 1.]])
    xp, yp, zp = np.dot(rot_mat, np.array([x, y, z]))
    rp = np.sqrt(xp**2 + yp**2)
    r1 = np.sqrt((xp/xb)**2 + (yp/yb)**2 + (zp/zb)**2)

    # Exponential cutoff outside r_max
    cutoff_fact = 1. if rp < r_max else np.exp(-0.5*(rp - r_max)**2/r0)

    return cutoff_fact * rho0b * np.exp(-r1)
```

```python
def rho_disk(x, y, z, coords="Cartesian"):
    if coords == "Spherical":
        x, y, z = to_cartesian(x, y, z)

    H = 2.75  # kpc
    h1 = r_GC/8. * 0.27  # kpc
    h2 = r_GC/8. * 0.44  # kpc
    beta = 0.565
    rho0d = density_conversion * 4.93e7

    r = np.sqrt(x**2 + y**2)
    eta = max(0.670, 0.114 + r/9.025)

    return rho0d/eta * np.exp(-(r-r_GC)/H) *((1. - beta) / np.cosh(z/(eta*h1))**2 + beta * np.exp(-np.abs(z)/(eta*h2)))
```

```python
def rho_tot(x, y, z, coords="Cartesian"):
    if coords == "Spherical":
        x, y, z = to_cartesian(x, y, z)

    return rho_NFW(np.sqrt(x**2+y**2+z**2)) + rho_disk(x, y, z) + rho_bulge(x, y, z)
```

Plot all the density profiles

```python
rs = np.logspace(-2, 2, 300)

plt.loglog(rs, [rho_NFW(r) for r in rs], label="NFW")
plt.loglog(rs, [rho_disk(r, 0, 0) for r in rs], label="Disk")
plt.loglog(rs, [rho_bulge(r, 0, 0) for r in rs], label="Bulge")
plt.loglog(rs, [rho_tot(r, 0, 0) for r in rs], "k", label="Total")

plt.ylim(1e-3, 1e3)
plt.xlabel("r (kpc)")
plt.ylabel(r"$\rho$ (GeV/cm$^3$)")
plt.legend()
plt.title("Density profiles")
```

## Gravitational potential
We need to find $\partial \Phi / \partial r$ so we can substitute it into the Jeans equation

The DM halo contribution is spherically symmetric part and thus easy to compute. Using Gauss' law,

\begin{align}
    \frac{d\Phi}{dr}(r) = \frac{G M_{\text{enclosed}}(r)}{r^2}.
\end{align}

The enclosed mass is

\begin{align}
    M_{\text{enclosed}}(r) &= \int_0^r dr'\ d\theta' d\phi'\ {r'}^2 \sin\theta' \rho(r')\\
    &= 4\pi \int_0^r dr'\ {r'}^2 \rho(r').
\end{align}

```python
def dPhi_dr_NFW(r):
    m_enc = integrate.quad(lambda rp: rp**2 * rho_NFW(rp), 0, r)[0]
    return m_enc / r**2
```

The disk density profile is azimuthally symmetric, and is invariant under $z \mapsto -z$. It can be treated by performing a multipole expansion:
\begin{align}
    \Phi(r, \theta) &= - G \sum_{l=0}^\infty \left[ \frac{A_l(r)}{r^{l+1}} + B_l(r) r^l \right] P_l(\cos\theta),\\
    A_l(r) &= 2\pi \int_0^r dr'\ \int_0^\pi d\theta'\ \sin\theta' {r'}^{l+2} \rho(r',\theta') P_l(\cos\theta')\\
    B_l(r) &= 2\pi \int_r^\infty dr'\ \int_0^\pi d\theta'\ \sin\theta' \frac{\rho(r',\theta')}{{r'}^{l-1}} P_l(\cos\theta').
\end{align}
The $r$ derivative is
\begin{align}
    \frac{\partial\Phi}{\partial r}(r, \theta) &= - G \sum_{l=0}^\infty \left[ \frac{r A_l'(r) - (l+1) A_l(r)}{r^{l+2}} + \left( r B_l'(r) + l B_l(r) \right) r^{l-1} \right] P_l(\cos\theta),\\
    A_l'(r) &= 2\pi \int_0^\pi d\theta'\ \sin\theta' r^{l+2} \rho(r,\theta') P_l(\cos\theta')\\
    B_l'(r) &= -2\pi \int_0^\pi d\theta'\ \sin\theta' \frac{\rho(r,\theta')}{r^{l-1}} P_l(\cos\theta').
\end{align}

```python
def dPhi_dr_disk_mp_coeff(r, l, r_max=100.):
    def Al_integrand(rp, thp):
        return 2.*np.pi * np.sin(thp) * rp**(l+2) * rho_disk(rp, thp, 0., "Spherical") * legendre(l)(np.cos(thp))

    def Bl_integrand(rp, thp):
        return 2.*np.pi * np.sin(thp) / rp**(l-1) * rho_disk(rp, thp, 0., "Spherical") * legendre(l)(np.cos(thp))

    # Legendre coefficients
    Al = 2. * integrate.dblquad(Al_integrand,
                                0, r,
                                lambda r: 0, lambda r: np.pi/2.)[0]
    # Don't need this if l=0!
    Bl = 0. if l == 0 else 2. * integrate.dblquad(Bl_integrand,
                                                  0, r,
                                                  lambda r: 0, lambda r: np.pi/2.)[0]

    # Radial derivatives of Legendre coefficients
    dAl_dr = 2. * integrate.quad(lambda thp: Al_integrand(r, thp),
                                 0, np.pi/2.)[0]
    dBl_dr = 2. * integrate.quad(lambda thp: -Bl_integrand(r, thp),
                                 0, np.pi/2.)[0]

    # Put it all together
    A_terms = -(r * dAl_dr - (l+1) * Al) / r**(l+2)
    B_terms = -(r * dBl_dr + l * Bl) * r**(l-1)

    return A_terms + B_terms
```

```python
print dPhi_dr_disk_mp_coeff(1., 0)
print dPhi_dr_disk_mp_coeff(1., 1)
print dPhi_dr_disk_mp_coeff(1., 2)
```

    17.6800305169
    -57.5446351891
    -10958.8287826

```python
print dPhi_dr_disk_mp_coeff(1., 3)
print dPhi_dr_disk_mp_coeff(1., 4)
```

    /Users/acoogan/.pyenv/versions/2.7.13/Python.framework/Versions/2.7/lib/python2.7/site-packages/scipy/integrate/quadpack.py:364: IntegrationWarning: The integral is probably divergent, or slowly convergent.
      warnings.warn(msg, IntegrationWarning)

    1968.2948524
    2640.51088841

```python
xs = np.linspace(-1, 1, 200)

for d in [0, 1, 2]:
    plt.plot(xs, legendre(d)(xs), label=d)
plt.legend()
```

```python
def dPhi_dr_disk(r):

```

```python
rs = np.logspace(-2, 2, 300)

plt.loglog(rs, [dPhi_dr_NFW(r) for r in rs], label="NFW")

plt.xlabel("r (kpc)")
plt.ylabel(r"$\partial \Phi / \partial r$")
plt.legend()
plt.title("Potential gradient")
```

```python

```

```python
integrate.tplquad(lambda x, y, z: rho_tot(x, y, z) * np.sqrt(x**2+y**2+z**2),
                  0., 10.,
                  lambda x: 0., lambda x: 10.,
                  lambda x, y: 0., lambda x, y: 10.)
```

```python

```


```
import numpy as np
import scipy as sp
import sympy as sy
from scipy.misc import imread, imsave
from matplotlib import pyplot as plt
from matplotlib import cm
%load_ext cythonmagic
```

```
#for nummexpr versions, if you would like to use them
import numexpr as ne
```

```cython
%%cython
#solution to problem 1
cimport numpy as np
import numpy as np
from libc.math cimport fabs, exp
cimport cython

cdef double g(double x, double sigma):
    #The absolute value isn't necessary here
    return exp(-(x/sigma)**2)

@cython.boundscheck(False)
@cython.wraparound(False)
def perona_malik3(np.ndarray[dtype=double,ndim=2] u, double lbda, double sigma, int reps):
    cdef int n, i, j, h=u.shape[0], w=u.shape[1]
    cdef double temp, d1, d2, d3, d4
    cdef np.ndarray[dtype=double,ndim=1] trow = np.empty(w)
    for n in xrange(reps):
        trow[:] = u[0] #python slicing, possibly slow. Maybe do it with a memoryview??
        for i in xrange(1, h-1):
            temp = u[i,0]
            for j in xrange(1, w-1):
                d1 = trow[j] - u[i,j]
                d2 = u[i+1,j] - u[i,j]
                d3 = trow[j-1] - u[i,j]
                d4 = u[i,j+1] - u[i,j]
                trow[j] = u[i,j]
                u[i,j] = u[i,j] + lbda*(g(d1,sigma)*d1 + g(d2,sigma)*d2 + g(d3,sigma)*d3 + g(d4,sigma)*d4)
```

```
lbda = .25
sigma = .01
iters = 50
image = np.array(imread("test_photo.jpg", flatten=True), dtype=np.float64)
image /= np.max(image)
%time perona_malik3(image, lbda, sigma, iters)
plt.imshow(image, cmap=cm.Greys_r)
plt.show()
```

    CPU times: user 8.57 s, sys: 0.00 s, total: 8.57 s
    Wall time: 8.57 s

```cython
%%cython
#solution to problem 2
cimport numpy as np
import numpy as np
from libc.math cimport fabs, exp
cimport cython

cdef inline double g(double x, double sv):
    #in this case the absolute value is not necessary.
    return exp(-(x*sv)**2)

@cython.boundscheck(False)
@cython.wraparound(False)
def perona_malik9(np.ndarray[dtype=double,ndim=2] u, double lb, double sigma, int reps):
    cdef int n, i, j, h=u.shape[0], w=u.shape[1]
    cdef double d1, d2, d3, d4, sv=1./sigma
    cdef np.ndarray[dtype=double,ndim=1] trow = np.empty(w)
    for n in xrange(reps):
        #do the first row
        #do the first item in the first row
        d1 = u[1,0] - u[0,0]
        d2 = u[0,1] - u[0,0]
        trow[0] = u[0,0]
        u[0,0] = u[0,0] + lb*(g(d1,sv)*d1 + g(d2,sv)*d2)
        #do the middle entries of the first row
        for j in xrange(1, w-1):
            d1 = u[i+1,j] - u[i,j]
            d2 = trow[j-1] - u[i,j]
            d3 = u[i,j+1] - u[i,j]
            trow[j] = u[i,j]
            u[i,j] = u[i,j] + lb*(g(d1,sv)*d1 + g(d2,sv)*d2 + g(d3,sv)*d3)
        #do the last entry in the first row
        d1 = u[1,w-1] - u[0,w-1]
        d2 = trow[w-2] - u[0,w-1]
        trow[w-1] = u[0,w-1]
        u[0,w-1] = u[0,w-1] + lb*(g(d1,sv)*d1 + g(d2,sv)*d2)
        #do the middle rows
        for i in xrange(1, h-1):
            #do the first entry of the current row
            d1 = trow[0] - u[i,0]
            d2 = u[i+1,0] - u[i,0]
            d3 = u[i,1] - u[i,0]
            trow[0] = u[i,0]
            u[i,0] = u[i,0] + lb*(g(d1,sv)*d1 + g(d2,sv)*d2 + g(d3,sv)*d3)
            #do the middle entries of the current row
            for j in xrange(1, w-1):
                d1 = trow[j] - u[i,j]
                d2 = u[i+1,j] - u[i,j]
                d3 = trow[j-1] - u[i,j]
                d4 = u[i,j+1] - u[i,j]
                trow[j] = u[i,j]
                u[i,j] = u[i,j] + lb*(g(d1,sv)*d1 + g(d2,sv)*d2 + g(d3,sv)*d3 + g(d4,sv)*d4)
            #do the last entry of the current row
            d1 = trow[w-1] - u[i,w-1]
            d2 = u[i+1,w-1] - u[i,w-1]
            d3 = trow[w-2] - u[i,w-1]
            trow[w-1] = u[i,w-1]
            u[i,w-1] = u[i,w-1] + lb*(g(d1,sv)*d1 + g(d2,sv)*d2 + g(d3,sv)*d3)
        #do the first entry of the last row
        d1 = trow[0] - u[h-1,0]
        d2 = u[h-1,1] - u[h-1,0]
        trow[0] = u[h-1,0]
        u[h-1,0] = u[h-1,0] + lb*(g(d1,sv)*d1 + g(d2,sv)*d2)
        #do the middle entries of the last row
        for j in xrange(1, w-1):
            d1 = trow[j] - u[h-1,j]
            d2 = trow[j-1] - u[h-1,j]
            d3 = u[h-1,j+1] - u[h-1,j]
            trow[j] = u[h-1,j]
            u[h-1,j] = u[h-1,j] + lb*(g(d1,sv)*d1 + g(d2,sv)*d2 + g(d3,sv)*d3)
        #do the last entry of the last row
        d1 = trow[w-1] - u[h-1,w-1]
        d2 = trow[w-2] - u[h-1,w-1]
        #we don't actually need to store the last temp value.
        u[h-1,w-1] += lb*(g(d1,sv)*d1 + g(d2,sv)*d2)
```

```
lbda = .25
sigma = .01
iters = 50
image = np.array(imread("test_photo.jpg", flatten=True), dtype=np.float64)
image /= np.max(image)
%time perona_malik9(image, lbda, sigma, iters)
plt.imshow(image, cmap=cm.Greys_r)
plt.show()
```

    CPU times: user 8.46 s, sys: 0.00 s, total: 8.46 s
    Wall time: 8.46 s

```
#problem 2
#vanilla numpy version
#based on diferent views of the same data which
#are made by slicing the arrays involved.
#It is less straightforward, but much shorter.
#This one could probably be better optimized by
#pre-allocation of all needed temporary arrays.
def perona_malik11(u, lbda, sigma, reps):
    unew = u.copy()
    #make some views of the same data as u
    uN = u[:-1]
    uS = u[1:]
    uW = u[:,:-1]
    uE = u[:,1:]
    #make similar views for unew
    uNnew = unew[:-1]
    uSnew = unew[1:]
    uWnew = unew[:,:-1]
    uEnew = unew[:,1:]
    temp = np.empty_like(u)
    vtemp = temp[:-1]
    htemp = temp[:,:-1]
    difs = np.empty_like(u)
    vdifs = difs[:-1]
    hdifs = difs[:,:-1]
    for i in xrange(reps):
        vdifs[:] = uS - uN
        vtemp[:] = vdifs / sigma
        np.square(vtemp, out=vtemp)
        vtemp *= -1
        np.exp(vtemp, out=vtemp)
        vtemp *= lbda
        vtemp *= vdifs
        uNnew += vtemp
        uSnew -= vtemp
        hdifs[:] = uE - uW
        htemp[:] = hdifs / sigma
        np.square(htemp, out=htemp)
        htemp *= -1
        np.exp(htemp, out=htemp)
        htemp *= lbda
        htemp *= hdifs
        uWnew += htemp
        uEnew -= htemp
        u[:] = unew
```

```
lbda = .25
sigma = .01
iters = 50
image = np.array(imread("test_photo.jpg", flatten=True), dtype=np.float32)
image /= np.max(image)
%time perona_malik11(image, lbda, sigma, iters)
plt.imshow(image, cmap=cm.Greys_r)
plt.show()
```

    CPU times: user 5.54 s, sys: 0.00 s, total: 5.54 s
    Wall time: 5.52 s

```
#numexpr-ized problem 2
#this one is blazing fast
def perona_malik12(u, lbda, sigma, reps):
    #expects array of 32 bit floats
    sinv = np.float32(1./sigma)
    lbda32 = np.float32(lbda)
    unew = u.copy()
    #make some views of the same data as u
    uN = u[:-1]
    uS = u[1:]
    uW = u[:,:-1]
    uE = u[:,1:]
    #make similar views for unew
    uNnew = unew[:-1]
    uSnew = unew[1:]
    uWnew = unew[:,:-1]
    uEnew = unew[:,1:]
    temp = np.empty_like(u)
    vtemp = temp[:-1]
    htemp = temp[:,:-1]
    for i in xrange(reps):
        ne.evaluate('lbda32*exp(-(sinv*(uS-uN))**2)*(uS-uN)', out=vtemp)
        ne.evaluate('uNnew+vtemp', out=uNnew)
        ne.evaluate('uSnew-vtemp', out=uSnew)
        ne.evaluate('lbda32*exp(-(sinv*(uE-uW))**2)*(uE-uW)', out=htemp)
        ne.evaluate('uWnew+htemp', out=uWnew)
        ne.evaluate('uEnew-htemp', out=uEnew)
        u[:] = unew
```

```
lbda = .25
sigma = .01
iters = 50
image = np.array(imread("test_photo.jpg", flatten=True), dtype=np.float32)
image /= np.max(image)
%time perona_malik12(image, lbda, sigma, iters)
plt.imshow(image, cmap=cm.Greys_r)
plt.show()
```

    CPU times: user 1.65 s, sys: 0.00 s, total: 1.65 s
    Wall time: 1.63 s

```cython
%%cython
#solution to problem 3
cimport numpy as np
import numpy as np
from libc.math cimport fabs, exp, fmax
cimport cython

cdef inline double g(double x, double sv):
    #absolute value isn't necessary here.
    #sv is 1./sigma
    return exp(-(x*sv)**2)

@cython.boundscheck(False)
@cython.wraparound(False)
cdef inline double nm(double[:] x, double[:] y):
    #this is the norm used to compare pixels
    #currently using the infinity norm
    return fmax(fmax(x[0]-y[0], x[1]-y[1]), x[2]-y[2])

@cython.boundscheck(False)
@cython.wraparound(False)
def perona_malik_color2(np.ndarray[dtype=double,ndim=3] U, double lb, double sigma, int reps):
    #this one isn't very readable, but works the same as the black and white version except
    #that instead of a single value for a pixel, now we have three to deal with 3 dimensional vectors
    #we use the norm function defined above for the computation of g at each point.
    cdef int n, i, j, k, h=U.shape[0], w=U.shape[1]
    cdef double g1, g2, g3, g4, d1, d2, d3, d4, cst=4./3., sv=1./sigma, temp
    #g1 thru g4 are temporary variables for the coefficients
    #d1 thru d4 are temporary variables for the differences between pixels
    #strictly speaking, d1 thru d4 are not needed and may actually slow things down
    #we have added them here for readability
    cdef double[:,:,:] u = U
    #here we use memory views so we can get the performance boost
    #from using inlined functions with the memoryview slices.
    cdef double[:,:] trow = np.empty((w,3))
    for n in xrange(reps):
        #process first pixel of first row
        g1 = g(nm(u[1,0], u[0,0]), sv)
        g2 = g(nm(u[0,1], u[0,0]), sv)
        for k in xrange(3):
            trow[0,k] = u[0,0,k]
            d1 = u[1,0,k] - u[0,0,k]
            d2 = u[0,1,k] - u[0,0,k]
            u[0,0,k] += lb * (g1*d1 + g2*d2)
        #there may be a bug in this for loop, but I don't see it...
        #the reason is that there appears to be some darkening at the top of the image.
        for j in xrange(1, w-1):
            #process middle pixels of the first row
            g1 = g(nm(u[1,j], u[0,j]), sv)
            g2 = g(nm(trow[j-1], u[0,j]), sv)
            g3 = g(nm(u[0,j+1], u[0,j]), sv)
            for k in xrange(3):
                trow[j,k] = u[0,j,k]
                d1 = u[1,j,k] - u[0,j,k]
                d2 = trow[j-1,k] - u[0,j,k]
                d3 = u[0,j+1,k] - u[0,j,k]
                u[0,j,k] += lb * (g1*d1 + g2*d2 + g3*d3)
        #last pixel of first row
        g1 = g(nm(u[1,w-1], u[0,w-1]), sv)
        g2 = g(nm(trow[w-2], u[0,w-1]), sv)
        for k in xrange(3):
            trow[w-1,k] = u[0,w-1,k]
            d1 = u[1,w-1,k] - u[0,w-1,k]
            d2 = trow[w-2,k] - u[0,w-1,k]
            u[0,w-1,k] = u[0,w-1,k] + lb * (g1*d1 + g2*d2)
        for i in xrange(1, h-1):
            #processing middle rows
            #first pixel of each row
            g1 = g(nm(trow[0], u[i,0]), sv)
            g2 = g(nm(u[i+1,0], u[i,0]), sv)
            g3 = g(nm(u[i,1], u[i,0]), sv)
            for k in xrange(3):
                d1 = trow[0,k] - u[i,0,k]
                d2 = u[i+1,0,k] - u[i,0,k]
                d3 = u[i,1,k] - u[i,0,k]
                trow[0,k] = u[i,0,k]
                u[i,0,k] += lb * (g1*d1 + g2*d2 + g3*d3)
            for j in xrange(1, w-1):
                #middle pixels of each row
                g1 = g(nm(trow[j], u[i,j]), sv)
                g2 = g(nm(u[i+1,j], u[i,j]), sv)
                g3 = g(nm(trow[j-1], u[i,j]), sv)
                g4 = g(nm(u[i,j+1], u[i,j]), sv)
                for k in xrange(3):
                    d1 = trow[j,k] - u[i,j,k]
                    d2 = u[i+1,j,k] - u[i,j,k]
                    d3 = trow[j-1,k] - u[i,j,k]
                    d4 = u[i,j+1,k] - u[i,j,k]
                    trow[j,k] = u[i,j,k]
                    u[i,j,k] += lb * (g1*d1 + g2*d2 + g3*d3 + g4*d4)
            #last pixel of each row
            g1 = g(nm(trow[w-1], u[i,w-1]), sv)
            g2 = g(nm(u[i+1,w-1], u[i,w-1]), sv)
            g3 = g(nm(trow[w-2], u[i,w-1]), sv)
            for k in xrange(3):
                d1 = trow[w-1,k] - u[i,w-1,k]
                d2 = u[i+1,w-1,k] - u[i,w-1,k]
                d3 = trow[w-2,k] - u[i,w-1,k]
                trow[w-1,k] = u[i,w-1,k]
                u[i,w-1,k] += lb * (g1*d1 + g2*d2 + g3*d3)
        #first pixel of last row
        g1 = g(nm(trow[0], u[h-1,0]), sv)
        g2 = g(nm(u[h-1,1], u[h-1,0]), sv)
        for k in xrange(3):
            d1 = trow[0,k] - u[h-1,0,k]
            d2 = u[h-1,1,k] - u[h-1,0,k]
            trow[0,k] = u[h-1,0,k]
            u[h-1,0,k] += lb * (g1*d1 + g2*d2)
        for j in xrange(1, w-1):
            #middle pixels of last row
            g1 = g(nm(trow[j], u[h-1,j]), sv)
            g2 = g(nm(trow[j-1], u[h-1,j]), sv)
            g3 = g(nm(u[h-1,j+1], u[h-1,j]), sv)
            for k in xrange(3):
                d1 = trow[j,k] - u[h-1,j,k]
                d2 = trow[j-1,k] - u[h-1,j,k]
                d3 = u[h-1,j+1,k] - u[h-1,j,k]
                trow[j,k] = u[h-1,j,k]
                u[h-1,j,k] += lb * (g1*d1 + g2*d2 + g3*d3)
        #last pixel of last row
        g1 = g(nm(trow[w-1], u[h-1,w-1]), sv)
        g2 = g(nm(trow[w-2], u[h-1,w-1]), sv)
        for k in xrange(3):
            d1 = trow[w-1,k] - u[h-1,w-1,k]
            d2 = trow[w-2,k] - u[h-1,w-1,k]
            #we don't actually need to store the last set of temp values
            u[h-1,w-1,k] += lb * (g1*d1 + g2*d2)
```

```
lbda = .25
sigma = .01
iters = 50
image = np.array(imread("test_photo.jpg"), dtype=np.float64)
image /= np.max(image)
%time perona_malik_color2(image, lbda, sigma, iters)
plt.imshow(image)
plt.show()
```

    CPU times: user 29.74 s, sys: 0.00 s, total: 29.74 s
    Wall time: 29.73 s

```
#problem 3
#vanilla numpy version based on array views
def perona_malik_color20(u, lbda, sigma, iters):
    unew = u.copy()
    #transposes are necessary so that broadcasting can work later.
    uNT = u[:-1].T
    uST = u[1:].T
    uWT = u[:,:-1].T
    uET = u[:,1:].T
    uNnewT = unew[:-1].T
    uSnewT = unew[1:].T
    uWnewT = unew[:,:-1].T
    uEnewT = unew[:,1:].T
    vdifs = np.empty_like(uNT[0])
    hdifs = np.empty_like(uWT[0])
    temp = np.empty_like(u)
    vtemp = temp[:-1].T
    htemp = temp[:,:-1].T
    temp2 = np.empty_like(temp)
    vtemp2 = temp2[:-1].T
    htemp2 = temp2[:,:-1].T
    for n in xrange(iters):
        #I lined the computations up in this way so as to
        #avoid unnecessary memory allocation and deallocation
        #as we go through the for loop.
        vtemp[:] = uST - uNT
        np.fabs(vtemp, out=vtemp2)
        np.max(vtemp2, axis=0, out=vdifs)
        vdifs /= sigma
        np.square(vdifs, out=vdifs)
        vdifs *= -1
        np.exp(vdifs, out=vdifs)
        vdifs *= lbda
        vtemp *= vdifs    #broadcasting across color dimension
        uNnewT += vtemp
        uSnewT -= vtemp
        htemp[:] = uET-uWT
        np.fabs(htemp, out=htemp2)
        np.max(htemp2, axis=0, out=hdifs)
        hdifs /= sigma
        np.square(hdifs, out=hdifs)
        hdifs *= -1
        np.exp(hdifs, out=hdifs)
        hdifs *= lbda
        htemp *= hdifs    #broadcasting across color dimension
        uWnewT += htemp
        uEnewT -= htemp
        u[:] = unew
```

```
lbda = .25
sigma = .01
iters = 50
image = np.array(imread("test_photo.jpg"), dtype=np.float32)
image /= np.max(image)
%time perona_malik_color20(image, lbda, sigma, iters)
plt.imshow(image)
plt.show()
```

    CPU times: user 16.24 s, sys: 0.00 s, total: 16.24 s
    Wall time: 16.24 s

```
#problem 3
#using numexpr and array slices
def perona_malik_color13(u, lbda, sigma, iters):
    sinv = np.float32(1./sigma)
    lbda32 = np.float32(lbda)
    unew = u.copy()
    uNT = u[:-1].T
    uST = u[1:].T
    uWT = u[:,:-1].T
    uET = u[:,1:].T
    uNnewT = unew[:-1].T
    uSnewT = unew[1:].T
    uWnewT = unew[:,:-1].T
    uEnewT = unew[:,1:].T
    vdifs = np.empty_like(uNT[0])
    hdifs = np.empty_like(uWT[0])
    temp = np.empty_like(u)
    vtemp = temp[:-1].T
    htemp = temp[:,:-1].T
    temp2 = np.empty_like(temp)
    vtemp2 = temp2[:-1].T
    htemp2 = temp2[:,:-1].T
    for n in xrange(iters):
        ne.evaluate('uST-uNT', out=vtemp)
        ne.evaluate('abs(vtemp)', out=vtemp2)
        np.max(vtemp2, axis=0, out=vdifs)
        ne.evaluate('lbda32*exp(-(sinv*(vdifs))**2)', out=vdifs)
        vtemp *= vdifs #broadcasting
        ne.evaluate('uNnewT+vtemp', out=uNnewT)
        ne.evaluate('uSnewT-vtemp', out=uSnewT)
        ne.evaluate('uET-uWT', out=htemp)
        ne.evaluate('abs(htemp)', out=htemp2)
        np.max(htemp2, axis=0, out=hdifs)
        ne.evaluate('lbda32*exp(-(sinv*(hdifs))**2)', out=hdifs)
        htemp *= hdifs #broadcasting
        ne.evaluate('uWnewT+htemp', out=uWnewT)
        ne.evaluate('uEnewT-htemp', out=uEnewT)
        u[:] = unew
```

```
lbda = .25
sigma = .01
iters = 50
image = np.array(imread("test_photo.jpg"), dtype=np.float32)
image /= np.max(image)
%time perona_malik_color13(image, lbda, sigma, iters)
plt.imshow(image)
plt.show()
```

    CPU times: user 11.10 s, sys: 0.00 s, total: 11.10 s
    Wall time: 11.10 s

```cython
%%cython
#this is a function needed in my solution of problem 4
import numpy as np
cimport numpy as np
cimport cython
from libc.math cimport fmin, fmax, sqrt, fabs

float32 = np.float32
ctypedef np.float32_t float32_T

@cython.boundscheck(False)
@cython.wraparound(False)
cdef inline bimin(float32_T[:] arr, int size):
    #changes the first two items of an array to be the
    #two smallest items in the array
    cdef int i
    cdef float32_T min1, min2
    if arr[0] < arr[1]:
        min1 = arr[0]
        min2 = arr[1]
    else:
        min2 = arr[0]
        min1 = arr[1]
    for i in xrange(2, size):
        if arr[i] < min1:
            min1 = arr[i]
            min2 = min1
        elif arr[1] < min2:
            min2 = arr[i]
    arr[0]=min1
    arr[1]=min2

@cython.boundscheck(False)
@cython.wraparound(False)
cpdef makecoefs(np.ndarray[dtype=float32_T,ndim=2] vdifs,
    np.ndarray[dtype=float32_T,ndim=2] hdifs,
    np.ndarray[dtype=float32_T,ndim=2] ldifs,
    np.ndarray[dtype=float32_T,ndim=2] udifs,
    np.ndarray[dtype=float32_T,ndim=2] coefs):
    #This one isn't particularly well optimized
    #it *shouldn't* be too terrible though.
    #the operations are in place to fill the coeffs array
    #I don't know of any nice way of doing this, so heres a tractable way.
    cdef int i, j, k, h=coefs.shape[0], w=coefs.shape[1]
    cdef float32_T[:] temp = np.empty(8,dtype=np.float32)
    #do first point
    temp[0] = vdifs[0,0]
    temp[1] = ldifs[0,0]
    temp[2] = hdifs[0,0]
    bimin(temp, 3)
    coefs[0,0] = .5 * (temp[0]+temp[1])
    for j in xrange(1, w-1):
        temp[0] = hdifs[0,j-1]
        temp[1] = hdifs[0,j]
        temp[2] = udifs[0,j-1]
        temp[3] = vdifs[0,j]
        temp[4] = ldifs[0,j]
        bimin(temp, 5)
        coefs[0,j] = .5 * (temp[0]+temp[1])
    temp[0] = hdifs[0,w-2]
    temp[1] = udifs[0,w-2]
    temp[2] = vdifs[0,w-1]
    bimin(temp, 3)
    coefs[0,0] = .5 * (temp[0]+temp[1])
    for i in xrange(1, h-1):
        temp[0] = vdifs[i-1,0]
        temp[1] = vdifs[i,0]
        temp[2] = udifs[i-1,0]
        temp[3] = hdifs[i,0]
        temp[4] = ldifs[i,0]
        bimin(temp, 5)
        coefs[i,0] = .5 * (temp[0]+temp[1])
        for j in xrange(1, w-1):
            temp[0] = ldifs[i-1,j-1]
            temp[1] = ldifs[i,j]
            temp[2] = vdifs[i-1,j]
            temp[3] = vdifs[i,j]
            temp[4] = udifs[i-1,j]
            temp[5] = udifs[i,j-1]
            temp[6] = hdifs[i,j-1]
            temp[7] = hdifs[i,j]
            bimin(temp, 8)
            coefs[i,j] = .5 * (temp[0]+temp[1])
        temp[0] = vdifs[i-1,w-1]
        temp[1] = vdifs[i,w-1]
        temp[2] = ldifs[i-1,w-2]
        temp[3] = hdifs[i,w-2]
        temp[4] = udifs[i,w-2]
        bimin(temp, 6)
        coefs[i,w-1] = .5 * (temp[0]+temp[1])
    temp[0] = vdifs[h-2,0]
    temp[1] = udifs[h-2,0]
    temp[2] = hdifs[h-1,0]
    bimin(temp, 3)
    coefs[h-1,0] = .5 * (temp[0]+temp[1])
    for j in xrange(1, w-1):
        temp[0] = hdifs[h-1,j-1]
        temp[1] = hdifs[h-1,j]
        temp[2] = ldifs[h-2,j-1]
        temp[3] = vdifs[h-2,j]
        temp[4] = udifs[h-2,j]
        bimin(temp, 5)
        coefs[h-1,j] = .5 * (temp[0]+temp[1])
    temp[0] = hdifs[h-1,w-2]
    temp[1] = ldifs[h-2,w-2]
    temp[2] = vdifs[h-2,w-1]
    bimin(temp, 3)
    coefs[h-1,w-1] = .5 * (temp[0]+temp[1])
```

```
#vanilla numpy version for problem 4
def min_bias_color2(u, lbda, reps):
    #this uses a good bit of excess memory...
    #the approach is based on the approach I took with the
    #optimized versions of problems 2 and 3 below
    difs = np.empty_like(u)
    uN = u[:-1]
    uS = u[1:]
    uW = u[:,:-1]
    uE = u[:,1:]
    uNW = u[:-1,:-1]
    uNE = u[:-1,1:]
    uSW = u[1:,:-1]
    uSE = u[1:,1:]
    Ndifs = difs[:-1]
    Sdifs = difs[1:]
    Wdifs = difs[:,:-1]
    Edifs = difs[:,1:]
    temp = np.empty_like(u)
    #coefs is used to store the coefficients that we use
    #to control how much diffusion we allow at each pixel
    coefs = np.empty_like(temp[:,:,0])
    vtemp = temp[:-1]
    htemp = temp[:,:-1]
    dtemp = temp[:-1,:-1]
    vdifs = np.empty_like(vtemp[:,:,0])
    hdifs = np.empty_like(htemp[:,:,0])
    ldifs = np.empty_like(dtemp[:,:,0])
    udifs = np.empty_like(dtemp[:,:,0])
    lastrow = difs[-1]
    difsT = difs.T
    coefsT = coefs.T
    for i in xrange(reps):
        #Calculate the difference between the old and new,
        #then add in place.
        vtemp[:] = uN-uS
        np.fabs(vtemp, out=vtemp)
        np.max(vtemp, axis=2, out=vdifs)
        htemp[:] = uW-uE
        np.fabs(htemp, out=htemp)
        np.max(htemp, axis=2, out=hdifs)
        dtemp[:] = uNW-uSE
        np.fabs(dtemp, out=dtemp)
        np.max(dtemp, axis=2, out=ldifs)
        dtemp[:] = uSW-uNE
        np.fabs(dtemp, out=dtemp)
        np.max(dtemp, axis=2, out=udifs)
        makecoefs(vdifs, hdifs, ldifs, udifs, coefs)
        coefs *= lbda
        #overwrite most of difs instead of reinitializing
        Ndifs[:] = uS-uN
        #initialize what's left
        lastrow[:] = 0.
        Sdifs += uN
        Sdifs -= uS
        Wdifs += uE
        Wdifs -= uW
        Edifs += uW
        Edifs -= uE
        difsT *= coefsT#broadcasting
        u += difs
```

```
lbda = .25
iters = 50
image = np.array(imread("test_photo.jpg"), dtype=np.float32)
image /= np.max(image)
%time min_bias_color2(image, lbda, iters)
plt.imshow(image)
plt.show()
```

    CPU times: user 24.95 s, sys: 0.00 s, total: 24.95 s
    Wall time: 24.95 s

```
#numexpr optimized version of problem 4
import numexpr as ne
def min_bias_color(u, lbda, reps):
    #this uses a good bit of excess memory...
    #the approach is based on the approach I took with the
    #optimized versions of problems 2 and 3 below
    difs = np.empty_like(u)
    lb32 = np.float32(lbda)
    uN = u[:-1]
    uS = u[1:]
    uW = u[:,:-1]
    uE = u[:,1:]
    uNW = u[:-1,:-1]
    uNE = u[:-1,1:]
    uSW = u[1:,:-1]
    uSE = u[1:,1:]
    Ndifs = difs[:-1]
    Sdifs = difs[1:]
    Wdifs = difs[:,:-1]
    Edifs = difs[:,1:]
    temp = np.empty_like(u)
    coefs = np.empty_like(temp[:,:,0])
    #coefs is used to store the coefficients that we use
    #to control how much diffusion we allow at each pixel
    vtemp = temp[:-1]
    htemp = temp[:,:-1]
    dtemp = temp[:-1,:-1]
    vdifs = np.empty_like(vtemp[:,:,0])
    hdifs = np.empty_like(htemp[:,:,0])
    ldifs = np.empty_like(dtemp[:,:,0])
    udifs = np.empty_like(dtemp[:,:,0])
    lastrow = difs[-1]
    difsT = difs.T
    coefsT = coefs.T
    for i in xrange(reps):
        #Calculate the difference between the old and new,
        #then add in place.
        ne.evaluate('abs(uN-uS)', out=vtemp)
        np.max(vtemp, axis=2, out=vdifs)
        ne.evaluate('abs(uW-uE)', out=htemp)
        np.max(htemp, axis=2, out=hdifs)
        ne.evaluate('abs(uNW-uSE)', out=dtemp)
        np.max(dtemp, axis=2, out=ldifs)
        ne.evaluate('abs(uSW-uNE)', out=dtemp)
        np.max(dtemp, axis=2, out=udifs)
        makecoefs(vdifs, hdifs, ldifs, udifs, coefs)
        ne.evaluate('lb32*coefs', out=coefs)
        #overwrite most of difs instead of reinitializing
        ne.evaluate('uS-uN', out=Ndifs)
        #initialize what's left
        lastrow[:] = 0.
        ne.evaluate('Sdifs+uN-uS', out=Sdifs)
        ne.evaluate('Wdifs+uE-uW', out=Wdifs)
        ne.evaluate('Edifs+uW-uE', out=Edifs)
        difsT *= coefsT#broadcasting
        difs *= lb32
        ne.evaluate('u+difs', out=u)
```

```
lbda = .25
iters = 50
image = np.array(imread("test_photo.jpg"), dtype=np.float32)
image /= np.max(image)
%time min_bias_color(image, lbda, iters)
plt.imshow(image)
plt.show()
```

    CPU times: user 16.18 s, sys: 0.00 s, total: 16.18 s
    Wall time: 16.18 s

```
#here are two versions using Theano
#currently, importing Theano breaks the %%cython magic
#so evaluate the above cells first
import theano as ta
from theano import tensor as te

#problem 2
#preliminaries
a = te.fmatrix('a')
b = te.fmatrix('b')
difference = a - b
dif = ta.function([a,b], difference)
lbda_scl = te.fscalar('lbda_s')
sinv_scl = te.fscalar('sinv_s')
operator14 = lbda_scl * te.exp(-((a-b)*sinv_scl)**2) * (a-b)
op14 = ta.function([a,b,lbda_scl,sinv_scl], operator14)

#here's the actual function
def perona_malik14(u, lbda, sigma, reps):
    unew = u.copy()
    sinv = np.float32(1./sigma)
    lb32 = np.float32(lbda)
    #make some views of the same data as u
    uN = u[:-1]
    uS = u[1:]
    uW = u[:,:-1]
    uE = u[:,1:]
    #make similar views for unew
    uNnew = unew[:-1]
    uSnew = unew[1:]
    uWnew = unew[:,:-1]
    uEnew = unew[:,1:]
    temp = np.empty_like(u)
    vtemp = temp[:-1]
    htemp = temp[:,:-1]
    for i in xrange(reps):
        vtemp[:] = op14(uS, uN, lb32, sinv)
        uNnew += vtemp
        uSnew -= vtemp
        htemp[:] = op14(uE, uW, lb32, sinv)
        uWnew += htemp
        uEnew -= htemp
        u[:] = unew

#problem 3
#preliminaries
fmatrix3 = te.TensorType('float32',(False,)*3)
c = fmatrix3('c')
d = fmatrix3('d')
lbda_scl = te.fscalar('lbda_scl')
sinv_scl = te.fscalar('sinv_scl')
difop = c - d
dif = ta.function([c,d], difop)
operator16 = lbda_scl * te.exp(-(a*sinv_scl)**2)
op16 = ta.function([a,lbda_scl,sinv_scl], operator16)

#now here's the function itself
def perona_malik_color16(u, lbda, sigma, reps):
    lb32 = np.float32(lbda)
    sinv = np.float32(1./sigma)
    unew = u.copy()
    uNT = u[:-1].T
    uST = u[1:].T
    uWT = u[:,:-1].T
    uET = u[:,1:].T
    uNTnew = unew[:-1].T
    uSTnew = unew[1:].T
    uWTnew = unew[:,:-1].T
    uETnew = unew[:,1:].T
    temp = np.empty_like(u)
    vtemp = temp[:-1].T
    htemp = temp[:,:-1].T
    temp2 = np.empty_like(temp)
    vtemp2 = temp2[:-1].T
    htemp2 = temp2[:,:-1].T
    cfs = np.empty(u.shape[:-1],dtype=u.dtype)
    vcfs = cfs[:-1].T
    hcfs = cfs[:,:-1].T
    for i in xrange(reps):
        vtemp[:] = dif(uST, uNT)
        np.fabs(vtemp, out=vtemp2)
        np.max(vtemp2, axis=0, out=vcfs)
        vcfs[:] = op16(vcfs, lb32, sinv)
        vtemp *= vcfs
        uNTnew += vtemp
        uSTnew -= vtemp
        htemp[:] = dif(uET, uWT)
        np.fabs(htemp, out=htemp2)
        np.max(htemp2, axis=0, out=hcfs)
        hcfs[:] = op16(hcfs, lb32, sinv)
        htemp *= hcfs
        uWTnew += htemp
        uETnew -= htemp
        u[:] = unew
```

```
lbda = .25
sigma = .01
iters = 50
image = np.array(imread("test_photo.jpg", flatten=True), dtype=np.float32)
image /= np.max(image)
%time perona_malik14(image, lbda, sigma, iters)
plt.imshow(image, cmap=cm.Greys_r)
plt.show()
```

    CPU times: user 5.57 s, sys: 0.00 s, total: 5.57 s
    Wall time: 5.57 s

```
#meh, mediocre performance for color
lbda = .25
sigma = .01
iters = 50
image = np.array(imread("test_photo.jpg"), dtype=np.float32)
image /= np.max(image)
%time perona_malik_color16(image, lbda, sigma, iters)
plt.imshow(image)
plt.show()
```

    CPU times: user 23.18 s, sys: 0.00 s, total: 23.18 s
    Wall time: 23.18 s

```
#here we generate the images in the lab
lbda = .2
sigma = .7
iters = 5
image = np.array(imread("baloon_resized_color.png", flatten=True), dtype=np.float32)
#strictly speaking, the max may not be the right way to do this, but it is good enough
image /= np.max(image)
imsave("baloon_resized_bw.png", image)
%time perona_malik11(image, lbda, sigma, iters)
imsave("baloon_resized_bw_5.png", image)
more_iters = 15
%time perona_malik11(image, lbda, sigma, more_iters)
imsave("baloon_resized_bw_20.png", image)
more_iters = 80
%time perona_malik11(image, lbda, sigma, more_iters)
imsave("baloon_resized_bw_100.png", image)
plt.imshow(image, cmap=cm.Greys_r)
plt.show()
```

```
lbda = .2
sigma = .7
iters = 5
image = np.array(imread("baloon_resized_color.png"), dtype=np.float32)
image /= np.max(image)
%time perona_malik_color20(image, lbda, sigma, iters)
imsave("baloon_resized_color_5.png", image)
more_iters = 15
%time perona_malik_color20(image, lbda, sigma, more_iters)
imsave("baloon_resized_color_20.png", image)
more_iters = 80
%time perona_malik_color20(image, lbda, sigma, more_iters)
imsave("baloon_resized_color_100.png", image)
plt.imshow(image)
plt.show()
```

```
lbda = .2
sigma = .1
iters = 50
image = np.array(imread("baloons_resized_color.png", flatten=True), dtype=np.float32)
image /= np.max(image)
imsave("baloons_resized_bw.png", image)
%time perona_malik11(image, lbda, sigma, iters)
imsave("baloons_resized_bw_50.png", image)
plt.imshow(image, cmap=cm.Greys_r)
plt.show()
```

```
n = 100000
lbda = .25
sigma = .04
iters = 300
image = np.array(imread("baloons_resized_color.png"), dtype=np.float32)
image /= np.max(image)
indices = sp.rand(n, 3)
indices[:,0] *= image.shape[0]
indices[:,1] *= image.shape[1]
indices[:,2] *= image.shape[2]
#indices to add the noise to
indices = np.array(indices, dtype=int)
noise = sp.rand(n)
#now actually add the noise
for i in xrange(n):
    image[indices[i,0],indices[i,1],indices[i,2]] = noise[i]
#save the noisy version
imsave('baloons_resized_noisy.png', image)
%time min_bias_color(image, lbda, iters)
imsave('baloons_resized_minbias.png', image)
iters = 8
%time perona_malik_color20(image, lbda, sigma, iters)
imsave('baloons_resized_both.png', image)
plt.imshow(image)
plt.show()
```

```

```