  GEOS 657: Microwave Remote Sensing

 Lab 8: Change Detection in SAR Amplitude Time Series Data  -- [20 Points]

  Franz J Meyer; University of Alaska Fairbanks & Josef Kellndorfer, Earth Big Data, LLC

 Due Date:  April 23, 2019

 This Lab is part of the UAF course GEOS 657: Microwave Remote Sensing. It is introducing you to the methods of change detection in deep multi-temporal SAR image data stacks. Specifically, the lab applies the method of Cumulative Sums to perform change detection in a 60 image deep Sentinel-1 data stack over Niamey, Niger.

As previously, the work will be done within the framework of a Jupyter Notebook.

In this chapter we introduce the following data analysis concepts:

- The concepts of time series slicing by month, year, and date.
- The concepts and workflow of Cumulative Sum-based change point detection.
- The identification of change dates for each identified change point.

  THIS NOTEBOOK INCLUDES FOUR HOMEWORK ASSIGNMENTS. Complete all four assignments to achieve full score.

 To submit your homework, please download your Jupyter Notebook from the server both as PDF (*.pdf) and Notebook file (*.ipynb) and submit them as a ZIP bundle via Blackboard or email (to fjmeyer@alaska.edu). To download, please select the following options in the main menu of the notebook interface:

  Save your notebook with all of its content by selecting  File / Save and Checkpoint

 To export in Notebook format, click on File / Download as / Notebook (.ipynb)  --- Downloading your file may take a bit depending on its size.

The best option to  export your notebook in PDF format is to print the content of the browser window to a PDF. To do so, right click in your browser window and select the print option in the pop-up menu.

Contact me at fjmeyer@alaska.edu should you run into any problems.

 Important Note about JupyterHub

 Your JupyterHub server will automatically shutdown when left idle for more than 1 hour. Your notebooks will not be lost but you will have to restart their kernels and re-run them from the beginning. You will not be able to seamlessly continue running a partially run notebook.

```javascript

%%javascript
var kernel = Jupyter.notebook.kernel;
var command = ["notebookUrl = ",
               "'", window.location, "'" ].join('')
kernel.execute(command)
```

```python
from IPython.display import Markdown
from IPython.display import display

user = !echo $JUPYTERHUB_USER
env = !echo $CONDA_PREFIX
if env[0] == '':
    env[0] = 'Python 3 (base)'
if env[0] != '/home/jovyan/.local/envs/rtc_analysis':
    display(Markdown(f'<text style=color:red><strong>WARNING:</strong></text>'))
    display(Markdown(f'<text style=color:red>This notebook should be run using the "rtc_analysis" conda environment.</text>'))
    display(Markdown(f'<text style=color:red>It is currently using the "{env[0].split("/")[-1]}" environment.</text>'))
    display(Markdown(f'<text style=color:red>Select the "rtc_analysis" from the "Change Kernel" submenu of the "Kernel" menu.</text>'))
    display(Markdown(f'<text style=color:red>If the "rtc_analysis" environment is not present, use <a href="{notebookUrl.split("/user")[0]}/user/{user[0]}/notebooks/conda_environments/Create_OSL_Conda_Environments.ipynb"> Create_OSL_Conda_Environments.ipynb </a> to create it.</text>'))
    display(Markdown(f'<text style=color:red>Note that you must restart your server after creating a new environment before it is usable by notebooks.</text>'))
```

  0. Importing Relevant Python Packages

 The first step of this lab exercise on SAR image time series analysis and change detection is to import the necessary python libraries into your Jupyter Notebook.

```python
%%capture
import os
from copy import deepcopy

import pandas as pd
from osgeo import gdal # for Info
import numpy as np

%matplotlib inline
import matplotlib.pylab as plt
import matplotlib.patches as patches

import asf_notebook as asfn
asfn.jupytertheme_matplotlib_format()
```

  1. Load Data Stack for this Lab

 This Lab will be using a 60-image deep C-band Sentinel-1 SAR data stack over Niamey, Niger to demonstrate the concepts of time series change detection. The data are available to us through the services of the Alaska Satellite Facility.

Specifically we will use a small image segment over the campus of AGRHYMET Regional Centre, a regional organization supporting West Africa in the use or remote sensing.

This site was picked as we had information about construction going on at this site sometime in the 2015 - 2017 time frame. Land was cleared and a building was erected. In this exercise we will see if we can detect the construction activity and if we are able to determine when construction began and when it ended.

In this case, we will retrieve the relevant data from an Amazon Web Service (AWS) cloud storage bucket.

  1.1 Download The Data:

 Before we download anything, create a working directory for this analysis and change into it:

```python
path = "/home/jovyan/notebooks/ASF/GEOS_657_Labs/2019/lab_8_data"
asfn.new_directory(path)
os.chdir(path)
print(f"Current working directory: {os.getcwd()}")
```

Download the data from the AWS bucket:

```python
!aws --region=us-east-1 --no-sign-request s3 cp s3://asf-jupyter-data/Niamey.zip Niamey.zip
```

Unzip the file and clean up:

```python
f = "Niamey.zip"
asfn.asf_unzip(path, f)
if asfn.path_exists(f):
    os.remove(f)
```

  1.2 Switch to the Data Directory:

 The following lines set variables that capture path variables needed for data processing. Change into the unzipped /cra directory and define variables for names of the files containing data and image information:

```python
cra_path = f"{path}/cra/"
if asfn.path_exists(cra_path):
    os.chdir(cra_path)
date_file = "S32631X402380Y1491460sS1_A_vv_0001_A_mtfil.dates"
image_file = "S32631X402380Y1491460sS1_A_vv_0001_A_mtfil.vrt"
```

  1.3 Assess Image Acquisition Dates

 Before we start analyzing the available image data, we want to examine the content of our data stack. To do so, we read the image acquisition dates for all files in the time series and create a *pandas* date index:

```python
with open(date_file, 'r') as d:
    dates = d.readlines()
time_index = pd.DatetimeIndex(dates)
j = 1
print('Bands and dates for', image_file)
for i in time_index:
    print("{:4d} {}".format(j, i.date()), end=' ')
    j += 1
    if j%5 == 1: print()
```

  1.4 Read in the Data Stack

 We Read in the time series raster stack from the entire data set.

```python
raster_stack = gdal.Open(image_file).ReadAsArray()
```

  1.5 Lay the groundwork for saving plots and level-3 products.

Create a directory in which store our output, and move into it:

```python
os.chdir(path)
product_path = 'plots_and_products'
asfn.new_directory(product_path)
if asfn.path_exists(product_path) and os.getcwd() != f"{path}/{product_path}":
    os.chdir(product_path)
print(f"Current working directory: {os.getcwd()}")
```

We will need the upper-left and lower-right corner coordinates when saving our products as GeoTiffs. In this situation, you have been given a pre-subset vrt image stack.

Retrieve the corner coordinates from the vrt using gdal.Info():

```python
vrt = gdal.Open(f"{cra_path}{image_file}")
vrt_info = gdal.Info(vrt, format='json')
coords = [vrt_info['cornerCoordinates']['upperLeft'], vrt_info['cornerCoordinates']['lowerRight']]
print(coords)
```

Retrieve the utm zone from the vrt:

```python
utm_zone = vrt_info['coordinateSystem']['wkt'].split(',')[-1][0:-2]
print(f"utm zone: {utm_zone}")
```

Write a function to convert our plots into GeoTiffs:

```python
# do not include a file extension in out_filename
# extent must be in the form of a list: [[upper_left_x, upper_left_y], [lower_right_x, lower_right_y]]
def geotiff_from_plot(source_image, out_filename, extent, utm_zone, cmap=None, vmin=None, vmax=None, interpolation=None, dpi=300):
    plt.figure()
    plt.axis('off')
    plt.imshow(source_image, cmap=cmap, vmin=vmin, vmax=vmax, interpolation=interpolation)
    temp = f"{out_filename}_temp.png"
    plt.savefig(temp, dpi=dpi, transparent='true', bbox_inches='tight', pad_inches=0)

    cmd = f"gdal_translate -of Gtiff -a_ullr {extent[0][0]} {extent[0][1]} {extent[1][0]} {extent[1][1]} -a_srs EPSG:{utm_zone} {temp} {out_filename}.tiff"
    !{cmd}
    try:
        os.remove(temp)
    except FileNotFoundError:
        pass
```

  2. Plot the Global Means of the Time Series

 To accomplish this task, complete the following steps:

Conversion to power-scale

Compute mean values

Convert to dB-scale

Create time series of means using Pandas

Plot time series of means

Convert to Power-scale:

```python
caldB = -83
calPwr = np.power(10.0, caldB/10.0)
raster_stack_pwr = np.power(raster_stack, 2.0) * calPwr
```

Compute means:

```python
rs_means_pwr = np.mean(raster_stack_pwr, axis=(1, 2))
```

Convert to dB-scale:

```python
rs_means_dB = 10.0 * np.log10(rs_means_pwr)
```

Make a pandas time series object:

```python
ts = pd.Series(rs_means_dB,index=time_index)
```

Use pandas to plot the time series object with band numbers as data point labels. Save the plot as a png (time_series_means.png):

```python
plt.rcParams.update({'font.size': 14})
plt.figure(figsize=(16, 8))
plt.title(f"Time Series of Means")
ts.plot()
xl = plt.xlabel('Date')
yl = plt.ylabel('$\overline{\gamma^o}$ [dB]')
for xyb in zip(ts.index, rs_means_dB, range(1, len(ts)+1)):
    plt.annotate(xyb[2], xy=xyb[0:2])
plt.grid()
plt.savefig('time_series_means.png', dpi=72)
```

   ASSIGNMENT #1:   Analyze Global Means Time Series Plot  -- [3 Points]

 Look at the global means time series plot above and determine from the time_index array at which dates you see  maximum and minimum values. Are relative peaks associated with seasons?

PROVIDE ANSWER HERE:

  3. Generate Time Series for Point Locations or Subsets

 In python, we can use the matrix slicing tools (Like Matlab) to obtain subsets of the data. For example to pick one pixel at a line/pixel location and obtain all band values, use:

>  [:,line,pixel] notation.

Or, if we are interested in a subset at a offset location we can use:

> [:,yoffset:(yoffset+yrange),xoffset:(xoffset+xrange)]

In the section below we will learn how to generate time series plots for point locations (pixels) or areas (e.g. a 5x5 window region). To show  individual bands, we define a show_image function which incorporates the matrix slicing from above.

  3.1 Plotting Time Series for Subset

Write a function to plot the calibrated time series for a pre-defined subset:

```python
# Preconditions:
# raster_stack must be a stack of images in SAR power units
# time_index must be a pandas date-time index
# band_number must represent a valid bandnumber in the raster_stack
def show_image(raster_stack, time_index, band_number, output_filename=None, subset=None, vmin=None, vmax=None):
    fig = plt.figure(figsize=(16, 8))
    ax1 = fig.add_subplot(121)
    ax2 = fig.add_subplot(122)

    # If vmin or vmax are None we use percentiles as limits:
    if vmin == None:
        vmin = np.percentile(raster_stack[band_number-1].flatten(), 5)
    if vmax == None:
        vmax = np.percentile(raster_stack[band_number-1].flatten(), 95)

    ax1.imshow(raster_stack[band_number-1], cmap='gray', vmin=vmin, vmax=vmax)
    ax1.set_title('Image Band {} {}'.format(band_number, time_index[band_number-1].date()))
    if subset == None:
        bands, ydim, xdim = raster_stack.shape
        subset = (0, 0, xdim, ydim)

    ax1.add_patch(patches.Rectangle((subset[0], subset[1]), subset[2], subset[3], fill=False, edgecolor='red'))
    ax1.xaxis.set_label_text('Pixel')
    ax1.yaxis.set_label_text('Line')

    ts_pwr = np.mean(raster_stack[:, subset[1]:(subset[1]+subset[3]), subset[0]:(subset[0]+subset[2])], axis=(1,2))
    ts_dB = 10.0 * np.log10(ts_pwr)
    ax2.plot(time_index, ts_dB)
    ax2.yaxis.set_label_text('$\gamma^o$ [dB]')
    ax2.set_title('$\gamma^o$ Backscatter Time Series')
    # Add a vertical line for the date where the image is displayed
    ax2.axvline(time_index[band_number-1], color='red')
    plt.grid()

    fig.autofmt_xdate()

    if output_filename:
        plt.savefig(output_filename, dpi=72)
        print(f"Saved plot: {output_filename}")
```

Call show_image() on different bands to compare the information content of different time steps in our area of interest.

 Call show_image() on band number 24:

```python
band_number = 24
subset = [5, 20, 3, 3]
show_image(raster_stack_pwr, time_index, band_number, subset=subset, output_filename=f"band_{band_number}.png")
```

Call show_image() on band number 43:

```python
band_number = 43
show_image(raster_stack_pwr, time_index, band_number, subset=subset, output_filename=f"band_{band_number}.png")
```

  3.2 Helper Function to Generate a Time Series Object

Write a function that creates an object representing the time series for an image subset:

```python
# Extract the means along the time series axes
# raster shape is time steps, lines, pixels.
# With axis=1,2, we average lines and pixels for each time step (axis 0)
# returns pandas time series object
def timeSeries(raster_stack_pwr, time_index, subset, ndv=0.0):
    raster = raster_stack_pwr.copy()
    if ndv != np.nan:
        raster[np.equal(raster, ndv)] = np.nan
    ts_pwr = np.nanmean(raster[:,subset[1]:(subset[1]+subset[3]), subset[0]:(subset[0]+subset[2])], axis=(1, 2))
    # convert the means to dB
    ts_dB = 10.0 * np.log10(ts_pwr)
    # make the pandas time series object
    ts = pd.Series(ts_dB, index=time_index)
    return ts
```

Call timeSeries() to make a time series object for the chosen subset:

```python
ts = timeSeries(raster_stack_pwr, time_index, subset)
```

Plot the time series object:

```python
fig = ts.plot(figsize=(16, 4))
fig.yaxis.set_label_text('mean dB')
fig.set_title('Time Series for Chosen Subset')
plt.grid()
```

  4. Create Seasonal Subsets of Time Series Records

 Let's expand upon SAR time series analysis. Often it is desirable to subset time series by season or months to compare data acquired under similar weather/growth/vegetation cover conditions. For example, in analyzing C-Band backscatter data, it might be useful to limit comparative analysis to dry season observations only as soil moisture might confuse signals during the wet seasons. To subset time series along the time axis we will make use of the following Pandas datatime index tools:

month

day of year

Extract a hectare-sized area around our subset location (5,20,5,5):

```python
subset = (5, 20, 5, 5)
time_series_1 = timeSeries(raster_stack_pwr, time_index, subset)
```

Convert the time series to a pandas DataFrame to allow for more processing options.

```python
data_frame = pd.DataFrame(time_series_1, index=ts.index, columns=['g0'])
```

Label the data value column as 'g0' for $\gamma^0$ and plot the time series backscatter profile:

```python
ylim = (-15, -5)
data_frame.plot(figsize=(16, 4))
plt.title('Sentinel-1 C-VV Time Series Backscatter Profile, Subset: 5,20,5,5  ')
plt.ylabel('$\gamma^o$ [dB]')
plt.ylim(ylim)
_ = plt.legend(["C-VV Time Series"])
plt.grid()
```

 4.1 Change Start Date of Time Series to November 2015

Plot the cropped time series and save it as a png (time_series_backscatter_profile.png):

```python
data_frame_sub1 = data_frame[data_frame.index>'2015-11-01']

# Plot
data_frame_sub1.plot(figsize=(16, 4))
plt.title('Sentinel-1 C-VV Time Series Backscatter Profile, Subset: {}'.format(subset))
plt.ylabel('$\gamma^o$ [dB]')
plt.ylim(ylim)
_ = plt.legend(["C-VV Time Series"])
plt.grid()
plt.savefig('time_series_backscatter_profile', dpi=72)
```

  4.2 Subset Time Series by Months

Using the Pandas DateTimeIndex object index.month and numpy's logical_and function, we can easily subset the time series by month:

Create subset data_frames. In one, replace the data from June-February with NaNs. In the other, replace the data from March-May with NaNs:

```python
data_frame_sub2 = deepcopy(data_frame_sub1)
for index, row in data_frame_sub2.iterrows():
    if index.month < 3 or index.month > 5:
        row['g0'] = np.nan

data_frame_sub3 = deepcopy(data_frame_sub1)
for index, row in data_frame_sub3.iterrows():
    if index.month > 2 and index.month < 6:
        row['g0'] = np.nan
```

Plot the time series backscatter profile for March - May. Save the plot as a png (march2may_time_series_backscatter_profile.png):

```python
# Plot
fig, ax = plt.subplots(figsize=(16, 4))
data_frame_sub2.plot(ax=ax)
plt.title(f'Sentinel-1 C-VV Time Series Backscatter Profile, Subset: {subset}')
plt.ylabel('$\gamma^o$ [dB]')
plt.ylim(ylim)
_ = plt.legend(["C-VV Time Series (March - May)"], loc='best')
plt.grid()
plt.savefig('march2may_time_series_backscatter_profile', dpi=72)
```

 Using numpy's **invert** function, we can invert a selection. In this example, we extract all other months from the time series.

Plot the time series backscatter profile for June - Feburary. Save the plot as a png (june2feb_time_series_backscatter_profile.png):

```python
# Plot
fig, ax = plt.subplots(figsize=(16, 4))
data_frame_sub3.plot(ax=ax)
plt.title(f'Sentinel-1 C-VV Time Series Backscatter Profile, Subset: {subset}')
plt.ylabel('$\gamma^o$ [dB]')
plt.ylim(ylim)
_ = plt.legend(["C-VV Time Series (June-February)"], loc='best')
plt.grid()
plt.savefig('june2feb_time_series_backscatter_profile', dpi=72)
```

  4.3 Split Time Series by Year to Compare Year-to-Year Patterns

 Sometimes it is useful to compare year-to-year $\sigma^0$ values to identify changes in backscatter characteristics. This helps to distinguish true change from seasonal variability.

    Split the time series into different years:

```python
data_frame_by_year = data_frame_sub1.groupby(pd.Grouper(freq="Y"))
```

Plot the split time series. Save the plot as a png (yearly_time_series_backscatter_profile.png):

```python
fig, ax = plt.subplots(figsize=(16, 4))
for label, df in data_frame_by_year:
    df.g0.plot(ax=ax, label=label.year)
plt.legend()
# data_frame_by_year.plot(ax=ax)
plt.title('Sentinel-1 C-VV Time Series Backscatter Profile, Subset: {}'.format(subset))
plt.ylabel('$\gamma^o$ [dB]')
plt.ylim(ylim)
plt.grid()
plt.savefig('yearly_time_series_backscatter_profile', dpi=72)
```

  4.4 Create a Pivot Table to Group Years and Sort Data for Plotting Overlapping Time Series

 Pivot Tables are  a technique in data processing. They enable a person to arrange and rearrange (or "pivot") statistics in order to draw attention to useful information. To do so, we first add columns for day-of-year and year to the data frame:

```python
# Add day of year
data_frame_sub1 = data_frame_sub1.assign(doy=data_frame_sub1.index.dayofyear)
# Add year
data_frame_sub1 = data_frame_sub1.assign(year=data_frame_sub1.index.year)
```

Create a pivot table which has day-of-year as the index and years as columns:

```python
pivot_table = pd.pivot_table(data_frame_sub1, index=['doy'], columns=['year'], values=['g0'])
# Set the names for the column indices
pivot_table.columns.set_names(['g0', 'year'], inplace=True)
print(pivot_table.head(10))
print('...\n',pivot_table.tail(10))
```

 As we can see, there are NaN values on the days in a year where no acquisition took place. Now we use time weighted interpolation to fill the dates for all the observations in any given year. For **time weighted interpolation** to work we need to create a dummy year as a date index, perform the interpolation, and reset the index to the day of year.

Create a dummy year as a date index:

```python
# Add fake dates for year 100 to enable time sensitive interpolation
# of missing values in the pivot table
year_doy = ['2100-{}'.format(x) for x in pivot_table.index]
y100_doy=pd.DatetimeIndex(pd.to_datetime(year_doy,format='%Y-%j'))

# make a copy of the piv table and add two columns
pivot_table_2 = pivot_table.copy()
pivot_table_2 = pivot_table_2.assign(d100=y100_doy) # add the fake year dates
pivot_table_2 = pivot_table_2.assign(doy=pivot_table_2.index) # add doy as a column to replace as index later again

# Set the index to the dummy year
pivot_table_2.set_index('d100', inplace=True, drop=True)
```

Perform the time-weighted interpolation:

```python
pivot_table_2 = pivot_table_2.interpolate(method='time')
```

Reset the index to the day of year:

```python
pivot_table_2.set_index('doy', inplace=True, drop=True)
```

Inspect the new pivot table and see whether we interpolated the NaN values where it made sense:

```python
print(pivot_table_2.head(10))
print('...\n',pivot_table_2.tail(10))
```

Plot the time series data with overlapping years. Save the plot as a png (overlapping_years_time_series_backscatter_profile.png):

```python
pivot_table_2.plot(figsize=(16, 8))
plt.title('Sentinel-1 C-VV Time Series Backscatter Profile,\
Subset: 5,20,5,5  ')
plt.ylabel('$\gamma^o$ [dB]')
plt.xlabel('Day of Year')
_ = plt.ylim(ylim)
plt.grid()
plt.savefig('overlapping_years_time_series_backscatter_profile', dpi=72)
```

   ASSIGNMENT #2:   Interpret the Year-to-Year Time Series Plot  -- [4 Points]

 Answer the following questions related to the year-to-year time series plot shown above:

Describe the $\gamma^0$ time series for year 2016. What kind of seasonal patterns do you see? Based on the observed seasonal patterns, what type of surface cover do you think was present at this area in 2016?  -- [2 Points]

Describe the $\gamma^0$ time series for year 2017. What kind of seasonal patterns do you see and how do they differ from the previous year?  -- [2 Points]

PROVIDE YOUR ANSWERS BELOW:

  5. Time Series Change Detection

 Now we are ready to perform efficient change detection on the time series data. We will discuss two approaches:

Year-to-year differencing of the subsetted time series

Cumulative Sum-based change detection

Set a dB change threshold.

```python
threshold = 3
```

Calculate the difference between years (2016 and 2017):

```python
diff_2017_2016 = pivot_table_2.g0[2017] - pivot_table_2.g0[2016]
```

  5.1 Change Detection based on Year-to-Year Differencing

Compute and plot the differences between the interpolated time series and look for change using a threshold value. Save the plot as a png (year2year_differencing_time_series.png):

```python
_ = diff_2017_2016.plot(kind='line', figsize=(16,8))
plt.title('Year-to-Year Difference Time Series')
plt.ylabel('$\Delta\gamma^o$ [dB]')
plt.xlabel('Day of Year')
plt.grid()
plt.savefig('year2year_differencing_time_series', dpi=72)
```

Calculate the days-of-year on which the threshold was exceeded:

```python
threshold_exceeded = diff_2017_2016[abs(diff_2017_2016) > threshold]
threshold_exceeded
```

 From the threshold_exceeded dataframe we can infer the first date at which the threshold was exceeded. We would label this date as a **change point**. As an additional criteria for labeling a change point, one can also consider the number of observations after an identified change point that also excided the threshold. If only one or two observations differed from the year before this could be considered an outlier. Additional smoothing of the time series may sometimes be useful to avoid false detections.

   ASSIGNMENT #3:   Perform Year-to-Year Differencing-based Change Detection for a Different Subset  -- [5 Points]

 Go back to the beginning of Section 4 and change the subset coordinates to a different subset (i.e., modify to subset=($X$,$Y$,5,5) with $X$ and $Y$ being the center of your modified subset). Work through the workbook from the beginning of Section 4 until the end of Section 5.1 with your modified subset. Discuss whether or not your new subset shows change according to the 3dB change threshold.

DISCUSS BELOW WHETHER YOUR NEW SUBSET SHOWS CHANGE:

  5.2 Cumulative Sums for Change Detection

 Another approach to detect change in regularly acquired data is employing the method of **cumulative sums**. Changes are determined by comparing the time series data against its mean. A full explanation and examples from the financial sector can be found at [http://www.variation.com/cpa/tech/changepoint.html](http://www.variation.com/cpa/tech/changepoint.html)

5.2.A First let's consider a time series and it's mean observation:

We look at two full years of observations from Sentinel-1 data for an area where we suspect change. In the following, we define $X$ as our time series

\begin{equation}
X = (X_1,X_2,...,X_n)
\end{equation}

with $X_i$ being the SAR backscatter values at times $i=1,...,n$ and $n$ is the number of observations in the time series.

Create a times series of the subset and calculate the backscatter values:

```python
subset = (5, 20, 3, 3)
#subset=(12,5,3,3)
time_series_1 = timeSeries(raster_stack_pwr, time_index, subset)
backscatter_values = time_series_1[time_series_1.index>'2015-10-31']
```

 5.2.B Filtering the time series for outliers:

It is advantageous in noisy SAR time series like those from C-Band Sentinel-1 data to reduce noise by applying a filter along the time axis. Pandas offers a "rolling" function for these purposes. Using the rolling function, we will apply a median filter to our data.

Calculate the median backscatter values and plot them against the original values. Save the plot as a png (Original vs. Median Time Series):

```python
backscatter_values_median = backscatter_values.rolling(5, center=True).median()
backscatter_values_median.plot(figsize=(16, 4))
_ = backscatter_values.plot()
plt.title('Original vs. Median Time Series')
plt.ylabel('$\gamma^o$ [dB]')
plt.xlabel('Time')
plt.grid()
plt.savefig('original_vs_median_time_series', dpi=72)
```

Calculate the time series' mean value and plot it against the original values. Save the plot as a png (original_time_series_vs_mean_val.png):

```python
fig, ax = plt.subplots(figsize=(16, 4))
backscatter_values.plot()
plt.title('Original Time Series vs. Mean Value')
plt.ylabel('$\gamma^o$ [dB]')
ax.axhline(backscatter_values.mean(), color='red')
_ = plt.legend(['$\gamma^o$', '$\overline{\gamma^o}$'])
plt.grid()
plt.savefig('original_time_series_vs_mean_val', dpi=72)
```

Calculate the time series' mean value and plot it against the median values. Save the plot as a png (median_time_series_vs_mean_val.png):

```python
backscatter_values_mean = backscatter_values.mean()

fig, ax = plt.subplots(figsize=(16, 4))
backscatter_values_median.plot()
plt.title('Median Time Series vs. Mean Value')
plt.ylabel('$\gamma^o$ [dB]')
ax.axhline(backscatter_values.mean(), color='red')
_ = plt.legend(['$\gamma^o$', '$\overline{\gamma^o}$'])
plt.grid()
plt.savefig('median_time_series_vs_mean_val', dpi=72)
```

 5.2.C Calculate the Residuals of the Time Series Against the Mean $\overline{\gamma^o}$:

To get to the residual, we calculate

\begin{equation}
R = X_i - \overline{X}
\end{equation}

Calculate the residuals:

```python
residuals = backscatter_values - backscatter_values_mean
```

 5.2.D Calculate Cumulative Sum of the Residuals:

The cumulative sum is defined as:

\begin{equation}
S = \displaystyle\sum_1^n{R_i}
\end{equation}

 Calculate and plot the cumulative sum of the residuals. Save the plot as a png (cumulative_sum_residuals.png):

```python
sums = residuals.cumsum()

_ = sums.plot(figsize=(16, 6))
plt.title('Cumulative Sum of the Residuals')
plt.ylabel('Cummulative Sum $S$ [dB]')
plt.xlabel('Time')
plt.grid()
plt.savefig('cumulative_sum_residuals', dpi=72)
```

 The **cumulative sum** is a good indicator of change in the time series. An estimator for the magnitude of change is given as the difference between the maximum and minimum value of the cumulative sum $S$:

\begin{equation}
S_{DIFF} = S_{MAX} - S_{MIN}
\end{equation}

Calculate the magnitude of change:

```python
change_mag = sums.max() - sums.min()
print('Change magnitude: %5.3f dB' % (change_mag))
```

 5.2.E Identify Change Point in the Time Series:

A candidate change point is identified from $S$ at the time where $S_{MAX}$ is found:

\begin{equation}
T_{{CP}_{before}} = T(S_i = S_{MAX})
\end{equation}

with $T_{{CP}_{before}}$ being the timestamp of the last observation before the identified change point, $S_i$ is the cumulative sum of $R$ with $i=1,...n$, and $n$ is the number of observations in the time series.

The first observation after a change occurred ($T_{{CP}_{after}}$) is then found as the first observation in the time series following $T_{{CP}_{before}}$.

Calculate $T_{{CP}_{before}}$:

```python
change_point_before = sums[sums==sums.max()].index[0]
print('Last date before change occurred: {}'.format(change_point_before.date()))
```

Calculate $T_{{CP}_{after}}$:

```python
change_point_after = sums[sums.index > change_point_before].index[0]
print('First date after change occurred: {}'.format(change_point_after.date()))
```

 5.2.F Determine our Confidence in the Identified Change Point using Bootstrapping:

We can determine if an identified change point is indeed a valid detection by randomly reordering the time series and comparing the various $S$ curves. During this "bootstrapping" approach, we count how many times the $S_{DIFF}$ values are greater than $S_{{DIFF}_{random}}$ of the identified change point.

After bootstrapping, we define the confidence level $CL$ in a detected change point according to:

\begin{equation}
CL = \frac{N_{GT}}{N_{bootstraps}}
\end{equation}

where $N_{GT}$ is the number of times $S_{DIFF}$ > $S_{{DIFF}_{random}}$ and $N_{bootstraps}$ is the number of bootstraps randomizing $R$.

As another quality metric we can also calculate the significance $CP_{significance}$ of a change point according to:

\begin{equation}
CP_{significance} = 1 - \left( \frac{\sum_{b=1}^{N_{bootstraps}}{S_{{DIFF}_{{random}_i}}}}{N_{bootstraps}} \middle/ S_{DIFF} \right)
\end{equation}

The closer $CP_{significance}$ is to 1, the more significant the change point.

Write a function that implements the bootstrapping algorithm:

```python
# pyplot must be imported as plt
import random
def bootstrap(n_bootstraps, sums, residuals, output_file=False):
    fig, ax = plt.subplots(figsize=(16,6))
    ax.set_ylabel('Cumulative Sums of the Residuals')
    change_mag_random_sum = 0
    change_mag_random_max = 0 # to keep track of the maximum change magnitude of the bootstrapped sample
    qty_change_mag_above_random = 0 # to keep track of the maxium Sdiff of the bootstrapped sample
    print("Running Bootstrapping for %4.1f iterations ..." % (n_bootstraps))
    colors = ['C0', 'C1', 'C2', 'C3', 'C4', 'C5', 'C6', 'C7', 'C8', 'C9', 'c', 'm', 'y', 'k', 'g']
    for i in range(n_bootstraps):
        residuals_random = residuals.sample(frac=1)  # Randomize the time steps of the residuals
        sums_random = residuals_random.cumsum()
        change_mag_random = sums_random.max() - sums_random.min()
        change_mag_random_sum += change_mag_random
        if change_mag_random > change_mag_random_max:
            change_mag_random_max = change_mag_random
        if change_mag > change_mag_random:
            qty_change_mag_above_random += 1
        sums_random.plot(ax=ax, color=random.choice(colors), label='_nolegend_')
        if ((i+1)/n_bootstraps*100) % 10 == 0:
            print("\r%4.1f percent completed ..." % ((i+1)/n_bootstraps*100), end='\r', flush=True)
    sums.plot(ax=ax, color='r', linewidth=3)
    fig.legend(['S Curve for Candidate Change Point'])
    print(f"Bootstrapping Complete")
    _ = ax.axhline(change_mag_random_sum/n_bootstraps, color='b')
    plt.grid()
    if output_file:
        plt.savefig(f"bootstrap_{n_bootstraps}", dpi=72)
        print(f"Saved plot: bootstrap_{n_bootstraps}.png")
    return [qty_change_mag_above_random, change_mag_random_sum]
```

Call the bootstrap function with a sample size of 2000:

```python
n_bootstraps = 2000
bootstrapped_change_mag = bootstrap(n_bootstraps, sums, residuals, output_file=True)
```

 Based on the bootstrapping results, we can now calculate Confidence Level $CL$ and Significance $CP_{significance}$ for our candidate change point.

Calculate the confidence level:

```python
confidence_level = 1.0 * bootstrapped_change_mag[0] / n_bootstraps
print('Confidence Level for change point {} percent'.format(confidence_level*100.0))
```

Calculate the change point significance:

```python
change_point_significance = 1.0 - (bootstrapped_change_mag[1]/n_bootstraps)/change_mag
print('Change point significance metric: {}'.format(change_point_significance))
```

 5.2.G TRICK: Detrending of Time Series Before Change Detection to Improve Robustness:

De-trending the time series with global image means improves the robustness of change point detection as global image time series anomalies stemming from calibration or seasonal trends are removed prior to time series analysis. This de-trending needs to be performed with large subsets so real change is not influencing the image statistics.

NOTE: Due to the small size of our subset, we will see some distortions when we detrend the time series.

Let's start by building a global image means time series and plot the global means. Save the plot as a png (global_means_time_series.png):

```python
means_pwr = np.mean(raster_stack_pwr, axis=(1, 2))
means_dB = 10.0 * np.log10(means_pwr)
global_means_ts = pd.Series(means_dB, index=time_index)
global_means_ts = global_means_ts[global_means_ts.index > '2015-10-31'] # filter dates
global_means_ts = global_means_ts.rolling(5, center=True).median()
global_means_ts.plot(figsize=(16, 6))
plt.title('Time Series of Global Means')
plt.ylabel('[dB]')
plt.xlabel('Time')
plt.grid()
plt.savefig(f"global_means_time_series", dpi=72)
```

Compare the time series of global means (above) to the time series of our small subset (below):

```python
backscatter_values.plot(figsize=(16, 6))
plt.title('Sentinel-1 C-VV Time Series Backscatter Profile,\
Subset: 5,20,5,5  ')
plt.ylabel('[dB]')
plt.xlabel('Time')
plt.grid()
```

 There are some signatures of the global seasonal trend in our subset time series. To remove these signatures and get a cleaner time series of change, we subtract the global mean time series from our subset time series.

De-trend the subset and re-plot the backscatter profile. Save the plot (detrended_time_series.png):

```python
backscatter_minus_seasonal = backscatter_values - global_means_ts
backscatter_minus_seasonal.plot(figsize=(16, 6))
plt.title('De-trended Sentinel-1 C-VV Time Series Backscatter Profile, Subset: 5,20,5,5')
plt.ylabel('[dB]')
plt.xlabel('Time')
plt.grid()
plt.savefig(f"detrended_time_series", dpi=72)
```

Save a plot comparing the original, global means, and detrended time-series (globalMeans_original_detrended_time_series.png):

```python
means_pwr = np.mean(raster_stack_pwr, axis=(1, 2))
means_dB = 10.0 * np.log10(means_pwr)
global_means_ts = pd.Series(means_dB, index=time_index)
global_means_ts = global_means_ts[global_means_ts.index > '2015-10-31'] # filter dates
global_means_ts = global_means_ts.rolling(5, center=True).median()
global_means_ts.plot(figsize=(16, 6))
backscatter_values.plot(figsize=(16, 6))
backscatter_minus_seasonal = (backscatter_values - global_means_ts)
backscatter_minus_seasonal.plot(figsize=(16, 6))
plt.title('Time Series of Global Means')
plt.ylabel('[dB]')
plt.xlabel('Time')
plt.grid()
plt.savefig(f"globalMeans_original_detrended_time_series", dpi=72)
```

Recalculate the residuals based on the de-trended data: and plot it:

```python
residuals = backscatter_minus_seasonal - backscatter_values_mean
```

Compute, plot, and save the cumulative sum of the detrended time series (cumualtive_sum_detrended_time_series.png):

```python
sums = residuals.cumsum()
_ = sums.plot(figsize=(16, 6))
plt.title("Cumulative Sum of the Detrended Time Series")
plt.ylabel('CumSum $S$ [dB]')
plt.xlabel('Time')
plt.grid()
plt.savefig(f"cumualtive_sum_detrended_time_series", dpi=72)
```

Detect Change Point and extract related change dates:

```python
change_point_before = sums[sums==sums.max()].index[0]
print('Last date before change occurred: {}'.format(change_point_before.date()))

change_point_after = sums[sums.index > change_point_before].index[0]
print('First date after change occurred: {}'.format(change_point_after.date()))
```

Perform bootstrapping:

```python
n_bootstraps = 2000
bootstrapped_change_mag = bootstrap(n_bootstraps, sums, residuals, output_file=True)
```

Calculate the confidence level:

```python
confidence_level = bootstrapped_change_mag[0] / n_bootstraps
print('Confidence Level for change point {} percent'.format(confidence_level*100.0))
```

Note how the change point significance $CP_{significance}$ has increased in the detrended time series:

```python
change_point_significance = 1.0 - (bootstrapped_change_mag[1]/n_bootstraps) / change_mag
print('Change point significance metric: {}'.format(change_point_significance))
```

   ASSIGNMENT #4:   Perform Cumulative Sum-based Change Detection for a Different Subset  -- [6 Points]

 Go back to the beginning of Section 5.2 and change the subset coordinates to a different subset (i.e., modify to subset=($X$,$Y$,5,5) with $X$ and $Y$ being the center of your selected subset). Work through the workbook from the beginning to the end of Section 5.2with your selected subset. Discuss whether or not your new subset shows change according to the Cumulative Sum approach.

DISCUSS BELOW WHETHER YOUR NEW SUBSET SHOWS CHANGE:

  6. Cumulative Sum-based Change Detection Across an Entire Image

 With numpy arrays we can apply the concept of **cumulative sum change detection** analysis effectively on the entire image stack. We take advantage of array slicing and axis-based computing in numpy. Axis 0 is the time domain in our raster stacks.

6.1 We first create our time series stack:

Filter out the first layer (Dates >= '2015-11-1'):

```python
raster_stack = raster_stack_pwr
raster_stack_sub = raster_stack_pwr[1:, :, :]
time_index_sub = time_index[1:]
```

Run the following code cell if you wish to change to dB scale:

```python
raster_stack = 10.0 * np.log10(raster_stack_sub)
```

Plot and save Band-1 (band_1.png):

```python
plt.figure(figsize=(12, 8))
band_number = 0
vmin = np.percentile(raster_stack[band_number], 5)
vmax = np.percentile(raster_stack[band_number], 95)
plt.title('Band  {} {}'.format(band_number+1, time_index_sub[band_number].date()))
plt.imshow(raster_stack[0], cmap='gray', vmin=vmin, vmax=vmax)
_ = plt.colorbar()
plt.savefig('band_1.png', dpi=300, transparent='true')
```

Save the plot as a GeoTiff (band_1.tiff):

```python
%%capture
geotiff_from_plot(raster_stack[0], 'band_1', coords, utm_zone, cmap='gray', dpi=600)
```

  6.2 Calculate Mean Across Time Series to Prepare for Calculation of Cumulative Sum $S$:

 Plot and save the the raster stack mean (raster_stack_mean.png):

```python
raster_stack_mean = np.mean(raster_stack, axis=0)
plt.figure(figsize=(12, 8))
plt.imshow(raster_stack_mean, cmap='gray')
_ = plt.colorbar()
plt.savefig('raster_stack_mean.png', dpi=300, transparent='true')
```

Save the raster stack mean as a GeoTiff (raster_stack_mean.tiff):

```python
%%capture
geotiff_from_plot(raster_stack_mean, 'raster_stack_mean', coords, utm_zone, cmap='gray', dpi=600)
```

Calculate the residuals:

```python
residuals = raster_stack - raster_stack_mean
```

Close img, as it is no longer needed in the notebook:

```python
raster_stack = None
```

Plot and save the residuals for band 1 (residuals_band_1.png):

```python
plt.figure(figsize=(12, 8))
plt.imshow(residuals[0])
plt.title('Residuals for Band  {} {}'.format(band_number+1, time_index_sub[band_number].date()))
_ = plt.colorbar()
plt.savefig('residuals_band_1', dpi=300, transparent='true')
```

Save the residuals for band 1 as a GeoTiff (residuals_band_1.tiff):

```python
%%capture
geotiff_from_plot(residuals[0], 'residuals_band_1', coords, utm_zone, dpi=600)
```

  6.3 Calculate Cumulative Sum $S$ as well as Change Magnitude $S_{diff}$:

Plot and save the cumulative sum max, min, and change magnitude (Smax_Smin_Sdiff.png):

```python
sums = np.cumsum(residuals, axis=0)
sums_max = np.max(sums, axis=0)
sums_min = np.min(sums, axis=0)
change_mag = sums_max - sums_min
fig, ax = plt.subplots(1, 3, figsize=(16, 4))
vmin = sums_min.min()
vmax = sums_max.max()
sums_max_plot = ax[0].imshow(sums_max, vmin=vmin, vmax=vmax)
ax[0].set_title('$S_{max}$')
ax[1].imshow(sums_min, vmin=vmin, vmax=vmax)
ax[1].set_title('$S_{min}$')
ax[2].imshow(change_mag, vmin=vmin, vmax=vmax)
ax[2].set_title('Change Magnitude')
fig.subplots_adjust(right=0.8)
cbar_ax = fig.add_axes([0.85, 0.15, 0.02, 0.7])
_ = fig.colorbar(sums_max_plot, cax=cbar_ax)
plt.savefig('Smax_Smin_Sdiff', dpi=300, transparent='true')
```

Save Smax as a GeoTiff (Smax.tiff):

```python
%%capture
geotiff_from_plot(sums_max, 'Smax', coords, utm_zone, vmin=vmin, vmax=vmax)
```

Save Smin as a GeoTiff (Smin.tiff):

```python
%%capture
geotiff_from_plot(sums_min, 'Smin', coords, utm_zone, vmin=vmin, vmax=vmax)
```

Save the change magnitude as a GeoTiff (Sdiff.tiff):

```python
%%capture
geotiff_from_plot(change_mag, 'Sdiff', coords, utm_zone, vmin=vmin, vmax=vmax)
```

  6.4 Mask $S_{diff}$ With a-priori Threshold To Idenfity Change Candidates:

To identified change candidate pixels, we can threshold $S_{diff}$ to reduce computation of the bootstrapping. For land cover change we would not expect more than 5-10% change pixels in a landscape. So, if the test region is reasonably large, setting a threshold for expected change to 10% is appropriate. In our example we'll start out with a very conservative threshold of 20%.

Plot and save the histogram for the change magnitude and the change magnitude cumulative distribution function (Sdiff_histogram_CDF.png):

```python
plt.rcParams.update({'font.size': 14})
fig = plt.figure(figsize=(14, 6)) # Initialize figure with a size
ax1 = fig.add_subplot(121)  # 121 determines: 2 rows, 2 plots, first plot
ax2 = fig.add_subplot(122)
# Second plot: Histogram
# IMPORTANT: To get a histogram, we first need to *flatten*
# the two-dimensional image into a one-dimensional vector.
histogram = ax1.hist(change_mag.flatten(), bins=200, range=(0, np.max(change_mag)))
ax1.xaxis.set_label_text('Change Magnitude')
ax1.set_title('Change Magnitude Histogram')
plt.grid()
n, bins, patches = ax2.hist(change_mag.flatten(), bins=200, range=(0, np.max(change_mag)), cumulative='True', density='True', histtype='step', label='Empirical')
ax2.xaxis.set_label_text('Change Magnitude')
ax2.set_title('Change Magnitude CDF')
plt.grid()
plt.savefig('Sdiff_histogram_CDF', dpi=72, transparent='true')
```

Using this threshold, we can create a plot to visualize our change candidate areas. Save the plot (change_candidate.png):

```python
precentile = 0.8
out_indicies = np.where(n>precentile)
threshold_indicies = np.min(out_indicies)
threshold = bins[threshold_indicies]
print('At the {}% percentile, the threshold value is {:2.2f}'.format(precentile*100,threshold))

change_mag_mask = change_mag < threshold
plt.figure(figsize=(12, 8))
plt.title('Change Candidate Areas (black)')
_ = plt.imshow(change_mag_mask, cmap='gray')
plt.savefig('change_candidate', dpi=300, transparent='true')
```

Save the change candidate plot as a GeoTiff (change_candidate.tiff):

```python
%%capture
geotiff_from_plot(change_mag_mask, 'change_candidate', coords, utm_zone, cmap='gray')
```

  6.5 Bootstrapping to Prepare for Change Point Selection:

We can now perform bootstrapping over the candidate pixels. The workflow is as follows:

Filter our residuals to the change candidate pixels

Perform bootstrapping over candidate pixels

For efficient computing we permutate the index of the time axis:

```python
residuals_mask = np.broadcast_to(change_mag_mask, residuals.shape)
residuals_masked = np.ma.array(residuals, mask=residuals_mask)
```

On the masked time series stack of residuals, we can re-compute the cumulative sums:

```python
sums_masked = np.ma.cumsum(residuals_masked, axis=0)
```

Plot the min sums, max sums, and change magnitude of the masked subset (masked_Smax_Smin_Sdiff.png):

```python
sums_masked_max = np.ma.max(sums_masked, axis=0)
sums_masked_min = np.ma.min(sums_masked, axis=0)
change_mag = sums_masked_max - sums_masked_min
fig, ax = plt.subplots(1, 3, figsize=(16, 4))
vmin = sums_masked_min.min()
vmax = sums_masked_max.max()
sums_masked_max_plot = ax[0].imshow(sums_masked_max, vmin=vmin, vmax=vmax)
ax[0].set_title('$S_{max}$')
ax[1].imshow(sums_masked_min, vmin=vmin, vmax=vmax)
ax[1].set_title('$S_{min}$')
ax[2].imshow(change_mag, vmin=vmin, vmax=vmax)
ax[2].set_title('Change Magnitude')
fig.subplots_adjust(right=0.8)
cbar_ax = fig.add_axes([0.85, 0.15, 0.02, 0.7])
_ = fig.colorbar(sums_masked_max_plot, cax=cbar_ax)
plt.savefig('masked_Smax_Smin_Sdiff', dpi=300, transparent='true')
```

Save the Smax of the masked subset as a GeoTiff (masked_Smax.tiff):

```python
%%capture
geotiff_from_plot(sums_masked_max, 'masked_Smax', coords, utm_zone, vmin=vmin, vmax=vmax)
```

Save the Smin of the masked subset as a GeoTiff (masked_Smin.tiff):

```python
%%capture
geotiff_from_plot(sums_masked_min, 'masked_Smin', coords, utm_zone, vmin=vmin, vmax=vmax)
```

Save the change magnitude of the masked subset as a GeoTiff (masked_Sdiff.tiff):

```python
%%capture
geotiff_from_plot(change_mag, 'masked_Sdiff', coords, utm_zone, vmin=vmin, vmax=vmax)
```

Perform bootstrapping::

```python
random_index = np.random.permutation(residuals_masked.shape[0])
residuals_random = residuals_masked[random_index,:,:]
n_bootstraps = 2000  # bootstrap sample size
# to keep track of the maxium Sdiff of the bootstrapped sample:
change_mag_random_max = np.ma.copy(change_mag)
change_mag_random_max[~change_mag_random_max.mask] = 0
# to compute the Sdiff sums of the bootstrapped sample:
change_mag_random_sum = np.ma.copy(change_mag)
change_mag_random_sum[~change_mag_random_max.mask] = 0
# to keep track of the count of the bootstrapped sample
qty_change_mag_above_random = change_mag_random_sum
qty_change_mag_above_random[~qty_change_mag_above_random.mask] = 0
print("Running Bootstrapping for %4.1f iterations ..." % (n_bootstraps))
for i in range(n_bootstraps):
    # For efficiency, we shuffle the time axis index and use that
    #to randomize the masked array
    random_index = np.random.permutation(residuals_masked.shape[0])
    # Randomize the time step of the residuals
    residuals_random = residuals_masked[random_index, :, :]
    sums_random = np.ma.cumsum(residuals_random, axis=0)
    sums_random_max = np.ma.max(sums_random, axis=0)
    sums_random_min = np.ma.min(sums_random, axis=0)
    change_mag_random = sums_random_max - sums_random_min
    change_mag_random_sum += change_mag_random
    change_mag_random_max[np.ma.greater(change_mag_random, change_mag_random_max)] = \
    change_mag_random[np.ma.greater(change_mag_random, change_mag_random_max)]
    qty_change_mag_above_random[np.ma.greater(change_mag, change_mag_random)] += 1
    if ((i+1)/n_bootstraps*100)%10 == 0:
        print("\r%4.1f percent completed ..." % ((i+1)/n_bootstraps*100), end='\r', flush=True)
print(f"Bootstrapping Complete.      ")
```

  6.6 Extract Confidence Metrics and Select Final Change Points:

We first compute for all pixels the confidence level $CL$, the change point significance metric $CP_{significance}$ and the product of the two as our confidence metric for identified change points. Plot and save them (confidence_change_point.png):

```python
confidence_level = qty_change_mag_above_random / n_bootstraps
change_point_significance = 1.0 - (change_mag_random_sum/n_bootstraps) / change_mag
#Plot
fig, ax = plt.subplots(1, 3 ,figsize=(16, 4))
a = ax[0].imshow(confidence_level*100)
fig.colorbar(a, ax=ax[0])
ax[0].set_title('Confidence Level %')
a = ax[1].imshow(change_point_significance)
fig.colorbar(a, ax=ax[1])
ax[1].set_title('Change Point Significance')
a = ax[2].imshow(confidence_level*change_point_significance)
fig.colorbar(a, ax=ax[2])
_ = ax[2].set_title('Confidence Level\nx\nChange Point Significance')
plt.savefig('confidence_change_point', dpi=300, transparent='true')
```

Save the confidence level of the masked subset as a GeoTiff (confidence_level.tiff):

```python
%%capture
geotiff_from_plot(confidence_level*100, 'confidence_level', coords, utm_zone)
```

Save the change point significance of the masked subset as a GeoTiff (change_point.tiff):

```python
%%capture
geotiff_from_plot(change_point_significance, 'change_point', coords, utm_zone)
```

Save the confidence level x change point significance of the masked subset as a GeoTiff (CL_x_CP.tiff):

```python
%%capture
geotiff_from_plot(confidence_level*change_point_significance, 'CL_x_CP', coords, utm_zone)
```

Set a change point threshold of 0.5:

```python
change_point_threshold = 0.5
```

Create and save a plot showing the final change points (change_point_thresh.png):

```python
fig = plt.figure(figsize=(12, 8))
ax = fig.add_subplot(1, 1, 1)
plt.title('Detected Change Pixels based on Threshold %2.1f' % (change_point_threshold))
a = ax.imshow(confidence_level*change_point_significance < change_point_threshold, cmap='cool')
plt.savefig('change_point_thresh', dpi=300, transparent='true')
```

Save the thresholded change point significance of the masked subset as a GeoTiff (change_point_thresh.tiff):

```python
%%capture
geotiff_from_plot(confidence_level*change_point_significance < change_point_threshold, 'change_point_thresh', coords, utm_zone, cmap='cool')
```

  6.7 Derive Timing of Change for Each Change Pixel:

Our last step in the identification of the change points is to extract the timing of the change. We will produce a raster layer that shows the band number of the first date after a change was detected. We will make use of the numpy indexing scheme.

Create a combined mask of the first threshold and the identified change points after the bootstrapping:

```python
# make a mask of our change points from the new threhold and the previous mask
change_point_mask = np.ma.mask_or(confidence_level*change_point_significance<change_point_threshold,
                                  confidence_level.mask)
# Broadcast the mask to the shape of the masked S curves
change_point_mask2 = np.broadcast_to(change_point_mask, sums_masked.shape)
# Make a numpy masked array with this mask
change_point_raster = np.ma.array(sums_masked.data, mask=change_point_mask2)
```

To retrieve the dates of the change points we find the band indices in the time series along the time axis where the maximum of the cumulative sums was located. Numpy offers the "argmax" function for this purpose.

```python
change_point_index = np.ma.argmax(change_point_raster, axis=0)
change_indices = list(np.unique(change_point_index))
change_indices.remove(0)
print(f"Change Indicies:\n{change_indices}")
# Look up the dates from the indices to get the change dates
all_dates = time_index[time_index>'2015-10-31']
change_dates = [str(all_dates[x].date()) for x in change_indices]
print(f"\nChange Dates:\n{change_dates}")
```

Lastly, we plot and the change dates using the change point index raster and labeling the change dates (change_dates.png):

```python
ticks = change_indices
tick_labels = change_dates

cmap = plt.cm.get_cmap('tab20',ticks[-1])
fig, ax = plt.subplots(figsize=(12, 12))
cax = ax.imshow(change_point_index, interpolation='nearest', cmap=cmap)
# fig.subplots_adjust(right=0.8)
# cbar_ax = fig.add_axes([0.85, 0.15, 0.05, 0.7])
# fig.colorbar(p,cax=cbar_ax)

ax.set_title('Dates of Change')
# cbar = fig.colorbar(cax, ticks=ticks)
cbar = fig.colorbar(cax, ticks=ticks, orientation='horizontal')
_ = cbar.ax.set_xticklabels(tick_labels, size=10, rotation=45, ha='right')
plt.savefig('change_dates', dpi=300, transparent='true')
```

Save the dates of change plot as a GeoTiff (change_dates.tiff):

```python
%%capture
geotiff_from_plot(change_point_index, 'change_dates', coords, utm_zone, interpolation='nearest', cmap=cmap)
```

 GEOS 657-Lab8-SARTSChangeDetection.ipynb - Version 1.3.0 - April 2021

        Version Changes:

namespace asf_notebook


# Quantum Ensemble for Classification

## Antonio Macaluso

#### Department of Computer Science and Engineering, University of Bologna

#### preprint arXiv:2007.01028

<link rel="stylesheet" href=reveal.js/css/theme/sky.css" id="theme>

## Contribution

* A novel quantum algorithm to perform  ensemble classification

* Superposition and Entanglement to generate a large number of different models

* Large ensembles feasible with relatively small circuits

* Experiments using qiskit

# Classical Ensemble Learning

## Supervised Machine Learning as Aggregator of Functions

Standard formulation for supervised ML:

\begin{align}
            y = f(x; \Theta) +  \epsilon.
\end{align}

where $x \in \mathbb{R}^p$ and $y$ is the target variable

Objectives:

* Useful approximation $\hat{f}(x; \cdot)$

* Optimal set of parameters $\Theta$

Alternative formulation:
\begin{align}
            \hat{f}(x;\beta, \theta) & = \sum_{i=1}^{B} \beta_i \hspace{0.2em} g(x;\theta_i) %= E_{\beta,\theta}[g(x; \cdot)]
\end{align}

where  $g(x; \cdot)$ is the base model and $\theta = \{ \theta_i\}_{i=1, \dots, B}$ and $\beta = \{ \beta_i\}_{i=1, \dots, B}$

## Ensemble Learning (Bagging)

+ $g(x, \cdot)$ is a ML model (under/over-fitting)

+ Fit the same model under different and independent training conditions

+ Necessary and sufficient conditions: the base models **accurate** and **diverse**

Ensemble prediction:
\begin{align}
    f_{\text{ens}}(x) = \frac{1}{B}\sum_{i=1}^B \beta_i \hspace{0.2em} g(x; \theta_i)
\end{align}

where $g(x;\cdot)$ is the base model

Approaches: *Bagging, Boosting, Randomisation, Stacking*

## Example of Bagging: Random Forest

Bagging averages the predictions over a collection of bootstrap samples

Random Forest uses a decision tree as based model

\begin{align}
            \hat{f}^B_{\text{rf}}(x) = \frac{1}{B} \sum_{i=1}^B  g(x; \theta_i)
            =
            \frac{1}{B} \sum_{i=1}^B \left[ \sum_{m=1}^{M} c^{(i)}_{m} \boldsymbol{I}_{\left(x\in R^{(i)}_m \right)}\right]
\end{align}

$\beta_i = \frac{1}{B}$ for $i=1, \dots, B$

$\theta_i$ characterises the $b$th tree in terms of split variables, cutpoints, terminal-node values

*Friedman Jerome, Trevor Hastie, and Robert Tibshirani. The elements of statistical learning. Vol. 1. No. 10. New York: Springer series in statistics, 2001*

## Time Complexity of Classical Ensemble (Bagging)

Classical Ensemble scaling:

\begin{align*}
            \underbrace{\mathcal{O}\left(B \times N^{a}p^{b}\right)}_\text{Training} + \underbrace{\mathcal{O}(Bp)}_\text{Testing} \qquad a, b \geq 1,
        \end{align*}

where $N$ is the number of training points and $p$ is the number of features

- Linearity in $B$ (ensemble size)

- Multiplicative cost of the classifier

$$\newcommand{\ket}[1]{\left|{#1}\right\rangle}$$
$$\newcommand{\bra}[1]{\left\langle{#1}\right|}$$
$$\newcommand{\braket}[1]{\left\langle{#1}\right\rangle}$$

# Quantum Algorithm for Ensemble Learning

## Quantum Computing Properties
Superposition
+ $n$ qubits $\xrightarrow{}$ $2^n$ states $\implies$ $n+1$ qubits $\xrightarrow{}$ $2^{n+1}$ states
+ single model/prediction $\rightarrow$ single quantum state

Entanglement
+ Each base model $g(x; \cdot)$ is strictly associated to a single weight

Interference
+ Train and Test "interact" each other to compute test prediction

## Quantum Ensemble: requirements

- Quantum gate for sampling:                             $\ket{x,y} \xrightarrow{V_b}\ket{x_b,y_b}$

- Quantum classifier $F$ (working via interference):    $\ket{x_b,y_b}\ket{\tilde{x}}\ket{0}\xrightarrow{F} \ket{x_b,y_b}\ket{\tilde{x}}\ket{f_b}$

Input:
- $d$-qubits $control$ register
- $data$ register encodes the training set
- $test$ register encodes the test set

Output:
+ The average of $B=2^d$ different predictions of the same classifier

## Quantum Ensemble using Bagging strategy

+ **(Step 1) State Preparation**

+ **(Step 2) Sampling in Superposition**

+ **(Step 3) Learning via Interference**

+ **(Step 4) Measurement**

## Step 1: State Preparation
 * The training set is encoded into the $data$ quantum register

 * The $control$ register is initialised into uniform superposition

\begin{align}
            \ket{\Phi_0} & = \big(W \otimes S_{(x,y)}\big)\overset{d}{\underset{j=1}{\otimes}} \ket{0} \otimes \ket{0} \nonumber \\
  & = \overset{d}{\underset{j=1}{\otimes}} \left(\frac{\ket{0}+\ket{1}}{\sqrt{2}}\right) \otimes \ket{x,y} =  \frac{1}{\sqrt{2^d}}\sum_{b=1}^{2^d} \ket{b} \otimes \ket{x,y},
        \end{align}

$S_{(x,y)}$ is the unitary that encodes data into a quantum state

$W=H^{\otimes d}$ is the Walsh-Hadamard gate

## Step 2: Sampling in Superposition

$2^d$ different transformations (*quantum trajectories*) of the input in superposition

Entanglement between $data$ and $control$ registers
\begin{align}
        \ket{\Phi_{d}}
        = & \frac{1}{\sqrt{2^d}} \sum_{b = 1}^{2^{d}} \ket{b} V_b\ket{x,y} = \frac{1}{\sqrt{2^d}} \sum_{b = 1}^{2^{d}} \ket{b}\ket{x_b,y_b}
\end{align}

Each $V_b$ represents a single "quantum trajectory"

\begin{equation}\label{eq:single_subsample}
  \ket{x,y} \xrightarrow{V_{b}} \ket{x_b, y_b} %\text{where} \quad U_{t_1}U_{t_2}\dots U_{t_q} = V_t
\end{equation}

$V_b$ transforms the original training set to obtain a random sub-sample of it

## Step 3: Learning via Interference

The *test* register is initialised to encode the test set, considering also a set of qubits to store the final predictions ($target$ sub-register)
\begin{align}
    (S_{\tilde{x}} \otimes \mathbb{1}) \ket{0}\ket{0} =\ket{\tilde{x}}\ket{0}
\end{align}

The $data$ and $test$ registers interact via interference to compute the target variable:
\begin{align}\label{eq:classification via interference}
\ket{\Phi_{f}} & = \Big(\mathbb{1}^{\otimes d} \otimes F \Big) \ket{\Phi_d} = (\mathbb{1}^{\otimes d} \otimes F )\Bigg[\frac{1}{\sqrt{2^d}}\sum_{b=1}^{2^d} \ket{b} \ket{x_b, y_b}\Bigg] \otimes
                        \ket{\tilde{x}}
                        \ket{0}  \nonumber \\
                        & =  \frac{1}{\sqrt{2^d}}\sum_{b=1}^{2^d} \ket{b} \ket{x_b, y_b} \ket{\tilde{x}}\ket{f_b}
            %CV_{t}\ket{\psi} = CV_{t}\ket{x, y}\ket{0} \ket{x^{(\text{test})}, 0} = C \ket{x, y}\ket{x_t, x_y} \ket{x^{(\text{test})}, 0} = \ket{x, y}\ket{x_t, x_t} \ket{x^{(\text{test})}, \tilde{y}} %= \ket{x, y, D_t, x^{(\text{test})}, f(x^{(\text{test})}|x_t,y_t)}
        \end{align}

where $\tilde{y} = f_{b}$ is computed through the quantum gate $F$

A single execution of $F$ allows propagating the use of the classifier to all $2^d$ sub-samples

## Step 4: Measurement

The expectation $measurement$ on the target qubit provides a sum of expectation values:

\begin{align}
    \left\langle M \right\rangle & =  \braket{\Phi_f|\mathbb{1}^{\otimes d} \otimes \mathbb{1} \otimes \mathbb{1} \otimes M|\Phi_f} \nonumber \\
    & =
    \frac{1}{2^d}\sum_{b=1}^{2^d}\braket{b|b} \otimes  \braket{(x_b,y_b)|(x_b,y_b)} \otimes\braket{\tilde{x}|\tilde{x}} \otimes\braket{f_b|M|f_b}  \nonumber \\
    & =  \frac{1}{2^d}\sum_{b=1}^{2^d}\left\langle M_b \right\rangle = \frac{1}{B} \sum_{b=1}^B \hat{f}_b = \hat{f}_{bag}(\tilde{x}|x,y)
\end{align}
E.g.

$$f_{bag} = \sqrt{a_0}\ket{0}+\sqrt{a_1}\ket{1} $$

# Quantum Circuit

## Quantum Circuit for Ensemble

The generic $i$-th step involves the following transformations:

\begin{align}
        \ket{\Phi_{i,1}} = \frac{1}{\sqrt{2}}\Big(\ket{0}\ket{x,y} + \ket{1} U_{(i,1)}\ket{x,y} \Big)
\end{align}

\begin{align}
    \ket{\Phi_{i,2}} = & ( X \otimes \mathbb{1}) \ket{\Phi_{i,1}}
    = \frac{1}{\sqrt{2}}\Big(\ket{1}\ket{x,y} + \ket{0} U_{(i,1)}\ket{x,y} \Big)
\end{align}

\begin{align}
    \ket{\Phi_{i}}= \frac{1}{\sqrt{2}}\Big(\ket{1}U_{(i,2)}\ket{x,y} + \ket{0} U_{(i,1)}\ket{x,y} \Big)
    \end{align}

## Sampling in superposition steps
The output of the State preparation is the following:
    $\ket{\Phi_0} = \overset{d}{\underset{i=1}{\otimes}} \left(\frac{\ket{0}+\ket{1}}{\sqrt{2}}\right) \otimes \ket{x,y}$

### Sampling in superposition steps ($1st$ step)

\begin{align}
    \ket{\Phi_{1}} = \overset{d-1}{\underset{i=1}{\otimes}} \left(\frac{\ket{0}+\ket{1}}{\sqrt{2}}\right) \otimes \frac{1}{\sqrt{2}}\Big(\ket{1}U_{(1,2)}\ket{x,y} + \ket{0} U_{(1,1)}\ket{x,y} \Big)    \end{align}

### Sampling in superposition steps ($2nd$ step)

\begin{align}
    \ket{\Phi_{2}}
            =  \overset{d-2}{\underset{i=1}{\otimes}} \left(\frac{\ket{0}+\ket{1}}{\sqrt{2}}\right) \otimes \frac{1}{2}\Big[
             \hspace{.2em} &\ket{00} U_{(2,1)}U_{(1,1)}\ket{x,y} +
            %\nonumber \\ + &
            \ket{01} U_{(2,1)}U_{(1,2)}\ket{x,y} +
            \nonumber \\ + &
            \ket{10} U_{(2,2)}U_{(1,1)}\ket{x,y} +
            %\nonumber \\ + &
            \ket{11} U_{(2,2)}U_{(1,2)}\ket{x,y}
                     \Big]  \nonumber \\
             \hspace{-10.75em} = \frac{1}{\sqrt{4}} \sum_{b=1}^{4} \ket{b} V_b\ket{x,y}
        \end{align}

where each $V_b$ is the product of $2$ unitaries $U_{(i,j)}$ for $j=1,2$ and $i=1,2$

### Sampling in Superposition ($3rd$ step)

\begin{align}
    \ket{\Phi_{3}}
    =  \overset{d-3}{\underset{i=1}{\otimes}} \left(\frac{\ket{0}+\ket{1}}{\sqrt{2}}\right) \otimes \frac{1}{\sqrt{8}}\Big[
    & \ket{000} U_{(3,1)}U_{(2,1)}U_{(1,1)}\ket{x,y} + \ket{001}U_{(3,1)}U_{(2,1)}U_{(1,2)}\ket{x,y}
    \nonumber \\ + &
     \ket{010} U_{(3,1)}U_{(2,2)}U_{(1,1)}\ket{x,y} + \ket{011}U_{(3,1)}U_{(2,2)}U_{(1,2)}\ket{x,y}
    \nonumber \\ + &
    \ket{100} U_{(3,2)}U_{(2,1)}U_{(1,1)}\ket{x,y} +
    \ket{101} U_{(3,2)}U_{(2,1)}U_{(1,2)}\ket{x,y}
    \nonumber \\ + &
    \ket{110} U_{(3,2)}U_{(2,2)}U_{(1,1)}\ket{x,y} + \ket{111}U_{(3,2)}U_{(2,2)}U_{(1,2)}\ket{x,y}
    \Big] \nonumber \\
 & \hspace{-11.5em} = \frac{1}{\sqrt{2^3}} \sum_{b=1}^{8} \ket{b} V_b\ket{x,y}
\end{align}

where each $V_b$ is the product of $2$ unitaries $U_{(i,j)}$ for $j=1,2$ and $i=1,2,3$

### Sampling in superposition after d steps:

\begin{align}
        \ket{\Phi_{d}}
        & = \frac{1}{\sqrt{2^d}} \sum_{b = 1}^{2^{d}} \ket{b} V_b\ket{x,y} = \frac{1}{\sqrt{2^d}} \sum_{b = 1}^{2^{d}} \ket{b}\ket{x_b,y_b}
\end{align}

where each $V_b$ is the product of $d$ unitaries $U_{(i,j)}$ for $i=1, \cdots, d$ and $j=1,2$.

**$2^d$ different transformations** (*quantum trajectories*) of the input in superposition in only **$d$ steps**

## Time Complexity

Classical Ensemble scaling:

\begin{align*}
            \underbrace{\mathcal{O}\left(B \times N^{a}p^{b}\right)}_\text{Training} + \underbrace{\mathcal{O}(Bp)}_\text{Testing} \qquad a, b \geq 1,
\end{align*}

Quantum Ensemble scaling:

\begin{align*}
         \mathcal{O}(\underbrace{d \times 2 cost({CU_{(i,j)}})}_\text{Sampling in Sup.} + \underbrace{cost(F)}_\text{Learning} )
          %\Big)
\end{align*}

$B =2^d$ is the ensemble size, $N$ is the number of obs., and $p$ is the number of features

Additive cost of the classifier (instead of multiplicative)

## Aggregation Strategy

Majority voting vs Simple averaging

\begin{align}
    f_{\text{avg}}^{(i)}(x) = \frac{1}{B}\sum_{b=1}^B f_b^{(i)}(x),
\end{align}

$B$ is the ensemble size and $f_b^{(i)}(x)$ is the probability for $x$ to be classified in the $i$-th class provided by the $b$-th classifier

Performance:

\begin{align}\label{perf_ens}
            E_{\text{ens}} = \frac{1+\rho(B-1)}{B}E_{\text{model}}
\end{align}
        where $E_{\text{model}}$ is the expected error of the single models and  $\rho$ is the average correlation among their errors

## Theoretical Performance

```python
%matplotlib inline

import sys
sys.path.insert(1, '../')
from Utils import *

# parameters
d = np.arange(6) # Number of control qubits

B = 2**d # Number of models: ensemble size

errs = np.array([0.1, 0.2, 0.3, 0.4]) # base model error

ro = [0.00, 0.25, 0.5] # correlation amontg errors

colors = sns.diverging_palette(220, 20, n=6)

# create figure
fig, ax = plt.subplots(1, figsize=(8,5))

#for each possible ensemble size
for i in np.arange(len(errs)):
    err = errs[i]
    color = colors[i]
    mean = ((1+ro[1]*(B-1))/B)*err
    lower = mean - ((1+ro[0]*(B-1))/B)*err
    upper = ((1 + ro[2] * (B - 1)) / B) * err - mean
    ax.plot(d, mean, lw=2, label = str(int(err*100))+'%', color=color)
    ax.fill_between(d, mean + upper, mean - lower, facecolor=color, alpha=0.5)

# title
ax.set_title('Theoretical performance of Bagging Ensemble', size=14).set_position([.5, 1.05])

# legend
ax.legend(loc='upper right', title = '$E_{model}$', prop=dict(size=12))

# label of x-axis
ax.set_xlabel(r'Number of control qubits $(d=log_2B)$', size=14)

## label of x-axis
ax.set_ylabel('Prediction Error ($E_{ens}$)', size=14)

# set grid
ax.grid(alpha=.3)
```

## Boosting and Randomisation

Quantum Boosting:
    \begin{align} \label{eq:quantum_boosting}
            \ket{\Phi_{f}}
            = \frac{1}{\sqrt{2^d}}\sum_{b=1}^{2^d} \alpha_b \ket{b} \ket*{f_b}
        \end{align}

where the contribution of $f_b$ to the ensemble is weighted by $\alpha_b$

Quantum Randomisation:

\begin{align}
    \ket{\Phi_{f}}
    =\frac{1}{\sqrt{2^d}}\sum_{b=1}^{2^d}\ket{b}  G_b  \ket{x,y}\ket{\tilde{x}}\ket{0} = \frac{1}{\sqrt{2^d}}\sum_{b=1}^{2^d} \ket{b} \ket{x,y}\ket{\tilde{x}}\ket{f_b}
\end{align}

where $G_b=V_b F$ is the quantum classifier

## (Theoretical) Advantages and Limitations

+
$2^d$ different transformations of the input in $d$ steps: exponential speed-up (+)

+ Quantum parallelism allows propagation of $F$ to all quantum trajectories (+)

+ Independence from the encoding strategy chosen for data (+)

+ Measurement is limited to a subset of qubits (+)

- Evaluation of the single functions implies exponential cost (-)

- Additional cost of state preparation (-)


# 3D Visualization of Molecules with Plotly
!pip3 install sympy --user

```python
import numpy as np
import pandas as pd

import plotly.graph_objs as go
from plotly.offline import init_notebook_mode, iplot
from sympy.geometry import Point3D

structures = pd.read_csv('../input/champs-scalar-coupling/structures.csv')

# initiate the plotly notebook mode
init_notebook_mode(connected=True)

def plot_molecule(molecule_name, structures_df):
    """Creates a 3D plot of the molecule"""

    atomic_radii = dict(C=0.77, F=0.71, H=0.38, N=0.75, O=0.73)
    cpk_colors = dict(C='black', F='green', H='white', N='blue', O='red')

    molecule = structures_df[structures_df.molecule_name == molecule_name]
    coordinates = molecule[['x', 'y', 'z']].values
    x_coordinates = coordinates[:, 0]
    y_coordinates = coordinates[:, 1]
    z_coordinates = coordinates[:, 2]
    elements = molecule.atom.tolist()
    radii = [atomic_radii[element] for element in elements]

    def get_bonds():
        """Generates a set of bonds from atomic cartesian coordinates"""
        ids = np.arange(coordinates.shape[0])
        bonds = dict()
        coordinates_compare, radii_compare, ids_compare = coordinates, radii, ids

        for _ in range(len(ids)):
            coordinates_compare = np.roll(coordinates_compare, -1, axis=0)
            radii_compare = np.roll(radii_compare, -1, axis=0)
            ids_compare = np.roll(ids_compare, -1, axis=0)
            distances = np.linalg.norm(coordinates - coordinates_compare, axis=1)
            bond_distances = (radii + radii_compare) * 1.3
            mask = np.logical_and(distances > 0.1, distances <  bond_distances)
            distances = distances.round(2)
            new_bonds = {frozenset([i, j]): dist for i, j, dist in zip(ids[mask], ids_compare[mask], distances[mask])}
            bonds.update(new_bonds)
        return bonds

    def atom_trace():
        """Creates an atom trace for the plot"""
        colors = [cpk_colors[element] for element in elements]
        markers = dict(color=colors, line=dict(color='lightgray', width=2), size=7, symbol='circle', opacity=0.8)
        trace = go.Scatter3d(x=x_coordinates, y=y_coordinates, z=z_coordinates, mode='markers', marker=markers,
                             text=elements, name='')
        return trace

    def bond_trace():
        """"Creates a bond trace for the plot"""
        trace = go.Scatter3d(x=[], y=[], z=[], hoverinfo='none', mode='lines',
                             marker=dict(color='grey', size=7, opacity=1))
        for i, j in bonds.keys():
            trace['x'] += (x_coordinates[i], x_coordinates[j], None)
            trace['y'] += (y_coordinates[i], y_coordinates[j], None)
            trace['z'] += (z_coordinates[i], z_coordinates[j], None)
        return trace

    bonds = get_bonds()

    zipped = zip(range(len(elements)), x_coordinates, y_coordinates, z_coordinates)
    annotations_id = [dict(text=num, x=x, y=y, z=z, showarrow=False, yshift=15)
                   for num, x, y, z in zipped]

    annotations_length = []
    for (i, j), dist in bonds.items():
        p_i, p_j = Point3D(coordinates[i]), Point3D(coordinates[j])
        p = p_i.midpoint(p_j)
        annotation = dict(text=dist, x=float(p.x), y=float(p.y), z=float(p.z), showarrow=False, yshift=15)
        annotations_length.append(annotation)

    updatemenus = list([
        dict(buttons=list([
                 dict(label = 'Atom indices',
                      method = 'relayout',
                      args = [{'scene.annotations': annotations_id}]),
                 dict(label = 'Bond lengths',
                      method = 'relayout',
                      args = [{'scene.annotations': annotations_length}]),
                 dict(label = 'Atom indices & Bond lengths',
                      method = 'relayout',
                      args = [{'scene.annotations': annotations_id + annotations_length}]),
                 dict(label = 'Hide all',
                      method = 'relayout',
                      args = [{'scene.annotations': []}])
                 ]),
                 direction='down',
                 xanchor = 'left',
                 yanchor = 'top'
            ),
    ])

    data = [atom_trace(), bond_trace()]
    axis_params = dict(showgrid=False, showticklabels=False, zeroline=False, titlefont=dict(color='white'))
    layout = dict(scene=dict(xaxis=axis_params, yaxis=axis_params, zaxis=axis_params, annotations=annotations_id),
                  margin=dict(r=0, l=0, b=0, t=0), showlegend=False, updatemenus=updatemenus)

    fig = go.Figure(data=data, layout=layout)
    iplot(fig)
```

```python
plot_molecule('dsgdb9nsd_133885', structures)
```

```python
plot_molecule('dsgdb9nsd_105227', structures)
```

```python
plot_molecule('dsgdb9nsd_099964', structures)
```

```python
plot_molecule('dsgdb9nsd_133621', structures)
```

```python
plot_molecule('dsgdb9nsd_122001', structures)
```

```python
plot_molecule('dsgdb9nsd_102650', structures)
```

```python
plot_molecule('dsgdb9nsd_100779', structures)
```

```python
plot_molecule('dsgdb9nsd_064272', structures)
```

```python
plot_molecule('dsgdb9nsd_080543', structures)
```

```python
plot_molecule('dsgdb9nsd_109581', structures)
```


```
from IPython.display import HTML
def typeset():
  """MathJax initialization for the current cell.

  This installs and configures MathJax for the current output.
  """
  display(HTML('''

      '''))
```

```
from IPython.display import HTML, Math
def mview(e):
  display(HTML(""))
  return Math(latex(e))
```

Those a

```
from sympy import *
import numpy as np
import math

x = Symbol('x')
y = Symbol('y')

limit(1/x**2,x,0)
```

    oo

```
# Find the limit of Functions
from sympy import Limit, Symbol, S

Limit(1/x, x, S.Infinity)
```

    Limit(1/x, x, oo, dir='-')

```

# To find the value
l = Limit(1/x, x, S.Infinity)
l.doit()
```

    0

```
# Differentiation
diff(15*x**100-3*x**12+5*x-46)
```

    1500*x**99 - 36*x**11 + 5

```
# Constant
diff(99)
```

    0

```

# Multiplication by Constant
diff(3*x)
```

    3

```
# Power Rule
diff(x**3)
```

    3*x**2

```
# Sum Rule
diff(x**2+3*x)
```

    2*x + 3

```
# Difference Rule
diff(x**2-3*x)
```

    2*x - 3

```

# Product Rule
diff(x**2*x)
```

    3*x**2

```
# Chain Rule
diff(ln(x**2))
```

    2/x

```

# Example
diff(9*(x+x**2))
```

    18*x + 9

```
# Matrix Calculus
a = diff(3*x**2*y,x)
b = diff(3*x**2*y,y)
c = diff(2*x+8*y**7,x)
d = diff(2*x+y**8,y)
```

```
print(a)
print(b)
print(c)
print(d)
```

    6*x*y
    3*x**2
    2
    8*y**7

```
import numpy as np
Matrix_Calculus = np.matrix([[a, c], [b,d]])
Matrix_Calculus
```

    matrix([[6*x*y, 2],
            [3*x**2, 8*y**7]], dtype=object)

```

# Chain Rule
mview(diff(ln(sin(x**3)**2)))
```

$$\frac{6 x^{2} \cos{\left (x^{3} \right )}}{\sin{\left (x^{3} \right )}}$$

```

# Sigmoid
# d/dx S(x)=S(x)(1−S(x))
mview(diff(1/(1+math.e**-x),x))
```

$$\frac{1.0 \cdot 2.71828182845905^{- x}}{\left(1 + 2.71828182845905^{- x}\right)^{2}}$$

```

```

```
e = Integral(cos(x)**2, (x, 0, pi))
mview(e)
```

$$\int_{0}^{\pi} \cos^{2}{\left (x \right )}\, dx$$

```

import sympy
sympy.init_printing()
mview(sympy.integrate(2*x, (x, 1, 0)))
```

$$-1$$

```
# Jacobian
from sympy import sin, cos, Matrix
from sympy.abc import rho, phi

X = Matrix([rho*cos(phi), rho*sin(phi), rho**2])
Y = Matrix([rho, phi])

mview(X.jacobian(Y))
```

$$\left[\begin{matrix}\cos{\left (\phi \right )} & - \rho \sin{\left (\phi \right )}\\\sin{\left (\phi \right )} & \rho \cos{\left (\phi \right )}\\2 \rho & 0\end{matrix}\right]$$

```
eq = x**2*y + 3/4*x*y + 10
mview(eq)
```

$$x^{2} y + 0.75 x y + 10$$

```
# Jacobian matrix
x, y, z = symbols('x y z')
mview(Matrix([sin(x) + y, cos(y) + x, z]).jacobian([x, y, z]))
```

$$\left[\begin{matrix}\cos{\left (x \right )} & 1 & 0\\1 & - \sin{\left (y \right )} & 0\\0 & 0 & 1\end{matrix}\right]$$

```

```

```

```

```

```

```
# (x,y)=(0,0)
x1 = -y
x2 = x - 2*y * (2-x**2)
J = sympy.Matrix([x1,x2])
mview(J.jacobian([x,y]))
```

$$\left[\begin{matrix}0 & -1\\4 x y + 1 & 2 x^{2} - 4\end{matrix}\right]$$

```
mview(J.jacobian([x,y]).subs([(x,0), (y,0)]))
```

$$\left[\begin{matrix}0 & -1\\1 & -4\end{matrix}\right]$$

```
# Derivatives of x
de_x = diff(x**2*y + 3/4*x*y + 10, x)
mview(de_x)
```

$$2 x y + 0.75 y$$

```
# Derivatives of y
de_y = diff(x**2*y + 3/4*x*y + 10, y)
mview(de_y)
```

$$x^{2} + 0.75 x$$

```
# Example
F = sympy.Matrix([de_x,de_y])
mview(F.jacobian([x,y]))
```

$$\left[\begin{matrix}2 y & 2 x + 0.75\\2 x + 0.75 & 0\end{matrix}\right]$$

```
# Hessian
from sympy import Function, hessian, pprint
from sympy.abc import x, y
f = Function('f')(x, y)
g1 = Function('g')(x, y)
g2 = x**2+y**2
pprint(hessian(f, (x,y), [g1, g2]))
```

    ⎡                   ∂               ∂            ⎤
    ⎢     0        0    ──(g(x, y))     ──(g(x, y))  ⎥
    ⎢                   ∂x              ∂y           ⎥
    ⎢                                                ⎥
    ⎢     0        0        2⋅x             2⋅y      ⎥
    ⎢                                                ⎥
    ⎢                     2               2          ⎥
    ⎢∂                   ∂               ∂           ⎥
    ⎢──(g(x, y))  2⋅x   ───(f(x, y))   ─────(f(x, y))⎥
    ⎢∂x                   2            ∂y ∂x         ⎥
    ⎢                   ∂x                           ⎥
    ⎢                                                ⎥
    ⎢                     2               2          ⎥
    ⎢∂                   ∂               ∂           ⎥
    ⎢──(g(x, y))  2⋅y  ─────(f(x, y))   ───(f(x, y)) ⎥
    ⎢∂y                ∂y ∂x              2          ⎥
    ⎣                                   ∂y           ⎦

```

import numpy as np
import matplotlib.pyplot as plt

def tanh(x):
    return np.tanh(x)

X = np.linspace(-5, 5, 100)
plt.plot(X, tanh(X),'b')
plt.xlabel('X Axis')
plt.ylabel('Y Axis')
plt.title('Neural Networks - Activation Function')
plt.grid()
plt.text(4, 0.8, r'$\sigma(x)=\tanh{(x)}}$', fontsize=16)
plt.show()
```

```
def sigma(x):
    return (np.exp(x) - np.exp(-x)) / (np.exp(x) + np.exp(-x))

X = np.linspace(-5, 5, 100)
plt.plot(X, sigma(X),'b')
plt.xlabel('X Axis')
plt.ylabel('Y Axis')
plt.title('Sigmoid Function')
plt.grid()
plt.text(4, 0.8, r'$\sigma(x)=\frac{e^{x}-e^{-x}}{e^{x}+e^{-x}}$', fontsize=16)
plt.show()
```

```
def derivatives_sigma(x):
    return 1 / (np.cosh(x))**2

X = np.linspace(-5, 5, 100)
plt.plot(X, derivatives_sigma(X),'b')
plt.xlabel('X Axis')
plt.ylabel('Y Axis')
plt.title('Sigmoid Function')
plt.grid()
plt.text(4, 0.8, r'$\sigma(x)=\frac{1}{cosh^2(x)}$', fontsize=16)
plt.show()
```

```

def derivatives_sigma(x):
    return 4 / (np.exp(x)+np.exp(-x))**2

X = np.linspace(-5, 5, 100)
plt.plot(X, derivatives_sigma(X),'b')
plt.xlabel('X Axis')
plt.ylabel('Y Axis')
plt.title('Sigmoid Function')
plt.grid()
plt.text(4, 0.8, r'$\sigma(x)=\frac{4}{(e^{x}+e^{-x})^2}$', fontsize=16)
plt.show()
```

```
# Newton_Raphson
# f(x) - the function of the polynomial
from sympy import *

def f(x):
    function = x**3 - x - 1
    return function

def derivative(x): #function to find the derivative of the polynomial
    derivative = diff(f(x), x)
    return derivative

def Newton_Raphson(x):
	return (x - (f(x) / derivative(x)))

mview(Newton_Raphson(x))
```

$$x - \frac{x^{3} - x - 1}{3 x^{2} - 1}$$

```
# Advanced Chain Rule
# F(y)=ln(1−5y2+y3)
from sympy import *

y = symbols('y')
F = symbols('F', cls=Function)

mview(Derivative(ln(1 - 5*y**2 + y**3)).doit())
```

$$\frac{3 y^{2} - 10 y}{y^{3} - 5 y^{2} + 1}$$

```
mview(diff(ln(1 - 5*y**2 + y**3)))
```

$$\frac{3 y^{2} - 10 y}{y^{3} - 5 y^{2} + 1}$$


```python
import numpy as np
from sympy import divisors, divisor_count, sieve
from tqdm import tqdm
import multiprocessing as mp
import pickle
```

```python
N = 1e8

try:
    prime_set = pickle.load(open('data/prime_set_1e8.pkl', 'rb'))
except FileNotFoundError:
    sieve._reset()
    sieve.extend(N)
    prime_set = set(sieve._list)

is_prime = lambda x: x in prime_set
```

```python
def compute(n):
    ds = divisors(n, generator=True)
    l  = divisor_count(n)

    if l % 2 == 1:
        return False

    for i in range(l//2):
        d = next(ds)
        if not is_prime(d + n//d):
            return False

    return True

def do_batched(batch):
    s = 0
    for n in tqdm(batch):
        if compute(n):
            s += n
    return s
```

```python
batches = np.arange(N, dtype=int).reshape(100, -1).tolist()

pool = mp.Pool(processes=6)
out = pool.map(do_batched, batches)

sum(out)
```


## Constraint Handling

### Inequality Constraints

**If somebody is interested in implementing or willing to make contribution(s) regarding constraint handling of inequality constraints please let us know.
The G problem is suite is already available to experiment with different algorithms. So far, mostly parameter-less constraint handling is used for our algorithms.**

### Equality Constraints

We got a couple of questions of how equality constraints should be handled in a genetic algorithm. In general, functions without any smoothness are challenging to handle for genetic algorithms. An equality constraint is basically an extreme case, where the constraint violation is 0 at exactly one point and otherwise 1.

Let us consider the following constraint $g(x)$ where $x$ represents a variable:

$g(x): x = 5$

An equality constraint can be expressed by an inequality constraint:

$g(x): |x - 5| \leq 0$

or

$g(x): (x-5)^2   \leq 0$

However, all of constraints above are very strict and make **most of the search space infeasible**. Without providing more information to the algorithm those constraint are very difficult to satisfy.

For this reason, the constraint can be smoothed by adding an epsilon to it and, therefore, having two inequality constraints:

$g'(x): 5 - \epsilon \leq x \leq 5 + \epsilon$

Also, it can be simply expressed in one inequality constraint by:

$g'(x): (x-5)^2  - \hat{\epsilon} \leq 0$

Depending on the $\epsilon$ the solutions will be more or less close to the desired value. However, the genetic algorithm does not know anything about the problem itself which makes it difficult to handle and focus the search in the infeasible space.

**Constraint Handling Through Repair**

A simple approach is to handle constraints through a repair function. This is only possible if the equation of the constraint is known. The repair makes sure every solution that is evaluated is in fact feasible. Let us consider the following example where
the equality constraints need to consider more than one variable:

\begin{align}
\begin{split}
\min \;\; & f_1(x) = (x_1^2 + x_2^2) \\
\max \;\; & f_2(x) = -(x_1-1)^2 - x_2^2 \\[1mm]
\text{s.t.} \;\; & g_1(x_1, x_3) : x_1 + x_3 = 2\\[2mm]
& -2 \leq x_1 \leq 2 \\
& -2 \leq x_2 \leq 2 \\
& -2 \leq x_3 \leq 2
\end{split}
\end{align}

We implement the problem using by squaring the term and using an $\epsilon$ as we have explained above. The source code for the problem looks as follows:

```python
import numpy as np

from pymoo.model.problem import Problem

class MyProblem(Problem):

    def __init__(self):
        super().__init__(n_var=3,
                         n_obj=2,
                         n_constr=1,
                         xl=np.array([-2, -2, -2]),
                         xu=np.array([2, 2, 2]))

    def _evaluate(self, x, out, *args, **kwargs):
        f1 = x[:, 0] ** 2 + x[:, 1] ** 2
        f2 = (x[:, 0] - 1) ** 2 + x[:, 1] ** 2
        g1 = (x[:, 0] + x[:, 2] - 2) ** 2 - 1e-5

        out["F"] = np.column_stack([f1, f2])
        out["G"] = g1

```

As you might have noticed the problem has similar characteristics to problem in our getting started.
Before a solution is evaluated a repair function is called. To make sure a solution is feasible, an approach would be to either set $x_3 = 2 - x_1$ or $x_1 = 2 - x_3$. Additionally, we need to consider that this repair might produce a variable to be out of bounds.

```python
from pymoo.model.repair import Repair

class MyRepair(Repair):

    def _do(self, problem, pop, **kwargs):
        for k in range(len(pop)):
            x = pop[k].X
            if np.random.random() < 0.5:

                x[2] = 2 - x[0]
                if x[2] > 2:
                    val = x[2] - 2
                    x[0] += val
                    x[2] -= val

            else:
                x[0] = 2 - x[2]
                if x[0] > 2:
                    val = x[0] - 2
                    x[2] += val
                    x[0] -= val

        return pop
```

Now the algorithm object needs to be initialized with the repair operator and then can be run to solve the problem:

```python
from pymoo.algorithms.nsga2 import NSGA2

algorithm = NSGA2(pop_size=100, repair=MyRepair(), eliminate_duplicates=True)

from pymoo.optimize import minimize
from pymoo.visualization.scatter import Scatter

res = minimize(MyProblem(),
               algorithm,
               ('n_gen', 20),
               seed=1,
               verbose=True)

plot = Scatter()
plot.add(res.F, color="red")
plot.show()
```

In our case it is easy to verify if the constraint is violated or not:

```python
print(res.X[:, 0] + res.X[:, 2])
```

    [2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2.
     2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2.
     2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2.
     2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2. 2.
     2. 2. 2. 2.]

If you would like to compare the solution without a repair you will see how searching only in the feasible space helps:

```python
algorithm = NSGA2(pop_size=100, eliminate_duplicates=True)

res = minimize(MyProblem(),
               algorithm,
               ('n_gen', 20),
               seed=1,
               verbose=True)

plot = Scatter()
plot.add(res.F, color="red")
plot.show()

```

```python
print(res.X[:, 0] + res.X[:, 2])
```

    [2.00206869]

Here in fact the $\epsilon$ term is necessary to find any feasible solution at all.


```python
import open3d as o3d
import numpy as np
import copy
import os
import sys

# monkey patches visualization and provides helpers to load geometries
sys.path.append('..')
import open3d_tutorial as o3dtut
# change to True if you want to interact with the visualization windows
o3dtut.interactive = not "CI" in os.environ
```

# Robust kernels

This tutorial demonstrates the use of robust kernels in the context of outlier rejection. For this particular tutorial, we will be using the ICP (Iterative Closest Point) registration algorithm as the _target_ problem where we want to deal with outliers. Even so, the theory applies to any given optimization problem and not just for ICP. For the moment the robust kernels have been only implemented for the `PointToPlane` ICP.

The notation and some of the kernels implemented in `Open3D` has been inspired by the publication "Analysis of Robust Functions for Registration Algorithms" [\[Babin2019\]](../reference.html#babin2019)

**Note:**

This tutorial and the original implementation of the robust kernels in `Open3D` was contributed by **Ignacio Vizzo** and **Cyrill Stachniss** from the University of Bonn.

## Input Data

The code below reads a source point cloud and a target point cloud from two files. A rough transformation is given.

**Note:**

The initial alignment is usually obtained by a global registration algorithm. See [Global registration](../pipelines/global_registration.rst) for examples.

```python
def draw_registration_result(source, target, transformation):
    source_temp = copy.deepcopy(source)
    target_temp = copy.deepcopy(target)
    source_temp.paint_uniform_color([1, 0.706, 0])
    target_temp.paint_uniform_color([0, 0.651, 0.929])
    source_temp.transform(transformation)
    o3d.visualization.draw_geometries([source_temp, target_temp],
                                      zoom=0.4459,
                                      front=[0.9288, -0.2951, -0.2242],
                                      lookat=[1.6784, 2.0612, 1.4451],
                                      up=[-0.3402, -0.9189, -0.1996])
```

```python
demo_icp_pcds = o3d.data.DemoICPPointClouds()
source = o3d.io.read_point_cloud(demo_icp_pcds.paths[0])
target = o3d.io.read_point_cloud(demo_icp_pcds.paths[1])
trans_init = np.asarray([[0.862, 0.011, -0.507, 0.5],
                         [-0.139, 0.967, -0.215, 0.7],
                         [0.487, 0.255, 0.835, -1.4], [0.0, 0.0, 0.0, 1.0]])
draw_registration_result(source, target, trans_init)
```

## Point-to-plane ICP using Robust Kernels

The standard point-to-plane ICP algorithm [\[ChenAndMedioni1992\]](../reference.html#chenandmedioni1992) minimizes this objective function:

\begin{equation}
E(\mathbf{T}) = \sum_{(\mathbf{p},\mathbf{q})\in\mathcal{K}}\big((\mathbf{p} - \mathbf{T}\mathbf{q})\cdot\mathbf{n}_{\mathbf{p}}\big)^{2},
\end{equation}

where $\mathbf{n}_{\mathbf{p}}$ is the normal of point $\mathbf{p}$ and $\mathcal{K}$ is the correspondence set between the target point cloud $\mathbf{P}$, and source point cloud $\mathbf{Q}$.

If we call $r_i(\mathbf{T})$ is $i^\mathrm{th}$ the residual, for a given pair of correspondences $(\mathbf{p},\mathbf{q})\in\mathcal{K}$ we can rewrite the objective function as.

\begin{equation}
E(\mathbf{T}) = \sum_{(\mathbf{p},\mathbf{q})\in\mathcal{K}}\big((\mathbf{p} - \mathbf{T}\mathbf{q})\cdot\mathbf{n}_{\mathbf{p}}\big)^{2} = \sum_{i=1}^{N} \big({r_i(\mathbf{T})}\big)^2
\end{equation}

The optimization problem above can also be solved by using the iteratively reweighted least-squares (IRLS) approach, which solves a sequence of weighted least squares problems:

\begin{equation}
E(\mathbf{T}) =  \sum_{i=1}^{N} w_i \big({r_i(\mathbf{T})}\big)^2
\end{equation}

### Outlier Rejection with Robust Kernels

The main idea of a robust loss is to downweight large residuals that are assumed to be caused from outliers such that their influence on the solution is reduced. This is achieved by optimizing $E(\mathbf{T})$ as:

\begin{equation}
E(\mathbf{T}) = \sum_{(\mathbf{p},\mathbf{q})\in\mathcal{K}}\rho\big((\mathbf{p} - \mathbf{T}\mathbf{q})\cdot\mathbf{n}_{\mathbf{p}}\big) = \sum_{i=1}^{N} \rho\big({r_i(\mathbf{T})}\big),
\end{equation}

where $\rho(r)$ is also called the robust loss or kernel.

We can see that exists a relation between the optimization formulation in IRLS and the one that uses the robust loss function. By setting the weight $w_i= \frac{1}{r_i(\mathbf{T})}\rho'(r_i(\mathbf{T}))$, we can solve the robust loss optimization problem using the existing techniques for weighted least-squares. Therefore, we can minimize the objective function using Gauss-Newton and determine increments by iteratively solving:

\begin{align}
\left(\mathbf{J}^\top \mathbf{W} \mathbf{J}\right)^{-1}\mathbf{J}^\top\mathbf{W}\vec{r},
\end{align}

where $\mathbf{W} \in \mathbb{R}^{N\times N}$ is a diagonal matrix containing weights $w_i$ for each residual $r_i$

### How to use Robust Kernels in Open3D

`registration_icp` can be called with a parameter `TransformationEstimationPointToPlane(loss)`. Where `loss` is a given loss function(also called robust kernel).

Internally, `TransormationEstimationPointToPlane(loss)` implements a function to compute the weighted residuals and Jacobian matrices of the point-to-plane ICP objective according to the provided robust kernel.

**Note:**

For more details on the ICP algorithm please see [ICP Registration](../pipelines/icp_registration.rst).

## Vanilla ICP vs Robust ICP

To better show the advantages of using robust kernels in the registration, we add some artificially generated gaussian noise to the source point cloud.

```python
def apply_noise(pcd, mu, sigma):
    noisy_pcd = copy.deepcopy(pcd)
    points = np.asarray(noisy_pcd.points)
    points += np.random.normal(mu, sigma, size=points.shape)
    noisy_pcd.points = o3d.utility.Vector3dVector(points)
    return noisy_pcd

mu, sigma = 0, 0.1  # mean and standard deviation
source_noisy = apply_noise(source, mu, sigma)

print("Source PointCloud + noise:")
o3d.visualization.draw_geometries([source_noisy],
                                  zoom=0.4459,
                                  front=[0.353, -0.469, -0.809],
                                  lookat=[2.343, 2.217, 1.809],
                                  up=[-0.097, -0.879, 0.467])
```

### Vanilla ICP

We inspect how the results will look if we use the exact same parameters that were used in the [ICP Registration Tutorial](../pipelines/icp_registration.rst).

```python
threshold = 0.02
print("Vanilla point-to-plane ICP, threshold={}:".format(threshold))
p2l = o3d.pipelines.registration.TransformationEstimationPointToPlane()
reg_p2l = o3d.pipelines.registration.registration_icp(source_noisy, target,
                                                      threshold, trans_init,
                                                      p2l)

print(reg_p2l)
print("Transformation is:")
print(reg_p2l.transformation)
draw_registration_result(source, target, reg_p2l.transformation)
```

### Tuning Vanilla ICP

Given the fact that we are now dealing with gaussian noise, we might try to increase the threshold to search for nearest neighbors with the aim of improving the registration result.

We can see that under these conditions and without a robust kernel, the traditional ICP has no chance to deal with  outliers.

```python
threshold = 1.0
print("Vanilla point-to-plane ICP, threshold={}:".format(threshold))
p2l = o3d.pipelines.registration.TransformationEstimationPointToPlane()
reg_p2l = o3d.pipelines.registration.registration_icp(source_noisy, target,
                                                      threshold, trans_init,
                                                      p2l)

print(reg_p2l)
print("Transformation is:")
print(reg_p2l.transformation)
draw_registration_result(source, target, reg_p2l.transformation)
```

### Robust ICP

Using the same `threshold=1.0` **and** a robust kernel, we can properly register the two point clouds:

```python
print("Robust point-to-plane ICP, threshold={}:".format(threshold))
loss = o3d.pipelines.registration.TukeyLoss(k=sigma)
print("Using robust loss:", loss)
p2l = o3d.pipelines.registration.TransformationEstimationPointToPlane(loss)
reg_p2l = o3d.pipelines.registration.registration_icp(source_noisy, target,
                                                      threshold, trans_init,
                                                      p2l)
print(reg_p2l)
print("Transformation is:")
print(reg_p2l.transformation)
draw_registration_result(source, target, reg_p2l.transformation)
```

**Note:**

For this example we use the **TukeyLoss**, available in `open3d.pipelines.registration.TukeyLoss`. For the parameter `k` we set it to match the std deviation of the noise model $k = \sigma$.
The parameter `k` used in the Robust Kernels it's usually pick to match the standard deviation of the noise model of the input data. In this sense, `k`, is the discriminator between **inlier**/**outlier**. Although this is not always trivial to define in real world data, for synthetic one, it's easy to fix in order to illustrate the benefits of the robust kernels.