## Trailing losses ##

Trailing losses occur for moving objects when their motion during an exposure (or visit) makes them cover an area larger than the PSF. This notebook investigates the SNR losses due to trailing, as expected for LSST (with its 2-snap per visit observations) and the effect of DM source detection on the limiting magnitude expected for a visit.
There are also some visualizations of what moving objects might look like in LSST visits (including the dip or gap that may occur in the trail due to 2 snaps per visit observations), and an estimate of what fraction of NEOs or PHAs may be affected by these trailing losses.

Note that for LSST, each visit is composed of 2 shorter exposures. The pair is used to reject cosmic rays, but will be simply added together for a single visit limiting magnitude equivalent to the combination of the two visits (LSST is sky-noise limited in all bands except u band, which also has a non-negligble read-noise contribution). The spacing between the two exposures is nominally 2 seconds (the readout time), however the shutter requires 1 second to move across the field of view. The shutter is composed of two blades which travel across the fov one after another (e.g. one blade 'opens' the camera, the second blade 'closes' it; for the next exposure, the second blade 'opens', followed by the first blade 'closing' the camera and returning to the starting positions). Thus, the midpoint of any particular exposure varies by 1 second across the fov but the total exposure time is constant. The 'gap' between a pair of exposures in a visit varies from 2 seconds to 4 seconds, depending on location in the fov relative to the shutter movement.

If the shutter is opening L-R then R-L, then a point on the L side of the camera will have an exposure midpoint 1 second earlier than a point on the R side of the camera, will have a 4 second gap between exposures instead of a 2 second gap, and have the midpoint of the second exposure 1 second later than the R side of the camera. This may complicate trailing calculations.

Each object will have the same trail length in the individual exposures, and the same overall 'central' location, but slightly different combined trail length due to variation in length of gap in middle of the visit.

---
Simple example of motion (just to show gap between exposures in a single visit).

```python
import matplotlib.pyplot as plt
%matplotlib inline
import numpy as np
```

```python
# simple motion - no PSF, just showing the 'gap' between snaps
velocity = 0.5 #deg/day
velocity = velocity /24.0 #arcseconds/second
exposuretime = np.arange(0, 15.01, 0.2) #seconds
timesL = np.concatenate([exposuretime, exposuretime+exposuretime.max()+4])
timesR = np.concatenate([exposuretime+1., exposuretime+exposuretime.max()+2.+1])
#print timesL.mean(), timesL.min(), timesL.max()
#print timesR.mean(), timesR.min(), timesR.max()
positionL = velocity * timesL
positionR = velocity * timesR
plt.figure()
plt.plot(positionL, np.zeros(len(positionL))+0.1, 'k.', label='RHS - 2s gap')
plt.plot(positionR, np.zeros(len(positionR))+0.2, 'r.', label='LHS - 4s gap')
plt.ylim(0, 0.3)
plt.xlim(-0.05, None)
plt.xlabel('Arcseconds')
plt.legend(loc='lower right', fontsize='smaller')
plt.axvline(positionL.mean(), color='g', linestyle=':')
print positionL.mean(), positionR.mean()
```

## PSF ##

Now let's add in the seeing distribution, to look at the flux profile of the sources.

Assume the PSF (for a stationary source) is consistent with Kolmogorov turbulence -- for LSST, this means it will actually be a von Karmen profile, but we can approximate this well enough here with a double gaussian. For moving objects, we can create many 'stationary' PSFs, at a series of locations along the trail of the object.

A good description of the PSF is a double-gaussian:
\begin{equation}
p_K(r | \alpha) \, =  \, 0.909 \left( p(r | \alpha) + 0.1 p(r | 2\alpha) \right) \\
p(r | \alpha) \, = \frac{1}{2 \pi \alpha^2} exp( \frac{-r^2}{2 \alpha^2})
\end{equation}

```python
from scipy import integrate

def sumPSF(x, y, flux):
    dx = np.min(np.diff(x))
    dy = np.min(np.diff(y))
    sumVal = integrate.simps(integrate.simps(flux, dx=dx), dx=dy)
    #sumVal = np.trapz(np.trapz(flux, dx=dx),dx=dy)
    return sumVal
```

```python
from scipy import interpolate

def zoomImage(x, y, flux, zoom=[-1, 1, -1, 1], zmax=None, nbins=200.0, pixelize=False, pixelscale=0.2):
    """Zoom in and show the image in region 'zoom'.
    'pixelize' translates x/y into pixels and displays the image as would-be-seen with pixels."""
    if zmax is None:
        zmax = flux.max()
    if pixelize:
        x_pix = x / pixelscale
        y_pix = y / pixelscale
        xg = np.arange(zoom[0], zoom[1]+0.5, 1)
        yg = np.arange(zoom[2], zoom[3]+0.5, 1)
        xgrid, ygrid = np.meshgrid(xg, yg)
        showflux = interpolate.interpn((y_pix, x_pix), flux, (ygrid, xgrid),
                                      method='splinef2d', bounds_error=False, fill_value=0)
        plt.imshow(showflux, extent=zoom, vmin=0, vmax=zmax, origin='lower', interpolation='none',
                  cmap='gray')
        plt.colorbar()
        plt.xlabel('Pixels')
        plt.ylabel('Pixels')
    else:
        nbins = float(nbins)
        binsize = (zoom[1]-zoom[0])/nbins
        xg = np.arange(zoom[0], zoom[1]+binsize, binsize)
        binsize = (zoom[3] - zoom[2])/nbins
        yg = np.arange(zoom[2], zoom[3]+binsize, binsize)
        xgrid, ygrid = np.meshgrid(xg, yg)
        showflux = interpolate.interpn((y, x), flux, (ygrid, xgrid),
                                       method='splinef2d', bounds_error=False, fill_value=0)
        plt.imshow(showflux, extent=zoom, vmin=0, vmax=zmax, origin='lower', interpolation='none', cmap='gray')
        plt.colorbar()
        plt.xlabel('Arcseconds')
        plt.ylabel('Arcseconds')
```

```python
def singleGaussianStationaryPSF(seeing, totalflux=1, xcen=0, ycen=0, stepsize=0.01, alpharad=10.0):
    "Distribute flux across PSF. seeing in arcseconds"
    # Translate 'seeing' FWHM to gaussian 'sigma' (alpha in the equations above)
    alpha = seeing / np.sqrt(8 * np.log(2))
    maxrad = alpha*alpharad
    x = np.arange(0, 2*maxrad, stepsize)
    x = x - x.mean() + xcen
    y = np.arange(0, 2*maxrad, stepsize)
    y = y - y.mean() + ycen
    xi, yi = np.meshgrid(x, y)
    radius = np.sqrt((xi-xcen)**2 + (yi-ycen)**2)
    p = 1.0/(2.0*np.pi*alpha**2) * np.exp(-radius**2/(2.0*alpha**2))
    flux = p / sumPSF(x, y, p) * totalflux
    # Flux = flux[y][x], although here it doesn't matter because len(x) = len(y)
    return x, y, flux
```

```python
def stationaryPSF(seeing, totalflux=1, xcen=0, ycen=0, stepsize=0.01, alpharad=10.0):
    "Distribute flux across PSF. seeing in arcseconds"
    # Translate 'seeing' FWHM to gaussian 'sigma' (alpha in the equations above)
    alpha = seeing / np.sqrt(8 * np.log(2))
    maxrad = alpha*alpharad
    x = np.arange(0, 2*maxrad, stepsize)
    x = x - x.mean() + xcen
    y = np.arange(0, 2*maxrad, stepsize)
    y = y - y.mean() + ycen
    xi, yi = np.meshgrid(x, y)
    radius = np.sqrt((xi-xcen)**2 + (yi-ycen)**2)
    p1 = 1.0/(2.0*np.pi*alpha**2) * np.exp(-radius**2/(2.0*alpha**2))
    p2 = 1.0/(2.0*np.pi*(2*alpha)**2) * np.exp(-radius**2/(2.0*(2*alpha)**2))
    p = 0.909*(p1 + 0.1*p2)
    flux = p / sumPSF(x, y, p) * totalflux
    # Flux = flux[y][x], although here it doesn't matter because len(x) = len(y)
    return x, y, flux
```

```python
def crossSection(x, y, flux, xi=None, yi=None):
    """Take a cross-section at xi/yi and return the flux values."""
    if xi is None:
        xi = np.mean(x)
    if yi is None:
        yi = np.mean(y)
    # Find closest point in x/y arrays.
    xindx = np.argmin(np.abs(x-xi))
    yindx = np.argmin(np.abs(y-yi))
    fluxx = flux[yindx][:]
    fluxy = np.swapaxes(flux, 0, 1)[xindx][:]
    return fluxx, fluxy
```

Check these tools out with stationary PSF.

```python
x, y, flux = stationaryPSF(0.7, totalflux=1)
sumPSF(x, y, flux)
```

    1.0000000000000002

```python
print sumPSF(x, y, flux)
plt.figure()
zoomImage(x, y, flux)
plt.title('0.7" seeing')
plt.figure()
fluxx, fluxy = crossSection(x, y, flux)
x1 = x
fx1 = fluxx
x, y, flux = stationaryPSF(0.7, xcen=1)
print sumPSF(x, y, flux)
plt.figure()
zoomImage(x, y, flux)
plt.title('0.7" seeing')
plt.figure()
fluxx, fluxy = crossSection(x, y, flux, xi=1)
x2 = x
fx2 = fluxx
y2 = y
fy2 = fluxy

x, y, flux = stationaryPSF(1.0, totalflux=1)
print sumPSF(x, y, flux)
plt.figure()
zoomImage(x, y, flux, zmax=1.6)
plt.title('1.0" seeing')
plt.figure()
fluxx, fluxy = crossSection(x, y, flux)
plt.plot(x1, fx1, 'r')
plt.plot(x2, fx2, 'r:')
plt.plot(y2, fy2, 'g:')
plt.plot(x, fluxx, 'b')
plt.xlim(-2, 2)
plt.xlabel('Arcseconds')
```

### Moving object PSF ###

Now simulate a moving object as a series of stationary PSFs, summing all the flux vlaues contributed by each stationary PSF.

```python
def movingPSF(velocity=1.0, seeing=0.7, totalflux=1., side='L'):
    "Simulate a moving object;  velocity (deg/day), seeing(arcsecond), side='L' or 'R' (L=4sec gap)"""
    velocity = velocity / 24.0 #arcsecond/second
    exposureTimeSteps = seeing/velocity/20.0
    exposuretime = np.arange(0, 15+exposureTimeSteps/2.0, exposureTimeSteps) #seconds
    timesL = np.concatenate([exposuretime, exposuretime+exposuretime.max()+4])
    timesR = np.concatenate([exposuretime+1., exposuretime+exposuretime.max()+2.+1])
    positionL = velocity * timesL
    positionR = velocity * timesR
    xlist = []
    ylist = []
    fluxlist = []
    if side=='L':
        positions = positionL
    else:
        positions = positionR
    for p in (positions):
        xcen = p
        x, y, flux = stationaryPSF(seeing, xcen=xcen, ycen=0)
        xlist.append(x)
        ylist.append(y)
        fluxlist.append(flux)

    xmin = np.array([x.min() for x in xlist]).min()
    xmax = np.array([x.max() for x in xlist]).max()
    ymin = np.array([y.min() for y in ylist]).min()
    ymax = np.array([y.max() for y in ylist]).max()
    stepsize = 0.01 #arcseconds
    x = np.arange(xmin, xmax+stepsize, stepsize)
    y = np.arange(ymin, ymax+stepsize, stepsize)
    xgrid, ygrid = np.meshgrid(x, y)
    flux = np.zeros(np.shape(xgrid), float)
    for xi, yi, fi in zip(xlist, ylist, fluxlist):
        f = interpolate.interpn((yi, xi), fi, (ygrid, xgrid), bounds_error=False, fill_value=0)
        flux += f
    fluxSum = sumPSF(x, y, flux)
    flux = flux / fluxSum * totalflux
    return x, y, flux
```

```python
velocity = 1.0 #deg/day
seeing = 0.7 #arcseconds
x, y, flux = movingPSF(velocity=velocity, seeing=seeing, totalflux=1000.0)
print sumPSF(x, y, flux)
```

    1000.0

```python
zoomImage(x, y, flux, zoom=[-1, 3, -1, 1])
plt.title('Velocity of %.2f deg/day with %.2f" seeing' %(velocity, seeing))
plt.ylabel('4 second gap')
plt.figure()
fluxx, fluxy = crossSection(x, y, flux)
plt.figure()
plt.plot(x, fluxx, 'r', label='Lengthwise')
plt.plot(y, fluxy, 'b', label='Crossection')
plt.legend(loc='upper right', fontsize='smaller', fancybox=True)
plt.xlabel('Arcseconds')
plt.title('Velocity of %.2f deg/day with %.2f" seeing' %(velocity, seeing))
```

Above is with a smooth sub-pixel sampling. At 1 deg/day, on the 4 second gap (L) hand side of the chip, there is a dip in the smoothly sampled flux. However, it's small in width - the tracks are only separated by about 0.1", which is less than a pixel (0.2"/pixel for LSST). We can look at how the flux would appear if it was only sampled at the center of each pixel.

```python
# try pixelizing the flux
pixelscale = 0.2
# 1 deg/day not nyquist sampled; 2 deg/day is definitely visible!
zoom=[-1, 3, -1, 1]
zoompix = [int(z/pixelscale) for z in zoom]
zoomImage(x, y, flux, zoom=zoompix, pixelize=True)
```

```python
x, y, flux = movingPSF(side='R', velocity=velocity, seeing=seeing)
zoomImage(x, y, flux, zoom=[-1, 3, -1, 1])
plt.title('Velocity of %.2f deg/day with %.2f" seeing' %(velocity, seeing))
plt.ylabel('2 second gap')
plt.figure()
fluxx, fluxy = crossSection(x, y, flux)

plt.figure()
plt.plot(x, fluxx, 'r', label='Lengthwise')
plt.plot(y, fluxy, 'b', label='Crossection')
plt.legend(loc='upper right', fontsize='smaller', fancybox=True)
plt.xlabel('Arcseconds')
plt.title('Velocity of %.2f deg/day with %.2f" seeing' %(velocity, seeing))
```

```python
velocity=0.5
seeing=1.0
x, y, flux = movingPSF(side='L', velocity=velocity, seeing=seeing)
zoomImage(x, y, flux, zoom=[-1, 2, -1, 1])
plt.title('Velocity of %.2f deg/day with %.2f" seeing' %(velocity, seeing))
plt.ylabel('4 second gap')
plt.figure()
fluxx, fluxy = crossSection(x, y, flux)
print fluxx.max(), fluxy.max()

plt.figure()
plt.plot(x, fluxx, 'r', label='Lengthwise')
plt.plot(y, fluxy, 'b', label='Center Crossection')
plt.legend(loc='upper right', fontsize='smaller', fancybox=True)
plt.xlabel('Arcseconds')
plt.title('Velocity of %.2f deg/day with %.2f" seeing' %(velocity, seeing))
```

---
## SNR ##

Calculating the SNR primarily comes down to calculating $n_{eff}$.

We can calculate SNR (eqn 41 in SNR doc) (assuming gain=1):
\begin{equation}
SNR = \frac {C_b} {(C_b + n_{eff} (B_b + \sigma_I^2))^{1/2}}
\end{equation}
And equation 45-46 of the SNR doc invert this to:
\begin{equation}
C_b(SNR) = \frac{SNR^2}{2} + \left( \frac{SNR^4}{4}+ SNR^2 V_n \right) ^{1/2} \\
V_n = n_{eff} (B_b + \sigma_I^2) \\
\end{equation}

And then, as shown in eqn 26 of the SNR doc, $n_{eff}$ can be calculated as follows:
\begin{equation}
n_{eff} = \sum_i w_i = \frac{1}{\sum_i p_i^2} \\
\end{equation}

For a double gaussian, this is approximately equivalent to
\begin{equation}
n_{eff} = 2.436 \, (FWHM / pixelscale)^2 \\
\end{equation}
and for the simplification of a single gaussian PSF, it is exactly equivalent to
\begin{equation}
n_{eff} = 2.266 \, (FWHM / pixelscale)^2 \\
\end{equation}

```python
from scipy import integrate

def calcNeff(x, y, psfprofile, pixelscale=0.2):
    # Find dx/dy intervals for integration. They should be uniform based on methods here.
    # numpy says they are, but somehow the diff returns multiple versions ?? (is this the bug?)
    dx = np.max(np.diff(x))
    dy = np.max(np.diff(y))
    # Make sure psfprofile normalizes to 1.
    psfSum = integrate.simps(integrate.simps(psfprofile, dx=dx), dx=dy)
    psfprofile /= psfSum
    # Calculate neff (area), in 'numerical steps'
    neff = 1.0 / integrate.simps(integrate.simps(psfprofile**2, dx=dx), dx=dy)
    # Convert to pixels (the 'neff' above is in arcseconds^2)
    neff = neff / (pixelscale**2)
    return neff
```

```python
# Calculate Neff for stationary sources.
pixelscale = 0.2 #arcseconds/pixel
FWHM = 0.7 #arcseconds  ('seeing')

# For a single gaussian PSF
x, y, flux = singleGaussianStationaryPSF(seeing=FWHM, totalflux=1.0, alpharad=20.0, stepsize=0.01)
neff = calcNeff(x, y, flux, pixelscale=pixelscale)
neff_analytic = 2.266 * (FWHM/pixelscale)**2
print 'Single Gaussian:'
print 'Analytic Neff', neff_analytic
print 'Calculated neff from PSF',  neff
print '% difference:', (neff-neff_analytic)/neff_analytic*100.0

# For a double gaussian PSF
# See note after equation 33 in SNR doc -
#  suggests seeing = 1.035 * FWHM for a double-gaussian.
seeing = FWHM * 1.035
neff_analytic = 2.436 * (seeing/pixelscale)**2
# Calculate Neff from sum(1/p) for each pixel.
x, y, flux = stationaryPSF(seeing=FWHM, totalflux=1.0, alpharad=20.0, stepsize=0.01)
neff = calcNeff(x, y, flux, pixelscale=pixelscale)
print 'Double Gaussian (adjusted FWHM/seeing value):'
print 'Analytic Neff', neff_analytic
print 'Calculated neff from PSF',  neff
print '% difference:', (neff-neff_analytic)/neff_analytic*100.0
```

    Single Gaussian:
    Analytic Neff 27.7585
    Calculated neff from PSF 27.7607058687
    % difference: 0.00794664229063
    Double Gaussian (adjusted FWHM/seeing value):
    Analytic Neff 31.966425225
    Calculated neff from PSF 31.0304425876
    % difference: -2.92801785237

```python
# So calculate Neff for moving sources - example:
velocity = 0.5
seeing = 0.7
x, y, flux = movingPSF(side='L', velocity=velocity, seeing=seeing, totalflux=1.0)
neff = calcNeff(x, y, flux)
print 'Calculated neff from PSF (LHS), velocity %.2f seeing %.1f: %f' %(velocity, seeing, neff)
x, y, flux = movingPSF(side='R', velocity=velocity, seeing=seeing, totalflux=1.0)
neff = calcNeff(x, y, flux)
print 'Calculated neff from PSF (RHS), velocity %.2f seeing %.1f: %f' %(velocity, seeing, neff)
```

    Calculated neff from PSF (LHS), velocity 0.50 seeing 0.7: 38.864290
    Calculated neff from PSF (RHS), velocity 0.50 seeing 0.7: 37.669208

```python
# Calculate totalflux equivalent to (optimally extracted) SNR=5.0 for this range of velocities.
SNR = 5.0
sky = 2000.
inst_noise = 10.0
Vn = neff*(sky + inst_noise)
counts =  SNR**2/2. + np.sqrt(SNR**4/4. + SNR**2 * Vn)
mags = 2.5*np.log10(counts)
# and for a stationary source.
x, y, flux = stationaryPSF(seeing=seeing, totalflux=1.)
neff_stat = calcNeff(x, y, flux)
Vn = neff_stat*(sky + inst_noise)
counts = SNR**2/2. + np.sqrt(SNR**4/4. + SNR**2 * Vn)
mag_stat = 2.5*np.log10(counts)
# Subtract the two to find the magnitude increase required to stay at SNR=5.0 (optimal extraction) as objects trail.
mag_diff = mags - mag_stat
print mag_stat, "=m5 for a stationary source (corresponds to SNR=5)"
print mags, "= m5 for for an object moving", velocity, "deg/day in", seeing, "arcsec seeing"
print "difference: ", mag_diff
```

    7.75202331664 =m5 for a stationary source (corresponds to SNR=5)
    7.85626751764 = m5 for for an object moving 0.5 deg/day in 0.7 arcsec seeing
    difference:  0.104244200996

This result matches closely with the trailing loss formula used in the LSST Overview paper, so it looks about right. This is the inevitable loss in SNR due to the fact that the moving object trails across more sky pixels, thus adding more sky noise into the measurement of its (optimally extracted) source.

---
We also need to look at the effect that results from DM's source detection algorithms. DM will only detect sources which are brighter than some threshhold (currently 5$\sigma$) in a PSF-convolved image. Because our sources are moving, their peak fluxes (convolved with the stationary PSF) will be lower than a similar stationary source.

```python
# Find counts at threshhold for stationary source.
seeing = 0.7
SNR = 5.0
sky = 2000.
inst_noise = 10.0
x, y, flux = stationaryPSF(seeing=seeing, totalflux=1)
neff_stat = calcNeff(x, y, flux)
Vn = neff_stat*(sky + inst_noise)
counts_stat = SNR**2/2. + np.sqrt(SNR**4/4. + SNR**2 * Vn)
x_stat, y_stat, flux_stat = stationaryPSF(seeing=seeing, totalflux=counts_stat)
zoomImage(x_stat, y_stat, flux_stat)
print counts_stat, sumPSF(x_stat, y_stat, flux_stat), flux_stat.sum()
```

```python
# Distribute same counts in moving object.
velocity = 1.0
x_mo, y_mo, flux_mo = movingPSF(seeing=seeing, velocity=velocity, totalflux=counts_stat)
zoomImage(x_mo, y_mo, flux_mo)
print counts_stat, sumPSF(x_mo, y_mo, flux_mo), flux_mo.sum()
```

```python
# Compare the peak brightness of the two (without correlation with PSF)
fx_stat, fy_stat = crossSection(x_stat, y_stat, flux_stat)
fx_mo, fy_mo = crossSection(x_mo, y_mo, flux_mo)
plt.plot(x_stat, fx_stat, 'g')
plt.plot(x_mo, fx_mo, 'r')
plt.xlabel("Arcseconds")
plt.title('Lengthwise cross-section')
print 'max counts for stationary / moving objects:', flux_stat.max(), '/', flux_mo.max()
print 'total flux across stationary object', sumPSF(x_stat, y_stat, flux_stat)
print 'total flux across moving object', sumPSF(x_mo, y_mo, flux_mo)
```

```python
# Generate a PSF profile that we will correlate with the stationary and moving object sources
# (this is the LSST detection filter)
x_psf, y_psf, psfprofile = stationaryPSF(seeing=seeing, totalflux=1.0, stepsize=0.01, alpharad=10.0)
```

```python
from scipy import signal
filtered_stat = signal.fftconvolve(flux_stat, psfprofile)
plt.imshow(filtered_stat, origin='lower')
plt.colorbar()
print 'Max counts in filtered image (~ sum of total flux)', filtered_stat.max()
print 'total flux in original image, simple sum without account for pixel size', flux_stat.sum()
print 'Max counts in original image', flux_stat.max()
```

```python
filtered_mo = signal.fftconvolve(flux_mo, psfprofile)
plt.imshow(filtered_mo, origin='lower')
plt.colorbar()
print 'Max counts in filtered image', filtered_mo.max()
print 'total counts in original image (sum without account for pixel size)', flux_mo.sum()
print 'Max counts in original image', flux_mo.max()
```

```python
# So how much brighter do we have to get as a moving object in order to hit the
#  out_stat.max() value, which is the detection threshhold?
ratio = filtered_stat.max() / filtered_mo.max()
print "increasing counts in moving object by", ratio
dmag = 2.5*np.log10(ratio)
print "equivalent to change in magnitude of", dmag
flux_mo2 = flux_mo * ratio
```

    increasing counts in moving object by 1.55539911197
    equivalent to change in magnitude of 0.479604616668

```python
# Just look at the cross-sections,
# see that even with the increase of 1.55 in flux that we're still below the pix level of the stationary PSF.
# this is because along the 'line' of the velocity, the flux doesn't fall as fast as a stationary PSF
fx_stat, fy_stat = crossSection(x_stat, y_stat, flux_stat)
fx_mo, fy_mo = crossSection(x_mo, y_mo, flux_mo)
fx_mo2, fy_mo2 = crossSection(x_mo, y_mo, flux_mo2)
plt.plot(x_stat, fx_stat, 'g', label='Stationary F=F0')
plt.plot(x_mo, fx_mo, 'r', label='Moving, F=F0')
plt.plot(x_mo, fx_mo2, 'k', label='Moving, F=F0*%.3f' %(ratio))
plt.legend(loc='upper right', fancybox=True, fontsize='smaller')
plt.xlabel("Arcseconds")
plt.title('Lengthwise cross-section')
```

Set up and repeat the experiment for a variety of different seeings, and both sides of the chip.
(warning  - this is pretty slow).

```python
det_loss = {}
trail_loss = {}
seeings = [0.7, 1.0, 1.2]
velocities = np.concatenate([np.arange(0.02, 2.5, 0.2), np.arange(2.8, 4.0, 0.3), np.arange(4.5, 10, 0.5)])
sides = ['L', 'R']
```

```python
# Find counts at threshhold for stationary source.
for side in sides:
    det_loss[side] = {}
    trail_loss[side] = {}
    for seeing in seeings:
        SNR = 5.0
        sky = 2000.  #these values should cancel
        inst_noise = 10.0
        x, y, flux = stationaryPSF(seeing=seeing, totalflux=1)
        neff_stat = calcNeff(x, y, flux)
        Vn = neff_stat*(sky + inst_noise)
        counts_stat = SNR**2/2. + np.sqrt(SNR**4/4. + SNR**2 * Vn)
        x_stat, y_stat, flux_stat = stationaryPSF(seeing=seeing, totalflux=counts_stat)
        # Determine the PSF Profile for convolution (correlation, but we're symmetric)
        x_psf, y_psf, psfprofile = stationaryPSF(seeing=seeing, totalflux=1.0, stepsize=0.01, alpharad=10.0)
        # Calculated the filtered peak value for stationary sources - this is what we have to match.
        filtered_stat = signal.fftconvolve(flux_stat, psfprofile)

        # Calculate how much brighter (than a stationary obj) a moving object has to be to match the
        #     peak level above in PSF filter (rather than moving object filter)
        # And calculate how much brighter (than stationary obj) a moving object has to be to hit SNR=5
        #   even with optimal extraction
        det_loss[side][seeing] = np.zeros(len(velocities), float)
        trail_loss[side][seeing] = np.zeros(len(velocities), float)
        for i, v in enumerate(velocities):
            x, y, flux = movingPSF(seeing=seeing, velocity=v, totalflux=counts_stat, side=side)
            filtered_mo = signal.fftconvolve(flux, psfprofile)
            det_loss[side][seeing][i] = filtered_stat.max() / filtered_mo.max()
            neff = calcNeff(x, y, flux)
            Vn = neff*(sky + inst_noise)
            counts_mo =  SNR**2/2. + np.sqrt(SNR**4/4. + SNR**2 * Vn)
            trail_loss[side][seeing][i] = counts_mo / counts_stat
```

```python
# We have the 'blue curve'= minimum SNR losses due to increased area == 'trail_loss'
# We have the 'red curve' = max detection loss due to detection on PSF-filtered image instead of trailed PSF
# 'diff_loss' == the difference between them (potentially recoverable with increased work by DM)
diff_loss = {}
for side in sides:
    diff_loss[side] = {}
    for seeing in seeings:
        diff_loss[side][seeing] = det_loss[side][seeing]/trail_loss[side][seeing]
```

```python
# red = detection losses due to detection on point-like PSF
# blue = snr losses due to increased area under moving object
# green = ratio between the two (red/blue)
# solid = LHS of focal plane (4s gap), dashed = RHS of focal plane (2s gap)
for side in sides:
    if side == 'L':
        linestyle = '-'
    if side == 'R':
        linestyle = ':'
    plt.figure(1)
    for seeing in seeings:
        plt.plot(velocities, det_loss[side][seeing], color='r', linestyle=linestyle)
        plt.plot(velocities, trail_loss[side][seeing], color='b', linestyle=linestyle)
        plt.plot(velocities, diff_loss[side][seeing], color='g', linestyle=linestyle)
        plt.xlabel('Velocity (deg/day)')
        plt.ylabel('Flux loss (ratio) - SNR loss')

    plt.figure(2)
    for seeing in seeings:
        plt.plot(velocities*30.0/seeing/24.0, det_loss[side][seeing], color='r', linestyle=linestyle)
        plt.plot(velocities*30.0/seeing/24.0, trail_loss[side][seeing], color='b', linestyle=linestyle)
        plt.plot(velocities*30.0/seeing/24.0, diff_loss[side][seeing], color='g', linestyle=linestyle)
        plt.xlabel('x')
        plt.ylabel('Flux loss (ratio) - SNR loss')

    plt.figure(3)
    for seeing in seeings:
        plt.plot(velocities*30.0/seeing/24.0, -2.5*np.log10(det_loss[side][seeing]), color='r', linestyle=linestyle)
        plt.plot(velocities*30.0/seeing/24.0, -2.5*np.log10(trail_loss[side][seeing]), color='b', linestyle=linestyle)
        plt.plot(velocities*30.0/seeing/24.0, -2.5*np.log10(diff_loss[side][seeing]), color='g', linestyle=linestyle)
        plt.xlabel('x')
        plt.ylabel('Delta mag')
```

Fit functions to the detection and trailing losses.  We're looking for something probably like:

\begin{equation}
x = \frac{v T_{exp}} {\theta} \\
flux ratio = \sqrt{1 + a x^2 / (1 + b x)}\\
\end{equation}

```python
from scipy.optimize import curve_fit
from scipy.special import erf, erfc
```

```python
def vToX(v, t, seeing):
    return v * t / seeing / 24.0

def fitfunc(x, c1, c2):
    # x = velocities * t / seeing (/24.0)
    func = np.sqrt(1. + c1*x**2 / (1. + c2 *x))
    return func

def fitfunc2(x, c1, c2):
    func = 1 + c1*x**2 / (1.+c2*x)
    return func

print vToX(1.0, 30, 0.7)
```

    1.78571428571

```python
tExp = 30.0

xall = {}
trailall = {}
detall = {}
diffall = {}
for side in sides:
    # combine the data so that we can fit it all at once.
    xall[side] = []
    detall[side] = []
    trailall[side] = []
    diffall[side] = []
    for s in seeings:
        x = vToX(velocities, tExp, s)
        xall[side].append(x)
        detall[side].append(det_loss[side][s])
        trailall[side].append(trail_loss[side][s])
        diffall[side].append(diff_loss[side][s])
    xall[side] = np.array(xall[side]).flatten()
    detall[side] = np.array(detall[side]).flatten()
    trailall[side] = np.array(trailall[side]).flatten()
    diffall[side] = np.array(diffall[side]).flatten()
    xarg = np.argsort(xall[side])
    detall[side] = detall[side][xarg]
    trailall[side] = trailall[side][xarg]
    xall[side] = xall[side][xarg]
    diffall[side] = diffall[side][xarg]

# Fit the data.
trailab = {}
detab = {}
diffab = {}
for side in sides:
    trailab[side] = {}
    detab[side] = {}
    diffab[side] = {}
    popt, pcov = curve_fit(fitfunc, xall[side], trailall[side])
    trailab[side]['a'] = popt[0]
    trailab[side]['b'] = popt[1]
    popt, pcov = curve_fit(fitfunc, xall[side], detall[side])
    detab[side]['a'] = popt[0]
    detab[side]['b'] = popt[1]
    popt, pcov = curve_fit(fitfunc, xall[side], diffall[side])
    diffab[side]['a'] = popt[0]
    diffab[side]['b'] = popt[1]

# Residuals?
dl = {}
tl = {}
dd = {}
for side in sides:
    dl[side] = fitfunc(xall[side], detab[side]['a'], detab[side]['b'])
    tl[side] = fitfunc(xall[side], trailab[side]['a'], trailab[side]['b'])
    dd[side] = fitfunc(xall[side], diffab[side]['a'], diffab[side]['b'])

# Plot data
for side in sides:
    plt.plot(xall[side], dl[side], 'r-')
    plt.plot(xall[side], tl[side], 'b-')
    plt.plot(xall[side], detall[side], 'r.')
    plt.plot(xall[side], trailall[side], 'b.')
    plt.xlabel('x')
    plt.ylabel('flux loss')

plt.figure()
for side in sides:
    plt.plot(xall[side], dd[side], 'g-')
    plt.plot(xall[side], diffall[side], 'g.')
    plt.xlabel('x')
    plt.ylabel('ratio (det / trail) flux')

# plot diffs.
plt.figure()
for side in sides:
    diff_dl = 2.5*np.log10(detall[side] / dl[side])
    diff_tl = 2.5*np.log10(trailall[side] / tl[side])
    eps = 1e-20
    diff_dd = 2.5*np.log10(diffall[side] / dd[side])

    plt.plot(xall[side], diff_dl, 'r-')
    plt.plot(xall[side], diff_tl, 'b-')
    plt.xlabel('x')
    plt.ylabel('$\Delta$ (mag_calc - mag_fit)')

    plt.plot(xall[side], diff_dd, 'g-')
    plt.xlabel('x')
    plt.ylabel('$\Delta$ (mag_calc - mag_fit)')
```

Fitting the trailing losses with a simple polynomial seems to do slightly better, but results in a more complicated result. A third order polynomial is not enough to capture the wiggles near x=0, so deg=4 it is. Actually, deg=5 does better at not adding additional losses near velocity=0, which is an important area.

```python
trailp = {}
detp = {}
diffp = {}
deg = 5
for side in sides:
    trailp[side] = np.polyfit(xall[side], trailall[side], deg=deg)
    detp[side] = np.polyfit(xall[side], detall[side], deg=deg)
    diffp[side] = np.polyfit(xall[side], diffall[side], deg=deg)

# Residuals?
dl = {}
tl = {}
dd = {}
for side in sides:
    dl[side] = np.polyval(detp[side], xall[side])
    tl[side] = np.polyval(trailp[side], xall[side])
    dd[side] = np.polyval(diffp[side], xall[side])

# plot
for side in sides:
    plt.plot(xall[side], dl[side], 'r-')
    plt.plot(xall[side], tl[side], 'b-')
    plt.plot(xall[side], detall[side], 'r.')
    plt.plot(xall[side], trailall[side], 'b.')
    plt.xlabel('x')
    plt.ylabel('flux loss')

plt.figure()
for side in sides:
    plt.plot(xall[side], dd[side], 'g-')
    plt.plot(xall[side], diffall[side], 'g.')
    plt.xlabel('x')
    plt.ylabel('ratio (det / trail) flux')

plt.figure()
for side in sides:
    plt.plot(xall[side], 2.5*np.log10(detall[side]/dl[side]), 'r-')
    plt.plot(xall[side], 2.5*np.log10(trailall[side]/tl[side]), 'b-')
    plt.xlabel('x')
    plt.ylabel('$\Delta$ (mag_calc - mag_fit)')
    eps = 1e-20
    diff_dd = 2.5*np.log10(diffall[side] / dd[side])
    plt.plot(xall[side], diff_dd, 'g-')
```

```python
for side in sides:
    print 'side', side
    print 'a/b'
    print 'trailing loss params : a,b', trailab[side]['a'], trailab[side]['b']
    print 'detection loss params: a,b', detab[side]['a'], detab[side]['b']
    print 'ratio params: a,b', diffab[side]['a'], diffab[side]['b']
    print 'p'
    print 'trailing loss params: p', trailp[side]
    print 'detection loss params: p, ', detp[side]
    print 'ratio params: p', diffp[side]

print 'Average a/b"s:'
print 'trailing loss:', (trailab['L']['a'] + trailab['R']['a'])/2.0, (trailab['L']['b'] + trailab['R']['b'])/2.0
print 'detection loss:', (detab['L']['a'] + detab['R']['a'])/2.0, (detab['L']['b'] + detab['R']['b'])/2.0
print 'difference det-trail', (diffab['L']['a'] + diffab['R']['a'])/2.0, (diffab['L']['b'] + diffab['R']['b'])/2.0
```

    side L
    a/b
    trailing loss params : a,b 0.920734046087 1.4169297694
    detection loss params: a,b 0.431955259706 0.00535711080589
    ratio params: a,b 0.217994245071 0.346470879895
    p
    trailing loss params: p [ -5.29500760e-06   2.19397497e-04  -2.95124468e-03   8.65400208e-03
       2.11945061e-01   9.51532325e-01]
    detection loss params: p,  [ -1.78979546e-05   7.56237969e-04  -1.16129979e-02   8.36873676e-02
       2.97513833e-01   8.81788708e-01]
    ratio params: p [ -8.89600663e-06   3.65178479e-04  -5.22005436e-03   2.82739025e-02
       1.08896878e-01   9.39462156e-01]
    side R
    a/b
    trailing loss params : a,b 0.601723287645 0.907305298359
    detection loss params: a,b 0.405893156869 0.000259020721574
    ratio params: a,b 0.179239610511 0.263157781724
    p
    trailing loss params: p [ -6.51485613e-06   2.82673664e-04  -4.19447694e-03   1.98842197e-02
       1.70661764e-01   9.58934388e-01]
    detection loss params: p,  [ -2.73651404e-05   1.22260780e-03  -2.00065226e-02   1.49305252e-01
       1.03333241e-01   9.42108957e-01]
    ratio params: p [ -1.54787202e-05   6.75787994e-04  -1.05047458e-02   6.64178860e-02
       9.35379777e-03   9.71380681e-01]
    Average a/b"s:
    trailing loss: 0.761228666866 1.16211753388
    detection loss: 0.418924208288 0.00280806576373
    difference det-trail 0.198616927791 0.30481433081

```python
# Summary:
def dmag(velocity, seeing, texp=30.):
    a_trail = 0.761
    b_trail = 1.162
    a_det = 0.420
    b_det = 0.003
    x = velocity * texp / seeing / 24.0
    dmag = {}
    dmag['trail'] = 1.25 * np.log10(1 + a_trail*x**2/(1+b_trail*x))
    dmag['detect'] = 1.25 * np.log10(1 + a_det*x**2 / (1+b_det*x))
    return dmag
```

```python
velocities = np.arange(0, 8, 0.1)
plt.figure(figsize=(8,6))
seeing = 0.7
dmags = dmag(velocities, seeing)
plt.plot(velocities, dmags['trail'], 'b:', label='SNR loss')
plt.plot(velocities, dmags['detect'], 'r-', label='Detection loss')
#plt.plot(velocities, 1.25*(np.log10(1+0.67*(velocities*30.0/seeing/24.0))), 'k:')
plt.legend(loc='upper left', fancybox=True, numpoints=1, fontsize='smaller')
plt.grid(True)
plt.ylim(0, None)
plt.xlim(0, None)
plt.xlabel(r'Velocity (deg/day)', fontsize='x-large')
plt.ylabel(r'$\Delta$ Mag', fontsize='x-large')
plt.title(r'Trailing Losses for $\Theta$ = %.1f"' % seeing, fontsize='x-large')
plt.savefig('_static/trailing_losses.png', format='png', dpi=600)
```

```python
velocities = np.arange(0, 30, 0.1)
plt.figure(figsize=(8,6))
seeing = 0.7
dmags = dmag(velocities, seeing)
plt.plot(velocities, dmags['trail'], 'b:', label='SNR loss')
plt.plot(velocities, dmags['detect'], 'r-', label='Detection loss')
#plt.plot(velocities, 1.25*(np.log10(1+0.67*(velocities*30.0/seeing/24.0))), 'k:')
plt.legend(loc='upper left', fancybox=True, numpoints=1, fontsize='smaller')
plt.grid(True)
plt.ylim(0, None)
plt.xlim(0, None)
plt.xlabel(r'Velocity (deg/day)', fontsize='x-large')
plt.ylabel(r'$\Delta$ Mag', fontsize='x-large')
plt.title(r'Trailing Losses for $\Theta$ = %.1f"' % seeing, fontsize='x-large')
plt.savefig('_static/trailing_losses_fast.png', format='png', dpi=600)
```

```python
velocities = np.arange(0, 20, 0.1)
plt.figure(figsize=(8,6))
seeing = 1.0
dmags = dmag(velocities, seeing)
plt.plot(velocities, dmags['trail'], 'b:', label='SNR loss')
plt.plot(velocities, dmags['detect'], 'r-', label='Detection loss')
plt.legend(loc='upper left', fancybox=True, numpoints=1, fontsize='smaller')
plt.grid(True)
plt.ylim(0, None)
plt.xlim(0, None)
plt.xlabel(r'Velocity (deg/day)', fontsize='x-large')
plt.ylabel(r'$\Delta$ Mag', fontsize='x-large')
plt.title(r'Trailing Losses for $\Theta$ = %.1f"' % seeing, fontsize='x-large')
plt.savefig('_static/trailing_losses_%.1f.png' % seeing, format='png', dpi=600)
```

```python

```


# Burnt area mapping using Sentinel-2 Near Real Time data

* [**Sign up to the DEA Sandbox**](https://docs.dea.ga.gov.au/setup/sandbox.html) to run this notebook interactively from a browser
* **Compatibility:** Notebook currently only compatible with `DEA Sandbox` environment as near real time data is not available on the NCI
* **Products used:**
[s2a_nrt_granule](https://explorer.sandbox.dea.ga.gov.au/s2a_nrt_granule),
[s2b_nrt_granule](https://explorer.sandbox.dea.ga.gov.au/s2b_nrt_granule),
[s2a_ard_granule](https://explorer.sandbox.dea.ga.gov.au/s2a_ard_granule),
[s2b_ard_granule](https://explorer.sandbox.dea.ga.gov.au/s2b_ard_granule)

## Background

### Normalized Burn Ratio

The Normalized Burn Ratio (NBR) is an index that uses the differences in the way healthy green vegetation and burnt vegetation reflect light to find burnt area.
It is calculated using the following Sentinel-2 bands: Near Infrared/Band 8 and Shortwave Infrared/Band 12. The equation is defined below:

\begin{equation}
NBR = \frac{(NIR - SWIR)}{(NIR + SWIR)}
\end{equation}

NBR returns values between -1 and 1.
**Healthy green vegetation will have a high NBR value while burnt vegetation will have a low value**.
Areas of dry, brown vegetation or bare soil will also return lower NBR values than green vegetation.

### Delta Normalized Burn Ratio

Change in Normalized Burn Ratio - also called Delta Normalized Burn Ratio (dNBR) - is calculated by subtracting the post-fire NBR value from the baseline NBR value as defined in this equation:

\begin{equation}
dNBR = NBR_{baseline} - NBR_{post fire}
\end{equation}

The dNBR value can be more useful than the NBR alone to determine what is burnt as it shows change from the baseline state.
**A burnt area will have a positive dNBR value** while an unburnt area will have a negative dNBR value or a value close to zero.

dNBR can also be used to describe burn severity (although this notebook does not look at burn severity).
A higher severity fire will burn more vegetation, resulting in a higher dNBR. More information on NBR, dNBR and using it to measure burn severity can be found [on the UN-SPIDER knowledge portal](http://un-spider.org/advisory-support/recommended-practices/recommended-practice-burn-severity/in-detail/normalized-burn-ratio).

### Defining Burnt From Unburnt Areas

[Rahman et al. 2018](https://doi.org/10.1109/IGARSS.2018.8518449) found a dNBR threshold value of +0.1 appropriate for differentiating burnt from unburnt areas when using Sentinel-2.
However, some exploration with different threshold levels may be needed to get a good result in areas with different vegetation types.

In the example presented in this notebook, which covers part of the Clyde Mountain fire in the area north of Batemans Bay, the fire occurred in heavily forested area, which returns a very strong dNBR result.
Using +0.1 as a threshold here results in many false positives being picked up in the unburnt urban and forest areas where vegetation drying has occurred prior to the fire.
A much more conservative threshold here of +0.3 produces a better result.
Keep in mind the limitations of remote sensing and that in an ideal situation ground truth data collected in the location of interest would be used to assist in selecting a threshold.

Some care should also be taken when interpreting results as a number of possible false positives can return a positive dNBR result:

* A lot of smoke in the post burn image can interfere with the dNBR value
* Areas that have been cleared of vegetation by some other means (logging, harvesting, and landslides) towards the end of the baseline period may incorrectly show up as burnt
* Drying out of bright green vegetation such as grasses.
If a fire event has been preceded by a rapid drying out of vegetation this can result in a low positive dNBR value in areas that have not burnt.

## Description

This notebook calculates the change in Normalized Burn Ratio between a baseline composite image of the pre-fire condition of a selected area and a post-fire event image, in order to find burnt area extent.
**Specifically this notebook has been designed for mapping fire extent in recent fires, and so assumes that NRT products will need to be used. If the fire of interest is historical, users should select the definitive Sentinel-2 Analysis Ready Product instead.**

The user can change the location over which this notebook is run and specify a different date between which pre and post fire condition will be compared.
The length of time over which the baseline composite image will be generated can be specified as 3, 6 or 12 months.
The code in this notebook will automatically generate the composite image over the set length of time using both Sentinel-2 Near Real Time data and the definitive Sentinel 2 Analysis Ready Product.

The notebook contains the following steps:

1.  Select a location for the analysis
2.  Define fire event date and length of composite image
3.  Load all baseline data
4.  Generate Normalized Burn Ratio for baseline period
5.  Load post-fire data from Near Real Time data
6.  Generate Normalized Burn Ratio for post fire image
7.  Caculate Delta Normalized Burn Ratio
8.  Apply threshold to Delta Normalized Burn Ratio
9.  Calculate the area burnt
10. Export results as a GeoTIFF

***

## Getting started

To run this analysis, run all the cells in the notebook, starting with the "Load packages" cell.

### Load packages
Import Python packages that are used for the analysis.

```python
import sys
import datacube
from datacube.utils import cog
from datetime import datetime
from datetime import timedelta
import pandas as pd
import xarray as xr
import numpy as np
import matplotlib.pyplot as plt

sys.path.append("../Scripts")
from dea_datahandling import load_ard
from dea_plotting import rgb
from dea_plotting import display_map
from dea_bandindices import calculate_indices
```

### Connect to the datacube

Connect to the datacube so we can access DEA data.
The `app` parameter is a unique name for the analysis which is based on the notebook file name.

```python
dc = datacube.Datacube(app="Burnt_area_mapping")
```

### Select location

The selected latitude and longitude will be displayed as a red box on the map below the next cell.
This map can be used to find coordinates of other places, simply scroll and click on any point on the map to display the latitude and longitude of that location.

```python
# Set the central latitude and longitude
central_lat = -35.653031
central_lon = 150.231667

# Set the buffer to load around the central coordinates
buffer = 0.1

# Compute the bounding box for the study area
study_area_lat = (central_lat - buffer, central_lat + buffer)
study_area_lon = (central_lon - buffer, central_lon + buffer)

display_map(x=study_area_lon, y=study_area_lat, margin=-0.2)
```

### Define fire event date and length of composite image

Delta Normalized Burn Ratio produces the best result when using a post-fire image that was collected before much re-growth has occured.
However, images collected while the fire is still active can be obscured by smoke and not show the full burn extent.
As a result some adjustment of the fire event date entered may be needed to get the best result.

The length of the baseline period can be automatically set to `3, 6 or 12 months`

```python
# Fire event date
fire_date = '2020-01-05'

# Length of baseline period
baseline_length = '3 months'
```

#### Automaticaly define date range for baseline composite image

```python
# Define dates for loading data
if baseline_length == '12 months':
    time_step = timedelta(days=365)
if baseline_length == '6 months':
    time_step = timedelta(days=182.5)
if baseline_length == '3 months':
    time_step = timedelta(days=91)

# Calculate the start and end date for baseline data load
start_date_pre = datetime.strftime(
    ((datetime.strptime(fire_date, '%Y-%m-%d')) - time_step), '%Y-%m-%d')
end_date_pre = datetime.strftime(
    ((datetime.strptime(fire_date, '%Y-%m-%d')) - timedelta(days=1)),
    '%Y-%m-%d')

# Calculate end date for post fire data load
start_date_post = datetime.strftime(
    ((datetime.strptime(fire_date, '%Y-%m-%d')) + timedelta(days=1)),
    '%Y-%m-%d')
end_date_post = datetime.strftime(
    ((datetime.strptime(fire_date, '%Y-%m-%d')) + timedelta(days=90)),
    '%Y-%m-%d')

# Print dates
print(f'start_date_pre:  {start_date_pre}')
print(f'end_date_pre:    {end_date_pre}')
print(f'fire_date:       {fire_date}')
print(f'start_date_post: {start_date_post}')
print(f'end_date_post:   {end_date_post}')
```

    start_date_pre:  2019-10-06
    end_date_pre:    2020-01-04
    fire_date:       2020-01-05
    start_date_post: 2020-01-06
    end_date_post:   2020-04-04

#### Create a function to combine ARD and NRT if both contain data

```python
def combine_if(nrt, ard):

    # If both ARD and NRT data is available
    if nrt is not None and ard is not None:
        print('Combining ARD and NRT data')

        # Print dates to see what images are available for both datasets
        ard_dates = ard.time.dt.strftime('%Y-%m-%d').values
        nrt_dates = nrt.time.dt.strftime('%Y-%m-%d').values
        print(f'ARD dates\n    {ard_dates}')
        print(f'NRT dates\n    {nrt_dates}')

        # Find duplicates (convert to single item if list has length 1)
        duplicate_dates = np.intersect1d(ard_dates, nrt_dates)
        duplicate_dates = (duplicate_dates[0] if
                           len(duplicate_dates) == 1
                           else duplicate_dates)

        # Only select NRT that is not duplicated in ARD
        if len(duplicate_dates) > 0:
            nrt = nrt.sel(time=~duplicate_dates)

        # Concantenate NRT and ARD data together
        dataset = xr.concat([ard, nrt], dim='time').sortby('time')

        # Print to see when images are available in output dataset
        dates = dataset.time.dt.strftime('%Y-%m-%d').values.tolist()
        print(f'Output dates after removing duplicates\n    {dates}')

        return dataset

    # If only ARD is available
    elif ard is not None:
        # Convert dates and print to see when images are available
        dates = ard.time.dt.strftime('%Y-%m-%d').values.tolist()
        print(f'Using ARD data\n    {dates}')

        return ard

    # If only NRT is available
    elif nrt is not None:
        # Convert dates and print to see when images are available
        dates = nrt.time.dt.strftime('%Y-%m-%d').values.tolist()
        print(f'Using NRT data\n    {dates}')

        return nrt

    # If no data of any kind is available
    else:
        print('No data avaliable')

```

#### Define load parameters

```python
resolution = (-10, 10)
measurements = ['nbart_blue', 'nbart_green', 'nbart_red',
                'nbart_nir_1', 'nbart_swir_3']
min_gooddata = 0.5
output_crs = 'EPSG:3577'
```

## Load all baseline data

```python
try:
    # Load all data in basline period avalible from ARD data
    baseline_ard = load_ard(dc=dc,
                            products=['s2a_ard_granule', 's2b_ard_granule'],
                            x=study_area_lon,
                            y=study_area_lat,
                            time=(start_date_pre, end_date_pre),
                            measurements=measurements,
                            min_gooddata=min_gooddata,
                            output_crs=output_crs,
                            resolution=resolution,
                            group_by='solar_day')
except:
    baseline_ard = None
    print('There is no baseline ARD data')

try:
    # Load remaining data in baseline period available from ARD collection
    baseline_nrt = load_ard(dc=dc,
                            products=['s2a_nrt_granule', 's2b_nrt_granule'],
                            x=study_area_lon,
                            y=study_area_lat,
                            time=(start_date_pre, end_date_pre),
                            measurements=measurements,
                            min_gooddata=min_gooddata,
                            output_crs=output_crs,
                            resolution=resolution,
                            group_by='solar_day')
except:
    baseline_nrt = None
    print('There is no baseline NRT data')
```

    Finding datasets
        s2a_ard_granule
        s2b_ard_granule
    Counting good quality pixels for each time step
    Filtering to 5 out of 9 time steps with at least 50.0% good quality pixels
    Applying pixel quality/cloud mask
    Loading 5 time steps
    Finding datasets
        s2a_nrt_granule
        s2b_nrt_granule
    There is no baseline NRT data

#### Create baseline dataset

```python
baseline = combine_if(baseline_nrt, baseline_ard)
baseline
```

    Using ARD data
        ['2019-10-17', '2019-10-22', '2019-11-01', '2019-11-21', '2019-12-01']

<svg style="position: absolute; width: 0; height: 0; overflow: hidden">
<defs>
<symbol id="icon-database" viewBox="0 0 32 32">
<path d="M16 0c-8.837 0-16 2.239-16 5v4c0 2.761 7.163 5 16 5s16-2.239 16-5v-4c0-2.761-7.163-5-16-5z"></path>
<path d="M16 17c-8.837 0-16-2.239-16-5v6c0 2.761 7.163 5 16 5s16-2.239 16-5v-6c0 2.761-7.163 5-16 5z"></path>
<path d="M16 26c-8.837 0-16-2.239-16-5v6c0 2.761 7.163 5 16 5s16-2.239 16-5v-6c0 2.761-7.163 5-16 5z"></path>
</symbol>
<symbol id="icon-file-text2" viewBox="0 0 32 32">
<path d="M28.681 7.159c-0.694-0.947-1.662-2.053-2.724-3.116s-2.169-2.030-3.116-2.724c-1.612-1.182-2.393-1.319-2.841-1.319h-15.5c-1.378 0-2.5 1.121-2.5 2.5v27c0 1.378 1.122 2.5 2.5 2.5h23c1.378 0 2.5-1.122 2.5-2.5v-19.5c0-0.448-0.137-1.23-1.319-2.841zM24.543 5.457c0.959 0.959 1.712 1.825 2.268 2.543h-4.811v-4.811c0.718 0.556 1.584 1.309 2.543 2.268zM28 29.5c0 0.271-0.229 0.5-0.5 0.5h-23c-0.271 0-0.5-0.229-0.5-0.5v-27c0-0.271 0.229-0.5 0.5-0.5 0 0 15.499-0 15.5 0v7c0 0.552 0.448 1 1 1h7v19.5z"></path>
<path d="M23 26h-14c-0.552 0-1-0.448-1-1s0.448-1 1-1h14c0.552 0 1 0.448 1 1s-0.448 1-1 1z"></path>
<path d="M23 22h-14c-0.552 0-1-0.448-1-1s0.448-1 1-1h14c0.552 0 1 0.448 1 1s-0.448 1-1 1z"></path>
<path d="M23 18h-14c-0.552 0-1-0.448-1-1s0.448-1 1-1h14c0.552 0 1 0.448 1 1s-0.448 1-1 1z"></path>
</symbol>
</defs>
</svg>

<xarray.Dataset>
Dimensions:       (time: 5, x: 2108, y: 2457)
Coordinates:
  * y             (y) float64 -4e+06 -4e+06 -4e+06 ... -4.024e+06 -4.024e+06
  * time          (time) datetime64[ns] 2019-10-17T00:02:39.024000 ... 2019-1...
  * x             (x) float64 1.633e+06 1.633e+06 ... 1.654e+06 1.654e+06
    spatial_ref   int32 3577
Data variables:
    nbart_blue    (time, y, x) float32 188.0 129.0 176.0 213.0 ... nan nan nan
    nbart_green   (time, y, x) float32 253.0 172.0 215.0 294.0 ... nan nan nan
    nbart_red     (time, y, x) float32 192.0 123.0 178.0 234.0 ... nan nan nan
    nbart_nir_1   (time, y, x) float32 2168.0 1716.0 2058.0 ... nan nan nan
    nbart_swir_3  (time, y, x) float32 398.0 355.0 355.0 423.0 ... nan nan nan
Attributes:
    crs:           EPSG:3577
    grid_mapping:  spatial_ref

xarray.Dataset

<input id='section-8966ba1f-4fc4-4d82-a822-b6264416df9d' class='xr-section-summary-in' type='checkbox' disabled >Dimensions:

time: 5

x: 2108

y: 2457

<input id='section-06697bce-9ae2-4a19-b516-b9482c06457d' class='xr-section-summary-in' type='checkbox'  checked>Coordinates: (4)

y

(y)

float64

-4e+06 -4e+06 ... -4.024e+06
<input id='attrs-5c7a6f11-4de7-4e6f-820e-2d6d8bdf20f3' class='xr-var-attrs-in' type='checkbox' ><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg><input id='data-44d23897-cc01-4ec4-8cb7-573a6edd0e58' class='xr-var-data-in' type='checkbox'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg>

units :

metre

resolution :

-10.0

crs :

EPSG:3577

array([-3999915., -3999925., -3999935., ..., -4024455., -4024465., -4024475.])

time

(time)

datetime64[ns]

2019-10-17T00:02:39.024000 ... 2...
<input id='attrs-9a706aa1-bd23-4df3-9946-6b44a7dddeab' class='xr-var-attrs-in' type='checkbox' ><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg><input id='data-4defa030-2a4b-4946-9ecd-0ebe492b5b61' class='xr-var-data-in' type='checkbox'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg>

units :

seconds since 1970-01-01 00:00:00

array(['2019-10-17T00:02:39.024000000', '2019-10-22T00:02:41.024000000',
       '2019-11-01T00:02:41.024000000', '2019-11-21T00:02:41.024000000',
       '2019-12-01T00:02:41.024000000'], dtype='datetime64[ns]')

x

(x)

float64

1.633e+06 1.633e+06 ... 1.654e+06
<input id='attrs-c4d5c255-4465-4c48-9edc-ae6b8c4fdbd2' class='xr-var-attrs-in' type='checkbox' ><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg><input id='data-21ebfb70-deb0-4097-a349-2ee2c2407717' class='xr-var-data-in' type='checkbox'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg>

units :

metre

resolution :

10.0

crs :

EPSG:3577

array([1633215., 1633225., 1633235., ..., 1654265., 1654275., 1654285.])

spatial_ref

()

int32

3577
<input id='attrs-09b06de1-8848-49f1-825c-005bbbf723b5' class='xr-var-attrs-in' type='checkbox' ><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg><input id='data-dad7150f-cba9-4d98-8374-6cee4e9bff12' class='xr-var-data-in' type='checkbox'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg>

spatial_ref :

PROJCS["GDA94 / Australian Albers",GEOGCS["GDA94",DATUM["Geocentric_Datum_of_Australia_1994",SPHEROID["GRS 1980",6378137,298.257222101,AUTHORITY["EPSG","7019"]],AUTHORITY["EPSG","6283"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.0174532925199433,AUTHORITY["EPSG","9122"]],AUTHORITY["EPSG","4283"]],PROJECTION["Albers_Conic_Equal_Area"],PARAMETER["latitude_of_center",0],PARAMETER["longitude_of_center",132],PARAMETER["standard_parallel_1",-18],PARAMETER["standard_parallel_2",-36],PARAMETER["false_easting",0],PARAMETER["false_northing",0],UNIT["metre",1,AUTHORITY["EPSG","9001"]],AXIS["Easting",EAST],AXIS["Northing",NORTH],AUTHORITY["EPSG","3577"]]

grid_mapping_name :

albers_conical_equal_area

array(3577, dtype=int32)

<input id='section-903d23a8-14e2-46df-98b0-4e033064b013' class='xr-section-summary-in' type='checkbox'  checked>Data variables: (5)

nbart_blue

(time, y, x)

float32

188.0 129.0 176.0 ... nan nan nan
<input id='attrs-b8ee1751-b2fb-4ee5-915d-35a639a1f4b3' class='xr-var-attrs-in' type='checkbox' ><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg><input id='data-3f812115-055a-48aa-9c08-702454b72156' class='xr-var-data-in' type='checkbox'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg>

units :

1

spectral_definition :

{'response': [0.001206, 0.00204, 0.002623, 0.002738, 0.002746, 0.002076, 0.003246, 0.002224, 0.002789, 0.003127, 0.002377, 0.003776, 0.002856, 0.003063, 0.00607, 0.009028, 0.019547, 0.038955, 0.084088, 0.176255, 0.292197, 0.364612, 0.382418, 0.385789, 0.393447, 0.400158, 0.410291, 0.424686, 0.449286, 0.481594, 0.505323, 0.523406, 0.529543, 0.534688, 0.533786, 0.534656, 0.5381, 0.543691, 0.557717, 0.578585, 0.601967, 0.616037, 0.621092, 0.613597, 0.596062, 0.575863, 0.558063, 0.546131, 0.542099, 0.553602, 0.571684, 0.598269, 0.633236, 0.67337, 0.711752, 0.738396, 0.758249, 0.768325, 0.773367, 0.780468, 0.788363, 0.795449, 0.809151, 0.824011, 0.837709, 0.844983, 0.847328, 0.840111, 0.825797, 0.804778, 0.78694, 0.771578, 0.761923, 0.765487, 0.781682, 0.810031, 0.850586, 0.901671, 0.95467, 0.987257, 1.0, 0.986389, 0.908308, 0.724, 0.478913, 0.286992, 0.169976, 0.102833, 0.065163, 0.04116, 0.02508, 0.013112, 0.002585, 0.001095, 0.000308, 0.000441, 0.0, 0.0, 0.000443], 'wavelength': [440, 441, 442, 443, 444, 445, 446, 447, 448, 449, 450, 451, 452, 453, 454, 455, 456, 457, 458, 459, 460, 461, 462, 463, 464, 465, 466, 467, 468, 469, 470, 471, 472, 473, 474, 475, 476, 477, 478, 479, 480, 481, 482, 483, 484, 485, 486, 487, 488, 489, 490, 491, 492, 493, 494, 495, 496, 497, 498, 499, 500, 501, 502, 503, 504, 505, 506, 507, 508, 509, 510, 511, 512, 513, 514, 515, 516, 517, 518, 519, 520, 521, 522, 523, 524, 525, 526, 527, 528, 529, 530, 531, 532, 533, 534, 535, 536, 537, 538]}

crs :

EPSG:3577

grid_mapping :

spatial_ref

array([[[ 188.,  129.,  176., ...,  173.,  176.,  212.],
        [ 165.,  144.,  167., ...,  155.,  171.,  175.],
        [ 178.,  212.,  187., ...,  200.,  230.,  175.],
        ...,
        [ 235.,  240.,  235., ...,  308.,  357.,  360.],
        [ 222.,  207.,  254., ...,  300.,  338.,  349.],
        [ 219.,  214.,  216., ...,  350.,  350.,  354.]],

       [[ 152.,  135.,  185., ...,  181.,  196.,  221.],
        [ 165.,  154.,  178., ...,  179.,  212.,  218.],
        [ 183.,  154.,  180., ...,  226.,  272.,  265.],
        ...,
        [ 235.,  244.,  254., ...,  379.,  336.,  353.],
        [ 223.,  249.,  256., ...,  350.,  340.,  364.],
        [ 238.,  243.,  237., ...,  355.,  375.,  339.]],

       [[ 349.,  308.,  324., ...,  349.,  351.,  342.],
        [ 343.,  317.,  329., ...,  344.,  361.,  355.],
        [ 347.,  368.,  360., ...,  394.,  388.,  369.],
        ...,
        [ 383.,  404.,  401., ...,   nan,   nan,   nan],
        [ 388.,  393.,  402., ...,   nan,   nan,   nan],
        [ 409.,  398.,  390., ...,   nan,   nan,   nan]],

       [[ 263.,  192.,  219., ...,  321.,  297.,  293.],
        [ 219.,  215.,  221., ...,  343.,  323.,  318.],
        [ 231.,  215.,  250., ...,  409.,  376.,  393.],
        ...,
        [ 283.,  281.,  295., ...,   nan,   nan,   nan],
        [ 282.,  307.,  306., ...,   nan,   nan,   nan],
        [ 306.,  301.,  283., ...,   nan,   nan,   nan]],

       [[ 625.,  599.,  616., ..., 1159., 1153., 1157.],
        [ 601.,  594.,  615., ..., 1170., 1160., 1172.],
        [ 595.,  609.,  644., ..., 1207., 1205., 1172.],
        ...,
        [1961., 1945., 1933., ...,   nan,   nan,   nan],
        [1956., 1950., 1938., ...,   nan,   nan,   nan],
        [1965., 1972., 1952., ...,   nan,   nan,   nan]]], dtype=float32)

nbart_green

(time, y, x)

float32

253.0 172.0 215.0 ... nan nan nan
<input id='attrs-33f1863e-9971-4323-ab9b-21ef50c0cee9' class='xr-var-attrs-in' type='checkbox' ><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg><input id='data-072b4bf7-0804-44f6-b5c9-aa5b8b24fd8b' class='xr-var-data-in' type='checkbox'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg>

units :

1

spectral_definition :

{'response': [0.00084, 0.016372, 0.037688, 0.080665, 0.169509, 0.341374, 0.573031, 0.746711, 0.828036, 0.868228, 0.888565, 0.891572, 0.87815, 0.860271, 0.843698, 0.834035, 0.832617, 0.844968, 0.867734, 0.897361, 0.933938, 0.966923, 0.990762, 1.0, 0.997812, 0.981107, 0.947088, 0.907183, 0.868656, 0.837622, 0.81291, 0.792434, 0.7851, 0.789606, 0.807011, 0.830458, 0.846433, 0.858402, 0.85799, 0.799824, 0.62498, 0.386994, 0.200179, 0.098293, 0.042844, 0.016512], 'wavelength': [537, 538, 539, 540, 541, 542, 543, 544, 545, 546, 547, 548, 549, 550, 551, 552, 553, 554, 555, 556, 557, 558, 559, 560, 561, 562, 563, 564, 565, 566, 567, 568, 569, 570, 571, 572, 573, 574, 575, 576, 577, 578, 579, 580, 581, 582]}

crs :

EPSG:3577

grid_mapping :

spatial_ref

array([[[ 253.,  172.,  215., ...,  252.,  263.,  290.],
        [ 252.,  167.,  205., ...,  201.,  236.,  249.],
        [ 229.,  289.,  268., ...,  238.,  267.,  238.],
        ...,
        [ 279.,  300.,  313., ...,  271.,  287.,  256.],
        [ 279.,  282.,  336., ...,  252.,  274.,  248.],
        [ 266.,  299.,  309., ...,  259.,  275.,  268.]],

       [[ 223.,  175.,  246., ...,  246.,  245.,  277.],
        [ 208.,  216.,  238., ...,  242.,  243.,  247.],
        [ 275.,  216.,  252., ...,  280.,  293.,  312.],
        ...,
        [ 295.,  281.,  333., ...,  276.,  258.,  239.],
        [ 280.,  318.,  333., ...,  249.,  236.,  248.],
        [ 291.,  300.,  293., ...,  249.,  241.,  265.]],

       [[ 402.,  339.,  365., ...,  360.,  364.,  378.],
        [ 412.,  345.,  350., ...,  345.,  387.,  368.],
        [ 384.,  418.,  424., ...,  438.,  421.,  366.],
        ...,
        [ 398.,  411.,  440., ...,   nan,   nan,   nan],
        [ 399.,  443.,  447., ...,   nan,   nan,   nan],
        [ 413.,  442.,  407., ...,   nan,   nan,   nan]],

       [[ 336.,  244.,  305., ...,  363.,  344.,  359.],
        [ 312.,  273.,  301., ...,  364.,  352.,  348.],
        [ 337.,  273.,  334., ...,  480.,  443.,  448.],
        ...,
        [ 346.,  332.,  376., ...,   nan,   nan,   nan],
        [ 360.,  345.,  377., ...,   nan,   nan,   nan],
        [ 378.,  353.,  357., ...,   nan,   nan,   nan]],

       [[ 627.,  565.,  585., ..., 1031., 1031., 1020.],
        [ 597.,  558.,  591., ..., 1030., 1014., 1016.],
        [ 571.,  606.,  656., ..., 1069., 1060., 1018.],
        ...,
        [1678., 1673., 1646., ...,   nan,   nan,   nan],
        [1677., 1684., 1660., ...,   nan,   nan,   nan],
        [1670., 1678., 1663., ...,   nan,   nan,   nan]]], dtype=float32)

nbart_red

(time, y, x)

float32

192.0 123.0 178.0 ... nan nan nan
<input id='attrs-4bba9e49-70c5-43a5-9cdc-4dafcfa77dfe' class='xr-var-attrs-in' type='checkbox' ><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg><input id='data-1c0091c9-f28a-46ae-947c-10dd875b1e45' class='xr-var-data-in' type='checkbox'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg>

units :

1

spectral_definition :

{'response': [0.002584, 0.034529, 0.14997, 0.464826, 0.817746, 0.965324, 0.983869, 0.9969, 1.0, 0.995449, 0.991334, 0.977215, 0.936802, 0.873776, 0.814166, 0.776669, 0.764864, 0.775091, 0.801359, 0.830828, 0.857112, 0.883581, 0.90895, 0.934759, 0.955931, 0.96811, 0.973219, 0.971572, 0.969003, 0.965712, 0.960481, 0.944811, 0.884152, 0.706167, 0.422967, 0.189853, 0.063172, 0.020615, 0.002034], 'wavelength': [646, 647, 648, 649, 650, 651, 652, 653, 654, 655, 656, 657, 658, 659, 660, 661, 662, 663, 664, 665, 666, 667, 668, 669, 670, 671, 672, 673, 674, 675, 676, 677, 678, 679, 680, 681, 682, 683, 684]}

crs :

EPSG:3577

grid_mapping :

spatial_ref

array([[[ 192.,  123.,  178., ...,  231.,  230.,  261.],
        [ 186.,  123.,  174., ...,  192.,  214.,  226.],
        [ 166.,  218.,  202., ...,  258.,  266.,  239.],
        ...,
        [ 295.,  283.,  312., ...,  238.,  252.,  223.],
        [ 284.,  273.,  347., ...,  239.,  248.,  229.],
        [ 271.,  305.,  308., ...,  226.,  232.,  222.]],

       [[ 163.,  116.,  184., ...,  239.,  232.,  272.],
        [ 161.,  146.,  172., ...,  230.,  243.,  243.],
        [ 214.,  146.,  198., ...,  265.,  329.,  307.],
        ...,
        [ 326.,  278.,  316., ...,  216.,  223.,  214.],
        [ 287.,  338.,  336., ...,  198.,  208.,  235.],
        [ 288.,  277.,  294., ...,  188.,  184.,  253.]],

       [[ 345.,  237.,  283., ...,  323.,  337.,  353.],
        [ 323.,  259.,  281., ...,  354.,  352.,  347.],
        [ 298.,  333.,  334., ...,  416.,  425.,  349.],
        ...,
        [ 401.,  389.,  404., ...,   nan,   nan,   nan],
        [ 393.,  407.,  421., ...,   nan,   nan,   nan],
        [ 388.,  416.,  387., ...,   nan,   nan,   nan]],

       [[ 253.,  180.,  256., ...,  340.,  335.,  330.],
        [ 243.,  219.,  260., ...,  393.,  365.,  360.],
        [ 269.,  219.,  281., ...,  506.,  455.,  489.],
        ...,
        [ 359.,  356.,  365., ...,   nan,   nan,   nan],
        [ 390.,  371.,  368., ...,   nan,   nan,   nan],
        [ 397.,  371.,  353., ...,   nan,   nan,   nan]],

       [[ 494.,  422.,  448., ...,  818.,  836.,  841.],
        [ 480.,  415.,  452., ...,  876.,  863.,  845.],
        [ 456.,  491.,  530., ...,  943.,  965.,  874.],
        ...,
        [1364., 1335., 1336., ...,   nan,   nan,   nan],
        [1362., 1331., 1336., ...,   nan,   nan,   nan],
        [1377., 1358., 1352., ...,   nan,   nan,   nan]]], dtype=float32)

nbart_nir_1

(time, y, x)

float32

2168.0 1716.0 2058.0 ... nan nan
<input id='attrs-48b793c2-62a3-4234-8608-b94cc2c7c4a4' class='xr-var-attrs-in' type='checkbox' ><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg><input id='data-88df640e-8441-4443-a0b6-be1c7f6acd16' class='xr-var-data-in' type='checkbox'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg>

units :

1

spectral_definition :

{'response': [0.000451, 0.007614, 0.019072, 0.033498, 0.056536, 0.087148, 0.13246, 0.203436, 0.314068, 0.450085, 0.587433, 0.714518, 0.81829, 0.902932, 0.960732, 0.993723, 1.0, 0.985213, 0.958376, 0.93655, 0.925816, 0.930376, 0.941281, 0.953344, 0.962183, 0.965631, 0.963105, 0.959009, 0.951205, 0.945147, 0.943136, 0.945814, 0.945357, 0.943762, 0.937084, 0.92789, 0.915897, 0.900979, 0.881577, 0.86216, 0.8432, 0.822684, 0.801819, 0.776911, 0.755632, 0.737893, 0.722217, 0.708669, 0.698416, 0.690211, 0.682257, 0.681494, 0.682649, 0.678491, 0.67595, 0.671304, 0.665538, 0.660812, 0.658739, 0.65831, 0.66407, 0.672731, 0.685501, 0.701159, 0.720686, 0.742292, 0.75953, 0.776608, 0.784061, 0.78772, 0.787693, 0.783525, 0.776161, 0.768339, 0.759264, 0.745943, 0.733022, 0.720589, 0.706663, 0.69087, 0.675896, 0.661558, 0.649339, 0.638008, 0.627424, 0.618963, 0.610945, 0.604322, 0.597408, 0.591724, 0.586961, 0.582746, 0.581202, 0.57948, 0.580197, 0.582505, 0.585308, 0.589481, 0.592827, 0.596749, 0.601372, 0.603234, 0.605476, 0.608972, 0.613463, 0.618715, 0.626841, 0.637436, 0.648791, 0.659233, 0.66734, 0.668624, 0.659924, 0.641666, 0.615841, 0.583567, 0.552076, 0.526407, 0.507116, 0.49653, 0.499119, 0.512088, 0.529093, 0.542739, 0.537964, 0.495953, 0.419305, 0.326791, 0.231085, 0.14854, 0.089683, 0.054977, 0.033246, 0.019774, 0.007848, 0.001286], 'wavelength': [773, 774, 775, 776, 777, 778, 779, 780, 781, 782, 783, 784, 785, 786, 787, 788, 789, 790, 791, 792, 793, 794, 795, 796, 797, 798, 799, 800, 801, 802, 803, 804, 805, 806, 807, 808, 809, 810, 811, 812, 813, 814, 815, 816, 817, 818, 819, 820, 821, 822, 823, 824, 825, 826, 827, 828, 829, 830, 831, 832, 833, 834, 835, 836, 837, 838, 839, 840, 841, 842, 843, 844, 845, 846, 847, 848, 849, 850, 851, 852, 853, 854, 855, 856, 857, 858, 859, 860, 861, 862, 863, 864, 865, 866, 867, 868, 869, 870, 871, 872, 873, 874, 875, 876, 877, 878, 879, 880, 881, 882, 883, 884, 885, 886, 887, 888, 889, 890, 891, 892, 893, 894, 895, 896, 897, 898, 899, 900, 901, 902, 903, 904, 905, 906, 907, 908]}

crs :

EPSG:3577

grid_mapping :

spatial_ref

array([[[2168., 1716., 2058., ..., 2045., 2307., 2521.],
        [2224., 1669., 1930., ..., 1573., 2038., 2154.],
        [2009., 2203., 2312., ..., 1704., 1771., 1774.],
        ...,
        [1770., 2063., 2213., ...,  226.,  246.,  244.],
        [1819., 1900., 2195., ...,  215.,  237.,  233.],
        [1908., 1965., 2060., ...,  224.,  240.,  233.]],

       [[1913., 1604., 2101., ..., 1918., 2019., 2159.],
        [2065., 1869., 2095., ..., 1447., 1731., 1799.],
        [2360., 1869., 2267., ..., 1719., 1757., 1700.],
        ...,
        [1726., 1888., 2177., ...,  219.,  197.,  183.],
        [1799., 1908., 2065., ...,  182.,  190.,  173.],
        [1877., 1821., 1808., ...,  196.,  205.,  197.]],

       [[2319., 1757., 1966., ..., 1938., 2005., 2133.],
        [2426., 1933., 1976., ..., 1676., 1824., 1833.],
        [2262., 2323., 2454., ..., 1837., 1837., 1697.],
        ...,
        [1801., 2014., 2178., ...,   nan,   nan,   nan],
        [1835., 1977., 2138., ...,   nan,   nan,   nan],
        [1897., 1927., 1983., ...,   nan,   nan,   nan]],

       [[2244., 1744., 2061., ..., 1961., 1941., 2000.],
        [2200., 1944., 2048., ..., 1620., 1855., 1917.],
        [2324., 1944., 2217., ..., 1809., 1931., 1942.],
        ...,
        [1692., 1819., 2144., ...,   nan,   nan,   nan],
        [1855., 1836., 1956., ...,   nan,   nan,   nan],
        [1970., 1820., 1835., ...,   nan,   nan,   nan]],

       [[2421., 1994., 1962., ..., 2179., 2160., 2215.],
        [2460., 2069., 1902., ..., 1993., 2082., 2129.],
        [2298., 2293., 2404., ..., 2164., 2198., 1966.],
        ...,
        [2065., 2211., 2271., ...,   nan,   nan,   nan],
        [2193., 2197., 2251., ...,   nan,   nan,   nan],
        [2240., 2174., 2191., ...,   nan,   nan,   nan]]], dtype=float32)

nbart_swir_3

(time, y, x)

float32

398.0 355.0 355.0 ... nan nan nan
<input id='attrs-08d3dd1b-33e6-4a75-ad2a-2b2d146f8398' class='xr-var-attrs-in' type='checkbox' ><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg><input id='data-0e1db2f5-1993-47a6-8ff1-84e972141c58' class='xr-var-data-in' type='checkbox'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg>

units :

1

spectral_definition :

{'response': [0.000639, 0.001023, 0.002885, 0.003997, 0.006597, 0.00766, 0.008004, 0.00854, 0.0093, 0.010002, 0.010972, 0.012089, 0.013364, 0.015017, 0.017126, 0.01978, 0.023336, 0.027668, 0.033216, 0.040217, 0.048883, 0.059642, 0.073175, 0.090535, 0.11147, 0.136903, 0.167811, 0.203461, 0.242871, 0.284898, 0.327178, 0.368404, 0.408003, 0.444778, 0.476537, 0.503107, 0.525318, 0.543352, 0.557253, 0.568634, 0.57903, 0.588684, 0.598891, 0.609981, 0.621362, 0.634283, 0.648546, 0.663707, 0.680046, 0.696165, 0.711964, 0.727011, 0.741301, 0.757405, 0.772071, 0.785581, 0.798238, 0.809677, 0.819702, 0.828599, 0.836722, 0.844443, 0.851107, 0.853252, 0.854746, 0.856174, 0.857821, 0.859532, 0.86146, 0.863257, 0.865139, 0.867319, 0.869696, 0.874302, 0.878588, 0.882439, 0.885929, 0.889473, 0.893226, 0.896696, 0.899897, 0.902596, 0.904831, 0.905525, 0.905665, 0.905503, 0.905158, 0.904783, 0.904082, 0.903347, 0.902761, 0.902377, 0.901983, 0.903424, 0.904313, 0.905315, 0.906446, 0.908092, 0.910123, 0.91295, 0.915585, 0.918444, 0.921302, 0.924337, 0.927219, 0.92974, 0.931922, 0.934142, 0.935906, 0.937086, 0.937641, 0.938301, 0.937652, 0.940441, 0.942518, 0.943259, 0.943031, 0.942117, 0.940632, 0.938428, 0.93666, 0.935256, 0.933022, 0.92688, 0.921057, 0.915483, 0.91102, 0.908293, 0.907283, 0.908191, 0.911169, 0.916189, 0.922855, 0.920605, 0.919482, 0.919489, 0.921276, 0.924526, 0.927733, 0.931974, 0.93677, 0.941483, 0.946802, 0.951203, 0.954437, 0.957047, 0.959729, 0.962539, 0.964858, 0.966042, 0.966647, 0.96631, 0.96546, 0.964841, 0.963656, 0.961698, 0.959454, 0.957327, 0.95514, 0.953558, 0.952732, 0.952181, 0.951731, 0.952146, 0.952641, 0.954057, 0.957078, 0.960639, 0.964222, 0.968307, 0.972691, 0.977423, 0.982898, 0.987144, 0.990734, 0.993983, 0.996787, 0.998753, 1.0, 0.999927, 0.999152, 0.997129, 0.993884, 0.989686, 0.983735, 0.976213, 0.967813, 0.958343, 0.94844, 0.938203, 0.927725, 0.917039, 0.905999, 0.893868, 0.881683, 0.868647, 0.854635, 0.84062, 0.826016, 0.809516, 0.791867, 0.772443, 0.749107, 0.720346, 0.688185, 0.651283, 0.61005, 0.566031, 0.52097, 0.474659, 0.429259, 0.385857, 0.342092, 0.30068, 0.263176, 0.227704, 0.195721, 0.16809, 0.14468, 0.124831, 0.108237, 0.094401, 0.082363, 0.0715, 0.062691, 0.054985, 0.048195, 0.042864, 0.038598, 0.034947, 0.031999, 0.029588, 0.027418, 0.025577, 0.023959, 0.021672, 0.019148, 0.016331, 0.010989, 0.007379, 0.006511, 0.00471, 0.002065], 'wavelength': [2078, 2079, 2080, 2081, 2082, 2083, 2084, 2085, 2086, 2087, 2088, 2089, 2090, 2091, 2092, 2093, 2094, 2095, 2096, 2097, 2098, 2099, 2100, 2101, 2102, 2103, 2104, 2105, 2106, 2107, 2108, 2109, 2110, 2111, 2112, 2113, 2114, 2115, 2116, 2117, 2118, 2119, 2120, 2121, 2122, 2123, 2124, 2125, 2126, 2127, 2128, 2129, 2130, 2131, 2132, 2133, 2134, 2135, 2136, 2137, 2138, 2139, 2140, 2141, 2142, 2143, 2144, 2145, 2146, 2147, 2148, 2149, 2150, 2151, 2152, 2153, 2154, 2155, 2156, 2157, 2158, 2159, 2160, 2161, 2162, 2163, 2164, 2165, 2166, 2167, 2168, 2169, 2170, 2171, 2172, 2173, 2174, 2175, 2176, 2177, 2178, 2179, 2180, 2181, 2182, 2183, 2184, 2185, 2186, 2187, 2188, 2189, 2190, 2191, 2192, 2193, 2194, 2195, 2196, 2197, 2198, 2199, 2200, 2201, 2202, 2203, 2204, 2205, 2206, 2207, 2208, 2209, 2210, 2211, 2212, 2213, 2214, 2215, 2216, 2217, 2218, 2219, 2220, 2221, 2222, 2223, 2224, 2225, 2226, 2227, 2228, 2229, 2230, 2231, 2232, 2233, 2234, 2235, 2236, 2237, 2238, 2239, 2240, 2241, 2242, 2243, 2244, 2245, 2246, 2247, 2248, 2249, 2250, 2251, 2252, 2253, 2254, 2255, 2256, 2257, 2258, 2259, 2260, 2261, 2262, 2263, 2264, 2265, 2266, 2267, 2268, 2269, 2270, 2271, 2272, 2273, 2274, 2275, 2276, 2277, 2278, 2279, 2280, 2281, 2282, 2283, 2284, 2285, 2286, 2287, 2288, 2289, 2290, 2291, 2292, 2293, 2294, 2295, 2296, 2297, 2298, 2299, 2300, 2301, 2302, 2303, 2304, 2305, 2306, 2307, 2308, 2309, 2310, 2311, 2312, 2313, 2314, 2315, 2316, 2317, 2318, 2319, 2320]}

crs :

EPSG:3577

grid_mapping :

spatial_ref

array([[[398., 355., 355., ..., 411., 417., 417.],
        [451., 380., 355., ..., 471., 414., 414.],
        [380., 380., 429., ..., 414., 414., 434.],
        ...,
        [538., 528., 549., ..., 193., 194., 176.],
        [538., 538., 548., ..., 193., 193., 179.],
        [532., 538., 548., ..., 197., 179., 179.]],

       [[414., 362., 362., ..., 484., 421., 421.],
        [440., 410., 362., ..., 484., 421., 421.],
        [440., 410., 410., ..., 458., 436., 436.],
        ...,
        [567., 545., 545., ..., 175., 175., 164.],
        [566., 537., 537., ..., 155., 175., 164.],
        [566., 537., 537., ..., 155., 155., 169.]],

       [[513., 408., 408., ..., 488., 439., 439.],
        [521., 438., 408., ..., 578., 514., 514.],
        [438., 438., 495., ..., 514., 514., 503.],
        ...,
        [603., 592., 590., ...,  nan,  nan,  nan],
        [603., 603., 583., ...,  nan,  nan,  nan],
        [623., 603., 583., ...,  nan,  nan,  nan]],

       [[561., 428., 428., ..., 642., 560., 560.],
        [555., 471., 428., ..., 642., 560., 560.],
        [555., 471., 471., ..., 611., 585., 585.],
        ...,
        [627., 587., 587., ...,  nan,  nan,  nan],
        [657., 618., 618., ...,  nan,  nan,  nan],
        [657., 618., 618., ...,  nan,  nan,  nan]],

       [[606., 513., 513., ..., 539., 527., 527.],
        [599., 570., 513., ..., 724., 641., 641.],
        [570., 570., 609., ..., 641., 641., 574.],
        ...,
        [661., 614., 610., ...,  nan,  nan,  nan],
        [661., 661., 611., ...,  nan,  nan,  nan],
        [676., 661., 611., ...,  nan,  nan,  nan]]], dtype=float32)

<input id='section-ba9f69f5-3454-4b10-8aec-942c688da288' class='xr-section-summary-in' type='checkbox'  checked>Attributes: (2)

crs :

EPSG:3577

grid_mapping :

spatial_ref

## Generate Normalized Burn Ratio for baseline period

```python
# Calculate NBR for the baseline images
baseline = calculate_indices(baseline,
                             index='NBR',
                             collection='ga_s2_1',
                             drop=False)

# Compute median using all observations in the dataset along the time axis
baseline_image = baseline.median(dim='time')

# Select NBR
baseline_NBR = baseline_image.NBR
```

Plot the baseline NBR data side-by-side with an RGB plot of the study area:

```python
# Set up subplots
f, axarr = plt.subplots(1, 2, figsize=(13, 7), squeeze=False)

# Visualise baseline image as true colour image
rgb(baseline_image,
    bands=['nbart_red', 'nbart_green', 'nbart_blue'],
    ax=axarr[0, 0])
axarr[0, 0].set_title('Baseline RGB')
axarr[0, 0].set_xlabel('X coordinate')
axarr[0, 0].set_ylabel('Y coordinate')

# Visualise baseline image as NBR image
baseline_NBR.plot(cmap='RdBu', vmin=-1, vmax=1, ax=axarr[0, 1])
axarr[0, 1].set_title('Baseline NBR')
axarr[0, 1].yaxis.set_visible(False)
axarr[0, 1].set_xlabel('X coordinate');
```

## Load post-fire data

```python
try:
    # Load post-fire NRT data from Sentinel-2A and 2B
    post_nrt = load_ard(dc=dc,
               products=['s2a_nrt_granule', 's2b_nrt_granule'],
               x=study_area_lon,
               y=study_area_lat,
               time=(start_date_post, end_date_post),
               min_gooddata=min_gooddata,
               measurements=measurements,
               output_crs=output_crs,
               resolution=resolution,
               group_by='solar_day')
except:
    post_nrt = None
    print('There is no baseline NRT data')

try:
    # Load post-fire ARD data from Sentinel-2A and 2B
    post_ard = load_ard(dc=dc,
               products=['s2a_ard_granule', 's2b_ard_granule'],
               x=study_area_lon,
               y=study_area_lat,
               time=('2020-01-06', '2020-04-04'),
               min_gooddata=min_gooddata,
               measurements=measurements,
               output_crs=output_crs,
               resolution=resolution,
               group_by='solar_day')
except:
    post_ard = None
    print('There is no baseline ARD data')
```

    Finding datasets
        s2a_nrt_granule
        s2b_nrt_granule
    There is no baseline NRT data
    Finding datasets
        s2a_ard_granule
        s2b_ard_granule
    Counting good quality pixels for each time step
    Filtering to 4 out of 7 time steps with at least 50.0% good quality pixels
    Applying pixel quality/cloud mask
    Loading 4 time steps

### Create post-fire dataset

```python
post_col = combine_if(post_nrt, post_ard)
```

### Generate Normalized Burn Ratio for post fire image

To calculate the post-fire NBR image, we can choose to use a single, clear image (if one exists), or to calculate a median composite using the time-series.  Comment out the code you **don't** want to use in the cells below.

```python
### Use a single image:

# Select the most recent image after the fire
post_image = post_col.isel(time=0)

# Calculate NBR
post_image = calculate_indices(post_image, index='NBR', collection='ga_s2_1', drop=False)

# Select NBR
post_NBR = post_image.NBR

### Or use a median composite:

# # # Calculate NBR on all post-fire images
# post_combined = calculate_indices(post_col, index='NBR', collection='ga_s2_1', drop=False)

# # Calculate the median post-fire image
# post_image = post_combined.median(dim='time')

# # Select NBR
# post_NBR = post_image.NBR
# post_NBR
```

Plot the post-fire NBR data side-by-side with an RGB plot of the study area:

```python
# Set up subplots
f, axarr = plt.subplots(1, 2, figsize=(13, 7), squeeze=False)

# Visualise post-fire image as a true colour image
rgb(post_image,
    bands=['nbart_red', 'nbart_green', 'nbart_blue'],
    ax=axarr[0, 0])
axarr[0, 0].set_title('Post-fire RGB')
axarr[0, 0].set_xlabel('X coordinate')
axarr[0, 0].set_ylabel('Y coordinate')

# Visualise post-fire image as NBR image
post_NBR.plot(cmap='RdBu', vmin=-1, vmax=1, ax=axarr[0, 1])
axarr[0, 1].set_title('Post-fire NBR')
axarr[0, 1].yaxis.set_visible(False)
axarr[0, 1].set_xlabel('X coordinate');
```

## Calculate Delta Normalized Burn Ratio
We can now compute delta NBR by subtracting our post-fire NBR data from our baseline NBR data:

```python
delta_NBR = baseline_NBR - post_NBR

# Visualise dNBR image
delta_NBR.plot(cmap='RdBu_r', vmin=-1, vmax=1, figsize=(11, 9))
plt.xlabel('X coordinate')
plt.ylabel('Y coordinate');
```

## Apply threshold to Delta Normalized Burn Ratio

Set and apply the NBR threshold.
Here we set it to 0.3, but this will need adjustment depending on the use case.

```python
# Set threshold
threshold = 0.3

# Apply threshold
burnt = delta_NBR > threshold

# Mask post-fire true colour image
masked = delta_NBR.where(burnt==1)
```

Re-visualize pre and post-fire true colour images to help adjust the NBR threshold:

```python
# Set up subplots
f, axarr = plt.subplots(2, 3, figsize=(13, 11))
bands=['nbart_red', 'nbart_green', 'nbart_blue']

baseline_NBR.plot(cmap='RdBu', vmin=-1, vmax=1,
                  add_colorbar=False, ax=axarr[0, 0])
axarr[0, 0].set_title('Baseline NBR')
axarr[0, 0].set_ylabel('Y coordinate')
axarr[0, 0].xaxis.set_visible(False)

post_NBR.plot(cmap='RdBu', vmin=-1, vmax=1,
              add_colorbar=False, ax=axarr[0, 1])
axarr[0, 1].set_title('Post-fire NBR')
axarr[0, 1].yaxis.set_visible(False)
axarr[0, 1].xaxis.set_visible(False)

delta_NBR.plot(cmap='RdBu_r', vmin=-1, vmax=1,
               add_colorbar=False, ax=axarr[0, 2])
axarr[0, 2].set_title('Delta NBR')
axarr[0, 2].yaxis.set_visible(False)
axarr[0, 2].xaxis.set_visible(False)

rgb(baseline_image, bands=bands, ax=axarr[1, 0])
axarr[1, 0].set_title('Baseline RGB')
axarr[1, 0].set_title('Baseline RGB')
axarr[1, 0].set_xlabel('X coordinate')
axarr[1, 0].set_ylabel('Y coordinate')

rgb(post_image, bands=bands, ax=axarr[1,1])
axarr[1, 1].set_title('Post-fire RGB')
axarr[1, 1].set_xlabel('X coordinate')
axarr[1, 1].yaxis.set_visible(False)

rgb(post_image.where(burnt==1), bands=bands, ax=axarr[1, 2])
axarr[1, 2].set_title('Burnt RGB')
axarr[1, 2].set_xlabel('X coordinate')
axarr[1, 2].yaxis.set_visible(False)
```

## Calculate area burnt

```python
# Constants for calculating burnt area
pixel_length = resolution[1]  # in metres
m_per_km = 1000  # conversion from metres to kilometres

# Area per pixel
area_per_pixel = pixel_length ** 2 / m_per_km ** 2

# Calculate areas
unburnt_area = (delta_NBR <= threshold).sum() * area_per_pixel
burnt_area = burnt.sum() * area_per_pixel
not_nan_area = delta_NBR.notnull().sum() * area_per_pixel
nan_area = delta_NBR.isnull().sum() * area_per_pixel
total_area = unburnt_area + burnt_area + nan_area

print(f'Unburnt area:            {unburnt_area.item():.2f} km^2')
print(f'Burnt area:              {burnt_area.item():.2f} km^2')
print(f'Nan area:                {nan_area.item():.2f} km^2')
print(f'Total area (no nans):    {not_nan_area.item():.2f} km^2')
print(f'Total area (with nans):  {total_area.item():.2f} km^2')
```

## Export results to GeoTIFF

The baseline reference image and the post fire image will both be saved as a multiband GeoTIFF with the following bands in the following order: Blue, Green, Red, NIR, SWIR.

The thresholded burnt area image will be saved as a single band image, where a value of 1 = burnt and a value of 0 = not burnt.

```python
# Define an area name to be used in saved file names
area_name = 'Example'

# Write baseline reference image to multi-band GeoTIFF
cog.write_cog(baseline_image.NBR, f'{area_name}_baseline_image.tif', overwrite=True)

# Write post fire image to multi-band GeoTIFF
cog.write_cog(post_image.NBR, f'{area_name}_post_fire_image.tif', overwrite=True)

# Turn delta NBR into a Xarray Dataset for export to GeoTIFF
cog.write_cog(delta_NBR, f'{area_name}_delta_NBR.tif', overwrite=True)
```

***

## Additional information

**License:** The code in this notebook is licensed under the [Apache License, Version 2.0](https://www.apache.org/licenses/LICENSE-2.0).
Digital Earth Australia data is licensed under the [Creative Commons by Attribution 4.0](https://creativecommons.org/licenses/by/4.0/) license.

**Contact:** If you need assistance, please post a question on the [Open Data Cube Slack channel](http://slack.opendatacube.org/) or on the [GIS Stack Exchange](https://gis.stackexchange.com/questions/ask?tags=open-data-cube) using the `open-data-cube` tag (you can view previously asked questions [here](https://gis.stackexchange.com/questions/tagged/open-data-cube)).
If you would like to report an issue with this notebook, you can file one on [Github](https://github.com/GeoscienceAustralia/dea-notebooks).

**Last modified:** October 2020

**Compatible datacube version:**

```python
print(datacube.__version__)
```

## Tags
Browse all available tags on the DEA User Guide's [Tags Index](https://docs.dea.ga.gov.au/genindex.html)