# Surface Water-Mass Transformation

In this notebook, we compute surface water-mass transformation rates (both in the net and partitioned into contributions from heat and salt fluxes) for the southern ocean, south of 60$^\circ$S.

Here we are looking specifically at ACCESS-OM2-01 output (0.1$^\circ$ resolution, latest spinup), as this configuration has demonstrated considerable success when simulating high latitude dense water formation processes.

## 1. Defining surface water-mass transformation

The surface water-mass transformation framework described here follows [Newsom *et al* (2016)](https://journals.ametsoc.org/doi/full/10.1175/JCLI-D-15-0513.1) and [Abernathey *et al* (2016)](https://www.nature.com/articles/ngeo2749). Surface water-mass transformation may be defined as the volume flux into a given density class ($\sigma$) from lighter density classes ($\sigma'<\sigma$) due to surface buoyancy forcing. Integrated over a region of the ocean surface, this volume flux can be expressed as,

\begin{equation}\Omega(\sigma,t)=\frac{\partial}{\partial\sigma}\iint_{\sigma'<\sigma}\Big(\frac{\partial\sigma}{\partial\theta}\theta + \frac{\partial\sigma}{\partial S}S\Big)\,dx\,dy\end{equation}

where $t$ is time, and the terms in the integrand are the potential temperature ($\theta$) flux and salinity ($S$) flux components of the surface buoyancy flux. The linearity of this expression means we can extract the relative contributions of heat ($\Omega_\text{heat}$) and salinity ($\Omega_\text{fw}$) fluxes to surface water-mass transformation, highlighting driving mechanisms.

## 2. Requirements

**The cosima-cookbook**: The `conda/analysis3-20.04` (or later) module on NCI (or your own up-to-date cookbook installation).

**Model diagnostics**: for this example I use some of the long RYF9091 spinup output, if you want to use do surface water-mass transformation analysis on a different experiment you'll need to check you have saved the required diagnostics.

#### Warning: if you are using a surface heat flux diagnostic computed online by an ACCESS-OM2 run, make sure that this diagnostic is computed correctly! Some issues were found in the online computation of net_sfc_heating (see this [git issue](https://github.com/COSIMA/access-om2/issues/139) from March 2019). In this example we compute net surface heat fluxes offline from component fluxes to avoid these problems - if in doubt use the component fluxes as done here. If you dont have these variables saved and can't re-run...net_sfc_heating isn't really so bad for near Antarctic surface water-mass transformation calculations, since transformation rates are strongly dominated by freshwater fluxes, so the errors in heat fluxes confer only a small error to transformation rates.

We'll need at least monthly resolution (in `diag_table` language...): `surface_temp`, `surface_salt`, `pme_river`, `sfc_hflux_from_runoff`, `sfc_hflux_coupler`, `sfc_hflux_pme` and `frazil_3d_int_z` (although the latter is not strictly a surface flux, instead frazil heat fluxes are vertically integrated over the column. BUT this variable is highly surface intensified and, in most cases, conceptually relevant to what we want to know from SWMT analysis) plus some basic gridding information.

## 3. Computing surface water-mass transformation

First, import some required modules and functions.

```python
%matplotlib inline
%config InlineBackend.figure_format='retina'

import xarray as xr
import numpy as np
import dask.array
import cosima_cookbook as cc
from gsw import alpha, SA_from_SP, p_from_z, CT_from_pt, beta, sigma1
# note these gsw numpy functions mean some care is needed to avoid memory issues in very high-res analysis

## plotting
import matplotlib.pyplot as plt
from matplotlib import gridspec
import cmocean.cm as cmo
import matplotlib.colors as col
import cartopy.crs as ccrs
import matplotlib.path as mpath
from matplotlib import rc
rc('text', usetex=True)
rc('xtick', labelsize=25)
rc('ytick', labelsize=25)
rc('axes', labelsize=25)
```

You should also set up a `dask` client - this code uses the vanilla method for doing this. I suggest using Gadi and an entire node (48 cores) for this calculation.

```python
## once you've set up a dask-worker, connect to it, click the dashboard link to check worker status
from dask.distributed import Client
client = Client()
client
```

Client

Scheduler: tcp://127.0.0.1:38655

Dashboard: /proxy/8787/status

Cluster

Workers: 8

Cores: 48

Memory: 202.49 GB

I'll be using output from the ACCESS-OM2-01 RYF9091 spinup, which is still running as I'm writing this example.

```python
db = '/g/data/ik11/databases/ryf9091.db'
session = cc.database.create_session(db)
expt = '01deg_jra55v13_ryf9091'
```

#### The function below will compute  surface water-mass transformation rates for the southern ocean, averaged over an integer number of years of monthly resolution output. Below I have flagged some possible modifications you may want/need to employ:

- If you are not computing net surface heat fluxes from its components (here we define `net_surface_heating = sfc_hflux_from_runoff + sfc_hflux_coupler + sfc_hflux_pme + frazil_3d_int_z`), the function below will need to be modified to account for this.
- If you are not looking at the southern ocean, alter the `yt_ocean` slicing regime.
- You may want to use a different density binning array (`isopycnal_bins`)  if the phenomena you're interested in occurs in a shifted density space.
- We use $\sigma_1$ here because We are interested in dense waters that subduct to the Antarctic continental shelf <1000 m depth. You may want to use $\sigma_0$ $\sigma_2$ or something else. If so, alter the `gsw` function used, and define a more appropriate `isopycnal_bins` array.
- If you're looking at a non-interval number of years, the time averaging scheme will need to be modified.
- If you're online computed temperatures (SST) are potential temp (as opposed to conservative temp), use the gsw function `CT_from_pt` before the `sigma1` function as `sigma1` takes conservative temperature input (more info on the TEOS-10 gsw functions [here](http://www.teos-10.org/pubs/gsw/v3_04/html/gsw_contents.html)). This is the case for older ACCESS-OM2-01 runs (e.g. spinup6).
- On this `dask` setup and with the memory constraints of the VDI we don't think you would be able to do much more than a 5 year average of 0.1$^\circ$ data using this function. You are memory-limited by the numpy TEOS-10 gsw functions. If you need to average over more years, you mey have to do the TEOS-10 gsw translations separately and save these prior to computing transformation rates.

When analysing 0.1$^\circ$ output we tend to compute + save and reload surface water-mass transformation diagnostics in separate steps. In part this is because the computations take a while (~13min for 5 years of data below), but we also find that there's some instability (my workers are always killed) if we instead `return net_transformation, heat_transformation, salt_transformation` and then manipulate these variables straight away (a mystery for another mind). Saving and reloading works fine however.

```python
test = cc.querying.getvar(expt,'surface_temp',session, ncfile='ocean_month.nc')
test.time ## want to see what the latest years look like
```

<svg style="position: absolute; width: 0; height: 0; overflow: hidden">
<defs>
<symbol id="icon-database" viewBox="0 0 32 32">
<title>Show/Hide data repr</title>
<path d="M16 0c-8.837 0-16 2.239-16 5v4c0 2.761 7.163 5 16 5s16-2.239 16-5v-4c0-2.761-7.163-5-16-5z"></path>
<path d="M16 17c-8.837 0-16-2.239-16-5v6c0 2.761 7.163 5 16 5s16-2.239 16-5v-6c0 2.761-7.163 5-16 5z"></path>
<path d="M16 26c-8.837 0-16-2.239-16-5v6c0 2.761 7.163 5 16 5s16-2.239 16-5v-6c0 2.761-7.163 5-16 5z"></path>
</symbol>
<symbol id="icon-file-text2" viewBox="0 0 32 32">
<title>Show/Hide attributes</title>
<path d="M28.681 7.159c-0.694-0.947-1.662-2.053-2.724-3.116s-2.169-2.030-3.116-2.724c-1.612-1.182-2.393-1.319-2.841-1.319h-15.5c-1.378 0-2.5 1.121-2.5 2.5v27c0 1.378 1.122 2.5 2.5 2.5h23c1.378 0 2.5-1.122 2.5-2.5v-19.5c0-0.448-0.137-1.23-1.319-2.841zM24.543 5.457c0.959 0.959 1.712 1.825 2.268 2.543h-4.811v-4.811c0.718 0.556 1.584 1.309 2.543 2.268zM28 29.5c0 0.271-0.229 0.5-0.5 0.5h-23c-0.271 0-0.5-0.229-0.5-0.5v-27c0-0.271 0.229-0.5 0.5-0.5 0 0 15.499-0 15.5 0v7c0 0.552 0.448 1 1 1h7v19.5z"></path>
<path d="M23 26h-14c-0.552 0-1-0.448-1-1s0.448-1 1-1h14c0.552 0 1 0.448 1 1s-0.448 1-1 1z"></path>
<path d="M23 22h-14c-0.552 0-1-0.448-1-1s0.448-1 1-1h14c0.552 0 1 0.448 1 1s-0.448 1-1 1z"></path>
<path d="M23 18h-14c-0.552 0-1-0.448-1-1s0.448-1 1-1h14c0.552 0 1 0.448 1 1s-0.448 1-1 1z"></path>
</symbol>
</defs>
</svg>

xarray.DataArray

'time'

time: 2400

<input id='section-100a8d04-decf-4a19-a88f-7399a8af9895' class='xr-array-in' type='checkbox' ><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg>
1900-01-16 12:00:00 1900-02-15 00:00:00 ... 2099-12-16 12:00:00

array([cftime.DatetimeNoLeap(1900, 1, 16, 12, 0, 0, 0, 4, 16),
       cftime.DatetimeNoLeap(1900, 2, 15, 0, 0, 0, 0, 6, 46),
       cftime.DatetimeNoLeap(1900, 3, 16, 12, 0, 0, 0, 0, 75), ...,
       cftime.DatetimeNoLeap(2099, 10, 16, 12, 0, 0, 0, 0, 289),
       cftime.DatetimeNoLeap(2099, 11, 16, 0, 0, 0, 0, 3, 320),
       cftime.DatetimeNoLeap(2099, 12, 16, 12, 0, 0, 0, 5, 350)], dtype=object)

<input id='section-cf3cb6c6-1972-4733-8aad-0c90a25fff6a' class='xr-section-summary-in' type='checkbox'  checked>Coordinates: (1)

time

(time)

object

1900-01-16 12:00:00 ... 2099-12-16 12:00:00
<input id='attrs-5d853a73-1048-4fdf-9ba3-e2d0b7638c54' class='xr-var-attrs-in' type='checkbox' ><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg><input id='data-627bafff-5a41-413e-9352-b53c7c5a6400' class='xr-var-data-in' type='checkbox'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg>

long_name :

time

cartesian_axis :

T

calendar_type :

NOLEAP

bounds :

time_bounds

array([cftime.DatetimeNoLeap(1900, 1, 16, 12, 0, 0, 0, 4, 16),
       cftime.DatetimeNoLeap(1900, 2, 15, 0, 0, 0, 0, 6, 46),
       cftime.DatetimeNoLeap(1900, 3, 16, 12, 0, 0, 0, 0, 75), ...,
       cftime.DatetimeNoLeap(2099, 10, 16, 12, 0, 0, 0, 0, 289),
       cftime.DatetimeNoLeap(2099, 11, 16, 0, 0, 0, 0, 3, 320),
       cftime.DatetimeNoLeap(2099, 12, 16, 12, 0, 0, 0, 5, 350)], dtype=object)

<input id='section-d388a572-b9c4-4f8c-bb0e-5276bc50e266' class='xr-section-summary-in' type='checkbox'  checked>Attributes: (4)

long_name :

time

cartesian_axis :

T

calendar_type :

NOLEAP

bounds :

time_bounds

Spinup is 200 years; we'll compute surface water-mass transformation averaged over years 186-189 (2086-2089).

```python
def save_SWMT(expt, session, start_time, end_time, outpath, lat_north = -59, n = None):
    '''
    Computes southern ocean surface water-mass transformation rates (partitioned into transformation from heat
    and freshwater) referenced to 1000 db from monthly ACCESS-OM2 output.
    Suitable for analysis of high-resolution (0.1 degree) output (the scattered .load()'s allowed this)

    expt - text string indicating the name of the experiment
    session - a database session created by cc.database.create_session()
    start_time - text string designating the start month of the analysis ('YYYY-MM', e.g. '1905-01')
    end_time - text string indicating the end month of the analysis ('YYY-MM', e.g. '1905-12')
    outpath - text string indicating directory where output databases are to be saved (3 xarray databases, can
    modify to combine these if memory permits)
    lat_north - function computed processes between lat = -90 and lat = lat_north
    n - designate if a subset of output files is to be considered (see cc.querying.getvar)

    NOTE: this function assumes you are averaging over an integer number of years (though the start month
    need not be january, e.g. can have start_time = '1905-05', end_time = '1907-04' etc), modify if otherwise.

    NOTE: assumes surface_temp and surface_salt variables are in potential temperature (K) and practical
    salinity (PSU), simplifications may be made if conservative temperature (C) and absolute salinity (g/kg)
    are computed online

    required modules:
    xarray as xr
    numpy as np
    cosima_cookbook as cc
    from gsw import alpha, SA_from_SP, p_from_z, CT_from_pt, beta, sigma1
    '''
    ## getvar all required variables
    SST = cc.querying.getvar(expt,'surface_temp',session,ncfile='ocean_month.nc') - 273.15 # SST - conservative temperature in K (sheck this is the case for your run)
    SSS_PSU = cc.querying.getvar(expt,'surface_salt',session,ncfile='ocean_month.nc') # SSS - practical salinity (not absolute)
    pme_river = cc.querying.getvar(expt,'pme_river',session,ncfile='ocean_month.nc') # mass flux of precip - evap + river
    ## getvar the components of the net surface heat fux instead of the net_surface_heating variable
    sfc_hflux_from_runoff = cc.querying.getvar(expt,'sfc_hflux_from_runoff',session,ncfile='ocean_month.nc') # W/m2
    sfc_hflux_coupler = cc.querying.getvar(expt,'sfc_hflux_coupler',session,ncfile='ocean_month.nc') # W/m2
    sfc_hflux_pme = cc.querying.getvar(expt,'sfc_hflux_pme',session,ncfile='ocean_month.nc') # W/m2
    frazil_3d_int_z = cc.querying.getvar(expt,'frazil_3d_int_z',session,ncfile='ocean_month.nc') # W/m2
    geolon_t = cc.querying.getvar(expt,'geolon_t',session, n=1)
    geolat_t = cc.querying.getvar(expt,'geolat_t',session, n=1)
    ## slice for time and latitudinal constraints
    time_slice = slice(start_time,end_time)
    lat_slice = slice(-90,lat_north)
    SST = SST.sel(time=time_slice, yt_ocean=lat_slice)
    SSS_PSU = SSS_PSU.sel(time=time_slice, yt_ocean=lat_slice)
    pme_river = pme_river.sel(time=time_slice, yt_ocean=lat_slice)
    sfc_hflux_from_runoff = sfc_hflux_from_runoff.sel(time=time_slice, yt_ocean=lat_slice)
    sfc_hflux_coupler = sfc_hflux_coupler.sel(time=time_slice, yt_ocean=lat_slice)
    sfc_hflux_pme = sfc_hflux_pme.sel(time=time_slice, yt_ocean=lat_slice)
    frazil_3d_int_z = frazil_3d_int_z.sel(time=time_slice, yt_ocean=lat_slice)
    lon_t = geolon_t.sel(yt_ocean=lat_slice)
    lat_t = geolat_t.sel(yt_ocean=lat_slice)
    ## extract coordinate arrays
    yt_ocean = SST.yt_ocean.values
    xt_ocean = SST.xt_ocean.values
    time_monthly = SST.time.values
    ## construct an xarray of days per month (check this is relevant to your run), simple modification if non integer number of years analysed
    start_month = int(start_time[5:7])
    end_month = int(end_time[5:7])
    n_years = int(len(SST.time)/12)
    months_standard_noleap = np.array([31, 28, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31])
    if start_month != 1:
        months_offset_noleap = np.append(months_standard_noleap[(start_month-1):],months_standard_noleap[:(start_month-1)])
    else:
        months_offset_noleap = months_standard_noleap
    days_per_month = xr.DataArray(np.tile(months_offset_noleap, n_years), coords = [time_monthly], dims = ['time'], name = 'days per month')
    ## compute net surface heat flux from its component terms
    net_surface_heating = sfc_hflux_from_runoff+ sfc_hflux_coupler+ sfc_hflux_pme+ frazil_3d_int_z # W/m2
    ## now I use some TEOS-10 gsw functions to compute absolute salinity, then potential density fields
    ## these are numpy functions, if you have memory errors this is a good step to check (though I have found
    ## this works on the VDI for 0.1 degree data, might be issues for very long time periods)
    depth = -0.541281 # st_ocean value of the uppermost cell
    depth_tile = (lat_t*0+1)*depth
    pressure = xr.DataArray(p_from_z(depth_tile,lat_t), coords = [yt_ocean, xt_ocean], dims = ['yt_ocean', 'xt_ocean'], name = 'pressure', attrs = {'units':'dbar'})
    # convert units to absolute salinity
    SSS = xr.DataArray(SA_from_SP(SSS_PSU,pressure,lon_t,lat_t), coords = [time_monthly, yt_ocean, xt_ocean], dims = ['time','yt_ocean', 'xt_ocean'], name = 'sea surface salinity', attrs = {'units':'Absolute Salinity (g/kg)'})
    ## SST is already saved as conservative temperature in this run, if you are working with an older run with potential
    ## temperature saved, conversion will be required (make sure you work with C not K)
    # SST = xr.DataArray(CT_from_pt(SSS_AS,SST_PT), coords = [time_monthly, yt_ocean, xt_ocean], dims = ['time','yt_ocean', 'xt_ocean'], name = 'sea surface temperature', attrs = {'units':'Conservative Temperature (C)'})
    # compute potential density referenced to 1000dbar (or referenced otherwise, depending on your purpose)
    pot_rho_1 = xr.DataArray(sigma1(SSS, SST), coords = [time_monthly, yt_ocean, xt_ocean], dims = ['time','yt_ocean', 'xt_ocean'], name = 'potential density ref 1000dbar', attrs = {'units':'kg/m^3 (-1000 kg/m^3)'})
    pot_rho_1 = pot_rho_1.load()
    # Compute salt transformation (no density binning)
    haline_contraction = xr.DataArray(beta(SSS, SST, pressure), coords = [time_monthly, yt_ocean, xt_ocean], dims = ['time','yt_ocean', 'xt_ocean'], name = 'saline contraction coefficient (constant conservative temp)', attrs = {'units':'kg/g'})
    salt_transformation = haline_contraction*SSS*pme_river*days_per_month #! before was PSU, why?
    salt_transformation = salt_transformation.load()
    # Compute heat transformation (no density binning)
    thermal_expansion = xr.DataArray(alpha(SSS, SST, pressure), coords = [time_monthly, yt_ocean, xt_ocean], dims = ['time','yt_ocean', 'xt_ocean'], name = 'thermal expansion coefficient (constant conservative temp)', attrs = {'units':'1/K'})
    heat_transformation =  thermal_expansion*net_surface_heating*days_per_month
    heat_transformation = heat_transformation.load()
    # Record the time bounds before summing through time (just to make sure it's consistent with requested years)
    time_bounds =  str(salt_transformation.coords['time.year'][0].values)+'_'+str(salt_transformation.coords['time.month'][0].values)+'-'+str(salt_transformation.coords['time.year'][-1].values)+'_'+str(salt_transformation.coords['time.month'][-1].values)

    # Next section does a few things. It cycles through isopycnal bins, determines which cells are
    # within the given bin for each month, finds the transformation values for those cells for each month,
    # and sums these through time. You are left with an array of shape (isopyncal bins * lats * lons)
    # where the array associated with a given isopycnal bin is NaN everywhere except where pot_rho_1
    # was within the bin, there it has a time summed transformation value.

    isopycnal_bins = np.arange(31, 33.5, 0.02) ## alter if this density range doesn't capture surface processes in your study region, or if a different density field (not sigma1) is used

    bin_bottoms = isopycnal_bins[:-1]
    binned_salt_transformation = xr.DataArray(np.zeros((len(bin_bottoms), len(yt_ocean), len(xt_ocean))), coords = [bin_bottoms, yt_ocean,xt_ocean], dims = ['isopycnal_bins', 'yt_ocean', 'xt_ocean'], name = 'salt transformation in isopycnal bins summed over time')
    binned_salt_transformation.chunk({'isopycnal_bins':1})
    for i in range(len(isopycnal_bins)-1):
        bin_mask = pot_rho_1.where(pot_rho_1 <= isopycnal_bins[i+1]).where(pot_rho_1 > isopycnal_bins[i]) * 0 + 1
        masked_transform = (salt_transformation * bin_mask).sum(dim = 'time')
        masked_transform = masked_transform.where(masked_transform != 0)
        masked_transform = masked_transform.load()
        binned_salt_transformation[i,:,:] = masked_transform
    print('salt_transformation binning done')

    binned_heat_transformation = xr.DataArray(np.zeros((len(bin_bottoms), len(yt_ocean), len(xt_ocean))), coords = [bin_bottoms, yt_ocean,xt_ocean], dims = ['isopycnal_bins', 'yt_ocean', 'xt_ocean'], name = 'heat transformation in isopycnal bins summed over time')
    binned_heat_transformation.chunk({'isopycnal_bins':1})

    for i in range(len(isopycnal_bins)-1):
        bin_mask = pot_rho_1.where(pot_rho_1 <= isopycnal_bins[i+1]).where(pot_rho_1 > isopycnal_bins[i]) * 0 + 1
        masked_transform = (heat_transformation * bin_mask).sum(dim = 'time')
        masked_transform = masked_transform.where(masked_transform != 0)
        masked_transform = masked_transform.load()
        binned_heat_transformation[i,:,:] = masked_transform
    print('heat_transformation binning done')

    ndays = days_per_month.sum().values
    salt_transformation = binned_salt_transformation/ndays
    c_p = 3992.1
    heat_transformation = binned_heat_transformation/c_p/ndays

    isopycnal_bin_diff = np.diff(isopycnal_bins)
    salt_transformation = salt_transformation/isopycnal_bin_diff[:,np.newaxis,np.newaxis]
    heat_transformation = heat_transformation/isopycnal_bin_diff[:,np.newaxis,np.newaxis]
    isopycnal_bin_mid = (isopycnal_bins[1:] + isopycnal_bins[:-1])/2

    # this procedure defines fluxes from lighter to denser classes as negative, I want the opposite
    salt_transformation = salt_transformation *-1
    heat_transformation = heat_transformation *-1

    # Convert the binned (and summed through time) salt and heat transformation DataArrays to Datasets (to save metadata) and save to netCDF
    ds = xr.Dataset({'binned_salt_transformation': salt_transformation, 'time_bounds': time_bounds})
    ds.to_netcdf(outpath+'/ST_'+time_bounds+'.nc')
    ds = xr.Dataset({'binned_heat_transformation': heat_transformation, 'time_bounds': time_bounds})
    ds.to_netcdf(outpath+'/HT_'+time_bounds+'.nc')
    #############################################################
    net_transformation = heat_transformation + salt_transformation
    #############################################################
    del(heat_transformation, salt_transformation) ## unecessary for lower res or smaller time
    # wanted to rename the isopycnal bin (bottom edge) coord with the isopycnal bin midpoints...
    net_transformation.coords['isopycnal_bins'] = isopycnal_bin_mid
    ds = xr.Dataset({'surface_water_mass_transformation': net_transformation, 'time_bounds': time_bounds})
    ds.to_netcdf(outpath+'/SWMT_'+time_bounds+'.nc')
    del(net_transformation)
    return outpath, time_bounds ## helpful for re-loading
```

```python
%%time
## Make a temporary directory to stash a few files
!mkdir -p SWMT_temp
outpath = 'SWMT_temp/' # use a path where you have write permission
outpath, time_bounds = save_SWMT(expt, session, '2086-01', '2089-12', outpath)
```

    salt_transformation binning done
    heat_transformation binning done
    CPU times: user 4min 58s, sys: 3min 55s, total: 8min 53s
    Wall time: 10min 25s

## 4. Reload computed surface water-mass transformation

```python
def get_SWMT(outpath, time_bounds):

    net_transformation = xr.open_dataset(outpath + '/SWMT_' + time_bounds + '.nc', chunks={'isopycnal_bins':1})
    net_transformation = net_transformation.surface_water_mass_transformation
    heat_transformation = xr.open_dataset(outpath + '/HT_' + time_bounds + '.nc', chunks={'isopycnal_bins':1})
    heat_transformation = heat_transformation.binned_heat_transformation
    salt_transformation = xr.open_dataset(outpath + '/ST_' + time_bounds + '.nc', chunks={'isopycnal_bins':1})
    salt_transformation = salt_transformation.binned_salt_transformation

    return net_transformation, heat_transformation, salt_transformation
```

## 5. Plotting options: summed over southern ocean, south of 59S

One useful way of visualising the surface water-mass transformation metric is to sum the transformation rates (in density bins) over the analysis region, which here is the entire southern ocean south of 59S.

```python
area_t = cc.querying.getvar(expt, 'area_t', session, n=1) # needed for the plots I'll provide.
swmt, heat, salt = get_SWMT(outpath, time_bounds)
## sum over region, convert to Sv
isopycnal_bin_mid = swmt.isopycnal_bins
swmt_sum = (swmt * area_t/1e6).sum(['xt_ocean', 'yt_ocean']).values
heat_sum = (heat * area_t/1e6).sum(['xt_ocean', 'yt_ocean']).values
salt_sum = (salt * area_t/1e6).sum(['xt_ocean', 'yt_ocean']).values
```

```python
figure = plt.figure(num=1, figsize = (10, 10))
plt.plot(swmt_sum,isopycnal_bin_mid, color = 'k', label='net')
plt.plot(heat_sum,isopycnal_bin_mid, color = 'r', label='heat')
plt.plot(salt_sum,isopycnal_bin_mid, color = 'b', label='salt')
plt.plot([0, 0],[31, 33.2], 'k', linewidth=0.5)
plt.ylim((33.2, 31))
plt.ylabel(r'$\sigma_{1}$ (kg m$^{-3}$)', fontsize = 30)
plt.xlabel('Surface water-mass transformation (Sv)', fontsize=30)
plt.legend(loc=3, fontsize = 25);
```

Here the positive peak in the high density classes indicates a rate of subduction due to surface densification processes (predominantly freshwater fluxes associated with sea-ice processes in this case). The negative peak indicates upwelling rates.

## 6. Plotting options: Antarctic shelf dense water formation

You might be interested in dense water formation on the Anarctic continental shelf. Below we've outlined a procedure whereby you can identify the density of subducting waters on the continental shelf, and map the locations where this sudbuction occurs.

You will need a way of masking for the continental shelf. You might just use a simple depth criterion, but here I define the shelf region using a mask that selects cells poleward of a continuous approximation of the 1000 m isobath surrounding Antartcica.

```python
def shelf_mask_isobath(var):
    '''
    Masks ACCESS-OM2-01 variables by the region polewards of the 1000m isobath as computed using
    a script contributed by Adele Morrison.
    Only to be used with ACCESS-OM2-0.1 output!
    '''
    contour_file = np.load('/g/data/ik11/grids/Antarctic_slope_contour_1000m.npz')

    shelf_mask = contour_file['contour_masked_above']
    yt_ocean = contour_file['yt_ocean']
    xt_ocean = contour_file['xt_ocean']

    # in this file the points along the isobath are given a positive value, the points outside (northwards)
    # of the isobath are given a value of -100 and all the points on the continental shelf have a value of 0
    # so we mask for the 0 values
    shelf_mask[np.where(shelf_mask!=0)] = np.nan
    shelf_mask = shelf_mask+1
    shelf_map = np.nan_to_num(shelf_mask)
    shelf_mask = xr.DataArray(shelf_mask, coords = [('yt_ocean', yt_ocean), ('xt_ocean', xt_ocean)])
    shelf_map = xr.DataArray(shelf_map, coords = [('yt_ocean', yt_ocean), ('xt_ocean', xt_ocean)])

    # then we want to multiply the variable with the mask so we need to account for the shape of the mask.
    # The mask uses a northern cutoff of 59S.
    masked_var = var.sel(yt_ocean = slice(-90, -59.03)) * shelf_mask
    return masked_var, shelf_map
```

Below we've plotted the positioning of this isobath boundary.

```python
ht = cc.querying.getvar(expt,'ht', session, n=1)
ht = ht.sel(yt_ocean = slice(-90, -59))
land_mask = (ht*0).fillna(1)
yt_ocean = ht.yt_ocean.values
xt_ocean = ht.xt_ocean.values
ht_shelf , shelf_mask = shelf_mask_isobath(ht)
```

```python
fig = plt.figure(num=1,figsize=(10, 10))
ax = plt.subplot(projection=ccrs.SouthPolarStereo())

ax.contour(xt_ocean, yt_ocean, land_mask,[0,1], colors = 'k', alpha = 0.4, transform=ccrs.PlateCarree())
ax.contour(xt_ocean, yt_ocean, shelf_mask.values, [0, 1], colors = 'r', transform=ccrs.PlateCarree())
ax.set_extent([-180, 180, -90, -59], ccrs.PlateCarree())
```

Now we'll repoduce the previous surface water-mass transformation plot but for the continental shelf region only.

```python
swmt_shelf, shelf_mask = shelf_mask_isobath(swmt)
heat_shelf, shelf_mask = shelf_mask_isobath(heat)
salt_shelf, shelf_mask = shelf_mask_isobath(salt)
area_t_shelf, shelf_mask = shelf_mask_isobath(area_t)
swmt_shelf_sum = (swmt_shelf * area_t_shelf/1e6).sum(['xt_ocean','yt_ocean']).values
heat_shelf_sum = (heat_shelf * area_t_shelf/1e6).sum(['xt_ocean','yt_ocean']).values
salt_shelf_sum = (salt_shelf * area_t_shelf/1e6).sum(['xt_ocean','yt_ocean']).values
```

```python
figure = plt.figure(num = 1, figsize = (10, 10))

plt.plot(swmt_shelf_sum, isopycnal_bin_mid, color = 'k',label='net')
plt.plot(heat_shelf_sum, isopycnal_bin_mid, color = 'r',label='heat')
plt.plot(salt_shelf_sum, isopycnal_bin_mid, color = 'b',label='salt')
plt.plot([0, 0], [31, 33.2], 'k', linewidth=0.5)
plt.ylim((33.2, 31))
plt.ylabel(r'$\sigma_{1}$ (kg m$^{-3}$)', fontsize = 30)
plt.xlabel('Surface water-mass transformation (Sv)', fontsize = 30)
plt.legend(loc=1, fontsize = 25);
```

This shows us that contienntal shelf surface waters are made denser (almost entirely by sea-ice freshwater fluxes) and subduct away from the surface at a rate of over 10 Sv! Where does this happen? If we map the surface water-mass transformation rate across the density class experiences the peak positive transformation rate when summed over the shelf, we will be able to see where waters are subducting.

```python
max_transformation_index = np.argmax(swmt_shelf_sum)
max_transformation_density = isopycnal_bin_mid[max_transformation_index]
print('Shelf subduction density:',max_transformation_density.values)

shelf_subduction_plot = swmt_shelf.isel(isopycnal_bins = max_transformation_index)*1e5
swmt_xt = shelf_subduction_plot.xt_ocean
swmt_yt = shelf_subduction_plot.yt_ocean
```

    Shelf subduction density: 32.46999999999997

```python
fig  = plt.figure(1, figsize = (20, 10))
gs = gridspec.GridSpec(1,2, width_ratios = [3, 2])
gs.update(wspace = 0.05)

ax, ax1 = plt.subplot(gs[0], projection=ccrs.SouthPolarStereo()), plt.subplot(gs[1])

ax.contour(xt_ocean, yt_ocean,land_mask, [0, 1],
           colors = 'k', alpha = 0.3, transform = ccrs.PlateCarree() )
ax.contour(shelf_mask.xt_ocean, shelf_mask.yt_ocean, shelf_mask, [0, 1],
           colors='k', alpha =0.8, transform = ccrs.PlateCarree())

norm = col.Normalize(vmin=0, vmax=2.5)

plot_swmt = ax.pcolormesh(swmt_xt, swmt_yt, shelf_subduction_plot,
                          vmin=0, vmax=2.5, cmap=cmo.matter,transform=ccrs.PlateCarree())

theta = np.linspace(0, 2*np.pi, 100)
center, radius = [0.5, 0.5], 0.5
verts = np.vstack([np.sin(theta), np.cos(theta)]).T
circle = mpath.Path(verts * radius + center)
#ax.set_boundary(circle, transform=ax.transAxes)
ax.set_extent([-180, 180, -90, -59], ccrs.PlateCarree())
ax.outline_patch.set_visible(False)

cax = fig.add_axes([0.27, 0.13, 0.2, 0.04])
cbar=plt.colorbar(plot_swmt, cax=cax, orientation='horizontal', shrink = 0.5, ticks = [0, 0.5, 1, 1.5, 2, 2.5, 3])
cbar.set_label(r'Surface water-mass transformation ($\times 10^{-5}$m s$^{-1}$)', fontsize = 25)

ax1.plot(swmt_shelf_sum,isopycnal_bin_mid, color = 'k',label='net')
ax1.plot(heat_shelf_sum,isopycnal_bin_mid, color = 'r',label='heat')
ax1.plot(salt_shelf_sum,isopycnal_bin_mid, color = 'b',label='salt')
ax1.plot([0, 0],[31, 33.2], 'k', linewidth=0.5)
ax1.plot([-5, 15],[max_transformation_density, max_transformation_density],'k--', linewidth=1)
ax1.set_ylim((33.2, 31))
ax1.set_xlim((-1.5, 12.1))
ax1.set_ylabel(r'$\sigma_{1}$ (kg m$^{-3}$)', fontsize = 30)
ax1.set_xlabel('Surface water-mass transformation (Sv)', fontsize = 30)
ax1.legend(loc=1, fontsize = 25);
```

The map in the left panel shows the spatial distribution of surface water-mass transformation across the density class indicated with the dashed line in the right panel.


```python
from numpy import roots, random, array, linspace
from numpy import fabs as fb
from math import atan, fabs, pi, tan, cos, sin, sqrt, tan
import sympy
from sympy import Symbol

def cubic (Qx, Qy, Ux, Uy):

    a = Uy
    b = (Ux + 2*Qy)
    c = (2*Qx - Uy)
    d = -Ux

    det = -4*b*b*b*d + b*b*c*c - 4*a*c*c*c + 18*a*b*c*d - 27*a*a*d*d

    if (det < 0):
        return 'c'

    if (det > 0):

        a = roots([a, b, c, d])
        a = a.real

        a = [atan(a[0]), atan(a[1]), atan(a[2])]

        U = [Ux*cos(a[0]) + Uy*sin(a[0]), Ux*cos(a[1]) + Uy*sin(a[1]), Ux*cos(a[2]) + Uy*sin(a[2])]
        rightU = [2*sin(a[0])*cos(a[0]), 2*sin(a[1])*cos(a[1]), 2*sin(a[2])*cos(a[2])]

        for i in range(0, 3):
            if (U[i] * rightU[i] < 0):
                a[i] = a[i] + pi

        print a[0]*(180/pi), a[1]*(180/pi), a[2]*(180/pi)

        a = sorted(a)
        a = [a[0] - a[0], a[1] - a[0], a[2] - a[0]]

        if (a[2] > pi):
            return 'a'
        else:
            return 'b'
```

```python
mu, sigma = 0, 1
```

```python
Ux = 1.0
Uy = 1.0
Qx = -1.0
Qy = 1.1
```

```python
print cubic(Qx, Qy, Ux, Uy)
```

    -75.6291843111 44.0609744917 165.171028792
    a

```python
U = array([-2*Ux + 2*Uy, -Ux + 2*Uy, 2*Uy, Ux + 2*Uy, 2*Ux + 2*Uy,-2*Ux + Uy, -Ux + Uy, Uy, Ux + Uy, 2*Ux + Uy,-2*Ux, -Ux, 0.0, Ux, 2*Ux,-2*Ux - Uy, -Ux - Uy, -Uy, Ux - Uy, 2*Ux - Uy,-2*Ux - 2*Uy, -Ux - 2*Uy, -2*Uy, Ux - 2*Uy, 2*Ux - 2*Uy])
Q = array([-2*Qx + 2*Qy, -Qx + 2*Qy, 2*Qy, Qx + 2*Qy, 2*Qx + 2*Qy,-2*Qx + Qy, -Qx + Qy, Qy, Qx + Qy, 2*Qx + Qy,-2*Qx, -Qx, 0.0, Qx, 2*Qx,-2*Qx - Qy, -Qx - Qy, -Qy, Qx - Qy, 2*Qx - Qy,-2*Qx - 2*Qy, -Qx - 2*Qy, -2*Qy, Qx - 2*Qy, 2*Qx - 2*Qy])
P_x = Ux * U + Q * Qx
P_y = Uy * U + Q * Qy
```

```python
P1_x = array([-2, -1, 0, 1, 2, -2, -1, 0, 1, 2, -2, -1, 0, 1, 2, -2, -1, 0, 1, 2, -2, -1, 0, 1, 2])
P1_y = array([2, 2, 2, 2, 2, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, -1, -1, -1, -1, -1, -2, -2, -2, -2, -2])

P2_x = P1_x + P_x*0.1*np.reciprocal(fb(P_x))
P2_y = P1_y + P_y*0.1*np.reciprocal(fb(P_y))

P3_x = P1_x - P_x*0.1*np.reciprocal(fb(P_x))
P3_y = P1_y - P_y*0.1*np.reciprocal(fb(P_y))
```

    /Users/heyfaraday/anaconda/envs/new2/lib/python2.7/site-packages/ipykernel/__main__.py:4: RuntimeWarning: divide by zero encountered in reciprocal
    /Users/heyfaraday/anaconda/envs/new2/lib/python2.7/site-packages/ipykernel/__main__.py:4: RuntimeWarning: invalid value encountered in multiply
    /Users/heyfaraday/anaconda/envs/new2/lib/python2.7/site-packages/ipykernel/__main__.py:5: RuntimeWarning: divide by zero encountered in reciprocal
    /Users/heyfaraday/anaconda/envs/new2/lib/python2.7/site-packages/ipykernel/__main__.py:5: RuntimeWarning: invalid value encountered in multiply
    /Users/heyfaraday/anaconda/envs/new2/lib/python2.7/site-packages/ipykernel/__main__.py:7: RuntimeWarning: divide by zero encountered in reciprocal
    /Users/heyfaraday/anaconda/envs/new2/lib/python2.7/site-packages/ipykernel/__main__.py:7: RuntimeWarning: invalid value encountered in multiply
    /Users/heyfaraday/anaconda/envs/new2/lib/python2.7/site-packages/ipykernel/__main__.py:8: RuntimeWarning: divide by zero encountered in reciprocal
    /Users/heyfaraday/anaconda/envs/new2/lib/python2.7/site-packages/ipykernel/__main__.py:8: RuntimeWarning: invalid value encountered in multiply

```python
P_x*0.1*np.reciprocal(fb(P_x))
```

    /Users/heyfaraday/anaconda/envs/new2/lib/python2.7/site-packages/ipykernel/__main__.py:1: RuntimeWarning: divide by zero encountered in reciprocal
      if __name__ == '__main__':
    /Users/heyfaraday/anaconda/envs/new2/lib/python2.7/site-packages/ipykernel/__main__.py:1: RuntimeWarning: invalid value encountered in multiply
      if __name__ == '__main__':

    array([-0.1, -0.1, -0.1,  0.1,  0.1, -0.1, -0.1, -0.1,  0.1,  0.1, -0.1,
           -0.1,  nan,  0.1,  0.1, -0.1, -0.1,  0.1,  0.1,  0.1, -0.1, -0.1,
            0.1,  0.1,  0.1])

```python
print P2_x
print P2_y
```

    [-2.1 -1.1 -0.1  1.1  2.1 -2.1 -1.1 -0.1  1.1  2.1 -2.1 -1.1  nan  1.1  2.1
     -2.1 -1.1  0.1  1.1  2.1 -2.1 -1.1  0.1  1.1  2.1]
    [ 2.1  2.1  2.1  2.1  2.1  1.1  1.1  1.1  1.1  1.1  0.1  0.1  nan -0.1 -0.1
     -1.1 -1.1 -1.1 -1.1 -1.1 -2.1 -2.1 -2.1 -2.1 -2.1]

```python
print P_x*np.reciprocal(P_y)
```

    [ -0.90909091  -0.48672566  -0.04524887   0.41666667   0.90047393
      -1.70124481  -0.90909091  -0.04524887   0.90047393   1.94029851 -20.         -20.
              nan -20.         -20.           1.94029851   0.90047393
      -0.04524887  -0.90909091  -1.70124481   0.90047393   0.41666667
      -0.04524887  -0.48672566  -0.90909091]

    /Users/heyfaraday/anaconda/envs/new2/lib/python2.7/site-packages/ipykernel/__main__.py:1: RuntimeWarning: divide by zero encountered in reciprocal
      if __name__ == '__main__':
    /Users/heyfaraday/anaconda/envs/new2/lib/python2.7/site-packages/ipykernel/__main__.py:1: RuntimeWarning: invalid value encountered in multiply
      if __name__ == '__main__':

```python
P2_x
```

    array([-2.1, -1.1, -0.1,  1.1,  2.1, -2.1, -1.1, -0.1,  1.1,  2.1, -2.1,
           -1.1,  nan,  1.1,  2.1, -2.1, -1.1,  0.1,  1.1,  2.1, -2.1, -1.1,
            0.1,  1.1,  2.1])

```python
import matplotlib.pyplot as plt
%matplotlib inline
```

```python
plt.plot(P1_x, P1_y, 'ro')
plt.plot(P2_x, P2_y, 'go')
plt.plot(P3_x, P3_y, 'bo')

plt.show()
```

```python
Ux = random.normal(mu, sigma, 1000000)
Uy = random.normal(mu, sigma, 1000000)
Qx = random.normal(mu, sigma, 1000000)
Qy = random.normal(mu, sigma, 1000000)
```

```python
a, b, c = 0, 0, 0

for i in range(0, 100000):
    ch = ''
    ch = cubic(Qx[i], Qy[i], Ux[i], Uy[i])
    if (ch == 'a'):
        a = a + 1
    if (ch == 'b'):
        b = b + 1
    if (ch == 'c'):
        c = c + 1

print a/100000.0, b/100000.0, c/100000.0
```

    0.52896 0.09177 0.37927

```python
P_x
```

    array([  6.,  11.,  16.,  21.,  26.,  -2.,   3.,   8.,  13.,  18., -10.,
            -5.,   0.,   5.,  10., -18., -13.,  -8.,  -3.,   2., -26., -21.,
           -16., -11.,  -6.])

```python
P_y
```

    array([ 10.,  18.,  26.,  34.,  42.,  -3.,   5.,  13.,  21.,  29., -16.,
            -8.,   0.,   8.,  16., -29., -21., -13.,  -5.,   3., -42., -34.,
           -26., -18., -10.])

```python

```


# Econometrics HW_01
**11510691 程远星$\DeclareMathOperator*{\argmin}{argmin}
\newcommand{\using}[1]{\stackrel{\mathrm{#1}}{=}}
\newcommand{\ffrac}{\displaystyle \frac}
\newcommand{\space}{\text{ }}
\newcommand{\bspace}{\;\;\;\;}
\newcommand{\QQQ}{\boxed{?\:}}
\newcommand{\void}{\left.\right.}
\newcommand{\CB}[1]{\left\{ #1 \right\}}
\newcommand{\SB}[1]{\left[ #1 \right]}
\newcommand{\P}[1]{\left( #1 \right)}
\newcommand{\dd}{\mathrm{d}}
\newcommand{\Tran}[1]{{#1}^{\mathrm{T}}}
\newcommand{\d}[1]{\displaystyle{#1}}
\newcommand{\EE}[2][\,\!]{\mathbb{E}_{#1}\left[#2\right]}
\newcommand{\Var}[2][\,\!]{\mathrm{Var}_{#1}\left[#2\right]}
\newcommand{\Cov}[2][\,\!]{\mathrm{Cov}_{#1}\left(#2\right)}
\newcommand{\Corr}[2][\,\!]{\mathrm{Corr}_{#1}\left(#2\right)}
\newcommand{\I}[1]{\mathrm{I}\left( #1 \right)}
\newcommand{\N}[1]{\mathrm{N} \left( #1 \right)}
\newcommand{\ow}{\text{otherwise}}$**

## Question 1

$\P{1}$

$\bspace\begin{cases}
H_0: \beta_3 = 0, & \text{null hypothesis} \\
H_1: \beta_3 \neq 0, & \text{alternative hypothesis}
\end{cases}$

$\P{2}$

$\bspace$Intuitively they should be positive. Because larger population will cause an increse in the demand of housing, or more specificly, rental since it's a college town. Moreover, when the average income is higher, it's nature to predict that rental market will adjust itself to a higher rent, thus $\beta_1$ and $\beta_2$ are both positive.

$\P{3}$

$\bspace$It's wrong since the coefficient for $\texttt{population}$ is an elasticity rather a normal slope parameter.

$\P{4}$

$\bspace$Referring the probability table with $\texttt{df} = 60$ and $1\%$ confidence level, the critical value is $2.660$. Then the $\texttt{t-}$statistics is:

$$t = \ffrac{\hat\beta_3} {\text{se}\P{\hat\beta_3}} = \ffrac{0.0056} {0.0017}=3.2941$$

$\bspace$way higher than the critical value, thus conclusion can be drawn that slope parameter $\beta_3$ is statistically different from $0$ at $1\%$ confidence level.

## Question 2

$\P{1}$

$$\begin{align}
\Var{\hat\beta_1 - 3 \hat\beta_2} &= \Cov{\hat\beta_1,\hat\beta_1} + \P{-3}^2 \Cov{\hat\beta_2,\hat\beta_2} -3 \Cov{\hat\beta_1,\hat\beta_2} -3 \Cov{\hat\beta_1,\hat\beta_2}\\
&= \Var{\hat\beta_1} + 9\Var{\hat\beta_2} -6 \Cov{\hat\beta_1,\hat\beta_2}
\end{align}$$

$\P{2}$

$$t = \ffrac{\hat\beta_1 - 3\hat\beta_2 -  1} {\text{se}\P{\hat\beta_1 - 3\hat\beta_2}} = \ffrac{\hat\beta_1 - 3\hat\beta_2 -  1} {\sqrt{\P{\text{se}\P{\hat\beta_1}^2} + 9\P{\text{se}\P{\hat\beta_2}^2} - 6\Cov{\hat\beta_1, \hat\beta_2}}}$$

$\P{3}$

$$\begin{align}
y &= \beta_0 + \P{\theta_1 + 3 \beta_2} x_1 + \beta_2 x_2 + \beta_3 x_3 + u \\
&= \beta_0 + \theta_1 x_1 + \beta_2 \P{x_2 + 3x_1} + \beta_3 x_3 + u \\
\end{align}$$

## Question 3

$\P{1}$

$\bspace$We first refer to the statistic table and find the critical value of $\text{df} = 120$ (closet one to $n = 142-4-1 = 137$) and $5\%$ confidence level, it's 2.45. Then the $\texttt{F-}$value is:

$$F = \ffrac{\ffrac{R^2} {k}} {\ffrac{1-R^2} {n-k-1}}=1.409$$

$\bspace$way less than the critical value, so that we fail to reject the null hypothesis, can't tell anything. Also,

$$t_1 = \ffrac{\hat\beta_1} {\text{se}\P{\hat\beta_1}} = \ffrac{0.321} {0.201}=1.597\\
t_2 = \ffrac{\hat\beta_2} {\text{se}\P{\hat\beta_2}} = \ffrac{0.043} {0.078}=0.551\\
t_3 = \ffrac{\hat\beta_3} {\text{se}\P{\hat\beta_3}} = \ffrac{0.0051} {0.0047}=1.085\\
t_4 = \ffrac{\hat\beta_4} {\text{se}\P{\hat\beta_4}} = \ffrac{0.0035} {0.0022}=1.591$$

$\bspace$no explanatory variable is individually significant at $5\%$ confidence level since the one with biggest statistic is still less than $1.980$ from the statistic table of $5\%$ confidence level and $120$ degree of freedom, two-sided.

$\P{2}$

$\bspace$Use the similar method we can find that $F = 1.169$. Boo-Nah... No changes from the previous conclusion.

$\P{3}$

$\bspace$There'll be error when we try to find the value of $\log\P{\texttt{dkr}}$! It's out of log functions' domain.

$\P{4}$

$\bspace$Week.

- At $5\%$ confidency level, no individual variables is significant, jointly as well
- Low $R^2$ meaning that the variance of the dependent variable almost can't be explained by the independent variables