# Managing Bond Portfolios
## Interest rate risk
1. There's an INVERSE relationship between prices and yields
2. **Convexity**: An increase in YTM results in a smaller price change than a decrease in YTM of equal magnitude.
3. Prices are **more sensitive** to interest rate changes for the bonds with:
    - Longer maturity.
    - Lower coupon rate.
    - Lower YTM.
4.

**Duration**: a measure of the effective maturity of a bond, defined as the **weighted average** of the times until each payment, with **weights proportional to the present value** of the payments.

$$D = \sum_{t=1}^{T}w_t \times t \\
w_t = \frac{\frac{\text{Cash Flow}} {(1+y)^t}} {\text{Bond Price}} \\
D_{Modified} = D^*= \frac{D} {1+\text{YTM}}$$

>**e.g.** $8\%$ coupon, annully payment, $\text{YTM} = 10\%$
>
>$$\begin{align}
\text{Bond Price} &= \frac{\text{Cash Flow}_1} {(1+y)} + \frac{\text{Cash Flow}_2} {(1+y)^2} + \frac{\text{Cash Flow}_3} {(1+y)^3} \\
&= \frac{80} {(1+0.1)} + \frac{80} {(1+0.1)^2} + \frac{1080} {(1+0.1)^3} \\
&= 950.26
\end{align}$$

>$$\begin{align}
\text{Duration} &= D \\
&= \frac{1} {\text{Bond Price}} \left( 1\text{ year } \times \frac{\text{Cash Flow}_1} {(1+y)} + 2\text{ year } \times \frac{\text{Cash Flow}_2} {(1+y)^2} + 3\text{ year } \times \frac{\text{Cash Flow}_3} {(1+y)^3} \right)\\
&= \frac{1} {950.26} \left( 1\times \frac{80} {(1+0.1)} + 2 \times \frac{80} {(1+0.1)^2} + 3 \times \frac{1080} {(1+0.1)^3} \right)\\
&= 2.7774\text{ year }
\end{align}$$

***
Modified Duration the sensitivity of bond price to interest rate risk, and we define

$$\frac{\Delta P} {P} = -D \times \frac{\Delta(1+y)} {(1+y)}= -D^* \times \Delta y \Longrightarrow D^* = - \frac{
\frac{\Delta P} {P}} {\Delta y} = - \frac{\text{percentage change in bond price}} {\text{change in YTM}}$$

>**e.g.** Comparison between several methods
>
>For a 8% coupon bond, with initial $\text{YTM} = 10\%$, now it changes to $10.01\%$, a small change.
>
>$$\begin{array}{c}\hline
\text{YTM} = 10\% \\ \hline
\begin{array}{ccccc}\hline
\text{Time} & \text{Payment} & \text{Payment Discount} & \text{Weight} & \text{Weight Sum} \\ \hline
1 & 80 & 72.727 & 0.0765 & 0.0765 \\
2 & 80 & 66.116 & 0.0696 & 0.1392 \\
3 & 1080 & 811.420 & 0.8539 & 2.5617 \\
\end{array} \\ \hline
\text{Sum} = 950.263 \\ \hline \hline
\text{YTM} = 10.01\% \\ \hline
\begin{array}{ccccc} \hline
\text{Time} & \text{Payment} & \text{Payment Discount} & \text{Weight} & \text{Weight Sum} \\ \hline
1 & 80 & 72.721 & 0.0765 & 0.0765 \\
2 & 80 & 66.104 & 0.0696 & 0.1392 \\
3 & 1080 & 811.199 & 0.8539 & 2.5616 \\
\end{array} \\ \hline
\text{Sum} = 950.0231 \\ \hline
\end{array}$$
>
>So that the actual percentage price change is $\newcommand{\ffrac}{\displaystyle \frac}\ffrac{\Delta P} {P} = \ffrac{950.0231 - 950.263} {950.263} = -0.0252\%$
>***
>Now we using zero-coupon bond. So we already know the maturity is $2.7774 \text{ years}$
>
>$$P_0 = \frac{1000} {1.10^2.7774} = 767.425, P_1 = \frac{1000} {1.1001^2.7774} = 767.2313$$
>
>$$\frac{\Delta P} {P} = \frac{767.2313 - 767.425} {767.425} = -0.0252\%$$
>***
>Now using the Duration Rule
>
>$$\frac{\Delta P} {P} = -D^* \times \Delta y = \frac{-2.7774} {1.10} \times 0.0001 = -0.0252\%$$
>
>So for this kind of small change in YTM, all these three methods get a similar result.

1. Equal-duration assets are equally sensitive to interest rate movements.
2. The duration rule works pretty well for small changes in YTM.
3. Bonds portfolio with Diversified term will lower your interest rate risk.

Several Rules
1. Duration of a zero-coupon bond equals its time to maturity.
2. Duration is higher when coupon rate is lower.
3. Duration increases with maturity for bonds selling at par or at a premium.
    - This means that coupon rate exceeds the market rate and a capital loss in the future
4. Duration is higher when YTM is lower.
    - And annual bond get higher duration than semi-annual bond
5. Duration of perpetuity: $\ffrac{1+y} {y}$

## Passive Bond Management
### Immunization

**Immunization**: A duration-matching strategy to shield net worth from interest rate movement.

**Interest rate risk**: mismatch between the maturities of assets and liabilities;
1. Liabilities: customers’ deposits, short-term, short duration.
2. Assets: commercial loans, long-term, long duration.

So that when interest rate goes up, your liability goes down and your asset goes up. However if it goes down, their net worth goes down, the assets falls more in the value than their liabilities.

>**e.g.** Interest rate initially $8\%$
>
>- Liability: $5$ years guaranteed investment contract, just like the zero-coupon bond, $D=5$; and promise to pay $10,000 \times (1.08)^5 = 14,693.28$ after that
- Asset: $6\text{-year}$ coupon bond, $8\%$ coupon rate and $\text{YTM} = 8\%$. After some calculation, $D=4.99$.
>
>So now no matter how rate changes, the accumulated value of invested payment actually doesn't change that much.
>
>- If the interest rate keeps at a $8\%$ level, the accumulated investment payment is
>
>$$ {800} \times {(1.08)^4} + {800} \times {(1.08)^3} + {800} \times {(1.08)^2} + {800} \times {(1.08)^1} + {800} \times {(1.08)^0} + \frac{10800} {(1.08)^1} = 14693.28$$
>
>- And if the interest rate falls to $7\%$, the accumulated investment payment is
>
>$$ {800} \times {(1.07)^4} + {800} \times {(1.07)^3} + {800} \times {(1.07)^2} + {800} \times {(1.07)^1} + {800} \times {(1.07)^0} + \frac{10800} {(1.07)^1} = 14694.05$$
>
>- And if the interest rate increase to $9\%$, the accumulated investment payment is
>
>$$ {800} \times {(1.09)^4} + {800} \times {(1.09)^3} + {800} \times {(1.09)^2} + {800} \times {(1.09)^1} + {800} \times {(1.09)^0} + \frac{10800} {(1.09)^1} = 14696.02$$

***
How to make that portfolio?
**Rebalancing**: Realigning the proportions of assets in a portfolio to rematch the durations of assets and liabilities.

>An insurance company must pay $\$19,487$ in $7$ years. The market interest rate is $10\%$. So the PV of the obligation is $\$10,000$. The manager wishes to fund the obligation using $3\text{-year}$ zero-coupon bonds and perpetuities paying annual coupons.
>
>$$D_{perpetuity} = \frac{1+y} {y} = 11 \\[1em]
D_{portfolio} = w \times 3 + (1-w) \times 11 = D_{liability} = 7 \Rightarrow w = 0.5$$
>
>The manager purchases $\$5,000$ PV of zero-coupon bonds and $\$5,000$ PV of perpetuity. Note that the face value of the zeros will be $\$5,000 \times 1.1^3 = \$6,655$; the perpetuity pays $\$5,000 \times 10\% = \$500$ annually.

### Cash Flow Matching and Dedication
**cash flow matching**: Matching cash flows from a fixed-income portfolio with those of an obligation.

**dedication strategy**: Refers to multiperiod cash flow matching.

The advantage of dedication over immunization is no need for rebalancing.

However it is not widely persued, because it's constraint on bonds while perpetuity is not available.

## Convexity

Duration only apply to small change, so when there's a big change in interest rate, **convexity** is required to be taken into account, which is the curvature of the price-yield relationship of a bond.

$$\frac{\Delta P} {P} = -D^* \times \Delta y + \frac{1} {2} \times Convex \times \left( \Delta y \right)^2 \\[1em]
D^* = \frac{1} {1+y} \sum_{t=1}^{T} w_t \times t \\[1em]
Convex = \frac{1} {\left( 1+y \right)^2} \sum_{t=1}^{T} w_t \times t(t+1) \\[1em]
w_t = \frac{CF_t} {(1+y)^t} \Big/ \text{Price}
$$

>**e.g.** $T=3$, Annual coupon rate: $10\%$, YTM: $10\%$, Par value: $100$.
>
>$$D^* = \frac{1} {1.1} \times \frac{\ffrac{10} {1.1} \times 1 + \ffrac{10} {1.1^2} \times 2 + \ffrac{110} {1.1^3} \times 3 }  {\ffrac{10} {1.1} + \ffrac{10} {1.1^2} + \ffrac{110} {1.1^3}} = 2.487$$
>
>$$Convex = \frac{1} {1.1^2} \times\frac{\ffrac{10} {1.1} \times 1 \times 2 + \ffrac{10} {1.1^2} \times 2 \times 3 + \ffrac{110} {11.1^3} \times 3 \times 4 } {\ffrac{10} {1.1} + \ffrac{10} {1.1^2} + \ffrac{110} {1.1^3}} = 8.756$$
>
>> $P_0 = \ffrac{10} {1.1} + \ffrac{10} {1.1^2} + \ffrac{110} {1.1^3} = 100$
>>
> small changes in YTM: $10\% \to 9.9\%$
>>
>> $P_1 = \ffrac{10} {1.099} + \ffrac{10} {1.099^2} + \ffrac{110} {1.099^3} = 100.2491$
>>
>> Real price change: $\ffrac{P_1 - P_0} {P_0} = 0.2491\%$
>>
>> Duration Rule: $\ffrac{\Delta P} {P} = -D^* \times \Delta y = -2.487 \times (-0.001) = 0.2487\%$

>> With Convexity: $\ffrac{\Delta P} {P} = -D^* \times \Delta y + \frac{1} {2} \times Convex \times \left( \Delta y \right)^2 = -2.487 \times (-0.001) + \ffrac{1} {2} \times 8.756 \times (-0.001)^2 = 0.2491\% $.
>>***
big changes in YTM: $10\% \to 5\%$
>>
>> $P_1 = \ffrac{10} {1.05} + \ffrac{10} {1.05^2} + \ffrac{110} {1.05^3} = 113.62$
>>
>> Real price change: $\ffrac{P_1 - P_0} {P_0} = 13.62\%$
>>
>> Duration Rule: $\ffrac{\Delta P} {P} = -D^* \times \Delta y = -2.487 \times (-0.05) = 12.44\%$

>> With Convexity: $\ffrac{\Delta P} {P} = -D^* \times \Delta y + \frac{1} {2} \times Convex \times \left( \Delta y \right)^2 = -2.487 \times (-0.05) + \ffrac{1} {2} \times 8.756 \times (-0.05)^2 = 13.53\% $.

For bond with greater complexity, graphically it means it curves heavier. So that no matter how interest rate changes, it always performs better.

## Active Bond Management
### Sources of Potential Profit
1. interest rate forecasting
2. identification of relative mispricing within the fixed-income market

**substitution swap**: Exchange of one bond for a bond with similar attributes but more attractively priced.

**intermarket spread swap**: Switching from one segment of the bond market to another.

**rate anticipation swap**: A switch made in response to forecasts of interest rate changes.

**pure yield pickup swap**: Moving to higher yield bonds, usually with longer maturities.

**tax swap**: Swapping two similar bonds to receive a tax benefit.

## SUMMARY
- Even default-free bonds such as Treasury issues are subject to interest rate risk. Longer-term bonds generally are more sensitive to interest rate shifts than short-term bonds. A measure of the average life of a bond is Macaulay’s duration, defined as the weighted average of the times until each payment made by the security, with weights proportional to the present value of the payment.
- Modified duration is a direct measure of the sensitivity of a bond’s price to a change in its yield. The proportional change in a bond’s price approximately equals the negative of modified duration times the change in the bond’s yield.
- Immunization strategies are characteristic of passive bond portfolio management. Such strategies attempt to render the individual or firm immune from movements in interest rates. This may take the form of immunizing net worth or, instead, immunizing the future accumulated value of a bond portfolio.
- Immunization of a fully funded plan is accomplished by matching the durations of assets and liabilities. To maintain an immunized position as time passes and interest rates change, the portfolio must be periodically rebalanced.
- Convexity refers to the curvature of a bond’s price-yield relationship. Accounting for convexity can substantially improve on the accuracy of the duration approximation for the response of bond prices to changes in yields.
- A more direct form of immunization is dedication or cash flow matching. If a portfolio is perfectly matched in cash flow with projected liabilities, rebalancing will be unnecessary.
- Active bond management can be decomposed into interest rate forecasting techniques and intermarket spread analysis. One popular taxonomy classifies active strategies as substitution swaps, intermarket spread swaps, rate anticipation swaps, and pure yield pickup swaps.

## KEY TERMS
- cash flow matching
- convexity
- dedication strategy
- horizon analysis
- immunization
- intermarket spread swap
- Macaulay’s duration
- modified duration
- pure yield pickup swap
- rate anticipation swap
- rebalancing
- substitution swap
- tax swap

## Key Equations
- Macaulay's Duration
$$D = \sum_{t=1}^{T} t \times w_t$$

- Modified duration and duration rule
$$\frac{\Delta P} {P} = -D \times\left[ \frac{\Delta(1+y)} {(1+y)}\right] = -D^* \times \Delta y$$

- Duration of perpetuity
$$D_{perpetuity} = \frac{1+y} {y}$$

- Convexity
$$\frac{\Delta P} {P} = -D^* \times \Delta y + \frac{1} {2} \times Convex \times \left( \Delta y \right)^2$$

## Assignment
1. All other things equal (YTM: $10\%$), which of the following has the longest duration?
    - A 30-year bond with a 10% coupon
    - ~~A 20-year bond with a 9% coupon~~
    - ~~A 20-year bond with a 7% coupon~~
    - ~~A 10-year zero-coupon bond~~
2. **VIP** You find a $5\text{-year}$ AA Xerox bond priced to yield $6\%$. You find a similar-risk $5\text{-year}$ Canon bond priced to yield $6.5\%$. If you expect interest rates to rise, which of the following should you do?
    - ~~Short the Canon bond, and buy the Xerox bond.~~
    - Buy the Canon bond, and short the Xerox bond.
    - ~~Short both the Canon bond and the Xerox bond.~~
    - ~~Buy both the Canon bond and the Xerox bond~~
3. Bond prices are ___ sensitive to changes in yield when the bond is selling at a ___ initial yield to maturity.
    - ~~more; higher~~
    - more; lower
    - ~~less; lower~~
    - ~~equally; higher or lower~~
4. Duration is a concept that is useful in assessing a bond's ___ .
    - ~~liquidity risk~~
    - price volatility
    - ~~convexity risk~~
    - ~~credit risk~~
5. All other things equal, a bond's duration is ___ .
    - ~~higher when the yield to maturity is higher~~
    - lower when the yield to maturity is higher
    - ~~the same at all yield rates~~
    - ~~indeterminable when the yield to maturity is high~~
6. A bond currently has a price of $\$1,050$. The yield on the bond is $6\%$. If the yield increases $25$ basis points, the price of the bond will go down to $\$1,030$. The duration of this bond is ___ years.
$$\text{ }\\
\frac{\Delta P} {P} = -D^* \cdot \Delta y \\
-20/1050 = -D^* \cdot 0.25\% \\
D^* = 7.6 = \frac{D} {1+y} \\
D = 7.6 \times 1.0625 = 8.075 \\[0.8em]
$$

7. A $20\text{-year}$ maturity bond pays interest of $\$90$ once per year and has a face value of $\$1,000$. Its yield to maturity is $10\%$. You expect that interest rates will decline over the upcoming year and that the yield to maturity on this bond will be only $8\%$ a year from now. Using horizon analysis, the return you expect to earn by holding this bond over the upcoming year is ___ .
$$\text{ }\\
\text{Price}_{\;\text{Today}} = \text{Price}(\text{N}=20, \text{YTM}=10\%, \text{Coupon}=90, \text{Par}=1000) = 914.86 \\
\text{Price}_{\;\text{Next Year}} = \text{Price}(\text{N}=19, \text{YTM}=8\%, \text{Coupon}=90, \text{Par}=1000) = 1096.04 \\
\text{Return} = \frac{90 + 1096.04 - 914.86} {914.86} = 0.2964\\[0.9em]
$$

8. An $8\%$, $30\text{-year}$ bond has a yield to maturity of $10\%$ and a modified duration of $8$ years. If the market yield drops by $15$ basis points, there will be a ___ in the bond's price.
$$\text{ }\\
\frac{\Delta P} {P} = -D^* \cdot \Delta y = -8 \times (-0.15\%) = 1.2\%\\[0.9em] $$

9. You have a $25\text{-year}$ maturity, $10\%$ coupon, $10\%$ yield bond with a duration of $10$ years and a convexity of $135.5$. If the interest rate were to fall $125$ basis points, your predicted new price for the bond (including convexity) is ___ .
$$\text{ }\\
\begin{align}
\frac{\Delta P} {P} &= -\frac{D} {1+y} \cdot \Delta y + \frac{1} {2} \times Convex \cdot (\Delta y)^2 \\
&= -\frac{10} {1.1} \times (-1.25\%) + \frac{1} {2} \times 135.5 \times (-1.25\%)^2 = 12.42\%\\
\text{Price} &= 1000 \times (1+12.42\%) = 1124.22
\end{align}\\[1em]
$$

10. You own a bond that has a duration of $7$ years. Interest rates are currently $8\%$, but you believe the Fed is about to increase interest rates by $21$ basis points. Your predicted price change on this bond is
$$\text{ }\\
D^* = \frac{7} {1.08} = 6.48 \\
\frac{\Delta P} {P} = -D^* \cdot \Delta y = -6.48 \times 0.21\% = -1.366\%\\[1em]
$$

11. A bond pays annual interest. Its coupon rate is $11.0\%$. Its value at maturity is $1,000$. It matures in $4$ years. Its yield to maturity is currently $8.0\%$. The modified duration of this bond is ___ years.
$$ ... $$

12. You have a $25\text{-year}$ maturity, $10.2\%$ coupon, $10.2\%$ yield bond with a duration of $10$ years and a convexity of $135.7$. If the interest rate were to fall $127$ basis points, your predicted new price for the bond (including convexity) is
$$ ... $$

## In class question
$15$ year maturity, $4\%$ coupon rate, $6\%$ yield bond with duration of $10.5$ years and a convexity of $128.75$. The bond is currently priced at $805.76$. If the interest rate were to increase $200$ basis points your predicted new price for the bond is

$$\begin{align}
\frac{\Delta P} {P} &= -\frac{D} {1+y} \cdot \Delta y + \frac{1} {2} \times Convex \cdot (\Delta y)^2 \\
&= -\frac{10.5} {1.06} \cdot 0.02 + \frac{1} {2} \times 128.75 \cdot (0.02)^2 \\
&= -17.236\%
\end{align} $$

$$P_{New} = \left( 1 - \frac{\Delta P} {P} \right) \times P_{Old} = 666.8766$$


```python
%matplotlib nbagg
from __future__ import division
from IPython.display import HTML
from IPython.display import display
from scipy.optimize import *
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
```

## Question 1: Linear Regression, Curvature Matrix

Consider the data listed below,
\begin{equation}
\begin{array}{lcccccc}
\hline
{\rm frequency~(Hz)} &10&20&30&40&50&60\\
{\rm voltage~(mV)} &16&45&64&75&70&115\\
{\rm error~(mV)}   &5&5&5&5&30&5\\
\hline
{\rm frequency~(Hz)} &70&80&90&100&110&\\
{\rm voltage~(mV)} &142&167&183&160&221&\\
{\rm error~(mV)}   &5&5&5&30&5&\\
\hline
\end{array}
\end{equation}

This data is also contained in the file 'linear_regression.csv'.

Required:
<bf>
> (i) Calculate the 4 elements of the curvature matrix.
<bf>
> (ii) Invert this to give the error matrix.
<bf>
> (iii) What are the uncertainties in the slope and intercept?
<bf>
> (iv) Comment on your answer.

### (i) Calculate the 4 elements of the curvature matrix.

```python
data = pd.read_csv('linear_regression.csv')
frequencies = data.iloc[:,0]
voltages = data.iloc[:,1]
voltage_errors = data.iloc[:,2]

def one_i():
    '''Your function should return something of the form np.matrix([[a_cc,a_cm],[a_mc,a_mm]])'''
    '''where m is the slope and c the intercept'''
    # YOUR CODE HERE
    a_cc = 0
    a_cm = 0
    a_mm = 0
    for i in range(len(frequencies)):
        a_cc += 1 / (voltage_errors[i] ** 2)
        a_cm += frequencies[i] / (voltage_errors[i] ** 2)
        a_mm += frequencies[i] ** 2 / (voltage_errors[i] ** 2)
    a_mc = a_cm
    curvature_matrix = []
    curvature_matrix.append([a_cc, a_cm])
    curvature_matrix.append([a_mc, a_mm])
    return(curvature_matrix)

print(one_i())
```

    [[0.3622222222222222, 20.56666666666667], [20.56666666666667, 1537.888888888889]]

```python
'''TEST CELL - DO NOT DELETE'''
assert isinstance(one_i(), (list, tuple, np.ndarray)), \
    'Please make sure that the return value is list/array of floats.'
assert len(one_i()) == 2 , 'Please make sure you return a matrix with dimensions 2x2'
```

### (ii) Invert this to give the error matrix.

```python
data = pd.read_csv('linear_regression.csv')
frequencies = data.iloc[:,0]
voltages = data.iloc[:,1]
voltage_errors = data.iloc[:,2]

def one_ii():
    '''Your function should return something of the form np.matrix([[a_cc,a_cm],[a_mc,a_mm]])'''
    # YOUR CODE HERE
    inverted_matrix = np.linalg.inv(one_i())
    return(inverted_matrix)

print(one_ii())
```

    [[ 1.14708117e+01 -1.53402734e-01]
     [-1.53402734e-01  2.70174453e-03]]

```python
'''TEST CELL - DO NOT DELETE'''
assert isinstance(one_ii(), (list, tuple, np.ndarray)), \
    'Please make sure that the return value is list/array of floats.'
assert len(one_ii()) == 2 , 'Please make sure you return a matrix with dimensions 2x2'
```

### (iii) What are the uncertainties in the slope and intercept?

```python
data = pd.read_csv('linear_regression.csv')
frequencies = data.iloc[:,0]
voltages = data.iloc[:,1]
voltage_errors = data.iloc[:,2]

def one_iii():
    slope_uncertainty = 0
    intercept_uncertainty = 0
    # YOUR CODE HERE
    error_matrix = one_ii()
    slope_uncertainty = np.sqrt(error_matrix[0, 0])
    intercept_uncertainty = np.sqrt(error_matrix[1, 1])
    return(slope_uncertainty,intercept_uncertainty)

print(one_iii())
```

    (3.3868586735217345, 0.051978308277218006)

```python
'''TEST CELL - DO NOT DELETE'''
assert isinstance(one_iii(), (list, tuple, np.ndarray)), \
    'Please make sure that the return value is list/array of floats.'
assert len(one_iii()) == 2 , 'Please make sure you return a list/array of length 2'
```

```python
'''TEST CELL - DO NOT DELETE'''
```

    'TEST CELL - DO NOT DELETE'

### (iv) Comment on your answer

It is truly convenient to calculate the uncertainties in te slope and intercept than using chi2 + 1.

## Question 2: Using a calibration curve

A frequently encountered case where the correlation of the uncertainties must be taken into account is that of a calibration curve.  Consider the following set of measurements from an optical-activity experiment, where the angle of rotation of a plane-polarized light beam, $\theta$, is measured as a function of the independent variable, the concentration, $C$, of a sucrose solution.

\begin{equation}
\begin{array}{lcccc}
\hline
C \mbox{ (g cm$^{-3}$)} &0.025&0.05&0.075&0.100\\
\theta \mbox{ (degrees)}&10.7&21.6&32.4&43.1\\
\hline
C \mbox{ (g cm$^{-3}$)}&0.125&0.150&0.175\\
\theta \mbox{ (degrees)}&53.9&64.9&75.4\\
\hline
\end{array}
\end{equation}

The errors in the angle measurement are all $0.1^{\circ}$, the errors in the concentration are negligible.  A straight line  fit to the data yields  a gradient of $431.7\,^{\circ}\mbox{ g$^{-1}$ cm$^{3}$}$, and intercept $-0.03^{\circ}$. This data is contained in 'optical_activity.csv'.

<bf>
Required:
<bf>
>(i) Show that the curvature matrix, $\mathsf{A}$, is given by

\begin{equation}
\mathsf{A}=\left[\begin{array}{cc}
700\left((^{\circ})^{-2}\right)&70\left((^{\circ})^{-2}\mbox{g cm$^{-3}$}\right)\\
70\left((^{\circ})^{-2}\mbox{g cm$^{-3}$}\right)&8.75\left((\mbox{g/$^\circ$ cm$^{3})^2$}\right)\\
\end{array}\right] ,
\end{equation}

>and that the error matrix  is

\begin{equation}
\mathsf{C}=\left[\begin{array}{cc}
0.00714\left((^{\circ})^2\right)&-0.0571\left((^{\circ})^2\mbox{g$^{-1}$cm$^{3}$}\right)\\
-0.0571\left((^{\circ})^2\mbox{g$^{-1}$cm$^{3}$}\right)&0.571\left((^{\circ})^2\mbox{g$^{-2}$ cm$^{6}$}\right)\\
\end{array}\right] .
\end{equation}

The entry for the intercept is in the top left-hand corner, that for the gradient in the bottom right-hand corner.
<bf>
>(ii) Calculate the associated correlation matrix.

Use the  entries of the error matrix to answer the following  questions:
<bf>
>(iii) What are the uncertainties in the best-fit intercept and gradient?
<bf>
>(iv) What optical rotation is expected for a known concentration of $C=0.080g cm^{-3}$, and what is the uncertainty?
<bf>
>(v) What is the concentration given a measured rotation of $\theta=70.3^{\circ}$ and what is the uncertainty?

### (i) Verify the curvature matrix and the error matrix above.

```python
data = pd.read_csv('optical_activity.csv')
concentrations = data.iloc[:,0]
angles = data.iloc[:,1]
angle_errors = data.iloc[:,2]

def two_i():
    '''Your function should return something of the form np.matrix([[a_cc,a_cm],[a_mc,a_mm]]). Must return the curvature and error matricies)'''
    curvature_matrix = 0
    error_matrix = 0
    # YOUR CODE HERE
    a_cc = 0
    a_cm = 0
    a_mm = 0
    for i in range(len(concentrations)):
        a_cc += 1 / (angle_errors[i] ** 2)
        a_cm += concentrations[i] / (angle_errors[i] ** 2)
        a_mm += concentrations[i] ** 2 / (angle_errors[i] ** 2)
    a_mc = a_cm
    curvature_matrix = []
    curvature_matrix.append([a_cc, a_cm])
    curvature_matrix.append([a_mc, a_mm])
    error_matrix = np.linalg.inv(curvature_matrix)
    return(curvature_matrix,error_matrix)

print(two_i())
```

    ([[699.9999999999999, 69.99999999999999], [69.99999999999999, 8.75]], array([[ 0.00714286, -0.05714286],
           [-0.05714286,  0.57142857]]))

```python
'''TEST CELL - DO NOT DELETE'''
assert isinstance(two_i(), (list, tuple, np.ndarray)), \
    'Please make sure that the return value is list/array of floats.'
assert len(two_i()) == 2 , 'Please make sure you return a list/array of two matrices'
assert len(two_i()[0]) == 2, 'Please make sure that the first entry is a 2x2 matrix'
assert len(two_i()[1]) == 2, 'Please make sure that the second entry is a 2x2 matrix'
```

```python
'''TEST CELL - DO NOT DELETE'''
```

    'TEST CELL - DO NOT DELETE'

### (ii) Calculate the associated correlation matrix.

```python
data = pd.read_csv('optical_activity.csv')

concentrations = data.iloc[:,0]
angles = data.iloc[:,1]
angle_errors = data.iloc[:,2]

def two_ii():
    '''Your function should return something of the form np.matrix([[a_cc,a_cm],[a_mc,a_mm]])'''
    # YOUR CODE HERE
    associated_correlation_matrix = []
    _, error_matrix = two_i()
    correlation_paramater = error_matrix[0][1] / np.sqrt(error_matrix[0][0] * error_matrix[1][1])
    associated_correlation_matrix.append([1, correlation_paramater])
    associated_correlation_matrix.append([correlation_paramater, 1])
    return associated_correlation_matrix

print(two_ii())
```

    [[1, -0.8944271909999156], [-0.8944271909999156, 1]]

```python
'''TEST CELL - DO NOT DELETE'''
assert isinstance(two_ii(), (list, tuple, np.ndarray)), \
    'Please make sure that the return value is a matrix.'
assert len(two_ii()) == 2
```

### (iii) What are the uncertainties in the best-fit intercept and gradient?

```python
data = pd.read_csv('optical_activity.csv')
concentrations = data.iloc[:,0]
angles = data.iloc[:,1]
angle_errors = data.iloc[:,2]

def f(x,a,b):
    return(a*x+b)

def two_iii():
    '''Your function should return the uncertainty in the gradient and intercept'''
    gradient_uncertainty = 0
    intercept_uncertainty = 0
    # YOUR CODE HERE
    _,error_matrix = two_i()
    gradient_uncertainty = np.sqrt(error_matrix[1, 1])
    intercept_uncertainty = np.sqrt(error_matrix[0, 0])
    return(gradient_uncertainty,intercept_uncertainty)

print(two_iii())
```

    (0.7559289460184541, 0.08451542547285162)

```python
'''TEST CELL - DO NOT DELETE'''
assert isinstance(two_iii(), (list, tuple, np.ndarray)), \
    'Please make sure that the return value is a list/array of floats.'
assert len(two_iii()) == 2
```

```python
'''TEST CELL - DO NOT DELETE'''
```

    'TEST CELL - DO NOT DELETE'

### (iv) What optical rotation is expected for a known concentration of $C=0.080g cm^{-3}$, and what is the uncertainty?

```python
data = pd.read_csv('optical_activity.csv')
concentrations = data.iloc[:,0]
angles = data.iloc[:,1]
angle_errors = data.iloc[:,2]

def two_iv():
    '''Your function should return the angle and the uncertainty'''
    angle = 0
    uncertainty = 0
    C = 0.080
    # YOUR CODE HERE
    _,error_matrix = two_i()
    gradient = 431.7
    intercept = -0.03
    angle =  C * gradient + intercept
    uncertainty = np.sqrt(C ** 2 * error_matrix[1, 1] + error_matrix[0, 0] + 2 * C * error_matrix[0, 1])
    return(angle,uncertainty)

print(two_iv())
```

    (34.506, 0.04070801956792859)

```python
'''TEST CELL - DO NOT DELETE'''
assert isinstance(two_iv(), (list, tuple, np.ndarray)), \
    'Please make sure that the return value is a list/array of floats.'
assert len(two_iv()) == 2
```

```python
'''TEST CELL - DO NOT DELETE'''
```

    'TEST CELL - DO NOT DELETE'

### (v) What is the concentration given a measured rotation of $\theta=70.3^{\circ}$ and what is the uncertainty? You must return your answer in $gcm^{-3}$

```python
data = pd.read_csv('optical_activity.csv')
concentrations = data.iloc[:,0]
angles = data.iloc[:,1]
angle_errors = data.iloc[:,2]

def two_v():
    '''Your function should return the concentration and uncertainty'''
    # YOUR CODE HERE
    angle = 70.3
    ret = curve_fit(f,concentrations,angles)
    gradient = ret[0][0]
    intercept = ret[0][1]
    _,error_matrix = two_i()
    concentration = (angle - intercept) / gradient
    gradient_error, intercept_error = two_iii()
    uncertainty = concentration * np.sqrt((intercept_error / (angle - intercept)) ** 2 + (gradient_error / gradient) ** 2 - 2 * error_matrix[0, 1] / ((angle - intercept) * gradient))
    return(concentration,uncertainty)

print(two_v())
```

    (0.16290536073443715, 0.0004685968620121708)

```python
'''TEST CELL - DO NOT DELETE'''
assert isinstance(two_v(), (list, tuple, np.ndarray)), \
    'Please make sure that the return value is a list/array of floats.'
assert len(two_v()) == 2
```

```python
'''TEST CELL - DO NOT DELETE'''
```

    'TEST CELL - DO NOT DELETE'

## Question 3: Error bars from a $\chi^2$ minimisation to a non-linear function

In this question we will analyse the data shown in the figure below, which is an X-ray spectrum as a function of angle.

The data is contained in the file 'LorentzianData.csv'. There are three columns: the angle, the signal (in counts per second), and the error.  The number of X-rays counted in 20 seconds was recorded.

The model to describe the data has four parameters:  the height of the Lorentzian lineshape, $S_0$; the angle at which the peak is centered, $\theta_{0}$;
 the angular width of the peak, $\Delta\theta$; and a constant background offset, $S_{\rm bgd}$. Mathematically, the signal, $S$, is of the form:
\begin{equation}
S=S_{\rm bgd}+\frac{S_{0}}{1+4\left(\frac{\theta-\theta_{0}}{\Delta\theta}\right)^2}.
\end{equation}

and the function is defined by lorentzian(theta, s_0, s_bgd,delta_theta,theta_0).

Required:
<bf>
>(i) Explain how the error in the count rate was calculated.
<bf>
>(ii) Perform a $\chi^2$ minimisation.  What are the best-fit parameters?
<bf>
>(iii) Evaluate the error matrix.
<bf>
>(iv) Calculate the correlation matrix.
<bf>
>(v) What are the uncertainties in the best-fit parameters?
<bf>
>(vi) If you can plot contour plots, show the $\chi^2$ contours for
<bf>
>>(a) background vs. peak centre.
<bf>
>>(b) background vs. peak width.
<bf>
>These figures are shown in figure 6.11 of Hughes and Hase. Comment on the shape of the contours. Only your comment will be graded.
<bf>

### (i) Explain how the error in the count rate was calculated.

It can be divided into two parts. The first part is A = S_bgd, B = S_0 / (1 + 4 * ((phi - phi0) / delta_phi)).
The second part is C = S_0, D = (1 + 4 * ((phi - phi0) / delta_phi)).
It can apply to two formula to calculate its answer S = A + B and B = C / D, and we can finally figure out the error.

### (ii) Perform a $\chi^2$ minimisation.  What are the best-fit parameters?

```python
data = pd.read_csv('LorentzianData.csv')

def lorentzian(theta, s_0, s_bgd,delta_theta,theta_0):
    return s_bgd+(s_0/(1+4*(((theta-theta_0)/delta_theta)**2)))

def func_solve(index, args):
    kai2 = 0
    for i in range(len(args[0])):
        kai2 += (args[1][i] - lorentzian(args[0][i], index[0], index[1], index[2], index[3])) ** 2 / (args[2][i] ** 2)
    return kai2

def three_ii():
    s_0 = 8
    s_bgd = 1.4
    delta_theta = 0.9
    theta_0 = 44.4
    covariance_matrix = 0
    angles = data.iloc[:,0]
    intensity = data.iloc[:,1]
    intensity_errors = data.iloc[:,2]
    index = [s_0, s_bgd, delta_theta, theta_0]
    args = [angles, intensity, intensity_errors]
    res = minimize(func_solve, index, args)
    s_0 = res.x[0]
    s_bgd = res.x[1]
    delta_theta = res.x[2]
    theta_0 = res.x[3]
    hess_inv = res.hess_inv
    curvature_matrix = np.linalg.inv(hess_inv) * 1 / 2
    covariance_matrix = np.linalg.inv(curvature_matrix)
    # YOUR CODE HERE
    return(s_0,s_bgd,delta_theta,theta_0,covariance_matrix)

print(three_ii())
```

    (5.426322905614177, 1.4044035346590564, 0.9498830699087861, 44.390120244013396, array([[ 5.46894368e-03, -1.66320605e-03,  7.03982956e-04,
             1.64449886e-04],
           [-1.66320605e-03,  5.92308717e-04, -2.43674125e-04,
            -1.68099865e-05],
           [ 7.03982956e-04, -2.43674125e-04,  6.29659088e-04,
            -4.01150268e-05],
           [ 1.64449886e-04, -1.68099865e-05, -4.01150268e-05,
             2.37812579e-04]]))

```python
'''TEST CELL - DO NOT DELETE'''
assert isinstance(three_ii(), (list, tuple, np.ndarray)), \
    'Please make sure that the return value is a list/array.'
assert len(three_ii()) == 5 , 'Please make sure that you return five values'
```

### (iii) Evaluate the error matrix.

```python
data = pd.read_csv('LorentzianData.csv')

def three_iii():
    '''Your function should return something of the form np.matrix([4x4])'''
    # YOUR CODE HERE
    _, _, _, _, error_matrix = three_ii()
    return(error_matrix)

print(three_iii())
```

    [[ 5.46894368e-03 -1.66320605e-03  7.03982956e-04  1.64449886e-04]
     [-1.66320605e-03  5.92308717e-04 -2.43674125e-04 -1.68099865e-05]
     [ 7.03982956e-04 -2.43674125e-04  6.29659088e-04 -4.01150268e-05]
     [ 1.64449886e-04 -1.68099865e-05 -4.01150268e-05  2.37812579e-04]]

```python
'''TEST CELL - DO NOT DELETE'''
assert isinstance(three_iii(), (list, tuple, np.ndarray)), \
    'Please make sure that the return value is a matrix.'
assert len(three_iii()) == 4 , 'Please make sure that you return a 4x4 matrix'
```

### (iv) Calculate the correlation matrix.

```python
data = pd.read_csv('LorentzianData.csv')

def three_iv():
    '''Your function should return something of the form np.matrix([[a_cc,a_cm],[a_mc,a_mm]])'''
    # YOUR CODE HERE
    error_matrix = three_iii()
    cor00 = 1
    cor01 = error_matrix[0][1] / np.sqrt(error_matrix[0][0] * error_matrix[1][1])
    cor02 = error_matrix[0][2] / np.sqrt(error_matrix[0][0] * error_matrix[2][2])
    cor03 = error_matrix[0][3] / np.sqrt(error_matrix[0][0] * error_matrix[3][3])
    cor10 = cor01
    cor11 = 1
    cor12 = error_matrix[1][2] / np.sqrt(error_matrix[1][1] * error_matrix[2][2])
    cor13 = error_matrix[1][3] / np.sqrt(error_matrix[1][1] * error_matrix[3][3])
    cor20 = cor02
    cor21 = cor12
    cor22 = 1
    cor23 = error_matrix[2][3] / np.sqrt(error_matrix[2][2] * error_matrix[3][3])
    cor30 = cor03
    cor31 = cor13
    cor32 = cor23
    cor33 = 1
    correlation_matrix = []
    correlation_matrix.append([cor00, cor01, cor02, cor03])
    correlation_matrix.append([cor10, cor11, cor12, cor13])
    correlation_matrix.append([cor20, cor21, cor22, cor23])
    correlation_matrix.append([cor30, cor31, cor32, cor33])
    return(correlation_matrix)

print(three_iv())
```

    [[1, -0.9241027617197095, 0.37936536733168097, 0.1441997277595128], [-0.9241027617197095, 1, -0.399008928950146, -0.04478946659620176], [0.37936536733168097, -0.399008928950146, 1, -0.10366614239816119], [0.1441997277595128, -0.04478946659620176, -0.10366614239816119, 1]]

```python
'''TEST CELL - DO NOT DELETE'''
assert isinstance(three_iv(), (list, tuple, np.ndarray)), \
    'Please make sure that the return value is a matrix.'
assert len(three_iv()) == 4 , 'Please make sure that you return a 4x4 matrix'
```

### (v) What are the uncertainties in the best-fit parameters?

```python
data = pd.read_csv('LorentzianData.csv')

def three_v():
    uncertainty_s_0 = 0
    uncertainty_s_bgd = 0
    uncertainty_delta_theta = 0
    uncertainty_theta_0 = 0
    error_matrix = three_iii()
    uncertainty_s_0 = np.sqrt(error_matrix[0][0])
    uncertainty_s_bgd = np.sqrt(error_matrix[1][1])
    uncertainty_delta_theta = np.sqrt(error_matrix[2][2])
    uncertainty_theta_0 = np.sqrt(error_matrix[3][3])
    return(uncertainty_s_0,uncertainty_s_bgd,uncertainty_delta_theta,uncertainty_theta_0)

print(three_v())
```

    (0.07395230678243828, 0.024337393391224975, 0.02509300875199448, 0.015421173059876989)

```python
'''TEST CELL - DO NOT DELETE'''
```

    'TEST CELL - DO NOT DELETE'

### (vi) These contours are shown in figure 6.11 of Hughes and Hase. Comment on the shape of the contours.

From the contours, we can know the peak, width, height of the data. Also, we will find that there may be some local minimal, which we might be trapped in when using minimization function, and we need to choose more initial values to get the true global minimal.

## Question 4: Prove the following properties:

Assume in this question that the uncertainties in $A$ and $B$ are correlated.
>(i) If $Z=A\pm B$, show that
${\displaystyle\alpha_{Z}^2=\alpha_{A}^2+\alpha_{B}^2\pm2\alpha_{AB}}$.
<bf>
>(ii) If $Z=A\times B$, show that
 ${\displaystyle\left(\frac{\alpha_Z}{Z}\right)^2=\left(\frac{\alpha_A}{A}\right)^2+\left(\frac{\alpha_B}{B}\right)^2+2\left(\frac{\alpha_{AB}}{AB}\right)}$.
<bf>
>(iii) If ${\displaystyle Z=\frac{A}{B}}$, show that
${\displaystyle\left(\frac{\alpha_Z}{Z}\right)^2=\left(\frac{\alpha_A}{A}\right)^2+\left(\frac{\alpha_B}{B}\right)^2-2\left(\frac{\alpha_{AB}}{AB}\right)}$.

Since $a_{z}^{2}=\left(\frac{\partial_{z}}{\partial A}\right)^{2} \alpha_{A}^{2}+\left(\frac{\partial_{z}}{\partial B}\right)^{2} \alpha_{B}^{2}+2 \times\left(\frac{\partial_{z}}{\partial A}\right)\left(\frac{\partial_{z}}{\partial B}\right) \times A B$, we know that $\mathrm{Z}=\mathrm{A}+\mathrm{B}$, we can easily get ${\displaystyle\alpha_{Z}^2=\alpha_{A}^2+\alpha_{B}^2\pm2\alpha_{AB}}$.
It is same for $\mathrm{Z}=\mathrm{A}-\mathrm{B}$

when$\mathrm{Z}=\mathrm{A} \times B$, we can get $\alpha_{Z}^{2}=B^{2} \alpha_{A}^{2}+A^{2} \alpha_{B}^{2}+2 A B \alpha_{A B}$, both sides are divided by AB. $\left(\frac{a_{Z}}{A B}\right)^{2}=\frac{\left(B^{2} \times \alpha_{A}^{2}+A^{2} \times \alpha_{B}^{2}+2 A B \times \alpha_{A B}\right)}{A^{2} B^{2}}$. Since Z = AB, we can get  ${\displaystyle\left(\frac{\alpha_Z}{Z}\right)^2=\left(\frac{\alpha_A}{A}\right)^2+\left(\frac{\alpha_B}{B}\right)^2+2\left(\frac{\alpha_{AB}}{AB}\right)}$.

Same for ${\displaystyle Z=\frac{A}{B}}$

```python

```

Core 1/data analysis/ipython notebook/Homework 3/Assignment 3.ipynb


# メモ

チャート式基礎からの数学 Ⅰ の二次関数 quadratic function
を読む。

# 関数とグラフ

**関数**

2つの変数 $x,y$ があって、$x$ の値が定まるとそれに対応して $y$ の値がただ 1つ定まるとき、$y$ は $x$ の関数である、という。$y=f(x)$ と表す。

**定義域・閾値**

関数 $y=f(x)$ において、変数 $x$ のとりうる値の範囲、$x$ の変域を、この関数の定義域という。 また、$x$ が定義域全体を動くとき、$f(x)$ のとりうる値の範囲、すなわち $y$ の変域を、この関数の値域という。

**$y = ax +b$ のグラフ**

$a \neq 0$ のとき 1次関数 $y=ax+b$ のグラフ。
傾きが $a$、$y$ 軸上の切片 ($y$ 切片) が $b$ の直線。
$a > 0$ なら右上がり $\quad a < 0$ なら右下がり

$a = 0$ のとき $y = b$ のグラフ。
傾きが 0、$y$ 切片が $b$ の $y$ 軸に垂直な直線。

**象限**

座標平面の右上を第1象限といい、反時計回りに第2象限、第3象限、第4象限という。

問題

点 $(2x-3, -3x+5)$ が第2象限にあるように $x$ の値の範囲を定める。

また、$x$ がどのような値であってもこの点が存在しない象限はどこか。

解答

{ 象限の定義より }
1.
2*x - 3 < 0
2 * x < 3
x < 3 / 2

2.
-3 * x + 5 > 0
-3 * x > -5
3 * x < 5
x < 5 / 3

```
from sympy import *
init_printing()
x = symbols('x')
solve(Eq(2*x-3, 0))

```


# Demo - Rayleigh Benard

**Mikael Mortensen** (email: `mikaem@math.uio.no`), Department of Mathematics, University of Oslo.

Date: **November 21, 2019**

**Summary.** Rayleigh-Benard convection arise
due to temperature gradients in a fluid. The governing equations are
Navier-Stokes coupled (through buoyancy) with an additional temperature
equation derived from the first law of thermodynamics, using a linear
correlation between density and temperature.

This is a demonstration of how the Python module [shenfun](https://github.com/spectralDNS/shenfun) can be used to solve for
these Rayleigh-Benard cells in a 2D channel with two walls of
different temperature in one direction, and periodicity in the other direction.
The solver described runs with MPI
without any further considerations required from the user.
Note that there is a more physically realistic 3D solver implemented within
[the spectralDNS project](https://github.com/spectralDNS/spectralDNS/blob/master/spectralDNS/solvers/KMMRK3_RB.py).
To allow for some simple optimizations, the solver described in this demo has been implemented in a class in the
[RayleighBenardRk3.py](https://github.com/spectralDNS/shenfun/blob/master/demo/RayleighBenardRK3.py)
module in the demo folder of shenfun. Below are two example solutions, where the first (movie)
has been run at a very high Rayleigh number (*Ra*), and the lower image with a low *Ra* (laminar).

Temperature fluctuations in the Rayleigh Benard flow. The top and bottom walls are kept at different temperatures and this sets up the Rayleigh-Benard convection. The simulation is run at Ra =100,000, Pr =0.7 with 100 and 256 quadrature points in x and y-directions, respectively.

Convection cells for a laminar flow. The simulation is run at Ra =100, Pr =0.7 with 40 and 128 quadrature points in x and y-directions, respectively.

## The Rayleigh Bénard equations

The governing equations solved in domain $\Omega=[-1, 1]\times [0, 2\pi]$ are

$$
\begin{equation}
    \frac{\partial \boldsymbol{u}}{\partial t} + (\boldsymbol{u} \cdot \nabla) \boldsymbol{u} = - \nabla p + \sqrt{\frac{Pr}{Ra}} \nabla^2 \boldsymbol{u}  + T \boldsymbol{i}, \label{eq:momentum} \tag{1}
\end{equation}
$$

$$
\begin{equation}
    \frac{\partial T}{\partial t} +\boldsymbol{u} \cdot \nabla T = \frac{1}{\sqrt{RaPr}} \nabla^2 T, \label{eq:T} \tag{2}
\end{equation}
$$

$$
\begin{equation}
    \nabla \cdot \boldsymbol{u} = 0, \label{eq:div} \tag{3}
\end{equation}
$$

where $\boldsymbol{u}(x, y, t) (= u\boldsymbol{i} + v\boldsymbol{j})$ is the velocity vector, $p(x, y, t)$ is pressure, $T(x, y, t)$ is the temperature, and $\boldsymbol{i}$ and
$\boldsymbol{j}$ are the unity vectors for the $x$ and $y$-directions, respectively.

The equations are complemented with boundary conditions $\boldsymbol{u}(\pm 1, y, t) = (0, 0), \boldsymbol{u}(x, 2 \pi, t) = \boldsymbol{u}(x, 0, t), T(1, y, t) = 1, T(-1, y, t) =  0, T(x, 2 \pi, t) = T(x, 0, t)$.
Note that these equations have been non-dimensionalized according to [[pandey18]](#pandey18), using dimensionless
Rayleigh number $Ra=g \alpha \Delta T h^3/(\nu \kappa)$ and Prandtl number $Pr=\nu/\kappa$. Here
$g \boldsymbol{i}$ is the vector accelleration of gravity, $\Delta T$ is the temperature difference between
the top and bottom walls, $h$ is the hight of the channel in $x$-direction, $\nu$ is the
dynamic viscosity coefficient, $\kappa$ is the heat transfer coefficient and $\alpha$ is the
thermal expansion coefficient. Note that the
governing equations have been non-dimensionalized using the free-fall velocityscale
$U=\sqrt{g \alpha \Delta T h}$. See [[pandey18]](#pandey18) for more details.

The governing equations contain a non-trivial coupling between velocity, pressure and temperature.
This coupling can be simplified by eliminating the pressure from the equation for the wall-normal velocity
component $u$. We accomplish this by taking the Laplace of the momentum equation in wall normal
direction, using the pressure from the divergence of the momentum equation
$\nabla^2 p = -\nabla \cdot \boldsymbol{H}+\partial T/\partial x$, where
$\boldsymbol{H} = (H_x, H_y) = (\boldsymbol{u} \cdot \nabla) \boldsymbol{u}$

$$
\begin{equation}
    \frac{\partial \nabla^2 {u}}{\partial t} = \frac{\partial^2 H_y}{\partial x \partial y} - \frac{\partial^2 H_x}{\partial y\partial y}  + \sqrt{\frac{Pr}{Ra}} \nabla^4 {u}  + \frac{\partial^2 T}{\partial y^2} . \label{eq:rb:u} \tag{4}
\end{equation}
$$

This equation is solved with $u(\pm 1) = \partial u/\partial x(\pm 1) = 0$, where the latter follows from the
divergence constraint. In summary, we now seem to have the following equations to solve:

$$
\begin{equation}
    \frac{\partial \nabla^2 {u}}{\partial t} = \frac{\partial^2 H_y}{\partial x \partial y} - \frac{\partial^2 H_x}{\partial y\partial y}  + \sqrt{\frac{Pr}{Ra}} \nabla^4 {u}  + \frac{\partial^2 T}{\partial y^2}, \label{eq:rb:u2} \tag{5}
\end{equation}
$$

$$
\begin{equation}
    \frac{\partial v}{\partial t} + H_y = -  \frac{\partial p}{\partial y} + \sqrt{\frac{Pr}{Ra}} \nabla^2 v, \label{eq:v} \tag{6}
\end{equation}
$$

$$
\begin{equation}
    \frac{\partial T}{\partial t} +\boldsymbol{u} \cdot \nabla T = \frac{1}{\sqrt{RaPr}} \nabla^2 T, \label{eq:T2} \tag{7}
\end{equation}
$$

$$
\begin{equation}
    \nabla \cdot \boldsymbol{u} = 0 \label{eq:div2} \tag{8}.
\end{equation}
$$

However, we note that Eqs. ([5](#eq:rb:u2)) and ([7](#eq:T2)) and ([8](#eq:div2)) do not depend on pressure, and,
apparently, on each time step we can solve ([5](#eq:rb:u2)) for $u$, then ([8](#eq:div2)) for $v$ and finally ([7](#eq:T2)) for $T$.
So what do we need ([6](#eq:v)) for? It appears to have become redundant from the elimination of the
pressure from Eq. ([5](#eq:rb:u2)). It turns out that this is actually almost completely true, but
([5](#eq:rb:u2)), ([7](#eq:T2)) and ([8](#eq:div2)) can only provide closure for all but one of the
Fourier coefficients. To see this it is necessary to introduce some discretization and basis functions
that will be used to solve the problem. To this end we use $P_N$, which is the set of all real polynomials
of degree less than or equal to N and introduce the following finite-dimensional approximation spaces

$$
\begin{equation}
  V_N^B(x) = \{v \in P_N | v(\pm 1) = v´(\pm 1) = 0\}, \label{eq:VB} \tag{9}
\end{equation}
$$

$$
\begin{equation}
  V_N^D(x) = \{v \in P_N | v(\pm 1) = 0\}, \label{eq:VD} \tag{10}
\end{equation}
$$

$$
\begin{equation}
  V_N^T(x) = \{v \in P_N | v(-1) = 0, v(1) = 1\}, \label{eq:VT} \tag{11}
\end{equation}
$$

$$
\begin{equation}
  V_N^W(x) = \{v \in P_N\}, \label{eq:VW} \tag{12}
\end{equation}
$$

$$
\begin{equation}
  V_M^F(y) = \{\exp(\imath l y) | l \in [-M/2, -M/2+1, \ldots M/2-1]\}. \label{eq:VF} \tag{13}
\end{equation}
$$

Here $\text{dim}(V_N^B) = N-4, \text{dim}(V_N^D) = \text{dim}(V_N^W) = N-2$, $\text{dim}(V_N^T) = N$
and $\text{dim}(V_M^F)=M$. We note that
$V_N^B, V_N^D, V_N^W$ and $V_N^T$ can be used to approximate $u, v, T$ and $p$, respectively, in the $x$-direction.
Also note that for $V_M^F$ it is assumed that $M$ is an even number.

We can now choose basis functions for the spaces, using Shen's composite bases for either Legendre or
Chebyshev polynomials. For the Fourier space the basis functions are already given. We leave the actual choice
of basis as an implementation option for later. For now we use $\phi^B(x), \phi^D(x), \phi^W$ and $\phi^T(x)$
as common notation for basis functions in spaces $V_N^B, V_N^D, V_N^W$ and $V_N^T$, respectively.

To get the required approximation spaces for the entire domain we use tensor products of the
one-dimensional spaces in ([9](#eq:VB))-([13](#eq:VF))

$$
\begin{equation}
  W_{BF} = V_N^B \otimes V_M^F, \label{eq:WBF} \tag{14}
\end{equation}
$$

$$
\begin{equation}
  W_{DF} = V_N^D \otimes V_M^F, \label{eq:WDF} \tag{15}
\end{equation}
$$

$$
\begin{equation}
  W_{TF} = V_N^T \otimes V_M^F, \label{eq:WTF} \tag{16}
\end{equation}
$$

$$
\begin{equation}
  W_{WF} = V_N^W \otimes V_M^F. \label{eq:WWF} \tag{17}
\end{equation}
$$

Space $W_{BF}$ has 2D tensor product basis functions $\phi_k^B(x) \exp (\imath l y)$ and
similar for the others. All in all
we get the following approximations for the unknowns

$$
\begin{equation}
    u_N(x, y, t) = \sum_{k \in \boldsymbol{k}_B} \sum_{l \in \boldsymbol{l}} \hat{u}_{kl}(t) \phi_k^B(x) \exp(\imath l y),
\label{_auto1} \tag{18}
\end{equation}
$$

$$
\begin{equation}
    v_N(x, y, t) = \sum_{k \in \boldsymbol{k}_D} \sum_{l \in \boldsymbol{l}} \hat{v}_{kl}(t) \phi_k^D(x) \exp(\imath l y),
\label{_auto2} \tag{19}
\end{equation}
$$

$$
\begin{equation}
    p_N(x, y, t) = \sum_{k \in \boldsymbol{k}_W} \sum_{l \in \boldsymbol{l}} \hat{p}_{kl}(t) \phi_k^W(x) \exp(\imath l y),
\label{_auto3} \tag{20}
\end{equation}
$$

$$
\begin{equation}
    T_N(x, y, t) = \sum_{k \in \boldsymbol{k}_T} \sum_{l \in \boldsymbol{l}} \hat{T}_{kl}(t) \phi_k^T(x) \exp(\imath l y),
\label{_auto4} \tag{21}
\end{equation}
$$

where $\boldsymbol{k}_{x} = \{0, 1, \ldots \text{dim}(V_N^x)-1\}, \, \text{for} \, x\in(B, D, W, T)$
and $\boldsymbol{l} = \{-M/2, -M/2+1, \ldots, M/2-1\}$.
Note that since the problem is defined in real space we will have Hermitian symmetry. This means
that $\hat{u}_{k, l} = \overline{\hat{u}}_{k, -l}$, with an overbar being a complex conjugate,
and similar for $\hat{v}_{kl}, \hat{p}_{kl}$ and
$\hat{T}_{kl}$. For this reason we can get away with
solving for only the positive $l$'s, as long as we remember that the sum in the end goes over both positive
and negative $l's$. This is actually automatically taken care of by the FFT provider and is
not much of an additional complexity in the implementation. So from now on $\boldsymbol{l} = \{0, 1, \ldots, M/2\}$.

We can now take a look at why Eq. ([6](#eq:v)) is needed. If we first solve ([5](#eq:rb:u2)) for
$\hat{u}_{kl}(t), (k, l) \in \boldsymbol{k}_B \times \boldsymbol{l}$, then we can use ([8](#eq:div2)) to
solve for $\hat{v}_{kl}(t)$. But here there is a problem. We can see this by creating the variational
form required to solve ([8](#eq:div2)) by the spectral Galerkin method. To this end make $v=v_N$ in ([8](#eq:div2))
a trial function, use $u=u_N$ a known function and take the weighted inner product over the
domain using test function $q \in W_{DF}$

$$
\begin{equation}
    \left < \frac{\partial u_N}{\partial x} + \frac{\partial v_N}{\partial y}, q \right > _w = 0.
\label{_auto5} \tag{22}
\end{equation}
$$

Here we are using the inner product notation

$$
\begin{equation}
    \left  _w = \int_{-1}^1 \int_0^{2\pi} a \overline{b} dx_wdy_w \left(\approx \sum_{i}\sum_{j} a(x_i, y_j) \overline{b}(x_i, y_j) w(x_i) w(y_j)\right),
\label{_auto6} \tag{23}
\end{equation}
$$

where the exact form of the
weighted scalar product depends on the chosen basis; Legendre has $dx_w=dx$, Chebyshev
$dx_w = dx/\sqrt{1-x^2}$ and Fourier $dy_w=dy/2/\pi$. The bases also have associated quadrature weights
$\{w(x_i) \}_{i=0}^{N-1}$ and $\{w(y_j)\}_{j=0}^{M-1}$ that are used to approximate the integrals.

Inserting now for the known $u_N$, the unknown $v_N$, and $q=\phi_m^D(x) \exp(\imath n y)$ the
continuity equation becomes

$$
\int_{-1}^1 \int_{0}^{2\pi} \frac{\partial}{\partial x} \left(\sum_{k \in \boldsymbol{k}_B} \sum_{l \in \boldsymbol{l}} \hat{u}_{kl}(t) \phi_k^B(x) \exp(\imath l y) \right) \phi_m^D(x) \exp(-\imath n y) dx_w dy_w + \\
  \int_{-1}^1 \int_{0}^{2\pi} \frac{\partial}{\partial y} \left(\sum_{k \in \boldsymbol{k}_D} \sum_{l \in \boldsymbol{l}} \hat{v}_{kl}(t) \phi_k^D(x) \exp(\imath l y) \right) \phi_m^D(x) \exp(-\imath n y) dx_w dy_w  = 0.
$$

The $x$ and $y$ domains are separable, so we can rewrite as

$$
\sum_{k \in \boldsymbol{k}_B} \sum_{l \in \boldsymbol{l}} \int_{-1}^1 \frac{\partial \phi_k^B(x)}{\partial x}  \phi_m^D(x) dx_w \int_{0}^{2\pi} \exp(\imath l y) \exp(-\imath n y) dy_w \hat{u}_{kl} + \\
    \sum_{k \in \boldsymbol{k}_D} \sum_{l \in \boldsymbol{l}} \int_{-1}^1 \phi_k^D(x) \phi_m^D(x) dx_w   \int_{0}^{2\pi} \frac{\partial \exp(\imath l y)}{\partial y} \exp(-\imath n y) dy_w \hat{v}_{kl} = 0.
$$

Now perform some exact manipulations in the Fourier direction and introduce the
1D inner product notation for the $x$-direction

$$
\begin{equation}
    \left(a, b\right)_w = \int_{-1}^1 a(x) b(x) dx_w \left(\approx \sum_{j = 0}^{N-1} a(x_j)b(x_j) w(x_j)\right).
\label{_auto7} \tag{24}
\end{equation}
$$

By also simplifying the notation using summation of repeated indices,
we get the following equation

$$
\begin{equation}
   \delta_{ln} \left(\frac{\partial \phi_k^B}{\partial x}, \phi_m^D \right)_w \hat{u}_{kl}
   + \imath l \delta_{ln} \left(\phi_k^D, \phi_m^D \right)_w \hat{v}_{kl}  = 0.
\label{_auto8} \tag{25}
\end{equation}
$$

Now $l$ must equal $n$ and we can simplify some more

$$
\begin{equation}
   \left(\frac{\partial \phi_k^B}{\partial x}, \phi_m^D \right)_w \hat{u}_{kl}
   + \imath l \left(\phi_k^D, \phi_m^D \right)_w \hat{v}_{kl}  = 0. \label{eq:div3} \tag{26}
\end{equation}
$$

We see that this equation can be solved for
$\hat{v}_{kl} \text{ for } (k, l) \in \boldsymbol{k}_D \times [1, 2, \ldots, M/2]$, but try with
$l=0$ and you hit division by zero, which obviously is not allowed. And this is the reason
why Eq. ([6](#eq:v)) is still needed, to solve for $\hat{v}_{k,0}$! Fortunately,
since $\exp(\imath 0 y) = 1$, the pressure derivative $\frac{\partial p}{\partial y} = 0$,
and as such the pressure is still not required. When used only for
Fourier coefficient 0, Eq. ([6](#eq:v)) becomes

$$
\begin{equation}
\frac{\partial v}{\partial t} + N_y = \sqrt{\frac{Pr}{Ra}} \nabla^2 v. \label{eq:vx} \tag{27}
\end{equation}
$$

There is still one more revelation to be made from Eq. ([26](#eq:div3)). When $l=0$ we get

$$
\begin{equation}
    \left(\frac{\partial \phi_k^B}{\partial x}, \phi_m^D \right)_w \hat{u}_{k,0} = 0,
\label{_auto9} \tag{28}
\end{equation}
$$

which is trivially satisfied if $\hat{u}_{k,0}=0$ for $k\in\boldsymbol{k}_B$. Bottom line is
that we only need to solve Eq. ([5](#eq:rb:u2)) for $l \in \boldsymbol{l}/\{0\}$, whereas we can use
directly $\hat{u}_{k,0}=0 \text{ for } k \in \boldsymbol{k}_B$.

To sum up, with the solution known at $t = t - \Delta t$, we solve

<colgroup>
<col style="width: 20%"/>
<col style="width: 30%"/>
<col style="width: 50%"/>
</colgroup>

       Equation      	      For unknown     	   With indices

 (5)        	        $\hat{u}_{kl}(t)$    	        $(k, l) \in \boldsymbol{k}_B \times \boldsymbol{l}/\{0\}$

   (8)    	        $\hat{v}_{kl}(t)$    	        $(k, l) \in \boldsymbol{k}_D \times \boldsymbol{l}/\{0\}$

   (27)        	        $\hat{v}_{kl}(t)$    	        $(k, l) \in \boldsymbol{k}_D \times \{0\}$

   (7)        	        $\hat{T}_{kl}(t)$    	        $(k, l) \in \boldsymbol{k}_T \times \boldsymbol{l}$

## Temporal discretization

The governing equations are integrated in time using a semi-implicit third order Runge Kutta method.
This method applies to any generic equation

$$
\begin{equation}
 \frac{\partial \psi}{\partial t} = \mathcal{N} + \mathcal{L}\psi \label{eq:genericpsi} \tag{29},
\end{equation}
$$

where $\mathcal{N}$ and $\mathcal{L}$ represents the nonlinear and linear contributions, respectively.
With time discretized as $t_n = n \Delta t, \, n = 0, 1, 2, ...$, the
Runge Kutta method also subdivides each timestep into stages
$t_n^k = t_n + c_k \Delta t, \, k = (0, 1, .., N_s-1)$, where $N_s$ is
the number of stages. The third order Runge Kutta method implemented here uses three stages.
On one timestep the generic equation ([29](#eq:genericpsi))
is then integrated from stage $k$ to $k+1$ according to

$$
\begin{equation}
    \psi^{k+1} = \psi^k + a_k \mathcal{N}^k + b_k \mathcal{N}^{k-1} + \frac{a_k+b_k}{2}\mathcal{L}(\psi^{k+1}+\psi^{k}),
\label{_auto10} \tag{30}
\end{equation}
$$

which should be rearranged with the unknowns on the left hand side and the
knowns on the right hand side

$$
\begin{equation}
    \big(1-\frac{a_k+b_k}{2}\mathcal{L}\big)\psi^{k+1} = \big(1 + \frac{a_k+b_k}{2}\mathcal{L}\big)\psi^{k} + a_k \mathcal{N}^k + b_k \mathcal{N}^{k-1}. \label{eq:rk3stages} \tag{31}
\end{equation}
$$

For the three-stage third order Runge Kutta method the constants are given as

<colgroup>
<col style="width: 33%"/>
<col style="width: 33%"/>
<col style="width: 34%"/>
</colgroup>

$a_n/\Delta t$	 $b_n/\Delta t$	 $c_n / \Delta t$

   8/15              	    0                 	    0

   5/12              	    −17/60            	    8/15

   3/4               	    −5/12             	    2/3

For the spectral Galerkin method used by `shenfun` the governing equation
is first put in a weak variational form. This will change the appearence of
Eq. ([31](#eq:rk3stages)) slightly. If $\phi$ is a test function, $\psi^{k+1}$
the trial function, and $\psi^{k}$ a known function, then the variational form
of ([31](#eq:rk3stages)) is obtained by multiplying ([31](#eq:rk3stages)) by $\phi$ and
integrating (with weights) over the domain

$$
\begin{equation}
    \Big < (1-\frac{a_k+b_k}{2}\mathcal{L})\psi^{k+1}, \phi \Big > _w = \Big < (1 + \frac{a_k+b_k}{2}\mathcal{L})\psi^{k}, \phi\Big > _w + \Big < a_k \mathcal{N}^k + b_k \mathcal{N}^{k-1}, \phi \Big > _w. \label{eq:rk3stagesvar} \tag{32}
\end{equation}
$$

Equation ([32](#eq:rk3stagesvar)) is the variational form implemented by `shenfun` for the
time dependent equations.

## Implementation

To get started we need instances of the approximation spaces discussed in
Eqs. ([9](#eq:VB)) - ([17](#eq:WWF)). When the spaces are created we also need
to specify the family and the dimension of each space. Here we simply
choose Chebyshev and Fourier with 100 and 256 quadrature points in $x$ and
$y$-directions, respectively. We could replace 'Chebyshev' by 'Legendre',
but the former is known to be faster due to the existence of fast transforms.

```
from shenfun import *

N, M = 100, 256
family = 'Chebyshev'
VB = FunctionSpace(N, family, bc=(0, 0, 0, 0))
VD = FunctionSpace(N, family, bc=(0, 0))
VW = FunctionSpace(N, family)
VT = FunctionSpace(N, family, bc=(0, 1))
VF = FunctionSpace(M, 'F', dtype='d')
```

And then we create tensor product spaces by combining these bases (like in Eqs. ([14](#eq:WBF))-([17](#eq:WWF))).

```
W_BF = TensorProductSpace(comm, (VB, VF))    # Wall-normal velocity
W_DF = TensorProductSpace(comm, (VD, VF))    # Streamwise velocity
W_WF = TensorProductSpace(comm, (VW, VF))    # No bc
W_TF = TensorProductSpace(comm, (VT, VF))    # Temperature
BD = VectorSpace([W_BF, W_DF])   # Velocity vector
DD = VectorSpace([W_DF, W_DF])   # Convection vector
```

Here the last two lines create mixed tensor product spaces by the
Cartesian products `BD = W_BF` $\times$ `W_DF` and `DD = W_DF` $\times$ `W_DF`.
These mixed space will be used to hold the velocity and convection vectors,
but we will not solve the equations in a coupled manner and continue in the
segregated approach outlined above.

We also need containers for the computed solutions. These are created as

```
u_  = Function(BD)     # Velocity vector, two components
u_1 = Function(BD)     # Velocity vector, previous step
T_  = Function(W_TF)   # Temperature
T_1 = Function(W_TF)   # Temperature, previous step
H_  = Function(DD)     # Convection vector
H_1 = Function(DD)     # Convection vector previous stage

# Need a container for the computed right hand side vector
rhs_u = Function(DD).v
rhs_T = Function(DD).v
```

In the final solver we will also use bases for dealiasing the nonlinear term,
but we do not add that level of complexity here.

### Wall-normal velocity equation

We implement Eq. ([5](#eq:rb:u2)) using the three-stage Runge Kutta equation ([32](#eq:rk3stagesvar)).
To this end we first need to declare some test- and trial functions, as well as
some model constants

```
u = TrialFunction(W_BF)
v = TestFunction(W_BF)
a = (8./15., 5./12., 3./4.)
b = (0.0, -17./60., -5./12.)
c = (0., 8./15., 2./3., 1)

# Specify viscosity and time step size using dimensionless Ra and Pr
Ra = 10000
Pr = 0.7
nu = np.sqrt(Pr/Ra)
kappa = 1./np.sqrt(Pr*Ra)
dt = 0.1

# Get one solver for each stage of the RK3
solver = []
for rk in range(3):
    mats = inner(div(grad(u)) - ((a[rk]+b[rk])*nu*dt/2.)*div(grad(div(grad(u)))), v)
    solver.append(chebyshev.la.Biharmonic(*mats))
```

Notice the one-to-one resemblance with the left hand side of ([32](#eq:rk3stagesvar)), where $\psi^{k+1}$
now has been replaced by $\nabla^2 u$ (or `div(grad(u))`) from Eq. ([5](#eq:rb:u2)).
For each stage we assemble a list of tensor product matrices `mats`, and in `chebyshev.la`
there is available a very fast direct solver for exactly this type of (biharmonic)
matrices. The solver is created with `chebyshev.la.Biharmonic(*mats)`, and here
the necessary LU-decomposition is carried out for later use and reuse on each time step.

The right hand side depends on the solution on the previous stage, and the
convection on two previous stages. The linear part (first term on right hand side of ([31](#eq:rk3stages)))
can be assembled as

```Python
        inner(div(grad(u_[0])) + ((a[rk]+b[rk])*nu*dt/2.)*div(grad(div(grad(u_[0])))), v)
```

The remaining parts $\frac{\partial^2 H_y}{\partial x \partial y} - \frac{\partial^2 H_x}{\partial y\partial y} + \frac{\partial^2 T}{\partial y^2}$
end up in the nonlinear $\mathcal{N}$. The nonlinear convection term $\boldsymbol{H}$ can be computed in many different ways.
Here we will make use of
the identity $(\boldsymbol{u} \cdot \nabla) \boldsymbol{u} = -\boldsymbol{u} \times (\nabla \times \boldsymbol{u}) + 0.5 \nabla\boldsymbol{u} \cdot \boldsymbol{u}$,
where $0.5 \nabla \boldsymbol{u} \cdot \boldsymbol{u}$ can be added to the eliminated pressure and as such
be neglected. Compute $\boldsymbol{H} = -\boldsymbol{u} \times (\nabla \times \boldsymbol{u})$ by first evaluating
the velocity and the curl in real space. The curl is obtained by projection of $\nabla \times \boldsymbol{u}$
to the no-boundary-condition space `W_TF`, followed by a backward transform to real space.
The velocity is simply transformed backwards.

**Notice.**

If dealiasing is required, it should be used here to create padded backwards transforms of the curl and the velocity,
before computing the nonlinear term in real space. The nonlinear product should then be forward transformed with
truncation. To get a space for dealiasing, simply use, e.g., `W_BF.get_dealiased()`.

```
# Get a mask for setting Nyquist frequency to zero
mask = W_DF.get_mask_nyquist()

def compute_convection(u, H):
    curl = project(Dx(u[1], 0, 1) - Dx(u[0], 1, 1), W_TF).backward()
    ub = u.backward()
    H[0] = W_DF.forward(-curl*ub[1])
    H[1] = W_DF.forward(curl*ub[0])
    H.mask_nyquist(mask)
    return H
```

Note that the convection has a homogeneous Dirichlet boundary condition in the
non-periodic direction. With convection computed we can assemble $\mathcal{N}$
and all of the right hand side, using the function `compute_rhs_u`

```

def compute_rhs_u(u, T, H, rhs, rk):
    v = TestFunction(W_BF)
    H = compute_convection(u, H)
    rhs[1] = 0
    rhs[1] += inner(v, div(grad(u[0])) + ((a[rk]+b[rk])*nu*dt/2.)*div(grad(div(grad(u[0])))))
    w0 = inner(v, Dx(Dx(H[1], 0, 1), 1, 1) - Dx(H[0], 1, 2))
    w1 = inner(v, Dx(T, 1, 2))
    rhs[1] += a[rk]*dt*(w0+w1)
    rhs[1] += b[rk]*dt*rhs[0]
    rhs[0] = w0+w1
    rhs.mask_nyquist(mask)
    return rhs
```

Note that we will only use `rhs` as a container, so it does not actually matter
which space it has here. We're using `.v` to only access the Numpy array view of the Function.
Also note that `rhs[1]` contains the right hand side computed at stage `k`,
whereas `rhs[0]` is used to remember the old value of the nonlinear part.

### Streamwise velocity

The streamwise velocity is computed using Eq. ([26](#eq:div3)) and ([27](#eq:vx)). For efficiency we
can here preassemble both matrices seen in ([26](#eq:div3)) and reuse them every
time the streamwise velocity is being computed. We will also need the
wavenumber $\boldsymbol{l}$, here retrived using `W_BF.local_wavenumbers(scaled=True)`.
For ([27](#eq:vx)) we preassemble the required Helmholtz solvers, one for
each RK stage.

```
# Assemble matrices and solvers for all stages
B_DD = inner(TestFunction(W_DF), TrialFunction(W_DF))
C_DB = inner(TestFunction(W_DF), Dx(TrialFunction(W_BF), 0, 1))
VD0 = FunctionSpace(N, family, bc=(0, 0))
v0 = TestFunction(VD0)
u0 = TrialFunction(VD0)
solver0 = []
for rk in range(3):
    mats0 = inner(v0, 2./(nu*(a[rk]+b[rk])*dt)*u0 - div(grad(u0)))
    solver0.append(chebyshev.la.Helmholtz(*mats0))

# Allocate work arrays and variables
u00 = Function(VD0)
b0 = np.zeros((2,)+u00.shape)
w00 = np.zeros_like(u00)
dudx_hat = Function(W_DF)
K = W_BF.local_wavenumbers(scaled=True)[1]

def compute_v(u, rk):
    if comm.Get_rank() == 0:
        u00[:] = u_[1, :, 0].real
    dudx_hat = C_DB.matvec(u[0], dudx_hat)
    with np.errstate(divide='ignore'):
        dudx_hat = 1j * dudx_hat / K
    u[1] = B_DD.solve(dudx_hat, u=u[1])

    # Still have to compute for wavenumber = 0
    if comm.Get_rank() == 0:
        b0[1] = inner(v0, 2./(nu*(a[rk]+b[rj])*dt)*Expr(u00) + div(grad(u00)))
        w00 = inner(v0, H_[1, :, 0])
        b0[1] -= (2.*a/nu/(a[rk]+b[rk]))*w00
        b0[1] -= (2.*b/nu/(a[rk]+b[rk]))*b0[0]
        u00 = solver0[rk](b0[1], u00)
        u[1, :, 0] = u00
        b0[0] = w00
    return u
```

### Temperature

The temperature equation ([2](#eq:T)) is implemented using a Helmholtz solver.
The main difficulty with the temperature is the non-homogeneous boundary
condition that requires special attention. A non-zero Dirichlet boundary
condition is implemented by adding two basis functions to the
basis of the function space

$$
\begin{equation}
    \phi^D_{N-2} = 0.5(1+x),
\label{_auto11} \tag{33}
\end{equation}
$$

$$
\begin{equation}
    \phi^D_{N-1} = 0.5(1-x),
\label{_auto12} \tag{34}
\end{equation}
$$

with the approximation now becoming

$$
\begin{equation}
    T_N(x, y, t) = \sum_{k=0}^{N-1} \sum_{l \in \boldsymbol{l}} \hat{T}_{kl} \phi^D_k(x)\exp(\imath l y),
\label{_auto13} \tag{35}
\end{equation}
$$

$$
\begin{equation}
                 = \sum_{k=0}^{N-3} \sum_{l \in \boldsymbol{l}} \hat{T}_{kl} \phi^D_k(x)\exp(\imath l y) + \sum_{k=N-2}^{N-1} \sum_{l \in \boldsymbol{l}} \hat{T}_{kl} \phi^D_k(x)\exp(\imath l y).
\label{_auto14} \tag{36}
\end{equation}
$$

The boundary condition requires

$$
\begin{equation}
T_N(1, y, t) = \sum_{k=N-2}^{N-1} \sum_{l \in \boldsymbol{l}} \hat{T}_{kl} \phi^D_k(1)\exp(\imath l y),
\label{_auto15} \tag{37}
\end{equation}
$$

$$
\begin{equation}
             = \sum_{l \in \boldsymbol{l}} \hat{T}_{N-2, l} \exp(\imath l y), \label{eq:TN0} \tag{38}
\end{equation}
$$

and

$$
\begin{equation}
T_N(-1, y, t) = \sum_{k=N-2}^{N-1} \sum_{l \in \boldsymbol{l}} \hat{T}_{kl} \phi^D_k(-1)\exp(\imath l y),
\label{_auto16} \tag{39}
\end{equation}
$$

$$
\begin{equation}
              = \sum_{l \in \boldsymbol{l}} \hat{T}_{N-1, l} \exp(\imath l y). \label{eq:TN1} \tag{40}
\end{equation}
$$

We find $\hat{T}_{N-2, l}$ and $\hat{T}_{N-1, l}$ using orthogonality. Multiply ([38](#eq:TN0)) and
([40](#eq:TN1)) by $\exp(-\imath m y)$ and integrate over the domain $[0, 2\pi]$. We get

$$
\begin{equation}
    \hat{T}_{N-2, l} = \int_{0}^{2\pi} T_N(1, y, t) \exp(-\imath l y) dy,
\label{_auto17} \tag{41}
\end{equation}
$$

$$
\begin{equation}
    \hat{T}_{N-1, l} = \int_{0}^{2\pi} T_N(-1, y, t) \exp(-\imath l y) dy.
\label{_auto18} \tag{42}
\end{equation}
$$

Using this approach it is easy to see that any inhomogeneous function $T_N(\pm 1, y, t)$
of $y$ and $t$ can be used for the boundary condition, and not just a constant.
To implement a non-constant Dirichlet boundary condition, the `FunctionSpace` function
can take any `sympy` function of `(y, t)`, for exampel by replacing the
creation of `VT` by

```
import sympy as sp
y, t = sp.symbols('y,t')
f = 0.9+0.1*sp.sin(2*(y))*sp.exp(-t)
VT = FunctionSpace(N, family, bc=(0, f))
```

For merely a constant `f` or a `y`-dependency, no further action is required.
However, a time-dependent approach requires the boundary values to be
updated each time step. To this end there is the function
`BoundaryValues.update_bcs_time`, used to update the boundary values to the new time.
Here we will assume a time-independent boundary condition, but the
final implementation will contain the time-dependent option.

Due to the non-zero boundary conditions there are also a few additional
things to be aware of. Assembling the coefficient matrices will also
assemble the matrices for the two boundary test functions. That is,
for the 1D mass matrix with $u=\sum_{k=0}^{N-1}\hat{T}_k \phi^D_k $ and $v=\phi^D_m$,
we will have

$$
\begin{equation}
    \left(u, v \right)_w = \left( \sum_{k=0}^{N-1} \hat{T}_k \phi^D_k(x), \phi^D_m \right)_w,
\label{_auto19} \tag{43}
\end{equation}
$$

$$
\begin{equation}
                         = \sum_{k=0}^{N-3} \left(\phi^D_k(x), \phi^D_m \right)_w \hat{T}_k + \sum_{k=N-2}^{N-1} \left( \phi^D_k(x), \phi^D_m \right)_w \hat{T}_k,
\label{_auto20} \tag{44}
\end{equation}
$$

where the first term on the right hand side is the regular mass matrix for a
homogeneous boundary condition, whereas the second term is due to the non-homogeneous.
Since $\hat{T}_{N-2}$ and $\hat{T}_{N-1}$ are known, the second term contributes to
the right hand side of a system of equations. All boundary matrices can be extracted
from the lists of tensor product matrices returned by `inner`. For
the temperature equation these boundary matrices are extracted using
`extract_bc_matrices` below. The regular solver is placed in the
`solverT` list, one for each stage of the RK3 solver.

```
q = TestFunction(W_TF)
p = TrialFunction(W_TF)
solverT = []
lhs_mat = []
for rk in range(3):
    matsT = inner(q, 2./(kappa*(a[rk]+b[rk])*dt)*p - div(grad(p)))
    lhs_mat.append(extract_bc_matrices([matsT]))
    solverT.append(chebyshev.la.Helmholtz(*matsT))
```

The boundary contribution to the right hand side is computed for each
stage as

```
w0 = Function(W_WF)
w0 = lhs_mat[rk][0].matvec(T_, w0)
```

The complete right hand side of the temperature equations can be computed as

```
def compute_rhs_T(u, T, rhs, rk):
    q = TestFunction(W_TF)
    rhs[1] = inner(q, 2./(kappa*(a[rk]+b[rk])*dt)*Expr(T)+div(grad(T)))
    rhs[1] -= lhs_mat[rk][0].matvec(T, w0)
    ub = u.backward()
    Tb = T.backward()
    uT_ = BD.forward(ub*Tb)
    w0[:] = 0
    w0 = inner(q, div(uT_), output_array=w0)
    rhs[1] -= (2.*a/kappa/(a[rk]+b[rk]))*w0
    rhs[1] -= (2.*b/kappa/(a[rk]+b[rk]))*rhs[0]
    rhs[0] = w0
    rhs.mask_nyquist(mask)
    return rhs
```

We now have all the pieces required to solve the Rayleigh Benard problem.
It only remains to perform an initialization and then create a solver
loop that integrates the solution forward in time.

```

# initialization
T_b = Array(W_TF)
X = W_TF.local_mesh(True)
T_b[:] = 0.5*(1-X[0]) + 0.001*np.random.randn(*T_b.shape)*(1-X[0])*(1+X[0])
T_ = T_b.forward(T_)
T_.mask_nyquist(mask)

def solve(t=0, tstep=0, end_time=100):
    while t < end_time-1e-8:
        for rk in range(3):
            rhs_u = compute_rhs_u(u_, T_, H_, rhs_u, rk)
            u_[0] = solver[rk](rhs_u[1], u_[0])
            if comm.Get_rank() == 0:
                u_[0, :, 0] = 0
            u_ = compute_v(u_, rk)
            u_.mask_nyquist(mask)
            rhs_T = compute_rhs_T(u_, T_, rhs_T, rk)
            T_ = solverT[rk](rhs_T[1], T_)
            T_.mask_nyquist(mask)

        t += dt
        tstep += 1
```

A complete solver implemented in a solver class can be found in
[RayleighBenardRk3.py](https://github.com/spectralDNS/shenfun/blob/master/demo/RayleighBenardRK3.py),
where some of the terms discussed in this demo have been optimized some more for speed.
Note that in the final solver it is also possible to use a $(y, t)$-dependent boundary condition
for the hot wall. And the solver can also be configured to store intermediate results to
an `HDF5` format that later can be visualized in, e.g., Paraview. The movie in the
beginning of this demo has been created in Paraview.

1.  **A. Pandey, J. D. Scheel and J. Schumacher**.
    Turbulent Superstructures in Rayleigh-B\'enard Convection,
    *Nature Communications*,
    9(1),
    pp. 2118,
    [doi: 10.1038/s41467-018-04478-0](http://dx.doi.org/10.1038/s41467-018-04478-0),
    2018.


# Linear Shooting Method
#### John S Butler john.s.butler@tudublin.ie   [Course Notes](https://johnsbutler.netlify.com/files/Teaching/Numerical_Analysis_for_Differential_Equations.pdf)    [Github](https://github.com/john-s-butler-dit/Numerical-Analysis-Python)

## Overview
This notebook illustates the implentation of a linear shooting method to a linear boundary value problem. The video below walks through the code.

```python
from IPython.display import HTML
HTML('')
```

## Introduction
To numerically approximate the Boundary Value Problem
 \begin{equation}
y^{''}=p(x)y^{'}+q(x)y+r(x) \ \ \ a < x < b \end{equation}
with the left and right boundary conditions:
 \begin{equation}y(a)=\alpha, \end{equation}
 \begin{equation}y(b) =\beta. \end{equation}

The Boundary Value Problem is divided into two
Initial Value Problems:
1. The first 2nd order Initial Value Problem is the same as the original Boundary Value Problem with an extra initial condtion $y_1^{'}(a)=0$.
\begin{equation}
 y^{''}_1=p(x)y^{'}_1+q(x)y_1+r(x), \ \    y_1(a)=\alpha, \ \ \color{green}{y^{'}_1(a)=0},\\
\end{equation}
2. The second 2nd order Initial Value Problem is the homogenous form of the original Boundary Value Problem, by removing $r(x)$, with the initial condtions $y_2(a)=0$ and $y_2^{'}(a)=1$.

\begin{equation}
y^{''}_2=p(x)y^{'}_2+q(x)y_2, \ \ \color{green}{y_2(a)=0, \ \ y^{'}_2(a)=1}.
\end{equation}

combining these intial values problems together to get the unique solution
\begin{equation}
y(x)=y_1(x)+\frac{\beta-y_1(b)}{y_2(b)}y_2(x),
\end{equation}
provided that $y_2(b)\not=0$.

The truncation error for the shooting method  is
$$ |y_i - y(x_i)| \leq K h^n\left|1+\frac{y_{2 i}}{y_{2 i}}\right| $$
$O(h^n)$ is the order of the numerical method used to approximate the solution of the Initial Value Problems.

```python
import numpy as np
import math
import matplotlib.pyplot as plt
import pandas as pd
import warnings
warnings.filterwarnings("ignore")

```

## Example Boundary Value Problem
To illustrate the shooting method we shall apply it to the Boundary Value Problem:
 \begin{equation}y^{''}=2y^{'}+3y-6,  \end{equation}
with boundary conditions
 \begin{equation}y(0) = 3,  \end{equation}
 \begin{equation}y(1) = e^3+2,  \end{equation}
with the exact solution is
 \begin{equation}y=e^{3x}+2.  \end{equation}
The __boundary value problem__ is broken into two second order __Initial Value Problems:__
1. The first 2nd order Intial Value Problem is the same as the original Boundary Value Problem with an extra initial condtion $u^{'}(0)=0$.
\begin{equation}
u^{''} =2u'+3u-6, \ \ \ \ u(0)=3, \ \ \ \color{green}{u^{'}(0)=0}
\end{equation}
2. The second 2nd order Intial Value Problem is the homogenous form of the original Boundary Value Problem with the initial condtions $w^{'}(0)=0$ and $w^{'}(0)=1$.
\begin{equation}
w^{''} =2w^{'}+3w, \ \ \ \ \color{green}{w(0)=0}, \ \ \ \color{green}{w^{'}(0)=1}
\end{equation}

combining these results of these two intial value problems as a linear sum
\begin{equation}
y(x)=u(x)+\frac{e^{3x}+2-u(1)}{w(1)}w(x)
\end{equation}
gives the solution of the Boundary Value Problem.

## Discrete Axis
The stepsize is defined as
 \begin{equation}h=\frac{b-a}{N}, \end{equation}
here it is
 \begin{equation}h=\frac{1-0}{10}, \end{equation}
giving
 \begin{equation}x_i=0+0.1 i, \end{equation}
for $i=0,1,...10.$

```python
## BVP
N=10
h=1/N
x=np.linspace(0,1,N+1)
fig = plt.figure(figsize=(10,4))
plt.plot(x,0*x,'o:',color='red')
plt.xlim((0,1))
plt.title('Illustration of discrete time points for h=%s'%(h))

plt.show()
```

## Initial conditions
The initial conditions for the discrete equations are:
$$ u_1[0]=3$$
$$ \color{green}{u_2[0]=0}$$
$$ \color{green}{w_1[0]=0}$$
$$ \color{green}{w_2[0]=1}$$

```python
U1=np.zeros(N+1)
U2=np.zeros(N+1)
W1=np.zeros(N+1)
W2=np.zeros(N+1)

U1[0]=3
U2[0]=0

W1[0]=0
W2[0]=1
```

## Numerical method
The Euler method is applied to numerically approximate the solution of the system of the two second order initial value problems they are converted in to two pairs of two first order initial value problems:

### 1. Inhomogenous Approximation
The plot below shows the numerical approximation for the two first order Intial Value Problems
\begin{equation}
u_1^{'} =u_2, \ \ \ \ u_1(0)=3,
\end{equation}
\begin{equation}
u_2^{'} =2u_2+3u_1-6, \ \ \ \color{green}{u_2(0)=0},
\end{equation}

that Euler approximate of the inhomogeneous two Initial Value Problems is :
 \begin{equation}u_{1}[i+1]=u_{1}[i] + h u_{2}[i] \end{equation}
 \begin{equation}u_{2}[i+1]=u_{2}[i] + h (2u_{2}[i]+3u_{1}[i] -6) \end{equation}
with $u_1[0]=3$ and $\color{green}{u_2[0]=0}$.

```python
for i in range (0,N):
    U1[i+1]=U1[i]+h*(U2[i])
    U2[i+1]=U2[i]+h*(2*U2[i]+3*U1[i]-6)

```

### Plots
The plot below shows the Euler approximation of the two intial value problems $u_1$ on the left and $u2$ on the right.

```python
fig = plt.figure(figsize=(12,4))
ax = fig.add_subplot(1,2,1)
plt.plot(x,U1,'^')
#plt.title(r"$u_1'=u_2, \ \  u_1(0)=3$",fontsize=16)
plt.grid(True)

ax = fig.add_subplot(1,2,2)
plt.plot(x,U2,'v')
#plt.title("U2", fontsize=16)

plt.grid(True)
plt.show()
```

### 2. Homogenous Approximation
The homogeneous Bounday Value Problem is divided into two first order Intial Value Problems
\begin{equation}
w_1^{'} =w_2, \ \ \ \ \color{green}{w_1(1)=0}
\end{equation}
\begin{equation}
w_2^{'} =2w_2+3w_1,  \ \ \ \color{green}{w_2(1)=1}
\end{equation}

The Euler approximation of the homogeneous of the two Initial Value Problem is
 \begin{equation}w_{1}[i+1]=w_{1}[i] + h w_{2}[i] \end{equation}
 \begin{equation}w_{2}[i+1]=w_{2}[i] + h (2w_{2}[i]+3w_{1}[i]) \end{equation}
with $\color{green}{w_1[0]=0}$ and $\color{green}{w_2[1]=1}$.

```python
for i in range (0,N):
    W1[i+1]=W1[i]+h*(W2[i])
    W2[i+1]=W2[i]+h*(2*W2[i]+3*W1[i])
```

### Homogenous Approximation

### Plots
The plot below shows the Euler approximation of the two intial value problems $u_1$ on the left and $u2$ on the right.

```python
fig = plt.figure(figsize=(12,4))
ax = fig.add_subplot(1,2,1)
plt.plot(x,W1,'^')
plt.grid(True)
plt.title("w_1'=w_2,  w_1(0)=0",fontsize=16)

ax = fig.add_subplot(1,2,2)
plt.plot(x,W2,'v')
plt.grid(True)
plt.title("w_2'=2w_2+3w_1,  w_2(0)=1",fontsize=16)
plt.tight_layout()
plt.subplots_adjust(top=0.85)

plt.show()
beta=math.exp(3)+2
y=U1+(beta-U1[N])/W1[N]*W1
```

## Approximate Solution
Combining together the numerical approximation of $u_1$ and $w_1$ as a weighted sum
 \begin{equation}y(x[i])\approx u_{1}[i] + \frac{e^3+2-u_{1}[N]}{w_1[N]}w_{1}[i] \end{equation}
gives the approximate solution of the Boundary Value Problem.

The truncation error for the shooting method using the Euler method is
 \begin{equation} |y_i - y(x[i])| \leq K h\left|1+\frac{w_{1}[i]}{u_{1}[i]}\right|  \end{equation}
$O(h)$ is the order of the method.

The plot below shows the approximate solution of the Boundary Value Problem (left), the exact solution (middle) and the error (right)

```python
Exact=np.exp(3*x)+2
fig = plt.figure(figsize=(12,4))
ax = fig.add_subplot(2,3,1)
plt.plot(x,y,'o')

plt.grid(True)
plt.title("Numerical",
          fontsize=16)

ax = fig.add_subplot(2,3,2)
plt.plot(x,Exact,'ks-')

plt.grid(True)
plt.title("Exact",
          fontsize=16)

ax = fig.add_subplot(2,3,3)
plt.plot(x,abs(y-Exact),'ro')
plt.grid(True)
plt.title("Error ",fontsize=16)

plt.tight_layout()
plt.subplots_adjust(top=0.85)
plt.show()
```

### Data
The Table below shows that output for $x$, the Euler numerical approximations $U1$, $U2$, $W1$ and $W2$ of the system of four Intial Value Problems, the shooting methods approximate solution $y_i=u_{1 i} + \frac{e^3+2-u_{1}(x_N)}{w_1(x_N)}w_{1 i}$ and the exact solution of the Boundary Value Problem.

```python
d = {'time x_i': x[0:10],
     'U1':np.round(U1[0:10],3),
     'U2':np.round(U2[0:10],3),
     'W1':np.round(W1[0:10],3),
     'W2':np.round(W2[0:10],3),
     'y_i':np.round(W2[0:10],3),
     'y(x_i)':np.round(W2[0:10],3)
    }
df = pd.DataFrame(data=d)
df
```

      time x_i
      U1
      U2
      W1
      W2
      y_i
      y(x_i)

      0
      0.0
      3.000
      0.000
      0.000
      1.000
      1.000
      1.000

      1
      0.1
      3.000
      0.300
      0.100
      1.200
      1.200
      1.200

      2
      0.2
      3.030
      0.660
      0.220
      1.470
      1.470
      1.470

      3
      0.3
      3.096
      1.101
      0.367
      1.830
      1.830
      1.830

      4
      0.4
      3.206
      1.650
      0.550
      2.306
      2.306
      2.306

      5
      0.5
      3.371
      2.342
      0.781
      2.932
      2.932
      2.932

      6
      0.6
      3.605
      3.222
      1.074
      3.753
      3.753
      3.753

      7
      0.7
      3.927
      4.347
      1.449
      4.826
      4.826
      4.826

      8
      0.8
      4.362
      5.795
      1.932
      6.226
      6.226
      6.226

      9
      0.9
      4.942
      7.663
      2.554
      8.050
      8.050
      8.050

```python

```

Chapter 06 - Boundary Value Problems/601_Linear Shooting Method.ipynb