___
# Machine Learning I.
## Problem Set 1: Linear Regression (14 points total)

### Instructions
The graded portion of problem set 1 consists of one notebook:
```
ps1_nb1-ml.ipynb
```
There is also a quick introduction to Jupyter notebooks and a non-graded tutorial of numpy arrays:
```
0-ten_minute_introduction_to_notebooks.ipynb
1-numpy_introduction.ipynb
```
Both are recommended.

### Due Date
* Group D 19.02.2020 before 23:59:59 (CET)

### Instructor
* Prof. Dr. Gregory Wheeler ([gregorywheeler.org](http://gregorywheeler.org))

---

### Declare your collaborators
You may work alone or in a group. The maximum group size is 4 people.

If you work in a group, use the next cell to enter the list of names (first, last) of your collaborators.
~~~python
# Example
COLLABORATORS = ['Stu Dent', 'May Bee', 'Ki Val Storr']
~~~
You should also familiarize yourself with the collaboration policy on the course Canvas page.

```python
# Be sure your names are strings
COLLABORATORS = ["Gaurav Sharma"]

```

---
## Money cannot buy happiness. Or can it?

The populations of some countries are happier than other countries. In this exercise, you will investigate the relationship between a country's per capita GDP and its "happiness score".  Since 2012, the annual [World Happiness Report](https://worldhappiness.report/), published by the United Nations' [Sustainable Development Solutions Network](https://www.unsdsn.org/), asks people around the world how happy they are. The happiness score uses the [Cantril scale](https://news.gallup.com/poll/122453/understanding-gallup-uses-cantril-scale.aspx), which is a subjective scale constructed by asking respondents to imagine a ladder on which the best possible life for themselves receives the score of 10, the worst possible life a score of 0, and then are asked to rate their own lives on that scale between 0 and 10.  The questionaire is given to a representative sample of each country each year.

We expect more individual wealth to correlate with higher happiness. But, how much of the variation in happiness is explained by individual wealth?  That is the question we will explore.

There are three main tasks in this notebook.

 - PART A. Implement Gradient Descent to calculate the coefficients of a univariate linear regression
 - PART B. Use a built-in library to check your implementation in A and perform some analysis
 - PART C. Prepare data yourself and apply your model.

You will be asked to write some code and also answer several questions about the model and data. The first step is to run the next cell to load the basic libraries we will use in this assignment.

```python
# import numpy and set up plotting parameters
import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
```

# PART A - Gradient Descent

## I. Loading and Formatting Data
In the ps1 directory you will find the data set for problem 1, `ps1_data.csv`. File loading methods vary depending on the format of the data file. This particular data set is a _comma-seperated text file_ (.csv), so we will use the numpy function `np.loadtxt` and specify that the columns are delimited by commas.

Run the next cell to load `ps1_data.csv` into a numpy array assigned to the variable `data`.

```python
data = np.loadtxt('ps1_data1.csv', delimiter=',')
#data
```

Recall that numpy arrays are Python's standard datatype for computing with vectors, matrices, and tensors, and the numerical methods of numpy -- such as `np.dot` and `np.sum` -- are element-wise operations. Although there is also a Python matrix package designed specifically to do linear algebra operations, we will use numpy.

(For those of you new to Python but who are familiar with linear algebra or with programming matrix operations in MATLAB, [this page](https://www.khanacademy.org/math/precalculus/precalc-matrices/multiplying-matrices-by-matrices/v/matrix-multiplication-intro) contains useful comparison between Python and MATLAB to help you get quickly up to speed.)

You can confirm that data is a numpy array by checking its type, i.e. by excuting `type(data)`. It is also important to know about the dimensions of `data` -- that is, the number of rows (training examples) and the number of columns (features and target). This information is given by using the method `.shape`, i.e., by executing `data.shape`.

Tip: You can insert a cell anywhere (+) and delete it (scissors) or any other cell that is not locked

However, before turning in your assignment, you should remove any cells you have added.

Tip: To clear the output of your notebook and your local memory, you can restart the kernel and clear  “Kernel >> Restart and Clear all Output":

You should see that `data` has two columns. The first column is the feature $x$, per Capita GDP of each country in `data`, and the second column is the target $y$, the corresponding Happiness index. The number of rows denotes the number of training examples, which by convention is assigned to the variable `m`.

In the next cell, you should complete the function `set_variables()` which will take an array as an argument and return three variables: `x` and `y` , each of which is itself an array; and `m`, an integer. When `set_variables()` is called specifically on `data`, the three variables are

    x - the features of psl_data; i.e., the first column, which are city populations
    y - the target variable of psl_data; i.e., the second column, which are revenues
    m - the number of training examples in psl_data.

Your code should work on any two-column array, not only on `data`.

Tip: Remember that arrays in Python are 0-indexed.

### Completing Functions

The next cell has a partially completed function, `set_variables()`.  Inside the code block, you will see two lines:

~~~python

# YOUR CODE HERE
raise NotImplementedError()
~~~

To complete the function, you should delete both of these lines and replace the comment `# YOUR CODE HERE` with your line or lines of code.  The second line raises a  "not implemented error" and is included automatically to guard against accidentally submitting an incomplete assignment.  To pass the final validation, you must remove each of these errors from your functions.

```python
def setVariables(data_array) :
    """

    The function set_variables() is called on one argument:

        data_array - an np.array

    and returns three global variables:
        x - the first column of data_array
        y - the second column of the data_array
        m - the number of rows of the data_array na
    Your code should run on any two-column data_array.
    """
    global x
    global y
    global m
    # YOUR CODE HERE

    x = [data_array[i][0] for i in range(len(data_array))]

    y = [data_array[i][1] for i in range(len(data_array))]

    #Pythonic way of coding!
    m = len(data_array)
    #raise NotImplementedError()
    return x, y, m
```

### Test Cells
The next three cells are test cells that will evaluate your implementation of `setVariables()`. You should run all three. (They will be run automatically when your notebook is graded.) Each of these cells has a line of code, which is a "public" test of your code, but there are also "hidden" tests of your code.  The public tests are designed to give you some initial feedback that you are on the right track.

```python
"""TEST CELL 1 for set_variables"""
setVariables(data)
assert x[34] == 1.870765686
```

```python
"""TEST CELL 2 for set_variables"""
setVariables(data)
assert y[71] == 5.429999828
```

```python

```

```python
"""TEST CELL 3 for m of set_variables"""
setVariables(data)
assert m < 200
```

## II. Data Visualization
The following block of code produces a basic scatterplot with a legend and title.

```python
# produce scatterplot

plt.scatter(x,y, label="ps1_data1")
plt.title('2017 World Happiness by Country')
plt.xlabel(r'Per Capita GPD')
plt.ylabel(r'Happiness Score')
plt.legend()
```

## III. Implementing Gradient Descent

Now we will implement gradient descent to fit the **linear regression** parameters $\begin{align} \theta & = (\theta_0,\theta_1)\end{align}$ to the data set, `data`.

### Univariate Linear Regression

A univariate linear regression has one _independent variable or regressor, x,_ which is called a _feature_ in machine learning. The _dependent variable_ or _target_ variable $y$, is the  true happiness score.  The goal is to fit a linear model to estimate or predict the value of $y$, happiness, as some linear function of $x$, per capita GDP. Specifically, the hypothesis $\begin{align} h(x; \theta) \end{align}$ is determined by $\begin{align} \boldsymbol{\theta} \end{align}$, a two-dimensionpal vector $\theta = (\theta_0,\theta_1)$, and the single feature $ x_1$, such that $$h(x; \theta) = \theta_0 + \theta_1x_1.$$

To fit a model of this form to data, we need a way to pick values for $\theta_0$ and $ \theta_1$ that "best fit" the data. Intuitively, we want to select parameters that minimize the error of its estimates of the known values for $y$ in the training data. Specifying, we will use residual squared error (RSS) as the numeric measure of performance and ordinary least squares as the definition of "best fit."  Defining the precise form of this minimization task is done by defining the optimization objective of the algorithm, which is the next step.

### Optimization Objective

The optimization objective of linear regression is to minimize the residual sum of squares, RSS,

$$RSS = \sum_{i=1}^m (\hat{\epsilon}^{(i)})^2 = \sum_{i=1}^{m} (y^{(i)} - (\hat{\theta}_0 + \hat{\theta}_1x^{(i)} ))^2$$

where RSS is the standard loss function used for fitting linear regression models.  The mean squared error (MSE) is $\frac{RSS}{m}$.  The cost function $J(\boldsymbol{\theta})$ we will minimize is

$$J(\boldsymbol{\theta}) = \frac{1}{2m}\sum_{i=1}^m (h(x^{(i)}; \boldsymbol{\theta}) - y^{(i)})^2$$

which is $\frac{1}{2} * MSE$.

The index $i$ ranges over the number of training examples, $m$, in `data`. Informally, the cost function $J$ applied to $\begin{align} \boldsymbol{\theta} & = (\hat{\theta}_0,\hat{\theta}_1)\end{align}$ assesses the "cost" of using particular values for $\hat{\theta}_0$ and $\hat{\theta}_1$ to fit a line to `data`, measured as squared-error loss, and the optimization problem is one of picking values for $\hat{\theta}_0$ and for $\hat{\theta}_1$ that makes those costs as close to zero as possible. Unpacking the right side of this equation further, for each training example $i$ (that is, for each $i$ of the $m$ rows of `data` ), a hypothesis $h$ parameterized by $\boldsymbol{\theta}$ looks at the feature $x$ of the $i$th training example, written $x^{(i)}$, and computes an estimate of the value of $y$ for this $i$th example, written $h(x^{(i)}; \boldsymbol{\theta})$. The total loss then is the average squared-error loss for all $m$ training examples.

Specifically, the optimization objective is to minimize the cost function $J(\boldsymbol\theta)$, written as $\min_{\boldsymbol{\theta}} J(\boldsymbol\theta)$. The fraction $\frac{1}{2}$ and changing $(y - h)^2$ in RSS to $(h- y)^2$ are each mathematical conveniences that make the next step simplier and more intuitive, respectively.

Notation 1: Recall that in frequentist statistics $\theta$ is a true but unknown parameter and $\hat{\theta}$ is an estimate of that unknown parameter. In short, the estimators that we construct, manipulate, and evaluate all wear hats. But, since we never deal directly with "true but unknown" parameters, when the discussion turns to coding we drop the hat notation.

### Implementation

The adjustable parameters of a univariate regression model $h(x^{(i)}; \boldsymbol{\theta})$  are $\theta_0$ and $\theta_1$ -- or, if you did not read the previous remark about notation, $\hat{\theta}_0$ and $\hat{\theta}_1$. The parameters $\theta_0$ and $\theta_1$  determine the $y$-intercept and slope, respectively.  They are the values that gradient descent changes to minimize the cost $J(\boldsymbol{\theta})$.

There are a family of gradient descent algorithms. We will implement **batch gradient descent**. In batch gradient descent, at each iteration of the algorithm the values for the parameter vector $\boldsymbol{\theta}$ (should!) change  closer to the optimal values that realize the lowest cost $ J(\boldsymbol\theta)$.

Notation 2:  The actual optimal parameter values that minimizes $J$ are unknown.  Were our discussion to dwell on the difference between the actual parameters and our estimated parameters, then we'd be compelled to put hats on the estimated parameters.

As we discussed in class, at each step batch gradient descent **simultaneously updates** all parameters $\begin{align} \theta_j \end{align}$:

              $\begin{align}\theta_j := \theta_j - \alpha \frac{1}{m} \sum_{i=1}^{m}(h(x^{(i)}; \boldsymbol{\theta}) - y^{(i)})x^{(i)} \end{align}$ simultaneous update of $\begin{align} \theta_j \end{align}$ for all _j_.

The next series of steps guides you through an implementation of gradient descent.

___

**Step 1**. Observe that there are two parameters, $\theta_0$ and $\theta_1$, but only one feature in `data`. (Check the shape of `x` if you are unconvinced.) Since $\theta_0$ is the y-intercept term, this is the same as if we multiplied $\theta_0$ by 1, for each m entries. We say that $x_0$ = 1 is a **intercept term**, and we add intercept terms in such cases to ensure that feature vectors and parameter vectors are comformable.

With this explanation in mind, add a column of 1's to the single column feature vector within data. We do this first by initializing an $(m, 2)$ array of 1's, then replacing the second column of X with the first column of `data`. This new row of ones is to accommodate the $y$-intercept parameter, $\theta_0$. The next step is to
initialize a $(1, 2)$ array of zeros, called `theta` , which initializes the values for $\theta_0$ and $\theta_1$, respectively. Our gradient descent algorithm will update `theta` in order to minimize the cost function J.

Running the next cell initializes the variables `x` and `theta` for you.

```python
# initialize data array
X = np.ones((m, 2))
X[:,1] = data[:,0]
# initalize model
theta = np.zeros(2)
```

---
## Question 1

Suppose that in addition to "per capita GDP" you also had the average "life expectancy" for each country to use as features in a linear model to estimate each country's happiness score.

Which of the following statements are true? Select all and only that apply.

* A) $h(\boldsymbol{x}; \boldsymbol{\theta}) = \theta_0 + \theta_1x_1 + \theta_2x_2$
* B) $h(\boldsymbol{x}; \boldsymbol{\theta}) = \theta_0 + \theta_1x_1 + x_2$
* C) $h(\boldsymbol{x}; \boldsymbol{\theta}) = \theta_0 + \theta_1x_1 x_2$

Suppose now you wish to initialize a data array and parameter vector for this model.  Which block of code could you use for this model?  (Assume as above that the last column of `data` is the target, $y$.)

* D)
~~~python
# initialize data array
X = np.ones((m, 3))
X[:,1] = data[:,0]
# initalize model
theta = np.zeros(2)
~~~

* E)
~~~python
# initialize data array
X = np.ones((m, 3))
X[:,0] = data[:,0]
X[:,1] = data[:,1]
# initalize model
theta = np.zeros(3)
~~~

* F)
~~~python
# initialize data array
X = np.ones((m, 3))
X[:,1] = data[:,0]
X[:,2] = data[:,1]
# initalize model
theta = np.zeros(3)
~~~

From the list of possible answers `['A', 'B', 'C', 'D', 'E', 'F]` , record your answer or answers as a list in the next cell by completing the function `ans_one()`.  For example, if the possible answers were `['P','Q','R']` and you selected P and Q as the correct answers, then you would simply write:
~~~python
ans = ['P', 'Q']
~~~
in the function `ans_one()`.

---

```python
def ans_one():
    """ Returns a list of your answers.

    Parameters
    ----------
    :ans:  list
        The list of your answers. Elements of :ans: are strings.
    Returns
    -------
    :ans:
    """

    # YOUR CODE HERE
    ans = ['A','F']
    #raise NotImplementedError()
    return ans
```

```python
# test cell for ans_one()
ans = ans_one()
possible_ans = ['A', 'B', 'C', 'D', 'E', 'F']
if type(ans) == list and all(ii in possible_ans for ii in ans):
    assert True
else:
    raise AssertionError("Inadmissible answer. Check that your answer to ans_one is the correct format.")
```

**Step 2.** Compute the squared-error cost function from above:
\begin{align} \frac{1}{2m}\sum_{i=1}^m (h(x^{(i)}; \boldsymbol{\theta}) - y^{(i)})^2 \end{align}

The function `costJ(X, y, theta)` takes as arguments the (_m_, 2)-dimension features array, `X` ,the (_m_, 1)-dimension target column vector, `y`, and the initial two-element parameter vector `theta`.

There are several ways to implement this code, but a good strategy is to break the operation down into several steps and implement those steps as separate lines of code. Here is one way to attack the problem in two lines of code.

First Line of Code

Setting aside the indices, notice the general form of the hypothesis: $$h(x; \boldsymbol{\theta}) = \theta_0x_0 + \theta_1x_1.$$ Our first operation, then, is to multiply each $x^{(i)}$ in the second column of X by $\theta_1$ and to keep $\theta_0$ as is, which is why we added a column of 1s to X above: that is, $\begin{align} x_0 =  1\end{align}$ for all _m_ training examples. Then, for each estimate you can subtract the corresponding values of $y$.

 Hint: We deliberately highlighted the dimensions of the arrays 'X' and ‘theta’ because there is a nice result from matrix algebra. Let $\boldsymbol{X}$ and $\boldsymbol{\theta}$ be:

 $$\boldsymbol{X} = \begin{bmatrix}
 				1 & x_1^{(1)}  \\
				1 & x_1^{(2)}   \\
				1 & x_1^{(3)}   \\
				\vdots & \vdots \\
				1 & x_1^{(m)}
			\end{bmatrix}
			%
			\quad
			%
			%
 \boldsymbol{\theta} = \begin{bmatrix} \theta_0 \\ \theta_1
	\end{bmatrix}$$

Then,
$$\begin{bmatrix}
 				1 \\
				1 \\
				\vdots \\
				1
		   \end{bmatrix}
\theta_0  +
%
 \begin{bmatrix}
 				x_1^{(1)}  \\
				x_1^{(2)}  \\
				\vdots \\
				x_1^{(m)}
		   \end{bmatrix}
\theta_1
=
\begin{bmatrix}
 				\theta_0 + \theta_1x^{(1)} \\
				\theta_0 + \theta_1x^{(2)} \\
				\vdots \\
				\theta_0 + \theta_1x^{(m)} \\
			\end{bmatrix} = \boldsymbol{X\theta}$$

where $\boldsymbol{X\theta}$ is an $m \times 1$ vector.

See the tutorial, `ps1-tutorial.ipynb`, for information about `np.dot` specifically, and about matrix multiplication in general.

The first step is to compute the difference between each of the $m$ estimates in the vector $\boldsymbol{X\theta}$ and the corresponding target values in the vector $y$. The next step is to square this difference, which we may use the built-in NumPy function `np.power(---,---)`.

So, the vector of quantities $(h(x; \boldsymbol{\theta}) - y^{(i)})^2$ for each _i_ of the _m_ examples can be computed and stored to a local variable `scores` with one line of code of the form

~~~python
# YOUR CODE HERE
scores = np.power(---, ---)
return
~~~

Second Line of Code

What remains is to sum `scores` and divide by the number of training examples multiplied by two. The result should be a scalar saved to the variable `ans`:

~~~python
# YOUR CODE HERE
scores = np.power(---, ---)
ans = ---
return ans
~~~

where `ans` will use the value computed in the first line of code and saved to the variable `scores`.

```python
def costJ(X, y, theta):
    """Implement the squared-error cost function, compute_cost_J.
    compute_cost_J(X, y, theta) computes the cost of using the
    theta to parameterize a linear regression hypothesis to fit
    the data in X and y.
    """
    # Your line of Code

    scores = np.power((np.dot(X,theta) - y),2) #Finding error cost

    ans = np.divide(np.sum(scores),np.multiply(2,m))

    #raise NotImplementedError()
    return ans
```

```python
"""Test cell 1 for costJ"""
assert round(costJ(X,y,theta), 1) == 15.0
```

```python
"""Test cell 2 for costJ"""
```

    'Test cell 2 for costJ'

**Step 3**. We now turn to the implementation of gradient descent. The function,

    batchGradientDescent(X, y, theta, alpha, num_iterations)

takes 5 arguments:

    X - an array of features from data
    y - an array of labels from data
    theta - an array of parameters that gradient descent adjust to minimize loss according to J_cost
    alpha - the learning rate
    num_iterations - the number of iterations that the algorithm runs before terminating.

The loop structure to iterate gradient descent `num_iterations` times is given for you. What is missing are the simultaneous updates of the two components of __$\theta$__, namely $\theta_0$ and $\theta_1$.

__Gradient Descent Algorithm__

    Enter loop
        Repeat:

$$\theta_0=:\theta_0 - \alpha \frac{\partial} {\partial \theta_0} J(\theta_0, \theta_1)$$
$$\theta_1=:\theta_1 - \alpha \frac{\partial} {\partial \theta_1} J(\theta_0, \theta_1)$$

        Stop at convergence
    Exit loop

What you will need to do complete two steps within the for loop, namely update theta and create a history of the computed cost function _J($\theta$)_ for each
iteration of the algorithm.

__1. Update theta__

The code initializes `theta_new` as a copy of theta. Within the for loop, your simultaeous update of `theta_new` should be completed in one line of
_vectorized code_:

```python
--- -= --- <operation> np.dot(---, ---) <operation> ---
```

which may involve the following additional pieces, not necessarily in order,  may be used more than once:

  * \`X`
  * \`y`
  * \`theta_new`
  * \`X.T`, the transposition of X
  * \`alpha`
  * \`np.dot(---,---) (operation) ----`
  * with \`(operation)` from the set {—, *, /}

 Hint: The partial derivative terms for each component $\theta_j$ of the parameter vector $\boldsymbol{\theta}$ for is:
$$
\theta_j:   {\frac{\partial}{\partial\theta_j}  J(\theta_1, \ldots, \theta_j \ldots, \theta_n)} =  \frac{1}{m} \sum_{i=1}^m (h(x^{(i)}; \boldsymbol{\theta}) - y^{(i)}) \cdot x_j^{(i)}, $$
where $x_j^{(i)} = 1$ for all $i$.

We saw above that $h(x^{(i)}; \boldsymbol{\theta}) - y^{(i)})$ is calculated by $X \theta$. Then, the partial derivatives are either computed by $(X \theta) X$ or $(X \theta) X^T$. Which is it? Why?

__2. Store history of the cost function J__

The second step is to write into the $j$th element of the vector `J_history` the result from computing the cost determined by executing costJ using `theta_new` that was just updated in the $j$th iternation. Bear in mind that `ii` is used as the index for `num_iterations`, to avoid confusing this for indices used to index rows ($i$) and columns ($j$) of our training set.
___

Now, complete the implementation of gradient descent.

```python
def batchGradientDescent(X, y, theta, alpha, num_iterations):
    """gradient_descent performs batch gradient descent to minimize theta and return a history
    of the gradient steps to plot.

    gradient_descent updates theta by taking a number of gradient steps, fixed by the
    parameter num_iterations, where the size of those steps is determined by the learning

    rate, alpha.
    """
    ## Initialize variables
    # m is the number of training samples
    m = len(y)

    # J_history is the cost function history, intialized to a vector of zeros
    # for each iteration step
    J_history = np.zeros(num_iterations)
    # initialize theta_new with a copy of theta.
    theta_new = theta.copy()

    for ii in range(num_iterations):
        # YOUR CODE HERE
        # 1st line: update theta_new
        theta_new = theta_new - np.multiply(np.dot(np.transpose(X),np.subtract(np.dot(X,theta_new),y)),(alpha / m))

        # 2nd line: update J_history with compute_cost using theta_new
        J_history[ii] = costJ(X,y,theta_new)

        #raise NotImplementedError()

    return theta_new, J_history
```

## III. Running Gradient Descent

The next step is to set the learning rate and number of iterations. Setting the correct values for these two parameters depends on the data set. In practise, you would need to try out different values of these parameters to see how your alhorithm performed. But,I provide you with parameter settings that will work if your code up to now is implemented correctly.

Specifically, be sure the next cell has the following parameter values:

~~~python
## gradient_descent adjustable parameters
# learning rate
alpha = 0.2
#number of iterations
iterations = 1100
~~~

```python
## gradient_descent adjustable parameters
# learning rate
alpha = 0.2
#number of iterations
iterations = 1100
```

Run the next cell to instantiate `alpha` and `iterations`.
Now we are ready to run gradient descent with the parameter settings for `alpha` and the number of `iterations` excuting the following line of code in the next cell.

```python
# run gradient descent
theta_new, J_history = batchGradientDescent(X, y, theta, alpha, iterations)

```

Gradient descent (correctly parameterized by `alpha` and `iterations`) returns values for $\theta_0$ and $\theta_1$ stored in `theta_new`,along with `J_history`.

Run the next cell to see the values for $\theta_0$ and $\theta_1$.

```python
# Check the output of batchGradientDescent
theta_new
```

    array([3.22558776, 2.164952  ])

When implemented correctly, running `batchGradientDescent` on `data` with learning rate `alpha = 0.2` and `iterations = 1300` should return a value for $\theta_0$ that is aproximately $3.20$ and a value for $\theta_1$ that is aproximately $2.18$, and `J_history` should be an array of monotonically decreasing values.

Run the following test cells to check. Note that there are hidden test cells that will be used for grading.

```python

```

```python

```

```python

```

## IV. Visualization, Tuning, and Debugging

### Plotting the history of J

The variable J_history stores the computed cost function for each iteration of batchGradientDescent , which for our example we set to 1100 iterations. How do we determine that number of iterations is long enough to converge? One way to do this is to plot the value of $J(\theta)$ at each iteration to see its
performance.

The next cell plots `J_history` with your initial parameters.  If your implementation is correct, your cost function should decrease with each iteration and loosely resemble the following plot:

Hint: You can use this plot to check whether you have implemented `batchGradientDescent` correctly, and perhaps to give you clues about how to correct
your code if you find something amiss.

```python
# Plot cost function vs. iteration
plt.plot(J_history)
plt.xlabel(r'Iteration number' )
plt.ylabel(r'$J(\theta)$')
```

### Selecting Learning Rates

We selected the learning rate `alpha = 0.2` and `num_iterations = 1100`. The previous plot should give you clues about how to improve the model.  What should you do? You can adjust either the learning rate `alpha` or the number of iterations `iters`.

Let's look at the case where `iters = 20` and ask whether a different learning rate would be better in terms of minimizing $J(\theta)$ within the first 20 iterations. To look at a several learning rates at once, we can define a list

~~~python
alphas = [ ---, ---, ---, ---, ---]
iters = 20
~~~

to compute and plot different learning rates at once.

In the next cell, supply a range of learning rates, to plot and compare with respect to a fixed number of iterations. Note that in this plot we only look at the first 20 iterations.

Run the next cell, then answer the Question 2.

```python
alphas = np.arange(0,1,0.01)
iters = 20
J = [] # For getting min value of J history
alpha_list = [] # For gettting alpha values

for aa in alphas:
    theta_new, J_history = batchGradientDescent(X, y, theta, aa, iters)
    #if np.min(J_history) <j:
    a = np.min(J_history)
    J.append(a)
    alpha_list.append(aa)
    plt.plot(J_history, label='learning rate={0}'.format(aa) )

print(alpha_list[np.argmin(J)])
```

---
## Question 2

What is the learning rate `alpha` that yields the lowest score $J(\theta)$ after exactly 20 iterations? Limit your search to values of `alpha` to two decimal places.  For example, if `alpha = 0.07` is your answer, you would finish function `ans_two()` by writing

~~~python
num = 0.07
~~~

Enter your answer below.

---

```python
def ans_two():
    """ Returns a list of your answers.

    Parameters
    ----------
    :ans:  list
        The list of your answers. Elements of :ans: are strings.
    Returns
    -------
    :ans:
    """

    num = 0.84
    #raise NotImplementedError()
    return num
```

```python
num = ans_two()
if type(num) == float and num == round(num,2):
    assert True
else:
    raise AssertionError("Check that your answer is a float and rounded to two decimal places")
```

```python

```

```python

```

---
## Question 3

Which are true? Mark all and only that apply.

Suppose that $J(\boldsymbol{\theta})$ is a continuously differentiable and convex function and $\theta$ is in $\mathbb{R}^2$.

  * A) Gradient Descent is guaranteed to converge to a local minima of h for all learning rates α.
  * B) Gradient Descent is guaranteed to converge to a local minima of h for some learning rates α
  * C) There is no learning rate α such that Gradient Descent is guaranteed to converge to a local minima of h
  *  D) The gradient of h is zero at the minimum.
  * E) The gradient of J points in the direction in $\mathbb{R}^2$ of the steepest increase in the value of J
---

```python
def ans_three():
    """ Returns a list of your answers.

    Parameters
    ----------
    :ans:  list
        The list of your answers. Elements of :ans: are strings.
    Returns
    -------
    :ans:
    """

    # YOUR CODE HERE
    ans = ['B','D','E']
    #raise NotImplementedError()
    return ans
```

```python
# test cell for ans_three()
ans = ans_three()
possible_ans = ['A', 'B', 'C', 'D', 'E']
if type(ans) == list and any(ii in ans for ii in possible_ans):
    assert True
else:
    raise AssertionError("Inadmissible answer. Check that your answer to ans_one is the correct format.")
```

---
## Question 4

Instead of Gradient Descent, which yields an approximate solution to OLS regression, we can instead use the Normal Equation to calculate an analytical solution.

Which of the following are true about the Normal Equation? Mark all and only that apply.

  * A) The number of features must be less than the number of training examples.
  * B) There is no learning rate hyperparameter to choose.
  * C) There is no need to iterate.
  * D) The speed of computing the normal equation slows exponentially as the number of features increases
  * E) The speed of computing the normal equation slows exponentially as the number of training examples increases

---

```python
def ans_four():
    """ Returns a list of your answers.

    Parameters
    ----------
    :ans:  list
        The list of your answers. Elements of :ans: are strings.
    Returns
    -------
    :ans:
    """

    # YOUR CODE HERE
    ans = ['B','C','D']
    #raise NotImplementedError()
    return ans
```

```python

```

# PART B - SciKit-Learn Linear Regression

Naturally, there are built-in libraries for regression models.  In fact, it just takes a few lines of code once we import `LinearRegression` from the `linear_model` methods in the `sklearn` (scikit-learn) library.

```python
# import sklearn linear regression
from sklearn.linear_model import LinearRegression
```

Sklearn uses a common [API](https://en.wikipedia.org/wiki/Application_programming_interface) for building models. So, the sequence we follow to build a simple linear model resembles the sequence of steps for building a wide range of models.  It is good, therefore, to become aquainted with this tool.

Basically, there are two steps:

 - Instantiate a model

 - Fit the model

Followed by plotting and some rudimentary anlysis of the model. Later in the course we will add a third step, which involves making predictions and evaluating the accuracy of those predictions.

```python
# instantiate model
model = LinearRegression()
# fit the model
model.fit(X, y)
```

    LinearRegression(copy_X=True, fit_intercept=True, n_jobs=1, normalize=False)

And that's it!

How does this method relate to what you did in PART A?  To retrieve the parameters, we apply the following methods to `model`:

 - `.intercept_`
 - `.coef_`

Note that the trailing underscore `_` is simply used to avoid naming conflicts in sklearn. You can interpret each as giving you the intercept ($\theta_0$) and the remaining coefficients ($\theta_1, \theta_2, \ldots$) in your model, respectively. Here (hopefully!) some of what you learned in PART A can be useful to understand what the next lines of code are doing.

Specifically, since sklearn linear regression can model multiple regression problems (i.e., problems with 2 or more features), the method `.coef_` returns an np.array rather than single number.  Hence, to retrieve the parameter value for $\theta_1$, we index by `[1]` into this array.

```python
#print theta0 and theta1
print('theta_0:', model.intercept_)
print('theta_1:', model.coef_[1])  # model.coef_ returns an array
```

    theta_0: 3.2255877706752263
    theta_1: 2.1649519939763175

Compare the values you computed here to those you computed above.

Next, we produce a plot.

```python
plt.scatter(x, y)
plt.title('2017 World Happiness by Country')
plt.xlabel(r'Per Capita GPD')
plt.ylabel(r'Happiness Score')
plt.plot(x, model.predict(X),color='r')
```

The coefficient of determination or "r-squared", $r^2$, is a measure of how well a linear regression model fits our data.  Put more carefully, $r^2$ measures how much of the variation in the data is explained by the (best fit) model $h(x; \theta)$:

$$ r^2 := 1- \frac{SE \, \hat{y}}{SE \, \overline{y}} $$

where $SE \,\hat{y}$ is the squared error of the estimator $\hat{y} = h(x; \theta)$, otherwise known as the residual sum of squares (RSS), and $SE \, \overline{y}$ is the total sum of squares, defined by

$$ SE \, \overline{y} = \sum_{i = 1}^{m}(y^{(i)} - \overline{y})$$

where $\overline{y}$ is the mean of the $m$ observed $y^{(i)}$'s.

Although we are working with squared residuals, notice that we are asking a different question.  Until now, we have used squared error as a performance measure to find the best fitting linear model to our data.  The coefficient of determination is used to evaluate a model selection question, not a model fitting question.  The coefficient of determination measures how much variation is explained by your linear model, ranging between 0 (none) to 1 (all).

The method `.score(X,y)` returns the $r^2$ of our model.  Note that we need `X`, which includes the column of intercept terms, rather than merely `x`.

```python
r_sq = model.score(X, y)
print('coefficient of determination:', r_sq)
```

    coefficient of determination: 0.6487826791386584

Lastly, the [seaborn](https://seaborn.pydata.org/) statistical visualization library is an extension of matplotlib that includes several built-in features that allow you to create appealing and informative graphics.  For instance, we might like to visualize the confidence interval around our model.  The next two lines of code produce a (default) 95% confidence interval, that is, that interval $[L, U]$ is calculated by

$$L = \theta_1 - 1.96(SE\,\theta_1) \qquad U = \theta_1 - 1.96(SE\,\theta_1) $$

```python
import seaborn as sns
sns.set_style("ticks")
sns.regplot(x,y,line_kws = {"color": 'r'})
```

# PART C - Your Turn

Let's review. In Part A you focused on the mechanics of gradient descent, how the mathematical components of ordinary least squares regression (OLS) regression can be represented and computed by NumPy element-wise array operations, and how those computational components fit together into an implementation of Batch Gradient Descent.  You then ran your implementation of gradient descent to find approximate solutions for the parameter of coefficients, $\boldsymbol{\theta}$, and recorded a step-wise history of values of the cost function, showing the progress of the algorithm in finding values of $\boldsymbol{\theta}$ taht minimize $J(\boldsymbol{\theta})$.  In Part B you used the SciKit-Learn library to confirm the parameter values, were introduced to the "logic" of the sklearn API (which you will see more of, later in the course), were introduced to a method -- $r^2$ to evaluate the model, and a brief introduction to the statistical plotting library seaborn

In this section, you will do some of the data preparation work that was done for you to fit a regression model on life expectancy.  The data set `ps1_data2.csv` contains the two columns of data in `ps1_data1.csv` plus a third  column recording the average life expectancy, in years, of each country.

 Note:
The average life expectancy data is scaled to values between 0 and 1. For example, a country whose average lifespan is 77.45 years will appear as 0.7745.

Your first task is to run the next cell to load `ps1_data2.csv`.  Then, on your own, you will need to prepare the data set to perform a univariate linear regression model to the explain the target variable happiness from the feature  average lifespan.  That is, $x$ will be the life expectancy feature, and $y$ will be the happiness score (as before).

```python
data2 = np.loadtxt('ps1_data2.csv', delimiter=',')
#data2
```

Once `data2` is in the right format to use the code you have written in Part A and the libraries introduced to you in Part B, use those tools (and those tools only) to answer each part of Question 5.

```python
# A free code cell

X = [(1,data2[i][1]) for i in range(len(data2))]

y = [data2[i][2] for i in range(len(data2))]

# instantiate model
# import sklearn linear regression
from sklearn.linear_model import LinearRegression
model = LinearRegression()
# fit the model
model.fit(X, y)

#print theta0 and theta1
print('theta_0:', model.intercept_)
print('theta_1:', model.coef_[1])  # model.coef_ returns an array

r_sq = model.score(X, y)
print('coefficient of determination:', r_sq)
```

    theta_0: 3.291682356622725
    theta_1: 3.7374824724355054
    coefficient of determination: 0.5997037800245257

```python
# Becaue data2 coontains two variables x2 is taken
x = [data2[i][1] for i in range(len(data2))]
plt.scatter(x, y)
plt.title('2017 World Happiness by Country')
plt.xlabel(r'Average Lifespan')
plt.ylabel(r'Happiness Score')
plt.plot(x, model.predict(X),color='g')
```

---
## Question 5

Run Gradient descent and sklearn linear regression on `data2`.  Be sure to pick new variables names to ensure that all variables correspond to data set `ps_data2` and not to `ps_data1`.

There are four quantities that you will need to report about your OLS univariate regression model explaining happiness scores as a linear function of average life expectancy, which we will refer to as "model 2":

 - a) The value of the intercept parameter $\theta_0$ of the fitted univariate regression model 2.
 - b) The value of the coefficient parameter $\theta_1$ of the fitted univariate regression model 2.
 - c) The value of the cost function, $J([\theta_0, \theta_1])$, on the fitted parameters for model 2.. Note that this is the last value stored in the J_history cost function.
 - d) The coefficient of determination score of model 2.

The test condition for a, b, c, rounds the correct answers to 5 decimal places, using the python function `round(--your answer --, 5)`; the test condition for answer 3 rounds to 3 decimal places.

In the following cell, enter the values for for those variables.  Here is the key:

~~~python
def ans_5():
    a =     # a: theta_0 from your fitted model, rounded to 5 decimal places
    b =     # b: theta_1 from your fitted model, rounded to 5 decimal places
    c =     # c: the last value of the array J_history, rounded to 5 decimal places
    d =     # d: r-squared value, rounded to 3 decimal places
~~~

 Hint:
If you use tools from Part A and Part B, it is important that they return the same answers (up to 5 decimal places) to those parts of Question 5 that they both can answer. Now the hint: You may want to aim for slightly higher precision for some quantities to ensure the test conditions for all four quantities are satisfied.

---

```python
def ans_five():
    """ Returns four numerical values of your linear model.

    Parameters
    ----------
    :a:  float
        The value of theta_0 from your fitted model, rounded to 5 decimal places.
    :b:  float
        The value of theta_1 from your fitted model, rounded to 5 decimal places
    :c:  float
        The last value of the array J_history, rounded to 5 decimal places
    :d:  float
        The calculated r-squared of your fitted model, rounded to 3 decimal places

    Returns
    -------
    :a:
    :b:
    :c:
    :d:
    """
    # YOUR CODE HERE
    a = model.intercept_
    b = model.coef_[1]
    c = np.round(batchGradientDescent(X, y, theta, alpha, iterations)[1][-1],5)
    d = model.score(X, y)
    #raise NotImplementedError()
    return a, b, c, d  # do not edit

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python
# Test for Collaborator policy before submission
%run -i 'collaboration_test.py'
```

    WARNING. Your collaborator list is empty: COLLABORATORS =  []

     	 By submitting this notebook with an empty collaborators list,
    	 you confirm that you completed this assignment entirely on your own.

Before turning this notebook in, you should do the following steps:

  1. __Restart Kernel__ (Kernel ⟶ Restart and Clear Output)
  2. __Run all Cells__ (Cell ⟶ Run All)
  3. __Validate__: Press the 'Validate' button
  4. __Save File__ (File ⟶ Save and Checkpoint)
  5. __Close and Shutdown Kernel__ (File ⟶ Close and Halt)


# *Thermal Transport in a Cylinder*

In this demo, we solve the three-dimensional diffusion equation in a short
cylinder, with Dirichlet boundary conditions $u_D$ and steady source term
$f$ along the central axis.

$$
\begin{align}
  u'   &= \nabla^2 u + f     \quad\text{in a short cylinder} \\
  u    &= u_D  \hphantom{u+f}\quad\text{on the boundary} \\
  u    &= u_0  \hphantom{u+f}\quad\;\text{at $t = 0$}
\end{align}
$$
with$$
\begin{align}
  u_D  &= 0 \\
  u_0  &= 0 \\
  f    &= A \exp\bigl( -(r/\rho)^2 - (z/\zeta)^2 \bigr).
\end{align}
$$

```python
import numpy as np
from fenics import *
from mshr import *   # now deprecated (?), transition to alternate mesh generator (?)
```

```python
## set up the problem and define (some of) the main parameters

# simulation parameters
# -- time step
T = 5.0           # total simulation time
n_steps = 100     # number of time steps
dt = T / n_steps  # size of time step
nip = 10          # number of intervals between plots
# -- cylinder
rad = 1.0         # radius
lh  = 0.1         # half-length
# --  mesh density
md = 24           # mesh density within cylinder

# create mesh: Cylinder(p_top, p_bot, r_top, r_bot)
domain = Cylinder(Point(0, 0, lh), Point(0., 0., -lh), rad, rad)
mesh = generate_mesh(domain, md)
# and define function space
V = FunctionSpace(mesh, 'P', 1)
```

```python
mesh
```

```python
# define boundary condition
def boundary(x, on_boundary):
    return on_boundary

bc = DirichletBC(V, Constant(0), boundary)

# define initial value
u_0 = Constant(0)
u_n = interpolate(u_0, V)

# define source term
A = 2.0          # amplitude
rho =  rad / 10  # radial decay length
zeta = lh / 2    # longitudinal decay length
f = Expression('A * exp( -(pow(x[0], 2) + pow(x[1], 2)) / pow(rho, 2) - pow(x[2] / zeta, 2) )',
               degree=1, A=A, rho=rho, zeta=zeta)

# define the variational problem
u = TrialFunction(V)
v = TestFunction(V)
F = u*v*dx + dt*dot(grad(u), grad(v))*dx - (u_n + dt*f)*v*dx
a, L = lhs(F), rhs(F)
```

```python
# define time-evolution function
def evolve():

    # report initial state
    yield u_n

    # time-stepping
    u = Function(V)
    t = 0
    for n in range(1, n_steps + 1):

        # update current time
        t += dt

        # compute solution
        solve(a == L, u, bc)

        # report solution at nip-step intervals
        if n % nip == 0:
            yield u

        # update previous solution
        u_n.assign(u)
```

```python
n_rows = 3
n_cols = 5
fig_wd = 15
# default sizing here yields unit aspect ratio
# plt.figure(figsize = (fig_wd, fig_wd * n_rows // n_cols))

idx = 0
for u in evolve():
    idx += 1
#     plot(u)
```

```python
xvals = mesh.coordinates()[:,0]
yvals = mesh.coordinates()[:,1]
zvals = mesh.coordinates()[:,2]
uvals = u.compute_vertex_values()
```

```python
import plotly.graph_objects

inds = []
# get the "facets" and build an array of the indices of their coordinates
for item in dolfin.cpp.mesh.facets(mesh):
    inds.append(item.entities(0).tolist())

# provide these indices to plotly so it can draw proper surfaces
inds = np.array(inds)
ii = inds[:, 0]
jj = inds[:, 1]
kk = inds[:, 2]

fig = plotly.graph_objects.Figure(
    data=[plotly.graph_objects.Mesh3d(
        x=xvals, y=yvals, z=zvals,
        i=ii, j=jj, k=kk,
        colorscale='jet',
        intensity=uvals,
        alphahull=-1,
        opacity=0.1
    )]
)
fig.update_layout(scene_aspectmode='data')
fig.show()
```


## Finite Elements

The finite element method (FEM) or analysis (FEA) is a *numerical method* for solving many engineering domain problems. It's areas include structural analysis, heat transfer and fluid flow around an object.

The method yields approximate values of the unknowns using a discrete number of points over the domain of the object in question. To solve the problem, it integrates by parts, dividing a large problem into smaller, simpler objects that are called *Finite Elements*. The simple equations that model the finite elements are then assembled into larger systems of equations to model the entire problem.

### 1D problem

The temperature $T$ of a bar of length 1 at equilibrium satisfies:

$$
\partial_{xx}T + f(x)=0
$$

The heat source is given by $f(x)$. We'll hold the temperature fixed at the right edge by setting $T(1)=0$. We'll allow heat to conduct through the left edge by setting $\partial_xT(0)=0$.

We need a way of handling the second-derivative term, so we can introduce a weighting function $w(x)$ and integrate by parts to get:

$$
  \left[ \partial_x T(x) w(x) \right]_0^1 - \int_0^1 \text{d}x \, \partial_x T(x) \partial_x w(x) = - \int_0^1 \text{d}x \, f(x) w(x).
$$

and after factoring the boundary conditions we get:

$$
\int_0^1 \text{d}x\partial_xT(x)\partial_xw(x)=\int_0^1 \text{d}xw(x)f(x)
$$

This is considered the **weak form** of the problem.

### Step 1. A simple 3-node example

To get an approximate solution, we split the domain into intervals/elements. Let's begin with a simple example where we choose $[0, 1/2]$ and $[1/2, 1]$, which means there are nodes at $\{0, 1/2, 1\}$. To repesent this, we'll introduce some *shape functions* $N_A(x)$ and write:

$$
T(x)=\sum_A T_AN_A(x)
$$

here $A$ is a counter associated with the nodes, so $A=\{0,1,2\}$: the nodes are $x_A$. The shape function $N_A(x)=1$ at the associated node, and $0$ at all other nodes. In this case, let's choose the shape functions to be piecewise linear, which are shape functions look as:

$$
\begin{align}
  N_0(x) &= \begin{cases} 1 - 2 x, & 0 \le x \le 1/2 \\ 0, & 1/2 \le x \le 1, \end{cases}, \\
  N_1(x) &= \begin{cases} 2 x, & 0 \le x \le 1/2 \\ 2 - 2 x, & 1/2 \le x \le 1, \end{cases}, \\
  N_2(x) &= \begin{cases} 0, & 0 \le x \le 1/2 \\ 2 x - 1, & 1/2 \le x \le 1. \end{cases}
\end{align}
$$

```python
import matplotlib.pyplot as plt
%matplotlib inline

x = np.linspace(0.,1.,1000)
# we use np.where to select x <= 0.5, then replace true with top function, false with bottom function
N0 = np.where(x <= .5, 1-2*x, np.zeros_like(x))
N1 = np.where(x <= .5, 2*x, 2-2*x)
N2 = np.where(x <= .5, np.zeros_like(x), 2*x-1)
plt.plot(x, N0, label='$N_0$')
plt.plot(x, N1, label='$N_1$')
plt.plot(x, N2, label='$N_2$')
plt.legend()
plt.show()
```

We can now write out the weak form of this problem as:

$$
  \sum_A \sum_B T_A w_B \int_0^1 \text{d}x \, \partial_x N_A(x) \partial_x N_B(x) = \sum_B w_B \int_0^1 \text{d}x \, N_A(x) f(x).
$$

This has to be true for *any* choice of weight function, so for *any* coefficients $w_B$. This is true only if

$$
  \sum_A T_A \int_0^1 \text{d}x \, \partial_x N_A(x) \partial_x N_B(x) = \int_0^1 \text{d}x \, N_B(x) f(x).
$$

which can be written as a linear system matrix problem:

$$
KT=F
$$

The coefficients of the **stiffness matrix** $K$ are given as:

$$
  K_{AB} = \int_0^1 \text{d}x \, \partial_x N_A(x) \partial_x N_B(x).
$$

The coefficients of the **force vector** $F$ are given by

$$
  F_{A} = \int_0^1 \text{d}x \, N_A(x) f(x).
$$

In order to calculate the **stiffness** matrix and **force** vectors, we need to calculate the derivatives of the shape functions:

$$
\begin{align}
  \partial_x N_0(x) &= \begin{cases} - 2, & 0 \le x \le 1/2 \\ 0, & 1/2 \le x \le 1, \end{cases}, \\
  \partial_x N_1(x) &= \begin{cases} 2, & 0 \le x \le 1/2 \\ - 2, & 1/2 \le x \le 1, \end{cases}, \\
  \partial_x N_2(x) &= \begin{cases} 0, & 0 \le x \le 1/2 \\ 2, & 1/2 \le x \le 1. \end{cases}
\end{align}
$$

Therefore the product $\partial_xN_A\partial_xN_B$ required for calculating the stiffness matrix is either $0$ or $\pm 4$:

$$
\begin{align}
  \partial_x N_0(x) \partial_x N_0(x) &= \begin{cases} 4, & 0 \le x \le 1/2 \\ 0, & 1/2 \le x \le 1, \end{cases}, \\
  \partial_x N_0(x) \partial_x N_1(x) &= \begin{cases} -4, & 0 \le x \le 1/2 \\ 0, & 1/2 \le x \le 1, \end{cases}, \\
  \partial_x N_1(x) \partial_x N_0(x) &= \begin{cases} -4, & 0 \le x \le 1/2 \\ 0, & 1/2 \le x \le 1, \end{cases}, \\
  \partial_x N_1(x) \partial_x N_1(x) &= \begin{cases} 0, & 0 \le x \le 1/2 \\ 4, & 1/2 \le x \le 1. \end{cases}
\end{align}
$$

This gives:

$$
K_{AB}= \begin{pmatrix} 2 & -2 \\ -2 & 4 \end{pmatrix}
$$

Meanwhile, the force vector is

$$
\begin{align}
  F_0 &= \int_0^{1/2} \text{d}x \, x - 2 x^2 \\ &= \frac{1}{8} - \frac{1}{12} = \frac{1}{24}, \\
  F_1 &= \int_0^{1/2} \text{d}x \, 2 x^2 + \int_{1/2}^1 \text{d}x \, 2x - 2 x^2 \\ &= \frac{1}{12} + 1 - \frac{2}{3} - \frac{1}{4} + \frac{1}{12} = \frac{1}{4}.
\end{align}
$$

### Step 2.

We will demonstrate this by computing analytically the **stiffness** matrix and **force vector** using *sympy*:

```python
import sympy as smp
smp.init_printing()
```

```python
x = smp.symbols("x")
N = [
    smp.Piecewise((1-2*x, x <=1/2), (0, x > 1/2)),
    smp.Piecewise((2*x, x<=1/2), (2-2*x, x>1/2)),
    smp.Piecewise((0, x<1/2), (2*x-1, x>1/2))
]
K = np.zeros((2,2))
for A in range(2):
    for B in range(2):
        K[A,B] = smp.integrate(smp.diff(N[A], x)*smp.diff(N[B], x), (x, 0 , 1))
print(K)
```

    [[ 2. -2.]
     [-2.  4.]]

And now for the force vector:

```python
f = lambda x : x
F = np.zeros(2)
for A in range(2):
    F[A] = smp.integrate(N[A] * f(x), (x, 0, 1))
print(F)
```

    [0.04166667 0.25      ]

And finally we will **numerically** solve $KT=F$.

```python
T_A = np.linalg.solve(K,F)

N_num = []
for A, N_A in enumerate(N):
    N_num.append(smp.lambdify((x,), N[A], "numpy"))

def T(x):
    Tvals = np.zeros_like(x)
    for A in range(2):
        Tvals += T_A[A] * N_num[A](x)
    return Tvals
```

```python
coords = np.linspace(0,1,1000)
plt.plot(coords, T(coords), label="Numerical")
plt.plot(coords, (1-coords**3)/6, label="Exact")
plt.legend()
plt.show()
```

```python

```


### Nomenclature:
$d$ means a derivative ALONG the saturation line,
$\partial$ means a partial derivative AT the saturation line (or anywhere in the single phase region).

### References:
Krafcik and Velasco, DOI 10.1119/1.4858403
Thorade and Saadat, DOI 10.1007/s12665-013-2394-z
https://en.wikipedia.org/wiki/Product_rule
https://en.wikipedia.org/wiki/Quotient_rule
https://en.wikipedia.org/wiki/Triple_product_rule

### Clausius-Clapeyron
Clausius-Clapeyron p/T

\begin{equation}
\frac{dp}{dT} = \frac{s''-s'}{v''-v'}
\end{equation}

derivative of Clausius-Clapeyron
\begin{equation}
\begin{split}
\frac{d^2 p}{d T^2} &=
    \frac{\left(\frac{ds''}{dT}-\frac{ds'}{dT}\right) (v''-v')}{(v''-v')^2}
  - \frac{(s''-s') \left(\frac{dv''}{dT}-\frac{dv'}{dT}\right)}{(v''-v')^2} \\
 &= \frac{\left(\frac{ds''}{dT}-\frac{ds'}{dT}\right)}{(v''-v')}
  - \frac{dp}{dT} \frac{\left(\frac{dv''}{dT}-\frac{dv'}{dT}\right)}{(v''-v')}
\end{split}
\end{equation}

Clausius-Clapeyron T/p
\begin{equation}
\frac{dT}{dp} = \frac{v''-v'}{s''-s'}
\end{equation}

derivative of Clausius-Clapeyron
\begin{equation}
\begin{split}
\frac{d^2 T}{d p^2} &=
    \frac{\left(\frac{dv''}{dp}-\frac{dv'}{dp}\right) (s''-s')}{(s''-s')^2}
  - \frac{(v''-v') \left(\frac{ds''}{dp}-\frac{ds'}{dp}\right)}{(s''-s')^2} \\
 &= \frac{\left(\frac{dv''}{dp}-\frac{dv'}{dp}\right)}{(s''-s')}
  - \frac{dT}{dp} \frac{\left(\frac{ds''}{dp}-\frac{ds'}{dp}\right)}{(s''-s')}
\end{split}
\end{equation}

where
\begin{equation}
\begin{split}
\frac{ds}{dT} &= \left(\frac{\partial s}{\partial T}\right)_p  + \left(\frac{\partial s}{\partial p}\right)_T \frac{dp}{dT}\\
\frac{dv}{dT} &= \left(\frac{\partial v}{\partial T}\right)_p  + \left(\frac{\partial v}{\partial p}\right)_T \frac{dp}{dT}\\
\frac{dv}{dp} &= \left(\frac{\partial v}{\partial p}\right)_T  + \left(\frac{\partial v}{\partial T}\right)_p \frac{dT}{dp}\\
\frac{ds}{dp} &= \left(\frac{\partial s}{\partial p}\right)_T  + \left(\frac{\partial s}{\partial T}\right)_p \frac{dT}{dp}
\end{split}
\end{equation}

### Temporary Names

Introduce temporary names for some of the partial derivatives wrt $p$ and $T$:
\begin{equation}
\begin{split}
A &= \left(\frac{\partial \rho}{\partial T}\right)_p \\
B &= \left(\frac{\partial \rho}{\partial p}\right)_T \\
C &= \left(\frac{\partial s}{\partial T}\right)_p \\
E &= \left(\frac{\partial s}{\partial p}\right)_T \\
G &= \left(\frac{\partial h}{\partial T}\right)_p \\
K &= \left(\frac{\partial h}{\partial p}\right)_T
\end{split}
\end{equation}

Introduce temporary names for some of the partial derivatives wrt $\rho$ and $T$:
\begin{equation}
\begin{split}
W &= \left(\frac{\partial p}{\partial T}\right)_{\rho} \\
X &= \left(\frac{\partial p}{\partial \rho}\right)_T \\
Y &= \left(\frac{\partial s}{\partial T}\right)_{\rho} \\
Z &= \left(\frac{\partial s}{\partial \rho}\right)_T
\end{split}
\end{equation}

Introduce temporary names for some of the derivatives along the saturation line:
\begin{equation}
\begin{split}
M &= \frac{d \rho}{d h} = \frac{{d \rho}/{dT}}{{dh}/{dT}} \\
N &= \frac{d s}{d h} = \frac{{ds}/{dT}}{{dh}/{dT}}
\end{split}
\end{equation}

Now the rest is not too hard, but with intermediate steps it is quite long.

### First example: $d^2 \rho / dT^2$
The corresponding first derivative can be written in two ways:
\begin{equation}
\begin{split}
\frac{d \rho}{dT}
  &= \left(\frac{\partial \rho}{\partial T}\right)_p + \left(\frac{\partial \rho}{\partial p}\right)_T \frac{d p}{dT}
   = A + B \frac{dp}{dT}\\
  &= {\frac{dp}{dT} \left(\frac{\partial p}{\partial T}\right)_{\rho}} / {\left(\frac{\partial p}{\partial \rho}\right)_T}
   = {\frac{dp}{dT} W}/X
\end{split}
\end{equation}
Both ways can be used as a starting point for the second derivative:
\begin{equation}
\begin{split}
\frac{d^2 \rho}{dT^2}
  &= \frac{dA}{dT} + \frac{dB}{dT}\frac{dp}{dT} + B\frac{d^2p}{dT^2}\\
  &= \frac{\left(\frac{d^2p}{dT^2} - \frac{dW}{dT}\right)X - \left(\frac{dp}{dT}-W\right)\frac{dX}{dT}}{X^2}
\end{split}
\end{equation}
The first variant seems to be nicer, because the first deriv did not have a fraction in it.
The derivatives along the saturation line that appeared can be written as (again, using two different approaches):
\begin{equation}
\begin{split}
\frac{dA}{dT} &= \left(\frac{\partial A}{\partial T}\right)_p + \left(\frac{\partial A}{\partial p}\right)_T \frac{d p}{dT}\\
\frac{dB}{dT} &= \left(\frac{\partial B}{\partial T}\right)_p + \left(\frac{\partial B}{\partial p}\right)_T \frac{d p}{dT}\\
\frac{dW}{dT} &= \left(\frac{\partial W}{\partial T}\right)_{\rho} + \left(\frac{\partial W}{\partial \rho}\right)_T \frac{d\rho}{dT}\\
\frac{dX}{dT} &= \left(\frac{\partial X}{\partial T}\right)_{\rho} + \left(\frac{\partial X}{\partial \rho}\right)_T \frac{d\rho}{dT}\\
\end{split}
\end{equation}
and then the partial derivatives of A, B, W and X wrt p and T can be written as:
\begin{equation}
\begin{split}
\left(\frac{\partial A}{\partial T}\right)_p &= \left(\frac{\partial^2 \rho}{\partial T^2}\right)_p \\
\left(\frac{\partial A}{\partial p}\right)_T &= \left(\frac{\partial^2 \rho}{\partial p \partial T}\right) \\
\left(\frac{\partial B}{\partial T}\right)_p &= \left(\frac{\partial^2 \rho}{\partial T \partial p}\right) = \left(\frac{\partial A}{\partial p}\right)_T \\
\left(\frac{\partial B}{\partial p}\right)_T &= \left(\frac{\partial^2 \rho}{\partial p^2}\right)_T \\
\left(\frac{\partial W}{\partial T}\right)_{\rho} &= \left(\frac{\partial^2 p}{\partial T^2}\right)_{\rho} \\
\left(\frac{\partial W}{\partial \rho}\right)_T &= \left(\frac{\partial^2 p}{\partial \rho \partial T}\right) \\
\left(\frac{\partial X}{\partial T}\right)_{\rho} &= \left(\frac{\partial^2 p}{\partial T \partial \rho}\right) = \left(\frac{\partial W}{\partial \rho}\right)_T \\
\left(\frac{\partial X}{\partial \rho}\right)_T &= \left(\frac{\partial^2 p}{\partial {\rho}^2}\right)_T
\end{split}
\end{equation}
So this time, the second way looks maybe nicer, because it uses partial derivatives wrt to density and temperature. On the other hand, all second partial derivatives are already implemented in Coolprop and the pT derivatives are internally rewritten as dT derivatives.

### Second example: $d^2s/dT^2$
The corresponding first derivative can be written in two ways:
\begin{equation}
\begin{split}
\frac{ds}{dT}
  &= \left(\frac{\partial s}{\partial T}\right)_p + \left(\frac{\partial s}{\partial p}\right)_T \frac{dp}{dT}
   = C + E\frac{dp}{dT}\\
  &= \left(\frac{\partial s}{\partial T}\right)_{\rho} + \left(\frac{\partial s}{\partial \rho}\right)_T \frac{d \rho}{dT}
   = Y + Z\frac{d \rho}{dT}
\end{split}
\end{equation}
Both can be used as starting point for the second derivatives.
\begin{split}
\frac{d^2 s}{dT^2}
  &= \frac{dC}{dT} + \frac{dE}{dT}\frac{dp}{dT} + E\frac{d^2p}{dT^2}\\
  &= \frac{dY}{dT} + \frac{dZ}{dT}\frac{dp}{dT} + Z\frac{d^2 \rho}{dT^2}
\end{split}
Now, which one is nicer to work with? Not sure.
Just as shown above, the derivatives along the saturation line that appeared can be written as (again, using two different approaches):
\begin{equation}
\begin{split}
\frac{dC}{dT} &= \left(\frac{\partial C}{\partial T}\right)_p + \left(\frac{\partial C}{\partial p}\right)_T \frac{d p}{dT}\\
\frac{E}{dT} &= \left(\frac{\partial E}{\partial T}\right)_p + \left(\frac{\partial E}{\partial p}\right)_T \frac{d p}{dT}\\
\frac{dY}{dT} &= \left(\frac{\partial Y}{\partial T}\right)_{\rho} + \left(\frac{\partial Y}{\partial \rho}\right)_T \frac{d\rho}{dT}\\
\frac{dZ}{dT} &= \left(\frac{\partial Z}{\partial T}\right)_{\rho} + \left(\frac{\partial Z}{\partial \rho}\right)_T \frac{d\rho}{dT}\\
\end{split}
\end{equation}
Rewriting the the partial derivatives works just like shown above.

### Third example: $d^2{\rho}/dp^2$
The corresponding first derivative can be written in many ways, here is one:
\begin{equation}
\frac{d \rho}{dp}
  = \left(\frac{\partial \rho}{\partial p}\right)_T + \left(\frac{\partial \rho}{\partial T}\right)_p \frac{dT}{dp}
  = B + A\frac{dT}{dp}
\end{equation}

The second derivative is then:
\begin{equation}
\frac{d^2 \rho}{dp^2}
  = \frac{dB}{dp} + \frac{dA}{dp}\frac{dT}{dp} + A\frac{d^2T}{dp^2}
\end{equation}
The derivatives wrt p along the saturation line can be written as:
\begin{equation}
\begin{split}
\frac{dB}{dp} &= \left(\frac{\partial B}{\partial p}\right)_T + \left(\frac{\partial B}{\partial T}\right)_p \frac{d T}{dp}\\
\frac{dA}{dp} &= \left(\frac{\partial A}{\partial p}\right)_T + \left(\frac{\partial A}{\partial T}\right)_p \frac{d T}{dp}
\end{split}
\end{equation}
Rewriting the partial derivatives works just like shown above.

### Fourth example: $d^2s/dp^2$
The corresponding first derivative can be written in many ways, here is one:
\begin{equation}
\frac{ds}{dp}
  = \left(\frac{\partial s}{\partial p}\right)_T + \left(\frac{\partial s}{\partial T}\right)_p \frac{dT}{dp}
  = E + C\frac{dT}{dp}
\end{equation}

The second derivative is then:
\begin{equation}
\frac{d^2 s}{dp^2}
  = \frac{dE}{dp} + \frac{dC}{dp}\frac{dT}{dp} + C\frac{d^2T}{dp^2}
\end{equation}
Rewriting the derivatives along the sat line and the partial derivatives works just like shown above.

### Fifth example: $d^2h/dT^2$
First:
\begin{equation}
\frac{dh}{dT}
  = \left(\frac{\partial h}{\partial T}\right)_p + \left(\frac{\partial h}{\partial p}\right)_T \frac{dp}{dT}
  = G + K\frac{dp}{dT}
\end{equation}

Second:
\begin{equation}
\frac{d^2 h}{dT^2}
  = \frac{dG}{dT} + \frac{dK}{dT}\frac{dp}{dT} + K\frac{d^2p}{dT^2}
\end{equation}

### Sixth example: $d^2{\rho}/dh^2$
This is a bit different, but also not difficult.
First:
\begin{equation}
  M = \frac{d\rho}{dh} = \frac{{d\rho}/{dT}}{{dh}/{dT}}
\end{equation}

Intermediate:
\begin{equation}
\frac{dM}{dT}
  = \frac{\frac{d^2 \rho}{dT^2}\frac{dh}{dT}-\frac{d \rho}{dT}\frac{d^2h}{dT^2}}{\left(\frac{dh}{dT}\right)^2}
\end{equation}

Second:
\begin{equation}
\frac{d^2 \rho}{dh^2}
  = \frac{dM}{dh}
  = \frac{{dM}/{dT}}{{dh}/{dT}}
  = \frac{\frac{d^2 \rho}{dT^2}\frac{dh}{dT}-\frac{d \rho}{dT}\frac{d^2h}{dT^2}}{\left(\frac{dh}{dT}\right)^3}
\end{equation}

### Seventh example: $d^2s/dh^2$
First:
\begin{equation}
  N = \frac{ds}{dh} = \frac{{ds}/{dT}}{{dh}/{dT}}
\end{equation}

Intermediate:
\begin{equation}
\frac{dN}{dT}
  = \frac{\frac{d^2s}{dT^2}\frac{dh}{dT}-\frac{ds}{dT}\frac{d^2h}{dT^2}}{\left(\frac{dh}{dT}\right)^2}
\end{equation}

Second:
\begin{equation}
\frac{d^2s}{dh^2}
  = \frac{dN}{dh}
  = \frac{{dN}/{dT}}{{dh}/{dT}}
  = \frac{\frac{d^2s}{dT^2}\frac{dh}{dT}-\frac{ds}{dT}\frac{d^2h}{dT^2}}{\left(\frac{dh}{dT}\right)^3}
\end{equation}

### Eigth example: $d^2s/(dh dp)$
First:
\begin{equation}
  N = \frac{ds}{dh} = \frac{{ds}/{dT}}{{dh}/{dT}}
\end{equation}

Intermediate:
\begin{equation}
\frac{dN}{dT}
  = \frac{\frac{d^2s}{dT^2}\frac{dh}{dT}-\frac{ds}{dT}\frac{d^2h}{dT^2}}{\left(\frac{dh}{dT}\right)^2}
\end{equation}

Second:
\begin{equation}
\frac{d^2s}{dh dp}
  = \frac{dN}{dp}
  = \frac{{dN}/{dT}}{{dp}/{dT}}
  = \frac{\frac{d^2s}{dT^2}\frac{dh}{dT}-\frac{ds}{dT}\frac{d^2h}{dT^2}}{\left(\frac{dh}{dT}\right)^2 \left(\frac{dp}{dT}\right)}
\end{equation}

```python
# load some bits and pieces
import sys
import numpy as np
import matplotlib
import matplotlib.pyplot as plt

import CoolProp as CP
from CoolProp.CoolProp import PropsSI

# Print the versions
print("      python: {}".format(sys.version))
print("       numpy: {}".format(np.__version__))
print("  matplotlib: {}".format(matplotlib.__version__))
print("    CoolProp: {}".format(CP.__version__))
print("CoolProp git: {}".format(CP.__gitrevision__))

# Constants
eps = 1e-9
kilo = 1e3
Mega = 1e6
golden = (1 + 5 ** 0.5) / 2
nPoints = 1000
width = 12.5

# helper function: get slope from two sorted arrays
def numSlopeAr(xAr, yAr):
 deltaX = np.ones(len(xAr))
 deltaY = np.ones(len(yAr))
 slopeAr = np.ones(len(yAr))
 for index in range(1,len(xAr)-1):
     deltaX[index] = xAr[index-1] - xAr[index+1]
     deltaY[index] = yAr[index-1] - yAr[index+1]
     slopeAr[index] = deltaY[index]/deltaX[index]
 # inaccurate, but who cares?
 slopeAr[0]=slopeAr[1]
 slopeAr[-1]=slopeAr[-2]
 return slopeAr

# helper function: draw tangent from array
def drawTangent(xAr, yAr, slopeAr, index: int):
 xVal = xAr[index]
 yVal = yAr[index]
 slopeVal = slopeAr[index]
 # line length as percentage of length of x axis
 xRange = 0.15*abs(max(xAr)-min(xAr))
 xRangeAr = np.arange(xVal-xRange, xVal+xRange)
 tang = yVal + slopeVal*(xRangeAr-xVal)
 plt.plot(xVal,yVal,'om',xRangeAr,tang,'-k')
```

          python: 3.6.5 |Anaconda, Inc.| (default, Mar 29 2018, 13:32:41) [MSC v.1900 64 bit (AMD64)]
           numpy: 1.14.3
      matplotlib: 2.2.2
        CoolProp: 6.2.1
    CoolProp git: 81dbcb71c512b5d57cdfe197e59ae87f0221e4e6

```python
# All calculations happen in this cell
# All following cells are used for plotting only
# Run this cell twice to get rid of error message

# Set FluidName
FluidName = 'R1234yf'

# Triple and critical data
T_crt = PropsSI('TCRIT',FluidName)
T_trp = PropsSI('TTRIPLE',FluidName)
p_crt = PropsSI('PCRIT',FluidName)
p_trp = PropsSI('PTRIPLE',FluidName)
d_crt = PropsSI('RHOCRIT',FluidName)
d_trp_liq = PropsSI('D','T',T_trp,'Q',0,FluidName)
d_trp_vap  = PropsSI('D','T',T_trp,'Q',1,FluidName)

print(f"T_trp = {T_trp}")
print(f"T_crt = {T_crt}")
print(f"p_trp = {p_trp}")
print(f"p_crt = {p_crt}")

# Properties at saturation, liq and vap
# All the way to the crt point or keep some distance?
T_sat = np.linspace(T_trp, T_crt, num=nPoints)
p_sat = CP.CoolProp.PropsSI('P','T',T_sat,'Q',0,FluidName)
d_sat_liq = CP.CoolProp.PropsSI('D','T',T_sat,'Q',0,FluidName)
d_sat_vap = CP.CoolProp.PropsSI('D','T',T_sat,'Q',1,FluidName)
v_sat_liq = 1/d_sat_liq
v_sat_vap = 1/d_sat_vap
s_sat_liq = CP.CoolProp.PropsSI('S','T',T_sat,'Q',0,FluidName)
s_sat_vap = CP.CoolProp.PropsSI('S','T',T_sat,'Q',1,FluidName)
h_sat_liq = CP.CoolProp.PropsSI('H','T',T_sat,'Q',0,FluidName)
h_sat_vap = CP.CoolProp.PropsSI('H','T',T_sat,'Q',1,FluidName)
u_sat_liq = CP.CoolProp.PropsSI('U','T',T_sat,'Q',0,FluidName)
u_sat_vap = CP.CoolProp.PropsSI('U','T',T_sat,'Q',1,FluidName)

# Clausius-Clapeyron
# at crt point, vap=liq, so D becomes zero
Ds_vl = (s_sat_vap-s_sat_liq)
Dv_vl = (v_sat_vap-v_sat_liq)
dp_dT_q =  Ds_vl/Dv_vl
dT_dp_q = Dv_vl/Ds_vl # = 1/dp_dT_q
dp_dT_qn = numSlopeAr(T_sat,p_sat)
dT_dp_qn = numSlopeAr(p_sat,T_sat)

# derivs wrt pT, single-phase AT the sat liq line
dd_dp_Tl = CP.CoolProp.PropsSI('d(Dmass)/d(P)|T','D',d_sat_liq,'T',T_sat,FluidName)
dd_dT_pl = CP.CoolProp.PropsSI('d(Dmass)/d(T)|P','D',d_sat_liq,'T',T_sat,FluidName)
ds_dp_Tl = CP.CoolProp.PropsSI('d(Smass)/d(P)|T','D',d_sat_liq,'T',T_sat,FluidName)
ds_dT_pl = CP.CoolProp.PropsSI('d(Smass)/d(T)|P','D',d_sat_liq,'T',T_sat,FluidName)
dh_dp_Tl = CP.CoolProp.PropsSI('d(Hmass)/d(P)|T','D',d_sat_liq,'T',T_sat,FluidName)
dh_dT_pl = CP.CoolProp.PropsSI('d(Hmass)/d(T)|P','D',d_sat_liq,'T',T_sat,FluidName)
dv_dp_Tl = -dd_dp_Tl/d_sat_liq**2
dv_dT_pl = -dd_dT_pl/d_sat_liq**2

# derivs wrt pT, single-phase AT the sat vap line
dd_dp_Tv = CP.CoolProp.PropsSI('d(Dmass)/d(P)|T','D',d_sat_vap,'T',T_sat,FluidName)
dd_dT_pv = CP.CoolProp.PropsSI('d(Dmass)/d(T)|P','D',d_sat_vap,'T',T_sat,FluidName)
ds_dp_Tv = CP.CoolProp.PropsSI('d(Smass)/d(P)|T','D',d_sat_vap,'T',T_sat,FluidName)
ds_dT_pv = CP.CoolProp.PropsSI('d(Smass)/d(T)|P','D',d_sat_vap,'T',T_sat,FluidName)
dh_dp_Tv = CP.CoolProp.PropsSI('d(Hmass)/d(P)|T','D',d_sat_vap,'T',T_sat,FluidName)
dh_dT_pv = CP.CoolProp.PropsSI('d(Hmass)/d(T)|P','D',d_sat_vap,'T',T_sat,FluidName)
dv_dp_Tv = -dd_dp_Tv/d_sat_vap**2
dv_dT_pv = -dd_dT_pv/d_sat_vap**2

# derivs wrt dT, single-phase AT the sat liq line
dp_dd_Tl = CP.CoolProp.PropsSI('d(P)/d(Dmass)|T','D',d_sat_liq,'T',T_sat,FluidName)
dp_dT_dl = CP.CoolProp.PropsSI('d(P)/d(T)|Dmass','D',d_sat_liq,'T',T_sat,FluidName)
ds_dd_Tl = CP.CoolProp.PropsSI('d(Smass)/d(D)|T','D',d_sat_liq,'T',T_sat,FluidName)
ds_dT_dl = CP.CoolProp.PropsSI('d(Smass)/d(T)|D','D',d_sat_liq,'T',T_sat,FluidName)
dh_dd_Tl = CP.CoolProp.PropsSI('d(Hmass)/d(D)|T','D',d_sat_liq,'T',T_sat,FluidName)
dh_dT_dl = CP.CoolProp.PropsSI('d(Hmass)/d(T)|D','D',d_sat_liq,'T',T_sat,FluidName)
dp_dv_Tl = -d_sat_liq**2 * dp_dd_Tl
dp_dT_vl = dp_dT_dl

# derivs wrt dT, single-phase AT the sat vap line
dp_dd_Tv = CP.CoolProp.PropsSI('d(P)/d(Dmass)|T','D',d_sat_vap,'T',T_sat,FluidName)
dp_dT_dv = CP.CoolProp.PropsSI('d(P)/d(T)|Dmass','D',d_sat_vap,'T',T_sat,FluidName)
ds_dd_Tv = CP.CoolProp.PropsSI('d(Smass)/d(D)|T','D',d_sat_vap,'T',T_sat,FluidName)
ds_dT_dv = CP.CoolProp.PropsSI('d(Smass)/d(T)|D','D',d_sat_vap,'T',T_sat,FluidName)
dh_dd_Tv = CP.CoolProp.PropsSI('d(Hmass)/d(D)|T','D',d_sat_vap,'T',T_sat,FluidName)
dh_dT_dv = CP.CoolProp.PropsSI('d(Hmass)/d(T)|D','D',d_sat_vap,'T',T_sat,FluidName)
dp_dv_Tv = -d_sat_vap**2 * dp_dd_Tv
dp_dT_vv = dp_dT_dv

# two ways for derivs along the sat line: using derivs wrt dT or pT

# derivs wrt T ALONG the sat liq line (that is, at q=const=0)
dd_dT_ql = (dp_dT_q - dp_dT_dl)/dp_dd_Tl
dv_dT_ql = dv_dT_pl + dv_dp_Tl*dp_dT_q
ds_dT_ql = ds_dT_dl + ds_dd_Tl*dd_dT_ql
#ds_dT_ql = ds_dT_pl + ds_dp_Tl*dp_dT_q
dh_dT_ql = dh_dT_pl + dh_dp_Tl*dp_dT_q

# derivs wrt T ALONG the sat vap line (that is, at q=const=1)
dd_dT_qv = (dp_dT_q - dp_dT_dv)/dp_dd_Tv
dv_dT_qv = (dp_dT_q - dp_dT_dv)/dp_dd_Tv
dv_dT_qv = dv_dT_pv + dv_dp_Tv*dp_dT_q
ds_dT_qv = ds_dT_dv + ds_dd_Tv*dd_dT_qv
dh_dT_qv = dh_dT_pv + dh_dp_Tv*dp_dT_q

# derivs wrt p ALONG the sat liq line (that is, at q=const=0)
dd_dp_ql = dd_dp_Tl + dd_dT_pl*dT_dp_q
ds_dp_ql = ds_dp_Tl + ds_dT_pl*dT_dp_q
dv_dp_ql = dv_dp_Tl + dv_dT_pl*dT_dp_q

# derivs wrt p ALONG the sat vap line (that is, at q=const=1)
dd_dp_qv = dd_dp_Tv + dd_dT_pv*dT_dp_q
ds_dp_qv = ds_dp_Tv + ds_dT_pv*dT_dp_q
dv_dp_qv = dv_dp_Tv + dv_dT_pv*dT_dp_q

# derivs wrt h ALONG the sat line
dd_dh_ql = dd_dT_ql/dh_dT_ql
dd_dh_qv = dd_dT_qv/dh_dT_qv
ds_dh_ql = ds_dT_ql/dh_dT_ql
ds_dh_qv = ds_dT_qv/dh_dT_qv

# derivs of Clausius-Clapeyron
# d2p_dT2_q = ((ds_dT_qv-ds_dT_ql)*Dv_vl - Ds_vl*(dv_dT_qv-dv_dT_ql)) / Dv_vl**2
d2p_dT2_q = (ds_dT_qv-ds_dT_ql)/Dv_vl - dp_dT_q*(dv_dT_qv-dv_dT_ql)/Dv_vl
d2T_dp2_q = (dv_dp_qv-dv_dp_ql)/Ds_vl - dT_dp_q*(ds_dp_qv-ds_dp_ql)/Ds_vl
d2p_dT2_qn = numSlopeAr(T_sat,dp_dT_q)
d2T_dp2_qn = numSlopeAr(p_sat,dT_dp_q)

# second derivs AT the sat line
# AT the liq line
# wrt pT
d2d_dT2_pl = np.empty(nPoints)
d2d_dp2_Tl = np.empty(len(T_sat))
d2d_dpTl   = np.empty(len(T_sat))
d2s_dT2_pl = np.empty(len(T_sat))
d2s_dp2_Tl = np.empty(len(T_sat))
d2s_dpTl   = np.empty(len(T_sat))
d2h_dT2_pl = np.empty(len(T_sat))
d2h_dp2_Tl = np.empty(len(T_sat))
d2h_dpTl   = np.empty(len(T_sat))
# wrt dT
d2s_dT2_dl = np.empty(len(T_sat))
d2s_dd2_Tl = np.empty(len(T_sat))
d2s_ddTl   = np.empty(len(T_sat))
# AT the vap line
# wrt pT
d2d_dT2_pv = np.empty(len(T_sat))
d2d_dp2_Tv = np.empty(len(T_sat))
d2d_dpTv   = np.empty(len(T_sat))
d2s_dT2_pv = np.empty(len(T_sat))
d2s_dp2_Tv = np.empty(len(T_sat))
d2s_dpTv   = np.empty(len(T_sat))
d2h_dT2_pv = np.empty(len(T_sat))
d2h_dp2_Tv = np.empty(len(T_sat))
d2h_dpTv   = np.empty(len(T_sat))
# wrt dT
d2s_dT2_dv = np.empty(len(T_sat))
d2s_dd2_Tv = np.empty(len(T_sat))
d2s_ddTv   = np.empty(len(T_sat))

HEOS = CP.AbstractState("HEOS", FluidName)
for idx in range(0,len(T_sat)):
    # AT the liq line
    HEOS.update(CP.QT_INPUTS, 0, T_sat[idx])
    # wrt pT
    d2d_dT2_pl[idx] = HEOS.second_partial_deriv(CP.iDmass, CP.iT, CP.iP, CP.iT, CP.iP)
    d2d_dp2_Tl[idx] = HEOS.second_partial_deriv(CP.iDmass, CP.iP, CP.iT, CP.iP, CP.iT)
    d2d_dpTl[idx] = HEOS.second_partial_deriv(CP.iDmass, CP.iP, CP.iT, CP.iT, CP.iP)
    d2s_dT2_pl[idx] = HEOS.second_partial_deriv(CP.iSmass, CP.iT, CP.iP, CP.iT, CP.iP)
    d2s_dp2_Tl[idx] = HEOS.second_partial_deriv(CP.iSmass, CP.iP, CP.iT, CP.iP, CP.iT)
    d2s_dpTl[idx] = HEOS.second_partial_deriv(CP.iSmass, CP.iP, CP.iT, CP.iT, CP.iP)
    d2h_dT2_pl[idx] = HEOS.second_partial_deriv(CP.iHmass, CP.iT, CP.iP, CP.iT, CP.iP)
    d2h_dp2_Tl[idx] = HEOS.second_partial_deriv(CP.iHmass, CP.iP, CP.iT, CP.iP, CP.iT)
    d2h_dpTl[idx] = HEOS.second_partial_deriv(CP.iHmass, CP.iP, CP.iT, CP.iT, CP.iP)
    # wrt dT
    d2s_dT2_dl[idx] = HEOS.second_partial_deriv(CP.iSmass, CP.iT, CP.iDmass, CP.iT, CP.iDmass)
    d2s_dd2_Tl[idx] = HEOS.second_partial_deriv(CP.iSmass, CP.iDmass, CP.iT, CP.iDmass, CP.iT)
    d2s_ddTl[idx] = HEOS.second_partial_deriv(CP.iSmass, CP.iDmass, CP.iT, CP.iT, CP.iDmass)

    # AT the vap line
    HEOS.update(CP.QT_INPUTS, 1, T_sat[idx])
    # wrt pT
    d2d_dT2_pv[idx] = HEOS.second_partial_deriv(CP.iDmass, CP.iT, CP.iP, CP.iT, CP.iP)
    d2d_dp2_Tv[idx] = HEOS.second_partial_deriv(CP.iDmass, CP.iP, CP.iT, CP.iP, CP.iT)
    d2d_dpTv[idx] = HEOS.second_partial_deriv(CP.iDmass, CP.iP, CP.iT, CP.iT, CP.iP)
    d2s_dT2_pv[idx] = HEOS.second_partial_deriv(CP.iSmass, CP.iT, CP.iP, CP.iT, CP.iP)
    d2s_dp2_Tv[idx] = HEOS.second_partial_deriv(CP.iSmass, CP.iP, CP.iT, CP.iP, CP.iT)
    d2s_dpTv[idx] = HEOS.second_partial_deriv(CP.iSmass, CP.iP, CP.iT, CP.iT, CP.iP)
    d2h_dT2_pv[idx] = HEOS.second_partial_deriv(CP.iHmass, CP.iT, CP.iP, CP.iT, CP.iP)
    d2h_dp2_Tv[idx] = HEOS.second_partial_deriv(CP.iHmass, CP.iP, CP.iT, CP.iP, CP.iT)
    d2h_dpTv[idx] = HEOS.second_partial_deriv(CP.iHmass, CP.iP, CP.iT, CP.iT, CP.iP)
    # wrt dT
    d2s_dT2_dv[idx] = HEOS.second_partial_deriv(CP.iSmass, CP.iT, CP.iDmass, CP.iT, CP.iDmass)
    d2s_dd2_Tv[idx] = HEOS.second_partial_deriv(CP.iSmass, CP.iDmass, CP.iT, CP.iDmass, CP.iT)
    d2s_ddTv[idx] = HEOS.second_partial_deriv(CP.iSmass, CP.iDmass, CP.iT, CP.iT, CP.iDmass)

# calculate 2nd derivs ALONG the sat line (analytically and numerically)

# A=dd_dT_p and B=dd_dp_T
# liq side
dA_dT_ql = d2d_dT2_pl + d2d_dpTl*dp_dT_q
dB_dT_ql = d2d_dpTl + d2d_dp2_Tl*dp_dT_q
d2d_dT2_qla = dA_dT_ql + dB_dT_ql*dp_dT_q + dd_dp_Tl*d2p_dT2_q
d2d_dT2_qln = numSlopeAr(T_sat, dd_dT_ql)
# vap side
dA_dT_qv = d2d_dT2_pv + d2d_dpTv*dp_dT_q
dB_dT_qv = d2d_dpTv + d2d_dp2_Tv*dp_dT_q
d2d_dT2_qva = dA_dT_qv + dB_dT_qv*dp_dT_q + dd_dp_Tv*d2p_dT2_q
d2d_dT2_qvn = numSlopeAr(T_sat, dd_dT_qv)
#print(d2d_dT2_qla - d2d_dT2_qln)
#print(d2d_dT2_qva - d2d_dT2_qln)

# C=ds_dT_p and E=ds_dp_T
# liq side
dC_dT_ql = d2s_dT2_dl + d2s_ddTl*dd_dT_ql
dE_dT_ql = d2s_ddTl + d2s_dd2_Tl*dd_dT_ql
d2s_dT2_qla = dC_dT_ql + dE_dT_ql*dd_dT_ql + ds_dd_Tl*d2d_dT2_qla
d2s_dT2_qln = numSlopeAr(T_sat, ds_dT_ql)
# vap side
dC_dT_qv = d2s_dT2_dv + d2s_ddTv*dd_dT_qv
dE_dT_qv = d2s_ddTv + d2s_dd2_Tv*dd_dT_qv
d2s_dT2_qva = dC_dT_qv + dE_dT_qv*dd_dT_qv + ds_dd_Tv*d2d_dT2_qva
d2s_dT2_qvn = numSlopeAr(T_sat, ds_dT_qv)
#print(d2s_dT2_qla - d2s_dT2_qln)
#print(d2s_dT2_qva - d2s_dT2_qvn)

# B=dd_dp_T and A=dd_dT_p
# liq side
dB_dp_ql = d2d_dp2_Tl + d2d_dpTl*dT_dp_q
dA_dp_ql = d2d_dpTl + d2d_dT2_pl*dT_dp_q
d2d_dp2_qla = dB_dp_ql + dA_dp_ql*dT_dp_q + dd_dT_pl*d2T_dp2_q
d2d_dp2_qln = numSlopeAr(T_sat, ds_dp_ql)
# vap side
dB_dp_qv = d2d_dp2_Tv + d2d_dpTv*dT_dp_q
dA_dp_qv = d2d_dpTv + d2d_dT2_pv*dT_dp_q
d2d_dp2_qva = dB_dp_qv + dA_dp_qv*dT_dp_q + dd_dT_pv*d2T_dp2_q
d2d_dp2_qvn = numSlopeAr(T_sat, ds_dp_qv)
#print(d2d_dp2_qla - d2d_dp2_qln)
#print(d2d_dp2_qva - d2d_dp2_qvn)

# E=ds_dp_T and C=ds_dT_p
# liq side
dE_dp_ql = d2s_dp2_Tl + d2s_dpTl*dT_dp_q
dC_dp_ql = d2s_dpTl + d2s_dT2_pl*dT_dp_q
d2s_dp2_qla = dE_dp_ql + dC_dp_ql*dT_dp_q + ds_dT_pl*d2T_dp2_q
d2s_dp2_qln = numSlopeAr(p_sat, ds_dp_ql)
# vap side
dE_dp_qv = d2s_dp2_Tv + d2s_dpTv*dT_dp_q
dC_dp_qv = d2s_dpTv + d2s_dT2_pv*dT_dp_q
d2s_dp2_qva = dE_dp_qv + dC_dp_qv*dT_dp_q + ds_dT_pv*d2T_dp2_q
d2s_dp2_qvn = numSlopeAr(p_sat, ds_dp_qv)
#print(d2s_dp2_qla - d2s_dp2_qln)
#print(d2s_dp2_qva - d2s_dp2_qvn)

# G=dh_dT_p and K=dh_dp_T
# liq side
dG_dT_ql = d2h_dT2_pl + d2h_dpTl*dp_dT_q
dK_dT_ql = d2h_dpTl + d2h_dp2_Tl*dp_dT_q
d2h_dT2_qla = dG_dT_ql + dK_dT_ql*dp_dT_q + dh_dp_Tl*d2p_dT2_q
d2h_dT2_qln = numSlopeAr(T_sat, dh_dT_ql)
# vap side
dG_dT_qv = d2h_dT2_pv + d2h_dpTv*dp_dT_q
dK_dT_qv = d2h_dpTv + d2h_dp2_Tv*dp_dT_q
d2h_dT2_qva = dG_dT_qv + dK_dT_qv*dp_dT_q + dh_dp_Tv*d2p_dT2_q
d2h_dT2_qvn = numSlopeAr(T_sat, dh_dT_qv)
#print(d2h_dT2_qla - d2h_dT2_qln)
#print(d2h_dT2_qva - d2h_dT2_qvn)

# liq side
d2d_dh2_qla = (d2d_dT2_qla*dh_dT_ql - dd_dT_ql*d2h_dT2_qla) / dh_dT_ql**3
d2d_dh2_qln = numSlopeAr(h_sat_liq, dd_dh_ql)
# vap side
d2d_dh2_qva = (d2d_dT2_qva*dh_dT_qv - dd_dT_qv*d2h_dT2_qva) / dh_dT_qv**3
d2d_dh2_qvn = numSlopeAr(h_sat_vap, dd_dh_qv)
#print(d2d_dh2_qla - d2d_dh2_qln)
#print(d2d_dh2_qva - d2d_dh2_qvn)

# liq side
d2s_dh2_qla = (d2s_dT2_qla*dh_dT_ql - ds_dT_ql*d2h_dT2_qla) / dh_dT_ql**3
d2s_dh2_qln = numSlopeAr(h_sat_liq, ds_dh_ql)
# vap side
d2s_dh2_qva = (d2s_dT2_qva*dh_dT_qv - ds_dT_qv*d2h_dT2_qva) / dh_dT_qv**3
d2s_dh2_qvn = numSlopeAr(h_sat_vap, ds_dh_qv)
#print(d2s_dh2_qla - d2s_dh2_qln)
#print(d2s_dh2_qva - d2s_dh2_qvn)
```

    T_trp = 220.0
    T_crt = 367.85
    p_trp = 31507.558848475514
    p_crt = 3382200.0

    C:\Users\matthis.thorade\Anaconda3\lib\site-packages\ipykernel_launcher.py:41: RuntimeWarning: invalid value encountered in true_divide
    C:\Users\matthis.thorade\Anaconda3\lib\site-packages\ipykernel_launcher.py:42: RuntimeWarning: invalid value encountered in true_divide

```python
# vapor pressure, Clausius-Clapeyron

# pick a point and set figure size
mySatPoint = next(i for i,v in enumerate(p_sat) if v >= 53400.6)
#mySatPoint = next(i for i,v in enumerate(T_sat) if v >= 101325)
print("p_sat = " + str(p_sat[mySatPoint]))
print("T_sat = " + str(T_sat[mySatPoint]))
plt.figure(figsize=(width,width*3/2/golden))

plt.subplot(3,2,1)
plt.plot(T_sat, p_sat, color='blue')
#plt.yscale('log')
plt.ticklabel_format(style='sci', axis='y', scilimits=(0,0))
plt.grid(b=True, linestyle=':')
plt.minorticks_on()
plt.xlabel('Temperature in K')
plt.ylabel('Pressure in Pa')
drawTangent(T_sat, p_sat, dp_dT_q, mySatPoint)
#print(dp_dT_q - dp_dT_qn)

plt.subplot(3,2,2)
plt.plot(p_sat, T_sat, color='blue')
#plt.yscale('log')
plt.ticklabel_format(style='sci', axis='x', scilimits=(0,0))
plt.grid(b=True, linestyle=':')
plt.minorticks_on()
plt.xlabel('Pressure in Pa')
plt.ylabel('Temperature in K')
drawTangent(p_sat, T_sat, dT_dp_q, mySatPoint)
#print(dT_dp_q - dT_dp_qn)

plt.subplot(3,2,3)
plt.plot(T_sat, dp_dT_q, color='blue')
#plt.yscale('log')
plt.grid(b=True, linestyle=':')
plt.minorticks_on()
plt.xlabel('Temperature in K')
plt.ylabel('dp/dT in Pa/K')
drawTangent(T_sat, dp_dT_q, d2p_dT2_q, mySatPoint)
#print(d2p_dT2_q - d2p_dT2_qn)

plt.subplot(3,2,4)
plt.plot(p_sat, dT_dp_q, color='blue')
plt.yscale('log')
plt.ticklabel_format(style='sci', axis='x', scilimits=(0,0))
plt.grid(b=True, linestyle=':')
plt.minorticks_on()
plt.xlabel('Pressure in Pa')
plt.ylabel('dT/dp in K/Pa')
drawTangent(p_sat, dT_dp_q, d2T_dp2_q, mySatPoint)
#print(d2T_dp2_q - d2T_dp2_qn)

plt.subplot(3,2,5)
plt.plot(T_sat, d2p_dT2_q, color='blue')
plt.grid(b=True, linestyle=':')
plt.minorticks_on()
plt.xlabel('Temperature in K')
plt.ylabel('d2p/dT2 in Pa/K2')

plt.subplot(3,2,6)
plt.plot(p_sat, abs(d2T_dp2_q), color='blue')
plt.yscale('log')
plt.ticklabel_format(style='sci', axis='x', scilimits=(0,0))
plt.grid(b=True, linestyle=':')
plt.minorticks_on()
plt.xlabel('Pressure in Pa')
plt.ylabel('|d2T/dp2| in K/Pa2')

plt.show()
```

```python
# Pick a point and set figure size
mySatPoint = int(nPoints-50)
print("T_sat = " + str(T_sat[mySatPoint]))
print("p_sat = " + str(p_sat[mySatPoint]))
plt.figure(figsize=(width,width*3/2/golden))

# d versus T
plt.subplot(3,2,1)
plt.plot(T_sat, d_sat_liq, color='blue')
plt.plot(T_sat, d_sat_vap, color='red')
#plt.plot(T_sat, (d_sat_liq+d_sat_vap)/2, color='black')
plt.xlabel('Temperature in K')
plt.ylabel('Density in kg/m³')
drawTangent(T_sat, d_sat_liq, dd_dT_ql, mySatPoint)
drawTangent(T_sat, d_sat_vap, dd_dT_qv, mySatPoint)

# s versus T
plt.subplot(3,2,2)
plt.plot(T_sat, s_sat_liq, color='blue')
plt.plot(T_sat, s_sat_vap, color='red')
plt.xlabel('Temperature in K')
plt.ylabel('Specific entropy in J/(kg·K)')
drawTangent(T_sat, s_sat_liq, ds_dT_ql, mySatPoint)
drawTangent(T_sat, s_sat_vap, ds_dT_qv, mySatPoint)

# d versus p
plt.subplot(3,2,3)
plt.plot(p_sat, d_sat_liq, color='blue')
plt.plot(p_sat, d_sat_vap, color='red')
plt.ticklabel_format(style='sci', axis='x', scilimits=(0,0))
plt.xlabel('Pressure in Pa')
plt.ylabel('Density in kg/m³')
drawTangent(p_sat, d_sat_liq, dd_dp_ql, mySatPoint)
drawTangent(p_sat, d_sat_vap, dd_dp_qv, mySatPoint)
#print(dd_dp_ql - numSlopeAr(p_sat, d_sat_liq))
#print(dd_dp_qv - numSlopeAr(p_sat, d_sat_vap))

# s versus p
plt.subplot(3,2,4)
plt.plot(p_sat, s_sat_liq, color='blue')
plt.plot(p_sat, s_sat_vap, color='red')
#plt.yscale('log')
plt.ticklabel_format(style='sci', axis='x', scilimits=(0,0))
plt.xlabel('Pressure in Pa')
plt.ylabel('Specific entropy in J/(kg·K)')
drawTangent(p_sat, s_sat_liq, ds_dp_ql, mySatPoint)
drawTangent(p_sat, s_sat_vap, ds_dp_qv, mySatPoint)
#print(ds_dp_ql - numSlopeAr(p_sat, s_sat_liq))
#print(ds_dp_qv - numSlopeAr(p_sat, s_sat_vap))

# d versus h
plt.subplot(3,2,5)
plt.plot(h_sat_liq, d_sat_liq, color='blue')
plt.plot(h_sat_vap, d_sat_vap, color='red')
plt.ticklabel_format(style='sci', axis='x', scilimits=(0,0))
plt.xlabel('Specific enthalpy in J/kg')
plt.ylabel('Density in kg/m³')
drawTangent(h_sat_liq, d_sat_liq, dd_dh_ql, mySatPoint)
drawTangent(h_sat_vap, d_sat_vap, dd_dh_qv, mySatPoint)

# s versus h
plt.subplot(3,2,6)
plt.plot(h_sat_liq, s_sat_liq, color='blue')
plt.plot(h_sat_vap, s_sat_vap, color='red')
plt.ticklabel_format(style='sci', axis='x', scilimits=(0,0))
plt.xlabel('Specific enthalpy in J/kg')
plt.ylabel('Specific entropy in J/(kg·K)')
drawTangent(h_sat_liq, s_sat_liq, ds_dh_ql, mySatPoint)
drawTangent(h_sat_vap, s_sat_vap, ds_dh_qv, mySatPoint)

plt.show()
```

```python
# Pick a point and set figure size
mySatPoint = int(nPoints-50)
print("T_sat = " + str(T_sat[mySatPoint]))
print("p_sat = " + str(p_sat[mySatPoint]))
plt.figure(figsize=(width,width*3/2/golden))

# dd_dT versus T
plt.subplot(3,2,1)
plt.plot(T_sat, dd_dT_ql, color='blue')
plt.plot(T_sat, dd_dT_qv, color='red')
plt.xlabel('Temperature in K')
plt.ylabel('dd/dT in kg/m³/K')
drawTangent(T_sat, dd_dT_ql, d2d_dT2_qla, mySatPoint)
drawTangent(T_sat, dd_dT_qv, d2d_dT2_qva, mySatPoint)

# ds_dT versus T
plt.subplot(3,2,2)
plt.plot(T_sat, ds_dT_ql, color='blue')
plt.plot(T_sat, ds_dT_qv, color='red')
plt.xlabel('Temperature in K')
plt.ylabel('ds/dT in J//(kg·K)/K')
drawTangent(T_sat, ds_dT_ql, d2s_dT2_qla, mySatPoint)
drawTangent(T_sat, ds_dT_qv, d2s_dT2_qva, mySatPoint)

# dd_dp versus p
plt.subplot(3,2,3)
#plt.plot(p_sat, dd_dp_ql, color='blue')
plt.plot(p_sat, dd_dp_qv, color='red')
plt.ticklabel_format(style='sci', axis='x', scilimits=(0,0))
plt.xlabel('Pressure in Pa')
plt.ylabel('dd/dp in kg/m³/Pa')
#drawTangent(p_sat, dd_dp_ql, d2d_dp2_qla, mySatPoint)
drawTangent(p_sat, dd_dp_qv, d2d_dp2_qva, mySatPoint)

# ds_dp versus p
plt.subplot(3,2,4)
plt.plot(p_sat, ds_dp_ql, color='blue')
#plt.plot(p_sat, ds_dp_qv, color='red')
#plt.yscale('log')
plt.ylim([0,10*min(ds_dp_ql)])
plt.ticklabel_format(style='sci', axis='x', scilimits=(0,0))
plt.xlabel('Pressure in Pa')
plt.ylabel('ds/dp in J//(kg·K)/Pa')
drawTangent(p_sat, ds_dp_ql, d2s_dp2_qla, mySatPoint)
#drawTangent(p_sat, ds_dp_qv, d2s_dp2_qva, mySatPoint)

# dd_dh versus h
plt.subplot(3,2,5)
plt.plot(h_sat_liq, dd_dh_ql, color='blue')
#plt.plot(h_sat_vap, dd_dh_qv, color='red')
plt.ticklabel_format(style='sci', axis='x', scilimits=(0,0))
plt.xlabel('Specific enthalpy in J/kg')
plt.ylabel('dd/dh')
drawTangent(h_sat_liq, dd_dh_ql, d2d_dh2_qla, mySatPoint)
#drawTangent(h_sat_vap, dd_dh_qv, d2d_dh2_qva, mySatPoint)

# ds_dh versus h
plt.subplot(3,2,6)
plt.plot(h_sat_liq, ds_dh_ql, color='blue')
#plt.plot(h_sat_vap, ds_dh_qv, color='red')
plt.ticklabel_format(style='sci', axis='x', scilimits=(0,0))
plt.xlabel('Specific enthalpy in J/kg')
plt.ylabel('ds/dh')
drawTangent(h_sat_liq, ds_dh_ql, d2s_dh2_qla, mySatPoint)
#drawTangent(h_sat_vap, ds_dh_qv, d2s_dh2_qva, mySatPoint)

plt.show()
```


### General Relativty describes the interaction between the geometry of spacetime and the energy-momentum of matter.

$G_{\mu\nu}$ + $\lambda g_{\mu\nu}$ = 8$\pi G T_{\mu\nu}$

Einstein's field equations are basically a set of coupled partial differential equations of components of the metric $g_{\mu\nu}$
. The metric defines the geometry of the space-time manifold.

On the left hand side we have the Einstein tensor $G_{\mu\nu}$ = $R_{\mu\nu}$ - 1/2 $g_{\mu\nu}$$R$ , where the Riemann tensor and Ricci scalar are complicated functions of the metric and its derivatives. Besides, we have $\lambda$ , which is a free constant parameter allowed by the theory, but which we now think is responsible for the accelerated expansion of the Universe.

The right hand side, with the tensor $T_{\mu\nu}$
 describes the energy and momentum of matter, a fluid, particles or the object in question (for example a star).

Youtube Video - https://www.youtube.com/watch?v=JRZgW1YjCKk (Stanford Lecture Series)

```python
import sympy as smp
import einsteinpy
```

#### Let's calculate the behavior of the "scale factor" of the Universe, that the Einstein equations predict, for a homogeneous and isotropic Universe filled with a perfect fluid.

```python
G_00, T_00, G, rho = smp.symbols("G_{00} T_{00} G rho")
```

```python
rho
```

$\displaystyle \rho$

```python
einst_eq = smp.Eq( G_00, 8 * smp.pi * G * T_00 )
einst_eq
```

$\displaystyle G_{00} = 8 \pi G T_{00}$

```python
# The energy-momentum tensor of a perfect fluid:

energy_momentum = smp.Eq(T_00, rho)
energy_momentum
```

$\displaystyle T_{00} = \rho$

```python
syms = smp.symbols("t x y z")
t, x, y, z = syms
```

```python
a = smp.Function("a")  ##scale factor
```

__The line element of a homogeneous and isotropic Universe with no curvature. The metric elements are the coefficients in front of the coordinates.__
$ds^{2}$ = $g_{\mu\nu}$ $dx^{\mu}$ $dx^{\nu}$ = -$dt^{2}$ + $a^{2}(t)$($dx^{2}$+$dy^{2}$+$dz^{2}$)

```python
from einsteinpy.symbolic import EinsteinTensor, MetricTensor
m = smp.diag(-1, a(t)**2, a(t)**2, a(t)** 2).tolist()
metric = MetricTensor(m, syms)
```

```python
metric.tensor()
```

$\displaystyle \left[\begin{matrix}-1 & 0 & 0 & 0\\0 & a^{2}{\left(t \right)} & 0 & 0\\0 & 0 & a^{2}{\left(t \right)} & 0\\0 & 0 & 0 & a^{2}{\left(t \right)}\end{matrix}\right]$

```python
einst = EinsteinTensor.from_metric(metric)
einst.simplify()
```

$\displaystyle \left[\begin{matrix}\frac{3.0 \left(\frac{d}{d t} a{\left(t \right)}\right)^{2}}{a^{2}{\left(t \right)}} & 0 & 0 & 0\\0 & - 2.0 a{\left(t \right)} \frac{d^{2}}{d t^{2}} a{\left(t \right)} - 1.0 \left(\frac{d}{d t} a{\left(t \right)}\right)^{2} & 0 & 0\\0 & 0 & - 2.0 a{\left(t \right)} \frac{d^{2}}{d t^{2}} a{\left(t \right)} - 1.0 \left(\frac{d}{d t} a{\left(t \right)}\right)^{2} & 0\\0 & 0 & 0 & - 2.0 a{\left(t \right)} \frac{d^{2}}{d t^{2}} a{\left(t \right)} - 1.0 \left(\frac{d}{d t} a{\left(t \right)}\right)^{2}\end{matrix}\right]$

```python
H = smp.symbols("H")
```

Let's define a new function, that we will cull the Hubble function and it will be the relative "velocity" of expansion of $a(t)$:

$H(t)$ = $a^{o}$/$a$

```python
smp.Eq( H, smp.diff(a(t), t)/a(t))
```

$\displaystyle H = \frac{\frac{d}{d t} a{\left(t \right)}}{a{\left(t \right)}}$

```python
G_00s = einst[0,0]
G_00s = G_00s.subs(smp.diff(a(t), t)/a(t), H)
G_00s
```

$\displaystyle 3.0 H^{2}$

```python
einst_eq.subs(G_00, G_00s).subs(T_00, rho)
```

$\displaystyle 3.0 H^{2} = 8 \pi G \rho$

```python

```


# Solar-powered pump

A solar-powered pump transports water from a source underground up to a tank. The PV solar panel converts solar energy into electrical power with an efficiency of $\eta_{\text{PV}} = 0.08$ and stores the electrical energy in a battery, which powers the pump continuously. The 24-hour annual average solar flux is SF = 225 W/m$^2$.

The isentropic efficiency of the pump is 0.58, and the pump performance is given as pressure head versus flow rate:

| Head (ft)     | Flow rate (gpm) |
| ------------- | --------------- |
| 60            | 0               |
| 60            | 2               |
| 57            | 4               |
| 52            | 6               |
| 44            | 8               |
| 33            | 10              |
| 18            | 12              |
| 0             | 14              |

The water source is 20 ft below ground level (where the tank is located), so the pump must provide a pressure rise equal to the 20 ft of water. There is also head loss due to friction in the pipe; the total head loss (pressure drop due to friction) between the inlet and outlet is:
\begin{equation}
\Delta P_{\text{pipe}} = 20 \left[ \text{ft H}_2 \text{O} \right] + 0.085 \left[ \frac{ \text{ft H}_2 \text{O} }{\text{gpm}^2} \right] \dot{V}^2
\end{equation}
where $\dot{V}$ is the volumetric flow rate.

We can assume water is an incompressible fluid here.

**Problem**:

- Determine the flow rate delivered by the pump
- Determine the area of the solar panel required by the system

```python
import numpy as np
from scipy import optimize
import cantera as ct

from pint import UnitRegistry
ureg = UnitRegistry()
Q_ = ureg.Quantity
```

```python
import matplotlib.pyplot as plt
%matplotlib inline

# these are mostly for making the saved figures nicer
import matplotlib_inline.backend_inline
matplotlib_inline.backend_inline.set_matplotlib_formats('pdf', 'png')
plt.rcParams['figure.dpi']= 150
plt.rcParams['savefig.dpi'] = 150
```

## Determine the flow rate of water delivered

First, specify the input information:

```python
solar_flux = Q_(225, 'W/m^2')
efficiency_pump = 0.58
efficiency_solar = 0.08
```

Next, to model the pump, we can fit the performance data to a third-order polynomial:
\begin{equation}
\Delta P_{\text{pump}} = a_0 + a_1 \dot{V} + a_2 \dot{V}^2 + a_3 \dot{V}^3
\end{equation}

We can find this fit using the NumPy function `polyfit()`, and place the result in a `poly1d` object so we can easily evaluate the polynomial:

```python
# Construct NumPy arrays with the pressure head and
# flow rate from the table
head = Q_(np.asarray([60, 60, 57, 52, 44, 33, 18, 0]), 'ft')
flow_rate = Q_(np.asarray([0, 2, 4, 6, 8, 10, 12, 14]), 'gallon per minute')

# Perform the fit with degree 3, and place in a poly1d object
pump_fit = np.poly1d(np.polyfit(flow_rate.magnitude, head.magnitude, 3))

fig, ax = plt.subplots(figsize=(5, 3))

# Plot the measurements
ax.plot(flow_rate.magnitude, head.magnitude, 'o')

# Generate a dense sample of flow rate values, then plot the polynomial fit
flow_rate_dense = np.linspace(flow_rate[0].magnitude, flow_rate[-1].magnitude, 100)
ax.plot(flow_rate_dense, pump_fit(flow_rate_dense))

plt.xlabel(f'Flow rate ({flow_rate.units})')
plt.ylabel(f'Pressure head ({head.units})')
plt.grid(True)
plt.legend(['Pump data', 'Best fit curve'])
fig.tight_layout()
plt.show()
```

To find the volumetric flow rate of the pump, we need to find the condition where the pressure head of the pump matches the pressure drop of the pipe between the inlet and outlet. We can do this by setting two $\Delta P$ expressions equal to each other, and the finding the root of this equation:
\begin{equation}
\Delta P_{\text{pump}} \left(\dot{V}\right) = \Delta P_{\text{pipe}} \left(\dot{V}\right)
\end{equation}

To find the root, we can use the [SciPy function `root_scalar`](https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.root_scalar.html), part of its `optimize` module. This needs needs us to provide a function that rearranges the above expression to equal zero, and returns that value. In other words, the function should return:
\begin{equation}
\Delta P_{\text{pump}} \left(\dot{V}\right) - \Delta P_{\text{pipe}} \left(\dot{V}\right)
\end{equation}
which will equal zero when the root is found.

```python
def pump_performance(volume_flow_rate, pump_fit):
    '''Find root of pump performance equations'''
    deltaP_pump = pump_fit(volume_flow_rate) * ureg.ft
    volume_flow_rate *= ureg('gal/min')
    deltaP_pipe = (
        Q_(20, 'ft') +
        Q_(0.085, 'ft/((gal/min)**2)') * volume_flow_rate**2
        )
    return (deltaP_pump - deltaP_pipe).to('ft').magnitude
```

Then, we call the `root_scalar` function, giving it the function we just created along with a range of possible values and the extra argument our function needs:

```python
sol = optimize.root_scalar(
    pump_performance, bracket=[0, 14],
    args=(pump_fit,),
    )

volume_flow_rate = Q_(sol.root, 'gal/min')
print(f'Pump flow rate: {volume_flow_rate: .2f}')
```

    Pump flow rate: 10.51 gallon / minute

## Determine area of solar panels required

To determine the area of solar panels required, we first need to calculate the work needed for the pump:

$$
\dot{W}_p = \frac{\dot{m} v \Delta P_{\text{pump}} }{\eta_p} = \frac{\dot{V} \Delta P_{\text{pump}} }{\eta_p}
$$

First, however, we'll need to convert the pressure head (given in feet of water) to an actual pressure difference, using $h = \frac{P}{\rho g}$.

```python
pressure_head = Q_(pump_fit(volume_flow_rate.magnitude), 'ft')
water = ct.Water()
pressure_drop = pressure_head * Q_(water.density, 'kg/m^3') * ureg.gravity

work_pump = (
    volume_flow_rate * pressure_drop.to('Pa') /
    efficiency_pump
    )
print(f'Pump work: {work_pump.to(ureg.watt): .2f}')
```

    Pump work: 100.13 watt

Finally, we can calculate the area required by relating the power generated by the solar panels to the rate of work required by the pump:

$$
\text{SF} \, A \eta_{\text{PV}} = \dot{W}_p
$$

```python
area_solar = work_pump / (solar_flux * efficiency_solar)
print(f'Area of solar panels required: {area_solar.to("m^2"): .2f}')
```

    Area of solar panels required: 5.56 meter ** 2


# Denton proportional procedure

Denton procedure will interpolate high frequency data into low frecuency. Let be $I$ a vector indicator with high frecuency data from t=1 to T. Lets assume that $A_n$ is a vector of low frecuency data where $A_n$ represent the n period and is length N. So the objetive is interpolate the vector $I$ into $A$. Lets assume that the length of I divide by A is equal to q (if q=4 => that is annual data with quarterly data). Lets assume that q=4.

Original source: https://www.imf.org/external/pubs/ft/qna/pdf/2017/chapter6.pdf

The minimization problem is define as:

$$
\min_{X_t} \sum_{t=2}^T (\frac{X_t}{I_t} - \frac{X_{t-1}}{I_{t-1}})^2
$$
Subject to:
$$
\sum_{t=4n-3}^{4n} X_t = A_n \text{    for n = 1,...,N}
$$

So we want the vector $X_t$ such that aggregate the same value  in annual data but minimize the variation growth of the quarterly data.

It is better to express this problem as a quadratic matricial minimization problem. Lets define:

\begin{equation}
D =
\begin{pmatrix}
-1 & 1 & 0 &\cdots & 0\\
0 & -1 & 1 & \cdots & 0\\
\vdots & \vdots & \vdots & \ddots & \vdots\\
0 & 0 & 0 & \cdots & 0
\end{pmatrix}
\end{equation}

D is square matrix (TxT) with -1 in the diagonal and 1 in the subsecuent element of the diagonal and the last rows has 0 in all of his elements.

\begin{equation}
J =
\begin{pmatrix}
1 & 1 & 1 & 1 & 0 & 0 & 0 & 0 &\cdots & 0\\
0 & 0 & 0 & 0 & 1 & 1 & 1 & 1 &\cdots & 0\\
\vdots & \vdots & \vdots  & \vdots & \vdots & \vdots & \vdots  & \vdots & \ddots & \vdots\\
0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & \cdots & 1
\end{pmatrix}
\end{equation}

J is matrix (N x T) with 1s in rows and 0 that it is use to aggregate the data of X (in the representation above it is assume to be year data with quarterly data).

Let $\tilde{I}$ be the diagonal inverse matrix of $I$. It is a square matrix of $TxT$ and $\tilde{X}= \tilde{I}X$. Therefore the problem can be represented as:

$$
\min_{X} (D\tilde{X})^T D\tilde{X}
$$
Subject to:
$$JX = A \text{ and } \tilde{X}= \tilde{I}X$$

Let assume that $M=\tilde{I^T} D^TD\tilde{I}$

Rewrite the problem as:

$$
\min_{X} X^T M X
$$
Subject to:
$$
JX = A
$$

The langrange is given by:

$$
L = X^T M X - \lambda^{T} (JX - A )
$$

The FOC under the lagrange multiplier will be $(M + M^T)X - J^T \lambda = 0$ and $JX =A$. In matricial form:

\begin{equation}
\begin{pmatrix}
(M+M^T) & -J^T \\
J & 0
\end{pmatrix}
\begin{pmatrix}
X \\
\lambda
\end{pmatrix}=
\begin{pmatrix}
0 \\
A
\end{pmatrix}
\end{equation}

The solution is given by:

\begin{equation}
\begin{pmatrix}
X \\
\lambda
\end{pmatrix}=
\begin{pmatrix}
(M+M^T) & -J^T \\
J & 0
\end{pmatrix}^{-1}
\begin{pmatrix}
0 \\
A
\end{pmatrix}
\end{equation}

```python
import denton
import numpy as np

help(denton.proportional_method)
```

    Help on function proportional_method in module denton:

    proportional_method(I, A)
        Inputs:
            I = indicator array to be use in the benchmark.
            Ordered from t0 to tn (high frequncy) ex. quarterly data
            A =  aggregate data represented as every element as mean of a the n subset of I
            (in the same order) (low frequncy) ex. annual data
            I/A has to be integer i.e the elements in the indicator data must the multiple of A.
        Output
            B = array with high frequency interpolation of the high frequency data in the low frequency

        Example:
        Taken from: https://www.imf.org/external/pubs/ft/qna/pdf/2017/chapter6.pdf
        Table: Example 6.2

        import numpy as np
        import denton

        I = np.array([99.4,99.6,100.1,100.9,101.7,102.2,102.9,
             103.8,104.9,106.3,107.3,107.8,107.9,
             107.5,107.2,107.5])
        A = np.array([1000, 1040, 1060.8, 1064.9])

        #the average of every 4 in the quarterly data = annual data
        B = denton.proportional_method(I, A)

        #to replicate the table then divide by 4
        B_imf = denton.proportional_method(I, A)/4

```python
I = np.array([99.4,99.6,100.1,100.9,101.7,102.2,102.9,
              103.8,104.9,106.3,107.3,107.8,107.9,
              107.5,107.2,107.5])
A = np.array([1000, 1040, 1060.8, 1064.9])

#the average of every 4 in the data annual data
B = denton.proportional_method(I, A)

#to replicate the table then divide by 4
B_imf = denton.proportional_method(I, A)/4

print(B_imf)

```

    [[247.47624703]
     [248.38181462]
     [250.44888312]
     [253.69305523]
     [257.37943434]
     [259.40742807]
     [261.02059637]
     [262.19254122]
     [262.88387148]
     [264.79745537]
     [266.21069991]
     [266.90797325]
     [267.15445131]
     [266.16323935]
     [265.41990401]
     [266.16240533]]

```python

```