# Chapter 1: Math Tools I (Vectors, Complex Numbers and Dirac Notation)

The aim of this chapter is not to explain the basics of quantum information theory or quantum mechanics. It is rather directed towards introducing the language that both of these fields are built on, linear algebra. Both Chapter 1 and Chapter 2 attempt to establish the mathematical foundation necessary to keep up with this (fairly introductory) series. The module that will be used to demonstrate applications to Python is  Numpy.

**Note:** While I do attempt to survey these topics briefly enough for one to grasp everything else mentioned in throughout the 8 Chapters, a solid understanding of both vector and matrix algebra is much advised.

  ##  1.1 Complex Numbers
  A complex number is a number which takes the following form:

  \begin{equation}
  c = a + ib
  \end{equation}
  where $a$ and $b$ are real numbers and $i = \sqrt{-1}$. This type of number represents a real component ($Re(c) = a)$ and an imaginary component ($Im(c) = b$).

  One can say that a complex number behaves exactly like a 2D vector, which raises interesting properties such as the ability to describe a complex number as a point in a 2D complex plane (_i.e_ a plane where the x-axis represents $a$, and the y-axis represents $ib$).

  For example, let us say we have the number, $c = a+bi$. That could be represented by the following:

  Another thing one can notice is that both real numbers and imaginary numbers are special cases of complex numbers. A real number is where $b = 0$ and an imaginary is where $a = 0$

  ### Properties of a complex number

  A complex number has two essential properties, Its length and its angle. Looking at the figure above, it should be easy to see that the length of a complex number is $|c| =  \sqrt{a^2 + b^2}$ (using the pythagorean theorem).

  The angle formed with the x-axis (also called the argument) can be obtained using basic trigonometry. One would find that its value is $\theta = \tan^{-1}(b/a)$.

  ### Operations on complex numbers

  Let's say we have 2 complex numbers, $c_1 = a + ib $ and $c_2 = d + ie$, and we want to apply basic mathematical operations on these two numbers. One would do that via the following methods:

  **Addition:**

  $c_3 = c_1 + c_2$

  $c_3 = a + ib + d + ie$

  $c_3 = (a+d) + i(b+e)$

  **Subtraction:**

  $c_3 = c_1 - c_2$

  $c_3 = a + ib - d - ie$

  $c_3 = (a-d) + i(b-e)$

   **Multiplitcation:**

  $c_3 = c_1 * c_2$

  $c_3 = (a + ib) * (d + ie)$

  $c_3 = ad + iae + ibd -be$

  $c_3 = (ad-be) + i(ae+bd)$

   **Division:**

  $c_3 = c_1 / c_2$

  $c_3 = \frac{a + ib}{d + ie}$

  $c_3 = \frac{a + ib}{d + ie} \frac{d - ie}{d - ie}$

   $c_3 = \frac{(ad+be) + i(ae-bd)}{d^2 + e^2}$

   ### Python Implementation

```python
#Importing numpy, a module very useful for all things math!
import numpy as np

#Python syntax for 1+2i
c1 = complex(1,2)

#Getting length and angle
length_c1 = abs(c1)
angle_c1 = np.angle(c1)

#Real and imaginary components

Re = c1.real
Im = c1.imag

print("The Length and Angle of " + str(c1), "is", str(length_c1), "and", str(angle_c1))
print('Re: ', Re)
print('Im: ', Im)
```

    The Length and Angle of (1+2j) is 2.23606797749979 and 1.1071487177940904
    Re:  1.0
    Im:  2.0

```python
#Operation on 2 complex numbers

c1 = complex(1,2)
c2 = complex(3,4)

add = c1 + c2
sub = c1 - c2
mult = c1*c2
div = c1/c2

print("Add:", str(add))
print("Subtract:", str(sub))
print("Multiply:", str(mult))
print("Divide:", str(div))
```

    Add: (4+6j)
    Subtract: (-2-2j)
    Multiply: (-5+10j)
    Divide: (0.44+0.08j)

##  1.2 Complex Conjugation

The concept of a complex conjugate is very simple, yet it seems to have a very important role in the usage of complex numbers. All the complex conjugate does is it transforms $i$ to $-i$. For example, if we have a complex number $c = a + ib$, its conjugate will be $c^* =  a-ib$. Visually, this can be described as flipping a point in the complex plane upon the x-axis, as shown bellow:

Looking at the Figure above, one can see that $|c| = |c^*|$ and $\phi_c = - \phi_{c^*}$.

  ### Rules of Conjugation

 **Product of complex number and its conjugate:**

  $cc^* = (a + ib)(a - ib)$

  $cc^* = (a^2 + iab - iab + b^2)$

  $cc^* = a^2 + b^2$

  $cc^* = |c|^2$

  **Product of Conjugates:**

  $(c_1 c_2)^* = c_1 ^* c_2 ^*$

  **Sum of Conjugates:**

  $(c_1 + c_2)^* = c_1 ^* + c_2 ^*$

  **Conjugate of Conjugates:**

  $c^{**} = c$

  ### Python Implementation

```python
#One can get the conjugate of a complex function using np.conj(). Let's say c2 = c1*

c1 = complex(1,1)
c2 = np.conj(c1)
print('c1: ', str(c1))
print('c1*: ', str(c2))

#Checking to see if product of complex number and conjugate is is length^2

cc_conj = c1*c2

if abs(cc_conj) == int(abs(c1)**2):
    print("Math works! c*c = |c|^2")

#Excercise: Test out other rules of conjugation
```

    c1:  (1+1j)
    c1*:  (1-1j)
    Math works! c*c = |c|^2

   ##  1.3 Vector Space

Before continuing with the mathematical pre-requisites of basic Quantum Information Theory, allow me to be honest about something. Writing this chapter is very difficult for me. Not because of the high level of technicality required to write about such topics, but rather because I am sacrificing a lot of interesting and beautiful ideas in exchange of practicality and rapidity so that we can move on to the topic of interest on an "ASAP basis". The topics of vector spaces and abstract algebra are infinitely more deep and compelling than what I paint their picture to be, so as I said in the introduction, I highly advise looking at other resources. Ok, enough with my rant and back to let's get back to oversimplifying the math.

There are several ways of defining a vector. One might introduce them as a Rank 1 Tensor, or as a quantity that has magnitude and direction, but for the sake of this chapter, we shall assume that you understand the basics and introduce it as an element of a vector space. A vector space is a collection of not only vectors, but a field of scalars and two operations, vector addition and scalar multiplication. The number of dimensions of a vector space can vary, and is based on the number of elements a vector in that space has. For instance, classically speaking, the position of a particle is usually described in a 3D vector-space with vector of form $\vec{r}(x,y,z)$, while relativistically speaking one would rather use what's called a four-vector to describe a particle's dynamics. A four-vector takes the following form: $\vec{R}(x,y,z,ct)$ or simply $\vec{R}(\vec{r},ct)$ (where $c$ is the speed of light and $t$ is time). This is what is meant by space-time. The reason I mention the relativistic case is to show that a vector space can also be a subspace of another vector space. In QIT, vector spaces will be used not to describe a particle's position, but rather the probabilities of qubits being in certain states. If this is not apparent now, it should be in Chapter 4.

### Operations

**Vector Addition:** Vector addition is the proccess of adding the $i$th component of vectors (i.e $[x_1, x_2, x_3,...x_i] + [y_1,y_2,y_3,...y_i]$ = $[x_1 + y_1, x_2 + y_2,x_3 + y_3...x_i + y_i]$).

 Given that u and v are vectors in vector space **V**,

  1. $u + v$ is also a vector in **V**
  2. $u + v = v + u$
  3. $(u + v) + w = u + (w + v)$
  4. There is a $\vec{0}$ vector in **V**, where $u + \vec{0} = u$
  5. For every $u$ in **V**, there is another vector $-u$, where $u + (-u) = 0$

  **Scalar Multiplication:** Scalar multiplication is the process of multiplying every item in a vector by a scalar.  (i.e $c*[x_1, x_2, x_3,...x_i]$ = $[c*x_1, c*x_2, c*x_3,...c*x_i]$).

 Given that u and v are vectors in vector space **V** and c and d are  scalars in a field which is also in **V**,

  1. $c*u$ is also a vector in **V**
  2. $c(u + v) = cv + cu$
  3. $c(du) = d(cu)$
  4. $(c+d)u = cu + du$
  5. $1(u) = u$

### Linear combination

   We just stated that in a vector space **V**, there are two operations that can be acted on vectors $u$ and $v$, vector addition (i.e $u+v$) and scalar multiplication (i.e $au$ or $bv$), where $a$ and $b$ are scalars. Any operation which takes the form $ua + bv$ is what is known as a linear combination. Linear combinations are very crucial to the topic of quantum information theory because it is essentially how one represents a superposition, but that will be further discussed later on.

### Python Implementation

```python
#What not to do: represent a vector by a python list.
#Let's see why:

print('List As a vector:')
notAVector1 = [1,2,3]
notAVector2 = [1,2,3]
scalar = 3

vecAdd = notAVector1 + notAVector2
scale = scalar*notAVector1
print("Doesn't satisfy the rules of vector addition: ", notAVector1, "+", notAVector2, "=", vecAdd)
print("Doesn't satisfy the rules of scalar multiplication: ", notAVector1, "*", scalar, "=", scale)

print('')

#What to do: Represent vectors as a 1D numpy array
#Let's see why:

print('1D Numpy array (np.array([vec])) as a vector:')
vector1 = np.array([1,2,3])
vector2 = np.array([1,2,3])
scalar  = 5
vecAdd = vector1 + vector2
scale = scalar*vector1

print("Satisfies the rules of vector addition: ", vector1, "+", vector2, "=", vecAdd)
print("Satisfies the rules of scalar multiplication: ", vector1, "*", scalar, "=", scale)

#Excercise: Use what we learned about vector spaces to build a python tool for complex numbers
#(which can be represented by 2D vectors)

```

    List As a vector:
    Doesn't satisfy the rules of vector addition:  [1, 2, 3] + [1, 2, 3] = [1, 2, 3, 1, 2, 3]
    Doesn't satisfy the rules of scalar multiplication:  [1, 2, 3] * 3 = [1, 2, 3, 1, 2, 3, 1, 2, 3]

    1D Numpy array (np.array([vec])) as a vector:
    Satisfies the rules of vector addition:  [1 2 3] + [1 2 3] = [2 4 6]
    Satisfies the rules of scalar multiplication:  [1 2 3] * 5 = [ 5 10 15]

   ##  1.4 Basis Set

   In the preceding sections, a vector was described as a set of coordinates in an $n$-dimensional vector space. However, That is not the only way to represent a vector. Another notable way to represent vectors is by viewing its components as scalars that act on what is called a set of unit vectors.

   For example, let us take the vector $\vec{v}(a,b,c)$. Using the basic laws of vector addition and scalar multiplication, we can show that $\vec{v}$ can be written as the following linear combination:

   \begin{equation}
   \vec{v}(a,b,c) = a(1,0,0) + b(0,1,0) + c(0,0,1)
   \end{equation}

where (1,0,0), (0,1,0), (0,0,1) are unit vectors. In 3 dimensions, unit vectors conventionally denoted by $\hat{i}$, $\hat{j}$ and $\hat{k}$, but it could of course extent to any $n$-dimensional vector space, $V$.

Formally, a set of unit vectors is defined as an orthonormal basis of $R^n$ (an $n$-dimensional vector space where components can be any real number), but roughly speaking, they could be defined as any subset of $n$ vectors that are all orthogonal to eachother (ortho) and all have a length of 1 (normal). In $R^3$ (i.e the example given above), the unit vectors look like the following:

When first learning about unit vectors, one usually does the mistake of thinking that they are special in the sense that any vector could be written in terms of them. However, the truth is that they are nothing but a special case of a broader set of vectors which all possess the same quality, the quality of being a basis set. In vector space **V**, a subset of vectors are considered basis vectors if they satisfy the following:

       1. Any vector in V can be written as a linear combination of the vectors in the basis set
       2. No vector in the basis set can be expressed as a linear combination of the other vectors in the basis set

The only special quality that unit vectors possess is their general convenience in representing other vectors.

### Finding General Unit Vectors

Let's say you have an arbitrary n-dimensional vector, $\vec{v}$, and you want to find a vector that has the same direction as $\vec{v}$ but has a length of 1 (i.e its unit vector). You would have to do what is called normalizing the vector, which could be done by scalar multiplying $v$ with the reciprocal of its length, $|v|$. According to the pythagorean theorem, the length (or *norm*) of a vector can be calculated by the following,

\begin{equation}
|v| = \sqrt{\sum_{i = 1}^{n} v_i ^2}
\end{equation}

or the square root of the sum of the square of its components.

So in a 3 dimensional case, the unit vector of $\vec{v}$ would be

\begin{equation}
\hat{v} = \frac{\vec{v}(v_1,v_2,v_3)}{\sqrt{v_1 ^2 + v_2 ^2 + v_3 ^2}}
\end{equation}

A more general approach to find the norm of a vector is using a vector operation called the inner product, which for real vectors (i.e vectors that are in subspace $R^n$), can be written as the sum of the products of $i$-th components of the two vectors, or

\begin{equation}
\vec{v}.\vec{u} = \sum_{i = 1}^{n} v_i* u_i
\end{equation}

Notice that the first equation used to calculate the norm is essentially the square root of the inner product of a vector and itself, ergo $|v| = \sqrt{v.v}$

### Inner Product of Complex Vectors

In quantum computing, the main purpose of vectors is to represent the state of a qubit (or many qubits). One important thing to note before beginning quantum computing is that, for reasons that one might find very elegant, the formulation of quantum mechanics and concequently quantum computing is based on complex numbers and complex vectors rather than real numbers and vectors. A complex vector $\vec{v} \in C^N$ can be represented by the following

\begin{equation}
v = (v_1, v_2, v_3, ..., v_{n})
\end{equation}

where $v_i \in C$ (are complex numbers). The inner product of two vectors was previously defined to be the sum of the
product of their corrersponding $i$th components. However, with complex vectors, a more general definition must be given, which is the following:

\begin{equation}
\vec{u}.\vec{v} = \sum_{i=1}^{n} u^*_i v_i
\end{equation}

where $\vec{u}$ and $\vec{v}$ are vectors and $u^*_i$ is the complex conjugate of the $i$th component of $\vec{u}$. One can see that the inner product of real vectors definition stated above is equivelent to the new, general one presented now because $a^* = a$, given that $a \in R$. Also, the norm of a complex vector is now

\begin{equation}
|v| = \sqrt{\sum_{i = 1}^{n} v_i^*v_i} = \sqrt{\sum_{i = 1}^{n} |v_i|^2 }
\end{equation}

Using the first rule of conjugation described in Section 1.2.

In the next section, a more elegant and useful notation for dealing with complex vectors will be presented. One that is used ubiquitously throughout fields such as quantum mechanics, quantum field theory and most important to us, quantum information theory.

### Python Implementation

```python
#Finding the Unit Vector of a vector, v

v = np.array([1,2,3])

norm = 1/np.sqrt(1**2 + 2**2 + 3**2)

unitV = norm*v

#Getting the length of unitV. If the math is right, it should be 1
lengthUnitV = np.array([i**2 for i in unitV]).sum()

#Results
print(unitV, 'is the normalized unit vector of v')

if lengthUnitV == 1:
    print('Math works! Unit vector length is 1')

#Excercise: Create a function that finds the unit vector of any given n-dimensional vector
#Hint: generalize what was done to obtain the normalizing factor, norm

```

    [0.26726124 0.53452248 0.80178373] is the normalized unit vector of v
    Math works! Unit vector length is 1

```python
#Inner Product. It's the same numpy function for any set of two vectors

#v and u ∈ R^3

v = np.array([5,1,3])
u = np.array([4,2,5])

inner_r3 = np.inner(u,v)

print('Inner product of', v, ' and ', u, ' is ', inner_r3)

#v and u ∈ C^4

v = np.array([2j, 2+3j, 3, 0])
u = np.array([1+1j, 2+4j, 1, 1j])

inner_c4 = np.inner(u,v)

print('Inner product of', v, ' and ', u, ' is ', inner_c4)

#Excercise: Create a function that gets the norm of a  complex vector. Bonus if you don't use np.inner()

def normalize(vector):
    return unitVector

```

    Inner product of [5 1 3]  and  [4 2 5]  is  37
    Inner product of [0.+2.j 2.+3.j 3.+0.j 0.+0.j]  and  [1.+1.j 2.+4.j 1.+0.j 0.+1.j]  is  (-7+16j)

   ##  1.5 Dirac Notation

  Anyone who has taken a course in physics knows that vectors are a very powerful tool for representing systems. In quantum mechanics and quantum computing, a vector is used to represent the state of a system (i.e a wavefunction/statevector). The vector notation that is generally used to describe such quantities is usually an idiosyncratic notation called Dirac Notation (or Bracket Notation). One might ask, "Why use a different notation? The ones used in the earlier sections and generally throughout math and physics seem fine enough. What benifit do we get out of using this new notation?" The reason Dirac Notation is used is because as you'll see in the later chapters, quantum states get messy to describe. Filled with hermition adjoints, tensor products, and hefty integrals and sums, it eventually gets difficult to keep track of everything that's going on. Dirac Notation does a good job of making it simpler to do all the manipulations required while explicitly separating the state from its dual (something we'll speak about later on).

  There are two main components in Dirac Notation, a bra vector $$. In simple terms, a ket is represented by a column matrix, i.e

  \begin{equation}
  |u> = \begin{pmatrix}u_1 \\\ u_2 \\\ \vdots \\\ u_n \end{pmatrix}
  \end{equation}

  and a bra is what is called the Hermitian Conjugate of a ket. Hermitian Conjugation is the act of taking the transpose of the complex conjugate of a matrix. While we have spoken about complex conjugation, we haven't yet defined what a transpose is. A transpose is an operation on a matrix that applies the following transformation:

  \begin{equation}
  M_{ij} \rightarrow M_{ji}
  \end{equation}

  where $M$ is a matrix and $i$ and $j$ represent the row and column index, respectively. Given that a ket vector is a column matrix, one can notice that the transpose of a ket vector is essentially a row matrix ($M_{1j} \rightarrow  M_{j1}$). And since a Hermitian conjugate, denoted by $M^{\dagger}$ does the following,

  \begin{equation}
  M^{\dagger} = M^{T*}
  \end{equation}
  one can find that a bra vector, $<u|$, takes the following form:

  \begin{equation}
  <u| = \begin{pmatrix} u_1^* & u_2^* & \ldots & u_n^* \end{pmatrix}
  \end{equation}

  So, in simple terms, a bra vector is a column matrix that contains the complex conjugates of the components of a ket matrix (which represents the state of the quantum system).

  So we defined what a bra is, and what a ket is, but what is a bra(c)ket? Say we have vectors $\vec{u}$ and vectors $\vec{v}$, where both $\vec{u}$ and $\vec{v} \in C^n$, the bracket would be evaluated as follows:

  \begin{equation}
   = \begin{pmatrix} u_1^* & u_2^* & \ldots & u_n^* \end{pmatrix} * \begin{pmatrix}v_1 \\\ v_2  \\\ \vdots \\\ u_n \end{pmatrix}
  \end{equation}
  According to matrix multiplication, this expands to
  \begin{equation}
   = u_1^*v_1 + u_2^*v_2 + \ldots + u_n^*v_n
  \end{equation}

  Or more generally,
  \begin{equation}
   = \sum_{i=1}^n u_i^*v_i
  \end{equation}

  This result is interesting because, as shown in Section 1.4, this is essentially the inner product of vectors $\vec{u}$ and $\vec{v}$. From that, we can derive that the norm of vector $\vec{u}$ could be described by

  \begin{equation}
  |\vec{u}| = \sqrt{}
  \end{equation}

  ### Ket Algebra

  Given that a ket is essentially just a representation of a vector, all the vector axioms and identities presented earlier apply in the same way. That is,

   #### Ket Addition
      1.
  \begin{equation}
  |\phi> + |\psi> = |\psi> + |\phi>
  \end{equation}

      2.
  \begin{equation}
  (|\psi> + |\phi>) = |\omega> = (|\psi> + |\omega>) + |\phi>
  \end{equation}

   #### Scalar Multiplication
       1.
  \begin{equation}
  (c_1 + c_2)|\psi> = c_1|\psi> + c_2|\psi>
  \end{equation}

       2.
  \begin{equation}
  c_1(c_2)|\psi> = c_2(c_1)|\psi>
  \end{equation}

       3.
  \begin{equation}
  (c_1)(|\psi> + |\phi>) = c_1|\psi> + c_1|\phi>
  \end{equation}

       4.

  \begin{equation}
  (1)|\psi> = |\psi>
  \end{equation}

       5.
  \begin{equation}
  |\psi> + 0 = |\psi>
  \end{equation}

 ### Application to Quantum Computing

 As we'll speak about in more detail throughout the later chapters, a quantum computer's state is essentially a linear combination of two vectors, the $|0>$ vector (which represents the 0 state) and the $|1>$ vector (which represents the 1 state). As we stated before, a linear combination can be expressed as follows:

 \begin{equation}
 |\psi> = c_1|0> + c_2|1>
 \end{equation}

 where $c_1$ and $c_2$ $\in C$. The two constants represent a quantity called the amplitude, which when squared, gives the probability of getting the state associated with it. In other words, $c_1^2$ is the probability of getting a 0 and $c_2^2$ is the probability of getting a 1. And since there's a 100% probability of getting either a 0 or a 1, then the following must be true

 \begin{equation}
 |c_1|^2 + |c_2|^2 = 1
 \end{equation}

This is essentially the language of quantum information theory, kets and amplitudes. Anything that changes the amplitudes is called an operator, but that's a lesson for another chapter.

 ### Python Implementation

```python
#Defining a bra and a ket using np.matrix

#In np.matrix(), the elements of the matrix are inputed as a string a ";" creates a new row and a " " creates a new column

print('Case 1, Real Vectors:')
#If the vectors are real, then the complex conjugate of the vector is the vector itself. So the hermitan conj is just the transpose

ket = np.matrix('1 ; 2 ; 3 ; 4')
bra = np.matrix('1 2 3 4')

bracket = bra*ket

print('|u> = \n ', ket)
print('')
print('<u| = ', bra)
print('')
print('<u|u> =', int(bracket))

print('')
print('Case 2, Complex Vectors:')

#To apply a hermitian operator automatically, one can use np.matrix()'s function, getH(). For example,

ket = np.matrix(' 1j; 0 ; 3j ; 2')
bra = ket.getH()

bracket = bra*ket

print('')
print('|u> = \n ', ket)
print('')
print('<u| = ', bra)
print('')
print('<u|u> =', int(abs(bracket)))

#Excercise: Create a function that takes in 2 np.array inputs,turn one into a bra and other into a ket and evaluate the inner product

```

    Case 1, Real Vectors:
    |u> =
      [[1]
     [2]
     [3]
     [4]]

    <u| =  [[1 2 3 4]]

     = 30

    Case 2, Complex Vectors:

    |u> =
      [[0.+1.j]
     [0.+0.j]
     [0.+3.j]
     [2.+0.j]]

    <u| =  [[0.-1.j 0.-0.j 0.-3.j 2.-0.j]]

     = 14

   ##  1.6 Inner Product

   The inner product of two vectors is something that was discussed briefly throughout the five previous sections but there are a few crucial ideas which haven't been discussed yet. As mentioned before, an inner product can be represented by a bra-ket, $$. The result of an inner product will always be a complex number, and the inner product must follow the following identities:

       1. Linearity:

\begin{equation}
<\psi|(a|\phi> + b|\omega>)  = a<\psi|\phi> + b<\psi|\omega>
\end{equation}

       2. Symmetry:

\begin{equation}
<\psi|\phi> = <\phi|\psi>^*
\end{equation}

       3. Positivity:

\begin{equation}
<\psi|\psi> \geq 0 (|\psi> \neq 0)
\end{equation}

Any vector space that both satisfies the axioms stated in the first few sections and is equipped with the inner product operation that satisfies the three identities stated above is called a Hilbert Space. In quantum mechanics, the wavefunction is a vector in an infinite dimensional Hilbert Space and in quantum information theory, a qubit's statevector is a vector in a 2D hilbert space (which takes the form [$c_1 c_2$], where $c_1$ and $c_2$ are the amplitudes described in Section 1.5).

The purpose of vectors in quantum mechanics is to represent the amplitudes of every possible state in the system. The square of this amplitude vector represents the probability distribution of every possible states. It should be obvious that the sum of all probabilities should always equal 1, meaning that the sum (or more specifically the norm) of the amplitude vector squared ($|\psi^2|$) must equal 1. Using dirac notation, this could be expressed as

\begin{equation}
<\psi|\psi> = 1
\end{equation}

Very often, one would find that a system's amplitude vector (i.e wavefunction/statevector) does not satisfy the relationship above, which is where the process of normalization, which was described in Section 1.4, comes into play. To normalize a vector (i.e set its norm to 1), one must scalar multiply $|\psi>$ with the normalizing constant $\frac{1}{\sqrt{<\psi|\psi>}}$, i.e

\begin{equation}
|\psi> \rightarrow \frac{1}{\sqrt{<\psi|\psi>}} |\psi>
\end{equation}

Another notable result one can obtain is the result of a bra-ket being zero. For instance, when

\begin{equation}
 = 0
\end{equation}

then vectors $|u>$ and $|v>$ are orthogonal, or form a 90$^\circ$ angle with eachother

```python
#Defining bra, ket, and bracket where |u> = (i,0,3i,2)

ket = np.matrix(' 1j; 0 ; 3j ; 2')
bra = ket.getH()

bracket = bra*ket

print("|u> unnormalized:\n ", ket)
print('Norm:', int(abs(bracket)))

print('')

#Normalizing it

norm_const = 1/np.sqrt(bracket)

ket*=norm_const
bra = ket.getH()

bracket = bra*ket

print("|u> normalized:\n ", ket)
print('Norm:', int(abs(bracket)))
print('\nIf this was a statevector, the net probability would be 1, so the system makes sense!')

#Excercise: Make a function that checks to see if two ket vectors are orthogonal

```

    |u> unnormalized:
      [[0.+1.j]
     [0.+0.j]
     [0.+3.j]
     [2.+0.j]]
    Norm: 14

    |u> normalized:
      [[0.        +0.26726124j]
     [0.        +0.j        ]
     [0.        +0.80178373j]
     [0.53452248+0.j        ]]
    Norm: 1

    If this was a statevector, the net probability would be 1, so the system makes sense!

   ##  1.7 Outer Product

   A curious reader might ask something such as "If a bra-ket gives the inner product of two vectors, what does a ket-bra give?" The answer is rather simple to find using, again, matrix multiplication. Say we have vectors $|u>$ and $|v>$, the ket-bra could be written out as follows

   \begin{equation}
   |u><v| =   \begin{pmatrix}u_1 \\\ u_2  \\\ \vdots \\\ u_n \end{pmatrix}* \begin{pmatrix} v_1^* & v_2^* & \ldots & v_n^* \end{pmatrix}
   \end{equation}

   The output of this matrix multiplication will be the following

   \begin{equation}
  |u><v| = \begin{pmatrix}v_1^*u_1 & v_2^*u_1 & \ldots & v_n^*u_1 \\\ v_1^*u_2 & v_2^*u_2 & \ldots & v_n^*u_2 \\\ \vdots & \vdots & \ddots &\vdots \\\ v_1^*u_n & v_2^*u_n & \ldots & v_n^*u_n\end{pmatrix}
   \end{equation}

   As you can clearly see, unlike a bra-ket, the result is a matrix. This matrix is what is known as the density matrix in quantum mechanics and is used as a projection operator. Let's say we have a density matrix $M = |v> = (|v> = |v>
   \end{equation}

   The outer product is also equivilent to the tensor product (denoted by $\vec{u}\otimes \vec{v}$) and will prove to be a very crucial operation later on, especially when studying how the state evolves.

   ### Python Implementation

```python
# |u><u|, where |u> = (i,i,1,i)

ket = np.matrix('1j ; 1j ; 1 ; 1j')
bra = ket.getH()

outer = ket*bra
print('|u><u| =')
print('')
print(outer)
print('')
print('The dimensions are',outer.shape, '(i.e n by n)')
```

    |u><u| =

    [[1.+0.j 1.+0.j 0.+1.j 1.+0.j]
     [1.+0.j 1.+0.j 0.+1.j 1.+0.j]
     [0.-1.j 0.-1.j 1.+0.j 0.-1.j]
     [1.+0.j 1.+0.j 0.+1.j 1.+0.j]]

    The dimensions are (4, 4) (i.e n by n)

## Dirac Notation Python Module

Here, we will build a small Python Module based on the concepts from the past 3 sections. It allows one to use bras and kets smoothly in Python, where the initial input is a list. This isn't anything special and its sole purpose is to offer a summary for the things we've learned about Dirac Notation so far. This also answers all the excercise questions above.

### Dirac Module

```python
#ket(): The user inputs a python list that is then converted into a numpy array, which is then reshaped to be a column
def ket(vec):
    ket = np.array(vec)
    return ket.reshape(-1,1)

#bra(): The user inputs a ket vector (np array) that is then transformed into a bra vector

def bra(vec):
    #Turning the vector into a ket in case it isn't
    Ket = ket(vec)
    #Getting The conjugates of the components of the ket and inserting them into a row vector (i.e turning it to a bra)
    ket_conj = [np.conj(i)[0] for i in Ket]
    #Reformatting it into an array
    bra = np.array(ket_conj)
    return bra

#braket(): Gets the inner product by evaluating <u|v>

def braket(bra,ket):
    #Gets the sum of all the u*v values from i=1 to n, where n is the number of dimensions
    inner_prod = 0
    for i in range(len(bra)):
        inner_prod += bra[i]*ket[i]
    return inner_prod

#ketbra(): Gets the outer product by evaluating |u><v|
def ketbra(bra,ket):
    return ket*bra

#normalize(): normalizes a ket

def normalize(ket):
    #Gets the bra
    Bra = bra(ket)
    #Gets the inner product
    norm = braket(Bra,ket)
    #Returns the normalized vector
    return 1/np.sqrt(norm)*ket

#isOrthogonal(): Checks if 2 vectors are orthogonal
def isOrthogonal(ket1, ket2):
    #Turns ket1 into a bra
    bra1 = bra(ket1)
    #Gets the inner product of bra1 and ket2
    inner = braket(bra1,ket2)
    #If inner == 0, Returns True, else, returns false
    return bool(inner == 0)
```

### Testing Dirac Module

```python
#Defining Two vectors (as lists)
u = [1+1j, 0, 1,-3]
v = [2j, 0, 0, 3+1j]

#Turning them into bras and kets
k = ket(u)
b = bra(v)

#Getting <v|u> and |u><v|
inner = braket(b,k)
outer = ketbra(b,k)

#Normalizing |u>
u_norm = normalize(k)

#Checking if |u> and |v> are orthogonal

orthog_checker = isOrthogonal(k,b)

#Outputs:

print('|u> =\n',k)
print('')
print('<v| =', b)

print('')

print('<v|u> =', inner)
print('')
print('|u><v| =\n', outer)

print('')

print('Normalized |u> =\n ', u_norm)

print('')

print('Are |u> and |v> orthogonal? (T/F):', orthog_checker)

```

    |u> =
     [[ 1.+1.j]
     [ 0.+0.j]
     [ 1.+0.j]
     [-3.+0.j]]

    <v| = [0.-2.j 0.-0.j 0.-0.j 3.-1.j]

    <v|u> = [-7.+1.j]

    |u><v| =
     [[ 2.-2.j  0.+0.j  0.+0.j  4.+2.j]
     [ 0.+0.j  0.+0.j  0.+0.j  0.+0.j]
     [ 0.-2.j  0.+0.j  0.+0.j  3.-1.j]
     [ 0.+6.j  0.+0.j  0.+0.j -9.+3.j]]

    Normalized |u> =
      [[ 0.28867513+0.28867513j]
     [ 0.        +0.j        ]
     [ 0.28867513+0.j        ]
     [-0.8660254 +0.j        ]]

    Are |u> and |v> orthogonal? (T/F): False

   ##  1.8  Linearly Dependent and Independent Vectors

   A set of vectors are said to be linearly independent if no vector in that set could be written as a linear combination of the other vectors. Let's say we have a vector-set **V** = {$v_1,v_2,v_3,...v_n$}, to be linearly independent, the only solution to the following equation

   \begin{equation}
  c_1 v_1  + c_2 v_2  + c_3 v_3  + \ldots c_n v_n  = \sum_{i=1}^n c_i v_i  = 0
   \end{equation}

   must be

   \begin{equation}
   c_1 = c_2 = c_3 = \ldots = c_n = 0
   \end{equation}

   If the equation above is satisfied when any $c$ value $\neq 0$, then the vectors are linearly dependent.

   ##  1.9 New Description of Basis Sets, Orthonormality and Dual Spaces

   In Section 1.4, a rudimentary definition of a $\textit{basis set}$ was given. The two conditions for a vector set  {$|v_1>,|v_2>,|v_3>, \ldots, |v_n>$} to be a basis set of subspace **V**  were the following:

    1. Any vector in V can be written as a linear combination of the vectors in the basis set
    2. No vector in the basis set can be expressed as a linear combination of the other vectors in the basis set

  Now that more advanced concepts have been laid out, a new, more formal definition can be introduced. That definition is that a basis set must

    1. Any vector in subspace V can be described as a linear combilation of the vectors in the set. If this condition is applied, the set is said to span the space. This would be described notationally as follows:

 \begin{equation}
 |\psi> = \sum_{i=1}^n c_i |v_i>
 \end{equation}
where $|\psi>$ is any vector in **V**

    2. The vectors in the set are linearly independent (Section 1.8)

    3. The vector set is complete, meaning that no additional basis set is required to describe any possible state of the system

In Section 1.4, a vector set was said to be orthonormal if their inner product is zero and the norm of each vector was 1. Using a function called the Kronecker-Delta, one can write a single expression that describes both of these conditions, namely a vector set {$v_1, v_2,\ldots, v_n$} is orthonormal if

\begin{equation}
<v_i|v_j> = \delta_{ij}
\end{equation}

where $\delta_{ij}$ is the Kronecker-Delta function, which is defined as

\begin{equation}
\delta_{ij} =
    \begin{cases}
            1, &         \text{if } i=j,\\
            0, &         \text{if } i\neq j.
    \end{cases}
\end{equation}

where $i$ and $j$ are integers between 1 and $n$ that represent the index of the two vectors undergoing the inner-product. The reason this works is because when $i \neq j$, the inner product of two different vectors must be zero (orthogonal) and when $i=j$, then the inner product $<v_i|v_j>$ represents the norm of $v_i$ which must be one (normalized) (i.e orthonormal)

The concept of a dual space is fairly detailed, but for the sake of this text all you need to know is that a bra vector $<\psi|$ is the dual of the ket vector $|\psi>$.

### Python Implementation

```python
#Kronicker-Delta Function which takes a list of numpy arrays

def kronDelta(vecSet):
    #Creating an empty list that will soon be filled with all kronicker-delta values for a vectorset
    delta_list = []
    #Setting condition: for every value of i and j, if i=j, 1, if i=/=j, 0
    for i in range(len(vecSet)):
        for j in range(len(vecSet)):
            if i==j:
                delta=1
            else:
                delta = 0
            #Appending every result of the KD function to the empty list defined above
            delta_list.append(delta)
    #Returning the list
    return delta_list

#Orthonormal Checker:takes the inner product of every 2 vectors in the set and compares it to the value obtained by KD

def orthonormalCheck(vecSet):
    #Gets the KD list of vecSet
    kdResult = kronDelta(vecSet)
    #Empty list that will soon be filled with all inner product values
    innerProds = []
    #Finding the all inner products of vecSet
    for i in range(len(vecSet)):
        for j in range(len(vecSet)):
            inner = np.inner(vecSet[i],vecset[j])
            #Appending every inner product result to the empty list defined above
            innerProds.append(inner)
    #Condition: if the inner-products of vecSet equals the KD of vecSet, True, else, False
    if innerProds == kdResult:
        return True
    else:
        return False

#Testing our functions on the unit vectors of R^4. This should obviously yield True

vecset = [np.array([1,0,0,0]), np.array([0,1,0,0]), np.array([0,0,1,0]), np.array([0,0,0,1])]

KD = kronDelta(vecset)

isOrthonormal = orthonormalCheck(vecset)

print('Kronicker-Delta Function of Unit Vectors of R^4:', KD)
print('Are they orthonormal? (T/F): ', isOrthonormal)

#Challenge: Build a function that shows whether a vector set forms a basis set of C^n
```

    Kronicker-Delta Function of Unit Vectors of R^4: [1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1]
    Are they orthonormal? (T/F):  True

   ##  1.10 Statevectors

You just went through a crash course of basic vector algebra and probably want a quick answer as to how you will use all of this stuff before having to go through the other two prerequisite chapters. I understand your concern, so here is your quick introduction to quantum computing:

A quantum computer is any device which, in one way or another, utilizes the laws of quantum mechanics to perform logical operations. As we will show in the next chapter, a key property of quantum mechanics is that the physical systems it describes are intrinsically random. In other words, they are not in a definite state, but rather in a linear combination of all possible states. This is what is known as superposition. As hinted throughout this section, one would use a vector called the statevector to describe the probability of finding a particle in any given state. The number of dimensions of the statevector is equal to the number of possible states a system can be in. This statevector is represented by a ket as follows:

\begin{equation}
|\Psi> = \begin{pmatrix}\psi_1 \\\ \psi_2  \\\ \vdots \\\ \psi_n \end{pmatrix}
\end{equation}

Where $\psi_i$ is a complex number that represents the a quantity called the amplitude for the $i$th state (where $i$ is between 1 and $n$). The importance of the amplitude is that when squared, it yields the probability of the system being in state $i$. Another way to represent this is by treating the main statevector |$\Psi$> as a linear combination of basis vectors in an n-dimensional Hilbert space (as you can see, around half of the concepts we learned came up in this one sentence... so read it a few times). This would look like this:

\begin{equation}
|\Psi> = \psi_1 |1> + \psi_2 |2> + \ldots + \psi_n |n> = \sum_{i=1}^n \psi_i|i>
\end{equation}

   As stated above, $|\psi_1|^2$ is the probability of finding a system in state 1, $|\psi_2|^2$ is the probability of finding a particle in state 2 and so on. From there, it would be easy to deduce that the sum of all the amplitudes squared, or $\sum_{i=1}^n |\psi_i|^2$ must be equal to the total probability of the system, which should be 1. However, remember that according to Section 1.2, $|\psi_i|^2 = \psi_i^* \psi_i$, which means that the sum of all amplitudes multiplied by their corresponding conjugates must equal 1. Recall that this essentially means that the norm of the vector is 1, which, when written out in Dirac Notation, takes the following form:

   \begin{equation}
   <\Psi|\Psi> = 1
   \end{equation}

 I know what you're thinking. "Ok, Adam... all this vector stuff sounds cool but what do you mean by state? State of what?". State could mean several things. It could mean position, momentum, or any property of the system. It is just a matter of giving the statevector context about that state (i.e defining what propery to apply it to). If one would want to study the amplitude distribution of a particle's position $x$, then they have to evaluate the following inner product:

 \begin{equation}
 <x|\psi>
 \end{equation}

 And similarly for momentum $p$, $
$. These inner products yield what is known as the wavefunction, which describe a system's amplitude as a function of the physical quantity (e.g $\psi(x)$ for position).

 ### Qubit

 The segment above was a (frankly poor) introduction to what a statevector and wavefunction mean in quantum mechanics. But how does that explain what a quantum computer does, or equally importantly, how quantum information is represented? In a classical computer, information is simply represented by two possible **definite** states: a 0 or a 1. Anything that is in one of those states is called a bit. The function of a classical computer is to manipulate a series of bits according to its given instruction in order to create either cool things like GTA 5, or uncool things like TikTok.

 However, as described above, the principles of quantum mechanics are inherently random (non-deterministic), meaning that information (such as a bit) cannot be in a definite state, but rather in a linear combination of states defined by a statevector $|\psi>$. In this case, given that there are only two possible states a, 1 or 0, a quantum bit's (or for short, qubit's) quantum state will be represented by a superposition (linear combination) of state $|0>$ and state |1>. Technically speaking, this would mean that a qubit is a two level quantum system represented by a ket in 2D Hilbert space:

 \begin{equation}
 |\psi> = \alpha |0\rangle + \beta |1>
 \end{equation}

 where $\alpha$ and $\beta$ are amplitudes that are $\in C$, and $|0>$ and $|1>$ are basis vectors which can be represented by the following matrices:

 \begin{equation}
 |0> = \begin{pmatrix}1 \\\ 0\end{pmatrix} and |1> = \begin{pmatrix}0 \\\ 1\end{pmatrix}
 \end{equation}

 If one were to expand $|0>$ and $|1>$ into their matrix representation in the superposition expression described above, then one would get the following:

 \begin{equation}
 |\psi> = \alpha \begin{pmatrix}1 \\\ 0\end{pmatrix} + \beta \begin{pmatrix}0 \\\ 1\end{pmatrix} \newline = \begin{pmatrix}\alpha \\\ 0\end{pmatrix} + \begin{pmatrix}0 \\\ \beta \end{pmatrix} \newline = \begin{pmatrix}\alpha \\\ \beta\end{pmatrix}
 \end{equation}

 I can go on and on about this topic but will have to stop here to save some information for the coming chapters

```python
#Go nuts here
zero = bra(np.array([1,0]))
one = ket(np.array([0,1]))

ketbra(zero,one)
```

    array([[0, 0],
           [1, 0]])

```python

```

Chapter 1 - Math Tools I (Vectors, Complex Numbers and Dirac Notation).ipynb


###### Content under Creative Commons Attribution license CC-BY 4.0, code under MIT license (c)2014 L.A. Barba, C.D. Cooper, G.F. Forsyth.

# Spreading out

Welcome to the second lesson of the course module: "_Spreading out: parabolic PDEs."_ We're studying the heat equation in one spatial dimension:

\begin{equation}
\frac{\partial T}{\partial t} = \alpha \frac{\partial^2 T}{\partial x^2}
\end{equation}

where $\alpha$ is the thermal diffusivity and $T$ is the temperature.

In the previous lesson, we reviewed the numerical solution of the 1D diffusion equation with a forward-time, centered-space scheme: an _explicit_ scheme. What does that mean?

The solution for $T$ at timestep $t^{n+1}$ was calculated using different combinations of $T$ values from the *previous* timestep $t^n$.  We have complete knowledge of the parts that feed into the solution update at each spatial point.

*Implicit* methods work differently: we will use more data from the "future" in the update, including several values of $T$ at $t^{n+1}$. This will make the scheme more difficult to apply, but there are several reasons why it may be worth the effort.

In lesson 1, we discussed two disadvantages of explicit methods: (1) boundary effects drag behind by one time step; (2) stability requirements constrain the time-step to very small values. Both of these issues are resolved by implicit schemes.

## Implicit schemes

Let's move things around a bit and try combining the Euler time step with an evaluation of the spatial derivative on the *updated* solution at $t^{n+1}$. The discretized form of the equation is now as follows:

\begin{equation}\frac{T_{i}^{n+1}-T_{i}^{n}}{\Delta t}=\alpha\frac{T_{i+1}^{n+1}-2T_{i}^{n+1}+T_{i-1}^{n+1}}{\Delta x^2}\end{equation}

The stencil for this discretization doesn't look anything like the other stencils we've used until now. Check it out.

.
#### Figure 1. Stencil of the implicit central scheme.

From the previous time-step, we only know $T_i^{n}$, but what about $T_i^{n+1}$, $T_{i-1}^{n+1}$ and $T_{i+1}^{n+1}$?  What can we do?

No need to panic! Let's start by putting what we *do know* on the right-hand side of the equation and what we *don't know* on the left. We get:

\begin{equation} -T_{i-1}^{n+1} + \left( 2 + \frac{\Delta x^2}{\alpha\Delta t}\right) T_{i}^{n+1} - T_{i+1}^{n+1} = T_{i}^{n}\frac{\Delta x^2}{\alpha\Delta t} \end{equation}

It looks like there are a lot of unknowns and just one equation!

What does it look like with $i=1$?

\begin{equation}-T_{0}^{n+1} + \left( 2 + \frac{\Delta x^2}{\alpha\Delta t}\right) T_{1}^{n+1} - T_{2}^{n+1} = T_{1}^{n}\frac{\Delta x^2}{\alpha\Delta t} \end{equation}

and $i=2$?

\begin{equation}-T_{1}^{n+1} + \left( 2 + \frac{\Delta x^2}{\alpha\Delta t}\right) T_{2}^{n+1} - T_{3}^{n+1} = T_{2}^{n}\frac{\Delta x^2}{\alpha\Delta t} \end{equation}

What about $i=3$?

\begin{equation}-T_{2}^{n+1} + \left( 2 + \frac{\Delta x^2}{\alpha\Delta t}\right) T_{3}^{n+1} - T_{4}^{n+1} = T_{3}^{n}\frac{\Delta x^2}{\alpha\Delta t} \end{equation}

Can you see the common element across equations? Here's a little help:

$T_{i}^{n+1}$ also appears in the equation for $T_{i-1}^{n+1}$ and $T_{i+1}^{n+1}$. We might have enough equations if we apply this for all $i$-values at the same time, don't you think? In fact, this is a linear system of equations for the unknown values $T_{i}^{n+1}$ on the spatial grid.

### What about boundary conditions?

Let's look at the boundary points of the example from the previous lesson with a Dirichlet BC at $x=0$ and a Neumann BC at $x=1$, discretizing with $N$ mesh points.

The value $T_0^{n+1}$ is known at every time-step from the BC, so putting all unknown terms on the left-hand side of the equation and the known values on the right side yields the following for the $i=1$ equation:

\begin{equation}-T_{2}^{n+1} + \left( 2 + \frac{\Delta x^2}{\alpha\Delta t}\right) T_{1}^{n+1} = T_{1}^{n}\frac{\Delta x^2}{\alpha\Delta t} + T_{0}^{n+1} \end{equation}

That was easy!

On the other hand, for $i=N-2$, the equation reads

\begin{equation} -T_{N-3}^{n+1} + \left( 2 + \frac{\Delta x^2}{\alpha\Delta t}\right) T_{N-2}^{n+1} - T_{N-1}^{n+1} = T_{N-2}^{n}\frac{\Delta x^2}{\alpha\Delta t} \end{equation}

The discretized Neumann boundary condition on the right side of the rod is

\begin{equation}\frac{T^{n}_{N-1} - T^{n}_{N-2}}{\Delta x} = q.\end{equation}

But we can just as easily write that at time step $n+1$ (the boundary conditions apply at every time-step):

\begin{equation}\frac{T^{n+1}_{N-1} - T^{n+1}_{N-2}}{\Delta x} = q.\end{equation}

Inserting the Neumann boundary condition in the equation for $i=N-2$ yields

\begin{equation} - T_{N-3}^{n+1} + \left( 1 + \frac{\Delta x^2}{\alpha\Delta t}\right) T_{N-2}^{n+1} = T_{N-2}^{n}\frac{\Delta x^2}{\alpha\Delta t} + \Delta x q \end{equation}

Make sure you work this out with pen and paper: it's important to recognize where these terms come from!

Now we can write the linear system of equations in matrix form as follows:

$$[A][x] = [b]+[b]_{b.c.}$$

where the matrix of coefficients $[A]$ is a sparse matrix—most of the matrix elements are zero—with three non-zero diagonals. We write below the system expanded out, so you can see the structure of the matrix, with $\sigma=\frac{\alpha\Delta t}{\Delta x^2}$:

\begin{align}\left[ \begin{array}{cccccc}
 \left(2 + \frac{1}{\sigma}\right) & -1 & 0 & \cdots & & 0 \\
 -1 & \left(2 + \frac{1}{\sigma}\right) & -1 & 0 & \cdots & 0 \\
 0 & & \ddots& & & \vdots \\
 \vdots & & & & \left(2 + \frac{1}{\sigma}\right)& \\
 0 & \cdots & & & -1 & \left(1 + \frac{1}{\sigma}\right) \end{array} \right]
 \cdot
 \left[ \begin{array}{c}
 T_1^{n+1} \\ T_2^{n+1} \\ \vdots \\ \\ T_{N-2}^{n+1} \end{array} \right]
 =
 \left[ \begin{array}{c}
T_1^n \frac{1}{\sigma} \\ T_2^{n}\frac{1}{\sigma} \\ \vdots \\ \\ T_{N-2}^{n}\frac{1}{\sigma} \end{array} \right]
 +
 \begin{bmatrix}
  T_0^{n+1}\\
  0\\\\
  \vdots\\\\
  0\\
  q\Delta x
 \end{bmatrix}
 \end{align}

Notice that the Dirichlet boundary condition adds only a term to the right-hand side of the system.  The Neumann boundary condition both adds a term to the right-hand side and modifies the matrix $[A]$.

### Problem set up

We'll re-use the problem from lesson 1: we have a graphite rod, with [thermal diffusivity](http://en.wikipedia.org/wiki/Thermal_diffusivity) $\alpha=1.22\times10^{-3} {\rm m}^2/{\rm s}$, length $L=1{\rm m}$, and temperature held at $T=100{\rm C}$ on the left end, $x=0$, and $0{\rm C}$ everywhere else intially. We'll compute the evolution of temperature on the length of the rod.

Let's start like we did in the previous lesson: import your libraries and set up the discretization.

```
import numpy
import matplotlib.pyplot as plt
%matplotlib inline
from matplotlib import rcParams
rcParams['font.family'] = 'serif'
rcParams['font.size'] = 16
```

```
L = 1.
nt = 100
nx = 51.
alpha = 1.22e-3

dx = L/(nx-1)

Ti = numpy.zeros(nx)
Ti[0] = 100
```

### Solving a linear system

We need to solve the linear system of equations written above to advance the solution in time. Luckily, we can rely on our friends from SciPy who have developed some nice linear solvers, so we don't need to write our own.

From `scipy.linalg`, let's import `solve`: a function to solve linear systems. Make sure to explore the documentation of [`scipy.linalg`](http://docs.scipy.org/doc/scipy-0.14.0/reference/linalg.html). We'll need to define our own custom functions to generate the coefficient matrix and the right-hand side of the linear system. You should carefully study the code below.

```
from scipy.linalg import solve
```

```
def generateMatrix(N, sigma):
    """ Computes the matrix for the diffusion equation with backward Euler
        Dirichlet condition at i=0, Neumann at i=-1

    Parameters:
    ----------
    T: array of float
        Temperature at current time step
    sigma: float
        alpha*dt/dx^2

    Returns:
    -------
    A: 2D numpy array of float
        Matrix for diffusion equation
    """

    # Setup the diagonal
    d = numpy.diag(numpy.ones(N-2)*(2+1./sigma))

    # Consider Neumann BC
    d[-1,-1] = 1+1./sigma

    # Setup upper diagonal
    ud = numpy.diag(numpy.ones(N-3)*-1, 1)

    # Setup lower diagonal
    ld = numpy.diag(numpy.ones(N-3)*-1, -1)

    A = d + ud + ld

    return A
```

```
def generateRHS(T, sigma):
    """ Computes right-hand side of linear system for diffusion equation
        with backward Euler

    Parameters:
    ----------
    T: array of float
        Temperature at current time step
    sigma: float
        alpha*dt/dx^2

    Returns:
    -------
    b: array of float
        Right-hand side of diffusion equation with backward Euler
    """
    b = numpy.zeros_like(T)

    b = T[1:-1]*1./sigma
    # Consider Dirichlet BC
    b[0] += T[0]

    return b
```

Next, we'll define a function that steps in time using the implicit central-space scheme.  Remember that for an implicit method, a step in time is performed by solving the entire linear system. This is a fundamental difference between implicit and explicit methods, and implies a considerable computational cost.

```
def implicit_ftcs(T, A, nt, sigma):
    """ Advances diffusion equation in time with implicit central scheme

    Parameters:
    ----------
    T: array of float
        initial temperature profile
    A: 2D array of float
        Matrix with discretized diffusion equation
    nt: int
        number of time steps
    sigma: float
        alpha*td/dx^2

    Returns:
    -------
    T: array of floats
        temperature profile after nt time steps
    """

    for t in range(nt):
        Tn = T.copy()
        b = generateRHS(Tn, sigma)
        # Use numpy.linalg.solve
        T_interior = solve(A,b)
        T[1:-1] = T_interior
        # Enforce Neumann BC (Dirichlet is enforced automatically)
        T[-1] = T[-2]

    return T
```

We solve the linear system for every time step, but the $A$ matrix does not change. Thus, you can generate it only once and then use it as an input parameter to the time-stepping function. Let's try this out!

```
sigma = 0.5
dt = sigma * dx*dx/alpha
nt = 1000

A = generateMatrix(nx, sigma)
```

It's always a good idea to check your matrix...

```
print A
```

    [[ 4. -1.  0. ...,  0.  0.  0.]
     [-1.  4. -1. ...,  0.  0.  0.]
     [ 0. -1.  4. ...,  0.  0.  0.]
     ...,
     [ 0.  0.  0. ...,  4. -1.  0.]
     [ 0.  0.  0. ..., -1.  4. -1.]
     [ 0.  0.  0. ...,  0. -1.  3.]]

Looks good!

Now, we apply the time iteration with our very own function `implicit_ftcs()`.

```
T = Ti.copy()
T = implicit_ftcs(T, A, nt, sigma)
```

Now plot the solution!

```
plt.plot(numpy.linspace(0,1,nx), T, color='#003366', ls='-', lw=3);
```

Not too impressive, this looks just like the result from *explicit* forward in time, centered in space for $\alpha\frac{\Delta t}{\Delta x^2} = \frac{1}{2}$.

But try $\alpha\frac{\Delta t}{\Delta x^2} = 5$, which violates the stability condition of the *explicit* scheme:

```
sigma = 5.0

T = Ti.copy()
A = generateMatrix(nx, sigma)

T = implicit_ftcs(T, A, nt, sigma)

plt.plot(numpy.linspace(0,1,nx), T, color='#003366', ls='-', lw=3);
```

**It didn't blow up!**

We were not able to use such a large time step with the explicit scheme. You can try out other values of `sigma` and you'll get a stable solution. In fact, this is an *unconditionally stable* scheme—the most valuable feature of implicit methods is that they give stable solutions without a constraint on the choice of time step.

Using the implicit scheme, we can always advance in time using larger time steps. But each time step requires the solution of a linear system, which is computationally expensive. This is the trade-off between explicit and implicit methods.

##### A word of warning

Implicit methods allow you to use significantly larger time steps, because they are not subject to stability constraints. But that doesn't mean you can use just _any_ large time step! Remember that Euler's method is a first-order method, so the _accuracy_ gets worse as you increase the time step, in direct proportion. In fact, you can lose the ability to capture the correct physics if your time step is too large. Numerical stability does not imply accuracy!

##### Dig deeper

You see how matrix `A` is mostly full of zeros?  We call such a matrix *sparse*, and there are many ways to make more efficient calculations taking advantage of their particular structure. First of all, you can optimize the memory usage. Check out SciPy's [sparse-matrix storage formats](http://docs.scipy.org/doc/scipy-0.14.0/reference/sparse.html): you don't need too store $(N-2)^2$ elements! For example, a `coo_matrix` format stores only $3*N_\text{nonzero}$, where $N_\text{nonzero}$ is the number of non-zero elements in `A`. Make sure to explore this topic a little more. It's an important topic in numerical PDEs.

---
###### The cell below loads the style of the notebook

```
from IPython.core.display import HTML
css_file = '../../styles/numericalmoocstyle.css'
HTML(open(css_file, "r").read())
```

<link href='http://fonts.googleapis.com/css?family=Alegreya+Sans:100,300,400,500,700,800,900,100italic,300italic,400italic,500italic,700italic,800italic,900italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Arvo:400,700,400italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=PT+Mono' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Shadows+Into+Light' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Nixie+One' rel='stylesheet' type='text/css'>
<style>

@font-face {
    font-family: "Computer Modern";
    src: url('http://mirrors.ctan.org/fonts/cm-unicode/fonts/otf/cmunss.otf');
}

#notebook_panel { /* main background */
    background: rgb(245,245,245);
}

div.cell { /* set cell width */
    width: 750px;
}

div #notebook { /* centre the content */
    background: #fff; /* white background for content */
    width: 1000px;
    margin: auto;
    padding-left: 0em;
}

#notebook li { /* More space between bullet points */
margin-top:0.8em;
}

/* draw border around running cells */
div.cell.border-box-sizing.code_cell.running {
    border: 1px solid #111;
}

/* Put a solid color box around each cell and its output, visually linking them*/
div.cell.code_cell {
    background-color: rgb(256,256,256);
    border-radius: 0px;
    padding: 0.5em;
    margin-left:1em;
    margin-top: 1em;
}

div.text_cell_render{
    font-family: 'Alegreya Sans' sans-serif;
    line-height: 140%;
    font-size: 125%;
    font-weight: 400;
    width:600px;
    margin-left:auto;
    margin-right:auto;
}

/* Formatting for header cells */
.text_cell_render h1 {
    font-family: 'Nixie One', serif;
    font-style:regular;
    font-weight: 400;
    font-size: 45pt;
    line-height: 100%;
    color: rgb(0,51,102);
    margin-bottom: 0.5em;
    margin-top: 0.5em;
    display: block;
}
.text_cell_render h2 {
    font-family: 'Nixie One', serif;
    font-weight: 400;
    font-size: 30pt;
    line-height: 100%;
    color: rgb(0,51,102);
    margin-bottom: 0.1em;
    margin-top: 0.3em;
    display: block;
}

.text_cell_render h3 {
    font-family: 'Nixie One', serif;
    margin-top:16px;
	font-size: 22pt;
    font-weight: 600;
    margin-bottom: 3px;
    font-style: regular;
    color: rgb(102,102,0);
}

.text_cell_render h4 {    /*Use this for captions*/
    font-family: 'Nixie One', serif;
    font-size: 14pt;
    text-align: center;
    margin-top: 0em;
    margin-bottom: 2em;
    font-style: regular;
}

.text_cell_render h5 {  /*Use this for small titles*/
    font-family: 'Nixie One', sans-serif;
    font-weight: 400;
    font-size: 16pt;
    color: rgb(163,0,0);
    font-style: italic;
    margin-bottom: .1em;
    margin-top: 0.8em;
    display: block;
}

.text_cell_render h6 { /*use this for copyright note*/
    font-family: 'PT Mono', sans-serif;
    font-weight: 300;
    font-size: 9pt;
    line-height: 100%;
    color: grey;
    margin-bottom: 1px;
    margin-top: 1px;
}

.CodeMirror{
        font-family: "PT Mono";
        font-size: 90%;
}

</style>


The Cobweb Model

Presentation follows Hommes, JEBO 1994.  Let $p_t$ denote the observed price of goods and $p_t^e$ the expected price of goods in period $t$. Similarly, let $q_t^d$ denote the quantity demanded of all goods in period $t$ and $q_t^s$ the quantity supplied of all goods in period $t$.

\begin{align}
    q_t^d =& D(p_t) \tag{1} \\
    q_t^s =& S(p_t^e) \tag{2} \\
    q_t^d =& q_t^s \tag{3} \\
    p_t^e =& p_{t-1}^e + w\big(p_{t-1} - p_{t-1}^e\big) = (1 - w)p_{t-1}^e + w p_{t-1} \tag{4}
\end{align}

Equation 1 says that the quantity demanded of goods in period $t$ is some function of the observed price in period $t$. Equation 2, meanwhile, states that the quantity of goods supplied in period $t$ is a function of the expected price in period $t$. Equation 3 is a market clearing equilibrium condition. Finally, equation 4 is an adaptive expectation formation rule that specifies how goods producers form their expectations about the price of goods in period $t$ as a function of past prices.

Combine the equations as follows. Note that equation 3 implies that...

$$ D(p_t) = q_t^d = q_t^s = S(p_t^e) $$

...and therefore, assuming the demand function $D$ is invertible, we can write the observed price of goods in period $t$ as...

$$ p_t = D^{-1}\big(S(p_t^e)\big). \tag{5}$$

Substituting equation 5 into equation 4 we arrive at the following difference equation

$$ p_{t+1}^e = w D^{-1}\big(S(p_t^e)\big) + (1 - w)p_t^e. \tag{7}$$

```python
%matplotlib inline
```

```python
%load_ext autoreload
```

```python
%autoreload 2
```

```python
import functools

import ipywidgets
import matplotlib.pyplot as plt
import numpy as np
from scipy import optimize
import seaborn as sns

import cobweb
```

```python
def observed_price(D_inverse, S, expected_price, **params):
    """The observed price of goods in a particular period."""
    actual_price = D_inverse(S(expected_price, **params), **params)
    return actual_price

def adaptive_expectations(D_inverse, S, expected_price, w, **params):
    """An adaptive expectations price forecasting rule."""
    actual_price = observed_price(D_inverse, S, expected_price, **params)
    price_forecast = w * actual_price + (1 - w) * expected_price
    return price_forecast

```

 Non-linear supply functions

When thinking about supply it helps to start with the following considerations...

 ...when prices are low, the quantity supplied increases slowly because of fixed costs of production (think startup costs, etc).

 ...when prices are high, supply also increases slowly because of capacity constraints.

These considerations motivate our focus on "S-shaped" supply functions...

$$ S_{\gamma}(p_t^e) = -tan^{-1}(-\gamma \bar{p}) + tan^{-1}(\gamma (p_t^e - \bar{p})). \tag{10}$$

The parameter $0 < \gamma < \infty$ controls the "steepness" of the supply function.

```python
def quantity_supply(expected_price, gamma, p_bar, **params):
    """The quantity of goods supplied in period t given the epxected price."""
    return -np.arctan(-gamma * p_bar) + np.arctan(gamma * (expected_price - p_bar))
```

 Exploring supply shocks

Interactively change the value of $\gamma$ to see the impact on the shape of the supply function.

```python
ipywidgets.interact?
```

```python
interactive_quantity_supply_plot = ipywidgets.interact(cobweb.quantity_supply_plot,
                                                       S=ipywidgets.fixed(quantity_supply),
                                                       gamma=cobweb.gamma_float_slider,
                                                       p_bar=cobweb.p_bar_float_slider)

```

 Special case: Linear demand functions

Suppose the the quantity demanded of goods is a simple, decresing linear function of the observed price.

$$ q_t^d = D(p_t) = a - b p_t \implies p_t = D^{-1}(q_t^d) = \frac{a}{b} - \frac{1}{b}q_t^d \tag{11} $$

...where $-\infty < a < \infty$ and $0 < b < \infty$.

```python
def quantity_demand(observed_price, a, b):
    """The quantity demand of goods in period t given the price."""
    quantity = a - b * observed_price
    return quantity

def inverse_demand(quantity_demand, a, b, **params):
    """The price of goods in period t given the quantity demanded."""
    price = (a / b) - (1 / b) * quantity_demand
    return price
```

 Exploring demand shocks

Interactively change the values of $a$ and $b$ to get a feel for how they impact demand.  Shocks to $a$ shift the entire demand curve; shocks to $b$ change the slope of the demand curve (higher $b$ implies greater sensitivity to price; lower $b$ implies less sensitivity to price).

```python
interactive_quantity_demand_plot = ipywidgets.interact(cobweb.quantity_demand_plot,
                                                       D=ipywidgets.fixed(quantity_demand),
                                                       a=cobweb.a_float_slider,
                                                       b=cobweb.b_float_slider)

```

 Supply and demand

Market clearing equilibrium price, $p^*$, satisfies...

$$ D(p_t) = S(p_t^e). $$

Really this is also an equilibrium in beliefs because we also require that $p_t = p_t^e$!

```python
interactive_supply_demand_plot = ipywidgets.interact(cobweb.supply_demand_plot,
                                                     D=ipywidgets.fixed(quantity_demand),
                                                     S=ipywidgets.fixed(quantity_supply),
                                                     a=cobweb.a_float_slider,
                                                     b=cobweb.b_float_slider,
                                                     gamma=cobweb.gamma_float_slider,
                                                     p_bar=cobweb.p_bar_float_slider)

```

 Analyzing dynamics of the model via simulation...

Model has no closed form solution (i.e., we can not solve for a function that describes $p_t^e$ as a function of time and model parameters).  BUT, we can simulate equation 7 above to better understand the dynamics of the model...

We can simulate our model and plot time series for different parameter values.  Questions for discussion...

 Can you find a two-cycle? What does this mean?

 Can you find higher cycles? Perhaps a four-cycle? Maybe even a three-cycle?

 Do simulations with similar initial conditions converge or diverge over time?

Can we relate these things to other SFI MOOCS on non-linear dynamics and chaos? Surely yes!

```python
model = functools.partial(adaptive_expectations, inverse_demand, quantity_supply)
interactive_time_series_plot = ipywidgets.interact(cobweb.time_series_plot,
                                                   F=ipywidgets.fixed(model),
                                                   X0=cobweb.initial_expected_price_slider,
                                                   T=cobweb.T_int_slider,
                                                   a=cobweb.a_float_slider,
                                                   b=cobweb.b_float_slider,
                                                   w=cobweb.w_float_slider,
                                                   gamma=cobweb.gamma_float_slider,
                                                   p_bar=cobweb.p_bar_float_slider)
```

 Forecast errors

How do we measure forecast error?  What does the distribution of forecast errors look like for different parameters? Could an agent learn to avoid chaos? Specifically, suppose an agent learned to tune the value of $w$ in order to minimize its mean forecast error. Would this eliminate chaotic dynamics?

```python
interactive_forecast_error_plot = ipywidgets.interact(cobweb.forecast_error_plot,
                                                      D_inverse=ipywidgets.fixed(inverse_demand),
                                                      S=ipywidgets.fixed(quantity_supply),
                                                      F=ipywidgets.fixed(model),
                                                      X0=cobweb.initial_expected_price_slider,
                                                      T=cobweb.T_int_slider,
                                                      a=cobweb.a_float_slider,
                                                      b=cobweb.b_float_slider,
                                                      w=cobweb.w_float_slider,
                                                      gamma=cobweb.gamma_float_slider,
                                                      p_bar=cobweb.p_bar_float_slider)
```

 Other things of possible interest?

Impulse response functions?
Compare constrast model predictions for rational expectations, naive expectations, adaptive expectations.  Depending on what Cars might have in mind, we could also add other expectation formation rules from his more recent work and have students analyze those...

```python

```


```javascript
%%javascript
MathJax.Hub.Config({TeX: { equationNumbers: { autoNumber: "AMS" } }});
```

    <IPython.core.display.Javascript object>

#  FLC - Fourier Linear Combiner

The [FLC](https://ieeexplore.ieee.org/document/18751) is designed for modeling and estimating a quasiperiodic signal of known frequency by adapting the amplitude and phase of a reference signal, in real-time with zero phase shift. The reference signal is generated artificially by a dynamic truncated Fourier series
\begin{equation}\label{eq:truncatedFourierSeries}
y_k=\sum_{r=1}^{n}a_{rk}\sin(r\omega_0 k )+b_{rk}\cos(r\omega_0k)
\end{equation}

where $k$ is the sampling instance, $y_k$ the is estimated signal, $n$ is the number of harmonics in the model, $\mu$ is the adaptive gain parameter, and the Fourier coefficients $a_{rk}$ and $b_{rk}$ are the adaptive weights for their respective harmonic frequency $r\omega_0$. Then $\textbf{x}_k$ and $\textbf{w}_k$ are defined

\begin{align}
\textbf{x}_k&=
\begin{bmatrix}\label{eq:FLCxVector}
[\sin(\omega_0k)&\sin(2\omega_0k)&\cdots&\sin(n\omega_0k)]^T \\
[\cos(\omega_0k)&\cos(2\omega_0k)&\cdots&\cos(n\omega_0k)]^T
\end{bmatrix}\\\label{eq:FLCfilterWeights}
\textbf{w}_k&=\begin{bmatrix}
[a_{1k}&a_{2k}&\cdots & a_{nk}]^T\\
[b_{1k}&b_{2k}&\cdots & b_{nk}]^T
\end{bmatrix}
\end{align}

Using \eqref{eq:FLCxVector} and \eqref{eq:FLCfilterWeights} the linear combiner can be written as

\begin{equation}\label{key}
y_k=\textbf{w}_k^T\textbf{x}_k
\end{equation}

The error between the input signal $s_k$ and the estimated signal, $y_k$ is

\begin{align}
\varepsilon_k&=s_k-y_k=s_k-\textbf{w}_k^T\textbf{x}_k
\end{align}

and the recursive LMS algorithm used to update the weights of $\textbf{w}_k$

\begin{align}
\textbf{w}_{k+1}&=\textbf{w}_k+2\mu\textbf{x}_k\epsilon_k
\end{align}

The FLC architecture can be seen in the following figure


###### Content under Creative Commons Attribution license CC-BY 4.0, code under MIT license (c)2014 L.A. Barba, G.F. Forsyth, C. Cooper. Based on [CFDPython](https://github.com/barbagroup/CFDPython), (c)2013 L.A. Barba, also under CC-BY license.

# Space & Time

## Stability and the CFL condition

Welcome back! This is the second IPython Notebook of the series *Space and Time — Introduction to Finite-difference solutions of PDEs*, the second module of ["Practical Numerical Methods with Python"](http://openedx.seas.gwu.edu/courses/GW/MAE6286/2014_fall/about).

In the first lesson of this series, we studied the numerical solution of the linear and non-linear convection equations, using the finite-difference method. Did you experiment there using different parameter choices? If you did, you probably ran into some unexpected behavior. Did your solution ever blow up (sometimes in a cool way!)?

In this IPython Notebook, we will explore why changing the discretization parameters can affect your solution in such a drastic way.

With the solution parameters we initially suggested, the spatial grid had 41 points and the timestep was 0.25.  Now, we're going to experiment with the number of points in the grid.  The code below corresponds to the linear convection case, but written into a function so that we can easily examine what happens as we adjust just one variable: **the grid size**.

```python
import numpy
from matplotlib import pyplot
%matplotlib inline
from matplotlib import rcParams
rcParams['font.family'] = 'serif'
rcParams['font.size'] = 16
```

```python
def linearconv(nx):
    """Solve the linear convection equation.

    Solves the equation d_t u + c d_x u = 0 where
    * the wavespeed c is set to 1
    * the domain is x \in [0, 2]
    * 20 timesteps are taken, with \Delta t = 0.025
    * the initial data is the hat function

    Produces a plot of the results

    Parameters
    ----------

    nx : integer
        number of internal grid points

    Returns
    -------

    None : none
    """
    dx = 2/(nx-1)
    nt = 20
    dt = .025
    c = 1

    x = numpy.linspace(0,2,nx)

    u = numpy.ones(nx)
    lbound = numpy.where(x >= 0.5)
    ubound = numpy.where(x <= 1)
    u[numpy.intersect1d(lbound, ubound)]=2

    un = numpy.ones(nx)

    for n in range(nt):
        un = u.copy()
        u[1:] = un[1:] -c*dt/dx*(un[1:] -un[0:-1])
        u[0] = 1.0

    pyplot.plot(x, u, color='#003366', ls='--', lw=3)
    pyplot.ylim(0,2.5);
```

Now let's examine the results of the linear convection problem with an increasingly fine mesh.  We'll try 41, 61 and 71 points ... then we'll shoot for 85. See what happens:

```python
linearconv(41) #convection using 41 grid points
```

```python
linearconv(61)
```

```python
linearconv(71)
```

So far so good—as we refine the spatial grid, the wave is more square, indicating that the discretization error is getting smaller. But what happens when we refine the grid even further? Let's try 85 grid points.

```python
linearconv(85)
```

Oops. This doesn't look anything like our original hat function. Something has gone awry. It's the same code that we ran each time, so it's not a bug!

### What happened?

To answer that question, we have to think a little bit about what we're actually implementing in the code when we solve the linear convection equation with the forward-time/backward-space method.

In each iteration of the time loop, we use the existing data about the solution at time $n$ to compute the solution in the subsequent time step, $n+1$.  In the first few cases, the increase in the number of grid points returned more accurate results.  There was less discretization error and the translating wave looked more like a square wave than it did in our first example.

Each iteration of the time loop advances the solution by a time-step of length $\Delta t$, which had the value 0.025 in the examples above. During this iteration, we evaluate the solution $u$ at each of the $x_i$ points on the grid.  But in the last plot, something has clearly gone wrong.

What has happened is that over the time period $\Delta t$, the wave is travelling a distance which is greater than `dx`, and we say that the solution becomes *unstable* in this situation (this statement can be proven formally, see below).  The length `dx` of grid spacing is inversely proportional to the number of total points `nx`: we asked for more grid points, so `dx` got smaller. Once `dx` got smaller than the $c\Delta t$—the distance travelled by the numerical solution in one time step—it's no longer possible for the numerical scheme to solve the equation correctly!

#### Graphical interpretation of the CFL condition.

Consider the illustration above. The green triangle represents the _domain of dependence_ of the numerical scheme. Indeed, for each time step, the variable $u_i^{n+1}$ only depends on the values $u_i^{n}$ and $u_{i-1}^{n}$.

When the distance $c\Delta t$ is smaller than $\Delta x$, the characteristic line traced from the grid coordinate $i, n+1$ lands _between_ the points $i-1,n$ and $i,n$ on the grid. We then say that the _mathematical domain of dependence_ of the solution of the original PDE is contained in the _domain of dependence_ of the numerical scheme.

On the contrary, if $\Delta x$ is smaller than $c\Delta t$, then the information about the solution needed for $u_i^{n+1}$ is not available in the _domain of dependence_ of the numerical scheme, because the characteristic line traced from the grid coordinate $i, n+1$ lands _behind_ the point $i-1,n$ on the grid.

The following condition thus ensures that the domain of dependence of the differential equation is contained in the _numerical_ domain of dependence:

\begin{equation}\sigma = \frac{c \Delta t}{\Delta x} \leq 1
\end{equation}

As can be proven formally, stability of the numerical solution requires that step size `dt` is calculated with respect to the size of `dx` to satisfy the condition above.

The value of $c\Delta t/\Delta x$ is called the **Courant-Friedrichs-Lewy number** (CFL number), often denoted by $\sigma$. The value $\sigma_{\text{max}}$ that will ensure stability depends on the discretization used; for the forward-time/backward-space scheme, the condition for stability is $\sigma<1$.

In a new version of our code—written _defensively_—, we'll use the CFL number to calculate the appropriate time-step `dt` depending on the size of `dx`.

```python
def linearconv(nx):
    """Solve the linear convection equation.

    Solves the equation d_t u + c d_x u = 0 where
    * the wavespeed c is set to 1
    * the domain is x \in [0, 2]
    * 20 timesteps are taken, with \Delta t computed using the CFL 0.5
    * the initial data is the hat function

    Produces a plot of the results

    Parameters
    ----------

    nx : integer
        number of internal grid points

    Returns
    -------

    None : none
    """
    dx = 2/(nx-1)
    nt = 20
    c = 1
    sigma = .5
    x = numpy.linspace(0,2,nx)

    dt = sigma*dx

    u = numpy.ones(nx)
    lbound = numpy.where(x >= 0.5)
    ubound = numpy.where(x <= 1)
    u[numpy.intersect1d(lbound, ubound)]=2

    un = numpy.ones(nx)

    for n in range(nt):
        un = u.copy()
        u[1:] = un[1:] -c*dt/dx*(un[1:] -un[0:-1])
        u[0] = 1.0

    pyplot.plot(x, u, color='#003366', ls='--', lw=3)
    pyplot.ylim(0,2.5);
```

Now, it doesn't matter how many points we use for the spatial grid: the solution will always be stable!

```python
linearconv(41)
```

```python
linearconv(61)
```

```python
linearconv(81)
```

```python
linearconv(101)
```

```python
linearconv(121)
```

Notice that as the number of points `nx` increases, the wave convects a shorter and shorter distance.  The number of time iterations we have advanced the solution to is held constant at `nt = 20`, but depending on the value of `nx` and the corresponding values of `dx` and `dt`, a shorter time window is being examined overall.

---

###### The cell below loads the style of the notebook.

```python
from IPython.core.display import HTML
css_file = '../../styles/numericalmoocstyle.css'
HTML(open(css_file, "r").read())
```

<link href='http://fonts.googleapis.com/css?family=Alegreya+Sans:100,300,400,500,700,800,900,100italic,300italic,400italic,500italic,700italic,800italic,900italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Arvo:400,700,400italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=PT+Mono' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Shadows+Into+Light' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Nixie+One' rel='stylesheet' type='text/css'>
<style>

@font-face {
    font-family: "Computer Modern";
    src: url('http://mirrors.ctan.org/fonts/cm-unicode/fonts/otf/cmunss.otf');
}

#notebook_panel { /* main background */
    background: rgb(245,245,245);
}

div.cell { /* set cell width */
    width: 750px;
}

div #notebook { /* centre the content */
    background: #fff; /* white background for content */
    width: 1000px;
    margin: auto;
    padding-left: 0em;
}

#notebook li { /* More space between bullet points */
    margin-top:0.8em;
}

/* draw border around running cells */
div.cell.border-box-sizing.code_cell.running {
    border: 1px solid #111;
}

/* Put a solid color box around each cell and its output, visually linking them*/
div.cell.code_cell {
    background-color: rgb(256,256,256);
    border-radius: 0px;
    padding: 0.5em;
    margin-left:1em;
    margin-top: 1em;
}

div.text_cell_render{
    font-family: 'Alegreya Sans' sans-serif;
    line-height: 140%;
    font-size: 125%;
    font-weight: 400;
    width:600px;
    margin-left:auto;
    margin-right:auto;
}

/* Formatting for header cells */
.text_cell_render h1 {
    font-family: 'Nixie One', serif;
    font-style:regular;
    font-weight: 400;
    font-size: 45pt;
    line-height: 100%;
    color: rgb(0,51,102);
    margin-bottom: 0.5em;
    margin-top: 0.5em;
    display: block;
}

.text_cell_render h2 {
    font-family: 'Nixie One', serif;
    font-weight: 400;
    font-size: 30pt;
    line-height: 100%;
    color: rgb(0,51,102);
    margin-bottom: 0.1em;
    margin-top: 0.3em;
    display: block;
}

.text_cell_render h3 {
    font-family: 'Nixie One', serif;
    margin-top:16px;
    font-size: 22pt;
    font-weight: 600;
    margin-bottom: 3px;
    font-style: regular;
    color: rgb(102,102,0);
}

.text_cell_render h4 {    /*Use this for captions*/
    font-family: 'Nixie One', serif;
    font-size: 14pt;
    text-align: center;
    margin-top: 0em;
    margin-bottom: 2em;
    font-style: regular;
}

.text_cell_render h5 {  /*Use this for small titles*/
    font-family: 'Nixie One', sans-serif;
    font-weight: 400;
    font-size: 16pt;
    color: rgb(163,0,0);
    font-style: italic;
    margin-bottom: .1em;
    margin-top: 0.8em;
    display: block;
}

.text_cell_render h6 { /*use this for copyright note*/
    font-family: 'PT Mono', sans-serif;
    font-weight: 300;
    font-size: 9pt;
    line-height: 100%;
    color: grey;
    margin-bottom: 1px;
    margin-top: 1px;
}

.CodeMirror{
    font-family: "PT Mono";
    font-size: 90%;
}

</style>


Consider a sky-diver falling towards the surface of Earth. The diver brought with them a temperature sensor that took multiple measurements along the descent. The diver is interested in determining their terminal velocity given these measurements.

Problem: Given a temperature measurement, can one estimate the velocity as a function of time?

```julia
using LinearAlgebra: diagm
using Random
using CairoMakie
using SymPy: Sym, @syms

CairoMakie.activate!()

include("kalmanflt.jl");
include("extendedkalmanflt.jl");
```

The system is a particle falling due to gravity WITH air resistance. For this simulation, the falling object will acheive terminal velocity. The velocity as a function of time is modeled as

$$
\begin{align}
v(t) &= v_{\infty} \tanh\left(\frac{g t}{v_{\infty}}\right)
\end{align}
$$

where $v_{\infty}$ is the terminal velocity as time approaches infinity and $g$ is the acceleration due to gravity.

What we need now is a mapping between temperature and altitude.
I will use the data from https://www.engineeringtoolbox.com/air-altitude-temperature-d_461.html and linearize the temperature model. For the terminal velocity, I make-up a weak, linear relationship between temperature and terminal velocity.

$$
\begin{align}
h_{n} &= h_{n}(t) = h_{n-1} + v_{n-1} \Delta t \\
v_{n} &= v_{n}(t,\Delta t) = v_{n-1} + \frac{\partial}{\partial t}\left[v_{\infty}\tanh\left(\frac{g t}{v_{\infty}}\right)\right] \Delta t = v_{n-1} +  g \Delta t \text{sech}^2\left(\frac{g t}{v_{\infty}}\right)\\
v_{\infty} &\rightarrow v_{\infty}(T) = m_v T + b_v \\
T &\rightarrow T(h) = m_T h + b_T
\end{align}
$$

```julia
time_steps = 150

@syms (
    t_s::Positive,
    T_K::Positive,
    altitude_m::Real,
    hnm1_m::Real,
    hn_pred_m::Real,
    vnm1_m__s::Real,
    time_delta_s::Positive,
    accel_m__s2::Real,
    VInf_m__s::Real,
    VInfS_m__Ks::Real,
    VInfI_m__s::Real,
    TS_K__m::Real,
    TI_K::Real
)

par_values_dict = Dict(
    time_delta_s=>0.1,
    VInfS_m__Ks=>-0.1481,
    VInfI_m__s=>94.44,
    TS_K__m=>-0.006_835,
    TI_K=>288.706,
    accel_m__s2=>-9.8,
)
```

$\begin{equation*}\begin{cases}accel^{s2}_{m} & \text{=>} &-9.8\\VInfS^{Ks}_{m} & \text{=>} &-0.1481\\TI_{K} & \text{=>} &288.706\\time_{\delta s} & \text{=>} &0.1\\TS^{m}_{K} & \text{=>} &-0.006835\\VInfI^{s}_{m} & \text{=>} &94.44\\\end{cases}\end{equation*}$

```julia
fTemperature_K(altitude_m, TS_K__m, TI_K) = TS_K__m * altitude_m + TI_K
fVInf_m__s(T_K, VInfS_m__Ks, VInfI_m__s) = VInfS_m__Ks * T_K + VInfI_m__s
```

    fVInf_m__s (generic function with 1 method)

```julia
vn_m__s(
    vnm1_m__s,
    time_delta_s,
    t_s,
    VInf_m__s,
    accel_m__s2
)  = (
    vnm1_m__s
    + time_delta_s
    * accel_m__s2 * sech(accel_m__s2 * t_s / VInf_m__s)^2
)
```

    vn_m__s (generic function with 1 method)

```julia
hn_m(
    hnm1_m,
    vnm1_m__s,
    time_delta_s
) = hnm1_m + vnm1_m__s * time_delta_s
```

    hn_m (generic function with 1 method)

```julia
height_state_m = StateVariable(
    hn_m,
    hnm1_m,
    Array([hnm1_m, vnm1_m__s, time_delta_s])
)

velocity_state_m__s = StateVariable(
    vn_m__s,
    vnm1_m__s,
    Array(
        [
            vnm1_m__s,
            time_delta_s,
            t_s,
            fVInf_m__s(  # Velocity infty as a fcn of temperature
                fTemperature_K(  # Temperature as a fcn of altitude
                    hnm1_m,
                    TS_K__m,
                    TI_K
                ),
                VInfS_m__Ks,
                VInfI_m__s
            ),
            accel_m__s2
        ]
    )
)

state_variables = Vector([height_state_m, velocity_state_m__s]);
```

```julia
trans_mat_sym = transition_matrix_sym(state_variables)
trans_mat_sym.doit()
```

    ┌ Warning: `vendor()` is deprecated, use `BLAS.get_config()` and inspect the output instead
    │   caller = npyinitialize() at numpy.jl:67
    └ @ PyCall /Users/mhogan/.julia/packages/PyCall/L0fLP/src/numpy.jl:67

$\left[ \begin{array}{rr}1&time_{\delta s}\\\frac{2 TS^{m}_{K} VInfS^{Ks}_{m} \left(accel^{s2}_{m}\right)^{2} t_{s} time_{\delta s} \tanh{\left(\frac{accel^{s2}_{m} t_{s}}{VInfI^{s}_{m} + VInfS^{Ks}_{m} \left(TI_{K} + TS^{m}_{K} hnm_{1 m}\right)} \right)} \operatorname{sech}^{2}{\left(\frac{accel^{s2}_{m} t_{s}}{VInfI^{s}_{m} + VInfS^{Ks}_{m} \left(TI_{K} + TS^{m}_{K} hnm_{1 m}\right)} \right)}}{\left(VInfI^{s}_{m} + VInfS^{Ks}_{m} \left(TI_{K} + TS^{m}_{K} hnm_{1 m}\right)\right)^{2}}&1\end{array}\right]$

```julia
plot_eval_dict = Dict(par_values_dict)

n_points = 31
time_axis_s = LinRange(0, 30, n_points)
altitude_axis_m = LinRange(2000, 3000, n_points)
dv_dh_sym = trans_mat_sym[2, 1].subs(plot_eval_dict).doit()
```

$- \frac{7.27924474253786 \cdot 10^{-6} t_{s} \tanh{\left(\frac{9.8 t_{s}}{0.0010122635 hnm_{1 m} + 51.6826414} \right)} \operatorname{sech}^{2}{\left(\frac{9.8 t_{s}}{0.0010122635 hnm_{1 m} + 51.6826414} \right)}}{\left(1.95861409668586 \cdot 10^{-5} hnm_{1 m} + 1\right)^{2}}$

```julia
dv_dh = [
    Float64(dv_dh_sym.subs(Dict(t_s=>t, hnm1_m=>a)).evalf())
        for t in time_axis_s, a in altitude_axis_m
];
```

```julia
wireframe(
    time_axis_s,
    altitude_axis_m,
    dv_dh,
    axis=(type=Axis3, xlabel = "Time (second)", ylabel = "Position (meters)", title=L"\frac{dv}{dh}"),
    color=:black
)

current_figure()
```

```julia
temperature_obs_K(altitude_m, TS_K__m, TI_K) = fTemperature_K(altitude_m, TS_K__m, TI_K)
temperature_observation = ObservationVariable(temperature_obs_K, Array([hnm1_m, TS_K__m, TI_K]))
observation_variables = Vector([temperature_observation]);
```

```julia
obs_mat_sym = observation_matrix_sym(observation_variables, state_variables)
obs_mat_sym.doit()
```

$\left[ \begin{array}{rr}TS^{m}_{K}&0\end{array}\right]$

```julia
obs_mat = float(obs_mat_sym.subs(par_values_dict).doit().evalf())
```

    1×2 Matrix{Float64}:
     -0.006835  0.0

```julia
"""Use these two variables to simulate differences variations between the truth and observation
for the temperature vs altitude
"""

"""Temperature variations"""
artificial_noise_K = 0.01  # Kelvin
observation_cov = diagm([1.0^2])  # Kelvin^2

system_noise_cov = diagm([100.0^2, 1.00^2]);
```

```julia
true_state_fcn_time = Array{Vector{Float64}}(undef, time_steps)
measurements_array = Array{Vector{Float64}}(undef, time_steps)
states_fcn_time = Array{KalmanState{Float64}}(undef, time_steps);
```

```julia
"""Define a system where a falling object is thought to be released at rest
at 3 km above the Earth's surface with wind resistence.
"""
true_state_vector = Vector{Float64}([2915.000, -1.1])
initial_state_vector = Vector([3000.0, 0.0])
initial_state_cov = [1000.0^2 10.0; 10.0 1.0^2]  # Diagonal([100.0^2, 1.0^2])
initial_state = KalmanState{Float64}(initial_state_vector, initial_state_cov)
curr_state = KalmanState{Float64}(initial_state.state, initial_state.cov)
true_par_values_dict = Dict(par_values_dict)

for step in 1:time_steps

    curr_step_time_s = (step) * true_par_values_dict[time_delta_s]

    # Set the true parameters for this time step
    true_par_values_dict[t_s] = curr_step_time_s
    true_par_values_dict[hnm1_m] = true_state_vector[1]
    true_par_values_dict[vnm1_m__s] = true_state_vector[2]

    # Set the measured/estimated parameters for this time step
    par_values_dict[t_s] = curr_step_time_s
    par_values_dict[hnm1_m] = curr_state.state[1]
    par_values_dict[vnm1_m__s] = curr_state.state[2]

    # Calculate the true state for this simulation
    true_state_vector = Vector([
        Float64(
            var.pred_func(var.pred_func_syms...).subs(true_par_values_dict).evalf()
        ) for var in state_variables
    ])
    true_state_fcn_time[step] = true_state_vector

    # Predict the next state and save those predictions into the parameter dictionary
    pred_state::KalmanState = predict_state(
        state_variables, curr_state, par_values_dict, system_noise_cov
    )
    par_values_dict[hnm1_m] = pred_state.state[1]
    par_values_dict[vnm1_m__s] = pred_state.state[2]

    # Get the measurement
    meas_vec = Vector([
        Float64(
            var.obs_func(var.obs_func_syms...).subs(true_par_values_dict).evalf()
        ) + (randn() * artificial_noise_K) for var in observation_variables
    ])
    measurements_array[step] = meas_vec

    # Get the corrected current state
    # NOTE: Observation matrix "H" is evaluated+calculated automatically in "correct_state" method if none is provided
    curr_state = correct_state(
        state_variables,
        observation_variables,
        par_values_dict,
        pred_state,
        KalmanObservation{Float64}(meas_vec, obs_mat, observation_cov)
    )
    states_fcn_time[step] = curr_state

end
```

```julia
time_axis_s = [(step * par_values_dict[time_delta_s]) for step in 1:time_steps]
scatter(
    time_axis_s,
    [measurements_array[step][1] for step in 1:time_steps],
    axis = (aspect = 2, xlabel = "Time (second)", ylabel = "Temperature (Kelvin)"),
    label="Measured Temperature"
)
lines!(
    time_axis_s,
    [measurements_array[step][1] for step in 1:time_steps]
)
axislegend()
current_figure()
```

```julia
true_positions_m = [tr_st[1] for tr_st in true_state_fcn_time]
esti_positions_m = [s.state[1] for s in states_fcn_time]

scatter(
    time_axis_s,
    true_positions_m,
    axis = (aspect = 2, xlabel = "Time (second)", ylabel = "Position (meters)"),
    label="True position"
)

lines!(
    time_axis_s,
    true_positions_m
)

scatter!(
    time_axis_s,
    esti_positions_m,
    label="Estimated position"
)

lines!(
    time_axis_s,
    esti_positions_m
)

errorbars!(
    time_axis_s,
    esti_positions_m,
    [sqrt(s.cov[1, 1]) for s in states_fcn_time]
)

axislegend()
current_figure()
```

```julia
true_velocities_m__s = [tr_st[2] for tr_st in true_state_fcn_time]
esti_velocities_m__s = [s.state[2] for s in states_fcn_time]

scatter(
    time_axis_s,
    true_velocities_m__s,
    axis = (aspect = 2, xlabel = "Time (second)", ylabel = "Velocity (meters / sec)"),
    label="True velocity"
)

lines!(
    time_axis_s,
    true_velocities_m__s
)

scatter!(
    time_axis_s,
    esti_velocities_m__s,
    label="Estimated position"
)

lines!(
    time_axis_s,
    esti_velocities_m__s
)

errorbars!(
    time_axis_s,
    esti_velocities_m__s,
    [sqrt(s.cov[2, 2]) for s in states_fcn_time]
)

axislegend()
current_figure()
```

```julia

```


```python
from sympy import *
init_printing()
import numpy as np
%matplotlib widget
import matplotlib as mpl
import matplotlib.pyplot as plt
import ipywidgets as widgets
```

# Phase portraits of planar linear ODEs

## Saddle point
First, let us look at the phase portrait of a saddle point $\dot x=x$, $\dot y=-\lambda y$.

```python
fig, ax = plt.subplots()

Y, X = np.mgrid[-5:5:100j, -5:5:100j]

@widgets.interact(λ=(0.1, 2.0))
def saddle(λ=1):
    ax.clear()
    U, V = X, -λ*Y
    ax.streamplot(X, Y, U, V)
```

    Canvas(toolbar=Toolbar(toolitems=[('Home', 'Reset original view', 'home', 'home'), ('Back', 'Back to previous …

    interactive(children=(FloatSlider(value=1.0, description='λ', max=2.0, min=0.1), Output()), _dom_classes=('wid…

More generally, any linear ODE with one positive and one negative eigenvalue defines a saddle point.

```python
fig, ax = plt.subplots()

Y, X = np.mgrid[-5:5:100j, -5:5:100j]

@widgets.interact(b=(0.6, 3.0))
def saddle(b=1):
    ax.clear()
    A = np.array([[1, b], [2, 1]])
    λ, v = np.linalg.eig(A)
    U = X + b * Y
    V = 2 * X + Y
    ax.streamplot(X, Y, U, V)
    for i in (0, 1):
        for s in (-3, 3):
            ax.arrow(0, 0, s * v[0,i], s * v[1, i])
```

    Canvas(toolbar=Toolbar(toolitems=[('Home', 'Reset original view', 'home', 'home'), ('Back', 'Back to previous …

    interactive(children=(FloatSlider(value=1.0, description='b', max=3.0, min=0.6), Output()), _dom_classes=('wid…

## Node

```python
fig, ax = plt.subplots()

Y, X = np.mgrid[-5:5:1000j, -5:5:1000j]

@widgets.interact(λ=(0.1, 2.0))
def unstable_node(λ=1):
    ax.clear()
    U, V = X, λ*Y
    ax.streamplot(X, Y, U, V)
```

    Canvas(toolbar=Toolbar(toolitems=[('Home', 'Reset original view', 'home', 'home'), ('Back', 'Back to previous …

    interactive(children=(FloatSlider(value=1.0, description='λ', max=2.0, min=0.1), Output()), _dom_classes=('wid…

## Center

```python
fig, ax = plt.subplots()

Y, X = np.mgrid[-5:5:1000j, -5:5:1000j]

@widgets.interact(b=(0.1, 2.0))
def unstable_node(b=1):
    ax.clear()
    U, V = -Y, b ** 2 * X
    ax.streamplot(X, Y, U, V)
```

    Canvas(toolbar=Toolbar(toolitems=[('Home', 'Reset original view', 'home', 'home'), ('Back', 'Back to previous …

    interactive(children=(FloatSlider(value=1.0, description='b', max=2.0, min=0.1), Output()), _dom_classes=('wid…

## Focus

```python
fig, ax = plt.subplots()

Y, X = np.mgrid[-5:5:1000j, -5:5:1000j]

@widgets.interact(a=(0.1, 2.0))
def unstable_node(a=1):
    ax.clear()
    U, V = a * X -Y, X + a * Y
    ax.streamplot(X, Y, U, V)
```

    Canvas(toolbar=Toolbar(toolitems=[('Home', 'Reset original view', 'home', 'home'), ('Back', 'Back to previous …

    interactive(children=(FloatSlider(value=1.0, description='a', max=2.0, min=0.1), Output()), _dom_classes=('wid…

## Jordan cell

```python
fig, ax = plt.subplots()

Y, X = np.mgrid[-5:5:1000j, -5:5:1000j]
U, V = X + Y, Y
ax.streamplot(X, Y, U, V)
```

    Canvas(toolbar=Toolbar(toolitems=[('Home', 'Reset original view', 'home', 'home'), ('Back', 'Back to previous …

    <matplotlib.streamplot.StreamplotSet at 0x7fa312e2d730>

## General linear ODE

The interactive widget in the next cell allows user to create any linear ODE.

```python
fig, ax = plt.subplots()

Y, X = np.mgrid[-5:5:1000j, -5:5:1000j]

@widgets.interact(a=widgets.IntText(value=1), b=widgets.IntText(value=2), c=widgets.IntText(value=2), d=widgets.IntText(value=1))
def unstable_node(a, b, c, d):
    ax.clear()
    U, V = a * X + b * Y, c * X + d * Y
    ax.streamplot(X, Y, U, V)
```

    Canvas(toolbar=Toolbar(toolitems=[('Home', 'Reset original view', 'home', 'home'), ('Back', 'Back to previous …

    interactive(children=(IntText(value=1, description='a'), IntText(value=2, description='b'), IntText(value=2, d…


## Imports

```python
import math
from sympy import Symbol
from sympy.solvers import solve
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
```

## Key

```python
percent_difficulty_drop = .2
number_of_blocks_till_eda = 6
number_of_minutes_till_eda = 12 * 60 # 12 hours * 60 min
Current_Hash_Power = 10606 * 1024 * 1024 * 1024 * 1024 * 1024 # hashes / sec
best_antminer = 4.73 * 1024 * 1024 * 1024 * 1024 # hashes / sec
best_antminer_price = 489.99 # dollars
cost_of_mining_equipment = ( 1.0 * Current_Hash_Power / best_antminer ) * best_antminer_price
```

## Helper Functions

```python
def eda_will_kick_in(min_to_mine_one_block):
    return number_of_blocks_till_eda * min_to_mine_one_block > number_of_minutes_till_eda
def get_alpha(total_hashpower,my_hashpower):
    return 1.0 * my_hashpower / total_hashpower
```

```python
class EDA_Calc:
    def __init__(self,percent_difficulty_drop,number_of_blocks_till_eda,number_of_minutes_till_eda):
        self.percent_difficulty_drop = percent_difficulty_drop
        self.number_of_blocks_till_eda = number_of_blocks_till_eda
        self.number_of_minutes_till_eda = number_of_minutes_till_eda
        self.average_minute_for_block = 10 # BTC has 10 min blocks
        self.calc_equation()

    def calc_equation(self):
        """
           b * x * ((1-p)**n) < m
           finds the smallest integer n that will solve this equation
        """
        x = Symbol('x') # min to mine one block
        n = Symbol('n') # number of edas that will happen
        m = Symbol('m') # min till eda could kick in
        b = Symbol('b') # number of blocks we look back on
        p = Symbol('p') # percent change
        equation = b * x * ((1-p)**n) - m # < 0
        equation = equation.subs(m,self.number_of_minutes_till_eda)
        equation = equation.subs(b,self.number_of_blocks_till_eda)
        equation = equation.subs(p,self.percent_difficulty_drop)
        self.equation = equation

    def how_many_edas_will_kick_in(self,var,solve_alpha=False):
        """
            b * x * ((1-p)**n) < m

            > (1-p)^n < m / (b*x)
            > n * log(1-p) < log(m/(b*x))
            > n < log(m/(b*x))/log(1-p)

            if solve_alpha is false, then var = x, else alpha

        """
        m = self.number_of_minutes_till_eda
        b = self.number_of_blocks_till_eda
        p = self.percent_difficulty_drop

        if solve_alpha:
            if var == 0:
                return np.inf
            x = self.average_minute_for_block / var
        else:
            if var == 0:
                return 0
            x = var
        top = math.log(1.0 * m/(b*x))
        bottom = math.log(1-p)
        return max(math.ceil( top / bottom ),0)
calc = EDA_Calc(percent_difficulty_drop,number_of_blocks_till_eda,number_of_minutes_till_eda)
```

## Plots

### # of EDAS that will kick in - Minutes to mine one block

```python
X = np.arange(0.0001,1000)
y = map(lambda e: calc.how_many_edas_will_kick_in(e),X)
plt.plot(X,y)
plt.xlabel('Minutes To Mine One Block')
plt.ylabel('Number Of EDAs That Will Kick In')
```

### # of EDAS that will kick in - alpha

```python
alpha = np.linspace(0.01,.02)
y = map(lambda e: calc.how_many_edas_will_kick_in(e,solve_alpha=True),alpha)
plt.plot(alpha,y)
plt.xlabel('Alpha')
plt.ylabel('Number Of EDAs That Will Kick In')
```

### Cost to get alpha of the current hashpower (in USD)

```python
alpha = np.linspace(.001,1)
cost = map(lambda a: cost_of_mining_equipment * a, alpha)
plt.plot(alpha,cost)
plt.xlabel('alpha')
plt.ylabel('cost in dollars')
ax = plt.gca()
ax.ticklabel_format(axis='y', style='sci', scilimits=(7,10))
```


```python
import numpy as np
import matplotlib.pyplot as plt
import sympy as sym

```

### 원의 방정식
Equation of a circle

원의 방정식을 `sympy`를 써서 나타내 보자.
Let's represent a circle in `sympy`.

먼저 사용할 기호를 준비한다.
First, let's prepare symbols to use as follows.

```python
x, y, r = sym.symbols("x y r", real=True)
x, y, r

```

`sympy.Eq()`는 방정식의 두 변을 매개변수로 받아들인다.
`sympy.Eq()` takes both sides of the equation as arguments.

```python
circle = sym.Eq((x * x) + (y * y), (r * r))
circle

```

위 방정식을 $y$에 대해 풀어보자.
Let's solve the equation above about $y$.

```python
sol = sym.solve(circle, y)

```

```python
sol[0]

```

```python
sol[1]

```

### 닫힌 형태
Closed form

위 두 닫힌 형태의 양함수를 그려보자.

Let's plot two closed form functions.

```python
r = 10
num = 51
fig, ax = plt.subplots(figsize=(10, 10))
x_array = np.linspace(-r, r, num)
y1_array = np.sqrt(r**2 - x_array**2)
y2_array = - np.sqrt(r**2 - x_array**2)

plt.plot(x_array, y1_array)
plt.plot(x_array, y2_array)
plt.grid(True);

```

### 열린 형태
Open form

```python
circle

```

$(x, y)$ 좌표 평면의 점들 가운데, 위 조건을 만족하는 점들을 모으면 원이 될 것이다.

Among the points of $(x, y)$ coordinates, set of all points satisfying the equation above would be the circle.

유한개의 $(x, y)$ 점이 위 조건에 얼마나 가까운지 한번 표시해 보자.
Let's plot a finite number of $(x, y)$ points; how close are these from satisfying the condition above?

```python
fig, ax = plt.subplots(figsize=(10, 10))

x = np.linspace(-15, 15, 61)
y = np.linspace(-15, 15, 61)

X, Y = np.meshgrid(x, y)

Z = -np.abs(10**2 - (X**2 + Y**2))

plt.pcolor(X, Y, Z, shading="auto")
plt.colorbar()
plt.axis("equal");

# plt.savefig("ㅇ.png", dpi=300)

```

### 매개변수 형태
Parametric form

$\theta$를 독립 변수로 사용하여 $(x, y)$ 를 표시해 보자.
Let's plot $(x, y)$ using $\theta$ as an independent variable.

$$
x(\theta) = r cos \theta \\
y(\theta) = r sin \theta
$$

```python
theta_deg = np.linspace(0, 360)
# num = 얼마가 좋겠습니까?
r = 10

x_array = 10 * np.cos(theta_deg)
y_array = 10 * np.sin(theta_deg)

```

```python
len(theta_deg)

```

```python
fig, ax = plt.subplots(figsize=(12, 12))
plt.plot(x_array, y_array, ".-")
plt.axis("equal")
plt.grid(True)

```

위 그림을 고쳐 보시오
Fix the figure above.

각각 몇개의 점을 계산하였는가

How many calculations each?

```python

```