# Probability
Many of the problems we try to solve using statistics are to do with *probability*. For example, what's the probable salary for a graduate who scored a given score in their final exam at school? Or, what's the likely height of a child given the height of his or her parents?

It therefore makes sense to learn some basic principles of probability as we study statistics.

## Probability Basics
Let's start with some basic definitions and principles.
- An ***experiment*** or ***trial*** is an action with an uncertain outcome, such as tossing a coin.
- A ***sample space*** is the set of all possible outcomes of an experiment. In a coin toss, there's a set of two possible oucomes (*heads* and *tails*).
- A ***sample point*** is a single possible outcome - for example, *heads*)
- An ***event*** is a specific outome of single instance of an experiment - for example, tossing a coin and getting *tails*.
- ***Probability*** is a value between 0 and 1 that indicates the likelihood of a particular event, with 0 meaning that the event is impossible, and 1 meaning that the event is inevitable. In general terms, it's calculated like this:

\begin{equation}\text{probability of an event} = \frac{\text{Number of sample points that produce the event}}{\text{Total number of sample points in the sample space}} \end{equation}

For example, the probability of getting *heads* when tossing as coin is 1/2 - there is only one side of the coin that is designated *heads*. and there are two possible outcomes in the sample space (*heads* and *tails*). So the probability of getting *heads* in a single coin toss is 0.5 (or 50% when expressed as a percentage).

Let's look at another example. Suppose you throw two dice, hoping to get 7.

The dice throw itself is an *experiment* - you don't know the outome until the dice have landed and settled.

The *sample space* of all possible outcomes is every combination of two dice - 36 *sample points*:

⚀+⚀	⚀+⚁	⚀+⚂	⚀+⚃	⚀+⚄	⚀+⚅

⚁+⚀	⚁+⚁	⚁+⚂	⚁+⚃	⚁+⚄	⚁+⚅

⚂+⚀	⚂+⚁	⚂+⚂	⚂+⚃	⚂+⚄	⚂+⚅

⚃+⚀	⚃+⚁	⚃+⚂	⚃+⚃	⚃+⚄	⚃+⚅

⚄+⚀	⚄+⚁	⚄+⚂	⚄+⚃	⚄+⚄	⚄+⚅

⚅+⚀	⚅+⚁	⚅+⚂	⚅+⚃	⚅+⚄	⚅+⚅

The *event* you want to happen is throwing a 7. There are 6 *sample points* that could produce this event:

⚀+⚀	⚀+⚁	⚀+⚂	⚀+⚃	⚀+⚄	⚀+⚅

⚁+⚀	⚁+⚁	⚁+⚂	⚁+⚃	⚁+⚄	⚁+⚅

⚂+⚀	⚂+⚁	⚂+⚂	⚂+⚃	⚂+⚄	⚂+⚅

⚃+⚀	⚃+⚁	⚃+⚂	⚃+⚃	⚃+⚄	⚃+⚅

⚄+⚀	⚄+⚁	⚄+⚂	⚄+⚃	⚄+⚄	⚄+⚅

⚅+⚀	⚅+⚁	⚅+⚂	⚅+⚃	⚅+⚄	⚅+⚅

The *probability* of throwing a 7 is therefore 6/36 which can be simplified to 1/6 or approximately 0.167 (16.7%).

### Probability Notation
When we express probability, we use an upper-case **P** to indicate *probability* and an upper-case letter to represent the event. So to express the probability of throwing a 7 as an event valled ***A***, we could write:

\begin{equation}P(A) = 0.167 \end{equation}

### The Complement of an Event
The *complement* of an event is the set of *sample points* that do ***not*** result in the event.

For example, suppose you have a standard deck of playing cards, and you draw one card, hoping for a *spade*. In this case, the drawing of a card is the *experiment*, and the *event* is drawing a spade. There are 13 cards of each suit in the deck. So the *sample space* contains 52 *sample points*:

13 x ♥	13 x ♠	13 x ♣	13 x ♦

There are 13 *sample points* that would satisfy the requirements of the event:

13 x ♥	13 x ♠	13 x ♣	13 x ♦

So the *probability* of the event (drawing a spade) is 13/52 which is 1/4 or 0.25 (25%).

The *complement* of the event is all of the possible outcomes that *don't* result in drawing a spade:

13 x ♥	13 x ♠	13 x ♣	13 x ♦

There are 39 sample points in the complement (3 x 13), so the probability of the complement is 39/52 which is 3/4 or 0.75 (75%).

Note that the probability of an event and the probability of its complement ***always add up to 1***.

This fact can be useful in some cases. For example, suppose you throw two dice and want to know the probability of throwing more than 4. You *could* count all of the outcomes that would produce this result, but there are a lot of them. It might be easier to identify the ones that *do not* produce this result (in other words, the complement):

⚀+⚀	⚀+⚁	⚀+⚂	⚀+⚃	⚀+⚄	⚀+⚅

⚁+⚀	⚁+⚁	⚁+⚂	⚁+⚃	⚁+⚄	⚁+⚅

⚂+⚀	⚂+⚁	⚂+⚂	⚂+⚃	⚂+⚄	⚂+⚅

⚃+⚀	⚃+⚁	⚃+⚂	⚃+⚃	⚃+⚄	⚃+⚅

⚄+⚀	⚄+⚁	⚄+⚂	⚄+⚃	⚄+⚄	⚄+⚅

⚅+⚀	⚅+⚁	⚅+⚂	⚅+⚃	⚅+⚄	⚅+⚅

Out of a total of 36 sample points in the sample space, there are 6 sample points where you throw a 4 or less (1+1, 1+2, 1+3, 2+1, 2+2, and 3+1); so the probability of the complement is 6/36 which is 1/6 or approximately 0.167 (16.7%).

Now, here's the clever bit. Since the probability of the complement and the event itself must add up to 1, the probability of the event must be **5/6** or **0.833** (**83.3%**).

We indicate the complement of an event by adding a **'** to the letter assigned to it, so:

\begin{equation}P(A) = 1 - P(A') \end{equation}

### Bias
Often, the sample points in the sample space do not have the same probability, so there is a *bias* that makes one outcome more likely than another. For example, suppose your local weather forecaster indicates the predominant weather for each day of the week like this:

Mon	Tue	Wed	Thu	Fri	Sat	Sun

☁	☂	☀	☀	☀	☁	☀

This forceast is pretty typical for your area at this time of the year. In fact, historically the weather is sunny on 60% of days, cloudy on 30% of days, and rainy on only 10% of days. On any given day, the sample space for the weather contains 3 sample points (*sunny*, *cloudy*, and *rainy*); but the probabities for these sample points are not the same.

If we assign the letter **A** to a sunny day event, **B** to a cloudy day event, and **C** to a rainy day event then we can write these probabilities like this:

\begin{equation}P(A)=0.6\;\;\;\; P(B)=0.3\;\;\;\; P(C)=0.1 \end{equation}

The complement of **A** (a sunny day) is any day where it is not sunny - it is either cloudy or rainy. We can work out the probability for this in two ways: we can subtract the probablity of **A** from 1:

\begin{equation}P(A') = 1 - P(A) = 1 - 0.6 = 0.4 \end{equation}

Or we can add together the probabilities for all events that do *not* result in a sunny day:

\begin{equation}P(A') = P(B) + P(C) = 0.3 + 0.1 = 0.4 \end{equation}

Either way, there's a 40% chance of it not being sunny!

## Conditional Probability and Dependence
Events can be:
- *Independent* (events that are not affected by other events)
- *Dependent* (events that are conditional on other events)
- *Mutually Exclusive* (events that can't occur together)

### Independent Events
Imagine you toss a coin. The sample space contains two possible outomes: heads (❂) or tails (♾).

The probability of getting *heads* is 1/2, and the probability of getting *tails* is also 1/2. Let's toss a coin...

❂

OK, so we got *heads*. Now, let's toss the coin again:

❂

It looks like we got *heads* again. If we were to toss the coin a third time, what's the probability that we'd get *heads*?

Although you might be tempted to think that a *tail* is overdue, the fact is that each coin toss is an independent event. The outcome of the first coin toss does not affect the second coin toss (or the third, or any number of other coin tosses). For each independent coin toss, the probability of getting *heads* (or *tails*) remains 1/2, or 50%.

Run the following Python code to simulate 10,000 coin tosses by assigning a random value of 0 or 1 to *heads* and *tails*. Each time the coin is tossed, the probability of getting *heads* or *tails* is 50%, so you should expect approximately half of the results to be *heads* and half to be *tails* (it won't be exactly half, due to a little random variation; but it should be close):

```python
%matplotlib inline
import random

# Create a list with 2 element (for heads and tails)
heads_tails = [0,0]

# loop through 10000 trials
trials = 10000
trial = 0
while trial < trials:
    trial = trial + 1
    # Get a random 0 or 1
    toss = random.randint(0,1)
    # Increment the list element corresponding to the toss result
    heads_tails[toss] = heads_tails[toss] + 1

print (heads_tails)

# Show a pie chart of the results
from matplotlib import pyplot as plt
plt.figure(figsize=(5,5))
plt.pie(heads_tails, labels=['heads', 'tails'])
plt.legend()
plt.show()
```

### Combining Independent Events
Now, let's ask a slightly different question. What is the probability of getting three *heads* in a row? Since the probability of a heads on each independent toss is 1/2, you might be tempted to think that the same probability applies to getting three *heads* in a row; but actually, we need to treat getting three *heads* as it's own event, which is the combination of three independent events. To combine independent events like this, we need to multiply the probability of each event. So:

❂ = 1/2

❂❂ = 1/2 x 1/2

❂❂❂ = 1/2 x 1/2 x 1/2

So the probability of tossing three *heads* in a row is 0.5 x 0.5 x 0.5, which is 0.125 (or 12.5%).

Run the code below to simulate 10,000 trials of flipping a coin three times:

```python
import random

# Count the number of 3xHeads results
h3 = 0

# Create a list of all results
results = []

# loop through 10000 trials
trials = 10000
trial = 0
while trial < trials:
    trial = trial + 1
    # Flip three coins
    result = ['H' if random.randint(0,1) == 1 else 'T',
              'H' if random.randint(0,1) == 1 else 'T',
              'H' if random.randint(0,1) == 1 else 'T']
    results.append(result)
    # If it's three heads, add it to the count
    h3 = h3 + int(result == ['H','H','H'])

# What proportion of trials produced 3x heads
print ("%.2f%%" % ((h3/trials)*100))

# Show all the results
print (results)

```

The output shows the percentage of times a trial resulted in three heads (which should be somewhere close to 12.5%). You can count the number of *['H', 'H', 'H']* entries in the full list of results to verify this if you like!

#### Probability Trees
You can represent a series of events and their probabilities as a probability tree:

                         ____H(0.5)  : 0.5 x 0.5 x 0.5 = 0.125
                        /
                   ____H(0.5)
                  /     \____T(0.5)  : 0.5 x 0.5 x 0.5 = 0.125
                 /
              __H(0.5)   ____H(0.5)  : 0.5 x 0.5 x 0.5 = 0.125
             /   \      /
            /     \____T(0.5)
           /            \____T(0.5)  : 0.5 x 0.5 x 0.5 = 0.125
          /
    _____/              _____H(0.5)  : 0.5 x 0.5 x 0.5 = 0.125
         \             /
          \        ___H(0.5)
           \      /    \_____T(0.5)  : 0.5 x 0.5 x 0.5 = 0.125
            \    /
             \__T(0.5)  _____H(0.5)  : 0.5 x 0.5 x 0.5 = 0.125
                 \     /
                  \___T(0.5)
                       \_____T(0.5)  : 0.5 x 0.5 x 0.5 = 0.125
                                                         _____
                                                          1.0

Starting at the left, you can follow the branches in the tree that represent each event (in this case a coin toss result of *heads* or *tails* at each branch). Multiplying the probability of each branch of your path through the tree gives you the combined probability for an event composed of all of the events in the path. In this case, you can see from the tree that you are equally likely to get any sequence of three *heads* or *tails* results (so three *heads* is just as likely as three *tails*, which is just as likely as *head-tail-head*, *tail-head-tail*, or any other combination!)

Note that the total probability for all paths through the tree adds up to 1.

#### Combined Event Probability Notation
When calculating the probability of combined events, we assign a letter such as **A** or **B** to each event, and we use the *intersection* (**∩**) symbol to indicate that we want the combined probability of multiple events. So we could assign the letters **A**, **B**, and **C** to each independent coin toss in our sequence of three tosses, and express the combined probability like this:

\begin{equation}P(A \cap B \cap C) = P(A) \times P(B) \times P(C) \end{equation}

#### Combining Events with Different Probabilities
Imagine you have created a new game that mixes the excitment of coin-tossing with the thrill of die-rolling! The objective of the game is to roll a die and get *6*, and toss a coin and get *heads*:

⚅ +❂

On each turn of the game, a player rolls the die and tosses the coin.

How can we calculate the probability of winning?

There are two independent events required to win: a die-roll of *6* (which we'll call event **A**), and a coin-toss of *heads* (which we'll call event **B**)

Our formula for combined independent events is:

\begin{equation}P(A \cap B) = P(A) \times P(B) \end{equation}

The probablilty of rolling a *6* on a fair die is 1/6 or 0.167;  and the probability of tossing a coin and getting *heads* is 1/2 or 0.5:

\begin{equation}P(A \cap B) = 0.167 \times 0.5 = 0.083 \end{equation}

So on each turn, there's an 8.3% chance to win the game.

#### Intersections and Unions

Previously you saw that we use the *intersection* (**∩**) symbol to represent "and" when combining event probabilities. This notation comes from a branch of mathematics called *set theory*, in which we work with sets of values. let's examine this in a little more detail.

Here's our deck of playing cards, with the full sample space for drawing any card:

A ♥	A ♠	A ♣A ♦

K ♥	K ♠	K ♣K ♦

Q ♥	Q ♠	Q ♣Q ♦

J ♥	J ♠	J ♣J ♦

10 ♥	10 ♠	10 ♣10 ♦

9 ♥	9 ♠	9 ♣9 ♦

8 ♥	8 ♠	8 ♣8 ♦

7 ♥	7 ♠	7 ♣7 ♦

6 ♥	6 ♠	6 ♣6 ♦

5 ♥	5 ♠	5 ♣5 ♦

4 ♥	4 ♠	4 ♣4 ♦

3 ♥	3 ♠	3 ♣3 ♦

2 ♥	2 ♠	2 ♣2 ♦

Now, let's look at two potential events:
- Drawing an ace (**A**)
- Drawing a red card (**B**)

The set of sample points for event **A** (drawing an ace) is:

A ♥	A ♠	A ♣A ♦

K ♥	K ♠	K ♣K ♦

Q ♥	Q ♠	Q ♣Q ♦

J ♥	J ♠	J ♣J ♦

10 ♥	10 ♠	10 ♣10 ♦

9 ♥	9 ♠	9 ♣9 ♦

8 ♥	8 ♠	8 ♣8 ♦

7 ♥	7 ♠	7 ♣7 ♦

6 ♥	6 ♠	6 ♣6 ♦

5 ♥	5 ♠	5 ♣5 ♦

4 ♥	4 ♠	4 ♣4 ♦

3 ♥	3 ♠	3 ♣3 ♦

2 ♥	2 ♠	2 ♣2 ♦

So the probability of drawing an ace is:

\begin{equation}P(A) = \frac{4}{52} = \frac{1}{13} = 0.077\end{equation}

Now let's look at the set of sample points for event **B** (drawing a red card)

A ♥	A ♠	A ♣A ♦

K ♥	K ♠	K ♣K ♦

Q ♥	Q ♠	Q ♣Q ♦

J ♥	J ♠	J ♣J ♦

10 ♥	10 ♠	10 ♣10 ♦

9 ♥	9 ♠	9 ♣9 ♦

8 ♥	8 ♠	8 ♣8 ♦

7 ♥	7 ♠	7 ♣7 ♦

6 ♥	6 ♠	6 ♣6 ♦

5 ♥	5 ♠	5 ♣5 ♦

4 ♥	4 ♠	4 ♣4 ♦

3 ♥	3 ♠	3 ♣3 ♦

2 ♥	2 ♠	2 ♣2 ♦

The probability of drawing a red card is therefore:

\begin{equation}P(A) = \frac{26}{52} = \frac{1}{2} = 0.5\end{equation}

##### Intersections

We can think of the sample spaces for these events as two sets, and we can show them as a Venn diagram:

Event A⚭Event B

Each circle in the Venn diagram represents a set of sample points. The set on the left contains the sample points for event **A** (drawing an ace) and the set on the right contains the sample points for event **B** (drawing a red card). Note that the circles overlap, creating an intersection that contains only the sample points that apply to event **A** *and* event **B**.

This intersected sample space looks like this:

A ♥	A ♠	A ♣A ♦

K ♥	K ♠	K ♣K ♦

Q ♥	Q ♠	Q ♣Q ♦

J ♥	J ♠	J ♣J ♦

10 ♥	10 ♠	10 ♣10 ♦

9 ♥	9 ♠	9 ♣9 ♦

8 ♥	8 ♠	8 ♣8 ♦

7 ♥	7 ♠	7 ♣7 ♦

6 ♥	6 ♠	6 ♣6 ♦

5 ♥	5 ♠	5 ♣5 ♦

4 ♥	4 ♠	4 ♣4 ♦

3 ♥	3 ♠	3 ♣3 ♦

2 ♥	2 ♠	2 ♣2 ♦

As you've seen previously, we write this as **A ∩ B**, and we can calculate its probability like this:

\begin{equation}P(A \cap B) = P(A) \times P(B) = 0.077 \times 0.5 = 0.0385 \end{equation}

So when you draw a single card from a full deck, there is a 3.85% chance it will be a red ace.

##### Unions
The intersection describes the sample space for  event **A** *and* event **B**; but what if we wanted to look at the probability of drawing an ace *or* a red card. In other words, any sample point that is in either of the Venn digram circles.

This set of sample points looks like this:

A ♥	A ♠	A ♣A ♦

K ♥	K ♠	K ♣K ♦

Q ♥	Q ♠	Q ♣Q ♦

J ♥	J ♠	J ♣J ♦

10 ♥	10 ♠	10 ♣10 ♦

9 ♥	9 ♠	9 ♣9 ♦

8 ♥	8 ♠	8 ♣8 ♦

7 ♥	7 ♠	7 ♣7 ♦

6 ♥	6 ♠	6 ♣6 ♦

5 ♥	5 ♠	5 ♣5 ♦

4 ♥	4 ♠	4 ♣4 ♦

3 ♥	3 ♠	3 ♣3 ♦

2 ♥	2 ♠	2 ♣2 ♦

We call this the *union* of the sets, and we write it as **A ∪ B**.

To calculate the probability of a card being either an ace (of any color) or a red card (of any value), we can work out the probability of A, add it to the probability of B, and subtract the probability of A ∩ B (to avoid double-counting the red aces):

\begin{equation}P(A \cup B) = P(A) + P(B) - P(A \cap B)\end{equation}

So:

\begin{equation}P(A \cup B) = 0.077 + 0.5 - 0.0385 = 0.5385\end{equation}

So when you draw a card from a full deck, there is a 53.85% probability that it will be either an ace or a red card.

### Dependent Events
Let's return to our deck of 52 cards from which we're going to draw one card. The sample space can be summarized like this:

13 x ♥	13 x ♠	13 x ♣	13 x ♦

There are two black suits (*spades* and *clubs*) and two red suits (*hearts* and *diamonds*); with 13 cards in each suit. So the probability of drawing a black card (event **A**) and the probability of drawing a red card (event **B**) can be calculated like this:

\begin{equation}P(A) = \frac{13 + 13}{52} = \frac{26}{52} = 0.5 \;\;\;\; P(B) = \frac{13 + 13}{52} = \frac{26}{52} = 0.5\end{equation}

Now let's draw a card from the deck:

♥

We drew a heart, which is red. So, assuming we don't replace the card back into the deck, this changes the sample space as follows:

12 x ♥	13 x ♠	13 x ♣	13 x ♦

The probabilities for **A** and **B** are now:

\begin{equation}P(A) = \frac{13 + 13}{51} = \frac{26}{51} = 0.51 \;\;\;\; P(B) = \frac{12 + 13}{51} = \frac{25}{51} = 0.49\end{equation}

Now let's draw a second card:

♦

We drew a diamond, so again this changes the sample space for the next draw:

12 x ♥	13 x ♠	13 x ♣	12 x ♦

The probabilities for **A** and **B** are now:

\begin{equation}P(A) = \frac{13 + 13}{50} = \frac{26}{50} = 0.52 \;\;\;\; P(B) = \frac{12 + 12}{50} = \frac{24}{50} = 0.48\end{equation}

So it's clear that one event can affect another; in this case, the probability of drawing a card of a particular color on the second draw depends on the color of card drawn on the previous draw. We call these *dependent* events.

Probability trees are particularly useful when looking at dependent events. Here's a probability tree for drawing red or black cards as the first three draws from a deck of cards:

                         _______R(0.48)
                        /
                   ____R(0.49)
                  /     \_______B(0.52)
                 /
              __R(0.50)  _______R(0.50)
             /   \      /
            /     \____B(0.51)
           /            \_______B(0.50)
          /
    _____/              ________R(0.50)
         \             /
          \        ___R(0.51)
           \      /    \________B(0.50)
            \    /
             \__B(0.50) ________R(0.52)
                 \     /
                  \___B(0.49)
                       \________B(0.48)

#### Calculating Probabilities for Dependent Events
Imagine a game in which you have to predict the color of the next card to be drawn. Suppose the first card drawn is a *spade*, which is black. What is the probability of the next card being red?

The notation for this is:

\begin{equation}P(B|A)\end{equation}

You can interpret this as *the probability of B, given A*. In other words, given that event **A** (drawing a black card) has already happened, what is the probability of **B** (drawing a red card). This is commonly referred to as the *conditional probability* of B given A; and it's formula is:

\begin{equation}P(B|A) = \frac{P(A \cap B)}{P(A)}\end{equation}

So to return to our example, the probability of the second card being red given that the first card was black is:

\begin{equation}P(B|A) = \frac{\frac{26}{52} \times \frac{26}{51}}{\frac{26}{52}}\end{equation}

Which simplifies to:

\begin{equation}P(B|A) = \frac{0.5 \times 0.51}{0.5}\end{equation}

So:

\begin{equation}P(B|A) = \frac{0.255}{0.5} = 0.51\end{equation}

Which is what we calculated previously - so the formula works!

Because this is an algebraic expression, we can rearrange it like this:

\begin{equation}P(A \cap B) = P(A) \times P(B|A)\end{equation}

We can use this form of the formula to calculate the probability that the first two cards drawn from a full deck of cards will both be jacks. In this case, event **A** is drawing a jack for the first card, and event **B** is drawing a jack for the second card.

The probability that the first drawn card will be a jack is:

\begin{equation}P(A) = \frac{4}{52} = \frac{1}{13}\end{equation}

We draw the first card:

J ♣

Success! it's the jack of clubs. Our chances of the first two cards being jacks are looking good so far

Now. we know that there are now only 3 jacks left, in a deck of 51 remaining cards; so the probability of drawing a jack as a second card, given that we drew a jack as the first card is:

\begin{equation}P(B|A) = \frac{3}{51}\end{equation}

So we can work out the probability of drawing two jacks from a deck like this:

\begin{equation}P(A \cap B) = \frac{1}{13} \times \frac{3}{51} = \frac{3}{663} = \frac{1}{221}\end{equation}

So there's a 1 in 221 (0.45%)  probability that the first two cards drawn from a full deck will be jacks.

### Mutually Exclusive Events
We've talked about dependent and independent events, but there's a third category to be considered: mutually exclusive events.

For example, when flipping a coin, what is the probability that in a single coin flip the result will be *heads* ***and*** *tails*? The answer is of course, 0; a single coin flip can only result in *heads* ***or*** *tails*; not both!

For mutually exclusive event, the probability of an intersection is:

\begin{equation}P(A \cap B) = 0\end{equation}

The probability for a union is:

\begin{equation}P(A \cup B) = P(A) + P(B)\end{equation}

Note that we don't need to subtract the intersection (*and*) probability to calculate the union (*or*) probability like we did previously, because there's no risk of double-counting the sample points that lie in both events - there are none. (The intersection probability for mutually exclusive events is always 0, so you can subtract it if you like - you'll still get the same result!)

Let's look at another two mutually exclusive events based on rolling a die:
- Rolling a 6 (event **A**)
- Rolling an odd number (event **B**)

The probabilities for these events are:

\begin{equation}P(A) = \frac{1}{6} \;\;\;\; P(B) = \frac{3}{6}\end{equation}

What's the probability of rolling a 6 *and* an odd number in a single roll? These are mutually exclusive, so:

\begin{equation}P(A \cap B) = 0\end{equation}

What's the probability of rolling a 6 *or* an odd number:

\begin{equation}P(A \cup B) = \frac{1}{6} + \frac{3}{6} = \frac{4}{6}\end{equation}

## Binomial Variables and Distributions
Now that we know something about probability, let's apply that to statistics. Statistics is about inferring measures for a full population based on samples, allowing for random variation; so we're going to have to consider the idea of a *random variable*.

A random variable us a number that can vary in value. For example, the temperature on a given day, or the number of students taking a class.

### Binomial Variables
One particular type of random variable that we use in statistics is a *binomial* variable. A binomial variable is used to count how frequently an event occurs in a fixed number of repeated independent experiments. The event in question must have the same probability of occurring in each experiment, and indicates the success or failure of the experiment; with a probability ***p*** of success, which has a complement of ***1 - p*** as the probability of failure (we often call this kind of experiment a *Bernoulli Trial* after Swiss mathematician Jacob Bernoulli).

For example, suppose we flip a coin three times, counting *heads* as success. We can define a binomial variable to represent the number of successful coin flips (that is, the number of times we got *heads*).

Let's examine this in more detail.

We'll call our variable ***X***, and as stated previously it represents the number of times we flip *heads* in a series of three coin flips. Let's start by examining all the possible values for ***X***.

We're flipping the coin three times, with a probability of 1/2 of success on each flip. The possibile results include none of the flips resulting in *heads*, all of the flips resulting in *heads*, or any combination in between. There are two possible outcomes from each flip, and there are three flips, so the total number of possible result sets is 23, which is 8. Here they are:

♾♾♾

♾❂♾

♾♾❂

♾❂❂

❂♾♾

❂❂♾

❂♾❂

❂❂❂

In these results, our variable ***X***, representing the number of successful events (getting *heads*), can vary from 0 to 3. We can write that like this:

\begin{equation}X=\{0,1,2,3\}\end{equation}

When we want to indicate a specific outcome for a random variable, we use write the variable in lower case, for example ***x*** So what's the probability that ***x*** = 0 (meaning that out of our three flips we got no *heads*)?

We can easily see, that there is 1 row in our set of possible outcomes that contains no *heads*, so:

\begin{equation}P(x=0) = \frac{1}{8}\end{equation}

OK, let's see if we can find the probability for 1 success. There are three sample points containing a single *heads* result, so:

\begin{equation}P(x=1) = \frac{3}{8}\end{equation}

Again, we can easily see that from our results; but it's worth thinking about this in a slightly different way that will make it easier to calculate this probability more generically when there are more sample points (for example, if we had based our binomial variable on 100 coin flips, there would be many more combinations!).

What we're actually saying here is that for **3** experiments (in this case coin flips), we want to *choose* **1** successful results. This is written as 3C1. More generically, this is known as *n choose k*, and it's written like this:

\begin{equation}_{n}C_{k}\end{equation}

or sometimes like this:

\begin{equation}\begin{pmatrix} n \\ k\end{pmatrix}\end{equation}

The formula to calculate this is:

\begin{equation}\begin{pmatrix} n \\ k\end{pmatrix} = \frac{n!}{k!(n-k)!}\end{equation}

The exclamation points indicate *factorials* - the product of all positive integers less than or equal to the specified integer  (with 0! having a value of 1).

In the case of our 3C1 calculation, this means:

\begin{equation}\begin{pmatrix} 3 \\ 1\end{pmatrix} = \frac{3!}{1!(3 - 1)!} = \frac{3!}{1!\times2!} =\frac{3 \times 2 \times 1}{1 \times(2 \times 1)} = \frac{6}{2} = 3 \end{equation}

That seems like a lot of work to find the number of successful experiments, but now that you know this general formula, you can use it to calculate the number of sample points for any value of *k* from any set of *n* cases. Let's use it to find the possibility of two successful *heads* out of 3 coin flips:

\begin{equation}P(x=2) = \frac{_{3}C_{2}}{8}\end{equation}

Let's work out the number of combinations for 3C2

\begin{equation}_{3}C_{2} = \frac{3!}{2!(3 - 2)!} = \frac{6}{2 \times 1} = \frac{6}{2} = 3\end{equation}

So:

\begin{equation}P(x=2) = \frac{3}{8}\end{equation}

Finally, what's the probability that all three flips were *heads*?

\begin{equation}P(x=3) = \frac{_{3}C_{3}}{8}\end{equation}

\begin{equation}_{3}C_{3} = \frac{3!}{3!(3 - 3)!} = \frac{6}{6} = 1\end{equation}

So:

\begin{equation}P(x=3) = \frac{1}{8}\end{equation}

In Python, there are a number of modules you can use to find the *n choose k* combinations, including the *scipy.special.**comb*** function.

In our coin flipping experiment, there is an equal probability of success and failure; so the probability calculations are relatively simple, and you may notice that there's a symmetry to the probability for each possible value of the binomial variable, as you can see by running the following Python code. You can increase the value of the **trials** variable to verify that no matter how many times we toss the coin, the probabilities of getting *heads* (or *tails* for that matter) form a symmetrical distribution, because there's an equal probability of success and failure in each trial.

```python
%matplotlib inline
from scipy import special as sps
from matplotlib import pyplot as plt
import numpy as np

trials = 3

possibilities = 2**trials
x = np.array(range(0, trials+1))

p = np.array([sps.comb(trials, i, exact=True)/possibilities for i in x])

# Set up the graph
plt.xlabel('Successes')
plt.ylabel('Probability')
plt.bar(x, p)
plt.show()
```

#### Allowing for Bias
Previously, we calculated the probability for each possible value of a random variable by simply dividing the number of combinations for that value by the total number of possible outcomes. This works if the probability of the event being tested is equal for failure and success; but of course, not all experiments have an equal chance of success or failure. Some include a bias that makes success more or less likely - so we need to be a little more thorough in our calculations to allow for this.

Suppose you're flying off to some exotic destination, and you know that there's a one in four chance that the airport security scanner will trigger a random search for each passenger that goes though. If you watch five passengers go through the scanner, how many will be stopped for a random search?

It's tempting to think that there's a one in four chance, so a quarter of the passengers will be stopped; but remember that the searches are triggered randomly for thousands of passengers that pass through the airport each day. It's possible that none of the next five passengers will be searched; all five of them will be searched, or some other value in between will be searched.

Even though the probabilities of being searched or not searched are not the same, this is still a binomial variable. There are a fixed number of independent experiments (five passengers passing through the security scanner), the outcome of each experiment is either success (a search is triggered) or failure (no search is triggered), and the probability of being searched does not change for each passenger.

There are five experiments in which a passenger goes through the security scanner, let's call this **n**.

For each passenger, the probability of being searched is 1/4 or 0.25. We'll call this **p**.

The complement of **p** (in other words, the probability of *not* being searched) is **1-p**, in this case 3/4 or 0.75.

So, what's the probability that out of our **n** experiments, three result in a search (let's call that **k**) and the remaining ones (there will be **n**-**k** of them, which is two) don't?

- The probability of three passengers being searched is 0.25 x 0.25 x 0.25 which is the same as 0.253. Using our generic variables, this is **pk**.
- The probability that the rest don't get searched is 0.75 x 0.75, or 0.752. In terms of our variables, this is **1-p(n-k)**.
- The combined probability of three searchs and two non-searches is therefore 0.253 x 0.752 (approximately 0.088). Using our variables, this is:

\begin{equation}p^{k}(1-p)^{(n-k)}\end{equation}

This formula enables us to calculate the probability for a single combination of ***n*** passengers in which ***k*** experiments had a successful outcome. In this case, it enables us to calculate that the probability of three passengers out of five being searched is approximately 0.088. However, we need to consider that there are multiple ways this can happen. The first three passengers could get searched; or the last three; or the first, third, and fifth, or any other possible combination of 3 from 5.

There are two possible outcomes for each experiment; so the total number of possible combinations of five passengers being searched or not searched is 25 or 32. So within those 32 sets of possible result combinations, how many have three searches? We can use the nCk formula to calculate this:

\begin{equation}_{5}C_{3} = \frac{5!}{3!(5 - 3)!} = \frac{120}{6\times 4} = \frac{120}{24} = 5\end{equation}

So 5 out of our 32 combinations had 3 searches and 2 non-searches.

To find the probability of any combination of 3 searches out of 5 passengers, we need to multiply the number of possible combinations by the probability for a single combination - in this case 5/32 x 0.088, which is 0.01375, or 13.75%.

So our complete formula to calculate the probabilty of ***k*** events from ***n*** experiments with probability ***p*** is:

\begin{equation}P(x=k)  = \frac{n!}{k!(n-k)!} p^{k}(1-p)^{(n-k)}\end{equation}

This is known as the *General Binomial Probability Formula*, and we use it to calculate the *probability mass function* (or *PMF*) for a binomial variable. In other words, the we can use it to calculate the probability for each possible value for the variable and use that information to determine the relative frequency of the variable values as a distribution.

In Python, the *scipy.stats.**binom.pmf*** function encapsulates the general binomial probability formula, and you can use it to calculate the probability of a random variable having a specific value (***k***) for a given number of experiments (***n***) where the event being tested has a given probability (***p***), as demonstrated in the following code:

```python
%matplotlib inline
from scipy.stats import binom
from matplotlib import pyplot as plt
import numpy as np

n = 5
p = 0.25
x = np.array(range(0, n+1))

prob = np.array([binom.pmf(k, n, p) for k in x])

# Set up the graph
plt.xlabel('x')
plt.ylabel('Probability')
plt.bar(x, prob)
plt.show()
```

You can see from the bar chart that with this small value for ***n***, the distribution is right-skewed.

Recall that in our coin flipping experiment, when the probability of failure vs success was equal, the resulting distribution was symmetrical. With an unequal probability of success in each experiment, the bias has the effect of skewing the overall probability mass.

However, try increasing the value of ***n*** in the code above to 10, 20, and 50; re-running the cell each time. With more observations, the *central limit theorem* starts to take effect and the distribution starts to look more symmetrical - with enough observations it starts to look like a *normal* distribution.

There is an important distinction here - the *normal* distribution applies to *continuous* variables, while the *binomial* distribution applies to *discrete* variables. However, the similarities help in a number of statistical contexts where the number of observations (experiments) is large enough for the *central limit theorem* to make the distribution of binomial variable values behave like a *normal* distribution.

### Working with the Binomial Distribution
Now that you know how to work out a binomial distribution for a repeated experiment, it's time to take a look at some statistics that will help us quantify some aspects of probability.

Let's increase our ***n*** value to 100 so that we're looking at the number of searches per 100 passengers. This gives us the binomial distribution graphed by the following code:

```python
%matplotlib inline
from scipy.stats import binom
from matplotlib import pyplot as plt
import numpy as np

n = 100
p = 0.25
x = np.array(range(0, n+1))

prob = np.array([binom.pmf(k, n, p) for k in x])

# Set up the graph
plt.xlabel('x')
plt.ylabel('Probability')
plt.bar(x, prob)
plt.show()
```

#### Mean (Expected Value)
We can calculate the mean of the distribution like this:

\begin{equation}\mu  = np\end{equation}

So for our airport passengers, this is:

\begin{equation}\mu  = 100 \times 0.25 = 25\end{equation}

When we're talking about a probability distribution, the mean is usually referred to as the *expected value*. In this case, for any 100 passengers we can reasonably expect 25 of them to be searched.

#### Variance and Standard Deviation
Obviously, we can't search a quarter of a passenger - the expected value reflects the fact that there is variation, and indicates an average value for our binomial random variable. To get an indication of how much variability there actually is in this scenario, we can can calculate the variance and standard deviation.

For variance of a binomial probability distribution, we can use this formula:

\begin{equation}\sigma^{2}  = np(1-p)\end{equation}

So for our airport passengers:

\begin{equation}\sigma^{2}  = 100 \times 0.25 \times 0.75 = 18.75\end{equation}

To convert this to standard deviation we just take the square root:

\begin{equation}\sigma  = \sqrt{np(1-p)}\end{equation}

So:

\begin{equation}\sigma  = \sqrt{18.75} \approx 4.33 \end{equation}

So for every 100 passengers, we can expect 25 searches with a standard deviation of 4.33

In Python, you can use the ***mean***, ***var***, and ***std*** functions from the *scipy.stats.**binom*** package to return binomial distribution statistics for given values of *n* and *p*:

```python
from scipy.stats import binom

n = 100
p = 0.25

print(binom.mean(n,p))
print(binom.var(n,p))
print(binom.std(n,p))
```


```
from sympy import init_printing, var, simplify, collect, solve
```

```
var("k a b")
```

    (k, a, b)

```
init_printing()
```

```
f= k**2*(-a-1) + k*(-a*b - 2*a - 1) + (b**2 + b)
```

```
fsimp = simplify(solve(f, k))
```

```
import sympy
```

```
fsimp
```

```
f
```

```
fsimp
```

```
fsimp[0].subs(a, 1).subs(b, 1)
```

```
fsimp[1].subs(a, 1).subs(b, 1)
```

```
fsimp[0].subs(a, 0).subs(b, 0)
```

```
fsimp[1].subs(a, 0).subs(b, 0)
```

```
infty
```

```
import control
```

```
var("s")
```

```
num = [0, 1, -2, 1]
den = [1, 2, 1, 0]
```

```
funcion = control.tf(num, den)
```

```
roots([1, 2.6, 1-1.2, .6])
```

    array([-2.75190589+0.j        ,  0.07595294+0.46071885j,
            0.07595294-0.46071885j])

```
x,y = control.rlocus(funcion, linspace(-20, 20, 10000))
```

```
x = linspace(0, 10, 100)
y = sin(x)
```

```
plot(x, y)
```

```
figsize(15, 3)
```

```

```


```python
import matplotlib.pyplot as plt
from mpl_toolkits import mplot3d
import numpy as np
import sympy as sym
```

```python
def f(X, Y):
    return X**2 + 2*Y**2
```

```python
X1 = np.arange(-5, 5, 0.1) # -5 ile +5 arasında 0.1'er artan 100 nokta
Y1 = np.arange(-5, 5, 0.1) # -5 ile +5 arasında 0.1'er artan 100 nokta
Z1 = np.zeros(len(X1))

for i in range(len(X1)):
    Z1[i] = X1[i]**2 + 2*Y1[i]**2 # x^2 + 2*y^2

X1, Y1 = np.meshgrid(X1, Y1)
Z1 = f(X1, Y1)
```

### Surface Plots

```python
fig = plt.figure()
ax = plt.axes(projection='3d')
ax.plot_surface(X1, Y1, Z1, rstride=1, cstride=1,
                cmap='viridis', edgecolor='none')
ax.set_title('surface')
plt.show()
```

```python
ax.view_init(60, 35)
fig
```

### Three-dimensional Contour Plots

```python
fig = plt.figure()
ax = plt.axes(projection='3d')
ax.contour3D(X1, Y1, Z1, 50, cmap='binary')
ax.set_xlabel('x')
ax.set_ylabel('y')
ax.set_zlabel('z');
plt.show()
```

```python
ax.view_init(60, 35)
fig
```

### Two-dimensional Contour Plots

```python
plt.figure()
cs = plt.contour(X1, Y1, Z1)
plt.clabel(cs, inline=1, fontsize=10)
plt.title('Two-dimensional contour plot')
plt.show()
```

```python

```


# Vertical Line Test

## 1.1 Create two graphs, one that passes the vertical line test and one that does not.

```
import matplotlib.pyplot as plt
import numpy as np

a = np.array([0,2])
b = np.array([1,0])

vectors = [a,b]

fig, axes = plt.subplots(nrows=2, ncols=1, figsize=(3,6),
                        sharex=True)

axes[0].axhline(y=0.5)
axes[0].set_title('Passes test')

axes[1].axvline(x=0.5)
axes[1].set_title('Fails test')

plt.show()
```

## 1.2 Why are graphs that don't pass the vertical line test not considered "functions?"

A single input (x) will match to more than one output (y)

# Functions as Relations

## 2.1 Which of the following relations are functions? Why?

\begin{align}
\text{Relation 1: } \{(1, 2), (3, 2), (1, 3)\}
\\
\text{Relation 2: } \{(1, 3), (2, 3), (6, 7)\}
\\
\text{Relation 3: } \{(9, 4), (2, 1), (9, 6)\}
\\
\text{Relation 4: } \{(6, 2), (8, 3), (6, 4)\}
\\
\text{Relation 5: } \{(2, 6), (2, 7), (2, 4)\}
\end{align}

Relation 2 is the only function.It's the only one in which every x gets related to a single y. The other relations all have a same value x to a different Y

# Functions as a mapping between dimensions

## 3.1 for the following functions what is the dimensionality of the domain (input) and codomain (range/output)?

\begin{align}
m(𝑥_1,𝑥_2,𝑥_3)=(x_1+x_2, x_1+x_3, x_2+x_3)
\\
n(𝑥_1,𝑥_2,𝑥_3,𝑥_4)=(x_2^2 + x_3, x_2x_4)
\end{align}

M: Input 3 to 3
N: Input 4 to 2

## 3.2 Do you think it's possible to create a function that maps from a lower dimensional space to a higher dimensional space? If so, provide an example.

# Vector Transformations

## 4.1 Plug the corresponding unit vectors into each function. Use the output vectors to create a transformation matrix.

\begin{align}
p(\begin{bmatrix}x_1 \\ x_2 \end{bmatrix}) = \begin{bmatrix} x_1 + 3x_2 \\2 x_2 - x_1 \\  \end{bmatrix}
\\
\\
q(\begin{bmatrix}x_1 \\ x_2 \\ x_3\end{bmatrix}) = \begin{bmatrix} 4x_1 + x_2 + 2x_3 \\2 x_2 - x_1 + 3x_3 \\ 5x_1 - 2x_3 + x_2  \end{bmatrix}
\end{align}

\begin{align}
T_p = \begin{bmatrix} 1&3\\-1&2  \end{bmatrix}
\\
\\
T_q = \begin{bmatrix} 4&1&2\\-1&2&3\\5&1&-2  \end{bmatrix}
\end{align}

## 4.2 Verify that your transformation matrices are correct by choosing an input matrix and calculating the result both via the traditional functions above and also via vector-matrix multiplication.

```
#For T_p

def p(x1, x2):
    return np.array([[x1 + 3*x2],[2*x2 - x1]])

T_p = np.array([[1,3],[-1,2]])

print('Input array:')
print(np.array([[1],[2]]))

print('\nResult from traditional functions:')
print(p(1,2))

print('\nResult from matrix multiplication:')
print(np.matmul(T_p, np.array([[1],[2]])))
```

    Input array:
    [[1]
     [2]]

    Result from traditional functions:
    [[7]
     [3]]

    Result from matrix multiplication:
    [[7]
     [3]]

```
#For T_q
def q(x1, x2, x3):
    return np.array([[4*x1 + x2 + 2*x3],[2*x2 - x1 + 3*x3],[5*x1 - 2*x3 + x2]])

T_q = np.array([[4,1,2],[-1,2,3],[5,1,-2]])

print('Input array:')
print(np.array([[1],[2],[3]]))

print('\nResult from traditional functions:')
print(q(1,2,3))

print('\nResult from matrix multiplication:')
print(np.matmul(T_q, np.array([[1],[2],[3]])))

```

    Input array:
    [[1]
     [2]
     [3]]

    Result from traditional functions:
    [[12]
     [12]
     [ 1]]

    Result from matrix multiplication:
    [[12]
     [12]
     [ 1]]

# Eigenvalues and Eigenvectors

## 5.1 In your own words, give an explanation for the intuition behind eigenvalues and eigenvectors.

An intuition for eigen values and eigen vectors

# The Curse of Dimensionality

## 6.1 What are some of the challenges of working with high dimensional spaces?

Its hard for humans to visualize the simulation of operations that happen in more than 3 dimensions.

## 6.2 What is the rule of thumb for how many observations you should have compared to parameters in your model?

Have at least 10-20 observations per parameter

# Principal Component Analysis

## 7.1 Load the UCI Machine Learning Repository's [Iris Dataset](https://gist.githubusercontent.com/curran/a08a1080b88344b0c8a7/raw/d546eaee765268bf2f487608c537c05e22e4b221/iris.csv) and use PCA to isolate the dataset's first and second principal components and plot them on a graph.

```
import pandas as pd
from sklearn.decomposition import PCA
iris = pd.read_csv('https://gist.githubusercontent.com/curran/a08a1080b88344b0c8a7/raw/d546eaee765268bf2f487608c537c05e22e4b221/iris.csv')
iris_x = iris.drop(columns='species').values # X values only.
```

```
# Retain only 2 components
pca = PCA(2)

# Fit to our X values
pca.fit(iris_x)

# access values and vectors
print("\n Eigenvectors: \n", pca.components_)
print("\n Eigenvalues: \n",pca.explained_variance_)

# transform data
iris_transformed = pca.transform(iris_x)
```

     Eigenvectors:
     [[ 0.36158968 -0.08226889  0.85657211  0.35884393]
     [ 0.65653988  0.72971237 -0.1757674  -0.07470647]]

     Eigenvalues:
     [4.22484077 0.24224357]

```
from mlxtend.plotting import category_scatter

iris_final = pd.DataFrame(iris_transformed, columns=['PC1', 'PC2'])

# Add back the plant Iris species
iris_final['Species'] = iris['species']

fig = category_scatter(data=iris_final,
                       x = 'PC1',
                       y = 'PC2',
                       label_col = 'Species')

plt.xlabel('PC1', fontsize=16)
plt.ylabel('PC2', fontsize=16)
plt.title('The main PCs of the Iris dataset', fontsize=16)

plt.show()
```

# Stretch Goal

## 1) Do NOT work on the stretch goal until you feel like you have a firm grasp of eigenvectors, eigenvalues, and PCA. Prioritize self-study over the stretch goal if you are not comfortable with those topics yet.

## 2) Explore further the intuition behind eigenvalues and eigenvectors by creating your very own eigenfaces:

You don't necessarily have to use this resource, but this will get you started:
[Eigenface Tutorial](https://sandipanweb.wordpress.com/2018/01/06/eigenfaces-and-a-simple-face-detector-with-pca-svd-in-python/)

DS-Unit-1-Sprint-4-Linear-Algebra-master/module3-dimensionality-reduction/High Dimensional Data Assignment.ipynb


[Sascha Spors](https://orcid.org/0000-0001-7225-9992),
Professorship Signal Theory and Digital Signal Processing,
[Institute of Communications Engineering (INT)](https://www.int.uni-rostock.de/),
Faculty of Computer Science and Electrical Engineering (IEF),
[University of Rostock, Germany](https://www.uni-rostock.de/en/)

# Tutorial Signals and Systems (Signal- und Systemtheorie)

Summer Semester 2021 (Bachelor Course #24015)

- lecture: https://github.com/spatialaudio/signals-and-systems-lecture
- tutorial: https://github.com/spatialaudio/signals-and-systems-exercises

WIP...
The project is currently under heavy development while adding new material for the summer semester 2021

Feel free to contact lecturer [frank.schultz@uni-rostock.de](https://orcid.org/0000-0002-3010-0294)

## Übung / Exercise 9

```python
import matplotlib.pyplot as plt
from matplotlib.ticker import MaxNLocator
import numpy as np
from scipy import signal
# digital angular frequency vector with 1024 samples
dw = 2*np.pi / 2**10
w = np.arange(0, 2*np.pi, dw)
```

## Sig Sys UE 7.2 X4(z)

ROC: $|z|>1$
\begin{align}
X_4(z) = \frac{z^2}{z^2+1} = \frac{1}{1+z^{-2}}
\end{align}

```python
# SigSys 7.2 X4(z) o-0 x4[k], oscillating signal with 2*pi/4
# if interpreted as a system transfer function, this would be semi-stable
# X(z) = z^2 / (z^2+1) = 1 / (1 + z^-2)
# thus b0 = 1, a0 = 1, a1 = 0, a2 = 1
b = [1, 0, 0]  # make sure b and a are of same length,
a = [1, 0, 1]  # since signal.dimpulse relies on that :-(
sys = signal.dlti(b, a, dt=1)  # handle k as integer, i.e. unit time step
ax = plt.subplot(1, 1, 1)
N = 2**5
[k, h] = signal.dimpulse(sys, n=N)
h = np.squeeze(h)
plt.stem(k, h, use_line_collection=True,
         linefmt='C0:', markerfmt='C0o', basefmt='C0:')
ax.xaxis.set_major_locator(MaxNLocator(integer=True))
plt.xticks(np.arange(0, N, 2))
plt.xlabel(r'$k$')
plt.ylabel(r'$x_4[k]$')
plt.grid(True)
```

## Sig Sys UE 7.2 X6(z)

\begin{align}
X_6(z) = \frac{z^2-z}{z^2-\sqrt{2} z +1} = \frac{1-z^{-1}}{1-\sqrt{2} z^{-1} + z^{-2}}
\end{align}

```python
# SigSys 7.2 X6(z) o-0 x6[k], oscillating signal with 2*pi/8
# if interpreted as a system transfer function, this would be semi-stable
# X(z) = z(z-1) / (z^2 - sqrt(2)z +1) = (z^2 - z) / (z^2 - sqrt(2)z +1)
# X(z) = (1 - z^-1) / (1 - sqrt(2)z^-1 + z^-2)
# thus b0 = 1, b1 = -1, a0 = 1, a1 = -sqrt(2), a2 = 1
b = [+1, -1, 0]  # make sure b and a are of same length,
a = [+1, -np.sqrt(2), +1]  # since signal.dimpulse relies on that :-(
sys = signal.dlti(b, a, dt=1)  # handle k as integer, i.e. unit time step
ax = plt.subplot(1, 1, 1)
N = 2**5
[k, h] = signal.dimpulse(sys, n=N)
h = np.squeeze(h)
plt.stem(k, h, use_line_collection=True,
         linefmt='C0:', markerfmt='C0o', basefmt='C0:')
ax.xaxis.set_major_locator(MaxNLocator(integer=True))
plt.xticks(np.arange(0, N, 2))
plt.xlabel(r'$k$')
plt.ylabel(r'$x_6[k]$')
plt.grid(True)
```

## Sig Sys UE 7.3 System H1

\begin{align}
H_1(z) = \frac{z^3 +z^2 -z + \frac{1}{2}}{z^3} = 1 + z^{-1} - z^{-2} + \frac{1}{2} z^{-3}
\end{align}

```python
# SigSys 7.3 H1, simple bandpass-like filter
# stable since this is a system with an FIR
# we migth call this a simple FIR filter of order 3 with 3+1 coefficients
b = [1, 1, -1, 1/2]  # make sure b and a are of same length,
a = [1, 0, 0, 0]  # since signal.dimpulse relies on that :-(
# b is identical to the weigthing coefficients of the shifted Diracs in h[k]
# for a: no recursive part, only a0 for weighting y[k]
sys = signal.dlti(b, a, dt=1)  # handle k as integer, i.e. unit time step

N = 10
plt.figure(figsize=(6, 5))
ax = plt.subplot(2, 1, 1)
[k, h] = signal.dimpulse(sys, n=N)
h = np.squeeze(h)
plt.stem(k, h, use_line_collection=True,
         linefmt='C0:', markerfmt='C0o', basefmt='C0:')
ax.xaxis.set_major_locator(MaxNLocator(integer=True))
# plt.xlabel(r'$k$')
plt.ylabel(r'$h[k]$')
plt.title('Impulse response')
plt.grid(True)

ax = plt.subplot(2, 1, 2)
[k, he] = signal.dstep(sys, n=N)
he = np.squeeze(he)
plt.stem(k, he, use_line_collection=True,
         linefmt='C0:', markerfmt='C0o', basefmt='C0:')
ax.xaxis.set_major_locator(MaxNLocator(integer=True))
plt.xlabel(r'$k$')
plt.ylabel(r'$h_\epsilon[k]$')
plt.title('Step response')
plt.grid(True)
```

```python
[w, level, phase] = signal.dbode(sys, w=w)

plt.figure(figsize=(6, 5))
ax = plt.subplot(2, 1, 1)
plt.plot(w / 2 / np.pi, level)
plt.xticks(np.arange(0, 1, 0.125))
plt.xlim(0, 1)
#plt.xlabel(r'$\frac{\Omega}{2\pi}= \frac{f}{f_s}$')
plt.ylabel(r'Level in dB')
plt.grid(True)

ax = plt.subplot(2, 1, 2)
plt.plot(w / 2 / np.pi, phase)
plt.xticks(np.arange(0, 1, 0.125))
plt.xlim(0, 1)
plt.xlabel(r'$\frac{\Omega}{2\pi}= \frac{f}{f_s}$')
plt.ylabel(r'Phase in degree')
plt.grid(True)
```

## Sig Sys UE 7.3 System H2

\begin{align}
H(z) = \frac{z^2}{z^2 - z - 1} = \frac{1}{1 - z^{-1} + z^{-2}}
\end{align}

```python
# SigSys 7.3 H2, simple bandstop-like filter, but unstable,
# due to pole outside unit circle!
b = [1, 0, 0]
a = [1, -1, -1]  # recursive part is the bad guy here
sys = signal.dlti(b, a, dt=1)  # handle k as integer, i.e. unit time step

N = 10
plt.figure(figsize=(6, 5))
ax = plt.subplot(2, 1, 1)
[k, h] = signal.dimpulse(sys, n=N)
h = np.squeeze(h)
plt.stem(k, h, use_line_collection=True,
         linefmt='C0:', markerfmt='C0o', basefmt='C0:')
ax.xaxis.set_major_locator(MaxNLocator(integer=True))
# plt.xlabel(r'$k$')
plt.ylabel(r'$h[k]$')
plt.title('Impulse response')
plt.grid(True)

ax = plt.subplot(2, 1, 2)
[k, he] = signal.dstep(sys, n=N)
he = np.squeeze(he)
plt.stem(k, he, use_line_collection=True,
         linefmt='C0:', markerfmt='C0o', basefmt='C0:')
ax.xaxis.set_major_locator(MaxNLocator(integer=True))
plt.xlabel(r'$k$')
plt.ylabel(r'$h_\epsilon[k]$')
plt.title('Step response')
plt.grid(True)
```

```python
[w, level, phase] = signal.dbode(sys, w=w)

plt.figure(figsize=(6, 5))
ax = plt.subplot(2, 1, 1)
plt.plot(w / 2 / np.pi, level)
plt.xticks(np.arange(0, 1, 0.125))
plt.xlim(0, 1)
#plt.xlabel(r'$\frac{\Omega}{2\pi}= \frac{f}{f_s}$')
plt.ylabel(r'Level in dB')
plt.grid(True)

ax = plt.subplot(2, 1, 2)
plt.plot(w / 2 / np.pi, phase)
plt.xticks(np.arange(0, 1, 0.125))
plt.xlim(0, 1)
plt.xlabel(r'$\frac{\Omega}{2\pi}= \frac{f}{f_s}$')
plt.ylabel(r'Phase in degree')
plt.grid(True)
```

## Sig Sys UE 7.3 System H3

\begin{align}
H_3(z) = \frac{2 z^2 + 1}{z^2-\frac{1}{2} z} = \frac{2 + z^{-2}}{1-\frac{1}{2} z^{-1}}
\end{align}

```python
# SigSys 7.3 H3, simple lowpass filter, stable
b = [2, 0, 1]
a = [1, -1/2, 0]
sys = signal.dlti(b, a, dt=1)  # handle k as integer, i.e. unit time step

N = 10
plt.figure(figsize=(6, 5))
ax = plt.subplot(2, 1, 1)
[k, h] = signal.dimpulse(sys, n=N)
h = np.squeeze(h)
plt.stem(k, h, use_line_collection=True,
         linefmt='C0:', markerfmt='C0o', basefmt='C0:')
ax.xaxis.set_major_locator(MaxNLocator(integer=True))
# plt.xlabel(r'$k$')
plt.ylabel(r'$h[k]$')
plt.title('Impulse response')
plt.grid(True)

ax = plt.subplot(2, 1, 2)
[k, he] = signal.dstep(sys, n=N)
he = np.squeeze(he)
plt.stem(k, he, use_line_collection=True,
         linefmt='C0:', markerfmt='C0o', basefmt='C0:')
ax.xaxis.set_major_locator(MaxNLocator(integer=True))
plt.xlabel(r'$k$')
plt.ylabel(r'$h_\epsilon[k]$')
plt.title('Step response')
plt.grid(True)
```

```python
[w, level, phase] = signal.dbode(sys, w=w)

plt.figure(figsize=(6, 5))
ax = plt.subplot(2, 1, 1)
plt.plot(w / 2 / np.pi, level)
plt.xticks(np.arange(0, 1, 0.125))
plt.xlim(0, 1)
#plt.xlabel(r'$\frac{\Omega}{2\pi}= \frac{f}{f_s}$')
plt.ylabel(r'Level in dB')
plt.grid(True)

ax = plt.subplot(2, 1, 2)
plt.plot(w / 2 / np.pi, phase)
plt.xticks(np.arange(0, 1, 0.125))
plt.xlim(0, 1)
plt.xlabel(r'$\frac{\Omega}{2\pi}= \frac{f}{f_s}$')
plt.ylabel(r'Phase in degree')
plt.grid(True)
```

For the last example, consider that this is an system that was designed (for example to simulate an old-fashioned analog tape recorder frequency repsonse) for $f_s = 44100$ Hz, i.e. the sampling frequency of the old-fashioned CD. What would be the -3dB cutoff-frequency and the maximum damping (smallest dB value) of this very simple lowpass filter.

For that we need to re-plot the graphs with proper numbering / labeling of the physical frequency. Since we know that $\Omega=2\pi \frac{f}{f_s}$, we should rearrange to $f = \frac{\Omega}{2\pi}\cdot f_s$.

Thus:

```python
fs = 44100  # sampling frequency in Hz

plt.figure(figsize=(6, 5))
ax = plt.subplot(2, 1, 1)
plt.plot(w / 2 / np.pi * fs / 1000, level)
plt.xlim(0, fs/2/1000)
#plt.xlabel(r'frequency in kHz (or f / kHz)')
plt.ylabel(r'Level in dB')
plt.grid(True)

ax = plt.subplot(2, 1, 2)
plt.plot(w / 2 / np.pi * fs / 1000, phase)
plt.xlim(0, fs/2/1000)
plt.xlabel(r'frequency in kHz (or f / kHz)')
plt.ylabel(r'Phase in degree')
plt.grid(True)
```

From the plot we might observe that the cut-frequency (-3 dB off to maximum at 0 Hz) is roughly at 3.7 kHz. The minimum magnitude of about -1 dB occurs at ca. 11.4 kHz.

For audio, since our hearing resolution is logarithmic for most of the frequency range, a logarithmic frequency axis is meaningfully used. We know this from Bode plots concept as well. The above plot then is visualized as:

```python
plt.figure(figsize=(6, 5))
ax = plt.subplot(2, 1, 1)
plt.semilogx(w / 2 / np.pi * fs, level)
plt.xlim(20e1, 20e3)
plt.xticks([20, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000],
           [20, 50, 100, 200, 500, '1k', '2k', '5k', '10k', '20k'])
#plt.xlabel(r'frequency in Hz (or f / Hz)')
plt.ylabel(r'Level in dB')
plt.grid(True)

ax = plt.subplot(2, 1, 2)
plt.semilogx(w / 2 / np.pi * fs, phase)
plt.xlim(20e1, 20e3)
plt.xticks([20, 50, 100, 200, 500, 1000, 2000, 5000, 10000, 20000],
           [20, 50, 100, 200, 500, '1k', '2k', '5k', '10k', '20k'])
plt.xlabel(r'frequency in Hz (or f / Hz)')
plt.ylabel(r'Phase in degree')
plt.grid(True)
```

You might elaborate that the two purely imaginary zeros at $\pm \frac{\mathrm{j}}{\sqrt{2}}$ are responsible for the damping at about $f_s / 4=11025$ Hz, i.e. $\Omega=\frac{\pi}{2}$.

## Copyright

This tutorial is provided as Open Educational Resource (OER), to be found at
https://github.com/spatialaudio/signals-and-systems-exercises
accompanying the OER lecture
https://github.com/spatialaudio/signals-and-systems-lecture.
Both are licensed under a) the Creative Commons Attribution 4.0 International
License for text and graphics and b) the MIT License for source code.
Please attribute material from the tutorial as *Frank Schultz,
Continuous- and Discrete-Time Signals and Systems - A Tutorial Featuring
Computational Examples, University of Rostock* with
``main file, github URL, commit number and/or version tag, year``.


# Compute fixedpoint of $f(x, y)= [x^2 + y^3; xy - 0.5]$

**Randall Romero Aguilar, PhD**

This demo is based on the original Matlab demo accompanying the  Computational Economics and Finance 2001 textbook by Mario Miranda and Paul Fackler.

Original (Matlab) CompEcon file: **demslv04.m**

Running this file requires the Python version of CompEcon. This can be installed with pip by running

    !pip install compecon --upgrade

Last updated: 2021-Oct-01

## About

Compute fixedpoint of

\begin{equation}
f(x, y)= \begin{bmatrix}x^2 + y^3 \\ xy - 0.5 \end{bmatrix}
\end{equation}

using Newton, Broyden, and function iteration methods.

Initial values generated randomly.  Some algorithms may fail to converge, depending on the initial value.

True fixedpoint is $x = -0.09$,  $y=-0.46$.

```python
import numpy as np
import pandas as pd
from compecon import NLP, tic, toc
np.random.seed(12)
```

### Set up the problem

```python
def g(z):
    x, y = z
    return np.array([x **2 + y ** 3, x * y - 0.5])

problem_as_fixpoint = NLP(g, maxit=1500)
```

### Equivalent Rootfinding Formulation

```python
def f(z):
    x, y = z
    fval = [x - x ** 2 - y ** 3,
            y - x * y + 0.5]
    fjac = [[1 - 2 * x, -3 * y **2],
            [-y, 1 - x]]

    return np.array(fval), np.array(fjac)

problem_as_zero = NLP(f, maxit=1500)
```

### Randomly generate starting point

```python
xinit = np.random.randn(2)
```

### Compute fixed-point using Newton method

```python
t0 = tic()
z1 = problem_as_zero.newton(xinit)
t1 = 100 * toc(t0)
n1 = problem_as_zero.fnorm
```

### Compute fixed-point using Broyden method

```python
t0 = tic()
z2 = problem_as_zero.broyden(xinit)
t2 = 100 * toc(t0)
n2 = problem_as_zero.fnorm
```

### Compute fixed-point using function iteration

```python
t0 = tic()
z3 = problem_as_fixpoint.fixpoint(xinit)
t3 = 100 * toc(t0)
n3 = np.linalg.norm(problem_as_fixpoint.fx - z3)
```

```python
print('Hundredths of seconds required to compute fixed-point of ')
print('\n\t\tg(x1,x2)=[x1^2+x2^3; x1*x2-0.5]')
print('\nusing Newton, Broyden, and function iteration methods, starting at')
print('\n\t\tx1 = {:4.2f}  x2 = {:4.2f}\\n\\n'.format(*xinit))

pd.DataFrame({
    'Time': [t1, t2, t3],
    'Norm of f': [n1, n2, n3],
    'x1': [z1[0], z2[0], z3[0]],
    'x2': [z1[1], z2[1], z3[1]]},
    index=['Newton', 'Broyden', 'Function']
)
```

    Hundredths of seconds required to compute fixed-point of

    		g(x1,x2)=[x1^2+x2^3; x1*x2-0.5]

    using Newton, Broyden, and function iteration methods, starting at

    		x1 = 0.47  x2 = -0.68\n\n

      Time
      Norm of f
      x1
      x2

      Newton
      0.102687
      1.665335e-15
      -0.088909
      -0.459175

      Broyden
      0.598288
      8.015544e-10
      -0.088909
      -0.459175

      Function
      0.099659
      8.421821e-09
      -0.088909
      -0.459175

_build/jupyter_execute/notebooks/slv/04 Compute fixedpoint of f(x1,x2)= [x1 2+x2 3; x1 x2 - 0.5].ipynb


```python
import sympy as sp
sp.init_printing()
```

Analyitc solution for CSTR of 2 A -> B

```python
symbs = t, f, fcA, fcB, IA, IB, k, c1, c2 = sp.symbols('t f phi_A phi_B I_A I_B k c1 c2', real=True, positive=True)
symbs
```

```python
def analytic(t, f, fcA, fcB, IA, IB, k, c1, c2):
    u = sp.sqrt(f*(f + 4*fcA*k))
    v = u*sp.tanh(u/2*(t - c1))
    return [
        (-f + v)/(2*k),
        sp.exp(-f*t)*c2 + (f + 2*(fcA + fcB)*k - v)/(2*k)
    ]
```

```python
exprs = analytic(*symbs)
exprs
```

```python
exprs0 = [expr.subs(t, 0) for expr in exprs]
exprs0
```

```python
sol = sp.solve([expr - c0 for expr, c0 in zip(exprs0, [IA, IB])], [c1, c2])
sol
```

```python
exprs2 = [expr.subs(dict(zip([c1, c2], sol[1]))) for expr in exprs]
exprs2
```

```python
[expr.subs(t, 0).simplify() for expr in exprs2]
```

```python
%matplotlib inline
import numpy as np
from chempy import ReactionSystem
from chempy.kinetics.ode import get_odesys
rsys = ReactionSystem.from_string("OH + OH -> H2O2; 'k'")
cstr, extra = get_odesys(rsys, include_params=False, cstr=True)
fr, fc = extra['cstr_fr_fc']
print(cstr.names, cstr.param_names)
cstr.exprs
```

```python
args = tout, c0, params = np.linspace(0, .17), {'OH': 2, 'H2O2': 3}, {'k': 5, fc['OH']: 42, fc['H2O2']: 11, fr: 13}
res = cstr.integrate(*args)
res.plot()
```

```python
def analytic_alt(t, f, fcA, fcB, IA, IB, k):
    u = sp.sqrt(f*(f + 4*fcA*k))
    q = sp.atanh(-u*(sp.sqrt(f) + 2*k*IA)/(f*(f + 4*fcA*k)))
    v = u*sp.tanh(u/2*t - q)
    w = sp.exp(-f*t)/2/k
    y = 2*k*(fcA + fcB)
    return [
        (-f + v)/(2*k),
        w*(sp.exp(f*t)*(f + y - v) - y + 2*k*(IA + IB))
    ]
```

```python
def analytic_alt2(t, f, fcA, fcB, IA, IB, k, n):
    one_point_five = sp.S(3)/2
    a0, b0, x, y = fcA, fcB, IA, IB
    Sqrt, Tanh, ArcTanh, E = sp.sqrt, sp.tanh, sp.atanh, sp.E
    return [
        (-f + Sqrt(f)*Sqrt(f + 8*a0*k)*
     Tanh((Sqrt(f)*Sqrt(f + 8*a0*k)*t - 2*ArcTanh((-(f**one_point_five*Sqrt(f + 8*a0*k)) - 4*Sqrt(f)*k*Sqrt(f + 8*a0*k)*x)/(f**2 + 8*a0*f*k)))/2)
    )/(4*k),

    (-8*b0*k + 8*b0*E**(f*t)*k + E**(f*t)*f*n - 4*a0*k*n + 4*a0*E**(f*t)*k*n + 4*k*n*x + 8*k*y -
    E**(f*t)*Sqrt(f)*Sqrt(f + 8*a0*k)*n*Tanh((Sqrt(f)*Sqrt(f + 8*a0*k)*
         (t - (2*ArcTanh((-(f**one_point_five*Sqrt(f + 8*a0*k)) - 4*Sqrt(f)*k*Sqrt(f + 8*a0*k)*x)/(f**2 + 8*a0*f*k)))/(Sqrt(f)*Sqrt(f + 8*a0*k))))
        /2))/(8*E**(f*t)*k)
    ]
#     return [
#     (-f + Sqrt(f)*Sqrt(f + 8*a0*k)*
#    Tanh((Sqrt(f)*Sqrt(f + 8*a0*k)*t - 2*ArcTanh((-(f**one_point_five*Sqrt(f + 8*a0*k)) - 4*Sqrt(f)*k*Sqrt(f + 8*a0*k)*x)/(f**2 + 8*a0*f*k)))/2)
#   )/(4*k),
# (E**(f*t)*f - 4*a0*k - 8*b0*k + 4*a0*E**(f*t)*k + 8*b0*E**(f*t)*k + 4*k*x + 8*k*y -
#   E**(f*t)*Sqrt(f)*Sqrt(f + 8*a0*k)*Tanh((Sqrt(f)*Sqrt(f + 8*a0*k)*
#        (t - (2*ArcTanh((-(f**one_point_five*Sqrt(f + 8*a0*k)) - 4*Sqrt(f)*k*Sqrt(f + 8*a0*k)*x)/(f**2 + 8*a0*f*k)))/(Sqrt(f)*Sqrt(f + 8*a0*k))))
#       /2))/(8*E**(f*t)*k)
#     ]

```

```python
n = sp.Symbol('n')
exprs_alt = analytic_alt2(*symbs[:-2], n)
exprs_alt
```

```python
cses, expr_cse = sp.cse([expr.subs({fcA: sp.Symbol('fr'), fcB: sp.Symbol('fp'), f: sp.Symbol('fv'),
                                     IA: sp.Symbol('r'), IB: sp.Symbol('p')}) for expr in exprs_alt])
```

```python
print(
    '\n'.join(['%s = %s' % (lhs, rhs) for lhs, rhs in cses] + ['return (\n    %s\n)' % str(expr_cse)[1:-1]]).replace(
        '3/2', 'three/2').replace(
        'exp', 'be.exp').replace(
        'sqrt', 'be.sqrt').replace(
        'atanh', 'ATANH').replace(
        'tanh', 'be.tanh\n    ').replace(
        'ATANH', 'atanh')
)
```

```python
[expr.subs(t, 0).simplify() for expr in exprs_alt]
```

```python
[expr.diff(t).subs(t, 0).simplify() for expr in exprs_alt]
```

```python
print(list(rsys.substances))
rsys
```

```python
print(cstr.names, cstr.param_names)
cstr.exprs
```

```python
symbs[:-2]
```

```python
_cb = sp.lambdify(symbs[:-2] + (n,), exprs_alt)
def calc_analytic(xout, y0, p):
    return _cb(xout, p[fr], p[fc['OH']], p[fc['H2O2']], y0['OH'], y0['H2O2'], p['k'], 1)
```

```python
def get_analytic(result):
    ref = calc_analytic(
        result.xout,
        {k: res.get_dep(k)[0] for k in result.odesys.names},
        {k: res.get_param(k) for k in result.odesys.param_names})
    return np.array([ref[{'OH': 0, 'H2O2': 1}[k]] for k in result.odesys.names]).T
```

```python
yref = get_analytic(res)
print(yref.shape)
```

```python
res.plot()
res.plot(y=yref)
```

```python
res.plot(y=res.yout - yref)
```

```python
from chempy.kinetics.integrated import binary_irrev_cstr
```

```python
def get_analytic2(result):
    ref = binary_irrev_cstr(result.xout, result.get_param('k'), result.get_dep('OH')[0],
                           result.get_dep('H2O2')[0], result.get_param(fc['OH']), result.get_param(fc['H2O2']),
                           result.get_param(fr))
    return np.array([ref[{'OH': 0, 'H2O2': 1}[k]] for k in result.odesys.names]).T
```

```python
res.plot(y=res.yout - get_analytic2(res))
```


###### Content provided under a Creative Commons Attribution license, CC-BY 4.0; code under BSD 3-Clause license. (c)2014 Lorena A. Barba, Olivier Mesnard. Thanks: NSF for support via CAREER award #1149784.

# Source panel method

We are now getting close to the finish line with *AeroPython*! Our first few lessons introduced the fundamental flow solutions of potential flow, and we quickly learned that using our superposition powers we could get some useful results in aerodynamics.

The superposition of a [doublet](03_Lesson03_doublet.ipynb) and a free stream gave the flow around a circular cylinder, and we learned about the *D'Alembert paradox*: the result of zero drag for potential flow around a cylinder. Adding a [vortex](06_Lesson06_vortexLift.ipynb) at the center of the cylinder, we learned about lift and the *Kutta-Joukowski theorem* stating that lift is proportional to circulation: $L=\rho U \Gamma$. A most important result!

Adding together fundamental solutions of potential flow and seeing what we get when interpreting a dividing streamline as a solid body is often called an *indirect method*. This method goes all the way back to Rankine in 1871! But its applicability is limited because we can't stipulate a geometry and find the flow associated to it.

In [Lesson 9](09_Lesson09_flowOverCylinder.ipynb), we learned that it is possible to stipulate first the geometry, and then solve for the source strengths on a panel discretization of the body that makes the flow tangent at the boundary. This is called a *direct method* and it took off in the 1960s with the work of Hess and Smith at Douglas Aircraft Company.

A set of panels (line segments in 2D) can represent the surface of any solid body immersed in a potential flow by making the source-sheet strengths such that the normal velocity at each panel is equal to zero. This is a very powerful idea! But you should realize that all the panel strengths are coupled to each other, which is why we end up with a linear system of equations.

For an arbitrary geometry, we need to build a set of panels according to some points that define  the geometry. In this lesson, we will read from a file a geometry definition corresponding to a **NACA0012 airfoil**, create a set of panels, and solve for the source-sheet strengths to get flow around the airfoil.

*Make sure you have studied [Lesson 9](09_Lesson09_flowOverCylinder.ipynb) carefully before proceeding!* We will not repeat the full mathematical formulation in this notebook, so refer back as needed.

First, load our favorite Python libraries, and the `integrate` module from SciPy:

```python
import os
import math
import numpy
from scipy import integrate
from matplotlib import pyplot
# display the figures in the Notebook
%matplotlib inline
```

Next, we read the body geometry from a file using the NumPy function [`loadtxt()`](http://docs.scipy.org/doc/numpy/reference/generated/numpy.loadtxt.html). The file comes from the [Airfoil Tools](http://airfoiltools.com/airfoil/details?airfoil=n0012-il) website and it contains a set of coordinates for the standard NACA0012 symmetric profile. We saved the file in the `resources` folder and load it from our local copy.

The geometry points get loaded into one NumPy array, so we separate the data into two arrays: `x,y` (for better code readability). The subsequent code will plot the geometry of the airfoil.

```python
# read of the geometry from a data file
naca_filepath = os.path.join('resources', 'naca0012.dat')
with open (naca_filepath, 'r') as file_name:
    x, y = numpy.loadtxt(file_name, dtype=float, delimiter='\t', unpack=True)

# plot the geometry
width = 10
fig = pyplot.figure(figsize=(width, width))
pyplot.grid()
pyplot.xlabel('x', fontsize=16)
pyplot.ylabel('y', fontsize=16)
pyplot.plot(x, y, color='k', linestyle='-', linewidth=2)
# pyplot.axis('scaled', adjustable='box')
pyplot.axis('scaled')
fig.axes[0].axes.set_adjustable(adjustable = 'box')
pyplot.xlim(-0.1, 1.1)
pyplot.ylim(-0.1, 0.1);
```

## Discretization into panels

Like in [Lesson 9](09_Lesson09_flowOverCylinder.ipynb), we will create a discretization of the body geometry into panels (line segments in 2D). A panel's attributes are: its starting point, end point and mid-point, its length and its orientation. See the following figure for the nomenclature used in the code and equations below.

We can modify the `Panel` class from our previous notebook slightly, to work better for our study of flow over an airfoil. The only difference is that we identify points on the top or bottom surfaces with the words `upper` and `lower`, which is only used later for plotting results with different colors for the top and bottom surfaces of the profile.

```python
class Panel:
    """
    Contains information related to a panel.
    """
    def __init__(self, xa, ya, xb, yb):
        """
        Initializes the panel.

        Sets the end-points and calculates the center, length,
        and angle (with the x-axis) of the panel.
        Defines if the panel is on the lower or upper surface of the geometry.
        Initializes the source-sheet strength, tangential velocity,
        and pressure coefficient to zero.

        Parameters
        ----------
        xa: float
            x-coordinate of the first end-point.
        ya: float
            y-coordinate of the first end-point.
        xb: float
            x-coordinate of the second end-point.
        yb: float
            y-coordinate of the second end-point.
        """
        self.xa, self.ya = xa, ya
        self.xb, self.yb = xb, yb

        self.xc, self.yc = (xa + xb) / 2, (ya + yb) / 2  # control-point (center-point)
        self.length = math.sqrt((xb - xa)**2 + (yb - ya)**2)  # length of the panel

        # orientation of the panel (angle between x-axis and panel's normal)
        if xb - xa <= 0.0:
            self.beta = math.acos((yb - ya) / self.length)
        elif xb - xa > 0.0:
            self.beta = math.pi + math.acos(-(yb - ya) / self.length)

        # location of the panel
        if self.beta <= math.pi:
            self.loc = 'upper'
        else:
            self.loc = 'lower'

        self.sigma = 0.0  # source strength
        self.vt = 0.0  # tangential velocity
        self.cp = 0.0  # pressure coefficient
```

For the circular cylinder, the discretization into panels was really easy. This is the part that gets more complicated when you want to compute the flow around a general geometry, while the solution part is effectively the same as in [Lesson 9](09_Lesson09_flowOverCylinder.ipynb).

The function below will create the panels from the geometry data that was read from a file. It is better to have small panels near the leading-edge and the trailing edge, where the curvature is large. One method to get a non uniform distribution around the airfoil is to first discretize a circle with diameter equal to the airfoil's chord, with the leading edge and trailing edge touching the circle at a node, as shown in the following sketch.

Then, we store the $x$-coordinates of the circle points, `x_circle`, which will also be the $x$-coordinates of the panel nodes, `x`, and project the $y$-coordinates of the circle points onto the airfoil by interpolation. We end up with a node distribution on the airfoil that is refined near the leading edge and the trailing edge. It will look like this:

With the discretization method just described, the function `define_panels()` returns an array of objects, each an instance of the class `Panel` and containing all information about a panel, given the desired number of panels and the set of body coordinates.

A few remarks about the implementation of the function `define_panels()`:

* we just need to compute the $x$-coordinates of the circle (`x_circle`) since the $y$-coordinates of the panel nodes will be computed by interpolation;
* we create a circle with `N+1` points, but the first and last points coincide;
* we extend our NumPy arrays by adding an extra value that is equal to the first one; thus we don't have to do anything special with the value `x[i+1]` in the different loops;
* the *while*-loop is used to find two consecutive points, (`x[I]`,`y[I]`) and (`x[I+1]`,`y[I+1]`), on the foil such that the interval [`x[I]`,`x[I+1]`] contains the value `x_ends[i]`; we use the keyword `break` to get out of the loop;
* once the two points have been identified, the value `y_ends[i]` is computed by interpolation.

```python
def define_panels(x, y, N=40):
    """
    Discretizes the geometry into panels using the 'cosine' method.

    Parameters
    ----------
    x: 1D array of floats
        x-coordinate of the points defining the geometry.
    y: 1D array of floats
        y-coordinate of the points defining the geometry.
    N: integer, optional
        Number of panels;
        default: 40.

    Returns
    -------
    panels: 1D Numpy array of Panel objects
        The discretization of the geometry into panels.
    """
    R = (x.max() - x.min()) / 2  # radius of the circle
    x_center = (x.max() + x.min()) / 2  # x-coord of the center
    # define x-coord of the circle points
    x_circle = x_center + R * numpy.cos(numpy.linspace(0.0, 2 * math.pi, N + 1))

    x_ends = numpy.copy(x_circle)  # projection of the x-coord on the surface
    y_ends = numpy.empty_like(x_ends)  # initialization of the y-coord Numpy array

    x, y = numpy.append(x, x[0]), numpy.append(y, y[0])  # extend arrays using numpy.append

    # computes the y-coordinate of end-points
    I = 0
    for i in range(N):
        while I < len(x) - 1:
            if (x[I] <= x_ends[i] <= x[I + 1]) or (x[I + 1] <= x_ends[i] <= x[I]):
                break
            else:
                I += 1
        a = (y[I + 1] - y[I]) / (x[I + 1] - x[I])
        b = y[I + 1] - a * x[I + 1]
        y_ends[i] = a * x_ends[i] + b
    y_ends[N] = y_ends[0]

    panels = numpy.empty(N, dtype=object)
    for i in range(N):
        panels[i] = Panel(x_ends[i], y_ends[i], x_ends[i + 1], y_ends[i + 1])

    return panels
```

Now we can use this function, calling it with a desired number of panels whenever we execute the cell below. We also plot the resulting geometry.

```python
import matplotlib
N = 40                           # number of panels
panels = define_panels(x, y, N)  # discretizes of the geometry into panels

# plot the geometry and the panels
width = 10
fig = pyplot.figure(figsize=(width, width))
pyplot.grid()
pyplot.xlabel('x', fontsize=16)
pyplot.ylabel('y', fontsize=16)
pyplot.plot(x, y, color='k', linestyle='-', linewidth=2)
pyplot.plot(numpy.append([panel.xa for panel in panels], panels[0].xa),
            numpy.append([panel.ya for panel in panels], panels[0].ya),
            linestyle='-', linewidth=1, marker='o', markersize=6, color='#CD2305')
# https://matplotlib.org/3.1.1/api/_as_gen/matplotlib.axes.Axes.set_adjustable.html
pyplot.axis('scaled')
# fig.ax.Axes.set_adjustable(adjustable = 'box')
fig.axes[0].axes.set_adjustable(adjustable = 'box')
pyplot.xlim(-0.1, 1.1)
pyplot.ylim(-0.1, 0.1);
```

```python

```

## Freestream conditions

The NACA0012 airfoil will be immersed in a uniform flow with velocity $U_\infty$ and an angle of attack $\alpha=0$. Even though it may seem like overkill to create a class for the freestream, we'll do it anyway. When creating a class, one is expecting to also create several instances of its objects. Here, we just have one freestream, so why define a class? Well, it makes the code more readable and does not block the programmer from using the variable names `u_inf` and `alpha` for something else outside of the class.
Also, every time we need the freestream condition as input to a function, we will just have to pass the object as an argument and not all the attributes of the freestream.

```python
class Freestream:
    """
    Freestream conditions.
    """
    def __init__(self, u_inf=1.0, alpha=0.0):
        """
        Sets the freestream speed and angle (with the x-axis).

        Parameters
        ----------
        u_inf: float, optional
            Freestream speed;
            default: 1.0.
        alpha: float, optional
            Angle of attack in degrees;
            default: 0.0.
        """
        self.u_inf = u_inf
        self.alpha = numpy.radians(alpha)  # degrees --> radians
```

```python
# define and creates the object freestream
u_inf = 1.0                            # freestream spee
alpha = 0.0                            # angle of attack (in degrees)
freestream = Freestream(u_inf, alpha)  # instantiation of the object freestream
```

## Flow tangency boundary condition

Enforcing the flow-tangency condition on each *control point* approximately makes the body geometry correspond to a dividing streamline (and the approximation improves if we represented the body with more and more panels). So, for each panel $i$, we make $u_n=0$ at $(x_{c_i},y_{c_i})$, which leads to the equation derived in the previous lesson:

$$
\begin{equation}
u_{n_i} = \frac{\partial}{\partial n_i}\left\lbrace \phi\left(x_{c_i},y_{c_i}\right) \right\rbrace = 0
\end{equation}
$$

i.e.

$$
\begin{equation}
\begin{split}
0 = & U_\infty \cos\beta_i + \frac{\sigma_i}{2} \\
& + \sum_{j=1,j\neq i}^{N_p} \frac{\sigma_j}{2\pi} \int \frac{\left(x_{c_i}-x_j(s_j)\right) \cos\beta_i + \left(y_{c_i}-y_j(s_j)\right) \sin\beta_i}{\left(x_{c_i}-x_j(s)\right)^2 + \left(y_{c_i}-y_j(s)\right)^2} {\rm d}s_j
\end{split}
\end{equation}
$$

In the equation above, we calculate the derivative of the potential in the normal direction to enforce the flow tangency condition on each panel. But later, we will have to calculate the derivative in the tangential direction to compute the surface pressure coefficient. And, when we are interested in plotting the velocity field onto a mesh, we will have to calculate the derivative in the $x$- and $y$-direction.

Therefore the function below is similar to the one implemented in [Lesson 9](09_Lesson09_flowOverCylinder.ipynb) to obtain the integrals along each panel, but we've generalized it to adapt to the direction of derivation (by means of two new arguments, `dxdz` and `dydz`, which respectively represent the value of $\frac{\partial x_{c_i}}{\partial z_i}$ and $\frac{\partial y_{c_i}}{\partial z_i}$, $z_i$ being the desired direction).

Moreover, the function is also more general in the sense of allowing any evaluation point, not just a control point on a panel (the argument `p_i` has been replaced by the coordinates `x` and `y` of the control-point, and `p_j` has been replaced with `panel`).

```python
def integral(x, y, panel, dxdz, dydz):
    """
    Evaluates the contribution of a panel at one point.

    Parameters
    ----------
    x: float
        x-coordinate of the target point.
    y: float
        y-coordinate of the target point.
    panel: Panel object
        Source panel which contribution is evaluated.
    dxdz: float
        Derivative of x in the z-direction.
    dydz: float
        Derivative of y in the z-direction.

    Returns
    -------
    Integral over the panel of the influence at the given target point.
    """
    def integrand(s):
        return (((x - (panel.xa - math.sin(panel.beta) * s)) * dxdz +
                 (y - (panel.ya + math.cos(panel.beta) * s)) * dydz) /
                ((x - (panel.xa - math.sin(panel.beta) * s))**2 +
                 (y - (panel.ya + math.cos(panel.beta) * s))**2) )
    return integrate.quad(integrand, 0.0, panel.length)[0]
```

## Building the linear system

Here, we build and solve the linear system of equations of the form

$$
\begin{equation}
[A][\sigma] = [b]
\end{equation}
$$

In building the matrix, below, we call the `integral()` function with the correct values for the last parameters: $\cos \beta_i$ and $\sin\beta_i$, corresponding to a derivative in the normal direction.

Finally, we use `linalg.solve()` from NumPy to solve the system and find the strength of each panel.

```python
def build_matrix(panels):
    """
    Builds the source matrix.

    Parameters
    ----------
    panels: 1D array of Panel object
        The source panels.

    Returns
    -------
    A: 2D Numpy array of floats
        The source matrix (NxN matrix; N is the number of panels).
    """
    N = len(panels)
    A = numpy.empty((N, N), dtype=float)
    numpy.fill_diagonal(A, 0.5)

    for i, p_i in enumerate(panels):
        for j, p_j in enumerate(panels):
            if i != j:
                A[i, j] = 0.5 / math.pi * integral(p_i.xc, p_i.yc, p_j,
                                                   math.cos(p_i.beta),
                                                   math.sin(p_i.beta))

    return A

def build_rhs(panels, freestream):
    """
    Builds the RHS of the linear system.

    Parameters
    ----------
    panels: 1D array of Panel objects
        The source panels.
    freestream: Freestream object
        The freestream conditions.

    Returns
    -------
    b: 1D Numpy array of floats
        RHS of the linear system.
    """
    b = numpy.empty(len(panels), dtype=float)

    for i, panel in enumerate(panels):
        b[i] = -freestream.u_inf * math.cos(freestream.alpha - panel.beta)

    return b
```

```python
A = build_matrix(panels)           # compute the singularity matrix
b = build_rhs(panels, freestream)  # compute the freestream RHS
```

```python
# solve the linear system
sigma = numpy.linalg.solve(A, b)

for i, panel in enumerate(panels):
    panel.sigma = sigma[i]
```

## Surface pressure coefficient

From Bernoulli's equation, the pressure coefficient on the $i$-th panel is

$$
\begin{equation}
C_{p_i} = 1-\left(\frac{u_{t_i}}{U_\infty}\right)^2
\end{equation}
$$

where $u_{t_i}$ is the tangential component of the velocity at the center point of the $i$-th panel,

$$
\begin{equation}
\begin{split}
u_{t_i} = & -U_\infty \sin\beta_i \\
& + \sum_{j=1}^{N_p} \frac{\sigma_j}{2\pi} \int \frac{\left(x_{c_i}-x_j(s_j)\right) \frac{\partial x_{c_i}}{\partial t_i} + \left(y_{c_i}-y_j(s_j)\right) \frac{\partial y_{c_i}}{\partial t_i}}{\left(x_{c_i}-x_j(s)\right)^2 + \left(y_{c_i}-y_j(s)\right)^2} {\rm d}s_j
\end{split}
\end{equation}
$$

with

$$
\begin{equation}
\frac{\partial x_{c_i}}{\partial t_i} = -\sin\beta_i \quad\text{and} \quad \frac{\partial y_{c_i}}{\partial t_i} = \cos\beta_i
\end{equation}
$$

Notice that below we call the function `integral()` with different arguments: $-\sin\beta_i$ and $\cos\beta_i$ to get the derivation in the tangential direction.

```python
def get_tangential_velocity(panels, freestream):
    """
    Computes the tangential velocity on the surface of the panels.

    Parameters
    ---------
    panels: 1D array of Panel objects
        The source panels.
    freestream: Freestream object
        The freestream conditions.
    """
    N = len(panels)
    A = numpy.empty((N, N), dtype=float)
    numpy.fill_diagonal(A, 0.0)

    for i, p_i in enumerate(panels):
        for j, p_j in enumerate(panels):
            if i != j:
                A[i, j] = 0.5 / math.pi * integral(p_i.xc, p_i.yc, p_j,
                                                   -math.sin(p_i.beta),
                                                   math.cos(p_i.beta))

    b = freestream.u_inf * numpy.sin([freestream.alpha - panel.beta
                                      for panel in panels])

    sigma = numpy.array([panel.sigma for panel in panels])

    vt = numpy.dot(A, sigma) + b

    for i, panel in enumerate(panels):
        panel.vt = vt[i]
```

```python
# compute the tangential velocity at the center-point of each panel
get_tangential_velocity(panels, freestream)
```

```python
def get_pressure_coefficient(panels, freestream):
    """
    Computes the surface pressure coefficients on the panels.

    Parameters
    ---------
    panels: 1D array of Panel objects
        The source panels.
    freestream: Freestream object
        The freestream conditions.
    """
    for panel in panels:
        panel.cp = 1.0 - (panel.vt / freestream.u_inf)**2
```

```python
# computes the surface pressure coefficients
get_pressure_coefficient(panels, freestream)
```

### Theoretical solution

There is a classical method to obtain the theoretical characteristics of airfoils, known as *Theodorsen's method*. It uses the Joukowski transformation but is able to deal with any airfoil by an additional transformation between a "near circle" and a circle. The method is hairy indeed! But the resulting values of pressure coefficient are provided for some airfoils in table form in the 1945 [NACA Report No.824](http://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/19930090976.pdf), available from the NASA web server (see p. 71).

The values of $(u/U_{\infty})^2$ are given for several stations along the chord length. We transcribed them here, saving them into an array:

```python
voverVsquared=numpy.array([0.0, 0.64, 1.01, 1.241, 1.378, 1.402, 1.411, 1.411,
                           1.399, 1.378, 1.35, 1.288, 1.228, 1.166, 1.109, 1.044,
                           0.956, 0.906, 0.0])
print(voverVsquared)
```

    [0.    0.64  1.01  1.241 1.378 1.402 1.411 1.411 1.399 1.378 1.35  1.288
     1.228 1.166 1.109 1.044 0.956 0.906 0.   ]

```python
xtheo=numpy.array([0.0, 0.5, 1.25, 2.5, 5.0, 7.5, 10.0, 15.0, 20.0, 25.0, 30.0,
                   40.0, 50.0, 60.0, 70.0, 80.0, 90.0, 95.0, 100.0])
xtheo /= 100
print(xtheo)
```

    [0.     0.005  0.0125 0.025  0.05   0.075  0.1    0.15   0.2    0.25
     0.3    0.4    0.5    0.6    0.7    0.8    0.9    0.95   1.    ]

### And plot the result!

We will use the values from the NACA report (also given in the book by Abbot and von Doenhoff, ["Theory of Wing Sections,"](http://books.google.com/books/about/Theory_of_Wing_Sections_Including_a_Summ.html?id=DPZYUGNyuboC) 1949) to visually compare the pressure distribution with the result of our source panel method. Let's see how it looks!

```python
# plot the surface pressure coefficient
pyplot.figure(figsize=(10, 6))
pyplot.grid()
pyplot.xlabel('x', fontsize=16)
pyplot.ylabel('$C_p$', fontsize=16)
pyplot.plot([panel.xc for panel in panels if panel.loc == 'upper'],
            [panel.cp for panel in panels if panel.loc == 'upper'],
            label='upper',
            color='r', linewidth=1, marker='x', markersize=8)
pyplot.plot([panel.xc for panel in panels if panel.loc == 'lower'],
            [panel.cp for panel in panels if panel.loc == 'lower'],
            label='lower',
            color='b', linewidth=0, marker='d', markersize=6)
pyplot.plot(xtheo, 1-voverVsquared,
            label='theoretical',
            color='k', linestyle='--',linewidth=2)
pyplot.legend(loc='best', prop={'size':14})
pyplot.xlim(-0.1, 1.1)
pyplot.ylim(1.0, -0.6)
pyplot.title('Number of panels : {}'.format(N));
```

That looks pretty good! The only place where the panel method doesn't quite match the tabulated data from Theordorsen's method is at the trailing edge. But note that the flow-tangency boundary condition in the panel method is applied at the control point of the panel (not at the endpoints), so this discrepancy is not surprising.

##### Accuracy check

For a closed body, the sum of all the source strengths must be zero. If not, it means the body would be adding or absorbing mass from the flow! Therefore, we should have

$$
\sum_{j=1}^{N} \sigma_j l_j = 0
$$

where $l_j$ is the length of the $j^{\text{th}}$ panel.

With this, we can get a get an idea of the accuracy of the source panel method.

```python
# calculate the accuracy
accuracy = sum([panel.sigma*panel.length for panel in panels])
print('--> sum of source/sink strengths: {}'.format(accuracy))
```

    --> sum of source/sink strengths: 0.0046170311752831

## Streamlines onto a mesh grid

To get a streamline plot, we have to create a mesh (like we've done in all *AeroPython* lessons!) and compute the velocity field onto it. Knowing the strength of every panel, we find the $x$-component of the velocity by taking derivative of the velocity potential in the $x$-direction, and the $y$-component by taking derivative in the $y$-direction:

$$
u\left(x,y\right) = \frac{\partial}{\partial x}\left\lbrace \phi\left(x,y\right) \right\rbrace
$$

$$
v\left(x,y\right) = \frac{\partial}{\partial y}\left\lbrace \phi\left(x,y\right) \right\rbrace
$$

Notice that here we call the function `integral()` with $1,0$ as the final arguments when calculating the derivatives in the $x$-direction, and $0,1$ for the derivatives in th $y$-direction.

In addition, we use the function `numpy.vectorize()` (as we did in [Lesson 8](08_Lesson08_sourceSheet.ipynb)) to avoid the nested loops over the domain.

```python
def get_velocity_field(panels, freestream, X, Y):
    """
    Computes the velocity field on a given 2D mesh.

    Parameters
    ---------
    panels: 1D array of Panel objects
        The source panels.
    freestream: Freestream object
        The freestream conditions.
    X: 2D Numpy array of floats
        x-coordinates of the mesh points.
    Y: 2D Numpy array of floats
        y-coordinate of the mesh points.

    Returns
    -------
    u: 2D Numpy array of floats
        x-component of the velocity vector field.
    v: 2D Numpy array of floats
        y-component of the velocity vector field.
    """
    # freestream contribution
    u = freestream.u_inf * math.cos(freestream.alpha) * numpy.ones_like(X, dtype=float)
    v = freestream.u_inf * math.sin(freestream.alpha) * numpy.ones_like(X, dtype=float)
    # add the contribution from each source (superposition powers!!!)
    vec_intregral = numpy.vectorize(integral)
    for panel in panels:
        u += panel.sigma / (2.0 * math.pi) * vec_intregral(X, Y, panel, 1.0, 0.0)
        v += panel.sigma / (2.0 * math.pi) * vec_intregral(X, Y, panel, 0.0, 1.0)

    return u, v
```

```python
# define a mesh grid
nx, ny = 20, 20  # number of points in the x and y directions
x_start, x_end = -1.0, 2.0
y_start, y_end = -0.3, 0.3
X, Y = numpy.meshgrid(numpy.linspace(x_start, x_end, nx),
                      numpy.linspace(y_start, y_end, ny))

# compute the velocity field on the mesh grid
u, v = get_velocity_field(panels, freestream, X, Y)
```

```python
# plot the velocity field
width = 10
fig = pyplot.figure(figsize=(width, width))
pyplot.xlabel('x', fontsize=16)
pyplot.ylabel('y', fontsize=16)
pyplot.streamplot(X, Y, u, v,
                  density=1, linewidth=1, arrowsize=1, arrowstyle='->')
pyplot.fill([panel.xc for panel in panels],
            [panel.yc for panel in panels],
            color='k', linestyle='solid', linewidth=2, zorder=2)
# pyplot.axis('scaled', adjustable='box')
pyplot.axis('scaled')
fig.axes[0].axes.set_adjustable(adjustable = 'box')
pyplot.xlim(x_start, x_end)
pyplot.ylim(y_start, y_end)
pyplot.title('Streamlines around a NACA 0012 airfoil (AoA = ${}^o$)'.format(alpha),
             fontsize=16);
```

We can now calculate the pressure coefficient. In Lesson 9, we computed the pressure coefficient on the surface of the circular cylinder. That was useful because we have an analytical solution for the surface pressure on a cylinder in potential flow. For an airfoil, we are interested to see how the pressure looks all around it, and we make a contour plot in the flow domain.

```python
# compute the pressure field
cp = 1.0 - (u**2 + v**2) / freestream.u_inf**2

# plot the pressure field
width = 10
pyplot.figure(figsize=(width, width))
pyplot.xlabel('x', fontsize=16)
pyplot.ylabel('y', fontsize=16)
contf = pyplot.contourf(X, Y, cp,
                        levels=numpy.linspace(-2.0, 1.0, 100), extend='both')
cbar = pyplot.colorbar(contf,
                       orientation='horizontal',
                       shrink=0.5, pad = 0.1,
                       ticks=[-2.0, -1.0, 0.0, 1.0])
cbar.set_label('$C_p$', fontsize=16)
pyplot.fill([panel.xc for panel in panels],
            [panel.yc for panel in panels],
            color='k', linestyle='solid', linewidth=2, zorder=2)
pyplot.axis('scaled', adjustable='box')
pyplot.xlim(x_start, x_end)
pyplot.ylim(y_start, y_end)
pyplot.title('Contour of pressure field', fontsize=16);
```

### Final words

We've learned to use a source-sheet to represent any solid body: first a [circular cylinder](09_Lesson09_flowOverCylinder.ipynb) (which we knew we could get by superposing a doublet and a freestream), and now an airfoil.

But what is the feature of airfoils that makes them interesting? Well, the fact that we can use them to generate lift and make things that fly, of course! But what do we need to generate lift? Think, think ... what is it?

## References

1. [Airfoil Tools](http://airfoiltools.com/index), website providing airfoil data.
1. Ira Herbert Abbott, Albert Edward Von Doenhoff and Louis S. Stivers, Jr. (1945), "Summary of Airfoil Data," NACA Report No.824, [PDF on the NASA web server](http://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/19930090976.pdf) (see p. 71)
1. Ira Herbert Abbott, Albert Edward Von Doenhoff, "Theory of Wing Sections, Including a Summary of Airfoil Data" (1949), Dover Press.

A further reference on Theodorsen's method is:

* Roland Schinzinger, Patricio A. A. Laura (1991), "Conformal Mapping: Methods and Applications." Dover edition in 2003. [Read on Google Books](https://books.google.com/books?id=qe-7AQAAQBAJ&lpg=PA128&ots=wbg0jLlqq5&dq=method%20theodorsen&pg=PA128#v=onepage&q=%22method%20of%20theodorsen%20and%20garrick%22&f=false)

---

###### Please ignore the cell below. It just loads our style for the notebook.

```python
from IPython.core.display import HTML
def css_styling(filepath):
    styles = open(filepath, 'r').read()
    return HTML(styles)
css_styling('../styles/custom.css')
```

<link href='http://fonts.googleapis.com/css?family=Fenix' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Alegreya+Sans:100,300,400,500,700,800,900,100italic,300italic,400italic,500italic,700italic,800italic,900italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Source+Code+Pro:300,400' rel='stylesheet' type='text/css'>
<style>

@font-face {
    font-family: "Computer Modern";
    src: url('http://mirrors.ctan.org/fonts/cm-unicode/fonts/otf/cmunss.otf');
}

#notebook_panel { /* main background */
    background: rgb(245,245,245);
}

div.cell { /* set cell width */
    width: 750px;
}

div #notebook { /* centre the content */
    background: #fff; /* white background for content */
    width: 1000px;
    margin: auto;
    padding-left: 0em;
}

#notebook li { /* More space between bullet points */
margin-top:0.8em;
}

/* draw border around running cells */
div.cell.border-box-sizing.code_cell.running {
    border: 1px solid #111;
}

/* Put a solid color box around each cell and its output, visually linking them*/
div.cell.code_cell {
    background-color: rgb(256,256,256);
    border-radius: 0px;
    padding: 0.5em;
    margin-left:1em;
    margin-top: 1em;
}

div.text_cell_render{
    font-family: 'Alegreya Sans' sans-serif;
    line-height: 140%;
    font-size: 125%;
    font-weight: 400;
    width:600px;
    margin-left:auto;
    margin-right:auto;
}

/* Formatting for header cells */
.text_cell_render h1 {
    font-family: 'Alegreya Sans', sans-serif;
    font-style:regular;
    font-weight: 200;
    font-size: 50pt;
    line-height: 100%;
    color:#CD2305;
    margin-bottom: 0.5em;
    margin-top: 0.5em;
    display: block;
}

.text_cell_render h2 {
    font-family: 'Fenix', serif;
    font-size: 22pt;
    line-height: 100%;
    margin-bottom: 0.1em;
    margin-top: 0.3em;
    display: block;
}

.text_cell_render h3 {
    font-family: 'Fenix', serif;
    margin-top:12px;
    font-size: 16pt;
    margin-bottom: 3px;
    font-style: regular;
}

.text_cell_render h4 {    /*Use this for captions*/
    font-family: 'Fenix', serif;
    font-size: 2pt;
    text-align: center;
    margin-top: 0em;
    margin-bottom: 2em;
    font-style: regular;
}

.text_cell_render h5 {  /*Use this for small titles*/
    font-family: 'Alegreya Sans', sans-serif;
    font-weight: 300;
    font-size: 16pt;
    color: #CD2305;
    font-style: italic;
    margin-bottom: .5em;
    margin-top: 0.5em;
    display: block;
}

.text_cell_render h6 { /*use this for copyright note*/
    font-family: 'Source Code Pro', sans-serif;
    font-weight: 300;
    font-size: 9pt;
    line-height: 100%;
    color: grey;
    margin-bottom: 1px;
    margin-top: 1px;
}

.CodeMirror{
        font-family: "Source Code Pro";
        font-size: 90%;
}

.warning{
    color: rgb( 240, 20, 20 )
    }
</style>