Text provided under a Creative Commons Attribution license, CC-BY. Code under MIT license.
(c)2014 Lorena A. Barba, Olivier Mesnard. Thanks: NSF for support via CAREER award #1149784.
[@LorenaABarba](https://twitter.com/LorenaABarba)

##### Version 0.3 -- February 2015

# Vortex

Let's recap. In the first three lessons of _AeroPython_, you computed:

1. [a source-sink pair](http://nbviewer.ipython.org/github/barbagroup/AeroPython/blob/master/lessons/01_Lesson01_sourceSink.ipynb)—you learned how to make a `streamplot()` and use `scatter()` to mark the location of the singular points. If you did your challenge task, you saw that equipotential lines are perpendicular to streamlines.

2. [a source-sink pair in a free stream](http://nbviewer.ipython.org/github/barbagroup/AeroPython/blob/master/lessons/02_Lesson02_sourceSinkFreestream.ipynb), creating a _Rankine oval_. For the first time, we see that we can model the flow around an object using potential flow solutions. You also learned to define custom functions, to make Python work for you and look even more like plain English.

3. [a doublet](http://nbviewer.ipython.org/github/barbagroup/AeroPython/blob/master/lessons/03_Lesson03_doublet.ipynb)—this one is wonderful, because the stream-line pattern gives the flow around a circular cylinder (in two dimensions). You encountered the _D'Alembert paradox_ and hopefully that got you thinking. (If not, go back to that lesson and _think_!)

Did you also do the [assignment](http://nbviewer.ipython.org/github/barbagroup/AeroPython/blob/master/lessons/03_Lesson03_Assignment.ipynb)? It shows you how a *distribution of sources* can be used to model potential flow around an  airfoil. This starts looking like aerodynamics!

But what is the most important thing we want from applied aerodynamics? We want to make things fly, of course! And to fly, there must be a force of *aerodynamic lift* to counteract the weight of the object.

In this section of the course, we learn about lift. First, we will compute the flow of a potential vortex. It turns out, vortex circulation and lift are intimately related.

## What's a vortex?

This question is deeper than you might think! The simple answer is that a vortex is motion in circular streamlines. Imagine streamlines that are concentric circles about a given point—what's confusing is that this _does not mean_ that fluid elements are themselves rotating!

In an irrotational vortex, the tangential velocity is constant along a (circular) streamline and inversely proportional to the radius, while the radial velocity is zero. In polar coordinates:

\begin{equation}
u_\theta\left(r,\theta\right) = \frac{\text{constant}}{r}  \quad \text{,} \quad  u_r\left(r,\theta\right) = 0
\end{equation}

The vorticity is zero everywhere, except at the location of the point vortex, where the derivative of $u_\theta$ is infinite.

We introduced the concept of circulation in the first lesson ([Source & Sink](http://nbviewer.ipython.org/urls/github.com/barbagroup/AeroPython/blob/master/lessons/01_Lesson01_sourceSink.ipynb)). Let's use that. Around any circular streamline enclosing the vortex, and using the sign convention that a negative vortex circulates anti-clockwise, we have:

\begin{equation}\Gamma = -\oint \mathbf{v}\cdot d\vec{l} = -u_\theta 2 \pi r\end{equation}

Thus, the constant in the expression for $u_\theta$ in $(1)$ is equal to $\Gamma/2\pi$, and we now write:

\begin{equation}u_\theta\left(r,\theta\right) = \frac{\Gamma}{2\pi r}\end{equation}

We can get the stream function by integrating the velocity components:

\begin{equation}\psi\left(r,\theta\right) = \frac{\Gamma}{2\pi}\ln r\end{equation}

In Cartesian coordinates, the stream function is

\begin{equation}\psi\left(x,y\right) = \frac{\Gamma}{4\pi}\ln\left(x^2+y^2\right)\end{equation}

while the velocity components would be:

\begin{equation}u\left(x,y\right) = \frac{\Gamma}{2\pi}\frac{y}{x^2+y^2} \qquad v\left(x,y\right) = -\frac{\Gamma}{2\pi}\frac{x}{x^2+y^2}\end{equation}

This vortex flow is irrotational everywhere, except at the vortex center, where it is infinite. The strenth of the point vortex is equal to the circulation $\Gamma$ around it.

## Let's compute a vortex

The set-up is the same as before: we load our favorite libraries, and we create a grid of points to evaluate the velocity field.

```python
import numpy
import math
from matplotlib import pyplot
```

```python
N = 50                                # Number of points in each direction
x_start, x_end = -2.0, 2.0            # x-direction boundaries
y_start, y_end = -1.0, 1.0            # y-direction boundaries
x = numpy.linspace(x_start, x_end, N)    # computes a 1D-array for x
y = numpy.linspace(y_start, y_end, N)    # computes a 1D-array for y
X, Y = numpy.meshgrid(x, y)              # generates a mesh grid
```

Give your vortex a strength $\Gamma=5$ and place it at the center of your domain:

```python
gamma = 5.0                      # strength of the vortex
x_vortex, y_vortex = 0.0, 0.0    # location of the vortex
```

We will define two functions,

* `get_velocity_vortex()` and
* `get_stream_function_vortex()`,

to compute the velocity components and the stream function on our Cartesian grid, given the strength and the location of the vortex. Then, we will use our custom functions to evaluate everything on the grid points. Let's write those functions first.

```python
def get_velocity_vortex(strength, xv, yv, X, Y):
    """Returns the velocity field generated by a vortex.

    Arguments
    ---------
    strength -- strength of the vortex.
    xv, yv -- coordinates of the vortex.
    X, Y -- mesh grid.
    """
    u = + strength/(2*math.pi)*(Y-yv)/((X-xv)**2+(Y-yv)**2)
    v = - strength/(2*math.pi)*(X-xv)/((X-xv)**2+(Y-yv)**2)

    return u, v
```

```python
def get_stream_function_vortex(strength, xv, yv, X, Y):
    """Returns the stream-function generated by a vortex.

    Arguments
    ---------
    strength -- strength of the vortex.
    xv, yv -- coordinates of the vortex.
    X, Y -- mesh grid.
    """
    psi = strength/(4*math.pi)*numpy.log((X-xv)**2+(Y-yv)**2)

    return psi
```

An now, call the functions with the vortex strength and position, plus the coordinates of the evaluation grid, to get the velocity and streamfunction of the vortex.

```python
# computes the velocity field on the mesh grid
u_vortex, v_vortex = get_velocity_vortex(gamma, x_vortex, y_vortex, X, Y)

# computes the stream-function on the mesh grid
psi_vortex = get_stream_function_vortex(gamma, x_vortex, y_vortex, X, Y)
```

We are now able to visualize the streamlines of a vortex, and they look like concentric circles around the vortex center, as expected.

```python
# plots the streamlines
%matplotlib inline

size = 10
pyplot.figure(figsize=(size, (y_end-y_start)/(x_end-x_start)*size))
pyplot.xlabel('x', fontsize=16)
pyplot.ylabel('y', fontsize=16)
pyplot.xlim(x_start, x_end)
pyplot.ylim(y_start, y_end)
pyplot.streamplot(X, Y, u_vortex, v_vortex, density=2, linewidth=1, arrowsize=1, arrowstyle='->')
pyplot.scatter(x_vortex, y_vortex, color='#CD2305', s=80, marker='o');
```

## Vortex & Sink

For fun, let's use our superposition powers. Add a vortex to a sink, using two new functions to compute the velocity components and the stream function of the sink, and adding to those of a vortex (remember that the sink can be easily replaced by a source by just changing the sign of the strength).

```python
strength_sink = -1.0            # strength of the sink
x_sink, y_sink = 0.0, 0.0       # location of the sink
```

```python
def get_velocity_sink(strength, xs, ys, X, Y):
    """Returns the velocity field generated by a sink.

    Arguments
    ---------
    strength -- strength of the sink.
    xs, ys -- coordinates of the sink.
    X, Y -- mesh grid.
    """
    u = strength/(2*math.pi)*(X-xs)/((X-xs)**2+(Y-ys)**2)
    v = strength/(2*math.pi)*(Y-ys)/((X-xs)**2+(Y-ys)**2)

    return u, v
```

```python
def get_stream_function_sink(strength, xs, ys, X, Y):
    """Returns the stream-function generated by a sink.

    Arguments
    ---------
    strength -- strength of the sink.
    xs, ys -- coordinates of the sink.
    X, Y -- mesh grid.
    """
    psi = strength/(2*math.pi)*numpy.arctan2((Y-ys), (X-xs))

    return psi
```

```python
# computes the velocity field on the mesh grid
u_sink, v_sink = get_velocity_sink(strength_sink, x_sink, y_sink, X, Y)

# computes the stream-function on the mesh grid
psi_sink = get_stream_function_sink(strength_sink, x_sink, y_sink, X, Y)
```

Now, let's visualize the streamlines of the vortex-sink, and admire our artistic creation:

```python
# superposition of the sink and the vortex
u = u_vortex + u_sink
v = v_vortex + v_sink
psi = psi_vortex + psi_sink

# plots the streamlines
size = 10
pyplot.figure(figsize=(size, (y_end-y_start)/(x_end-x_start)*size))
pyplot.xlabel('x', fontsize=16)
pyplot.ylabel('y', fontsize=16)
pyplot.xlim(x_start, x_end)
pyplot.ylim(y_start, y_end)
pyplot.streamplot(X, Y, u, v, density=2, linewidth=1, arrowsize=1, arrowstyle='->')
pyplot.scatter(x_vortex, y_vortex, color='#CD2305', s=80, marker='o');
```

Very cool op-art. (And a good model for your typical bath-tub vortex.) But is all this useful? Yes!

First, you will take your training wheels off and—on your own—compute the flow of an [infinite row of vortices](http://nbviewer.ipython.org/github/barbagroup/AeroPython/blob/master/lessons/05_Lesson05_InfiniteRowOfVortices.ipynb). Your mission, should you choose to accept it, is in the next notebook.

After that, we will learn about the connection between a vortex, and the force of lift. This turns out to be very important in aerodynamics!

## What's this "irrotational" vortex thing?

I know what you are thinking.

What does it mean that the vortex is *irrotational*? Surely if there's a vortex, there's rotation!

You are not crazy. It's natural to think this way, but the potential vortex is a flow where streamlines are circular, yet fluid elements do not rotate around themselves—they just go around the circular path.

This classic video will help you understand ... just watch the 25 seconds of video after time 4m 25s, and see a "vorticity meter" go around a free vortex without rotating itself.

```python
from IPython.display import YouTubeVideo
from datetime import timedelta

start=int(timedelta(hours=0, minutes=4, seconds=25).total_seconds())

YouTubeVideo("loCLkcYEWD4", start=start)
```

Remember: vorticity measures the local angular velocity of each fluid element. If the fluid elements go around a circular path, but do not spin themselves, there is no vorticity!

This animation from [Wikipedia](http://en.wikipedia.org/wiki/Vortex#Irrotational_vortices) helps further illustrate what happens in an irrotational vortex: the orange markers with a line across them are going around in circles, but they are not themselves rotating (notice the white lines keep their orientation).

## Next ...

The next IPython Notebook of the *AeroPython* series presents the exercise ["Infinite row of vortices,"](http://nbviewer.ipython.org/github/barbagroup/AeroPython/blob/master/lessons/05_Lesson05_InfiniteRowOfVortices.ipynb) as independent student work.

---
Please ignore the cell below. It just loads our style for the notebook.

```python
from IPython.core.display import HTML
def css_styling():
    styles = open('../styles/custom.css', 'r').read()
    return HTML(styles)
css_styling()
```

<link href='http://fonts.googleapis.com/css?family=Fenix' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Alegreya+Sans:100,300,400,500,700,800,900,100italic,300italic,400italic,500italic,700italic,800italic,900italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Source+Code+Pro:300,400' rel='stylesheet' type='text/css'>
<style>
    @font-face {
        font-family: "Computer Modern";
        src: url('http://mirrors.ctan.org/fonts/cm-unicode/fonts/otf/cmunss.otf');
    }
    div.cell{
        width:800px;
        margin-left:16% !important;
        margin-right:auto;
    }
    h1 {
        font-family: 'Alegreya Sans', sans-serif;
    }
    h2 {
        font-family: 'Fenix', serif;
    }
    h3{
		font-family: 'Fenix', serif;
        margin-top:12px;
        margin-bottom: 3px;
       }
	h4{
		font-family: 'Fenix', serif;
       }
    h5 {
        font-family: 'Alegreya Sans', sans-serif;
    }
    div.text_cell_render{
        font-family: 'Alegreya Sans',Computer Modern, "Helvetica Neue", Arial, Helvetica, Geneva, sans-serif;
        line-height: 135%;
        font-size: 120%;
        width:600px;
        margin-left:auto;
        margin-right:auto;
    }
    .CodeMirror{
            font-family: "Source Code Pro";
			font-size: 90%;
    }
/*    .prompt{
        display: None;
    }*/
    .text_cell_render h1 {
        font-weight: 200;
        font-size: 50pt;
		line-height: 100%;
        color:#CD2305;
        margin-bottom: 0.5em;
        margin-top: 0.5em;
        display: block;
    }
    .text_cell_render h5 {
        font-weight: 300;
        font-size: 16pt;
        color: #CD2305;
        font-style: italic;
        margin-bottom: .5em;
        margin-top: 0.5em;
        display: block;
    }

    .warning{
        color: rgb( 240, 20, 20 )
        }
</style>


## In Class Part

#### Q1: What does this statement represent: "x//y" ?
**A) Divides value in x by value in y**

**B) Divides value in x by value in y, truncate result to whole number**

**C) Returns remainder when value in x is divided by value in y**

**D) Raises value in x by value in y**

#### Q2: x*=3 is equivalent to ?
**A) x = x+3**

**B) x = x*3**

__C) x = x**3__

**D) x = x/3**

#### Q3: Which function can be used to convert an integer to a float ?
**A) int()**

**B) float()**

__C) hex()__

**D) type()**

#### Q4: Python does not allow users to have integers, floats, and strings in the same list.
**A) TRUE**

**B) FALSE**

#### Q5: What is the difference between a list and a tuple?
**A) a list can only hold one type of objects**

**B) a tuple cannot hold strings**

__C) a list cannot be changed once defined__

**D) a tuple cannot be changed once defined**

#### Q6: In a dictionary, the first item in each pair is called the_________ and the second item is the ______.
**A) counter , object**

**B) object , counter**

__C) key , data__

**D) data , key**

#### Q7: In a dictionary, the key should be unique.
**A) TRUE**

**B) FALSE**

#### Q8: The data associated with a key in a dictionary, cannot contain a list.
**A) TRUE**

**B) FALSE**

#### Q9: Which scoping delimiter can be used to define a dictionary?
**A) ()**

**B) {}**

__C) []__

**D) <>**

#### Q10: Under what condition will an "or" statement return a FALSE?
**A) If and only if all conditions are False**

**B) If one of the conditions is false**

__C) If the conditions cannot be true together__

**D) The "or" statement will never returns a False**

#### Q11: Python uses indentation to ...
**A) Provide easier readability**

**B) Separate "FOR loops" from other forms of syntax**

__C) Isolate blocks of code__

**D) Repel the weak from programming!**

#### Q12: Which one is the correct syntax for Inline `if` statement?
**A) `if` condition is TRUE do TaskA  `elif` do TaskB**

**B) do TaskA `if` condition is TRUE `elif` do TaskB**

__C) `if` condition is TRUE do TaskA  `else` do TaskB__

**D) do TaskA `if` condition is TRUE `else` do TaskB**

#### Q13: What is the difference between count-controlled and sentinel-controlled repetition?
**A) In count-controlled repetition, the number of iterations is known prior to definition of the loop**

**B) In sentinel-controlled repetition, the number of iterations is known prior to definition of the loop**

__C) Count-controlled repetition is faster than sentinel-controlled repetition__

**D) Sentinel-controlled repetition is faster than count-controlled repetition**

#### Q14: Count-controlled repetition requires

**A) control variable (or loop counter)**

**B) initial value of the control variable**

__C) increment (or decrement) by which the control variable is modified each iteration through the loop__

**D) All of the above**

#### Q15: Which one is true about the range() function?
**A) The syntax is (start,increment,number of steps)**

**B) Can only be used in combination with "For loops"**

__C) Does not work with float objects__

**D) Can produce sequences with inconsistant increments**

#### Q16: Which one is correct about the break statement?
**A) The break statement terminates the current loop and resumes execution at the next statement outside the current loop**

**B) The break statement can only be used with "While loops"**

__C) The break statement can only be used with "For loops"__

**D) The break statement is a necessary part of count-controlled repetition**

#### Q17: Which statement is true about the try, except structure?
**A) It is a control structure**

**B) It is  useful to trap likely errors**

__C) It allows the user to set specific instruction in case of an exception__

**D) All of the above**

#### Q18: Which one is not a built-in Python function?
**A) print()**

**B) math.exp()**

__C) float()__

**D) dict()**

#### Q19: Which statement is true about user-built functions?
**A) The keyword return must be followed by a string**

**B) There can be multiple return statements in a function**

__C) The keyword return cannot be omitted from the function definition__

**D) All of the above**

#### Q20: Which one is NOT true about local and program variables?
**A) Any variable declared within a function is a local variable**

**B) Any variable declared outside a function is known as a program variable**

__C) A local variable cannot be accessed outside its function__

**D) A program variable cannot be used in a function**

#### Q21: The script below is intended to print out all the leap years from 1952 to 1996 (inclusive). However, the programmer keeps getting an error and has become frustrated with it! Please have a look at the code, the error(s), and the output and fix it so that it does what the user wants. Then, explain what was wrong with the script and how you have managed to fix it.

```python
for years in range(1952,1996,4.0):
  print(years)
```

#### Explain the diagnosis and the way you fixed it:

#### Q22: The script below is intended to compute the cosine of values from 0 to 10 (inclusive), store them in a separate list called ylist, and ultimately print out a table. However, the programmer keeps getting an error and has become frustrated with it! Please have a look at the code, the error(s), and the output and fix it so that it does what the user wants. Then, explain what was wrong with the script and how you have managed to fix it.

```python
xlist = ()
ylist = ()

print("     Cosines     ")
print("   x   ","|"," cos(x) ")
print("--------|--------")

for x in range(0,10):
    xlist.append(x*1.0)
    ylist.append(math.cos(x*1.0))
    print("%.3f" % xlist[x], "  |", " %.2f "  % ylist[x])
```

```python

```

****************************************************************************************************************

#########################################################################################################

****************************************************************************************************************

## Take Home Part

#### Question 2: Follow the steps below. Add comments to your script and signify when each step and each task is done. *hint: For this problem you will need the numpy and pandas libraries.
- __STEP1: There are 8 digits in your R#. Define a 2x4 array with these 8 digits, name it "Rarray", and print it__
- __STEP2: Find the maximum value of the "Rarray" and its position__
- __STEP3: Sort the "Rarray" along the rows, store it in a new array named "Rarraysort", and print the new array out__
- __STEP4: Define and print a 4x4 array that has the "Rarray" as its two first rows, and "Rarraysort" as its next rows. Name this new array "DoubleRarray"__
- __STEP5: Slice and print a 4x3 array from the "DoubleRarray" that contains the last three columns of it. Name this new array "SliceRarray".__
- __STEP6: Define the "SliceRarray" as a panda dataframe:__
    - name it "Rdataframe",
    - name the rows as "Row A","Row B","Row C", and "Row D"
    - name the columns as "Column 1", "Column 2", and "Column 3"
- __STEP7: Print  the first few rows of the "Rdataframe".__
- __STEP8: Create a new dataframe object ("R2dataframe") by adding a column to the "Rdataframe", name it "Column X" and fill it with "None" values. Then, use the appropriate descriptor function and print the data model (data column count, names, data types) of the "R2dataframe"__
- __STEP9: Replace the **'None'** in the "R2dataframe" with 0. Then, print the summary statistics of each numeric column in the data frame.__
- __STEP10: Define a function based on the equation below, apply on the entire "R2dataframe", store the results in a new dataframe ("R3dataframe"), and print the results and the summary statistics again.__
$$ y = x^2 - 5x +7 $$
- __STEP11: Print the number of occurrences of each unique value in "Column 3"__
- __STEP12: Sort the data frame with respect to "Column 1" with a descending order and print it__
- __STEP13: Write the final format of the "R3dataframe" on a CSV file, named "Rfile.csv"__
- __STEP14: Read the "Rfile.csv" and print its content.__

** __Make sure to attach the "Rfile.csv" file to your midterm exam submission.__

```python
#Suggested Solution:

#Step0:
import numpy as np
import pandas as pd
#Step1:
Rarray = np.array([[1,6,7,4],[5,2,3,8]])     #Define Rarray
print(Rarray)
#Step2:
print(np.max(Rarray))          #Find the maximum Value
print(np.argmax(Rarray))       #Find the posirtion of the maximum value
#Step3:
Rarraysort = np.sort(Rarray,axis = 1) #Sort Rarray along the rows and define a new array
print(Rarraysort)
#Step4:
DoubleRarray = np.array([[1,6,7,4],[5,2,3,8],[1,4,6,7],[2,3,5,8]])     #Define DoubleRarray
print(DoubleRarray)
#Step5:
SliceRarray = DoubleRarray[0:4,1:4]     #Slice DoubleRarray and Define SliceRarray
print(SliceRarray)
#Step6:
myrowname = ["Row A","Row B","Row C","Row D"]
mycolname = ["Column 1", "Column 2","Column 3"]
Rdataframe = pd.DataFrame(SliceRarray,myrowname,mycolname)     #Define Rdataframe
#Step7:
print(Rdataframe.head())     #Print the first few rows of the Rdataframe
#Step8:
Rdataframe['Column X']= None     #Add a new column, "Column X"
R2dataframe = Rdataframe         #Define R2dataframe
print(R2dataframe.info())               #Get the info
#Step9:
R2dataframe = R2dataframe.fillna(0)     #Replace NAs with 0
print(R2dataframe.describe())                 #Get the summary statistics
#Step10:
def myfunc(x):  # A user-built function
    y = (x**2) - (10*x) +7
    return(y)
R3dataframe = R2dataframe.apply(myfunc)     #Apply the function on the entire R2dataframe
print(R3dataframe)
print(R3dataframe.describe())
#Step11:
print(R3dataframe['Column 3'].value_counts())  #Returns the number of occurences of each unique value in Column 3
#Step12:
print(R3dataframe.sort_values('Column 1', ascending = False)) #Sorting based on Column 1
#Step13:
R3dataframe.to_csv('Rfile.csv')     #Write R3dataframe on a CSV file
#Step14:
readfilecsv = pd.read_csv('Rfile.csv') #Read the Rfile.csv
print(readfilecsv)         #Print the contents of the Rfile.csv
```

    [[1 6 7 4]
     [5 2 3 8]]
    8
    7
    [[1 4 6 7]
     [2 3 5 8]]
    [[1 6 7 4]
     [5 2 3 8]
     [1 4 6 7]
     [2 3 5 8]]
    [[6 7 4]
     [2 3 8]
     [4 6 7]
     [3 5 8]]
           Column 1  Column 2  Column 3
    Row A         6         7         4
    Row B         2         3         8
    Row C         4         6         7
    Row D         3         5         8
    <class 'pandas.core.frame.DataFrame'>
    Index: 4 entries, Row A to Row D
    Data columns (total 4 columns):
     #   Column    Non-Null Count  Dtype
    ---  ------    --------------  -----
     0   Column 1  4 non-null      int64
     1   Column 2  4 non-null      int64
     2   Column 3  4 non-null      int64
     3   Column X  0 non-null      object
    dtypes: int64(3), object(1)
    memory usage: 160.0+ bytes
    None
           Column 1  Column 2  Column 3  Column X
    count  4.000000  4.000000  4.000000       4.0
    mean   3.750000  5.250000  6.750000       0.0
    std    1.707825  1.707825  1.892969       0.0
    min    2.000000  3.000000  4.000000       0.0
    25%    2.750000  4.500000  6.250000       0.0
    50%    3.500000  5.500000  7.500000       0.0
    75%    4.500000  6.250000  8.000000       0.0
    max    6.000000  7.000000  8.000000       0.0
           Column 1  Column 2  Column 3  Column X
    Row A       -17       -14       -17         7
    Row B        -9       -14        -9         7
    Row C       -17       -17       -14         7
    Row D       -14       -18        -9         7
            Column 1   Column 2   Column 3  Column X
    count   4.000000   4.000000   4.000000       4.0
    mean  -14.250000 -15.750000 -12.250000       7.0
    std     3.774917   2.061553   3.947573       0.0
    min   -17.000000 -18.000000 -17.000000       7.0
    25%   -17.000000 -17.250000 -14.750000       7.0
    50%   -15.500000 -15.500000 -11.500000       7.0
    75%   -12.750000 -14.000000  -9.000000       7.0
    max    -9.000000 -14.000000  -9.000000       7.0
    -9     2
    -14    1
    -17    1
    Name: Column 3, dtype: int64
           Column 1  Column 2  Column 3  Column X
    Row B        -9       -14        -9         7
    Row D       -14       -18        -9         7
    Row A       -17       -14       -17         7
    Row C       -17       -17       -14         7
      Unnamed: 0  Column 1  Column 2  Column 3  Column X
    0      Row A       -17       -14       -17         7
    1      Row B        -9       -14        -9         7
    2      Row C       -17       -17       -14         7
    3      Row D       -14       -18        -9         7

### Question3:
__create a class to compute the average grade (out of 10) of the students based on their grades in Quiz1, Quiz2, the Mid-term, Quiz3, and the Final exam.__

| Student Name  |   Quiz 1   |   Quiz 2   |   Mid-term   |   Quiz 3   |  Final Exam  |
| ------------- | -----------| -----------| -------------| -----------| -------------|
| Harry         | 8          | 9          | 8            | 10         | 9            |
| Ron           | 7          | 8          | 8            | 7          | 9            |
| Hermione      | 10         | 10         | 9            | 10         | 10           |
| Draco         | 8          | 7          | 9            | 8          | 9            |
| Luna          | 9          | 8          | 7            | 6          | 5            |

1. __Use docstrings to describe the purpose of the class.__
2. __Create an object for each car brand and display the output as shown below.__

"Student Name": **Average Grade**

3. __Create and print out a dictionary with the student names as keys and their average grades as data.__

```python
#Suggested Solution:

class Hogwarts:
    """This class calculates the average grade of the students"""
    def __init__(self, Name,Quiz1,Quiz2,MidTerm,Quiz3,Final):
        self.Name = Name
        self.Quiz1 = Quiz1
        self.Quiz2 = Quiz2
        self.MidTerm = MidTerm
        self.Quiz3 = Quiz3
        self.Final= Final

    def average(self):
        return (self.Quiz1 + self.Quiz2 + self.MidTerm + self.Quiz3 + self.Final) /5

S1 = Hogwarts('Harry',8,9,8,10,9)                      #Fill the instances
S2 = Hogwarts('Ron',7,8,8,7,9)
S3 = Hogwarts('Hermione',10,10,9,10,10)
S4 = Hogwarts('Draco',8,7,9,8,9)
S5 = Hogwarts('Luna',9,8,7,6,5)

print("Harry", S1.average())
print("Ron", S2.average())
print("Hermione", S3.average())
print("Draco", S4.average())
print("Luna", S5.average())

GradeDict = {"Harry":S1.average(),"Ron":S2.average(),"Hermione":S3.average(),"Draco":S4.average(),"Luna":S5.average()}
print(GradeDict)
```

    Harry 8.8
    Ron 7.8
    Hermione 9.8
    Draco 8.2
    Luna 7.0
    {'Harry': 8.8, 'Ron': 7.8, 'Hermione': 9.8, 'Draco': 8.2, 'Luna': 7.0}

#### Question 4:
- __When it is 8:00 in Lubbock,__
    - __It is 9:00 in New York__
    - __It is 14:00 in London__
    - __It is 15:00 in Cairo__
    - __It is 16:00 in Istanbul__
    - __It is 19:00 in Hyderabad__
    - __It is 22:00 in Tokyo__

__Write a function that reports the time in New York, London, Cairo, Istanbul, Hyderabad, and Tokyo based on the time in Lubbock. Use a 24-hour time format. Include error trapping that:__

1- Issues a message like "Please Enter A Number from 00 to 23" if the first input is numeric but outside the range of [0,23].

2- Takes any numeric input for "Lubbock time" selection , and forces it into an integer.

3- Issues an appropriate message if the user's selection is non-numeric.

__Check your function for these times:__
- 8:00
- 17:00
- 0:00

```python
#Suggested Solution:

def LBBtime():
    try:
        LBK = int(input('What hour is it in Lubbock?- Please enter a number from 0 to 23'))
        if LBK>23:
            print('Please Enter A Number from 00 to 23')
        if LBK<23 and LBK>=0:
            print("Time in New York is",LBK+1,":00")
            print("Time in London is",LBK+6,":00")
            print("Time in Cairo is",LBK+7,":00")
            print("Time in Istanbul is",LBK+8,":00")
            print("Time in Hyderabad is",LBK+11,":00")
            print("Time in Tokyo is",LBK+14,":00")
            return #null return
    except:
        print("Please Enter an Appropriate Input")
```

```python
LBBtime()
```

    What hour is it in Lubbock?- Please enter a number from 0 to 23 TokyoPlease Enter an Appropriate Input

### Bonus Question for extra credit!
#### Write the VOLUME Function to compute the volume of Cylinders, Spheres, Cones, and Rectangular Boxes. This function should:
- First, ask the user about __the shape of the object__ of interest using this statement:

*"Please choose the shape of the object. Enter 1 for "Cylinder", 2 for "Sphere", 3 for "Cone", or 4 for "Rectangular Box""*

- Second, based on user's choice in the previous step, __ask for the right inputs__.
- Third, print out an statement with __the input values and the calculated volumes__.

#### Include error trapping that:

1. Issues a message that *"The object should be either a Cylinder, a Sphere, a Cone, or a Rectangular Box. Please Enter A Number from 1,2,3, and 4!"* if the first input is non-numeric.
2. Takes any numeric input for the initial selection , and force it into an integer.
4. Issues an appropriate message if the user's selection is numeric but outside the range of [1,4]
3. Takes any numeric input for the shape characteristics , and force it into a float.
4. Issues an appropriate message if the object characteristics are as non-numerics.

#### Test the script for:
1. __Sphere, r=10__
2. __r=10 , Sphere__
3. __Rectangular Box, w=5, h=10, l=0.5__

- __Volume of a Cylinder = πr²h__
- __Volume of a Sphere = 4(πr3)/3__
- __Volume of a Cone = (πr²h)/3__
- __Volume of a Rectangular Box = whl__

```python
#Suggested Solution:

import numpy as np # import NumPy: for large, multi-dimensional arrays and matrices, along with  high-level mathematical functions to operate on these arrays.
pi = np.pi #pi value from the np package

def VOLUME():
    try:
        UI = input('Please choose the shape of the object. Enter 1 for "Cylinder", 2 for "Sphere", 3 for "Cone", or 4 for "Rectangular Box"')
        UI =int(UI)
        if UI==1:
                try:
                    UI2 = input('Please enter the radius of the Cylinder')
                    r= float(UI2)
                    UI3 = input('Please enter the height of the Cylinder')
                    h= float(UI3)
                    V= pi*h*r**2
                    print("The volume of the Cylinder with the radius of ",r," and the height of ",h," is equal to", V)
                except:
                    print("The radius and height of the Cylinder must be numerics. Please Try Again!")
        elif UI==2:
                try:
                    UI2 = input('Please enter the radius of the Sphere')
                    r= float(UI2)
                    V= (4*pi*r**3)/3
                    print("The volume of the Sphere with the radius of ",r," is equal to", V)
                except:
                    print("The radius of the Sphere must be numeric. Please Try Again!")
        elif UI==3:
                try:
                    UI2 = input('Please enter the radius of the Cone')
                    r= float(UI2)
                    UI3 = input('Please enter the height of the Cone')
                    h= float(UI3)
                    V= (pi*h*r**2)/3
                    print("The volume of the Cone with the radius of ",r," and the height of ",h," is equal to", V)
                except:
                    print("The radius and height of the Cone must be numerics. Please Try Again!")
        elif UI==4:
                try:
                    UI2 = input('Please enter the width of the Rectangular Box')
                    w= float(UI2)
                    UI3 = input('Please enter the height of the Rectangular Box')
                    h= float(UI3)
                    UI4 = input('Please enter the length of the Rectangular Box')
                    l= float(UI4)
                    V= w*h*l
                    print("The volume of the Rectangular Box with the width of ",w," and the height of ",h," and the length of ",l," is equal to", V)
                except:
                    print("The width, height, and length of the Rectangular Box must be numerics. Please Try Again!")
        else:
            print("Please Enter A Number from 1,2,3, and 4!")

    except:
        print("The object should be either a Cylinder, a Sphere, a Cone, or a Rectangular Box. Please Enter A Number from 1,2,3, and 4!")

```

```python
VOLUME()
```

    Please choose the shape of the object. Enter 1 for "Cylinder", 2 for "Sphere", 3 for "Cone", or 4 for "Rectangular Box" 4Please enter the width of the Rectangular Box 2Please enter the height of the Rectangular Box 5Please enter the length of the Rectangular Box 10The volume of the Rectangular Box with the width of  2.0  and the height of  5.0  and the length of  10.0  is equal to 100.0

# Problem XX

Graphing Functions Special Functions

Consider the two functions listed below:

\begin{equation}
f(x) = e^{-\alpha x}
\label{eqn:fofx}
\end{equation}

\begin{equation}
g(x) = \gamma sin(\beta  x)
\label{eqn:gofx}
\end{equation}

Prepare a plot of the two functions on the same graph.

Use the values in Table below for $\alpha$, $\beta$, and $\gamma$.

|Parameter|Value|
|:---|---:|
|$\alpha$|0.50|
|$\beta$|3.00|
|$\gamma$|$\frac{\pi}{2}$|

The plot should have $x$ values ranging from $0$ to $10$ (inclusive) in sufficiently small increments to see curvature in the two functions as well as to identify the number and approximate locations of intersections. In this problem, intersections are locations in the $x-y$ plane where the two "curves" cross one another of the two plots.

```python
# By-hand evaluate f(x) for x=1, alpha = 1/2
```

```python
# By-hand evaluate g(x) for x=3.14, beta = 1/2, gamma = 2
```

```python
# Define the first function f(x,alpha), test the function using your by hand answer
def f(x,alpha):
    import math
    f = math.exp(-1.0*alpha*x)
    return f

f(1,0.5)
```

    0.6065306597126334

```python
# Define the second function g(x,beta,gamma), test the function using your by hand answer
def g(x,beta,gamma):
    import math
    f = gamma*math.sin(beta*x)
    return f

g(3.14,0.5,2.0)
```

    1.9999993658636692

```python
# Built a list for x that ranges from 0 to 10, inclusive, with adjustable step sizes for plotting later on
howMany = 10
scale = 10.0/howMany
xvector = []
for i in range(0,howMany+1):
    xvector.append(scale*i)
#xvector # activate to display
```

```python
# Build a plotting function that plots both functions on the same chart
alpha = 0.5
beta = 0.5
gamma = 2.0
yf = []
yg = []
for i in range(0,howMany+1):
    yf.append(f(xvector[i],alpha))
    yg.append(g(xvector[i],beta,gamma))

def plot2lines(list11,list21,list12,list22,strx,stry,strtitle): # plot list1 on x, list2 on y, xlabel, ylabel, title
    from matplotlib import pyplot as plt # import the plotting library from matplotlibplt.show()
    plt.plot( list11, list21, color ='green', marker ='s', linestyle ='dashdot' , label = "Observed" ) # create a line chart, years on x-axis, gdp on y-axis
    plt.plot( list12, list22, color ='red', marker ='o', linestyle ='solid' , label = "Model") # create a line chart, years on x-axis, gdp on y-axis
    plt.legend()
    plt.title(strtitle)# add a title
    plt.ylabel(stry)# add a label to the x and y-axes
    plt.xlabel(strx)
    plt.show() # display the plot
    return #null return

plot2lines(xvector,yf,xvector,yg,'x-value','y-value','plot of f and g')
```

```python

```

```python

```

#### Question 1: Substitute Intersection Plot
- __Define a function, called "myplotfunc" that takes__
    - A list of numeric for x
    - A list of numeric for y
    - A string for the plot title
    - A string for the x-axis label
    - A string for the y-axis label

 __as inputs and plots a curve with "red" color, '*' shaped marker, and 'dashdot' linestyle__

- __Use the "myplotfunc" function to create a plot of__
$$ y = -5x^5 +4sin(x)^4 -3x^3 +2cos(x)^2 -x $$
__for x raging from -10 to 10 (inclusive) in steps of 0.5.__

```python
#Suggested Solution:

def plotfunc(list1,list2,strtitle,strx,stry):
    from matplotlib import pyplot as plt
    plt.plot( list1, list2, color ='red', marker ='*', linestyle ='dashdot')
    plt.title(strtitle)
    plt.xlabel(strx)
    plt.ylabel(stry)
    plt.show()
    return

import math
xxx = [] # null list
yyy = [] # null list
for i in range(-20,21,1):
    x=0.5*i
    xxx.append(x)
    y= -5*(x**5) +4*(math.sin(x))**4 -3*(x**3) +2*(math.cos(x))**2 -x
    yyy.append(y)
plotfunc(xxx,yyy,"y = -5x^5 +4sin(x)^4 -3x^3 +2cos(x)^2 -x","X","Y")
```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```


# Vertical Line Test

## 1.1 Create two graphs, one that passes the vertical line test and one that does not.

```
import matplotlib.pyplot as plt

```

```
fig, ax = plt.subplots(figsize = (8,8))
ax.grid()
ax.set_xlim(-10,10)
ax.set_ylim(-10,10)
ax.axhline(0, color = 'k')
ax.axvline(0, color = 'k')
ax.arrow(0,0,9,9)
ax.arrow(0,0,-7.5,-2.5)
ax.arrow(-7.5,-2.5,5,-2.5)
ax.arrow(-2.5,-5,-5,-2.5)
ax.axvline(-5, color = 'r')
ax.axvline(5, color='r')
plt.show()
```

## 1.2 Why are graphs that don't pass the vertical line test not considered "functions?"

because a function is a mapping from a list of inputs to a list of outputs where **no input is mapped to multiple outputs**

# Functions as Relations

## 2.1 Which of the following relations are functions? Why?

\begin{align}
\text{Relation 1: } \{(1, 2), (3, 2), (1, 3)\}
\\
\text{Relation 2: } \{(1, 3), (2, 3), (6, 7)\}
\\
\text{Relation 3: } \{(9, 4), (2, 1), (9, 6)\}
\\
\text{Relation 4: } \{(6, 2), (8, 3), (6, 4)\}
\\
\text{Relation 5: } \{(2, 6), (2, 7), (2, 4)\}
\end{align}

Relation 2 because no input value is mapped with multiple output values

# Functions as a mapping between dimensions

## 3.1 for the following functions what is the dimensionality of the domain (input) and codomain (range/output)?

\begin{align}
m(𝑥_1,𝑥_2,𝑥_3)=(x_1+x_2, x_1+x_3, x_2+x_3)
\\
n(𝑥_1,𝑥_2,𝑥_3,𝑥_4)=(x_2^2 + x_3, x_2x_4)
\end{align}

3 dimentions shown in 3 dimentional space
4 dimentions shown in 2 dimentional space

## 3.2 Do you think it's possible to create a function that maps from a lower dimensional space to a higher dimensional space? If so, provide an example.

i think everythings possible with enough time, i would think that a unit or basis for the additional dimentions are required. like going from a square to a cube adding depth as a dimention, how could that be determined with only r2 as data points?

# Vector Transformations

## 4.1 Plug the corresponding unit vectors into each function. Use the output vectors to create a transformation matrix.

\begin{align}
p(\begin{bmatrix}x_1 \\ x_2 \end{bmatrix}) = \begin{bmatrix} x_1 + 3x_2 \\2 x_2 - x_1 \\  \end{bmatrix}
\\
\\
q(\begin{bmatrix}x_1 \\ x_2 \\ x_3\end{bmatrix}) = \begin{bmatrix} 4x_1 + x_2 + 2x_3 \\2 x_2 - x_1 + 3x_3 \\ 5x_1 - 2x_3 + x_2  \end{bmatrix}
\end{align}

## 4.2 Verify that your transformation matrices are correct by choosing an input matrix and calculating the result both via the traditional functions above and also via vector-matrix multiplication.

```
import numpy as np
r2 = np.array([[1,0],
               [0,1]])
p = np.array([[1],
              [2]])

print('P:')
print(p)

mp = mp.array([4],
              [5])

[19]
[6]

```

# Eigenvalues and Eigenvectors

## 5.1 In your own words, give an explanation for the intuition behind eigenvalues and eigenvectors.

eigenvectors dont change the direction they point during transformation

eigenvalues are the values that may be scaled in size but always stay along the same line of direction

# The Curse of Dimensionality

## 6.1 What are some of the challenges of working with high dimensional spaces?

the distance between data points becomes abstract. the data points loose meaning the more they are pulled in each dimention

## 6.2 What is the rule of thumb for how many observations you should have compared to parameters in your model?

5 times more observations than parameters

# Principal Component Analysis

## 7.1 Code for loading and cleaning the 2013 national dataset from the [Housing Affordability Data System (HADS)](https://www.huduser.gov/portal/datasets/hads/hads.html) --housing data, can be found below.

## Perform PCA on the processed dataset `national_processed` (Make sure you standardize your data!) and then make a scatterplot of PC1 against PC2. Some of our discussion and work around PCA with this dataset will continue during tomorrow's lecture and assignment.

Not only does this dataset have decent amount columns to begin with (99), but in preparing the data for PCA we have also [one-hot-encoded](https://hackernoon.com/what-is-one-hot-encoding-why-and-when-do-you-have-to-use-it-e3c6186d008f#targetText=One%20hot%20encoding%20is%20a,the%20entry%20in%20the%20dataset.) all of the categorical variables. This has the effect of creating a new column for each individual category of each categorical variable. After processing this dataset has 64738 columns. --Das a lot of columns.

Don't worry too much about the mechanics of one-hot encoding right now, you will learn and experiment with a whole bunch of categorical encoding approaches in unit 2.

The code below will read in the dataset and perform the one-hot encoding of the categorical variables. Start adding your PCA code at the bottom of the provided code.

```
from urllib.request import urlopen
from zipfile import ZipFile
from io import BytesIO
import os.path
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

# Read Natinal Data
national_url = 'https://www.huduser.gov/portal/datasets/hads/hads2013n_ASCII.zip'
national_file = 'thads2013n.txt'

if os.path.exists(national_file):
    national = pd.read_csv(national_file)
else:
    z_national = urlopen(national_url)
    zip_national = ZipFile(BytesIO(z_national.read())).extract(national_file)
    national = pd.read_csv(zip_national)

print(national.shape)
national.head()
```

    (64535, 99)

      CONTROL
      AGE1
      METRO3
      REGION
      LMED
      FMR
      L30
      L50
      L80
      IPOV
      BEDRMS
      BUILT
      STATUS
      TYPE
      VALUE
      VACANCY
      TENURE
      NUNITS
      ROOMS
      WEIGHT
      PER
      ZINC2
      ZADEQ
      ZSMHC
      STRUCTURETYPE
      OWNRENT
      UTILITY
      OTHERCOST
      COST06
      COST12
      COST08
      COSTMED
      TOTSAL
      ASSISTED
      GLMED
      GL30
      GL50
      GL80
      APLMED
      ABL30
      ...
      COST08RELPOVCAT
      COST08RELFMRPCT
      COST08RELFMRCAT
      COST12RELAMIPCT
      COST12RELAMICAT
      COST12RELPOVPCT
      COST12RELPOVCAT
      COST12RELFMRPCT
      COST12RELFMRCAT
      COSTMedRELAMIPCT
      COSTMedRELAMICAT
      COSTMedRELPOVPCT
      COSTMedRELPOVCAT
      COSTMedRELFMRPCT
      COSTMedRELFMRCAT
      FMTZADEQ
      FMTMETRO3
      FMTBUILT
      FMTSTRUCTURETYPE
      FMTBEDRMS
      FMTOWNRENT
      FMTCOST06RELPOVCAT
      FMTCOST08RELPOVCAT
      FMTCOST12RELPOVCAT
      FMTCOSTMEDRELPOVCAT
      FMTINCRELPOVCAT
      FMTCOST06RELFMRCAT
      FMTCOST08RELFMRCAT
      FMTCOST12RELFMRCAT
      FMTCOSTMEDRELFMRCAT
      FMTINCRELFMRCAT
      FMTCOST06RELAMICAT
      FMTCOST08RELAMICAT
      FMTCOST12RELAMICAT
      FMTCOSTMEDRELAMICAT
      FMTINCRELAMICAT
      FMTASSISTED
      FMTBURDEN
      FMTREGION
      FMTSTATUS

      0
      '100003130103'
      82
      '3'
      '1'
      73738
      956
      15738
      26213
      40322
      11067
      2
      2006
      '1'
      1
      40000
      -6
      '1'
      1
      6
      3117.394239
      1
      18021
      '1'
      533
      1
      '1'
      169.000000
      213.750000
      648.588189
      803.050535
      696.905247
      615.156712
      0
      -9
      73738
      15738
      26213
      40322
      51616.6
      20234.571429
      ...
      4
      72.898038
      2
      48.402635
      2
      290.250487
      4
      84.001102
      2
      37.077624
      2
      222.339102
      4
      64.346936
      2
      '1 Adequate'
      '-5'
      '2000-2009'
      '1 Single Family'
      '2 2BR'
      '1 Owner'
      '4 200%+ Poverty'
      '4 200%+ Poverty'
      '4 200%+ Poverty'
      '4 200%+ Poverty'
      '3 150-200% Poverty'
      '2 50.1 - 100% FMR'
      '2 50.1 - 100% FMR'
      '2 50.1 - 100% FMR'
      '2 50.1 - 100% FMR'
      '1 LTE 50% FMR'
      '2 30 - 50% AMI'
      '2 30 - 50% AMI'
      '2 30 - 50% AMI'
      '2 30 - 50% AMI'
      '2 30 - 50% AMI'
      '.'
      '2 30% to 50%'
      '-5'
      '-5'

      1
      '100006110249'
      50
      '5'
      '3'
      55846
      1100
      17165
      28604
      45744
      24218
      4
      1980
      '1'
      1
      130000
      -6
      '1'
      1
      6
      2150.725544
      4
      122961
      '1'
      487
      1
      '1'
      245.333333
      58.333333
      1167.640781
      1669.643405
      1324.671218
      1058.988479
      123000
      -9
      55846
      17165
      28604
      45744
      55846.0
      19911.400000
      ...
      4
      120.424656
      3
      103.094063
      6
      275.768999
      4
      151.785764
      3
      65.388468
      4
      174.909320
      3
      96.271680
      2
      '1 Adequate'
      '-5'
      '1980-1989'
      '1 Single Family'
      '4 4BR+'
      '1 Owner'
      '3 150-200% Poverty'
      '4 200%+ Poverty'
      '4 200%+ Poverty'
      '3 150-200% Poverty'
      '4 200%+ Poverty'
      '3 GT FMR'
      '3 GT FMR'
      '3 GT FMR'
      '2 50.1 - 100% FMR'
      '3 GT FMR'
      '4 60 - 80% AMI'
      '4 60 - 80% AMI'
      '6 100 - 120% AMI'
      '4 60 - 80% AMI'
      '7 120% AMI +'
      '.'
      '1 Less than 30%'
      '-5'
      '-5'

      2
      '100006370140'
      53
      '5'
      '3'
      55846
      1100
      13750
      22897
      36614
      15470
      4
      1985
      '1'
      1
      150000
      -6
      '1'
      1
      7
      2213.789404
      2
      27974
      '1'
      1405
      1
      '1'
      159.000000
      37.500000
      1193.393209
      1772.627006
      1374.582175
      1068.025168
      28000
      -9
      55846
      13750
      22897
      36614
      44676.8
      19937.500000
      ...
      4
      124.962016
      3
      109.452905
      6
      458.339239
      4
      161.147910
      3
      65.946449
      4
      276.153890
      4
      97.093197
      2
      '1 Adequate'
      '-5'
      '1980-1989'
      '1 Single Family'
      '4 4BR+'
      '1 Owner'
      '4 200%+ Poverty'
      '4 200%+ Poverty'
      '4 200%+ Poverty'
      '4 200%+ Poverty'
      '3 150-200% Poverty'
      '3 GT FMR'
      '3 GT FMR'
      '3 GT FMR'
      '2 50.1 - 100% FMR'
      '2 50.1 - 100% FMR'
      '4 60 - 80% AMI'
      '5 80 - 100% AMI'
      '6 100 - 120% AMI'
      '4 60 - 80% AMI'
      '4 60 - 80% AMI'
      '.'
      '3 50% or More'
      '-5'
      '-5'

      3
      '100006520140'
      67
      '5'
      '3'
      55846
      949
      13750
      22897
      36614
      13964
      3
      1985
      '1'
      1
      200000
      -6
      '1'
      1
      6
      2364.585097
      2
      32220
      '1'
      279
      1
      '1'
      179.000000
      70.666667
      1578.857612
      2351.169341
      1820.442900
      1411.700224
      0
      -9
      55846
      13750
      22897
      36614
      44676.8
      17875.000000
      ...
      4
      191.827492
      3
      161.926709
      7
      673.494512
      4
      247.752301
      3
      97.224801
      5
      404.382763
      4
      148.756610
      3
      '1 Adequate'
      '-5'
      '1980-1989'
      '1 Single Family'
      '3 3BR'
      '1 Owner'
      '4 200%+ Poverty'
      '4 200%+ Poverty'
      '4 200%+ Poverty'
      '4 200%+ Poverty'
      '4 200%+ Poverty'
      '3 GT FMR'
      '3 GT FMR'
      '3 GT FMR'
      '3 GT FMR'
      '2 50.1 - 100% FMR'
      '6 100 - 120% AMI'
      '7 120% AMI +'
      '7 120% AMI +'
      '5 80 - 100% AMI'
      '4 60 - 80% AMI'
      '.'
      '1 Less than 30%'
      '-5'
      '-5'

      4
      '100007130148'
      26
      '1'
      '3'
      60991
      737
      14801
      24628
      39421
      15492
      2
      1980
      '1'
      1
      -6
      -6
      '2'
      100
      4
      2314.524902
      2
      96874
      '1'
      759
      5
      '2'
      146.000000
      12.500000
      759.000000
      759.000000
      759.000000
      759.000000
      96900
      0
      60991
      14801
      24628
      39421
      48792.8
      16651.125000
      ...
      3
      102.985075
      3
      55.308707
      3
      195.972115
      3
      102.985075
      3
      55.308707
      3
      195.972115
      3
      102.985075
      3
      '1 Adequate'
      'Central City'
      '1980-1989'
      '5 50+ units'
      '2 2BR'
      '2 Renter'
      '3 150-200% Poverty'
      '3 150-200% Poverty'
      '3 150-200% Poverty'
      '3 150-200% Poverty'
      '4 200%+ Poverty'
      '3 GT FMR'
      '3 GT FMR'
      '3 GT FMR'
      '3 GT FMR'
      '3 GT FMR'
      '3 50 - 60% AMI'
      '3 50 - 60% AMI'
      '3 50 - 60% AMI'
      '3 50 - 60% AMI'
      '7 120% AMI +'
      '0 Not Assisted'
      '1 Less than 30%'
      '-5'
      '-5'

5 rows × 99 columns

```
# Look at datatypes
# a lot of object datatypes even though they seem to be strings of numbers.
national.dtypes
```

    CONTROL            object
    AGE1                int64
    METRO3             object
    REGION             object
    LMED                int64
                        ...
    FMTINCRELAMICAT    object
    FMTASSISTED        object
    FMTBURDEN          object
    FMTREGION          object
    FMTSTATUS          object
    Length: 99, dtype: object

```
# check for null values
national.isnull().sum().any()
```

    False

```
# check for number of categorical vs numeric columns
cat_cols = national.columns[national.dtypes=='object']
num_cols = national.columns[national.dtypes!='object']

print(f'{len(cat_cols)} categorical columns')
print(f'{len(num_cols)} numerical columns')
```

    32 categorical columns
    67 numerical columns

```
# We're making a copy of our data in case we mess something up.
national_processed = national.copy()

# Categorically Encode our Variables:
# They need to all be numeric before we do PCA.
# https://pbpython.com/categorical-encoding.html

# Cast categorical columns to "category" data type
national_processed[cat_cols] = national_processed[cat_cols].astype('category')

national_processed.dtypes
```

    CONTROL            category
    AGE1                  int64
    METRO3             category
    REGION             category
    LMED                  int64
                         ...
    FMTINCRELAMICAT    category
    FMTASSISTED        category
    FMTBURDEN          category
    FMTREGION          category
    FMTSTATUS          category
    Length: 99, dtype: object

```
# Replace all category cell values with their numeric category codes
for col in cat_cols:
  national_processed[col] = national_processed[col].cat.codes

print(national_processed.shape)
national_processed.head(10)
```

    (64535, 99)

      CONTROL
      AGE1
      METRO3
      REGION
      LMED
      FMR
      L30
      L50
      L80
      IPOV
      BEDRMS
      BUILT
      STATUS
      TYPE
      VALUE
      VACANCY
      TENURE
      NUNITS
      ROOMS
      WEIGHT
      PER
      ZINC2
      ZADEQ
      ZSMHC
      STRUCTURETYPE
      OWNRENT
      UTILITY
      OTHERCOST
      COST06
      COST12
      COST08
      COSTMED
      TOTSAL
      ASSISTED
      GLMED
      GL30
      GL50
      GL80
      APLMED
      ABL30
      ...
      COST08RELPOVCAT
      COST08RELFMRPCT
      COST08RELFMRCAT
      COST12RELAMIPCT
      COST12RELAMICAT
      COST12RELPOVPCT
      COST12RELPOVCAT
      COST12RELFMRPCT
      COST12RELFMRCAT
      COSTMedRELAMIPCT
      COSTMedRELAMICAT
      COSTMedRELPOVPCT
      COSTMedRELPOVCAT
      COSTMedRELFMRPCT
      COSTMedRELFMRCAT
      FMTZADEQ
      FMTMETRO3
      FMTBUILT
      FMTSTRUCTURETYPE
      FMTBEDRMS
      FMTOWNRENT
      FMTCOST06RELPOVCAT
      FMTCOST08RELPOVCAT
      FMTCOST12RELPOVCAT
      FMTCOSTMEDRELPOVCAT
      FMTINCRELPOVCAT
      FMTCOST06RELFMRCAT
      FMTCOST08RELFMRCAT
      FMTCOST12RELFMRCAT
      FMTCOSTMEDRELFMRCAT
      FMTINCRELFMRCAT
      FMTCOST06RELAMICAT
      FMTCOST08RELAMICAT
      FMTCOST12RELAMICAT
      FMTCOSTMEDRELAMICAT
      FMTINCRELAMICAT
      FMTASSISTED
      FMTBURDEN
      FMTREGION
      FMTSTATUS

      0
      0
      82
      2
      0
      73738
      956
      15738
      26213
      40322
      11067
      2
      2006
      0
      1
      40000
      -6
      1
      1
      6
      3117.394239
      1
      18021
      1
      533
      1
      0
      169.000000
      213.750000
      648.588189
      803.050535
      696.905247
      615.156712
      0
      -9
      73738
      15738
      26213
      40322
      51616.6
      20234.571429
      ...
      4
      72.898038
      2
      48.402635
      2
      290.250487
      4
      84.001102
      2
      37.077624
      2
      222.339102
      4
      64.346936
      2
      1
      0
      5
      1
      2
      0
      4
      4
      4
      4
      3
      1
      1
      1
      1
      1
      1
      1
      1
      1
      2
      0
      2
      0
      0

      1
      1
      50
      4
      2
      55846
      1100
      17165
      28604
      45744
      24218
      4
      1980
      0
      1
      130000
      -6
      1
      1
      6
      2150.725544
      4
      122961
      1
      487
      1
      0
      245.333333
      58.333333
      1167.640781
      1669.643405
      1324.671218
      1058.988479
      123000
      -9
      55846
      17165
      28604
      45744
      55846.0
      19911.400000
      ...
      4
      120.424656
      3
      103.094063
      6
      275.768999
      4
      151.785764
      3
      65.388468
      4
      174.909320
      3
      96.271680
      2
      1
      0
      3
      1
      4
      0
      3
      4
      4
      3
      4
      2
      2
      2
      1
      3
      3
      3
      5
      3
      7
      0
      1
      0
      0

      2
      2
      53
      4
      2
      55846
      1100
      13750
      22897
      36614
      15470
      4
      1985
      0
      1
      150000
      -6
      1
      1
      7
      2213.789404
      2
      27974
      1
      1405
      1
      0
      159.000000
      37.500000
      1193.393209
      1772.627006
      1374.582175
      1068.025168
      28000
      -9
      55846
      13750
      22897
      36614
      44676.8
      19937.500000
      ...
      4
      124.962016
      3
      109.452905
      6
      458.339239
      4
      161.147910
      3
      65.946449
      4
      276.153890
      4
      97.093197
      2
      1
      0
      3
      1
      4
      0
      4
      4
      4
      4
      3
      2
      2
      2
      1
      2
      3
      4
      5
      3
      4
      0
      3
      0
      0

      3
      3
      67
      4
      2
      55846
      949
      13750
      22897
      36614
      13964
      3
      1985
      0
      1
      200000
      -6
      1
      1
      6
      2364.585097
      2
      32220
      1
      279
      1
      0
      179.000000
      70.666667
      1578.857612
      2351.169341
      1820.442900
      1411.700224
      0
      -9
      55846
      13750
      22897
      36614
      44676.8
      17875.000000
      ...
      4
      191.827492
      3
      161.926709
      7
      673.494512
      4
      247.752301
      3
      97.224801
      5
      404.382763
      4
      148.756610
      3
      1
      0
      3
      1
      3
      0
      4
      4
      4
      4
      4
      2
      2
      2
      2
      2
      5
      6
      6
      4
      4
      0
      1
      0
      0

      4
      4
      26
      0
      2
      60991
      737
      14801
      24628
      39421
      15492
      2
      1980
      0
      1
      -6
      -6
      2
      100
      4
      2314.524902
      2
      96874
      1
      759
      5
      1
      146.000000
      12.500000
      759.000000
      759.000000
      759.000000
      759.000000
      96900
      0
      60991
      14801
      24628
      39421
      48792.8
      16651.125000
      ...
      3
      102.985075
      3
      55.308707
      3
      195.972115
      3
      102.985075
      3
      55.308707
      3
      195.972115
      3
      102.985075
      3
      1
      1
      3
      5
      2
      1
      3
      3
      3
      3
      4
      2
      2
      2
      2
      3
      2
      2
      2
      2
      7
      1
      1
      0
      0

      5
      5
      56
      1
      2
      62066
      657
      13170
      21924
      35073
      12005
      1
      1985
      0
      1
      -6
      -6
      2
      32
      3
      2482.655916
      1
      14987
      1
      695
      4
      1
      94.750000
      0.000000
      695.000000
      695.000000
      695.000000
      695.000000
      15000
      1
      62066
      13170
      21924
      35073
      43446.2
      14110.714286
      ...
      4
      105.783866
      3
      59.721372
      3
      231.570179
      4
      105.783866
      3
      59.721372
      3
      231.570179
      4
      105.783866
      3
      1
      0
      3
      4
      1
      1
      4
      4
      4
      4
      2
      2
      2
      2
      2
      2
      2
      2
      2
      2
      2
      2
      3
      0
      0

      6
      6
      50
      0
      2
      60991
      988
      16646
      27713
      44340
      18050
      3
      1985
      0
      1
      260000
      -6
      1
      1
      6
      4084.310118
      3
      69962
      1
      1165
      1
      0
      236.000000
      75.000000
      2038.948229
      3042.953477
      2353.009103
      1821.643625
      70001
      -9
      60991
      16646
      27713
      44340
      54891.9
      19235.377778
      ...
      4
      238.158816
      3
      191.891709
      7
      674.338721
      4
      307.991243
      3
      114.874680
      6
      403.688338
      4
      184.376885
      3
      1
      1
      3
      1
      3
      0
      4
      4
      4
      4
      4
      2
      2
      2
      2
      3
      6
      6
      6
      5
      7
      0
      1
      0
      0

      7
      7
      26
      3
      3
      52322
      773
      13489
      22471
      35929
      15992
      2
      1980
      0
      1
      -6
      -6
      2
      8
      5
      2823.395990
      2
      32000
      2
      976
      3
      1
      81.000000
      0.000000
      976.000000
      976.000000
      976.000000
      976.000000
      20000
      0
      52322
      13489
      22471
      35929
      41857.6
      15175.125000
      ...
      4
      126.261320
      3
      82.905428
      4
      244.122061
      4
      126.261320
      3
      82.905428
      4
      244.122061
      4
      126.261320
      3
      2
      0
      3
      3
      2
      1
      4
      4
      4
      4
      4
      2
      2
      2
      2
      3
      3
      3
      3
      3
      4
      1
      2
      1
      0

      8
      8
      60
      4
      3
      50296
      1125
      13115
      21859
      34939
      15452
      3
      1985
      0
      1
      170000
      -6
      1
      1
      7
      2552.762241
      2
      118987
      1
      1156
      1
      0
      184.083333
      47.500000
      1361.395637
      2017.860607
      1566.743131
      1219.311857
      107000
      -9
      50296
      13115
      21859
      34939
      40236.8
      17049.500000
      ...
      4
      139.266056
      3
      154.306552
      7
      522.355839
      4
      179.365387
      3
      93.241232
      5
      315.638586
      4
      108.383276
      3
      1
      0
      3
      1
      3
      0
      4
      4
      4
      4
      4
      2
      2
      2
      2
      3
      5
      5
      6
      4
      7
      0
      1
      1
      0

      9
      9
      26
      3
      1
      63221
      552
      13338
      22199
      35501
      12005
      1
      1985
      0
      1
      -6
      -6
      2
      24
      3
      2845.454432
      1
      47987
      1
      1100
      4
      1
      0.000000
      0.000000
      1100.000000
      1100.000000
      1100.000000
      1100.000000
      48000
      0
      63221
      13338
      22199
      35501
      44254.7
      14290.714286
      ...
      4
      199.275362
      3
      92.796170
      5
      366.513953
      4
      199.275362
      3
      92.796170
      5
      366.513953
      4
      199.275362
      3
      1
      0
      3
      4
      1
      1
      4
      4
      4
      4
      4
      2
      2
      2
      2
      3
      4
      4
      4
      4
      6
      1
      1
      0
      0

10 rows × 99 columns

```
# Now we only have numeric columns (ints and floats)
national_processed.dtypes
```

    CONTROL            int32
    AGE1               int64
    METRO3              int8
    REGION              int8
    LMED               int64
                       ...
    FMTINCRELAMICAT     int8
    FMTASSISTED         int8
    FMTBURDEN           int8
    FMTREGION           int8
    FMTSTATUS           int8
    Length: 99, dtype: object

```
### Your Code Here
from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA
import matplotlib.pyplot as plt
import numpy as np

```

```
# seperate data into x and y variables

X = np.array([national_processed['ROOMS'],national_processed['BEDRMS']])
#              national_processed['COSTMED']])

```

```
X
```

    array([[6, 6, 7, ..., 5, 3, 3],
           [2, 4, 4, ..., 3, 1, 1]])

```

centered1 = []
#centered2 = []
#centered3 = []
for i in range(len(X[0])):
  centr = X[0][i] - X[0].mean()
  centered1.append(centr)
#  centr2 = X[1][i]- X[1].mean()
#  centered2.append(centr2)
#  centr3 = X[2][i]-X[1].mean()
#  centered3.append(centr3)
std1 = np.std(X[0],axis =1)
#std2 = np.std(X[1],axis=1)
#std3 = np.std(X[2])
standardized = []
#standardized2 = []
#standardized3 = []
for i in range(len(centered1)):
  stndrzed = centered1[i] / std1
  standardized.append(stndrzed)
#  stndrzed2 = centered2[i] / std2
#  standardized2.append(stndrzed2)
#  stndrzed3 = centered3[i] / std3
#  standardized3.append(stndrzed3)

Z = np.array([standardized,standardized2])

```

```
means = np.mean(X.T, axis=1)
centered_data = X - means

stand_devs = np.std(X.T,axis=1)

standard_data = centered_data / stand_devs

```

```
Z = np.array(standard_data.T)

```

```
covariance_mat = np.cov(Z)

```

```
vals, vects = np.linalg.eig(covariance_mat)
print("eigenvalues:", vals)
print("eigenvectors:")
print(vects)
```

```
projected_d = vects.dot(standard_data)
pd.DataFrame(projected_d)
```

```
covariance_matrix = np.cov(Z)
evalues, evectors = np.linalg.eig(covariance_matrix)

print("eigenvalues:", evalues)
print("eigenvectors:")
print(evectors)
```

```
projected = evectors.dot(Z)
projected_df = pd.DataFrame(projected.T)
projected_df.head()
```

```
x = projected_d[0]
y = projected_d[1]

fig, ax = plt.subplots(figsize=(8,8))
ax.grid()
ax.set_xlim(-10,10)
ax.set_ylim(-10,10)
ax.axhline(0,color = 'k')
ax.axvline(0,color = 'k')
plt.scatter(x,y)
plt.show()
```

```

```

```
# Principal Component Analysis
from numpy import array
from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA
# define a matrix

print("Data: \n", X)
# Standardize the Data
# Instantiate a Standard Scaler object
scaler = StandardScaler()
# Use the object to fit_transform our data
Z = scaler.fit_transform(X)
print("\n Standardized Data: \n", Z)
# create the PCA instance
pca = PCA(2)
# fit on data
pca.fit(Z)
# access values and vectors
print("\n Eigenvectors: \n", pca.components_)
print("\n Eigenvalues: \n",pca.explained_variance_)
# transform data
B = pca.transform(Z)
print("\n Projected Data: \n", B)
```

    Data:
     [[6 6 7 ... 5 3 3]
     [2 4 4 ... 3 1 1]]

     Standardized Data:
     [[ 1.  1.  1. ...  1.  1.  1.]
     [-1. -1. -1. ... -1. -1. -1.]]

     Eigenvectors:
     [[ 3.93645878e-03  3.93645878e-03  3.93645878e-03 ...  3.93645878e-03
       3.93645878e-03  3.93645878e-03]
     [ 9.99992252e-01 -1.54958277e-05 -1.54958277e-05 ... -1.54958277e-05
      -1.54958277e-05 -1.54958277e-05]]

     Eigenvalues:
     [1.29068000e+05 2.67203605e-21]

     Projected Data:
     [[ 2.54035431e+02  9.74548766e-11]
     [-2.54035431e+02 -9.74548766e-11]]

# Stretch Goals

## 1) Perform further data exploration on the HADS national dataset (the version before we one-hot encoded it) Make scatterplots and see if you can see any resemblance between the original scatterplots and the plot of the principal components that you made in 7.1.

(You may or may not not see very much resemblance depending on the variables you choose, and that's ok!)

## 2) Study "Scree Plots" and then try and make one for your PCA dataset. How many principal conponents do you need to retain in order for your PCs to contain 90% of the explained variance?

We will present this topic formally at the beginning of tomorrow's lecture, so if you figure this stretch goal out, you're ahead of the game.

## 3) Explore further the intuition behind eigenvalues and eigenvectors by creating your very own eigenfaces:

Prioritize self-study over this stretch goal if you are not semi-comfortable with the topics of PCA, Eigenvalues, and Eigenvectors.

You don't necessarily have to use this resource, but this will get you started:
[Eigenface Tutorial](https://sandipanweb.wordpress.com/2018/01/06/eigenfaces-and-a-simple-face-detector-with-pca-svd-in-python/)