# Lab 4: Control flow-Loops

In this notebook, we propose and solve some exercises about loops.

**As a good programming practice, our test cases should ensure that all branches of the code are executed at least once.**

## List of exercises

1. Write a program that prints out the numbers from 0 to 10 (only even numbers). Use both: while and for loops.

* Input: no input
* Expected output:

```
0
2
4
6
8
10
```

```
i = 0
while i <= 10:
  print(i)
  i = i + 2

for i in range(0,12,2):
  print(i)

```

2. Write a program that prints out the numbers from 0 to 10 (only odd numbers). Use both: while and for loops.

* Input: no input
* Expected output:

```
1
3
5
7
9
```

```
i = 1
while i < 11:
  print(i)
  i = i + 2

for i in range(1,11,2):
  print(i)
```

3. Write a program that prints out the numbers from 10 to 0 (only odd numbers). Use both: while and for loops.

* Input: no input
* Expected output:

```
9
7
5
3
1
```

```
i = 9
while i >= 0:
  print(i)
  i = i - 2

for i in range(9,0,-2):
  print(i)
```

4. Write a program to calculate the factorial of an integer number. Use both: while and for loops.
  * factorial (n) = n * factorial (n-1)

* Input: an integer number, 5
* Expected output:

```
120
```

```
number = 5
fact = 1

if number < 0:
  fact = -1
elif number == 0 or number == 1:
  fact = 1
else:
  for i in range(1,number+1):
    fact *= i

print(fact)

```

5. Write a program to calculate the exponentiation of a number with base b, and exponent n, both: while and for loops.
  * $base^{exponent} = a * a* a...*a$

* Input: base 2, exponent 3
* Expected output:

```
8
```

```
base = 2
exponent = 3
mipow = 1

if exponent > 0:
  for i in range(0,exponent):
    mipow *= base

print(mipow)
```

6. Given an integer number, n, make the sum of all the numbers from 0 to n.

* Input: 5
* Expected output:

```
15
```

```
n = 5
add_n = 0
for i in range(0,n+1,1):
  add_n += i

print(add_n)
```

7. Given an integer number, n, print a cross of radius n as follows.

* Input: 9
* Expected output:

```
 * . . . . . . . *
 . * . . . . . * .
 . . * . . . * . .
 . . . * . * . . .
 . . . . * . . . .
 . . . * . * . . .
 . . * . . . * . .
 . * . . . . . * .
 * . . . . . . . *
```

```
n = 9
for i in range(0, n, 1):
  for j in range (0, n, 1):
    if (i==j or n-i-1 == j):
      print(" *", end="")
    else:
      print(" .", end="")
  print("")
```

8. Given an integer number, n, print a wedge of stars as follows.

* Input: 5
* Expected output:

```
*****
****
***
**
*
```

```
n_stars = 5
for i in range (n_stars, 0, -1):
  for j in range (0, i, 1):
    print("*", end="")
  print("")

#Pythonic
for i in range (n_stars, 0, -1):
    print("*"*i)
```

9. Given an integer number, n, print a wedge of stars as follows.

* Input: 5
* Expected output:

```
*
**
***
****
*****
```

```
n_stars = 5
for i in range (0, n_stars+1, 1):
    print("*"*i)
```

10. Make a program to display the multiplication table (1-10).

* Input: no input
* Expected output:

```
1	  2	 3	 4	  5	 6	 7	 8	 9	10
2	  4	 6	 8	 10	12	14	16	18	20
3	  6	 9	 12	15	18	21	24	27	30
4	  8	 12	16	20	24	28	32	36	40
5	  10	15	20	25	30	35	40	45	50
6	  12	18	24	30	36	42	48	54	60
7	  14	21	28	35	42	49	56	63	70
8	  16	24	32	40	48	56	64	72	80
9	  18	27	36	45	54	63	72	81	90
10	 20	30	40	50	60	70	80	90	100
```

```
for i in range(1,11,1):
  for j in range (1, 11, 1):
    print(i*j, end = "")
    print("\t", end = "")
  print("")

```

    1	2	3	4	5	6	7	8	9	10
    2	4	6	8	10	12	14	16	18	20
    3	6	9	12	15	18	21	24	27	30
    4	8	12	16	20	24	28	32	36	40
    5	10	15	20	25	30	35	40	45	50
    6	12	18	24	30	36	42	48	54	60
    7	14	21	28	35	42	49	56	63	70
    8	16	24	32	40	48	56	64	72	80
    9	18	27	36	45	54	63	72	81	90
    10	20	30	40	50	60	70	80	90	100

11. Write a program to detect if a number is a prime number.

* Input: 5, 8
* Expected output:

```
The number 5 is prime.
The number 8 is not prime.
```

```
n = 1
n_divisors = 1
divisor = 2
while divisor < n and n_divisors <= 2:
  if n % divisor == 0:
    n_divisors = n_divisors + 1
  divisor = divisor + 1

if n_divisors > 2:
  print("The number {} is not prime.".format(n))
else:
  print("The number {} is prime.".format(n))
```

12. Write a program to sum all odd numbers between n and 100 (included).

* Input: 11
* Expected output:

```
The sum between 11-100 is 4995.
```

```
top = 100
n = 11
add_top = 0
for value in range(11, top+1):
    add_top = add_top + value

print("The sum between {}-{} is {}".format(n,top,add_top))
```

    The sum between 11-100 is 4995

13. Write a program to show the square of the first 10 numbers.

* Input: no input
* Expected output:

```
The square of 0 is 0
The square of 1 is 1
The square of 2 is 4
The square of 3 is 9
The square of 4 is 16
The square of 5 is 25
The square of 6 is 36
The square of 7 is 49
The square of 8 is 64
The square of 9 is 81
The square of 10 is 100
```

```
for i in range(0, 11):
  print("The square of {} is {}".format(i,i**2))
```

    The square of 0 is 0
    The square of 1 is 1
    The square of 2 is 4
    The square of 3 is 9
    The square of 4 is 16
    The square of 5 is 25
    The square of 6 is 36
    The square of 7 is 49
    The square of 8 is 64
    The square of 9 is 81
    The square of 10 is 100

14. Write a program to show the Fibonnacci sequence of a given number n.

		fibonacci(n) =
						fibonacci (0) = 0
						fibonacci (1) = 1
						fibonacci (n) = fibonacci(n-1) + fibonacci (n-2)

* Input: a positive number n
* Expected output:

```
0
1
1
2
3
```

```
numbersToGenerate=5
fn2 = 0
fn1 = 1
print(fn2)
print(fn1)
for i in range(2, numbersToGenerate):
  fcurrent = fn1 + fn2
  temp = fn1
  fn1 = fcurrent
  fn2 = temp
  print(fn1)
```

15. Write a program to check if an integer number is a palindrome.

* Input: 121
* Expected output:

```
The number 121 is palindrome: True
```

```
number = 121
palindrome = number
reverse = 0
while (palindrome != 0):
  remainder = palindrome % 10
  reverse = reverse * 10 + remainder
  palindrome = palindrome // 10

print("The number "+str(number)+" is palindrome: "+str((number==reverse)))
```

    The number 121 is palindrome: True

16. Write a program to check if an integer number of three digit is an Armstrong number.
   *  An Armstrong number of three digit is a number whose sum of cubes of its digit is equal to its number."

* Input: $153 = 1^3+5^3+3^3$ or $1+125+27=153$
* Expected output:

```
The number 153 is an Armstrong number.
```

```
number = 153
initial_number = number
result = 0
if number>=100 and number <= 999:
  while number != 0:
    remainder = number % 10
    result = result + remainder**3
    number = number // 10
  if result == initial_number:
    print("The number {} is an Armstrong number.".format(result))
  else:
    print("The number {} is NOT an Armstrong number.".format(result))
else:
  print("Number is not valid.")
```

17. Write a program to show the first N prime numbers.

* Input: N = 10
* Expected output:

```
The number 1 is prime.
The number 2 is prime.
The number 3 is prime.
The number 5 is prime.
The number 7 is prime.
The number 11 is prime.
The number 13 is prime.
The number 17 is prime.
The number 19 is prime.
The number 23 is prime.
```

```
counter = 0
MAX_PRIMES = 10
n = 1

while counter < MAX_PRIMES:
  n_divisors = 1
  divisor = 1
  #Loop if n is prime
  while divisor < n and n_divisors <= 2:
    if n % divisor == 0:
      n_divisors = n_divisors + 1
    divisor = divisor + 1
  #End loop if n is prime
  #Step second loop
  if n_divisors <= 2:
    print("The number {} is prime.".format(n))
    counter = counter + +1

  n = n + 1
```

18. Write a program to calculate the combinatorial number

$\binom{m}{n} = \frac{m!}{n! (m-n)!}$

* Input: m = 10, n = 5
* Expected output:

```
252
```

```
m = 10
n = 5
factorialmn = 1
factorialn = 1
factorialm = 1

if (m < n):
  print("M must be >= n");
else:
  #Factorial m
  if m == 0 or m == 1:
    factorialm = 1
  else:
    i = m
    while i > 0:
      factorialm = i * factorialm
      i = i - 1
  #Factorial n
  if n == 0 or n == 1:
    factorialn = 1
  else:
    i = n
    while i>0 :
      factorialn = i * factorialn
      i = i - 1
  #Factorial m-n
  if (m - n == 0 or m - n == 1):
    factorialmn = 1
  else:
    i =  m - n
    while i>0 :
      factorialmn = i * factorialmn
      i = i - 1
  print("The result is: ",(factorialm / (factorialn * factorialmn)))
```

19. Write a program to print a Christmas tree of 15 base stars.

* Input: base_stars = 15
* Expected output:

```
       *
      ***
     *****
    *******
   *********
  ***********
 *************
      ***
      ***
      ***
```

```
base_stars = 15
half_blank_spaces = 0;
for i in range(1,base_stars, 2):
  half_blank_spaces = (base_stars-i) // 2
  for j in range(0, half_blank_spaces):
    print(" ",end="")
  for k in range(0, i):
    print("*",end="")
  for j in range(0, half_blank_spaces):
    print(" ",end="")
  print("")

half_blank_spaces = (base_stars//2)-1;
for i in range(3):
  for j in range(0,half_blank_spaces):
    print(" ",end="")
  for k in range(0, 3):
    print("*",end="")
  for j in range(0, half_blank_spaces):
    print(" ",end="")
  print("")
```

           *
          ***
         *****
        *******
       *********
      ***********
     *************
          ***
          ***
          ***

20. Write a program to print an * in the upper diagonal of an implicit matrix of size n.

* Input: n = 3
* Expected output:

```
* * *
. * *
. . *
```

```
n = 3
for i in range(1,n+1):
  for j in range (1,n+1):
    if (i <= j):
      print("*", end = "")
    else:
      print(".", end = "")
  print("")
```

21. Write a program to print an * in the lower diagonal of an implicit matrix of size n.

* Input: n = 3
* Expected output:

```
* . .
* * .
* * *
```

```
n = 3
for i in range(1,n+1):
  for j in range (1,n+1):
    if (i >= j):
      print("*", end = "")
    else:
      print(".", end = "")
  print("")
```

22. Write a program to print a diamond of radius n.

* Input: n = 7
* Expected output:

```
 . . . * . . .
 . . * * * . .
 . * * * * * .
 * * * * * * *
 . * * * * * .
 . . * * * . .
 . . . * . . .
```

```
n = 7
mid = n//2
lmin=0
rmax=0
if (n % 2 != 0):
  for i in range(0,n):
    if i <= mid:
      lmin=mid-i
      rmax=mid+i
    else:
      lmin=mid-(n-i)+1
      rmax=mid+(n-i)-1
    for j in range (0,n):
      if j>=lmin and j<=rmax:
        print(" *", end = "")
      else:
        print(" .", end = "")
    print("")
```

23. Write a program that displays a table of size (n x n) in which each cell will have * if i is a divisor of j or "" otherwise.

* Input: n = 10
* Expected output:

```
* * * * * * * * * * 1
* *   *   *   *   * 2
*   *     *     *   3
* *   *       *     4
*       *         * 5
* * *     *         6
*           *       7
* *   *       *     8
*   *           *   9
* *     *         * 10

```

```
N = 10
for i in range (1, N+1):
  for j in range (1, N+1):
    if (i%j == 0 or j%i == 0):
      print(" *", end = "")
    else:
      print("  ", end = "")
  print(" ",i)
  print("")
```

24. Write a program to display a menu with 3 options (to say hello in English, Spanish and French) and to finish, the user shall introduce the keyword "quit". If any other option is introduced, the program shall display that the input value is not a valid option.

* Input: test the options
* Expected output:

```
----------MENU OPTIONS----------
1-Say Hello!
2-Say ¡Hola!
3-Say Salut!
> introduce an option or quit to exit...
```

```
option = "1"
while option != "quit":
  print("----------MENU OPTIONS----------")
  print("1-Say Hello!")
  print("2-Say ¡Hola!")
  print("3-Say Salut!")
  option = input("> introduce an option or quit to exit...")
  if option == "1":
    print("Hello!")
  elif option == "2":
    print("¡Hola!")
  elif option == "3":
    print("Salut!")
  elif option == "quit":
    print("...finishing...")
  else:
    print("Not a valid option: ", option)

```

25. The number finder: write a program that asks the user to find out a target number. If the input value is less than the target number, the program shall say "the target value is less than X", otherwise, the program shall say "the target value is greater than X. The program shall finish once the user finds out the target number.

* Input: target = 10
* Expected output:

```
Introduce a number: 3
The target value is greater than  3
Introduce a number: 5
The target value is greater than  5
Introduce a number: 12
The target value is less than  12
Introduce a number: 10
```

```
target = 10
found = False
while (not found):
  value = int(input("Introduce a number: "))
  if target < value:
    print("The target value is less than ", value)
  elif target > value:
    print("The target value is greater than ", value)
  else:
    found = True

if found:
  print("You have found out the target value!")
```

26. Modify the program in 20 to give only 5 attempts.

* Input: target = 10
* Expected output:

```
Introduce a number: 6
The target value is greater than  6
Introduce a number: 7
The target value is greater than  7
Introduce a number: 12
The target value is less than  12
Introduce a number: 9
The target value is greater than  9
Introduce a number: 11
The target value is less than  11
You have consumed the max number of attempts.
```

```
target = 10
found = False
attempts = 0
MAX_ATTEMPTS = 5
while (not found and attempts < MAX_ATTEMPTS):
  value = int(input("Introduce a number: "))
  if target < value:
    print("The target value is less than ", value)
  elif target > value:
    print("The target value is greater than ", value)
  else:
    found = True
  attempts = attempts + 1

if found:
  print("You have found out the target value!")
else:
  print("You have consumed the max number of attempts.")
```

27. Write a program to print the following pattern:

```
1
22
333
4444
55555
```

```
for i in range (1,6):
  print(str(i)*i)
```

28. Write a program to find greatest common divisor (GCD) of two numbers.

* Input: x = 54, y = 24
* Expected output:
* Tip: try to improve the algorithm following the strategies here: https://en.wikipedia.org/wiki/Greatest_common_divisor

```
The GCD of 54 and 24 is: 6
```

```
#Python version in the math library
def gcd(a, b):
    """Calculate the Greatest Common Divisor of a and b.

    Unless b==0, the result will have the same sign as b (so that when
    b is divided by it, the result comes out positive).
    """
    while b:
        a, b = b, a%b
    return a
```

```
x = 54
y = 24
gcd = 1
#find x divisor
current_divisor = 1
if x != 0 and y != 0:
  if x > y:
    top = y
  else:
    top = x
  while (current_divisor < top):
    if (x % current_divisor == 0) and (y % current_divisor == 0):
      gcd = current_divisor
    current_divisor = current_divisor + 1

  print("The GCD of {} and {} is: {}".format(x,y,gcd))
else:
  print("The input must be different to 0.")

```

29. Write a program that counts the number of primer numbers between 1-MAX.

* Input: MAX = 100
* Expected output:

```
The number of primer numbers between 1 and 100 is 26.
```

```
MAX = 100
n_primes = 0
n = 1

while n < MAX:
  n_divisors = 1
  divisor = 1
  #Loop if n is prime
  while divisor < n and n_divisors <= 2:
    if n % divisor == 0:
      n_divisors = n_divisors + 1
    divisor = divisor + 1
  #End loop if n is prime
  #Step second loop
  if n_divisors <= 2:
    n_primes = n_primes + 1

  n = n + 1

print("The number of primer numbers between 1 and {} is {}.".format(MAX, n_primes))
```

    The number of primer numbers between 1 and 100 is 26.

30. Write a program that sums all primer numbers between 1-MAX.

* Input: MAX = 100
* Expected output:

```
The sum of all primer numbers between 1 and 100 is 1061.
```

```
MAX = 100
sum_primes = 0
n = 1

while n < MAX:
  n_divisors = 1
  divisor = 1
  #Loop if n is prime
  while divisor < n and n_divisors <= 2:
    if n % divisor == 0:
      n_divisors = n_divisors + 1
    divisor = divisor + 1
  #End loop if n is prime
  #Step second loop
  if n_divisors <= 2:
    sum_primes = sum_primes + n

  n = n + 1

print("The sum of all primer numbers between 1 and {} is {}.".format(MAX, sum_primes))
```

    The sum of all primer numbers between 1 and 100 is 1061.

31. Write a program to approximate the value of $e$ using the Taylor series for $e^x$.

$\begin{align}
   \sum_{n=0}^\infty \frac{x^n}{n!} &= \frac{x^0}{0!} + \frac{x^1}{1!} + \frac{x^2}{2!} + \frac{x^3}{3!} + \frac{x^4}{4!} + \frac{x^5}{5!}+ \cdots
 \end{align}$

 * Input: x = 1, MAX (N) = 100
 * Expected output:

```
The value of e is 2.7182818284590455
```

```
sum_e = 0
MAX = 100
x = 1
for n in range (1, MAX+1):
 factorial_n = 1
 for i in range(1,n):
   factorial_n = i * factorial_n
 sum_e = sum_e + (1/factorial_n)

print("The value of e is {}".format(sum_e))
```

32. Write a program to detect whether a number is a perfect number.

* A number is perfect if the addition of all positive divisors is equal to the number.

* Input: n = 6
* Expected output:

```
The number 6 is perfect.
```

```
n = 6
perfect = 0
for i in range (1,n):
  if n%i == 0:
    perfect = perfect + i

if perfect == n:
  print("The number {} is perfect.".format(n))
else:
  print("The number {} is NOT perfect.".format(n))
```


```python
import sympy as sp
```

```python
from sympy.physics.vector import init_vprinting
init_vprinting(use_latex='mathjax', pretty_print=False)
```

```python
from IPython.display import Image
Image('fig/2rp_new.png', width=300)
```

```python
from sympy.physics.mechanics import dynamicsymbols
```

```python
theta1, theta2, l1, l2, theta, alpha, a, d = dynamicsymbols('theta1 theta2 l1 l2 theta alpha a d')
theta1, theta2, l1, l2, theta, alpha, a, d
```

$\displaystyle \left( \theta_{1}, \  \theta_{2}, \  l_{1}, \  l_{2}, \  \theta, \  \alpha, \  a, \  d\right)$

```python
rot = sp.Matrix([[sp.cos(theta), -sp.sin(theta)*sp.cos(alpha), sp.sin(theta)*sp.sin(alpha)],
                 [sp.sin(theta), sp.cos(theta)*sp.cos(alpha), -sp.cos(theta)*sp.sin(alpha)],
                 [0, sp.sin(alpha), sp.cos(alpha)]])

trans = sp.Matrix([a*sp.cos(theta), a*sp.sin(theta),d])

last_row = sp.Matrix([[0, 0, 0, 1]])

m = sp.Matrix.vstack(sp.Matrix.hstack(rot, trans), last_row)
m
```

$\displaystyle \left[\begin{matrix}\cos{\left(\theta \right)} & - \sin{\left(\theta \right)} \cos{\left(\alpha \right)} & \sin{\left(\alpha \right)} \sin{\left(\theta \right)} & a \cos{\left(\theta \right)}\\\sin{\left(\theta \right)} & \cos{\left(\alpha \right)} \cos{\left(\theta \right)} & - \sin{\left(\alpha \right)} \cos{\left(\theta \right)} & a \sin{\left(\theta \right)}\\0 & \sin{\left(\alpha \right)} & \cos{\left(\alpha \right)} & d\\0 & 0 & 0 & 1\end{matrix}\right]$

```python
# Transformation: frame '0' to '1'

m01 = m.subs({alpha:0, a:l1, theta: theta1, d:0})
m01
```

$\displaystyle \left[\begin{matrix}\cos{\left(\theta_{1} \right)} & - \sin{\left(\theta_{1} \right)} & 0 & l_{1} \cos{\left(\theta_{1} \right)}\\\sin{\left(\theta_{1} \right)} & \cos{\left(\theta_{1} \right)} & 0 & l_{1} \sin{\left(\theta_{1} \right)}\\0 & 0 & 1 & 0\\0 & 0 & 0 & 1\end{matrix}\right]$

```python
# Transformation: frame '1' to '2'

m12 = m.subs({alpha:0, a:l2, theta: theta2, d:0})
m12
```

$\displaystyle \left[\begin{matrix}\cos{\left(\theta_{2} \right)} & - \sin{\left(\theta_{2} \right)} & 0 & l_{2} \cos{\left(\theta_{2} \right)}\\\sin{\left(\theta_{2} \right)} & \cos{\left(\theta_{2} \right)} & 0 & l_{2} \sin{\left(\theta_{2} \right)}\\0 & 0 & 1 & 0\\0 & 0 & 0 & 1\end{matrix}\right]$

```python
# Homogenous transformation: frame '0' to '2'

m02 = (m01*m12)
m02
```

$\displaystyle \left[\begin{matrix}- \sin{\left(\theta_{1} \right)} \sin{\left(\theta_{2} \right)} + \cos{\left(\theta_{1} \right)} \cos{\left(\theta_{2} \right)} & - \sin{\left(\theta_{1} \right)} \cos{\left(\theta_{2} \right)} - \sin{\left(\theta_{2} \right)} \cos{\left(\theta_{1} \right)} & 0 & l_{1} \cos{\left(\theta_{1} \right)} - l_{2} \sin{\left(\theta_{1} \right)} \sin{\left(\theta_{2} \right)} + l_{2} \cos{\left(\theta_{1} \right)} \cos{\left(\theta_{2} \right)}\\\sin{\left(\theta_{1} \right)} \cos{\left(\theta_{2} \right)} + \sin{\left(\theta_{2} \right)} \cos{\left(\theta_{1} \right)} & - \sin{\left(\theta_{1} \right)} \sin{\left(\theta_{2} \right)} + \cos{\left(\theta_{1} \right)} \cos{\left(\theta_{2} \right)} & 0 & l_{1} \sin{\left(\theta_{1} \right)} + l_{2} \sin{\left(\theta_{1} \right)} \cos{\left(\theta_{2} \right)} + l_{2} \sin{\left(\theta_{2} \right)} \cos{\left(\theta_{1} \right)}\\0 & 0 & 1 & 0\\0 & 0 & 0 & 1\end{matrix}\right]$

```python
# simplify the transformation as

mbee = sp.Matrix([[m02[0,0].simplify(), m02[0,1].simplify(), sp.trigsimp(m02[0,3].simplify())],
                  [m02[1,0].simplify(), m02[1,1].simplify(), sp.trigsimp(m02[1,3].simplify())],
                  [m02[2,0].simplify(), m02[2,1].simplify(), m02[2,2].simplify() ]])

mbee
```

$\displaystyle \left[\begin{matrix}\cos{\left(\theta_{1} + \theta_{2} \right)} & - \sin{\left(\theta_{1} + \theta_{2} \right)} & l_{1} \cos{\left(\theta_{1} \right)} + l_{2} \cos{\left(\theta_{1} + \theta_{2} \right)}\\\sin{\left(\theta_{1} + \theta_{2} \right)} & \cos{\left(\theta_{1} + \theta_{2} \right)} & l_{1} \sin{\left(\theta_{1} \right)} + l_{2} \sin{\left(\theta_{1} + \theta_{2} \right)}\\0 & 0 & 1\end{matrix}\right]$

```python
# Forward Kinematic Equations

# position in x-direction

px = mbee[0,2]
px
```

$\displaystyle l_{1} \cos{\left(\theta_{1} \right)} + l_{2} \cos{\left(\theta_{1} + \theta_{2} \right)}$

```python
# position in y-direction

py = mbee[1,2]
py
```

$\displaystyle l_{1} \sin{\left(\theta_{1} \right)} + l_{2} \sin{\left(\theta_{1} + \theta_{2} \right)}$

```python
# Evaluation of tip position

fx = sp.lambdify([l1, l2, theta1, theta2], px, 'numpy')
fy = sp.lambdify([l1, l2, theta1, theta2], py, 'numpy')
```

```python
# plotting tip position in X & Y Plane

import matplotlib.pyplot as plt
%matplotlib inline
import numpy as np
d2r = np.deg2rad
```

```python
theta1s = np.linspace(d2r(0), d2r(360)) #desired range of motion for joint 1
theta2s = np.linspace(d2r(0), d2r(360)) #desired range of motion for joint 2

zx = np.array(fx(15.0, 15.0, theta1s, theta2s))
zy = np.array(fy(15.0, 15.0, theta1s, theta2s))

fig, ax1 = plt.subplots()
ax1.plot(np.rad2deg(theta1s), zx, label = r'$p_x$')
ax1.plot(np.rad2deg(theta1s), zy, label = r'$p_y$')
ax1.set_xlabel(r'($\theta_1$, $\theta_2$) [deg]')
ax1.set_ylabel(r' tip position [mm]')
plt.legend()
plt.grid()
```

```python
# Manipulator Workspace

from plotly.offline import download_plotlyjs, init_notebook_mode, plot, iplot
init_notebook_mode(connected=True) # for offline mode in Jupyter Notebook use

import plotly.offline as py
import plotly.graph_objs as go
from numpy import cos, sin
```

```python
theta11 = np.linspace(d2r(0), d2r(90))
theta22 = np.linspace(d2r(0), d2r(360))
theta1, theta2 = np.meshgrid(theta11, theta22)
l_range = [5]

px1 = {}
py1 = {}
pz1 = {}
for i in l_range:
    l1 = i
    l2 = i-4

    pxa = l1*cos(theta1) + l2*cos(theta1 + theta2)
    pya = l1*sin(theta1) + l2*sin(theta1 + theta2)

    px1[f'x{i}'] = pxa
    py1[f'x{i}'] = pya
```

```python
pxx = px1['x5']
pyy = px1['x5']
pzz = pyy*0 #dummy zero points for z-axis, as it doesn't exist
```

```python
trace1 = go.Surface(z=pzz, x= pxx, y=pyy,
                    colorscale='Reds',
                    showscale=False,
                    opacity=0.7,
                   )
data = [trace1]
```

```python
layout = go.Layout(scene=dict(xaxis = dict(title='X (mm)'),
                              yaxis = dict(title='Y (mm)'),
                              zaxis = dict(title='Z (mm)'),
                             ),
                  )
```

```python
fig = go.Figure(data=data, layout=layout)
py.iplot(fig)
```


### Uniform Distribution
#### Continous
Consider,
$$\mathbb{U}[N_{bin},r'_{min},r'_{max},r_{min},r_{max}] = \frac{r'_{max}-r'_{min}}{N_{bin}(r_{max}-r_{min})} $$
Since in Matlab rand, the uniform distribution is defined in the range of $[0,1]$,
$$r\sim \mathbb{U}[0,1]$$
To extend the range $r\in [0,1]$ to $r'\in [a,b]$,
$$r' = r(b-a)+a$$

#### Discrete
\begin{equation}
p(x)=\left\{
  \begin{array}{@{}ll@{}}
    0.25, & \text{if}\ x\in\{1,2,3,4\} \\
    0, & \text{otherwise}
  \end{array}\right.
\end{equation}

```python
import numpy as np
import matplotlib.pyplot as plt
```

```python
s = np.random.uniform(1,5,10000)
count, bins, ignored = plt.hist(s, 100, density=True, align='left',rwidth=0.5)
plt.plot(bins, 0.25*np.ones_like(bins), linewidth=2, color='r')
```

```python
s = np.random.randint(1,5,100000)
plt.hist(s,   density=True, align='mid', rwidth=0.5)
```

### Degenerate Distribution
\begin{equation}
p(x)=\left\{
  \begin{array}{@{}ll@{}}
    1, & \text{if}\ x=1 \\
    0, & \text{if}\ x\in\{2,3,4\}
  \end{array}\right.
\end{equation}

```python
s = np.random.uniform(0,1,1000)
s = np.concatenate((s,np.zeros(5000)))
count, bins, ignored = plt.hist(s, 1, density=True, align='right')
plt.plot(range(0,6), np.ones(6), linewidth=2, color='r')
```

```python

```

Machine Learning A Probabilistic Perspective/2Probability/.ipynb_checkpoints/2.1discreteProbDistFig-checkpoint.ipynb


# Unsupervised Learning

```python
# Setup
from utils.lecture10 import *
%matplotlib inline
```

### Supervised vs Unsupervised Learning

The difference between *supervised learning* and *unsupervised learning* is that in the first case we have a variable $y$ which we want to predict, given a set of variables $\{ X_1, . . . , X_p \}$.

In *unsupervised learning* are not interested in prediction, because we do not have an associated response variable $y$. Rather, the goal is to discover interesting properties about the measurements on $\{ X_1, . . . , X_p \}$.

Questions that we are usually interested in are
- Clustering
- Dimensionality reduction

In general, unsupervised learning can be viewed as an extention of exploratory data analysis.

### Dimensionality Reduction

Working in high-dimensional spaces can be undesirable for many reasons; raw data are often sparse as a consequence of the curse of dimensionality, and analyzing the data is usually computationally intractable (hard to control or deal with).

Dimensionality reduction can also be useful to plot high-dimensional data.

### Clustering

Clustering refers to a very broad set of techniques for finding subgroups, or clusters, in a data set. When we cluster the observations of a data set, we seek to partition them into distinct groups so that the observations within each group are quite similar to each other, while observations in different groups are quite different from each other.

In this section we focus on the following algorithms:

1. **K-means clustering**
2. **Hierarchical clustering**
3. **Gaussian Mixture Models**

## Principal Component Analysis

Suppose that we wish to visualize $n$ observations with measurements on a set of $p$ features, $\{X_1, . . . , X_p\}$, as part of an exploratory data analysis.

We could do this by examining two-dimensional scatterplots of the data, each of which contains the n observations’ measurements on two of the features. However, there are $p(p−1)/2$ such scatterplots; for example,
with $p = 10$ there are $45$ plots!

PCA provides a tool to do just this. It finds a low-dimensional represen- tation of a data set that contains as much as possible of the variation.

### First Principal Component

The **first principal component** of a set of features $\{X_1, . . . , X_p\}$ is the normalized linear combination of the features $Z_1$

$$
Z_1 = \phi_{11} X_1 + \phi_{21} X_2 + ... + \phi_{p1} X_p
$$

that has the largest variance.

By normalized, we mean that $\sum_{i=1}^p \phi^2_{i1} = 1$.

### PCA Computation

In other words, the first principal component loading vector solves the optimization problem

$$
\underset{\phi_{11}, \ldots, \phi_{p 1}}{\max} \ \Bigg \lbrace \frac{1}{n} \sum _ {i=1}^{n}\left(\sum _ {j=1}^{p} \phi _ {j1} x _ {ij} \right)^{2} \Bigg \rbrace \quad \text { subject to } \quad \sum _ {j=1}^{p} \phi _ {j1}^{2}=1
$$

The objective that we are maximizing is just the sample variance of the $n$ values of $z_{i1}$.

After the first principal component $Z_1$ of the features has been determined, we can find the second principal component $Z_2$. The **second principal component** is the linear combination of $\{X_1, . . . , X_p\}$ that has maximal variance out of all linear combinations that are *uncorrelated* with $Z_1$.

### Example

We illustrate the use of PCA on the `USArrests` data set.

For each of the 50 states in the United States, the data set contains the number of arrests per $100,000$ residents for each of three crimes: `Assault`, `Murder`, and `Rape.` We also record the percent of the population in each state living in urban areas, `UrbanPop`.

```python
# Load crime data
df = pd.read_csv('data/USArrests.csv', index_col=0)
df.head()
```

      Murder
      Assault
      UrbanPop
      Rape

      State

      Alabama
      13.2
      236
      58
      21.2

      Alaska
      10.0
      263
      48
      44.5

      Arizona
      8.1
      294
      80
      31.0

      Arkansas
      8.8
      190
      50
      19.5

      California
      9.0
      276
      91
      40.6

### Data Scaling

To make all the features comparable, we first need to scale them. In this case, we use the `sklearn.preprocessing.scale()` function to normalize each variable to have zero mean and unit variance.

```python
# Scale data
X_scaled = pd.DataFrame(scale(df), index=df.index, columns=df.columns).values
```

We will see later what are the practical implications of (not) scaling.

### Fitting

Let's fit PCA with 2 components.

```python
# Fit PCA with 2 components
pca2 = PCA(n_components=2).fit(X_scaled)
```

```python
# Get weights
weights = pca2.components_.T
df_weights = pd.DataFrame(weights, index=df.columns, columns=['PC1', 'PC2'])
df_weights
```

      PC1
      PC2

      Murder
      0.535899
      0.418181

      Assault
      0.583184
      0.187986

      UrbanPop
      0.278191
      -0.872806

      Rape
      0.543432
      -0.167319

### Projecting the data

What does the trasformed data looks like?

```python
# Transform X to get the principal components
X_dim2 = pca2.transform(X_scaled)
df_dim2 = pd.DataFrame(X_dim2, columns=['PC1', 'PC2'], index=df.index)
df_dim2.head()
```

      PC1
      PC2

      State

      Alabama
      0.985566
      1.133392

      Alaska
      1.950138
      1.073213

      Arizona
      1.763164
      -0.745957

      Arkansas
      -0.141420
      1.119797

      California
      2.523980
      -1.542934

### Visualization

The advantage og PCA is that it allows us to see the variation in lower dimesions.

```python
make_figure_10_1a(df_dim2, df_weights)
```

### PCA and Spectral Analysis

In case you haven't noticed, calculating principal components, is equivalent to calculating the eigenvectors of the design matrix $X'X$, i.e. the variance-covariance matrix of $X$. Indeed what we performed above is a decomposition of the variance of $X$ into orthogonal components.

The constrained maximization problem above can be re-written in matrix notation as

$$
\max \ \phi' X'X \phi \quad \text{ s. t. } \quad \phi'\phi = 1
$$

Which has the following dual representation

$$
\mathcal L (\phi, \lambda) = \phi' X'X \phi - \lambda (\phi'\phi - 1)
$$

If we take the first order conditions

$$
\begin{align}
& \frac{\partial \mathcal L}{\partial \lambda} = \phi'\phi - 1 \\
& \frac{\partial \mathcal L}{\partial \phi} = 2 X'X \phi - 2 \lambda \phi
\end{align}
$$

Setting the derivatives to zero at the optimum, we get

$$
\begin{align}
& \phi'\phi = 1 \\
& X'X \phi = \lambda \phi
\end{align}
$$

Thus, $\phi$ is an **eigenvector** of the covariance matrix $X'X$, and the maximizing vector will be the one associated with the largest **eigenvalue** $\lambda$.

### Eigenvalues and eigenvectors

We can now double-check it using `numpy` linear algebra package.

```python
eigenval, eigenvec = np.linalg.eig(X_scaled.T @ X_scaled)
data = np.concatenate((eigenvec,eigenval.reshape(1,-1)))
idx = list(df.columns) + ['Eigenvalue']
df_eigen = pd.DataFrame(data, index=idx, columns=['PC1', 'PC2','PC3','PC4'])

df_eigen
```

      PC1
      PC2
      PC3
      PC4

      Murder
      0.535899
      0.418181
      0.649228
      -0.341233

      Assault
      0.583184
      0.187986
      -0.743407
      -0.268148

      UrbanPop
      0.278191
      -0.872806
      0.133878
      -0.378016

      Rape
      0.543432
      -0.167319
      0.089024
      0.817778

      Eigenvalue
      124.012079
      49.488258
      8.671504
      17.828159

The spectral decomposition of the variance of $X$ generates a set of orthogonal vectors (eigenvectors) with different magnitudes (eigenvalues). The eigenvalues tell us the amount of variance of the data in that direction.

If we combine the eigenvectors together, we form a projection matrix $P$ that we can use to transform the original variables: $\tilde X = P X$

```python
X_transformed = X_scaled @ eigenvec
df_transformed = pd.DataFrame(X_transformed, index=df.index, columns=['PC1', 'PC2','PC3','PC4'])

df_transformed.head()
```

      PC1
      PC2
      PC3
      PC4

      State

      Alabama
      0.985566
      1.133392
      0.156267
      -0.444269

      Alaska
      1.950138
      1.073213
      -0.438583
      2.040003

      Arizona
      1.763164
      -0.745957
      -0.834653
      0.054781

      Arkansas
      -0.141420
      1.119797
      -0.182811
      0.114574

      California
      2.523980
      -1.542934
      -0.341996
      0.598557

This is exactly the dataset that we obtained before.

### Scaling the Variables

The results obtained when we perform PCA will also depend on whether the variables have been individually scaled. In fact, the variance of a variable depends on its magnitude.

```python
# Variables variance
df.var(axis=0)
```

    Murder        18.970465
    Assault     6945.165714
    UrbanPop     209.518776
    Rape          87.729159
    dtype: float64

Consequently, if we perform PCA on the unscaled variables, then the first principal component loading vector will have a very large loading for `Assault`, since that variable has by far the highest variance.

```python
# Fit PCA with unscaled varaibles
X = df.values
pca2_u = PCA(n_components=2).fit(X)
```

```python
# Get weights
weights_u = pca2_u.components_.T
df_weights_u = pd.DataFrame(weights_u, index=df.columns, columns=['PC1', 'PC2'])
df_weights_u
```

      PC1
      PC2

      Murder
      0.041704
      0.044822

      Assault
      0.995221
      0.058760

      UrbanPop
      0.046336
      -0.976857

      Rape
      0.075156
      -0.200718

```python
# Transform X to get the principal components
X_dim2_u = pca2_u.transform(X)
df_dim2_u = pd.DataFrame(X_dim2_u, columns=['PC1', 'PC2'], index=df.index)
```

### Plotting

We can compare the lower dimensional representations with and without scaling.

```python
make_figure_10_1b(df_dim2, df_dim2_u, df_weights, df_weights_u)
```

As predicted, the first principal component loading vector places almost all of its weight on `Assault`, while the second principal component loading vector places almost all of its weight on `UrpanPop`. Comparing this to the left-hand plot, we see that scaling does indeed have a substantial effect on the results obtained. However, this result is simply a consequence of the scales on which the variables were measured.

### The Proportion of Variance Explained

We can now ask a natural question: how much of the information in a given data set is lost by projecting the observations onto the first few principal components? That is, how much of the variance in the data is not contained in the first few principal components? More generally, we are interested in knowing the **proportion of variance explained (PVE)** by each principal component.

```python
# Four components
pca4 = PCA(n_components=4).fit(X_scaled)
```

```python
# Variance of the four principal components
pca4.explained_variance_
```

    array([2.53085875, 1.00996444, 0.36383998, 0.17696948])

### Interpretation

We can compute it in percentage of the total variance.

```python
# As a percentage of the total variance
pca4.explained_variance_ratio_
```

    array([0.62006039, 0.24744129, 0.0891408 , 0.04335752])

In the `Arrest` dataset, the first principal component explains $62.0\%$ of the variance in the data, and the next principal component explains $24.7\%$ of the variance. Together, the first two principal components explain almost $87\%$ of the variance in the data, and the last two principal components explain only $13\%$ of the variance.

### Plotting

We can plot in a graph the percentage of the variance explained, relative to the number of components.

```python
make_figure_10_2(pca4)
```

### How Many Principal Components?

In general, a $n \times p$ data matrix $X$ has $\min\{n − 1, p\}$ distinct principal components. However, we usually are not interested in all of them; rather, we would like to use just the first few principal components in order to visualize or interpret the data.

We typically decide on the number of principal components required to visualize the data by examining a *scree plot*.

However, there is no well-accepted objective way to decide how many principal com- ponents are enough.

## K-Means Clustering

The idea behind K-means clustering is that a good clustering is one for which the within-cluster variation is as small as possible. Hence we want to solve the problem

$$
\underset{C_{1}, \ldots, C_{K}}{\operatorname{minimize}} \Bigg\lbrace \sum_{k=1}^{K} W\left(C_{k}\right) \Bigg\rbrace
$$

where $C_k$ is a cluster and $ W(C_k)$ is a measure of the amount by which the observations within a cluster differ from each other.

There are many possible ways to define this concept, but by far the most common choice involves **squared Euclidean distance**. That is, we define

$$
W\left(C_{k}\right)=\frac{1}{\left|C_{k}\right|} \sum_{i, i^{\prime} \in C_{k}} \sum_{j=1}^{p}\left(x_{i j}-x_{i^{\prime} j}\right)^2
$$

where $|C_k|$ denotes the number of observations in the $k^{th}$ cluster.

### Algorithm

1. Randomly assign a number, from $1$ to $K$, to each of the observations. These serve as initial cluster assignments for the observations.

2. Iterate until the cluster assignments stop changing:

    a) For each of the $K$ clusters, compute the cluster centroid. The kth cluster centroid is the vector of the $p$ feature means for the observations in the $k^{th}$ cluster.

    b) Assign each observation to the cluster whose centroid is closest (where closest is defined using Euclidean distance).

### Generate the data

We first generate a 2-dimensional dataset.

```python
# Simulate data
np.random.seed(123)
X = np.random.randn(50,2)
X[0:25, 0] = X[0:25, 0] + 3
X[0:25, 1] = X[0:25, 1] - 4
```

```python
make_new_figure_1(X)
```

### Step 1: random assignement

Now let's randomly assign the data to two clusters, at random.

```python
# Init clusters
K = 2
clusters0 = np.random.randint(K,size=(np.size(X,0)))
```

```python
make_new_figure_2(X, clusters0)
```

### Step 2: estimate distributions

What are the new centroids?

```python
# Compute new centroids
def compute_new_centroids(X, clusters):
    K = len(np.unique(clusters))
    centroids = np.zeros((K,np.size(X,1)))
    for k in range(K):
        if sum(clusters==k)>0:
            centroids[k,:] = np.mean(X[clusters==k,:], axis=0)
        else:
            centroids[k,:] = np.mean(X, axis=0)
    return centroids
```

```python
# Print
centroids0 = compute_new_centroids(X, clusters0)
print(centroids0)
```

    [[ 1.54179703 -1.65922379]
     [ 1.67917325 -2.36272948]]

### Plotting the centroids

Let's add the centroids to the graph.

```python
# Plot
plot_assignment(X, centroids0, clusters0, 0, 0)
```

### Step 3: assign data to clusters

Now we can assign the data to the clusters, according to the closest centroid.

```python
# Assign X to clusters
def assign_to_cluster(X, centroids):
    K = np.size(centroids,0)
    dist = np.zeros((np.size(X,0),K))
    for k in range(K):
        dist[:,k] = np.mean((X - centroids[k,:])**2, axis=1)
    clusters = np.argmin(dist, axis=1)

    # Compute inertia
    inertia = 0
    for k in range(K):
        if sum(clusters==k)>0:
            inertia += np.sum((X[clusters==k,:] - centroids[k,:])**2)
    return clusters, inertia
```

### Plotting assigned data

```python
# Get cluster assignment
[clusters1,d] = assign_to_cluster(X, centroids0)
```

```python
# Plot
plot_assignment(X, centroids0, clusters1, d, 1)
```

### Full Algorithm

We now have all the components to proceed iteratively.

```python
def kmeans_manual(X, K):

    # Init
    i = 0
    d0 = 1e4
    d1 = 1e5
    clusters = np.random.randint(K,size=(np.size(X,0)))

    # Iterate until convergence
    while np.abs(d0-d1) > 1e-10:
        d1 = d0
        centroids = compute_new_centroids(X, clusters)
        [clusters, d0] = assign_to_cluster(X, centroids)
        plot_assignment(X, centroids, clusters, d0, i)
        i+=1
```

### Plotting k-means clustering

```python
# Test
kmeans_manual(X, K)
```

Here the observations can be easily plotted because they are two-dimensional.
If there were more than two variables then we could instead perform PCA
and plot the first two principal components score vectors.

### More clusters

In the previous example, we knew that there really were two clusters because
we generated the data. However, for real data, in general we do not know
the true number of clusters. We could instead have performed K-means
clustering on this example with `K  =  3`. If we do this, K-means clustering will split up the two "real" clusters, since it has no information about them:

```python
# K=3
kmeans_manual(X, 3)
```

### Sklearn package

The automated function in `sklearn` to persorm $K$-means clustering is `KMeans`.

```python
# SKlearn algorithm
km1 = KMeans(n_clusters=3, n_init=1, random_state=1)
km1.fit(X)
```

    KMeans(n_clusters=3, n_init=1, random_state=1)

### Plotting

We can plot the asssignment generated by the `KMeans` function.

```python
# Plot
plot_assignment(X, km1.cluster_centers_, km1.labels_, km1.inertia_, km1.n_iter_)
```

As we can see, the results are different in the two algorithms? Why? $K$-means is susceptible to the initial values. One way to solve this problem is to run the algorithm multiple times and report only the best results

### Initial Assignment

To run the `Kmeans()` function in python with multiple initial cluster assignments, we use the `n_init` argument (default: 10). If a value of `n_init` greater than one is used, then K-means clustering will be performed using multiple random assignments, and the `Kmeans()` function will report only the best results.

```python
# 30 runs
km_30run = KMeans(n_clusters=3, n_init=30, random_state=1).fit(X)
plot_assignment(X, km_30run.cluster_centers_, km_30run.labels_, km_30run.inertia_, km_30run.n_iter_)
```

### Best Practices

It is generally recommended to always run K-means clustering with a large value of `n_init`, such as 20 or 50 to avoid getting stuck in an undesirable local optimum.

When performing K-means clustering, in addition to using multiple initial cluster assignments, it is also important to set a random seed using the `random_state` parameter. This way, the initial cluster assignments can be replicated, and the K-means output will be fully reproducible.

## Hierarchical Clustering

One potential disadvantage of K-means clustering is that it requires us to pre-specify the number of clusters $K$.

Hierarchical clustering is an alternative approach which does not require that we commit to a particular choice of $K$.

### The Dendogram

Hierarchical clustering has an added advantage over K-means clustering in that it results in an attractive tree-based representation of the observations, called a **dendrogram**.

```python
d = dendrogram(
        linkage(X, "complete"),
        leaf_rotation=90.,  # rotates the x axis labels
        leaf_font_size=8.,  # font size for the x axis labels
    )
```

### Interpretation

Each leaf of the *dendrogram* represents one observation.

As we move up the tree, some leaves begin to fuse into branches. These correspond to observations that are similar to each other. As we move higher up the tree, branches themselves fuse, either with leaves or other branches. The earlier (lower in the tree) fusions occur, the more similar the groups of observations are to each other.

We can use de *dendogram* to understand how similar two observations are: we can look for the point in the tree where branches containing those two obse rvations are first fused. The height of this fusion, as measured on the vertical axis, indicates how different the two observations are. Thus, observations that fuse at the very bottom of the tree are quite similar to each other, whereas observations that fuse close to the top of the tree will tend to be quite different.

The term **hierarchical** refers to the fact that clusters obtained by cutting the dendrogram at a given height are necessarily nested within the clusters obtained by cutting the dendrogram at any greater height.

### The Hierarchical Clustering Algorithm

1. Begin with $n$ observations and a measure (such as Euclidean distance) of all the $n(n − 1)/2$ pairwise dissimilarities. Treat each 2 observation as its own cluster.

2. For $i=n,n−1,...,2$

    a) Examine all pairwise inter-cluster dissimilarities among the $i$ clusters and identify the **pair of clusters that are least dissimilar** (that is, most similar). Fuse these two clusters. The dissimilarity between these two clusters indicates the height in the dendrogram at which the fusion should be placed.

    b) Compute the new pairwise inter-cluster dissimilarities among the $i−1$ remaining clusters.

### The Linkage Function

We have a concept of the dissimilarity between pairs of observations, but how do we define the dissimilarity between two clusters if one or both of the clusters contains multiple observations?

The concept of dissimilarity between a pair of observations needs to be extended to a pair of groups of observations. This extension is achieved by developing the notion of **linkage**, which defines the dissimilarity between two groups of observations.

### Linkages

The four most common types of linkage are:

1. **Complete**: Maximal intercluster dissimilarity. Compute all pairwise dissimilarities between the observations in cluster A and the observations in cluster B, and record the largest of these dissimilarities.
2. **Single**: Minimal intercluster dissimilarity. Compute all pairwise dissimilarities between the observations in cluster A and the observations in cluster B, and record the smallest of these dissimilarities.
3. **Average**: Mean intercluster dissimilarity. Compute all pairwise dissimilarities between the observations in cluster A and the observations in cluster B, and record the average of these dissimilarities.
4. **Centroid**: Dissimilarity between the centroid for cluster A (a mean vector of length p) and the centroid for cluster B. Centroid linkage can result in undesirable inversions.

Average, complete, and single linkage are most popular among statisticians. Average and complete linkage are generally preferred over single linkage, as they tend to yield more balanced dendrograms. Centroid linkage is often used in genomics, but suffers from a major drawback in that an inversion can occur, whereby two clusters are fused at a height below either of the individual clusters in the dendrogram. This can lead to difficulties in visualization as well as in interpretation of the dendrogram.

```python
# Init
linkages = [hierarchy.complete(X), hierarchy.average(X), hierarchy.single(X)]
titles = ['Complete Linkage', 'Average Linkage', 'Single Linkage']
```

### Plotting

```python
make_new_figure_4(linkages, titles)
```

For this data, both *complete* and *average* linkage generally separates the observations into their correct groups.

## Gaussian Mixture Models

Clustering methods such as hierarchical clustering and K-means are based on heuristics and rely primarily on finding clusters whose members are close to one another, as measured directly with the data (no probability model involved).

*Gaussian Mixture Models* assume that the data was generated by multiple multivariate gaussian distributions. The objective of the algorithm is to recover these latent distributions.

The advantages with respect to K-means are

- a structural interpretaion of the parameters
- automatically generates class probabilities
- can generate clusters of observations that are not necessarily close to each other

### Algorithm

1. Randomly assign a number, from $1$ to $K$, to each of the observations. These serve as initial cluster assignments for the observations.

2. Iterate until the cluster assignments stop changing:

    a) For each of the $K$ clusters, compute its mean and variance. The main difference with K-means is that we also compute the variance matrix.

    b) Assign each observation to its most likely cluster.

### Dataset

Let's use the same data we have used for k-means, for a direct comparison.

```python
make_new_figure_1(X)
```

### Step 1: random assignement

Let's also use the same random assignment of the K-means algorithm.

```python
make_new_figure_2(X, clusters0)
```

### Step 2: compute distirbutions

What are the new distributions?

```python
# Compute new centroids
def compute_distributions(X, clusters):
    K = len(np.unique(clusters))
    distr = []
    for k in range(K):
        if sum(clusters==k)>0:
            distr += [multivariate_normal(np.mean(X[clusters==k,:], axis=0), np.cov(X[clusters==k,:].T))]
        else:
            distr += [multivariate_normal(np.mean(X, axis=0), np.cov(X.T))]
    return distr
```

```python
# Print
distr0 = compute_distributions(X, clusters0)
print("Mean of the first distribution: \n", distr0[0].mean)
print("\nVariance of the first distribution: \n", distr0[0].cov)
```

    Mean of the first distribution:
     [ 1.54179703 -1.65922379]

    Variance of the first distribution:
     [[ 3.7160256  -2.27290036]
     [-2.27290036  4.67223237]]

### Plotting the distributions

Let's add the distributions to the graph.

```python
# Plot
plot_assignment_gmm(X, clusters0, distr0, i=0, logL=0.0)
```

### Likelihood

The main difference with respect with K-means is that we can now compute the probability that each observation belongs to each cluster. This is the probability that each observation was generated by one of the two bi-variate normal distributions. These probabilities are called **likelihoods**.

```python
# Print first 5 likelihoods
pdfs0 = np.stack([d.pdf(X) for d in distr0], axis=1)
pdfs0[:5]
```

    array([[0.03700522, 0.05086876],
           [0.00932081, 0.02117353],
           [0.04092453, 0.04480732],
           [0.00717854, 0.00835799],
           [0.01169199, 0.01847373]])

### Step 3: assign data to clusters

Now we can assign the data to the clusters, via maximum likelihood.

```python
# Assign X to clusters
def assign_to_cluster_gmm(X, distr):
    pdfs = np.stack([d.pdf(X) for d in distr], axis=1)
    clusters = np.argmax(pdfs, axis=1)
    log_likelihood = 0
    for k, pdf in enumerate(pdfs):
        log_likelihood += np.log(pdf[clusters[k]])
    return clusters, log_likelihood
```

```python
# Get cluster assignment
clusters1, logL1 = assign_to_cluster_gmm(X, distr0)
```

### Plotting assigned data

```python
# Compute new distributions
distr1 = compute_distributions(X, clusters1)
```

```python
# Plot
plot_assignment_gmm(X, clusters1, distr1, 1, logL1);
```

### Expectation - Maximization

The two steps we have just seen, are part of a broader family of algorithms to maximize likelihoods called **expectation**-**maximization** algorithms.

In the expectation step, we computed the expectation of the parameters, given the current cluster assignment.

In the maximization step, we assigned observations to the cluster that maximized the likelihood of the single observation.

The alternative, and more computationally intensive procedure, would have been to specify a global likelihood function and find the mean and variance paramenters of the two normal distributions that maximized those likelihoods.

### Full Algorithm

We can now deploy the full algorithm.

```python
def gmm_manual(X, K):

    # Init
    i = 0
    logL0 = 1e4
    logL1 = 1e5
    clusters = np.random.randint(K,size=(np.size(X,0)))

    # Iterate until convergence
    while np.abs(logL0-logL1) > 1e-10:
        logL1 = logL0
        distr = compute_distributions(X, clusters)
        clusters, logL0 = assign_to_cluster_gmm(X, distr)
        plot_assignment_gmm(X, clusters, distr, i, logL0)
        i+=1
```

### Plotting k-means clustering

```python
# Test
gmm_manual(X, K)
```

In this case, GMM does a very poor job identifying the original clusters.

### Overlapping Clusters

Let's now try with a different dataset, where the data is drawn from two overlapping bi-variate gaussian distributions, forming a cross.

```python
# Simulate data
X = np.random.randn(50,2)
X[0:25, :] = np.random.multivariate_normal([0,0], [[50,0],[0,1]], size=25)
X[25:, :] = np.random.multivariate_normal([0,0], [[1,0],[0,50]], size=25)
```

```python
make_new_figure_1(X)
```

### GMM with overlapping distributions

```python
# GMM
gmm_manual(X, K)
```

As we can see, GMM is able to correctly recover the original clusters.

### K-means with overlapping distributions

```python
# K-means
kmeans_manual(X, K)
```

K-means generates completely different clusters.


```python
import sympy as sp
from sympy.matrices import Matrix
#import numpy as np

# pretty print numpy matrices
#np.set_printoptions(formatter={'float': '{: 0.5f}'.format})
```

```python
th1, th2, th3, th4, th5, th6 = sp.symbols('th1:7')

# the DH paramters for KUKA robot as per written document
# alpha, a, d, theta (adjustmet)
# the angles we will use to control the robot will be added to the theta paramters
DH = [[0       , 0     , 0.75 , th1          ],
      [-sp.pi/2, 0.35  , 0    , th2 - sp.pi/2],
      [0       , 1.25  , 0    , th3          ],
      [-sp.pi/2, -0.054, 1.5  , th4          ],
      [sp.pi/2 , 0     , 0    , th5          ],
      [-sp.pi/2, 0     , 0    , th6          ],
      [0       , 0     , 0.303, 0            ]]
```

```python
# a help function that builds a transformation matrix given the 4 DH paramters for that joint
def LinkTransform(params):
    # params is a list of length 4: alpha, a, d, theta in this order
    alpha = params[0]
    a = params[1]
    d = params[2]
    theta = params[3]
    # returns a Matrix of transformation for the given DH paramters
    return Matrix([[sp.cos(theta)              , -sp.sin(theta)             , 0             , a],
                   [sp.sin(theta)*sp.cos(alpha), sp.cos(theta)*sp.cos(alpha), -sp.sin(alpha), -sp.sin(alpha)*d],
                   [sp.sin(theta)*sp.sin(alpha), sp.cos(theta)*sp.sin(alpha), sp.cos(alpha) , sp.cos(alpha)*d],
                   [0                          , 0                          , 0             , 1]])
```

```python
T34 = LinkTransform(DH[3])
T45 = LinkTransform(DH[4])
T56 = LinkTransform(DH[5])
T36 = sp.simplify(T34 * T45 * T56)
print(T36[0:3,0:3])
```

    Matrix([
    [-sin(th4)*sin(th6) + cos(th4)*cos(th5)*cos(th6), -sin(th4)*cos(th6) - sin(th6)*cos(th4)*cos(th5), -sin(th5)*cos(th4)],
    [                              sin(th5)*cos(th6),                              -sin(th5)*sin(th6),           cos(th5)],
    [-sin(th4)*cos(th5)*cos(th6) - sin(th6)*cos(th4),  sin(th4)*sin(th6)*cos(th5) - cos(th4)*cos(th6),  sin(th4)*sin(th5)]])

```python
# builds a transformation matrix for the orientation adjustment of the gripper
# so that we are consistent with the URDF representaiton of the gripper
def GripperAdjust(r_z = sp.pi, r_y = -sp.pi/2):
    R_z = Matrix([[sp.cos(r_z), -sp.sin(r_z), 0, 0],
                  [sp.sin(r_z), sp.cos(r_z) , 0, 0],
                  [0          , 0           , 1, 0],
                  [0          , 0           , 0, 1]])
    R_y = Matrix([[sp.cos(r_y) , 0          , sp.sin(r_y), 0],
                  [0           , 1          , 0          , 0],
                  [-sp.sin(r_y), 0          , sp.cos(r_y), 0],
                  [0           , 0          , 0          , 1]])
    return sp.simplify(R_z * R_y)
```

```python
T01 = LinkTransform(DH[0])
print('T01 = '+str(sp.simplify(T01))+'\n')
T12 = LinkTransform(DH[1])
print('T12 = '+str(sp.simplify(T12))+'\n')
T23 = LinkTransform(DH[2])
print('T23 = '+str(sp.simplify(T23))+'\n')
T34 = LinkTransform(DH[3])
print('T34 = '+str(sp.simplify(T34))+'\n')
T45 = LinkTransform(DH[4])
print('T45 = '+str(sp.simplify(T45))+'\n')
T56 = LinkTransform(DH[5])
print('T56 = '+str(sp.simplify(T56))+'\n')
T6G = LinkTransform(DH[6])
print('T6G = '+str(sp.simplify(T6G))+'\n')
print('TGA = '+str(GripperAdjust()))
```

    T01 = Matrix([
    [cos(th1), -sin(th1), 0,    0],
    [sin(th1),  cos(th1), 0,    0],
    [       0,         0, 1, 0.75],
    [       0,         0, 0,    1]])

    T12 = Matrix([
    [sin(th2),  cos(th2), 0, 0.35],
    [       0,         0, 1,    0],
    [cos(th2), -sin(th2), 0,    0],
    [       0,         0, 0,    1]])

    T23 = Matrix([
    [cos(th3), -sin(th3), 0, 1.25],
    [sin(th3),  cos(th3), 0,    0],
    [       0,         0, 1,    0],
    [       0,         0, 0,    1]])

    T34 = Matrix([
    [ cos(th4), -sin(th4), 0, -0.054],
    [        0,         0, 1,    1.5],
    [-sin(th4), -cos(th4), 0,      0],
    [        0,         0, 0,      1]])

    T45 = Matrix([
    [cos(th5), -sin(th5),  0, 0],
    [       0,         0, -1, 0],
    [sin(th5),  cos(th5),  0, 0],
    [       0,         0,  0, 1]])

    T56 = Matrix([
    [ cos(th6), -sin(th6), 0, 0],
    [        0,         0, 1, 0],
    [-sin(th6), -cos(th6), 0, 0],
    [        0,         0, 0, 1]])

    T6G = Matrix([
    [1, 0, 0,     0],
    [0, 1, 0,     0],
    [0, 0, 1, 0.303],
    [0, 0, 0,     1]])

    TGA = Matrix([
    [0,  0, 1, 0],
    [0, -1, 0, 0],
    [1,  0, 0, 0],
    [0,  0, 0, 1]])

```python
T0G = sp.simplify(T01*T12*T23*T34*T45*T56*T6G)
print('T0G unajusted = '+str(T0G))
```

    T0G unajusted = Matrix([
    [((sin(th1)*sin(th4) + sin(th2 + th3)*cos(th1)*cos(th4))*cos(th5) + sin(th5)*cos(th1)*cos(th2 + th3))*cos(th6) - (-sin(th1)*cos(th4) + sin(th4)*sin(th2 + th3)*cos(th1))*sin(th6), -((sin(th1)*sin(th4) + sin(th2 + th3)*cos(th1)*cos(th4))*cos(th5) + sin(th5)*cos(th1)*cos(th2 + th3))*sin(th6) + (sin(th1)*cos(th4) - sin(th4)*sin(th2 + th3)*cos(th1))*cos(th6), -(sin(th1)*sin(th4) + sin(th2 + th3)*cos(th1)*cos(th4))*sin(th5) + cos(th1)*cos(th5)*cos(th2 + th3), -0.303*sin(th1)*sin(th4)*sin(th5) + 1.25*sin(th2)*cos(th1) - 0.303*sin(th5)*sin(th2 + th3)*cos(th1)*cos(th4) - 0.054*sin(th2 + th3)*cos(th1) + 0.303*cos(th1)*cos(th5)*cos(th2 + th3) + 1.5*cos(th1)*cos(th2 + th3) + 0.35*cos(th1)],
    [ ((sin(th1)*sin(th2 + th3)*cos(th4) - sin(th4)*cos(th1))*cos(th5) + sin(th1)*sin(th5)*cos(th2 + th3))*cos(th6) - (sin(th1)*sin(th4)*sin(th2 + th3) + cos(th1)*cos(th4))*sin(th6), -((sin(th1)*sin(th2 + th3)*cos(th4) - sin(th4)*cos(th1))*cos(th5) + sin(th1)*sin(th5)*cos(th2 + th3))*sin(th6) - (sin(th1)*sin(th4)*sin(th2 + th3) + cos(th1)*cos(th4))*cos(th6), -(sin(th1)*sin(th2 + th3)*cos(th4) - sin(th4)*cos(th1))*sin(th5) + sin(th1)*cos(th5)*cos(th2 + th3),  1.25*sin(th1)*sin(th2) - 0.303*sin(th1)*sin(th5)*sin(th2 + th3)*cos(th4) - 0.054*sin(th1)*sin(th2 + th3) + 0.303*sin(th1)*cos(th5)*cos(th2 + th3) + 1.5*sin(th1)*cos(th2 + th3) + 0.35*sin(th1) + 0.303*sin(th4)*sin(th5)*cos(th1)],
    [                                                                       -(sin(th5)*sin(th2 + th3) - cos(th4)*cos(th5)*cos(th2 + th3))*cos(th6) - sin(th4)*sin(th6)*cos(th2 + th3),                                                                         (sin(th5)*sin(th2 + th3) - cos(th4)*cos(th5)*cos(th2 + th3))*sin(th6) - sin(th4)*cos(th6)*cos(th2 + th3),                                         -sin(th5)*cos(th4)*cos(th2 + th3) - sin(th2 + th3)*cos(th5),                                                                                -0.303*sin(th5)*cos(th4)*cos(th2 + th3) - 0.303*sin(th2 + th3)*cos(th5) - 1.5*sin(th2 + th3) + 1.25*cos(th2) - 0.054*cos(th2 + th3) + 0.75],
    [                                                                                0,                                                                                0,                                                                                0,                                                                                1]])

```python
T0GA = sp.simplify(T0G*GripperAdjust())
print('T0G ajusted = '+str(T0GA))
```

    T0G ajusted = Matrix([
    [-(sin(th1)*sin(th4) + sin(th2 + th3)*cos(th1)*cos(th4))*sin(th5) + cos(th1)*cos(th5)*cos(th2 + th3), ((sin(th1)*sin(th4) + sin(th2 + th3)*cos(th1)*cos(th4))*cos(th5) + sin(th5)*cos(th1)*cos(th2 + th3))*sin(th6) - (sin(th1)*cos(th4) - sin(th4)*sin(th2 + th3)*cos(th1))*cos(th6), ((sin(th1)*sin(th4) + sin(th2 + th3)*cos(th1)*cos(th4))*cos(th5) + sin(th5)*cos(th1)*cos(th2 + th3))*cos(th6) + (sin(th1)*cos(th4) - sin(th4)*sin(th2 + th3)*cos(th1))*sin(th6), -0.303*sin(th1)*sin(th4)*sin(th5) + 1.25*sin(th2)*cos(th1) - 0.303*sin(th5)*sin(th2 + th3)*cos(th1)*cos(th4) - 0.054*sin(th2 + th3)*cos(th1) + 0.303*cos(th1)*cos(th5)*cos(th2 + th3) + 1.5*cos(th1)*cos(th2 + th3) + 0.35*cos(th1)],
    [-(sin(th1)*sin(th2 + th3)*cos(th4) - sin(th4)*cos(th1))*sin(th5) + sin(th1)*cos(th5)*cos(th2 + th3), ((sin(th1)*sin(th2 + th3)*cos(th4) - sin(th4)*cos(th1))*cos(th5) + sin(th1)*sin(th5)*cos(th2 + th3))*sin(th6) + (sin(th1)*sin(th4)*sin(th2 + th3) + cos(th1)*cos(th4))*cos(th6), ((sin(th1)*sin(th2 + th3)*cos(th4) - sin(th4)*cos(th1))*cos(th5) + sin(th1)*sin(th5)*cos(th2 + th3))*cos(th6) - (sin(th1)*sin(th4)*sin(th2 + th3) + cos(th1)*cos(th4))*sin(th6),  1.25*sin(th1)*sin(th2) - 0.303*sin(th1)*sin(th5)*sin(th2 + th3)*cos(th4) - 0.054*sin(th1)*sin(th2 + th3) + 0.303*sin(th1)*cos(th5)*cos(th2 + th3) + 1.5*sin(th1)*cos(th2 + th3) + 0.35*sin(th1) + 0.303*sin(th4)*sin(th5)*cos(th1)],
    [                                        -sin(th5)*cos(th4)*cos(th2 + th3) - sin(th2 + th3)*cos(th5),                                                                       -(sin(th5)*sin(th2 + th3) - cos(th4)*cos(th5)*cos(th2 + th3))*sin(th6) + sin(th4)*cos(th6)*cos(th2 + th3),                                                                       -(sin(th5)*sin(th2 + th3) - cos(th4)*cos(th5)*cos(th2 + th3))*cos(th6) - sin(th4)*sin(th6)*cos(th2 + th3),                                                                                -0.303*sin(th5)*cos(th4)*cos(th2 + th3) - 0.303*sin(th2 + th3)*cos(th5) - 1.5*sin(th2 + th3) + 1.25*cos(th2) - 0.054*cos(th2 + th3) + 0.75],
    [                                                                                0,                                                                                0,                                                                                0,                                                                                1]])

```python
# builds the transformation matrices based on the DH and the theta paramters
# it resurns the final transformation matrix
def ChainLinkTransform(DH, show = False):
    for i in range(len(DH)):
        T = LinkTransform(DH[i])
        if i == 0:
            result = T
        else:
            result = sp.simplify(result * T)

        if show:
            print("T%d_%d = " % (i, i+1))
            print(T)
            print("T0_%d = " % (i+1))
            print(result)

    return sp.simplify(result * GripperAdust())
```

```python
# extracts the position and orientation from the quaternion matrix
def OrientationFromQuaternion(Q):
    pos = Q[:,3].T
    orient = [sp.atan2(Q[2,1], Q[2,2]),
              sp.atan2(-Q[2,0], sp.sqrt(Q[0,0]**2 + Q[1,0]**2)),
              sp.atan2(Q[1,0], Q[0,0])]

    return pos[0:3], orient
```

```python
def CalculateEffector(sym, thetas):
    res = sym.evalf(subs={th1: thetas[0], th2: thetas[1], th3: thetas[2],
                          th4: thetas[3], th5: thetas[4], th6: thetas[5]})
    pos, orient = OrientationFromQuaternion(res)
    print("pos    = "+str(pos))
    print("orient = "+str(orient))
```

```python
symtransf = ChainLinkTransform(DH)
```

```python
th = [0, 0, 0, 0, 0, 0]
CalculateEffector(symtransf, th)
```

ROS Reported:

`Trans = [2.153, 0.000, 1.947]
Rot = [0.000, 0.000, 0.000]`

```python
th = [0.99, 0, 0, 0, 0, 0, 0]
CalculateEffector(symtransf, th)
```

ROS Reported:

Trans = [1.173, 1.805, 1.947]

Rot = [0.000, 0.000, 0.994]

```python
th = [0.99, 0, 0, 0, 0, 0.72, 0]
CalculateEffector(DH, th)
```

ROS Reported:

Trans = [1.173, 1.805, 1.947]

Rot = [0.720, 0.0, 0.994]

```python
th = [0.99, 0, 0, 0, 0.49, 0.72, 0]
CalculateEffector(DH, th)
```

ROS Reported:

Trans = [1.154, 1.775, 1.804]

Rot   = [0.720, 0.489, 0.994]

Term 1/Project 2 - Robotic Arm - Pick and Place/submission/forward_kinematics-sympy.ipynb


# Basic stats using `Scipy`
In this example we will go over how to draw samples from various built in probability distributions and define your own custom distributions.

## Packages being used
+ `scipy`: has all the stats stuff
+ `numpy`: has all the array stuff

## Relevant documentation
+ `scipy.stats`: http://docs.scipy.org/doc/scipy/reference/tutorial/stats.html, http://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.rv_continuous.html#scipy.stats.rv_continuous, http://docs.scipy.org/doc/scipy/reference/stats.html#module-scipy.stats

```python
import numpy as np
import scipy.stats as st
# some special functions we will make use of later on
from scipy.special import erfc
from matplotlib import pyplot as plt
from astropy.visualization import hist
import mpl_style
%matplotlib inline
plt.style.use(mpl_style.style1)
```

There are many probability distributions that are already available in `scipy`: http://docs.scipy.org/doc/scipy/reference/stats.html#module-scipy.stats.  These classes allow for the evaluations of PDFs, CDFs, PPFs, moments, random draws, and fitting.  As an example lets take a look at the normal distribution.

```python
norm = st.norm(loc=0, scale=1)
x = np.linspace(-5, 5, 1000)

plt.figure(1, figsize=(8, 10))
plt.subplot2grid((2, 2), (0, 0))
plt.plot(x, norm.pdf(x))
plt.xlabel('x')
plt.ylabel('PDF(x)')
plt.xlim(-5, 5)

plt.subplot2grid((2, 2), (0, 1))
plt.plot(x, norm.cdf(x))
plt.xlabel('x')
plt.ylabel('CDF(x)')
plt.xlim(-5, 5)

plt.subplot2grid((2, 2), (1, 0))
sample_norm = norm.rvs(size=100000)
hist(sample_norm, bins='knuth', histtype='step', lw=1.5, density=True)
plt.xlabel('x')
plt.ylabel('Random Sample')
plt.tight_layout()
```

You can calculate moments and fit data:

```python
for i in range(4):
    print('moment {0}: {1}'.format(i+1, norm.moment(i+1)))

print('best fit: {0}'.format(st.norm.fit(sample_norm)))
```

    moment 1: 0.0
    moment 2: 1.0
    moment 3: 0.0
    moment 4: 3.0
    best fit: (0.0023590267327017016, 1.0016772649523067)

# Custom probability distributions
Sometimes you need to use obscure PDFs that are not already in `scipy` or `astropy`.  When this is the case you can make your own subclass of `st.rv_continuous` and overwrite the `_pdf` or `_cdf` methods.  This new sub class will act exactly like the built in distributions.

The methods you can override in the subclass are:

+ \_rvs: create a random sample drawn from the distribution
+ \_pdf: calculate the PDF at any point
+ \_cdf: calculate the CDF at any point
+ \_sf: survival function, a.k.a. 1-CDF(x)
+ \_ppf: percent point function, a.k.a. inverse CDF
+ \_isf: inverse survival function
+ \_stats: function that calculates the first 4 moments
+ \_munp: function that calculates the nth moment
+ \_entropy: differential entropy
+ \_argcheck: function to check the input arguments are valid (e.g. var>0)

You should override any method you have analytic functions for, otherwise (typically slow) numerical integration, differentiation, and function inversion are used to transform the ones that are specified.

## The exponentially modified Gaussian distribution
As and example lets create a class for the EMG distribution (https://en.wikipedia.org/wiki/Exponentially_modified_Gaussian_distribution).  This is the distributions resulting from the sum of a Gaussian random variable and an exponential random variable.  The PDF and CDF are:

\begin{align}
f(x;\mu,\sigma, \lambda) & = \frac{\lambda}{2} \exp{\left( \frac{\lambda}{2} \left[ 2\mu+\lambda\sigma^{2}-2x \right] \right)} \operatorname{erfc}{\left( \frac{\mu + \lambda\sigma^{2}-x}{\sigma\sqrt{2}} \right)} \\
F(x; \mu, \sigma, \lambda) & = \Phi(u, 0, v) - \Phi(u, v^2, v) \exp{\left( -u + \frac{v^2}{2} \right)} \\
\Phi(x, a, b) & = \frac{1}{2} \left[ 1 + \operatorname{erf}{\left( \frac{x - a}{b\sqrt{2}} \right)} \right] \\
u & = \lambda(x - \mu) \\
v & = \lambda\sigma
\end{align}

```python
# create a generating class
class EMG_gen1(st.rv_continuous):
    def _pdf(self, x, mu, sig, lam):
        u = 0.5 * lam * (2 * mu + lam * sig**2 - 2 * x)
        v = (mu + lam * sig**2 - x)/(sig * np.sqrt(2))
        return 0.5 * lam * np.exp(u) * erfc(v)
    def _cdf(self, x, mu, sig, lam):
        u = lam * (x - mu)
        v = lam * sig
        phi1 = st.norm.cdf(u, loc=0, scale=v)
        phi2 = st.norm.cdf(u, loc=v**2, scale=v)
        return phi1 - phi2 * np.exp(-u + 0.5 * v**2)
    def _stats(self, mu, sig, lam):
        # reutrn the mean, variance, skewness, and kurtosis
        mean = mu + 1 / lam
        var = sig**2 + 1 / lam**2
        sl = sig * lam
        u = 1 + 1 / sl**2
        skew = (2 / sl**3) * u**(-3 / 2)
        v = 3 * (1 + 2 / sl**2 + 3 / sl**4) / u**2
        kurt = v - 3
        return mean, var, skew, kurt
    def _argcheck(self, mu, sig, lam):
        return np.isfinite(mu) and (sig > 0) and (lam > 0)

class EMG_gen2(EMG_gen1):
    def _ppf(self, q, mu, sig, lam):
        # use linear interpolation to solve this faster (not exact, but much faster than the built in method)
        # pick range large enough to fit the full cdf
        var = sig**2 + 1 / lam**2
        x = np.arange(mu - 50 * np.sqrt(var), mu + 50 * np.sqrt(var), 0.01)
        y = self.cdf(x, mu, sig, lam)
        return np.interp(q, y, x)

class EMG_gen3(EMG_gen1):
    def _rvs(self, mu, sig, lam):
        # redefine the random sampler to sample based on a normal and exp dist
        return st.norm.rvs(loc=mu, scale=sig, size=self._size) + st.expon.rvs(loc=0, scale=1/lam, size=self._size)

# use generator to make the new class
EMG1 = EMG_gen1(name='EMG1')
EMG2 = EMG_gen2(name='EMG2')
EMG3 = EMG_gen3(name='EMG3')
```

Lets look at how long it takes to create readom samples for each of these version of the EMG:

```python
%time EMG1.rvs(0, 1, 0.5, size=1000)
print('=========')
%time EMG2.rvs(0, 1, 0.5, size=1000)
print('=========')
%time EMG3.rvs(0, 1, 0.5, size=1000)
print('=========')
```

    CPU times: user 3.88 s, sys: 7.93 ms, total: 3.88 s
    Wall time: 3.87 s
    =========
    CPU times: user 3.38 ms, sys: 0 ns, total: 3.38 ms
    Wall time: 3.34 ms
    =========
    CPU times: user 973 µs, sys: 1.62 ms, total: 2.6 ms
    Wall time: 20 ms
    =========

    /mnt/lustre/shared_python_environment/DataLanguages/lib/python3.8/site-packages/scipy/stats/_distn_infrastructure.py:1083: VisibleDeprecationWarning: The signature of <bound method EMG_gen3._rvs of <__main__.EMG_gen3 object at 0x7f779ee89760>> does not contain a "size" keyword.  Such signatures are deprecated.
      warnings.warn(

As you can see, the numerical inversion of the CDF is very slow, the approximation to the inversion is much faster, and defining `_rvs` in terms of the `normal` and `exp` distributions is the fastest.

Lets take a look at the results for `EMG3`:

```python
dist = EMG3(0, 1, 0.5)
x = np.linspace(-5, 20, 1000)

plt.figure(2, figsize=(8, 10))
plt.subplot2grid((2, 2), (0, 0))
plt.plot(x, dist.pdf(x))
plt.xlabel('x')
plt.ylabel('PDF(x)')

plt.subplot2grid((2, 2), (0, 1))
plt.plot(x, dist.cdf(x))
plt.xlabel('x')
plt.ylabel('CDF(x)')

plt.subplot2grid((2, 2), (1, 0))
sample_emg = dist.rvs(size=10000)
hist(sample_emg, bins='knuth', histtype='step', lw=1.5, density=True)
plt.xlabel('x')
plt.ylabel('Random Sample')
plt.tight_layout()
```

As with the built in functions we can calculate moments and do fits to data.  **Note** Since we are not using the built in `loc` and `scale` params they are fixed to 0 and 1 in the fit below.

```python
for i in range(4):
    print('moment {0}: {1}'.format(i+1, dist.moment(i+1)))

print('best fit: {0}'.format(EMG3.fit(sample_emg, floc=0, fscale=1)))
```

    moment 1: 2.0
    moment 2: 9.0
    moment 3: 54.0
    moment 4: 435.0
    best fit: (-0.02693863999005528, 1.0210387739316098, 0.49836860851157133, 0, 1)

For reference here is how `scipy` defines this distribution (found under the name `exponnorm`):

```python
import scipy.stats._continuous_distns as cd
np.source(cd.exponnorm_gen)
```

    In file: /mnt/lustre/shared_python_environment/DataLanguages/lib/python3.8/site-packages/scipy/stats/_continuous_distns.py

    class exponnorm_gen(rv_continuous):
        r"""An exponentially modified Normal continuous random variable.

        Also known as the exponentially modified Gaussian distribution [1]_.

        %(before_notes)s

        Notes
        -----
        The probability density function for `exponnorm` is:

        .. math::

            f(x, K) = \frac{1}{2K} \exp\left(\frac{1}{2 K^2} - x / K \right)
                      \text{erfc}\left(-\frac{x - 1/K}{\sqrt{2}}\right)

        where :math:`x` is a real number and :math:`K > 0`.

        It can be thought of as the sum of a standard normal random variable
        and an independent exponentially distributed random variable with rate
        ``1/K``.

        %(after_notes)s

        An alternative parameterization of this distribution (for example, in
        the Wikpedia article [1]_) involves three parameters, :math:`\mu`,
        :math:`\lambda` and :math:`\sigma`.

        In the present parameterization this corresponds to having ``loc`` and
        ``scale`` equal to :math:`\mu` and :math:`\sigma`, respectively, and
        shape parameter :math:`K = 1/(\sigma\lambda)`.

        .. versionadded:: 0.16.0

        References
        ----------
        .. [1] Exponentially modified Gaussian distribution, Wikipedia,
               https://en.wikipedia.org/wiki/Exponentially_modified_Gaussian_distribution

        %(example)s

        """
        def _rvs(self, K, size=None, random_state=None):
            expval = random_state.standard_exponential(size) * K
            gval = random_state.standard_normal(size)
            return expval + gval

        def _pdf(self, x, K):
            return np.exp(self._logpdf(x, K))

        def _logpdf(self, x, K):
            invK = 1.0 / K
            exparg = invK * (0.5 * invK - x)
            return exparg + _norm_logcdf(x - invK) - np.log(K)

        def _cdf(self, x, K):
            invK = 1.0 / K
            expval = invK * (0.5 * invK - x)
            logprod = expval + _norm_logcdf(x - invK)
            return _norm_cdf(x) - np.exp(logprod)

        def _sf(self, x, K):
            invK = 1.0 / K
            expval = invK * (0.5 * invK - x)
            logprod = expval + _norm_logcdf(x - invK)
            return _norm_cdf(-x) + np.exp(logprod)

        def _stats(self, K):
            K2 = K * K
            opK2 = 1.0 + K2
            skw = 2 * K**3 * opK2**(-1.5)
            krt = 6.0 * K2 * K2 * opK2**(-2)
            return K, opK2, skw, krt

```python
%time st.exponnorm.rvs(0.5, size=1000)
print('=========')
```

    CPU times: user 868 µs, sys: 0 ns, total: 868 µs
    Wall time: 576 µs
    =========

```python

```


# Homework 14: Nonlinear Equations

### Problem 1
Use fsolve to find the roots of the polynomial $f(x) = 2x^2 + 3x - 10$.

```python
import numpy as np
from scipy.optimize import fsolve
```

```python

```

### Problem 2

Use fsolve to find the solution of the following two equations:
\begin{align}
f(x,y) &= 2x^{2/3}+y^{2/3}-9^{1/3} \\
g(x,y) &= \frac{x^2}{4} + \sqrt{y} - 1.
\end{align}
Use an initial guess of $x_0=1$, $y_0$ = 1.

```python

```

### Problem 3

```python
# import or install wget
try:
    import wget
except:
    try:
        from pip import main as pipmain
    except:
        from pip._internal import main as pipmain
    pipmain(['install','wget'])
    import wget

# retrieve thermoData.yaml
url = 'https://apmonitor.com/che263/uploads/Main/thermoData.yaml'
filename = wget.download(url)
print('')
print('Retrieved thermoData.yaml')
```

    100% [..............................................................................] 14985 / 14985
    Retrieved thermoData.yaml

Compute the adiabatic flame temperature for a stoichiometric methane-air flame. The code is given below. There is a thermo class that is modified from your last homework. Also, you'll need thermoData.yaml again. Then there is a function to define. Fill in the blanks as indicated. You should also read all of the code given below and make sure you understand it.

**Equation Summary:**
* Your function (started for you below) is: ```f_flame(Ta) = 0```.
* That is, $f_{flame}(T_a) = 0 = H_r(T_r) - H_p(T_a) = 0$.
    * $T_a$ is the unknown.
    * $T_r = 300\,K$
    * $H_r(T_r) = y_{CH4}h_{CH4}(T_r) + y_{O2}h_{O2}(T_r) + y_{N2}h_{N2}(T_r)$.
    * $H_p(T_a) = y_{CO2}h_{CO2}(T_a) + y_{H2O}h_{H2O}(T_a) + y_{N2}h_{N2}(T_a)$.
    * $y_i = m_i/m_t$.
        * $m_i = n_iM_i$.
        * $n_i$ and $M_i$ are given.
        * $m_t = \sum_im_i$.
        * **Do these separately for reactants and products** That is: $m_t = m_{O2}+m_{N2}+m_{CH4}$ for the reactants. (Also $m_t$ is the same for products since mass is conserved.)
    * $h_i$ is computed using the thermo class. So, if ```t_CO2``` is my thermo class object for $CO_2$, then ```h_CO2=t_CO2.h_mass(T)```.

**Description:**
* We have a chemical reaction:
    * $CH_4 + 2O_2 + 7.52N_2 \rightarrow CO_2 + 2H_2O$ + 7.52$N_2$.
* You can think of the burning as potential energy stored in the reactant bonds being released as kinetic energy in the products so the product temperature is higher.
* Adiabatic means there is no enthalpy loss. You can think of enthalpy as energy. This means the products have the same enthalpy as the reactants. And this is just a statement that energy is conserved, like mass is.
* The idea is to take a known reactant temperature, find the reactant enthalpy (which is an easy explicit equation you can calculate directly), then set the product enthalpy equal to the reactant enthalpy and find the corresponding product temperature (which is a harder nonlinear solve).
    * $T_r\rightarrow h_r = h_p \rightarrow T_p$.
* The reactants start at room temperature, $T=300\,K$, so we can compute their enthalpy.
    * We know the moles of reactants: $n_{ch4}=1$, $n_{O2}=2$, $n_{N2}=7.52$.
    * So, we can compute the corresponding masses using the molecular weights.
    * Then we sum the masses of each species to get the total mass, and compute the mass fractions.
    * Then we can compute the enthalpy as $h=\sum_iy_ih_i$. That is, the total enthalpy is the sum of the enthalpy per unit mass of each species times the mass fraction of each species.
        * For reactants we have $h_r = y_{CH4}h_{CH4}+y_{O2}h_{O2}+y_{N2}h_{N2}$, where $h_i$ are evaluated using the class function h_mass(T), and T=300 for reactants.
* Now, $h_p=h_r$. For products, we have $h_p = y_{CO2}h_{CO2}+y_{H2O}h_{H2O}+y_{N2}h_{N2}$, where we evaluate the class function h_mass(Tp), where Tp is the product temperature we are trying to compute.
    * Solving for $T_p$ amounts to solving $f(T_p)=0$, where $$f(T_p) = h_p - y_{CO2}h_{CO2}(T_p)+y_{H2O}h_{H2O}(T_p)+y_{N2}h_{N2}(T_p)$$.

```python
import numpy as np
from scipy.optimize import fsolve
import yaml

class thermo:
    def __init__(self, species, MW) :
        """
        species: input string name of species in thermoData.yaml
        M: input (species molecular weight, kg/kmol)
        """
        self.Rgas = 8314.46      # J/kmol*K
        self.M    = MW
        with open("thermoData.yaml") as yfile :
           yfile = yaml.load(yfile)
        self.a_lo = yfile[species]["a_lo"]
        self.a_hi = yfile[species]["a_hi"]
        self.T_lo = 300.
        self.T_mid = 1000.
        self.T_hi = 3000.

    #--------------------------------------------------------
    def h_mole(self,T) :
        """
        return enthalpy in units of J/kmol
        T: input (K)
        """
        if T<=self.T_mid and T>=self.T_lo :
            a = self.a_lo
        elif T>self.T_mid and T<=self.T_hi :
            a = self.a_hi
        else :
            print ("ERROR: temperature is out of range")
        hrt = a[0] + a[1]/2.0*T + a[2]/3.0*T*T + a[3]/4.0*T**3.0 + a[4]/5.0*T**4.0 + a[5]/T
        return hrt * self.Rgas * T

    #--------------------------------------------------------
    def h_mass(self,T) :
        """
        return enthalpy in units of J/kg
        T: input (K)
        """
        return self.h_mole(T)/self.M
```

```python
def f_flame(Ta) :
    """
    We are solving for hp = sum_i y_i*h_i. In f=0 form this is f = hp - sum_i y_i*h_i
    We know the reactant temperature, so we can compute enthalpy (h). Then we know hp = hr (adiabatic).
    Vary T until sum_i y_i*h_i = hp.
    Steps:
        1. Given moles --> mass --> mass fractions.
        2. Make thermo classes for each species.
        3. Compute hr = sum_i y_i*h_i.
        ... Do this for the reactants, then products.
    """
    no2 = 2.                        # kmol
    nch4 = 1.
    nn2  = 7.52
    nco2 = 1.
    nh2o = 2.
    Mo2 = 32.                       # kg/kmol
    Mch4 = 16.
    Mn2  = 28.
    Mco2 = 44.
    Mh2o = 18.
    mo2 = no2*Mo2                   # mass
    mch4 = nch4*Mch4                # mass
    mn2 = nn2*Mn2                   # mass
    mh2o = nh2o*Mh2o
    mco2 = nco2*Mco2
    t_o2 = thermo("O2",Mo2)         # thermo object; use as: t_o2.h_mass(T) to get h_O2, etc.
    t_ch4 = thermo("CH4",Mch4)
    t_n2 = thermo("N2",Mn2)
    t_co2 = thermo("CO2",Mco2)
    t_h2o = thermo("H2O",Mh2o)

    #-------- Reactants
    # TO DO: compute total mass, then mass fractions
    # TO DO: Set reactant temperature, then compute reactant enthalpy

    #---------- Products
    # TO DO: Set the product enthalpy = reactant enthalpy
    # TO DO: Set the product mass fractions
    # TO DO: Compute the enthalpy of the products corresponding to the current Tp
    #        Then return the function: f(Tp) = hp - hp_based_on_current_Tp
```

```python
# TO DO: Set a guess temperature, then solve for the product temperature
```

### Problem 4

**Example: Solve a system of 6 equations in 6 unknowns**

This is solving a parallel pipe network where we have three pipes that are connected at the beginning and the end. The pipes can be of different lengths and diameter and pipe roughness.  Given the total flow rate, and the pipe properties, find the flow rate through each of three parallel pipes.
* **Unknowns: three flow rates: $Q_1$, $Q_2$, $Q_3$**.
* We need ***three equations***.
    * We'll label the pipes 1, 2, and 3.
    * **Eq. 1:** $Q_{tot} = Q_1+Q_2+Q_3$.
        * That is, the total flow rate is just the sum through each pipe.
    * Because the pipes are connected, the pressure drop across each pipe is the same:
        * **Eq. 2:** $\Delta P_1=\Delta P_2,$
        * **Eq. 3:** $\Delta P_1=\Delta P_3$
* Now we need to relate the pressure drop equations to the unknowns. The pressure is related to the flow rate by:
    * $\Delta P=\frac{fL\rho v^2}{2D}$, and we use $Q=Av=\frac{\pi}{4}D^2v\rightarrow v=\frac{4Q}{\pi D^2}$, where $Q$ is volumetric flow rate. Then, substitute for v to get: $$\Delta P=\frac{fL\rho}{2D}\left(\frac{4Q}{\pi D^2}\right)^2$$
* Here, $f$ is the friction factor in the pipe. We treat it as an unknown so we have **three more unknowns: $f_1$, $f_2$, $f_3$**. The Colbrook equation relates $f$ to $Q$ for given pipe properties. So, we have **three more equations**.

* Here are the **six equations** in terms of the **six unknowns: $Q_1$, $Q_2$, $Q_3$, $f_1$, $f_2$, $f_3$**.
    1. $Q_1+Q_2+Q_3-Q_{tot} = 0$.
    2.  $\frac{f_1L_1\rho}{2D_1}\left(\frac{4Q_1}{\pi D_1^2}\right)^2 - \frac{f_2L_2\rho}{2D_2}\left(\frac{4Q_2}{\pi D_2^2}\right)^2 = 0$
    3. $\frac{f_1L_1\rho}{2D_1}\left(\frac{4Q_1}{\pi D_1^2}\right)^2 - \frac{f_3L_3\rho}{2D_3}\left(\frac{4Q_3}{\pi D_3^2}\right)^2 = 0$
    4. Colbrook equation relating $f_1$ to $Q_1$:
    $$\frac{1}{\sqrt{f_1}}+2\log_{10}\left(\frac{\epsilon_1}{3.7D_1} + \frac{2.51\mu\pi D_1}{\rho 4Q_1\sqrt{f_1}}\right).$$
    5. Colbrook equation relating $f_2$ to $Q_2$.
    6. Colbrook equation relating $f_3$ to $Q_3$.

* All units are SI.

```python
def F_pipes(x) :
    Q1 = x[0]           # rename the vars so we can read our equations below.
    Q2 = x[1]
    Q3 = x[2]
    f1 = x[3]
    f2 = x[4]
    f3 = x[5]

    Qt = 0.01333       # Given total volumetric flow rate
    e1 = 0.00024       # pipe roughness (m) (epsilon in the equation)
    e2 = 0.00012
    e3 = 0.0002
    L1 = 100           # pipe length (m)
    L2 = 150
    L3 = 80
    D1 = 0.05          # pipe diameter (m)
    D2 = 0.045
    D3 = 0.04
    mu = 1.002E-3      # viscosity (kg/m*s)
    rho = 998.         # density (kg/m3)

    F = np.zeros(6)    # initialize the function array

    # TO DO: Define the functions here

    return F

#--------------------------------------
# TO DO: make a guess array for the unknowns: Q1, Q2, Q3, f1, f2, f3
#     (use Q3 = Qtot-Q1-Q2 in your guess, for consistency)
# TO DO: Solve the problem and print the results.
```

```python

```