# Tutorial on Python for scientific computing

> Marcos Duarte, Renato Naville Watanabe
> [Laboratory of Biomechanics and Motor Control](http://pesquisa.ufabc.edu.br/bmclab)
> Federal University of ABC, Brazil

This will be a very brief tutorial on Python.
For a complete (and much better) tutorial about Python see [A Whirlwind Tour of Python](https://github.com/jakevdp/WhirlwindTourOfPython) and [Python Data Science Handbook](https://jakevdp.github.io/PythonDataScienceHandbook/) for a specific tutorial about Python for scientific computing.

To use Python for scientific computing we need the Python program itself with its main modules and specific packages for scientific computing. [See this notebook on how to install Python for scientific computing](http://nbviewer.ipython.org/github/demotu/BMC/blob/master/notebooks/PythonInstallation.ipynb).
Once you get Python and the necessary packages for scientific computing ready to work, there are different ways to run Python, the main ones are:

- open a terminal window in your computer and type `python` or `ipython` that the Python interpreter will start
- run the `Jupyter notebook` and start working with Python in a browser
- run `Spyder`, an interactive development environment (IDE)
- run the `Jupyter qtconsole`, a more featured terminal
- run Python online in a website such as [https://www.pythonanywhere.com/](https://www.pythonanywhere.com/) or [Colaboratory](https://colab.research.google.com/notebooks/welcome.ipynb)
- run Python using any other Python editor or IDE

We will use the Jupyter Notebook for this tutorial but you can run almost all the things we will see here using the other forms listed above.

## Python as a calculator

Once in the Jupyter notebook, if you type a simple mathematical expression and press `Shift+Enter` it will give the result of the expression:

```python
1 + 2 - 25
```

    -22

```python
4/7
```

    0.5714285714285714

Using the `print` function, let's explore the mathematical operations available in Python:

```python
print('1+2 = ', 1+2, '\n', '4*5 = ', 4*5, '\n', '6/7 = ', 6/7, '\n', '8**2 = ', 8**2, sep='')
```

    1+2 = 3
    4*5 = 20
    6/7 = 0.8571428571428571
    8**2 = 64

And if we want the square-root of a number:

```python
sqrt(9)
```

We get an error message saying that the `sqrt` function if not defined. This is because `sqrt` and other mathematical functions are available with the `math` module:

```python
import math
```

```python
math.sqrt(9)
```

    3.0

```python
from math import sqrt
```

```python
sqrt(9)
```

    3.0

## The import function

We used the command '`import`' to be able to call certain functions. In Python functions are organized in modules and packages and they have to be imported in order to be used.

A module is a file containing Python definitions (e.g., functions) and statements. Packages are a way of structuring Python’s module namespace by using “dotted module names”. For example, the module name A.B designates a submodule named B in a package named A. To be used, modules and packages have to be imported in Python with the import function.

Namespace is a container for a set of identifiers (names), and allows the disambiguation of homonym identifiers residing in different namespaces. For example, with the command import math, we will have all the functions and statements defined in this module in the namespace '`math.`', for example, '`math.pi`' is the π constant and '`math.cos()`', the cosine function.

By the way, to know which Python version you are running, we can use one of the following modules:

```python
import sys
sys.version
```

    '3.6.6 |Anaconda, Inc.| (default, Jun 28 2018, 17:14:51) \n[GCC 7.2.0]'

And if you are in an IPython session:

```python
from IPython import sys_info
print(sys_info())
```

    {'commit_hash': 'd774f565b',
     'commit_source': 'installation',
     'default_encoding': 'UTF-8',
     'ipython_path': '/opt/miniconda3/lib/python3.6/site-packages/IPython',
     'ipython_version': '7.4.0',
     'os_name': 'posix',
     'platform': 'Linux-4.15.0-47-generic-x86_64-with-debian-buster-sid',
     'sys_executable': '/opt/miniconda3/bin/python',
     'sys_platform': 'linux',
     'sys_version': '3.6.6 |Anaconda, Inc.| (default, Jun 28 2018, 17:14:51) \n'
                    '[GCC 7.2.0]'}

The first option gives information about the Python version; the latter also includes the IPython version, operating system, etc.

## Object-oriented programming

Python is designed as an object-oriented programming (OOP) language. OOP is a paradigm that represents concepts as "objects" that have data fields (attributes that describe the object) and associated procedures known as methods.

This means that all elements in Python are objects and they have attributes which can be acessed with the dot (.) operator after the name of the object. We already experimented with that when we imported the module `sys`, it became an object, and we acessed one of its attribute: `sys.version`.

OOP as a paradigm is much more than defining objects, attributes, and methods, but for now this is enough to get going with Python.

## Python and IPython help

To get help about any Python command, use `help()`:

```python
help(math.degrees)
```

    Help on built-in function degrees in module math:

    degrees(...)
        degrees(x)

        Convert angle x from radians to degrees.

Or if you are in the IPython environment, simply add '?' to the function that a window will open at the bottom of your browser with the same help content:

```python
math.degrees?
```

And if you add a second '?' to the statement you get access to the original script file of the function (an advantage of an open source language), unless that function is a built-in function that does not have a script file, which is the case of the standard modules in Python (but you can access the Python source code if you want; it just does not come with the standard program for installation).

So, let's see this feature with another function:

```python
import scipy.fftpack
scipy.fftpack.fft??
```

To know all the attributes of an object, for example all the functions available in `math`, we can use the function `dir`:

```python
print(dir(math))
```

    ['__doc__', '__loader__', '__name__', '__package__', '__spec__', 'acos', 'acosh', 'asin', 'asinh', 'atan', 'atan2', 'atanh', 'ceil', 'copysign', 'cos', 'cosh', 'degrees', 'e', 'erf', 'erfc', 'exp', 'expm1', 'fabs', 'factorial', 'floor', 'fmod', 'frexp', 'fsum', 'gamma', 'gcd', 'hypot', 'inf', 'isclose', 'isfinite', 'isinf', 'isnan', 'ldexp', 'lgamma', 'log', 'log10', 'log1p', 'log2', 'modf', 'nan', 'pi', 'pow', 'radians', 'sin', 'sinh', 'sqrt', 'tan', 'tanh', 'tau', 'trunc']

### Tab completion in IPython

IPython has tab completion: start typing the name of the command (object) and press `tab` to see the names of objects available with these initials letters. When the name of the object is typed followed by a dot (`math.`), pressing `tab` will show all available attribites, scroll down to the desired attribute and press `Enter` to select it.

### The four most helpful commands in IPython

These are the most helpful commands in IPython (from [IPython tutorial](http://ipython.org/ipython-doc/dev/interactive/tutorial.html)):

 - `?` : Introduction and overview of IPython’s features.
 - `%quickref` : Quick reference.
 - `help` : Python’s own help system.
 - `object?` : Details about ‘object’, use ‘object??’ for extra details.

### Comments

Comments in Python start with the hash character, #, and extend to the end of the physical line:

```python
# Import the math library to access more math stuff
import math
math.pi  # this is the pi constant; a useless comment since this is obvious
```

    3.141592653589793

To insert comments spanning more than one line, use a multi-line string with a pair of matching triple-quotes: `"""` or `'''` (we will see the string data type later). A typical use of a multi-line comment is as documentation strings and are meant for anyone reading the code:

```python
"""Documentation strings are typically written like that.

A docstring is a string literal that occurs as the first statement
in a module, function, class, or method definition.

"""
```

    'Documentation strings are typically written like that.\n\nA docstring is a string literal that occurs as the first statement\nin a module, function, class, or method definition.\n\n'

A docstring like above is useless and its output as a standalone statement looks uggly in IPython Notebook, but you will see its real importance when reading and writting codes.

Commenting a programming code is an important step to make the code more readable, which Python cares a lot.
There is a style guide for writting Python code ([PEP 8](https://www.python.org/dev/peps/pep-0008/)) with a session about [how to write comments](https://www.python.org/dev/peps/pep-0008/#comments).

### Magic functions

IPython has a set of predefined ‘magic functions’ that you can call with a command line style syntax.
There are two kinds of magics, line-oriented and cell-oriented.
Line magics are prefixed with the % character and work much like OS command-line calls: they get as an argument the rest of the line, where arguments are passed without parentheses or quotes.
Cell magics are prefixed with a double %%, and they are functions that get as an argument not only the rest of the line, but also the lines below it in a separate argument.

## Assignment and expressions

The equal sign ('=') is used to assign a value to a variable. Afterwards, no result is displayed before the next interactive prompt:

```python
x = 1
```

Spaces between the statements are optional but it helps for readability.

To see the value of the variable, call it again or use the print function:

```python
x
```

    1

```python
print(x)
```

    1

Of course, the last assignment is that holds:

```python
x = 2
x = 3
x
```

    3

In mathematics '=' is the symbol for identity, but in computer programming '=' is used for assignment, it means that the right part of the expresssion is assigned to its left part.
For example, 'x=x+1' does not make sense in mathematics but it does in computer programming:

```python
x = x + 1
print(x)
```

    6

A value can be assigned to several variables simultaneously:

```python
x = y = 4
print(x)
print(y)
```

    4
    4

Several values can be assigned to several variables at once:

```python
x, y = 5, 6
print(x)
print(y)
```

    5
    6

And with that, you can do (!):

```python
x, y = y, x
print(x)
print(y)
```

    6
    5

Variables must be “defined” (assigned a value) before they can be used, or an error will occur:

```python
x = z
```

## Variables and types

There are different types of built-in objects in Python (and remember that everything in Python is an object):

```python
import types
print(dir(types))
```

    ['AsyncGeneratorType', 'BuiltinFunctionType', 'BuiltinMethodType', 'CodeType', 'CoroutineType', 'DynamicClassAttribute', 'FrameType', 'FunctionType', 'GeneratorType', 'GetSetDescriptorType', 'LambdaType', 'MappingProxyType', 'MemberDescriptorType', 'MethodType', 'ModuleType', 'SimpleNamespace', 'TracebackType', '_GeneratorWrapper', '__all__', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', '_ag', '_calculate_meta', '_collections_abc', '_functools', 'coroutine', 'new_class', 'prepare_class']

Let's see some of them now.

### Numbers: int, float, complex

Numbers can an integer (int), float, and complex (with imaginary part).
Let's use the function `type` to show the type of number (and later for any other object):

```python
type(6)
```

    int

A float is a non-integer number:

```python
math.pi
```

    3.141592653589793

```python
type(math.pi)
```

    float

Python (IPython) is showing `math.pi` with only 15 decimal cases, but internally a float is represented with higher precision.
Floating point numbers in Python are implemented using a double (eight bytes) word; the precison and internal representation of floating point numbers are machine specific and are available in:

```python
sys.float_info
```

    sys.float_info(max=1.7976931348623157e+308, max_exp=1024, max_10_exp=308, min=2.2250738585072014e-308, min_exp=-1021, min_10_exp=-307, dig=15, mant_dig=53, epsilon=2.220446049250313e-16, radix=2, rounds=1)

Be aware that floating-point numbers can be trick in computers:

```python
0.1 + 0.2
```

    0.30000000000000004

```python
0.1 + 0.2 - 0.3
```

    5.551115123125783e-17

These results are not correct (and the problem is not due to Python). The error arises from the fact that floating-point numbers are represented in computer hardware as base 2 (binary) fractions and most decimal fractions cannot be represented exactly as binary fractions. As consequence, decimal floating-point numbers are only approximated by the binary floating-point numbers actually stored in the machine. [See here for more on this issue](http://docs.python.org/2/tutorial/floatingpoint.html).

A complex number has real and imaginary parts:

```python
1+2j
```

    (1+2j)

```python
print(type(1+2j))
```

    <class 'complex'>

Each part of a complex number is represented as a floating-point number. We can see them using the attributes `.real` and `.imag`:

```python
print((1+2j).real)
print((1+2j).imag)
```

    1.0
    2.0

### Strings

Strings can be enclosed in single quotes or double quotes:

```python
s = 'string (str) is a built-in type in Python'
s
```

    'string (str) is a built-in type in Python'

```python
type(s)
```

    str

String enclosed with single and double quotes are equal, but it may be easier to use one instead of the other:

```python
'string (str) is a Python's built-in type'
```

```python
"string (str) is a Python's built-in type"
```

    "string (str) is a Python's built-in type"

But you could have done that using the Python escape character '\':

```python
'string (str) is a Python\'s built-in type'
```

    "string (str) is a Python's built-in type"

Strings can be concatenated (glued together) with the + operator, and repeated with *:

```python
s = 'P' + 'y' + 't' + 'h' + 'o' + 'n'
print(s)
print(s*5)
```

    Python
    PythonPythonPythonPythonPython

Strings can be subscripted (indexed); like in C, the first character of a string has subscript (index) 0:

```python
print('s[0] = ', s[0], '  (s[index], start at 0)')
print('s[5] = ', s[5])
print('s[-1] = ', s[-1], '  (last element)')
print('s[:] = ', s[:], '  (all elements)')
print('s[1:] = ', s[1:], '  (from this index (inclusive) till the last (inclusive))')
print('s[2:4] = ', s[2:4], '  (from first index (inclusive) till second index (exclusive))')
print('s[:2] = ', s[:2], '  (till this index, exclusive)')
print('s[:10] = ', s[:10], '  (Python handles the index if it is larger than the string length)')
print('s[-10:] = ', s[-10:])
print('s[0:5:2] = ', s[0:5:2], '  (s[ini:end:step])')
print('s[::2] = ', s[::2], '  (s[::step], initial and final indexes can be omitted)')
print('s[0:5:-1] = ', s[::-1], '  (s[::-step] reverses the string)')
print('s[:2] + s[2:] = ', s[:2] + s[2:], '  (because of Python indexing, this sounds natural)')
```

    s[0] =  P   (s[index], start at 0)
    s[5] =  n
    s[-1] =  n   (last element)
    s[:] =  Python   (all elements)
    s[1:] =  ython   (from this index (inclusive) till the last (inclusive))
    s[2:4] =  th   (from first index (inclusive) till second index (exclusive))
    s[:2] =  Py   (till this index, exclusive)
    s[:10] =  Python   (Python handles the index if it is larger than the string length)
    s[-10:] =  Python
    s[0:5:2] =  Pto   (s[ini:end:step])
    s[::2] =  Pto   (s[::step], initial and final indexes can be omitted)
    s[0:5:-1] =  nohtyP   (s[::-step] reverses the string)
    s[:2] + s[2:] =  Python   (because of Python indexing, this sounds natural)

### len()

Python has a built-in functon to get the number of itens of a sequence:

```python
help(len)
```

    Help on built-in function len in module builtins:

    len(obj, /)
        Return the number of items in a container.

```python
s = 'Python'
len(s)
```

    6

The function len() helps to understand how the backward indexing works in Python.
The index s[-i] should be understood as s[len(s) - i] rather than accessing directly the i-th element from back to front. This is why the last element of a string is s[-1]:

```python
print('s = ', s)
print('len(s) = ', len(s))
print('len(s)-1 = ',len(s) - 1)
print('s[-1] = ', s[-1])
print('s[len(s) - 1] = ', s[len(s) - 1])
```

    s =  Python
    len(s) =  6
    len(s)-1 =  5
    s[-1] =  n
    s[len(s) - 1] =  n

Or, strings can be surrounded in a pair of matching triple-quotes: """ or '''. End of lines do not need to be escaped when using triple-quotes, but they will be included in the string. This is how we created a multi-line comment earlier:

```python
"""Strings can be surrounded in a pair of matching triple-quotes: \""" or '''.

End of lines do not need to be escaped when using triple-quotes,
but they will be included in the string.

"""
```

    'Strings can be surrounded in a pair of matching triple-quotes: """ or \'\'\'.\n\nEnd of lines do not need to be escaped when using triple-quotes,\nbut they will be included in the string.\n\n'

### Lists

Values can be grouped together using different types, one of them is list, which can be written as a list of comma-separated values between square brackets. List items need not all have the same type:

```python
x = ['spam', 'eggs', 100, 1234]
x
```

    ['spam', 'eggs', 100, 1234]

Lists can be indexed and the same indexing rules we saw for strings are applied:

```python
x[0]
```

    'spam'

The function len() works for lists:

```python
len(x)
```

    4

### Tuples

A tuple consists of a number of values separated by commas, for instance:

```python
t = ('spam', 'eggs', 100, 1234)
t
```

    ('spam', 'eggs', 100, 1234)

The type tuple is why multiple assignments in a single line works; elements separated by commas (with or without surrounding parentheses) are a tuple and in an expression with an '=', the right-side tuple is attributed to the left-side tuple:

```python
a, b = 1, 2
print('a = ', a, '\nb = ', b)
```

    a =  1
    b =  2

Is the same as:

```python
(a, b) = (1, 2)
print('a = ', a, '\nb = ', b)
```

    a =  1
    b =  2

### Sets

Python also includes a data type for sets. A set is an unordered collection with no duplicate elements.

```python
basket = ['apple', 'orange', 'apple', 'pear', 'orange', 'banana']
fruit = set(basket)  # create a set without duplicates
fruit
```

    {'apple', 'banana', 'orange', 'pear'}

As set is an unordered collection, it can not be indexed as lists and tuples.

```python
set(['orange', 'pear', 'apple', 'banana'])
'orange' in fruit  # fast membership testing
```

    True

### Dictionaries

Dictionary is a collection of elements organized keys and values. Unlike lists and tuples, which are indexed by a range of numbers, dictionaries are indexed by their keys:

```python
tel = {'jack': 4098, 'sape': 4139}
tel
```

    {'jack': 4098, 'sape': 4139}

```python
tel['guido'] = 4127
tel
```

    {'guido': 4127, 'jack': 4098, 'sape': 4139}

```python
tel['jack']
```

    4098

```python
del tel['sape']
tel['irv'] = 4127
tel
```

    {'guido': 4127, 'irv': 4127, 'jack': 4098}

```python
tel.keys()
```

    dict_keys(['jack', 'guido', 'irv'])

```python
'guido' in tel
```

    True

The dict() constructor builds dictionaries directly from sequences of key-value pairs:

```python
tel = dict([('sape', 4139), ('guido', 4127), ('jack', 4098)])
tel
```

    {'guido': 4127, 'jack': 4098, 'sape': 4139}

## Built-in Constants

- **False** : false value of the bool type
- **True** : true value of the bool type
- **None** : sole value of types.NoneType. None is frequently used to represent the absence of a value.

In computer science, the Boolean or logical data type is composed by two values, true and false, intended to represent the values of logic and Boolean algebra. In Python, 1 and 0 can also be used in most situations as equivalent to the Boolean values.

## Logical (Boolean) operators

### and, or, not

- **and** : logical AND operator. If both the operands are true then condition becomes true.	 (a and b) is true.
- **or** : logical OR Operator. If any of the two operands are non zero then condition becomes true.	 (a or b) is true.
- **not** : logical NOT Operator. Reverses the logical state of its operand. If a condition is true then logical NOT operator will make false.

### Comparisons

The following comparison operations are supported by objects in Python:

- **==** : equal
- **!=** : not equal
- **<**	: strictly less than
- **<=** : less than or equal
- **\>** : strictly greater than
- **\>=** : greater than or equal
- **is** : object identity
- **is not** : negated object identity

```python
True == False
```

    False

```python
not True == False
```

    True

```python
1 < 2 > 1
```

    True

```python
True != (False or True)
```

    False

```python
True != False or True
```

    True

## Indentation and whitespace

In Python, statement grouping is done by indentation (this is mandatory), which are done by inserting whitespaces, not tabs. Indentation is also recommended for alignment of function calling that span more than one line for better clarity.
We will see examples of indentation in the next session.

## Control of flow

### `if`...`elif`...`else`

Conditional statements (to peform something if another thing is True or False) can be implemmented using the `if` statement:
```
if expression:
   statement
elif:
   statement
else:
   statement
```
`elif` (one or more) and `else` are optionals.
The indentation is obligatory.
For example:

```python
if True:
    pass
```

Which does nothing useful.

Let's use the `if`...`elif`...`else` statements to categorize the [body mass index](http://en.wikipedia.org/wiki/Body_mass_index) of a person:

```python
# body mass index
weight = 100  # kg
height = 1.70  # m
bmi = weight / height**2
```

```python
if bmi < 15:
    c = 'very severely underweight'
elif 15 <= bmi < 16:
    c = 'severely underweight'
elif 16 <= bmi < 18.5:
    c = 'underweight'
elif 18.5 <= bmi < 25:
    c = 'normal'
elif 25 <= bmi < 30:
    c = 'overweight'
elif 30 <= bmi < 35:
    c = 'moderately obese'
elif 35 <= bmi < 40:
    c = 'severely obese'
else:
    c = 'very severely obese'

print('For a weight of {0:.1f} kg and a height of {1:.2f} m,\n\
the body mass index (bmi) is {2:.1f} kg/m2,\nwhich is considered {3:s}.'\
      .format(weight, height, bmi, c))
```

    For a weight of 100.0 kg and a height of 1.70 m,
    the body mass index (bmi) is 34.6 kg/m2,
    which is considered moderately obese.

### for

The `for` statement iterates over a sequence to perform operations (a loop event).
```
for iterating_var in sequence:
    statements
```

```python
for i in [3, 2, 1, 'go!']:
    print(i, end=', ')
```

    3, 2, 1, go!,

```python
for letter in 'Python':
    print(letter),
```

    P
    y
    t
    h
    o
    n

#### The `range()` function

The built-in function range() is useful if we need to create a sequence of numbers, for example, to iterate over this list. It generates lists containing arithmetic progressions:

```python
help(range)
```

    Help on class range in module builtins:

    class range(object)
     |  range(stop) -> range object
     |  range(start, stop[, step]) -> range object
     |
     |  Return an object that produces a sequence of integers from start (inclusive)
     |  to stop (exclusive) by step.  range(i, j) produces i, i+1, i+2, ..., j-1.
     |  start defaults to 0, and stop is omitted!  range(4) produces 0, 1, 2, 3.
     |  These are exactly the valid indices for a list of 4 elements.
     |  When step is given, it specifies the increment (or decrement).
     |
     |  Methods defined here:
     |
     |  __bool__(self, /)
     |      self != 0
     |
     |  __contains__(self, key, /)
     |      Return key in self.
     |
     |  __eq__(self, value, /)
     |      Return self==value.
     |
     |  __ge__(self, value, /)
     |      Return self>=value.
     |
     |  __getattribute__(self, name, /)
     |      Return getattr(self, name).
     |
     |  __getitem__(self, key, /)
     |      Return self[key].
     |
     |  __gt__(self, value, /)
     |      Return self>value.
     |
     |  __hash__(self, /)
     |      Return hash(self).
     |
     |  __iter__(self, /)
     |      Implement iter(self).
     |
     |  __le__(self, value, /)
     |      Return self<=value.
     |
     |  __len__(self, /)
     |      Return len(self).
     |
     |  __lt__(self, value, /)
     |      Return self<value.
     |
     |  __ne__(self, value, /)
     |      Return self!=value.
     |
     |  __new__(*args, **kwargs) from builtins.type
     |      Create and return a new object.  See help(type) for accurate signature.
     |
     |  __reduce__(...)
     |      helper for pickle
     |
     |  __repr__(self, /)
     |      Return repr(self).
     |
     |  __reversed__(...)
     |      Return a reverse iterator.
     |
     |  count(...)
     |      rangeobject.count(value) -> integer -- return number of occurrences of value
     |
     |  index(...)
     |      rangeobject.index(value, [start, [stop]]) -> integer -- return index of value.
     |      Raise ValueError if the value is not present.
     |
     |  ----------------------------------------------------------------------
     |  Data descriptors defined here:
     |
     |  start
     |
     |  step
     |
     |  stop

```python
range(10)
```

    range(0, 10)

```python
range(1, 10, 2)
```

    range(1, 10, 2)

```python
for i in range(10):
    n2 = i**2
    print(n2),
```

    0
    1
    4
    9
    16
    25
    36
    49
    64
    81

### while

The `while` statement is used for repeating sections of code in a loop until a condition is met (this different than the `for` statement which executes n times):
```
while expression:
    statement
```
Let's generate the Fibonacci series using a `while` loop:

```python
# Fibonacci series: the sum of two elements defines the next
a, b = 0, 1
while b < 1000:
    print(b, end='   ')
    a, b = b, a+b
```

    1   1   2   3   5   8   13   21   34   55   89   144   233   377   610   987

## Function definition

A function in a programming language is a piece of code that performs a specific task. Functions are used to reduce duplication of code making easier to reuse it and to decompose complex problems into simpler parts. The use of functions contribute to the clarity of the code.

A function is created with the `def` keyword and the statements in the block of the function must be indented:

```python
def function():
    pass
```

As per construction, this function does nothing when called:

```python
function()
```

The general syntax of a function definition is:
```
def function_name( parameters ):
   """Function docstring.

   The help for the function

   """

   function body

   return variables
```
A more useful function:

```python
def fibo(N):
    """Fibonacci series: the sum of two elements defines the next.

    The series is calculated till the input parameter N and
    returned as an ouput variable.

    """

    a, b, c = 0, 1, []
    while b < N:
        c.append(b)
        a, b = b, a + b

    return c
```

```python
fibo(100)
```

    [1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89]

```python
if 3 > 2:
       print('teste')
```

    teste

Let's implemment the body mass index calculus and categorization as a function:

```python
def bmi(weight, height):
    """Body mass index calculus and categorization.

    Enter the weight in kg and the height in m.
    See http://en.wikipedia.org/wiki/Body_mass_index

    """

    bmi = weight / height**2

    if bmi < 15:
     c = 'very severely underweight'
    elif 15 <= bmi < 16:
        c = 'severely underweight'
    elif 16 <= bmi < 18.5:
        c = 'underweight'
    elif 18.5 <= bmi < 25:
        c = 'normal'
    elif 25 <= bmi < 30:
      c = 'overweight'
    elif 30 <= bmi < 35:
        c = 'moderately obese'
    elif 35 <= bmi < 40:
        c = 'severely obese'
    else:
        c = 'very severely obese'

    s = 'For a weight of {0:.1f} kg and a height of {1:.2f} m,\
    the body mass index (bmi) is {2:.1f} kg/m2,\
    which is considered {3:s}.'\
    .format(weight, height, bmi, c)

    print(s)
```

```python
bmi(73, 1.70)
```

    For a weight of 73.0 kg and a height of 1.70 m,    the body mass index (bmi) is 25.3 kg/m2,    which is considered overweight.

## Numeric data manipulation with Numpy

Numpy is the fundamental package for scientific computing in Python and has a N-dimensional array package convenient to work with numerical data. With Numpy it's much easier and faster to work with numbers grouped as 1-D arrays (a vector), 2-D arrays (like a table or matrix), or higher dimensions. Let's create 1-D and 2-D arrays in Numpy:

```python
import numpy as np
```

```python
x1d = np.array([1, 2, 3, 4, 5, 6])
print(type(x1d))
x1d
```

    <class 'numpy.ndarray'>

    array([1, 2, 3, 4, 5, 6])

```python
x2d = np.array([[1, 2, 3], [4, 5, 6]])
x2d
```

    array([[1, 2, 3],
           [4, 5, 6]])

len() and the Numpy functions size() and shape() give information aboout the number of elements and the structure of the Numpy array:

```python
print('1-d array:')
print(x1d)
print('len(x1d) = ', len(x1d))
print('np.size(x1d) = ', np.size(x1d))
print('np.shape(x1d) = ', np.shape(x1d))
print('np.ndim(x1d) = ', np.ndim(x1d))
print('\n2-d array:')
print(x2d)
print('len(x2d) = ', len(x2d))
print('np.size(x2d) = ', np.size(x2d))
print('np.shape(x2d) = ', np.shape(x2d))
print('np.ndim(x2d) = ', np.ndim(x2d))
```

    1-d array:
    [1 2 3 4 5 6]
    len(x1d) =  6
    np.size(x1d) =  6
    np.shape(x1d) =  (6,)
    np.ndim(x1d) =  1

    2-d array:
    [[1 2 3]
     [4 5 6]]
    len(x2d) =  2
    np.size(x2d) =  6
    np.shape(x2d) =  (2, 3)
    np.ndim(x2d) =  2

Create random data

```python
x = np.random.randn(4,3)
x
```

    array([[-0.36123769,  0.18896133, -0.53809885],
           [-0.7332364 ,  0.47109317,  1.06194556],
           [ 0.07331805,  0.72426922, -1.74606307],
           [-0.48601252, -0.72308218, -0.98513516]])

Joining (stacking together) arrays

```python
x = np.random.randint(0, 5, size=(2, 3))
print(x)
y = np.random.randint(5, 10, size=(2, 3))
print(y)
```

    [[1 2 3]
     [3 2 2]]
    [[5 5 5]
     [7 5 9]]

```python
np.vstack((x,y))
```

    array([[1, 2, 3],
           [3, 2, 2],
           [5, 5, 5],
           [7, 5, 9]])

```python
np.hstack((x,y))
```

    array([[1, 2, 3, 5, 5, 5],
           [3, 2, 2, 7, 5, 9]])

Create equally spaced data

```python
np.arange(start = 1, stop = 10, step = 2)
```

    array([1, 3, 5, 7, 9])

```python
np.linspace(start = 0, stop = 1, num = 11)
```

    array([ 0. ,  0.1,  0.2,  0.3,  0.4,  0.5,  0.6,  0.7,  0.8,  0.9,  1. ])

### Interpolation

Consider the following data:

```python
y = [5,  4, 10,  8,  1, 10,  2,  7,  1,  3]
```

Suppose we want to create data in between the given data points (interpolation); for instance, let's try to double the resolution of the data by generating twice as many data:

```python
t = np.linspace(0, len(y), len(y))       # time vector for the original data
tn = np.linspace(0, len(y), 2 * len(y))  # new time vector for the new time-normalized data
yn = np.interp(tn, t, y)                 # new time-normalized data
yn
```

    array([ 5.        ,  4.52631579,  4.05263158,  6.52631579,  9.36842105,
            9.26315789,  8.31578947,  5.78947368,  2.47368421,  3.36842105,
            7.63157895,  8.31578947,  4.52631579,  2.78947368,  5.15789474,
            6.36842105,  3.52631579,  1.10526316,  2.05263158,  3.        ])

The key is the Numpy `interp` function, from its help:

    interp(x, xp, fp, left=None, right=None)
    One-dimensional linear interpolation.
    Returns the one-dimensional piecewise linear interpolant to a function with given values at discrete data-points.

A plot of the data will show what we have done:

```python
%matplotlib inline
import matplotlib.pyplot as plt
plt.figure(figsize=(10,5))
plt.plot(t, y, 'bo-', lw=2, label='original data')
plt.plot(tn, yn, '.-', color=[1, 0, 0, .5], lw=2, label='interpolated')
plt.legend(loc='best', framealpha=.5)
plt.show()
```

For more about Numpy, see [http://www.numpy.org/](http://www.numpy.org/).

## Read and save files

There are two kinds of computer files: text files and binary files:
> Text file: computer file where the content is structured as a sequence of lines of electronic text. Text files can contain plain text (letters, numbers, and symbols) but they are not limited to such. The type of content in the text file is defined by the Unicode encoding (a computing industry standard for the consistent encoding, representation and handling of text expressed in most of the world's writing systems).
>
> Binary file: computer file where the content is encoded in binary form, a sequence of integers representing byte values.

Let's see how to save and read numeric data stored in a text file:

**Using plain Python**

```python
f = open("newfile.txt", "w")           # open file for writing
f.write("This is a test\n")            # save to file
f.write("And here is another line\n")  # save to file
f.close()
f = open('newfile.txt', 'r')           # open file for reading
f = f.read()                           # read from file
print(f)
```

    This is a test
    And here is another line

```python
help(open)
```

    Help on built-in function open in module io:

    open(file, mode='r', buffering=-1, encoding=None, errors=None, newline=None, closefd=True, opener=None)
        Open file and return a stream.  Raise IOError upon failure.

        file is either a text or byte string giving the name (and the path
        if the file isn't in the current working directory) of the file to
        be opened or an integer file descriptor of the file to be
        wrapped. (If a file descriptor is given, it is closed when the
        returned I/O object is closed, unless closefd is set to False.)

        mode is an optional string that specifies the mode in which the file
        is opened. It defaults to 'r' which means open for reading in text
        mode.  Other common values are 'w' for writing (truncating the file if
        it already exists), 'x' for creating and writing to a new file, and
        'a' for appending (which on some Unix systems, means that all writes
        append to the end of the file regardless of the current seek position).
        In text mode, if encoding is not specified the encoding used is platform
        dependent: locale.getpreferredencoding(False) is called to get the
        current locale encoding. (For reading and writing raw bytes use binary
        mode and leave encoding unspecified.) The available modes are:

        ========= ===============================================================
        Character Meaning
        --------- ---------------------------------------------------------------
        'r'       open for reading (default)
        'w'       open for writing, truncating the file first
        'x'       create a new file and open it for writing
        'a'       open for writing, appending to the end of the file if it exists
        'b'       binary mode
        't'       text mode (default)
        '+'       open a disk file for updating (reading and writing)
        'U'       universal newline mode (deprecated)
        ========= ===============================================================

        The default mode is 'rt' (open for reading text). For binary random
        access, the mode 'w+b' opens and truncates the file to 0 bytes, while
        'r+b' opens the file without truncation. The 'x' mode implies 'w' and
        raises an `FileExistsError` if the file already exists.

        Python distinguishes between files opened in binary and text modes,
        even when the underlying operating system doesn't. Files opened in
        binary mode (appending 'b' to the mode argument) return contents as
        bytes objects without any decoding. In text mode (the default, or when
        't' is appended to the mode argument), the contents of the file are
        returned as strings, the bytes having been first decoded using a
        platform-dependent encoding or using the specified encoding if given.

        'U' mode is deprecated and will raise an exception in future versions
        of Python.  It has no effect in Python 3.  Use newline to control
        universal newlines mode.

        buffering is an optional integer used to set the buffering policy.
        Pass 0 to switch buffering off (only allowed in binary mode), 1 to select
        line buffering (only usable in text mode), and an integer > 1 to indicate
        the size of a fixed-size chunk buffer.  When no buffering argument is
        given, the default buffering policy works as follows:

        * Binary files are buffered in fixed-size chunks; the size of the buffer
          is chosen using a heuristic trying to determine the underlying device's
          "block size" and falling back on `io.DEFAULT_BUFFER_SIZE`.
          On many systems, the buffer will typically be 4096 or 8192 bytes long.

        * "Interactive" text files (files for which isatty() returns True)
          use line buffering.  Other text files use the policy described above
          for binary files.

        encoding is the name of the encoding used to decode or encode the
        file. This should only be used in text mode. The default encoding is
        platform dependent, but any encoding supported by Python can be
        passed.  See the codecs module for the list of supported encodings.

        errors is an optional string that specifies how encoding errors are to
        be handled---this argument should not be used in binary mode. Pass
        'strict' to raise a ValueError exception if there is an encoding error
        (the default of None has the same effect), or pass 'ignore' to ignore
        errors. (Note that ignoring encoding errors can lead to data loss.)
        See the documentation for codecs.register or run 'help(codecs.Codec)'
        for a list of the permitted encoding error strings.

        newline controls how universal newlines works (it only applies to text
        mode). It can be None, '', '\n', '\r', and '\r\n'.  It works as
        follows:

        * On input, if newline is None, universal newlines mode is
          enabled. Lines in the input can end in '\n', '\r', or '\r\n', and
          these are translated into '\n' before being returned to the
          caller. If it is '', universal newline mode is enabled, but line
          endings are returned to the caller untranslated. If it has any of
          the other legal values, input lines are only terminated by the given
          string, and the line ending is returned to the caller untranslated.

        * On output, if newline is None, any '\n' characters written are
          translated to the system default line separator, os.linesep. If
          newline is '' or '\n', no translation takes place. If newline is any
          of the other legal values, any '\n' characters written are translated
          to the given string.

        If closefd is False, the underlying file descriptor will be kept open
        when the file is closed. This does not work when a file name is given
        and must be True in that case.

        A custom opener can be used by passing a callable as *opener*. The
        underlying file descriptor for the file object is then obtained by
        calling *opener* with (*file*, *flags*). *opener* must return an open
        file descriptor (passing os.open as *opener* results in functionality
        similar to passing None).

        open() returns a file object whose type depends on the mode, and
        through which the standard file operations such as reading and writing
        are performed. When open() is used to open a file in a text mode ('w',
        'r', 'wt', 'rt', etc.), it returns a TextIOWrapper. When used to open
        a file in a binary mode, the returned class varies: in read binary
        mode, it returns a BufferedReader; in write binary and append binary
        modes, it returns a BufferedWriter, and in read/write mode, it returns
        a BufferedRandom.

        It is also possible to use a string or bytearray as a file for both
        reading and writing. For strings StringIO can be used like a file
        opened in a text mode, and for bytes a BytesIO can be used like a file
        opened in a binary mode.

**Using Numpy**

```python
import numpy as np
data = np.random.randn(3,3)
np.savetxt('myfile.txt', data, fmt="%12.6G")    # save to file
data = np.genfromtxt('myfile.txt', unpack=True) # read from file
data
```

    array([[-0.141613 ,  0.0158789,  1.09553  ],
           [-0.538208 ,  0.370273 , -0.0108841],
           [-0.327891 ,  0.117738 ,  1.63674  ]])

## Ploting with matplotlib

Matplotlib is the most-widely used packge for plotting data in Python. Let's see some examples of it.

```python
import matplotlib.pyplot as plt
```

Use the IPython magic `%matplotlib inline` to plot a figure inline in the notebook with the rest of the text:

```python
%matplotlib inline
```

```python
import numpy as np
```

```python
t = np.linspace(0, 0.99, 100)
x = np.sin(2 * np.pi * 2 * t)
n = np.random.randn(100) / 5
plt.Figure(figsize=(12,8))
plt.plot(t, x, label='sine', linewidth=2)
plt.plot(t, x + n, label='noisy sine', linewidth=2)
plt.annotate(s='$sin(4 \pi t)$', xy=(.2, 1), fontsize=20, color=[0, 0, 1])
plt.legend(loc='best', framealpha=.5)
plt.xlabel('Time [s]')
plt.ylabel('Amplitude')
plt.title('Data plotting using matplotlib')
plt.show()
```

Use the IPython magic `%matplotlib qt` to plot a figure in a separate window (from where you will be able to change some of the figure proprerties):

```python
%matplotlib qt
```

    Warning: Cannot change to a different GUI toolkit: qt. Using notebook instead.

```python
mu, sigma = 10, 2
x = mu + sigma * np.random.randn(1000)
fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(12, 4))
ax1.plot(x, 'ro')
ax1.set_title('Data')
ax1.grid()

n, bins, patches = ax2.hist(x, 25, normed=True, facecolor='r') # histogram
ax2.set_xlabel('Bins')
ax2.set_ylabel('Probability')
ax2.set_title('Histogram')
fig.suptitle('Another example using matplotlib', fontsize=18, y=1)
ax2.grid()

plt.tight_layout()
plt.show()
```

    <IPython.core.display.Javascript object>

    <IPython.core.display.Javascript object>

And a window with the following figure should appear:

```python
from IPython.display import Image
Image(url="./../images/plot.png")
```

You can switch back and forth between inline and separate figure using the `%matplotlib` magic commands used above. There are plenty more examples with the source code in the [matplotlib gallery](http://matplotlib.org/gallery.html).

```python
# get back the inline plot
%matplotlib inline
```

## Signal processing with Scipy

The Scipy package has a lot of functions for signal processing, among them: Integration (scipy.integrate), Optimization (scipy.optimize), Interpolation (scipy.interpolate), Fourier Transforms (scipy.fftpack), Signal Processing (scipy.signal), Linear Algebra (scipy.linalg), and Statistics (scipy.stats). As an example, let's see how to use a low-pass Butterworth filter to attenuate high-frequency noise and how the differentiation process of a signal affects the signal-to-noise content. We will also calculate the Fourier transform of these data to look at their frequencies content.

```python
from scipy.signal import butter, filtfilt
import scipy.fftpack
freq = 100.
t = np.arange(0,1,.01);
w = 2*np.pi*1 # 1 Hz
y = np.sin(w*t)+0.1*np.sin(10*w*t)
# Butterworth filter
b, a = butter(4, (5/(freq/2)), btype = 'low')
y2 = filtfilt(b, a, y)
# 2nd derivative of the data
ydd = np.diff(y,2)*freq*freq   # raw data
y2dd = np.diff(y2,2)*freq*freq # filtered data
# frequency content
yfft = np.abs(scipy.fftpack.fft(y))/(y.size/2);   # raw data
y2fft = np.abs(scipy.fftpack.fft(y2))/(y.size/2); # filtered data
freqs = scipy.fftpack.fftfreq(y.size, 1./freq)
yddfft = np.abs(scipy.fftpack.fft(ydd))/(ydd.size/2);
y2ddfft = np.abs(scipy.fftpack.fft(y2dd))/(ydd.size/2);
freqs2 = scipy.fftpack.fftfreq(ydd.size, 1./freq)
```

And the plots:

```python
fig, ((ax1,ax2),(ax3,ax4)) = plt.subplots(2, 2, figsize=(12, 6))

ax1.set_title('Temporal domain', fontsize=14)
ax1.plot(t, y, 'r', linewidth=2, label = 'raw data')
ax1.plot(t, y2, 'b', linewidth=2, label = 'filtered @ 5 Hz')
ax1.set_ylabel('f')
ax1.legend(frameon=False, fontsize=12)

ax2.set_title('Frequency domain', fontsize=14)
ax2.plot(freqs[:int(yfft.size/4)], yfft[:int(yfft.size/4)],'r',  lw=2,label='raw data')
ax2.plot(freqs[:int(yfft.size/4)],y2fft[:int(yfft.size/4)],'b--',lw=2,label='filtered @ 5 Hz')
ax2.set_ylabel('FFT(f)')
ax2.legend(frameon=False, fontsize=12)

ax3.plot(t[:-2], ydd, 'r', linewidth=2, label = 'raw')
ax3.plot(t[:-2], y2dd, 'b', linewidth=2, label = 'filtered @ 5 Hz')
ax3.set_xlabel('Time [s]'); ax3.set_ylabel("f ''")

ax4.plot(freqs[:int(yddfft.size/4)], yddfft[:int(yddfft.size/4)], 'r', lw=2, label = 'raw')
ax4.plot(freqs[:int(yddfft.size/4)],y2ddfft[:int(yddfft.size/4)],'b--',lw=2, label='filtered @ 5 Hz')
ax4.set_xlabel('Frequency [Hz]'); ax4.set_ylabel("FFT(f '')")
plt.show()
```

For more about Scipy, see [https://docs.scipy.org/doc/scipy/reference/tutorial/](https://docs.scipy.org/doc/scipy/reference/tutorial/).

## Symbolic mathematics with Sympy

Sympy is a package to perform symbolic mathematics in Python. Let's see some of its features:

```python
from IPython.display import display
import sympy as sym
from sympy.interactive import printing
printing.init_printing()
```

Define some symbols and the create a second-order polynomial function (a.k.a., parabola):

```python
x, y = sym.symbols('x y')
y = x**2 - 2*x - 3
y
```

Plot the parabola at some given range:

```python
from sympy.plotting import plot
%matplotlib inline
plot(y, (x, -3, 5));
```

And the roots of the parabola are given by:

```python
sym.solve(y, x)
```

We can also do symbolic differentiation and integration:

```python
dy = sym.diff(y, x)
dy
```

```python
sym.integrate(dy, x)
```

For example, let's use Sympy to represent three-dimensional rotations. Consider the problem of a coordinate system xyz rotated in relation to other coordinate system XYZ. The single rotations around each axis are illustrated by:

```python
from IPython.display import Image
Image(url="./../images/rotations.png")
```

The single 3D rotation matrices around Z, Y, and X axes can be expressed in Sympy:

```python
from IPython.core.display import Math
from sympy import symbols, cos, sin, Matrix, latex
a, b, g = symbols('alpha beta gamma')

RX = Matrix([[1, 0, 0], [0, cos(a), -sin(a)], [0, sin(a), cos(a)]])
display(Math(latex('\\mathbf{R_{X}}=') + latex(RX, mat_str = 'matrix')))

RY = Matrix([[cos(b), 0, sin(b)], [0, 1, 0], [-sin(b), 0, cos(b)]])
display(Math(latex('\\mathbf{R_{Y}}=') + latex(RY, mat_str = 'matrix')))

RZ = Matrix([[cos(g), -sin(g), 0], [sin(g), cos(g), 0], [0, 0, 1]])
display(Math(latex('\\mathbf{R_{Z}}=') + latex(RZ, mat_str = 'matrix')))
```

$$\mathbf{R_{X}}=\left[\begin{matrix}1 & 0 & 0\\0 & \cos{\left (\alpha \right )} & - \sin{\left (\alpha \right )}\\0 & \sin{\left (\alpha \right )} & \cos{\left (\alpha \right )}\end{matrix}\right]$$

$$\mathbf{R_{Y}}=\left[\begin{matrix}\cos{\left (\beta \right )} & 0 & \sin{\left (\beta \right )}\\0 & 1 & 0\\- \sin{\left (\beta \right )} & 0 & \cos{\left (\beta \right )}\end{matrix}\right]$$

$$\mathbf{R_{Z}}=\left[\begin{matrix}\cos{\left (\gamma \right )} & - \sin{\left (\gamma \right )} & 0\\\sin{\left (\gamma \right )} & \cos{\left (\gamma \right )} & 0\\0 & 0 & 1\end{matrix}\right]$$

And using Sympy, a sequence of elementary rotations around X, Y, Z axes is given by:

```python
RXYZ = RZ*RY*RX
display(Math(latex('\\mathbf{R_{XYZ}}=') + latex(RXYZ, mat_str = 'matrix')))
```

$$\mathbf{R_{XYZ}}=\left[\begin{matrix}\cos{\left (\beta \right )} \cos{\left (\gamma \right )} & \sin{\left (\alpha \right )} \sin{\left (\beta \right )} \cos{\left (\gamma \right )} - \sin{\left (\gamma \right )} \cos{\left (\alpha \right )} & \sin{\left (\alpha \right )} \sin{\left (\gamma \right )} + \sin{\left (\beta \right )} \cos{\left (\alpha \right )} \cos{\left (\gamma \right )}\\\sin{\left (\gamma \right )} \cos{\left (\beta \right )} & \sin{\left (\alpha \right )} \sin{\left (\beta \right )} \sin{\left (\gamma \right )} + \cos{\left (\alpha \right )} \cos{\left (\gamma \right )} & - \sin{\left (\alpha \right )} \cos{\left (\gamma \right )} + \sin{\left (\beta \right )} \sin{\left (\gamma \right )} \cos{\left (\alpha \right )}\\- \sin{\left (\beta \right )} & \sin{\left (\alpha \right )} \cos{\left (\beta \right )} & \cos{\left (\alpha \right )} \cos{\left (\beta \right )}\end{matrix}\right]$$

Suppose there is a rotation only around X ($\alpha$) by $\pi/2$; we can get the numerical value of the rotation matrix by substituing the angle values:

```python
r = RXYZ.subs({a: np.pi/2, b: 0, g: 0})
r
```

$$\left[\begin{matrix}1 & 0 & 0\\0 & 6.12323399573677 \cdot 10^{-17} & -1.0\\0 & 1.0 & 6.12323399573677 \cdot 10^{-17}\end{matrix}\right]$$

And we can prettify this result:

```python
display(Math(latex(r'\mathbf{R_{(\alpha=\pi/2)}}=') +
             latex(r.n(chop=True, prec=3), mat_str = 'matrix')))
```

$$\mathbf{R_{(\alpha=\pi/2)}}=\left[\begin{matrix}1.0 & 0 & 0\\0 & 0 & -1.0\\0 & 1.0 & 0\end{matrix}\right]$$

For more about Sympy, see [http://docs.sympy.org/latest/tutorial/](http://docs.sympy.org/latest/tutorial/).

## Data analysis with pandas

> "[pandas](http://pandas.pydata.org/) is a Python package providing fast, flexible, and expressive data structures designed to make working with “relational” or “labeled” data both easy and intuitive. It aims to be the fundamental high-level building block for doing practical, real world data analysis in Python."

To work with labellled data, pandas has a type called DataFrame (basically, a matrix where columns and rows have may names and may be of different types) and it is also the main type of the software [R](http://www.r-project.org/). Fo ezample:

```python
import pandas as pd
```

```python
x = 5*['A'] + 5*['B']
x
```

    ['A', 'A', 'A', 'A', 'A', 'B', 'B', 'B', 'B', 'B']

```python
df = pd.DataFrame(np.random.rand(10,2), columns=['Level 1', 'Level 2'] )
df['Group'] = pd.Series(['A']*5 + ['B']*5)
plot = df.boxplot(by='Group')
```

```python
from pandas.plotting import scatter_matrix
df = pd.DataFrame(np.random.randn(100, 3), columns=['A', 'B', 'C'])
plot = scatter_matrix(df, alpha=0.5, figsize=(8, 6), diagonal='kde')
```

pandas is aware the data is structured and give you basic statistics considerint that and nicely formatted:

```python
df.describe()
```

<div>
<style>
    .dataframe thead tr:only-child th {
        text-align: right;
    }

    .dataframe thead th {
        text-align: left;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>A</th>
      <th>B</th>
      <th>C</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>count</th>
      <td>100.000000</td>
      <td>100.000000</td>
      <td>100.000000</td>
    </tr>
    <tr>
      <th>mean</th>
      <td>0.101437</td>
      <td>-0.004921</td>
      <td>0.078482</td>
    </tr>
    <tr>
      <th>std</th>
      <td>1.017746</td>
      <td>1.084727</td>
      <td>1.023564</td>
    </tr>
    <tr>
      <th>min</th>
      <td>-2.323379</td>
      <td>-2.708453</td>
      <td>-2.866506</td>
    </tr>
    <tr>
      <th>25%</th>
      <td>-0.584179</td>
      <td>-0.752746</td>
      <td>-0.583681</td>
    </tr>
    <tr>
      <th>50%</th>
      <td>0.074468</td>
      <td>-0.050982</td>
      <td>0.115910</td>
    </tr>
    <tr>
      <th>75%</th>
      <td>0.845898</td>
      <td>0.705415</td>
      <td>0.674648</td>
    </tr>
    <tr>
      <th>max</th>
      <td>2.713227</td>
      <td>2.302115</td>
      <td>2.585530</td>
    </tr>
  </tbody>
</table>
</div>

For more on pandas, see this tutorial: [http://pandas.pydata.org/pandas-docs/stable/10min.html](http://pandas.pydata.org/pandas-docs/stable/10min.html).

## To learn more about Python

There is a lot of good material in the internet about Python for scientific computing, here is a small list of interesting stuff:

 - [How To Think Like A Computer Scientist](http://www.openbookproject.net/thinkcs/python/english2e/) or [the interactive edition](http://interactivepython.org/courselib/static/thinkcspy/index.html) (book)
 - [Python Scientific Lecture Notes](http://scipy-lectures.github.io/) (lecture notes)
 - [A Whirlwind Tour of Python](https://github.com/jakevdp/WhirlwindTourOfPython) (tutorial/book)
 - [Python Data Science Handbook](https://jakevdp.github.io/PythonDataScienceHandbook/) (tutorial/book)
 - [Lectures on scientific computing with Python](https://github.com/jrjohansson/scientific-python-lectures#lectures-on-scientific-computing-with-python) (lecture notes)


# Announcements
- No Problem Set this week, Problem Set 4 will be posted on 9/28.
- Solutions to Problem Set 3 posted on D2L.

# Ordinary Differential Equations - higher order methods in practice

Based on notes and notebooks by Niels Henrik Aase, Thorvald Ballestad, Vasilis Paschalidis and Jon Andreas Støvneng

### Higher Order Derivatives and Sets of 1st order ODEs

The trick to solving ODEs with higher derivatives is turning them into systems of first-order ODEs.

As a simple example, consider the second-order differential equation describing the van der Pol oscillator

$$ \frac{d^2 x}{dt^2} - a (1-x^2) \frac{dx}{dt} + x = 0 $$

We turn this into a pair of first-order ODEs by defining an auxiliary function $v(t) = dx/dt$ and writing the system as

\begin{align}
\begin{split}
\frac{dv}{dt} &= a (1-x^2) v - x\\
\frac{dx}{dt} &= v
\end{split}
\end{align}

Note that there are only functions (and the independent variable) on the RHS; all "differentials" are on the LHS.

Now that we have a system of first-order equations ,we can proceed as above. A function describing
the RHS of this system is

```python
import numpy as np
import matplotlib.pyplot as plt
def RK4_step(t, y, h, g, *P):
    """
    Implements a single step of a fourth-order, explicit Runge-Kutta scheme
    """
    thalf = t + 0.5*h
    k1 = h * g(t, y, *P)
    k2 = h * g(thalf, y + 0.5*k1, *P)
    k3 = h * g(thalf, y + 0.5*k2, *P)
    k4 = h * g(t + h, y + k3, *P)
    return y + (k1 + 2*k2 + 2*k3 + k4)/6
```

```python
def vdpRHS(t, y, a):
    # we store our function as the array [x, x']
    return np.array([

        y[1],                          # dx/dt = v
        a*(1-y[0]**2)*y[1] - y[0]      # dv/dt = a*(1-x**2)*v - x

    ])
```

```python
def odeSolve(t0, y0, tmax, h, RHS, method, *P):
    """
    ODE driver with constant step-size, allowing systems of ODE's
    """
    # make array of times and find length of array
    t = np.arange(t0,tmax+h,h)
    ntimes,  = t.shape

    # find out if we are solving a scalar ODE or a system of ODEs, and allocate space accordingly

    if type(y0) in [int, float]:  # check if primitive type -- means only one eqn
        neqn = 1
        y = np.zeros( ntimes )
    else:                         # otherwise assume a numpy array -- a system of more than one eqn
        neqn, = y0.shape
        y = np.zeros( (ntimes, neqn) )

    # set first element of solution to initial conditions (possibly a vector)
    y[0] = y0

    # march on...
    for i in range(0,ntimes-1):
        y[i+1] = method(t[i], y[i], h, RHS, *P)

    return t,y
```

```python
a = 15 # parameter

h = 0.01
t0 = 0.0
y0 = np.array([ 0,  1])
tmax = 3

# call the solver
t, y = odeSolve(t0, y0, tmax, h, vdpRHS, RK4_step, a)

fig,ax = plt.subplots()
ax.plot(t,y[:,0],'b', label='x')
ax.plot(t,y[:,1],'r--', label='v')
ax.set_xlabel('time')
ax.legend()
ax.set_title(f"van der Pol Oscillator for a={a}")

plt.tight_layout()
plt.show()

```

A somewhat more complex example is the Lane-Emden equation, which is really just Poisson's equation in spherical symmetry for the graviational potential of a self-gravitating fluid whose pressure is related to its density as $P\propto\rho^\gamma$. Such a system is called a _polytrope_, and is often used in astrophysics as a simple model for the structure of system such as a a star in which outward pressure and inward gravity are in equilibrium.

Let $\xi$ be the dimensionless radius of the system, and let $\theta$ be related to the density as
$\rho = \rho_c \theta^n$, where $\rho_c$ is the density at the origin and $n = 1/(\gamma-1)$. We then have the dimensionless second-order differential equation

$$ \frac{1}{\xi^2}\frac{d}{d\xi}\left(\xi^2\frac{d\theta}{d\xi}\right) + \theta^n = 0 $$

Note that the first term is just the divergence $\nabla\cdot\theta$ in spherical symmetry.

If we expand out the first term, we have

$$ \frac{d^2\theta}{d\xi^2} + \frac{2}{\xi}\frac{d\theta}{d\xi} + \theta^n = 0 $$

Defining an auxiliary function $v(\xi) = d\theta/d\xi$, we can then convert this into a system of two first-order ODEs:

\begin{align}
\begin{split}
\frac{dv}{d\xi} &= -\frac{2}{\xi} v - \theta^n \\
\frac{d\theta}{d\xi} &= v
\end{split}
\end{align}

Again, we have "derivatives" only on the LHS and no derivatives on the RHS of our system.

Looking at this expression, one can see right away that at the origin $\xi=0$ we will have a numerical problem; we are dividing by zero.
Analytically, this is not a problem, since $v/\xi\rightarrow0$ as $\xi\rightarrow0$, but here we need to address this numerically.

The first approach is to take care of the problem in our RHS function:

```python
def leRHS(x, y, n):

    dthetadx = y[1]

    if x==0:
        dvdx = -y[0]**n
    else:
        dvdx = -2/x*y[1] - y[0]**n

    return np.array([ dthetadx, dvdx ])

```

This is somewhat clunky, however, and you would first have to convince yourself that in fact $v(\xi)\rightarrow0$ faster than $\xi$ (don't just take my word for it!).

Instead, we could use a more direct RHS function

```python
def leRHS(x, y, n):

    dthetadx = y[1]

    dvdx = -2/x*y[1] - y[0]**n

    return np.array([ dthetadx, dvdx ])
```

and expand the solution in a Taylor series about the origin to get a starting value for our numerical integration at a small distance away from the origin. To do this, write

$$\theta(\xi) = a_0 + a_1 \xi + a_2 \xi^2 + \dots$$

The first thing to notice is that, by symmetry, only even powers of $\xi$ will appear in the solution.
Thus we will have

$$ \theta(\xi) = a_0 + a_2 \xi^2 + a_4 \xi^4 + \dots$$

By the boundary condition $\theta(0) = 1$, we have immediately that $a_0 = 1$.

Next, substitute $\theta(\xi) = 1 + a_2 \xi^2 + a_4 \xi ^4 + O(\xi^6)$ into the Lane-Emden equation. $\theta$ and its first two derivatives are

\begin{align}
\begin{split}
\theta(\xi) &= 1 + a_2 \xi^2 + a_4 \xi^4 + O(\xi^6)\\
\theta'(\xi) &= 2 a_2 \xi + 4 a_4 \xi^3 + O(\xi^5) \\
\theta''(\xi) &= 2 a_2 + 12 a_4 \xi^2 + O(\xi^4)
\end{split}
\end{align}

Putting these into the Lane-Emden equation, we have
\begin{align}
\begin{split}
2 a_2 + 12 a_4 \xi^2 + O(\xi^4) + \frac{2}{\xi} (2 a_2 x + 4 a_4 \xi^3 + O(\xi^5))  &= -\theta^n \\
6 a_2 + 20 a_4 \xi^2 + O(\xi^4) &= -\theta^n
\end{split}
\end{align}

A boundary condition $\theta(0)=1$, and thus we have $a_2 = -1/6$. Away from zero, then, we have

\begin{align}
\begin{split}
-1 + 20 a_4 \xi^2 + O(\xi^4) &= -\left(1 - 1/6 \xi^2 + a_4 \xi^4 + O(\xi^6)\right)^n
\end{split}
\end{align}

The term on the RHS is $ 1 - n \xi^2/6 + O(\xi^4)$, and so we must have $a_4 = n/120$.

Thus, the series expansion of the solution around the origin is

$$ \theta(\xi) = 1 - \frac{1}{6}\xi^2 + \frac{n}{120} \xi^4 + \dots $$

We can now use this expansion to take a first step slightly away from the origin before beginning our
numerical integration, thus avoiding the divide by zero. Note that this series solution near the origin is $O(h^5)$ and so is a good match for RK4 if we take the same (or smaller) step-size.

```python
n = 3

xi0 = 0.01                            # starting value of xi for our numerical integration
theta0 = 1 - xi0**2/6 + n*xi0**4/120  # Taylor series solution to the DE near zero derived above
theta0p = -xi0/3 + n*xi0**3/30
y0 = np.array([ theta0,  theta0p])    # set IC's for numerical integration
print(f"IC at {xi0:10.5e}: {y0[0]:10.5e}, {y0[1]:10.5e}")

h = 0.1
tmax = 8

# call the solver
t, y = odeSolve(xi0, y0, tmax, h, leRHS, RK4_step, n)

fig,ax = plt.subplots()
ax.plot(t,y[:,0],'b', label=r'$\theta(\xi)$')
ax.plot(t,y[:,1],'r--', label=r'$\frac{d\theta}{d\xi}$')
ax.plot([0,tmax],[0,0],'k')
ax.set_xlabel(r'$\xi$')
ax.set_title(f"Lane Emden Equation for n={n}")
ax.legend()

plt.tight_layout()
plt.show()
```

For values of $n\le5$, the solutions of the Lane Emden equation (the so-called Lane-Emden functions of index $n$) decrease to zero at finite $\xi$. Since this is the radius at which the density goes to zero, we can interpret it as the surface of the self-gravitating body (for example, the radius of the star). Knowing this value $\xi_1$ is thus interesting... Let us see how to determine it numerically.

Cleary, we are looking for the solution to $\theta(\xi_1)=0$; this is just root-finding, which we already know how to do. Instead of using some closed-form function, however, the value of the function $\theta(\xi)$ must in this case be determined numerically. But we have just figured out how to do this!

Let's use the bisection method for our root-finding algorithm; here is a quick version (no error checking!)

```python
def bisection(func, low, high, eps, *P):
    flow = func(low, *P)
    fhigh = func(high, *P)
    mid = 0.5*(low+high)
    fmid = func(mid,*P)

    while (high-low)> eps:
        if fmid*flow < 0:
            high = mid
            fhigh = fmid
        else:
            low = mid
            flow = mid

        mid = 0.5*(low+high)
        fmid = func(mid,*P)

    return low
```

Now let us make a function which returns $\theta(\xi)$, the solution to the Lane-Emden equation at $\xi$

```python
def theta(xi, n):
    h = 1e-4
    xi0 = 1e-4
    theta0 = 1 - xi0**2/6 + n*xi0**4/120
    theta0p = -xi0/3 + n*xi0**3/30
    y0 = np.array([ theta0,  theta0p])
    t, y = odeSolve(xi0, y0, xi, h, leRHS, RK4_step, n)

    return y[-1,0]
```

Using these, we can compute the surface radius of the polytrope

```python
n = 3
xi1 = bisection(theta, 6, 8, 1e-5, n)
print(f"xi_1 = {xi1:7.5f}")
```

    xi_1 = 6.89680

A more careful treatment gives a value $\xi_1 = 6.89685...$, so we are doing pretty well...

# Breaktout exercise: Projectile motion

The trajectory of a projectile in a uniform gravity field is a classic and dear physics problem. However, in most introductory physics courses, one works under the assumption that the frictional forces can be neglected. This is of course not always the case, especially not at high speeds. If frictional forces is included, the path of the projectile can no longer be described analytically, and the trajectory must be calculated numerically. To do this, one must solve a coupled set of differential equations, which provides us with an excellent opportunity to implement an ODE-solver on vector form. This turns out to be an elegant and easy implementation, that can be used for solving both differential equations of higher order and coupled differential equations.

## Theory
### Equations of motion
The governing equations for a projectile is derived from Newtons second law, $\vec{F} = m\vec{a}$, where $\vec{F}$ is the force acting on the projectile, $m$ is the mass, and $\vec{a}$ is the acceleration of the ball. In this example there are two forces acting on the ball. The gravitational force

$$\vec{F_G} = m\vec{g},$$

where $\vec{g}$ denotes the the standard gravity. The other force is the drag force

$$\vec{F_D} = -\frac{1}{2}\rho ACv_r^2\hat{v_r}.$$

$A$ is the cross-sectional area, $C$ is the drag coefficient, and $v_r$ is the velocity of the projectile relative to the fluid it is traveling through, $\rho$ is the air density and $\hat{v_r}$ is the direction of the projectile with length unity. A more practical way of writing this equation is

$$\vec{F_D} = - \frac{B}{\rho_0}\rho v_r\left(\vec{v}-\vec{V}\right),$$

where $\rho_0$ is the air density at sea level altitude, $B$ is some constant, $\vec{v}$ is the velocity of the projectile, and $\vec{V}$ is the velocity of the wind.

In this example we will use the shells from the German cannon "the Paris gun" as the projectiles, and the numerical constants mentioned above will be based on historical data of the cannon. The Paris gun operated during World War I and bombarded Paris from far away, the distances often exceeded 100 kilometers, with the velocities of the shells being correspondingly high.

The complete equation for the drag force includes an extra term, which is not mentioned above, namely the term originating from Stokes' law. For a spherical projectile the term takes the form

$$\vec{F_D'} = -6 \pi \eta \vec{v},$$

where $\vec{F_D'}$ denotes the extra term, and $\eta$ denotes the air's viscosity. At high velocities this term is dwarfed by $\vec{F_D}$ as $\vec{F_D}$ is proportional to $v^2$, thus making the drag force $\vec{F_D}$ an excellent approximation for the total drag force.

The last part of the theory needed before we can find the equations of motion is an expression for the air density. As the shells of the Paris gun will reach the stratosphere, it is not reasonable to treat $\rho$ as a constant. In this notebook we will use an adiabatic air density model. The air density is then given by

$$\rho = \rho_0 \left(1 - \frac{ay}{T_0}\right)^\alpha,$$

where $\rho$ and $T_0$ denotes the air density and temperature at sea level, $y$ denotes the altitude, $a$ and $\alpha$ are constants $a= 6.5 \cdot 10^{-3}$ K/m, $\alpha$ = 2.5. At high altitudes the expression in the parentheses will become negative, thus making the air density an imaginary number. To avoid this, the air density is set equal to zero when reaching these altitudes. This critical height is around 44 kilometers, well into the stratosphere.

In a two-dimensional coordinate system, where $x$ and $y$ are the displacement from the origin in horizontal and vertical direction respectively, the equations of motion become
$$
\begin{align}
m\ddot{x} &= -\frac{B}{\rho_0}\rho\left(\dot{x}-V_x\right)\left|\vec{v}-\vec{V}\right|,\\
m\ddot{y} &= -\frac{B}{\rho_0}\rho\left(\dot{y}-V_y\right)\left|\vec{v}-\vec{V}\right|-mg,
\end{align}
$$

where $V_x$ and $V_y$ denotes the velocity of the wind in the horizontal and vertical direction. Simplifying even further, and introducing $u$ and $v$ as the velocities in the horizontal and vertical direction yields four coupled differential equations

\begin{equation}
\begin{aligned}
\dot{x} &= u, \\
\dot{u} &= -\frac{B}{m}\Big[ 1 - \frac{ay}{T_0} \Big]^\alpha \left(u-V_x\right)\Big[ (u-V_x)^2+(v-V_y)^2 \Big]^{1/2},\\
\dot{y} &= w, \\
\dot{w} &= -\frac{B}{m}\Big[ 1 - \frac{ay}{T_0} \Big]^\alpha \left(v-V_y\right)\Big[ (u-V_x)^2+(v-V_y)^2 \Big]^{1/2}-g.\\
\end{aligned}
\label{ODEs}
\end{equation}

### Discretization: Runge-Kutta 4th order

First we define a set of $N$ discrete time values $$
t_n=t_0+n\cdot h~~~~~\mathrm{with}~~~~n=0,1,2,3,...,N.
$$

$h$ is the time step length and is calculated by the formula $h = \frac{t_N-t_0}{N}$, where $t_0$ and $t_N$ denotes the initial and final time values of the projectile's flight. These $N$ time values correspond to $N$ number of $x$, $y$, $u$, and $w$ values, each denoted $x_n$, $y_n$, $u_n$ and $w_n$. A more practical way of denoting this collection of values is on vector form
$$
\vec{w_n} = \begin{bmatrix}x_n \\ y_n \\ u_n \\ v_n\end{bmatrix}.
$$

This notation makes it possible to implement the Runge-Kutta method in an elegant way. It is always possible to reduce an ordinary differential equation of order $q$ to a problem of solving a set of two $q-1$ order coupled differential equations. We have a coupled set of two second order differential equations, thus we need to solve a set of four first order coupled differential equations. By introducing a function $f$ that transforms $\vec{w_n}$ in the way described by equation \eqref{ODEs}, one can implement the Runge-Kutta method directly. In our case, $f$ is given by

$$
f(\vec{w_n} ) = f \begin{bmatrix}x_n \\ y_n \\ u_n \\ v_n\end{bmatrix} = \begin{bmatrix}u_n \\ v_n \\ -\frac{B}{m}\Big[ 1 - \frac{ay_n}{T_0} \Big]^\alpha \left(u_n-V_x\right)\Big[ (u_n-V_x)^2+(v_n-V_y)^2 \Big]^{1/2} \\ -\frac{B}{m}\Big[ 1 - \frac{ay_n}{T_0} \Big]^\alpha \left(v_n-V_y\right)\Big[ (u_n-V_x)^2+(v_n-V_y)^2 \Big]^{1/2}-mg\end{bmatrix},
$$
with no time dependence, such that
$$
\dot{\vec{w_n}}=f(\vec{w_n})
$$

describes the complete system of differential equations. The following code implements this, and can easily be expanded to solve either an even higher order ODE or an ODE describing a three-dimensional problem.

```python
# Importing necessary packages
import numpy as np
import matplotlib.pyplot as plt
from matplotlib import rc
%matplotlib inline

newparams = {'figure.figsize': (15, 7), 'axes.grid': False,
             'lines.markersize': 10, 'lines.linewidth': 2,
             'font.size': 15, 'mathtext.fontset': 'stix',
             'font.family': 'STIXGeneral', 'figure.dpi': 200}
plt.rcParams.update(newparams)

# Constants

g = 9.81             # Force of gravity per kilo on earth's surface
alpha = 2.5          # Parameter in the adiabatic air density model
a = 6.5 * 10 ** (-3) # Parameter in the adiabatic air density model
T_0 = 288            # Temperature at sea level
m = 50               # Mass of the projectile
B = 2 * 10 ** (-3)   # Constant based on the Paris gun
v_0 = 1640           # Initial velocity

# Wind strength. This is a constant that never varies
V = np.zeros(2) # Starting of with no wind

def f(t,w,*P):
    """ A function describing the right hand side of the equations
    of motion/the set of ODEs.
    Parameter:
        t       (time parameter, not needed in this case)
        w       vector containg the needed coordinates and their velocities
    """
    # V : the vector describing the wind strength and direction
    temp_vector = np.zeros(4)
    temp_vector[0] = w[2]
    temp_vector[1] = w[3]
    # Saving the next parameters in order to optimize run time
    k = (1 - a * w[1] / T_0)   # Air density factor
    s = np.sqrt((w[2]-V[0]) ** 2 + (w[3]-V[1]) ** 2 )
    if k>0:
        temp_vector[2] = - B / m * k ** (alpha) * (w[2]-V[0]) * s
        temp_vector[3] = - B / m * k ** (alpha) * (w[3]-V[1]) * s - g
    else:
        temp_vector[2] = 0
        temp_vector[3] = - g
    return temp_vector
```

Now we can finally start to explore the trajectories of the Paris gun. The functions below will be helpful going forward.

```python
def shoot(theta, v0):
    """ Initializes the vector w (x and y position and velocity of the
    projectile) given a initial shooting angle, theta, and
    absolute velocity, v0.
    """
    w = np.zeros(4)
    w[2] = v0 * np.cos(np.deg2rad(theta))
    w[3] = v0 * np.sin(np.deg2rad(theta))
    return w

def projectile_motion(h, theta):
    """ Calculates the motion of the projectile using the functions
    defined above. While the projectile is in the air (w[1] >=0) the
    position and velocity is updated using a single step of RK4.
    Parameters:
        h        unit/integration step length
        theta    initial shooting angle
    Returns:
        X_list   array of the projectile's x-position
        Y_list   array of the porjectile's y-position
    """
    w = shoot(theta, v_0)
    X_list = np.zeros(0)
    Y_list = np.zeros(0)
    while  w[1] >= 0:
        w = RK4_step(0, w, h,f,0)
        X_list = np.append(X_list, w[0])
        Y_list = np.append(Y_list, w[1])
    return X_list, Y_list
```

### Finding the optimal launch angle
With no drag force, it is trivial to show that the angle maximizing the range of a projectile is $\theta = 45^\circ$, where $\theta$ is the angle between the horizontal axis and the direction of the initial velocity. It is interesting to se how the air drag affects this maximum angle, both with and without wind. The following code finds the optimal angle to one decimal precision, and shows how the range depends on the firing angle. It is also instructive to see some of the trajectories of the projectile, so some of the trajectories are included.

```python
def find_optimal_angle(h):
    """ Given an integration time step, this function calculates the optimal initial
    shooting angle for the projectile to obtain maximum range, in x-direction. The
    set of angles tested, with their corresponding range, along with the optimal
    angle are returned.
    """

    record = 0        # Placeholder variable that holds the maximum range
    optimal_angle = 0 # Placeholder variable that holds the angle yielding the maximum range
    # Lists containing the initial angle and its corresponding range
    theta_list = np.zeros(0)
    range_list = np.zeros(0)
    for theta in range (1,90,2):
        x_list, y_list = projectile_motion(h, theta)
        # Using linear interpolation do determine the landing point more precisely
        m = (y_list[-1] - y_list[-2]) / (x_list[-1] - x_list[-2])   # The landing point
        x_range = - y_list[-1] / m + x_list[-1]
        theta_list = np.append(theta_list, theta)
        range_list = np.append(range_list, x_range)
        # Update records
        if x_range >= record:
            record = x_range
            optimal_angle = theta

    # Rerunning the same code on a smaller interval in order to approximate the optimal angle
    # more precicely
    return theta_list, range_list, optimal_angle

theta, x , best = find_optimal_angle(0.1)

print("The optimal angle is: ", best, " degrees")
```

    The optimal angle is:  55  degrees

```python
plt.plot(theta, x/1000)
plt.title(r"Projectile range as a function of shooting angle, $\theta$")
plt.xlabel(r"$\theta $ [$\degree$]")
plt.ylabel(r"range [km]")

def trajectories(h):
    plt.figure()
    plt.title("Projectile trajectories by alternating shooting angle")
    plt.xlabel(r"x [km]")
    plt.ylabel(r"y [km]")
    theta_list = np.arange(30.0,75,5)
    for angle in theta_list:
        x_list, y_list = projectile_motion(h, angle)
        plt.plot(x_list/1000, y_list/1000, '--', label=r'$\theta = $%.i $\degree$'%(angle))
    plt.legend(loc='best')
    plt.gca().set_ylim(bottom=0)
    plt.show()
trajectories(0.1)
```

In contrast to projectile motion with no friction, maximum range is not obtained with firing angle $\theta = 45^\circ$. Because  the drag force is lower at higher altitudes, the range increases when the projectile spends the majority of it's flight at higher altitudes. This explains why the maximum range is obtained at $\theta = 55^\circ$.

### Numerical validity

In order to determine the validity of the numerical implementation, we can use conservation of energy. During the projectile's trajectory the drag force $\vec{F_D}$ will do work on the projectile. The cumulative work done on the projectile is given by

$$
W = \int \vec{F_D} \cdot \vec{dr}.
$$

In our discrete model, we can easily calculate this integral by summing over all the infinitesimal contributions, and using the fact that $\vec{F_D}$ always is antiparallel with $\vec{dr}$ such that the cumulative work at time $t_M$, $M<N$, is equal to

$$
W = \sum_{n=1}^{M} |\vec{F_{D_{n-1}}}| \cdot |\vec{r_n} - \vec{r_{n-1}}|,
$$
where $\vec{F_{D_{n-1}}}$ denotes the drag force at position $\vec{r_{n-1}}$. However, this can again be written to the form

$$
W = h \sum_{n=1}^{M} |\vec{F_{D_{n-1}}}| \cdot |\vec{v_{n-1}}|,
$$
which is the most convenient for our purposes. It must be noted that the choice of  index that is used for the force and speed respectively is non-trivial, as they both will vary during the interval $dr$. Here both the speed of the projectile and the force at point $\vec{r_{n-1}}$ is used. This is mainly a pragmatic choice, as this combination yields the lowest, and thus the most realistic energy loss. Thus we can finally express the energy of the projectile (which of course is equal to it's kinetic and potential energy) and the work done on the air at time $t_M$

$$
E(t_M) = h \sum_{n=1}^{M} |\vec{F_{D_{n-1}}}| \cdot |\vec{v_{n-1}}| + \frac{1}{2} m v_m^2 + mgy_m,
$$

where $v_m$ is the absolute velcoity of the projectile, and $y_m$ is the height of the projectile over the ground (zero potential reference point).

Now all that remains is to check if this expression for the energy indeed remains constant while it is in the air.

In this notebook, the Runge-Kutta method is implemented without much difficulties. However, it is interesting to see whether or not it was necessary to use a 4th order differential equation solver, instead of using the most primitive ODE solver, Euler's method. The following code will use both Runge-Kutta and Euler's method, and compare the energy loss during the motion for both of them.

```python
def euler_step(t,w ,h, *P):
    """Simple implementation of Euler's method on vector form
    """
    return w + h * f(t,w)

def energy_conservation(h, theta, func):
    """ This function performs the same steps as "projectile_motion()" and "f()"
    but extracts the value for the drag, F_D, at every step. This enables the
    calculation of work and storage of energy at each time step.
    Parameters:
        h           integration time step
        theta       initial shooting angle
        func        a function defining the integration step (euler or RK4)
    Returns:
        x_list      array of the trajectory x-coordinates
        y_list      array of the trajectory y-coordinates
        time_steps  array of the elapsed time at each step
        energy      array of the energy at each step
        work        the total work by the drag force
    """
    w = shoot(theta, v_0)

    x_list = np.zeros(1)
    y_list = np.zeros(1)
    work = 0

    # Initial total energy
    v = np.sqrt(w[2] ** 2 + w[3] ** 2)
    energy = np.array([0.5 * m * v ** 2])

    while w[1] >= 0:
        w = func(0, w, h,f,0)

        # Updating lists
        x_list = np.append(x_list, w[0])
        y_list = np.append(y_list, w[1])

        # Temporary parameters for checking if high altitude
        k = (1 - a * y_list[-2] / T_0)
        s = np.sqrt((w[2] - V[0]) ** 2 + (w[3] - V[1]) ** 2)
        if k > 0:
            F_D = B * k ** (alpha) * s ** 2
        else:
            F_D = 0

        # Add the amount of work done by drag this time interval
        work +=  v * F_D * h
        # Updating the speed in order to calculate the total energy
        v = np.sqrt(w[2] ** 2 + w[3] ** 2)
        # Use np.sum on the work_list to get the cumulative work
        energy = np.append(energy, 0.5 * m * v ** 2 + m * g * w[1] + work)

    n = len(x_list)
    time_steps = np.linspace(0, n * h, n)

    return x_list, y_list, time_steps, energy, work

# Setting initial values
theta = 45
h = 0.1
# Getting the results
x_list_euler, y_list_euler, time_steps_euler, energy_euler, work = energy_conservation(h, theta, euler_step)
x_list_RK4, y_list_RK4, time_steps_RK4, energy_RK4, work = energy_conservation(h, theta, RK4_step)

print("At firing angle %.1f degrees and with time step h = %.2f seconds:" %(theta, h))
print("Energy lost with RK4: %.3E = %.2f percent of total energy" %(energy_RK4[0] - energy_RK4[-1],
                                          (energy_RK4[0] - energy_RK4[-1]) / energy_RK4[0] * 100))
print("Energy lost with Euler's method: %.3E = %.2f percent of total energy" %(energy_euler[0] - energy_euler[-1],
                                        (energy_euler[0] - energy_euler[-1]) / energy_euler[0] * 100))

# Plotting the separate trajectories
plt.plot(x_list_euler/1000, y_list_euler/1000, label ="Euler")
plt.plot(x_list_RK4/1000, y_list_RK4/1000, label ="RK4")
plt.title("Firing angle = " + str(theta) + "$\degree$")
plt.xlabel(r"$x$ [km]")
plt.ylabel(r"$y$ [km]")
plt.gca().set_ylim(bottom=0)
plt.legend()
plt.show()
```

From the figure above, it is clear that Euler's method work surprisingly well, even though it us just ODE-solver of first order. It produces nearly the same trajectory as Runge Kutta 4th method. The most likely explanation of the small difference between Runge Kutta 4th order and Euler's method is that the coupled set of ODEs we are solving are quite linear, in other words, the derivatives vary quite slowly. Thus Euler's method produce satisfactory results, even though it is only of first order.

It might seem like loosing around $10^3$ joules is a lot, but one needs to remember the enormous dimensions of the Paris gun, and that this is a small fraction of the original energy of the projectile.

```python

```


```python
from IPython.display import Image
from IPython.core.display import HTML
Image(url= "https://i.imgur.com/WBsZZjO.png") # power rule of exponents + an estimation
```

```python
Image(url= "https://i.imgur.com/OKzEasd.png")
```

```python
from sympy import *
from sympy import euler
x = symbols('x')
c = symbols('c', real = True)
sympify(c).is_real
```

    True

```python
eq5 = c*E**(x)
Eq(eq5,100) #solving for c when x = 10 because the plot looks nice
```

$\displaystyle c e^{x} = 100$

```python
plot(E**(x-5.4)) # after plotting this I noticed that it would be nice if this y= 100 right when x = 10
                 # the final solution may be useful for conceptualizing percentage of exponential growths
```

```python
Image(url= "https://i.imgur.com/F2uZGSM.png")
```

```python
eq11 = Eq(c*E**(10),100)
eq11
# This is what I want to solve.
```

$\displaystyle c e^{10} = 100$

```python
eq1 = 0.00453999208646404*exp(x)
Eq(eq1,eq1.subs(x,10).evalf())
```

$\displaystyle 0.00453999208646404 e^{x} = 99.9999804011934$

```python
eq2 = 0.00454*exp(x)
Eq(eq2,eq2.subs(x,10).evalf())
```

$\displaystyle 0.00454 e^{x} = 100.000154708422$

```python
p0 = plot(eq1,eq2,show = False,xlim = (0,10),size = (12,5),legend = True,autoscale = True)
p0[0].line_color = 'r'
p0[1].line_color = 'b'
p0.show()
```

```python
s = solve(eq11)[0]
s # one possible solution for real number c when x = 10
```

$\displaystyle \frac{100}{e^{10}}$

```python
s.evalf()
```

$\displaystyle 0.00453999297624849$

```python
final_solution = (s*E**x)
phi = symbols('phi')
Eq(phi,final_solution)
```

$\displaystyle \phi = \frac{100 e^{x}}{e^{10}}$

```python
final_solution.subs(x,10)
```

$\displaystyle 100$

```python
final_equation = s.evalf()*E**x
Eq(final_equation,100)
```

$\displaystyle 0.00453999297624849 e^{x} = 100$

```python
(0.00453999297624849*E**10).evalf()
```

$\displaystyle 100.0$

```python
p0 = plot(eq1,eq2,final_solution,show = False,xlim = (0,10),size = (12,5),legend = True,autoscale = True)
p0[0].line_color = 'r'
p0[1].line_color = 'b'
p0[2].line_color = 'g'
p0.show()
```

```python
eq14 = Eq(0.00453999297624849*E**(x),100)
solve(eq14)
```

    [10.0000000000000]

```python
0.00453999208646404 < 0.00453999297624849 < 0.00454
```

    True


#  Gaussian Mixture Models (GMM) tutorial
Followed from: [link](https://www.python-course.eu/expectation_maximization_and_gaussian_mixture_models.php)

- GMM algorithm is an unsupervised clustering algorithm.
- It's constituents are Gaussian distributions.
- The algorithm tries to fit a mixture of Gaussians to the training set.
- While SGD could be used to build a GMM, Expectation Maximization (EM) offers better performance at the same task.
- The EM algorithm has two main parts:
    1. E-Step: for calculating the densities of the datapoints.
    2. M-Step: for updating or estimating the parameters of the GMM.

## Initial Step
1. Decide how many sources or clusters (c) in the dataset.
2. Initialize the parameters mean $\mu_c$, covariance $\Sigma_c$, and a cluster weights $\pi_c$, subject to $\sum_c\pi_c=1$

##  E - Step
Calculate for each datapoint $x_i$ the probability $r_{ic}$ that datapoint $x_i$ belongs to cluster c with:
\begin{equation}
r_{ic}= \frac{\pi_cN\left(x_i\mid\mu_c,\Sigma_c\right)}{\Sigma_{k=1}^K\pi_kN\left(x_i\mid\mu_k,\Sigma_k\right)}
\end{equation}
where $N\left(x\mid\mu,\sum\right)$ describes the multivariate Gaussian with:
\begin{equation}
N\left(x\mid\mu,\Sigma\right)=\frac{1}{(2\pi)^{\frac{n}{2}}\mid\Sigma_c\mid^{\frac{1}{2}}}exp\left(-\frac{1}{2}\left(x_i-\mu_c\right)^T\Sigma^{-1}_c\left(x_i-\mu_c\right)\right)
\end{equation}

MathJax.Hub.Config({
    TeX: { equationNumbers: { autoNumber: "AMS" } }
});

##  M - Step
For each cluster $c$:
1. Calculate the total weight $m_c$, the fraction of points allocated to cluster $c$ and update the GMM parameters with $r_{ic}$:
\begin{align}
m_c &= \sum_ir_{ic}\\
\pi &= \frac{m_c}{m}\\
\mu_c &= \frac{1}{m_c}\sum_ir_{ic}x_i\\
\Sigma_c&=\frac{1}{m_c}\sum_ir_{ic}(x_i-\mu_c)^T(x_i-\mu_c)
\end{align}

**The updated means are used in the last formula**. Repeat E and M steps till convergence.
### Likelihood computation / loss function
Log likelihood:
\begin{equation}
ln~p(X\mid\pi,\mu,\Sigma)=\sum_{i=1}^Nln\left(\sum^K_{k=1}\pi_kN\left(x_i\mid\mu_k,\Sigma_k\right)\right)
\end{equation}

##  Implementation with sklearn

```python
import matplotlib.pyplot as plt
from matplotlib import style
from sklearn.datasets.samples_generator import make_blobs
import numpy as np
from scipy.stats import multivariate_normal
from sklearn.mixture import GaussianMixture
```

```python
style.use('seaborn')
```

###  Create dataset

```python
X, Y = make_blobs(cluster_std=0.5, random_state=20, n_samples=1000, centers=5)
```

###  Stretch dataset to get ellipsoid data

```python
X = np.dot(X, np.random.RandomState(0).randn(2,2))
x, y = np.meshgrid(np.sort(X[:, 0]), np.sort(X[:, 1]))
```

```python
XY = np.array([x.flatten(), y.flatten()]).T
```

```python
GMM = GaussianMixture(n_components=5).fit(X) # instantiate and fit the model
print('Converged:', GMM.converged_) # check if the model has converged
```

    Converged: True

```python
means = GMM.means_
```

```python
means
```

    array([[ -0.49282149, -12.37705278],
           [ 10.94510475,  18.59634417],
           [-12.56857912,   4.85978424],
           [ -3.92027834,  -0.16528677],
           [ 19.85757916,  17.34078884]])

```python
covariances = GMM.covariances_
```

```python
print(str(covariances))
```

    [[[1.02906033 0.8413495 ]
      [0.8413495  1.5081849 ]]

     [[0.96755872 0.79622775]
      [0.79622775 1.43355228]]

     [[0.94633179 0.69274809]
      [0.69274809 1.38106594]]

     [[0.95004033 0.72007886]
      [0.72007886 1.37181252]]

     [[0.86698786 0.56596329]
      [0.56596329 1.09824226]]]

```python
# predict
Y = np.array([[0.5],[0.5]])
prediction = GMM.predict_proba(Y.T)
print(prediction)
```

    [[4.00098007e-33 5.02664415e-44 9.36305075e-82 1.00000000e+00
      1.94756664e-93]]

### Plot

```python
fig = plt.figure(figsize=(10,10))
ax0 = fig.add_subplot(111)
ax0.scatter(X[:,0],X[:,1])
ax0.scatter(Y[0,:],Y[1,:],c='orange',zorder=10,s=100)
for m,c in zip(means,covariances):
    multi_normal = multivariate_normal(mean=m,cov=c)
    ax0.contour(np.sort(X[:,0]),np.sort(X[:,1]),multi_normal.pdf(XY).reshape(len(X),len(X)),colors='black',alpha=0.3)
    ax0.scatter(m[0],m[1],c='grey',zorder=10,s=100)

plt.show()
```

```python
XY.shape
```

    (1000000, 2)

```python

```


Solution to: [Day 0: Weighted Mean](https://www.hackerrank.com/challenges/s10-weighted-mean)

Table of Contents

```javascript
%%javascript
$.getScript('https://kmahelona.github.io/ipython_notebook_goodies/ipython_notebook_toc.js')
```

    <IPython.core.display.Javascript object>

- Table of Contents
    - Wiki definition
    - Equation
- Solution
    - Imports
    - Constants
    - Input
    - Weighted mean
    - Scale
    - Main

## Wiki definition
The weighted arithmetic mean is similar to an ordinary arithmetic mean (the most common type of average),
except that instead of each of the data points contributing equally to the final average, some data points contribute more than others.

The notion of weighted mean plays a role in descriptive statistics and also occurs in a more general form in several other areas of mathematics.

If all the weights are equal, then the weighted mean is the same as the arithmetic mean.

## Equation

\begin{equation}
\huge
\mu_{w} = \frac
{\sum_{i=1}^{n}(x_{i} * w_{i})}
{\sum_{i=1}^{n}w_{i}}
\end{equation}

# Solution

## Imports

```python
from typing import List, Tuple, Union
```

## Constants

```python
SCALE = 1	# number of decimals
```

## Input

```python
def get_input() -> Tuple[int, list, list]:
	"""Returns 3 variabless:
		n: int, length of list
		nums: list, numbers to calculate
		weights: list, weight of numbers.
		"""
	n = int(input())
	nums = [int(val) for val in input().split()]
	weights = [int(val) for val in input().split()]
	return (n, nums, weights)
```

## Weighted mean

```python
def calc_weighted_mean(n: int, nums: List[int], weights: List[int]) -> float:
	"""Returns weighted mean from list of nums and weights with length n."""
	assert (len(nums) == n) and (len(weights) == n)

	weighted_total = 0
	for i in range(n):
		weighted_total += nums[i] * weights[i]

	return format_scale( weighted_total / sum(weights))
```

## Scale

```python
def format_scale(num: Union[int, float]) -> Union[int, float]:
	if isinstance(num, int):
		return num

	return round(num, 1)
```

## Main

```python
def main():
	n, nums, weights = get_input()
	print( calc_weighted_mean(n, nums, weights))
```

```python
if __name__ == "__main__":
	main()
```

    5
    10 40 30 50 20
    1 2 3 4 5
    32.0


# Continued rational function approximations
> Mixing continued fractions with Taylor approximations.

- toc: false
- badges: true
- comments: true
- categories: [math, jupyter]

```python
#hide_input
import numpy as np
import matplotlib.pyplot as plt
```

A Taylor series of a function is kind of like a decimal representation of a real number, where instead of base 10 we use "base x" to represent functions of x.

What happens if we replace "decimal representation" with "[continued fraction](https://en.wikipedia.org/wiki/Continued_fraction) representation"?

## Warm-up: Taylor approximations

Say we want to approximate $y = \log(1 + x)$ near x = 0. Here's a graph of this function:

```python
#hide_input
x = np.linspace(-0.5, 0.5)
y = np.log1p(x)

plt.plot(x, y)
plt.axhline(0, color='black')
plt.axvline(0, color='black')
```

Our first order approximation is $y = x$. This matches the slope and intercept at x = 0:

```python
#hide_input
plt.plot(x, y)
plt.plot(x, x)
plt.axhline(0, color='black')
plt.axvline(0, color='black')
```

Suppose now we want more terms in the Taylor approximation. We could take derivatives and plug it into the general formula for Taylor series but here's a more heuristic approach that leads to the same result (which we will modify later).

If $ y \approx x $, then $ y/x \approx 1 $. Define $z_1 = y/x$ and plot it as a function of x. We find that we can get a better approximation for $z_1$ by matching *its* slope at x = 0:

```python
#hide_input
plt.plot(x, y/x)
plt.plot(x, 1 - x/2)
plt.axvline(0, color='black')
```

Plugging in this approximation for $z_1$ and working backward to solve for $y$ this gives us a second order Taylor approximation.

$$ \begin{align}
y/x &= z_1 \approx 1 - x/2 \\
y &= xz_1 \\
y &\approx x(1 - x/2) \\
&\approx x - x^2/2
\end{align} $$

```python
#hide_input
plt.plot(x, y)
plt.plot(x, x - x*x/2)
plt.axhline(0, color='black')
plt.axvline(0, color='black')
```

We can keep going to get the third term.

$$ \begin{align}
y &\approx x + x^2(-1/2) \\
y &= x + x^2 z_2 \\
y/x &= 1 + xz_2 \\
y/x - 1 &= xz_2 \\
(y/x - 1)/x &= z_2
\end{align} $$

so we plot $(y/x - 1)/x$ and once again match its slope

```python
#hide_input
plt.plot(x, (y/x-1)/x)
plt.plot(x, -1/2 + x/3)
plt.axvline(0, color='black')
```

and plug it back in to get the third order Taylor expansion

$$ \begin{align}
y &= x + x^2 z_2 \\
y &\approx x + x^2 (-1/2 + x/3) \\
&= x - x^2/2 + x^3/3
\end{align} $$

```python
#hide_input
plt.plot(x, y)
plt.plot(x, x - x*x/2 + x*x*x/3)
plt.axhline(0, color='black')
plt.axvline(0, color='black')
```

And we can keep going like this to get more and more terms in the Taylor series.

To summarize what we've been doing so far:

* Approximate a function as constant + linear in x
* Subtract out the constant and divide what's left by x, to get something approximately constant
* Replace "constant" with "constant + linear" and repeat

## Enter continued fractions

Now to this basic procedure we're going to make one change:

* Approximate a function as constant + linear in x
* Subtract out the constant and **divide x by what's left**, to get something approximately constant
* Replace "constant" with "constant + linear" and repeat

This is inspired by continued fractions as it's analogous to how you would generate a continued fraction expansion for a real number.

Let's see what happens when we apply the modified procedure to our function from before. In the first step we have $y \approx x$. Last time we went from this to $y = xz_1$, now let's do $y = x/z_1$ and plot $z_1 = x/y$:

```python
#hide_input
plt.plot(x, x/y)
plt.plot(x, 1 + x/2)
plt.axvline(0, color='black')
```

We can approximate $z_1$ better with 1 + x/2. So

$$ \begin{align}
y &= x/z_1 \\
y &\approx x/(1 + x/2)
\end{align} $$

And that's our second order continued fraction approximation!

```python
#hide_input
plt.plot(x, y)
plt.plot(x, x/(1 + x/2))
plt.axhline(0, color='black')
plt.axvline(0, color='black')
```

Continuing,

$$ \begin{align}
y &\approx x/(1 + x/2) \\
y &= x/(1 + x/z_2) \\
x/y &= 1 + x/z_2 \\
x/y - 1 &= x/z_2 \\
x/(x/y - 1) &= z_2
\end{align} $$

so we plot and approximate $x/(x/y - 1)$

```python
#hide_input
plt.plot(x, x/(x/y - 1))
plt.plot(x, 2 + x/3)
plt.axvline(0, color='black')
```

and plug it back in to get the third order continued fraction approximation

$$ \begin{align}
y &= x/(1 + x/z_2) \\
y &\approx x/(1 + x/(2 + x/3))
\end{align} $$

```python
#hide_input
plt.plot(x, y)
plt.plot(x, x/(1 + x/(2 + x/3)))
plt.axhline(0, color='black')
plt.axvline(0, color='black')
```

It seems to work pretty well! I don't have a proof of convergence or anything, but for this example, it seems to produce better approximations than the same order Taylor series.

Here's a bigger plot of the second order Taylor and continued fraction approximations, together:

```python
#hide_input
plt.figure(figsize=(15, 10))
plt.plot(x, x - x*x/2, color='#ff7f00', label="2nd order Taylor approximation")
plt.plot(x, x/(1 + x/2), color='#00cc00', label="2nd order continued fraction approximation")
plt.plot(x, y, label="y = ln(1 + x)")
plt.axhline(0, color='black')
plt.axvline(0, color='black')
plt.legend()
```

And the relative error of the second order and third order approximations. In both cases, continued fraction (green) beats Taylor expansion (orange).

```python
#hide_input
x = np.linspace(-0.5, 0.5, 200)
y = np.log1p(x)

plt.figure(figsize=(15, 10))
plt.plot(x, np.abs((x - x*x/2)/y-1), color='#ff7f00', label="2nd order Taylor approximation")
plt.plot(x, np.abs((x/(1 + x/2))/y-1), color='#00cc00', label="2nd order continued fraction approximation")
plt.ylabel('relative error')
plt.yscale('log')
plt.axhline(0, color='black')
plt.axvline(0, color='black')
plt.legend()

plt.figure(figsize=(15, 10))
plt.plot(x, np.abs((x - x*x/2 + x*x*x/3)/y-1), color='#ff7f00', label="3rd order Taylor approximation")
plt.plot(x, np.abs((x/(1 + x/(2 + x/3)))/y-1), color='#00cc00', label="3rd order continued fraction approximation")
plt.ylabel('relative error')
plt.yscale('log')
plt.axhline(0, color='black')
plt.axvline(0, color='black')
plt.legend()
```

We can do this for other functions too! Here's $y = e^x$

```python
#hide_input
x = np.linspace(-1, 1)
y = np.exp(x)
plt.figure(figsize=(15, 10))
plt.plot(x, y, label="y = e^x")
plt.plot(x, 1 + x/(1 - x/2), label="2nd order continued fraction approximation")
plt.plot(x, 1 + x/(1 - x/(2 + x/3)), label="3rd order continued fraction approximation")
plt.axvline(0, color='black')
plt.legend()
```

And $y = \sin(x)$

```python
#hide_input
x = np.linspace(-2, 2)
y = np.sin(x)
plt.figure(figsize=(15, 10))
plt.plot(x, y, label="y = sin(x)")
plt.plot(x, x, label="1st order Taylor (or continued fraction) approximation")
plt.plot(x, x/(1 + x*x/6), label="3rd order continued fraction approximation")
plt.axvline(0, color='black')
plt.legend()
```

In the case of $y = \sin(x)$, this seems to be worse than the same order Taylor approximation. Is there a theory for when this method does well?

I searched for information about this and I came across [Padé approximant](https://en.wikipedia.org/wiki/Pad%C3%A9_approximant). Does the procedure I've described compute a Padé approximant?