# Tutorial on Python for scientific computing

Marcos Duarte

This tutorial is a short introduction to programming and a demonstration of the basic features of Python for scientific computing. To use Python for scientific computing we need the Python program itself with its main modules and specific packages for scientific computing. [See this notebook on how to install Python for scientific computing](http://nbviewer.ipython.org/github/demotu/BMC/blob/master/notebooks/PythonInstallation.ipynb).
Once you get Python and the necessary packages for scientific computing ready to work, there are different ways to run Python, the main ones are:

- open a terminal window in your computer and type `python` or `ipython` that the Python interpreter will start
- run the IPython notebook and start working with Python in a browser
- run Spyder, an interactive development environment (IDE)
- run the IPython qtconsole, a more featured terminal
- run IPython completely in the cloud with for example, [https://cloud.sagemath.com](https://cloud.sagemath.com) or [https://www.wakari.io](https://www.wakari.io)
- run Python online in a website such as [https://www.pythonanywhere.com/](https://www.pythonanywhere.com/)
- run Python using any other Python editor or IDE

We will use the IPython Notebook for this tutorial but you can run almost all the things we will see here using the other forms listed above.

## Python as a calculator

Once in the IPython notebook, if you type a simple mathematical expression and press `Shift+Enter` it will give the result of the expression:

```python
1 + 2 - 30
```

    -27

```python
4/5
```

    0.8

If you are using Python version 2.x instead of Python 3.x, you should have got 0 as the result of 4 divided by 5, which is wrong! The problem is that for Python versions up to 2.x, the operator '/' performs division with integers and the result will also be an integer (this behavior was changed in version 3.x).

If you want the normal behavior for division, in Python 2.x you have two options: tell Python that at least one of the numbers is not an integer or import the new division operator (which is inoffensive if you are already using Python 3), let's see these two options:

```python
4/5.
```

    0.8

```python
from __future__ import division
```

```python
4/5
```

    0.8

I prefer to use the import division option (from future!); if we put this statement in the beginning of a file or IPython notebook, it will work for all subsequent commands.

Another command that changed its behavior from Python 2.x to 3.x is the `print` command.
In Python 2.x, the print command could be used as a statement:

```python
print 4/5
```

With Python 3.x, the print command bahaves as a true function and has to be called with parentheses. Let's also import this future command to Python 2.x and use it from now on:

```python
from __future__ import print_function
```

```python
print(4/5)
```

    0.8

With the `print` function, let's explore the mathematical operations available in Python:

```python
print('1+2 = ', 1+2, '\n', '4*5 = ', 4*5, '\n', '6/7 = ', 6/7, '\n', '8**2 = ', 8**2, sep='')
```

    1+2 = 3
    4*5 = 20
    6/7 = 0.8571428571428571
    8**2 = 64

And if we want the square-root of a number:

```python
sqrt(9)
```

We get an error message saying that the `sqrt` function if not defined. This is because `sqrt` and other mathematical functions are available with the `math` module:

```python
import math
```

```python
math.sqrt(9)
```

    3.0

```python
from math import sqrt
```

```python
sqrt(9)
```

    3.0

## The import function

We used the command '`import`' to be able to call certain functions. In Python functions are organized in modules and packages and they have to be imported in order to be used.

A module is a file containing Python definitions (e.g., functions) and statements. Packages are a way of structuring Python’s module namespace by using “dotted module names”. For example, the module name A.B designates a submodule named B in a package named A. To be used, modules and packages have to be imported in Python with the import function.

Namespace is a container for a set of identifiers (names), and allows the disambiguation of homonym identifiers residing in different namespaces. For example, with the command import math, we will have all the functions and statements defined in this module in the namespace '`math.`', for example, '`math.pi`' is the π constant and '`math.cos()`', the cosine function.

By the way, to know which Python version you are running, we can use one of the following modules:

```python
import sys
sys.version
```

    '3.4.1 |Anaconda 2.0.1 (64-bit)| (default, May 19 2014, 13:02:30) [MSC v.1600 64 bit (AMD64)]'

And if you are in an IPython session:

```python
from IPython import sys_info
print(sys_info())
```

    {'commit_hash': '681fd77',
     'commit_source': 'installation',
     'default_encoding': 'cp1252',
     'ipython_path': 'C:\\Anaconda3\\lib\\site-packages\\IPython',
     'ipython_version': '2.1.0',
     'os_name': 'nt',
     'platform': 'Windows-7-6.1.7601-SP1',
     'sys_executable': 'C:\\Anaconda3\\python.exe',
     'sys_platform': 'win32',
     'sys_version': '3.4.1 |Anaconda 2.0.1 (64-bit)| (default, May 19 2014, '
                    '13:02:30) [MSC v.1600 64 bit (AMD64)]'}

The first option gives information about the Python version; the latter also includes the IPython version, operating system, etc.

## Object-oriented programming

Python is designed as an object-oriented programming (OOP) language. OOP is a paradigm that represents concepts as "objects" that have data fields (attributes that describe the object) and associated procedures known as methods.

This means that all elements in Python are objects and they have attributes which can be acessed with the dot (.) operator after the name of the object. We already experimented with that when we imported the module `sys`, it became an object, and we acessed one of its attribute: `sys.version`.

OOP as a paradigm is much more than defining objects, attributes, and methods, but for now this is enough to get going with Python.

## Python and IPython help

To get help about any Python command, use `help()`:

```python
help(math.degrees)
```

    Help on built-in function degrees in module math:

    degrees(...)
        degrees(x)

        Convert angle x from radians to degrees.

Or if you are in the IPython environment, simply add '?' to the function that a window will open at the bottom of your browser with the same help content:

```python
math.degrees?
```

And if you add a second '?' to the statement you get access to the original script file of the function (an advantage of an open source language), unless that function is a built-in function that does not have a script file, which is the case of the standard modules in Python (but you can access the Python source code if you want; it just does not come with the standard program for installation).

So, let's see this feature with another function:

```python
import scipy.fftpack
scipy.fftpack.fft??
```

To know all the attributes of an object, for example all the functions available in `math`, we can use the function `dir`:

```python
print(dir(math))
```

    ['__doc__', '__loader__', '__name__', '__package__', '__spec__', 'acos', 'acosh', 'asin', 'asinh', 'atan', 'atan2', 'atanh', 'ceil', 'copysign', 'cos', 'cosh', 'degrees', 'e', 'erf', 'erfc', 'exp', 'expm1', 'fabs', 'factorial', 'floor', 'fmod', 'frexp', 'fsum', 'gamma', 'hypot', 'isfinite', 'isinf', 'isnan', 'ldexp', 'lgamma', 'log', 'log10', 'log1p', 'log2', 'modf', 'pi', 'pow', 'radians', 'sin', 'sinh', 'sqrt', 'tan', 'tanh', 'trunc']

### Tab completion in IPython

IPython has tab completion: start typing the name of the command (object) and press `tab` to see the names of objects available with these initials letters. When the name of the object is typed followed by a dot (`math.`), pressing `tab` will show all available attribites, scroll down to the desired attribute and press `Enter` to select it.

### The four most helpful commands in IPython

These are the most helpful commands in IPython (from [IPython tutorial](http://ipython.org/ipython-doc/dev/interactive/tutorial.html)):

 - `?` : Introduction and overview of IPython’s features.
 - `%quickref` : Quick reference.
 - `help` : Python’s own help system.
 - `object?` : Details about ‘object’, use ‘object??’ for extra details.

[See these IPython Notebooks for more on IPython and the Notebook capabilities](http://nbviewer.ipython.org/github/ipython/ipython/tree/master/examples/Notebook/).

### Comments

Comments in Python start with the hash character, #, and extend to the end of the physical line:

```python
# Import the math library to access more math stuff
import math
math.pi  # this is the pi constant; a useless comment since this is obvious
```

    3.141592653589793

To insert comments spanning more than one line, use a multi-line string with a pair of matching triple-quotes: `"""` or `'''` (we will see the string data type later). A typical use of a multi-line comment is as documentation strings and are meant for anyone reading the code:

```python
"""Documentation strings are typically written like that.

A docstring is a string literal that occurs as the first statement
in a module, function, class, or method definition.

"""
```

    'Documentation strings are typically written like that.\n\nA docstring is a string literal that occurs as the first statement\nin a module, function, class, or method definition.\n\n'

A docstring like above is useless and its output as a standalone statement looks uggly in IPython Notebook, but you will see its real importance when reading and writting codes.

Commenting a programming code is an important step to make the code more readable, which Python cares a lot.
There is a style guide for writting Python code ([PEP 8](http://www.python.org/dev/peps/pep-0008/)) with a session about [how to write comments](http://www.python.org/dev/peps/pep-0008/#comments).

### Magic functions

IPython has a set of predefined ‘magic functions’ that you can call with a command line style syntax.
There are two kinds of magics, line-oriented and cell-oriented.
Line magics are prefixed with the % character and work much like OS command-line calls: they get as an argument the rest of the line, where arguments are passed without parentheses or quotes.
Cell magics are prefixed with a double %%, and they are functions that get as an argument not only the rest of the line, but also the lines below it in a separate argument.

## Assignment and expressions

The equal sign ('=') is used to assign a value to a variable. Afterwards, no result is displayed before the next interactive prompt:

```python
x = 1
```

Spaces between the statements are optional but it helps for readability.

To see the value of the variable, call it again or use the print function:

```python
x
```

    1

```python
print(x)
```

    1

Of course, the last assignment is that holds:

```python
x = 2
x = 3
x
```

    3

In mathematics '=' is the symbol for identity, but in computer programming '=' is used for assignment, it means that the right part of the expresssion is assigned to its left part.
For example, 'x=x+1' does not make sense in mathematics but it does in computer programming:

```python
x = 1
print(x)
x = x + 1
print(x)
```

    1
    2

A value can be assigned to several variables simultaneously:

```python
x = y = 4
print(x)
print(y)
```

    4
    4

Several values can be assigned to several variables at once:

```python
x, y = 5, 6
print(x)
print(y)
```

    5
    6

And with that, you can do (!):

```python
x, y = y, x
print(x)
print(y)
```

    6
    5

Variables must be “defined” (assigned a value) before they can be used, or an error will occur:

```python
x = z
```

## Variables and types

There are different types of built-in objects in Python (and remember that everything in Python is an object):

```python
import types
print(dir(types))
```

    ['BuiltinFunctionType', 'BuiltinMethodType', 'CodeType', 'DynamicClassAttribute', 'FrameType', 'FunctionType', 'GeneratorType', 'GetSetDescriptorType', 'LambdaType', 'MappingProxyType', 'MemberDescriptorType', 'MethodType', 'ModuleType', 'SimpleNamespace', 'TracebackType', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', '_calculate_meta', 'new_class', 'prepare_class']

Let's see some of them now.

### Numbers: int, float, complex

Numbers can an integer (int), float, and complex (with imaginary part).
Let's use the function `type` to show the type of number (and later for any other object):

```python
type(6)
```

    int

A float is a non-integer number:

```python
math.pi
```

    3.141592653589793

```python
type(math.pi)
```

    float

Python (IPython) is showing `math.pi` with only 15 decimal cases, but internally a float is represented with higher precision.
Floating point numbers in Python are implemented using a double (eight bytes) word; the precison and internal representation of floating point numbers are machine specific and are available in:

```python
sys.float_info
```

    sys.float_info(max=1.7976931348623157e+308, max_exp=1024, max_10_exp=308, min=2.2250738585072014e-308, min_exp=-1021, min_10_exp=-307, dig=15, mant_dig=53, epsilon=2.220446049250313e-16, radix=2, rounds=1)

Be aware that floating-point numbers can be trick in computers:

```python
0.1 + 0.2
```

    0.30000000000000004

```python
0.1 + 0.2 - 0.3
```

    5.551115123125783e-17

These results are not correct (and the problem is not due to Python). The error arises from the fact that floating-point numbers are represented in computer hardware as base 2 (binary) fractions and most decimal fractions cannot be represented exactly as binary fractions. As consequence, decimal floating-point numbers are only approximated by the binary floating-point numbers actually stored in the machine. [See here for more on this issue](http://docs.python.org/2/tutorial/floatingpoint.html).

A complex number has real and imaginary parts:

```python
1+2j
```

    (1+2j)

```python
print(type(1+2j))
```

    <class 'complex'>

Each part of a complex number is represented as a floating-point number. We can see them using the attributes `.real` and `.imag`:

```python
print((1+2j).real)
print((1+2j).imag)
```

    1.0
    2.0

### Strings

Strings can be enclosed in single quotes or double quotes:

```python
s = 'string (str) is a built-in type in Python'
s
```

    'string (str) is a built-in type in Python'

```python
type(s)
```

    str

String enclosed with single and double quotes are equal, but it may be easier to use one instead of the other:

```python
'string (str) is a Python's built-in type'
```

```python
"string (str) is a Python's built-in type"
```

    "string (str) is a Python's built-in type"

But you could have done that using the Python escape character '\':

```python
'string (str) is a Python\'s built-in type'
```

    "string (str) is a Python's built-in type"

Strings can be concatenated (glued together) with the + operator, and repeated with *:

```python
s = 'P' + 'y' + 't' + 'h' + 'o' + 'n'
print(s)
print(s*5)
```

    Python
    PythonPythonPythonPythonPython

Strings can be subscripted (indexed); like in C, the first character of a string has subscript (index) 0:

```python
print('s[0] = ', s[0], '  (s[index], start at 0)')
print('s[5] = ', s[5])
print('s[-1] = ', s[-1], '  (last element)')
print('s[:] = ', s[:], '  (all elements)')
print('s[1:] = ', s[1:], '  (from this index (inclusive) till the last (inclusive))')
print('s[2:4] = ', s[2:4], '  (from first index (inclusive) till second index (exclusive))')
print('s[:2] = ', s[:2], '  (till this index, exclusive)')
print('s[:10] = ', s[:10], '  (Python handles the index if it is larger than the string length)')
print('s[-10:] = ', s[-10:])
print('s[0:5:2] = ', s[0:5:2], '  (s[ini:end:step])')
print('s[::2] = ', s[::2], '  (s[::step], initial and final indexes can be omitted)')
print('s[0:5:-1] = ', s[::-1], '  (s[::-step] reverses the string)')
print('s[:2] + s[2:] = ', s[:2] + s[2:], '  (because of Python indexing, this sounds natural)')
```

    s[0] =  P   (s[index], start at 0)
    s[5] =  n
    s[-1] =  n   (last element)
    s[:] =  Python   (all elements)
    s[1:] =  ython   (from this index (inclusive) till the last (inclusive))
    s[2:4] =  th   (from first index (inclusive) till second index (exclusive))
    s[:2] =  Py   (till this index, exclusive)
    s[:10] =  Python   (Python handles the index if it is larger than the string length)
    s[-10:] =  Python
    s[0:5:2] =  Pto   (s[ini:end:step])
    s[::2] =  Pto   (s[::step], initial and final indexes can be omitted)
    s[0:5:-1] =  nohtyP   (s[::-step] reverses the string)
    s[:2] + s[2:] =  Python   (because of Python indexing, this sounds natural)

### len()

Python has a built-in functon to get the number of itens of a sequence:

```python
help(len)
```

    Help on built-in function len in module builtins:

    len(...)
        len(object)

        Return the number of items of a sequence or mapping.

```python
s = 'Python'
len(s)
```

    6

The function len() helps to understand how the backward indexing works in Python.
The index s[-i] should be understood as s[len(s) - i] rather than accessing directly the i-th element from back to front. This is why the last element of a string is s[-1]:

```python
print('s = ', s)
print('len(s) = ', len(s))
print('len(s)-1 = ',len(s) - 1)
print('s[-1] = ', s[-1])
print('s[len(s) - 1] = ', s[len(s) - 1])
```

    s =  Python
    len(s) =  6
    len(s)-1 =  5
    s[-1] =  n
    s[len(s) - 1] =  n

Or, strings can be surrounded in a pair of matching triple-quotes: """ or '''. End of lines do not need to be escaped when using triple-quotes, but they will be included in the string. This is how we created a multi-line comment earlier:

```python
"""Strings can be surrounded in a pair of matching triple-quotes: \""" or '''.

End of lines do not need to be escaped when using triple-quotes,
but they will be included in the string.

"""
```

    'Strings can be surrounded in a pair of matching triple-quotes: """ or \'\'\'.\n\nEnd of lines do not need to be escaped when using triple-quotes,\nbut they will be included in the string.\n\n'

### Lists

Values can be grouped together using different types, one of them is list, which can be written as a list of comma-separated values between square brackets. List items need not all have the same type:

```python
x = ['spam', 'eggs', 100, 1234]
x
```

    ['spam', 'eggs', 100, 1234]

Lists can be indexed and the same indexing rules we saw for strings are applied:

```python
x[0]
```

    'spam'

The function len() works for lists:

```python
len(x)
```

    4

### Tuples

A tuple consists of a number of values separated by commas, for instance:

```python
t = ('spam', 'eggs', 100, 1234)
t
```

    ('spam', 'eggs', 100, 1234)

The type tuple is why multiple assignments in a single line works; elements separated by commas (with or without surrounding parentheses) are a tuple and in an expression with an '=', the right-side tuple is attributed to the left-side tuple:

```python
a, b = 1, 2
print('a = ', a, '\nb = ', b)
```

    a =  1
    b =  2

Is the same as:

```python
(a, b) = (1, 2)
print('a = ', a, '\nb = ', b)
```

    a =  1
    b =  2

### Sets

Python also includes a data type for sets. A set is an unordered collection with no duplicate elements.

```python
basket = ['apple', 'orange', 'apple', 'pear', 'orange', 'banana']
fruit = set(basket)  # create a set without duplicates
fruit
```

    {'apple', 'banana', 'orange', 'pear'}

As set is an unordered collection, it can not be indexed as lists and tuples.

```python
set(['orange', 'pear', 'apple', 'banana'])
'orange' in fruit  # fast membership testing
```

    True

### Dictionaries

Dictionary is a collection of elements organized keys and values. Unlike lists and tuples, which are indexed by a range of numbers, dictionaries are indexed by their keys:

```python
tel = {'jack': 4098, 'sape': 4139}
tel
```

    {'sape': 4139, 'jack': 4098}

```python
tel['guido'] = 4127
tel
```

    {'guido': 4127, 'sape': 4139, 'jack': 4098}

```python
tel['jack']
```

    4098

```python
del tel['sape']
tel['irv'] = 4127
tel
```

    {'guido': 4127, 'irv': 4127, 'jack': 4098}

```python
tel.keys()
```

    dict_keys(['guido', 'irv', 'jack'])

```python
'guido' in tel
```

    True

The dict() constructor builds dictionaries directly from sequences of key-value pairs:

```python
tel = dict([('sape', 4139), ('guido', 4127), ('jack', 4098)])
tel
```

    {'guido': 4127, 'sape': 4139, 'jack': 4098}

## Built-in Constants

- **False** : false value of the bool type
- **True** : true value of the bool type
- **None** : sole value of types.NoneType. None is frequently used to represent the absence of a value.

In computer science, the Boolean or logical data type is composed by two values, true and false, intended to represent the values of logic and Boolean algebra. In Python, 1 and 0 can also be used in most situations as equivalent to the Boolean values.

## Logical (Boolean) operators

### and, or, not

- **and** : logical AND operator. If both the operands are true then condition becomes true.	 (a and b) is true.
- **or** : logical OR Operator. If any of the two operands are non zero then condition becomes true.	 (a or b) is true.
- **not** : logical NOT Operator. Reverses the logical state of its operand. If a condition is true then logical NOT operator will make false.

### Comparisons

The following comparison operations are supported by objects in Python:

- **==** : equal
- **!=** : not equal
- **<**	: strictly less than
- **<=** : less than or equal
- **\>** : strictly greater than
- **\>=** : greater than or equal
- **is** : object identity
- **is not** : negated object identity

```python
True == False
```

    False

```python
not True == False
```

    True

```python
1 < 2 > 1
```

    True

```python
True != (False or True)
```

    False

```python
True != False or True
```

    True

## Indentation and whitespace

In Python, statement grouping is done by indentation (this is mandatory), which are done by inserting whitespaces, not tabs. Indentation is also recommended for alignment of function calling that span more than one line for better clarity.
We will see examples of indentation in the next session.

## Control of flow

### `if`...`elif`...`else`

Conditional statements (to peform something if another thing is True or False) can be implemmented using the `if` statement:
```
if expression:
   statement
elif:
   statement
else:
   statement
```
`elif` (one or more) and `else` are optionals.
The indentation is obligatory.
For example:

```python
if True:
    pass
```

Which does nothing useful.

Let's use the `if`...`elif`...`else` statements to categorize the [body mass index](http://en.wikipedia.org/wiki/Body_mass_index) of a person:

```python
# body mass index
weight = 100  # kg
height = 1.70  # m
bmi = weight / height**2
```

```python
if bmi < 15:
    c = 'very severely underweight'
elif 15 <= bmi < 16:
    c = 'severely underweight'
elif 16 <= bmi < 18.5:
    c = 'underweight'
elif 18.5 <= bmi < 25:
    c = 'normal'
elif 25 <= bmi < 30:
    c = 'overweight'
elif 30 <= bmi < 35:
    c = 'moderately obese'
elif 35 <= bmi < 40:
    c = 'severely obese'
else:
    c = 'very severely obese'

print('For a weight of {0:.1f} kg and a height of {1:.2f} m,\n\
the body mass index (bmi) is {2:.1f} kg/m2,\nwhich is considered {3:s}.'\
      .format(weight, height, bmi, c))
```

    For a weight of 100.0 kg and a height of 1.70 m,
    the body mass index (bmi) is 34.6 kg/m2,
    which is considered moderately obese.

### for

The `for` statement iterates over a sequence to perform operations (a loop event).
```
for iterating_var in sequence:
    statements
```

```python
for i in [3, 2, 1, 'go!']:
    print(i),
```

    3
    2
    1
    go!

```python
for letter in 'Python':
    print(letter),
```

    P
    y
    t
    h
    o
    n

#### The `range()` function

The built-in function range() is useful if we need to create a sequence of numbers, for example, to iterate over this list. It generates lists containing arithmetic progressions:

```python
help(range)
```

    Help on class range in module builtins:

    class range(object)
     |  range(stop) -> range object
     |  range(start, stop[, step]) -> range object
     |
     |  Return a sequence of numbers from start to stop by step.
     |
     |  Methods defined here:
     |
     |  __contains__(self, key, /)
     |      Return key in self.
     |
     |  __eq__(self, value, /)
     |      Return self==value.
     |
     |  __ge__(self, value, /)
     |      Return self>=value.
     |
     |  __getattribute__(self, name, /)
     |      Return getattr(self, name).
     |
     |  __getitem__(self, key, /)
     |      Return self[key].
     |
     |  __gt__(self, value, /)
     |      Return self>value.
     |
     |  __hash__(self, /)
     |      Return hash(self).
     |
     |  __iter__(self, /)
     |      Implement iter(self).
     |
     |  __le__(self, value, /)
     |      Return self<=value.
     |
     |  __len__(self, /)
     |      Return len(self).
     |
     |  __lt__(self, value, /)
     |      Return self<value.
     |
     |  __ne__(self, value, /)
     |      Return self!=value.
     |
     |  __new__(*args, **kwargs) from builtins.type
     |      Create and return a new object.  See help(type) for accurate signature.
     |
     |  __reduce__(...)
     |
     |  __repr__(self, /)
     |      Return repr(self).
     |
     |  __reversed__(...)
     |      Return a reverse iterator.
     |
     |  count(...)
     |      rangeobject.count(value) -> integer -- return number of occurrences of value
     |
     |  index(...)
     |      rangeobject.index(value, [start, [stop]]) -> integer -- return index of value.
     |      Raise ValueError if the value is not present.
     |
     |  ----------------------------------------------------------------------
     |  Data descriptors defined here:
     |
     |  start
     |
     |  step
     |
     |  stop

```python
range(10)
```

    range(0, 10)

```python
range(1, 10, 2)
```

    range(1, 10, 2)

```python
for i in range(10):
    n2 = i**2
    print(n2),
```

    0
    1
    4
    9
    16
    25
    36
    49
    64
    81

### while

The `while` statement is used for repeating sections of code in a loop until a condition is met (this different than the `for` statement which executes n times):
```
while expression:
    statement
```
Let's generate the Fibonacci series using a `while` loop:

```python
# Fibonacci series: the sum of two elements defines the next
a, b = 0, 1
while b < 1000:
    print(b, end='   ')
    a, b = b, a+b
```

    1   1   2   3   5   8   13   21   34   55   89   144   233   377   610   987

## Function definition

A function in a programming language is a piece of code that performs a specific task. Functions are used to reduce duplication of code making easier to reuse it and to decompose complex problems into simpler parts. The use of functions contribute to the clarity of the code.

A function is created with the `def` keyword and the statements in the block of the function must be indented:

```python
def function():
    pass
```

As per construction, this function does nothing when called:

```python
function()
```

The general syntax of a function definition is:
```
def function_name( parameters ):
   """Function docstring.

   The help for the function

   """

   function body

   return variables
```
A more useful function:

```python
def fibo(N):
    """Fibonacci series: the sum of two elements defines the next.

    The series is calculated till the input parameter N and
    returned as an ouput variable.

    """

    a, b, c = 0, 1, []
    while b < N:
        c.append(b)
        a, b = b, a + b

    return c
```

```python
fibo(100)
```

    [1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89]

```python
if 3 > 2:
       print('teste')
```

    teste

Let's implemment the body mass index calculus and categorization as a function:

```python
def bmi(weight, height):
    """Body mass index calculus and categorization.

    Enter the weight in kg and the height in m.
    See http://en.wikipedia.org/wiki/Body_mass_index

    """

    bmi = weight / height**2

    if bmi < 15:
     c = 'very severely underweight'
    elif 15 <= bmi < 16:
        c = 'severely underweight'
    elif 16 <= bmi < 18.5:
        c = 'underweight'
    elif 18.5 <= bmi < 25:
        c = 'normal'
    elif 25 <= bmi < 30:
      c = 'overweight'
    elif 30 <= bmi < 35:
        c = 'moderately obese'
    elif 35 <= bmi < 40:
        c = 'severely obese'
    else:
        c = 'very severely obese'

    s = 'For a weight of {0:.1f} kg and a height of {1:.2f} m,\
    the body mass index (bmi) is {2:.1f} kg/m2,\
    which is considered {3:s}.'\
    .format(weight, height, bmi, c)

    print(s)
```

```python
bmi(73, 1.70)
```

    For a weight of 73.0 kg and a height of 1.70 m,    the body mass index (bmi) is 25.3 kg/m2,    which is considered overweight.

## Numeric data manipulation with Numpy

Numpy is the fundamental package for scientific computing in Python and has a N-dimensional array package convenient to work with numerical data. With Numpy it's much easier and faster to work with numbers grouped as 1-D arrays (a vector), 2-D arrays (like a table or matrix), or higher dimensions. Let's create 1-D and 2-D arrays in Numpy:

```python
import numpy as np
```

```python
x1d = np.array([1, 2, 3, 4, 5, 6])
print(type(x1d))
x1d
```

    <class 'numpy.ndarray'>

    array([1, 2, 3, 4, 5, 6])

```python
x2d = np.array([[1, 2, 3], [4, 5, 6]])
x2d
```

    array([[1, 2, 3],
           [4, 5, 6]])

len() and the Numpy functions size() and shape() give information aboout the number of elements and the structure of the Numpy array:

```python
print('1-d array:')
print(x1d)
print('len(x1d) = ', len(x1d))
print('np.size(x1d) = ', np.size(x1d))
print('np.shape(x1d) = ', np.shape(x1d))
print('np.ndim(x1d) = ', np.ndim(x1d))
print('\n2-d array:')
print(x2d)
print('len(x2d) = ', len(x2d))
print('np.size(x2d) = ', np.size(x2d))
print('np.shape(x2d) = ', np.shape(x2d))
print('np.ndim(x2d) = ', np.ndim(x2d))
```

    1-d array:
    [1 2 3 4 5 6]
    len(x1d) =  6
    np.size(x1d) =  6
    np.shape(x1d) =  (6,)
    np.ndim(x1d) =  1

    2-d array:
    [[1 2 3]
     [4 5 6]]
    len(x2d) =  2
    np.size(x2d) =  6
    np.shape(x2d) =  (2, 3)
    np.ndim(x2d) =  2

Create random data

```python
x = np.random.randn(4,3)
x
```

    array([[-0.84313748,  0.7128172 ,  0.26805518],
           [ 1.66552011,  0.81119094,  0.72512201],
           [-2.6924966 ,  1.75984096,  0.58488695],
           [ 0.64630267, -0.8719821 , -0.8510621 ]])

Joining (stacking together) arrays

```python
x = np.random.randint(0, 5, size=(2, 3))
print(x)
y = np.random.randint(5, 10, size=(2, 3))
print(y)
```

    [[0 4 0]
     [3 1 2]]
    [[6 9 6]
     [6 5 5]]

```python
np.vstack((x,y))
```

    array([[0, 4, 0],
           [3, 1, 2],
           [6, 9, 6],
           [6, 5, 5]])

```python
np.hstack((x,y))
```

    array([[0, 4, 0, 6, 9, 6],
           [3, 1, 2, 6, 5, 5]])

Create equally spaced data

```python
np.arange(start = 1, stop = 10, step = 2)
```

    array([1, 3, 5, 7, 9])

```python
np.linspace(start = 0, stop = 1, num = 11)
```

    array([ 0. ,  0.1,  0.2,  0.3,  0.4,  0.5,  0.6,  0.7,  0.8,  0.9,  1. ])

### Interpolation

Consider the following data:

```python
y = [5,  4, 10,  8,  1, 10,  2,  7,  1,  3]
```

Suppose we want to create data in between the given data points (interpolation); for instance, let's try to double the resolution of the data by generating twice as many data:

```python
t = np.linspace(0, len(y), len(y))       # time vector for the original data
tn = np.linspace(0, len(y), 2 * len(y))  # new time vector for the new time-normalized data
yn = np.interp(tn, t, y)                 # new time-normalized data
yn
```

    array([ 5.        ,  4.52631579,  4.05263158,  6.52631579,  9.36842105,
            9.26315789,  8.31578947,  5.78947368,  2.47368421,  3.36842105,
            7.63157895,  8.31578947,  4.52631579,  2.78947368,  5.15789474,
            6.36842105,  3.52631579,  1.10526316,  2.05263158,  3.        ])

The key is the Numpy `interp` function, from its help:

    interp(x, xp, fp, left=None, right=None)
    One-dimensional linear interpolation.
    Returns the one-dimensional piecewise linear interpolant to a function with given values at discrete data-points.

A plot of the data will show what we have done:

```python
%matplotlib inline
import matplotlib.pyplot as plt
plt.figure(figsize=(10,5))
plt.plot(t, y, 'bo-', lw=2, label='original data')
plt.plot(tn, yn, '.-', color=[1, 0, 0, .5], lw=2, label='interpolated')
plt.legend(loc='best', framealpha=.5)
plt.show()
```

For more about Numpy, see [http://wiki.scipy.org/Tentative_NumPy_Tutorial](http://wiki.scipy.org/Tentative_NumPy_Tutorial).

## Read and save files

There are two kinds of computer files: text files and binary files:
> Text file: computer file where the content is structured as a sequence of lines of electronic text. Text files can contain plain text (letters, numbers, and symbols) but they are not limited to such. The type of content in the text file is defined by the Unicode encoding (a computing industry standard for the consistent encoding, representation and handling of text expressed in most of the world's writing systems).
>
> Binary file: computer file where the content is encoded in binary form, a sequence of integers representing byte values.

Let's see how to save and read numeric data stored in a text file:

**Using plain Python**

```python
f = open("newfile.txt", "w")           # open file for writing
f.write("This is a test\n")            # save to file
f.write("And here is another line\n")  # save to file
f.close()
f = open('newfile.txt', 'r')           # open file for reading
f = f.read()                           # read from file
print(f)
```

    This is a test
    And here is another line

```python
help(open)
```

    Help on built-in function open in module io:

    open(...)
        open(file, mode='r', buffering=-1, encoding=None,
             errors=None, newline=None, closefd=True, opener=None) -> file object

        Open file and return a stream.  Raise IOError upon failure.

        file is either a text or byte string giving the name (and the path
        if the file isn't in the current working directory) of the file to
        be opened or an integer file descriptor of the file to be
        wrapped. (If a file descriptor is given, it is closed when the
        returned I/O object is closed, unless closefd is set to False.)

        mode is an optional string that specifies the mode in which the file
        is opened. It defaults to 'r' which means open for reading in text
        mode.  Other common values are 'w' for writing (truncating the file if
        it already exists), 'x' for creating and writing to a new file, and
        'a' for appending (which on some Unix systems, means that all writes
        append to the end of the file regardless of the current seek position).
        In text mode, if encoding is not specified the encoding used is platform
        dependent: locale.getpreferredencoding(False) is called to get the
        current locale encoding. (For reading and writing raw bytes use binary
        mode and leave encoding unspecified.) The available modes are:

        ========= ===============================================================
        Character Meaning
        --------- ---------------------------------------------------------------
        'r'       open for reading (default)
        'w'       open for writing, truncating the file first
        'x'       create a new file and open it for writing
        'a'       open for writing, appending to the end of the file if it exists
        'b'       binary mode
        't'       text mode (default)
        '+'       open a disk file for updating (reading and writing)
        'U'       universal newline mode (deprecated)
        ========= ===============================================================

        The default mode is 'rt' (open for reading text). For binary random
        access, the mode 'w+b' opens and truncates the file to 0 bytes, while
        'r+b' opens the file without truncation. The 'x' mode implies 'w' and
        raises an `FileExistsError` if the file already exists.

        Python distinguishes between files opened in binary and text modes,
        even when the underlying operating system doesn't. Files opened in
        binary mode (appending 'b' to the mode argument) return contents as
        bytes objects without any decoding. In text mode (the default, or when
        't' is appended to the mode argument), the contents of the file are
        returned as strings, the bytes having been first decoded using a
        platform-dependent encoding or using the specified encoding if given.

        'U' mode is deprecated and will raise an exception in future versions
        of Python.  It has no effect in Python 3.  Use newline to control
        universal newlines mode.

        buffering is an optional integer used to set the buffering policy.
        Pass 0 to switch buffering off (only allowed in binary mode), 1 to select
        line buffering (only usable in text mode), and an integer > 1 to indicate
        the size of a fixed-size chunk buffer.  When no buffering argument is
        given, the default buffering policy works as follows:

        * Binary files are buffered in fixed-size chunks; the size of the buffer
          is chosen using a heuristic trying to determine the underlying device's
          "block size" and falling back on `io.DEFAULT_BUFFER_SIZE`.
          On many systems, the buffer will typically be 4096 or 8192 bytes long.

        * "Interactive" text files (files for which isatty() returns True)
          use line buffering.  Other text files use the policy described above
          for binary files.

        encoding is the name of the encoding used to decode or encode the
        file. This should only be used in text mode. The default encoding is
        platform dependent, but any encoding supported by Python can be
        passed.  See the codecs module for the list of supported encodings.

        errors is an optional string that specifies how encoding errors are to
        be handled---this argument should not be used in binary mode. Pass
        'strict' to raise a ValueError exception if there is an encoding error
        (the default of None has the same effect), or pass 'ignore' to ignore
        errors. (Note that ignoring encoding errors can lead to data loss.)
        See the documentation for codecs.register or run 'help(codecs.Codec)'
        for a list of the permitted encoding error strings.

        newline controls how universal newlines works (it only applies to text
        mode). It can be None, '', '\n', '\r', and '\r\n'.  It works as
        follows:

        * On input, if newline is None, universal newlines mode is
          enabled. Lines in the input can end in '\n', '\r', or '\r\n', and
          these are translated into '\n' before being returned to the
          caller. If it is '', universal newline mode is enabled, but line
          endings are returned to the caller untranslated. If it has any of
          the other legal values, input lines are only terminated by the given
          string, and the line ending is returned to the caller untranslated.

        * On output, if newline is None, any '\n' characters written are
          translated to the system default line separator, os.linesep. If
          newline is '' or '\n', no translation takes place. If newline is any
          of the other legal values, any '\n' characters written are translated
          to the given string.

        If closefd is False, the underlying file descriptor will be kept open
        when the file is closed. This does not work when a file name is given
        and must be True in that case.

        A custom opener can be used by passing a callable as *opener*. The
        underlying file descriptor for the file object is then obtained by
        calling *opener* with (*file*, *flags*). *opener* must return an open
        file descriptor (passing os.open as *opener* results in functionality
        similar to passing None).

        open() returns a file object whose type depends on the mode, and
        through which the standard file operations such as reading and writing
        are performed. When open() is used to open a file in a text mode ('w',
        'r', 'wt', 'rt', etc.), it returns a TextIOWrapper. When used to open
        a file in a binary mode, the returned class varies: in read binary
        mode, it returns a BufferedReader; in write binary and append binary
        modes, it returns a BufferedWriter, and in read/write mode, it returns
        a BufferedRandom.

        It is also possible to use a string or bytearray as a file for both
        reading and writing. For strings StringIO can be used like a file
        opened in a text mode, and for bytes a BytesIO can be used like a file
        opened in a binary mode.

**Using Numpy**

```python
import numpy as np
data = np.random.randn(3,3)
np.savetxt('myfile.txt', data, fmt="%12.6G")    # save to file
data = np.genfromtxt('myfile.txt', unpack=True) # read from file
data
```

    array([[-0.113108,  0.510521, -0.240193],
           [-0.448538, -0.348518, -1.97899 ],
           [ 0.21628 , -0.572211, -1.49621 ]])

## Ploting with matplotlib

Matplotlib is the most-widely used packge for plotting data in Python. Let's see some examples of it.

```python
import matplotlib.pyplot as plt
```

Use the IPython magic `%matplotlib inline` to plot a figure inline in the notebook with the rest of the text:

```python
%matplotlib notebook
```

    C:\Miniconda3\lib\site-packages\IPython\kernel\__init__.py:13: ShimWarning: The `IPython.kernel` package has been deprecated. You should import from ipykernel or jupyter_client instead.
      "You should import from ipykernel or jupyter_client instead.", ShimWarning)

```python
import numpy as np
```

```python
t = np.linspace(0, 0.99, 100)
x = np.sin(2 * np.pi * 2 * t)
n = np.random.randn(100) / 5
plt.Figure(figsize=(12,8))
plt.plot(t, x, label='sine', linewidth=2)
plt.plot(t, x + n, label='noisy sine', linewidth=2)
plt.annotate(s='$sin(4 \pi t)$', xy=(.2, 1), fontsize=20, color=[0, 0, 1])
plt.legend(loc='best', framealpha=.5)
plt.xlabel('Time [s]')
plt.ylabel('Amplitude')
plt.title('Data plotting using matplotlib')
plt.show()
```

    <IPython.core.display.Javascript object>

Use the IPython magic `%matplotlib qt` to plot a figure in a separate window (from where you will be able to change some of the figure proprerties):

```python
%matplotlib qt
```

```python
mu, sigma = 10, 2
x = mu + sigma * np.random.randn(1000)
fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(12, 4))
ax1.plot(x, 'ro')
ax1.set_title('Data')
ax1.grid()

n, bins, patches = ax2.hist(x, 25, normed=True, facecolor='r') # histogram
ax2.set_xlabel('Bins')
ax2.set_ylabel('Probability')
ax2.set_title('Histogram')
fig.suptitle('Another example using matplotlib', fontsize=18, y=1)
ax2.grid()

plt.tight_layout()
plt.show()
```

And a window with the following figure should appear:

```python
from IPython.display import Image
Image(url="./../images/plot.png")
```

You can switch back and forth between inline and separate figure using the `%matplotlib` magic commands used above. There are plenty more examples with the source code in the [matplotlib gallery](http://matplotlib.org/gallery.html).

```python
# get back the inline plot
%matplotlib inline
```

## Signal processing with Scipy

The Scipy package has a lot of functions for signal processing, among them: Integration (scipy.integrate), Optimization (scipy.optimize), Interpolation (scipy.interpolate), Fourier Transforms (scipy.fftpack), Signal Processing (scipy.signal), Linear Algebra (scipy.linalg), and Statistics (scipy.stats). As an example, let's see how to use a low-pass Butterworth filter to attenuate high-frequency noise and how the differentiation process of a signal affects the signal-to-noise content. We will also calculate the Fourier transform of these data to look at their frequencies content.

```python
from scipy.signal import butter, filtfilt
import scipy.fftpack
freq = 100.
t = np.arange(0,1,.01);
w = 2*np.pi*1 # 1 Hz
y = np.sin(w*t)+0.1*np.sin(10*w*t)
# Butterworth filter
b, a = butter(4, (5/(freq/2)), btype = 'low')
y2 = filtfilt(b, a, y)
# 2nd derivative of the data
ydd = np.diff(y,2)*freq*freq   # raw data
y2dd = np.diff(y2,2)*freq*freq # filtered data
# frequency content
yfft = np.abs(scipy.fftpack.fft(y))/(y.size/2);   # raw data
y2fft = np.abs(scipy.fftpack.fft(y2))/(y.size/2); # filtered data
freqs = scipy.fftpack.fftfreq(y.size, 1./freq)
yddfft = np.abs(scipy.fftpack.fft(ydd))/(ydd.size/2);
y2ddfft = np.abs(scipy.fftpack.fft(y2dd))/(ydd.size/2);
freqs2 = scipy.fftpack.fftfreq(ydd.size, 1./freq)
```

And the plots:

```python
fig, ((ax1,ax2),(ax3,ax4)) = plt.subplots(2, 2, figsize=(10, 4))

ax1.set_title('Temporal domain', fontsize=14)
ax1.plot(t, y, 'r', linewidth=2, label = 'raw data')
ax1.plot(t, y2, 'b', linewidth=2, label = 'filtered @ 5 Hz')
ax1.set_ylabel('f')
ax1.legend(frameon=False, fontsize=12)

ax2.set_title('Frequency domain', fontsize=14)
ax2.plot(freqs[:yfft.size/4], yfft[:yfft.size/4],'r',  lw=2,label='raw data')
ax2.plot(freqs[:yfft.size/4],y2fft[:yfft.size/4],'b--',lw=2,label='filtered @ 5 Hz')
ax2.set_ylabel('FFT(f)')
ax2.legend(frameon=False, fontsize=12)

ax3.plot(t[:-2], ydd, 'r', linewidth=2, label = 'raw')
ax3.plot(t[:-2], y2dd, 'b', linewidth=2, label = 'filtered @ 5 Hz')
ax3.set_xlabel('Time [s]'); ax3.set_ylabel("f ''")

ax4.plot(freqs[:yddfft.size/4], yddfft[:yddfft.size/4], 'r', lw=2, label = 'raw')
ax4.plot(freqs[:yddfft.size/4],y2ddfft[:yddfft.size/4],'b--',lw=2, label='filtered @ 5 Hz')
ax4.set_xlabel('Frequency [Hz]'); ax4.set_ylabel("FFT(f '')")
plt.show()
```

For more about Scipy, see [http://docs.scipy.org/doc/scipy/reference/tutorial/](http://docs.scipy.org/doc/scipy/reference/tutorial/).

## Symbolic mathematics with Sympy

Sympy is a package to perform symbolic mathematics in Python. Let's see some of its features:

```python
from IPython.display import display
import sympy as sym
from sympy.interactive import printing
printing.init_printing()
```

Define some symbols and the create a second-order polynomial function (a.k.a., parabola):

```python
x, y = sym.symbols('x y')
y = x**2 - 2*x - 3
y
```

Plot the parabola at some given range:

```python
from sympy.plotting import plot
%matplotlib inline
plot(y, (x, -3, 5));
```

And the roots of the parabola are given by:

```python
sym.solve(y, x)
```

$$\begin{bmatrix}-1, & 3\end{bmatrix}$$

We can also do symbolic differentiation and integration:

```python
dy = sym.diff(y, x)
dy
```

```python
sym.integrate(dy, x)
```

For example, let's use Sympy to represent three-dimensional rotations. Consider the problem of a coordinate system xyz rotated in relation to other coordinate system XYZ. The single rotations around each axis are illustrated by:

```python
from IPython.display import Image
Image(url="./../images/rotations.png")
```

The single 3D rotation matrices around Z, Y, and X axes can be expressed in Sympy:

```python
from IPython.core.display import Math
from sympy import symbols, cos, sin, Matrix, latex
a, b, g = symbols('alpha beta gamma')

RX = Matrix([[1, 0, 0], [0, cos(a), -sin(a)], [0, sin(a), cos(a)]])
display(Math(latex('\\mathbf{R_{X}}=') + latex(RX, mat_str = 'matrix')))

RY = Matrix([[cos(b), 0, sin(b)], [0, 1, 0], [-sin(b), 0, cos(b)]])
display(Math(latex('\\mathbf{R_{Y}}=') + latex(RY, mat_str = 'matrix')))

RZ = Matrix([[cos(g), -sin(g), 0], [sin(g), cos(g), 0], [0, 0, 1]])
display(Math(latex('\\mathbf{R_{Z}}=') + latex(RZ, mat_str = 'matrix')))
```

$$\mathbf{R_{X}}=\left[\begin{matrix}1 & 0 & 0\\0 & \cos{\left (\alpha \right )} & - \sin{\left (\alpha \right )}\\0 & \sin{\left (\alpha \right )} & \cos{\left (\alpha \right )}\end{matrix}\right]$$

$$\mathbf{R_{Y}}=\left[\begin{matrix}\cos{\left (\beta \right )} & 0 & \sin{\left (\beta \right )}\\0 & 1 & 0\\- \sin{\left (\beta \right )} & 0 & \cos{\left (\beta \right )}\end{matrix}\right]$$

$$\mathbf{R_{Z}}=\left[\begin{matrix}\cos{\left (\gamma \right )} & - \sin{\left (\gamma \right )} & 0\\\sin{\left (\gamma \right )} & \cos{\left (\gamma \right )} & 0\\0 & 0 & 1\end{matrix}\right]$$

And using Sympy, a sequence of elementary rotations around X, Y, Z axes is given by:

```python
RXYZ = RZ*RY*RX
display(Math(latex('\\mathbf{R_{XYZ}}=') + latex(RXYZ, mat_str = 'matrix')))
```

$$\mathbf{R_{XYZ}}=\left[\begin{matrix}\cos{\left (\beta \right )} \cos{\left (\gamma \right )} & \sin{\left (\alpha \right )} \sin{\left (\beta \right )} \cos{\left (\gamma \right )} - \sin{\left (\gamma \right )} \cos{\left (\alpha \right )} & \sin{\left (\alpha \right )} \sin{\left (\gamma \right )} + \sin{\left (\beta \right )} \cos{\left (\alpha \right )} \cos{\left (\gamma \right )}\\\sin{\left (\gamma \right )} \cos{\left (\beta \right )} & \sin{\left (\alpha \right )} \sin{\left (\beta \right )} \sin{\left (\gamma \right )} + \cos{\left (\alpha \right )} \cos{\left (\gamma \right )} & - \sin{\left (\alpha \right )} \cos{\left (\gamma \right )} + \sin{\left (\beta \right )} \sin{\left (\gamma \right )} \cos{\left (\alpha \right )}\\- \sin{\left (\beta \right )} & \sin{\left (\alpha \right )} \cos{\left (\beta \right )} & \cos{\left (\alpha \right )} \cos{\left (\beta \right )}\end{matrix}\right]$$

Suppose there is a rotation only around X ($\alpha$) by $\pi/2$; we can get the numerical value of the rotation matrix by substituing the angle values:

```python
r = RXYZ.subs({a: np.pi/2, b: 0, g: 0})
r
```

$$\left[\begin{matrix}1 & 0 & 0\\0 & 6.12323399573677 \cdot 10^{-17} & -1.0\\0 & 1.0 & 6.12323399573677 \cdot 10^{-17}\end{matrix}\right]$$

And we can prettify this result:

```python
display(Math(latex(r'\mathbf{R_{(\alpha=\pi/2)}}=') +
             latex(r.n(chop=True, prec=3), mat_str = 'matrix')))
```

$$\mathbf{R_{(\alpha=\pi/2)}}=\left[\begin{matrix}1.0 & 0 & 0\\0 & 0 & -1.0\\0 & 1.0 & 0\end{matrix}\right]$$

For more about Sympy, see [http://docs.sympy.org/latest/tutorial/](http://docs.sympy.org/latest/tutorial/).

## Data analysis with pandas

> "[pandas](http://pandas.pydata.org/) is a Python package providing fast, flexible, and expressive data structures designed to make working with “relational” or “labeled” data both easy and intuitive. It aims to be the fundamental high-level building block for doing practical, real world data analysis in Python."

To work with labellled data, pandas has a type called DataFrame (basically, a matrix where columns and rows have may names and may be of different types) and it is also the main type of the software [R](http://www.r-project.org/). Fo ezample:

```python
import pandas as pd
```

```python
x = 5*['A'] + 5*['B']
x
```

    ['A', 'A', 'A', 'A', 'A', 'B', 'B', 'B', 'B', 'B']

```python
df = pd.DataFrame(np.random.rand(10,2), columns=['Level 1', 'Level 2'] )
df['Group'] = pd.Series(['A']*5 + ['B']*5)
plot = df.boxplot(by='Group')
```

```python
from pandas.tools.plotting import scatter_matrix
df = pd.DataFrame(np.random.randn(100, 3), columns=['A', 'B', 'C'])
plot = scatter_matrix(df, alpha=0.5, figsize=(8, 6), diagonal='kde')
```

pandas is aware the data is structured and give you basic statistics considerint that and nicely formatted:

```python
df.describe()
```

<div style="max-height:1000px;max-width:1500px;overflow:auto;">
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>A</th>
      <th>B</th>
      <th>C</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>count</th>
      <td> 100.000000</td>
      <td> 100.000000</td>
      <td> 100.000000</td>
    </tr>
    <tr>
      <th>mean</th>
      <td>   0.176822</td>
      <td>  -0.028189</td>
      <td>   0.079938</td>
    </tr>
    <tr>
      <th>std</th>
      <td>   0.962097</td>
      <td>   1.056000</td>
      <td>   0.955360</td>
    </tr>
    <tr>
      <th>min</th>
      <td>  -1.909003</td>
      <td>  -2.515748</td>
      <td>  -2.564882</td>
    </tr>
    <tr>
      <th>25%</th>
      <td>  -0.434171</td>
      <td>  -0.851826</td>
      <td>  -0.531646</td>
    </tr>
    <tr>
      <th>50%</th>
      <td>   0.113800</td>
      <td>   0.053892</td>
      <td>   0.088606</td>
    </tr>
    <tr>
      <th>75%</th>
      <td>   0.661160</td>
      <td>   0.782833</td>
      <td>   0.600897</td>
    </tr>
    <tr>
      <th>max</th>
      <td>   2.673605</td>
      <td>   1.989213</td>
      <td>   2.621580</td>
    </tr>
  </tbody>
</table>
</div>

For more on pandas, see this tutorial: [http://pandas.pydata.org/pandas-docs/stable/10min.html](http://pandas.pydata.org/pandas-docs/stable/10min.html).

## To learn more about Python

There is a lot of good material in the internet about Python for scientific computing, here is a small list of interesting stuff:

 - [How To Think Like A Computer Scientist](http://www.openbookproject.net/thinkcs/python/english2e/) or [the interactive edition](http://interactivepython.org/courselib/static/thinkcspy/index.html) (book)
 - [Python Scientific Lecture Notes](http://scipy-lectures.github.io/) (lecture notes)
 - [Lectures on scientific computing with Python](https://github.com/jrjohansson/scientific-python-lectures#lectures-on-scientific-computing-with-python) (lecture notes)
 - [IPython in depth: high-productivity interactive and parallel python](http://youtu.be/bP8ydKBCZiY) (video lectures)


# Week 3 January 18-22: Building a Variational Monte Carlo program

**Morten Hjorth-Jensen  Email morten.hjorth-jensen@fys.uio.no**, Department of Physics and Center fo Computing in Science Education, University of Oslo, Oslo, Norway and Department of Physics and Astronomy and Facility for Rare Ion Beams, Michigan State University, East Lansing, Michigan, USA

Date: **Jan 21, 2021**

Copyright 1999-2021, Morten Hjorth-Jensen  Email morten.hjorth-jensen@fys.uio.no. Released under CC Attribution-NonCommercial 4.0 license

## Overview of week 3
**Topics.**

* Variational Monte Carlo methods, Metropolis Algorithm, statistics and Markov Chain theory

* How to structure the VMC code

## Setting up a VMC code

In setting up a C++ code for variational Monte Carlo calculations, we will use an excellent framework developed by a former Computational Physics student, Morten Ledum, now PhD student at the Hylleraas center for Quantum Chemistry.
The GitHub repository is at <https://github.com/mortele/variational-monte-carlo-fys4411>.

We will discuss this both in our forthcoming lectures and during our lab sessions.

## Introduction

### Structure and Aims

These notebooks serve the aim of linking traditional variational Monte
Carlo VMC calculations methods with recent progress on solving
many-particle problems using Machine Learning algorithms.

### This notebook

In this notebook the aim is to give you an introduction as well as an
understanding of the basic elements that are needed in order to
develop a professional variational Monte Carlo code. We will focus on
a simple system of two particles in an oscillator trap (or
alternatively two fermions moving in a Coulombic potential). The particles can
interact via a repulsive or an attrative force.  It is thus similar to the system described in project 1.

The advantage of these systems is that for two particles (boson or
fermions) we have analytical solutions for the eigenpairs of the
non-interacting case. Furthermore, for a two- or three-dimensional
system of two electrons moving in a harmonic oscillator trap, we have
[analytical solutions for the interacting case as well](https://iopscience.iop.org/article/10.1088/0305-4470/27/3/040/meta).

Having analytical eigenpairs is an invaluable feature that allows us
to assess the physical relevance of the trial wave functions, be
these either from a standard VMC procedure, from Boltzmann Machines or
from Shadow Wave functions.

In this notebook we start with the basics of a VMC calculation and
introduce concepts like Markov Chain Monte Carlo methods and the
Metropolis algorithm, importance sampling and Metropolis-Hastings
algorithm, resampling methods to obtain better estimates of the
statistical errors and minimization of the expectation values of the
energy and the variance. The latter is done in order to obtain the
best possible variational parameters. Furthermore it will define the
so-called **cost** function, a commonly encountered quantity in Machine
Learning algorithms. Minimizing the latter is the one which leads to
the determination of the optimal parameters in basically all Machine Learning algorithms.
This is a possible topic for project 2.

Topics like Markov Chain Monte Carlo and various resampling techniques
are also central to Machine Learning methods. Presenting them in the
context of VMC approaches leads hopefully to an easier starting point
for the understanding of these methods for those of you who will do the machine lerning variant of project 2.

## Basic Quantum Monte Carlo

We start with the variational principle.
Given a hamiltonian $H$ and a trial wave function $\Psi_T(\boldsymbol{R};\boldsymbol{\alpha})$, the variational principle states that the expectation value of $\cal{E}[H]$, defined through

$$
\cal {E}[H] =
   \frac{\int d\boldsymbol{R}\Psi^{\ast}_T(\boldsymbol{R};\boldsymbol{\alpha})H(\boldsymbol{R})\Psi_T(\boldsymbol{R};\boldsymbol{\alpha})}
        {\int d\boldsymbol{R}\Psi^{\ast}_T(\boldsymbol{R};\boldsymbol{\alpha})\Psi_T(\boldsymbol{R};\boldsymbol{\alpha})},
$$

is an upper bound to the ground state energy $E_0$ of the hamiltonian $H$, that is

$$
E_0 \le {\cal E}[H].
$$

In general, the integrals involved in the calculation of various
expectation values are multi-dimensional ones. Traditional integration
methods such as Gauss-Legendre quadrature will not be adequate for say the
computation of the energy of a many-body system.

Here we have defined the vector $\boldsymbol{R} = [\boldsymbol{r}_1,\boldsymbol{r}_2,\dots,\boldsymbol{r}_n]$  as an array that contains the positions of all particles $n$ while the vector $\boldsymbol{\alpha} = [\alpha_1,\alpha_2,\dots,\alpha_m]$ contains the variational parameters of the model, $m$ in total.

The trial wave function can be expanded in the eigenstates $\Psi_i(\boldsymbol{R})$
of the hamiltonian since they form a complete set, viz.,

$$
\Psi_T(\boldsymbol{R};\boldsymbol{\alpha})=\sum_i a_i\Psi_i(\boldsymbol{R}),
$$

and assuming that the set of eigenfunctions are normalized, one obtains

$$
\frac{\sum_{nm}a^*_ma_n \int d\boldsymbol{R}\Psi^{\ast}_m(\boldsymbol{R})H(\boldsymbol{R})\Psi_n(\boldsymbol{R})}
        {\sum_{nm}a^*_ma_n \int d\boldsymbol{R}\Psi^{\ast}_m(\boldsymbol{R})\Psi_n(\boldsymbol{R})} =\frac{\sum_{n}a^2_n E_n}
        {\sum_{n}a^2_n} \ge E_0,
$$

where we used that $H(\boldsymbol{R})\Psi_n(\boldsymbol{R})=E_n\Psi_n(\boldsymbol{R})$.
In general, the integrals involved in the calculation of various  expectation
values  are multi-dimensional ones.
The variational principle yields the lowest energy of states with a  given symmetry.

In most cases, a wave function has only small values in large parts of
configuration space, and a straightforward procedure which uses
homogenously distributed random points in configuration space
will most likely lead to poor results. This may suggest that some kind
of importance sampling combined with e.g., the Metropolis algorithm
may be  a more efficient way of obtaining the ground state energy.
The hope is then that those regions of configurations space where
the wave function assumes appreciable values are sampled more
efficiently.

The tedious part in a VMC calculation is the search for the variational
minimum. A good knowledge of the system is required in order to carry out
reasonable VMC calculations. This is not always the case,
and often VMC calculations
serve rather as the starting
point for so-called diffusion Monte Carlo calculations (DMC). Diffusion Monte Carlo  is a way of
solving exactly the many-body Schroedinger equation by means of
a stochastic procedure. A good guess on the binding energy
and its wave function is however necessary.
A carefully performed VMC calculation can aid in this context.

The basic procedure of a Variational Monte Carlo calculations consists thus of

1. Construct first a trial wave function $\psi_T(\boldsymbol{R};\boldsymbol{\alpha})$,  for a many-body system consisting of $n$ particles located at positions  $\boldsymbol{R}=(\boldsymbol{R}_1,\dots ,\boldsymbol{R}_n)$. The trial wave function depends on $\alpha$ variational parameters $\boldsymbol{\alpha}=(\alpha_1,\dots ,\alpha_M)$.

2. Then we evaluate the expectation value of the hamiltonian $H$

$$
\overline{E}[\boldsymbol{\alpha}]=\frac{\int d\boldsymbol{R}\Psi^{\ast}_{T}(\boldsymbol{R},\boldsymbol{\alpha})H(\boldsymbol{R})\Psi_{T}(\boldsymbol{R},\boldsymbol{\alpha})}
        {\int d\boldsymbol{R}\Psi^{\ast}_{T}(\boldsymbol{R},\boldsymbol{\alpha})\Psi_{T}(\boldsymbol{R},\boldsymbol{\alpha})}.
$$

1. Thereafter we vary $\boldsymbol{\alpha}$ according to some minimization algorithm and return eventually to the first step if we are not satisfied with the results.

Here we have used the notation $\overline{E}$ to label the expectation value of the energy.

### Linking with standard statistical expressions for expectation values

In order to bring in the Monte Carlo machinery, we define first a likelihood distribution, or probability density distribution (PDF). Using our ansatz for the trial wave function $\psi_T(\boldsymbol{R};\boldsymbol{\alpha})$ we define a PDF

$$
P(\boldsymbol{R})= \frac{\left|\psi_T(\boldsymbol{R};\boldsymbol{\alpha})\right|^2}{\int \left|\psi_T(\boldsymbol{R};\boldsymbol{\alpha})\right|^2d\boldsymbol{R}}.
$$

This is our model for  probability distribution function.
The approximation to the expectation value of the Hamiltonian is now

$$
\overline{E}[\boldsymbol{\alpha}] =
   \frac{\int d\boldsymbol{R}\Psi^{\ast}_T(\boldsymbol{R};\boldsymbol{\alpha})H(\boldsymbol{R})\Psi_T(\boldsymbol{R};\boldsymbol{\alpha})}
        {\int d\boldsymbol{R}\Psi^{\ast}_T(\boldsymbol{R};\boldsymbol{\alpha})\Psi_T(\boldsymbol{R};\boldsymbol{\alpha})}.
$$

We define a new quantity

$$
E_L(\boldsymbol{R};\boldsymbol{\alpha})=\frac{1}{\psi_T(\boldsymbol{R};\boldsymbol{\alpha})}H\psi_T(\boldsymbol{R};\boldsymbol{\alpha}),
\label{eq:locale1} \tag{1}
$$

called the local energy, which, together with our trial PDF yields a new expression (and which look simlar to the the expressions for moments in statistics)

$$
\overline{E}[\boldsymbol{\alpha}]=\int P(\boldsymbol{R})E_L(\boldsymbol{R};\boldsymbol{\alpha}) d\boldsymbol{R}\approx \frac{1}{N}\sum_{i=1}^NE_L(\boldsymbol{R_i};\boldsymbol{\alpha})
\label{eq:vmc1} \tag{2}
$$

with $N$ being the number of Monte Carlo samples. The expression on the right hand side follows from Bernoulli's law of large numbers, which states that the sample mean, in the limit $N\rightarrow \infty$ approaches the true mean

The Algorithm for performing a variational Monte Carlo calculations runs as this

   * Initialisation: Fix the number of Monte Carlo steps. Choose an initial $\boldsymbol{R}$ and variational parameters $\alpha$ and calculate $\left|\psi_T^{\alpha}(\boldsymbol{R})\right|^2$.

   * Initialise the energy and the variance and start the Monte Carlo calculation.

      * Calculate  a trial position  $\boldsymbol{R}_p=\boldsymbol{R}+r*step$ where $r$ is a random variable $r \in [0,1]$.

      * Metropolis algorithm to accept or reject this move  $w = P(\boldsymbol{R}_p)/P(\boldsymbol{R})$.

      * If the step is accepted, then we set $\boldsymbol{R}=\boldsymbol{R}_p$.

      * Update averages

   * Finish and compute final averages.

Observe that the jumping in space is governed by the variable *step*. This is called brute-force sampling and is normally replaced by what is called **importance sampling**, discussed in more detail below here.

### Simple example, the hydrogen atom

The radial Schroedinger equation for the hydrogen atom can be
written as (when we have gotten rid of the first derivative term in the kinetic energy and used $rR(r)=u(r)$)

$$
-\frac{\hbar^2}{2m}\frac{d^2 u(r)}{d r^2}-
\left(\frac{ke^2}{r}-\frac{\hbar^2l(l+1)}{2mr^2}\right)u(r)=Eu(r).
$$

We will specialize to the case with $l=0$ and end up with

$$
-\frac{\hbar^2}{2m}\frac{d^2 u(r)}{d r^2}-
\left(\frac{ke^2}{r}\right)u(r)=Eu(r).
$$

Then we introduce a dimensionless variable $\rho=r/a$ where $a$ is a constant with dimension length.
Multiplying with $ma^2/\hbar^2$ we can rewrite our equations as

$$
-\frac{1}{2}\frac{d^2 u(\rho)}{d \rho^2}-
\frac{ke^2ma}{\hbar^2}\frac{u(\rho)}{\rho}-\lambda u(\rho)=0.
$$

Since $a$ is just a parameter we choose to set

$$
\frac{ke^2ma}{\hbar^2}=1,
$$

which leads to $a=\hbar^2/mke^2$, better known as the Bohr radius with value $0.053$ nm. Scaling the equations this way does not only render our numerical treatment simpler since we avoid carrying with us all physical parameters, but we obtain also a **natural** length scale. We will see this again and again. In our discussions below with a harmonic oscillator trap, the **natural** lentgh scale with be determined by the oscillator frequency, the mass of the particle and $\hbar$. We have also defined a dimensionless 'energy' $\lambda = Ema^2/\hbar^2$.
With the rescaled quantities, the ground state energy of the hydrogen atom is $1/2$.
The equation we want to solve is now defined by the Hamiltonian

$$
H=-\frac{1}{2}\frac{d^2 }{d \rho^2}-\frac{1}{\rho}.
$$

As trial wave function we peep now into the analytical solution for
the hydrogen atom and use (with $\alpha$ as a variational parameter)

$$
u_T^{\alpha}(\rho)=\alpha\rho \exp{-(\alpha\rho)}.
$$

Inserting this wave function into the expression for the
local energy $E_L$ gives

$$
E_L(\rho)=-\frac{1}{\rho}-
              \frac{\alpha}{2}\left(\alpha-\frac{2}{\rho}\right).
$$

To have analytical local energies saves us from computing numerically
the second derivative, a feature which often increases our numerical
expenditure with a factor of three or more. Integratng up the local energy (recall to bring back the PDF in the integration) gives  $\overline{E}[\boldsymbol{\alpha}]=\alpha(\alpha/2-1)$.

### Second example, the harmonic oscillator in one dimension

We present here another well-known example, the harmonic oscillator in
one dimension for one particle. This will also serve the aim of
introducing our next model, namely that of interacting electrons in a
harmonic oscillator trap.

Here as well, we do have analytical solutions and the energy of the
ground state, with $\hbar=1$, is $1/2\omega$, with $\omega$ being the
oscillator frequency. We use the following trial wave function

$$
\psi_T(x;\alpha) = \exp{-(\frac{1}{2}\alpha^2x^2)},
$$

which results in a local energy

$$
\frac{1}{2}\left(\alpha^2+x^2(1-\alpha^4)\right).
$$

We can compare our numerically calculated energies with the exact energy as function of $\alpha$

$$
\overline{E}[\alpha] = \frac{1}{4}\left(\alpha^2+\frac{1}{\alpha^2}\right).
$$

Similarly, with the above ansatz, we can also compute the exact variance which reads

$$
\sigma^2[\alpha]=\frac{1}{4}\left(1+(1-\alpha^4)^2\frac{3}{4\alpha^4}\right)-\overline{E}.
$$

Our code for computing the energy of the ground state of the harmonic oscillator follows here. We start by defining directories where we store various outputs.

```python
# Common imports
import os

# Where to save the figures and data files
PROJECT_ROOT_DIR = "Results"
FIGURE_ID = "Results/FigureFiles"
DATA_ID = "Results/VMCHarmonic"

if not os.path.exists(PROJECT_ROOT_DIR):
    os.mkdir(PROJECT_ROOT_DIR)

if not os.path.exists(FIGURE_ID):
    os.makedirs(FIGURE_ID)

if not os.path.exists(DATA_ID):
    os.makedirs(DATA_ID)

def image_path(fig_id):
    return os.path.join(FIGURE_ID, fig_id)

def data_path(dat_id):
    return os.path.join(DATA_ID, dat_id)

def save_fig(fig_id):
    plt.savefig(image_path(fig_id) + ".png", format='png')

outfile = open(data_path("VMCHarmonic.dat"),'w')
```

We proceed with the implementation of the Monte Carlo algorithm but list first the ansatz for the wave function and the expression for the local energy

```python
%matplotlib inline

# VMC for the one-dimensional harmonic oscillator
# Brute force Metropolis, no importance sampling and no energy minimization
from math import exp, sqrt
from random import random, seed
import numpy as np
import matplotlib.pyplot as plt
from numba import jit
from decimal import *
# Trial wave function for the Harmonic oscillator in one dimension
def WaveFunction(r,alpha):
    return exp(-0.5*alpha*alpha*r*r)

# Local energy  for the Harmonic oscillator in one dimension
def LocalEnergy(r,alpha):
    return 0.5*r*r*(1-alpha**4) + 0.5*alpha*alpha
```

Note that in the Metropolis algorithm there is no need to compute the
trial wave function, mainly since we are just taking the ratio of two
exponentials. It is then from a computational point view, more
convenient to compute the argument from the ratio and then calculate
the exponential. Here we have refrained from this purely of
pedagogical reasons.

```python
# The Monte Carlo sampling with the Metropolis algo
# The jit decorator tells Numba to compile this function.
# The argument types will be inferred by Numba when the function is called.
@jit
def MonteCarloSampling():

    NumberMCcycles= 100000
    StepSize = 1.0
    # positions
    PositionOld = 0.0
    PositionNew = 0.0

    # seed for rng generator
    seed()
    # start variational parameter
    alpha = 0.4
    for ia in range(MaxVariations):
        alpha += .05
        AlphaValues[ia] = alpha
        energy = energy2 = 0.0
        #Initial position
        PositionOld = StepSize * (random() - .5)
        wfold = WaveFunction(PositionOld,alpha)
        #Loop over MC MCcycles
        for MCcycle in range(NumberMCcycles):
            #Trial position
            PositionNew = PositionOld + StepSize*(random() - .5)
            wfnew = WaveFunction(PositionNew,alpha)
            #Metropolis test to see whether we accept the move
            if random() <= wfnew**2 / wfold**2:
                PositionOld = PositionNew
                wfold = wfnew
            DeltaE = LocalEnergy(PositionOld,alpha)
            energy += DeltaE
            energy2 += DeltaE**2
        #We calculate mean, variance and error
        energy /= NumberMCcycles
        energy2 /= NumberMCcycles
        variance = energy2 - energy**2
        error = sqrt(variance/NumberMCcycles)
        Energies[ia] = energy
        Variances[ia] = variance
        outfile.write('%f %f %f %f \n' %(alpha,energy,variance,error))
    return Energies, AlphaValues, Variances
```

Finally, the results are presented here with the exact energies and variances as well.

```python
#Here starts the main program with variable declarations
MaxVariations = 20
Energies = np.zeros((MaxVariations))
ExactEnergies = np.zeros((MaxVariations))
ExactVariance = np.zeros((MaxVariations))
Variances = np.zeros((MaxVariations))
AlphaValues = np.zeros(MaxVariations)
(Energies, AlphaValues, Variances) = MonteCarloSampling()
outfile.close()
ExactEnergies = 0.25*(AlphaValues*AlphaValues+1.0/(AlphaValues*AlphaValues))
ExactVariance = 0.25*(1.0+((1.0-AlphaValues**4)**2)*3.0/(4*(AlphaValues**4)))-ExactEnergies*ExactEnergies

#simple subplot
plt.subplot(2, 1, 1)
plt.plot(AlphaValues, Energies, 'o-',AlphaValues, ExactEnergies,'r-')
plt.title('Energy and variance')
plt.ylabel('Dimensionless energy')
plt.subplot(2, 1, 2)
plt.plot(AlphaValues, Variances, '.-',AlphaValues, ExactVariance,'r-')
plt.xlabel(r'$\alpha$', fontsize=15)
plt.ylabel('Variance')
save_fig("VMCHarmonic")
plt.show()
#nice printout with Pandas
import pandas as pd
from pandas import DataFrame
data ={'Alpha':AlphaValues, 'Energy':Energies,'Exact Energy':ExactEnergies,'Variance':Variances,'Exact Variance':ExactVariance,}
frame = pd.DataFrame(data)
print(frame)
```

For $\alpha=1$ we have the exact eigenpairs, as can be deduced from the
table here. With $\omega=1$, the exact energy is $1/2$ a.u. with zero
variance, as it should. We see also that our computed variance follows rather well the exact variance.
Increasing the number of Monte Carlo cycles will improve our statistics (try to increase the number of Monte Carlo cycles).

The fact that the variance is exactly equal to zero when $\alpha=1$ is that
we then have the exact wave function, and the action of the hamiltionan
on the wave function

$$
H\psi = \mathrm{constant}\times \psi,
$$

yields just a constant. The integral which defines various
expectation values involving moments of the hamiltonian becomes then

$$
\langle H^n \rangle =
   \frac{\int d\boldsymbol{R}\Psi^{\ast}_T(\boldsymbol{R})H^n(\boldsymbol{R})\Psi_T(\boldsymbol{R})}
        {\int d\boldsymbol{R}\Psi^{\ast}_T(\boldsymbol{R})\Psi_T(\boldsymbol{R})}=
\mathrm{constant}\times\frac{\int d\boldsymbol{R}\Psi^{\ast}_T(\boldsymbol{R})\Psi_T(\boldsymbol{R})}
        {\int d\boldsymbol{R}\Psi^{\ast}_T(\boldsymbol{R})\Psi_T(\boldsymbol{R})}=\mathrm{constant}.
$$

**This gives an important information: the exact wave function leads to zero variance!**
As we will see below, many practitioners perform a minimization on both the energy and the variance.

## The Metropolis algorithm

Till now we have not yet discussed the derivation of the Metropolis algorithm. We assume the reader has some familiarity with the mathematics of Markov chains.

The Metropolis algorithm , see [the original article](http://scitation.aip.org/content/aip/journal/jcp/21/6/10.1063/1.1699114), was invented by Metropolis et. al
and is often simply called the Metropolis algorithm.
It is a method to sample a normalized probability
distribution by a stochastic process. We define $\mathbf{P}_i^{(n)}$ to
be the probability for finding the system in the state $i$ at step $n$.
The algorithm is then

* Sample a possible new state $j$ with some probability $T_{i\rightarrow j}$.

* Accept the new state $j$ with probability $A_{i \rightarrow j}$ and use it as the next sample. With probability $1-A_{i\rightarrow j}$ the move is rejected and the original state $i$ is used again as a sample.

We wish to derive the required properties of $T$ and $A$ such that
$\mathbf{P}_i^{(n\rightarrow \infty)} \rightarrow p_i$ so that starting
from any distribution, the method converges to the correct distribution.
Note that the description here is for a discrete probability distribution.
Replacing probabilities $p_i$ with expressions like $p(x_i)dx_i$ will
take all of these over to the corresponding continuum expressions.

The dynamical equation for $\mathbf{P}_i^{(n)}$ can be written directly from
the description above. The probability of being in the state $i$ at step $n$
is given by the probability of being in any state $j$ at the previous step,
and making an accepted transition to $i$ added to the probability of
being in the state $i$, making a transition to any state $j$ and
rejecting the move:

$$
\mathbf{P}^{(n)}_i = \sum_j \left [
\mathbf{P}^{(n-1)}_jT_{j\rightarrow i} A_{j\rightarrow i}
+\mathbf{P}^{(n-1)}_iT_{i\rightarrow j}\left ( 1- A_{i\rightarrow j} \right)
\right ] \,.
$$

Since the probability of making some transition must be 1,
$\sum_j T_{i\rightarrow j} = 1$, and the above equation becomes

$$
\mathbf{P}^{(n)}_i = \mathbf{P}^{(n-1)}_i +
 \sum_j \left [
\mathbf{P}^{(n-1)}_jT_{j\rightarrow i} A_{j\rightarrow i}
-\mathbf{P}^{(n-1)}_iT_{i\rightarrow j}A_{i\rightarrow j}
\right ] \,.
$$

For large $n$ we require that $\mathbf{P}^{(n\rightarrow \infty)}_i = p_i$,
the desired probability distribution. Taking this limit, gives the
balance requirement

$$
\sum_j \left [
p_jT_{j\rightarrow i} A_{j\rightarrow i}
-p_iT_{i\rightarrow j}A_{i\rightarrow j}
\right ] = 0 \,.
$$

The balance requirement is very weak. Typically the much stronger detailed
balance requirement is enforced, that is rather than the sum being
set to zero, we set each term separately to zero and use this
to determine the acceptance probabilities. Rearranging, the result is

$$
\frac{ A_{j\rightarrow i}}{A_{i\rightarrow j}}
= \frac{p_iT_{i\rightarrow j}}{ p_jT_{j\rightarrow i}} \,.
$$

The Metropolis choice is to maximize the $A$ values, that is

$$
A_{j \rightarrow i} = \min \left ( 1,
\frac{p_iT_{i\rightarrow j}}{ p_jT_{j\rightarrow i}}\right ).
$$

Other choices are possible, but they all correspond to multilplying
$A_{i\rightarrow j}$ and $A_{j\rightarrow i}$ by the same constant
smaller than unity.\footnote{The penalty function method uses just such
a factor to compensate for $p_i$ that are evaluated stochastically
and are therefore noisy.}

Having chosen the acceptance probabilities, we have guaranteed that
if the  $\mathbf{P}_i^{(n)}$ has equilibrated, that is if it is equal to $p_i$,
it will remain equilibrated. Next we need to find the circumstances for
convergence to equilibrium.

The dynamical equation can be written as

$$
\mathbf{P}^{(n)}_i = \sum_j M_{ij}\mathbf{P}^{(n-1)}_j
$$

with the matrix $M$ given by

$$
M_{ij} = \delta_{ij}\left [ 1 -\sum_k T_{i\rightarrow k} A_{i \rightarrow k}
\right ] + T_{j\rightarrow i} A_{j\rightarrow i} \,.
$$

Summing over $i$ shows that $\sum_i M_{ij} = 1$, and since
$\sum_k T_{i\rightarrow k} = 1$, and $A_{i \rightarrow k} \leq 1$, the
elements of the matrix satisfy $M_{ij} \geq 0$. The matrix $M$ is therefore
a stochastic matrix.

The Metropolis method is simply the power method for computing the
right eigenvector of $M$ with the largest magnitude eigenvalue.
By construction, the correct probability distribution is a right eigenvector
with eigenvalue 1. Therefore, for the Metropolis method to converge
to this result, we must show that $M$ has only one eigenvalue with this
magnitude, and all other eigenvalues are smaller.

## The system: two electrons in a harmonic oscillator trap in two dimensions

The Hamiltonian of the quantum dot is given by

$$
\hat{H} = \hat{H}_0 + \hat{V},
$$

where $\hat{H}_0$ is the many-body HO Hamiltonian, and $\hat{V}$ is the
inter-electron Coulomb interactions. In dimensionless units,

$$
\hat{V}= \sum_{i < j}^N \frac{1}{r_{ij}},
$$

with $r_{ij}=\sqrt{\mathbf{r}_i^2 - \mathbf{r}_j^2}$.

This leads to the  separable Hamiltonian, with the relative motion part given by ($r_{ij}=r$)

$$
\hat{H}_r=-\nabla^2_r + \frac{1}{4}\omega^2r^2+ \frac{1}{r},
$$

plus a standard Harmonic Oscillator problem  for the center-of-mass motion.
This system has analytical solutions in two and three dimensions ([M. Taut 1993 and 1994](https://journals.aps.org/pra/abstract/10.1103/PhysRevA.48.3561)).

We want to perform  a Variational Monte Carlo calculation of the ground state of two electrons in a quantum dot well with different oscillator energies, assuming total spin $S=0$.
Our trial wave function has the following form

$$
\begin{equation}
   \psi_{T}(\boldsymbol{r}_1,\boldsymbol{r}_2) =
   C\exp{\left(-\alpha_1\omega(r_1^2+r_2^2)/2\right)}
   \exp{\left(\frac{r_{12}}{(1+\alpha_2 r_{12})}\right)},
\label{eq:trial} \tag{3}
\end{equation}
$$

where the $\alpha$s represent our variational parameters, two in this case.

Why does the trial function look like this? How did we get there?
**This will be one of our main motivations** for switching to Machine Learning later.

To find an ansatz for the correlated part of the wave function, it is
useful to rewrite the two-particle local energy in terms of the
relative and center-of-mass motion.
Let us denote the distance
between the two electrons as $r_{12}$. We omit the center-of-mass
motion since we are only interested in the case when $r_{12}
\rightarrow 0$. The contribution from the center-of-mass (CoM)
variable $\boldsymbol{R}_{\mathrm{CoM}}$ gives only a finite contribution.  We
focus only on the terms that are relevant for $r_{12}$ and for three
dimensions.

The relevant local energy becomes then

$$
\lim_{r_{12} \rightarrow 0}E_L(R)= \frac{1}{{\cal R}_T(r_{12})}\left(2\frac{d^2}{dr_{ij}^2}+\frac{4}{r_{ij}}\frac{d}{dr_{ij}}+\frac{2}{r_{ij}}-\frac{l(l+1)}{r_{ij}^2}+2E \right){\cal R}_T(r_{12})
= 0.
$$

Set $l=0$ and we have the so-called **cusp** condition

$$
\frac{d {\cal R}_T(r_{12})}{dr_{12}} = -\frac{1}{2(l+1)} {\cal R}_T(r_{12})\qquad r_{12}\to 0
$$

The above  results in

$$
{\cal R}_T  \propto \exp{(r_{ij}/2)},
$$

for anti-parallel spins and

$$
{\cal R}_T  \propto \exp{(r_{ij}/4)},
$$

for anti-parallel spins.
This is the so-called cusp condition for the relative motion, resulting in a minimal requirement
for the correlation part of the wave fuction.
For general systems containing more than say two electrons, we have this
condition for each electron pair $ij$.

### First code attempt for the two-electron case

First, as with the hydrogen case, we declare where to store files.

```python
# Common imports
import os

# Where to save the figures and data files
PROJECT_ROOT_DIR = "Results"
FIGURE_ID = "Results/FigureFiles"
DATA_ID = "Results/VMCQdotMetropolis"

if not os.path.exists(PROJECT_ROOT_DIR):
    os.mkdir(PROJECT_ROOT_DIR)

if not os.path.exists(FIGURE_ID):
    os.makedirs(FIGURE_ID)

if not os.path.exists(DATA_ID):
    os.makedirs(DATA_ID)

def image_path(fig_id):
    return os.path.join(FIGURE_ID, fig_id)

def data_path(dat_id):
    return os.path.join(DATA_ID, dat_id)

def save_fig(fig_id):
    plt.savefig(image_path(fig_id) + ".png", format='png')

outfile = open(data_path("VMCQdotMetropolis.dat"),'w')
```

Thereafter we set up the analytical expressions for the wave functions and the local energy

```python
# 2-electron VMC for quantum dot system in two dimensions
# Brute force Metropolis, no importance sampling and no energy minimization
from math import exp, sqrt
from random import random, seed
import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
from matplotlib import cm
from matplotlib.ticker import LinearLocator, FormatStrFormatter
import sys
#from numba import jit

# Trial wave function for the 2-electron quantum dot in two dims
def WaveFunction(r,alpha,beta):
    r1 = r[0,0]**2 + r[0,1]**2
    r2 = r[1,0]**2 + r[1,1]**2
    r12 = sqrt((r[0,0]-r[1,0])**2 + (r[0,1]-r[1,1])**2)
    deno = r12/(1+beta*r12)
    return exp(-0.5*alpha*(r1+r2)+deno)

# Local energy  for the 2-electron quantum dot in two dims, using analytical local energy
def LocalEnergy(r,alpha,beta):

    r1 = (r[0,0]**2 + r[0,1]**2)
    r2 = (r[1,0]**2 + r[1,1]**2)
    r12 = sqrt((r[0,0]-r[1,0])**2 + (r[0,1]-r[1,1])**2)
    deno = 1.0/(1+beta*r12)
    deno2 = deno*deno
    return 0.5*(1-alpha*alpha)*(r1 + r2) +2.0*alpha + 1.0/r12+deno2*(alpha*r12-deno2+2*beta*deno-1.0/r12)
```

The Monte Carlo sampling without importance sampling is set up here.

```python
# The Monte Carlo sampling with the Metropolis algo
# The jit decorator tells Numba to compile this function.
# The argument types will be inferred by Numba when the function is called.
#@jit
def MonteCarloSampling():

    NumberMCcycles= 10000
    StepSize = 1.0
    # positions
    PositionOld = np.zeros((NumberParticles,Dimension), np.double)
    PositionNew = np.zeros((NumberParticles,Dimension), np.double)

    # seed for rng generator
    seed()
    # start variational parameter
    alpha = 0.9
    for ia in range(MaxVariations):
        alpha += .025
        AlphaValues[ia] = alpha
        beta = 0.2
        for jb in range(MaxVariations):
            beta += .01
            BetaValues[jb] = beta
            energy = energy2 = 0.0
            DeltaE = 0.0
            #Initial position
            for i in range(NumberParticles):
                for j in range(Dimension):
                    PositionOld[i,j] = StepSize * (random() - .5)
            wfold = WaveFunction(PositionOld,alpha,beta)

            #Loop over MC MCcycles
            for MCcycle in range(NumberMCcycles):
                #Trial position moving one particle at the time
                for i in range(NumberParticles):
                    for j in range(Dimension):
                        PositionNew[i,j] = PositionOld[i,j] + StepSize * (random() - .5)
                    wfnew = WaveFunction(PositionNew,alpha,beta)

                    #Metropolis test to see whether we accept the move
                    if random() < wfnew**2 / wfold**2:
                       for j in range(Dimension):
                           PositionOld[i,j] = PositionNew[i,j]
                       wfold = wfnew
                DeltaE = LocalEnergy(PositionOld,alpha,beta)
                energy += DeltaE
                energy2 += DeltaE**2
            #We calculate mean, variance and error ...
            energy /= NumberMCcycles
            energy2 /= NumberMCcycles
            variance = energy2 - energy**2
            error = sqrt(variance/NumberMCcycles)
            Energies[ia,jb] = energy
            Variances[ia,jb] = variance
            outfile.write('%f %f %f %f %f\n' %(alpha,beta,energy,variance,error))
    return Energies, Variances, AlphaValues, BetaValues
```

And finally comes the main part with the plots as well.

```python
#Here starts the main program with variable declarations
NumberParticles = 2
Dimension = 2
MaxVariations = 10
Energies = np.zeros((MaxVariations,MaxVariations))
Variances = np.zeros((MaxVariations,MaxVariations))
AlphaValues = np.zeros(MaxVariations)
BetaValues = np.zeros(MaxVariations)
(Energies, Variances, AlphaValues, BetaValues) = MonteCarloSampling()
outfile.close()

# Prepare for plots
fig = plt.figure()
ax = fig.gca(projection='3d')
# Plot the surface.
X, Y = np.meshgrid(AlphaValues, BetaValues)
surf = ax.plot_surface(X, Y, Energies,cmap=cm.coolwarm,linewidth=0, antialiased=False)
# Customize the z axis.
zmin = np.matrix(Energies).min()
zmax = np.matrix(Energies).max()
ax.set_zlim(zmin, zmax)
ax.set_xlabel(r'$\alpha$')
ax.set_ylabel(r'$\beta$')
ax.set_zlabel(r'$\langle E \rangle$')
ax.zaxis.set_major_locator(LinearLocator(10))
ax.zaxis.set_major_formatter(FormatStrFormatter('%.02f'))
# Add a color bar which maps values to colors.
fig.colorbar(surf, shrink=0.5, aspect=5)
save_fig("QdotMetropolis")
plt.show()
```

```python

```


# DOF gradients

Compute the final gradients needed for the jacobian calculation.

```python
import numpy as np
import sympy
import tensor_tools as tt
```

```python
zero = sympy.S(0)
one  = sympy.S(1)

delta_Kalpha = tt.form_tensor('_',[3,12])

for i in range(3):
    for j in range(12):
        delta_Kalpha[i][j] = zero

delta_Kalpha[0][0] = one
delta_Kalpha[1][1] = one
delta_Kalpha[2][2] = one

delta_Kalpha
```

    [[1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
     [0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
     [0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0]]

```python
delta_kKalpha = tt.form_tensor('_',[9,12])

for i in range(9):
    for j in range(12):
        delta_kKalpha[i][j] = zero

for i in range(9):
    delta_kKalpha[i][i+3] = one

delta_kKalpha
```

    [[0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0],
     [0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0],
     [0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0],
     [0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0],
     [0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0],
     [0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0],
     [0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0],
     [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0],
     [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1]]

```python
sot_voigt_map = tt.get_voigt_mapping(2)
tot_voigt_map = tt.get_voigt_mapping(3)
```

```python
N = sympy.symbols('N')
```

```python
dNdx = [sympy.symbols('dNdx'+str(v+1)) for v in range(3)]
```

Compute voigt matrix form of $\frac{\partial u_{K,k}}{\partial \hat{u}_{\alpha}}$

```python
dgrad_udU = tt.form_tensor('_',[9,12])

string = ''
for K in range(3):
    for k in range(3):
        Khat = sot_voigt_map.index([K,k])

        for alpha in range(12):

            dgrad_udU[Khat][alpha] = delta_Kalpha[K][alpha]*dNdx[k]

            if(dgrad_udU[Khat][alpha] != zero):
                string += " "*8 + "tripletList.push_back(T({0},{1},dNdx[{2}]));\n".format(Khat,alpha,k)
print(string)
string = ''
for K in range(3):
    for k in range(3):
        Khat = sot_voigt_map.index([K,k])

        for alpha in range(12):

            dgrad_udU[Khat][alpha] = delta_Kalpha[K][alpha]*dNdx[k]

            if(dgrad_udU[Khat][alpha] != zero):
                string += " "*8 + "dgrad_udU({0},{1}) = detadx[{2}];\n".format(Khat,alpha,k)
print(string)
dgrad_udU
```

            tripletList.push_back(T(0,0,dNdx[0]));
            tripletList.push_back(T(5,0,dNdx[1]));
            tripletList.push_back(T(4,0,dNdx[2]));
            tripletList.push_back(T(8,1,dNdx[0]));
            tripletList.push_back(T(1,1,dNdx[1]));
            tripletList.push_back(T(3,1,dNdx[2]));
            tripletList.push_back(T(7,2,dNdx[0]));
            tripletList.push_back(T(6,2,dNdx[1]));
            tripletList.push_back(T(2,2,dNdx[2]));

            dgrad_udU(0,0) = detadx[0];
            dgrad_udU(5,0) = detadx[1];
            dgrad_udU(4,0) = detadx[2];
            dgrad_udU(8,1) = detadx[0];
            dgrad_udU(1,1) = detadx[1];
            dgrad_udU(3,1) = detadx[2];
            dgrad_udU(7,2) = detadx[0];
            dgrad_udU(6,2) = detadx[1];
            dgrad_udU(2,2) = detadx[2];

    [[dNdx1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
     [0, dNdx2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
     [0, 0, dNdx3, 0, 0, 0, 0, 0, 0, 0, 0, 0],
     [0, dNdx3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
     [dNdx3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
     [dNdx2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
     [0, 0, dNdx2, 0, 0, 0, 0, 0, 0, 0, 0, 0],
     [0, 0, dNdx1, 0, 0, 0, 0, 0, 0, 0, 0, 0],
     [0, dNdx1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]]

Compute voigt matrix form of $\frac{\partial \phi_{kK} \phi}{\partial \hat{u}_{\alpha}}$

```python
dphidU = tt.form_tensor('_',[9,12])

string = ''
for K in range(3):
    for k in range(3):
        Khat = sot_voigt_map.index([K,k])

        for alpha in range(12):

            dphidU[Khat][alpha] = delta_kKalpha[Khat][alpha]*N

            if(dphidU[Khat][alpha] != zero):
                string += " "*8 + "tripletList.push_back(T({0},{1},N));\n".format(Khat,alpha)
print(string)
string = ''
for K in range(3):
    for k in range(3):
        Khat = sot_voigt_map.index([K,k])

        for alpha in range(12):

            dphidU[Khat][alpha] = delta_kKalpha[Khat][alpha]*N

            if(dphidU[Khat][alpha] != zero):
                string += " "*8 + "dphidU({0:2},{1:2}) = eta;\n".format(Khat,alpha)
print(string)
dphidU
```

            tripletList.push_back(T(0,3,N));
            tripletList.push_back(T(5,8,N));
            tripletList.push_back(T(4,7,N));
            tripletList.push_back(T(8,11,N));
            tripletList.push_back(T(1,4,N));
            tripletList.push_back(T(3,6,N));
            tripletList.push_back(T(7,10,N));
            tripletList.push_back(T(6,9,N));
            tripletList.push_back(T(2,5,N));

            dphidU( 0, 3) = eta;
            dphidU( 5, 8) = eta;
            dphidU( 4, 7) = eta;
            dphidU( 8,11) = eta;
            dphidU( 1, 4) = eta;
            dphidU( 3, 6) = eta;
            dphidU( 7,10) = eta;
            dphidU( 6, 9) = eta;
            dphidU( 2, 5) = eta;

    [[0, 0, 0, N, 0, 0, 0, 0, 0, 0, 0, 0],
     [0, 0, 0, 0, N, 0, 0, 0, 0, 0, 0, 0],
     [0, 0, 0, 0, 0, N, 0, 0, 0, 0, 0, 0],
     [0, 0, 0, 0, 0, 0, N, 0, 0, 0, 0, 0],
     [0, 0, 0, 0, 0, 0, 0, N, 0, 0, 0, 0],
     [0, 0, 0, 0, 0, 0, 0, 0, N, 0, 0, 0],
     [0, 0, 0, 0, 0, 0, 0, 0, 0, N, 0, 0],
     [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, N, 0],
     [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, N]]

Compute voigt matrix form of $\frac{\partial \phi_{kK} \phi_{,l}}{\partial \hat{u}_{\alpha}}$

```python
dgrad_phidU = tt.form_tensor('_',[27,12])

string = ''
n=0
for K in range(3):
    for k in range(3):
        Khat = sot_voigt_map.index([k,K])
        for l in range(3):
            Khatl = tot_voigt_map.index([k,K,l])
            for alpha in range(12):

                dgrad_phidU[Khatl][alpha] = delta_kKalpha[Khat][alpha]*dNdx[l]

                if(dgrad_phidU[Khatl][alpha] != zero):
                    string += " "*8 + "tripletList.push_back(T({0},{1},dNdx[{2}]));\n".format(Khatl,alpha,l)
                    n += 1
print(n)
print(string)

string = ''
n=0
for K in range(3):
    for k in range(3):
        Khat = sot_voigt_map.index([k,K])
        for l in range(3):
            Khatl = tot_voigt_map.index([k,K,l])
            for alpha in range(12):

                dgrad_phidU[Khatl][alpha] = delta_kKalpha[Khat][alpha]*dNdx[l]

                if(dgrad_phidU[Khatl][alpha] != zero):
                    string += " "*8 + "dgrad_phidU({0:2},{1:2}) = detadx[{2}];\n".format(Khatl,alpha,l)
                    n += 1
print(n)
print(string)
dgrad_phidU
```

    27
            tripletList.push_back(T(0,3,dNdx[0]));
            tripletList.push_back(T(5,3,dNdx[1]));
            tripletList.push_back(T(4,3,dNdx[2]));
            tripletList.push_back(T(9,11,dNdx[0]));
            tripletList.push_back(T(14,11,dNdx[1]));
            tripletList.push_back(T(13,11,dNdx[2]));
            tripletList.push_back(T(18,10,dNdx[0]));
            tripletList.push_back(T(23,10,dNdx[1]));
            tripletList.push_back(T(22,10,dNdx[2]));
            tripletList.push_back(T(8,8,dNdx[0]));
            tripletList.push_back(T(1,8,dNdx[1]));
            tripletList.push_back(T(3,8,dNdx[2]));
            tripletList.push_back(T(17,4,dNdx[0]));
            tripletList.push_back(T(10,4,dNdx[1]));
            tripletList.push_back(T(12,4,dNdx[2]));
            tripletList.push_back(T(26,9,dNdx[0]));
            tripletList.push_back(T(19,9,dNdx[1]));
            tripletList.push_back(T(21,9,dNdx[2]));
            tripletList.push_back(T(7,7,dNdx[0]));
            tripletList.push_back(T(6,7,dNdx[1]));
            tripletList.push_back(T(2,7,dNdx[2]));
            tripletList.push_back(T(16,6,dNdx[0]));
            tripletList.push_back(T(15,6,dNdx[1]));
            tripletList.push_back(T(11,6,dNdx[2]));
            tripletList.push_back(T(25,5,dNdx[0]));
            tripletList.push_back(T(24,5,dNdx[1]));
            tripletList.push_back(T(20,5,dNdx[2]));

    27
            dgrad_phidU( 0, 3) = detadx[0];
            dgrad_phidU( 5, 3) = detadx[1];
            dgrad_phidU( 4, 3) = detadx[2];
            dgrad_phidU( 9,11) = detadx[0];
            dgrad_phidU(14,11) = detadx[1];
            dgrad_phidU(13,11) = detadx[2];
            dgrad_phidU(18,10) = detadx[0];
            dgrad_phidU(23,10) = detadx[1];
            dgrad_phidU(22,10) = detadx[2];
            dgrad_phidU( 8, 8) = detadx[0];
            dgrad_phidU( 1, 8) = detadx[1];
            dgrad_phidU( 3, 8) = detadx[2];
            dgrad_phidU(17, 4) = detadx[0];
            dgrad_phidU(10, 4) = detadx[1];
            dgrad_phidU(12, 4) = detadx[2];
            dgrad_phidU(26, 9) = detadx[0];
            dgrad_phidU(19, 9) = detadx[1];
            dgrad_phidU(21, 9) = detadx[2];
            dgrad_phidU( 7, 7) = detadx[0];
            dgrad_phidU( 6, 7) = detadx[1];
            dgrad_phidU( 2, 7) = detadx[2];
            dgrad_phidU(16, 6) = detadx[0];
            dgrad_phidU(15, 6) = detadx[1];
            dgrad_phidU(11, 6) = detadx[2];
            dgrad_phidU(25, 5) = detadx[0];
            dgrad_phidU(24, 5) = detadx[1];
            dgrad_phidU(20, 5) = detadx[2];

    [[0, 0, 0, dNdx1, 0, 0, 0, 0, 0, 0, 0, 0],
     [0, 0, 0, 0, 0, 0, 0, 0, dNdx2, 0, 0, 0],
     [0, 0, 0, 0, 0, 0, 0, dNdx3, 0, 0, 0, 0],
     [0, 0, 0, 0, 0, 0, 0, 0, dNdx3, 0, 0, 0],
     [0, 0, 0, dNdx3, 0, 0, 0, 0, 0, 0, 0, 0],
     [0, 0, 0, dNdx2, 0, 0, 0, 0, 0, 0, 0, 0],
     [0, 0, 0, 0, 0, 0, 0, dNdx2, 0, 0, 0, 0],
     [0, 0, 0, 0, 0, 0, 0, dNdx1, 0, 0, 0, 0],
     [0, 0, 0, 0, 0, 0, 0, 0, dNdx1, 0, 0, 0],
     [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, dNdx1],
     [0, 0, 0, 0, dNdx2, 0, 0, 0, 0, 0, 0, 0],
     [0, 0, 0, 0, 0, 0, dNdx3, 0, 0, 0, 0, 0],
     [0, 0, 0, 0, dNdx3, 0, 0, 0, 0, 0, 0, 0],
     [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, dNdx3],
     [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, dNdx2],
     [0, 0, 0, 0, 0, 0, dNdx2, 0, 0, 0, 0, 0],
     [0, 0, 0, 0, 0, 0, dNdx1, 0, 0, 0, 0, 0],
     [0, 0, 0, 0, dNdx1, 0, 0, 0, 0, 0, 0, 0],
     [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, dNdx1, 0],
     [0, 0, 0, 0, 0, 0, 0, 0, 0, dNdx2, 0, 0],
     [0, 0, 0, 0, 0, dNdx3, 0, 0, 0, 0, 0, 0],
     [0, 0, 0, 0, 0, 0, 0, 0, 0, dNdx3, 0, 0],
     [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, dNdx3, 0],
     [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, dNdx2, 0],
     [0, 0, 0, 0, 0, dNdx2, 0, 0, 0, 0, 0, 0],
     [0, 0, 0, 0, 0, dNdx1, 0, 0, 0, 0, 0, 0],
     [0, 0, 0, 0, 0, 0, 0, 0, 0, dNdx1, 0, 0]]

```python

```