# Chapter 3: Classical Computation and Logic

So far, we have laid out the mathematical foundation necessary for introductory quantum computing, which as shown before, is mere vector and matrix algebra. This third, and short, chapter will be dedicated towards defining a few basic concepts of classical computation, not because it is a necessary prerequisite (although it does help), but rather to describe how information could be processed using classical deterministic rules, as opposed to the ones that will be presented in the next chapter. The Python Module that will be used here is Truth-Table-Generator which, as the name suggests, is used to generate truth tables by simply passing the logic operations through the function ```ttg.Truths()```.

## 3.1 What is Computation?

   According to _Google_, a computer is a machine that can be programmed to carry out sequences of arithmetic or logical operations automatically. In other words, it is a device that one can, in one way or another, use to process information using a series of steps that one defines. These series of steps are called algorithms, and can be as simple as adding two numbers or as complex as suggesting which cat videos you would prefer to watch on YouTube. However, all of these terms are ambiguous, as one could ask questions such as "What is meant by information? How is it represented? How are these logical operations implemented to begin with?" The answer is that there are an infinite  number of ways that computation can be performed. The model that is ubiquitously used, however, is the method of using digital circuits to represent and process binary data (i.e information that could be in one of two states, typically represented by 1 or 0). Physically, this model of information proccessing could be implemented by using a device called a transistor which has the ability to switch between high voltage signals (i.e 1) and low voltage signals (i.e 0). Your computer likely has billions of those, which, using complex architecture, work together to process the information and run the algorithms necessary to output programs like operating systems, web browsers, amazing Jupyter Notebooks about Quantum Information, etc.

### Why Binary?

   It is common knowledge that computers process information in binary form. To answer why this is, let us think about one of the simplest algorithms: counting. Historically, people have counted in multiple ways. The most primitive way to count is by using a system that represents the number $n$ with $n$ units of, say, a stripe **I**. For example, the number 5 could be represented by **IIIII**, and the number 13 by **IIIIIIIIIIIII**. If a computer were to implement this _tally_ system for counting, then it could represent the number $n$ by $n$ high voltage signals (i.e $n$ 1s). This is a very inefficient model, and even with an algorithm as simple as counting, will very quickly prove to be inept. For this reason, a digital system must be deployed, as it offers an exponentially more efficient way of representing data. For instance, the following image shows the number of digits (and concequently, number of transistor-like devices) necessary to represent the number 13.

 As you can see, it takes 13 digits to represent 13 in base 1 (tally), four digits to represent it in base 2 (binary), three digits to represent it in base 3 (trinary), two digits to represent it in base 4 to 13 (quaternary to tridecimal) and only 1 digit to represent it in base 14+. The graph almost looks negative exponential, which suggests that the higher the base, the more efficient the model. So why base two and not a higher base? Can't we define a system with low voltage, high voltage and even higher voltage? The answer is that while that is possible, and we can theoretically build a computer with any number of possible states, the two state system will prove to be most efficient, maybe not in terms of number of transistors, but in terms of design. For this reason, the bit (a 1 or 0) is called the base unit of classical information.

 ## Python Implementation

```python
#Converting a number n from base 1 to base 10
import numpy as np

def tally_to_dec(n):
    n_list = [int(i) for i in str(n)] #Converting n to a list of all the digits in it
    return len(n_list) #Returning the number of digits in n

#Converting a number n from base 2 to 10

def bin_to_dec(n):
    n_list = [int(i) for i in str(n)] #Same as before (100 --> [1,0,0])
    num_digits = len(n_list)
    ones = [2**(num_digits - i) for i in range(num_digits) if n_list[i]==1] # 2^i if n_list(i) is 1
    result = np.array(ones).sum() #Sums all the 2^i where n_list[i] == 1
    return result

a = tally_to_dec(11111111111)
b = bin_to_dec(11010)

print('11111111111 in base 1 is ', a, ' in base 10')
print('11010 in base 2 is ', b, ' in base 10')

#Excercise: Convert from base 8 to 15
```

    11111111111 in base 1 is  11  in base 10
    11010 in base 2 is  52  in base 10

## 3.2 Boolean Algebra

In the Section above, it was made clear that computers operate on bits, or **b**inary dig**its**. In other words, everything that a computer does can be traced down to some series of 1s and 0s. A bit is also called boolean data. Formally, one can say that $x$ is Boolean if $x \in \eta$, where $\eta = $ {0,1}.

Boolean algebra is defined as any operation on $x$ and $y$, for $x,y \in \eta$. There are two main operations (addition and multiplication) that are governed by the following seven axioms:

1.**Closure:** If $x, y \in \eta$, then $O(x,y) \in \eta$, where $O$ is some operation (i.e $x\bullet y$ or $x + y$).

2.**Cardinality:** There are two elements $x$ and $y$ of set $\eta$ for $x \neq y$ (e.g 1 and 0, ```True``` and ```False```, etc.) This was already stated above.

3.**Commutativity:** $O(x,y) = O(y,x)$, where O is some operation that could either be multiplication or addition

4.**Associativity:** $x + (y + z) = (x + y) + z $ and $x \bullet (y \bullet z) = (x \bullet y) \bullet z$. Or generally, $O(x, O(y,z)) = O(O(x,y),z)$.

5.**Distributivity:** $x + (y\bullet z) = (x+y)\bullet(x+z)$ and $x \bullet (y + z) = (x\bullet y) + (x\bullet z)$

6.**Identity Element**: Each operation has its own identity element. For example, (+) has the identity element 0, and ($\bullet$) has the identity element 1. In other words

\begin{equation}
x + 0 = x \newline
x\bullet 1 = x
\end{equation}

7.**Complement Element** for every $x$, there is a $y$ to satisfy

\begin{equation}
x + y = 1 \newline
x \bullet y = 0
\end{equation}

### Examples

Given that there are two operations and two elements, one can find that there are 8 possible solutions to $O(x,y)$ for $x,y \in \eta$. Using the axioms above, one would find them to be

\begin{equation}
0 \bullet 0 = 0 \newline
0 \bullet 1 = 0 \newline
1 \bullet 0 = 0 \newline
1 \bullet 1 = 1 \newline
\newline
0 + 0 = 0 \newline
0 + 1 = 1 \newline
1 + 0 = 1 \newline
1 + 1 = 1 \newline
\end{equation}

A more organized way to represent such operations is with a _truth table_, which will be used for more complex boolean operations in Section 3.3.

### DeMorgan's Law

From the seven axioms presented above, many theorems about boolean operations can be derived. One of the most notable relationships is deMorgan's Law, which states the following:

\begin{equation}
(x + y)' = x' \bullet y'
\end{equation}

where $x'$ is essentially the "inverse" of $x$ (i.e 0' = 1 and 1' = 0). I would encourage one to prove DeMorgan's law from the axioms above, but a more efficient way of proving it is by using a truth table, which will be done in the next chapter

### Python Implementation

```python
#To represent boolean data, python uses True and False. For example:

if True:
    print('Cool')
if False:
    print('Uncool')

#One can use this to build conditions, but I'll stop here before this turns into an intro to Python course

#Excercise: Try to build a function that adds, multiplies and inverses two boolean values
#(Without looking at the next section)

def add(a,b):
    return
def mult(a,b):
    return
def invert(a):
    return
```

    Cool

## 3.3 Logic Gates and Circuits

The axioms presented above are the only things that computers understand, and as stated before, they understand it using transistors which act as logic gates. A logic gate is essentially just a way of representing boolean operations in terms of  a series of inputs and one output. For example, the AND gate will return a 1 if and only if all of its inputs are 1. Notice that it is equivilent to the ($\bullet$) operation where 1$\bullet$1 = 1, and all other scenarios are 0.

### Truth Tables

A key property of logic gates is that while there can be any number of input boolean data, there will always be one boolean output. A truth table is a table that shows what the output will be for any possible combination of inputs, which is $2^n$ where $n$ is the number of input bits. For example, the AND gate, which has been said to represent $x \bullet y$ will have the following truth table:

Inputs are every possible $x$ and $y$ and outputs are $x \bullet y$. Truth tables can be generated in Python by using the ```truth-table-generator``` module as shown bellow:

```python
import ttg
truthTable = ttg.Truths(['x', 'y'], ['x and y'], ascending=True)

print(truthTable)
```

    +-----+-----+-----------+
    |  x  |  y  |  x and y  |
    |-----+-----+-----------|
    |  0  |  0  |     0     |
    |  0  |  1  |     0     |
    |  1  |  0  |     0     |
    |  1  |  1  |     1     |
    +-----+-----+-----------+

```ttg.Truths()``` can take several parameters, but only two are necessary. The first argument, which is a list of strings, specifies the inputs ```['x','y']```. The second is the operations, in this case ```['x and y']```. The third argument is optional, but pretty much sets the inputs in an ascending order (i.e 00,01,10,11) as oppose to the default (11,10,01,00).

## Logic Gate Examples:

Technically, any logic gate could be constructed by a series of boolean operations on any number of bits. However, the following are the most commonly found ones

### _**Buffer Gate:**_

The Buffer gate is a single bit gate that outputs whatever is inputted to it.

**Boolean Operation:** $BUFFER(x) = x$

**Symbol:**

**Truth Table:**

```python
buffer_gate = ttg.Truths(['x'], ['x'], ascending=True)
print('Buffer Gate')
print(buffer_gate)
```

    Buffer Gate
    +-----+-----+
    |  x  |  x  |
    |-----+-----|
    |  0  |  0  |
    |  1  |  1  |
    +-----+-----+

### _**NOT Gate:**_

The NOT gate is a single bit gate that outputs the inverse of the input. It is essentially the opposite of a buffer gate.

**Boolean Operation:** $NOT(x) = x'$

**Symbol:**

**Truth Table:**

```python
not_gate = ttg.Truths(['x'], ['not x'], ascending=True)
print('Not Gate')
print(not_gate)
```

    Not Gate
    +-----+---------+
    |  x  |  not x  |
    |-----+---------|
    |  0  |    1    |
    |  1  |    0    |
    +-----+---------+

### _**AND Gate:**_

The AND gate is a 2 bit gate that outputs 1 only if input signal 1 AND input signal 2 are 1.

**Boolean Operation:** $AND(x,y) = x \bullet y $

**Symbol and Truth Table:**

### _**NAND Gate:**_

The NAND gate is a 2 bit gate that outputs 1 only if input signal 1 AND input signal 2 are 0. It is the equivilent of a NOT AND.

**Boolean Operation:** $NAND(x,y) = (x \bullet y)' $

**Symbol and Truth Table:**

### _**OR Gate:**_

The OR gate is a 2 bit gate that outputs 1 if either input 1 OR input 2 are 1

**Boolean Operation:** $OR(x,y) = x + y $

**Symbol and Truth Table:**

### _**NOR Gate:**_

The NOR gate is a 2 bit gate that outputs 1 if either input 1 OR input 2 are 0. It is the equivilent of a NOT OR

**Boolean Operation:** $NOR(x,y) = (x + y)' $

**Symbol and Truth Table:**

### _**XOR Gate:**_

The XOR, or the Exclusive OR gate is a 2 bit gate that outputs 1 if ONLY either input 1 or input 2 is 1

**Boolean Operation:** $XOR(x,y) = (x + y) \bullet (x \bullet y)'$

Note: if you want to check this out with *ttg*, the syntax is ```ttg.Truths(['x','y'], ['(x or y) and not(x and y)'], ascending=True)```

**Symbol and Truth Table:**

### Example of Using Logic Gates for Computation

Let's say you run a school cafetaria and your one job is to give kids a sandwich and salad. For the sake of simplicity, let's say there are two types of sandwiches, say, vegetable and chicken, and that there are no preferences (i.e there are no students that are exclusively vegitarians). You also happen to be a master of robotics and decide to create a robot that replaces you. Now remember that computers (such as robots) only understand data in boolean form, so you would have to write the operation above in boolean logic, which is either vegetable or chicken sandwich and a salad, or in the language of logic gates: ```(Vegetable XOR Chicken) AND Salad```. The truth table for this will be:

```python
roboLunchLady = ttg.Truths(['V','C','S'], ['(V xor C) and S'])
print('Robo-Lunch-Lady Instructions:')
print(roboLunchLady)
```

    Robo-Lunch-Lady Instructions:
    +-----+-----+-----+-------------------+
    |  V  |  C  |  S  |  (V xor C) and S  |
    |-----+-----+-----+-------------------|
    |  1  |  1  |  1  |         0         |
    |  1  |  1  |  0  |         0         |
    |  1  |  0  |  1  |         1         |
    |  1  |  0  |  0  |         0         |
    |  0  |  1  |  1  |         1         |
    |  0  |  1  |  0  |         0         |
    |  0  |  0  |  1  |         0         |
    |  0  |  0  |  0  |         0         |
    +-----+-----+-----+-------------------+

Robo-Lunch-Lady now knows to only serve a kid if they're asking for a chicken or vegetable sandwich, and a salad. Notice how we used XOR instead of OR so these sneaky kids wouldn't be able to hack their way into having two sandwiches.

### Python Implementation

```python
#Proving deMorgan's theorem with truth tables, as promised in 3.2

deMorgan_left = ttg.Truths(['x','y'], ['not (x or y)'])

deMorgan_right = ttg.Truths(['x','y'], ['not x and not y'])

print('(x + y)\':')
print(deMorgan_left)

print('x\'.y\':')
print(deMorgan_right)

print('They are the same. Note that x and y could also be made up of any arbitrary boolean operations as well (e.g x = a+b, y = a*b')

```

    (x + y)':
    +-----+-----+----------------+
    |  x  |  y  |  not (x or y)  |
    |-----+-----+----------------|
    |  1  |  1  |       0        |
    |  1  |  0  |       0        |
    |  0  |  1  |       0        |
    |  0  |  0  |       1        |
    +-----+-----+----------------+
    x'.y':
    +-----+-----+-------------------+
    |  x  |  y  |  not x and not y  |
    |-----+-----+-------------------|
    |  1  |  1  |         0         |
    |  1  |  0  |         0         |
    |  0  |  1  |         0         |
    |  0  |  0  |         1         |
    +-----+-----+-------------------+
    They are the same. Note that x and y could also be made up of any arbitrary boolean operations as well (e.g x = a+b, y = a*b

```python
#Implementing logic gates manually

def and_gate(x,y):
    if x==1 and y==1:
        return 1
    else:
        return 0

def nand_gate(x,y):
    if and_gate(x,y) == 0:
        return 1
    else:
        return 0

#Excercise 1: Create the three other gates presented above.

def or_gate(x,y):
    return

def nor_gate(x,y):
    return

def xor_gate(x,y):
    return

#Excercise 2: Assume there are vegitarian students, and those specifically can't have chicken. Generate a truth table.
#Hint: treat vegitarian students as an input bit
```

## 3.4 Quantum Computing and Its Purpose

A couple of weeks into my quantum computing journey, I had the privilege of meeting Prof. Scott Aaronson with a few friends of mine over a web conference. My memory is a bit faint but I remember one of the first questions asked went along the lines of "Can you solve [some problem] with a quantum computer?" Dr. Aaronson responded the following in a manner so instinctive, it would perhaps be safe for one to assume that he gets asked similar questions on a daily basis: "Well, yeah you can do that. Except there wouldn't be any noticable speedup. Quantum Computing doesn't change the theory of computability, meaning that everything that can be solved on a quantum computer, can also be solved on a classical computer, it's just that for some problems, a classical computer may take exponentially more time to compute it". In my opinion, this is a great "one-liner" which explains a lot about why quantum computing is being so heavily investigated, because by using the logic that will later on be described, one can potentially design an algorithm which highly outperforms the current classical versions in terms of efficiency.

Richard Feynman noticed this in a talk he gave in 1982, where he said that the current model of computation (i.e classical computers) cannot efficiently model quantum systems in a scalable manner. He gave the following expample: Suppose you have an electron which could either be spin up or spin down. To represent every possible states, one would need two bits (1 for spin up and 1 for spin down), now if there are two electrons, one would need 4 bits, and for $n$ electrons, one would need $2^n$ bits... this gets exponentially less practical and is not an issue of scalability, but rather in the model itself. With a series of quantum bits (i.e bits that could be in a superposition of 1 and 0 at the same time), the number of qubits necessary to model $n$-electron spins would be reduced to $n$.

However, problems outside of modeling quantum systems are also being investigated at the moment and there is a whole field dedicated towards what can and can't be efficiently solved by a quantum computer

```python
# Go nuts here:

```

Chapter 3 - Classical Computation and Logic.ipynb


# Calculating Term Frequency(tf) and Inverse Document Frequency(idf) in Information Retrieval

```python
import nltk
import numpy as np
nltk.download('stopwords')
from nltk.tokenize import RegexpTokenizer
from sklearn.feature_extraction.stop_words import ENGLISH_STOP_WORDS
stopWords = set(list(ENGLISH_STOP_WORDS) + nltk.corpus.stopwords.words('english'))
regex = RegexpTokenizer('\w+')
```

    [nltk_data] Downloading package stopwords to C:\Users\Divay
    [nltk_data]     Sharma\AppData\Roaming\nltk_data...
    [nltk_data]   Package stopwords is already up-to-date!
    c:\users\divay sharma\appdata\local\programs\python\python37-32\lib\site-packages\sklearn\utils\deprecation.py:144: FutureWarning: The sklearn.feature_extraction.stop_words module is  deprecated in version 0.22 and will be removed in version 0.24. The corresponding classes / functions should instead be imported from sklearn.feature_extraction.text. Anything that cannot be imported from sklearn.feature_extraction.text is now part of the private API.
      warnings.warn(message, FutureWarning)

```python
file = open("assi3_data.txt","r").readlines()
file
```

    ["1.Boolean model (Doc1) '''The (standard) Boolean model of information retrieval (BIR)[1] is a classical information retrieval (IR) model and, at the same time, the first and most-adopted one. It is used by many IR systems to this day.[citation needed] The BIR is based on Boolean logic and classical set theory in that both the documents to be searched and the user's query are conceived as sets of terms. Retrieval is based on whether or not the documents contain the query terms. '''\n",
     "2.probabilistic relevance model(doc2) '''The probabilistic relevance model[1][2] was devised by Stephen E. Robertson and Karen SpÃ¤rck Jones as a framework for probabilistic models to come. It is a formalism of information retrieval useful to derive ranking functions used by search engines and web search engines in order to rank matching documents according to their relevance to a given search query.It is a theoretical model estimating the probability that a document dj is relevant to a query q. The model assumes that this probability of relevance depends on the query and document representations. Furthermore, it assumes that there is a portion of all documents that is preferred by the user as the answer set for query q. Such an ideal answer set is called R and should maximize the overall probability of relevance to that user. The prediction is that documents in this set R are relevant to the query, while documents not present in the set are non-relevant. '''\n",
     "3.Vector space model(doc3) '''Vector space model or term vector model is an algebraic model for representing text documents (and any objects, in general) as vectors of identifiers (such as index terms). It is used in information filtering, information retrieval, indexing and relevancy rankings. Its first use was in the SMART Information Retrieval System. Each dimension corresponds to a separate term. If a term occurs in the document, its value in the vector is non-zero. Several different ways of computing these values, also known as (term) weights, have been developed. One of the best known schemes is tf-idf weighting (see the example below).The definition of term depends on the application. Typically terms are single words, keywords, or longer phrases. If words are chosen to be the terms, the dimensionality of the vector is the number of words in the vocabulary (the number of distinct words occurring in the corpus).Vector operations can be used to compare documents with queries. '''"]

The tf-idf weight is a weight often used in information retrieval and text mining. This weight is a statistical measure used to evaluate how important a word is to a document in a collection or corpus. The importance increases proportionally to the number of times a word appears in the document but is offset by the frequency of the word in the corpus (data-set).

#### Term Frequency
This measures the frequency of a word in a document. This highly depends on the length of the document and the generality of word. TF is individual to each document and word, hence we can formulate TF as follows.

#### Document Frequency
This measures the importance of document in whole set of corpus, this is very similar to TF. The only difference is that TF is frequency counter for a term t in document d, where as DF is the count of occurrences of term t in the document set N.

#### Inverse Document Frequency
IDF is the inverse of the document frequency which measures the informativeness of term t.

\begin{align}
tf-idf(t, d) = tf(t, d) * log(N/(df + 1))
\end{align}

```python
def calculate_tf_idf(file):
    N = len(file)
    tf = []
    word_count = []
    idf = {}
    for doc in file:
        words = regex.tokenize(doc)
        doc_dict = {}
        count = 0
        for word in words:
            word = word.lower()
            if word not in stopWords:
                if word not in idf:
                    idf[word] = 0
                count = count + 1
                if word not in doc_dict:
                    doc_dict[word] = 1
                else:
                    doc_dict[word] = doc_dict[word] + 1
        tf.append(doc_dict)
        word_count.append(count)

    #Calculate tf
    for i in range(N):
        for word in tf[i]:
            tf[i][word] = tf[i][word]/word_count[i]

    #Calculate tdf
    for word in idf:
        for i in range(N):
            if word in tf[i]:
                idf[word] = idf[word]+1
        idf[word] = np.log(N/(idf[word]+1) +1)

    #Calculate tf-idf
    tf_idf = tf
    for i in range(N):
        for word in tf_idf[i]:
            tf_idf[i][word] = tf[i][word]*idf[word]

    return tf_idf
```

```python
tf_idf = calculate_tf_idf(file)
```

tf-idf value for 1st document

```python
tf_idf[0]
```

    {'1': 0.03150669002545206,
     'boolean': 0.06247436808232875,
     'model': 0.03815562190468791,
     'doc1': 0.020824789360776252,
     'standard': 0.020824789360776252,
     'information': 0.02543708126979194,
     'retrieval': 0.03815562190468791,
     'bir': 0.041649578721552505,
     'classical': 0.041649578721552505,
     'ir': 0.041649578721552505,
     'time': 0.020824789360776252,
     'adopted': 0.020824789360776252,
     'used': 0.01271854063489597,
     'systems': 0.020824789360776252,
     'day': 0.020824789360776252,
     'citation': 0.020824789360776252,
     'needed': 0.020824789360776252,
     'based': 0.041649578721552505,
     'logic': 0.020824789360776252,
     'set': 0.01575334501272603,
     'theory': 0.020824789360776252,
     'documents': 0.02543708126979194,
     'searched': 0.020824789360776252,
     'user': 0.01575334501272603,
     'query': 0.03150669002545206,
     'conceived': 0.020824789360776252,
     'sets': 0.020824789360776252,
     'terms': 0.03150669002545206,
     'contain': 0.020824789360776252}

```python
def chcek_query(tf_idf,query):
    N = len(tf_idf)
    vocab = []
    query = regex.tokenize(query)
    for word in query:
        word = word.lower()
        if word not in stopWords:
            vocab.append(word)
    doc_score = []
    for i in range(N):
        score = 0
        for word in vocab:
            if word in tf_idf[i]:
                score = score + tf_idf[i][word]
        doc_score.append(score)
    print(doc_score)
    return doc_score.index(max(doc_score))
```

Testing with queries

```python
query = "Vector space model"
print("Doc " + str(chcek_query(tf_idf,query)))
```

    [0.03815562190468791, 0.024331121214583595, 0.10631987785261035]
    Doc 2

```python
query = "Stephen E. Robertson"
print("Doc " + str(chcek_query(tf_idf,query)))
```

    [0, 0.029879045604592015, 0]
    Doc 1

```python
query = "classical information retrieval"
print("Doc " + str(chcek_query(tf_idf,query)))
```

    [0.10524228189603235, 0.012165560607291797, 0.03108976599641237]
    Doc 0

```python

```


Equações e teoria da página 99 do Miranda.

Equação do volume de cauda horizontal
$$
\begin{equation}
V_{HT} = \frac{I_{HT}.S_{HT}}{\bar c.S} [2.62]
\end{equation}
$$

Valores recomendados

$$
\begin{equation}
0,35 < V_{HT} < 0,5
\end{equation}
$$
Equação do volume de cauda horizontal
 vertical
$$
V_{VT} = \frac{I_{VT}.S_{VT}}{b.S}    [2.63]
$$

Valores recomendados

$$
0,04 < V_{VT} < 0,06
$$

# Código antigo do sorriso

```python
import numpy as np
import matplotlib.pyplot as plt

calc = (3.2-1.86) - ((1 / 4) * 0.526)

x = 889.43 / 1000
S = 0.843
b = 1.86
c_medium = 0.463
AR = 3
teta = np.linspace(np.pi/36, np.pi/18, 20)
# l_ht = 0.67874
cht = np.linspace(.150,.400, 20)
s_it = (cht ** 2) * AR
# bht = (s_it / cht)

V_ht = [0.35, 0.5]

# s_it = np.linspace(0,1,200)
Vht = list(); Sht_old = list(); lht_save = list(); lht_f = list(); Sht_f = list(); teta_save = list()

for k in teta:
    l_ht = (x - cht * (3 / 4)) / np.cos(k)
    for i, S_ht in enumerate(s_it):
        Vht_test = ((l_ht) * S_ht) / (c_medium * S)
        for j in range(0, len(Vht_test)):
            x_test = l_ht[j] * np.cos(k) + (3 / 4) * (S_ht / AR) ** (1 / 2)
            if (Vht_test[j] >= V_ht[0] and Vht_test[j] <= V_ht[1] and x_test <= x and x_test >= x*0.99):
                Vht.append(Vht_test[j])
                Sht_old.append(S_ht)
                lht_save.append(l_ht[j])
                teta_save.append(k)

Sht_n = np.array(Sht_old)
lht_n = np.array(lht_save)

# for indx, i in enumerate(lht_save):
#     if i >= (max(lht_save) * 0.99):
#         lht_f.append(i)
#         Sht_f.append(Sht_old[indx])

# Sht_n = np.array(Sht_f)
# lht_n = np.array(lht_f)

c_ht = (Sht_n / AR) ** (1 / 2)
b_ht = (Sht_n / c_ht)

data = np.array((Vht, lht_n, c_ht, b_ht, teta_save))

cont = 0
for i in range(0, len(data[1])):
    it = i - cont
    if (data[1][it] < (0.999 * max(data[1]))):
        data = np.delete(data, it, 1)
        cont += 1

cont = 0
for i in range(0, len(data[0])):
    it = i - cont
    if (data[0][it] < (0.999 * max(data[0]))):
        data = np.delete(data, it, 1)
        cont += 1

x_cord = np.empty((len(data[4])))
y_cord = np.empty((len(data[4])))

for i in range(0, len(data[4])):
    y_cord[i] = data[1][i] * np.sin(data[4][i])
    x_cord[i] = data[1][i] * np.cos(data[4][i])

fig, (ax1, ax2) = plt.subplots(2,1)
for i in range(0,len(data[0])):

    ax1.plot([0, data[3][i]], [0, 0])
    ax1.plot([data[3][i], data[3][i]], [0, data[2][i]])
    ax1.plot([0, data[3][i]], [data[2][i], data[2][i]])
    ax1.plot([0, 0], [0, data[2][i]])
    ax2.plot([(x_cord[i] - (data[2][i] / 4)), x_cord[i] + ((3/4) * data[2][i])], [y_cord[i], y_cord[i]])

ax2.plot([-(1/4)*0.526,(3/4)*0.526],[0,0])
ax2.plot([0,0], [0,-0.16])
ax2.set_xlim(0, 1)
plt.axis('equal')

# Vvt = (l_vt * S_vt) / (b * S)

```

```python
import numpy as np
import pandas as pd

def volumes(L, S, k='b ou c', Sw='área da asa'):
    v = []
    for i in range(len(L)):
        v.append(L[i]*S/(k*Sw))
    return filtragem(v)

def filtragem(m, lim_min=0.35, lim_max=0.5):
    Vx = np.ravel(m)
    V = list()
    indice = list()
    p = len(m[0])
    for n in range(len(Vx)):
        if lim_min <= Vx[n] <= lim_max:
            V.append(Vx[n])
            indice.append(n)
    valor = np.array((indice)) / p
    v_pos_x = valor.astype(int)
    c = (valor - v_pos_x) * p
    v_pos_y = (np.around(c)).astype(int)
    return V, v_pos_x, v_pos_y

#Inputs
S = np.linspace(0.1,0.5) #m^2
xlim = 2.5; ylim = 0.3   #m
teta = 10                #graus
deg2rad = np.pi/180
AR = 2

#Processamento
c = (S / AR) **(1 / 2)
b = (S / c)
y_pos = np.linspace(0.3*ylim, ylim)
x_pos = y_pos/np.tan(teta * deg2rad)
l  =  np.sqrt((x_pos*2)+(y_pos*2))

values = np.array((S, l))

valor, indice_l, indice_S = volumes(values[1], values[0], k=0.4, Sw=1)

print(valor[0], l[indice_l[0]], S[indice_S[0]])

print(x_pos[indice_l[0]], y_pos[indice_l[0]])
print(c[indice_S[0]], b[indice_S[0]])

```

# Solução

```python
import numpy as np
import ZebraLib as Lzb

class plane:

    def _init_(self, Sw=1, AR=2, teta=10, cw=0.5, bw=2):
        #Asa
        self.Sw = Sw
        self.teta = teta
        self.cw = cw
        self.bw = bw
                #Profundor e leme
        self.AR = AR
        self.S = np.linspace(0.05, 0.5)
        self.c = (self.S / self.AR) **(1 / 2)
        self.b = self.S/self.c

        #Calculos
        self.__calc_L()

    def __calc_L(self, xlim=2.5, ylim=0.3):
        self.y_pos = np.linspace(0.3*ylim, ylim)
        self.x_pos = self.y_pos/np.tan(np.radians(self.teta))
        self.L = np.sqrt((self.x_pos*2)+(self.y_pos*2))

    def __filtragem(self, m, lim_min, lim_max):
        Vx = np.ravel(m)
        V = list()
        indice = list()
        p = len(m[0])
        for n in range(len(Vx)):
            if lim_min <= Vx[n] <= lim_max:
                V.append(Vx[n])
                indice.append(n)
        valor = np.array((indice)) / p
        v_pos_x = valor.astype(int)
        c = (valor - v_pos_x) * p
        v_pos_y = (np.around(c)).astype(int)
        return V, v_pos_x, v_pos_y
```


```python
from IPython.display import HTML

HTML('''
<form action="javascript:code_toggle()"><input type="submit" value="Code Toggle"></form>''')
```

<form action="javascript:code_toggle()"><input type="submit" value="Code Toggle"></form>

```python
from IPython.display import HTML

HTML('''
<a href="https://github.com/wd15/chimad-phase-field/raw/master/benchmarks/benchmark3.ipynb"
   download="benchmark3.ipynb">
<button type="submit">Download Notebook</button>
</a>''')
```

<a href="https://github.com/wd15/chimad-phase-field/raw/master/benchmarks/benchmark3.ipynb"
   download="benchmark3.ipynb">
<button type="submit">Download Notebook</button>
</a>

# Benchmark Problem 3: Dendritic Growth

```python
from IPython.display import HTML

HTML('''{% include jupyter_benchmark_table.html num="[3]" revision=1 %}''')
```

{% include jupyter_benchmark_table.html num="[3]" revision=1 %}

See the journal publication entitled ["Phase Field Benchmark Problems for Dendritic Growth and Linear Elasticity"][nolink] for more details about the benchmark problems. Furthermore, read [the extended essay][benchmarks] for a discussion about the need for benchmark problems.

[benchmarks]: ../
[nolink]: https://pages.nist.gov/chimad-phase-field/

## Overview

Dendritic growth simulations are useful as benchmark problems being highly sensitive to both the phase field model formulation and the particular numerical implementation employed (see, for example,
[[1][karma1998quantitative], [2][warren1995prediction]]). Historically, dendritic growth was one of the first applications of phase field modeling [[3][Fix], [4][Langer]], and remains a significant area of research today.  Previous analyses of both the sharp [[4][Langer], [5][collins1985diffuse], [6][kobayashi1993modeling], [7][mcfadden1993phase], [8][wang1993thermodynamically]] and thin interface limits [[10][karma1996phase], [1][karma1998quantitative], [11][mcfadden2000thin], [12][karma2001phase]] have demonstrated that the diffuse-interface phase field formulation is asymptotically equivalent to the sharp-interface Stefan formulation. In 2001, the introduction of an "anti-trapping current" to correct for solute trapping due to the jump in chemical potential at the solid/liquid interface [[13][karma2001phase], [14][echebarria2004quantitative]] facilitated quantitative phase field modeling of alloy solidification using unphysically large diffuse interface widths.  Today, massive increases in computing power and the advent of scientific computing on graphical processing units enable large-scale, quantitative 3D phase field simulations of growing dendrites (see, for example, [[15][hotzer2015large], [16][shibuta2015solidification]] and reviews [[17][takaki2014phase], [18][granasy2014phase]]).

[karma1998quantitative]: https://doi.org/10.1103/PhysRevE.57.4323
[warren1995prediction]: https://doi.org/10.1016/0956-7151(94)00285-P
[Fix]: https://doi.org/10.1007/978-3-0348-7893-7
[Langer]: https://doi.org/10.1142/9789814415309_0005
[collins1985diffuse]: https://doi.org/10.1103/PhysRevB.31.6119
[kobayashi1993modeling]: https://doi.org/10.1016/0167-2789(93)90120-P
[mcfadden1993phase]: https://doi.org/10.1103/PhysRevE.48.2016
[wang1993thermodynamically]: https://doi.org/10.1016/0167-2789(93)90189-8
[karma1996phase]: https://doi.org/10.1103/PhysRevE.53.R3017
[karma1998quantitative]: https://doi.org/10.1103/PhysRevE.57.4323
[mcfadden2000thin]: https://doi.org/10.1016/S0167-2789(00)00064-6
[karma2001phase]: https://doi.org/10.1103/PhysRevLett.87.115701
[echebarria2004quantitative]: https://doi.org/10.1103/PhysRevE.70.061604
[hotzer2015large]: https://doi.org/10.1016/j.actamat.2015.03.051
[shibuta2015solidification]: https://doi.org/10.1007/s11837-015-1452-2
[takaki2014phase]: https://doi.org/10.2355/isijinternational.54.437
[granasy2014phase]: https://doi.org/10.1007/s11661-013-1988-0

## Model Formulation

In this formulation, one order parameter, $\phi$, and one additional field variable, $U$, are evolved.  The phase of the material is described by $\phi$, which takes a value of -1 in the liquid and +1 in the solid.  In addition, the nondimensionalized temperature is indicated by $U$,
\begin{equation}
U=\frac{T-T_m}{L/c_p},
\end{equation}
where $T$ is the local temperature, $T_m$ is the melting temperature, $L$ is the latent heat of fusion, and $c_p$ is the specific heat at constant pressure, such that $U=0$ is the nondimensionalized melting temperature (note that, for this particular problem, we do not need to supply values for $T_m$, $L$, and $c_p$). The free energy of the system, $\mathcal{F}$, is expressed as [[1][karma1998quantitative]]

$$
\mathcal{F}=\int_{V}\left[\frac{1}{2} \left[W(\textbf{n})\right]^2|\nabla \phi|^2+f_{chem}(\phi,U)\right]\,dV
$$

where $\left[W(\textbf{n})\right]^2$ is the gradient energy coefficient, $\textbf{n}\equiv \frac{\nabla\phi}{|\nabla\phi|}$ is the normal direction to the interface, and $f_{chem}$ is the chemical free energy density.  In this formulation, $f_{chem}$ is a double-well potential with a simple polynomial formulation [[1][karma1998quantitative]],

$$
f_{chem}=-\frac{1}{2}\phi^2+\frac{1}{4}\phi^4
+\lambda U\phi\left[1-\frac{2}{3}\phi^2+\frac{1}{5}\phi^4\right],
$$

where $\lambda$ is a dimensionless coupling constant.  The interface thickness and directional anisotropy are controlled by $W(\textbf{n})$, which takes the form $W(\textbf{n})=W_0a(\textbf{n})$ in two dimensions. We use a simple form for $a(\textbf{n})$ to reflect in-plane symmetry [[1][karma1998quantitative]],

\begin{equation}
a(\textbf{n})=1+\epsilon_m\cos \left[m(\theta-\theta_0) \right],
\end{equation}

where $m$ is a non-negative integer and $\theta$ is the in-plane azimuthal angle, $\tan\theta=n_y/n_x$; $\theta_0$ is an offset azimuthal angle that specifies the misorientation of the crystalline lattice relative to the laboratory frame of reference (in this case, the $x$-axis in the simulation coordinate system). We take

\begin{equation}
\lambda=\frac{D\tau_0}{0.6267W_0^2}
\end{equation}

because this choice gives quantitative agreement in the so-called ``thin interface limit'' with sharp interface models of dendritic growth in the limit of vanishing interface kinetics [[1][karma1998quantitative]].

The time scale for the evolution of $\phi$ and $U$ are similar, so both must be described with time-dependent equations.  The evolution of $\phi$, a non-conserved quantity, is governed by the Allen-Cahn equation [[1][karma1998quantitative]],

\begin{equation}
\tau(\textbf{n})\frac{\partial\phi}{\partial t} = -\frac{\delta \mathcal{F}}{\delta \phi},
\end{equation}

where the kinetic coefficient $\tau(\textbf{n})=\tau_0\left[a(\textbf{n})\right]^2$. The functional derivative is given as [[1][karma1998quantitative]]

\begin{eqnarray}
\tau(\textbf{n})\frac{\partial\phi}{\partial t} & = & \left[\phi-\lambda U\left(1-\phi^2\right)\right]\left(1-\phi^2\right)+\nabla\cdot\left( \left[W(\textbf{n})\right]^2\nabla\phi\right)\nonumber\\
&&+\frac{\partial}{\partial x}\left[|\nabla\phi|^2W(\textbf{n})\frac{\partial W(\textbf{n})}{\partial \left(\frac{\partial\phi}{\partial x}\right)}\right]
+\frac{\partial}{\partial y}\left[|\nabla\phi|^2W(\textbf{n})\frac{\partial W(\textbf{n})}{\partial \left(\frac{\partial\phi}{\partial y}\right)}\right]
\end{eqnarray}

for two dimensions.  We remind the reader that although  $a(\textbf{n})$, $W(\textbf{n})$ and $\tau(\textbf{n})$ have a vectorial argument, they are scalar quantities. Furthermore, the time evolution of $U$ is governed by thermal diffusion and the release of latent heat at the interface during solidification [[1][karma1998quantitative]],

\begin{equation}
\frac{\partial U}{\partial t} = D\nabla^2U+\frac{1}{2}\frac{\partial \phi}{\partial t}
\end{equation}

where $D$ is a thermal diffusion constant.

[karma1998quantitative]: https://doi.org/10.1103/PhysRevE.57.4323

## Parameterization and simulation conditions

This section presents the specific details for the solidification and dendritic growth benchmark problem, including the model parameterization, initial conditions, boundary conditions, and computational domain size. The model is parameterized with dimensionless units, see the table below. While the diffuse interface width depends on orientation, it varies between four and five units, where the width is defined as the distance over which $-0.9 < \phi < 0.9$. The benchmark problem is formulated for two dimensions.  To further reduce computational cost, we simulate one-quarter of a growing dendrite, as is commonly done in earlier works (e.g., Ref. [[1][karma1998quantitative], [2][warren1995prediction]]). One-quarter of a solid seed with a radius of eight units (with the position of the interface defined as $\phi=0$) and a diffuse interface width of one unit is placed in the lower-left corner of the computational domain, surrounded by liquid.  Initially, the entire system is uniformly undercooled with $U \left(t=0\right)=\Delta$.  This undercooling is chosen to challenge numerical solvers somewhat because it increases the thermal diffusion length and requires a larger computational domain size relative to more negative undercoolings.  We select a square computational domain of $(960 \textrm{ units})^2$, which is two times longer than the long dimension used in Ref. [[1][karma1998quantitative]] for the same model parameterization. No-flux boundary conditions are chosen for $\phi$ and $U$ on all domain boundaries.

| Quantity                  | Symbol       | Value |
|:--------------------------|:-------------|:------|
| Interface thickness       | $W_0$        | 1     |
| Rotational symmetry order | $m$          | 4     |
| Anisotropy strength       | $\epsilon_4$ | 0.05  |
| Offset angle              | $\theta_0$   | 0     |
| Relaxation time           | $\tau_0$     | 1     |
| Diffusion coefficient     | $D$          | 10    |
| Undercooling              | $\Delta$     | -0.3  |

[karma1998quantitative]: https://doi.org/10.1103/PhysRevE.57.4323
[warren1995prediction]: https://doi.org/10.1016/0956-7151(94)00285-P

## Example Result at $t=1500$

## Submission Guidelines

All benchmark solutions should be run to at least $t=1500$. The following data should be collected for upload:

 - The solid fraction in the domain (the integral of $\int_V \phi dV$)

 - The free energy, $\mathcal{F}$

 - The estimated tip position versus time (the $\phi=0.5$ contour on either the $x=0$ or $y=0$ planes)

These values should be obtained as frequently as possible, but at least 20 values during the simulation would be good. The data should be stored in a CSV file with the following format.

```
time,solid_fraction,free_energy,tip_position
0.1,1000.0,500.0,0.1
...
1502.0,2000.0,100.0,50.0
```

Further data to upload can include a Youtube video or images of the dendrite at different times. These are not requied, but will help others view your work.

Please use the [upload form]({{ site.baseurl/simulations/upload_form/ }}) to upload your results.


TensorFlow

Laboratory on Neural Networks

***

## Introduction

A few years back, Google has open-sourced its machine learning framework: **TensorFlow**. Since then, it has become one of the most used tools in machine learning, both for research purposes and for production systems. TensorFlow is very flexible and can handle a wide range of neural architectures.

In this lecture we will see how we can build a simple Neural Network capable of categorizing images from the *MNIST* dataset into digits. This tutorial
is a mix of some guides hosted on the [tensorflow website](https://www.tensorflow.org/tutorials).

This lecture assumes familiarity with the Python programming language and the libraries we have seen in the previous lecture: Numpy, Matplotlib and Scikit-learn.

## Google Colab

Google Colab is a  cloud service based on Jupyter  that enables to execute notebooks in the cloud, using resources from Google's datacenters.
Google even provides limited access to GPU, in order to accelerate tasks such as
deep learning.

## References

- https://www.tensorflow.org
- https://www.tensorflow.org/tutorials/quickstart/beginner
- https://www.tensorflow.org/tutorials/quickstart/advanced
- http://yann.lecun.com/exdb/mnist/

# Object-oriented programming in Python
Before start playing with tensorflow we need to briefly introduce objects in Python.

## Classes and Objects
Python is an object oriented programming language.
Almost everything in Python is an object, with its properties and methods.

An object is a combination of variables and functions used to perform
operations over them. Each object is an instance of a class.

A class is like an object constructor, or a "blueprint" for creating objects.
A class defines both object's variables (usually referred as `attributes`) and
funtions (referred as `methods`) that will operate over them.

Among many others, there exists a special method, the __\_\_init\_\___ method.
In the __\_\_init\_\___ method  we define and initialize the
attributes of the objetcs we will create starting from the class.

As the language mandates, the first argument of each method of a class
is `self`, which is a pointer to the instance of the class itself.
This enables to perform computation involving attributes:

```python
class Person: # the class definition
  def __init__(self, p_name, p_age):
    self.name = p_name # create some attributes
    self.age = p_age

  def hello(self, greetings="Hello"): # define a method
    print(greetings + " my name is " + self.name)

p1 = Person("John", 36) # create object from class Person
print(p1.age)
p1.hello(greetings="Hi")
p1.hello()
```

    36
    Hi my name is John
    Hello my name is John

Then we can reuse the class to create other persons

```python
p2 = Person("Alice", 27)

print(p2.age)
p2.hello()
```

    27
    Hello my name is Alice

## Inheritance

Inheritance enables us to define a class that inherits all the methods and attributes from another class.

The __parent__ class is the class being inherited from, also called base or super class.

The __child__ class is the class that inherits from another class, also called derived  or sub class.

We can specify the parent class by putting its name
in brackets after the name of the child class.

In the child's __\_\_init\_\___ method we must call the parent's __\_\_init\_\___
method, in order to initialize the parent correctly. This can
be done by using the `super()` function

```python
class Student(Person): # Student inherits from Person
  def __init__(self, name, age, year):
    super().__init__(name, age) # call __init__ method of the parent class
    self.graduationyear = year

  def welcome(self):
    print("Welcome", self.name, "to the class of", self.graduationyear)

s1 = Student("Bob", 24, 2021)

print(s1.age) # inherited from Person
s1.hello(greetings="Hi") # inherited from Person

print(s1.graduationyear)
s1.welcome()

```

    24
    Hi my name is Bob
    2021
    Welcome Bob to the class of 2021

# TensorFlow

Let's dive into how to declare and train a Neural Network with *TensorFlow*.

Instruct Google Colab to load TensorFlow __version 2__:

```python
try:
  # %tensorflow_version only exists in Colab.
  %tensorflow_version 2.x
except Exception:
  pass

```

Import TensorFlow  into your program:

```python
import tensorflow as tf

tf.random.set_seed(0) # set seed for reproducibility, still not deterministic if a GPU is used

```

As it was for the numpy library (which we used in the last tutorial),
even  TensorFlow has its basic data structure, called __Tensor__.
A Tensor is the equivalent of the numpy array, a fixed-size structure containing
an ordered collection of homogeneous data.

```python
t = tf.convert_to_tensor([1,2,3,4])
t
```

    <tf.Tensor: shape=(4,), dtype=int32, numpy=array([1, 2, 3, 4], dtype=int32)>

There are many ways to declare tensors, many of them quite
similar to numpy:

```python
import numpy as np
a = np.arange(5)
t = tf.range(5)

print(a)
print(t)

print(a.shape)
print(t.shape)
```

    [0 1 2 3 4]
    tf.Tensor([0 1 2 3 4], shape=(5,), dtype=int32)
    (5,)
    (5,)

The library offers easy ways to switch between Tensors and numpy arrays:

```python
an_array = np.array([1,2,3])
a_tensor = tf.convert_to_tensor(an_array) # from array to tensor

another_array = a_tensor.numpy() # from tensor to array

print(an_array)
print(a_tensor)
print(another_array)
```

    [1 2 3]
    tf.Tensor([1 2 3], shape=(3,), dtype=int64)
    [1 2 3]

Tensors can be directly used within the numpy library

```python
np.sqrt(an_array * a_tensor)
```

    array([1., 2., 3.])

## MNIST dataset

*MNIST* is a dataset containing labelled images of handwritten digits. This dataset is similar to the *digits* dataset we have seen in the previous lecture, but the images here are larger, 28 by 28 pixels.

We can use TensorFlow directly to download and read the MNIST data:

```python
mnist = tf.keras.datasets.mnist

(data_train, targets_train), (data_test, targets_test) = mnist.load_data()
```

    Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz
    11493376/11490434 [==============================] - 3s 0us/step

The data instances in MNIST are images represented by  arrays contaning 784 numbers between 0 and 255, i.e. the gray scale relative intensity of the pixels (0 means white, 255 means black). Later we will  normalize them, obtaining values between 0 and 1. The shape of the loaded data is
(number of images, 28, 28).

***

MNIST already comes with a train-test split. The training data is a *numpy array* of 60000 images, while the test data contains 10000 images.

***

In the current version of Tensorflow the labels are encoded
as categorical values (integers from 0 to 9). For this tutorial we will use the so-called
"one-hot" representation. Each label `y` is a vector of 10 0-1 elements, in which only the position associated with the true digit is set to 1, the others are set to 0. Thus, the training labels will be encoded as a  tensor of shape (number of images, 10).

Once loaded, we can visualize the data using Matplotlib

```python
%matplotlib inline
import matplotlib.pyplot as plt

plt.gray()
plt.matshow(255 - data_train[0]) # 255 - x simply inverts the fading direction of the image
plt.show()
```

While the label is:

```python
targets_train[0]
```

    5

We can use the `tf.one_hot` function in order to obtain the one hot encoding

```python
one_hot = tf.one_hot(targets_train[0], 10) # we must specify the number of classes, 10 for MNIST
one_hot
```

    <tf.Tensor: id=12, shape=(10,), dtype=float32, numpy=array([0., 0., 0., 0., 0., 1., 0., 0., 0., 0.], dtype=float32)>

Let's cast the data to floating points, normalize and reshape it (we need to add an empty dimension at the end, in order to use it later to train  our neural networks)

```python

x_train = tf.cast(data_train, tf.float32) / 255.0
x_test = tf.cast(data_test, tf.float32) / 255.0

# Add a fourth dimension
x_train = x_train[..., tf.newaxis]
x_test = x_test[..., tf.newaxis]

x_train.shape
```

    TensorShape([60000, 28, 28, 1])

```python
y_train = tf.one_hot(targets_train, 10)
y_test = tf.one_hot(targets_test, 10)
```

## Softmax Regression

The first thing we are going to implement using TensorFlow is a simple softmax regression model to predict the digits from the images. Softmax regression is a generalization of Logistic Regression to multi-class classification.

In softmax regression we define a linear model for each possible class. These models provide evidence supporting the image `x` being of the particular class `i`:

\begin{equation}
    \theta_i = \sum_{1 \le j \le 784} W_{i,j} \cdot x_j + b_i
\end{equation}

A softmax function is then applied to each linear model in order to get the probability $y_i$ of the image being of class `i`:

\begin{equation}
    y_i = \text{softmax}(\theta_i) = \frac{e^{\theta_i}}{\sum_{j} e^{\theta_j}}
\end{equation}

This is a schematic view of the operations we are a doing:

***

Spelling out the equations of our model:

***

The vectorized form:

***

The vectorized equation is therefore the following:

\begin{equation}
    y = \text{softmax}(W \cdot x + b)
\end{equation}

Where $y$ is a $1 \times 10$ column vector, $W$ is a $10 \times 784$ matrix, $x$ is a $1 \times 784$ column vector and $b$ is a $1 \times 10$ column vector. While we used column vectors for ease of exposition, in the implementation is much more convenient to use row vectors for $x$, $b$ and $y$. Hence, the matrix `W` in the code below will be of shape (784, 10). Let's now dive into the definition of this perceptron with TensorFlow.

First we need to import some objects that we will use to compose
our models:

```python
from tensorflow.keras import Model
from tensorflow.keras.layers import Softmax, Flatten
```

Then we can start defining our custom perceptron.

In Tensorflow, in order to create a custom layer or network we must
extend the class `tensorflow.keras.Model`, by defining
all the weights (or layers) and the operations of our model.
Our network will be a child class of  `tensorflow.keras.Model`, inheriting
all the capabilities of that class, for instance the ability to
perform automatic gradient computation over its parameters.

Let's define our perceptron. In the __\_\_init\_\___ method we need to declare our matrix of weights `self.W` and biases `self.b`, plus two more layers: a  `Flatten` layer, which is used to trasform
images, represented as 28x28 matrices, to vectors of 784 elements,
and a `Softmax` layer that, as the name suggests, will compute the softmax over
the output of the perceptron.

In the __call__ method we use the components we declared in order to
compose the series of operations needed to perform inference over
a batch of examples `x`, represented as a tensor whose shape at runtime will be
(batch size, 28, 28, 1). First we apply the flatten layer, in order to obtain
a batch with shape (batch size,  784), then we perform the matrix multiplication
with `self.W` and the addition of the bias vector `self.b`.

The result of the matrix multiplication has shape (batch size, 10), while
the bias vector has shape (1, 10). In order to execute this addition ,
tensorflow performs an automatic  *broadcasting* of the biases by repetition, in order
to obtain the same shape of the other operand. You can find
all the broadcasting rules in the [documentation](https://www.tensorflow.org/xla/broadcasting).

```python
class MnistPerceptron(Model): # inherit from Model
  def __init__(self):
    super().__init__() # initialize Model
    self.flatten = Flatten() # used to flatten pixels of images
    self.W = tf.Variable(tf.zeros([784, 10])) # declare weights with shape :(748, 10)
    self.b = tf.Variable(tf.zeros([1, 10]))   # declare biases, with shape :(1, 10)
    self.softmax = Softmax()

  def call(self, x,training=False):
    # the  training argument is unused in this model, we will need it later

    flat = self.flatten(x) # flatten images

    multiplied = tf.matmul(flat, self.W) # matmul, output shape : (batch, 10)
    # we can equivalently do:
    #multiplied = tf.transpose(tf.matmul(tf.traspose(self.W), tf.traspose(flat)))

    fwded = multiplied + self.b # broadcast self.b to (batch, 10) and add

    prob = self.softmax(fwded) # softmax
    return prob

# Create an instance of the model
perceptron = MnistPerceptron()
```

## Training

After we define the model, we want to train it using the MNIST data, using gradient descent. To do so, we first need to define a **loss function**. For this task we are going to use the *cross-entropy* loss:

\begin{equation}
    H_{y}(\hat{y}) = - \sum_i y_i \log(\hat{y}_i)
\end{equation}

Where $\hat{y}$ is the distribution predicted by our model and $y$ is the true probability distribution of the classes (i.e. the one-hot vector).

```python
perceptron_loss = tf.keras.losses.CategoricalCrossentropy()
```

Then we must create a `Dataset`, starting from the MNIST data, that we will use to train end evaluate our models. A Dataset is an object that can be used to
organize our raw data in batches of examples.

Let's create a train and test datasets. Then we will be able to iterate over them, obtaining a batch at each iteration that we will use to perform gradient descent.

```python
train_ds = tf.data.Dataset.from_tensor_slices(
    (x_train, y_train)).shuffle(10000).batch(100)

test_ds = tf.data.Dataset.from_tensor_slices((x_test, y_test)).batch(50)
```

Choose an optimizer for training.
In this case we will use Stocastic Gadient Descent (SGD) with a leraning rate of 0.01

```python
perceptron_optimizer = tf.keras.optimizers.SGD(learning_rate=1e-2)
```

Select metrics to measure the loss and the accuracy of the model. These metrics accumulate the values over different batches and then print the overall result.

```python
train_loss_metric = tf.keras.metrics.Mean()
train_accuracy_metric = tf.keras.metrics.CategoricalAccuracy()

test_loss_metric = tf.keras.metrics.Mean()
test_accuracy_metric = tf.keras.metrics.CategoricalAccuracy()
```

Then we can define a series of functions that we will use to train our model.

Our training framework is an iterative procedure that starts by sampling a
batch of examples from the training set, coupled with their labels.
The data is fed to the model, whose predictions are matched with the
true labels via a loss function. Then the gradient of the loss with
respect to the parameters of  the model is computed, and used to
update the parameters (following the directions in which it decreases).

The tensorflow library is able to automatically compute these
operations. In order to
do this we must create a disposable object, called __tape__.
This tape keeps track of the operations performed during inference
and computes the gradient with respect the loss function.
The gradient is then passed to the optimizer in order to
perform the update.

As last operation we update our accumulators to keep track of the
loss and accuracy metrics.

```python
def train_step(images, labels, model, loss_fn, optimizer):
  with tf.GradientTape() as tape: # all the operations within this scope will be recorded in tape
    predictions = model(images, training=True)
    loss = loss_fn(labels, predictions)
  gradients = tape.gradient(loss, model.trainable_variables)
  optimizer.apply_gradients(zip(gradients, model.trainable_variables))

  train_loss_metric(loss)
  train_accuracy_metric(labels, predictions)
```

We can then use our train step in order to perform training, by creating a training loop that iterates for a given number of times
(usually referred as *epochs*) trought
the dataset, printing the metrics at the end of each step.

```python
from datetime import datetime

def train_loop(epochs, train_ds, model, loss_fn, optimizer):
  for epoch in range(epochs):
      # reset the metrics for the next epoch
    train_loss_metric.reset_states()
    train_accuracy_metric.reset_states()

    start = datetime.now() # save start time
    for images, labels in train_ds:
      train_step(images, labels, model, loss_fn, optimizer)

    template = 'Epoch {}, Time {}, Loss: {}, Accuracy: {}'
    print(template.format(epoch+1,
                          datetime.now() - start,
                          train_loss_metric.result(),
                          train_accuracy_metric.result()*100))

```

Now we are ready to perform training

```python
EPOCHS = 10
train_loop(EPOCHS, train_ds, perceptron, perceptron_loss, perceptron_optimizer)
```

    Epoch 1, Time 0:00:08.748695, Loss: 1.1449335813522339, Accuracy: 80.3550033569336
    Epoch 2, Time 0:00:07.976085, Loss: 0.6415689587593079, Accuracy: 85.72833251953125
    Epoch 3, Time 0:00:07.845040, Loss: 0.5356608629226685, Accuracy: 87.04499816894531
    Epoch 4, Time 0:00:07.905150, Loss: 0.48453623056411743, Accuracy: 87.8066635131836
    Epoch 5, Time 0:00:07.948479, Loss: 0.45318278670310974, Accuracy: 88.3066635131836
    Epoch 6, Time 0:00:07.982059, Loss: 0.43141448497772217, Accuracy: 88.69332885742188
    Epoch 7, Time 0:00:07.944890, Loss: 0.41525769233703613, Accuracy: 88.97167205810547
    Epoch 8, Time 0:00:07.874499, Loss: 0.4026186764240265, Accuracy: 89.22000122070312
    Epoch 9, Time 0:00:07.900007, Loss: 0.39235949516296387, Accuracy: 89.45000457763672
    Epoch 10, Time 0:00:07.942112, Loss: 0.38381922245025635, Accuracy: 89.62000274658203

## Evaluation

Let's now evaluate our model on the test set. We need to define another couple of functions, in order to compute the metrics over each batch and then to
aggregate the results.

```python
def test_step(images, labels, model, loss_fn):
  predictions = model(images, training=False)
  t_loss = loss_fn(labels, predictions)

  test_loss_metric(t_loss)
  test_accuracy_metric(labels, predictions)
```

```python
def test_loop(test_ds, model, loss_fn):
    # reset the metrics for the next epoch
  test_loss_metric.reset_states()
  test_accuracy_metric.reset_states()

  for test_images, test_labels in test_ds:
    test_step(test_images, test_labels, model, loss_fn)

  template = 'Test Loss: {}, Test Accuracy: {}'
  print(template.format(test_loss_metric.result(),
                        test_accuracy_metric.result()*100))
```

Then we can perform evaluation of our trained perceptron

```python
test_loop(test_ds, perceptron, perceptron_loss)
```

    Test Loss: 0.36067456007003784, Test Accuracy: 90.36000061035156

Good! Our model works (sort of). These performance on MNIST are way too low. The state-of-the-art on this dataset has an accuracy of $0.9979$. Let's try to achieve a better accuracy by refining a bit our model.

## Deep architectures

We are now going to define a deep neural network capable of classifying digits much better than the shallow softmax we used before. We are going to use two convolutional layers alternated with 2 max pool layers, followed by a fully connected layer regularized with dropout, and finally we'll get predictions using again a softmax layer. Here you see a schematic view of the architecture (bottom up).

***

### Fully Connected Layer
In a fully connected layer each neuron is connected with each neuron of the previous layer. This is the most straightforward way to implement a Neural Network.

***

### Convolutional layer

Convolutional Neural Networks are based on a method for extracting meaningfull features from images, exploiting locality and parameter sharing between "close" pixels. In particular, a convolutional layer takes as input a matrix and divides it into smaller "patches". It outputs a number of features for each patch. The resulting tensor will be similar to the one in the following picture.

Applying a convolution reduces the height and the width of the
input image. This effect is not always desirable.
In order to avoid it, we must add some empty data around our images.
This data (usually containing zeros) is called *padding*

***

### Max pool layer

The max pool layer is a filter that is used to reduce the number of the input dimensions between two convolutional layers. It is useful for dimensionality reduction, it avoids overfitting, etc. The following is a simple example:

***

### ReLU layer

Rectified Linear Unit (ReLU) is a particular activation function of the type: $max(0, \theta)$. ReLUs have become very wide spread thanks to the fact, differently from sigmoids, that they do not saturate the gradient when used with backpropagation. This has enabled neural networks to become deeper and to increase their representational power. The following image compares several activation functions and their gradients.

***

### Dropout

Dropout is a regularization technique for neural networks. When using dropout, we set a probability for dropping random nodes from the networks at each gradient descent step. In this way, the network is going to learn a more robust model, not relying too much on a single node for a prediction. In some sense, using dropout we are "sampling" a lot of different networks and learning them, and finally we are left with the "average" network, which is less susceptible to outliers. Dropouts also helps in avoiding overfitting.

***

Let's start by defining two modules, one encoding a convolutional layer, another for the fully connected layer.

```python
class MnistConvolutional(Model):
  def __init__(self, in_channels, out_channels, size):
    super().__init__() # setup the moedl basic functionalities (mandatory)
    initial = tf.random.truncated_normal([size, size, in_channels, out_channels], stddev=0.1)
    self.filters = tf.Variable(initial) # create weights for the filters

  def call(self, x):
    res = tf.nn.conv2d(x, self.filters, 1, padding="SAME")
    return res

class MnistFullyConnected(Model):
  def __init__(self, input_shape, output_shape):
    super().__init__() # initialize the model
    self.W = tf.Variable(tf.random.truncated_normal([input_shape, output_shape], stddev=0.1)) # declare weights
    self.b = tf.Variable(tf.constant(0.1, shape=[1, output_shape]))  # declare biases

  def call(self, x):
    res = tf.matmul(x, self.W) + self.b
    return res

```

Then we can define our complete network.

Note that, due to the presence of the dropout,
in this case the behavior of the network at evalution phase
is different than the one at training phase.

During evaluation, the dropout layer must rescale the output
instead of randomly dropping part of it. For this reason,
the dropout at inference time takes in input an argument `training`
that controls the behavior of the layer. We can pass to it the
value our network receives from the caller, enabling us to
control it from the outside (see `treain_step` and `test_step` functions).

```python
from tensorflow.keras.layers import MaxPool2D, Dropout

class MnistDeepModel(Model):
  def __init__(self):
    super().__init__()                            #input shape: (batch, 28, 28, 1)
    self.conv1 = MnistConvolutional(1, 32, 5)  # out shape : (batch, 28, 28, 32)
    self.pool1 = MaxPool2D([2,2])                 # out shape : (batch, 14, 14, 32)
    self.conv2 = MnistConvolutional(32, 64, 5) # out shape : (batch, 14, 14, 64)
    self.pool2 = MaxPool2D([2,2])                 # out shape : (batch, 7, 7, 64)
    self.flatten = Flatten()                      # out shape : (batch, 7*7*64)
    self.fc1 = MnistFullyConnected(7*7*64, 1024)  # out shape : (batch, 1024)
    self.dropout = Dropout(0.5)                   # out shape : unchanged
    self.fc2 = MnistFullyConnected(1024, 10)      # out shape : (batch, 10)
    self.softmax = Softmax()                      # out shape : unchanged

  def call(self, x, training=False):
    x = tf.nn.relu(self.conv1(x))
    x = self.pool1(x)
    x = tf.nn.relu(self.conv2(x))
    x = self.pool2(x)

    x = self.flatten(x)
    x = tf.nn.relu(self.fc1(x))

    x = self.dropout(x, training=training) # behavior of dropout changes between train and test

    x = self.fc2(x)
    prob = self.softmax(x)

    return prob

# Create an instance of the model
network = MnistDeepModel()
```

Choose an optimizer and loss function for training:

```python
network_loss = tf.keras.losses.CategoricalCrossentropy()

network_optimizer = tf.keras.optimizers.Adam(learning_rate=1e-4)
```

Train the model using the same helper function we created for the perceptron

```python
EPOCHS = 10
train_loop(EPOCHS, train_ds,  network, network_loss, network_optimizer)
```

    Epoch 1, Time 0:00:12.641701, Loss: 0.5832469463348389, Accuracy: 83.6816635131836
    Epoch 2, Time 0:00:11.714608, Loss: 0.15075995028018951, Accuracy: 95.40833282470703
    Epoch 3, Time 0:00:11.438412, Loss: 0.09863907843828201, Accuracy: 96.92666625976562
    Epoch 4, Time 0:00:11.406362, Loss: 0.07229878753423691, Accuracy: 97.79166412353516
    Epoch 5, Time 0:00:11.483109, Loss: 0.0608312226831913, Accuracy: 98.08833312988281
    Epoch 6, Time 0:00:11.546565, Loss: 0.048298757523298264, Accuracy: 98.49832916259766
    Epoch 7, Time 0:00:11.366544, Loss: 0.03957162797451019, Accuracy: 98.77333068847656
    Epoch 8, Time 0:00:11.354394, Loss: 0.03575403615832329, Accuracy: 98.79500579833984
    Epoch 9, Time 0:00:11.544217, Loss: 0.031041104346513748, Accuracy: 98.98666381835938
    Epoch 10, Time 0:00:11.939246, Loss: 0.02749975211918354, Accuracy: 99.08333587646484

Then perform evaluation over the trained network

```python
test_loop(test_ds, network, network_loss)
```

    Test Loss: 0.026255810633301735, Test Accuracy: 99.0999984741211

The image classifier is now trained to ~99% accuracy on this dataset.
By selecting a better set of hyperparameters
(number of epochs, batch size, number of neurons in the various  layers, number of layers, etc..)
we could obtain even better results.
To learn more, read the [TensorFlow tutorials](https://www.tensorflow.org/tutorials).


```python
import numpy as np
import matplotlib.pyplot as plt
plt.style.use('classic')
%matplotlib inline
```

# Class 5: Managing Data with Pandas

Pandas is a Python library for managing datasets. Documentation and examples are available on the website for Pandas: http://pandas.pydata.org/.

In this Notebook, we'll make use of a dataset containing long-run averages of inflation, money growth, and real GDP. The dataset is available here: https://raw.githubusercontent.com/letsgoexploring/economic-data/master/quantity-theory/csv/quantity_theory_data.csv (Python code to generate the dataset: https://github.com/letsgoexploring/economic-data). Recall that the quantity theory of money implies the following linear relationship between the long-run rate of money growth, the long-run rate of inflation, and the long-run rate of real GDP growth in a country:

\begin{align}
\text{inflation} & = \text{money growth} - \text{real GDP growth},
\end{align}

Generally, we treat real GDP growth and money supply growth as exogenous so this is a theory about the determination of inflation.

### Import Pandas

```python
# Import the Pandas module as pd
import pandas as pd
```

### Import data from a csv file

Pandas has a function called `read_csv()` for reading data from a csv file into a Pandas `DataFrame` object.

```python
# Import quantity theory data into a Pandas DataFrame called 'df' with country names as the index.

# Directly from internet
df = pd.read_csv('https://raw.githubusercontent.com/letsgoexploring/economic-data/master/quantity-theory/csv/quantity_theory_data.csv')

# From current working directory
# df = pd.read_csv('qtyTheoryData.csv')
```

```python
# Print the first 5 rows
print(df.head())
```

                   country iso code  observations  inflation  money growth  \
    0              Albania      ALB            21    0.05453       0.12879
    1              Algeria      DZA            51    0.10913       0.16370
    2               Angola      AGO            20    0.88679       1.08457
    3  Antigua and Barbuda      ATG            38    0.04052       0.09903
    4            Argentina      ARG            54    0.74972       0.79170

       gdp growth
    0     0.04829
    1     0.03896
    2     0.08592
    3     0.03673
    4     0.02468

```python
# Print the last 10 rows
print(df.tail(10))
```

                      country iso code  observations  inflation  money growth  \
    168               Ukraine      UKR            23    0.61867       0.58537
    169  United Arab Emirates      ARE            40    0.03659       0.14033
    170         United States      USA            54    0.03442       0.05753
    171               Uruguay      URY            54    0.37658       0.41383
    172               Vanuatu      VUT            36    0.03611       0.08343
    173         Venezuela, RB      VEN            53    0.20726       0.27581
    174    West Bank and Gaza      PSE            17    0.03864       0.12528
    175           Yemen, Rep.      YEM            24    0.14770       0.13852
    176                Zambia      ZMB            27    0.22850       0.24470
    177              Zimbabwe      ZWE            32   -0.00128      -0.13307

         gdp growth
    168    -0.01070
    169     0.04811
    170     0.03140
    171     0.02294
    172     0.02972
    173     0.02793
    174     0.03389
    175     0.04008
    176     0.01208
    177     0.00839

```python
# Print the type of variable 'df'
print(type(df))
```

    <class 'pandas.core.frame.DataFrame'>

### Properties of `DataFrame` objects

Like entries in a spreadsheet file, elements in a `DataFrame` object have row (or *index*) and column coordinates. Column names are always strings. Index elements can be integers, strings, or dates.

```python
# Print the columns of df
print(df.columns)
```

    Index(['country', 'iso code', 'observations', 'inflation', 'money growth',
           'gdp growth'],
          dtype='object')

```python
# Create a new variable called 'money' equal to the 'money growth' column and print
money = df['money growth']
print(money)
```

    0      0.12879
    1      0.16370
    2      1.08457
    3      0.09903
    4      0.79170
            ...
    173    0.27581
    174    0.12528
    175    0.13852
    176    0.24470
    177   -0.13307
    Name: money growth, Length: 178, dtype: float64

```python
# Print the type of the variable money
print(type(money))
```

    <class 'pandas.core.series.Series'>

A Pandas `Series` stores one column of data. Like a `DataFrame`, a `Series` object has an index. Note that `money` has the same index as `df`. Instead of having a column, the `Series` has a `name` attribute.

```python
# Print the name of the 'money' variable
print(money.name)
```

    money growth

Select multiple columns of a `DataFrame` by puting the desired column names in a set a of square brackets (i.e., in a `list`).

```python
# Print the first 5 rows of just the inflation, money growth, and gdp growth columns
print(df[['inflation','money growth','gdp growth']].head())
```

       inflation  money growth  gdp growth
    0    0.05453       0.12879     0.04829
    1    0.10913       0.16370     0.03896
    2    0.88679       1.08457     0.08592
    3    0.04052       0.09903     0.03673
    4    0.74972       0.79170     0.02468

As mentioned, the set of row coordinates is the index. Unless specified otherwise, Pandas automatically assigns an integer index starting at 0 to rows of the `DataFrame`.

```python
# Print the index of 'df'
print(df.index)
```

    RangeIndex(start=0, stop=178, step=1)

Note that in the index of the `df` is the numbers 0 through 177. We could have specified a different index when we imported the data using `read_csv()`. For example, suppose we want to the country names to be the index of `df`. Since country names are in the first column of the data file, we can pass the argument `index_col=0` to `read_csv()`

```python
# Import quantity theory data into a Pandas DataFrame called 'df' with country names as the index.
df = pd.read_csv('https://raw.githubusercontent.com/letsgoexploring/economic-data/master/quantity-theory/csv/quantity_theory_data.csv',index_col=0)

# Print first 5 rows of df
print(df.head())
```

                        iso code  observations  inflation  money growth  \
    country
    Albania                  ALB            21    0.05453       0.12879
    Algeria                  DZA            51    0.10913       0.16370
    Angola                   AGO            20    0.88679       1.08457
    Antigua and Barbuda      ATG            38    0.04052       0.09903
    Argentina                ARG            54    0.74972       0.79170

                         gdp growth
    country
    Albania                 0.04829
    Algeria                 0.03896
    Angola                  0.08592
    Antigua and Barbuda     0.03673
    Argentina               0.02468

Use the `loc` attribute to select rows of the `DataFrame` by index *values*.

```python
# Create a new variable called 'usa_row' equal to the 'United States' row and print
usa_row = df.loc['United States']
print(usa_row)
```

    iso code            USA
    observations         54
    inflation       0.03442
    money growth    0.05753
    gdp growth       0.0314
    Name: United States, dtype: object

Use `iloc` attribute to select row based on integer location (starting from 0).

```python
# Create a new variable called 'third_row' equal to the third row in the DataFrame and print
third_row = df.iloc[2]
print(third_row)
```

    iso code            AGO
    observations         20
    inflation       0.88679
    money growth    1.08457
    gdp growth      0.08592
    Name: Angola, dtype: object

There are several ways to return a single element of a Pandas `DataFrame`. For example, here are three that we want to return the value of inflation for the United States from the DataFrame `df`:

1. `df.loc['United States','inflation']`
2. `df.loc['United States']['inflation']`
3. `df['inflation']['United States']`

The first method points directly to the element in the `df` while the second and third methods return *copies* of the element. That means that you can modify the value of inflation for the United States by running:

    df.loc['United States','inflation'] = new_value

But running either:

    df.loc['United States']['inflation'] = new_value

or:

    df['inflation']['United States'] = new_value

will return a warning from Pandas.

```python
# Print the inflation rate of the United States  (By index and column together)
print('Long-run average inflation in US: ',df.loc['United States','inflation'])
```

    Long-run average inflation in US:  0.03442

```python
# Print the inflation rate of the United States (first by index, then by column)
print('Long-run average inflation in US: ',df.loc['United States']['inflation'])
```

    Long-run average inflation in US:  0.03442

```python
# Print the inflation rate of the United States  (first by column, then by index)
print('Long-run average inflation in US: ',df['inflation']['United States'])
```

    Long-run average inflation in US:  0.03442

New columns are easily created as functions of existing columns.

```python
# Create a new column called 'difference' equal to the money growth column minus
# the inflation column and print the modified DataFrame
df['difference'] = df['money growth'] - df['inflation']
print(df['difference'])
```

    country
    Albania                0.07426
    Algeria                0.05457
    Angola                 0.19778
    Antigua and Barbuda    0.05851
    Argentina              0.04198
                            ...
    Venezuela, RB          0.06855
    West Bank and Gaza     0.08664
    Yemen, Rep.           -0.00918
    Zambia                 0.01620
    Zimbabwe              -0.13179
    Name: difference, Length: 178, dtype: float64

```python
# Print the average difference between money growth and inflation
print(df.difference.mean())
```

    0.06339837078651682

```python
# Remove the following columns from the DataFrame: 'iso code','observations','difference'
df = df.drop(['iso code','observations','difference'],axis=1)

# Print the modified DataFrame
print(df)
```

                         inflation  money growth  gdp growth
    country
    Albania                0.05453       0.12879     0.04829
    Algeria                0.10913       0.16370     0.03896
    Angola                 0.88679       1.08457     0.08592
    Antigua and Barbuda    0.04052       0.09903     0.03673
    Argentina              0.74972       0.79170     0.02468
    ...                        ...           ...         ...
    Venezuela, RB          0.20726       0.27581     0.02793
    West Bank and Gaza     0.03864       0.12528     0.03389
    Yemen, Rep.            0.14770       0.13852     0.04008
    Zambia                 0.22850       0.24470     0.01208
    Zimbabwe              -0.00128      -0.13307     0.00839

    [178 rows x 3 columns]

### Methods

A Pandas `DataFrame` has a bunch of useful methods defined for it. `describe()` returns some summary statistics.

```python
# Print the summary statistics for 'df'
print(df.describe())
```

            inflation  money growth  gdp growth
    count  178.000000    178.000000  178.000000
    mean     0.128568      0.191966    0.037594
    std      0.172497      0.170733    0.022616
    min     -0.001280     -0.133070   -0.025240
    25%      0.042493      0.102785    0.024382
    50%      0.070050      0.140935    0.037745
    75%      0.132060      0.210498    0.048245
    max      1.277050      1.224430    0.165480

The `corr()` method returns a `DataFrame` containing the correlation coefficients of the specified `DataFrame`.

```python
# Create a variable called 'correlations' containg the correlation coefficients for columns in 'df'
correlations = df.corr()

# Print the correlation coefficients
print(correlations)
```

                  inflation  money growth  gdp growth
    inflation      1.000000      0.971200   -0.041515
    money growth   0.971200      1.000000    0.088061
    gdp growth    -0.041515      0.088061    1.000000

```python
# Print the correlation coefficient for inflation and money growth
print('corr of inflation and money growth: ',round(correlations.loc['inflation','money growth'],4))

# Print the correlation coefficient for inflation and real GDP growth
print('corr of inflation and gdp growth:   ',round(correlations.loc['inflation','gdp growth'],4))

# Print the correlation coefficient for money growth and real GDP growth
print('corr of money growth and gdp growth:',round(correlations.loc['money growth','gdp growth'],4))
```

    corr of inflation and money growth:  0.9712
    corr of inflation and gdp growth:    -0.0415
    corr of money growth and gdp growth: 0.0881

`sort_values()` returns a copy of the original `DataFrame` sorted along the given column. The optional argument `ascending` is set to `True` by default, but can be changed to `False` if you want to print the lowest first.

```python
# Print rows for the countries with the 10 lowest inflation rates
print(df.sort_values('inflation').head(10))
```

                          inflation  money growth  gdp growth
    country
    Zimbabwe               -0.00128      -0.13307     0.00839
    Djibouti                0.00035       0.07940     0.01683
    Germany                 0.01120       0.07852     0.01233
    Hong Kong SAR, China    0.01452       0.10295     0.03778
    Ireland                 0.01467       0.13223     0.03552
    France                  0.01560       0.07038     0.01316
    Kosovo                  0.01659       0.09355     0.03622
    Greece                  0.01736       0.01533    -0.00487
    Switzerland             0.01754       0.06243     0.01750
    Austria                 0.01757       0.08010     0.01543

```python
# Print rows for the countries with the 10 highest inflation rates
print(df.sort_values('inflation',ascending=False).head(10))
```

                      inflation  money growth  gdp growth
    country
    Congo, Dem. Rep.    1.27705       1.22443    -0.00239
    Brazil              0.88891       0.92023     0.04208
    Angola              0.88679       1.08457     0.08592
    Argentina           0.74972       0.79170     0.02468
    Nicaragua           0.62762       0.67775     0.02522
    Belarus             0.62519       0.65668     0.05171
    Ukraine             0.61867       0.58537    -0.01070
    Peru                0.48310       0.54905     0.03510
    Armenia             0.46680       0.48970     0.05726
    Azerbaijan          0.45091       0.52949     0.05724

Note that `sort_values` and `sort_index` return *copies* of the original `DataFrame`. If, in the previous example, we had wanted to actually modify `df`, we would have need to explicitly overwrite it:

    df = df.sort_index(ascending=False)

```python
# Print first 10 rows with the index sorted in descending alphabetical order
print(df.sort_index(ascending=False).head(10))
```

                          inflation  money growth  gdp growth
    country
    Zimbabwe               -0.00128      -0.13307     0.00839
    Zambia                  0.22850       0.24470     0.01208
    Yemen, Rep.             0.14770       0.13852     0.04008
    West Bank and Gaza      0.03864       0.12528     0.03389
    Venezuela, RB           0.20726       0.27581     0.02793
    Vanuatu                 0.03611       0.08343     0.02972
    Uruguay                 0.37658       0.41383     0.02294
    United States           0.03442       0.05753     0.03140
    United Arab Emirates    0.03659       0.14033     0.04811
    Ukraine                 0.61867       0.58537    -0.01070

### Quick plotting example

Construct a graph that visually confirms the quantity theory of money by making a scatter plot with average money growth on the horizontal axis and average inflation on the vertical axis. Set the marker size `s` to 50 and opacity (`alpha`) 0.25. Add a 45 degree line, axis labels, and a title. Lower and upper limits for the horizontal and vertical axes should be -0.2 and 1.2.

```python
# Create data for 45 degree line
x45 = [-0.2,1.2]
y45 = [-0.2,1.2]

# Create figure and axis
fig = plt.figure()
ax = fig.add_subplot(1,1,1)

# Plot 45 degree line and create legend in lower right corner
ax.plot(x45,y45,'-r',label = '$45^{\circ}$')
ax.legend(loc='lower right')

# Scatter plot of data inflation against money growth
ax.scatter(df['money growth'],df['inflation'],s=50,alpha = 0.25)
ax.set_xlim([-0.2,1.2])
ax.set_ylim([-0.2,1.2])
ax.set_xlabel('money growth')
ax.set_ylabel('inflation')
ax.set_title('Average inflation against average money growth \nfor '+str(len(df.index))+' countries.')
ax.grid()
```

### Exporting a `DataFrame` to csv

Use the DataFrame method `to_csv()` to export DataFrame to a csv file.

```python
# Export the DataFrame 'df' to a csv file called 'modified_data.csv'.
df.to_csv('modified_data.csv')
```


This is a test of the builder system.

```julia
using Plots
plot(rand(10,10))
```

```math
\begin{equation}
 y'(t) = \frac{0.2y(t-14)}{1 + y(t-14)^{10}} - 0.1y(t)
\end{equation}
```

$\alpha$

``u_0``

## Appendix

```julia
using SciMLBenchmarks
SciMLBenchmarks.bench_footer(WEAVE_ARGS[:folder],WEAVE_ARGS[:file])
```