# **Quantum Computing 101: it's easier than it sounds!**

By the end of this notebook we will write 2 simple quantum programs that will demonstrate the basic components of quantum mechanics which may make quantum computers faster than traditional computers:

*   Superposition (Quantum systems can be in multiple states at once)
*   Entanglement (Quantum computers can do multiple operations at once)

So let's jump right in.
# **The code**

Let's start by importing the shor library, and necessary components.

```
# Install shor library from pypi
!pip install shor

# Import the Circuit object for constructing Quantum Programs
from shor.quantum import Circuit

# The quantum gates necessary for these examples
from shor.gates import CNOT, Hadamard

# The input layer for the program
from shor.layers import Qbits

# The measure operation to get a binary output (classical) for the program
from shor.operations import Measure
```

    Requirement already satisfied: shor in /usr/local/lib/python3.6/dist-packages (0.0.6)
    Requirement already satisfied: matplotlib<4.0.0,>=3.3.3 in /usr/local/lib/python3.6/dist-packages (from shor) (3.3.4)
    Requirement already satisfied: qiskit<0.24.0,>=0.23.1 in /usr/local/lib/python3.6/dist-packages (from shor) (0.23.5)
    Requirement already satisfied: numpy<2.0.0,>=1.19.4 in /usr/local/lib/python3.6/dist-packages (from shor) (1.19.5)
    Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib<4.0.0,>=3.3.3->shor) (1.3.1)
    Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.3 in /usr/local/lib/python3.6/dist-packages (from matplotlib<4.0.0,>=3.3.3->shor) (2.4.7)
    Requirement already satisfied: pillow>=6.2.0 in /usr/local/lib/python3.6/dist-packages (from matplotlib<4.0.0,>=3.3.3->shor) (7.0.0)
    Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.6/dist-packages (from matplotlib<4.0.0,>=3.3.3->shor) (0.10.0)
    Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib<4.0.0,>=3.3.3->shor) (2.8.1)
    Requirement already satisfied: qiskit-aqua==0.8.2 in /usr/local/lib/python3.6/dist-packages (from qiskit<0.24.0,>=0.23.1->shor) (0.8.2)
    Requirement already satisfied: qiskit-ibmq-provider==0.11.1 in /usr/local/lib/python3.6/dist-packages (from qiskit<0.24.0,>=0.23.1->shor) (0.11.1)
    Requirement already satisfied: qiskit-ignis==0.5.2 in /usr/local/lib/python3.6/dist-packages (from qiskit<0.24.0,>=0.23.1->shor) (0.5.2)
    Requirement already satisfied: qiskit-aer==0.7.4 in /usr/local/lib/python3.6/dist-packages (from qiskit<0.24.0,>=0.23.1->shor) (0.7.4)
    Requirement already satisfied: qiskit-terra==0.16.4 in /usr/local/lib/python3.6/dist-packages (from qiskit<0.24.0,>=0.23.1->shor) (0.16.4)
    Requirement already satisfied: six in /usr/local/lib/python3.6/dist-packages (from cycler>=0.10->matplotlib<4.0.0,>=3.3.3->shor) (1.15.0)
    Requirement already satisfied: fastdtw in /usr/local/lib/python3.6/dist-packages (from qiskit-aqua==0.8.2->qiskit<0.24.0,>=0.23.1->shor) (0.3.4)
    Requirement already satisfied: yfinance in /usr/local/lib/python3.6/dist-packages (from qiskit-aqua==0.8.2->qiskit<0.24.0,>=0.23.1->shor) (0.1.55)
    Requirement already satisfied: quandl in /usr/local/lib/python3.6/dist-packages (from qiskit-aqua==0.8.2->qiskit<0.24.0,>=0.23.1->shor) (3.6.0)
    Requirement already satisfied: retworkx>=0.5.0 in /usr/local/lib/python3.6/dist-packages (from qiskit-aqua==0.8.2->qiskit<0.24.0,>=0.23.1->shor) (0.7.2)
    Requirement already satisfied: docplex==2.15.194 in /usr/local/lib/python3.6/dist-packages (from qiskit-aqua==0.8.2->qiskit<0.24.0,>=0.23.1->shor) (2.15.194)
    Requirement already satisfied: h5py in /usr/local/lib/python3.6/dist-packages (from qiskit-aqua==0.8.2->qiskit<0.24.0,>=0.23.1->shor) (2.10.0)
    Requirement already satisfied: pandas in /usr/local/lib/python3.6/dist-packages (from qiskit-aqua==0.8.2->qiskit<0.24.0,>=0.23.1->shor) (1.1.5)
    Requirement already satisfied: scipy>=1.4 in /usr/local/lib/python3.6/dist-packages (from qiskit-aqua==0.8.2->qiskit<0.24.0,>=0.23.1->shor) (1.4.1)
    Requirement already satisfied: setuptools>=40.1.0 in /usr/local/lib/python3.6/dist-packages (from qiskit-aqua==0.8.2->qiskit<0.24.0,>=0.23.1->shor) (53.0.0)
    Requirement already satisfied: scikit-learn>=0.20.0 in /usr/local/lib/python3.6/dist-packages (from qiskit-aqua==0.8.2->qiskit<0.24.0,>=0.23.1->shor) (0.22.2.post1)
    Requirement already satisfied: dataclasses; python_version < "3.7" in /usr/local/lib/python3.6/dist-packages (from qiskit-aqua==0.8.2->qiskit<0.24.0,>=0.23.1->shor) (0.8)
    Requirement already satisfied: dlx in /usr/local/lib/python3.6/dist-packages (from qiskit-aqua==0.8.2->qiskit<0.24.0,>=0.23.1->shor) (1.0.4)
    Requirement already satisfied: sympy>=1.3 in /usr/local/lib/python3.6/dist-packages (from qiskit-aqua==0.8.2->qiskit<0.24.0,>=0.23.1->shor) (1.7.1)
    Requirement already satisfied: psutil>=5 in /usr/local/lib/python3.6/dist-packages (from qiskit-aqua==0.8.2->qiskit<0.24.0,>=0.23.1->shor) (5.4.8)
    Requirement already satisfied: requests>=2.19 in /usr/local/lib/python3.6/dist-packages (from qiskit-ibmq-provider==0.11.1->qiskit<0.24.0,>=0.23.1->shor) (2.23.0)
    Requirement already satisfied: websockets>=8 in /usr/local/lib/python3.6/dist-packages (from qiskit-ibmq-provider==0.11.1->qiskit<0.24.0,>=0.23.1->shor) (8.1)
    Requirement already satisfied: nest-asyncio!=1.1.0,>=1.0.0 in /usr/local/lib/python3.6/dist-packages (from qiskit-ibmq-provider==0.11.1->qiskit<0.24.0,>=0.23.1->shor) (1.5.1)
    Requirement already satisfied: requests-ntlm>=1.1.0 in /usr/local/lib/python3.6/dist-packages (from qiskit-ibmq-provider==0.11.1->qiskit<0.24.0,>=0.23.1->shor) (1.1.0)
    Requirement already satisfied: urllib3>=1.21.1 in /usr/local/lib/python3.6/dist-packages (from qiskit-ibmq-provider==0.11.1->qiskit<0.24.0,>=0.23.1->shor) (1.24.3)
    Requirement already satisfied: networkx>=2.2 in /usr/local/lib/python3.6/dist-packages (from qiskit-ignis==0.5.2->qiskit<0.24.0,>=0.23.1->shor) (2.5)
    Requirement already satisfied: cython>=0.27.1 in /usr/local/lib/python3.6/dist-packages (from qiskit-aer==0.7.4->qiskit<0.24.0,>=0.23.1->shor) (0.29.21)
    Requirement already satisfied: pybind11>=2.4 in /usr/local/lib/python3.6/dist-packages (from qiskit-aer==0.7.4->qiskit<0.24.0,>=0.23.1->shor) (2.6.2)
    Requirement already satisfied: contextvars>=2.4; python_version < "3.7" in /usr/local/lib/python3.6/dist-packages (from qiskit-terra==0.16.4->qiskit<0.24.0,>=0.23.1->shor) (2.4)
    Requirement already satisfied: dill>=0.3 in /usr/local/lib/python3.6/dist-packages (from qiskit-terra==0.16.4->qiskit<0.24.0,>=0.23.1->shor) (0.3.3)
    Requirement already satisfied: fastjsonschema>=2.10 in /usr/local/lib/python3.6/dist-packages (from qiskit-terra==0.16.4->qiskit<0.24.0,>=0.23.1->shor) (2.15.0)
    Requirement already satisfied: ply>=3.10 in /usr/local/lib/python3.6/dist-packages (from qiskit-terra==0.16.4->qiskit<0.24.0,>=0.23.1->shor) (3.11)
    Requirement already satisfied: python-constraint>=1.4 in /usr/local/lib/python3.6/dist-packages (from qiskit-terra==0.16.4->qiskit<0.24.0,>=0.23.1->shor) (1.4.0)
    Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.6/dist-packages (from qiskit-terra==0.16.4->qiskit<0.24.0,>=0.23.1->shor) (2.6.0)
    Requirement already satisfied: multitasking>=0.0.7 in /usr/local/lib/python3.6/dist-packages (from yfinance->qiskit-aqua==0.8.2->qiskit<0.24.0,>=0.23.1->shor) (0.0.9)
    Requirement already satisfied: lxml>=4.5.1 in /usr/local/lib/python3.6/dist-packages (from yfinance->qiskit-aqua==0.8.2->qiskit<0.24.0,>=0.23.1->shor) (4.6.2)
    Requirement already satisfied: inflection>=0.3.1 in /usr/local/lib/python3.6/dist-packages (from quandl->qiskit-aqua==0.8.2->qiskit<0.24.0,>=0.23.1->shor) (0.5.1)
    Requirement already satisfied: more-itertools in /usr/local/lib/python3.6/dist-packages (from quandl->qiskit-aqua==0.8.2->qiskit<0.24.0,>=0.23.1->shor) (8.7.0)
    Requirement already satisfied: pytz>=2017.2 in /usr/local/lib/python3.6/dist-packages (from pandas->qiskit-aqua==0.8.2->qiskit<0.24.0,>=0.23.1->shor) (2018.9)
    Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.6/dist-packages (from scikit-learn>=0.20.0->qiskit-aqua==0.8.2->qiskit<0.24.0,>=0.23.1->shor) (1.0.0)
    Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.6/dist-packages (from sympy>=1.3->qiskit-aqua==0.8.2->qiskit<0.24.0,>=0.23.1->shor) (1.1.0)
    Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.6/dist-packages (from requests>=2.19->qiskit-ibmq-provider==0.11.1->qiskit<0.24.0,>=0.23.1->shor) (2020.12.5)
    Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.6/dist-packages (from requests>=2.19->qiskit-ibmq-provider==0.11.1->qiskit<0.24.0,>=0.23.1->shor) (2.10)
    Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.6/dist-packages (from requests>=2.19->qiskit-ibmq-provider==0.11.1->qiskit<0.24.0,>=0.23.1->shor) (3.0.4)
    Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python3.6/dist-packages (from requests-ntlm>=1.1.0->qiskit-ibmq-provider==0.11.1->qiskit<0.24.0,>=0.23.1->shor) (3.4.5)
    Requirement already satisfied: ntlm-auth>=1.0.2 in /usr/local/lib/python3.6/dist-packages (from requests-ntlm>=1.1.0->qiskit-ibmq-provider==0.11.1->qiskit<0.24.0,>=0.23.1->shor) (1.5.0)
    Requirement already satisfied: decorator>=4.3.0 in /usr/local/lib/python3.6/dist-packages (from networkx>=2.2->qiskit-ignis==0.5.2->qiskit<0.24.0,>=0.23.1->shor) (4.4.2)
    Requirement already satisfied: immutables>=0.9 in /usr/local/lib/python3.6/dist-packages (from contextvars>=2.4; python_version < "3.7"->qiskit-terra==0.16.4->qiskit<0.24.0,>=0.23.1->shor) (0.15)
    Requirement already satisfied: cffi>=1.12 in /usr/local/lib/python3.6/dist-packages (from cryptography>=1.3->requests-ntlm>=1.1.0->qiskit-ibmq-provider==0.11.1->qiskit<0.24.0,>=0.23.1->shor) (1.14.4)
    Requirement already satisfied: pycparser in /usr/local/lib/python3.6/dist-packages (from cffi>=1.12->cryptography>=1.3->requests-ntlm>=1.1.0->qiskit-ibmq-provider==0.11.1->qiskit<0.24.0,>=0.23.1->shor) (2.20)

### **Program 1: A Quantum Coin - Superposition on a quantum computer**

This program can be written with just a single qbit, and demonstrates quantum superposition.

```
# Program 1: A Quantum Coin - Superposition on a quantum computer

# Build the circuit.
qc = Circuit()
qc.add(Qbits(1))
qc.add(Hadamard(0))
qc.add(Measure(0))

# Draw the circuit diagram.
print(qc.draw())

# Run the progam using a quantum simulator
job = qc.run(100)
print(job.result.counts)
print(f'With our quantum coin, we flipped {job.result[0]} heads and {job.result[1]} tails')

```

               ┌───┐ ░ ┌─┐
          q_0: ┤ H ├─░─┤M├
               └───┘ ░ └╥┘
          c: 1/═════════╬═
                        ║
    measure: 1/═════════╩═
                        0
    {0: 47, 1: 53}
    With our quantum coin, we flipped 47 heads and 53 tails

# **Understanding the code**
Quantum computing can be a bit daunting at first, but the rest of this notebook should help give you a basic understanding of the math behind quantum computers.

To understand these programs, and quantum computers in general, it is simpler to start with something familiar, everyday classical computers, and work our way up from there.

## **What is a computer?**
A device which stores information (bits), and changes that information by following a set of rules (computation) to do something clever or useful.

### ***Bit: A tiny switch, represents information***
When we say information, we mean a thing or a concept. To define a thing or a concept, you must define what makes that thing or concept different.

Bits represent differences at the simplest level by enhabiting one of two possibilites: this or that, left or right, on or off, etc. A bit is a "two state system" and can represent any two things, states, or classes.

Typically we will see bits representing states like:

_0, off, or false_

_1, on , or true_

It can be helpful to think of bits like tiny light switches.

### ***Multiple Bits: Represent more information***
In order to represent more complicated things, like numbers, you need multiple bits.

For the numbers 0 through 7 (an 8 state system) we need 3 bits, since 3 bits have 8 possible combinations:

$$Zero: 000	\quad	One: 001	\quad	Two: 010	\quad	Three: 011	\quad	Four: 100	\quad	Five: 101	\quad	Six: 110 \quad Seven: 111$$

In general, $n$ bits can represent one of $2 ^ n$ possible states.

### ***Computation / Gates: Flipping bits***
A computation, or gate, changes information (a list of bits) from one state to another. It flips bits.

A simple computational gate, NOT, flips a single bit:
$$0 \rightarrow 1$$
$$1 \rightarrow 0$$

More complicated gates use multiple input bits, and can operate on seperate output bits.

The "and" gate uses 2 input bits and 1 output bit. It outputs 1 if and only if both the input bits are 1:

$$00 \rightarrow 0$$
$$01 \rightarrow 0$$
$$10 \rightarrow 0$$
$$11 \rightarrow 1$$

## **State Vector: Another way to view bits**
A useful mathematical way to view bits is to use a list of numbers called the "state vector" (also called a "one hot" encoding.)

The state vector is a list of numbers, all of which are either zero or one.
It has two properties:
- Only a single number is set to 1, everything else is 0
- The length of the list is equal to the number of possible states it represents

For example, a single bit is 2 state system. The state vector representation would look like this:

$\vec{0} = |0\rangle=\begin{bmatrix}1 \\ 0 \end{bmatrix}\qquad$$\vec{1} = |1\rangle = \begin{bmatrix}0 \\1 \end{bmatrix}$

For some numbers between 0 - 7 (8 state system) their state vector representation looks like this:

$Six=\vec{6}=|110\rangle=\begin{bmatrix}0 \\0 \\0 \\0 \\0 \\0 \\1 \\0\end{bmatrix}\quad$
$Two=\vec{2}=|010\rangle=\begin{bmatrix}0 \\0 \\1 \\0 \\0 \\0 \\0 \\0\end{bmatrix}\quad$
$Zero=\vec{0}=|010\rangle=\begin{bmatrix}1 \\0 \\0 \\0 \\0 \\0 \\0 \\0\end{bmatrix}\quad$

### ***Computing with state vectors***
Computations can be represented using matrix multiplication when using the state vector representation.

The NOT gate can be represented as the following 2 x 2 matrix:
$$NOT =\bigoplus=\begin{pmatrix}0 & 1 \\ 1 & 0\end{pmatrix}$$

As we can see, :
$$NOT\; |0\rangle = \begin{pmatrix}0 & 1 \\ 1 & 0\end{pmatrix}  \begin{bmatrix}1 \\ 0\end{bmatrix} = \begin{bmatrix}0 \\ 1\end{bmatrix} = |1\rangle$$

$$NOT \, |1\rangle = \begin{pmatrix}0 & 1 \\ 1 & 0\end{pmatrix} \begin{bmatrix}0 \\ 1\end{bmatrix}= \begin{bmatrix}1 \\ 0\end{bmatrix} = |0\rangle$$

## **What is a quantum computer?**
A device which stores information (qbits), and changes that information by following a set of rules (quantum computation) to do something clever or useful.

### ***Qbit / qubit: A tiny wheel, represent information***
The biggest difference between qbits and bits are the two properties:
- Superposition
- Entanglement

### ***A Single Qbit: Superposition***
Superposition refers to the ability of the qbit to be in multiple positions at the same time. But, this is an overly complicated way of viewing it.

More simply put:

Bits are like traditional light switches:

Qbits are like circular dimmers (before they are measured):

With a bit, you have two possible values: on or off, 1 or 0.

With a qbit, you can spin it around, you have infinitely many rotational states. Straight up means the qbit is all the way on ($|1\rangle$ state), and straight down means it is all the way off ($|0\rangle$ state), but every other position on that dimmer is partially on and partially off.

In a sense, the qbit can be both on and off at the same time. This is known as superposition.

We are going from digital to analog, from discrete to continuous, and because of that, we can store a LOT more (infinitely more) information in a single qbit than we could hope to store in a regular bit. However, there is a catch:

### ***Measurement: Changes the qbit***

When we actually measure a qbit, it snaps to either $|1\rangle$ or $|0\rangle$ and nothing in between.

It is as if we rotate the dimmer switch, and then open our eyes to find the lights either all the way on or all the off, no matter how we rotate the dimmer switch.

That was a lot, so let's use our familiar state vector to make this more concrete.

### ***State Vector: Another way to view qbits***

We can represent qbits using the state vector representation, in fact, we were doing quantum comuting all along! The very same representation we used for bits applies to qbits:

$\vec{0} = |0\rangle=\begin{bmatrix}1 \\ 0 \end{bmatrix}\qquad$$\vec{1} = |1\rangle = \begin{bmatrix}0 \\1 \end{bmatrix}$

These are the 0 and 1 states for the qbit, but more generally, since the qbit is like a wheel, it can take on many more states:

$\begin{bmatrix}\alpha \\ \beta \end{bmatrix}\qquad$ where $\qquad \alpha ^2 + \beta ^2 = 1$

The second term is the pythagorean theorem. It ensures that the qbit's components have a fixed radius of 1, or in other words, a qbit's state is a rotation on a circle.

All "superposition" is referring to, is that we have a continuous set of values ($\alpha$ and $\beta$) instead of 2 states: just 0 or just 1.

### ***Multiple Qbits: Entanglement***

In a similar way to bits, we can store WAY more information with multiple qbits than with a single qbit.

If we look at the state vector representation for 3 qbits:
$$\begin{bmatrix}a \\b \\c \\d \\e \\f \\g \\h\end{bmatrix}$$
where:
$$a^2 + b^2 + c ^2 + d ^2 + e ^2 + f ^2 + g^2 + h^2 = 1$$

Amazingly, if we have $n$ qbits, we have $2^n$ continuous angles to work with!! This scaling factor is the second reason that quantum computers have an advantage.

Even with just 30 qbits, that gives:

$2^{30} = 1,073,741,824$ continuous variables.

# **Making the connection**

Armed with our mathematical representation, we can re-visit our first program.

## Program 1: A Quantum Coin - Superposition

#### ***Building the circuit***
* We will create a `Circuit()` with a single qbit, which by convention starts in the $|0\rangle$ state.

* Then we will add a `Hadamard` gate, which rotates the qbit precisely halfway between $|0\rangle$ and $|1\rangle$.

* Then we add a `Measure` operation our qbit, which will "collapse" the qbit into either $|0\rangle$ or $|1\rangle$.

* Finally, we will run the circuit 100 times`

#### ***What should we expect?***
Since the qbit is rotated halfway between $|0\rangle$ and $|1\rangle$, we expect the measured qbit to "collapse" to those states with equal probability (about 50 times each).

#### ***Why is this "superposition"?***
Because our circuit is moving the qbit into a state that is an approximately equal linear combination of the basis states $|0\rangle$ or $|1\rangle$

Rather than just being 0, or just being 1 there is an equal chance of measuring either.

```
# Program 1: A Quantum Coin - Superposition on a quantum computer

qc = Circuit()
qc.add(Qbits(1))      # Add 1 Qbit, which by convention starts in the 0 / down / off state.
qc.add(Hadamard(0))   # Apply the Hadamard gate to the 1st qbit (index 0), rotating it to a 50 - 50 superposition
qc.add(Measure(0))    # Measure the 1st qbit (index 0)

print(qc.draw())

job = qc.run(100)  # Run the circuit 100 times (using a simulator)
print(job.result.counts)
```

               ┌───┐ ░ ┌─┐
          q_0: ┤ H ├─░─┤M├
               └───┘ ░ └╥┘
          c: 1/═════════╬═
                        ║
    measure: 1/═════════╩═
                        0
    {0: 46, 1: 54}

As expected, we see about half of the 100 results are in the $|0\rangle$ state and half are in the $|1\rangle$!

That is it! We just demonstrated quantum superposition.

### **Program 2: Quantum Entanglement**

Now we will introduce a second qbit into our circuit, and a new gate `CNOT`

#### ***CNOT gate***
`CNOT` or the "controlled not" gate, takes 2 qbits as input. It flips the second (target) qbit if and only if the first (control) qbit is 1.

The `CNOT` transitions the input states to the following outputs:
$$|00\rangle \rightarrow |00\rangle$$
$$|01\rangle \rightarrow |01\rangle$$
$$|10\rangle \rightarrow |11\rangle$$
$$|11\rangle \rightarrow |10\rangle$$

#### ***Building the circuit***
* In our circuit, we will add 2 qbits which start in the $|0\rangle$ state.
* Then, we apply the `Hadamard` gate to move the first (control) qbit into a 50 - 50 superposition state
* Then, we will apply the entanglement gate `CNOT` to the first (control) qbit, and the second (target) qbit
* Then we add a `Measurement` operation on both of our qbits
* Finally, we run the circuit 100 times

#### ***What should we expect?***
As before, we expect the first qbit to be $|0\rangle$ about half of the time, and $|1\rangle$ the other half of the time.

As for the second qbit, it should become "entangled" with the first, through the `CNOT` gate. Referring to the state transitions above, since we expect the inputs at that point in the circuit to be $|00\rangle$ or $|10\rangle$ we expect the output states to be $|00\rangle$ and $|11\rangle$.

#### ***Why is this "entanglement"?***
Entanglement means the state of the system only makes sense with multiple qbits, or in other words, we can't factor the state  back into individual qbits.

In our case, the second qbit's state depends strongly on the first qbits. We can't replicate the result if we try to seperate the circuit into two sub-circutis with 1 qbit each.

```
# Program 2: Entanglement Quantum Circuit with 2 Qbits

qc = Circuit()
qc.add(Qbits(2))
qc.add(Hadamard(0))
qc.add(CNOT(0, 1))   # Add a control not gate with the 0th qbit as the control bit
qc.add(Measure(0, 1))

# Draw the circuit diagramt
print(qc.draw())

# Run the program 100 times
print(qc.run(100).result)
```

               ┌───┐      ░ ┌─┐
          q_0: ┤ H ├──■───░─┤M├───
               └───┘┌─┴─┐ ░ └╥┘┌─┐
          q_1: ─────┤ X ├─░──╫─┤M├
                    └───┘ ░  ║ └╥┘
          c: 2/══════════════╬══╬═
                             ║  ║
    measure: 2/══════════════╩══╩═
                             0  1
    {'00000': 57, '00011': 43}

## ***We just demonstrated quantum entanglement***

As you can see, the results are as expected, and are tied together (entangled) such that only $|00\rangle$ and $|11\rangle$ are observed.


# Batch Normalization
One way to make deep networks easier to train is to **use more sophisticated optimization procedures such as SGD+momentum, RMSProp, or Adam**. Another strategy is to **change the architecture of the network to make it easier to train**.
One idea along these lines is batch normalization which was proposed by [1] in 2015.

The idea is relatively straightforward. **Machine learning methods tend to work better when their input data consists of uncorrelated features with zero mean and unit variance**. When training a neural network, we can preprocess the data before feeding it to the network to explicitly decorrelate its features; this will ensure that the first layer of the network sees data that follows a nice distribution. However, even if we preprocess the input data, the activations at deeper layers of the network will likely no longer be decorrelated and will no longer have zero mean or unit variance since they are output from earlier layers in the network. Even worse, during the training process the distribution of features at each layer of the network will shift as the weights of each layer are updated.

The authors of [1] hypothesize that the shifting distribution of features inside deep neural networks may make training deep networks more difficult. To overcome this problem, [1] proposes to insert **batch normalization layers** into the network. At training time, a **batch normalization layer uses a minibatch of data to estimate the mean and standard deviation of each feature. These estimated means and standard deviations are then used to center and normalize the features of the minibatch. A running average of these means and standard deviations is kept during training, and at test time these running averages are used to center and normalize features**.

It is possible that this normalization strategy could reduce the representational power of the network, since it may sometimes be optimal for certain layers to have features that are not zero-mean or unit variance. To this end, the batch normalization layer includes learnable shift and scale parameters for each feature dimension.

[1] [Sergey Ioffe and Christian Szegedy, "Batch Normalization: Accelerating Deep Network Training by Reducing
Internal Covariate Shift", ICML 2015.](https://arxiv.org/abs/1502.03167)

```python
# As usual, a bit of setup
import time
import numpy as np
import matplotlib.pyplot as plt
from cs231n.classifiers.fc_net import *
from cs231n.data_utils import get_CIFAR10_data
from cs231n.gradient_check import eval_numerical_gradient, eval_numerical_gradient_array
from cs231n.solver import Solver

%matplotlib inline
plt.rcParams['figure.figsize'] = (10.0, 8.0) # set default size of plots
plt.rcParams['image.interpolation'] = 'nearest'
plt.rcParams['image.cmap'] = 'gray'

# for auto-reloading external modules
# see http://stackoverflow.com/questions/1907993/autoreload-of-modules-in-ipython
%load_ext autoreload
%autoreload 2

def rel_error(x, y):
    """ returns relative error """
    return np.max(np.abs(x - y) / (np.maximum(1e-8, np.abs(x) + np.abs(y))))

def print_mean_std(x,axis=0):
    print('  means: ', x.mean(axis=axis))
    print('  stds:  ', x.std(axis=axis))
    print()
```

```python
# Load the (preprocessed) CIFAR10 data.
data = get_CIFAR10_data()
for k, v in data.items():
  print('%s: ' % k, v.shape)
```

    X_train:  (49000, 3, 32, 32)
    y_train:  (49000,)
    X_val:  (1000, 3, 32, 32)
    y_val:  (1000,)
    X_test:  (1000, 3, 32, 32)
    y_test:  (1000,)

## Batch normalization: forward
In the file `cs231n/layers.py`, implement the batch normalization forward pass in the function `batchnorm_forward`. Once you have done so, run the following to test your implementation.

Referencing the paper linked to above in [1] may be helpful!

```python
# Check the training-time forward pass by checking means and variances
# of features both before and after batch normalization

# Simulate the forward pass for a two-layer network
np.random.seed(231)
N, D1, D2, D3 = 200, 50, 60, 3
X = np.random.randn(N, D1)
W1 = np.random.randn(D1, D2)
W2 = np.random.randn(D2, D3)
a = np.maximum(0, X.dot(W1)).dot(W2)

print('Before batch normalization:')
print_mean_std(a,axis=0)

gamma = np.ones((D3,))
beta = np.zeros((D3,))
# Means should be close to zero and stds close to one
print('After batch normalization (gamma=1, beta=0)')
a_norm, _ = batchnorm_forward(a, gamma, beta, {'mode': 'train'})
print_mean_std(a_norm,axis=0)

gamma = np.asarray([1.0, 2.0, 3.0])
beta = np.asarray([11.0, 12.0, 13.0])
# Now means should be close to beta and stds close to gamma
print('After batch normalization (gamma=', gamma, ', beta=', beta, ')')
a_norm, _ = batchnorm_forward(a, gamma, beta, {'mode': 'train'})
print_mean_std(a_norm,axis=0)
```

    Before batch normalization:
      means:  [ -2.3814598  -13.18038246   1.91780462]
      stds:   [27.18502186 34.21455511 37.68611762]

    After batch normalization (gamma=1, beta=0)
      means:  [5.99520433e-17 6.93889390e-17 8.32667268e-19]
      stds:   [0.99999999 1.         1.        ]

    After batch normalization (gamma= [1. 2. 3.] , beta= [11. 12. 13.] )
      means:  [11. 12. 13.]
      stds:   [0.99999999 1.99999999 2.99999999]

```python
# Check the test-time forward pass by running the training-time
# forward pass many times to warm up the running averages, and then
# checking the means and variances of activations after a test-time
# forward pass.

np.random.seed(231)
N, D1, D2, D3 = 200, 50, 60, 3
W1 = np.random.randn(D1, D2)
W2 = np.random.randn(D2, D3)

bn_param = {'mode': 'train'}
gamma = np.ones(D3)
beta = np.zeros(D3)

for t in range(50):
  X = np.random.randn(N, D1)
  a = np.maximum(0, X.dot(W1)).dot(W2)
  batchnorm_forward(a, gamma, beta, bn_param)

bn_param['mode'] = 'test'
X = np.random.randn(N, D1)
a = np.maximum(0, X.dot(W1)).dot(W2)
a_norm, _ = batchnorm_forward(a, gamma, beta, bn_param)

# Means should be close to zero and stds close to one, but will be
# noisier than training-time forward passes.
print('After batch normalization (test-time):')
print_mean_std(a_norm,axis=0)
```

    After batch normalization (test-time):
      means:  [-0.03927354 -0.04349152 -0.10452688]
      stds:   [1.01531428 1.01238373 0.97819988]

## Batch normalization: backward
Now implement the backward pass for batch normalization in the function `batchnorm_backward`.

To derive the backward pass you should write out the computation graph for batch normalization and backprop through each of the intermediate nodes. Some intermediates may have multiple outgoing branches; make sure to sum gradients across these branches in the backward pass.

Once you have finished, run the following to numerically check your backward pass.

```python
# Gradient check batchnorm backward pass
np.random.seed(231)
N, D = 4, 5
x = 5 * np.random.randn(N, D) + 12
gamma = np.random.randn(D)
beta = np.random.randn(D)
dout = np.random.randn(N, D)

bn_param = {'mode': 'train'}
fx = lambda x: batchnorm_forward(x, gamma, beta, bn_param)[0]
fg = lambda a: batchnorm_forward(x, a, beta, bn_param)[0]
fb = lambda b: batchnorm_forward(x, gamma, b, bn_param)[0]

dx_num = eval_numerical_gradient_array(fx, x, dout)
da_num = eval_numerical_gradient_array(fg, gamma.copy(), dout)
db_num = eval_numerical_gradient_array(fb, beta.copy(), dout)

_, cache = batchnorm_forward(x, gamma, beta, bn_param)
dx, dgamma, dbeta = batchnorm_backward(dout, cache)
#You should expect to see relative errors between 1e-13 and 1e-8
print('dx error: ', rel_error(dx_num, dx))
print('dgamma error: ', rel_error(da_num, dgamma))
print('dbeta error: ', rel_error(db_num, dbeta))
```

    dx error:  1.6674604875341426e-09
    dgamma error:  7.417225040694815e-13
    dbeta error:  2.379446949959628e-12

## Batch normalization: alternative backward
In class we talked about two different implementations for the sigmoid backward pass. One strategy is to write out a computation graph composed of simple operations and backprop through all intermediate values. Another strategy is to work out the derivatives on paper. For example, you can derive a very simple formula for the sigmoid function's backward pass by simplifying gradients on paper.

Surprisingly, it turns out that you can do a similar simplification for the batch normalization backward pass too!

In the forward pass, given a set of inputs $X=\begin{bmatrix}x_1\\x_2\\...\\x_N\end{bmatrix}$,

we first calculate the mean $\mu$ and variance $v$.
With $\mu$ and $v$ calculated, we can calculate the standard deviation $\sigma$  and normalized data $Y$.
The equations and graph illustration below describe the computation ($y_i$ is the i-th element of the vector $Y$).

\begin{align}
& \mu=\frac{1}{N}\sum_{k=1}^N x_k  &  v=\frac{1}{N}\sum_{k=1}^N (x_k-\mu)^2 \\
& \sigma=\sqrt{v+\epsilon}         &  y_i=\frac{x_i-\mu}{\sigma}
\end{align}

The meat of our problem during backpropagation is to compute $\frac{\partial L}{\partial X}$, given the upstream gradient we receive, $\frac{\partial L}{\partial Y}.$ To do this, recall the chain rule in calculus gives us $\frac{\partial L}{\partial X} = \frac{\partial L}{\partial Y} \cdot \frac{\partial Y}{\partial X}$.

The unknown/hart part is $\frac{\partial Y}{\partial X}$. We can find this by first deriving step-by-step our local gradients at
$\frac{\partial v}{\partial X}$, $\frac{\partial \mu}{\partial X}$,
$\frac{\partial \sigma}{\partial v}$,
$\frac{\partial Y}{\partial \sigma}$, and $\frac{\partial Y}{\partial \mu}$,
and then use the chain rule to compose these gradients (which appear in the form of vectors!) appropriately to compute $\frac{\partial Y}{\partial X}$.

If it's challenging to directly reason about the gradients over $X$ and $Y$ which require matrix multiplication, try reasoning about the gradients in terms of individual elements $x_i$ and $y_i$ first: in that case, you will need to come up with the derivations for $\frac{\partial L}{\partial x_i}$, by relying on the Chain Rule to first calculate the intermediate $\frac{\partial \mu}{\partial x_i}, \frac{\partial v}{\partial x_i}, \frac{\partial \sigma}{\partial x_i},$ then assemble these pieces to calculate $\frac{\partial y_i}{\partial x_i}$.

You should make sure each of the intermediary gradient derivations are all as simplified as possible, for ease of implementation.

After doing so, implement the simplified batch normalization backward pass in the function `batchnorm_backward_alt` and compare the two implementations by running the following. Your two implementations should compute nearly identical results, but the alternative implementation should be a bit faster.

```python
np.random.seed(231)
N, D = 100, 500
x = 5 * np.random.randn(N, D) + 12
gamma = np.random.randn(D)
beta = np.random.randn(D)
dout = np.random.randn(N, D)

bn_param = {'mode': 'train'}
out, cache = batchnorm_forward(x, gamma, beta, bn_param)

t1 = time.time()
dx1, dgamma1, dbeta1 = batchnorm_backward(dout, cache)
t2 = time.time()
dx2, dgamma2, dbeta2 = batchnorm_backward_alt(dout, cache)
t3 = time.time()

print('dx difference: ', rel_error(dx1, dx2))
print('dgamma difference: ', rel_error(dgamma1, dgamma2))
print('dbeta difference: ', rel_error(dbeta1, dbeta2))
print('speedup: %.2fx' % ((t2 - t1) / (t3 - t2)))
```

    dx difference:  3.423369562270315e-13
    dgamma difference:  0.0
    dbeta difference:  0.0
    speedup: 1.96x

## Fully Connected Nets with Batch Normalization
Now that you have a working implementation for batch normalization, go back to your `FullyConnectedNet` in the file `cs231n/classifiers/fc_net.py`. Modify your implementation to add batch normalization.

Concretely, when the `normalization` flag is set to `"batchnorm"` in the constructor, you should insert a batch normalization layer before each ReLU nonlinearity. The outputs from the last layer of the network should not be normalized. Once you are done, run the following to gradient-check your implementation.

HINT: You might find it useful to define an additional helper layer similar to those in the file `cs231n/layer_utils.py`. If you decide to do so, do it in the file `cs231n/classifiers/fc_net.py`.

```python
np.random.seed(231)
N, D, H1, H2, C = 2, 15, 20, 30, 10
X = np.random.randn(N, D)
y = np.random.randint(C, size=(N,))

# You should expect losses between 1e-4~1e-10 for W,
# losses between 1e-08~1e-10 for b,
# and losses between 1e-08~1e-09 for beta and gammas.
for reg in [0, 3.14]:
  print('Running check with reg = ', reg)
  model = FullyConnectedNet([H1, H2], input_dim=D, num_classes=C,
                            reg=reg, weight_scale=5e-2, dtype=np.float64,
                            normalization='batchnorm')

  loss, grads = model.loss(X, y)
  print('Initial loss: ', loss)

  for name in sorted(grads):
    f = lambda _: model.loss(X, y)[0]
    grad_num = eval_numerical_gradient(f, model.params[name], verbose=False, h=1e-5)
    print('%s relative error: %.2e' % (name, rel_error(grad_num, grads[name])))
  if reg == 0: print()
```

    Running check with reg =  0
    Initial loss:  2.2611955101340957
    W1 relative error: 1.10e-04
    W2 relative error: 3.11e-06
    W3 relative error: 4.05e-10
    b1 relative error: 4.44e-08
    b2 relative error: 2.22e-08
    b3 relative error: 1.01e-10
    beta1 relative error: 7.33e-09
    beta2 relative error: 1.89e-09
    gamma1 relative error: 6.96e-09
    gamma2 relative error: 2.41e-09

    Running check with reg =  3.14
    Initial loss:  6.996533220108303
    W1 relative error: 1.98e-06
    W2 relative error: 2.29e-06
    W3 relative error: 1.11e-08
    b1 relative error: 5.55e-09
    b2 relative error: 2.22e-08
    b3 relative error: 2.10e-10
    beta1 relative error: 6.65e-09
    beta2 relative error: 3.39e-09
    gamma1 relative error: 6.27e-09
    gamma2 relative error: 5.28e-09

# Batchnorm for deep networks
Run the following to train a six-layer network on a subset of 1000 training examples both with and without batch normalization.

```python
np.random.seed(231)
# Try training a very deep net with batchnorm
hidden_dims = [100, 100, 100, 100, 100]

num_train = 1000
small_data = {
  'X_train': data['X_train'][:num_train],
  'y_train': data['y_train'][:num_train],
  'X_val': data['X_val'],
  'y_val': data['y_val'],
}

weight_scale = 2e-2
bn_model = FullyConnectedNet(hidden_dims, weight_scale=weight_scale, normalization='batchnorm')
model = FullyConnectedNet(hidden_dims, weight_scale=weight_scale, normalization=None)

print('Solver with batch norm:')
bn_solver = Solver(bn_model, small_data,
                num_epochs=10, batch_size=50,
                update_rule='adam',
                optim_config={
                  'learning_rate': 1e-3,
                },
                verbose=True,print_every=20)
bn_solver.train()

print('\nSolver without batch norm:')
solver = Solver(model, small_data,
                num_epochs=10, batch_size=50,
                update_rule='adam',
                optim_config={
                  'learning_rate': 1e-3,
                },
                verbose=True, print_every=20)
solver.train()
```

    Solver with batch norm:
    (Epoch 0 / 10) train acc: 0.107000; val_acc: 0.115000
    (Iteration 20 / 200) loss: 1.940574
    (Epoch 1 / 10) train acc: 0.314000; val_acc: 0.266000
    (Iteration 40 / 200) loss: 1.733858
    (Epoch 2 / 10) train acc: 0.385000; val_acc: 0.279000
    (Iteration 60 / 200) loss: 1.588897
    (Epoch 3 / 10) train acc: 0.493000; val_acc: 0.309000
    (Iteration 80 / 200) loss: 1.532039
    (Epoch 4 / 10) train acc: 0.531000; val_acc: 0.307000
    (Iteration 100 / 200) loss: 1.309579
    (Epoch 5 / 10) train acc: 0.574000; val_acc: 0.313000
    (Iteration 120 / 200) loss: 1.090331
    (Epoch 6 / 10) train acc: 0.634000; val_acc: 0.338000
    (Iteration 140 / 200) loss: 1.044100
    (Epoch 7 / 10) train acc: 0.683000; val_acc: 0.323000
    (Iteration 160 / 200) loss: 0.726503
    (Epoch 8 / 10) train acc: 0.769000; val_acc: 0.327000
    (Iteration 180 / 200) loss: 0.856740
    (Epoch 9 / 10) train acc: 0.793000; val_acc: 0.335000
    (Iteration 200 / 200) loss: 0.810437
    (Epoch 10 / 10) train acc: 0.783000; val_acc: 0.327000

    Solver without batch norm:
    (Epoch 0 / 10) train acc: 0.129000; val_acc: 0.131000
    (Iteration 20 / 200) loss: 2.011618
    (Epoch 1 / 10) train acc: 0.283000; val_acc: 0.250000
    (Iteration 40 / 200) loss: 1.853587
    (Epoch 2 / 10) train acc: 0.316000; val_acc: 0.277000
    (Iteration 60 / 200) loss: 1.903742
    (Epoch 3 / 10) train acc: 0.373000; val_acc: 0.282000
    (Iteration 80 / 200) loss: 1.557937
    (Epoch 4 / 10) train acc: 0.390000; val_acc: 0.310000
    (Iteration 100 / 200) loss: 1.309605
    (Epoch 5 / 10) train acc: 0.434000; val_acc: 0.300000
    (Iteration 120 / 200) loss: 1.414160
    (Epoch 6 / 10) train acc: 0.535000; val_acc: 0.345000
    (Iteration 140 / 200) loss: 1.308890
    (Epoch 7 / 10) train acc: 0.530000; val_acc: 0.304000
    (Iteration 160 / 200) loss: 1.029080
    (Epoch 8 / 10) train acc: 0.628000; val_acc: 0.339000
    (Iteration 180 / 200) loss: 0.934465
    (Epoch 9 / 10) train acc: 0.654000; val_acc: 0.342000
    (Iteration 200 / 200) loss: 0.793320
    (Epoch 10 / 10) train acc: 0.712000; val_acc: 0.328000

Run the following to visualize the results from two networks trained above. You should find that using batch normalization helps the network to converge much faster.

```python
def plot_training_history(title, label, baseline, bn_solvers, plot_fn, bl_marker='.', bn_marker='.', labels=None):
    """utility function for plotting training history"""
    plt.title(title)
    plt.xlabel(label)
    bn_plots = [plot_fn(bn_solver) for bn_solver in bn_solvers]
    bl_plot = plot_fn(baseline)
    num_bn = len(bn_plots)
    for i in range(num_bn):
        label='with_norm'
        if labels is not None:
            label += str(labels[i])
        plt.plot(bn_plots[i], bn_marker, label=label)
    label='baseline'
    if labels is not None:
        label += str(labels[0])
    plt.plot(bl_plot, bl_marker, label=label)
    plt.legend(loc='lower center', ncol=num_bn+1)

plt.subplot(3, 1, 1)
plot_training_history('Training loss','Iteration', solver, [bn_solver], \
                      lambda x: x.loss_history, bl_marker='o', bn_marker='o')
plt.subplot(3, 1, 2)
plot_training_history('Training accuracy','Epoch', solver, [bn_solver], \
                      lambda x: x.train_acc_history, bl_marker='-o', bn_marker='-o')
plt.subplot(3, 1, 3)
plot_training_history('Validation accuracy','Epoch', solver, [bn_solver], \
                      lambda x: x.val_acc_history, bl_marker='-o', bn_marker='-o')

plt.gcf().set_size_inches(15, 15)
plt.show()
```

# Batch normalization and initialization
We will now run a small experiment to study the interaction of batch normalization and weight initialization.

The first cell will train 8-layer networks both with and without batch normalization using different scales for weight initialization. The second layer will plot training accuracy, validation set accuracy, and training loss as a function of the weight initialization scale.

```python
np.random.seed(231)
# Try training a very deep net with batchnorm
hidden_dims = [50, 50, 50, 50, 50, 50, 50]
num_train = 1000
small_data = {
  'X_train': data['X_train'][:num_train],
  'y_train': data['y_train'][:num_train],
  'X_val': data['X_val'],
  'y_val': data['y_val'],
}

bn_solvers_ws = {}
solvers_ws = {}
weight_scales = np.logspace(-4, 0, num=20)
for i, weight_scale in enumerate(weight_scales):
  print('Running weight scale %d / %d' % (i + 1, len(weight_scales)))
  bn_model = FullyConnectedNet(hidden_dims, weight_scale=weight_scale, normalization='batchnorm')
  model = FullyConnectedNet(hidden_dims, weight_scale=weight_scale, normalization=None)

  bn_solver = Solver(bn_model, small_data,
                  num_epochs=10, batch_size=50,
                  update_rule='adam',
                  optim_config={
                    'learning_rate': 1e-3,
                  },
                  verbose=False, print_every=200)
  bn_solver.train()
  bn_solvers_ws[weight_scale] = bn_solver

  solver = Solver(model, small_data,
                  num_epochs=10, batch_size=50,
                  update_rule='adam',
                  optim_config={
                    'learning_rate': 1e-3,
                  },
                  verbose=False, print_every=200)
  solver.train()
  solvers_ws[weight_scale] = solver
```

    Running weight scale 1 / 20
    Running weight scale 2 / 20
    Running weight scale 3 / 20
    Running weight scale 4 / 20
    Running weight scale 5 / 20
    Running weight scale 6 / 20
    Running weight scale 7 / 20
    Running weight scale 8 / 20
    Running weight scale 9 / 20
    Running weight scale 10 / 20
    Running weight scale 11 / 20
    Running weight scale 12 / 20
    Running weight scale 13 / 20
    Running weight scale 14 / 20
    Running weight scale 15 / 20
    Running weight scale 16 / 20
    Running weight scale 17 / 20
    Running weight scale 18 / 20
    Running weight scale 19 / 20
    Running weight scale 20 / 20

```python
# Plot results of weight scale experiment
best_train_accs, bn_best_train_accs = [], []
best_val_accs, bn_best_val_accs = [], []
final_train_loss, bn_final_train_loss = [], []

for ws in weight_scales:
  best_train_accs.append(max(solvers_ws[ws].train_acc_history))
  bn_best_train_accs.append(max(bn_solvers_ws[ws].train_acc_history))

  best_val_accs.append(max(solvers_ws[ws].val_acc_history))
  bn_best_val_accs.append(max(bn_solvers_ws[ws].val_acc_history))

  final_train_loss.append(np.mean(solvers_ws[ws].loss_history[-100:]))
  bn_final_train_loss.append(np.mean(bn_solvers_ws[ws].loss_history[-100:]))

plt.subplot(3, 1, 1)
plt.title('Best val accuracy vs weight initialization scale')
plt.xlabel('Weight initialization scale')
plt.ylabel('Best val accuracy')
plt.semilogx(weight_scales, best_val_accs, '-o', label='baseline')
plt.semilogx(weight_scales, bn_best_val_accs, '-o', label='batchnorm')
plt.legend(ncol=2, loc='lower right')

plt.subplot(3, 1, 2)
plt.title('Best train accuracy vs weight initialization scale')
plt.xlabel('Weight initialization scale')
plt.ylabel('Best training accuracy')
plt.semilogx(weight_scales, best_train_accs, '-o', label='baseline')
plt.semilogx(weight_scales, bn_best_train_accs, '-o', label='batchnorm')
plt.legend()

plt.subplot(3, 1, 3)
plt.title('Final training loss vs weight initialization scale')
plt.xlabel('Weight initialization scale')
plt.ylabel('Final training loss')
plt.semilogx(weight_scales, final_train_loss, '-o', label='baseline')
plt.semilogx(weight_scales, bn_final_train_loss, '-o', label='batchnorm')
plt.legend()
plt.gca().set_ylim(1.0, 3.5)

plt.gcf().set_size_inches(15, 15)
plt.show()
```

## Inline Question 1:
Describe the results of this experiment. How does the scale of weight initialization affect models with/without batch normalization differently, and why?

## Answer:
- model with batch normalization is less sensitive to the sacle of weight initialization, since the change with scale is smoother than that of model without batch normalization
- model with batchnorm can work with small weight initialization scale ($10^{-4} - 10^{-2.5}$), but model without batchnorm performs badly when small weight initialization scale is applied
- the best weight initialization scale for model with batchnorm is around $10^{-2}$, while it is around $10^{-1}$ for model without batchnorm

# Batch normalization and batch size
We will now run a small experiment to study the interaction of batch normalization and batch size.

The first cell will train 6-layer networks both with and without batch normalization using different batch sizes. The second layer will plot training accuracy and validation set accuracy over time.

```python
def run_batchsize_experiments(normalization_mode):
    np.random.seed(231)
    # Try training a very deep net with batchnorm
    hidden_dims = [100, 100, 100, 100, 100]
    num_train = 1000
    small_data = {
      'X_train': data['X_train'][:num_train],
      'y_train': data['y_train'][:num_train],
      'X_val': data['X_val'],
      'y_val': data['y_val'],
    }
    n_epochs=10
    weight_scale = 2e-2
    batch_sizes = [5,10,50]
    lr = 10**(-3.5)
    solver_bsize = batch_sizes[0]

    print('No normalization: batch size = ',solver_bsize)
    model = FullyConnectedNet(hidden_dims, weight_scale=weight_scale, normalization=None)
    solver = Solver(model, small_data,
                    num_epochs=n_epochs, batch_size=solver_bsize,
                    update_rule='adam',
                    optim_config={
                      'learning_rate': lr,
                    },
                    verbose=False)
    solver.train()

    bn_solvers = []
    for i in range(len(batch_sizes)):
        b_size=batch_sizes[i]
        print('Normalization: batch size = ',b_size)
        bn_model = FullyConnectedNet(hidden_dims, weight_scale=weight_scale, normalization=normalization_mode)
        bn_solver = Solver(bn_model, small_data,
                        num_epochs=n_epochs, batch_size=b_size,
                        update_rule='adam',
                        optim_config={
                          'learning_rate': lr,
                        },
                        verbose=False)
        bn_solver.train()
        bn_solvers.append(bn_solver)

    return bn_solvers, solver, batch_sizes

batch_sizes = [5,10,50]
bn_solvers_bsize, solver_bsize, batch_sizes = run_batchsize_experiments('batchnorm')
```

    No normalization: batch size =  5
    Normalization: batch size =  5
    Normalization: batch size =  10
    Normalization: batch size =  50

```python
plt.subplot(2, 1, 1)
plot_training_history('Training accuracy (Batch Normalization)','Epoch', solver_bsize, bn_solvers_bsize, \
                      lambda x: x.train_acc_history, bl_marker='-^', bn_marker='-o', labels=batch_sizes)
plt.subplot(2, 1, 2)
plot_training_history('Validation accuracy (Batch Normalization)','Epoch', solver_bsize, bn_solvers_bsize, \
                      lambda x: x.val_acc_history, bl_marker='-^', bn_marker='-o', labels=batch_sizes)

plt.gcf().set_size_inches(15, 10)
plt.show()
```

## Inline Question 2:
Describe the results of this experiment. What does this imply about the relationship between batch normalization and batch size? Why is this relationship observed?

## Answer:
- generally, model with batchnorm and large batch size converges faster, and is easier to overfit the trainning data, compared with model without batchnorm
- when the batch size is small (e.g. 5), model without batchnorm could be better, that is, model with batchnorm and small batch size (e.g. 5) converges slower than model with the same batch size but without batchnorm
- within models with batchnorm, large batch size makes model converges faster but introduces sever overfitting

# Layer Normalization
Batch normalization has proved to be effective in making networks easier to train, but the dependency on batch size makes it less useful in complex networks which have a cap on the input batch size due to hardware limitations.

Several alternatives to batch normalization have been proposed to mitigate this problem; one such technique is Layer Normalization [2]. Instead of normalizing over the batch, we normalize over the features. In other words, when using **Layer Normalization, each feature vector corresponding to a single datapoint is normalized based on the sum of all terms within that feature vector**.

[2] [Ba, Jimmy Lei, Jamie Ryan Kiros, and Geoffrey E. Hinton. "Layer Normalization." stat 1050 (2016): 21.](https://arxiv.org/pdf/1607.06450.pdf)

## Inline Question 3:
Which of these data preprocessing steps is analogous to batch normalization, and which is analogous to layer normalization?

1. Scaling each image in the dataset, so that the RGB channels for each row of pixels within an image sums up to 1.
2. Scaling each image in the dataset, so that the RGB channels for all pixels within an image sums up to 1.
3. Subtracting the mean image of the dataset from each image in the dataset.
4. Setting all RGB values to either 0 or 1 depending on a given threshold.

## Answer:
- 3 is analogous to batch normalization
- 2 is analogous to layer normalization

# Layer Normalization: Implementation

Now you'll implement layer normalization. This step should be relatively straightforward, as conceptually the implementation is almost identical to that of batch normalization. One significant difference though is that for layer normalization, we do not keep track of the moving moments, and the testing phase is identical to the training phase, where **the mean and variance are directly calculated per datapoint**.

Here's what you need to do:

* In `cs231n/layers.py`, implement the forward pass for layer normalization in the function `layernorm_forward`.

Run the cell below to check your results.
* In `cs231n/layers.py`, implement the backward pass for layer normalization in the function `layernorm_backward`.

Run the second cell below to check your results.
* Modify `cs231n/classifiers/fc_net.py` to add layer normalization to the `FullyConnectedNet`. When the `normalization` flag is set to `"layernorm"` in the constructor, you should insert a layer normalization layer before each ReLU nonlinearity.

Run the third cell below to run the batch size experiment on layer normalization.

```python
# Check the training-time forward pass by checking means and variances
# of features both before and after layer normalization

# Simulate the forward pass for a two-layer network
np.random.seed(231)
N, D1, D2, D3 =4, 50, 60, 3
X = np.random.randn(N, D1)
W1 = np.random.randn(D1, D2)
W2 = np.random.randn(D2, D3)
a = np.maximum(0, X.dot(W1)).dot(W2)

print('Before layer normalization:')
print_mean_std(a,axis=1)

gamma = np.ones(D3)
beta = np.zeros(D3)
# Means should be close to zero and stds close to one
print('After layer normalization (gamma=1, beta=0)')
a_norm, _ = layernorm_forward(a, gamma, beta, {'mode': 'train'})
print_mean_std(a_norm,axis=1)

gamma = np.asarray([3.0,3.0,3.0])
beta = np.asarray([5.0,5.0,5.0])
# Now means should be close to beta and stds close to gamma
print('After layer normalization (gamma=', gamma, ', beta=', beta, ')')
a_norm, _ = layernorm_forward(a, gamma, beta, {'mode': 'train'})
print_mean_std(a_norm,axis=1)
```

    Before layer normalization:
      means:  [-59.06673243 -47.60782686 -43.31137368 -26.40991744]
      stds:   [10.07429373 28.39478981 35.28360729  4.01831507]

    After layer normalization (gamma=1, beta=0)
      means:  [ 4.81096644e-16 -7.40148683e-17  2.22044605e-16 -5.92118946e-16]
      stds:   [0.99999995 0.99999999 1.         0.99999969]

    After layer normalization (gamma= [3. 3. 3.] , beta= [5. 5. 5.] )
      means:  [5. 5. 5. 5.]
      stds:   [2.99999985 2.99999998 2.99999999 2.99999907]

```python
# Gradient check layer normalization backward pass
np.random.seed(231)
N, D = 4, 5
x = 5 * np.random.randn(N, D) + 12
gamma = np.random.randn(D)
beta = np.random.randn(D)
dout = np.random.randn(N, D)

ln_param = {}
fx = lambda x: layernorm_forward(x, gamma, beta, ln_param)[0]
fg = lambda a: layernorm_forward(x, a, beta, ln_param)[0]
fb = lambda b: layernorm_forward(x, gamma, b, ln_param)[0]

dx_num = eval_numerical_gradient_array(fx, x, dout)
da_num = eval_numerical_gradient_array(fg, gamma.copy(), dout)
db_num = eval_numerical_gradient_array(fb, beta.copy(), dout)

_, cache = layernorm_forward(x, gamma, beta, ln_param)
dx, dgamma, dbeta = layernorm_backward(dout, cache)

#You should expect to see relative errors between 1e-12 and 1e-8
print('dx error: ', rel_error(dx_num, dx))
print('dgamma error: ', rel_error(da_num, dgamma))
print('dbeta error: ', rel_error(db_num, dbeta))
```

    dx error:  2.107277492956569e-09
    dgamma error:  4.519489546032799e-12
    dbeta error:  2.5842537629899423e-12

# Layer Normalization and batch size

We will now run the previous batch size experiment with layer normalization instead of batch normalization. Compared to the previous experiment, **you should see a markedly smaller influence of batch size on the training history!**

```python
ln_solvers_bsize, solver_bsize, batch_sizes = run_batchsize_experiments('layernorm')

plt.subplot(2, 1, 1)
plot_training_history('Training accuracy (Layer Normalization)','Epoch', solver_bsize, ln_solvers_bsize, \
                      lambda x: x.train_acc_history, bl_marker='-^', bn_marker='-o', labels=batch_sizes)
plt.subplot(2, 1, 2)
plot_training_history('Validation accuracy (Layer Normalization)','Epoch', solver_bsize, ln_solvers_bsize, \
                      lambda x: x.val_acc_history, bl_marker='-^', bn_marker='-o', labels=batch_sizes)

plt.gcf().set_size_inches(15, 10)
plt.show()
```

## Inline Question 4:
When is layer normalization likely to not work well, and why?

1. Using it in a very deep network
2. Having a very small dimension of features
3. Having a high regularization term

## Answer:
2, a few features is not enough to estimated mean and variance


 INF285/ILI285 Computación Científica

 Tarea N°1, v1.00

## Instrucciones

Felipe Montero Concha 201473611-8

* La tarea es individual.
* Las consultas sobre las tareas se deben realizar por medio de la plataforma Aula.
* La tarea debe ser realizada en `Jupyter Notebook` (`Python3`).
* Se evaluará la correcta utilización de librerias `NumPy`, `SciPy`, entre otras, así como la correcta implementación de algoritmos de forma vectorizada.
*  **El archivo de entrega debe denominarse ROL-tarea-numero.ipynb**. _De no respetarse este formato existirá un descuento de **50 puntos**_
* La fecha de entrega es el jueves 30 de Abril a las **18:00 hrs**.  Se aceptarán entregas hasta las 19:00 hrs sin descuento en caso de existir algun problema, posteriormente existirá un descuento lineal hasta las 20:00 hrs del mismo día.
* Las tareas que sean entregadas antes del jueves a mediodía recibirán una bonificación de 10 puntos
* Debe citar cualquier código ajeno utilizado (incluso si proviene de los Jupyter Notebooks del curso).

## Introducción

En esta primera tarea de INF/ILI-285, versión 2020-1, estudiaremos la importancia de los primeros temas estudiados en el curso, los cuales son: Representación de Punto Flotante, Pérdida de Significancia, Errores de Cancelación y Búsqueda de Ceros. El desarrollo de cada uno de esos temas se presenta en una serie de preguntas, donde ustedes deben ir decidiendo, pregunta a pregunta, como cada uno de los temas se aplica. En general, los temas no se analizan por separado, sino de manera acoplada. Es muy importante que cada uno de los problemas sea analizado teóricamente primero, para luego poner su conocimiento en acción. Cada problema puede ser desarrollado de diversas formas, sin embargo, es muy importante determinar al final si el camino elegido resuelve la pregunta presentada.

## Problemas

```python
import numpy as np
import bitstring
import scipy.special
```

### 1. Simulador (50 ptos)

Dada la familia de polinomios de grado 3 con la forma:

\begin{equation}
f(x)=x^3 - A\,x^2 + A\,x - 1
\end{equation}

Se pide implementar un algoritmo que reciba como parámetros el valor de $A$ (con $|A|>>1$) y la cantidad de bits que tiene en la mantisa un computador ficticio, el manejo del exponente debe considerarse de las misma forma que lo maneja _double precision_.
Esta implementación debe calcular las raíces de $f$ sin perdida de significancia con la cantidad de bits disponibles para la mantisa.
Para obtener las raíces de $f$ usted debe encontrar de forma algebraica sus raíces y luego proponer un algoritmo basado en las _fórmulas_ obtenidas.

Considere que en ese computador ficticio cuenta con las operaciones matemáticas necesarias para obtener las raíces.  Considere el siguiente ejemplo:

```python
# Alg. Base
a = 9819824.624837
b = 148736.523476
c = a+b

# Alg. con Representación de Punto Flotante de 'bits_mant' bits en la mantisa.
am = f_new_rep(9819824.624837,bits_mant) # Aproximar el input en la nueva representación.
bm = f_new_rep(148736.523476,bits_mant) # Aproximar el input en la nueva representación.
cm = f_new_rep(m,exp,am+bm) # Aproximar el output de la suma y cada operación en la nueva representación.
```

```python
"""
input
x         : (double) valor a evaluar
bits_mant : (int) cantidad de bits de la mantisa
output
eval      : (double) resultado obtenido
"""
#Esta funcion lo unico que hace es sumar el 1 en caso de que sea necesario a un numero binario,
#acarreando el 1 cuando es necesario
def sumar_1_bin(num):
    a_sumar = str(num)[::-1]
    result = ""
    debe = True
    for bit in a_sumar:
        if bit == "1" and debe:
            result += "0"
        elif bit == "0" and debe:
            result += "1"
            debe = False
        else:
            result += bit
    return result[::-1]

def f_new_rep(x,bits_mant):
  # Algoritmo de representación de punto flotante modificada.
    rep_IEEE = bitstring.BitArray(float= x, length = 64)
    ceros_unos = rep_IEEE.bin

    signo = ceros_unos[0]
    exponente = ceros_unos[1:12]
    mantisa = ceros_unos[12:]

    new_mantisa = mantisa[0:bits_mant]
    check_mant = mantisa[bits_mant:53]

    if check_mant[0] == "0":
        new_mantisa = new_mantisa
    elif check_mant[0] == "1":
        if "1" in check_mant[1:]:
            new_mantisa = sumar_1_bin(new_mantisa)
        elif "1" not in check_mant[1:]:
            if new_mantisa[-1] == "1":
                new_mantisa = sumar_1_bin(new_mantisa)
    dec_exponente = int(exponente,2)-1023
    total = 2**dec_exponente
    k = dec_exponente - 1
    for i in range(0,bits_mant):
        total += int(new_mantisa[i])*(2**(k))
        k-=1
    if int(signo):
        total = total*-1
    return  total

print(f_new_rep(90,32))

```

```python
#Para todo polinomio cubico de esta caracteristica, el 1 siempre es raiz por lo tanto se procede a hacer ruffini en el codigo
# dividiendo el polinomio en 1, obteniendo los coeficientes de la cuadratica correspondiente para luego obtener a traves de
#la formula de soluciones cuadraticas el resto de las raices del polinomio cubico original.
def f_find_roots(A,bits_mant):

    coeficientes = [1,-A,A,-1]
    c_cuadraticos = [coeficientes[0]]
    for i in range(0,len(coeficientes)):
        if i == 0:
            a_sumar = coeficientes[0]*1
        else:
            c_cuadraticos.append(coeficientes[i]+a_sumar)
            a_sumar = coeficientes[i] + a_sumar
    print(c_cuadraticos)

    a = f_new_rep(c_cuadraticos[0],bits_mant)
    b = f_new_rep(c_cuadraticos[1],bits_mant)
    c = f_new_rep(c_cuadraticos[2],bits_mant)

    b_cuadrado = f_new_rep(b**2,bits_mant)
    dos_a = f_new_rep(2*a,bits_mant)
    ace = f_new_rep(4*a*c,bits_mant)

    resta_raiz = f_new_rep(b_cuadrado-ace,bits_mant)

    raiz = f_new_rep(np.sqrt(resta_raiz),bits_mant)

    sum1 = f_new_rep(-b + raiz , bits_mant)
    sum2 = f_new_rep(-b - raiz , bits_mant)

    x1 = f_new_rep(sum1/dos_a,bits_mant)
    x2 = f_new_rep(sum2/dos_a,bits_mant)

    x_roots = (1,x1,x2)
    return x_roots

f_find_roots(90,32)
```

### Polinomios de Legendre (50 puntos)
Dada la función compuesta $f$
\begin{equation}
f_{n,m}(x) = L_n(C_m(x)),
\end{equation}
donde $L_n$ es conocido como el polinomio de Legendre de grado $n$ definido de la siguiente forma:
\begin{equation}
L_{n}(x)=\frac{1}{2^{n}} \sum_{k=0}^{n}\left(\begin{array}{l}
n \\
k
\end{array}\right)^{2}(x-1)^{n-k}(x+1)^{k},
\end{equation}
y $C_m$ es el polinomio de Chebyshev
\begin{equation}
C_m(x) = \cos(m \cdot \arccos(x)).
\end{equation}

Utilizando el método de Bisección y Punto fijo se pide obtener la raíz de $f$ más cercana a $0.5$ dado un valor de $m$ y $n$

*Hint: Las raíces de Legendre son conocidas*

```python
#Por bisección se procede conociendo que el chebyshev de grado n se comporta de forma muy similar al legendre de grado n
# por lo tanto se le realiza una bisección en base a una raiz encontrada al chebyshev de grado n , la cual actua como
# x_0 para la función utilizada a comparar con el chebyshev de grado m
"""
input
n: (int) grado del polinomio de Legendre
m: (int) grado del polinomio de Chebyshev
tol: (double) tolerancia para la detención del algoritmo
output
root: (double) raiz obtenida
"""
def f_Biseccion(n, m, tol):
    i = 0
    menor_dif = float("inf")
    while i < n:
            valor_a_buscar = np.cos((np.pi*((2*i)-1))/(2*n))
            if abs(valor_a_buscar-0.5) < menor_dif:
                elegido = valor_a_buscar
                menor_dif = abs(valor_a_buscar-0.5)
            i+=1

    b = 1
    a = np.cos((np.pi/n)*(n-2))

    funcioncita = lambda x : elegido - np.cos(m*np.arccos(x))

    i = 0
    c = 0
    anterior = float("inf")

    while (abs(a-b) > tol) and (anterior != c):
        anterior = c
        c = (b+a)/2
        if funcioncita(c) == 0:
            return c
        else:
            if funcioncita(a)*funcioncita(c) < 0:
                b = c
            else:
                a = c
        i+=1

    x_m = np.cos(m*np.arccos(c))
    return x_m
"""
def f_FPI(n, m, tol):
    x =  x_0
    iteraciones=[]
    errores = []
    i = 0
    while abs(x_calc-g(x)) < tol:
        try:
            x_calc = g(x)
        except OverflowError:
            return None
        if x_calc == x and (x_calc not in iteraciones):
            iteraciones.append(x_calc)
        elif x_calc == x and (x_calc in iteraciones):
            i = n_iter
        else:
            iteraciones.append(x_calc)
        x = x_calc
        i+=1

    return root
"""
f_Biseccion(4,2,1e-8)
```

###  Determinantes (20 puntos)

Dada una matriz de dimensiones $ n \times n$ de la forma:
\begin{equation}
A
=
\begin{pmatrix}
a_{1,1} & a_{1,2}  & \dots & a_{1,n} \\
a_{2,1} & a_{2,2}  & \dots & a_{2,n} \\
\vdots &  \vdots   & \ddots & \vdots \\
a_{n,1} & a_{n,2}  & \dots & a_{n,n}
\end{pmatrix}
=
\begin{pmatrix}
\mathbf{r}_1 \\
\mathbf{r}_2 \\
\vdots \\
\mathbf{r}_n \\
\end{pmatrix},
\end{equation}
donde $\mathbf{r}_k$ representa la $k$-ésima fila de la matriz $A$.
Considere la siguiente matriz $C_{i,j}(\alpha)$,
\begin{equation}
C_{i,j}(\alpha)
=
\begin{pmatrix}
\mathbf{r}_1 \\
\vdots \\
\mathbf{r}_i\,(1-\alpha)+\mathbf{r}_j\,\alpha \\
\vdots \\
\mathbf{r}_j\,(1-\alpha)+\mathbf{r}_i\,\alpha \\
\vdots \\
\mathbf{r}_n \\
\end{pmatrix},
\end{equation}
de lo cual sabemos que $C_{i,j}(0)=A$ y que $C_{i,j}(1)$ es la matriz $A$ donde se intercambiaron las filas $i$ y $j$, es decir:
\begin{equation}
C_{i,j}(1)
=
\begin{pmatrix}
\mathbf{r}_1 \\
\vdots \\
\mathbf{r}_j \\
\vdots \\
\mathbf{r}_i \\
\vdots \\
\mathbf{r}_n \\
\end{pmatrix}.
\end{equation}
De las relaciones anteriores podemos concluir que el determinante de la matriz $A$, denominado $D=\det(A)$, es igual al determinante de $C_{i,j}(0)$, es decir $\det(C_{i,j}(0))=\det(A)=D$.
Por el otro lado, el determinante de $C_{i,j}(1)$ es $-D$, dado que es el intercambio de las filas $i$ y $j$ de la matriz $A$.
Por lo cual podemos concluir que $-D\leq \det(C_{i,j}(\alpha))\leq D$.

Utilizando el método de Bisección debe encontrar el valor de $\alpha$ con $p$ decimales de precisión que permitan que, dado los índices de las filas $i$, $j$, y $i\neq j$, el determinante de la matriz sea igual a $d$, donde $d\in[-D,D]$.

Para esto se debe implementar una función que reciba la matriz $A$, las filas $i$ y $j$, y $p$; y retorne $\widehat{\alpha}$ tal que $\det(C_{i,j}(\widehat{\alpha}))=d$.

```python
#Aca se determino que hay que calcular la biusección a parti de la función d - det(M(alpha))
# el intervalo de itaración de alpha es entre 0 y 1 ya que no puede salir de esos valores, considerando que
#los valores alpha y 1-alpha estan relacionados.
"""
input
A: (array n x n) matriz
i: (int) índice de la fila "i".
j: (int) índice de la fila "j".
p: (int) cantidad de decimales de precision
d: (double) valor requerido del determinante de $C_{i,j}(\alpha)$, i.e. $\det(C_{i,j}(\widehat{\alpha}))=d$.
output
alpha_hat: (double) alpha_hat tal que det(C_{i,j}(alpha_hat))=d.
"""
def generate_c(A,i,j,alpha):
    a_aux = np.array(A)
    fila_i = np.array(A[i])
    fila_j = np.array(A[j])
    aux = 0
    filas = []
    for x in range(0,len(A)):
        if x == i:
            aux = fila_i*alpha + fila_j*(1.-alpha)
            filas.append(aux)
        elif x == j:
            aux = fila_i*alpha + fila_j*(1.-alpha)
            filas.append(aux)
        else:
            filas.append(a_aux[x])
    print("este es el alfa de la funcion"+str(alpha))
    retorno = np.array(filas)
    print(retorno)
    return retorno

def find_alpha_hat(A,i,j,p,d):
    tolerancia =10**(-p-1)

    funcioncita = lambda C,i,j,alpha_i : d - np.linalg.det(generate_c(C,i,j,alpha_i))
    a = 0
    b = 1
    anterior = float("inf")
    print(generate_c(A,i,j,0.75))

    while (abs(a-b) > tolerancia) :
        c = (b+a)/2
        if funcioncita(A,i,j,c) == 0:
            alpha_hat = c
            return alpha_hat
        else:
            if funcioncita(A,i,j,a)*funcioncita(A,i,j,c) < 0:
                b = c
            else:
                a = c
    alpha_hat = c
    print(np.linalg.det(generate_c(A,i,j,alpha_hat)))
    return alpha_hat

A = np.array([[19,12,33,9],[3,4,6,4],[7,8,9,2],[5,9,6,1]])
print(np.linalg.det(A))
print(find_alpha_hat(A,0,2,7,4))
```

# Referencias

Si corresponde

```python

```


# Circuitos de 2ª Ordem
Jupyter Notebook desenvolvido por [Gustavo S.S.](https://github.com/GSimas)

> "Todos os que podem cursar um mestrado em engenharia devem fazê-lo a fim de estender o sucesso de sua
carreira! Se você quer trabalhar com pesquisa, o estado da arte em engenharia, lecionar em uma universidade
ou iniciar seu próprio negócio, você realmente precisa cursar um doutorado!" - **Charles K. Alexander**

Circuitos contendo dois elementos de armazenamento,
que são conhecidos como circuitos de segunda ordem, porque
suas respostas são descritas como equações diferenciais contendo derivadas
segundas.

Exemplos comuns de circuitos de segunda ordem são os RLC, onde estão
presentes os três tipos de elementos passivos, como mostram Figuras 8.1a e b.
Outros exemplos são circuitos RL e RC, como os indicados nas Figuras 8.1c e
d.

Um circuito com amplificadores operacionais com dois elementos de armazenamento
também pode ser um circuito de segunda ordem. Assim como nos
circuitos de primeira ordem, o de segunda ordem pode conter vários resistores
e fontes dependentes e independentes.

**Um circuito de segunda ordem é caracterizado por uma equação diferencial
de segunda ordem. Ele é formado por resistores e o equivalente de dois
elementos de armazenamento.**

## Determinação dos valores inicial e final

Para a análise de circuitos de 2ª Ordem é preciso obter v(0), i(0), dv(0)/dt, di(0)/dt, i(∞) e v(∞), onde v é a tensão no capacitor e i a corrente no indutor.
**A tensão no capacitor é sempre contínua
de modo que**

\begin{align}
{\Large v(0^+) = v(0^-) = v(0)}
\end{align}

**E a corrente no indutor é sempre contínua de modo que**

\begin{align}
{\Large i(0^+) = i(0^-) = i(0)}
\end{align}

onde t = 0– representa o instante imediatamente anterior ao evento de comutação
e t = 0+ é o instante imediatamente após o evento de comutação, supondo
que esse evento ocorra em t = 0.

**Exemplo 8.1**

A chave na Figura 8.2 foi fechada há um bom tempo. Ela é aberta em t = 0. Determine:

a) v(0+), i(0+)

b) dv(0+)/dt, di(0+)/dt

c) i(∞) e v(∞)

```python
print("Exemplo 8.1")

L = 0.25
C = 0.1
Vs = 12

#para t < 0
i0 = Vs/(4 + 2)
v0 = i0*2
print("Corrente i(0+):",i0,"A")
print("Tensão v(0+):",v0,"V")

#para t = 0+
#vl = L*di/dt
#di/dt = vl/L
vl = Vs - i0*4 - v0
di = vl/L
#ic = C*dv/dt
#dv/dt = ic/C
dv = i0/C
print("Taxa di/dt:",di,"A/s")
print("Taxa dv/dt:",dv,"V/s")

#para t = infinito
v = Vs
i = 0
print("Para t infinito, v:",v,"V")
print("Para t infinito, i:",i,"A")
```

    Exemplo 8.1
    Corrente i(0+): 2.0 A
    Tensão v(0+): 4.0 V
    Taxa di/dt: 0.0 A/s
    Taxa dv/dt: 20.0 V/s
    Para t infinito, v: 12 V
    Para t infinito, i: 0 A

**Problema Prático 8.1**

A chave na Figura 8.4 foi aberta há um bom tempo, entretanto, foi fechada em t = 0.

a) v(0+), i(0+)

b) dv(0+)/dt, di(0+)/dt

c) i(∞) e v(∞)

```python
print("Problema Prático 8.1")

L = 0.4
C = 1/20
Vs = 24

#para t < 0
i0 = Vs/(10 + 2)
v0 = i0*2
print("Corrente i(0+):",i0,"A")
print("Tensão v(0+):",v0,"V")

#para t = 0+
#di/dt = vl/L
vl = Vs - v0
di = vl/L
#dv/dt = ic/C
dv = 0
print("Taxa di/dt:",di,"A/s")
print("Taxa dv/dt:",dv,"V/s")

#para t = infinito
i = Vs/2
v = i*2
print("Corrente i(∞)",i,"A")
print("Tensão v(∞)",v,"V")
```

    Problema Prático 8.1
    Corrente i(0+): 2.0 A
    Tensão v(0+): 4.0 V
    Taxa di/dt: 50.0 A/s
    Taxa dv/dt: 0 V/s
    Corrente i(∞) 12.0 A
    Tensão v(∞) 24.0 V

**Exemplo 8.2**

No circuito da Figura 8.5, calcule:

(a) iL(0+), vC(0+); vR(0+);

(b) diL(0+)/dt, dvC(0+)/dt, dvR(0+)/dt;

(c) iL(∞), vC(∞), vR(∞).

```python
print("Exemplo 8.2")

L = 0.6
C = 1/2
Vs = 20
Cs = 3

#para t < 0
v0 = -Vs
i0 = 0
vr0 = Cs*4/(2 + 4) * 2
print("Corrente i(0+):",i0,"A")
print("Tensão v(0+):",v0,"V")
print("Tensão Vr(0+):",vr0,"V")

#para t = 0+
#di/dt = vl/L
vl = 0
di = vl/L
#dv/dt = ic/C
ic = Cs*2/(2 + 4)
dv = ic/C
#3 = vr/2 + v0/4
#0 = 2dvr/dt + dv0/dt
#-vr + v0 + vc + 20 = 0
#-dvr + dv0 + 2 = 0 => dv0 = dvr - 2
dvr = 2/3
print("Taxa di/dt:",di,"A/s")
print("Taxa dv/dt:",dv,"V/s")
print("Taxa dvr/dt:",dvr,"V/s")

#para t = ∞
i = Cs*2/(4 + 2)
v = -Vs
vr = (Cs - i)*2
print("Tensão i(∞):",i,"A")
print("Corrente v(∞):",v,"V")
print("Tensão vr(∞):",vr,"V")
```

    Exemplo 8.2
    Corrente i(0+): 0 A
    Tensão v(0+): -20 V
    Tensão Vr(0+): 4.0 V
    Taxa di/dt: 0.0 A/s
    Taxa dv/dt: 2.0 V/s
    Taxa dvr/dt: 0.6666666666666666 V/s
    Tensão i(∞): 1.0 A
    Corrente v(∞): -20 V
    Tensão vr(∞): 4.0 V

**Problema Prático 8.2**

Para o circuito da Figura 8.7, determine:

(a) iL(0+), vC(0+); vR(0+);

(b) diL(0+)/dt, dvC(0+)/dt, dvR(0+)/dt;

(c) iL(∞), vC(∞), vR(∞).

```python
print("Problema Prático 8.2")

C = 1/5
L = 2
Cs1 = 6
Cs2 = 4

#para t < 0
i0 = -Cs1
v0 = 0
vr0 = 0
print("Corrente i(0+):",i0,"A")
print("Tensão v(0+):",v0,"V")
print("Tensão vr(0+):",vr0,"V")

#para t = 0+
#di/dt = vl/L
vl = 0
di = vl/L
#dv/dt = ic/C
ic = 4
dv = ic/C
#vr = vc - vl = 0
#dvr = 20 - dvl
#6 = vr/5 + il
#0 = dvr + 5di
#0 = dvr
dvr = 0
print("Taxa di/dt:",di,"A/s")
print("Taxa dv/dt:",dv,"V/s")
print("Taxa dvr/dt:",dvr,"V/s")

#para t = ∞
i = Cs2 - Cs1
vr = Cs2*5
vc = vr
print("Corrente i(∞):",i,"A")
print("Tensão v(∞):",v,"V")
print("Tensão vr(∞):",vr,"V")
```

    Problema Prático 8.2
    Corrente i(0+): -6 A
    Tensão v(0+): 0 V
    Tensão vr(0+): 0 V
    Taxa di/dt: 0.0 A/s
    Taxa dv/dt: 20.0 V/s
    Taxa dvr/dt: 0 V/s
    Corrente i(∞): -2 A
    Tensão v(∞): -20 V
    Tensão vr(∞): 20 V

# Circuito RLC Série sem fonte

O circuito
é excitado pela energia inicialmente armazenada no capacitor e indutor, representada
pela tensão inicial V0 no capacitor e pela corrente inicial I0 no indutor.
Portanto, em t = 0.

\begin{align}
{\Large v(0) = \frac{1}{C} \int_{- \infty}^{0} idt = V_0}
\end{align}

\begin{align}
{\Large i(0) = I_0}
\end{align}

Aplicando a LKT no circuito da Figura 8.8:

\begin{align}
{\Large Ri + L \frac{di}{dt} + \frac{1}{C} \int_{- \infty}^{0} i(\tau)d \tau = 0}
\end{align}

Para eliminar a integral, diferenciamos em relação a t e reorganizamos os termos:

\begin{align}
{\Large \frac{d^2i}{dt^2} + \frac{R}{L} \frac{di}{dt} + \frac{i}{LC} = 0}
\end{align}

Esta é uma equação diferencial de segunda ordem e é o motivo para os circuitos
RLC neste capítulo serem chamados circuitos de segunda ordem. Obtemos o valor inicial da
derivada de i a partir de:

\begin{align}
{\Large Ri(0) + L \frac{di(0)}{dt} + V_0 = 0}
\end{align}

Nossa experiência no capítulo anterior sobre circuitos de primeira ordem nos sugere que a solução é na forma exponencial.
Portanto, façamos:

\begin{align}
{\Large i = Ae^{st}}
\end{align}

onde A e s são constantes a serem determinadas. Substituindo nas equações anteriores:

\begin{align}
{\Large As^2e^{st} + \frac{AR}{L}se^{st} + \frac{A}{LC}e^{st} = 0}
\end{align}

ou:

\begin{align}
{\Large Ae^{st} (s^2 + \frac{R}{L}s + \frac{1}{LC}) = 0}
\end{align}

Já que i = Ae^(st) é a solução pressuposta de que estamos tentando encontrar,
apenas a expressão entre parênteses pode ser zero:

\begin{align}
{\Large s^2 + \frac{R}{L}s + \frac{1}{LC} = 0}
end{align}

Esta equação quadrática é conhecida como **equação característica da Equação
diferencial** uma vez que as raízes da equação ditam as características
básicas de i. As duas raízes são:

\begin{align}
{\Large s_1 = -\frac{R}{2L} + \sqrt{ (\frac{R}{2L})^2 - \frac{1}{LC}}}
\\{\Large s_2 = -\frac{R}{2L} - \sqrt{ (\frac{R}{2L})^2 - \frac{1}{LC}}}
\end{align}

**Uma forma mais condensada de expressar as raízes é**:

\begin{align}
{\Large s_1 = -\alpha + \sqrt{ \alpha^2 - \omega_0^2 }}
\\{\Large s_2 = -\alpha - \sqrt{ \alpha^2 - \omega_0^2 }}
\end{align}

**onde**

\begin{align}
{\Large \alpha = \frac{R}{2L} }
\\{\Large \omega_0 = \frac{1}{ \sqrt{LC} }}
\end{align}

As raízes s1 e s2 são chamadas **frequências naturais**, medidas em nepers
por segundo (Np/s), pois estão associadas à resposta natural do circuito; v0 é
conhecida como **frequência ressonante** ou estritamente como a **frequência natural
não amortecida** expressa em radianos por segundo (rad/s); e a é a frequência
de neper ou fator de amortecimento expresso em nepers por segundo.

Assim, pode-se escrever:

\begin{align}
{\Large s^2 + 2 \alpha s + \omega_0^2 = 0}
\end{align}

A razão α/ω0 é conhecida como fator de amortecimento, ζ.

## Resposta do circuito RLC série:

Consequentemente, a resposta natural do circuito RLC em série é:

\begin{align}
{\Large i(t) = A_1 e^{s_1 t} + A_2 e^{s_2 t}}
\end{align}

onde as constantes A1 e A2 são determinadas a partir dos valores iniciais i(0) e
di(0)/dt.

Assim, podemos inferir que existem três tipos de soluções:

1. Se α > ω0 temos o caso de **amortecimento supercrítico**
2. Se α = ω0 temos o caso de **amortecimento crítico**
3. Se α < ω0 temos o caso de **subamortecimento**

## Amortecimento Supercrítico (α > ω0)

Se α > ω0 implica C > 4L/R^2. Quando isso acontece, ambas as raízes s1 e s2 são negativas e reais. A resposta é:

\begin{align}
{\Large i(t) = A_1 e^{s_1 t} + A_2 e^{s_2 t}}
\end{align}

que decai e se aproxima de zero à medida que t aumenta.

## Amortecimento Crítico (α = ω0)

Quando α = ω0, C = 4L/R^2 e:

\begin{align}
{\Large s_1 = s_2 = -\alpha = -\frac{R}{2L}}
\end{align}

Assim:

\begin{align}
{\Large i(t) = A_1 e^{-\alpha t} + A_2 e^{-\alpha t} = A_3 e^{-\alpha t}}
\end{align}

Isso não pode ser a solução, pois as duas condições iniciais não podem ser satisfeitas com a constante única A3. Assim, de acordo com a resolucao de equações diferenciais de segunda ordem, resposta natural de um circuito com amortecimento
crítico é a soma de dois termos: exponencial negativa e exponencial
negativa multiplicada por um termo linear:

\begin{align}
{\Large i(t) = (A_2 + A_1 t)e^{-\alpha t}}
\end{align}

essa figura é um esboço de i(t) = te^(–αt), que
atinge um valor máximo igual a e^(–1)/α em t = 1/α, uma constante de tempo, e
então decresce até chegar a zero.

## Subamortecimento

Para α < ω0, temos C < 4L/R^2. As raízes podem ser escritas como:

\begin{align}
{\Large s_1 = -\alpha + \sqrt{ \alpha^2 - \omega_0^2 } = -\alpha + j\omega_d}
\\{\Large s_2 = -\alpha - \sqrt{ \alpha^2 - \omega_0^2 } = -\alpha - j\omega_d}
\end{align}

onde **j = √-1 e ωd = √(ω0^2 - α^2)** que é chamada **frequência de amortecimento**.
Tanto v0 como vd são frequências naturais, porque ajudam a determinar
a resposta natural; enquanto ω0 é muitas vezes denominada frequência
natural não amortecida, ωd é chamada frequência natural amortecida. A resposta
natural é:

\begin{align}
{\Large i(t) = e^{-\alpha t} (B_1 cos(\omega_d t) + B_2 sen(\omega_d t))}
\end{align}

onde B1 = A1 + A2 e B2 = j(A1 - A2)

## Considerações

R = 0 produz uma resposta
**perfeitamente senoidal**. Essa
resposta não pode ser concretizada
na prática com L e C em virtude
das perdas inerentes nesses
elementos. Um dispositivo eletrônico chamado
**oscilador** é capaz de produzir uma
resposta perfeitamente senoidal.

Concluímos esta seção enfatizando as seguintes propriedades interessantes
e peculiares de um circuito RLC:

1. O comportamento de um circuito destes pode ser compreendido pelo conceito de **amortecimento**, que é a **perda gradual da energia inicial armazenada**. Se R = 0 diz-se que o circuito está sem perdas, pois o elemento amortecedor ou dissipador (R) não está presente. Ajustando-se o valor de R, a resposta pode ser não amortecida, com amortecimento supercrítico, com amortecimento crítico ou então subamortecida.

2. A **resposta oscilatória** é possível em razão da presença de dois tipos de elementos de armazenamento. Ter tanto L como C possibilita que o fluxo de energia fique indo e vindo entre eles. A oscilação amortecida, exibida pela resposta subamortecida, é conhecida como **oscilação circular**.

3. O caso com amortecimento crítico é a fronteira entre os casos de subamortecimento e de amortecimento supercrítico e ela cai de forma mais rápida. Com as mesmas condições iniciais, o caso de amortecimento **supercrítico tem o tempo de acomodação mais longo**, pois ele leva o maior tempo para dissipar a energia inicialmente armazenada. Se desejarmos uma resposta que se aproxime do valor final mais **rapidamente sem oscilação ou com oscilação circular**, o circuito com **amortecimento crítico** é o mais indicado.

**Exemplo 8.3**

Na Figura 8.8, R = 40 Ω , L = 4 H e C = 1/4 F. Calcule as raízes características do circuito.
A resposta natural é com amortecimento supercrítico, com subamortecimento ou
com amortecimento crítico?

```python
print("Exemplo 8.3")
R = 40
L = 4
C = 1/4

def sqrt(x):
    r = x**(1/2)
    return r

alpha = R/(2*L)
omega = 1/(sqrt(L*C))
s1 = -alpha + sqrt(alpha**2 - omega**2)
s2 = -alpha - sqrt(alpha**2 - omega**2)

resposta = ""

if alpha > omega:
    resposta = "superamortecimento"
elif alpha == omega:
    resposta = "amortecimento crítico"
else:
    resposta = "subamortecimento"

print("Raiz s1:",s1)
print("Raiz s2:",s2)
print("Resposta:",resposta)
```

    Exemplo 8.3
    Raiz s1: -0.10102051443364424
    Raiz s2: -9.898979485566356
    Resposta: superamortecimento

**Problema Prático 8.3**

Se R = 10, L = 5 H e C = 2 mF na Figura 8.8, determine alpha, omega0, s1 e s2. Qual é o tipo
de resposta natural que o circuito apresentará?

```python
print("Problema Prático 8.3")

m = 10**(-3) ##definicao de mili

R = 10
L = 5
C = 2*m

alpha = R/(2*L)
omega = 1/(sqrt(L*C))
s1 = -alpha + sqrt(alpha**2 - omega**2)
s2 = -alpha - sqrt(alpha**2 - omega**2)

resposta = ""

if alpha > omega:
    resposta = "superamortecimento"
elif alpha == omega:
    resposta = "amortecimento crítico"
else:
    resposta = "subamortecimento"

print("Alpha:",alpha)
print("Omega:",omega)
print("Raiz s1:",s1)
print("Raiz s2:",s2)
print("Resposta:",resposta)
```

    Problema Prático 8.3
    Alpha: 1.0
    Omega: 10.0
    Raiz s1: (-0.9999999999999994+9.9498743710662j)
    Raiz s2: (-1.0000000000000007-9.9498743710662j)
    Resposta: subamortecimento

**Exemplo 8.4**

Determine i(t) no circuito da Figura 8.10. Suponha que o circuito tenha atingido o estado
estável em t = 0–.

```python
print("Exemplo 8.4")

from sympy import *

t = symbols('t')
A2 = symbols('A2')

C = 20*m
L = 0.5
Vs = 10

#tensao e corrente iniciais
v0 = Vs*6/(6 + 4)
i0 = v0/6

#t > 0
R = 3 + 6
alpha = R/(2*L)
omega = 1/(sqrt(L*C))

s1 = -alpha + sqrt(alpha**2 - omega**2)
s2 = -alpha - sqrt(alpha**2 - omega**2)

def resposta_rlc(alpha, omega): #funcao para verificar tipo de resposta
    resposta = ""
    if alpha > omega:
        resposta = "superamortecimento"
    elif alpha == omega:
        resposta = "amortecimento crítico"
    else:
        resposta = "subamortecimento"
    return resposta

resposta = resposta_rlc(alpha,omega)

omegad = sqrt(omega**2 - alpha**2)

#Tipo de resposta:
#alpha < omega -> subamortecido -> i(t) = e^(-alpha t)*(A1*cos(wd*t) + A2*sen(wd*t))
#i(0) = 1 -> A1 = 1

print("Tensao inicial capacitor:",v0)
print("Corrente inicial indutor:", i0)
print("Alpha:",alpha)
print("Omega:",omega)
print("Raiz s1:",s1)
print("Raiz s2:",s2)
print("Resposta:",resposta)

#resposta completa
r = exp(-alpha*t)*(cos(omegad*t) + A2*sin(omegad*t))
#di/dt = -1/L * [Ri(0) + v(0)] = -6 A/s
#porem
r2 = r.diff(t)
print("i(0):",r2.subs(t,0))
#assim
A2 = (-6 + 9)/4.35
print("Constante A2:",A2)
r = exp(-alpha*t)*(cos(omegad*t) + A2*sin(omegad*t))
print("Resposta completa:",r)
```

    Exemplo 8.4
    Tensao inicial capacitor: 6.0
    Corrente inicial indutor: 1.0
    Alpha: 9.0
    Omega: 10.0000000000000
    Raiz s1: -9.0 + 4.35889894354067*I
    Raiz s2: -9.0 - 4.35889894354067*I
    Resposta: subamortecimento
    i(0): 4.35889894354067*A2 - 9.0
    Constante A2: 0.6896551724137931
    Resposta completa: (0.689655172413793*sin(4.35889894354067*t) + cos(4.35889894354067*t))*exp(-9.0*t)

**Problema Prático 8.4**

O circuito da Figura 8.12 atingiu o estado estável em t = 0–. Se o interruptor muda para
a posição b em t = 0, calcule i(t) para t > 0.

```python
print("Problema Prático 8.4")

A2 = symbols('A2')

L = 1
C = 1/9
Vs = 100

#para t < 0
i0 = Vs/10
v0 = 0

#para t > 0
R = 5
alpha = R/(2*L)
omega = 1/(sqrt(L*C))
resposta = resposta_rlc(alpha,omega)
omegad = sqrt(omega**2 - alpha**2)

#Encontrar constantes A1 e A2

#i(0) = Vs/10 = 10 A
#i(t) = exp(-alpha*t)*(A1*cos(omegad*t) + A2*sin(omegad*t))
#i(0) = A1
A1 = 10
#di/dt = -1/L * [Ri(0) + v(0)]
di = -1/L * (R*i0 + v0)
print("di/dt:",di) #di = -50
r = exp(-alpha*t)*(A1*cos(omegad*t) + A2*sin(omegad*t))
r2 = r.diff(t)
print("i(0):",r2.subs(t,0)) #1.65831*A2 - 25
A2 = (di + 25)/1.65831
print("Constante A2:",A2)

#Resposta completa
r = exp(-alpha*t)*(A1*cos(omegad*t) + A2*sin(omegad*t))

print("Alpha:",alpha)
print("Omega0:",omega)
print("Resposta:",resposta)
print("i(t):",r)
```

    Problema Prático 8.4
    di/dt: -50.0
    i(0): 1.6583123951777*A2 - 25.0
    Constante A2: -15.075589003262358
    Alpha: 2.5
    Omega0: 3.00000000000000
    Resposta: subamortecimento
    i(t): (-15.0755890032624*sin(1.6583123951777*t) + 10*cos(1.6583123951777*t))*exp(-2.5*t)

Aula 13 - Circuitos de Segunda Ordem.ipynb


```python
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
```

```python
df1 = pd.read_csv('data/data.csv')
df1
```

      day
      Humidity(%)
      Tmax(oC)
      Tmin(oc)
      Rain(mm)

      0
      1
      94.7
      13.0
      5.6
      0.0

      1
      2
      100.0
      16.4
      2.5
      9.9

      2
      3
      98.4
      22.0
      2.3
      0.3

      3
      4
      98.4
      19.5
      1.0
      0.0

      4
      5
      96.8
      20.0
      1.4
      0.0

      ...
      ...
      ...
      ...
      ...
      ...

      355
      356
      95.4
      20.5
      2.7
      0.0

      356
      357
      94.2
      20.5
      3.0
      0.0

      357
      358
      95.6
      20.0
      3.7
      0.0

      358
      359
      91.1
      19.2
      2.2
      0.0

      359
      360
      96.9
      19.1
      1.4
      0.0

360 rows × 5 columns

```python
df1 = df1.set_index('day')
df1
```

      Humidity(%)
      Tmax(oC)
      Tmin(oc)
      Rain(mm)

      day

      1
      94.7
      13.0
      5.6
      0.0

      2
      100.0
      16.4
      2.5
      9.9

      3
      98.4
      22.0
      2.3
      0.3

      4
      98.4
      19.5
      1.0
      0.0

      5
      96.8
      20.0
      1.4
      0.0

      ...
      ...
      ...
      ...
      ...

      356
      95.4
      20.5
      2.7
      0.0

      357
      94.2
      20.5
      3.0
      0.0

      358
      95.6
      20.0
      3.7
      0.0

      359
      91.1
      19.2
      2.2
      0.0

      360
      96.9
      19.1
      1.4
      0.0

360 rows × 4 columns

```python
df1['temp(oC)'] = (df1['Tmax(oC)']+df1['Tmin(oc)'])/2
df1
```

      Humidity(%)
      Tmax(oC)
      Tmin(oc)
      Rain(mm)
      temp(oC)

      day

      1
      94.7
      13.0
      5.6
      0.0
      9.30

      2
      100.0
      16.4
      2.5
      9.9
      9.45

      3
      98.4
      22.0
      2.3
      0.3
      12.15

      4
      98.4
      19.5
      1.0
      0.0
      10.25

      5
      96.8
      20.0
      1.4
      0.0
      10.70

      ...
      ...
      ...
      ...
      ...
      ...

      356
      95.4
      20.5
      2.7
      0.0
      11.60

      357
      94.2
      20.5
      3.0
      0.0
      11.75

      358
      95.6
      20.0
      3.7
      0.0
      11.85

      359
      91.1
      19.2
      2.2
      0.0
      10.70

      360
      96.9
      19.1
      1.4
      0.0
      10.25

360 rows × 5 columns

```python
df1.iloc[1,4]
```

    9.45

```python
df1.describe().T
```

      count
      mean
      std
      min
      25%
      50%
      75%
      max

      Humidity(%)
      360.0
      88.215556
      8.913733
      52.1
      82.775
      89.900
      95.225
      100.00

      Tmax(oC)
      360.0
      25.894722
      4.120923
      11.2
      22.650
      27.150
      29.225
      33.10

      Tmin(oc)
      360.0
      12.822222
      6.513911
      -1.3
      7.500
      13.400
      19.500
      21.50

      Rain(mm)
      360.0
      4.068611
      9.943038
      0.0
      0.000
      0.000
      1.600
      75.00

      temp(oC)
      360.0
      19.358472
      5.067583
      9.1
      14.775
      20.725
      23.825
      26.55

```python
plt.figure(figsize = [8,6])
plt.plot(df1.index,df1['Humidity(%)'])
plt.xlabel('Day no.')
plt.ylabel('Humidity(%)')
plt.title('Daily variation of Humidity')
plt.show()
```

```python
plt.figure(figsize = [8,6])
plt.plot(df1.index,df1['Tmax(oC)'],label='Max. temp')
plt.plot(df1.index,df1['Tmin(oc)'],label='Min. temp')
plt.xlabel('Day no.')
plt.ylabel('Temperature($^o$C)')
plt.title('Daily variation of  temperature')
plt.legend()
plt.show()
```

```python
day=[0,30,60,90,120,150,180,210,240,270,300,330,360]
mtemp=[]
mRH=[]
mrain=[]
for i in range(12):
    mRH.append(np.sum(df1.iloc[day[i]+1:day[i+1],0])/30)
    mtemp.append(np.sum(df1.iloc[day[i]+1:day[i+1],4])/30)
    mrain.append(np.sum(df1.iloc[day[i]+1:day[i+1],3]))
```

```python
m = np.arange(1,13)
month=['Jan','Feb','March','April','May','June','July','Aug','Sept','Oct','Nov','Dec']
plt.bar(m,mtemp,color='cyan',edgecolor='black')
plt.xticks(m, month )
plt.xlabel('Month', fontsize=14)
plt.ylabel('Temperature($^o$C)', fontsize=14)
plt.tick_params(axis='x', rotation= 45)
plt.title('Monthly variation of temperature', fontsize = 18 ,color='green')
plt.tight_layout()
plt.show()
```

```python
plt.bar(m,mRH,color='green',edgecolor='black')
plt.xticks(m, month )
plt.xlabel('Month', fontsize=14)
plt.ylabel('Humidity(%)', fontsize=14)
plt.tick_params(axis='x', rotation= 45)
plt.title('Monthly variation of Humidity', fontsize = 18 ,color='green')
plt.tight_layout()
plt.show()
```

```python
plt.bar(m,mrain,color='yellow',edgecolor='black')
plt.xticks(m, month )
plt.xlabel('Month', fontsize=14)
plt.ylabel('Rainfall(mm)', fontsize=14)
plt.tick_params(axis='x', rotation= 45)
plt.title('Monthly variation of rainfall', fontsize = 18 ,color='green')
plt.tight_layout()
plt.show()
```

\begin{equation}
dew~point=\frac{b(aT/(b+T)+ln(RH))}{a-(aT/(b+T)+ln(RH)}
\end{equation}

```python
def DewPoin(t,rh):
    a=17.27
    b=237.7
    x=a*t/(b+t)
    nm=b*(x+np.log(rh/100))
    dm=a-(x+np.log(rh/100))
    y=nm/dm
    return y
```

```python
dt=DewPoin(26,78)
dt
```

    21.85732599954956

```python
df1['dewpoint(oC)']=DewPoin(df1['temp(oC)'],df1['Humidity(%)'])
df1
```

      Humidity(%)
      Tmax(oC)
      Tmin(oc)
      Rain(mm)
      temp(oC)
      dewpoint(oC)

      day

      1
      94.7
      13.0
      5.6
      0.0
      9.30
      8.493324

      2
      100.0
      16.4
      2.5
      9.9
      9.45
      9.450000

      3
      98.4
      22.0
      2.3
      0.3
      12.15
      11.904965

      4
      98.4
      19.5
      1.0
      0.0
      10.25
      10.008675

      5
      96.8
      20.0
      1.4
      0.0
      10.70
      10.212111

      ...
      ...
      ...
      ...
      ...
      ...
      ...

      356
      95.4
      20.5
      2.7
      0.0
      11.60
      10.889071

      357
      94.2
      20.5
      3.0
      0.0
      11.75
      10.847578

      358
      95.6
      20.0
      3.7
      0.0
      11.85
      11.169239

      359
      91.1
      19.2
      2.2
      0.0
      10.70
      9.306803

      360
      96.9
      19.1
      1.4
      0.0
      10.25
      9.779279

360 rows × 6 columns

```python
plt.figure(figsize = [8,6])
plt.plot(df1.index,df1['dewpoint(oC)'])
plt.xlabel('Day no.')
plt.ylabel('dew point($^o$C)')
plt.title('Daily variation of Dew point')
plt.show()
```

```python

```