# Week 2
## Introduction to Solid State

```python
import numpy as np
import matplotlib.pyplot as plt
import os
import subprocess
import MSD as msd

from scipy import stats

def get_diffusion(file, atom):

    with open(file) as f:
        y = False
        for line in f:
            if str("atom        D ") in line:
                y = True
            if y == True and str(atom) in line:
                d = line.split()
                break
    return d
```

Now that you are familiar with molecular dynamics, you are now going to use it to tackle some real world problems. In the next three weeks you will investigate the transport properties of a simple fluorite material - Ca$F_2$. The transport properties of a material determine many properties that are utilised for modern technological applications. For example, solid oxide fuel cell (SOFCs - Alternative to batteries) materials are dependent on the movement of charge carriers through the solid electrolyte and  nuclear fuel materials oxidise and fall apart and this corrosive behaviour is dependent on the diffusion of oxygen into the lattice. Due to the importance of the transport properties of these materials, scientists and engineers spend large amounts of their time tring to optomise these properties using different stoichiometries, introducing defects and by using different syntheisis techniques. Over the next three weeks you will investigate how the transport properties of Ca$F_2$ are affected by temperature, structural defects (Schottky and Frenkel) and by chemcial dopants (e.g. different cations). A rough breakdown looks as follows
- Week 2
    - Introduction to DL_POLY
    - Tutorial on the calculation of diffusion coefficients
    - Tutorial on the Arhennius equation
    - Molecular dynamics simulations of stoichiomteric Ca$F_2$
- Week 3
    - Frenkel and Schottky defects
- Week 4
    - Dopants

## Introduction to DL_POLY

DL_POLY is a molecular dynamics program maintained by Daresbury laboratories. In contrast to pylj, DL_POLY is a three dimensional molecular dynamics code that is used worldwide by computational scientists for molecular simulation, but it should be noted that the theory is exactly the same and any understanding gained from pylj is completely applicable to DL_POLY. For the next three weeks you will use DL_POLY to run short molecular dynamics simulations on Ca$F_2$. You first need to understand the input files required for DL_POLY.
- CONTROL - This is the file that contains all of the simulation parameters, e.g. simulation temperature, pressure, number of steps e.t.c
- CONFIG - This is the file that contains the structure - i.e. the atomic coordinates of each atom.
- FIELD - This is the file that contains the force field or potential model e.g. Lennard Jones.

Contained within the folder "Input" you will find a file called input.txt. This is the main file that you will interact with over the next three weeks and is used to generate the FIELD, CONTROL and CONFIG. Essentially it is easier to meddle with input.txt than it is to meddle with the 3 DL_POLY files everytime you want to change something. To run metadise we will use the subprocess python module. You specify what program you want to run and the file that you want to run it in, you will need to ensure the file path is correct.

```python
subprocess.call('H:/Third_year_lab/Advanced_Practical_Chemistry_Teaching-master/progs/metadise.exe', cwd='H:/Third_year_lab/Advanced_Practical_Chemistry_Teaching-master/course/week_2/Input/')
os.rename('Input/control_o0001.dlp', 'Input/CONTROL')
os.rename('Input/config__o0001.dlp', 'Input/CONFIG')
os.rename('Input/field___o0001.dlp', 'Input/FIELD')
```

Now you should have a CONFIG, CONTROL and FIELD file within the input directory. In theory you could just call the DL_POLY program on this directory and your simulation would run. However we need to tweak the CONTROL file in order to set up our desired simulation. Make a new subdirectory in the week 2 directory named "Example" and copy CONFIG, CONTROL and FIELD to that subdirectory. Now edit the CONTROL file.

We want to change the following

`Temperature 300 ---> Temperature 1500`
`Steps 5001 ---> Steps 40000`
`ensemble nve ---> ensemble npt hoover 0.1 0.5`
`trajectory nstraj= 1    istraj=   250 keytrj=0 ---> trajectory nstraj= 0    istraj= 100  keytrj=0`

Now your simulation is ready. As a point of interest it is always good to check your structure before and after the simulation. You can view the CONFIG file in three dimensions using the VESTA program. It is available for free at http://www.jp-minerals.org/vesta/en/download.html . Download it and use it to view your CONFIG, a demonstrator can help if necessary. VESTA can generate nice pictures which will look very good in a lab report.

    Figure 1. Fluorite Ca$F_2$ unit cell visualised in VESTA.

To run DL_POLY from within a notebook use the below command. Keep in mind that this simulation will take 20 or so minutes so be patient.

```python
subprocess.call('H:/Third_year_lab/Advanced_Practical_Chemistry_Teaching-master/progs/dlpoly_classic.exe', cwd='H:/Third_year_lab/Advanced_Practical_Chemistry_Teaching-master/course/week_2/Example/')
```

    0

Once DL_POLY has completed you will find several files relating to your simulaton.
- HISTORY - This file contains the configuration of your system at each step during the simulation. You can view this as a movie using the VMD program - Ask a demonstrator for details
- REVCON - This is the configuration at the end of the simulation - Can be viewed in VESTA - why not check to see how it has changed.
- STATIS - Contains the stats at each step in the simulation.
- OUTPUT - Contains properties

It is now important to understand how we can actually use the details of the simulation to get some information on the properties of the material, e.g. Diffusion coefficients and activation energies.

## Mean Squared Displacements - Calculating diffusion coefficients

As we have seen molecules in liquds, gases and solids do not stay in the same place and move constantly. Think about a drop of dye in a glass of water, as time passes the dye distributes throughout the water. This process is called diffusion and is common throughout nature.

Using the dye as an example, the motion of a dye molecule is not simple. As it moves it is jostled by collisions with other molecules, preventing it from moving in a straight path. If the path is examined in close detail, it will be seen to be a good approximation to a random walk. In mathmatics a random walk is a series of steps, each taken in a random direction. This was analysed by Albert Einstein in a study of Brownian motion and he showed that the mean square of the distance travelled by a particle following a random walk is proportional to the time elapsed.
\begin{align}
\Big \langle r^2 \big \rangle & = 6 D_t + C
\end{align}

where $\Big \langle r^2 \big \rangle$ is the mean squared distance, t is time, D is the diffusion rate and C is a constant.

## What is the mean squared displacement

Going back to the example of the dye in water, lets assume for the sake of simplicity that we are in one dimension. Each step can either be forwards or backwards and we cannot predict which. From a given starting position, what distance is our dye molecule likely to travel after 1000 steps? This can be determined simply by adding together the steps, taking into account the fact that steps backwards subtract from the total, while steps forward add to the total. Since both forward and backward steps are equally probable, we come to the surprising conclusion that the probable distance travelled sums up to zero.

By adding the square of the distance we will always be adding positive numbers to our total which now increases linearly with time. Based upon equation 1 it should now be clear that a plot of $\Big \langle r^2 \big \rangle$ vs time with produce a line, the gradient of which is equal to 6D. Giving us direct access to the diffusion coefficient of the system.

Lets try explore this with an example. Run a short DL_POLY simulation on the input files provided.

You will a small MSD program called MSD.py to analyse your simulation results. First you need to read in the data, the HISTORY file contains a list of the atomic coordiantes held by the atoms during the simulation.

```python
# Read in the HISTORY file

## Provide the path to the simulation and the atom that you want data for.
data = msd.read_history("Example/HISTORY", "F")
```

data is a dictionary variable containing the atomic trajectories, lattice vectors, total number of atoms, and total number of timesteps.

data = {'trajectories':trajectories, 'lv':lv, 'timesteps':timesteps, 'natoms':natoms}

The next step is to calculate the MSD.

```python
# Run the MSD calculation
msd_data = msd.run_msd(data)
```

run_msd returns a dictionary containing the total MSD, the dimensional MSD values and the time.
msd_data = {'msd': msd, 'xmsd': xmsd, 'ymsd': ymsd, 'zmsd': zmsd, 'time': time}

This can then be plotted to give a nice linear relationship.

```python
plt.plot(msd_data['time'], msd_data['msd'], lw=2, color="red", label="MSD")
plt.plot(msd_data['time'], msd_data['xmsd'], lw=2, color="blue", label="X-MSD")
plt.plot(msd_data['time'], msd_data['ymsd'], lw=2, color="green", label="Y-MSD")
plt.plot(msd_data['time'], msd_data['zmsd'], lw=2, color="black", label="Z-MSD")

plt.ylabel("MSD (" r'$\AA$' ")", fontsize=15)
plt.xlabel("Time / ps", fontsize=15)
plt.ylim(0, np.amax(msd_data['msd']))
plt.xlim(0, np.amax(msd_data['time']))
plt.legend(loc=2, frameon=False)
plt.show()
```

To calculate the gradient we need to perform a linear regression on the data.

```python
slope, intercept, r_value, p_value, std_err = stats.linregress(msd_data['time'], msd_data['msd'])
```

The gradient is equal to 6D (6 = dimensionality). So our final diffusion coefficient for the simulation is given by

```python
diffusion_coefficient = (np.average(slope) / 6)
print("Diffusion Coefficient: ", diffusion_coefficient, " X 10 ^-9 (m^-2)")
```

    Diffusion Coefficient:  1.4550282026323467  X 10 ^-9 (m^-2)

## Simulation Length

It is important to consider the lenght of your simulation (Number of steps). Create a new folder called "Example_2", copy the CONFIG, FIELD and CONTROL files from your previous simulation but this time change the number of steps to 10000. Now rerun the simulation.

```python
subprocess.call('H:/Third_year_lab/Advanced_Practical_Chemistry_Teaching-master/progs/dlpoly_classic.exe', cwd='H:/Third_year_lab/Advanced_Practical_Chemistry_Teaching-master/course/week_2/Example_2/')
```

    0

```python
data = msd.read_history("Example_2/HISTORY", "F")
msd_data = msd.run_msd(data)

plt.plot(msd_data['time'], msd_data['msd'], lw=2, color="red", label="MSD")
plt.plot(msd_data['time'], msd_data['xmsd'], lw=2, color="blue", label="X-MSD")
plt.plot(msd_data['time'], msd_data['ymsd'], lw=2, color="green", label="Y-MSD")
plt.plot(msd_data['time'], msd_data['zmsd'], lw=2, color="black", label="Z-MSD")

plt.ylabel("MSD (" r'$\AA$' ")", fontsize=15)
plt.xlabel("Time / ps", fontsize=15)
plt.ylim(0, np.amax(msd_data['msd']))
plt.xlim(0, np.amax(msd_data['time']))
plt.legend(loc=2, frameon=False)
plt.show()
```

```python
slope, intercept, r_value, p_value, std_err = stats.linregress(msd_data['time'], msd_data['msd'])

diffusion_coefficient = (np.average(slope) / 6)
print("Diffusion Coefficient: ", diffusion_coefficient, " X 10 ^-9 (m^-2)")
```

    Diffusion Coefficient:  1.3602930070828145  X 10 ^-9 (m^-2)

You will hopefully see that your MSD plot has become considerably less linear. This shows that your simulation has not run long enough and your results will be unrealiable. You will hopefully also see a change to the value of your diffusion coefficient.  The length of your simulation is something that you should keep in mind for the next 3 weeks.

## Arrhenius

The next thing is to use the diffusion coefficients to calcaulte the activation energy for F diffusion. This rea=quires diffusion coefficients from a temperature range. Common sense and chemical intuition suggest that the higher the temperature, the faster a given chemical reaction will proceed. Quantitatively this relationship between the rate a reaction proceeds and its temperature is determined by the Arrhenius Equation. At higher temperatures, the probability that two molecules will collide is higher. This higher collision rate results in a higher kinetic energy, which has an effect on the activation energy of the reaction. The activation energy is the amount of energy required to ensure that a reaction happens.

\begin{align}
k = A * e^{(-Ea / RT)}
\end{align}

where k is the rate coefficient, A is a constant, Ea is the activation energy, R is the universal gas constant, and T is the temperature (in kelvin).

## Week 2 Exercise

Using what you have learned over the last 45 mins your task this week is to calculate the activation energy of F diffusion in Ca$F_2$. You will need to select a temperature range and carry out simulations at different temperatures within that range.
#### Questions to answer
- In what temperature range is Ca$F_2$ completely solid i.e. no diffusion?
- In what range is fluorine essentially liquid i.e. fluorine diffusion with no calcium diffusion?
- What is the melting temperature?
- Plot an Arrhenius plot and determine the activation energies in temperature range - You will need to rearange the equation.

You are encouraged to split the work up within your group and to learn how to view the simulation "movie" using VMD (Ask a demonstrator). VMD is a fantastic program that allows you to visualise your simulation, included below is a video showing a short snippet of an MD simulation of Ca$F_2$. A single F atom has been highlighted to show that diffusion is occuring.

```python
%%HTML
<div align="middle">
</div>
```

Furthermore, VMD can also be used to generate images showing the entire trajectory of the simulation, e.g.

    Figure 2. A figure showing all positions occupied by F during an MD simulation at 1500 K. F positions are shown in orange and Ca atoms are shown in green.

  To save you the time you can use the function declared at the start of this notebook to pull out a diffusion coefficient directly from the simulation output file. MSD.py is a small code to allow visualisation of the MSD plot but it is not neccesary every time you want the diffusion coefficient.

It is up to you how you organise/create your directories but it is reccomended that you start a new notebook. Use the commands/functions used in this notebook to generate your input files, run DL_POLY and extract the diffusion coefficients. The write your own code to generate an Arrhenius plot and calculate the activation energies.
If you finish early then feel free to start week 3.


```python
!pip install qiskit
```

    Collecting qiskit
      Downloading https://files.pythonhosted.org/packages/6f/61/cb7506e17a2566dc8a31a3e1924d91ac0bdd8ff07c71ec698c06647b6306/qiskit-0.26.2.tar.gz
    Collecting qiskit-terra==0.17.4
      Downloading https://files.pythonhosted.org/packages/b3/0c/3c7a8dd451dae0907263e9de9e3e34909e15e18c88a589b44581972c8511/qiskit_terra-0.17.4-cp37-cp37m-manylinux2010_x86_64.whl (6.0MB)
         |████████████████████████████████| 6.0MB 2.8MB/s
    Collecting qiskit-aer==0.8.2
      Downloading https://files.pythonhosted.org/packages/c2/d2/6ff15c370b5465b32529b528bf3f4ce1e01f74498be16203aa1c04b67022/qiskit_aer-0.8.2-cp37-cp37m-manylinux2010_x86_64.whl (18.0MB)
         |████████████████████████████████| 18.0MB 246kB/s
    Collecting qiskit-ibmq-provider==0.13.1
      Downloading https://files.pythonhosted.org/packages/be/99/74bbb901f88603a7d850d4889abc06d81ba702e4227151f4a5b66f2631fe/qiskit_ibmq_provider-0.13.1-py3-none-any.whl (228kB)
         |████████████████████████████████| 235kB 42.0MB/s
    Collecting qiskit-ignis==0.6.0
      Downloading https://files.pythonhosted.org/packages/54/be/a13c828e457e09d979667a61bddbd8c7246aafa94e2501b6a9154429cbea/qiskit_ignis-0.6.0-py3-none-any.whl (207kB)
         |████████████████████████████████| 215kB 41.1MB/s
    Collecting qiskit-aqua==0.9.1
      Downloading https://files.pythonhosted.org/packages/88/79/392c57b978decbb24b902344b536af52c40a751aed0ebbaefa8bc2964cb5/qiskit_aqua-0.9.1-py3-none-any.whl (2.1MB)
         |████████████████████████████████| 2.1MB 34.8MB/s
    Requirement already satisfied: python-dateutil>=2.8.0 in /usr/local/lib/python3.7/dist-packages (from qiskit-terra==0.17.4->qiskit) (2.8.1)
    Requirement already satisfied: psutil>=5 in /usr/local/lib/python3.7/dist-packages (from qiskit-terra==0.17.4->qiskit) (5.4.8)
    Collecting fastjsonschema>=2.10
      Downloading https://files.pythonhosted.org/packages/d1/fb/ea090e917b18320f79be31d754bbe496b715175e865603cfce1eaed2e774/fastjsonschema-2.15.1-py3-none-any.whl
    Collecting ply>=3.10
      Downloading https://files.pythonhosted.org/packages/a3/58/35da89ee790598a0700ea49b2a66594140f44dec458c07e8e3d4979137fc/ply-3.11-py2.py3-none-any.whl (49kB)
         |████████████████████████████████| 51kB 6.3MB/s
    Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.7/dist-packages (from qiskit-terra==0.17.4->qiskit) (2.6.0)
    Requirement already satisfied: dill>=0.3 in /usr/local/lib/python3.7/dist-packages (from qiskit-terra==0.17.4->qiskit) (0.3.3)
    Collecting retworkx>=0.8.0
      Downloading https://files.pythonhosted.org/packages/1b/92/f007f8b9d88dcd5b90e363967e5d54431a68c5fe06d83400732e3b438084/retworkx-0.8.0-cp37-cp37m-manylinux2010_x86_64.whl (1.0MB)
         |████████████████████████████████| 1.0MB 35.6MB/s
    Requirement already satisfied: sympy>=1.3 in /usr/local/lib/python3.7/dist-packages (from qiskit-terra==0.17.4->qiskit) (1.7.1)
    Collecting python-constraint>=1.4
      Downloading https://files.pythonhosted.org/packages/37/8b/5f1bc2734ca611943e1d6733ee244238679f6410a10cd45ede55a61a8402/python-constraint-1.4.0.tar.bz2
    Requirement already satisfied: scipy>=1.4 in /usr/local/lib/python3.7/dist-packages (from qiskit-terra==0.17.4->qiskit) (1.4.1)
    Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.7/dist-packages (from qiskit-terra==0.17.4->qiskit) (1.19.5)
    Collecting pybind11>=2.6
      Downloading https://files.pythonhosted.org/packages/8d/43/7339dbabbc2793718d59703aace4166f53c29ee1c202f6ff5bf8a26c4d91/pybind11-2.6.2-py2.py3-none-any.whl (191kB)
         |████████████████████████████████| 194kB 47.8MB/s
    Requirement already satisfied: requests>=2.19 in /usr/local/lib/python3.7/dist-packages (from qiskit-ibmq-provider==0.13.1->qiskit) (2.23.0)
    Collecting requests-ntlm>=1.1.0
      Downloading https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl
    Requirement already satisfied: nest-asyncio!=1.1.0,>=1.0.0 in /usr/local/lib/python3.7/dist-packages (from qiskit-ibmq-provider==0.13.1->qiskit) (1.5.1)
    Collecting websockets>=8
      Downloading https://files.pythonhosted.org/packages/84/64/78c2b3fe37730b30dca3c93d1f7f4a4286767f86e7c04cf3571b39bc2fb7/websockets-9.1-cp37-cp37m-manylinux2010_x86_64.whl (103kB)
         |████████████████████████████████| 112kB 50.7MB/s
    Requirement already satisfied: urllib3>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from qiskit-ibmq-provider==0.13.1->qiskit) (1.24.3)
    Requirement already satisfied: setuptools>=40.1.0 in /usr/local/lib/python3.7/dist-packages (from qiskit-ignis==0.6.0->qiskit) (56.1.0)
    Collecting dlx<=1.0.4
      Downloading https://files.pythonhosted.org/packages/54/c0/b8fb5bb727e983b6f5251433ef941b48f38c65bb0bd6ec509e9185bcd406/dlx-1.0.4.tar.gz
    Collecting quandl<=3.6.0
      Downloading https://files.pythonhosted.org/packages/c2/58/9f0e69d836045e3865d263e9ed49f42b23a58526fdabb30f74c430baee3f/Quandl-3.6.0-py2.py3-none-any.whl
    Requirement already satisfied: scikit-learn<=0.24.1,>=0.20.0 in /usr/local/lib/python3.7/dist-packages (from qiskit-aqua==0.9.1->qiskit) (0.22.2.post1)
    Collecting yfinance<=0.1.55
      Downloading https://files.pythonhosted.org/packages/7a/e8/b9d7104d3a4bf39924799067592d9e59119fcfc900a425a12e80a3123ec8/yfinance-0.1.55.tar.gz
    Requirement already satisfied: fastdtw<=0.3.4 in /usr/local/lib/python3.7/dist-packages (from qiskit-aqua==0.9.1->qiskit) (0.3.4)
    Requirement already satisfied: pandas<=1.2.3 in /usr/local/lib/python3.7/dist-packages (from qiskit-aqua==0.9.1->qiskit) (1.1.5)
    Collecting docplex<=2.20.204; sys_platform != "darwin"
      Downloading https://files.pythonhosted.org/packages/87/99/6f7c219b39fd58c84688ad0713eb932bfcf6be81fc74519e43ea9c915b56/docplex-2.20.204.tar.gz (611kB)
         |████████████████████████████████| 614kB 36.2MB/s
    Requirement already satisfied: h5py<=3.1.0 in /usr/local/lib/python3.7/dist-packages (from qiskit-aqua==0.9.1->qiskit) (3.1.0)
    Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/dist-packages (from python-dateutil>=2.8.0->qiskit-terra==0.17.4->qiskit) (1.15.0)
    Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.7/dist-packages (from sympy>=1.3->qiskit-terra==0.17.4->qiskit) (1.2.1)
    Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests>=2.19->qiskit-ibmq-provider==0.13.1->qiskit) (2.10)
    Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests>=2.19->qiskit-ibmq-provider==0.13.1->qiskit) (3.0.4)
    Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests>=2.19->qiskit-ibmq-provider==0.13.1->qiskit) (2020.12.5)
    Collecting ntlm-auth>=1.0.2
      Downloading https://files.pythonhosted.org/packages/ff/84/97c550164b54942b0e908c31ef09d9469f3ba4cd7332a671e2125732f63b/ntlm_auth-1.5.0-py2.py3-none-any.whl
    Collecting cryptography>=1.3
      Downloading https://files.pythonhosted.org/packages/b2/26/7af637e6a7e87258b963f1731c5982fb31cd507f0d90d91836e446955d02/cryptography-3.4.7-cp36-abi3-manylinux2014_x86_64.whl (3.2MB)
         |████████████████████████████████| 3.2MB 27.3MB/s
    Requirement already satisfied: more-itertools in /usr/local/lib/python3.7/dist-packages (from quandl<=3.6.0->qiskit-aqua==0.9.1->qiskit) (8.7.0)
    Collecting inflection>=0.3.1
      Downloading https://files.pythonhosted.org/packages/59/91/aa6bde563e0085a02a435aa99b49ef75b0a4b062635e606dab23ce18d720/inflection-0.5.1-py2.py3-none-any.whl
    Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.7/dist-packages (from scikit-learn<=0.24.1,>=0.20.0->qiskit-aqua==0.9.1->qiskit) (1.0.1)
    Requirement already satisfied: multitasking>=0.0.7 in /usr/local/lib/python3.7/dist-packages (from yfinance<=0.1.55->qiskit-aqua==0.9.1->qiskit) (0.0.9)
    Collecting lxml>=4.5.1
      Downloading https://files.pythonhosted.org/packages/30/c0/d0526314971fc661b083ab135747dc68446a3022686da8c16d25fcf6ef07/lxml-4.6.3-cp37-cp37m-manylinux2014_x86_64.whl (6.3MB)
         |████████████████████████████████| 6.3MB 30.2MB/s
    Requirement already satisfied: pytz>=2017.2 in /usr/local/lib/python3.7/dist-packages (from pandas<=1.2.3->qiskit-aqua==0.9.1->qiskit) (2018.9)
    Requirement already satisfied: cached-property; python_version < "3.8" in /usr/local/lib/python3.7/dist-packages (from h5py<=3.1.0->qiskit-aqua==0.9.1->qiskit) (1.5.2)
    Requirement already satisfied: cffi>=1.12 in /usr/local/lib/python3.7/dist-packages (from cryptography>=1.3->requests-ntlm>=1.1.0->qiskit-ibmq-provider==0.13.1->qiskit) (1.14.5)
    Requirement already satisfied: pycparser in /usr/local/lib/python3.7/dist-packages (from cffi>=1.12->cryptography>=1.3->requests-ntlm>=1.1.0->qiskit-ibmq-provider==0.13.1->qiskit) (2.20)
    Building wheels for collected packages: qiskit, python-constraint, dlx, yfinance, docplex
      Building wheel for qiskit (setup.py) ... done
      Created wheel for qiskit: filename=qiskit-0.26.2-cp37-none-any.whl size=10491 sha256=4c9777ea47a1179a3ef2032714d406efc22601986d91079e5ab754c186cec162
      Stored in directory: /root/.cache/pip/wheels/89/89/34/524839952d5a58a7be9789e580bfc1ca883bf6579152444568
      Building wheel for python-constraint (setup.py) ... done
      Created wheel for python-constraint: filename=python_constraint-1.4.0-py2.py3-none-any.whl size=24079 sha256=4d6021d1c3044625952a4f3d13941fa70fc366a3fca94c17a139e16a421f3ef0
      Stored in directory: /root/.cache/pip/wheels/34/31/15/7b070b25d0a549d20ce2e9fe6d727471c2c61ef904720fd40c
      Building wheel for dlx (setup.py) ... done
      Created wheel for dlx: filename=dlx-1.0.4-cp37-none-any.whl size=5712 sha256=0c747ac3a816c7f566fc5e413f4323a2d6171e35523b0eab1417fa9f271ab9de
      Stored in directory: /root/.cache/pip/wheels/bb/ba/15/fdd0deb104df3254912998150ba9245668db06b00af5912d1a
      Building wheel for yfinance (setup.py) ... done
      Created wheel for yfinance: filename=yfinance-0.1.55-py2.py3-none-any.whl size=22616 sha256=58d254f8a73997cea5cc11d60c93af01927565167ff0ba2f2442d0037f823e5b
      Stored in directory: /root/.cache/pip/wheels/04/98/cc/2702a4242d60bdc14f48b4557c427ded1fe92aedf257d4565c
      Building wheel for docplex (setup.py) ... done
      Created wheel for docplex: filename=docplex-2.20.204-cp37-none-any.whl size=675362 sha256=e3e3560717a797eb15db1c32555d12e5c96e41fead51074967d1978ba8fc8e49
      Stored in directory: /root/.cache/pip/wheels/ae/2c/e2/a099ebb6fda8adeba9c5fc2e25659d195ad2f5c6cc5fb75fd4
    Successfully built qiskit python-constraint dlx yfinance docplex
    Installing collected packages: fastjsonschema, ply, retworkx, python-constraint, qiskit-terra, pybind11, qiskit-aer, ntlm-auth, cryptography, requests-ntlm, websockets, qiskit-ibmq-provider, qiskit-ignis, dlx, inflection, quandl, lxml, yfinance, docplex, qiskit-aqua, qiskit
      Found existing installation: lxml 4.2.6
        Uninstalling lxml-4.2.6:
          Successfully uninstalled lxml-4.2.6
    Successfully installed cryptography-3.4.7 dlx-1.0.4 docplex-2.20.204 fastjsonschema-2.15.1 inflection-0.5.1 lxml-4.6.3 ntlm-auth-1.5.0 ply-3.11 pybind11-2.6.2 python-constraint-1.4.0 qiskit-0.26.2 qiskit-aer-0.8.2 qiskit-aqua-0.9.1 qiskit-ibmq-provider-0.13.1 qiskit-ignis-0.6.0 qiskit-terra-0.17.4 quandl-3.6.0 requests-ntlm-1.1.0 retworkx-0.8.0 websockets-9.1 yfinance-0.1.55

```python
from qiskit.providers.aer import QasmSimulator
from qiskit.providers.aer import AerSimulator
from qiskit.circuit.library import TwoLocal
from qiskit.algorithms import VQE
from qiskit.algorithms.optimizers import COBYLA
from qiskit.circuit.library import EfficientSU2
# opflow is Qiskit's module for creating operators like yours
from qiskit import *
from qiskit.opflow import OperatorBase
from qiskit.opflow import Z, X, I  # Pauli Z, X matrices and identity
import pylab
import matplotlib.pyplot as plt
import numpy as np

%matplotlib inline
```

```python
counts = []
values = []

def store_intermediate_result(eval_count, parameters, mean, std):
  counts.append(eval_count)
  values.append(mean)
```

```python
def run(B_X = 1, J_z = 1, B_Z = 1):
    master_counts = []
    master_values = []
    # for h in range (1,5,1):
    #     h=+h

    # Initialization
    B_X = B_X
    J_z = J_z
    B_Z = B_Z

    # or whatever value you have for h
    #H = - B_X * ((X ^ I ^ I ^ I) + (I ^ X ^ I ^ I) + (I ^ I ^ X ^ I) + (I ^ I ^ I ^ X)) + J_z * ((Z ^ Z ^ I ^ I ) + (I ^ Z ^ Z ^ I) + (I ^ I ^ Z ^ Z) + (Z ^ I ^ I ^ Z)) - B_Z * ((Z ^ I ^ I ^ I) + (I ^ Z ^ I ^ I) + (I ^ I ^ Z ^ I ) + (I ^ I ^ I ^ Z))

    # for 25 qubits
    H = - B_Z * (( Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z)
    ) + J_z * ((( Z ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ Z ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ Z ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ Z ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ Z ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ Z ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ Z ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ Z ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ Z ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ Z ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ Z ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ Z ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ Z ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ Z) + ( Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z)))
    - B_X * (( X ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ X ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ X ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ X ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ X ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ X ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ X ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ X ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ X ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ X ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ X ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ X ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ X ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ X ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ X ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ X ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ X ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ X ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ X ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ X ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ X ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ X ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ X ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ X ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ X)
    )

    # you can swap this for a real quantum device and keep the rest of the code the same!
    backend = QasmSimulator()

    # COBYLA usually works well for small problems like this one
    optimizer = COBYLA(maxiter=200)

    # EfficientSU2 is a standard heuristic chemistry ansatz from Qiskit's circuit library
    #ansatz = EfficientSU2(4, reps=1)

    # for 25 qubits
    # ansatz = EfficientSU2(25, reps=1)
    ansatz = TwoLocal(num_qubits=25, rotation_blocks=['ry', 'rz'], entanglement_blocks=None, entanglement='full', reps=1, skip_unentangled_qubits=False, skip_final_rotation_layer=True)

    # set the algorithm
    vqe = VQE(ansatz, optimizer, quantum_instance=backend, callback=store_intermediate_result)

    # run it with the Hamiltonian we defined above
    result = vqe.compute_minimum_eigenvalue(H)

    # print the result (it contains lot's of information)
    return result

```

```python
result = run()
print(result)
```

```python
print(result.optimal_value)
print(result.eigenvalue)
```

```python
counts_values = {}

for i in range(0, 20, 1):
    for j in range(0, 20, 1):
        print(f"Running VQE for BX : {i/10} & BZ : {j/10}, \t\t Optimal Value : {result.optimal_value}")
        counts = []
        values = []
        result = run(B_X = i/10, J_z = 1, B_Z = j/10)
        # counts_values[f"BX_{i/10} BZ_{j/10}"] = {"counts": counts, "values": values}
        counts_values[f"BX_{i/10} BZ_{j/10}"] = {'result': result}
```

```python
import pickle

print("Saving Optimization History")
with open('optimization_data.pickle', 'wb') as handle:
    pickle.dump(counts_values, handle, protocol=pickle.HIGHEST_PROTOCOL)
```

```python
print("Loading Optimization History")

with open('optimization_data.pickle', 'rb') as handle:
    counts_values = pickle.load(handle)
```

```python
arr = []
for i in range(0, 20, 1):
    r = []
    for j in range(0, 20, 1):
        cv = counts_values[f"BX_{i/10} BZ_{j/10}"]['result']
        r += [cv.optimal_value]
    arr += [r]
data = np.asarray(arr)
data.shape
```

```python
X = np.asarray([ x for x in range(0, 20, 1) ])
Y = np.asarray([ y for y in range(-10, 10, 1) ])
Z = data
```

```python
import numpy as np
import seaborn as sns
import matplotlib.pylab as plt

plt.imshow(Z, cmap='hot', interpolation='nearest')
plt.colorbar()
plt.show()
```

```python
import numpy
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D

nx, ny = 20, 20
x = range(nx)
y = range(ny)

hf = plt.figure()
ha = hf.add_subplot(111, projection='3d')

X, Y = numpy.meshgrid(x, y)
ha.plot_surface(X, Y, Z)

plt.show()
```

```python
import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d.axes3d import Axes3D

x = np.arange(0,20)
y = np.arange(0,20)

xs, ys = np.meshgrid(x, y)

fig = plt.figure()
ax = Axes3D(fig)
ax.plot_surface(xs, ys, Z, rstride=1, cstride=1, cmap='hot')
plt.show()
```

```python
import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import axes3d
# if using a Jupyter notebook, include:
%matplotlib inline

fig = plt.figure(figsize=(12, 6))
ax1 = fig.add_subplot(121, projection='3d')
ax2 = fig.add_subplot(122, projection='3d')

x = np.arange(0,20)
y = np.arange(0,20)
X,Y = np.meshgrid(x,y)

# Plot a basic wireframe
ax1.plot_wireframe(Y, X, Z, rstride=10, cstride=10, cmap='hot')
ax1.set_title('row step size 10, column step size 10')

ax2.plot_wireframe(Y, X, Z, rstride=20, cstride=20, cmap='hot')
ax2.set_title('row step size 20, column step size 20')

plt.show()

```

```python
import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import axes3d
from matplotlib import cm

# if using a Jupyter notebook, include:
%matplotlib inline

fig = plt.figure(figsize=(12, 4))
ax1 = fig.add_subplot(121, projection='3d')
ax2 = fig.add_subplot(122, projection='3d')

x = np.arange(0,20)
y = np.arange(0,20)
X,Y = np.meshgrid(x,y)

# Plot a basic wireframe
ax1.plot_surface(X, Y, Z, rstride=5, cstride=5, cmap='hot')

ax2.plot_surface(Y, X, Z, rstride=5, cstride=5, cmap='hot')
ax2.contourf(Y, X, Z, zdir='z', offset=np.min(Z), cmap=cm.ocean)

plt.show()

```

```python
import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import axes3d
from matplotlib import cm

# if using a Jupyter notebook, include:
%matplotlib inline

fig = plt.figure(figsize=(12, 4))
ax1 = fig.add_subplot(121, projection='3d')
ax2 = fig.add_subplot(122, projection='3d')

x = np.arange(0,20)
y = np.arange(0,20)
X,Y = np.meshgrid(x,y)

# Plot a basic wireframe
ax1.plot_surface(X, Y, Z, rstride=5, cstride=5, cmap='hot')

ax2.plot_surface(Y, X, Z, rstride=5, cstride=5, cmap='hot')

plt.show()

```

```python
# HAMILTONIAN FOR 25 QUBITS

# H = - B_Z * (( Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z)
# ) + J_z * ((( Z ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ Z ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ Z ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ Z ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ Z ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ Z ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ Z ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ Z ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ Z ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ Z ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ Z ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ Z ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ Z ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z ^ Z) + ( Z ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ Z)))
# - B_X * (( X ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ X ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ X ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ X ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ X ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ X ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ X ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ X ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ X ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ X ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ X ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ X ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ X ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ X ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ X ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ X ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ X ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ X ^ I ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ X ^ I ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ X ^ I ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ X ^ I ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ X ^ I ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ X ^ I ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ X ^ I) + ( I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ I ^ X)
# )
```

```python

```


```python
%matplotlib inline
```

`Introduction <introyt1_tutorial.html>`_ ||
`Tensors <tensors_deeper_tutorial.html>`_ ||
**Autograd** ||
`Building Models <modelsyt_tutorial.html>`_ ||
`TensorBoard Support <tensorboardyt_tutorial.html>`_ ||
`Training Models <trainingyt.html>`_ ||
`Model Understanding <captumyt.html>`_

The Fundamentals of Autograd
============================

Follow along with the video below or on `youtube <https://www.youtube.com/watch?v=M0fX15_-xrY>`__.

.. raw:: html

PyTorch’s *Autograd* feature is part of what make PyTorch flexible and
fast for building machine learning projects. It allows for the rapid and
easy computation of multiple partial derivatives (also referred to as
*gradients)* over a complex computation. This operation is central to
backpropagation-based neural network learning.

The power of autograd comes from the fact that it traces your
computation dynamically *at runtime,* meaning that if your model has
decision branches, or loops whose lengths are not known until runtime,
the computation will still be traced correctly, and you’ll get correct
gradients to drive learning. This, combined with the fact that your
models are built in Python, offers far more flexibility than frameworks
that rely on static analysis of a more rigidly-structured model for
computing gradients.

What Do We Need Autograd For?
-----------------------------

A machine learning model is a *function*, with inputs and outputs. For
this discussion, we’ll treat the inputs a as an *i*-dimensional vector
$\vec{x}$, with elements $x_{i}$. We can then express the
model, *M*, as a vector-valued function of the input: $\vec{y} =
\vec{M}(\vec{x})$. (We treat the value of M’s output as
a vector because in general, a model may have any number of outputs.)

Since we’ll mostly be discussing autograd in the context of training,
our output of interest will be the model’s loss. The *loss function*
L($\vec{y}$) = L($\vec{M}$\ ($\vec{x}$)) is a
single-valued scalar function of the model’s output. This function
expresses how far off our model’s prediction was from a particular
input’s *ideal* output. *Note: After this point, we will often omit the
vector sign where it should be contextually clear - e.g.,* $y$
instead of $\vec y$.

In training a model, we want to minimize the loss. In the idealized case
of a perfect model, that means adjusting its learning weights - that is,
the adjustable parameters of the function - such that loss is zero for
all inputs. In the real world, it means an iterative process of nudging
the learning weights until we see that we get a tolerable loss for a
wide variety of inputs.

How do we decide how far and in which direction to nudge the weights? We
want to *minimize* the loss, which means making its first derivative
with respect to the input equal to 0:
$\frac{\partial L}{\partial x} = 0$.

Recall, though, that the loss is not *directly* derived from the input,
but a function of the model’s output (which is a function of the input
directly), $\frac{\partial L}{\partial x}$ =
$\frac{\partial {L({\vec y})}}{\partial x}$. By the chain rule of
differential calculus, we have
$\frac{\partial {L({\vec y})}}{\partial x}$ =
$\frac{\partial L}{\partial y}\frac{\partial y}{\partial x}$ =
$\frac{\partial L}{\partial y}\frac{\partial M(x)}{\partial x}$.

$\frac{\partial M(x)}{\partial x}$ is where things get complex.
The partial derivatives of the model’s outputs with respect to its
inputs, if we were to expand the expression using the chain rule again,
would involve many local partial derivatives over every multiplied
learning weight, every activation function, and every other mathematical
transformation in the model. The full expression for each such partial
derivative is the sum of the products of the local gradient of *every
possible path* through the computation graph that ends with the variable
whose gradient we are trying to measure.

In particular, the gradients over the learning weights are of interest
to us - they tell us *what direction to change each weight* to get the
loss function closer to zero.

Since the number of such local derivatives (each corresponding to a
separate path through the model’s computation graph) will tend to go up
exponentially with the depth of a neural network, so does the complexity
in computing them. This is where autograd comes in: It tracks the
history of every computation. Every computed tensor in your PyTorch
model carries a history of its input tensors and the function used to
create it. Combined with the fact that PyTorch functions meant to act on
tensors each have a built-in implementation for computing their own
derivatives, this greatly speeds the computation of the local
derivatives needed for learning.

A Simple Example
----------------

That was a lot of theory - but what does it look like to use autograd in
practice?

Let’s start with a straightforward example. First, we’ll do some imports
to let us graph our results:

```python
# %matplotlib inline

import torch

import matplotlib.pyplot as plt
import matplotlib.ticker as ticker
import math
```

Next, we’ll create an input tensor full of evenly spaced values on the
interval $[0, 2{\pi}]$, and specify ``requires_grad=True``. (Like
most functions that create tensors, ``torch.linspace()`` accepts an
optional ``requires_grad`` option.) Setting this flag means that in
every computation that follows, autograd will be accumulating the
history of the computation in the output tensors of that computation.

```python
a = torch.linspace(0., 2. * math.pi, steps=25, requires_grad=True)
print(a)
```

Next, we’ll perform a computation, and plot its output in terms of its
inputs:

```python
b = torch.sin(a)
plt.plot(a.detach(), b.detach())
```

Let’s have a closer look at the tensor ``b``. When we print it, we see
an indicator that it is tracking its computation history:

```python
print(b)
```

This ``grad_fn`` gives us a hint that when we execute the
backpropagation step and compute gradients, we’ll need to compute the
derivative of $sin(x)$ for all this tensor’s inputs.

Let’s perform some more computations:

```python
c = 2 * b
print(c)

d = c + 1
print(d)
```

Finally, let’s compute a single-element output. When you call
``.backward()`` on a tensor with no arguments, it expects the calling
tensor to contain only a single element, as is the case when computing a
loss function.

```python
out = d.sum()
print(out)
```

Each ``grad_fn`` stored with our tensors allows you to walk the
computation all the way back to its inputs with its ``next_functions``
property. We can see below that drilling down on this property on ``d``
shows us the gradient functions for all the prior tensors. Note that
``a.grad_fn`` is reported as ``None``, indicating that this was an input
to the function with no history of its own.

```python
print('d:')
print(d.grad_fn)
print(d.grad_fn.next_functions)
print(d.grad_fn.next_functions[0][0].next_functions)
print(d.grad_fn.next_functions[0][0].next_functions[0][0].next_functions)
print(d.grad_fn.next_functions[0][0].next_functions[0][0].next_functions[0][0].next_functions)
print('\nc:')
print(c.grad_fn)
print('\nb:')
print(b.grad_fn)
print('\na:')
print(a.grad_fn)
```

With all this machinery in place, how do we get derivatives out? You
call the ``backward()`` method on the output, and check the input’s
``grad`` property to inspect the gradients:

```python
out.backward()
print(a.grad)
plt.plot(a.detach(), a.grad.detach())
```

Recall the computation steps we took to get here:

::

   a = torch.linspace(0., 2. * math.pi, steps=25, requires_grad=True)
   b = torch.sin(a)
   c = 2 * b
   d = c + 1
   out = d.sum()

Adding a constant, as we did to compute ``d``, does not change the
derivative. That leaves $c = 2 * b = 2 * sin(a)$, the derivative
of which should be $2 * cos(a)$. Looking at the graph above,
that’s just what we see.

Be aware than only *leaf nodes* of the computation have their gradients
computed. If you tried, for example, ``print(c.grad)`` you’d get back
``None``. In this simple example, only the input is a leaf node, so only
it has gradients computed.

Autograd in Training
--------------------

We’ve had a brief look at how autograd works, but how does it look when
it’s used for its intended purpose? Let’s define a small model and
examine how it changes after a single training batch. First, define a
few constants, our model, and some stand-ins for inputs and outputs:

```python
BATCH_SIZE = 16
DIM_IN = 1000
HIDDEN_SIZE = 100
DIM_OUT = 10

class TinyModel(torch.nn.Module):

    def __init__(self):
        super(TinyModel, self).__init__()

        self.layer1 = torch.nn.Linear(1000, 100)
        self.relu = torch.nn.ReLU()
        self.layer2 = torch.nn.Linear(100, 10)

    def forward(self, x):
        x = self.layer1(x)
        x = self.relu(x)
        x = self.layer2(x)
        return x

some_input = torch.randn(BATCH_SIZE, DIM_IN, requires_grad=False)
ideal_output = torch.randn(BATCH_SIZE, DIM_OUT, requires_grad=False)

model = TinyModel()
```

One thing you might notice is that we never specify
``requires_grad=True`` for the model’s layers. Within a subclass of
``torch.nn.Module``, it’s assumed that we want to track gradients on the
layers’ weights for learning.

If we look at the layers of the model, we can examine the values of the
weights, and verify that no gradients have been computed yet:

```python
print(model.layer2.weight[0][0:10]) # just a small slice
print(model.layer2.weight.grad)
```

Let’s see how this changes when we run through one training batch. For a
loss function, we’ll just use the square of the Euclidean distance
between our ``prediction`` and the ``ideal_output``, and we’ll use a
basic stochastic gradient descent optimizer.

```python
optimizer = torch.optim.SGD(model.parameters(), lr=0.001)

prediction = model(some_input)

loss = (ideal_output - prediction).pow(2).sum()
print(loss)
```

Now, let’s call ``loss.backward()`` and see what happens:

```python
loss.backward()
print(model.layer2.weight[0][0:10])
print(model.layer2.weight.grad[0][0:10])
```

We can see that the gradients have been computed for each learning
weight, but the weights remain unchanged, because we haven’t run the
optimizer yet. The optimizer is responsible for updating model weights
based on the computed gradients.

```python
optimizer.step()
print(model.layer2.weight[0][0:10])
print(model.layer2.weight.grad[0][0:10])
```

You should see that ``layer2``\ ’s weights have changed.

One important thing about the process: After calling
``optimizer.step()``, you need to call ``optimizer.zero_grad()``, or
else every time you run ``loss.backward()``, the gradients on the
learning weights will accumulate:

```python
print(model.layer2.weight.grad[0][0:10])

for i in range(0, 5):
    prediction = model(some_input)
    loss = (ideal_output - prediction).pow(2).sum()
    loss.backward()

print(model.layer2.weight.grad[0][0:10])

optimizer.zero_grad()

print(model.layer2.weight.grad[0][0:10])
```

After running the cell above, you should see that after running
``loss.backward()`` multiple times, the magnitudes of most of the
gradients will be much larger. Failing to zero the gradients before
running your next training batch will cause the gradients to blow up in
this manner, causing incorrect and unpredictable learning results.

Turning Autograd Off and On
---------------------------

There are situations where you will need fine-grained control over
whether autograd is enabled. There are multiple ways to do this,
depending on the situation.

The simplest is to change the ``requires_grad`` flag on a tensor
directly:

```python
a = torch.ones(2, 3, requires_grad=True)
print(a)

b1 = 2 * a
print(b1)

a.requires_grad = False
b2 = 2 * a
print(b2)
```

In the cell above, we see that ``b1`` has a ``grad_fn`` (i.e., a traced
computation history), which is what we expect, since it was derived from
a tensor, ``a``, that had autograd turned on. When we turn off autograd
explicitly with ``a.requires_grad = False``, computation history is no
longer tracked, as we see when we compute ``b2``.

If you only need autograd turned off temporarily, a better way is to use
the ``torch.no_grad()``:

```python
a = torch.ones(2, 3, requires_grad=True) * 2
b = torch.ones(2, 3, requires_grad=True) * 3

c1 = a + b
print(c1)

with torch.no_grad():
    c2 = a + b

print(c2)

c3 = a * b
print(c3)
```

``torch.no_grad()`` can also be used as a function or method dectorator:

```python
def add_tensors1(x, y):
    return x + y

@torch.no_grad()
def add_tensors2(x, y):
    return x + y

a = torch.ones(2, 3, requires_grad=True) * 2
b = torch.ones(2, 3, requires_grad=True) * 3

c1 = add_tensors1(a, b)
print(c1)

c2 = add_tensors2(a, b)
print(c2)
```

There’s a corresponding context manager, ``torch.enable_grad()``, for
turning autograd on when it isn’t already. It may also be used as a
decorator.

Finally, you may have a tensor that requires gradient tracking, but you
want a copy that does not. For this we have the ``Tensor`` object’s
``detach()`` method - it creates a copy of the tensor that is *detached*
from the computation history:

```python
x = torch.rand(5, requires_grad=True)
y = x.detach()

print(x)
print(y)
```

We did this above when we wanted to graph some of our tensors. This is
because ``matplotlib`` expects a NumPy array as input, and the implicit
conversion from a PyTorch tensor to a NumPy array is not enabled for
tensors with requires_grad=True. Making a detached copy lets us move
forward.

Autograd and In-place Operations
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

In every example in this notebook so far, we’ve used variables to
capture the intermediate values of a computation. Autograd needs these
intermediate values to perform gradient computations. *For this reason,
you must be careful about using in-place operations when using
autograd.* Doing so can destroy information you need to compute
derivatives in the ``backward()`` call. PyTorch will even stop you if
you attempt an in-place operation on leaf variable that requires
autograd, as shown below.

Note

The following code cell throws a runtime error. This is expected.

::

   a = torch.linspace(0., 2. * math.pi, steps=25, requires_grad=True)
   torch.sin_(a)

Autograd Profiler
-----------------

Autograd tracks every step of your computation in detail. Such a
computation history, combined with timing information, would make a
handy profiler - and autograd has that feature baked in. Here’s a quick
example usage:

```python
device = torch.device('cpu')
run_on_gpu = False
if torch.cuda.is_available():
    device = torch.device('cuda')
    run_on_gpu = True

x = torch.randn(2, 3, requires_grad=True)
y = torch.rand(2, 3, requires_grad=True)
z = torch.ones(2, 3, requires_grad=True)

with torch.autograd.profiler.profile(use_cuda=run_on_gpu) as prf:
    for _ in range(1000):
        z = (z / x) * y

print(prf.key_averages().table(sort_by='self_cpu_time_total'))
```

The profiler can also label individual sub-blocks of code, break out the
data by input tensor shape, and export data as a Chrome tracing tools
file. For full details of the API, see the
`documentation <https://pytorch.org/docs/stable/autograd.html#profiler>`__.

Advanced Topic: More Autograd Detail and the High-Level API
-----------------------------------------------------------

If you have a function with an n-dimensional input and m-dimensional
output, $\vec{y}=f(\vec{x})$, the complete gradient is a matrix of
the derivative of every output with respect to every input, called the
*Jacobian:*

\begin{align}J
     =
     \left(\begin{array}{ccc}
     \frac{\partial y_{1}}{\partial x_{1}} & \cdots & \frac{\partial y_{1}}{\partial x_{n}}\\
     \vdots & \ddots & \vdots\\
     \frac{\partial y_{m}}{\partial x_{1}} & \cdots & \frac{\partial y_{m}}{\partial x_{n}}
     \end{array}\right)\end{align}

If you have a second function, $l=g\left(\vec{y}\right)$ that
takes m-dimensional input (that is, the same dimensionality as the
output above), and returns a scalar output, you can express its
gradients with respect to $\vec{y}$ as a column vector,
$v=\left(\begin{array}{ccc}\frac{\partial l}{\partial y_{1}} & \cdots & \frac{\partial l}{\partial y_{m}}\end{array}\right)^{T}$
- which is really just a one-column Jacobian.

More concretely, imagine the first function as your PyTorch model (with
potentially many inputs and many outputs) and the second function as a
loss function (with the model’s output as input, and the loss value as
the scalar output).

If we multiply the first function’s Jacobian by the gradient of the
second function, and apply the chain rule, we get:

\begin{align}J^{T}\cdot v=\left(\begin{array}{ccc}
   \frac{\partial y_{1}}{\partial x_{1}} & \cdots & \frac{\partial y_{m}}{\partial x_{1}}\\
   \vdots & \ddots & \vdots\\
   \frac{\partial y_{1}}{\partial x_{n}} & \cdots & \frac{\partial y_{m}}{\partial x_{n}}
   \end{array}\right)\left(\begin{array}{c}
   \frac{\partial l}{\partial y_{1}}\\
   \vdots\\
   \frac{\partial l}{\partial y_{m}}
   \end{array}\right)=\left(\begin{array}{c}
   \frac{\partial l}{\partial x_{1}}\\
   \vdots\\
   \frac{\partial l}{\partial x_{n}}
   \end{array}\right)\end{align}

Note: You could also use the equivalent operation $v^{T}\cdot J$,
and get back a row vector.

The resulting column vector is the *gradient of the second function with
respect to the inputs of the first* - or in the case of our model and
loss function, the gradient of the loss with respect to the model
inputs.

**``torch.autograd`` is an engine for computing these products.** This
is how we accumulate the gradients over the learning weights during the
backward pass.

For this reason, the ``backward()`` call can *also* take an optional
vector input. This vector represents a set of gradients over the tensor,
which are multiplied by the Jacobian of the autograd-traced tensor that
precedes it. Let’s try a specific example with a small vector:

```python
x = torch.randn(3, requires_grad=True)

y = x * 2
while y.data.norm() < 1000:
    y = y * 2

print(y)
```

If we tried to call ``y.backward()`` now, we’d get a runtime error and a
message that gradients can only be *implicitly* computed for scalar
outputs. For a multi-dimensional output, autograd expects us to provide
gradients for those three outputs that it can multiply into the
Jacobian:

```python
v = torch.tensor([0.1, 1.0, 0.0001], dtype=torch.float) # stand-in for gradients
y.backward(v)

print(x.grad)
```

(Note that the output gradients are all related to powers of two - which
we’d expect from a repeated doubling operation.)

The High-Level API
~~~~~~~~~~~~~~~~~~

There is an API on autograd that gives you direct access to important
differential matrix and vector operations. In particular, it allows you
to calculate the Jacobian and the *Hessian* matrices of a particular
function for particular inputs. (The Hessian is like the Jacobian, but
expresses all partial *second* derivatives.) It also provides methods
for taking vector products with these matrices.

Let’s take the Jacobian of a simple function, evaluated for a 2
single-element inputs:

```python
def exp_adder(x, y):
    return 2 * x.exp() + 3 * y

inputs = (torch.rand(1), torch.rand(1)) # arguments for the function
print(inputs)
torch.autograd.functional.jacobian(exp_adder, inputs)
```

If you look closely, the first output should equal $2e^x$ (since
the derivative of $e^x$ is $e^x$), and the second value
should be 3.

You can, of course, do this with higher-order tensors:

```python
inputs = (torch.rand(3), torch.rand(3)) # arguments for the function
print(inputs)
torch.autograd.functional.jacobian(exp_adder, inputs)
```

The ``torch.autograd.functional.hessian()`` method works identically
(assuming your function is twice differentiable), but returns a matrix
of all second derivatives.

There is also a function to directly compute the vector-Jacobian
product, if you provide the vector:

```python
def do_some_doubling(x):
    y = x * 2
    while y.data.norm() < 1000:
        y = y * 2
    return y

inputs = torch.randn(3)
my_gradients = torch.tensor([0.1, 1.0, 0.0001])
torch.autograd.functional.vjp(do_some_doubling, inputs, v=my_gradients)
```

The ``torch.autograd.functional.jvp()`` method performs the same matrix
multiplication as ``vjp()`` with the operands reversed. The ``vhp()``
and ``hvp()`` methods do the same for a vector-Hessian product.

For more information, including preformance notes on the `docs for the
functional
API <https://pytorch.org/docs/stable/autograd.html#functional-higher-level-api>`__


# Analyzing, Visualizing & Classifying Fake & Real News Data with Logistic Regression

In this project, we aim to analyze and visualize news data before predicting whether the article is fake or real news. We will be using Logistic Regression for classification purposes since it's good at text-classifying. Data used can be found [here](https://www.kaggle.com/clmentbisaillon/fake-and-real-news-dataset).

The data is split into two datasets, containing 44898 articles in total. The columns include:

* title: Contains the title of each article.
* text: Contains the context of each article.
* subject: Contains the subject of each article.
* date: Contains the date each article was posted in a Month DD, YYYY format.

## 1. Reading the data in

We will not be using the text column for this project. For other columns, we will be familiarizing ourselves with them in this step. Let's import, concatenate and explore the datasets.

```python
import pandas as pd
import numpy as np
import re
import datetime as dt
import matplotlib.pyplot as plt
import seaborn as sns
import warnings
from sklearn.metrics import accuracy_score
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.feature_extraction.text import CountVectorizer

%matplotlib inline

warnings.simplefilter(action='ignore', category=Warning)
pd.options.mode.chained_assignment = None
```

```python
fake = pd.read_csv("Fake.csv", usecols=["title", "subject", "date"]).copy()
real = pd.read_csv("True.csv", usecols=["title", "subject", "date"]).copy()
```

```python
fake["label"] = "fake"
real["label"] = "real"
fake.head()
```

      title
      subject
      date
      label

      0
      Donald Trump Sends Out Embarrassing New Year’...
      News
      December 31, 2017
      fake

      1
      Drunk Bragging Trump Staffer Started Russian ...
      News
      December 31, 2017
      fake

      2
      Sheriff David Clarke Becomes An Internet Joke...
      News
      December 30, 2017
      fake

      3
      Trump Is So Obsessed He Even Has Obama’s Name...
      News
      December 29, 2017
      fake

      4
      Pope Francis Just Called Out Donald Trump Dur...
      News
      December 25, 2017
      fake

```python
real.head()
```

      title
      subject
      date
      label

      0
      As U.S. budget fight looms, Republicans flip t...
      politicsNews
      December 31, 2017
      real

      1
      U.S. military to accept transgender recruits o...
      politicsNews
      December 29, 2017
      real

      2
      Senior U.S. Republican senator: 'Let Mr. Muell...
      politicsNews
      December 31, 2017
      real

      3
      FBI Russia probe helped by Australian diplomat...
      politicsNews
      December 30, 2017
      real

      4
      Trump wants Postal Service to charge 'much mor...
      politicsNews
      December 29, 2017
      real

We have labelled the data, we can now combine the datasets. We will do sampling on the entire dataset after combining with a seed. We will use the seed "1" for the random number generator for reproducable results.

```python
news = pd.concat([fake, real], axis=0).sample(frac=1, random_state=1).reset_index(drop=True)
news.head()
```

      title
      subject
      date
      label

      0
      Trump Calls For This Racist Policy To Be Forc...
      News
      September 21, 2016
      fake

      1
      Republican ex-defense secretary Cohen backs Hi...
      politicsNews
      September 7, 2016
      real

      2
      TEACHER QUITS JOB After 5th, 6th Grade Muslim ...
      politics
      May 9, 2017
      fake

      3
      LAURA INGRAHAM RIPS INTO THE PRESS…Crowd Goes ...
      politics
      Jul 21, 2016
      fake

      4
      Germany's Merkel suffers state vote setback as...
      worldnews
      October 14, 2017
      real

```python
news.label.value_counts(dropna=False)
```

    fake    23481
    real    21417
    Name: label, dtype: int64

Now that we have combined the data, we will first clean and prepare it.

## 2. Preprocessing the data

We will first take a look at the columns subject and date.

```python
news.subject.value_counts(dropna=False)
```

    politicsNews       11272
    worldnews          10145
    News                9050
    politics            6841
    left-news           4459
    Government News     1570
    US_News              783
    Middle-east          778
    Name: subject, dtype: int64

As we can see, there are no missing values in this column. However, some subject names are written in camel case with others written in snake case and more... We will replace the column names to fix this problem.

```python
news["subject"] = news["subject"].replace({"politicsNews": "Politics",
                                           "worldnews": "World",
                                           "politics": "Politics",
                                           "News": "All",
                                           "left-news": "Left",
                                           "Government News": "Government",
                                           "US_News": "US",
                                           "Middle-east": "Middle East"})
news.subject.value_counts()
```

    Politics       18113
    World          10145
    All             9050
    Left            4459
    Government      1570
    US               783
    Middle East      778
    Name: subject, dtype: int64

We will now take a look at the dates.

```python
news.date.value_counts(dropna=False)
```

    December 20, 2017      182
    December 6, 2017       166
    November 30, 2017      162
    November 9, 2017       158
    October 13, 2017       155
                          ...
    December 22, 2017        1
    June 24, 2017            1
    May 22, 2016             1
    January 24, 2016         1
    September 11, 2016       1
    Name: date, Length: 2397, dtype: int64

As we can see, there are some values which are not dates. We will explore these values.

```python
news[news.date.str.extract(r"^((?!\w+ \d+, \d+))*", expand=False).notnull()]
```

      title
      subject
      date
      label

      328
      JUST IN: CROOKED DOJ OFFICIAL Didn’t Reveal Th...
      Politics
      14-Feb-18
      fake

      1352
      LOCAL REPORTER IN DEEP BLUE STATE Stuns Libera...
      Politics
      17-Feb-18
      fake

      1883
      JUST IN: BADASS GENERAL JOHN KELLY Shoved Chin...
      Politics
      18-Feb-18
      fake

      5157
      https://fedup.wpengine.com/wp-content/uploads/...
      Government
      https://fedup.wpengine.com/wp-content/uploads/...
      fake

      5295
      https://fedup.wpengine.com/wp-content/uploads/...
      Left
      https://fedup.wpengine.com/wp-content/uploads/...
      fake

      7127
      HEY, BERNIE SANDERS: 18 SCHOOL SHOOTINGS Have ...
      Politics
      16-Feb-18
      fake

      8116
      IT BEGINS…RINO MEGA-DONOR Threatens Jeb Bush: ...
      Politics
      18-Feb-18
      fake

      8554
      “PROUD TRANSGENDER” Democrat Candidate For Con...
      Politics
      15-Feb-18
      fake

      8567
      FACEBOOK’S VP Of Ads BLASTS MEDIA: Majority Of...
      Politics
      17-Feb-18
      fake

      8768
      BUSTED! Maxine Waters’ Ties to Black Nationali...
      Politics
      18-Feb-18
      fake

      10414
      HIGH SCHOOL TEACHER Makes “Shooting At Trump” ...
      Politics
      18-Feb-18
      fake

      10756
      WATCH: SNOWFLAKES ASKED Communist Party Platfo...
      Politics
      19-Feb-18
      fake

      13545
      JUST IN: PRESIDENT TRUMP AND FIRST LADY Make S...
      Politics
      16-Feb-18
      fake

      13997
      SC STATE REP Who is a Former Cop Proposes Smar...
      Politics
      15-Feb-18
      fake

      14060
      The DIRTY TRUTH About DACA Recipients…Where Th...
      Politics
      15-Feb-18
      fake

      14553
      BREAKING: FL GOV RICK SCOTT Calls for FBI Dire...
      Politics
      16-Feb-18
      fake

      15848
      TWO HIGH SCHOOL JROTC Members Tell INCREDIBLE ...
      Politics
      15-Feb-18
      fake

      16603
      FACEBOOK RELEASES Ads Paid For By Russians…And...
      Politics
      17-Feb-18
      fake

      20181
      JUST IN: Senate Rejects Immigration Bill…Trump...
      Politics
      15-Feb-18
      fake

      20587
      WATCH SHERIFF’S Honest Response After a School...
      Politics
      16-Feb-18
      fake

      20832
      MEGHAN MCCAIN Cut-Off After Saying the Florida...
      Politics
      17-Feb-18
      fake

      21047
      POLITICS FIRST: LEFTIST PARENTS Of Former Comb...
      Politics
      15-Feb-18
      fake

      22074
      LOOK WHO Approved Visas For Russian Operatives...
      Politics
      16-Feb-18
      fake

      22375
      GOTCHA! A Stammering John Podesta Stunned by a...
      Politics
      18-Feb-18
      fake

      22686
      https://100percentfedup.com/served-roy-moore-v...
      Politics
      https://100percentfedup.com/served-roy-moore-v...
      fake

      23296
      MSNBC ANCHOR Flabbergasted at What Texas Teach...
      Politics
      19-Feb-18
      fake

      24487
      Flashback Video: Susan Rice Says Trump’s Claim...
      Politics
      16-Feb-18
      fake

      25464
      https://fedup.wpengine.com/wp-content/uploads/...
      Left
      https://fedup.wpengine.com/wp-content/uploads/...
      fake

      26221
      https://fedup.wpengine.com/wp-content/uploads/...
      Politics
      https://fedup.wpengine.com/wp-content/uploads/...
      fake

      27124
      TUCKER CARLSON Rips The Left On Gun Control Hy...
      Politics
      16-Feb-18
      fake

      28436
      Homepage
      Left
      MSNBC HOST Rudely Assumes Steel Worker Would N...
      fake

      28582
      WATCH: Mitt Romney Announces Run for Senate…Ba...
      Politics
      16-Feb-18
      fake

      28834
      https://100percentfedup.com/video-hillary-aske...
      Politics
      https://100percentfedup.com/video-hillary-aske...
      fake

      29012
      BROTHER OF HERO COACH Breaks Down Over Tribute...
      Politics
      17-Feb-18
      fake

      29380
      DOJ’s JEFF SESSIONS Opens Investigation Into W...
      Politics
      18-Feb-18
      fake

      30269
      Democrat Senator Warns Mueller Not To Release ...
      Politics
      19-Feb-18
      fake

      33547
      WATCH: ’’Carpetbagger” Mitt Romney Takes Swipe...
      Politics
      18-Feb-18
      fake

      34168
      https://fedup.wpengine.com/wp-content/uploads/...
      Politics
      https://fedup.wpengine.com/wp-content/uploads/...
      fake

      34299
      How The Left, The Anti Defamation League and M...
      Politics
      17-Feb-18
      fake

      35062
      https://fedup.wpengine.com/wp-content/uploads/...
      Government
      https://fedup.wpengine.com/wp-content/uploads/...
      fake

      40390
      MEDIA PROMOTES Fake News That Shooter Connecte...
      Politics
      15-Feb-18
      fake

      42154
      HUH? RAPPER JAY-Z Compares Trayvon Martin to M...
      Politics
      15-Feb-18
      fake

      43011
      “THE TRUMP CURSE IS REAL”…Social Media Respond...
      Politics
      17-Feb-18
      fake

      43617
      https://100percentfedup.com/12-yr-old-black-co...
      Politics
      https://100percentfedup.com/12-yr-old-black-co...
      fake

      44457
      WHAT’S WRONG WITH PELOSI? Nancy Stutters, Stam...
      Politics
      15-Feb-18
      fake

Here are some observations about our recent exploration:

* The articles that have falsely formatted dates and other values for dates are all labelled fake.
* They are mostly about politics.

What we will do is, we will replace these values with null values.

```python
news.loc[news.date.str.extract(r"^((?!\w+ \d+, \d+))*", expand=False).notnull(), "date"] = np.nan
```

```python
news[news.date.isnull()].shape[0]
```

    45

Before classification, we will visualize our data to see if we're missing anything so far. We will do some further preprocessing later on.

## 3. Visualizing the data for gaining further insights

We will now visualize and analyze the data we have to see if there are any patterns we might be missing. We will begin with visualizing the time series data for frequencies of fake vs. real news articles.

```python
news.info()
```

    <class 'pandas.core.frame.DataFrame'>
    RangeIndex: 44898 entries, 0 to 44897
    Data columns (total 4 columns):
     #   Column   Non-Null Count  Dtype
    ---  ------   --------------  -----
     0   title    44898 non-null  object
     1   subject  44898 non-null  object
     2   date     44853 non-null  object
     3   label    44898 non-null  object
    dtypes: object(4)
    memory usage: 1.4+ MB

```python
news.date = pd.to_datetime(news.date, errors="coerce")
news_grouped = news[["date", "subject", "label"]].groupby(["date", "label"]).count().reset_index()

fig, ax = plt.subplots(figsize=(16,10))
sns.lineplot(x="date", y="subject", hue="label", data=news_grouped, palette="Set2", ax=ax)
plt.title("News Articles Labelled Fake vs. Real")
plt.xlabel("Time")
plt.ylabel("Count")
```

As we are able to see from the line chart above, we have no real news article data prior to the beginning of 2016. However, even after that, the amount of fake news articles is dominating compared to the amount of real news articles for a while. We see a sudden increase in number of real news articles in the last quarter of 2016 followed by another unexpected peak nearing May, 2017. Afterwards, we see a sudden and massive increase in the amount of real articles' data with a more subtle decrease in number of fake articles over time.

Let's take a look at the subjects.

```python
news_group_by_subj_and_label = news.groupby(by=["label", "subject"]).count().reset_index()

fig1, ax1 = plt.subplots(figsize=(16, 8))
sns.barplot(x="subject", y="title", hue="label", data=news_group_by_subj_and_label, palette="Set2", saturation=0.5, ax=ax1)
plt.title("Fake vs. Real News Articles by Subjects")
plt.xticks(rotation=45, horizontalalignment='center', fontweight='light', fontsize='x-large')
plt.yticks(horizontalalignment='center', fontweight='light', fontsize='large')
plt.xlabel("Subjects", fontsize="large")
plt.ylabel("Amount", fontsize="large")
plt.legend(fontsize="large")
```

As we can see, "real" article group contains articles that have either "Politics" or "World" as their subjects while "fake" articles consist of a variety of subjects. Fake news articles seem to have a tendency to belong into the subject category "All", followed by "Politics" whereas not showing any interest in "World" category. In the meanwhile, real news articles belong into either one of these categories: "Politics" and "World". Both real and fake articles have many articles in "Politics" category.

## 4. Further Preprocessing

Before we move on to the classification, we will be working on the data further.

Our first step will be dropping some data. We will drop articles that were released before 2016 because there are no real news articles in that period, therefore, the data is not representative enough. Talking about representation, we will also be dropping articles from subjects "Government", "Left", "Middle East" and "US". That's because the only common subject both fake and real articles have is "Politics" and we will be dropping too much data if we drop articles with "All" and "World" subjects. Also, their frequencies and names also indicate that the "All" subject for fake articles might be the equivalent of the "World" subject in real news.

```python
news_clean = news[news.date > dt.datetime(2016,1,1)]
news_clean.date.value_counts().sort_index()
```

    2016-01-02    35
    2016-01-03    31
    2016-01-04    34
    2016-01-05    30
    2016-01-06    43
                  ..
    2017-12-27    78
    2017-12-28    11
    2017-12-29    26
    2017-12-30    16
    2017-12-31    12
    Name: date, Length: 730, dtype: int64

```python
news_clean = news_clean[news_clean.subject.isin(["All", "Politics", "World"])]
news_clean.subject.value_counts()
```

    Politics    16910
    World       10145
    All          9030
    Name: subject, dtype: int64

```python
news_clean.shape[0]
```

    36085

As we can see, we still ended up with a decent amount of data! As a next step, we will be working on our features.

## 5. Feature Weighting

As our features, we are going to use words in the article titles. We will be assigning weights to these words using term-frequencies of them. We will begin with splitting the dat into training and testing sets.

```python
training_data, testing_data = train_test_split(news_clean, random_state=1) #seed for reproducibility

Y_train = training_data["label"].values
Y_test = testing_data["label"].values

def word_counter(data, column, training_set, testing_set):

    cv = CountVectorizer(binary=False, max_df=0.95)
    cv.fit_transform(training_data[column].values)

    train_feature_set = cv.transform(training_data[column].values)
    test_feature_set = cv.transform(testing_data[column].values)

    return train_feature_set, test_feature_set, cv

X_train, X_test, feature_transformer = word_counter(news_clean, "title", training_data, testing_data)
```

## 6. Training the model, prediction & accuracy

Now that we have prepared our features, we will be training our model. Then, we will predict values for our testing set. We will be using the accuracy metric after that. Accuracy is measured using the formula below:

\begin{equation}
\text{Accuracy} = \frac{\text{number of correctly classified articles}}{\text{total number of classified articles}}
\end{equation}

```python
classifier = LogisticRegression(solver="newton-cg", C=5, penalty="l2", multi_class="multinomial", max_iter=1000)
model = classifier.fit(X_train, Y_train)
```

```python
predictions = model.predict(X_test)
accuracy = accuracy_score(Y_test, predictions, normalize=True)
print("Our model has {}% prediction accuracy.".format(round(accuracy, 2) * 100))
```

    Our model has 96.0% prediction accuracy.

## 7. Conclusion

Through this project, we have cleaned, analyzed, visualized and classified fake and real news articles.

* We have cleaned our data to prepare it for further analysis.
* Later, we have visualized and analyzed the data before fitting it to make sure that our data is ready.
* We assigned term-frequencies of the words in the title column as their weights while feature weighting.
* We have used Multinomial Logistic Regression for classification and accuracy as our metric. Our model successfully classified 96% of the testing data in the end.


# Introduction

The following is a brief introduction to Python and the IPython Notebook. There is much more to learn than what is covered here. This is just enough to get you started with the tutorial.

# The IPython Notebook

IPython consists of two processes: a kernel and a frontend. The kernel executes code while the frontend provides an interface for the user enter their code. The IPython notebook is a frontend for Python which provides an interactive web-environment for executing code and displaying rich media.

### Using the Notebook

To start an IPython notebook session open a command prompt, navigate to a desired working directory then issue the following command:

`ipython notebook`

A new window will open in your web browser where you can open an existing notebook or start a new one.
Notebooks are organized with cells. You may have a code cell for inputing commands followed by its result cell which contains the output of the code. You may also have a text cell, such as this one you're reading right now. Cell type can be changed in the above dropdown menu.

There is the menubar above for navigating a notebook but you will find the following shortcuts helpful:

- `Enter` : Create a new line with in cell
- `Shift + Enter` : Execute cell and advance to next cell
- `Ctrl + Enter` : Execute cell in place (do not advance to the next cell)
- Press `esc` (command mode) then `h` to display keyboard shortcuts

At times you might run code that gets stuck in an infinite loop or simply you want to clear all your workspace variables and start over. To solve each of these problems you can click on the menu:

`Kernel -> Interrupt`

`Kernel -> Restart`

```python
from __future__ import print_function
```

### Magic Commands

These are commands to control IPython itself.

```python
# list available magic commands
%lsmagic
```

```python
%load utils.py
```

### Need Help?

In case you're lost help isn't far. The following commands should provide assistance.

```python
# Displays an overview of IPython's features
?
```

```python
# A Quick Reference for IPython
%quickref
```

```python
# For details about an object
# object_name?
round??
```

```python
# list functions available in your workspace
dir()
```

```python
# list variables available in your workspace
%whos
```

# Python

### Basic Data Types

```python
a = 5
b = 5.0
c = float(5)
d = 'dee'
e = 'e'

type(a), type(b), type(c), type(d), type(e)
```

### Data Structures

Python offers several builtin data structures for arranging data in memory. We will be making use of lists, dictionaries, tuples during this tutorial.

#### Lists

A list is a versatile container that holds objects in the order given. Lists are typically used to group similar items but may contain heterogenous data types.

```python
empty_list = []

string_list = ['lions', 'tigers', 'bears', 'sharks', 'hamsters']

int_list = [0, 1, 2, 3, 4]
int_list2 = range(5,10)

list_from_variables = [a,b,c,d,e]

list_of_lists = [empty_list,
                 string_list,
                 list_from_variables,
                 int_list,
                 int_list2]

print(list_of_lists)
```

Elements of a list are accessible by their index.

```python
print(string_list[0])
print(string_list[1:4])
print(int_list[::2])  # get every 2nd element
print(list_of_lists[1][4])  # get a nested item
```

List are mutable, meaning after a list is created we can change, add, or remove elements.

```python
int_list[2] = 222

int_list.append(5)

string_list.remove('lions')

list_from_variables.extend(int_list)

print(int_list)
print(string_list)
print(list_from_variables)
```

#### Tuples

Tuples share similarites with lists. A tuple is good for organizing related data that may be of different types. Notice they are defined with parenthesis, `()`, rather than brackets.

```python
joe_blow = (32, 'tall', 'likes hats')

print(joe_blow[1])
```

Unlike lists, tuples are immutable. They cannot be changed once defined.

```python
# this won't work
joe_blow.append('married')

# neither will this
joe_blow[2] = 'not really a fan of hats'
```

In python a function can return multiple values. These ouputs are packed into a tuple. Tuple unpacking assigns individual elements of a tuple to separate variables.

```python
pets = ('elephant', 'cow', 'rock')

pet1, pet2, pet3 = pets
```

A peculiar thing about tuples in python is defining a single element tuple. Note the trialing comma. This is necessary for python to know you want a one-element tuple.

```python
(pet1,)
```

#### Dictionaries

A dictionary is an unordered set of *key:value* pairs. Much like a language dictionary where you look up a *word* and get its *definition* in a python dictionary you look up a *key* and get its *value*.

```python
# numbers or strings may be used as keys
dictionary0 = {'key1':'value1', 'key2':'value2', 'key3':'value3'}
dictionary1 = {1:'value1', 2:'value2', 3:'value3'}

cylinder = {'mass':50, 'base':10, 'height':100}

print(dictionary0)
print(dictionary1.keys())
print(cylinder['mass'])
```

The zip function is a convenient function to help generate a dictionary. It takes sequence objects and combines them into a list of tuples. We can subsequently use the list of two element tuples to create a dictionary.

```python
keys = ['mass01', 'inertia01', 'mass02', 'inertia02']
values = [10, 1, 50, 5]

dict(zip(keys, values))
```

### Functions

Python does not use braces, `{}`, or `end` statements to seperate blocks of code. Rather, code blocks are initialized with colon, `:`, and defined by their indentation. It is convention to use four spaces for each level of indentation.

```python
def abs_value(A):
    if A < 0:
        A = -A
    return A

abs_value(-100)
```

```python
def long_div(dividend, divisor):
    quotient = dividend // divisor # // : floor division
    remainder = dividend % divisor # % : modulo
    return quotient, remainder

a = 430
b = 25

# an example of tuple unpacking
quo, rem = long_div(a, b)

print('%d divided %d is %d remainder %d' % (a, b, quo, rem))
```

### Modules

Modules add additional functionality not present in the default installation of python. Throughout this tutorial we will either import an entire module or import specific functions from a module.

```python
# import object from sympy into the current namespace
from numpy import array

# import multiple objects from sympy
from sympy.physics.mechanics import dynamicsymbols, ReferenceFrame, Point
```

Objects from these modules are now available in your namespace.

```python
# from numpy
arr = array([1,2,3,4,5])

# from sympy.physics.mechanics
inertial_frame = ReferenceFrame('I')
```

# The SciPy Stack

SciPy is a collection of open source software that brings scientific computing to Python.

- IPython : Enhanced Interactive Console
- Numpy : N-dimensional array package
- Matplotlib : a fully featured 2D plotting package
- Scipy : a collection of toolboxes for signal processing, optimization, statisitics, etc.
- Sympy : symbolic mathematics and computer algebra
- and much more...

## Numpy

### Setup

```python
from numpy import random, linspace, zeros, arange
```

### Creating Arrays

```python
# An array from a list
print(array([5, 12, -2, 9.3, 7]))

# random values
print(random.random((5)))

# linearly spaced values
# 5 values between 0 and 10 inclusive
print(linspace(0,10,5))

# range of values with a defined stepsize
# start at 0 and increase by 3
print(arange(0,14,3))
```

### Accessing Array Elements

```python
P = random.random((3,5))

# individual element
print(P[0,3])

# entire row
print(P[2])

# entire column
print(P[:,4])

# every third element
print(P[::3])
```

### Operations on Arrays

```python
# mathematical operations are always elementwise
x = arange(5)

print(x)

# the double asterisk represents exponentiation
print(x + x**2)
```

## Matplotlib

Matplotlib provides an API similar to MATLAB's

Use the magic command `%matplotlib inline` to work with matplotlib interactively. The `inline` argument allows for plots to be embedded within the IPython notebook

```python
%matplotlib inline
```

```python
from matplotlib.pyplot import plot, subplot, xlabel, ylabel, legend, tight_layout
from numpy import sin, cos, pi
```

### Examples

```python
x = arange(-pi,pi,0.1)
y1 = 2*sin(x)
y2 = x + cos(4*x)

plot(x, y1, 'r', x, y2, '--b')
plot(x[::5], y1[::5], 'og') # plot every 5th point
xlabel('x axis')
ylabel('y axis')
legend(['red', 'blue', 'green'], loc='upper left')
```

```python
x = linspace(-100,100)

for i in range(1,5):
    subplot(2,2,i)
    plot(x, x**i)

tight_layout() # this prevents the axis labels from overlapping
```

### Exercise

Use the provided function to create 3 different sine waves at various frequencies. Plot the 3 functions with labeled axis.

```python
def three_sine_waves(t, A, B, C):
    """
    t : (type: array) an monotonically increasing array of time values
    A,B,C : (type: float) frequency of sine waves
    """
    y1 = sin(A*t)
    y2 = sin(B*t)
    y3 = sin(C*t)

    return y1, y2, y3
```

```python
t =
y1, y2, y3 = three_sine_waves()
```

```python
%load exercise_solutions/n00_python_intro_data-plotting.py
```

## Scipy odeint

Scipy provides a routine for integrating first order ordinary differential equations.

### Setup

```python
from scipy.integrate import odeint
```

### Examples

```python
def dy(y,x):
    return x

y0 = 0.0
x = linspace(-5.0, 5.0, 1000)

y = odeint(dy,y0,x)

plot(x,y)
```

```python
def dy(y,t,coeff):
    A = coeff['A']
    B = coeff['B']
    C = coeff['C']
    D = coeff['D']
    return A*t**3 + B*t**2 + C*t + D

y0 = 2.0
t = linspace(-5.0, 3.0, 1000)
sys = {'A' : 0.25,
       'B' : 0.75,
       'C' : -1.5,
       'D' : -2.0}

y = odeint(dy, y0, t, args=(sys,))

plot(t,y)
```

## SymPy

### Setup

```python
from sympy import *

# This is for prettier printing of equations
interactive.printing.init_printing()
```

### Creating Symbolic Variables

```python
a = symbols('a')
b = symbols('b')
gravity, mass, spring_const, time = symbols('g, m, k, t')
```

### Expressions Using Symbolic Variables

```python
a**2 + b/pi
```

```python
simplify(4*(a*a*a)*(b+b+b))
```

```python
diff(-gravity*time**2/2, time)
```

```python
# indefinte integral

integrate(-gravity,time)
```

```python
# definite integral

v0 = 5
t1 = 0
t2 = .35
position = integrate(-gravity*time + v0,(time,t1,t2))
position.subs(gravity, 9.81)
```

# Additional Resources

[1] http://docs.python.org/2/tutorial

[2] http://nbviewer.ipython.org/github/ipython/ipython/blob/master/examples/notebooks/Cell%20Magics.ipynb

[3] http://www.scipy.org/index.html