## **Surface Crack Detection using Quantum-classical Neural Networks with PyTorch and Qiskit**

CNN is a type of neural network which uses a special operation called convolution and works best on image dataset.

There are a lot of preprocessing methods that can be used in this pipeline.

Convolution is a multiplication operation between an input and a filter. It's not a simple multiplication process, rather, it is computed by multiplying the input with the filter over a delayed period of time.

**Input * filter -> Answer**

As more and more ideas based on Hybrid Learning Environment come up with combining classical and quantum world to solve various problems.
Here is one approach that uses a machine learning framework, PyTorch to create a Hybrid Learning Environment.
 With this Hybrid approach, a system can be created  with classical neural network, but with some of the nodes replaced with quantum circuit.

**Implementation:**

>Step 0 : Installing & Importing Required Python Modules
step 1 : Load the image dataset.
 step 2 : Preprocessing the image dataset.
 step 3 : Then we define the model architecture and add a custom layer for training our model.
 step 4 : Train model for 20 epochs.
 step 5 : Train model to make inference of our test dataset.
 step 6 : Visualize the results of our data

##**Installing Required Modules**

```python
!pip install qiskit
!pip install qiskit-aer-gpu
!pip install qiskit_machine_learning
!pip install torch
!pip install torchvision
!pip install kaggle
```

    Requirement already satisfied: qiskit in /usr/local/lib/python3.7/dist-packages (0.30.0)
    Requirement already satisfied: qiskit-aer==0.9.0 in /usr/local/lib/python3.7/dist-packages (from qiskit) (0.9.0)
    Requirement already satisfied: qiskit-ibmq-provider==0.16.0 in /usr/local/lib/python3.7/dist-packages (from qiskit) (0.16.0)
    Requirement already satisfied: qiskit-ignis==0.6.0 in /usr/local/lib/python3.7/dist-packages (from qiskit) (0.6.0)
    Requirement already satisfied: qiskit-terra==0.18.2 in /usr/local/lib/python3.7/dist-packages (from qiskit) (0.18.2)
    Requirement already satisfied: qiskit-aqua==0.9.5 in /usr/local/lib/python3.7/dist-packages (from qiskit) (0.9.5)
    Requirement already satisfied: pybind11>=2.6 in /usr/local/lib/python3.7/dist-packages (from qiskit-aer==0.9.0->qiskit) (2.7.1)
    Requirement already satisfied: scipy>=1.0 in /usr/local/lib/python3.7/dist-packages (from qiskit-aer==0.9.0->qiskit) (1.4.1)
    Requirement already satisfied: numpy>=1.16.3 in /usr/local/lib/python3.7/dist-packages (from qiskit-aer==0.9.0->qiskit) (1.19.5)
    Requirement already satisfied: yfinance>=0.1.62 in /usr/local/lib/python3.7/dist-packages (from qiskit-aqua==0.9.5->qiskit) (0.1.63)
    Requirement already satisfied: quandl in /usr/local/lib/python3.7/dist-packages (from qiskit-aqua==0.9.5->qiskit) (3.6.1)
    Requirement already satisfied: scikit-learn>=0.20.0 in /usr/local/lib/python3.7/dist-packages (from qiskit-aqua==0.9.5->qiskit) (0.22.2.post1)
    Requirement already satisfied: psutil>=5 in /usr/local/lib/python3.7/dist-packages (from qiskit-aqua==0.9.5->qiskit) (5.4.8)
    Requirement already satisfied: retworkx>=0.8.0 in /usr/local/lib/python3.7/dist-packages (from qiskit-aqua==0.9.5->qiskit) (0.10.2)
    Requirement already satisfied: dlx<=1.0.4 in /usr/local/lib/python3.7/dist-packages (from qiskit-aqua==0.9.5->qiskit) (1.0.4)
    Requirement already satisfied: sympy>=1.3 in /usr/local/lib/python3.7/dist-packages (from qiskit-aqua==0.9.5->qiskit) (1.7.1)
    Requirement already satisfied: h5py<3.3.0 in /usr/local/lib/python3.7/dist-packages (from qiskit-aqua==0.9.5->qiskit) (3.1.0)
    Requirement already satisfied: pandas in /usr/local/lib/python3.7/dist-packages (from qiskit-aqua==0.9.5->qiskit) (1.1.5)
    Requirement already satisfied: fastdtw<=0.3.4 in /usr/local/lib/python3.7/dist-packages (from qiskit-aqua==0.9.5->qiskit) (0.3.4)
    Requirement already satisfied: docplex>=2.21.207 in /usr/local/lib/python3.7/dist-packages (from qiskit-aqua==0.9.5->qiskit) (2.22.213)
    Requirement already satisfied: setuptools>=40.1.0 in /usr/local/lib/python3.7/dist-packages (from qiskit-aqua==0.9.5->qiskit) (57.4.0)
    Requirement already satisfied: urllib3>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from qiskit-ibmq-provider==0.16.0->qiskit) (1.24.3)
    Requirement already satisfied: requests>=2.19 in /usr/local/lib/python3.7/dist-packages (from qiskit-ibmq-provider==0.16.0->qiskit) (2.23.0)
    Requirement already satisfied: python-dateutil>=2.8.0 in /usr/local/lib/python3.7/dist-packages (from qiskit-ibmq-provider==0.16.0->qiskit) (2.8.2)
    Requirement already satisfied: requests-ntlm>=1.1.0 in /usr/local/lib/python3.7/dist-packages (from qiskit-ibmq-provider==0.16.0->qiskit) (1.1.0)
    Requirement already satisfied: websocket-client>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from qiskit-ibmq-provider==0.16.0->qiskit) (1.2.1)
    Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.7/dist-packages (from qiskit-terra==0.18.2->qiskit) (2.6.0)
    Requirement already satisfied: symengine>0.7 in /usr/local/lib/python3.7/dist-packages (from qiskit-terra==0.18.2->qiskit) (0.8.1)
    Requirement already satisfied: fastjsonschema>=2.10 in /usr/local/lib/python3.7/dist-packages (from qiskit-terra==0.18.2->qiskit) (2.15.1)
    Requirement already satisfied: dill>=0.3 in /usr/local/lib/python3.7/dist-packages (from qiskit-terra==0.18.2->qiskit) (0.3.4)
    Requirement already satisfied: tweedledum<2.0,>=1.1 in /usr/local/lib/python3.7/dist-packages (from qiskit-terra==0.18.2->qiskit) (1.1.1)
    Requirement already satisfied: ply>=3.10 in /usr/local/lib/python3.7/dist-packages (from qiskit-terra==0.18.2->qiskit) (3.11)
    Requirement already satisfied: python-constraint>=1.4 in /usr/local/lib/python3.7/dist-packages (from qiskit-terra==0.18.2->qiskit) (1.4.0)
    Requirement already satisfied: six in /usr/local/lib/python3.7/dist-packages (from docplex>=2.21.207->qiskit-aqua==0.9.5->qiskit) (1.15.0)
    Requirement already satisfied: cached-property in /usr/local/lib/python3.7/dist-packages (from h5py<3.3.0->qiskit-aqua==0.9.5->qiskit) (1.5.2)
    Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests>=2.19->qiskit-ibmq-provider==0.16.0->qiskit) (2021.5.30)
    Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests>=2.19->qiskit-ibmq-provider==0.16.0->qiskit) (3.0.4)
    Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests>=2.19->qiskit-ibmq-provider==0.16.0->qiskit) (2.10)
    Requirement already satisfied: ntlm-auth>=1.0.2 in /usr/local/lib/python3.7/dist-packages (from requests-ntlm>=1.1.0->qiskit-ibmq-provider==0.16.0->qiskit) (1.5.0)
    Requirement already satisfied: cryptography>=1.3 in /usr/local/lib/python3.7/dist-packages (from requests-ntlm>=1.1.0->qiskit-ibmq-provider==0.16.0->qiskit) (3.4.8)
    Requirement already satisfied: cffi>=1.12 in /usr/local/lib/python3.7/dist-packages (from cryptography>=1.3->requests-ntlm>=1.1.0->qiskit-ibmq-provider==0.16.0->qiskit) (1.14.6)
    Requirement already satisfied: pycparser in /usr/local/lib/python3.7/dist-packages (from cffi>=1.12->cryptography>=1.3->requests-ntlm>=1.1.0->qiskit-ibmq-provider==0.16.0->qiskit) (2.20)
    Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.7/dist-packages (from scikit-learn>=0.20.0->qiskit-aqua==0.9.5->qiskit) (1.0.1)
    Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.7/dist-packages (from sympy>=1.3->qiskit-aqua==0.9.5->qiskit) (1.2.1)
    Requirement already satisfied: lxml>=4.5.1 in /usr/local/lib/python3.7/dist-packages (from yfinance>=0.1.62->qiskit-aqua==0.9.5->qiskit) (4.6.3)
    Requirement already satisfied: multitasking>=0.0.7 in /usr/local/lib/python3.7/dist-packages (from yfinance>=0.1.62->qiskit-aqua==0.9.5->qiskit) (0.0.9)
    Requirement already satisfied: pytz>=2017.2 in /usr/local/lib/python3.7/dist-packages (from pandas->qiskit-aqua==0.9.5->qiskit) (2018.9)
    Requirement already satisfied: more-itertools in /usr/local/lib/python3.7/dist-packages (from quandl->qiskit-aqua==0.9.5->qiskit) (8.9.0)
    Requirement already satisfied: inflection>=0.3.1 in /usr/local/lib/python3.7/dist-packages (from quandl->qiskit-aqua==0.9.5->qiskit) (0.5.1)
    Requirement already satisfied: qiskit-aer-gpu in /usr/local/lib/python3.7/dist-packages (0.9.0)
    Requirement already satisfied: pybind11>=2.6 in /usr/local/lib/python3.7/dist-packages (from qiskit-aer-gpu) (2.7.1)
    Requirement already satisfied: scipy>=1.0 in /usr/local/lib/python3.7/dist-packages (from qiskit-aer-gpu) (1.4.1)
    Requirement already satisfied: numpy>=1.16.3 in /usr/local/lib/python3.7/dist-packages (from qiskit-aer-gpu) (1.19.5)
    Requirement already satisfied: qiskit-terra>=0.17.0 in /usr/local/lib/python3.7/dist-packages (from qiskit-aer-gpu) (0.18.2)
    Requirement already satisfied: sympy>=1.3 in /usr/local/lib/python3.7/dist-packages (from qiskit-terra>=0.17.0->qiskit-aer-gpu) (1.7.1)
    Requirement already satisfied: tweedledum<2.0,>=1.1 in /usr/local/lib/python3.7/dist-packages (from qiskit-terra>=0.17.0->qiskit-aer-gpu) (1.1.1)
    Requirement already satisfied: retworkx>=0.9.0 in /usr/local/lib/python3.7/dist-packages (from qiskit-terra>=0.17.0->qiskit-aer-gpu) (0.10.2)
    Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.7/dist-packages (from qiskit-terra>=0.17.0->qiskit-aer-gpu) (2.6.0)
    Requirement already satisfied: dill>=0.3 in /usr/local/lib/python3.7/dist-packages (from qiskit-terra>=0.17.0->qiskit-aer-gpu) (0.3.4)
    Requirement already satisfied: symengine>0.7 in /usr/local/lib/python3.7/dist-packages (from qiskit-terra>=0.17.0->qiskit-aer-gpu) (0.8.1)
    Requirement already satisfied: python-constraint>=1.4 in /usr/local/lib/python3.7/dist-packages (from qiskit-terra>=0.17.0->qiskit-aer-gpu) (1.4.0)
    Requirement already satisfied: fastjsonschema>=2.10 in /usr/local/lib/python3.7/dist-packages (from qiskit-terra>=0.17.0->qiskit-aer-gpu) (2.15.1)
    Requirement already satisfied: python-dateutil>=2.8.0 in /usr/local/lib/python3.7/dist-packages (from qiskit-terra>=0.17.0->qiskit-aer-gpu) (2.8.2)
    Requirement already satisfied: psutil>=5 in /usr/local/lib/python3.7/dist-packages (from qiskit-terra>=0.17.0->qiskit-aer-gpu) (5.4.8)
    Requirement already satisfied: ply>=3.10 in /usr/local/lib/python3.7/dist-packages (from qiskit-terra>=0.17.0->qiskit-aer-gpu) (3.11)
    Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/dist-packages (from python-dateutil>=2.8.0->qiskit-terra>=0.17.0->qiskit-aer-gpu) (1.15.0)
    Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.7/dist-packages (from sympy>=1.3->qiskit-terra>=0.17.0->qiskit-aer-gpu) (1.2.1)
    Requirement already satisfied: qiskit_machine_learning in /usr/local/lib/python3.7/dist-packages (0.2.1)
    Requirement already satisfied: qiskit-terra>=0.18.0 in /usr/local/lib/python3.7/dist-packages (from qiskit_machine_learning) (0.18.2)
    Requirement already satisfied: setuptools>=40.1.0 in /usr/local/lib/python3.7/dist-packages (from qiskit_machine_learning) (57.4.0)
    Requirement already satisfied: scipy>=1.4 in /usr/local/lib/python3.7/dist-packages (from qiskit_machine_learning) (1.4.1)
    Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.7/dist-packages (from qiskit_machine_learning) (1.19.5)
    Requirement already satisfied: fastdtw in /usr/local/lib/python3.7/dist-packages (from qiskit_machine_learning) (0.3.4)
    Requirement already satisfied: scikit-learn>=0.20.0 in /usr/local/lib/python3.7/dist-packages (from qiskit_machine_learning) (0.22.2.post1)
    Requirement already satisfied: psutil>=5 in /usr/local/lib/python3.7/dist-packages (from qiskit_machine_learning) (5.4.8)
    Requirement already satisfied: symengine>0.7 in /usr/local/lib/python3.7/dist-packages (from qiskit-terra>=0.18.0->qiskit_machine_learning) (0.8.1)
    Requirement already satisfied: retworkx>=0.9.0 in /usr/local/lib/python3.7/dist-packages (from qiskit-terra>=0.18.0->qiskit_machine_learning) (0.10.2)
    Requirement already satisfied: sympy>=1.3 in /usr/local/lib/python3.7/dist-packages (from qiskit-terra>=0.18.0->qiskit_machine_learning) (1.7.1)
    Requirement already satisfied: tweedledum<2.0,>=1.1 in /usr/local/lib/python3.7/dist-packages (from qiskit-terra>=0.18.0->qiskit_machine_learning) (1.1.1)
    Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.7/dist-packages (from qiskit-terra>=0.18.0->qiskit_machine_learning) (2.6.0)
    Requirement already satisfied: fastjsonschema>=2.10 in /usr/local/lib/python3.7/dist-packages (from qiskit-terra>=0.18.0->qiskit_machine_learning) (2.15.1)
    Requirement already satisfied: python-constraint>=1.4 in /usr/local/lib/python3.7/dist-packages (from qiskit-terra>=0.18.0->qiskit_machine_learning) (1.4.0)
    Requirement already satisfied: dill>=0.3 in /usr/local/lib/python3.7/dist-packages (from qiskit-terra>=0.18.0->qiskit_machine_learning) (0.3.4)
    Requirement already satisfied: python-dateutil>=2.8.0 in /usr/local/lib/python3.7/dist-packages (from qiskit-terra>=0.18.0->qiskit_machine_learning) (2.8.2)
    Requirement already satisfied: ply>=3.10 in /usr/local/lib/python3.7/dist-packages (from qiskit-terra>=0.18.0->qiskit_machine_learning) (3.11)
    Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/dist-packages (from python-dateutil>=2.8.0->qiskit-terra>=0.18.0->qiskit_machine_learning) (1.15.0)
    Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.7/dist-packages (from scikit-learn>=0.20.0->qiskit_machine_learning) (1.0.1)
    Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.7/dist-packages (from sympy>=1.3->qiskit-terra>=0.18.0->qiskit_machine_learning) (1.2.1)
    Requirement already satisfied: torch in /usr/local/lib/python3.7/dist-packages (1.9.0+cu102)
    Requirement already satisfied: typing-extensions in /usr/local/lib/python3.7/dist-packages (from torch) (3.7.4.3)
    Requirement already satisfied: torchvision in /usr/local/lib/python3.7/dist-packages (0.10.0+cu102)
    Requirement already satisfied: pillow>=5.3.0 in /usr/local/lib/python3.7/dist-packages (from torchvision) (7.1.2)
    Requirement already satisfied: torch==1.9.0 in /usr/local/lib/python3.7/dist-packages (from torchvision) (1.9.0+cu102)
    Requirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from torchvision) (1.19.5)
    Requirement already satisfied: typing-extensions in /usr/local/lib/python3.7/dist-packages (from torch==1.9.0->torchvision) (3.7.4.3)
    Requirement already satisfied: kaggle in /usr/local/lib/python3.7/dist-packages (1.5.12)
    Requirement already satisfied: certifi in /usr/local/lib/python3.7/dist-packages (from kaggle) (2021.5.30)
    Requirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from kaggle) (2.23.0)
    Requirement already satisfied: tqdm in /usr/local/lib/python3.7/dist-packages (from kaggle) (4.62.2)
    Requirement already satisfied: six>=1.10 in /usr/local/lib/python3.7/dist-packages (from kaggle) (1.15.0)
    Requirement already satisfied: urllib3 in /usr/local/lib/python3.7/dist-packages (from kaggle) (1.24.3)
    Requirement already satisfied: python-dateutil in /usr/local/lib/python3.7/dist-packages (from kaggle) (2.8.2)
    Requirement already satisfied: python-slugify in /usr/local/lib/python3.7/dist-packages (from kaggle) (5.0.2)
    Requirement already satisfied: text-unidecode>=1.3 in /usr/local/lib/python3.7/dist-packages (from python-slugify->kaggle) (1.3)
    Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->kaggle) (2.10)
    Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests->kaggle) (3.0.4)

##**Importing Modules**

```python
## If local dataset is not available
# ! mkdir ~/.kaggle                           # Make directory named kaggle
# ! cp kaggle.json ~/.kaggle/                 # Copy kaggle.json file there
# ! chmod 600 ~/.kaggle/kaggle.json           # Change the permissions of the file.
```

```python
# Importing Required Python Modules
import numpy as np
import os
cwd = os.getcwd()
from PIL import Image
import time
import copy
import random
import cv2
import re
import shutil
import matplotlib.pyplot as plt
import matplotlib.image as mpimg
%matplotlib inline

#Importing Pytorch Specific Modules
import torch
from torch.autograd import Function
import torch.nn as nn
import torch.optim as optim
from torch.utils.data import DataLoader,ConcatDataset
from torch.utils.data import sampler
from torch.optim import lr_scheduler
from torchvision import datasets, transforms,models
import torch.nn.functional as F
import torchvision
from torchvision.datasets import MNIST
from sklearn.model_selection import KFold

#Importing Qiskit Specific Modules
import qiskit
from qiskit import transpile, assemble
from qiskit.visualization import *
from qiskit  import Aer, QuantumCircuit
from qiskit.utils import QuantumInstance
from qiskit.circuit import Parameter
## If local dataset is not available then uncomment below line
#import kaggle
```

#**Checking for GPU Availability**

```python
simulator = qiskit.Aer.get_backend('aer_simulator')

use_cuda = torch.cuda.is_available()

print('CUDA available:', use_cuda)

if use_cuda:
    device = torch.device('cuda')
    print('Training on GPU...')
    simulator.set_options(device='GPU')
else:
    device = torch.device('cpu')
    print('Training on CPU...')
```

    CUDA available: False
    Training on CPU...

## **Step 1: Load the Dataset and Visualize it**
* Link to the cracks data set:  https://www.kaggle.com/arunrk7/surface-crack-detection/metadata

```python
!kaggle datasets download -d arunrk7/surface-crack-detection     # Downloading the zip file using Kaggle Credentials
```

    surface-crack-detection.zip: Skipping, found more recently modified local copy (use --force to force download)

```python
!unzip surface-crack-detection.zip                            # Unzip the data
from IPython.display import clear_output
clear_output()
```

```python
crack_images = os.listdir('Positive/')
print("Number of Crack Images: ", len(crack_images))
```

    Number of Crack Images:  20000

```python
no_crack_images = os.listdir('Negative/')
print("Number of No Crack Images: ", len(no_crack_images))
```

    Number of No Crack Images:  20000

Visualize Random images with cracks

```python
random_indices = np.random.randint(0, len(crack_images), size=4)
print("*****************Random Images with Cracks**************************")
random_images = np.array(crack_images)[random_indices.astype(int)]

f, axarr = plt.subplots(2,2)
axarr[0,0].imshow(mpimg.imread(os.path.join(cwd, 'Positive', random_images[0])))
axarr[0,1].imshow(mpimg.imread(os.path.join(cwd, 'Positive', random_images[1])))
axarr[1,0].imshow(mpimg.imread(os.path.join(cwd, 'Positive', random_images[2])))
axarr[1,1].imshow(mpimg.imread(os.path.join(cwd, 'Positive', random_images[3])))
```

Visualize Random images with no cracks

```python
random_indices = np.random.randint(0, len(no_crack_images), size=4)
print("*****************Random Images without Cracks**************************")
random_images = np.array(no_crack_images)[random_indices.astype(int)]

f, axarr = plt.subplots(2,2)
axarr[0,0].imshow(mpimg.imread(os.path.join(cwd, 'Negative', random_images[0])))
axarr[0,1].imshow(mpimg.imread(os.path.join(cwd, 'Negative', random_images[1])))
axarr[1,0].imshow(mpimg.imread(os.path.join(cwd, 'Negative', random_images[2])))
axarr[1,1].imshow(mpimg.imread(os.path.join(cwd, 'Negative', random_images[3])))
```

## **Create Train and Val Data sets**

### Create training folder

```python
!rm -rf train
base_dir = cwd
files = os.listdir(base_dir)
```

```python
##Considering only 1000 images from each class in order to restrict Compute resource utilization
def create_training_data(folder_name):
    files_list = os.listdir(os.path.join(base_dir,folder_name))
    files_list = files_list[0:1000]
    train_dir = f"{base_dir}/train/{folder_name}"
    os.makedirs(train_dir)
    for f in files:
          search_object = re.search(folder_name, f)
          if search_object:
            for file_name in files_list:
              shutil.copy(f'{base_dir}/{folder_name}/{file_name}', train_dir)
```

```python
create_training_data('Positive')
create_training_data('Negative')
!rm -rf val
```

### Move images randomly from training to val folders

```python
os.makedirs('val/Positive')
os.makedirs('val/Negative')
```

```python
positive_train = base_dir + "/train/Positive/"
positive_val = base_dir + "/val/Positive/"
negative_train = base_dir + "/train/Negative/"
negative_val = base_dir + "/val/Negative/"

positive_files = os.listdir(positive_train)
negative_files = os.listdir(negative_train)
```

```python
print(len(positive_files), len(negative_files))             # Checking for the total count of file to be unchanged. Expected output: 20000 20000
```

    1000 1000

```python
for f in positive_files:
    if random.random() > 0.80:
        shutil.move(f'{positive_train}/{f}', positive_val)   # moving files to positivr_val
```

```python
for f in negative_files:
    if random.random() > 0.80:
        shutil.move(f'{negative_train}/{f}', negative_val)   # moving files to negative_val
```

## **Step 2: Preprocessing the Image Dataset**
## Pytorch data loader and Transforms

### Compute mean and std deviation for the dataset

There are 3 channels in a RGB IMAGE.
> 1. Red
2. Green
3. Blue

```python
mean_nums = [0.5, 0.5, 0.5]
std_nums = [0.5, 0.5, 0.5]
```

### Define Data Augmentation and Transforms

```python
import torchvision.transforms
```

```python
data_path_train = os.path.join(cwd, 'train')
data_path_test = os.path.join(cwd, 'val')
chosen_transforms = {'train': transforms.Compose([
        transforms.RandomResizedCrop(size=227),
        transforms.RandomRotation(degrees=10),
        transforms.RandomHorizontalFlip(),
        transforms.RandomVerticalFlip(),
        transforms.ColorJitter(brightness=0.15, contrast=0.15),
        transforms.ToTensor(),
        transforms.Normalize(mean_nums, std_nums)
]), 'val': transforms.Compose([
        transforms.Resize(227),
        transforms.CenterCrop(227),
        transforms.ToTensor(),
        transforms.Normalize(mean_nums, std_nums)
]),
}
```

### Create the Data Loader

```python
def load_dataset(format, batch_size):
    data_path = os.path.join(cwd, format)
    dataset = datasets.ImageFolder(
        root=data_path,
        transform= chosen_transforms[format]
    )
    data_loader = DataLoader(
        dataset,
        batch_size=batch_size,
        num_workers=2,
        shuffle=True
    )
    return data_loader, len(dataset), dataset.classes
```

```python
train_loader, train_size, class_names = load_dataset('train', 8)
print("Train Data Set size is: ", train_size)
print("Class Names are: ", class_names)
inputs, classes = next(iter(train_loader))
print(inputs.shape, classes.shape)
```

    Train Data Set size is:  1578
    Class Names are:  ['Negative', 'Positive']
    torch.Size([8, 3, 227, 227]) torch.Size([8])

```python
test_loader, test_size, class_names = load_dataset('val', 8)
print("Test Data Set size is: ", test_size)
print("Class Names are: ", class_names)
inputs_test, classes_test = next(iter(test_loader))
print(inputs_test.shape, classes_test.shape)
```

    Test Data Set size is:  422
    Class Names are:  ['Negative', 'Positive']
    torch.Size([8, 3, 227, 227]) torch.Size([8])

### Visualize the Augmented Data Set

```python
# Visualize some images
def imshow(inp, title=None):
    inp = inp.numpy().transpose((1, 2, 0))
    mean = np.array([mean_nums])
    std = np.array([std_nums])
    inp = std * inp + mean
    inp = np.clip(inp, 0, 1)
    plt.imshow(inp)
    if title is not None:
        plt.title(title)
    plt.pause(0.001)  # Pause a bit so that plots are updated
```

```python
# Grab some of the training data to visualize
inputs, classes = next(iter(train_loader))         # class_names = chosen_datasets['train'].classes

out = torchvision.utils.make_grid(inputs)          # Now we construct a grid from batch

idx_to_class = {0:'Negative', 1:'Positive'}
plt.figure(figsize=(20,10))
imshow(out, title=[x.data.numpy() for x in classes])
```

## **Step 3: Define the architecture to train the model**

##Define Quantum Circuit and Layer

```python
class QuantumCircuit:
    """
    This class provides a simple interface for interaction
    with the quantum circuit
    """

    def __init__(self, n_qubits, backend, shots):
        # --- Circuit definition ---
        self._circuit = qiskit.QuantumCircuit(n_qubits)

        all_qubits = [i for i in range(n_qubits)]
        self.theta = qiskit.circuit.Parameter('theta')

        self._circuit.h(all_qubits)
        self._circuit.barrier()
        self._circuit.ry(self.theta, all_qubits)

        self._circuit.measure_all()
        # ---------------------------

        self.backend = backend
        self.shots = shots

    def run(self, thetas):
        t_qc = transpile(self._circuit,
                         self.backend)
        qobj = assemble(t_qc,
                        shots=self.shots,
                        parameter_binds = [{self.theta: theta} for theta in thetas])
        job = self.backend.run(qobj)
        result_circ = job.result().get_counts()
        y = list(map(int, result_circ.values()))
        # counts = np.array(list(result_circ.values()))
        counts= np.array(y)
        states = np.array(list(result_circ.keys())).astype(float)

        # Compute probabilities for each state
        probabilities = counts / self.shots
        # Get state expectation
        expectation = np.sum(states * probabilities)

        return np.array([expectation])
```

Let's test the implementation

```python
circuit = QuantumCircuit(1, simulator, 100)
print('Expected value for rotation pi {}'.format(circuit.run([np.pi])[0]))
circuit._circuit.draw()
```

    /usr/local/lib/python3.7/dist-packages/qiskit/utils/deprecation.py:62: DeprecationWarning: Using a qobj for run() is deprecated as of qiskit-aer 0.9.0 and will be removed no sooner than 3 months from that release date. Transpiled circuits should now be passed directly using `backend.run(circuits, **run_options).
      return func(*args, **kwargs)

    Expected value for rotation pi 0.43

        ┌───┐ ░ ┌───────────┐ ░ ┌─┐
   q_0: ┤ H ├─░─┤ Ry(theta) ├─░─┤M├
        └───┘ ░ └───────────┘ ░ └╥┘
meas: 1/═════════════════════════╩═
                                 0

### Creating a "Quantum-Classical Class" with PyTorch

```python
class HybridFunction(Function):
    """ Hybrid quantum - classical function definition """

    @staticmethod
    def forward(ctx, inputs, quantum_circuit, shift):
        """ Forward pass computation """
        ctx.shift = shift
        ctx.quantum_circuit = quantum_circuit

        expectation_z = []
        for input in inputs:
            expectation_z.append(ctx.quantum_circuit.run(input.tolist()))
        result = torch.tensor(expectation_z).to(device)

        ctx.save_for_backward(inputs, result)
        return result

    @staticmethod
    def backward(ctx, grad_output):
        """ Backward pass computation """
        input, expectation_z = ctx.saved_tensors
        input_list = np.array(input.tolist())

        shift_right = input_list + np.ones(input_list.shape) * ctx.shift
        shift_left = input_list - np.ones(input_list.shape) * ctx.shift

        gradients = []
        for i in range(len(input_list)):
            expectation_right = ctx.quantum_circuit.run(shift_right[i])
            expectation_left  = ctx.quantum_circuit.run(shift_left[i])

            gradient = torch.tensor([expectation_right]).to(device) - torch.tensor([expectation_left]).to(device)
            gradients.append(gradient)

        # gradients = np.array([gradients]).T
        gradients = torch.tensor([gradients]).to(device)
        gradients = torch.transpose(gradients, 0, 1)

        # return torch.tensor([gradients]).float() * grad_output.float(), None, None
        return gradients.float() * grad_output.float(), None, None

class Hybrid(nn.Module):
    """ Hybrid quantum - classical layer definition """

    def __init__(self, backend, shots, shift):
        super(Hybrid, self).__init__()
        self.quantum_circuit = QuantumCircuit(1, backend, shots)
        self.shift = shift

    def forward(self, input):
        return HybridFunction.apply(input, self.quantum_circuit, self.shift)
```

```python
class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.conv1 = nn.Conv2d(3, 16, kernel_size=3)
        self.conv2 = nn.Conv2d(16, 32, kernel_size=3)
        self.dropout = nn.Dropout2d(p=0.5)
        self.fc1 = nn.Linear(1152, 256)
        self.fc2 = nn.Linear(256, 64)
        self.fc3 = nn.Linear(64, 2)
        self.hybrid = [Hybrid(simulator, 100, np.pi / 2) for i in range(2)]

    def forward(self, x):
        x = F.relu(self.conv1(x))
        x = F.max_pool2d(x, 8)
        x = F.relu(self.conv2(x))
        x = F.max_pool2d(x, 4)
        x = self.dropout(x)
        # x = x.view(-1, 256)
        x = torch.flatten(x, start_dim=1)
        x = F.relu(self.fc1(x))
        x = self.fc2(x)
        x=  self.fc3(x)
        x = torch.chunk(x, 10, dim=1)
        x = tuple([hy(x_) for hy, x_ in zip(self.hybrid, x)])
        return torch.cat(x, -1)
```

```python
# Convert model to be used on GPU
model = Net().to(device)
from torchsummary import summary
print(summary(model, (3, 227, 227)))
```

    ----------------------------------------------------------------
            Layer (type)               Output Shape         Param #
    ================================================================
                Conv2d-1         [-1, 16, 225, 225]             448
                Conv2d-2           [-1, 32, 26, 26]           4,640
             Dropout2d-3             [-1, 32, 6, 6]               0
                Linear-4                  [-1, 256]         295,168
                Linear-5                   [-1, 64]          16,448
                Linear-6                    [-1, 2]             130
    ================================================================
    Total params: 316,834
    Trainable params: 316,834
    Non-trainable params: 0
    ----------------------------------------------------------------
    Input size (MB): 0.59
    Forward/backward pass size (MB): 6.36
    Params size (MB): 1.21
    Estimated Total Size (MB): 8.15
    ----------------------------------------------------------------
    None

    /usr/local/lib/python3.7/dist-packages/torch/nn/functional.py:718: UserWarning: Named tensors and all their associated APIs are an experimental feature and subject to change. Please do not use them for anything important until they are released as stable. (Triggered internally at  /pytorch/c10/core/TensorImpl.h:1156.)
      return torch.max_pool2d(input, kernel_size, stride, padding, dilation, ceil_mode)
    /usr/local/lib/python3.7/dist-packages/qiskit/utils/deprecation.py:62: DeprecationWarning: Using a qobj for run() is deprecated as of qiskit-aer 0.9.0 and will be removed no sooner than 3 months from that release date. Transpiled circuits should now be passed directly using `backend.run(circuits, **run_options).
      return func(*args, **kwargs)

## **Step 4: Training our Model**

```python
dataloaders = {}
dataset_sizes = {}
batch_size = 256
dataloaders['train'], dataset_sizes['train'], class_names = load_dataset('train', batch_size)
dataloaders['val'], dataset_sizes['val'], _ = load_dataset('val', batch_size)
idx_to_class = {0:'Negative', 1:'Positive'}

```

```python
def train_model(model, criterion, optimizer, scheduler, num_epochs=20):
    since = time.time()

    best_model_wts = copy.deepcopy(model.state_dict())
    best_acc = 0.0
    loss_list = []
    train_acc = []
    val_acc =[]
    for epoch in range(num_epochs):
        print('Epoch {}/{}'.format(epoch, num_epochs - 1))
        print('-' * 10)

        # Each epoch has a training and validation phase
        for phase in ['train', 'val']:
            if phase == 'train':
                scheduler.step()
                acc_hist = train_acc
                model.train()  # Set model to training mode
            else:
                model.eval()   # Set model to evaluate mode
                acc_hist = val_acc
            current_loss = 0.0
            current_corrects = 0

            # Here's where the training happens
            print('Iterating through data...')
            # training starts...
            total_loss = []
            for inputs, labels in dataloaders[phase]:
                inputs = inputs.to(device)
                labels = labels.to(device)

                # We need to zero the gradients, don't forget it
                # Re-setting all of model parameters

                optimizer.zero_grad()

                # Time to carry out the forward training poss
                # We only need to log the loss stats if we are in training phase
                with torch.set_grad_enabled(phase == 'train'):
                    outputs = model(inputs).to(device)
                    # this computes the class that the model belongs to...
                    _, preds = torch.max(outputs, 1)

                    # compute the loss here...
                    # we compute the difference between true label and the predicted output...
                    loss = criterion(outputs, labels)

                    # backward + optimize only if in training phase
                    if phase == 'train':
                        loss.backward() # this is the back-propragation step. we

                        # minimize our model loss and optimize model in this phase
                        optimizer.step()
                        # optimizer updates the model paramters...

                # We want variables to hold the loss statistics
                # combine our model losses and compute the wrongly classified examples
                current_loss += loss.item() * inputs.size(0)
                current_corrects += torch.sum(preds == labels.data)
                total_loss.append(loss.item())

            # compute the loss and accuracy for each of our epoch
            epoch_loss = current_loss / dataset_sizes[phase]
            epoch_acc = current_corrects.double() / dataset_sizes[phase]
            acc_hist.append(epoch_acc)
            loss_list.append(sum(total_loss)/len(total_loss))
            print('{} Loss: {:.4f} Acc: {:.4f}'.format(
                phase, epoch_loss, epoch_acc))

            # Make a copy of the model if the accuracy on the validation set has improved
            if phase == 'val' and epoch_acc > best_acc:
                best_acc = epoch_acc
                best_model_wts = copy.deepcopy(model.state_dict())

        print()

    time_since = time.time() - since
    print('Training complete in {:.0f}m {:.0f}s'.format(
        time_since // 60, time_since % 60))
    print('Best val Acc: {:4f}'.format(best_acc))

    #Plotting the training graph
    plt.figure(figsize = (15,10))
    plt.plot(loss_list)
    plt.title('Convolutional NN Training Convergence')
    plt.xlabel('Training Iterations')
    plt.ylabel('Neg Log Likelihood Loss')
    plt.show()

    #Plotting the learning curve - Accuracy
    plt.figure(figsize = (15,10))
    plt.title('Training Vs Validation Accuracy')
    plt.plot(train_acc, label='train_accuracy')
    plt.plot(val_acc, label='val_accuracy')
    plt.legend(loc='best')
    plt.show()
    # Now we'll load in the best model weights and return it
    model.load_state_dict(best_model_wts)
    return model

```

```python
def visualize_model(model, num_images=6):
    was_training = model.training
    model.eval()
    images_handeled = 0
    fig = plt.figure()

    with torch.no_grad():
        for i, (inputs, labels) in enumerate(dataloaders['val']):
            inputs = inputs.to(device)
            labels = labels.to(device)

            outputs = model(inputs).to(device)
            _, preds = torch.max(outputs, 1)

            for j in range(inputs.size()[0]):
                images_handeled += 1
                ax = plt.subplot(num_images//2, 2, images_handeled)
                ax.axis('off')
                ax.set_title('predicted: {}'.format(class_names[preds[j]]))
                imshow(inputs.cpu().data[j])

                if images_handeled == num_images:
                    model.train(mode=was_training)
                    return
        model.train(mode=was_training)
```

###**Define Optimizer and Loss Function**

Learning rate decay- As we train the model, we can decide to change the learning rate dynamically. The schedular helps us in doing that.

In the 1st epoch, the learning rate was 0.1, then the learning rate in epoch 4 would be 0.01.

```python
criterion = nn.CrossEntropyLoss().to(device)
optimizer = optim.Adam(model.parameters())
exp_lr_scheduler = lr_scheduler.StepLR(optimizer, step_size=3, gamma=0.1)
```

### **Start training**

```python
base_model = train_model(model, criterion, optimizer, exp_lr_scheduler, num_epochs=20)
visualize_model(base_model)
plt.show()
# # an epoch is the total time during which one training step completes. In other words, it is the total
# # time that our model takes to process the input data once...
```

## **Step 5: Using trained model to make inference of our test dataset**

```python
def predict(model, test_image, print_class = False):
     # it uses the model to predict on test_image...
    transform = chosen_transforms['val']

    test_image_tensor = transform(test_image)
    if torch.cuda.is_available(): # checks if we have a gpu available
        test_image_tensor = test_image_tensor.view(1, 3, 227, 227).to(device)
    else:
        test_image_tensor = test_image_tensor.view(1, 3, 227, 227)

    with torch.no_grad():
        model.eval()
        # Model outputs log probabilities
        # this computes the output of the model
        out = model(test_image_tensor).to(device)
        # this computes the probability of each classes.
        ps = torch.exp(out)
        # we choose the top class. That is, the class with highest probability
        topk, topclass = ps.topk(1, dim=1)
        class_name = idx_to_class[topclass.cpu().numpy()[0][0]]
        if print_class:
            print("Output class :  ", class_name)
    return class_name
```

```python
def predict_on_crops(input_image, height=32, width=32, save_crops = False):
    im = cv2.imread(input_image)
    imgheight, imgwidth, channels = im.shape
    k=0
    output_image = np.zeros_like(im)
    for i in range(0,imgheight,height):
        for j in range(0,imgwidth,width):
            a = im[i:i+height, j:j+width]
            ## discard image cropss that are not full size
            predicted_class = predict(base_model,Image.fromarray(a))
            ## save image
            file, ext = os.path.splitext(input_image)
            image_name = file.split('/')[-1]
            folder_name = 'out_' + image_name
            ## Put predicted class on the image
            if predicted_class == 'Positive':
                color = (0,0, 255)
            else:
                color = (0, 255, 0)
            cv2.putText(a, predicted_class, (50,50), cv2.FONT_HERSHEY_SIMPLEX , 0.7, color, 1, cv2.LINE_AA)
            b = np.zeros_like(a, dtype=np.uint8)
            b[:] = color
            add_img = cv2.addWeighted(a, 0.9, b, 0.1, 0)
            ## Save crops
            if save_crops:
                if not os.path.exists(os.path.join('real_images', folder_name)):
                    os.makedirs(os.path.join('real_images', folder_name))
                filename = os.path.join('real_images', folder_name,'img_{}.png'.format(k))
                cv2.imwrite(filename, add_img)
            output_image[i:i+height, j:j+width,:] = add_img
            k+=1
    ## Save output image
    cv2.imwrite(os.path.join('real_images','predictions', folder_name+ '.jpg'), output_image)
    return output_image
```

```python
# Specify a path
PATH = "qnn_state_dict_model.pt"

# Save
torch.save(base_model.state_dict(), PATH)
# torch.save(model.state_dict(), PATH)
```

## **Step 6:  Visualize the results of our data**

```python
plt.figure()
output_image = predict_on_crops('/content/val/Negative/00097.jpg')
plt.imshow(cv2.cvtColor(output_image, cv2.COLOR_BGR2RGB))
```

```python
plt.figure()
output_image = predict_on_crops('/content/train/Positive/00043.jpg')
plt.imshow(cv2.cvtColor(output_image, cv2.COLOR_BGR2RGB))
```

```python
plt.figure()
output_image = predict_on_crops('/content/val/Negative/01773.jpg')
plt.imshow(cv2.cvtColor(output_image, cv2.COLOR_BGR2RGB))
```

```python

```


# Idaho Policy Institute (IPI) Report
* _This copy is not intended to be compiled, some models may become inconsistent with documentation._

Exploring & Modeling of Municipial Dataset: This is a project presented to my partners and me by our client, The Idaho Policy Institute, In which they provided a dataset containing information on finances, population and crime for virtually every city and/or county in Idaho. The goal of the project was to find relationships between financial decisions with crime and employment trends in cities of Idaho with efferent sizes.

**Notebook Author**

>Dominik Huffield

**Contributors**

>Sammy Bhushan

Amir Abbas Kazemzadeh Farizhandi

### Importing libraries

```python
import support.load_data as load
import support.supporting_funcs as funcs
# import support.plotting_funcs as ipiplot

import pandas as pd
import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt
import matplotlib.lines as mlines
from sklearn.model_selection import train_test_split
from sklearn.feature_selection import SelectKBest, chi2
from sklearn.feature_selection import f_regression
from sklearn.linear_model import LassoCV
from linearmodels.panel import PanelOLS
import sympy
from sklearn.metrics import accuracy_score
from sklearn.metrics import r2_score
from sklearn.metrics import mean_squared_log_error
from sklearn.linear_model import BayesianRidge
from sklearn.ensemble import RandomForestRegressor
from statsmodels.stats.outliers_influence import variance_inflation_factor
from statsmodels.tools.tools import add_constant

from sklearn.pipeline import Pipeline
from sklearn.model_selection import GridSearchCV
```

> If local enviorment does not contain any of the modules then the syntax to download through jupyter is

`!conda install [package]`

       or

`!pip install [package]`

Installation instructions may vary and the official docs should be referenced

## Dataset Classification Manual

Along with the dataset IPI also provided a manual to navigate it's contents, held within this projects repository. This may need to be referenced to understand what some columns are measuring for they are abbreviated however, this notebook should be sufficient in explaining itself.

## Data Sources

We felt this dataset was useful relevant information e.g. employment and business statistics. After importing we merged these tables with the one provided by IPI, or in one case made an additional table. Listed below are all additional sources.

> * U.S. Census Bureau (USC)

        Provides population measurments at various entity levels including state, county and city
* Willamette Government Finance Database (GFD)
        Provides goverment cash flow information for each city/county in Idaho
* Federal Bureau of Investigation (FBI)
        Describes various crime variables
* Bureau of Labor Statistics (BLS)
        Allows for adjustment for inflation and provides employment related statistics

## Functions We Created

We created many functions and methods to augment our work such as converting to real dollars using the current year as a parameter. We also did this to regularize some of our variables. Furthermore there are many plotting functions such as heat maps, matricies of distributions and scatter plots, choropleth maps , etc. These methods and functions are all stored within the support folder and have their own documentation. These were hidden for aesthetics and need not be read to understand the process of this notebook.

## Exploring Data

The data given to us by IPI was panel data, Cities or counties being the major axis while the minor axis was indexed by years. There are 200 cities in the set ranging over about 20 years. We clearly had an unbalanced panel for most cities & counties were missing data for serveral years with little consistency amongst themselves. Luckily we were able to identify four years that had data for a decent amount of variables, so we chose to keep those years and remove the rest. This is shown in more depth further down the notebook.

Along with poor years there were also many cities that contained a small amount of data. We did not incorporate these within our models and the method of removal is shown below. Further more there was not a useful quantity of data at the county level so these variables were removed and the process is stored within a support funtion.

The financial data was collected in nominal dollars. We used data from BLS to account for inflation and converted these values into real dollars base 2019 so that we would have the best understanding of the value of each dollar. This was done and stored in one of the support functions.

## Objective for Models

The amount of columns contained within the set describing crime amounted to 7 however there were over 500 variables describing financial revenue or expenditure. Using this knowledge we thought it best to attempt to describe crime using financial information as our independent variables (along with additional information). We chose to do this so that we would not limit the features available for our models along with the ability of this type of model to provide a good description of the relation between government finances and crime. The variables describing crime were quantities of occurances so to use this as our dependent variable we needed to construct a regression model.

A future iteration of this project one might create a classifier that describes whether an increase or decrease in a feature will result in an increase or decease in crime. We do not address this effect of each feature directly to our dependent as to avoid feature phishing.

**Selecting A Model**

Since this a panel dataset we wanted to account for the serial correlation resulting from the time series. Additionally we thought a linear model the easiest to interpret and understand the inner workings of the data itself. This lead us to a time effects model for our initial build, however the world is not linear and features will likely contain heteroskedasticity so past comprehension a non-linear approach will likely be a more robust and accurate model.

## Load Dataset

Using one of our functions within the support folder, we will take the dataset and convert the financial values from nominal to real dollars creating a dataframe. We use the CPI for 2019 but future analysis should adjust to the most recent CPI value to allow for full comprehension of a dollar.

```python
df = load.all_data(norm=False)
```

    Loading IPI data
    Adjusting for Inflation
    Getting GPS
    Getting Employees
    Merge Everything
    Categorize City Size

## Understanding Data Structure

```python
df['Year4'].sort_values().unique()
```

    array([1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005,
           2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014])

As you can see the range of the time series contains values from 1995 to 2014

```python
df['Name'].unique().size
```

    200

Along the major axis there are 200 different cities

```python
df.set_index(['Name', 'Year4']).loc[('BOISE CITY',np.arange(1995, 2016)), :]
```

      County
      YearofData
      SurveyYr
      ID
      IDChanged
      Type_Code
      County
      FIPS_Code_State
      FIPS_County
      FIPS_Place
      ...
      County/State
      County FIPS Year
      County Labor Force
      County Labor Employed
      County Labor Unemployed
      Unemployment Rate (%)
      postalcode
      latitude
      longitude
      size

      Name
      Year4

      BOISE CITY
      1996
      Ada
      96.0
      96.0
      132001001.0
      0.0
      2.0
      Ada
      16.0
      1.0
      8830.0
      ...
      Ada County, ID
      1996.0
      150733.0
      145670.0
      5063.0
      3.4
      83701
      43.6038
      -116.2729
      urban

      1997
      Ada
      97.0
      97.0
      132001001.0
      0.0
      2.0
      Ada
      16.0
      1.0
      8830.0
      ...
      Ada County, ID
      1997.0
      156552.0
      151759.0
      4793.0
      3.1
      83701
      43.6038
      -116.2729
      urban

      1998
      Ada
      98.0
      98.0
      132001001.0
      0.0
      2.0
      Ada
      16.0
      1.0
      8830.0
      ...
      Ada County, ID
      1998.0
      163205.0
      157989.0
      5216.0
      3.2
      83701
      43.6038
      -116.2729
      urban

      1999
      Ada
      99.0
      99.0
      132001001.0
      0.0
      2.0
      Ada
      16.0
      1.0
      8830.0
      ...
      Ada County, ID
      1999.0
      166095.0
      160959.0
      5136.0
      3.1
      83701
      43.6038
      -116.2729
      urban

      2000
      Ada
      0.0
      0.0
      132001001.0
      0.0
      2.0
      Ada
      16.0
      1.0
      8830.0
      ...
      Ada County, ID
      2000.0
      169932.0
      164171.0
      5761.0
      3.4
      83701
      43.6038
      -116.2729
      urban

      2001
      Ada
      NaN
      1.0
      132001001.0
      0.0
      2.0
      Ada
      16.0
      1.0
      8830.0
      ...
      Ada County, ID
      2001.0
      175653.0
      168876.0
      6777.0
      3.9
      83701
      43.6038
      -116.2729
      urban

      2002
      Ada
      NaN
      2.0
      132001001.0
      0.0
      2.0
      Ada
      16.0
      1.0
      8830.0
      ...
      Ada County, ID
      2002.0
      175742.0
      167437.0
      8305.0
      4.7
      83701
      43.6038
      -116.2729
      urban

      2003
      Ada
      NaN
      3.0
      132001001.0
      0.0
      2.0
      Ada
      16.0
      1.0
      8830.0
      ...
      Ada County, ID
      2003.0
      177056.0
      168453.0
      8603.0
      4.9
      83701
      43.6038
      -116.2729
      urban

      2004
      Ada
      NaN
      4.0
      132001001.0
      0.0
      2.0
      Ada
      16.0
      1.0
      8830.0
      ...
      Ada County, ID
      2004.0
      178289.0
      171048.0
      7241.0
      4.1
      83701
      43.6038
      -116.2729
      urban

      2005
      Ada
      NaN
      5.0
      132001001.0
      0.0
      2.0
      Ada
      16.0
      1.0
      8830.0
      ...
      Ada County, ID
      2005.0
      187127.0
      180914.0
      6213.0
      3.3
      83701
      43.6038
      -116.2729
      urban

      2006
      Ada
      NaN
      6.0
      132001001.0
      0.0
      2.0
      Ada
      16.0
      1.0
      8830.0
      ...
      Ada County, ID
      2006.0
      193828.0
      188617.0
      5211.0
      2.7
      83701
      43.6038
      -116.2729
      urban

      2007
      Ada
      NaN
      7.0
      132001001.0
      NaN
      2.0
      Ada
      16.0
      1.0
      8830.0
      ...
      Ada County, ID
      2007.0
      196740.0
      191500.0
      5240.0
      2.7
      83701
      43.6038
      -116.2729
      urban

      2008
      Ada
      NaN
      8.0
      132001001.0
      NaN
      2.0
      Ada
      16.0
      1.0
      8830.0
      ...
      Ada County, ID
      2008.0
      196434.0
      186848.0
      9586.0
      4.9
      83701
      43.6038
      -116.2729
      urban

      2009
      Ada
      NaN
      9.0
      132001001.0
      NaN
      2.0
      Ada
      16.0
      1.0
      8830.0
      ...
      Ada County, ID
      2009.0
      196383.0
      178892.0
      17491.0
      8.9
      83701
      43.6038
      -116.2729
      urban

      2010
      Ada
      NaN
      10.0
      132001001.0
      NaN
      2.0
      Ada
      16.0
      1.0
      8830.0
      ...
      Ada County, ID
      2010.0
      196945.0
      180585.0
      16360.0
      8.3
      83701
      43.6038
      -116.2729
      urban

      2011
      Ada
      NaN
      11.0
      132001001.0
      NaN
      2.0
      Ada
      16.0
      1.0
      8830.0
      ...
      Ada County, ID
      2011.0
      200304.0
      185600.0
      14704.0
      7.3
      83701
      43.6038
      -116.2729
      urban

      2012
      Ada
      NaN
      12.0
      132001001.0
      NaN
      2.0
      Ada
      16.0
      1.0
      8830.0
      ...
      Ada County, ID
      2012.0
      204353.0
      191714.0
      12639.0
      6.2
      83701
      43.6038
      -116.2729
      urban

      2013
      Ada
      NaN
      13.0
      132001001.0
      NaN
      2.0
      Ada
      16.0
      1.0
      8830.0
      ...
      Ada County, ID
      2013.0
      206575.0
      195843.0
      10732.0
      5.2
      83701
      43.6038
      -116.2729
      urban

      2014
      Ada
      NaN
      14.0
      132001001.0
      NaN
      2.0
      Ada
      16.0
      1.0
      8830.0
      ...
      Ada County, ID
      2014.0
      211675.0
      202897.0
      8778.0
      4.1
      83701
      43.6038
      -116.2729
      urban

19 rows × 629 columns

Here is a visual representation of one of these cities. Notice that for this city there is not data for 1995 and that we have an unbalaced panel. Some cities are missing many more values than this one and we will address this further down.

```python
df.shape
```

    (1509, 631)

Here you can see that we have over 600 variables

```python
df = df.drop(columns= funcs.search_all(df, 'coun'))
```

    County
    County
    FIPS_County
    Counties - Assault- Total
    Counties - Burglary- Total
    Counties - Manslaughter- Total
    Counties - Murder- Total
    Counties - Rape-Total
    Counties - Robbery- Total
    Counties - Total- All Thefts
    County FIPS Code
    County/State
    County FIPS Year
    County Labor Force
    County Labor Employed
    County Labor Unemployed

Again we reference a supporting function which we use to search for and drop all columns representing county level data. During our exploratory analysis we concluded that there was not enough county level data to be beneficial, cities become our major axis

```python
df.shape
```

    (1509, 615)

After refining the granularity of available variables to only consist of city level data we are left with 615 potential features

```python
plt.style.use('seaborn')
sns.barplot(x='Year4', y='Population', data=df.groupby('Year4').count().reset_index())
fig = plt.gcf()
fig.set_size_inches(10,7)
plt.ylabel('Quantity of Data Points')
plt.xlabel('Year')
plt.title('Frequency of gathered datapoints by Year')
plt.tight_layout()
```

```python
df = df.loc[df['Year4'].isin([1997, 2002, 2007, 2012])]
```

This plot only shows data points for population but during our analysis we identified that this trend was consistent for most variables. Being an unbalanced panel we removed every year other than 1997, 2002, 2007 and 2012. This provides us measurements on the max number of features while also maintaining balance. Removing these years was necessisary to maintain high quality input data. Notice these years are all evenly spaced so change in variables between years will be clear.

## Selecting a Dependant Variable

As stated under the _"Objective of Model"_ section above; the set contains many statistics desribing the financial activity of each city but only seven variables describing crime. These crime statsistics are unique measuements relative to the financial majority. There are a few other columns not describing finances, such as population, but we concluded that these columns would not provide the same value in identifying relations describing criminal activities and goverment financials.

Distinct from crime we also had information on unemployment data and would have liked to use it as the dependent in its own model . We were unable to address this within the time constraint but doing so would likely prove insightful.

```python
df.shape
```

    (797, 615)

After dropping those years from the set we are left with about 800 rows

```python
cities = (df.groupby('Name')['Assault- Total - Municipalities'].count() == 4)
cities = cities[cities == True].index
df = df[df['Name'].isin(cities)]
df.shape
```

    (236, 615)

Having already removed useless data on our minor axis (years) we then sought to do the same to our major axis (cities). Above we removed any city that did not have data for all four years, this reduced the total records in our dataframe to 236. This unfortunately removed a large portion of the original set but greatly improved the overall quality of our scope.

```python
tmp = df.notna().sum()
tmp = tmp[tmp > 206]
df = df.loc[:,tmp.index]
df.shape
```

    (236, 101)

Reducing our set an additional time we removed any feature that did not contain values for at least 87% of the total records. This is the penultimate step in our data purification but will will leave us with 236 records or 59 cities each containing data for 4 years and roughly 100 features.

```python
df = df.set_index(['Name', 'Year4'])
```

We are now done addressing missing data so we set the indices of our dataframe so that it will be recognized as a panel in future manipulations.

```python
df.drop(columns=['LAUS Code', 'latitude', 'longitude'],inplace=True)
```

The column LAUS Code is an object data type and contained little varience so we remnoved it from our set so that we didnt not have to waste time attempting to represent it as quantitative value. Additionally longitude contained negative values that could not be handled in futer models so we chose to remove it along with latitude.

```python
dummies = pd.get_dummies(df['size'])
df.drop(columns=['size'],inplace=True)
df = pd.concat([df, dummies], axis=1)
```

Size is a catagorical variable and depicts the size of a city so we replaced it with three dummy variables representing its three catagories.

## Initial Modeling

Our analyses only addresses Total Thefts as our dependent but with minimal reworking one could construct a loop iterating through each crime statistic as the model's dependent variable.

```python
dep_var = 'Total- All Thefts - Municipalities'
y = df[dep_var]
y = y.rename(dep_var)
y = pd.DataFrame(y)
y = y.astype(int)
```

Here we save Total Thefts as the dependent variable

```python
X = df.drop(columns = ['Robbery- Total - Municipalities',
       'Rape-Total - Municipalities',
       'Murder- Total - Municipalities',
       'Total- All Thefts - Municipalities',
       'Manslaughter-Total - Municipalities',
       'Burglary-Total - Municipalities',
       'Assault- Total - Municipalities'])
```

Now dropping every other crime variable from our data frame as to prevent them from causing undesired noise in our model.

## Interpolation

```python
X = X.interpolate(method='linear')
```

Finally we believe we have derived the optimal set to attempt to describe theft. However there are still a few null values, so we leverage the panel structure and use interpolate to fill in these nulls. Calculating our own values for these nulls is potentially harmful but we minimzed damage done here because we already dropped features that did not contain values for at least 87% of total records. Additionally notice we do not interpolate on the dependent for we do not want to predict predicted values, regardless "y" contained no missing values and we were able to keep the remaining records.

## Split

```python
train_X, val_X, train_y, val_y = train_test_split(X, y, test_size=0.2)
```

We perform a 20 percent test split but after preprocessing our dataset is much smaller, this is not desired but is prefered over noisy input data.

## Lasso

We need to perform feature selection. We want our features to be independent of one another and we want to use SelectKBest to perform a Chi Squared Test however K is a required parameter to do this. We will use a Lasso Regression to identify how many variables we whould use for SelectKBest. This is our initial model so it will be a little bit of a hack job but further down we will construct a pipeline to optimize K and some other hyperparameters.

```python
reg = LassoCV(cv=3, tol=0.009, random_state=201)
reg.fit(train_X, train_y[dep_var])
print("Best alpha using built-in LassoCV: %f" % reg.alpha_)
print("Best score using built-in LassoCV: %f" %reg.score(train_X,train_y))
```

    Best alpha using built-in LassoCV: 1077134.221116
    Best score using built-in LassoCV: 0.902847

```python
coef = pd.Series(reg.coef_, index = train_X.columns)
k_esi = sum(coef != 0)
print("Lasso picked " + str(k_esi) + " variables")
```

    Lasso picked 7 variables

We will use this value for K in SelectKBest

## SelectKBest

```python
selector = SelectKBest(score_func=chi2, k=k_esi)
selector.fit(train_X, train_y)
```

    SelectKBest(k=7, score_func=<function chi2 at 0x1a14cf26a8>)

```python
cols = selector.get_support(indices=True)

X_chi = train_X.iloc[:,cols]
val_X = val_X.iloc[:,cols]
```

We need the features of our model to only contain features that are independent of one another so to achieve this we use SelectKBest to perform a Chi Squared test and then select K values where K is determined by the previous Lasso regression.

## Handling Multicollinearity

```python
reduced_form, inds = sympy.Matrix(X_chi.values).rref()
X_chi = X_chi.iloc[:, list(inds)]
```

Handling linear dependence

```python
# For each X, calculate VIF and save in dataframe
vif = pd.DataFrame()
vif["VIF Factor"] = [variance_inflation_factor(X_chi.values, i) for i in range(X_chi.shape[1])]
vif["features"] = X_chi.columns
```

Leveraging python's _"Variance Inflation Factor"_ module to systematically remove collinear variables

```python
indep_var = vif.iloc[vif['VIF Factor'].drop_duplicates().index]['features']
X_chi = X_chi.loc[:,indep_var]
val_X = val_X.loc[:,indep_var]
train_X = X_chi
```

Transforming our train and test dataframes to only contain desired features.

## Time Effects Regression (OLS)

Having cleaned our data we believe OLS to be BLUE so we can now perform our regression

```python
paneols = PanelOLS(train_y[dep_var],X_chi[indep_var], time_effects=True)
res = paneols.fit()
res
```

PanelOLS Estimation Summary

  Dep. Variable:	    Total- All Thefts - Municipalities	   R-squared:         	    0.9019

  Estimator:	                     PanelOLS	                R-squared (Between):	   0.9450

  No. Observations:	                 188	                  R-squared (Within):	    -0.3319

  Date:	                      Fri, Jul 10 2020	            R-squared (Overall):	   0.9047

  Time:	                          22:54:55	                Log-likelihood     	    -1417.7

  Cov. Estimator:	               Unadjusted

  	                                   	                    F-statistic:       	    232.42

  Entities:	                         59	                   P-value            	    0.0000

  Avg Obs:	                        3.1864	                 Distribution:      	   F(7,177)

  Min Obs:	                        1.0000

  Max Obs:	                        4.0000	                 F-statistic (robust):	  232.42

  	                                   	                    P-value            	    0.0000

  Time periods:	                      4	                   Distribution:      	   F(7,177)

  Avg Obs:	                        47.000

  Min Obs:	                        45.000

  Max Obs:	                        49.000

Parameter Estimates

            	            Parameter	 Std. Err.	 T-stat	  P-value	 Lower CI	 Upper CI

  Total_Revenue	          -0.0136	   0.0196	   -0.6944	 0.4883	   -0.0523	  0.0251

  General_Revenue	        0.0056	    0.0216	   0.2594	  0.7957	   -0.0370	  0.0482

  Total_Expenditure	      0.0163	    0.0391	   0.4167	  0.6774	   -0.0608	  0.0934

  Direct_Expenditure	     0.0070	    0.0424	   0.1652	  0.8690	   -0.0767	  0.0908

  General_Expenditure	    -0.4118	   0.4934	   -0.8347	 0.4050	   -1.3854	  0.5618

  Direct_General_Expend	  0.4237	    0.4915	   0.8620	  0.3898	   -0.5463	  1.3938

  Total_LTD_Out	          0.0115	    0.0025	   4.6543	  0.0000	   0.0066	   0.0163

F-test for Poolability: 5.7287
P-value: 0.0009
Distribution: F(3,177)

Included effects: Time
id: 0x101897f28

This model appears to be a good representation of the data. From this we can see that Population, Total_Cash__Securities, Nonin_Trust_Cash__Sec, Total_Expenditure, ect. all are largly statistically significant in identifying Total Theft

Population and Total Expenditure seem to be accurate in describing Total Theft with low standard errors. This makes logical sense in that the more people in an area the more crime. Oth_Nonin_Fd_Cash__Sec, Tot_Chgs_and_Misc_Rev and Nonin_Trust_Cash___Sec also have low std. errors.

Our final P-value appears to be statistically significant so lets use our test data to see how it performs with unseen data.

```python
pred_y_pane = res.predict(val_X)
r2_score(val_y, pred_y_pane)
```

    0.8332998470474992

The R squared using unseen data.

Our R square dropped a bit but it still appears to be decent. We will make some improvements further down.

```python
fig, ax = plt.subplots()

ax.scatter(x=val_y, y=pred_y_pane['predictions'])
line = mlines.Line2D([0,1], [0,1], color='red')
transform = ax.transAxes
line.set_transform(transform)
ax.add_line(line)
ax.set_title('Parity Plot')
ax.set_xlabel('Actual Values')
ax.set_ylabel('Predicted Values')
plt.show()
```

Above is a parity plot simply to give a visual representation of how our model is predicting. From it you can see that the model is optimistically biased and predicting high.

## Scatter Plots of Dependent against Independent variables

```python
fig1, axs= plt.subplots(figsize=(15,15),nrows=5, ncols=3, sharey=True)

sns.regplot(y=y['Total- All Thefts - Municipalities'], x=X['Total_Revenue'], ax=axs[0,0], color='g')
axs[0, 0].set_xlabel("Total_Revenue")
axs[0, 0].set_ylabel('All Thefts')

sns.regplot(y=y['Total- All Thefts - Municipalities'], x=X['Total_Expenditure'], ax=axs[0,1], color='g')
axs[0, 1].set_xlabel("Total_Expenditure")
axs[0, 1].set_ylabel('All Thefts')

sns.regplot(y=y['Total- All Thefts - Municipalities'], x=X['Beg_LTD_Out_All_Other'], ax=axs[0,2], color='g')
axs[0, 2].set_xlabel("Beg_LTD_Out_All_Other")
axs[0, 2].set_ylabel('All Thefts')

###############

sns.regplot(y=y['Total- All Thefts - Municipalities'], x=X['Direct_Expenditure'], ax=axs[1,0], color='g')
axs[1, 0].set_xlabel("Direct_Expenditure")
axs[1, 0].set_ylabel('All Thefts')

sns.regplot(y=y['Total- All Thefts - Municipalities'], x=X['General_Expenditure'], ax=axs[1,1], color='g')
axs[1, 1].set_xlabel("General_Expenditure")
axs[1, 1].set_ylabel('All Thefts')

sns.regplot(y=y['Total- All Thefts - Municipalities'], x=X['LTD_Out_All_Other'], ax=axs[1,2], color='r')
axs[1, 2].set_xlabel("LTD_Out_All_Other")
axs[1, 2].set_ylabel('All Thefts')
##################

sns.regplot(y=y['Total- All Thefts - Municipalities'], x=X['Direct_General_Expend'], ax=axs[2,0], color='g')
axs[2, 0].set_xlabel("Direct_General_Expend")
axs[2, 0].set_ylabel('All Thefts')

sns.regplot(y=y['Total- All Thefts - Municipalities'], x=X['General_Current_Expend'], ax=axs[2,1], color='g')
axs[2, 1].set_xlabel("General_Current_Expend")
axs[2, 1].set_ylabel('All Thefts')

sns.regplot(y=y['Total- All Thefts - Municipalities'], x=X['Total_Cash___Securities'], ax=axs[2,2], color='g')
axs[2, 2].set_xlabel("Total_Cash___Securities")
axs[2, 2].set_ylabel('All Thefts')
###############

sns.regplot(y=y['Total- All Thefts - Municipalities'], x=X['General_Current_Oper'], ax=axs[3,0], color='g')
axs[3, 0].set_xlabel("General_Current_Oper")
axs[3, 0].set_ylabel('All Thefts')

sns.regplot(y=y['Total- All Thefts - Municipalities'], x=X['Total_Debt_Outstanding'], ax=axs[3,1], color='r')
axs[3, 1].set_xlabel("Total_Debt_Outstanding")
axs[3, 1].set_ylabel('All Thefts')

sns.regplot(y=y['Total- All Thefts - Municipalities'], x=X['Nonin_Trust_Cash___Sec'], ax=axs[3,2], color='g')
axs[3, 2].set_xlabel("Nonin_Trust_Cash___Sec")
axs[3, 2].set_ylabel('All Thefts')
##################

sns.regplot(y=y['Total- All Thefts - Municipalities'], x=X['Total_Long_Term_Debt_Out'], ax=axs[4,0], color='r')
axs[4, 0].set_xlabel("Total_Long_Term_Debt_Out")
axs[4, 0].set_ylabel('All Thefts')

sns.regplot(y=y['Total- All Thefts - Municipalities'], x=X['Total_Beg_LTD_Out'], ax=axs[4,1], color='g')
axs[4, 1].set_xlabel("Total_Beg_LTD_Out")
axs[4, 1].set_ylabel('All Thefts')

plt.tight_layout()
```

Taking a few of the independent varibles and ploting them against the dependent there arises some heteroskedasticity, as shown above, so a non-linear estimator will likely produce the best model. One might perform an F-test for validation but lets run with this.

## Bayesian Ridge Regression

```python
clf = BayesianRidge(compute_score=True)
clf.fit(train_X, train_y)
pred_y_baye = clf.predict(val_X)
```

    /Users/dom/anaconda3/lib/python3.7/site-packages/sklearn/utils/validation.py:761: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
      y = column_or_1d(y, warn=True)

```python
r2_score(val_y, pred_y_baye)
```

    0.8631852866124068

The R squared using unseen data

Compared to our time effects model this provided a pretty decent prediction

## Random Forest

```python
regr = RandomForestRegressor(n_estimators=100, random_state=201)
regr.fit(train_X, train_y)
pred_y_rand = regr.predict(val_X)
```

    /Users/dom/anaconda3/lib/python3.7/site-packages/ipykernel_launcher.py:2: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples,), for example using ravel().

```python
r2_score(val_y, pred_y_rand)
```

    0.9310881679861482

The R squared using unseen data

Slightly lower than the ridge model but being a random model we should attempt to optimize the parameter n_estimators.

```python
np.sqrt(mean_squared_log_error(val_y, pred_y_rand))
```

    1.230177740407641

The RSME using unseen data

## Optimizing HyperParameters with GridSearchCV

```python
train_X, val_X, train_y, val_y = train_test_split(X, y, test_size=0.2)
```

We will redo the split because we altered the training data during the Chi Squared test earlier

```python
tuned_params = [{'kbest__k':np.arange(5,X.columns.size//2, 5), 'rand__n_estimators':[5,10,50,100,500,1000]}]
model = SelectKBest(score_func=chi2, k=2)

pipeline = Pipeline([('kbest',model), ('rand', RandomForestRegressor(n_estimators=5, random_state=201))])

clf = GridSearchCV(estimator=pipeline, param_grid=tuned_params, cv=5, refit=True, scoring='neg_mean_squared_error')

clf.fit(train_X, train_y[dep_var])
```

    /Users/dom/anaconda3/lib/python3.7/site-packages/sklearn/model_selection/_search.py:841: DeprecationWarning: The default of the `iid` parameter will change from True to False in version 0.22 and will be removed in 0.24. This will change numeric results when test-set sizes are unequal.
      DeprecationWarning)

    GridSearchCV(cv=5, error_score='raise-deprecating',
           estimator=Pipeline(memory=None,
         steps=[('kbest', SelectKBest(k=2, score_func=<function chi2 at 0x1a14cf26a8>)), ('rand', RandomForestRegressor(bootstrap=True, criterion='mse', max_depth=None,
               max_features='auto', max_leaf_nodes=None,
               min_impurity_decrease=0.0, min_impurity...mators=5, n_jobs=None,
               oob_score=False, random_state=201, verbose=0, warm_start=False))]),
           fit_params=None, iid='warn', n_jobs=None,
           param_grid=[{'kbest__k': array([ 5, 10, 15, 20, 25, 30, 35, 40]), 'rand__n_estimators': [5, 10, 50, 100, 500, 1000]}],
           pre_dispatch='2*n_jobs', refit=True, return_train_score='warn',
           scoring='neg_mean_squared_error', verbose=0)

Using a pipeline with SelectKBest, RandomForest and GridSearchCV we optimized K of SelectKBest and the number of estimators (n) of RandomForest. We used GridSearchCV which is rather exhaustive but could likely become more efficient if replaced by RandomSearch

```python
clf.best_params_
```

    {'kbest__k': 35, 'rand__n_estimators': 5}

The optimal values according to our pipeline

```python
pred_y_grid = clf.predict(val_X)
r2_score(val_y, pred_y_grid)
```

    0.8375486072443916

The R squared using unseen data

This provided us with the best model, furthermore this pipeline could easily be isolated with the preprocessing steps and turned into a pickle for further use.

```python
np.sqrt(mean_squared_log_error(val_y, pred_y_grid))
```

    0.6822454745533553

The RSME using unseen data

Our RSME got slightly worse so this model could become more robust.

## Potential Future Steps

From this analysis we have provided a linear model estimating total theft of Idaho cities. Furthermore we constructed a pipeline that produces a more robust model. For further iterations of the project there are serveral places of potential improvement.

>- Feature engineering & importing relevent data.

* Perform a better imputation technique.

- Perform further test to determine optimal features.

- Replace GridSearch with RandomSearch or some other better method to optimize hyper parameters.

- Improve models by using a more effective algorithm or better yet applying deep learning techniques.

- Package everything into a pickle

- Construct models for the other variables measuring crime


```python
%matplotlib inline

import matplotlib
import numpy as np
import matplotlib.pyplot as plt

plt.rcParams["figure.figsize"] = (12, 9)

font = {'family' : 'normal',
        'weight' : 'normal',
        'size'   : 22}

matplotlib.rc('font', **font)

```

# Enrichment

Enrichment is the stage at which natural uranium is processed to acheive higher concentrations of fissile $^{235}U$ isotopes.

## Learning Objectives

- Recall the isotopic content of natural uranium.
- Identify target enrichments for various applications.
- Distinguish among enrichment technologies.
- Discuss the history of enrichment technology development.
- Calculate separation potentials.
- Quantify the relationship between feed, product, and tails during enrichment.
- Calculate SWU, Feed, and Waste Factors.
- Explain the physical and chemical mechanisms of various enrichment technologies.
- Discuss the nonproliferation implications of enrichment technology export control.

## Natural Uranium

Natural uranium, $U_{nat}$ is uranium found in nature. By weight it contains :

- 0.711\% $^{235}U$
- 99.284\% $^{238}U$
- trace amount (0.0055\%) of $^{234}U$

The halflife of 238U is different (longer) than 235U, so the ratio changes over time. (Where time is measured in millions of years.)

## Enrichment Level

_Commercial LWR Fuel_: Commercial nuclear fuel for light water reactors is typically in the range of 3-5%

_Research Reactor Fuel_: Research reactor fuel (such as TRIGA reactors in many universities) was previously often as high as 40%. However, a downblending effort was undertaken nationally and the fuel is now much less enriched.

_Small Modular Reactor Fuel_: Small modular reactors often boast long running times, such that the fuel rarely needs to be replaced. Though these typically rely on breeding for long term sustainability, the initial enrichment is usually relatively high in these designs.

_Navy Fuel_: Classified, but high.

_Weapons_: Very high. This varies by weapon type. Does anyone recall the $^{325}U$ enrichment of the Godiva device? (_Hint_: It was 30cm in diameter.)

```python
enr_sources = {'natural U':0.007,
               'recycled U': 0.01}

enr_uses = {'CANDU':0.007,
            'LWR': 0.045,
            'Research':0.20,
            'Weapons': 0.9}

import collections
od_enr_uses = collections.OrderedDict(sorted(enr_uses.items()))
```

```python
import numpy as np

x=[]
y=[]
delta_x=0.02
delta_y=-0.02

for k,v in od_enr_uses.items():
    x.append(v)
    y.append(v)
    plt.text(v + delta_x, v + delta_y, k)
t = x
area = 60*np.pi
plt.scatter(x, y, s=area, c=t, cmap='viridis', alpha=0.6)
plt.axis([0,1,0,1])
plt.xlabel('Enrichment $(\% ^{235}U)$')
plt.ylabel('Effort $(SWU)$')
plt.title('Enrichment for various uses')
plt.plot(0.007*np.ones(2), 'g--')
plt.text(0.5, 0.007, 'Natural U', color='g')

plt.show()
```

## Gaseous Diffusion

$$kT = \frac{1}{2}mv^2$$

```python
from scipy import constants as sc
sc.Boltzmann
import scipy
scipy.constants.unit('Boltzmann constant')

import numpy as np
k= sc.Boltzmann

def f_to_k(f):
    return (f + 459.67)*5./9.

T = np.arange(300.0, 700.)

def vel(temp, m):
    import math
    k= sc.Boltzmann
    return math.sqrt(2*k*temp/m)

v_235 = np.arange(0.0, 400.0)
v_238 = np.arange(0.0, 400.0)
ratio = np.arange(0.0, 400.0)

m_235 = 235
m_238 = 238

for i in range(0, 400):
    v_235[i] = vel(T[i], m_235)
    v_238[i] = vel(T[i], m_238)
    ratio[i] = v_235[i]/v_238[i]

plt.plot(v_235, 'g--')
plt.plot(v_238, 'r--')
```

```python
f_to_k(700.)
```

    644.2611111111112

```python
plt.plot(ratio)
```

$^{238}UF_6$ and $^{235}UF_6$ have different effusive rates. This is used to separate them through a porous barrier.

$$\frac{V_L}{V_H} = \sqrt{\frac{M_H}{M_L}} = \alpha$$

In the case of $^{238}UF_6$ and $^{235}UF_6$:

$$M_H = M_{^{238}U} + 6\times M_F$$
$$\Rightarrow M_H = 238 + 6\times 19$$

$$M_L = M_{^{235}U} + 6\times M_F$$
$$\Rightarrow M_H = 235 + 6\times 19$$

So, $\alpha$ is the separative capacity of a single gaseous diffusion stage.

### Math of Enrichment

\begin{align}
F &= \mbox{rate of feed material (per time)}\\
P &= \mbox{rate of enriched product material (per time)}\\
W &= \mbox{rate of waste material (tails)}\\
x_f &= \mbox{weight percent of the target isotope in feed }\\
x_p &= \mbox{weight percent of the target isotope in product}\\
x_w &= \mbox{weight percent of the target isotope remains in waste}\\
\end{align}

The core equations in the math of enrichment are these two:

\begin{align}
F &= P + W\\
x_fF &= x_pP + x_wW\\
\end{align}

We try to make these more useful by defining the following derived equations.

\begin{align}
FF &= \frac{F}{P}\\
   &= \frac{x_p - x_w}{x_f - x_w}\\
WF &= \frac{W}{P}\\
   &= \frac{x_p - x_f}{x_f - x_w}
\end{align}

```python
def feed(P, W):
    return P + W

def prod(F, W):
    return F - W

def tail(P, F):
    return F - P

def ff(x_p, x_w, x_f):
    num = x_p - x_w
    denom = x_f - x_w
    return num/denom

def wf(x_p, x_w, x_f):
    num = x_p - x_f
    denom = x_f - x_w
    return num/denom

def x_p(f, p, x_f, x_w):
    return (f/p)*(x_f - x_w) + x_w

def x_w(f, p, x_f, x_p):
    num = x_p - (f/p)*x_f
    denom = 1-(f/p)
    return num/denom

x_p = np.arange(0.1, 0.5, 0.1)
to_plot = np.arange(1.,5.)
for i in range(0,4):
    to_plot[i] = ff(x_p[i], 0.005, 0.00711)

plt.plot(x_p, to_plot, label="FF")
plt.plot(x_p, to_plot - 1, label="WF")
plt.xlabel("Desired enrichment")
plt.ylabel("kg")
plt.legend(loc=2)
```

### Separation Potentials

\begin{align}
V(x_i) &= (2x_i -1)ln\left(\frac{x_i}{1-x_i}\right)\\
\Rightarrow V(x_f) &= (2x_f -1)ln\left(\frac{x_f}{1-x_f}\right)\\
\end{align}

```python
from math import log
def v(x):
    """separation potential"""
    return (2*x - 1) * log(x/(1-x))

x_i = np.arange(0.001, 0.9, 0.001)
to_plot = [v(x) for x in x_i]
plt.plot(x_i, to_plot)
plt.xlabel(r'$x_i$', fontsize=24)
plt.ylabel(r'$v(x_i)$', fontsize=24)
```

$$SWU = \tau(PV(x_p) + WV(x_w) - FV(x_f)))$$

```python
def swu(x_p, x_w, x_f, p, t):
    w = p*wf(x_p, x_w, x_f)
    f = p*ff(x_p, x_w, x_f)
    return t*(p*v(x_p) + w*v(x_w) - f*v(x_f))

x_i = np.arange(0.001, 0.1, 0.001)

v_x_p = 0.03
v_x_w = 0.001
v_x_f = 0.00711
prod = 50
t = 1
swu_p = [swu(v_x_p, v_x_w, v_x_f, x, 1) for x in range(1, 100)]
swu_x_p = [swu(x, v_x_w, v_x_f, prod, 1) for x in x_i]
swu_x_w = [swu(v_x_p, x, v_x_f, prod, 1) for x in x_i]
swu_x_f = [swu(v_x_p, v_x_w, x, prod, 1) for x in x_i]

fig, ax1 = plt.subplots()

ax1.plot(x_i, swu_x_p, label= "SWU(x_p)", marker='o', linestyle='dashed')
ax1.plot(x_i, swu_x_w, label= "SWU(x_w)", linestyle='dashed')
ax1.plot(x_i, swu_x_f, label= "SWU(x_f)", marker='+', linestyle='solid')
ax1.set_xlabel(r'$x_i$', color='k', fontsize=24)

ax1.legend(loc=2)

ax2 = ax1.twiny()
ax2.plot(swu_p, label="SWU(P)", color='c')
ax2.set_xlabel(r'$P$', color='c')
for tl in ax2.get_xticklabels():
    tl.set_color('c')

ax2.legend()

```

```python

```

### SWU Factor

The SWU factor is the number of SWUs required per unit product (in our case, kg of enriched U).

\begin{align}
SF &= \frac{SWU}{P\tau}\\
   &= V(x_p) + \frac{W}{P}V(x_w) - \frac{F}{P}V(x_f)
\end{align}

```python
plt.clf()
def swu_factor(x_p, x_w, x_f):
    sf = v(x_p) + wf(x_p, x_w, x_f)*v(x_w) - ff(x_p, x_w, x_f)*v(x_f)
    return sf

x_i = np.arange(0.001, 0.1, 0.001)
x_p = 0.03
x_w = 0.002
x_f = 0.00711

sf_x_p = [swu_factor(x,   x_w, x_f) for x in x_i]
sf_x_w = [swu_factor(x_p, x,   x_f) for x in x_i]
sf_x_f = [swu_factor(x_p, x_w, x) for x in x_i]

plt.plot(x_i, sf_x_p, label= "SF(x_p)")
plt.plot(x_i, sf_x_w, label= "SF(x_w)")
plt.plot(x_i, sf_x_f, label= "SF(x_f)")
plt.legend(loc=2)
```

```python
plt.clf()
def swu_factor(x_p, x_w, x_f):
    sf = v(x_p) + wf(x_p, x_w, x_f)*v(x_w) - ff(x_p, x_w, x_f)*v(x_f)
    return sf

x_i = np.arange(0.001, 0.9, 0.001)
x_p = 0.03
x_w = 0.002
x_f = 0.00711

sf_x_p = [swu_factor(x, x_w, x_f) for x in x_i]
ff_x_p = [ff(x, x_w, x_f) for x in x_i]

plt.plot(x_i, sf_x_p, label= "SF(x_p)")
plt.plot(x_i, ff_x_p, label= "FF(x_p)")
plt.xlabel(r'$x_p$', fontsize=24)
plt.ylabel(r'$kg$', fontsize=24)
plt.legend(loc=2)
```

## Centrifugal Enrichment

### Centrifugal Force

Centrifugal enrichment relies on the __centrifugal force__. The centrifugal force is only apparent in the rotating reference frame. In the inertial reference frame, it is not apparent, so it is a 'fictitious' force. Fictitious or not, though, in the stationary reference frame, things move.

#### Velocity
Velocity in a rotating frame of reference, with the absolute angular velocity, $\omega$, gives

$$\frac{\operatorname{d}\boldsymbol{r}}{\operatorname{d}t} = \left[\frac{\operatorname{d}\boldsymbol{r}}{\operatorname{d}t}\right] + \boldsymbol{\omega} \times \boldsymbol{r}$$

Note that the square brackets in this notation indicate that the argumet is evaluated in the  stationary frame. So, the velocity in the rotating reference frame is dependent on the rotation, $\omega$.

#### Acceleration

Acceleration in a rotating reference frame simply undergoes the same derivative transformation a second time.

$$\begin{align}
\boldsymbol{a} &=\frac{\operatorname{d}^2\boldsymbol{r}}{\operatorname{d}t^2} = \frac{\operatorname{d}}{\operatorname{d}t}\frac{\operatorname{d}\boldsymbol{r}}{\operatorname{d}t} = \frac{\operatorname{d}}{\operatorname{d}t} \left( \left[\frac{\operatorname{d}\boldsymbol{r}}{\operatorname{d}t}\right] + \boldsymbol{\omega} \times \boldsymbol{r}\ \right) \\
 &= \left[ \frac{\operatorname{d}^2 \boldsymbol{r}}{\operatorname{d}t^2} \right] + \frac{\operatorname{d} \boldsymbol{\omega}}{\operatorname{d}t}\times\boldsymbol{r} + 2 \boldsymbol{\omega}\times \left[ \frac{\operatorname{d} \boldsymbol{r}}{\operatorname{d}t} \right] + \boldsymbol{\omega}\times ( \boldsymbol{\omega} \times \boldsymbol{r}) \ .
\end{align}
$$

But, that's all in the rotating frame. When an observer views this, the forces must appear to cancel. So, we arrive at the observed form of the equation of motion, where :

$$\boldsymbol{F} - m\frac{\operatorname{d} \boldsymbol{\omega}}{\operatorname{d}t}\times\boldsymbol{r} - 2m \boldsymbol{\omega}\times \left[ \frac{\operatorname{d} \boldsymbol{r}}{\operatorname{d}t} \right] - m\boldsymbol{\omega}\times (\boldsymbol{\omega}\times \boldsymbol{r}) = m\left[ \frac{\operatorname{d}^2 \boldsymbol{r}}{\operatorname{d}t^2} \right]$$

#### Questions:

- What direction is represented by the vector $\boldsymbol{r}$?
- What direction is represented by the vector $\boldsymbol{\omega}$?
- Combining these facts with the equation above, what can be said about the direction of the velocity vector $\vec{\omega}\times\vec{r}$?
- What about the direction of the final term of the observed force $-\vec{\omega}\times(\vec{\omega}\times\vec{r})$?

```python
# Let's see a classic image of a bank of centrifuges
from IPython.display import Image
Image("http://world-nuclear.org/uploadedImages/org/info/Nuclear_Fuel_Cycle/Enrichment_and_Conversion/centrfge.jpg")
```

### Separation Factor per Centrifuge Stage

$$\alpha = 1+ \frac{M_H - M_L}{2RT}\omega^2a^2$$

In the case of $^{238}UF_6$ and $^{235}UF_6$:

$$M_H = M_{^{238}U} + 6\times M_F$$
$$\Rightarrow M_H = 238 + 6\times 19$$

$$M_L = M_{^{235}U} + 6\times M_F$$
$$\Rightarrow M_L= 235 + 6\times 19$$

```python
from scipy import constants as sc
import math
def alpha(m_h, m_l, omega, a, t):
    """
    :param omega: angular speed of rotating bowls (rad/s)
    :param a: inside radius of rotor [m]
    :param m_h: molecular weight of \[^{238}U^{19}F_6\]
    :param m_l: molecular weight of \[^{235}U^{19}F_6\]
    :param r: R, molar gas constant
    :param t: temperature in Kelvin
    """
    return 1 + ((m_h-m_l)*(omega**2)*(a**2))/(2*sc.R*t)

def sep_cap(l, m_h, m_l, omega, a, t):
    """
    :param l: length of the bowl [m]
    :param omega: angular speed of rotating bowls (rad/s)
    :param a: inside radius of rotor [m?]
    :param m_h: molecular weight of \[^{238}U^{19}F_6\]
    :param m_l: molecular weight of \[^{235}U^{19}F_6\]
    :param r: R, molar gas constant
    :param t: temperature in Kelvin
    """
    return math.exp((l*(m_h-m_l)**2*(omega*a)**4))/(t**2)

m_h = (238 + 6*19)
m_l = (235 + 6*19)
a = 0.3048
t = 300
#l = 4
omega = np.arange(2300., 4000.)

alpha_plot = [alpha(m_h, m_l, o, a, t) for o in omega]
#plt.plot(alpha_plot, label=r'$\alpha$ (separation factor)')

#sep_cap_plot = [sep_cap(l, m_h, m_l, o, a, t) for o in omega]

plt.plot(omega, alpha_plot, label="alpha")
#plt.plot(sep_cap_plot, label="sep cap")
plt.xlabel(r'$\omega$', fontsize=24)
plt.legend(loc=2)
```

## The Zippe Centrifuge

> A Zippe-type centrifuge has a hollow, cylindrical rotor filled with gaseous uranium in the form of its hexafluoride. A pulsating magnetic field at the bottom of the rotor, similar to that used in an electric motor, is able to spin it quickly enough that the U-238 is thrown towards the edge. The lighter U-235 collects near the center. The bottom of the gaseous mix is heated, producing convection currents that move the U-238 down. The U-235 moves up, where scoops collect it. Each centrifuge has one inlet and two output lines (corresponding to the heavy and the light fractions).

> At the high speed of rotation, the gas is compressed close to the wall of the rotor. The rotor can be almost a meter in length (diameter is likely to be less than 10 cm) and a temperature gradient of 300 °C between the top and bottom of the rotor produces a very strong convection current. In addition, the very strong coriolis forces produced add to the separation efficiency.

> To reduce friction, the rotor spins in a vacuum. A magnetic bearing holds the top of the rotor steady, and the only physical contact is the needle-like bearing that the rotor sits on. The three gas lines must be concentric with the fixed axis as the outer rim is spinning very quickly, and the seal is very important.

## History

### Calutron Electromagnetic Separation

1940s schematic of Calutron separation physics.

Electromagnetic separation cartoon.

Alpha I racetrack, Y-12 Uranium 235 electromagnetic separation plant, Manhattan Project, Clinton Engineer Works, Oak Ridge

Y-12, Oak Ridge.

Calutron operators.

## Separation Nozzle Method

The nozzle method is another type of enrichment technology that relies on mass.

In particular, lighter elements are deflected upward more easily and are separated physically from their heavier counterparts.

## Laser Enrichment

Recall the hyperfine structure of the atom.

### AVLIS
Atomic vapor laser isotope separation.

### SILEX
In Australia, a combination of the two methods, abbreviated to SILEX (separation of isotopes via laser excitation). Cold gas, pulsed $CO_2$ laser with an amplified $16 \mu m$ wavelength laser which preferentially excites the $^{235}UF_6$.

## Economics of Enrichment

The enrichment technologies used by industry must be energy effecient and economically feasible. Cost of enrichment is typically given in $\$/SWU$, so to find the cost of enrichment, one must multiply the price of separation with the number of SWUs to be purchased.

$$\$ = \frac{\$}{SWU}SWU$$

### Discussion: If a $\$/kg$ unit is needed, one must multiply the price of separation in $\$/SWU$ with what value?

## World Enrichment Capacity

## Aside: Enrichment of Heavy Water
$^2H$ is also known as D (deuterium).

### Discussion: Are $D$ (deuterium) and $H$ (protium) easier or harder to separate than isotopes of U?

### Why?

```python

```


```python
from sympy import *
from sympy.abc import *
import numpy as np
import matplotlib.pyplot as plt

def Kw_(C):
    return ((4*C - 1)/(4*C - 4)) + 0.625/C

def Ks_(C):
    return (0.5/C) + 1

def Fm_(Fmax, Fmin):
    return (Fmax + Fmin)/2

def Fa_(Fmax, Fmin):
    return (Fmax - Fmin)/2

Ks, Kw, Fmin, Fmax, Fm, Sf, Fa, nf, Ytrab = symbols("K_s K_w F_{min} F_{max} F_m S_f F_a n_f y_{trab}")

Y = ((d**(b+2))*0.67*np.pi*A)
Z = (Y/(8*C)) - Ks*Fmin
X = Ks*(Fm-Fmin) + (1.34*((A*d**b)/Sf)-1)*Kw*Fa

e = Z/X
e.evalf(3)
```

$\displaystyle \frac{\frac{0.263 A d^{b + 2.0}}{C} - F_{min} K_{s}}{F_{a} K_{w} \left(\frac{1.34 A d^{b}}{S_{f}} - 1.0\right) + K_{s} \left(F_{m} - F_{min}\right)}$

### Resumo de molas helicoidais - Segunda - Aula 10/01
Indice da mola $C$

$$K_s = 1 + \frac{0.5}{C}$$
$$K_w = \frac{4C - 1}{4C - 4} + \frac{0.625}{C}$$

Forças médias e alternadas

$$F_m = \frac{F_{max + F_{min}}}{2}$$
$$F_a = \frac{F_{max - F_{min}}}{2}$$

Determinando $d$ pela relação iterativa - Começar i com $d=1mm$
$$d = \left(\frac{11.9 C n_{f} \left(F_{a} K_{w} \left(\frac{1.34 A d^{b}}{S_{f}} - 1.0\right) + K_{s} \left(F_{m} - \frac{F_{min} \left(n_{f} - 1.0\right)}{n_{f}}\right)\right)}{A \pi}\right)^{\frac{1}{b + 2.0}}$$

Diâmetro médio $D$ e externo $D_0$
$$D = Cd$$
$$D_0 = D + d$$

Rigidez da mola $K$
$$K = \frac{F_{max - F_{min}}}{y_{trabalho}}$$

Número de espiras ativas $N_a$ e totais $N_t$
$$N_a = \frac{d^{4}G}{8D^3K}$$
$$N_t = N_a + 2$$

Rigidez da mola $K$ considerando $Na$ inteiro
$$K = \frac{d^{4}G}{8D^3N_a}$$

Deflexão inicial $y_{inicial}$
$$y_{inic} = \frac{F_{min}}{K}$$

Comprimento livre $L_f$
$$L_f = dN_t + 1.15 + y_{inic}$$

```python
#Equação iterativo de 'd'
U = Fm-((nf-1)/nf)*Fmin
U_ = 1.34*((A*d**b)/Sf) - 1
J = ((8*C*nf)/(0.67*pi*A))

d_ = (J*(Ks*U + U_*Kw*Fa))**(1/(b+2))
d_.evalf(3)
```

$\displaystyle \left(\frac{11.9 C n_{f} \left(F_{a} K_{w} \left(\frac{1.34 A d^{b}}{S_{f}} - 1.0\right) + K_{s} \left(F_{m} - \frac{F_{min} \left(n_{f} - 1.0\right)}{n_{f}}\right)\right)}{A \pi}\right)^{\frac{1}{b + 2.0}}$

```python
#Dados do problema
fmax, fmin, ytrab, c = 600, 300, 25E-3, 8
A_, b_, sf = 1909.9, -0.1453, 310

#Cálculo dos coeficientes
ks, kw = Ks_(c), Kw_(c)

#Cálculo das forças médias e alternadas
fm = Fm_(fmax, fmin)
fa = Fa_(fmax, fmin)

#Cálculo iterativo do diâmetro
solução = []
din = 1 #chute inicial
for i in range(10):
    s = d_.subs({d:din, Kw:kw, Ks:ks, Fm:fm, Fa:fa, nf:1.5, Fmin:fmin, Fmax:fmax,
                b:b_, A:A_, Sf:sf, C:c, pi:np.pi})
    solução.append(s)
    din = (s.evalf(5))

np.array(solução, dtype=float)
```

    array([7.27672017, 6.35883075, 6.41626819, 6.41241659, 6.41267379,
           6.41265647, 6.41265749, 6.41265749, 6.41265749, 6.41265749])

```python
#Diâmetro comercial e dados numéricos
C, G = c, 80.8E9
dc = 6.5E-3

#Diâmetro médio D
D = C*dc

#Diâmetro externo D0
D0 = D + dc

#Rigidez da mola
K = (fmax-fmin)/ytrab

K, D, D0
```

    (12000.0, 0.052, 0.058499999999999996)

```python
#Qtd de espiras ativas
Na_ = (G*dc**4)/(8*K*(D)**3)

#aprox pra cima
Na = int(Na_) + 1
(Na_, Na)
```

    (10.685221354166664, 11)

```python
#Rigidez da mola de novo
K_ = (G*dc**4)/(8*Na*(D)**3)
K_
```

    11656.605113636362

```python
#Deflexão inicial
yinic = fmin/K_
yinic
```

    0.02573648134044174

```python
#Comprimento livre
Nt = Na + 2
Lf = (dc)*Nt + 1.15*(ytrab) + yinic
Lf
```

    0.13898648134044173

### Resumo de molas de compressão/tração - Terça - Aula 11/01

Comprimento livre $L_f$
$$L_f = L_b + 2L_g$$
$$L_f = N_t + 2L_g$$

Espiras ativas $N_a$
$$N_t = N_a + 1$$

Equação transcendental pra determinar o diâmetro do fio
$$d = \left(G \left(\frac{2.98507462686567 F_{a} n_{f} \left(0.38659 A d^{b} - 0.2885 S_{f}\right)}{S_{f}} + F_{m} n_{f} - F_{min} \left(n_{f} - 1\right)\right)\right)^{\frac{1}{b + 2}}$$

```python
from sympy import *
from sympy.abc import *
import numpy as np
import matplotlib.pyplot as plt

Ks, Kw, Fmin, Fmax, Fm, Sf, Fa, nf, Ytrab = symbols("K_s K_w F_{min} F_{max} F_m S_f F_a n_f y_{trab}")
Kb = symbols("K_b")

C=8
G_ = (4*(4*Kb*C+1))/(pi*A)
H_ = nf*Fm - (nf-1)*Fmin
T_ = (0.577*(-0.5*Sf + 0.67*A*d**b))/(0.5*0.67*Sf)
d_ = (G_*(H_ + nf*Fa*T_))**(1/(b+2))
d_.evalf(3)
```

$\displaystyle \left(\frac{\left(128.0 K_{b} + 4.0\right) \left(\frac{2.98 F_{a} n_{f} \left(0.387 A d^{b} - 0.289 S_{f}\right)}{S_{f}} + F_{m} n_{f} - F_{min} \left(n_{f} - 1.0\right)\right)}{A \pi}\right)^{\frac{1}{b + 2.0}}$

```python
#Dados dos materiais
dic = { 'A227':[-0.1822, 1753.3],
        'A228':[-0.1625, 2153.5],
        'A229':[-0.1833, 1831.2],
        'A232':[-0.1453, 1909.9],
        'A232':[-0.0934, 2059.2]}

material = 'A228'
fmax, fmin, C = 150, 50, 8

b_, A_, sf = dic[material][0], dic[material][1], 310
fm = Fm_(fmax, fmin)
fa = Fa_(fmax, fmin)
kb = (-1-C+4*C**2)/(4*C*(C-1))

#Cálculo iterativo do diâmetro
solução = []
din = 1 #chute inicial
for i in range(10):
    s = d_.subs({d:din, Kb:kb, Fm:fm, Fa:fa, nf:2, Fmin:fmin, Fmax:fmax,
                b:b_, A:A_, Sf:sf, C:8, pi:np.pi})
    solução.append(s)
    din = (s.evalf(5))
solução
#np.array(solução, dtype=float)
```

    [4.90339015850936,
     4.31041980252648,
     4.35523200896826,
     4.35161560058455,
     4.35190591019958,
     4.35188262669210,
     4.35188446480197,
     4.35188446480197,
     4.35188446480197,
     4.35188446480197]

```python

```