# Alzheimer’s Analysis

## Introduction
### About Dementia
Dementia is not a specific disease. It’s an overall term that describes a group of symptoms associated with a decline in memory or other thinking skills severe enough to reduce a person’s ability to perform everyday activities. Alzheimer’s disease accounts for 60 to 80 percent of cases. Vascular dementia, which occurs after a stroke, is the second most common dementia type. But there are many other conditions that can cause symptoms of dementia, including some that are reversible, such as thyroid problems and vitamin deficiencies.

Dementia is a general term for loss of memory and other mental abilities severe enough to interfere with daily life. It is caused by physical changes in the brain. Alzheimer's is the most common type of dementia, but there are many kinds.

### Alzheimer

- Alzheimer's is a type of dementia that causes problems with memory, thinking and behavior. Symptoms usually develop slowly and get worse over time, becoming severe enough to interfere with daily tasks.

- Alzheimer's is not a normal part of aging. The greatest known risk factor is increasing age, and the majority of people with Alzheimer's are 65 and older. But Alzheimer's is not just a disease of old age. Approximately 200,000 Americans under the age of 65 have younger-onset Alzheimer’s disease (also known as early-onset Alzheimer’s).

-  Alzheimer's is the sixth leading cause of death in the United States. Those with Alzheimer's live an average of eight years after their symptoms become noticeable to others, but survival can range from four to 20 years, depending on age and other health conditions.

- Alzheimer's has no current cure, but treatments for symptoms are available and research continues. Although current Alzheimer's treatments cannot stop Alzheimer's from progressing, they can temporarily slow the worsening of dementia symptoms and improve quality of life for those with Alzheimer's and their caregivers. Today, there is a worldwide effort under way to find better ways to treat the disease, delay its onset, and prevent it from developing.

### Understanding the data
> **Summary**: This set consists of a longitudinal collection of 150 subjects aged 60 to 96. Each subject was scanned on two or more visits, separated by at least one year for a total of 373 imaging sessions. For each subject, 3 or 4 individual T1-weighted MRI scans obtained in single scan sessions are included. The subjects are all right-handed and include both men and women. 72 of the subjects were characterized as nondemented throughout the study. 64 of the included subjects were characterized as demented at the time of their initial visits and remained so for subsequent scans, including 51 individuals with mild to moderate Alzheimer’s disease. Another 14 subjects were characterized as nondemented at the time of their initial visit and were subsequently characterized as demented at a later visit.

**Dataset Description**
- We will be using the longitudinal MRI data.
- The dataset consists of a longitudinal MRI data of 150 subjects aged 60 to 96.
- Each subject was scanned at least once.
- Everyone is right-handed.
- 72 of the subjects were grouped as 'Nondemented' throughout the study.
- 64 of the subjects were grouped as 'Demented' at the time of their initial visits and remained so throughout the study.
- 14 subjects were grouped as 'Nondemented' at the time of their initial visit and were subsequently characterized as 'Demented' at a later visit. These fall under the 'Converted' category.

| COL  | Description                         |
|------|-------------------------------------|
| EDUC | Years of Education                  |
| SES  | Socioeconomic Status                |
| MMSE | Mini Mental State Examination       |
| CDR  | Clinical Dementia Rating            |
| eTIV | Estimated Total Intracranial Volume |
| nWBV | Normalize Whole Brain Volume        |
| ASF  | Atlas Scaling Factor                |

**Clinical Info:**
- MMSE - Mini-Mental State Examination score (range is from 0 = worst to 30 = best)
- CDR - Clinical Dementia Rating (0 = no dementia, 0.5 = very mild AD, 1 = mild AD, 2 = moderate AD)
- Derived anatomic volumes
- eTIV - Estimated total intracranial volume, mm3
- nWBV - Normalized whole-brain volume, expressed as a percent of all voxels in the atlas-masked image that are labeled as gray or white matter by the automated tissue segmentation process
- ASF - Atlas scaling factor (unitless). Computed scaling factor that transforms native-space brain and skull to the atlas target (i.e., the determinant of the transform matrix)

#### Mini–Mental State Examination (MMSE)
> The Mini–Mental State Examination (MMSE) or Folstein test is a 30-point questionnaire that is used extensively in clinical and research settings to measure cognitive impairment. It is commonly used in medicine and allied health to screen for dementia. It is also used to estimate the severity and progression of cognitive impairment and to follow the course of cognitive changes in an individual over time; thus making it an effective way to document an individual's response to treatment. The MMSE's purpose has been not, on its own, to provide a diagnosis for any particular nosological entity.
> - **Cognitive Impairment:** Cognitive impairment is when a person has trouble remembering, learning new things, concentrating, or making decisions that affect their everyday life. Cognitive impairment ranges from mild to severe.

> Any score greater than or equal to 24 points (out of 30) indicates a normal cognition. Below this, scores can indicate severe (≤9 points), moderate (10–18 points) or mild (19–23 points) cognitive impairment. The raw score may also need to be corrected for educational attainment and age. That is, a maximal score of 30 points can never rule out dementia.

|   **Method**  |**Score**|  **Interpretation**                   |
|---------------|---------|--------------------------------------|
| Single Cutoff |   <24   |        Abnormal                      |
|     Range     |   <21   |  Increased Odds of Dementia          |
|               |   <25   |  Decreased Odds of Dementia          |
|   Education   |    21   |  Abnormal for 8th Grade Education    |
|               |   <23   |  Abnormal for High School Education  |
|               |   <24   |  Abnormal for College Education      |
|   Severity    |  24-30  |  No Cognitive Impairment             |
|               |  18-23  |  Mild Cognitive Impairment           |
|               |   0-17  |  Severe Cognitive Impairment         |

#### Clinical Dementia Rating (CDR)
> The CDR is a 5-point scale used to characterize six domains of cognitive and functional performance applicable to Alzheimer disease and related dementias: Memory, Orientation, Judgment & Problem Solving, Community Affairs, Home & Hobbies, and Personal Care. The necessary information to make each rating is obtained through a semi-structured interview of the patient and a reliable informant or collateral source (e.g., family member).

> The CDR table provides descriptive anchors that guide the clinician in making appropriate ratings based on interview data and clinical judgment. In addition to ratings for each domain, an overall CDR score may be calculated through the use of an algorithm. This score is useful for characterizing and tracking a patient's level of impairment/dementia:

| Score  | Description               |
|--------|---------------------------|
|    0   |        Normal             |
|    0.5 | Very Mild Dementia        |
|    1   | Mild Dementia             |
|    2   | Moderate Dementia         |
|    3   | Severe Dementia           |

##### Estimated Total Intracranial Volume(eTIV)
> Total intracranial volume (TIV/ICV) is an important covariate for volumetric analyses of the brain and brain regions, especially in the study of neurodegenerative diseases, where it can provide a proxy of maximum pre-morbid brain volume.

> Unlike brain atrophy in the patients with AD, TIV did not vary over time. Mean TIV did not differ significantly between any of the subject groups. There was no association between TIV and age or age at symptom onset. The only significant predictor of TIV was sex. Men showed an approximately ∼12% larger eTIV than women.

```python
import pandas as pd
import seaborn as sns
import numpy as np
import matplotlib.pyplot as plt

import xgboost as xgb
from sklearn.datasets import make_blobs
import pandas as pd
```

```python
import plotly.graph_objects as go
import plotly.express as px
from jupyter_dash import JupyterDash
import dash_core_components as dcc
import dash_html_components as html
from dash.dependencies import Input, Output
```

```python
## Read the CSV File Using Pandas read_csv function
#
from google.colab import files
data_to_load = files.upload()
df = pd.read_csv('oasis_longitudinal_original.csv')
# print the concise summery of the dataset

```

<input type="file" id="files-27d41aec-4c0b-4e97-ab41-169b23e2c344" name="files[]" multiple disabled
   style="border:none" />
<output id="result-27d41aec-4c0b-4e97-ab41-169b23e2c344">
 Upload widget is only available when the cell has been executed in the
 current browser session. Please rerun this cell to enable.
 </output>

```python
df.info()
```

    <class 'pandas.core.frame.DataFrame'>
    RangeIndex: 373 entries, 0 to 372
    Data columns (total 20 columns):
     #   Column       Non-Null Count  Dtype
    ---  ------       --------------  -----
     0   Subject ID   373 non-null    object
     1   MRI ID       373 non-null    object
     2   Group        373 non-null    object
     3   Visit        373 non-null    int64
     4   MR Delay     373 non-null    int64
     5   M/F          373 non-null    object
     6   Hand         373 non-null    object
     7   Glucose      373 non-null    int64
     8   Insulin      373 non-null    int64
     9   Age          373 non-null    int64
     10  EDUC         373 non-null    int64
     11  SES          354 non-null    float64
     12  MMSE         371 non-null    float64
     13  CDR          373 non-null    float64
     14  eTIV         373 non-null    int64
     15  nWBV         373 non-null    float64
     16  ASF          373 non-null    float64
     17  Unnamed: 17  0 non-null      float64
     18  Unnamed: 18  0 non-null      float64
     19  Unnamed: 19  3 non-null      object
    dtypes: float64(7), int64(7), object(6)
    memory usage: 58.4+ KB

```python
print("Tota Rows and Columns (Rows,Columns) : ",df.shape)
#print first five rows of the dataset
df.head(5)
```

    Tota Rows and Columns (Rows,Columns) :  (373, 20)

      Subject ID
      MRI ID
      Group
      Visit
      MR Delay
      M/F
      Hand
      Glucose
      Insulin
      Age
      EDUC
      SES
      MMSE
      CDR
      eTIV
      nWBV
      ASF
      Unnamed: 17
      Unnamed: 18
      Unnamed: 19

      0
      OAS2_0001
      OAS2_0001_MR1
      0
      1
      0
      M
      R
      89
      94
      87
      14
      2.0
      27.0
      0.0
      1987
      0.696
      0.883
      NaN
      NaN
      NaN

      1
      OAS2_0001
      OAS2_0001_MR2
      0
      2
      457
      M
      R
      137
      168
      88
      14
      2.0
      30.0
      0.0
      2004
      0.681
      0.876
      NaN
      NaN
      NaN

      2
      OAS2_0002
      OAS2_0002_MR1
      1
      1
      0
      M
      R
      116
      0
      75
      12
      NaN
      23.0
      0.5
      1678
      0.736
      1.046
      NaN
      NaN
      NaN

      3
      OAS2_0002
      OAS2_0002_MR2
      1
      2
      560
      M
      R
      78
      88
      76
      12
      NaN
      28.0
      0.5
      1738
      0.713
      1.010
      NaN
      NaN
      NaN

      4
      OAS2_0002
      OAS2_0002_MR3
      1
      3
      1895
      M
      R
      115
      0
      80
      12
      NaN
      22.0
      0.5
      1698
      0.701
      1.034
      NaN
      NaN
      NaN

```python
#print concise summery of the dataset
df.describe()
```

      Visit
      MR Delay
      Glucose
      Insulin
      Age
      EDUC
      SES
      MMSE
      CDR
      eTIV
      nWBV
      ASF
      Unnamed: 17
      Unnamed: 18

      count
      373.000000
      373.000000
      373.000000
      373.000000
      373.000000
      373.000000
      354.000000
      371.000000
      373.000000
      373.000000
      373.000000
      373.000000
      0.0
      0.0

      mean
      1.882038
      595.104558
      120.667560
      81.286863
      77.013405
      14.597855
      2.460452
      27.342318
      0.290885
      1488.128686
      0.729568
      1.195461
      NaN
      NaN

      std
      0.922843
      635.485118
      32.793537
      119.914377
      7.640957
      2.876339
      1.134005
      3.683244
      0.374557
      176.139286
      0.037135
      0.138092
      NaN
      NaN

      min
      1.000000
      0.000000
      0.000000
      0.000000
      60.000000
      6.000000
      1.000000
      4.000000
      0.000000
      1106.000000
      0.644000
      0.876000
      NaN
      NaN

      25%
      1.000000
      0.000000
      99.000000
      0.000000
      71.000000
      12.000000
      2.000000
      27.000000
      0.000000
      1357.000000
      0.700000
      1.099000
      NaN
      NaN

      50%
      2.000000
      552.000000
      116.000000
      43.000000
      77.000000
      15.000000
      2.000000
      29.000000
      0.000000
      1470.000000
      0.729000
      1.194000
      NaN
      NaN

      75%
      2.000000
      873.000000
      142.000000
      120.000000
      82.000000
      16.000000
      3.000000
      30.000000
      0.500000
      1597.000000
      0.756000
      1.293000
      NaN
      NaN

      max
      5.000000
      2639.000000
      197.000000
      846.000000
      98.000000
      23.000000
      5.000000
      30.000000
      2.000000
      2004.000000
      0.837000
      1.587000
      NaN
      NaN

```python
#since the dataset contain null values also
#count total rows in each column which contain null values
df.isna().sum()
```

    Subject ID       0
    MRI ID           0
    Group            0
    Visit            0
    MR Delay         0
    M/F              0
    Hand             0
    Glucose          0
    Insulin          0
    Age              0
    EDUC             0
    SES             19
    MMSE             2
    CDR              0
    eTIV             0
    nWBV             0
    ASF              0
    Unnamed: 17    373
    Unnamed: 18    373
    Unnamed: 19    370
    dtype: int64

```python
#'duplicated()' function in pandas return the duplicate row as True and othter as False
#for counting the duplicate elements we sum all the rows
sum(df.duplicated())
```

    0

```python
#fill null value with their column mean and median
df["SES"].fillna(df["SES"].median(), inplace=True)
df["MMSE"].fillna(df["MMSE"].mean(), inplace=True)
```

```python
#see how many people have Alzheimer
#same person visits two or more time so only take the single visit data
sns.set_style("whitegrid")
ex_df = df.loc[df['Visit'] == 1]
sns.countplot(x='Group', data=ex_df)
```

```python
#We have three groups so convert Converted Group Into Demented

ex_df['Group'] = ex_df['Group'].replace(['Converted'], ['Demented'])
df['Group'] = df['Group'].replace(['Converted'], ['Demented'])
sns.countplot(x='Group', data=ex_df)
```

```python
# bar drawing function
def bar_chart(feature):
    Demented = ex_df[ex_df['Group']=='Demented'][feature].value_counts()
    Nondemented = ex_df[ex_df['Group']=='Nondemented'][feature].value_counts()
    df_bar = pd.DataFrame([Demented,Nondemented])
    df_bar.index = ['Demented','Nondemented']
    df_bar.plot(kind='bar',stacked=True, figsize=(8,5))
    print(df_bar)

# Gender  and  Group ( Female=0, Male=1)
bar_chart('M/F')
plt.xlabel('Group',fontsize=13)
plt.xticks(rotation=0,fontsize=12)
plt.ylabel('Number of patients',fontsize=13)
plt.legend()
plt.title('Gender and Demented rate',fontsize=14)
```

```python
plt.figure(figsize=(10,5))
sns.violinplot(x='M/F', y='Glucose', data=df)
plt.title('Violin plots of Glucose by Gender',fontsize=14)
plt.xlabel('Gender',fontsize=13)
plt.ylabel('Glucose',fontsize=13)
plt.show()
```

```python
ax = sns.countplot(x='Glucose', data=ex_df)
ax.figure.set_size_inches(18.5, 5)
```

```python
#MMSE Vs Non-demented and demented
facet= sns.FacetGrid(df,hue="Group", aspect=3)
facet.map(sns.kdeplot,'Glucose',shade= True)
facet.set(xlim=(0, df['Glucose'].max()))
facet.add_legend()
plt.xlim(15.30)
```

>**The chart shows Nondemented group got much more higher Glucose values than Demented group.**

```python
#find the outliers in each of the column
def outliers_iqr(ys):
    quartile_1, quartile_3 = np.percentile(ys, [25, 75])
    iqr = quartile_3 - quartile_1
    lower_bound = quartile_1 - (iqr * 1.5)
    upper_bound = quartile_3 + (iqr * 1.5)
    return np.where((ys > upper_bound) | (ys < lower_bound))

list_atributes = ['MR Delay','EDUC', "SES", "MMSE", 'eTIV', "nWBV", "ASF","Glucose","Insulin"]
print("Outliers: \n")
for item in list_atributes:
    print(item,': ',outliers_iqr(df[item]))
```

    Outliers:

    MR Delay :  (array([ 32,  71,  75, 153, 159, 160, 265, 369]),)
    EDUC :  (array([107, 108, 109]),)
    SES :  (array([136, 137, 138, 161, 162, 179, 180]),)
    MMSE :  (array([  4,  25,  26,  43,  44,  51,  52,  60,  88,  89,  90,  93,  94,
            97,  98,  99, 100, 101, 105, 106, 138, 162, 172, 173, 184, 185,
           186, 222, 225, 226, 231, 232, 234, 251, 299, 300, 316, 317, 328,
           332, 360, 366]),)
    eTIV :  (array([0, 1]),)
    nWBV :  (array([], dtype=int64),)
    ASF :  (array([282]),)
    Glucose :  (array([ 58, 218, 225]),)
    Insulin :  (array([  5,   7,  39,  92, 119, 133, 165, 178, 198, 206, 209, 236, 246,
           251, 268, 285, 288, 291, 355]),)

```python
from pylab import rcParams
rcParams['figure.figsize'] = 8,5
cols = ['Insulin','Glucose','Age','MMSE']
x=df.fillna('')
sns_plot = sns.pairplot(x[cols])
```

```python
#boxplots which shows the IQR(Interquartile Range )
fig, axes = plt.subplots(2,3,figsize = (16,6))
fig.suptitle("Box Plot",fontsize=14)
sns.set_style("whitegrid")
sns.boxplot(data=df['Glucose'], orient="v",width=0.4, palette="colorblind",ax = axes[0][0]);
sns.boxplot(data=df['Insulin'], orient="v",width=0.4, palette="colorblind",ax = axes[0][1]);
sns.boxplot(data=df['MMSE'], orient="v",width=0.4, palette="colorblind",ax = axes[0][2]);
sns.boxplot(data=df['CDR'], orient="v",width=0.4, palette="colorblind",ax = axes[1][0]);
sns.boxplot(data=df['eTIV'], orient="v",width=0.4, palette="colorblind",ax = axes[1][1]);
sns.boxplot(data=df['ASF'], orient="v",width=0.4, palette="colorblind",ax = axes[1][2]);
#xlabel("Time");
```

```python
def plot_correlation_map( df ):
    corr = df.corr()
    _ , ax = plt.subplots( figsize =( 12 , 10 ) )
    cmap = sns.diverging_palette( 240 , 10 , as_cmap = True )
    _ = sns.heatmap(corr,cmap = cmap,square=True, cbar_kws={ 'shrink' : .9 }, ax=ax, annot = True, annot_kws = { 'fontsize' : 12 })
```

```python
plot_correlation_map(df)
```

# XG Boost Classifier

```python
import pandas as pd
from xgboost import XGBClassifier
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score
```

```python
from google.colab import files
upload=files.upload()

diabetes = pd.read_csv('oasis_longitudinal_original.csv')
cols = diabetes.columns
X = diabetes[["Age", "EDUC", "SES", "MMSE", "eTIV", "nWBV", "ASF"]]
y = []
for i in diabetes['Group']:
    if i=="Nondemented":
        y.append(0)
    else:
        y.append(1)
print(y)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33)
model = XGBClassifier()
model.fit(X_train, y_train)
y_pred = model.predict(X_test)
predictions = [round(value) for value in y_pred]
accuracy = accuracy_score(y_test, predictions)
print("Accuracy: %.2f%%" % (accuracy * 100.0))
```

<input type="file" id="files-a391be07-13a2-4c07-a336-aaef42057042" name="files[]" multiple disabled
   style="border:none" />
<output id="result-a391be07-13a2-4c07-a336-aaef42057042">
 Upload widget is only available when the cell has been executed in the
 current browser session. Please rerun this cell to enable.
 </output>

    Saving oasis_longitudinal_original.csv to oasis_longitudinal_original (3).csv
    [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
    Accuracy: 100.00%

```python
from google.colab import files
upload=files.upload()

```

<input type="file" id="files-a209a9cf-0ab8-4200-b9d1-b3d78ff38e48" name="files[]" multiple disabled
   style="border:none" />
<output id="result-a209a9cf-0ab8-4200-b9d1-b3d78ff38e48">
 Upload widget is only available when the cell has been executed in the
 current browser session. Please rerun this cell to enable.
 </output>

    Saving oasis_longitudinal_original_1.xlsx to oasis_longitudinal_original_1 (1).xlsx

```python
df.head(5)
```

      Subject ID
      MRI ID
      Group
      Visit
      MR Delay
      M/F
      Hand
      Glucose
      Insulin
      Age
      EDUC
      SES
      MMSE
      CDR
      eTIV
      nWBV
      ASF
      Unnamed: 17
      Unnamed: 18
      Unnamed: 19

      0
      0
      0
      0
      0
      0
      1
      0
      24
      46
      27
      5
      1
      14
      0
      284
      35
      1
      0
      0
      NaN

      1
      0
      1
      0
      1
      15
      1
      0
      72
      77
      28
      5
      1
      18
      0
      285
      20
      0
      252
      252
      NaN

      2
      1
      2
      1
      0
      0
      1
      0
      51
      0
      15
      3
      1
      10
      1
      231
      73
      52
      251
      251
      NaN

      3
      1
      3
      1
      1
      32
      1
      0
      13
      42
      16
      3
      1
      16
      1
      254
      51
      32
      250
      250
      NaN

      4
      1
      4
      1
      2
      185
      1
      0
      50
      0
      20
      3
      1
      9
      1
      238
      40
      46
      249
      249
      NaN

```python
from sklearn.model_selection import train_test_split

feature_col_names = ["M/F","Insulin","Glucose","Age", "EDUC", "SES", "MMSE", "eTIV", "nWBV", "ASF"]
predicted_class_names = ['Group']

X = df[feature_col_names].values
y = df[predicted_class_names].values

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.30, random_state=42)
```

```python
from sklearn.preprocessing import label_binarize
```

```python
y = label_binarize(y, classes=[0, 1, 2, 3])
```

```python
X_test
```

    array([[  1,   0,  69, ..., 181,  17, 101],
           [  0,   0,  41, ..., 104,  53, 167],
           [  1,  68,  78, ..., 135, 128, 137],
           ...,
           [  0,  58, 102, ..., 109,  76, 162],
           [  0, 101,  62, ...,  67,  89, 202],
           [  0,   0,  70, ..., 163,  63, 116]])

```python
from sklearn import metrics
def plot_confusion_metrix(y_test,model_test):
    cm = metrics.confusion_matrix(y_test, model_test)
    plt.figure(1)
    plt.clf()
    plt.imshow(cm, interpolation='nearest', cmap=plt.cm.Wistia)
    classNames = ['Nondemented','Demented']
    plt.title('Confusion Matrix')
    plt.ylabel('True label')
    plt.xlabel('Predicted label')
    tick_marks = np.arange(len(classNames))
    plt.xticks(tick_marks, classNames)
    plt.yticks(tick_marks, classNames)
    s = [['TN','FP'], ['FN', 'TP']]
    for i in range(2):
        for j in range(2):
            plt.text(j,i, str(s[i][j])+" = "+str(cm[i][j]))
    plt.show()
```

```python
from sklearn.metrics import roc_curve, auc
def report_performance(model):

    model_test = model.predict(X_test)

    print("\n\nConfusion Matrix:")
    print("{0}".format(metrics.confusion_matrix(y_test, model_test)))
    print("\n\nClassification Report: ")
    print(metrics.classification_report(y_test, model_test))
    #cm = metrics.confusion_matrix(y_test, model_test)
    plot_confusion_metrix(y_test, model_test)

def roc_curves(model):
    predictions_test = model.predict(X_test)
    fpr, tpr, thresholds = roc_curve(predictions_test,y_test)
    roc_auc = auc(fpr, tpr)

    plt.figure()
    plt.plot(fpr, tpr, color='darkorange', lw=1, label='ROC curve (area = %0.2f)' % roc_auc)
    plt.plot([0, 1], [0, 1], color='navy', lw=2, linestyle='--')
    plt.xlim([0.0, 1.0])
    plt.ylim([0.0, 1.05])
    plt.xlabel('False Positive Rate')
    plt.ylabel('True Positive Rate')
    plt.title('Receiver operating characteristic')
    plt.legend(loc="lower right")
    plt.show()
```

```python
total_models = ['GridSearchCV','SVC','DecisionTreeClassifier','XGBClassifier','LGBMClassifier',
                    'ExtraTreesClassifier','GradientBoostingClassifier','AdaBoostClassifier','KNeighborsClassifier',
                    'MLPClassifier','GaussianNB','LogisticRegression']
total_accuracy = {}
for i in total_models:
    total_accuracy[str(i)] = 0
def accuracy(model):
    pred = model.predict(X_test)
    accu = metrics.accuracy_score(y_test,pred)
    print("\nAcuuracy Of the Model: ",accu,"\n\n")
    total_accuracy[str((str(model).split('(')[0]))] = accu
```

```python
from sklearn.gaussian_process import GaussianProcessClassifier
from sklearn.neighbors import KNeighborsClassifier
from sklearn.svm import SVC
from sklearn.tree import DecisionTreeClassifier
from sklearn import metrics
from xgboost import XGBClassifier
from sklearn.ensemble import ExtraTreesClassifier
from sklearn.tree import ExtraTreeClassifier
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import GradientBoostingClassifier
from sklearn.ensemble import BaggingClassifier
from sklearn.ensemble import AdaBoostClassifier
from sklearn.ensemble import RandomForestClassifier
from sklearn.linear_model import LogisticRegression
from lightgbm import LGBMClassifier
from sklearn.model_selection import GridSearchCV
from sklearn.linear_model import LogisticRegression
from sklearn.naive_bayes import GaussianNB
from sklearn.neural_network import MLPClassifier
```

```python
from sklearn.feature_selection import RFECV
from sklearn.model_selection import GridSearchCV
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
```

```python
import numpy as np
from sklearn.svm import SVC
from sklearn.multiclass import OneVsRestClassifier
from sklearn.preprocessing import MultiLabelBinarizer
```

```python
import matplotlib.pyplot as plt
import numpy as np

from sklearn.metrics import accuracy_score
from sklearn.linear_model import LogisticRegression
from sklearn.svm import SVC
from sklearn.gaussian_process import GaussianProcessClassifier
from sklearn.gaussian_process.kernels import RBF
from sklearn import datasets
```

```python
from google.colab import files
upload=files.upload()
```

<input type="file" id="files-f9fc7955-8382-44d2-b49a-7c2194f29836" name="files[]" multiple disabled
   style="border:none" />
<output id="result-f9fc7955-8382-44d2-b49a-7c2194f29836">
 Upload widget is only available when the cell has been executed in the
 current browser session. Please rerun this cell to enable.
 </output>

    Saving oasis_longitudinal_original_1.xlsx to oasis_longitudinal_original_1 (2).xlsx

# Receiver Operating Characteristic (ROC)

```python
import numpy as np
import matplotlib.pyplot as plt
from itertools import cycle

from sklearn import svm, datasets
from sklearn.metrics import roc_curve, auc
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import label_binarize
from sklearn.multiclass import OneVsRestClassifier
from scipy import interp
from sklearn.metrics import roc_auc_score
```

```python
# Binarize the output
y = label_binarize(y, classes=[0, 1, 2,3])
n_classes = y.shape[1]
```

```python
# Add noisy features to make the problem harder
random_state = np.random.RandomState(0)
n_samples, n_features = X.shape
X = np.c_[X, random_state.randn(n_samples, 200 * n_features)]
```

```python
# shuffle and split training and test sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=.5,
                                                    random_state=0)
```

```python
# Learn to predict each class against the other
classifier = OneVsRestClassifier(svm.SVC(kernel='linear', probability=True,
                                 random_state=random_state))
y_score = classifier.fit(X_train, y_train).decision_function(X_test)
```

    /usr/local/lib/python3.7/dist-packages/sklearn/multiclass.py:75: UserWarning: Label not 3 is present in all training examples.
      str(classes[c]))

```python
# Compute ROC curve and ROC area for each class
fpr = dict()
tpr = dict()
roc_auc = dict()
for i in range(n_classes):
    fpr[i], tpr[i], _ = roc_curve(y_test[:, i], y_score[:, i])
    roc_auc[i] = auc(fpr[i], tpr[i])
```

    /usr/local/lib/python3.7/dist-packages/sklearn/metrics/_ranking.py:808: UndefinedMetricWarning: No positive samples in y_true, true positive value should be meaningless
      UndefinedMetricWarning)

```python
# Compute micro-average ROC curve and ROC area
fpr["micro"], tpr["micro"], _ = roc_curve(y_test.ravel(), y_score.ravel())
roc_auc["micro"] = auc(fpr["micro"], tpr["micro"])
```

```python
plt.figure()
lw = 2
plt.plot(fpr[2], tpr[2], color='darkorange',
         lw=lw, label='ROC curve (area = %0.2f)' % roc_auc[2])
plt.plot([0, 1], [0, 1], color='navy', lw=lw, linestyle='--')
plt.xlim([0.0, 1.0])
plt.ylim([0.0, 1.05])
plt.xlabel('False Positive Rate')
plt.ylabel('True Positive Rate')
plt.title('Receiver operating characteristic example')
plt.legend(loc="lower right")
plt.show()
```

```python
# First aggregate all false positive rates
all_fpr = np.unique(np.concatenate([fpr[i] for i in range(n_classes)]))

# Then interpolate all ROC curves at this points
mean_tpr = np.zeros_like(all_fpr)
for i in range(n_classes):
    mean_tpr += interp(all_fpr, fpr[i], tpr[i])

# Finally average it and compute AUC
mean_tpr /= n_classes

fpr["macro"] = all_fpr
tpr["macro"] = mean_tpr
roc_auc["macro"] = auc(fpr["macro"], tpr["macro"])

# Plot all ROC curves
plt.figure()
plt.plot(fpr["micro"], tpr["micro"],
         label='micro-average ROC curve (area = {0:0.2f})'
               ''.format(roc_auc["micro"]),
         color='deeppink', linestyle=':', linewidth=4)

plt.plot(fpr["macro"], tpr["macro"],
         label='macro-average ROC curve (area = {0:0.2f})'
               ''.format(roc_auc["macro"]),
         color='navy', linestyle=':', linewidth=4)

colors = cycle(['aqua', 'darkorange', 'cornflowerblue'])
for i, color in zip(range(n_classes), colors):
    plt.plot(fpr[i], tpr[i], color=color, lw=lw,
             label='ROC curve of class {0} (area = {1:0.2f})'
             ''.format(i, roc_auc[i]))

plt.plot([0, 1], [0, 1], 'k--', lw=lw)
plt.xlim([0.0, 1.0])
plt.ylim([0.0, 1.05])
plt.xlabel('False Positive Rate')
plt.ylabel('True Positive Rate')
plt.title('Some extension of Receiver operating characteristic to multi-class')
plt.legend(loc="lower right")
plt.show()
```

```python
print(__doc__)
import matplotlib.pyplot as plt
import seaborn as sns
```

    Automatically created module for IPython interactive environment

```python
sns.scatterplot(
    x=X[:, 0], y=X[:, 1], hue=y,
    marker='o', s=25, edgecolor='k', legend=False
).set_title("Data")
plt.show()
```

```python
from sklearn.model_selection import GridSearchCV, RepeatedStratifiedKFold
from sklearn.svm import SVC

param_grid = [
    {'kernel': ['linear']},
    {'kernel': ['poly'], 'degree': [2, 3]},
    {'kernel': ['rbf']}
]

svc = SVC(random_state=0)

cv = RepeatedStratifiedKFold(
    n_splits=10, n_repeats=10, random_state=0
)

search = GridSearchCV(
    estimator=svc, param_grid=param_grid,
    scoring='roc_auc', cv=cv
)
search.fit(X, y)
```

    GridSearchCV(cv=RepeatedStratifiedKFold(n_repeats=10, n_splits=10, random_state=0),
                 error_score=nan,
                 estimator=SVC(C=1.0, break_ties=False, cache_size=200,
                               class_weight=None, coef0=0.0,
                               decision_function_shape='ovr', degree=3,
                               gamma='scale', kernel='rbf', max_iter=-1,
                               probability=False, random_state=0, shrinking=True,
                               tol=0.001, verbose=False),
                 iid='deprecated', n_jobs=None,
                 param_grid=[{'kernel': ['linear']},
                             {'degree': [2, 3], 'kernel': ['poly']},
                             {'kernel': ['rbf']}],
                 pre_dispatch='2*n_jobs', refit=True, return_train_score=False,
                 scoring='roc_auc', verbose=0)

```python
import pandas as pd

results_df = pd.DataFrame(search.cv_results_)
results_df = results_df.sort_values(by=['rank_test_score'])
results_df = (
    results_df
    .set_index(results_df["params"].apply(
        lambda x: "_".join(str(val) for val in x.values()))
    )
    .rename_axis('kernel')
)
results_df[
    ['params', 'rank_test_score', 'mean_test_score', 'std_test_score']
]
```

      params
      rank_test_score
      mean_test_score
      std_test_score

      kernel

      rbf
      {'kernel': 'rbf'}
      1
      0.9400
      0.079297

      linear
      {'kernel': 'linear'}
      2
      0.9300
      0.077846

      3_poly
      {'degree': 3, 'kernel': 'poly'}
      3
      0.9044
      0.098776

      2_poly
      {'degree': 2, 'kernel': 'poly'}
      4
      0.6852
      0.169106

# We can see that the estimator using the 'rbf' kernel performed best, closely followed by 'linear'. Both estimators with a 'poly' kernel performed worse, with the one using a two-degree polynomial achieving a much lower perfomance than all other models.

```python
# create df of model scores ordered by perfomance
model_scores = results_df.filter(regex=r'split\d*_test_score')

# plot 30 examples of dependency between cv fold and AUC scores
fig, ax = plt.subplots()
sns.lineplot(
    data=model_scores.transpose().iloc[:30],
    dashes=False, palette='Set1', marker='o', alpha=.5, ax=ax
)
ax.set_xlabel("CV test fold", size=12, labelpad=10)
ax.set_ylabel("Model AUC", size=12)
ax.tick_params(bottom=True, labelbottom=False)
plt.show()

# print correlation of AUC scores across folds
print(f"Correlation of models:\n {model_scores.transpose().corr()}")
```

# We can observe that the performance of the models highly depends on the fold.

As a consequence, if we assume independence between samples we will be underestimating the variance computed in our statistical tests, increasing the number of false positive errors (i.e. detecting a significant difference between models when such does not exist) [1]_.

Several variance-corrected statistical tests have been developed for these cases. In this example we will show how to implement one of them (the so called Nadeau and Bengio's corrected t-test) under two different statistical frameworks: frequentist and Bayesian.

```python
import numpy as np
from scipy.stats import t

def corrected_std(differences, n_train, n_test):
    """Corrects standard deviation using Nadeau and Bengio's approach.

    Parameters
    ----------
    differences : ndarray of shape (n_samples, 1)
        Vector containing the differences in the score metrics of two models.
    n_train : int
        Number of samples in the training set.
    n_test : int
        Number of samples in the testing set.

    Returns
    -------
    corrected_std : int
        Variance-corrected standard deviation of the set of differences.
    """
    n = n_train + n_test
    corrected_var = (
        np.var(differences, ddof=1) * ((1 / n) + (n_test / n_train))
    )
    corrected_std = np.sqrt(corrected_var)
    return corrected_std

def compute_corrected_ttest(differences, df, n_train, n_test):
    """Computes right-tailed paired t-test with corrected variance.

    Parameters
    ----------
    differences : array-like of shape (n_samples, 1)
        Vector containing the differences in the score metrics of two models.
    df : int
        Degrees of freedom.
    n_train : int
        Number of samples in the training set.
    n_test : int
        Number of samples in the testing set.

    Returns
    -------
    t_stat : float
        Variance-corrected t-statistic.
    p_val : float
        Variance-corrected p-value.
    """
    mean = np.mean(differences)
    std = corrected_std(differences, n_train, n_test)
    t_stat = mean / std
    p_val = t.sf(np.abs(t_stat), df)  # right-tailed t-test
    return t_stat, p_val
```

```python
model_1_scores = model_scores.iloc[0].values  # scores of the best model
model_2_scores = model_scores.iloc[1].values  # scores of the second-best model

differences = model_1_scores - model_2_scores

n = differences.shape[0]  # number of test sets
df = n - 1
n_train = len(list(cv.split(X, y))[0][0])
n_test = len(list(cv.split(X, y))[0][1])

t_stat, p_val = compute_corrected_ttest(differences, df, n_train, n_test)
print(f"Corrected t-value: {t_stat:.3f}\n"
      f"Corrected p-value: {p_val:.3f}")
```

    Corrected t-value: 0.750
    Corrected p-value: 0.227

```python
t_stat_uncorrected = (
    np.mean(differences) / np.sqrt(np.var(differences, ddof=1) / n)
)
p_val_uncorrected = t.sf(np.abs(t_stat_uncorrected), df)

print(f"Uncorrected t-value: {t_stat_uncorrected:.3f}\n"
      f"Uncorrected p-value: {p_val_uncorrected:.3f}")
```

    Uncorrected t-value: 2.611
    Uncorrected p-value: 0.005

# Using the conventional significance alpha level at p=0.05, we observe that the uncorrected t-test concludes that the first model is significantly better than the second.

With the corrected approach, in contrast, we fail to detect this difference.

In the latter case, however, the frequentist approach does not let us conclude that the first and second model have an equivalent performance. If we wanted to make this assertion we need to use a Bayesian approach.

## Comparing two models: Bayesian approach
We can use Bayesian estimation to calculate the probability that the first
model is better than the second. Bayesian estimation will output a
distribution followed by the mean $\mu$ of the differences in the
performance of two models.

To obtain the posterior distribution we need to define a prior that models
our beliefs of how the mean is distributed before looking at the data,
and multiply it by a likelihood function that computes how likely our
observed differences are, given the values that the mean of differences
could take.

Bayesian estimation can be carried out in many forms to answer our question,
but in this example we will implement the approach suggested by Benavoli and
collegues [4]_.

One way of defining our posterior using a closed-form expression is to select
a prior conjugate to the likelihood function. Benavoli and collegues [4]_
show that when comparing the performance of two classifiers we can model the
prior as a Normal-Gamma distribution (with both mean and variance unknown)
conjugate to a normal likelihood, to thus express the posterior as a normal
distribution.
Marginalizing out the variance from this normal posterior, we can define the
posterior of the mean parameter as a Student's t-distribution. Specifically:

\begin{align}St(\mu;n-1,\overline{x},(\frac{1}{n}+\frac{n_{test}}{n_{train}})
   \hat{\sigma}^2)\end{align}

where $n$ is the total number of samples,
$\overline{x}$ represents the mean difference in the scores,
$n_{test}$ is the number of samples used for testing,
$n_{train}$ is the number of samples used for training,
and $\hat{\sigma}^2$ represents the variance of the observed
differences.

Notice that we are using Nadeau and Bengio's corrected variance in our
Bayesian approach as well.

Let's compute and plot the posterior:

```python
# intitialize random variable
t_post = t(
    df, loc=np.mean(differences),
    scale=corrected_std(differences, n_train, n_test)
)
```

```python
x = np.linspace(t_post.ppf(0.001), t_post.ppf(0.999), 100)

plt.plot(x, t_post.pdf(x))
plt.xticks(np.arange(-0.04, 0.06, 0.01))
plt.fill_between(x, t_post.pdf(x), 0, facecolor='blue', alpha=.2)
plt.ylabel("Probability density")
plt.xlabel(r"Mean difference ($\mu$)")
plt.title("Posterior distribution")
plt.show()
```

We can calculate the probability that the first model is better than the second by computing the area under the curve of the posterior distirbution from zero to infinity. And also the reverse: we can calculate the probability that the second model is better than the first by computing the area under the curve from minus infinity to zero.

```python
better_prob = 1 - t_post.cdf(0)

print(f"Probability of {model_scores.index[0]} being more accurate than "
      f"{model_scores.index[1]}: {better_prob:.3f}")
print(f"Probability of {model_scores.index[1]} being more accurate than "
      f"{model_scores.index[0]}: {1 - better_prob:.3f}")
```

    Probability of rbf being more accurate than linear: 0.773
    Probability of linear being more accurate than rbf: 0.227

```python
rope_interval = [-0.01, 0.01]
rope_prob = t_post.cdf(rope_interval[1]) - t_post.cdf(rope_interval[0])

print(f"Probability of {model_scores.index[0]} and {model_scores.index[1]} "
      f"being practically equivalent: {rope_prob:.3f}")
```

    Probability of rbf and linear being practically equivalent: 0.432

```python
x_rope = np.linspace(rope_interval[0], rope_interval[1], 100)

plt.plot(x, t_post.pdf(x))
plt.xticks(np.arange(-0.04, 0.06, 0.01))
plt.vlines([-0.01, 0.01], ymin=0, ymax=(np.max(t_post.pdf(x)) + 1))
plt.fill_between(x_rope, t_post.pdf(x_rope), 0, facecolor='blue', alpha=.2)
plt.ylabel("Probability density")
plt.xlabel(r"Mean difference ($\mu$)")
plt.title("Posterior distribution under the ROPE")
plt.show()
```

```python
cred_intervals = []
intervals = [0.5, 0.75, 0.95]

for interval in intervals:
    cred_interval = list(t_post.interval(interval))
    cred_intervals.append([interval, cred_interval[0], cred_interval[1]])

cred_int_df = pd.DataFrame(
    cred_intervals,
    columns=['interval', 'lower value', 'upper value']
).set_index('interval')
cred_int_df
```

      lower value
      upper value

      interval

      0.50
      0.000977
      0.019023

      0.75
      -0.005422
      0.025422

      0.95
      -0.016445
      0.036445

```python
from itertools import combinations
from math import factorial

n_comparisons = (
    factorial(len(model_scores))
    / (factorial(2) * factorial(len(model_scores) - 2))
)
pairwise_t_test = []

for model_i, model_k in combinations(range(len(model_scores)), 2):
    model_i_scores = model_scores.iloc[model_i].values
    model_k_scores = model_scores.iloc[model_k].values
    differences = model_i_scores - model_k_scores
    t_stat, p_val = compute_corrected_ttest(
        differences, df, n_train, n_test
    )
    p_val *= n_comparisons  # implement Bonferroni correction
    # Bonferroni can output p-values higher than 1
    p_val = 1 if p_val > 1 else p_val
    pairwise_t_test.append(
        [model_scores.index[model_i], model_scores.index[model_k],
         t_stat, p_val]
    )

pairwise_comp_df = pd.DataFrame(
    pairwise_t_test,
    columns=['model_1', 'model_2', 't_stat', 'p_val']
).round(3)
pairwise_comp_df
```

      model_1
      model_2
      t_stat
      p_val

      0
      rbf
      linear
      0.750
      1.000

      1
      rbf
      3_poly
      1.657
      0.302

      2
      rbf
      2_poly
      4.565
      0.000

      3
      linear
      3_poly
      1.111
      0.807

      4
      linear
      2_poly
      4.276
      0.000

      5
      3_poly
      2_poly
      3.851
      0.001

We observe that after correcting for multiple comparisons, the only model
that significantly differs from the others is `'2_poly'`.
`'rbf'`, the model ranked first by
:class:`~sklearn.model_selection.GridSearchCV`, does not significantly
differ from `'linear'` or `'3_poly'`.

```python
pairwise_bayesian = []

for model_i, model_k in combinations(range(len(model_scores)), 2):
    model_i_scores = model_scores.iloc[model_i].values
    model_k_scores = model_scores.iloc[model_k].values
    differences = model_i_scores - model_k_scores
    t_post = t(
        df, loc=np.mean(differences),
        scale=corrected_std(differences, n_train, n_test)
    )
    worse_prob = t_post.cdf(rope_interval[0])
    better_prob = 1 - t_post.cdf(rope_interval[1])
    rope_prob = t_post.cdf(rope_interval[1]) - t_post.cdf(rope_interval[0])

    pairwise_bayesian.append([worse_prob, better_prob, rope_prob])

pairwise_bayesian_df = (pd.DataFrame(
    pairwise_bayesian,
    columns=['worse_prob', 'better_prob', 'rope_prob']
).round(3))

pairwise_comp_df = pairwise_comp_df.join(pairwise_bayesian_df)
pairwise_comp_df
```

      model_1
      model_2
      t_stat
      p_val
      worse_prob
      better_prob
      rope_prob

      0
      rbf
      linear
      0.750
      1.000
      0.068
      0.500
      0.432

      1
      rbf
      3_poly
      1.657
      0.302
      0.018
      0.882
      0.100

      2
      rbf
      2_poly
      4.565
      0.000
      0.000
      1.000
      0.000

      3
      linear
      3_poly
      1.111
      0.807
      0.063
      0.750
      0.187

      4
      linear
      2_poly
      4.276
      0.000
      0.000
      1.000
      0.000

      5
      3_poly
      2_poly
      3.851
      0.001
      0.000
      1.000
      0.000

# Using the Bayesian approach we can compute the probability that a model performs better, worse or practically equivalent to another.

Results show that the model ranked first by :class:~sklearn.model_selection.GridSearchCV 'rbf', has approximately a 6.8% chance of being worse than 'linear', and a 1.8% chance of being worse than '3_poly'. 'rbf' and 'linear' have a 43% probability of being practically equivalent, while 'rbf' and '3_poly' have a 10% chance of being so.

Similarly to the conclusions obtained using the frequentist approach, all models have a 100% probability of being better than '2_poly', and none have a practically equivalent performance with the latter.


Table of Contents

3  Final Exam Part 3

3.1  🔁 Describe the following effects/experiments/equations, and explain their importance to the origins of quantum mechanics.

3.2  🔁 Explain the following topics.

3.3  🔁 What are the eigenfunctions and eigenvalues for the following Hamiltonians.

3.4  🤔 Discuss the following in your own words.

# Final Exam Part 3
👨🏫 **Overview:**
This is Part 3 of 3 Parts of your Final Exam/Project.
- **Part 1.** Questions on the Course Material. (40%)
- **Part 2.** Applying the Course material. (40%)
- **Part 3.** Discussing your exam and the Course Material. (20%)

**You must submit your Jupyter notebooks for Part 1 and Part 2 at least 48 hours prior to your appointment for Part 3.** You will be given your grade on Part 1 and Part 2 before the oral exam, so that you know what your status is. *For late submission of Part 1 and/or Part 2, I will deduct 2 points per hour.

📖 **Rules for "Open Book" Exam:**
**Like all other portions of this exam, this part of the exam is open notes and open library. It is "open internet search" but you (obviously!) can't post questions on an internet discussion board or homework/problem/exam help site. You are not allowed to communicate with your classmates or any other human being (except me) about these questions or your responses, and this includes human beings (singular or plural, known or anonymous) online.**

💯 **Grading:**
When we discuss Parts 1 and 2 of the exam, your grade will be "plus", "minus" or "neutral." You can gain, or lose, up to 20 points based on your performance on the oral portion of the exam.
- A "plus" response indicates that you know the answer to the question, and can expound upon it and related topics with mastery.
- A "minus" response indicates that while you may have answered a question correctly, you "got the right answer for the wrong reason" or your understanding of the topic being tested is only superficial. A "minus" response is also indicative of cases where you simply can't answer a question. If for whatever reason you do not schedule an oral exam, that will be taken as a "minus" response for *all* questions. Don't do this: it will be -40% on your final exam grade.
- A "neutral" response is the default, and is anything except mastery ("plus") or naïvté ("minus"). I anticipate that most responses will be "neutral".

In addition to discussing your answers in Parts 1 and 2, we will discuss the following questions. Each question group has 6 questions in it. I will randomly select one question from each group; we will then discuss that question. Each question will be graded "plus" (5 points) "minus" (0 points) or "neutral" (3 points). You can score up to 20 points on this portion of the exam, and the "default" score would be about 12 points.

📜 **Instructions:** Be prepared to discuss the following questions/topics. I will randomly select 1 question from each group.

## 🔁 Describe the following effects/experiments/equations, and explain their importance to the origins of quantum mechanics.
1. Black-Body Radiation and the Ultraviolet Catastrophe.
1. The Davisson-Germer experiment and Electron Scattering.
1. Compton Scattering.
1. The Photoelectric Effect
1. The Rydberg Formula
1. The Stern-Gerlach Experiment

## 🔁 Explain the following topics.
1. What are the key postulates of quantum mechanics?
1. What is the Heisenberg Uncertainty Principle? What does it say about the fundamentals of quantum mechanics? What is the commutator between two operators, and how does it relate to the Heisenberg Uncertainty Principle?
1. Explain what the Born-Oppenheimer Approximation is and why it is important.
1. Explain what the united-atom and separated-atom limit are for a diatomic molecule. How do these limits relate to molecular-orbital theory and the linear-combination-of-atomic-orbitals technique.
1. What are the quantum numbers that describe the state of an atom (neglecting relativity)? What do these quantum numbers have to do with term symbols? What do these quantum numbers have to do with simultaneous observables?
1. What is the Stern Gerlach experiment? Can you describe what is happening in the following diagram?

## 🔁 What are the eigenfunctions and eigenvalues for the following Hamiltonians.
You should be able to explain key characteristics of the systems and the general strategy for (possibly approximate) solving the associated time-independent Schrödinger equation.

\begin{align}
\hat{H}_1 &= -\frac{\hbar^2}{2m} \frac{d^2}{dx^2} -\frac{\hbar^2}{2m} \frac{d^2}{dy^2} + V_{a_x}(x) + V_{a_y}(y) \\
\hat{H}_2(\lambda) &= -\frac{\hbar^2}{2m} \frac{d^2}{dx_1^2} -\frac{\hbar^2}{2m} \frac{d^2}{dx_2^2} + V_{a}(x_1) + V_{a}(x_2)
- \lambda |x_1 - x_2|\\
\hat{H}_3(\lambda) &= -\tfrac{1}{2} \nabla_1^2 -\tfrac{1}{2} \nabla_2^2- \tfrac{Z}{r_1} - \tfrac{Z}{r_2} + \tfrac{\lambda}{|\mathbf{r}_1 - \mathbf{r}_2|}\\
\hat{H}_4 &= -\frac{\hbar^2}{2m} \frac{d^2}{dx^2} -\frac{\hbar^2}{2m} \frac{d^2}{dy^2} -\frac{\hbar^2}{2m} \frac{d^2}{dz^2} + V_{a}\left(\sqrt{x^2 + y^2 +z^2}\right) \\
\hat{H}_5(\mathbf{R}_B) &=  -\tfrac{1}{2} \nabla_{\mathbf{r}}^2 - \tfrac{Z_A}{r} - \tfrac{Z_B}{|\mathbf{r} - \mathbf{R}_B|} \\
\hat{H}_6 &= -\frac{\hbar^2}{2m} \frac{d^2}{dx^2} -\frac{\hbar^2}{2m} \frac{d^2}{dy^2} -\frac{\hbar^2}{2m} \frac{d^2}{dz^2} + V_{a_z}(z) + V_{a_r}(\sqrt{x^2+y^2})
\end{align}
where
$$
V_a(x) =
\begin{cases}
    +\infty & x\leq 0\\
    0       & 0\lt x \lt a\\
    +\infty & a \leq x
\end{cases}
$$

## 🤔 Discuss the following in your own words.
1. What is a Slater determinant? Why does one use a Slater determinant for a many-fermion system? When is the exact ground-state wavefunction for a many-fermion system a Slater determinant?
1. What is the variational principle? How is it used to approximate the ground-state energy and wavefunction of realistic, not-exactly-solvable, Hamiltonians? Can you give an example of how it is used?
1. What is the Hartree-Fock method? What is the intuition behind it, and when is it used?
1. What is Perturbation Theory? When is perturbation theory most appropriate? Least appropriate?
1. What is the secular equation? How does it relate to the technique of expanding a wavefunction in a basis set?
1. What is electron correlation? How can one approximate the effects of electron correlation?


# Scientific Computing: Numpy, Scipy and Graphics

## Python shells, scripts and editors

These are instructions how to run Python as scripts, an alternative to working in a Notebook:

Up to now we have used Python only in interactive mode, i.e. by directly typing commands at the Notebook/IPython prompt, which then get executed right away. While this is simple, and often convenient, it is also somewhat limiting. An alternative approach is to run Python as scripts/programs or notebooks like this one. A script is basically a list of Python commands saved in a file, which can then be executed together. Doing this makes it easy to save them for future re-use, re-run them with different input, etc. You can also use scripts written by others. Over the duration of our course there will be many such scripts provided to you, and you will also be writing some of your own.

Creating a script is easy - you should open a text editor and enter the commands in it, taking proper care of any required indentations, etc. Any text editor program is fine for this (but only ones that handle pure text, NOT Word, Notepad, etc.). If you have never used one before I recommend 'gedit' as it has a simple, intuitive graphical interface. Examples of more advanced and powerful editors are 'emacs' and 'vi',  but those have a somewhat steeper initial learning curve. Nevertheless, feel free to try them out. Some of the nice Python programming environments like Geany and Spyder come with their own integrated editor and shell, and same is true for the Jupyter notebooks.

You can start the editor from the Xterm window by typing:

gedit &

(or emacs, or vi; the '&' sign above is to put the program in background, so you can still use the window - for IPython, or anything else you wish).

Alternatively, you can start gedit (or other installed software) from within the ipython shell itself by the same command as above, but preceded by '!' (no quotes), i.e.

!gedit &

The preceding exclamation tells ipython that this is a normal shell command, not a Python statement. You can also start gedit from the Jupyter notebook, by doing

!gedit

'gedit' is Python-aware, thus has features like color coding of Python keywords and automatic indentation, which makes your work easier. Before a program can be run, it must be saved as a Python file with the '.py' extension, for example, 'myprog.py'. The program can then be executed by typing

python myprog.py

Note: On Windows double-clicking on the program icon will usually also work. But beware: the program window closes immediately after execution, before you get a chance to read the output. To prevent this from happening, conclude the program with the line

raw_input(’press return’)

Double-clicking the program icon also works in Unix and Linux if the first line of the program specifies the path to the Python interpreter (or a shell script that provides a link to Python). The path name must be preceded by the symbols '#!' (no quotes!). On my computer and the teaching machines the path is '/usr/bin/python',  so that all my programs should start with the line

#!/usr/bin/python

On multiuser systems the path is sometimes '/usr/local/bin/python'.

From within IPython a script can be ran using the magic command 'run':

%run prog.py

Note that the script file should be in your current IPython working
folder/directory. If you are not sure which folder you are currently in, enter 'pwd' at the IPython prompt, which tells you the current directory. If you are not at the correct place, you can change it using the

cd directory_name

command.

Once a module (script/function) is saved in a file, you can do 'import'  just like for any other Python module. This is a way to use external codes and functions, as well as creating your own. When a module is loaded into a program for the first time with the 'import' statement, it is compiled into bytecode and written in a file with the extension '.pyc'. The next time the program is run, for efficiency the interpreter loads the bytecode rather than the original Python file. If in the meantime changes have been made to the module, the module is automatically recompiled.

It is a good idea to document your modules by adding a docstring at the
beginning of each module. The docstring, which is enclosed in triple quotes, should explain what the module does. Here is an example that documents the module 'error':

```python
## module error
''' err(string).

    Prints 'string' and terminates program.

'''
import sys
def err(string):
    print(string)
    raw_input('Press return to exit')
    sys.exit()

```
The docstring of a module can be printed with the statement:

```python
import math
print(math.__doc__)
```

    This module is always available.  It provides access to the
    mathematical functions defined by the C standard.

# NumPy (Numerical Python) module
The NumPy module is not a part of the standard Python release. It must be obtained separately and installed on your computer. It is already installed and available on the university machines. You can use it by loading it as a module. This module introduces array objects (vectors, matrices) that are similar to lists, but can be manipulated by numerous functions contained in the module. The size of an array is immutable (i.e. once array is defined you cannot change its size), and no empty elements are allowed.

The complete set of functions in NumPy is quite long, you can check what is available using:

```python
import numpy
dir(numpy)
```

    ['ALLOW_THREADS',
     'BUFSIZE',
     'CLIP',
     'ComplexWarning',
     'DataSource',
     'ERR_CALL',
     'ERR_DEFAULT',
     'ERR_IGNORE',
     'ERR_LOG',
     'ERR_PRINT',
     'ERR_RAISE',
     'ERR_WARN',
     'FLOATING_POINT_SUPPORT',
     'FPE_DIVIDEBYZERO',
     'FPE_INVALID',
     'FPE_OVERFLOW',
     'FPE_UNDERFLOW',
     'False_',
     'Inf',
     'Infinity',
     'MAXDIMS',
     'MAY_SHARE_BOUNDS',
     'MAY_SHARE_EXACT',
     'MachAr',
     'ModuleDeprecationWarning',
     'NAN',
     'NINF',
     'NZERO',
     'NaN',
     'PINF',
     'PZERO',
     'PackageLoader',
     'RAISE',
     'RankWarning',
     'SHIFT_DIVIDEBYZERO',
     'SHIFT_INVALID',
     'SHIFT_OVERFLOW',
     'SHIFT_UNDERFLOW',
     'ScalarType',
     'Tester',
     'TooHardError',
     'True_',
     'UFUNC_BUFSIZE_DEFAULT',
     'UFUNC_PYVALS_NAME',
     'VisibleDeprecationWarning',
     'WRAP',
     '_NoValue',
     '__NUMPY_SETUP__',
     '__all__',
     '__builtins__',
     '__config__',
     '__doc__',
     '__file__',
     '__git_revision__',
     '__mkl_version__',
     '__name__',
     '__package__',
     '__path__',
     '__version__',
     '_import_tools',
     '_mat',
     'abs',
     'absolute',
     'absolute_import',
     'add',
     'add_docstring',
     'add_newdoc',
     'add_newdoc_ufunc',
     'add_newdocs',
     'alen',
     'all',
     'allclose',
     'alltrue',
     'alterdot',
     'amax',
     'amin',
     'angle',
     'any',
     'append',
     'apply_along_axis',
     'apply_over_axes',
     'arange',
     'arccos',
     'arccosh',
     'arcsin',
     'arcsinh',
     'arctan',
     'arctan2',
     'arctanh',
     'argmax',
     'argmin',
     'argpartition',
     'argsort',
     'argwhere',
     'around',
     'array',
     'array2string',
     'array_equal',
     'array_equiv',
     'array_repr',
     'array_split',
     'array_str',
     'asanyarray',
     'asarray',
     'asarray_chkfinite',
     'ascontiguousarray',
     'asfarray',
     'asfortranarray',
     'asmatrix',
     'asscalar',
     'atleast_1d',
     'atleast_2d',
     'atleast_3d',
     'average',
     'bartlett',
     'base_repr',
     'bench',
     'binary_repr',
     'bincount',
     'bitwise_and',
     'bitwise_not',
     'bitwise_or',
     'bitwise_xor',
     'blackman',
     'bmat',
     'bool',
     'bool8',
     'bool_',
     'broadcast',
     'broadcast_arrays',
     'broadcast_to',
     'busday_count',
     'busday_offset',
     'busdaycalendar',
     'byte',
     'byte_bounds',
     'bytes_',
     'c_',
     'can_cast',
     'cast',
     'cbrt',
     'cdouble',
     'ceil',
     'cfloat',
     'char',
     'character',
     'chararray',
     'choose',
     'clip',
     'clongdouble',
     'clongfloat',
     'column_stack',
     'common_type',
     'compare_chararrays',
     'compat',
     'complex',
     'complex128',
     'complex256',
     'complex64',
     'complex_',
     'complexfloating',
     'compress',
     'concatenate',
     'conj',
     'conjugate',
     'convolve',
     'copy',
     'copysign',
     'copyto',
     'core',
     'corrcoef',
     'correlate',
     'cos',
     'cosh',
     'count_nonzero',
     'cov',
     'cross',
     'csingle',
     'ctypeslib',
     'cumprod',
     'cumproduct',
     'cumsum',
     'datetime64',
     'datetime_as_string',
     'datetime_data',
     'deg2rad',
     'degrees',
     'delete',
     'deprecate',
     'deprecate_with_doc',
     'diag',
     'diag_indices',
     'diag_indices_from',
     'diagflat',
     'diagonal',
     'diff',
     'digitize',
     'disp',
     'divide',
     'division',
     'dot',
     'double',
     'dsplit',
     'dstack',
     'dtype',
     'e',
     'ediff1d',
     'einsum',
     'emath',
     'empty',
     'empty_like',
     'equal',
     'errstate',
     'euler_gamma',
     'exp',
     'exp2',
     'expand_dims',
     'expm1',
     'extract',
     'eye',
     'fabs',
     'fastCopyAndTranspose',
     'fft',
     'fill_diagonal',
     'find_common_type',
     'finfo',
     'fix',
     'flatiter',
     'flatnonzero',
     'flexible',
     'fliplr',
     'flipud',
     'float',
     'float128',
     'float16',
     'float32',
     'float64',
     'float_',
     'floating',
     'floor',
     'floor_divide',
     'fmax',
     'fmin',
     'fmod',
     'format_parser',
     'frexp',
     'frombuffer',
     'fromfile',
     'fromfunction',
     'fromiter',
     'frompyfunc',
     'fromregex',
     'fromstring',
     'full',
     'full_like',
     'fv',
     'generic',
     'genfromtxt',
     'get_array_wrap',
     'get_include',
     'get_printoptions',
     'getbuffer',
     'getbufsize',
     'geterr',
     'geterrcall',
     'geterrobj',
     'gradient',
     'greater',
     'greater_equal',
     'half',
     'hamming',
     'hanning',
     'histogram',
     'histogram2d',
     'histogramdd',
     'hsplit',
     'hstack',
     'hypot',
     'i0',
     'identity',
     'iinfo',
     'imag',
     'in1d',
     'index_exp',
     'indices',
     'inexact',
     'inf',
     'info',
     'infty',
     'inner',
     'insert',
     'int',
     'int0',
     'int16',
     'int32',
     'int64',
     'int8',
     'int_',
     'int_asbuffer',
     'intc',
     'integer',
     'interp',
     'intersect1d',
     'intp',
     'invert',
     'ipmt',
     'irr',
     'is_busday',
     'isclose',
     'iscomplex',
     'iscomplexobj',
     'isfinite',
     'isfortran',
     'isinf',
     'isnan',
     'isneginf',
     'isposinf',
     'isreal',
     'isrealobj',
     'isscalar',
     'issctype',
     'issubclass_',
     'issubdtype',
     'issubsctype',
     'iterable',
     'ix_',
     'kaiser',
     'kron',
     'ldexp',
     'left_shift',
     'less',
     'less_equal',
     'lexsort',
     'lib',
     'linalg',
     'linspace',
     'little_endian',
     'load',
     'loads',
     'loadtxt',
     'log',
     'log10',
     'log1p',
     'log2',
     'logaddexp',
     'logaddexp2',
     'logical_and',
     'logical_not',
     'logical_or',
     'logical_xor',
     'logspace',
     'long',
     'longcomplex',
     'longdouble',
     'longfloat',
     'longlong',
     'lookfor',
     'ma',
     'mafromtxt',
     'mask_indices',
     'mat',
     'math',
     'matmul',
     'matrix',
     'matrixlib',
     'max',
     'maximum',
     'maximum_sctype',
     'may_share_memory',
     'mean',
     'median',
     'memmap',
     'meshgrid',
     'mgrid',
     'min',
     'min_scalar_type',
     'minimum',
     'mintypecode',
     'mirr',
     'mod',
     'modf',
     'moveaxis',
     'msort',
     'multiply',
     'nan',
     'nan_to_num',
     'nanargmax',
     'nanargmin',
     'nanmax',
     'nanmean',
     'nanmedian',
     'nanmin',
     'nanpercentile',
     'nanprod',
     'nanstd',
     'nansum',
     'nanvar',
     'nbytes',
     'ndarray',
     'ndenumerate',
     'ndfromtxt',
     'ndim',
     'ndindex',
     'nditer',
     'negative',
     'nested_iters',
     'newaxis',
     'newbuffer',
     'nextafter',
     'nonzero',
     'not_equal',
     'nper',
     'npv',
     'numarray',
     'number',
     'obj2sctype',
     'object',
     'object0',
     'object_',
     'ogrid',
     'oldnumeric',
     'ones',
     'ones_like',
     'outer',
     'packbits',
     'pad',
     'partition',
     'percentile',
     'pi',
     'piecewise',
     'pkgload',
     'place',
     'pmt',
     'poly',
     'poly1d',
     'polyadd',
     'polyder',
     'polydiv',
     'polyfit',
     'polyint',
     'polymul',
     'polynomial',
     'polysub',
     'polyval',
     'power',
     'ppmt',
     'print_function',
     'prod',
     'product',
     'promote_types',
     'ptp',
     'put',
     'putmask',
     'pv',
     'r_',
     'rad2deg',
     'radians',
     'random',
     'rank',
     'rate',
     'ravel',
     'ravel_multi_index',
     'real',
     'real_if_close',
     'rec',
     'recarray',
     'recfromcsv',
     'recfromtxt',
     'reciprocal',
     'record',
     'remainder',
     'repeat',
     'require',
     'reshape',
     'resize',
     'restoredot',
     'result_type',
     'right_shift',
     'rint',
     'roll',
     'rollaxis',
     'roots',
     'rot90',
     'round',
     'round_',
     'row_stack',
     's_',
     'safe_eval',
     'save',
     'savetxt',
     'savez',
     'savez_compressed',
     'sctype2char',
     'sctypeDict',
     'sctypeNA',
     'sctypes',
     'searchsorted',
     'select',
     'set_numeric_ops',
     'set_printoptions',
     'set_string_function',
     'setbufsize',
     'setdiff1d',
     'seterr',
     'seterrcall',
     'seterrobj',
     'setxor1d',
     'shape',
     'shares_memory',
     'short',
     'show_config',
     'sign',
     'signbit',
     'signedinteger',
     'sin',
     'sinc',
     'single',
     'singlecomplex',
     'sinh',
     'size',
     'sometrue',
     'sort',
     'sort_complex',
     'source',
     'spacing',
     'split',
     'sqrt',
     'square',
     'squeeze',
     'stack',
     'std',
     'str',
     'str_',
     'string0',
     'string_',
     'subtract',
     'sum',
     'swapaxes',
     'take',
     'tan',
     'tanh',
     'tensordot',
     'test',
     'testing',
     'tile',
     'timedelta64',
     'trace',
     'transpose',
     'trapz',
     'tri',
     'tril',
     'tril_indices',
     'tril_indices_from',
     'trim_zeros',
     'triu',
     'triu_indices',
     'triu_indices_from',
     'true_divide',
     'trunc',
     'typeDict',
     'typeNA',
     'typecodes',
     'typename',
     'ubyte',
     'ufunc',
     'uint',
     'uint0',
     'uint16',
     'uint32',
     'uint64',
     'uint8',
     'uintc',
     'uintp',
     'ulonglong',
     'unicode',
     'unicode0',
     'unicode_',
     'union1d',
     'unique',
     'unpackbits',
     'unravel_index',
     'unsignedinteger',
     'unwrap',
     'ushort',
     'vander',
     'var',
     'vdot',
     'vectorize',
     'version',
     'void',
     'void0',
     'vsplit',
     'vstack',
     'warnings',
     'where',
     'who',
     'zeros',
     'zeros_like']

## Creating an Array
Arrays can be created in several ways. One of them is to start from a list and use the 'array' function to turn it into an array:

array(list,dtype = type specification)

Notes: Lists can hold any sequence of any Python objects, while arrays can only hold objects of the same type. Arrays are most efficient when the elements are of the basic number types (float, int, complex).

Here are two examples of creating a 2x2 array with floating-point elements:

```python
from numpy import array,float
a = array([[2.0, -1.0],[-1.0, 3.0]])
print(a)
```

    [[ 2. -1.]
     [-1.  3.]]

```python
b = array([[2, -1],[-1, 3]],dtype = float)
print(b)
```

    [[ 2. -1.]
     [-1.  3.]]

A couple of quite useful functions which also create arrays are:

zeros((dim1,dim2),dtype = type specification)

which creates a dim1 by dim2 array and fills it with zeroes, and

ones((dim1,dim2),dtype = type specification)

which fills the array with ones. The default type in both cases is float. Finally, there is the function

arange(from,to,increment)

which works just like the range function, but returns an array rather than a list. Here are examples of creating arrays:

```python
import numpy
print(numpy.arange(2,10,2))
```

    [2 4 6 8]

```python
print(numpy.arange(2.0,10.0,2.0))
```

    [ 2.  4.  6.  8.]

```python
print(numpy.zeros(3))
```

    [ 0.  0.  0.]

```python
print(numpy.zeros((3),dtype=int))
```

    [0 0 0]

```python
print(numpy.ones((2,2)))
```

    [[ 1.  1.]
     [ 1.  1.]]

An alternative, similar to Matlab, is to use

numpy.linspace(0,n,num=n+1,endpoint=True)

We can then 'reshape' the array, converting it from 1D (vector) to 2D (matrix), note that the total sizes of the two should be exactly the same (in the example below it is 9 numbers):

```python
x = numpy.linspace(0,8,9).reshape(3,3)
print(x)
```

    [[ 0.  1.  2.]
     [ 3.  4.  5.]
     [ 6.  7.  8.]]

Note of caution: Reshape does not copy - it does not create a new array, but rather creates a different view of the same data.

We can also do the above as follows:

```python
x = numpy.arange(9).reshape(3,3)
print(x)
```

    [[0 1 2]
     [3 4 5]
     [6 7 8]]

with the same resulting matrix (but made of integers! If you want real numbers you can do:

```python
x=numpy.arange(9.).reshape(3,3)
print(x)
```

    [[ 0.  1.  2.]
     [ 3.  4.  5.]
     [ 6.  7.  8.]]

Here is another example, where we first create lists using a function and then convert them into arrays:

```python
def f(x):
  return x**3     # sample function

n = 5           # no of points in [0,1]
dx = 1.0/(n-1)  # x spacing
xlist = [i*dx for i in range(n)]
ylist = [f(x) for x in xlist]
```
Note that once we have defined the function we can use it as any other function. We can then turn these lists into Numerical Python (NumPy) arrays:

```python
import numpy as np
x2 = np.array(xlist)
y2 = np.array(ylist)
print(x2,y2)
```

    (array([ 0.  ,  0.25,  0.5 ,  0.75,  1.  ]), array([ 0.      ,  0.015625,  0.125   ,  0.421875,  1.      ]))

Alternatively, instead of first making lists with x and y=f(x) data, and then turning lists into arrays, we can make NumPy arrays directly:

```python
n = 5                     # number of points
x3 = np.linspace(0, 1, n) # n points in [0, 1]
y3 = np.zeros(n)          # n zeros (float data type)
for i in xrange(n):
     y3[i] = f(x3[i])
print(x3,y3)
```

    (array([ 0.  ,  0.25,  0.5 ,  0.75,  1.  ]), array([ 0.      ,  0.015625,  0.125   ,  0.421875,  1.      ]))

Note: 'xrange'  is similar to 'range' but faster; 'xrange' in Python 2.x is called 'range' in Python 3.x.Caution regarding Vectors, Column and Row Matrices. Note that an n vector, an n by 1 (column) and 1 by n (row) matrix are three different
objects, even if they may contain the same data.
## Exercise:
Create the following arrays (with correct data types):

[[ 1  1  1  1]
 [ 1  1  1  1]
 [ 1  1  1  2]
 [ 1  6  1  1]]

[[0. 0. 0. 0. 0.]
 [2. 0. 0. 0. 0.]
 [0. 3. 0. 0. 0.]
 [0. 0. 4. 0. 0.]
 [0. 0. 0. 5. 0.]
 [0. 0. 0. 0. 6.]]

## Accessing and Changing Array Elements
If a is a rank-2 array (i.e. a matrix), then a[i,j] accesses the element in row i and column j, whereas a[i] refers to row i. The elements of an array can be changed by assignment:

```python
import numpy
a = numpy.zeros((3,3),dtype=int)
print(a)
```

    [[0 0 0]
     [0 0 0]
     [0 0 0]]

```python
a[0]=[2,3,5]    # Change a row
a[1,1]=6        # Change an element
a[2,0:2]=[8,-3] # Change part of a row
print(a)
```

    [[ 2  3  5]
     [ 0  6  0]
     [ 8 -3  0]]

## Exercises:
Form the following rank-2 array (without typing it in explicitly):

[[1  6 11]
[2  7 12]
[3  8 13]
[4  9 14]
[5 10 15]]

and generate a new array containing just its 2nd and 4th rows.

Change the fourth row to [1, 4, 9].
## Operations on Arrays
Arithmetic operators work differently on arrays than they do on tuples and lists -- the operation is broadcast to all the elements of the array; that is, the operation is applied to each element in the array individually. Array operations like that are generally very efficient since they are done by underlying optimized and fast C language functions. Here are some examples:

```python
from numpy import array
a = array([0.0, 4.0, 9.0, 16.0])
print(a/16.0)
print a - 4.0
```

    [ 0.      0.25    0.5625  1.    ]
    [ -4.   0.   5.  12.]

All basic arithmetic operations are also performed element-wise (vectors should have the same length, matrices the same dimensions).

```python
v1=array([6.,7.,9.])
v2=array([2.,3.,5.])
print('v1/v2=',v1/v2)
print('v1*v2=',v1*v2)
print('v1+v2=',v1+v2)
print('v1-v2=',v1-v2)
```

    ('v1/v2=', array([ 3.        ,  2.33333333,  1.8       ]))
    ('v1*v2=', array([ 12.,  21.,  45.]))
    ('v1+v2=', array([  8.,  10.,  14.]))
    ('v1-v2=', array([ 4.,  4.,  4.]))

The mathematical functions available in NumPy are also broadcast:

```python
from numpy import array,sqrt,sin
a = array([1.0, 4.0, 9.0, 16.0])
print(sqrt(a))
print(sin(a))
```

    [ 1.  2.  3.  4.]
    [ 0.84147098 -0.7568025   0.41211849 -0.28790332]

Functions imported from the math module will of course work on the individual array elements, but not on the array itself, in which they differ from the NumPy functions (even if they sometimes have the same name). Here is an example:

```python
from numpy import array
from math import sqrt
a = array([1.0, 4.0, 9.0, 16.0])

print sqrt(a[1])
```

    2.0

```python
print sqrt(a)
```

## Exercise:

Create an array x containing 100 equally-spaced numbers from 0 to $2\pi$.
Create another array y whose first column contains x, the second one contains $\sin(x)$ and the third one contains $\sin^2(x)$.

## Array Functions
There are numerous functions in NumPy that perform array operations and other useful tasks. Here are a few examples:

```python
import numpy as np
A = np.array([[4,-2,1],[-2,4,-2],[1,-2,3]],dtype=float)
b = np.array([1,4,3],dtype=float)

print(np.diagonal(A)) # Principal diagonal of A
```

```python
print(np.trace(A)) # Sum of diagonal elements of A
```

```python
print(np.argmax(b)) # Index of largest element of b
```

```python
print(np.argmin(A,axis=0)) # Indices of smallest column elements
```

```python
print(np.identity(3)) # Identity matrix
```
There are three functions in NumPy that compute matrix/vector products. They are illustrated by the program listed below. For more details, see Appendix A2 of the book.

```python
import numpy as np
x = np.array([7,3])
y = np.array([2,1])
A = np.array([[1,2],[3,2]])
B = np.array([[1,1],[2,2]])

# Dot product
print "dot(x,y) =\n",np.dot(x,y) #{x}.{y} - scalar (dot) product of 2 vectors
print "dot(A,x) =\n",np.dot(A,x) #[A]{x}  - maxtrix-vector dot product
print "dot(A,B) =\n",np.dot(A,B) #[A][B]  - matrix-matrix dot product

# Inner product
print "inner(x,y) =\n",np.inner(x,y) # {x}.{y}
print "inner(A,x) =\n",np.inner(A,x) # [A]{x}
print "inner(A,B) =\n",np.inner(A,B) # [A][B_transpose]

# Outer product
print "outer(x,y) =\n",np.outer(x,y)
print "outer(A,x) =\n",np.outer(A,x)
print "Outer(A,B) =\n",np.outer(A,B)
```

## Exercises:

1) Taking the y array from the previous exercise, find the values
of its largest and smallest elements in each row.

2) What do 'diagonal' and 'trace' give you if a 2D array is not with two equal dimensions?

## Linear Algebra Module
NumPy comes with a linear algebra module called 'linalg' that contains routine tasks such as matrix inversion and solution of simultaneous equations (which we will be discussing in more detail soon). For example:

```python
from numpy import array
from numpy.linalg import inv,solve

A = array([[ 4.0, -2.0, 1.0], \
[-2.0, 4.0, -2.0], \
[ 1.0, -2.0, 3.0]])

# Note: The backslash '\' above is Python's line continuation character.

b = array([1.0, 4.0, 2.0])
```

```python
print(inv(A)) # Matrix inverse
```

```python
print(solve(A,b)) # Solve the system of equations [A]{x} = {b}
```

## Exercises:

1) Set up the matrix

\begin{equation}
A=[[1. 2. 3.]\\
     [4. 5. 6.]\\
     [7. 8. 1.]]
\end{equation}

starting with 'linspace' and then reshaping the array into a 3x3 one.

2) Calculate the inverse of A. Check that $AA^{-1}=A^{-1}A=1$.

3) Create a vector b=(6,15,16). Solve the equation Ax=b for x. Check your answer.

## Copying Arrays
We explained before that if 'a' is a mutable object, such as a list, the assignment statement 'b = a' does not result in a new object b, but simply creates a new reference to 'a', called a deep copy. This also applies to arrays. To make an independent copy of an array a, use the copy method in the NumPy module:

```python
b = a.copy()
```

## Exercise:

Starting from the array A from the previous exercise, create arrays
B=A and C=A.copy(). Now change A[0,0] to 5 and print out both B and C. Did they change?

## Vectorizing Algorithms

Sometimes the broadcasting properties of the mathematical functions in the NumPy module can be utilised to replace loops in the code. This procedure is known as vectorisation. Consider, for example, the sum:

\begin{equation}
s=\sum_{i=0}^{100}\sqrt{\frac{i\pi}{100}}\sin\frac{i\pi}{100}
\end{equation}

The direct approach is to evaluate the sum in a loop, resulting in the following “scalar” code:

```python
from math import sqrt,sin,pi
x=0.0; sum = 0.0
for i in range(0,101):
    sum = sum + sqrt(x)*sin(x)
    x = x + 0.01*pi
print(sum)
```

    77.5138979892

The vectorised version of algorithm is:

```python
from numpy import sqrt,sin,arange,sum
from math import pi
x = arange(0.0,1.001*pi,0.01*pi)
print(sum(sqrt(x)*sin(x)))
```

    77.5138979892

Note that the first algorithm uses the scalar versions of 'sqrt' and
'sin' functions in the math module, whereas the second algorithm imports these functions from the NumPy. The vectorised algorithm is faster, so generally preferable, but uses more memory. Also, the code is more elegant, short and reabable.

Mathematical functions in Python without any 'if' tests automatically work for both scalar and array (vector) arguments (i.e., no vectorization by the programmer is necessary).
## Exercises:
1) Make a function which takes as an input a number 'n' and returns the sum of i^2*ln(i) for i=1 to n using a vectorized algorithm.

2) How much faster is Numpy with vectorised operations compared to standard Python? Try the following two ways to add 1 to a million numbers, first is a standard implicit for loop, the second is a vectorized Numpy array operation:

In [1]: lst=range(1000000)
In [2]: %timeit [i+1 for i in lst]
In [3]: arr=arange(1000000)
In [4]: %timeit arr+1

The 'magic' command %timeit tests and reports how fast the statement execution is. How much faster is the vectorised Numpy one?

3) The execution speed advantage is even better for standard Python
functions vs. Numpy functions ('ufunc' in optimized, pre-compiled C-code). Try the following:

In [5]: %timeit [sin(i)**2 for i in arr]
In [6]: %timeit np.sin(arr)**2

How much faster is the second one?

```python
lst=range(1000000)
%timeit [i+1 for i in lst]
```

    10 loops, best of 3: 42.2 ms per loop

```python
arr=arange(1000000)
%timeit arr+1
```

    The slowest run took 5.21 times longer than the fastest. This could mean that an intermediate result is being cached.
    1000 loops, best of 3: 1.28 ms per loop

```python
%timeit [sin(i)**2 for i in arr]
%timeit np.sin(arr)**2
```

    1 loop, best of 3: 1.2 s per loop
    10 loops, best of 3: 36.5 ms per loop

## Scoping of Variables (An important point!)
'Namespace' is a dictionary that contains the names of the variables and their values. The namespaces are automatically created and updated as a Python program runs. There are three levels of namespaces in Python:

- Local namespace, which is created when a function is called. It contains the variables passed to the function as arguments and the variables created within the function. The namespace is deleted when the function terminates. If a variable is created inside a function, its scope is the function's local namespace. It is not visible outside the function.

- A global namespace is created when a module is loaded. Each module has its own namespace. Variables assigned in a global namespace are visible to any function within the module.

- Built-in namespace is created when the interpreter starts. It contains the functions that come with the Python interpreter. These functions can be accessed by any program unit.

When a name is encountered during execution of a function, the Python interpreter tries to resolve it by searching the following in the order shown:
(1) local namespace,
(2) global namespace,
and (3) built-in namespace.

If the name cannot be resolved, Python raises a 'NameError' exception (i.e. lets you know this variable does not exist).

Because the variables residing in a global namespace (i.e. not within a function) are visible to functions within the module, it is not necessary to pass them explicitly to the functions as arguments (although it is good programming practice to do so), as the following program illustrates:

```python
def divide():
    c = a/b
    print 'a/b =',c
a = 100.0
b = 5.0
divide()
```

    a/b = 20.0

Note that the variable 'c' is created inside the function 'divide' and is thus not accessible to statements outside the function. Hence an attempt to move the print statement out of the function fails:

```python
def divide():
    c = a/b
a = 100.0
b = 5.0
divide()
print 'a/b =',c
```

## Exercises:
Create a file called 'numpytest.py'. First define two vectors,
called 'x' and 'y'. 'x' should be initialized to the vector

(1, 5, -2, 6, 8, 10)

and 'y' should be

(-1, 0, 1, 1, 2, 3).

Compute and store in a value 'a' the dot product of these two vectors. Store in a vector 'z1' a copy of 'x' but where every value less than zero has been replaced with zero (Hint: you can do this using slices, or the 'where' function). Store in a vector 'z2' all of the elements of 'x' that correspond to positions of 'y' that are strictly greater than zero. Finally, store in a vector 'z3' the sum of all values in 'y' that correspond to even elements of 'x'. (Hint: you can use modulus to get evenness: this is '%' in Python)
# Visualising Data
Work through the Entought Matplotlib slides (available on Study Direct) and then through the online tutorial at

http://cs.smith.edu/dftwiki/index.php/MatPlotLib_Tutorial_1

(there is a link on Study Direct). You can skip any repeated info.
## Exercises:
1) Plot a simple graph of a sine function in the range 0 to 3 with a
step size of 0.01. Make the line red. Add diamond-shaped markers with size of 5. Add a legend and a grid to the plot.

2) Add to the previous plot a line showing the cosine of the same vector. Make the line blue and dashed, with circular markers. Change line color and thickness as well as the size and the kind of the marker. Experiment with different styles.

3) Now draw a figure with two sub-plots on separate panels showing the
sinus and cosine. Add labels and ticks only to the outermost axes. Place in one of these plots a smaller inset plot showing the exponent of 'x' over the same 'x' interval. Make sure the small plot does not overlap the lines on the larger plot. Use the pylab help and manual if needed.

4) Using the 'random' module.

import random
a=random.random() # creates a random number between 0 and 1

or 'randn(n)', create 2 random vectors 'x' and 'y', each of length 100.
Using the plot command make a scatter plot of 'x' vs. 'y' (hint just use ’bo’ as the format).

5) Do the same scatter plot but using the semilogx, semilogy, and loglog plots (check the help). These produce a plot where one or both axes use logscale.

6) Go to

http://matplotlib.sourceforge.net/gallery.html and

http://www.scipy.org/Cookbook/Matplotlib

and make a plot (or several) that you think looks interesting.

```python

```

ExamPrep/01 Intro to Python/__EXAMPLES 01 - Intro to Numpy, Scipy & Gradient.ipynb