# OpenMC introduction

Please indicate your name below, since you will need to submit this notebook completed latest the day after the datalab.

Don't forget to save your progress during the datalab to avoid any loss due to crashes.

```python
name=''
```

In our previous endeavours we experienced that performing an accurate simulation of particle transport is not trivial. There are several softwares developed to solve this task, and for someone who is more interested in performing the analysis of a reactor core, it is often better to use an already verified and validated code instead of developing a new one from scratch. There are several such codes solving neutron transport either relying on deterministic or stochastic methods. During this datalab we will use a rather young code called OpenMC, which was originally developed at MIT.

OpenMC is a Monte Carlo neutron and photon transport simulation code. It can be used both for fixed source and for criticality calculations. The user can define complicated geometries with constructive solid geometries. OpenMC is written in C++, however it has a rich Python API, which will allow us to comminicate with the code through jupyter notebooks.

OpenMC is not yet an industrial standard code, however our motivation to use it for this course was that it is

1. freely available
2. the important physics we could not tackle within this course ourselves (eg. thermal scattering and several reactions) are implemented in it
3. since we can interact with the code through Python, we can deepen our programming skills
4. for small problems it runs relatively fast on a single PC

That said, today we are going to walk through a simple openMC input file. During this course we will only work with pincell models (ie. a single fuel pin infinite in the axial length with reflective boundaries, so neutrons are reflected back from the boundary in this way approximating an infinite core). This is of course a simplification of a complete reactor core, but it will still allow us to see a lot of cool physics. One could of course perform simulations at the assembly level or for a full-core, however such simulations would be more time consuming. There are several further examples at https://docs.openmc.org/.

**Note that this notebook should be opened from an environment where openmc is available!**

## The input file

During the datalab we will define the following problem which roughly matches a typical western-type pressurized water reactor's conditions:

- PWR pincell with reflective boundaries
- Fuel radius: 0.41 cm
- Fuel material: enriched UO2 with density 10.5 g/cm3
- Cladding inner/outer radius: 0.42/0.45 cm
- Cladding material: zirconium  with density 6.6 g/cm3
- Coolant/moderator: pressurized water with density 0.75 g/cm3
- Cell pitch: 1.26 cm

Note that in practice, PWR fuel cladding is made of an alloy called Zircaloy, nevertheless it has little influence on the neutron economy, therefore we will approximate it as pure zirconium with its natural abundance of isotopes.

So what do we need in order to define an openMC input?

1. Materials: which materials are included in our geometry (nuclide content, temperature, density)?
2. Geometry: how does the geometry look like?
3. Tally: what are the quantities of interest? For example flux, or fission rate?
4. Settings: how many neutrons we would like to include in the calculation?

For each of these steps we will use the python API to export an xml file, which the code will use once we execute it. Note however that one could directly write xml files without using python, and run the code outside of Jupyter. For larger problems this is sometimes better. However we will want to analyse the output results in python, and sometimes we will script the input, so for us it is handy to use the python API.

Let's get started!

### Import

We will need first to import openmc, and of course anything else we might need.

```python
import openmc
import numpy as np
import matplotlib.pyplot as plt
import os
```

### Materials

Do you remember the example `Material()` class we developed before? That is pretty similar to the `Material()` class of openMC. We create an instance of `Material()`, with a numeric id of the material, and then we can add nuclides, and set the density. We can also set the temperature (which can be 294 K, 600 K, 900 K, 1200 K, 1500 K, the temperatures at which the cross sections are evaluated). There are other options as well (for example one can directly set the enrichment), what you can find in the documentation.

If the nuclide inventory is not relevant for a material, we can add elements, in which case openMC will assume the natural abundance of isotopes.

Notice that for water we will need to link to the $S(\alpha,\beta)$ laws, since we need to take into account the molecular bounds for scattering on hydrogen.

Finally, we create an instance of `Materials()` (note, it is plural now!), and link the materials of the problem to this object. Then with the `export_to_xml()` method we export the information into an xml file.

```python
uo2 = openmc.Material(1, "uo2",temperature=1200)
# Add nuclides to uo2
uo2.add_nuclide('U235', 0.04)
uo2.add_nuclide('U238', 0.96)
uo2.add_nuclide('O16', 2.0)
uo2.set_density('g/cm3', 10.5)

#cladding
zirconium = openmc.Material(2, "zirconium",temperature=900)
zirconium.add_element('Zr', 1.0)
zirconium.set_density('g/cm3', 6.6)

#coolant
water = openmc.Material(3, "h2o")
water.add_nuclide('H1', 2.0)
water.add_nuclide('O16', 1.0)
water.set_density('g/cm3', 0.75)

water.add_s_alpha_beta('c_H_in_H2O')

#creating Materials() and exporting to xml
mats = openmc.Materials([uo2, zirconium, water])
mats.export_to_xml()
```

We can print the content of the created xml file, with the linux command `cat`. (The `!` sign let's the python interpreter know that this is a command of the operating system.

You can see how all the information was structured in an xml (which is probably familiar from the previous datalab).

```python
!cat materials.xml
```

### Geometry

Defining the geometry is only marginally more complicated. We will need to define surfaces (eg. `ZCylinder()`, `XPlane()`, and `YPlane()`), and regions bounded by the surfaces. Then we link the regions and the filling material to `Cell()` objects, which all have a numeric ID.

Our geometry is infinite in the axial dimension. First we define three cylinders (the bounding surfaces of the fuel, and the cladding). Then we use constuctive solid geometry rules to define the regions between the surfaces, and create the fuel pin. Finally we will define the water region around the fuel.

We will then need to define a `Universe()` (the usefulness of this is not apparent in our simple exercise, but when defining reactor cores with repetitive patterns of various types of fuel assemblies, then "universes" are the way to handle this in most neutron transport code. You do not need to worry about this in this course). And finally we create a `Geometry()`, and link the universe. We will then export the information into an xml file.

We further commented the lines to explain them

```python
fuel_or = openmc.ZCylinder(r=0.41) #fuel cylinder with outer radius
clad_ir = openmc.ZCylinder(r=0.42) #clad inner cylinder with inner radius
clad_or = openmc.ZCylinder(r=0.45) #clad outer cylinder with outer radius

fuel_region = -fuel_or                #inside the fuel cylinder
gap_region = +fuel_or & -clad_ir      #outside of fuel cylinder and inside of clad inner cylinder
clad_region = +clad_ir & -clad_or     #outside of clad inner cylinder and inside of clad outer cylinder

fuel = openmc.Cell(1, 'fuel')
fuel.fill = uo2
fuel.region = fuel_region

#notice that for the gap between the fuel and the clad we do not need to link a material
# we consider that there is void there (considering the low density of the filling gas this is a fair approximation)
gap = openmc.Cell(2, 'air gap')
gap.region = gap_region

clad = openmc.Cell(3, 'clad')
clad.fill = zirconium
clad.region = clad_region

pitch = 1.26
#we define the x and y planes with boundary condition
left = openmc.XPlane(x0=-pitch/2, boundary_type='reflective')
right = openmc.XPlane(x0=pitch/2, boundary_type='reflective')
bottom = openmc.YPlane(y0=-pitch/2, boundary_type='reflective')
top = openmc.YPlane(y0=pitch/2, boundary_type='reflective')

#outside of left and inside of right, outside of bottom, and inside of top and outside of clad outer cylinder
water_region = +left & -right & +bottom & -top & +clad_or

moderator = openmc.Cell(4, 'moderator')
moderator.fill = water
moderator.region = water_region

root = openmc.Universe(cells=[fuel, gap, clad, moderator])

geom = openmc.Geometry()
geom.root_universe = root
geom.export_to_xml()
```

We can again inspect the xml file.

```python
!cat geometry.xml
```

### Tally

We can describe the physical quantities of interest with tallies. If no tally is defined the code will only calculate the k-effective.

As we can see in the openMC documentation (in the user's guide) a tally is a quantity defined as

\begin{equation}
X = \underbrace{\int d\mathbf{r} \int d\mathbf{\Omega} \int
dE}_{\text{filters}} \underbrace{f(\mathbf{r}, \mathbf{\Omega},
E)}_{\text{scores}} \psi (\mathbf{r}, \mathbf{\Omega}, E)
\end{equation}

For example if function $f()$ would be unity, we would score the flux in a certain part of the phase space. If function $f()$ is a macroscopic cross section, we would score reaction rates.

Therefore we have to specify two things for a tally, which is created with a `Tally()` object:

1. *filters*: which part of the phase space we would like score at (for example only a certain part of the geometry is of interest, or certain energy bins)?
2. *scores*: what is the physical quantity of interest (for example flux, fission rate etc)

You can find a detailed description of the available scores and filters in the documentation.

Once the tallies are defined, we will need to link them into a `Tallies()` object, and export an xml file.

In the following code block we define three tallies.

1. tally for the neutron spectrum and the fission rate vs energy in the fuel
2. tally for the neutron spectrum in the moderator
3. a Mesh tally along the x-axis to tally the spatial dependence of the flux with three energy groups (thermal: 0-1eV, epithermal: 1eV-100keV, fast: 100keV-20MeV)

The reason why we tally the fission rate as well in the 1st tally is to see how openMC will store the results.

```python
# Tally 1: energy spectrum in fuel
cell_filter1 = openmc.CellFilter(fuel)
energybins=np.logspace(-2,7,1001) #1000 bins between 1e-2 eV and 1e7 eV
energy_filter = openmc.EnergyFilter(energybins)

t1 = openmc.Tally(1)
t1.filters = [cell_filter1,energy_filter]
t1.scores = ['flux','fission']

# Tally 2: energy spectrum in moderator
# we can use the same energy_filter as before
cell_filter2 = openmc.CellFilter(moderator)

t2 = openmc.Tally(2)
t2.filters = [cell_filter2,energy_filter]
t2.scores = ['flux']

# Tally 3: mesh tally in dimension x in three energy groups
energy_filter2 = openmc.EnergyFilter([0., 1, 100e3, 20.0e6])
myMesh=openmc.Mesh(name='xmesh')
myMesh.dimension=[100] #number of spatial bins along x axis
myMesh.lower_left=[-pitch/2]
myMesh.upper_right=[pitch/2]
mesh_filter = openmc.MeshFilter(myMesh)

t3 = openmc.Tally(3)
t3.filters = [mesh_filter,energy_filter2]
t3.scores = ['flux']

tallies = openmc.Tallies([t1,t2,t3])
tallies.export_to_xml()
```

We can inspect the xml file.

```python
!cat tallies.xml
```

### Settings

There are some parameters we will need to set: we have to include the original source location and specify the number of batches and particles per batches for the criticality mode calculation. We also have to specify the number of "inactive" batches (these cycles are used to spread the fission source over the geometry, but the scores are still biased due to the original source not sampling all the possible fission sites properly).

Since we have no better guess for the moment, we will place the source first in the center of the geometry. The number of batches and particles are set so that the results are reasonably accurate. If you wanted to have more accurate results you could increase these numbers, if you wanted to have shorter calculations, you could decrease these numbers.

Again, the instance of the `Setting()` object is exported to xml.

```python
point = openmc.stats.Point((0, 0, 0))
src = openmc.Source(space=point)

settings = openmc.Settings()
settings.source = src
settings.batches = 100
settings.inactive = 10
settings.particles = 5000
settings.export_to_xml()
```

```python
!cat settings.xml
```

## Plotting the geometry

There are several ways of plotting the geometry. The simplest inline plotting option is to call the `plot()` method of the `Universe()` object (in our case this was called `root`). This is a wrapper of matplotlib. However, there is a more advanced options by setting up an instance of the `Plot()` class, and running openMC directly to generate a plot. Since in this course we work with a rather simple geometry, we will not look at the advanced plotting option, but you are welcome to check the user's guide.

```python
root.plot()
```

## Calculation

Finally, we are done, and nothing left just to run openMC and wait for the results. For this we have to specify where the cross section files are (this we could do in the system as well). Then call the `openmc.run()` method. And you immediately see how the k-effective is being estimated.

**Note 1**: The results are being stored in a file named 'statepoint.100.h5' (100 refers to the number of batches), also a 'summary.h5' and some other files are created. Sometimes openMC is complaining if there are already existing h5 files, so in this case we can remove (`rm`) these files. For this we make a system call with `os.system` to remove any file for which the name starts with 's' and ands with 'h5'. Nevertheless, be careful with removing files, not to loose data.

**Note 2**: Sometimes the python API breaks for no apparent reason, you can just restart the kernel, if that happens.

```python
import os
os.system('rm s*h5')

openmc.run()
```

## Post-processing

Alright, we did our calculation, we even saw that some k-effective values were printed, but we still haven't seen any more results. For this we will need to read in the statepoint file. We will read this file into an object called `sp`. If we hit TAB while typing `sp.` we can review the available methods and attributes. For example we can get the final k-effective as below.

```python
sp = openmc.StatePoint('statepoint.100.h5')
sp.k_combined
```

And the values we can access for further processing or plotting:

```python
keff=sp.k_combined.nominal_value
keff_error=sp.k_combined.std_dev
print(keff,keff_error)
```

The tallies are stored in a dictionary where the keys are the numeric IDs we defined previously:

```python
sp.tallies
```

We can convert the tally results into pandas dataframes. So at the end we have a nice table, for each energy bin there is a row for the flux and a row for the fission rate. If we prefer we can split the dataframe based on conditions to store separately the flux and the fission rate. For this we can apply conditions as we have seen during the previous datalabs.

The results are stored in the 'mean' column.

```python
tallydf1=sp.tallies[1].get_pandas_dataframe()
tallydf1.head() #prints the first 5 rows
```

```python
tallydf1flux=tallydf1[tallydf1['score']=='flux']
tallydf1fiss=tallydf1[tallydf1['score']=='fission']
tallydf1flux.head()
```

Let's plot these results.

```python
energy=(tallydf1flux['energy low [eV]']+tallydf1flux['energy high [eV]'])/2
plt.figure()
plt.loglog(energy,tallydf1flux['mean'])
plt.xlabel('energy (eV)')
plt.ylabel('Group flux per source particle')
plt.show()

```

These of course does not look like the spectrum we saw during the lecture. But here actually each value is not the flux at a certain energy, but the integral of the flux between energies. If we divide the integral flux with the width of the bins (`deltaE`) we get the more familiar shape for the spectrum.

We can clearly see the Maxwellien thermal component, the 1/E part with the self-shielding effect of the resonances, and the Watt-spectrum at high energies.

The very first "negative" peak is due to the 6.67 eV resonance of U-238.

```python
deltaE=(tallydf1flux['energy high [eV]']-tallydf1flux['energy low [eV]'])
plt.figure()
plt.loglog(energy,tallydf1flux['mean']/deltaE,lw=2)

plt.ylabel('Spectrum per source particle (1/eV)')
plt.xlabel('Energy (eV)')
plt.show()
```

Now it is your turn to plot the spectrum in the moderator (preferably include both the fuel and the moderator spectrum in the same figure). You will need to `get_pandas_dataframe` from tally 2, and then plot the results. Compare with the spectrum in the fuel, what is the most noticable difference?

```python
# your code comes here
```

Your conclusion comes here.

Finally, we can look at the spatial dependence of the flux. We read in the 3rd tally. Try to split the dataset according to the energy groups and plot the flux vs x-coordinate for the three energy groups separately. Normalize each curve by their maximum, so they are comparable.

Conclude your findings!

```python
tallydf3=sp.tallies[3].get_pandas_dataframe()
tallydf3.head()
```

```python
# your code comes here
```

Your conclusion comes here!

## Experiment time!

If time permits, modify your input, and see how the results change. Points of interest:

- Change the void content (ie. decrease the moderator density), and observe how the k-eff changes. What happens with the spectrum?
- What is the impact on the k-eff if you change the UO2 temperature?
- What is the impact of increasing the pitch of the pincell?

You can also take a look at the third set of home assignments, where you will need to implement various geometries in openMC, and already give it a try.

```python

```


# [Applied Statistics](https://lamastex.github.io/scalable-data-science/as/2019/)
## 1MS926, Spring 2019, Uppsala University
©2019 Raazesh Sainudiin. [Attribution 4.0 International (CC BY 4.0)](https://creativecommons.org/licenses/by/4.0/)

# Assignment 1 for Course 1MS926
Fill in your Personal Number, make     sure you pass the `# ... Test` cells and
 submit by email with *Subject line*     **1MS926 Assignment1**.
You can submit multiple times before the deadline     and your highest score will be used.

```python
# Enter your 12 digit personal number here and evaluate this cell
MyPersonalNumber = 'YYYYMMDDXXXX'

#tests
assert(isinstance(MyPersonalNumber, basestring))
assert(MyPersonalNumber.isdigit())
assert(len(MyPersonalNumber)==12)
```

---
---
## Assignment 1, Problem 0
Maximum Points = 3

Given that you are in the civil engineering programme in *systems in technology and society*, spend some time reading the following:
- [https://anatomyof.ai/](https://anatomyof.ai/), "Anatomy of an AI System" By Kate Crawford and Vladan Joler (2018)

Answer whether each of the following statements is `True` or `False` *according to the authors* by appropriately replacing `Xxxxx` coresponding to `TruthValueOfStatement0a`, `TruthValueOfStatement0b` and `TruthValueOfStatement0c`, respectively, in the next cell to demonstrate your reading comprehension.

1. `Statement0a =` *Each small moment of convenience (provided by Amazon's Echo) – be it answering a question, turning on a light, or playing a song – requires a vast planetary network, fueled by the extraction of non-renewable materials, labor, and data.*
- `Statement0b =` *The Echo user is simultaneously a consumer, a resource, a worker, and a product*
- `Statement0c =` *Many of the assumptions about human life made by machine learning systems are narrow, normative and laden with error. Yet they are inscribing and building those assumptions into a new world, and will increasingly play a role in how opportunities, wealth, and knowledge are distributed.*

```python
# Replace Xxxxx with True or False; Don't modify anything else in this cell!

TruthValueOfStatement0a = Xxxxx

TruthValueOfStatement0b = Xxxxx

TruthValueOfStatement0c = Xxxxx
```

```python
# ASSIGNMENT 1, Test 0, POINTS 3
# Test locally to ensure an acceptable answer, True or False
assert(isinstance(TruthValueOfStatement0a, bool))
assert(isinstance(TruthValueOfStatement0b, bool))
assert(isinstance(TruthValueOfStatement0c, bool))
```

```python
# ASSIGNMENT 1, SOLUTION 0, POINTS 3
assert(isinstance(TruthValueOfStatement0a, bool))
assert(TruthValueOfStatement0a == True)
assert(isinstance(TruthValueOfStatement0b, bool))
assert(TruthValueOfStatement0b == True)
assert(isinstance(TruthValueOfStatement0c, bool))
assert(TruthValueOfStatement0c == True)
```

---
---
## Assignment 1, Problem 1
Maximum Points = 1

#### NonAutoGraded
#### Summarise in 50-100 words your thoughts after reading [https://anatomyof.ai/](https://anatomyof.ai/), "Anatomy of an AI System" By Kate Crawford and Vladan Joler (2018)

You can double-click this cell and start writing your summary below between the two `---` lines in English.

---

---

---
---
## Assignment 1, Problem 2
Maximum Points = 2

### Finding out the number of lines and characters in a file

Evaluate the following two cells by replacing `X` with the right command-line option to `wc` command in order to find:

1. the number of lines in `data/earthquakes_small.csv` and
2. the number of characters in `data/earthquakes_small.csv`

Finally, update the following cell by replacing `XXX` with the right integer answers, respectively, for:

1. `NumberOfLinesIn_earthquakes_small_csv_file` and
2. `NumberOfCharactersIn_earthquakes_small_csv_file`

Here is a brief synopsis of `wc` that you would get from running `man wc` as follows:
```
%%sh
man wc
```

```
WC(1)                     BSD General Commands Manual                    WC(1)

NAME
     wc -- word, line, character, and byte count

SYNOPSIS
     wc [-clmw] [file ...]

DESCRIPTION
     The wc utility displays the number of lines, words, and bytes contained in each input file, or standard input (if no file is specified) to the standard output.  A line is defined as a string of characters delimited by a <newline> character.  Characters beyond the final <newline> character will not be included in the line count.

     A word is defined as a string of characters delimited by white space characters.  White space characters are the set of characters for which the iswspace(3) function returns true.  If more than one input file is specified, a line of cumulative counts for all the files is displayed on a separate line after the output for the last file.

     The following options are available:

     -c      The number of bytes in each input file is written to the standard output.  This will cancel out any prior usage of the -m option.

     -l      The number of lines in each input file is written to the standard output.

     -m      The number of characters in each input file is written to the standard output.  If the current locale does not support multibyte
             characters, this is equivalent to the -c option.  This will cancel out any prior usage of the -c option.

     -w      The number of words in each input file is written to the standard output.

     When an option is specified, wc only reports the information requested by that option.  The order of output always takes the form of line, word, byte, and file name.  The default action is equivalent to specifying the -c, -l and -w options.
```

```sh
%%sh
# replace X in the next line with the right option to find the number of lines
wc -X data/earthquakes_small.csv
```

```sh
%%sh
# replace X in the next line with the right option to find the number of characters
wc -X data/earthquakes_small.csv
```

```python
# write your answer below by replacing XXX don't modify anything else!

NumberOfLinesIn_earthquakes_small_csv_file = XXX
NumberOfCharactersIn_earthquakes_small_csv_file = XXX
```

```python
# ASSIGNMENT 1, Test 2, POINTS 2
# Run this test locally to make sure you have the answer as a non-negative integer
assert(NumberOfLinesIn_earthquakes_small_csv_file > -1)
assert(NumberOfCharactersIn_earthquakes_small_csv_file > -1)
```

```python
# ASSIGNMENT 1, SOLUTION 2, POINTS 2
NumberOfLinesIn_earthquakes_small_csv_file = 411
NumberOfCharactersIn_earthquakes_small_csv_file = 77786
```

```python
# ASSIGNMENT 1, TEST 2, POINTS 2
assert(NumberOfLinesIn_earthquakes_small_csv_file==411)
assert(NumberOfCharactersIn_earthquakes_small_csv_file==77786)
```

---
---
## Assignment 1, Problem 3
Maximum Points = 1

Consider the experiment where we roll two fair dice independently.

Let $D$ be the event that "the sum of the two dice is 8" and let $C$ be the event that "the first die is 2".

What is the probability of D given C, i.e. what is $P(D|C)$?

Do the calculation by hand and write the answer in the next cell by assigning the variable `ProbOfDGivenC`.

```python
# Replace XXX below with the correct answer to Assignment 1 Problem 3
# Do NOT change the name of the variable ProbOfDGivenC
ProbOfDGivenC = XXX
```

```python
# ASSIGNMENT 1, Test 3, POINTS 1
# test that your answer is indeed a probability!
assert(ProbOfDGivenC >= 0 and ProbOfDGivenC <= 1)
```

```python
# ASSIGNMENT 1, SOLUTION 3, POINTS 1
ProbOfDGivenC = 1/5
```

```python
# ASSIGNMENT 1, TEST 3, POINTS 1
assert(ProbOfDGivenC = 1/5)
```

---
---
## Assignment 1, Problem 4
Maximum Points = 2

Recall that for a given parameter $\theta \in [0,1]$, the probability mass function (PMF) for the $Bernoulli(\theta)$ RV $X$ is:

$$
\begin{equation}
f(x;\theta)= \theta^x (1-\theta)^{1-x} \mathbf{1}_{\{0,1\}}(x) =
\begin{cases}
\theta & \text{if $x=1$,}\\
1-\theta & \text{if $x=0$,}\\
0 & \text{otherwise}
\end{cases}
\end{equation}
$$

In the next cell write a function named `pmfOfBernoulli` that takes in two arguments:

- the first argument is `x` and
- the second argument is `theta`

and returns the value for $f(x; \theta)$.

```python
# Replace RRR...RRR below Do NOT change the name of the function `pmfOfBernoulli`!

def pmfOfBernoulli(x, theta):
    '''RRR ... RRR'''
    RRR
    RRR
    RRR
    ...
    RRR
```

```python
# ASSIGNMENT 1, Test 4, POINTS 2
# Use this to locally test that your solution is returning probabilities
assert(pmfOfBernoulli(0, 1/2) >=0)
assert(pmfOfBernoulli(1, 1/2) <=1)
```

```python
# ASSIGNMENT 1, SOLUTION 4, POINTS 2
def pmfOfBernoulli(x, theta):
    '''return the probability of a Bernoulli(theta) RV with outcome x, i.e., the PMF f(x; theta)'''

    # first check that theta is in the allowed parameter range [0,1]
    # you can do by a simple if then statement after doing some try-catch to make sure it was a number
    try:
        param_theta = RR(theta)
    except TypeError:
        print "Input Error: parameter theta should be a real number"
    # next try-catch to see if the outcome_x is convertible into a real number
    try:
        outcome_x = RR(x)
    except TypeError:
        print "Input Error: outcome x should be a real number as the domain of the PMF is the real line"
    # you will get some credits without try-catch but it's important to check the edge cases and constraints

    if (theta < 0 or theta > 1):
        print "Input Error: Parameter theta Out of Range as theta \in [0,1], but the input value of theta = ", theta
        return "parameter error, check input" # one can do more sophisticated error outputs
    else:
        if x == 0:
            return 1-theta
        if x==1:
            return theta
        else:
            return 0
```

```python
# ASSIGNMENT 1, TEST 4, POINTS 2
assert(pmfOfBernoulli(1, 0.5)==0.5)
assert(pmfOfBernoulli(0, 0.5)==0.5)
assert(pmfOfBernoulli(1, 0.25)==0.25)
assert(pmfOfBernoulli(0, 0.25)==0.75)
assert(pmfOfBernoulli(10, 0.25)==0)
assert(pmfOfBernoulli(-10, 0.25)==0)

# due to more try-catch statements the following calls will make sense, uncomment and evaluate one by one
#pmfOfBernoulli(1, 1.25)
#pmfOfBernoulli('1', 0.5)
#pmfOfBernoulli('outcome is 1', 0.5)
```


```python
import sys
sys.path = ['/Users/sebastian/github/mlxtend/'] + sys.path
```

```python
#load watermark
%load_ext watermark
%watermark -a 'Gopala KR' -u -d -v -p watermark,numpy,pandas,matplotlib,nltk,sklearn,tensorflow,theano,mxnet,chainer,seaborn,keras,tflearn,bokeh,gensim
```

    The watermark extension is already loaded. To reload it, use:
      %reload_ext watermark
    Gopala KR
    last updated: 2018-02-13

    CPython 3.6.3
    IPython 6.2.1

    watermark 1.6.0
    numpy 1.13.1
    pandas 0.20.3
    matplotlib 2.0.2
    nltk 3.2.5
    sklearn 0.19.0
    tensorflow 1.3.0
    theano 1.0.1
    mxnet 1.0.0
    chainer 3.3.0
    seaborn 0.8.1
    keras 2.1.3
    tflearn n
    bokeh 0.12.14
    gensim 3.3.0

# Artificial Neurons and Single-Layer Neural Networks

## - How Machine Learning Algorithms Work Part 1

**This article offers a brief glimpse of the history and basic concepts of machine learning. We will take a look at the first algorithmically described neural network and the gradient descent algorithm in context of adaptive linear neurons, which will not only introduce the principles of machine learning but also serve as the basis for modern multilayer neural networks in future articles.**

## Sections

- [Introduction](#Introduction)
- [Artificial Neurons and the McCulloch-Pitts Model](#Artificial-Neurons-and-the-McCulloch-Pitts-Model)
- [Frank Rosenblatt's Perceptron](#Frank-Rosenblatt's-Perceptron)
    - [The Unit Step Function](#The-Unit-Step-Function)
    - [The Perceptron Learning Rule](#The-Perceptron-Learning-Rule)
    - [Implementing the Perceptron Rule in Python](#Implementing-the-Perceptron-Rule-in-Python)
    - [Problems with Perceptrons](#Problems-with-Perceptrons)
- [Adaptive Linear Neurons and the Delta Rule](#Adaptive-Linear-Neurons-and-the-Delta-Rule)
    - [Gradient Descent](#Gradient-Descent)
    - [The Gradient Descent Rule in Action](#The-Gradient-Descent-Rule-in-Action)
    - [Online Learning via Stochastic Gradient Descent](#Online-Learning-via-Stochastic-Gradient-Descent)
- [What's Next?](#What's-Next?)
- [References](#References)

# Introduction

[[back to top](#Sections)]

Machine learning is one of the hottest and most exciting fields in the modern age of technology. Thanks to machine learning, we enjoy robust email spam filters, convenient text and voice recognition, reliable web search engines, challenging chess players, and, hopefully soon, safe and efficient self-driving cars.

Without any doubt, machine learning has become a big and popular field, and sometimes it may be challenging to see the (random) forest for the (decision) trees. Thus, I thought that it might be worthwhile to explore different machine learning algorithms in more detail by not only discussing the theory but also by implementing them step by step.

To briefly summarize what machine learning is all about: "[Machine learning is the] field of study that gives computers the ability to learn without being explicitly programmed" (Arthur Samuel, 1959). Machine learning is about the development and use of algorithms that can recognize patterns in data in order to make decisions based on statistics, probability theory, combinatorics, and optimization.

The first article in this series will introduce perceptrons and the adaline (ADAptive LINear NEuron), which fall into the category of single-layer neural networks. The perceptron is not only the first algorithmically described learning algorithm [[1](#References)], but it is also very intuitive, easy to implement, and a good entry point to the (re-discovered) modern state-of-the-art machine learning algorithms: Artificial neural networks (or "deep learning" if you like). As we will see later, the adaline is a consequent improvement of the perceptron algorithm and offers a good opportunity to learn about a popular optimization algorithm in machine learning: gradient descent.

# Artificial Neurons and the McCulloch-Pitts Model

[[back to top](#Sections)]

The initial idea of the perceptron dates back to the work of Warren McCulloch and Walter Pitts in 1943 [[2](#References)], who drew an analogy between biological neurons and simple logic gates with binary outputs. In more intuitive terms, neurons can be understood as the subunits of a neural network in a biological brain. Here, the signals of variable magnitudes arrive at the dendrites. Those input signals are then accumulated in the cell body of the neuron, and if the accumulated signal exceeds a certain threshold, a output signal is generated that which will be passed on by the axon.

# Frank Rosenblatt's Perceptron

[[back to top](#Sections)]

To continue with the story, a few years after McCulloch and Walter Pitt, Frank Rosenblatt published the first concept of the Perceptron learning rule [[1](#References)]. The main idea was to define an algorithm in order to learn the values of the weights $w$ that are then multiplied with the input features in order to make a decision whether a neuron fires or not. In context of pattern classification, such an algorithm could be useful to determine if a sample belongs to one class or the other.

To put the perceptron algorithm into the broader context of machine learning: The perceptron belongs to the category of supervised learning algorithms, single-layer binary linear classifiers to be more specific. In brief, the task is to predict to which of two possible categories a certain data point belongs based on a set of input variables. In this article, I don't want to discuss the concept of predictive modeling and classification in too much detail, but if you prefer more background information, please see my previous article "[Introduction to supervised learning](http://sebastianraschka.com/Articles/2014_intro_supervised_learning.html)".

## The Unit Step Function

[[back to top](#Sections)]

Before we dive deeper into the algorithm(s) for learning the weights of the artificial neuron, let us take a brief look at the basic notation. In the following sections, we will label the *positive* and *negative* class in our binary classification setting as "1" and "-1", respectively. Next, we define an activation function $g(\mathbf{z})$ that takes a linear combination of the input values $\mathbf{x}$ and weights $\mathbf{w}$ as input ($\mathbf{z} = w_1x_{1} + \dots + w_mx_{m}$), and if $g(\mathbf{z})$ is greater than a defined threshold $\theta$ we predict 1 and -1 otherwise; in this case, this activation function $g$ is an alternative form of a simple "unit step function," which is sometimes also called "Heaviside step function."

*(Please note that the *unit step* is classically defined as being equal to 0 if $ z < 0$ and 1 for $z \ge 0$; nonetheless, we will refer to the following piece-wise linear function with -1 if $z < \theta$ and 1 for $z \ge \theta$ as *unit step function* for simplicity).*

$$
 g(\mathbf{z}) =\begin{cases}
    1 & \text{if $\mathbf{z} \ge \theta$}\\
    -1 & \text{otherwise}.
  \end{cases}
$$

where

$$\mathbf{z} =  w_1x_{1} + \dots + w_mx_{m} = \sum_{j=1}^{m} x_{j}w_{j} \\ = \mathbf{w}^T\mathbf{x}$$

$\mathbf{w}$ is the feature vector, and $\mathbf{x}$ is an $m$-dimensional sample from the training dataset:

$$
\mathbf{w} = \begin{bmatrix}
    w_{1}  \\
    \vdots \\
    w_{m}
\end{bmatrix}
\quad  \mathbf{x} = \begin{bmatrix}
    x_{1}  \\
    \vdots \\
    x_{m}
\end{bmatrix}$$

In order to simplify the notation, we bring $\theta$ to the left side of the equation and define $w_0 = -\theta  \text{ and } x_0=1$

so that

$$\begin{equation}
 g({\mathbf{z}}) =\begin{cases}
    1 & \text{if $\mathbf{z} \ge 0$}\\
    -1 & \text{otherwise}.
  \end{cases}
\end{equation}$$

and

$$\mathbf{z} = w_0x_{0} + w_1x_{1} + \dots + w_mx_{m} = \sum_{j=0}^{m} x_{j}w_{j} \\ = \mathbf{w}^T\mathbf{x}.$$

## The Perceptron Learning Rule

[[back to top](#Sections)]

It might sound like extreme case of a reductionist approach, but the idea behind this "thresholded" perceptron was to mimic how a single neuron in the brain works: It either "fires" or not. To summarize the main points from the previous section: A perceptron receives multiple input signals, and if the sum of the input signals exceed a certain threshold it either returns a signal or  remains "silent" otherwise. What made this a "machine learning" algorithm was Frank Rosenblatt's idea of the perceptron learning rule: The perceptron algorithm is about learning the weights for the input signals in order to draw linear decision boundary that allows us to discriminate between the two linearly separable classes +1 and -1.

Rosenblatt's initial perceptron rule is fairly simple and can be summarized by the following steps:

1. Initialize the weights to 0 or small random numbers.
2. For each training sample $\mathbf{x^{(i)}}$:
    2. Calculate the *output* value.
    2. Update the weights.

The output value is the class label predicted by the unit step function that we defined earlier (output $=g(\mathbf{z})$) and the weight update can be written more formally as  $w_j := w_j + \Delta w_j$.

The value for updating the weights at each increment is calculated by the learning rule

$\Delta w_j = \eta \; (\text{target}^{(i)} - \text{output}^{(i)})\;x^{(i)}_{j}$

where $\eta$ is the learning rate (a constant between 0.0 and 1.0), "target" is the true class label, and the "output" is the predicted class label.

It is important to note that all weights in the weight vector are being updated simultaneously. Concretely, for a 2-dimensional dataset, we would write the update as:

$\Delta w_0 = \eta(\text{target}^{(i)} - \text{output}^{(i)})$
$\Delta w_1 = \eta(\text{target}^{(i)} - \text{output}^{(i)})\;x^{(i)}_{1}$
$\Delta w_2 = \eta(\text{target}^{(i)} - \text{output}^{(i)})\;x^{(i)}_{2}$

Before we implement the perceptron rule in Python, let us make a simple thought experiment to illustrate how beautifully simple this learning rule really is. In the two scenarios where the perceptron predicts the class label correctly, the weights remain unchanged:

- $\Delta w_j = \eta(-1^{(i)} - -1^{(i)})\;x^{(i)}_{j} = 0$
- $\Delta w_j = \eta(1^{(i)} - 1^{(i)})\;x^{(i)}_{j} = 0$

However, in case of a wrong prediction, the weights are being "pushed" towards the direction of the positive or negative target class, respectively:

- $\Delta w_j = \eta(1^{(i)} - -1^{(i)})\;x^{(i)}_{j} = \eta(2)\;x^{(i)}_{j}$
- $\Delta w_j = \eta(-1^{(i)} - 1^{(i)})\;x^{(i)}_{j} = \eta(-2)\;x^{(i)}_{j}$

It is important to note that the convergence of the perceptron is only guaranteed if the two classes are linearly separable. If the two classes can't be separated by a linear decision boundary, we can set a maximum number of passes over the training dataset ("epochs") and/or a threshold for the number of tolerated misclassifications.

## Implementing the Perceptron Rule in Python

[[back to top](#Sections)]

In this section, we will implement the simple perceptron learning rule in Python to classify flowers in the Iris dataset.
Please note that I omitted some "safety checks" for clarity, for a more "robust" version please see the following [code on GitHub](https://github.com/rasbt/mlxtend/blob/master/mlxtend/classifier/perceptron.py).

```python
import numpy as np

class Perceptron(object):

    def __init__(self, eta=0.01, epochs=50):
        self.eta = eta
        self.epochs = epochs

    def train(self, X, y):

        self.w_ = np.zeros(1 + X.shape[1])
        self.errors_ = []

        for _ in range(self.epochs):
            errors = 0
            for xi, target in zip(X, y):
                update = self.eta * (target - self.predict(xi))
                self.w_[1:] +=  update * xi
                self.w_[0] +=  update
                errors += int(update != 0.0)
            self.errors_.append(errors)
        return self

    def net_input(self, X):
        return np.dot(X, self.w_[1:]) + self.w_[0]

    def predict(self, X):
        return np.where(self.net_input(X) >= 0.0, 1, -1)
```

For the following example, we will load the Iris data set from the [UCI Machine Learning Repository](http://archive.ics.uci.edu/ml/) and only focus on the two flower species *Setosa* and *Versicolor*. Furthermore, we will only use the two features *sepal length* and *petal length* for visualization purposes.

```python
import pandas as pd
df = pd.read_csv('https://archive.ics.uci.edu/ml/machine-learning-databases/iris/iris.data', header=None)

# setosa and versicolor
y = df.iloc[0:100, 4].values
y = np.where(y == 'Iris-setosa', -1, 1)

# sepal length and petal length
X = df.iloc[0:100, [0,2]].values
```

```python
!pip install mlxtend
```

    Collecting mlxtend
      Downloading mlxtend-0.10.0-py2.py3-none-any.whl (1.3MB)
        100% |████████████████████████████████| 1.3MB 927kB/s ta 0:00:01
    Requirement already satisfied: scikit-learn>=0.18 in /srv/venv/lib/python3.6/site-packages (from mlxtend)
    Requirement already satisfied: setuptools in /srv/venv/lib/python3.6/site-packages (from mlxtend)
    Requirement already satisfied: matplotlib>=1.5.1 in /srv/venv/lib/python3.6/site-packages (from mlxtend)
    Requirement already satisfied: numpy>=1.10.4 in /srv/venv/lib/python3.6/site-packages (from mlxtend)
    Requirement already satisfied: pandas>=0.17.1 in /srv/venv/lib/python3.6/site-packages (from mlxtend)
    Requirement already satisfied: scipy>=0.17 in /srv/venv/lib/python3.6/site-packages (from mlxtend)
    Requirement already satisfied: pyparsing!=2.0.0,!=2.0.4,!=2.1.2,!=2.1.6,>=1.5.6 in /srv/venv/lib/python3.6/site-packages (from matplotlib>=1.5.1->mlxtend)
    Requirement already satisfied: pytz in /srv/venv/lib/python3.6/site-packages (from matplotlib>=1.5.1->mlxtend)
    Requirement already satisfied: python-dateutil in /srv/venv/lib/python3.6/site-packages (from matplotlib>=1.5.1->mlxtend)
    Requirement already satisfied: cycler>=0.10 in /srv/venv/lib/python3.6/site-packages (from matplotlib>=1.5.1->mlxtend)
    Requirement already satisfied: six>=1.10 in /srv/venv/lib/python3.6/site-packages (from matplotlib>=1.5.1->mlxtend)
    Installing collected packages: mlxtend
    Successfully installed mlxtend-0.10.0

```python
%matplotlib inline
import matplotlib.pyplot as plt
from mlxtend.plotting import plot_decision_regions

ppn = Perceptron(epochs=10, eta=0.1)

ppn.train(X, y)
print('Weights: %s' % ppn.w_)
plot_decision_regions(X, y, clf=ppn)
plt.title('Perceptron')
plt.xlabel('sepal length [cm]')
plt.ylabel('petal length [cm]')
plt.show()

plt.plot(range(1, len(ppn.errors_)+1), ppn.errors_, marker='o')
plt.xlabel('Iterations')
plt.ylabel('Misclassifications')
plt.show()

```

As we can see, the perceptron converges after the 6th iteration and separates the two flower classes perfectly.

## Problems with Perceptrons

[[back to top](#Sections)]

Although the perceptron classified the two Iris flower classes perfectly, convergence is one of the biggest problems of the perceptron. Frank Rosenblatt proofed mathematically that the perceptron learning rule converges if the two classes can be separated by linear hyperplane, but problems arise if the classes cannot be separated perfectly by a linear classifier. To demonstrate this issue, we will use two different classes and features from the Iris dataset.

```python
# versicolor and virginica
y2 = df.iloc[50:150, 4].values
y2 = np.where(y2 == 'Iris-virginica', -1, 1)

# sepal width and petal width
X2 = df.iloc[50:150, [1,3]].values

ppn = Perceptron(epochs=25, eta=0.01)
ppn.train(X2, y2)

plot_decision_regions(X2, y2, clf=ppn)
plt.show()

plt.plot(range(1, len(ppn.errors_)+1), ppn.errors_, marker='o')
plt.xlabel('Iterations')
plt.ylabel('Misclassifications')
plt.show()
```

```python
print('Total number of misclassifications: %d of 100' % (y2 != ppn.predict(X2)).sum())
```

    Total number of misclassifications: 43 of 100

Even at a lower training rate, the perceptron failed to find a good decision boundary since one or more samples will always be misclassified in every epoch so that the learning rule never stops updating the weights.

It may seem paradoxical in this context that another shortcoming of the perceptron algorithm is that it stops updating the weights as soon as all samples are classified correctly. Our intuition tells us that a decision boundary with a large margin between the classes (as indicated by the dashed line in the figure below) likely has a better generalization error than the decision boundary of the perceptron. But large-margin classifiers such as Support Vector Machines are a topic for another time.

# Adaptive Linear Neurons and the Delta Rule

[[back to top](#Sections)]

The perceptron surely was very popular at the time of its discovery, however, it only took a few years until Bernard Widrow and his doctoral student Tedd Hoff proposed the idea of the Adaptive Linear Neuron (adaline) [[3](#References)].

In contrast to the perceptron rule, the delta rule of the adaline (also known as Widrow-Hoff" rule or Adaline rule) updates the weights based on a linear activation function rather than a unit step function; here, this linear activation function $g(\mathbf{z})$ is just the identity function of the net input $g(\mathbf{w}^T\mathbf{x}) = \mathbf{w}^T\mathbf{x}$. In the next section, we will see why this linear activation is an improvement over the perceptron update and where the name "delta rule" comes from.

## Gradient Descent

[[back to top](#Sections)]

Being a continuous function, one of the biggest advantages of the linear activation function over the unit step function is that it is differentiable. This property allows us to define a cost function $J(\mathbf{w})$ that we can minimize in order to update our weights. In the case of the linear activation function, we can define the cost function $J(\mathbf{w})$ as the *sum of squared errors* (SSE), which is similar to the cost function that is minimized in ordinary least squares (OLS) linear regression.

$$J(\mathbf{w})  = \frac{1}{2} \sum_{i} (\text{target}^{(i)} - \text{output}^{(i)})^2 \quad \quad \text{output}^{(i)} \in \mathbb{R}$$

(The fraction $\frac{1}{2}$ is just used for convenience to derive the gradient as we will see in the next paragraphs.)

In order to minimize the SSE cost function, we will use gradient descent, a simple yet useful optimization algorithm that is often used in machine learning to find the local minimum of linear systems.

Before we get to the fun part (calculus), let us consider a convex cost function for one single weight. As illustrated in the figure below, we can describe the principle behind gradient descent as "climbing down a hill" until a local or global minimum is reached. At each step, we take a step into the opposite direction of the gradient, and the step size is determined by the value of the learning rate as well as the slope of the gradient.

Now, as promised, onto the fun part -- deriving the Adaline learning rule.
As mentioned above, each update is updated by taking a step into the opposite direction of the gradient $\Delta \mathbf{w} = - \eta \nabla J(\mathbf{w})$, thus, we have to compute the partial derivative of the cost function for each weight in the weight vector:  $\Delta w_j = - \eta \frac{\partial J}{\partial w_j}$.

The partial derivative of the SSE cost function for a particular weight can be calculated as follows:

$$\begin{equation}
 \frac{\partial J}{\partial w_j} = \frac{\partial }{\partial w_j} \frac{1}{2} \sum_i  (t^{(i)} - o^{(i)})^2 \\
= \frac{1}{2} \sum_i \frac{\partial}{\partial w_j} (t^{(i)} - o^{(i)})^2 \\
= \frac{1}{2} \sum_i 2 (t^{(i)} - o^{(i)}) \frac{\partial}{\partial w_j} (t^{(i)} - o^{(i)}) \\
=  \sum_i (t^{(i)} - o^{(i)}) \frac{\partial}{\partial w_j} \bigg(t^{(i)} - \sum_j w_j x^{(i)}_{j}\bigg) \\
= \sum_i  (t^{(i)} - o^{(i)})(-x^{(i)}_{j})
\end{equation}$$

(t = target, o = output)

And if we plug the results back into the learning rule, we get

$\Delta w_j = - \eta \frac{\partial J}{\partial w_j} = - \eta \sum_i  (t^{(i)} - o^{(i)})(- x^{(i)}_{j}) = \eta \sum_i (t^{(i)} - o^{(i)})x^{(i)}_{j}$,

Eventually, we can apply a simultaneous weight update similar to the perceptron rule:

$\mathbf{w} := \mathbf{w} + \Delta \mathbf{w}$.

**Although, the learning rule above looks identical to the perceptron rule, we shall note the two main differences:**

1. Here, the output "o" is a real number and not a class label as in the perceptron learning rule.
2. The weight update is calculated based on all samples in the training set (instead of updating the weights incrementally after each sample), which is why this approach is also called "batch" gradient descent.

## The Gradient Descent Rule in Action

[[back to top](#Sections)]

Now, it's time to implement the gradient descent rule in Python.

```python
import numpy as np

class AdalineGD(object):

    def __init__(self, eta=0.01, epochs=50):
        self.eta = eta
        self.epochs = epochs

    def train(self, X, y):

        self.w_ = np.zeros(1 + X.shape[1])
        self.cost_ = []

        for i in range(self.epochs):
            output = self.net_input(X)
            errors = (y - output)
            self.w_[1:] += self.eta * X.T.dot(errors)
            self.w_[0] += self.eta * errors.sum()
            cost = (errors**2).sum() / 2.0
            self.cost_.append(cost)
        return self

    def net_input(self, X):
        return np.dot(X, self.w_[1:]) + self.w_[0]

    def activation(self, X):
        return self.net_input(X)

    def predict(self, X):
        return np.where(self.activation(X) >= 0.0, 1, -1)
```

In practice, it often requires some experimentation to find a good learning rate for optimal convergence, thus, we will start by plotting the cost for two different learning rates.

```python
ada = AdalineGD(epochs=10, eta=0.01).train(X, y)
plt.plot(range(1, len(ada.cost_)+1), np.log10(ada.cost_), marker='o')
plt.xlabel('Iterations')
plt.ylabel('log(Sum-squared-error)')
plt.title('Adaline - Learning rate 0.01')
plt.show()

ada = AdalineGD(epochs=10, eta=0.0001).train(X, y)
plt.plot(range(1, len(ada.cost_)+1), ada.cost_, marker='o')
plt.xlabel('Iterations')
plt.ylabel('Sum-squared-error')
plt.title('Adaline - Learning rate 0.0001')
plt.show()
```

The two plots above nicely emphasize the importance of plotting learning curves by illustrating two most common problems with gradient descent:

1. If the learning rate is too large, gradient descent will overshoot the minima and diverge.
2. If the learning rate is too small, the algorithm will require too many epochs to converge and can become trapped in local minima more easily.

Gradient descent is also a good example why feature scaling is important for many machine learning algorithms.
It is not only easier to find an appropriate learning rate if the features are on the same scale, but it also often leads to faster convergence and can prevent the weights from becoming too small (numerical stability).

A common way of feature scaling is standardization

$$\mathbf{x}_{j, std} = \frac{\mathbf{x}_j - \mathbf{\mu}_j}{\mathbf{\sigma}_j}$$

where $\mathbf{\mu}_j$ is the sample mean of the feature $\mathbf{x}_{j}$ and $\mathbf{\sigma}_j$ the standard deviation, respectively. After standardization, the features will have unit variance and are centered around mean zero.

```python
# standardize features
X_std = np.copy(X)
X_std[:,0] = (X[:,0] - X[:,0].mean()) / X[:,0].std()
X_std[:,1] = (X[:,1] - X[:,1].mean()) / X[:,1].std()
```

```python
%matplotlib inline
import matplotlib.pyplot as plt
from mlxtend.evaluate import plot_decision_regions

ada = AdalineGD(epochs=15, eta=0.01)

ada.train(X_std, y)
plot_decision_regions(X_std, y, clf=ada)
plt.title('Adaline - Gradient Descent')
plt.xlabel('sepal length [standardized]')
plt.ylabel('petal length [standardized]')
plt.show()

plt.plot(range(1, len( ada.cost_)+1), ada.cost_, marker='o')
plt.xlabel('Iterations')
plt.ylabel('Sum-squared-error')
plt.show()
```

## Online Learning via Stochastic Gradient Descent

[[back to top](#Sections)]

The previous section was all about "batch" gradient descent learning. The "batch" updates refers to the fact that the cost function is minimized based on the complete training data set. If we think back to the perceptron rule, we remember that it performed the weight update incrementally after each individual training sample. This approach is also called "online" learning, and in fact, this is also how Adaline was first described by Bernard Widrow et al. [[3](#References)]

The process of incrementally updating the weights is also called "stochastic" gradient descent since it approximates the minimization of the cost function. Although the stochastic gradient descent approach might sound inferior to gradient descent due its "stochastic" nature and the "approximated" direction (gradient), it can have certain advantages in practice. Often, stochastic gradient descent converges much faster than gradient descent since the updates are applied immediately after each training sample; stochastic gradient descent is computationally more efficient, especially for very large datasets. Another advantage of online learning is that the classifier can be immediately updated as new training data arrives, e.g., in web applications, and old training data can be discarded if storage is an issue. In large-scale machine learning systems, it is also common practice to use so-called "mini-batches", a compromise with smoother convergence than stochastic gradient descent.

In the interests of completeness let us also implement the stochastic gradient descent Adaline and confirm that it converges on the linearly separable iris dataset.

```python
import numpy as np

class AdalineSGD(object):

    def __init__(self, eta=0.01, epochs=50):
        self.eta = eta
        self.epochs = epochs

    def train(self, X, y, reinitialize_weights=True):

        if reinitialize_weights:
            self.w_ = np.zeros(1 + X.shape[1])
        self.cost_ = []

        for i in range(self.epochs):
            for xi, target in zip(X, y):
                output = self.net_input(xi)
                error = (target - output)
                self.w_[1:] += self.eta * xi.dot(error)
                self.w_[0] += self.eta * error

            cost = ((y - self.activation(X))**2).sum() / 2.0
            self.cost_.append(cost)
        return self

    def net_input(self, X):
        return np.dot(X, self.w_[1:]) + self.w_[0]

    def activation(self, X):
        return self.net_input(X)

    def predict(self, X):
        return np.where(self.activation(X) >= 0.0, 1, -1)
```

One more advice before we let the adaline learn via stochastic gradient descent is to shuffle the training dataset to iterate over the training samples in random order.

We shall note that the "standard" stochastic gradient descent algorithm uses sampling "with replacement," which means that at each iteration, a training sample is chosen randomly from the entire training set. In contrast, sampling "without replacement," which means that each training sample is evaluated exactly once in every epoch, is not only easier to implement but also shows a better performance in empirical comparisons. A more detailed discussion about this topic can be found in Benjamin Recht and Christopher Re's paper *Beneath the valley of the noncommutative arithmetic-geometric mean inequality: conjectures, case-studies, and consequences* [[4](#References)].

```python
ada = AdalineSGD(epochs=15, eta=0.01)

# shuffle data
np.random.seed(123)
idx = np.random.permutation(len(y))
X_shuffled, y_shuffled =  X_std[idx], y[idx]

# train and adaline and plot decision regions
ada.train(X_shuffled, y_shuffled)
plot_decision_regions(X_shuffled, y_shuffled, clf=ada)
plt.title('Adaline - Gradient Descent')
plt.xlabel('sepal length [standardized]')
plt.ylabel('petal length [standardized]')
plt.show()

plt.plot(range(1, len(ada.cost_)+1), ada.cost_, marker='o')
plt.xlabel('Iterations')
plt.ylabel('Sum-squared-error')
plt.show()
```

# What's Next?

[[back to top](#Sections)]

Although we covered many different topics during this article, we just scratched the surface of artificial neurons.

In later articles, we will take a look at different approaches to dynamically adjust the learning rate, the concepts of "One-vs-All" and "One-vs-One" for multi-class classification, regularization to overcome overfitting by introducing additional information, dealing with nonlinear problems and multilayer neural networks, different activation functions for artificial neurons, and related concepts such as logistic regression and support vector machines.

### References

[[back to top](#Sections)]

[1] F. Rosenblatt. The perceptron, a perceiving and recognizing automaton Project Para. Cornell Aeronautical Laboratory, 1957.

[2] W. S. McCulloch and W. Pitts. A logical calculus of the ideas immanent in nervous activity. The bulletin of mathematical biophysics, 5(4):115–133, 1943.

[3] B. Widrow et al. Adaptive ”Adaline” neuron using chemical ”memistors”. Number Technical Report 1553-2. Stanford Electron. Labs., Stanford, CA, October 1960.

[4] B. Recht and C. R ́e. Beneath the valley of the noncommutative arithmetic-geometric mean inequality: conjectures, case-studies, and consequences. arXiv preprint arXiv:1202.4184, 2012.

```python

```

```python

```

```python

```

```python
tested; gopal
```