## You don't know what to choose between Matlab and Python :

# Try Julia !

## Why a talk on programming language ?

- [Better Software, Better Research](https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6886129&tag=1) (Goble, C. (2014). IEEE Internet Computing)

>  Of  2,000  scientists surveyed  online, **91%** said  **using**  scientific  software  is  important  for  their own research, **84%** said **developing** scientific  software  is  important  for  their  own  research [...] and **38% spend at least one  fifth  of  their  time  developing  software**.

>   “You  can download our code from the URL supplied. Good luck downloading the only
> postdoc who can get it to run, though” (Ian  Holmes)

- Big Data -> Big Software
- Loi pour une république numérique (2017) $\rightarrow$ new codes must (more or less) be open source (cf https://scinfolex.com/2017/12/08/les-logiciels-produits-par-les-administrations-sont-passes-en-open-source-par-defaut-et-voici-pourquoi/)

# Play with the slides

https://tinyurl.com/introjuliagipsa

or https://mybinder.org/v2/gh/raphbacher/julia-intro/master?filepath=presentation_julia.ipynb

Sources:

https://github.com/raphbacher/julia-intro

# Outline

- State of the art : the two language problem
- What is Julia
- Why Julia is fast
- Julia Ecosystem & interop
- (Parallelism)

## State of the art : Matlab

Pros :
- Polished product with support
- Simulink
- High level syntax

Cons :
- Closed : not everyone has access to it, impossible to put a demonstrator online.
- Slow loops (better since 2015) : not everything is pretty once vectorized
- Not fast per se (Fortran bindings), memory management is difficult
- Not a generalist language (cumbersome to put a demo on the web e.g.)
- 12 000€ each year for the lab

## State of the art : Python

Pros :
- Free and open-source, generalist, widly used outside scientific community
- Lot of scientific communities are embracing it
- Lots of efforts to make it fast (numba, ...)

Cons :
- Scientific computing is not native :
    - all fallback to C/Fortran black-boxes -> limit flexibility
- Object Oriented paradigm can be cumbersome for scientific code

## Science and the two languages problem

Scientists need to easily explore new ideas :

- Need for mathematical abstractions
- Need for customizations (avoid black boxes, try variations)

- But also need for performance (intrinsic goal or to allow re-use)

What is done now (when you need to go further than using existing packages, for e.g. data analysis)

1) Prototyping in R/Python/Matlab

2) Rewriting whole or parts (if money/man power) to a performant low-level language as C/Fortran

# Yet another language

# Here comes Julia :

* innovative open-source generalist programming language with scientific computing at its core
* **easy as Matlab, fast as Fortran, flexible as Python, deep as LISP**
* leverages (for now...) major C/Fortran science packages, e.g. LAPACK, MPI, FFTW...
* 5th language to run at 1 petaflops (Celeste project), after assembly, Fortran, C, C++
* State of the art packages in Optimization (JUMP), Differential Equations (DifEq), ... Well positioned for ML (Flux, Knet, autodiff...)
* solves the "two-language problem"

# Background

* origins at MIT, UCSB applied math & computer science, 2010
* founders Viral Shah, Jeff Bezanson, Stefan Karpinsky (now at Julia Computing LLC), Alan Edelman (MIT)
* ~10-person core language team, ~600 contributors, ~1500 registered packages
* support from Intel, Microsoft, Wall St., Moore Foundation, NumFocus
* julia-0.1 released in 2012, julia-1.0 in August 2018

**Its goal : Solve the languages problem by having a dynamic, high level language with good mathematical expressivity able to be as fast as C/Fortran.**

# Disclaimer

- I am not a Julia expert : I've been following the language for ~ one year and started porting Python codes a few months ago.
- Julia (like Python...) is not a drop-in replacement for Matlab, it is an alternative.
- Don't fix that ain't broken :
    - if Matlab/R/Python/C is good for you, no need to change
    - but if you are sometimes faced with some limitations of these languages, give Julia a try

# Julia: expressive as Matlab

Julia can as expressive as Matlab for interactive calculations with built-in numerical libraries and plotting. A few examples...

### Arrays

```julia
using LinearAlgebra
A = randn(4,4)       # 4 x 4 matrix of normally distributed random numbers
```

```julia
A[:,1]                # familiar colon syntax --extract first column of A
```

## Linear Algebra

```julia
b = randn(4)
x = A\b               # solve Ax=b for x using backslash operator
norm(A*x-b)
```

```julia
U, σ, V = svd(A);     # singular value decomposition --note unicode variable name σ
```

```julia
Σ = diagm(0 => σ)
norm(A - U*Σ*V')      # compute error of SVD factorization A = U Σ V'
```

# Fast as Fortran

```julia
function loop()
    a=0
    for i in 1:1_000_000
        a+=1;
    end
end
@time loop # compile time
@time loop # now fast
```

## Compare with Matlab (R2018)

    function [] = loop()
        a=0;
        for i=1:1000000
            a=i+1;
        end
    end

    f = @() loop();

    timeit(f) => 0.0018s

**2000x slower !**

### Benchmarks on common code pattterns

Takeaway: Julia timings are clustered around C timing, Matlab/Python/R orders of magnitude slower.

### KS-CNAB2 benchmark: CPU time of identical algorithm in six languages.

Takeaway: Julia is nearly as fast as C and Fortran, within a few percent. Codes: [ks.py](https://github.com/johnfgibson/whyjulia/tree/master/ks-codes/ksbenchmark.py#48)
 [ks.m](https://github.com/johnfgibson/whyjulia/tree/master/ks-codes/ksbenchmark.m#46)
 [ks.jl](https://github.com/johnfgibson/whyjulia/tree/master/ks-codes/ksbenchmark.jl#L129) [ks.c](https://github.com/johnfgibson/whyjulia/tree/master/ks-codes/ksbenchmark.c#L82) [ks.cpp](https://github.com/johnfgibson/whyjulia/tree/master/ks-codes/ksbenchmark.cpp#L87) [ks.f90](https://github.com/johnfgibson/whyjulia/tree/master/ks-codes/ksbenchmark.f90#L84)

### KS-CNAB2 benchmark: CPU time versus lines of code

Takeaway: The Julia PDE code is almost as compact as Matlab/Python, almost as fast as C/Fortran.

## Julia: easy, dynamic, and fast. How?

 * **Just-in-time compilation** (JIT)
    - user-level code is compiled to machine code on-the-fly

 * **Meticulous type system**
    - designed to maximize impact of JIT
    - type inference: compiler determines types of variables

 * **Multiple dispatch**
    - functions are compiled for each set of argument types
    - function dispatch determined at compile time when possible, run time when not

### Just-in-time compilation

Functions are compiled to machine code when first run. Subsequent runs are as fast as compiled C, C++, Fortran.

# Caveats

- It's a marathon not a sprint : Julia can be slow at first
- It's an open-source language with lots of good willing contributors, it's not a product (see JuliaPro...)
- If you directly translate from Matlab to Julia you will not always see an improvement, you have exploit Julia strengths (loops to avoid allocations, type stability, ...)

# A julia type

Like a Python object but without methods

```julia
struct Spaceship
           speed::Float64
           Position::Array{Float64,2}
   end
```

## Multiple dispatch

```julia
collide(a::Asteroid, s::Spaceship) # a first method of the function collide
collide(s1::Spaceship, s2::Spaceship) # another method of the function collide
```

#### Logistic map example
$$f(x) = 4\, x\, (1-x)$$

```julia
f(x) = 4x*(1-x)     # define logistic map
@time f(0.3);       # first run is slow
@time f(0.4);       # second run is a thousand times faster
```

#### Observe the compilation of $f(x)$ by stages

user Julia -> generic Julia expression -> typed Julia expression -> intermediate compiler language -> machine code

```julia
include("macros.jl");
```

```julia
@code_lowered f(0.3)  # show f(x) as generic Julia expression
```

```julia
@code_typed f(0.3)    # show f(x) as Julia expr with inferred types, based on the arg types
```

## Type inference and dispatch

We can see that if we call f on an Integer we get a code specialised for integer (i64)

```julia
@code_llvm_nocomment f(0.3)    # show f(x) in intermediate compiler language (LLVM)
```

```julia
@code_native_nocomment f(0.3) # show f(x) in IA-64 assembly language
```

```julia
@code_llvm_nocomment f(3)    # show f(x) in intermediate compiler language (LLVM)
```

# Type stability

No need to declare type but need to be type stable...

```julia
function g1(a,n)
  x = 0
  for i in 1:n
    x += a
  end
end

@code_warntype g1(2.0,10) # see the Union{Float64, Int64}
# @code_llvm_nocomment g1(2.0,10)
```

```julia
function g2(a,n)
  x = zero(a) # get the zero of the type of a
  for i in 1:n
    x += a
  end
end

@code_warntype g2(2.0,10) # the type stable version is shorter thus faster
#@code_llvm_nocomment g2(2.0,10)
```

### Millionth iterate of logistic map: comparison of Julia, C, Matlab

Given $f(x) = 4\,x \, (1-x)$,

let $~~f^N(x) = f(\,f(\,...(\,f(\,f(x)))))$ be the millionth iterate of $f$, with $N=10^6$.

#### $f^N$ in Julia: run-time code generation & compilation!

```julia
# define function that, given function g(x),
# returns iterated function gᴺ(x) = g(g(...(g(g(x)))))
function iterator(g, N)

    # construct gᴺ, the Nth iterate of g
    function gᴺ(x)
       for i ∈ 1:N
          x = g(x)
       end
       return x
    end

    return gᴺ
end

fᴺ = iterator(f, 10^6);  # generate millionth iterate funcion fᴺ(x) for f(x) = 4x(1-x)

fᴺ(0.67)                 # run fᴺ(x) once to compile it
```

**$f^N$ in C++, compiled**

```julia
; cat codes/fN.cpp
```

### Compare execution time of $f^N$ in Julia, C++, Matlab

#### Julia

```julia
print("t=");
@time x = fᴺ(0.67);         # run the Julia fᴺ on x = 0.67
@show x;
```

#### C++

```julia
; g++ -O3 -o fN codes/fN.cpp -lm
```

```julia
; ./fN 0.67
```

Julia and C++ get the same result `x` in roughly same execution time `t`. Sometimes Julia is faster, sometimes C++.

#### Matlab
```
>> tic(); x=fN(0.67); t=toc();
>> t, x
t = 0.048889000000000
x = 0.101168853345475
```
Matlab gets the same result $x$, but ten to twenty times slower than Julia or C++.

# A short break

## An ecosystem of packages

- Most of packages are available on Github (ease collaboration)
- Main fields are grouped in Github Organizations (see https://julialang.org/community/)
- Julia comes with a powerful package/envrionment manager

```julia
using Pkg #load the Pkg stdlib
Pkg.activate(".") #activate the local environment
```

### Singular values of Hilbert matrix and arbitrary precision

The Hilbert matrix $A_{ij} = (i+j-1)^{-1}$ is notoriously ill-conditioned.

```julia
using LinearAlgebra #load stdlib

#install package : no need online, needed on your computer
#Pkg.add(PackageSpec(name="GenericSVD",rev="master"));

using GenericSVD #load package
```

```julia
m = 8
A = [1//(i+j-1) for i=1:m, j=1:m]    # 8 x 8 Hilbert matrix of Rationals
```

```julia
# 16 x 16 Hilbert is too ill-conditioned for 64-bit arithmetic
m = 16 ; A = [1/(i+j-1) for i=1:m, j=1:m]    # 16 x 16 Hilbert matrix of Float64s
@show cond(A)
@show eps(Float64);
```

```julia
σ = svdvals(A)
σ[end-5:end] # last 5 values
```

```julia
# Make 32 x 32 Hilbert matrix of 256-bit BigFloats and show a few elements
setprecision(256)
m = 32
A = [BigFloat(1//(i+j-1)) for i=1:m, j=1:m];
@show A[1,1]
@show A[2,1]
@show A[3,1]
;
```

```julia
# Compute singular values of 32 x 32 Hilbert matrix in 256-bit arithmetic
σ = svdvals!(A);
σ[end-5:end] # last 5 values
```

## User-defined type: Galois field GF(p)

Finite scalar field GF(p) is $\mathbb{Z}/p$, the integers modulo p, where p is prime. Example by Andreas Noack, Julia Computing.

```julia
# Type definition: Galois fields GF(p), where p is prime modulus, T is integer type
struct GF{p,T} <: Number where {p,T<:Integer}
    rep::T  # representative integer which holds the value of a GF(p) variable
    function GF{p,T}(x::Integer) where {p,T<:Integer}
        return new(mod(x, p))
    end
end
GF{p}(x::T) where {p,T<:Integer} = GF{p,T}(x)

# Define some basic methods for GF(p) that all Julia Numbers must have
import Base: convert, inv, one, promote_rule, show, zero

function call(::Type{GF{p}}, x::Integer) where {p}
    if !isprime(p)
        throw(ArgumentError("p must be prime"))
    end
    return GF{p,typeof(x)}(mod(x, p))
end

convert(::Type{GF{p,T}}, x::Integer) where {p,T} = GF{p}(x)
convert(::Type{GF{p}}, x::Integer) where {p} = GF{p}(x)
convert(::Type{GF{p,T}}, x::GF{p}) where {p,T} = GF{p,T}(x.rep)
promote_rule(::Type{GF{p,T1}}, ::Type{T2}) where {p,T1,T2<:Integer} = GF{p,promote_type(T1,T2)}
show(io::IO, x::GF) = show(io, x.rep);

```

```julia
# Define arithmetic operations on GF(p)
import Base: +, -, *, /,conj

for op in (:+, :-, :*)   # loop over ops, defining each in terms of integer ops mod p
    @eval begin
        ($op)(x::GF{p,T}, y::GF{p,T}) where {p,T} = GF{p,T}($(op)(x.rep, y.rep))
    end
end

# Define inverse and ÷. Requires more care than +, -, * because dividing by zero throws error
function inv(x::GF{p,T}) where {p,T}
    if x == zero(x)
        throw(DivideError())
    end
    r, u, v = gcdx(x.rep, p)
    GF{p,T}(u)
end

function conj(x::GF{p,T}) where {p,T}
    u = conj(x.rep)
    GF{p,T}(u)
end

(/)(x::GF{p}, y::GF{p}) where {p}= x*inv(y);
```

```julia
# Create some GF(7) variables and do arithmetic

x = GF{7}(9)   # x =  9 mod 7 = 2
y = GF{7}(12)  # y = 12 mod 7 = 5
@show x
@show y
@show x + y     # 2 + 5 mod 7 = 0
@show x - y     # 2 - 5 mod 7 = 4
@show x * y     # 2 * 5 mod 7 = 3
@show x / y     # 2 ÷ 5 mod 7 = 6, because 2 = 6*5 mod 7
;
```

### Linear algebra over Galois field

```julia
import Random: seed!
seed!(1234)
A = [GF{7}(rand(0:6)) for i = 1:4, j = 1:4]; # matrix of random GF(7) elems
```

```julia
b = [GF{7}(rand(0:6)) for i = 1:4];         # random vector b for Ax=b problem
```

```julia
x = A\b;   # solve Ax=b over GF(p)!
```

```julia
A*x - b   # check that x satisfies Ax=b
```

**Whoa! Built-in backslash operator on matrix of GF(p) worked!**

Julia generated and compiled a GF(p)-specific version of its generic LU decomp function. All it needs is definitions of $+, -, \times, \div$.

# Solving Differential Equations

 Solve the Lorenz equations:

$$
\begin{align}
\frac{dx}{dt} &= σ(y-x) \\
\frac{dy}{dt} &= x(ρ-z) - y \\
\frac{dz}{dt} &= xy - βz \\
\end{align}
$$

Fair Warning ⚠ : the next cell is gonna take some time. Like really. You can go grab a coffee.

This compilation time only occurs the first time you load this version of this package on your machine.
If you are trying this online on Mybinder, remember that each time you connect, you are on a fresh julia install
just built for you, so this precompilation time occurs. And Plots and DifferentialEquations are pretty big libraries (DifEq regroups a lot of solvers).

```julia
#Pkg.add("DifferentialEquations") # add the Differential equation suite
using DifferentialEquations # first time is very slow (precompilation)
using Plots
```

```julia
function lorenz(du,u,p,t)
 du[1] = 10.0*(u[2]-u[1])
 du[2] = u[1]*(28.0-u[3]) - u[2]
 du[3] = u[1]*u[2] - (8/3)*u[3]
end

u0 = [1.0;0.0;0.0] ; tspan = (0.0,100.0);
prob = ODEProblem(lorenz,u0,tspan)
sol = solve(prob)
Plots.plot(sol,vars=(1,2,3))
```

### Add a bit of interaction

```julia
#Pkg.add("IJulia")
#Pkg.add("Interact")
#Pkg.add("Plots")

using Interact #load library
using Plots
using IJulia
```

```julia
x = y = 0:0.1:30

freqs = OrderedDict(zip(["pi/4", "π/2", "3π/4", "π"], [π/4, π/2, 3π/4, π]))

mp = @manipulate for freq1 in freqs, freq2 in slider(0.01:0.1:4π; label="freq2")
    y = @. sin(freq1*x) * sin(freq2*x)
    plot(x, y)
end

display(mp) # do not work online on mybinder for now.

```

## Automatic differentiation

```julia
#Pkg.add("ForwardDiff")
using ForwardDiff

f(x::Vector) = sum(sin, x) + prod(tan, x) * sum(sqrt, x);

x = rand(5) # small size for example's sake

g = x -> ForwardDiff.gradient(f, x); # g = ∇f

@show g(x)

@show ForwardDiff.hessian(f, x)
```

# Interop

Interop possible avec Python, Matlab, R, Java, C/Fortran,...

```julia
#Pkg.add("PyCall") # add python binding package

using PyCall
@pyimport math
math.sin(math.pi / 4) - sin(pi / 4) # python - julia = 0
```

### Macros: code that transforms code

```julia
# @time macro inserts timing and memory profiling into expression, then evaluates, and prints
@time f(2//3)
```

```julia
# @which macro determines which function is called, provides link to source code on GitHub
@which exp(π)
```

```julia
@which exp(π*im)
```

Macros enable run-time code generation and transformation.

Applications :

  * generation and execution of boilerplate code
  * run-time generation and optimization of algorithms, e.g. FFTW, ATLAS
  * symbolic mathematics, automatic differentiation
  * *all written like high-level Python, running like compiled C !!!*

## Parallelism in Julia: just change the array type

Some very trivial examples of Julia's built-in parallelism

```julia
using Distributed
#Pkg.add("DistributedArrays")

addprocs(2);
```

```julia
; cat codes/count_heads.jl
```

```julia
@everywhere include("codes/count_heads.jl")

a = @spawn count_heads(10000000)
b = @spawn count_heads(10000000)
fetch(a)+fetch(b)
```

### Parallel loops with reduction

```julia
nheads = @distributed (+) for i=1:200000000
  Int(rand(Bool))
end
```

And more :
- Distributed arrays
- GPUArrays
- TPUArrays ...

Just changing the behavior of the underlying structure can bring new hardware support for all packages

# Helpful materials

- Main site https://julialang.org/
- Docs https://docs.julialang.org/en/v1/
- Forum https://discourse.julialang.org/
- Book https://benlauwens.github.io/ThinkJulia.jl/latest/book.html
- Blog http://www.stochasticlifestyle.com/
- All-in-one package : https://juliacomputing.com/products/juliapro.html
- Try online (not yet 1.0) : https://juliabox.com/

# Code as a first-class citizen product of research

- The (new version) Julia package manager has reproductibility at its core (each code project is linked to a deterministic set of dependencies)
- Creating a Julia package comes with tools for documentation, unit testing, continuous integration
- New scientific collaborations can be based on software (see the Github organizations such as JuliaDiff, JuliaStats, etc...)

# Food for thoughts

- Cost and open source
    - Matlab licences cost 12000€ each year to the lab
    - Julia (and Python and R) come for free but development is not free
    - A part of Matlab costs could go to financing open source software that is critical for science (see e.g. https://bitbucket.org/paugier/etude-asso-pynumfr/src/default/etude_asso_python_sciences_fr.rst?fileviewer=file-view-default)
- The combo C(++) low-level libraries and high level bindings (as Tensorflow, Keras...) lead to black box workflows...

# Conclusions

Julia
* **Easy as Matlab, fast as Fortran, flexible as Python, deep as Lisp**
*  Scientific computing, from interactive exploration to HPC
*  Paradigms (multiple dispatch) that enforce collaboration

Not covered
* large-scale programming, development ecosystem, environments, debuggers, etc.
* Abstract Types, compositions, ...
* rough edges: plotting, static compilation  (not quite there), package loading times, young 1.0

*Thanks* : the Julia communityfor most of the examples, xkcd

Julia website: http://www.julialang.org, this talk: https://github.com/raphbacher/julia-intro


```python
! pip3 install matplotlib
```

```python
import sys
!{sys.executable} -m pip install matplotlib
```

    Collecting matplotlib
      Using cached https://files.pythonhosted.org/packages/c3/8b/af9e0984f5c0df06d3fab0bf396eb09cbf05f8452de4e9502b182f59c33b/matplotlib-3.1.1-cp37-cp37m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
    Collecting kiwisolver>=1.0.1 (from matplotlib)
      Using cached https://files.pythonhosted.org/packages/df/93/8bc9b52a8846be2b9572aa0a7c881930939b06e4abe1162da6a0430b794f/kiwisolver-1.1.0-cp37-cp37m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
    Collecting cycler>=0.10 (from matplotlib)
      Using cached https://files.pythonhosted.org/packages/f7/d2/e07d3ebb2bd7af696440ce7e754c59dd546ffe1bbe732c8ab68b9c834e61/cycler-0.10.0-py2.py3-none-any.whl
    Collecting numpy>=1.11 (from matplotlib)
      Using cached https://files.pythonhosted.org/packages/c1/4b/78119133136c20e5ad2e01bf72b0633241defd619939908223cd394a9c32/numpy-1.17.0-cp37-cp37m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
    Requirement already satisfied: python-dateutil>=2.1 in /usr/local/Cellar/ipython/7.7.0/libexec/vendor/lib/python3.7/site-packages (from matplotlib) (2.8.0)
    Collecting pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 (from matplotlib)
      Using cached https://files.pythonhosted.org/packages/11/fa/0160cd525c62d7abd076a070ff02b2b94de589f1a9789774f17d7c54058e/pyparsing-2.4.2-py2.py3-none-any.whl
    Requirement already satisfied: setuptools in /usr/local/lib/python3.7/site-packages (from kiwisolver>=1.0.1->matplotlib) (41.0.1)
    Requirement already satisfied: six in /usr/local/Cellar/ipython/7.7.0/libexec/vendor/lib/python3.7/site-packages (from cycler>=0.10->matplotlib) (1.12.0)
    Installing collected packages: kiwisolver, cycler, numpy, pyparsing, matplotlib
    Successfully installed cycler-0.10.0 kiwisolver-1.1.0 matplotlib-3.1.1 numpy-1.17.0 pyparsing-2.4.2

```python
import matplotlib.pyplot as plt
```

```python
plt.plot(range(100))
```

```python
%time range(100)
```

    CPU times: user 3 µs, sys: 0 ns, total: 3 µs
    Wall time: 5.96 µs

    range(0, 100)

```python
%time x=range(1000000)
```

    CPU times: user 5 µs, sys: 1e+03 ns, total: 6 µs
    Wall time: 8.11 µs

```python
%%timeit x=range(1000000)
max(x)
```

    28.6 ms ± 370 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)

```python
%%writefile test.txt
this is written from jupyter
notebook
```

    Writing test.txt

```python
%pwd
```

    '/Users/mildo/MyDocuments/datascienceWithPython/module2'

```python
%cat test.txt
```

    this is written from jupyter
    notebook

```python
pwd
```

    '/Users/mildo/MyDocuments/datascienceWithPython/module2'

```python
%pwd
```

    '/Users/mildo/MyDocuments/datascienceWithPython/module2'

```python
%%HTML
<i> hola ale </i>
```

 hola ale

```latex
%%latex
\begin{equation}
Gradient: \nabla J = - 2H^T(Y-HW)
\end{equation}
```

\begin{equation}
Gradient: \nabla J = - 2H^T(Y-HW)
\end{equation}

```latex
%%latex
The equation environment creates numbered formulas you can
label and refer to elsewhere:

\begin{equation}\label{eq:pythagoras}
a^2 + b^2 = c^2 .
\end{equation}
Equation \ref{eq:pythagoras} is the heart of the Pythagorean theorem.

Use the \eqref macro to put parentheses around equation
references: \eqref{eq:pythagoras}.

```

The equation environment creates numbered formulas you can
label and refer to elsewhere:

\begin{equation}\label{eq:pythagoras}
a^2 + b^2 = c^2 .
\end{equation}
Equation \ref{eq:pythagoras} is the heart of the Pythagorean theorem.

Use the \eqref macro to put parentheses around equation
references: \eqref{eq:pythagoras}.

```latex
%%latex

For equations with no numbers, use \verb|equation*|:

\begin{equation*}
2 + 2 = 4 .
\end{equation*}
```

Use the \verb|\eqref| macro to put parentheses around equation
references: \eqref{eq:pythagoras}.

For equations with no numbers, use \verb|equation*|:

\begin{equation*}
2 + 2 = 4 .
\end{equation*}

```python
!pip install ipython-sql
```

    Collecting ipython-sql
      Downloading https://files.pythonhosted.org/packages/ab/df/427e7cf05ffc67e78672ad57dce2436c1e825129033effe6fcaf804d0c60/ipython_sql-0.3.9-py2.py3-none-any.whl
    Requirement already satisfied: ipython>=1.0 in /usr/local/Cellar/ipython/7.7.0/libexec/lib/python3.7/site-packages (from ipython-sql) (7.7.0)
    Collecting prettytable (from ipython-sql)
      Downloading https://files.pythonhosted.org/packages/ef/30/4b0746848746ed5941f052479e7c23d2b56d174b82f4fd34a25e389831f5/prettytable-0.7.2.tar.bz2
    Collecting sqlalchemy>=0.6.7 (from ipython-sql)
      Downloading https://files.pythonhosted.org/packages/c2/75/6217c626fa22ad56ae5ccb1a36e7c4f17f5ca31543887e00179468d10464/SQLAlchemy-1.3.7.tar.gz (5.9MB)
         |████████████████████████████████| 5.9MB 3.0MB/s eta 0:00:01
    Requirement already satisfied: ipython-genutils>=0.1.0 in /usr/local/Cellar/ipython/7.7.0/libexec/vendor/lib/python3.7/site-packages (from ipython-sql) (0.2.0)
    Requirement already satisfied: six in /usr/local/Cellar/ipython/7.7.0/libexec/vendor/lib/python3.7/site-packages (from ipython-sql) (1.12.0)
    Collecting sqlparse (from ipython-sql)
      Downloading https://files.pythonhosted.org/packages/ef/53/900f7d2a54557c6a37886585a91336520e5539e3ae2423ff1102daf4f3a7/sqlparse-0.3.0-py2.py3-none-any.whl
    Requirement already satisfied: setuptools>=18.5 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from ipython>=1.0->ipython-sql) (40.8.0)
    Requirement already satisfied: jedi>=0.10 in /usr/local/Cellar/ipython/7.7.0/libexec/vendor/lib/python3.7/site-packages (from ipython>=1.0->ipython-sql) (0.14.1)
    Requirement already satisfied: decorator in /usr/local/Cellar/ipython/7.7.0/libexec/vendor/lib/python3.7/site-packages (from ipython>=1.0->ipython-sql) (4.4.0)
    Requirement already satisfied: pickleshare in /usr/local/Cellar/ipython/7.7.0/libexec/vendor/lib/python3.7/site-packages (from ipython>=1.0->ipython-sql) (0.7.5)
    Requirement already satisfied: traitlets>=4.2 in /usr/local/Cellar/ipython/7.7.0/libexec/vendor/lib/python3.7/site-packages (from ipython>=1.0->ipython-sql) (4.3.2)
    Requirement already satisfied: prompt_toolkit<2.1.0,>=2.0.0 in /usr/local/Cellar/ipython/7.7.0/libexec/vendor/lib/python3.7/site-packages (from ipython>=1.0->ipython-sql) (2.0.9)
    Requirement already satisfied: pygments in /usr/local/Cellar/ipython/7.7.0/libexec/vendor/lib/python3.7/site-packages (from ipython>=1.0->ipython-sql) (2.4.2)
    Requirement already satisfied: backcall in /usr/local/Cellar/ipython/7.7.0/libexec/vendor/lib/python3.7/site-packages (from ipython>=1.0->ipython-sql) (0.1.0)
    Requirement already satisfied: appnope in /usr/local/Cellar/ipython/7.7.0/libexec/vendor/lib/python3.7/site-packages (from ipython>=1.0->ipython-sql) (0.1.0)
    Requirement already satisfied: pexpect in /usr/local/Cellar/ipython/7.7.0/libexec/vendor/lib/python3.7/site-packages (from ipython>=1.0->ipython-sql) (4.7.0)
    Requirement already satisfied: parso>=0.5.0 in /usr/local/Cellar/ipython/7.7.0/libexec/vendor/lib/python3.7/site-packages (from jedi>=0.10->ipython>=1.0->ipython-sql) (0.5.1)
    Requirement already satisfied: wcwidth in /usr/local/Cellar/ipython/7.7.0/libexec/vendor/lib/python3.7/site-packages (from prompt_toolkit<2.1.0,>=2.0.0->ipython>=1.0->ipython-sql) (0.1.7)
    Requirement already satisfied: ptyprocess>=0.5 in /usr/local/Cellar/ipython/7.7.0/libexec/vendor/lib/python3.7/site-packages (from pexpect->ipython>=1.0->ipython-sql) (0.6.0)
    Installing collected packages: prettytable, sqlalchemy, sqlparse, ipython-sql
      Running setup.py install for prettytable ... done
      Running setup.py install for sqlalchemy ... done
    Successfully installed ipython-sql-0.3.9 prettytable-0.7.2 sqlalchemy-1.3.7 sqlparse-0.3.0

```python
import sys
!{sys.executable} -m pip install ipython-sql
```

    Collecting ipython-sql
      Using cached https://files.pythonhosted.org/packages/ab/df/427e7cf05ffc67e78672ad57dce2436c1e825129033effe6fcaf804d0c60/ipython_sql-0.3.9-py2.py3-none-any.whl
    Collecting prettytable (from ipython-sql)
      Using cached https://files.pythonhosted.org/packages/ef/30/4b0746848746ed5941f052479e7c23d2b56d174b82f4fd34a25e389831f5/prettytable-0.7.2.tar.bz2
    Requirement already satisfied: ipython-genutils>=0.1.0 in /usr/local/Cellar/ipython/7.7.0/libexec/vendor/lib/python3.7/site-packages (from ipython-sql) (0.2.0)
    Collecting sqlalchemy>=0.6.7 (from ipython-sql)
      Using cached https://files.pythonhosted.org/packages/c2/75/6217c626fa22ad56ae5ccb1a36e7c4f17f5ca31543887e00179468d10464/SQLAlchemy-1.3.7.tar.gz
    Collecting sqlparse (from ipython-sql)
      Using cached https://files.pythonhosted.org/packages/ef/53/900f7d2a54557c6a37886585a91336520e5539e3ae2423ff1102daf4f3a7/sqlparse-0.3.0-py2.py3-none-any.whl
    Requirement already satisfied: ipython>=1.0 in /usr/local/Cellar/ipython/7.7.0/libexec/lib/python3.7/site-packages (from ipython-sql) (7.7.0)
    Requirement already satisfied: six in /usr/local/Cellar/ipython/7.7.0/libexec/vendor/lib/python3.7/site-packages (from ipython-sql) (1.12.0)
    Requirement already satisfied: setuptools>=18.5 in /usr/local/lib/python3.7/site-packages (from ipython>=1.0->ipython-sql) (41.0.1)
    Requirement already satisfied: jedi>=0.10 in /usr/local/Cellar/ipython/7.7.0/libexec/vendor/lib/python3.7/site-packages (from ipython>=1.0->ipython-sql) (0.14.1)
    Requirement already satisfied: decorator in /usr/local/Cellar/ipython/7.7.0/libexec/vendor/lib/python3.7/site-packages (from ipython>=1.0->ipython-sql) (4.4.0)
    Requirement already satisfied: pickleshare in /usr/local/Cellar/ipython/7.7.0/libexec/vendor/lib/python3.7/site-packages (from ipython>=1.0->ipython-sql) (0.7.5)
    Requirement already satisfied: traitlets>=4.2 in /usr/local/Cellar/ipython/7.7.0/libexec/vendor/lib/python3.7/site-packages (from ipython>=1.0->ipython-sql) (4.3.2)
    Requirement already satisfied: prompt_toolkit<2.1.0,>=2.0.0 in /usr/local/Cellar/ipython/7.7.0/libexec/vendor/lib/python3.7/site-packages (from ipython>=1.0->ipython-sql) (2.0.9)
    Requirement already satisfied: pygments in /usr/local/Cellar/ipython/7.7.0/libexec/vendor/lib/python3.7/site-packages (from ipython>=1.0->ipython-sql) (2.4.2)
    Requirement already satisfied: backcall in /usr/local/Cellar/ipython/7.7.0/libexec/vendor/lib/python3.7/site-packages (from ipython>=1.0->ipython-sql) (0.1.0)
    Requirement already satisfied: appnope in /usr/local/Cellar/ipython/7.7.0/libexec/vendor/lib/python3.7/site-packages (from ipython>=1.0->ipython-sql) (0.1.0)
    Requirement already satisfied: pexpect in /usr/local/Cellar/ipython/7.7.0/libexec/vendor/lib/python3.7/site-packages (from ipython>=1.0->ipython-sql) (4.7.0)
    Requirement already satisfied: parso>=0.5.0 in /usr/local/Cellar/ipython/7.7.0/libexec/vendor/lib/python3.7/site-packages (from jedi>=0.10->ipython>=1.0->ipython-sql) (0.5.1)
    Requirement already satisfied: wcwidth in /usr/local/Cellar/ipython/7.7.0/libexec/vendor/lib/python3.7/site-packages (from prompt_toolkit<2.1.0,>=2.0.0->ipython>=1.0->ipython-sql) (0.1.7)
    Requirement already satisfied: ptyprocess>=0.5 in /usr/local/Cellar/ipython/7.7.0/libexec/vendor/lib/python3.7/site-packages (from pexpect->ipython>=1.0->ipython-sql) (0.6.0)
    Building wheels for collected packages: prettytable, sqlalchemy
      Building wheel for prettytable (setup.py) ... done
      Stored in directory: /Users/mildo/Library/Caches/pip/wheels/80/34/1c/3967380d9676d162cb59513bd9dc862d0584e045a162095606
      Building wheel for sqlalchemy (setup.py) ... done
      Stored in directory: /Users/mildo/Library/Caches/pip/wheels/80/e0/68/3d11cc7209b5bd2c7d55cbb56c6bda843cc82f77c8387468ea
    Successfully built prettytable sqlalchemy
    Installing collected packages: prettytable, sqlalchemy, sqlparse, ipython-sql
    Successfully installed ipython-sql-0.3.9 prettytable-0.7.2 sqlalchemy-1.3.7 sqlparse-0.3.0

```python
%load_ext sql
```

    The sql extension is already loaded. To reload it, use:
      %reload_ext sql

```python
%sql sqlite://
```

    'Connected: @None'

```sql
%%sql
CREATE TABLE classroom (name, age, totalmarks);
insert into classroom values ('raj',10,10);
insert into classroom values ('ale',30,30);
insert into classroom values ('ser',20,20);
```

     * sqlite://
    Done.
    1 rows affected.
    1 rows affected.
    1 rows affected.

    []

```python
%sql select * from classroom
```

     * sqlite://
    Done.

        name
        age
        totalmarks

        raj
        10
        10

        ale
        30
        30

        ser
        20
        20

```python
%lsmagic
```

    Available line magics:
    %alias  %alias_magic  %autoawait  %autocall  %automagic  %autosave  %bookmark  %cat  %cd  %clear  %colors  %conda  %config  %connect_info  %cp  %debug  %dhist  %dirs  %doctest_mode  %ed  %edit  %env  %gui  %hist  %history  %killbgscripts  %ldir  %less  %lf  %lk  %ll  %load  %load_ext  %loadpy  %logoff  %logon  %logstart  %logstate  %logstop  %ls  %lsmagic  %lx  %macro  %magic  %man  %matplotlib  %mkdir  %more  %mv  %notebook  %page  %pastebin  %pdb  %pdef  %pdoc  %pfile  %pinfo  %pinfo2  %pip  %popd  %pprint  %precision  %prun  %psearch  %psource  %pushd  %pwd  %pycat  %pylab  %qtconsole  %quickref  %recall  %rehashx  %reload_ext  %rep  %rerun  %reset  %reset_selective  %rm  %rmdir  %run  %save  %sc  %set_env  %sql  %store  %sx  %system  %tb  %time  %timeit  %unalias  %unload_ext  %who  %who_ls  %whos  %xdel  %xmode

    Available cell magics:
    %%!  %%HTML  %%SVG  %%bash  %%capture  %%debug  %%file  %%html  %%javascript  %%js  %%latex  %%markdown  %%perl  %%prun  %%pypy  %%python  %%python2  %%python3  %%ruby  %%script  %%sh  %%sql  %%svg  %%sx  %%system  %%time  %%timeit  %%writefile

    Automagic is ON, % prefix IS NOT needed for line magics.

```python

```


```python
from IPython.core.display import HTML
def css_styling():
    styles = open("./styles/custom.css", "r").read()
    return HTML(styles)
css_styling()
```

<link href='http://fonts.googleapis.com/css?family=Alegreya+Sans:100,300,400,500,700,800,900,100italic,300italic,400italic,500italic,700italic,800italic,900italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Arvo:400,700,400italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=PT+Mono' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Shadows+Into+Light' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Philosopher:400,700,400italic,700italic' rel='stylesheet' type='text/css'>

<style>

@font-face {
    font-family: "Computer Modern";
    src: url('http://mirrors.ctan.org/fonts/cm-unicode/fonts/otf/cmunss.otf');
}

#notebook_panel { /* main background */
    background: #888;
    color: #f6f6f6;
}

div #notebook { /* centre the content */
    background: #fff;
    color: #333;
    width: 1200px;
    margin: auto;
    padding-left: 0em;
}

#notebook li { /* More space between bullet points */
margin-top:0.8em;
}

div.cell { /* set cell width to about 80 chars */
    width: 1000px;
}

div.cell.border-box-sizing.code_cell.running {
    /* draw border around running cells */
    border: 3px solid #111;
}

h1 {
    font-family: 'Philosopher', sans-serif;
}
h2 {
    font-family: 'Philosopher', serif;
}
h3{
    font-family: 'Philosopher', serif;
    font-style: 'italic';
    margin-top:12px;
    margin-bottom: 3px;
}
h4{
    font-family: 'Philosopher', serif;
}
h5 {
    font-family: 'Alegreya Sans', sans-serif;
}
h6 {
    font-family: 'PT Mono', sans-serif;
}

div.text_cell_render{
    font-family: 'Arvo' sans-serif;
    line-height: 130%;
    font-size: 115%;
    width:900px;
    margin-left:auto;
    margin-right:auto;
}

.CodeMirror{
        font-family: "PT Mono";
                    font-size: 100%;
            }

.text_cell_render h1 {
    font-weight: 400;
    font-size: 64pt;
    line-height: 100%;
    color: rgb(12,85,97);
    margin-bottom: 0.1em;
    margin-top: 0.1em;
    display: block;
}
.text_cell_render h2 {
    font-weight: 700;
    font-size: 24pt;
    line-height: 100%;
    color: rgb(171,165,131);
    margin-bottom: 0.1em;
    margin-top: 0.1em;
    display: block;
}

.text_cell_render h3 {
    font-style: italic;
    color: rgb(95,92,72);
}

.text_cell_render h5 {
    font-weight: 300;
    font-size: 16pt;
    color: grey;
    font-style: italic;
    margin-bottom: .1em;
    margin-top: 0.1em;
    display: block;
}

.text_cell_render h6 {
    font-weight: 300;
    font-size: 10pt;
    color: grey;
    margin-bottom: 1px;
    margin-top: 1px;
}
div.cell.code_cell {  /* Group area containing both code and output */
background-color: rgba(171,165,131,0.3);
border-radius: 10px; /* rounded borders */
padding: 1em;
}

</style>

# Characteristics and scalar problems

Based on earlier lessons **add explicit links** we expect that the solution of the Riemann problem obeys certain key rules.

1. The solution can be written as ${\bf q}(\xi)$ where $\xi = x/t$ is the self-similarity coordinate.
2. This means that, in regions where the solution is smooth, it must obey

$$
  \left( \frac{\partial {\bf f}}{\partial {\bf q}} - \xi \textrm{Id} \right) \partial_{\xi} {\bf q} = 0.
$$

3. This means that the self-similarity coordinate matches an eigenvalue $\lambda^{(p)}$ of the Jacobian matrix $\partial {\bf f} / \partial {\bf q}$, and that $\partial_{\xi} {\bf q}$ is an eigenvector.
4. When the solution is not continuous, the solution (if it exists) must obey the Rankine-Hugoniot condition

$$
  {\bf f}({\bf q}_r) - {\bf f}({\bf q}_l) = s \left( {\bf q}_r - {\bf q}_l \right)
$$
where $s$ is the speed of the shock, which is a constant.

## Scalar case

If we consider the key conditions in the scalar case, we see that the Jacobian is simply $f'(q)$. For example, given Burgers' equation
$$
  \partial_t q + \partial_x \left( \tfrac{1}{2} q^2 \right) = 0
$$
we have $f'(q) = q$, and so the characteristic speed is given by the value of the data itself.

We know that the solution is constant along characteristic lines. Let us plot those lines and see where the solution is determined.

```python
%matplotlib inline
import matplotlib
import matplotlib.pyplot as plt
import numpy as np
```

```python
def df_burgers(q):
    """The derivative of the flux function, here defined for Burger's equation."""

    return q
```

```python
def plot_characteristics(df, ql, qr):
    """Plot the characteristic lines for the initial data of a scalar Riemann problem."""

    fig = plt.figure(figsize=(12,6))
    ax1 = fig.add_subplot(121)

    dq = np.max([abs(qr-ql), 0.1])
    qmin = np.min([ql, qr])
    qmax = np.max([ql, qr])
    ax1.plot([-1.0, 0.0], [ql, ql], 'b-')
    ax1.plot([0.0, 1.0], [qr, qr], 'g-')
    ax1.set_xbound(-1.0, 1.0)
    ax1.set_ybound(qmin - 0.1*dq, qmax + 0.1*dq)
    ax1.set_xlabel(r"$x$")
    ax1.set_ylabel(r"$q(x, t=0)$")

    ax2 = fig.add_subplot(122)

    slope_l = df(ql)
    slope_r = df(qr)
    left_edge = np.min([-1.0, -1.0 - slope_l])
    right_edge = np.max([1.0, 1.0 - slope_r])
    x_start_points_l = np.linspace(left_edge, 0.0, 20)
    x_start_points_r = np.linspace(0.0, right_edge, 20)
    x_end_points_l = x_start_points_l + slope_l
    x_end_points_r = x_start_points_r + slope_r

    for xs, xe in zip(x_start_points_l, x_end_points_l):
        ax2.plot([xs, xe], [0.0, 1.0], 'b-')
    for xs, xe in zip(x_start_points_r, x_end_points_r):
        ax2.plot([xs, xe], [0.0, 1.0], 'g-')

    x_fill = [x_end_points_l[-1], x_start_points_l[-1], x_end_points_r[0]]
    t_fill = [1.0, 0.0, 1.0]
    ax2.fill_between(x_fill, t_fill, 1.0, facecolor = 'red', alpha = 0.5)

    ax2.set_xbound(-1.0, 1.0)
    ax2.set_ybound(0.0, 1.0)
    ax2.set_xlabel(r"$x$")
    ax2.set_ylabel(r"$t$")

    fig.tight_layout()
```

```python
plot_characteristics(df_burgers, 0.1, -0.2)
```

In the red shaded area the characteristics cross - there is characteristic focusing. The solution is not determined here as it could take the value of the initial data either from the left or from the right, which would give different values.

```python
plot_characteristics(df_burgers, -0.1, 0.2)
```

Alternatively the characteristics could separate. In this case the red shaded region is not determined by the initial data at all.

Try modifying the initial data and see what the results are.

```python
from ipywidgets import interactive, FloatSlider
def interactive_characteristics(ql, qr):
    return plot_characteristics(df_burgers, ql, qr)
interactive(interactive_characteristics,
            ql = FloatSlider(min=-2.0, max=2.0, step=0.1, value=-0.1),
            qr = FloatSlider(min=-2.0, max=2.0, step=0.1, value=0.2))
```

### Exercise

Try repeating this characteristic analysis using different scalar problems, such as the LWR traffic equation

\begin{equation}
  \partial_t q + \partial_x \left( q (1 - q) \right) = 0.
\end{equation}

## Shocks

When the characteristics cross, we have to find a weak solution that gives a unique self-similar solution. A shock satisfying the Rankine-Hugoniot conditions is such a solution.

For Burger's equation we can solve this explicitly. We have
$$
\begin{align}
  && {\bf f}({\bf q}_r) - {\bf f}({\bf q}_l) & = s \left( {\bf q}_r - {\bf q}_l \right) \\
  \implies && \frac{1}{2} \left( q_r^2 - q_l^2 \right) & = s \left( q_r - q_l \right) \\
  \implies && s & = \frac{1}{2} \frac{ q_r^2 - q_l^2 }{ q_r - q_l } \\
  && & = \frac{q_r + q_l}{2}.
\end{align}
$$
Given the shock speed, we then have the explicit solution to the Riemann problem for Burger's equation in the case of a shock, and when it occurs:
$$
  q_l > q_r \quad \implies \quad q(x, t) = \begin{cases} q_l & x < s t \\ q_r & x > s t \end{cases}
$$
where $s = (q_r + q_l) / 2$.

```python
def plot_shock_burgers(ql, qr):
    """Plot the characteristic lines for the shock solution to Burger's equation."""

    assert(ql > qr)

    fig = plt.figure(figsize=(12,6))
    ax1 = fig.add_subplot(121)

    dq = np.max([abs(qr-ql), 0.1])
    qmin = np.min([ql, qr])
    qmax = np.max([ql, qr])
    ax1.plot([-1.0, 0.0], [ql, ql], 'b-')
    ax1.plot([0.0, 1.0], [qr, qr], 'g-')
    ax1.set_xbound(-1.0, 1.0)
    ax1.set_ybound(qmin - 0.1*dq, qmax + 0.1*dq)
    ax1.set_xlabel(r"$x$")
    ax1.set_ylabel(r"$q(x, t=0)$")

    ax2 = fig.add_subplot(122)

    slope_l = ql
    slope_r = qr
    s = 0.5 * (qr + ql)
    ax2.plot([0.0, s], [0.0, 1.0], 'k-', linewidth=3)

    left_edge = np.min([-1.0, -1.0 - slope_l])
    right_edge = np.max([1.0, 1.0 - slope_r])
    x_start_points_l = np.linspace(left_edge, 0.0, 20)
    x_start_points_r = np.linspace(0.0, right_edge, 20)
    t_end_points_l = np.ones_like(x_start_points_l)
    t_end_points_r = np.ones_like(x_start_points_r)

    # Look for intersections
    t_end_points_l = np.minimum(t_end_points_l, x_start_points_l / (s - slope_l))
    t_end_points_r = np.minimum(t_end_points_r, x_start_points_r / (s - slope_r))
    x_end_points_l = x_start_points_l + slope_l * t_end_points_l
    x_end_points_r = x_start_points_r + slope_r * t_end_points_r

    for xs, xe, te in zip(x_start_points_l, x_end_points_l, t_end_points_l):
        ax2.plot([xs, xe], [0.0, te], 'b-')
    for xs, xe, te in zip(x_start_points_r, x_end_points_r, t_end_points_r):
        ax2.plot([xs, xe], [0.0, te], 'g-')

    ax2.set_xbound(-1.0, 1.0)
    ax2.set_ybound(0.0, 1.0)
    ax2.set_xlabel(r"$x$")
    ax2.set_ylabel(r"$t$")

    fig.tight_layout()
```

```python
interactive(plot_shock_burgers,
            ql = FloatSlider(min=-2.0, max=2.0, step=0.1, value=0.1),
            qr = FloatSlider(min=-2.0, max=2.0, step=0.1, value=-0.2))
```

### Exercise

Again, repeat this calculation for the LWR traffic equation.

## Rarefactions

When the characteristics do not cross, we have a rarefaction. Again, for Burger's equation we can solve for this explicitly.

We must have that $\xi = x/t$ is an eigenvalue of the Jacobian $\partial {\bf f} / \partial {\bf q}$. For Burger's equation, that simply means that $\xi = q$.

This holds true within the rarefaction wave - the gap in the $x-t$ plane where the characteristics do not reach. Therefore, we can write the solution in the rarefaction case, and when it holds, as:
$$
  q_l < q_r \quad \implies \quad q(x, t) = \begin{cases} q_l & x < \xi_l t \\ \frac{x}{t} & \xi_l t < x < \xi_r t \\ q_r & x > \xi_r t \end{cases}
$$
where the edges of the rarefaction wave (sometimes known as the *head* and *tail* of the rarefaction) are given by the characteristic speed there, which for Burger's equation means that $\xi_l = q_l$ and $\xi_r = q_r$.

```python
def plot_rarefaction_burgers(ql, qr):
    """Plot the characteristic lines for the rarefaction solution to Burger's equation."""

    assert(ql < qr)

    fig = plt.figure(figsize=(12,6))
    ax1 = fig.add_subplot(121)

    dq = np.max([abs(qr-ql), 0.1])
    qmin = np.min([ql, qr])
    qmax = np.max([ql, qr])
    ax1.plot([-1.0, 0.0], [ql, ql], 'b-')
    ax1.plot([0.0, 1.0], [qr, qr], 'g-')
    ax1.set_xbound(-1.0, 1.0)
    ax1.set_ybound(qmin - 0.1*dq, qmax + 0.1*dq)
    ax1.set_xlabel(r"$x$")
    ax1.set_ylabel(r"$q(x, t=0)$")

    ax2 = fig.add_subplot(122)

    slope_l = ql
    slope_r = qr

    left_edge = np.min([-1.0, -1.0 - slope_l])
    right_edge = np.max([1.0, 1.0 - slope_r])
    x_start_points_l = np.linspace(left_edge, 0.0, 20)
    x_start_points_r = np.linspace(0.0, right_edge, 20)
    x_end_points_l = x_start_points_l + slope_l
    x_end_points_r = x_start_points_r + slope_r

    # Rarefaction wave
    xi_l = ql
    xi_r = qr
    xi = np.linspace(xi_l, xi_r, 7)
    x_end_rarefaction = xi

    for xs, xe in zip(x_start_points_l, x_end_points_l):
        ax2.plot([xs, xe], [0.0, 1.0], 'b-')
    for xs, xe in zip(x_start_points_r, x_end_points_r):
        ax2.plot([xs, xe], [0.0, 1.0], 'g-')
    for xe in x_end_rarefaction:
        ax2.plot([0.0, xe], [0.0, 1.0], 'r--')

    ax2.set_xbound(-1.0, 1.0)
    ax2.set_ybound(0.0, 1.0)
    ax2.set_xlabel(r"$x$")
    ax2.set_ylabel(r"$t$")

    fig.tight_layout()
```

```python
interactive(plot_rarefaction_burgers,
            ql = FloatSlider(min=-2.0, max=2.0, step=0.1, value=-0.1),
            qr = FloatSlider(min=-2.0, max=2.0, step=0.1, value=0.2))
```

### Exercise

Again, repeat this calculation for the LWR traffic equation.

## Solution to the Riemann Problem

For Burger's equation, these two cases are sufficient for a complete solution of the Riemann problem. If $q_l < q_r$ then we have a rarefaction. If $q_l > q_r$ then we have a shock. If $q_l = q_r$ then the solution is trivial (although care should be taken in the code!).

```python
def plot_solution_burgers(ql, qr):
    """Plot the characteristic lines for the full solution to Burger's equation."""

    fig = plt.figure(figsize=(12,6))
    ax1 = fig.add_subplot(121)

    dq = np.max([abs(qr-ql), 0.1])
    qmin = np.min([ql, qr])
    qmax = np.max([ql, qr])
    ax1.plot([-1.0, 0.0], [ql, ql], 'b-')
    ax1.plot([0.0, 1.0], [qr, qr], 'g-')
    ax1.set_xbound(-1.0, 1.0)
    ax1.set_ybound(qmin - 0.1*dq, qmax + 0.1*dq)
    ax1.set_xlabel(r"$x$")
    ax1.set_ylabel(r"$q(x, t=0)$")

    ax2 = fig.add_subplot(122)

    slope_l = ql
    slope_r = qr

    if (ql <= qr):
        # Rarefaction case

        left_edge = np.min([-1.0, -1.0 - slope_l])
        right_edge = np.max([1.0, 1.0 - slope_r])
        x_start_points_l = np.linspace(left_edge, 0.0, 20)
        x_start_points_r = np.linspace(0.0, right_edge, 20)
        x_end_points_l = x_start_points_l + slope_l
        x_end_points_r = x_start_points_r + slope_r

        # Rarefaction wave
        xi_l = ql
        xi_r = qr
        xi = np.linspace(xi_l, xi_r, 7)
        x_end_rarefaction = xi

        for xs, xe in zip(x_start_points_l, x_end_points_l):
            ax2.plot([xs, xe], [0.0, 1.0], 'b-')
        for xs, xe in zip(x_start_points_r, x_end_points_r):
            ax2.plot([xs, xe], [0.0, 1.0], 'g-')
        for xe in x_end_rarefaction:
            ax2.plot([0.0, xe], [0.0, 1.0], 'r--')

    else:
        # Shock case
        s = 0.5 * (qr + ql)
        ax2.plot([0.0, s], [0.0, 1.0], 'k-', linewidth=3)

        left_edge = np.min([-1.0, -1.0 - slope_l])
        right_edge = np.max([1.0, 1.0 - slope_r])
        x_start_points_l = np.linspace(left_edge, 0.0, 20)
        x_start_points_r = np.linspace(0.0, right_edge, 20)
        t_end_points_l = np.ones_like(x_start_points_l)
        t_end_points_r = np.ones_like(x_start_points_r)

        # Look for intersections
        t_end_points_l = np.minimum(t_end_points_l, x_start_points_l / (s - slope_l))
        t_end_points_r = np.minimum(t_end_points_r, x_start_points_r / (s - slope_r))
        x_end_points_l = x_start_points_l + slope_l * t_end_points_l
        x_end_points_r = x_start_points_r + slope_r * t_end_points_r

        for xs, xe, te in zip(x_start_points_l, x_end_points_l, t_end_points_l):
            ax2.plot([xs, xe], [0.0, te], 'b-')
        for xs, xe, te in zip(x_start_points_r, x_end_points_r, t_end_points_r):
            ax2.plot([xs, xe], [0.0, te], 'g-')

    ax2.set_xbound(-1.0, 1.0)
    ax2.set_ybound(0.0, 1.0)
    ax2.set_xlabel(r"$x$")
    ax2.set_ylabel(r"$t$")

    fig.tight_layout()
```

```python
interactive(plot_solution_burgers,
            ql = FloatSlider(min=-2.0, max=2.0, step=0.1, value=-0.1),
            qr = FloatSlider(min=-2.0, max=2.0, step=0.1, value=0.2))
```

### Exercise

Again, repeat this exercise for the LWR traffic equation.

## Going further

For many nonlinear problems this type of construction is sufficient. However, there are cases where it fails.

Consider the *Buckley-Leverett* equation with flux
$$
  f(q) = \frac{q^2}{q^2 + a (1 - q)^2}
$$
where $a \in (0, 1)$ is a constant. We know that to solve within a rarefaction wave we need the derivative, which is
$$
  f'(q) = \frac{2 a q (1 - q)}{ \left[ q^2 + a (1 - q)^2 \right]}.
$$
As $q$ varies from $0$ to $1$ we see that the derivative has a maximum:

```python
a = 0.5
q = np.linspace(0.0, 1.0)
f = q**2 / (q**2 + a * (1.0 - q)**2)
df = 2.0 * a * q * (1.0 - q) / (q**2 + a * (1.0 - q)**2)
fig = plt.figure(figsize=(12,6))
ax1 = fig.add_subplot(121)
ax1.plot(q, f)
ax1.set_xlabel(r"$q$")
ax1.set_ylabel(r"$f$")
ax2 = fig.add_subplot(122)
ax2.plot(q, df)
ax2.set_xlabel(r"$q$")
ax2.set_ylabel(r"$\partial f / \partial q$")
fig.tight_layout()
plt.show()
```

This is a problem, as within a rarefaction wave we have to solve $\xi = f'(q)$. More formally we invert the derivative so that, given the characteristic coordinate $\xi$ we have the solution

\begin{equation}
  q = \left( f' \right)^{-1} (\xi)
\end{equation}

But clearly here there are two possible solutions!

This will be a problem for any nonlinear problem where the second derivative, $f''$, vanishes. For problems such as these, we need to look again at the characteristic structure to see what an allowable solution should be.

### Compound waves

We'll consider the Riemann problem for the Buckley-Leverett problem with $q_l = 1$ and $q_r = 0$.

First let's look at the characteristic speeds of the left and right states. Given $f'$ as above we see that $\xi_l = 0 = \xi_r$. So the characteristics are parallel. At first glance we would not expect either a shock or a rarefaction, but a linear discontinuity. However, such a discontinuity would satisfy the Rankine-Hugoniot conditions with the same speed as the neighbouring characteristics - here 0. Let us check:
$$
\begin{align}
  && f_r - f_l & = s \left( q_r - q_l \right) \\
  \implies && s & = \frac{0 - 1}{0 - 1} \\
  && & = 1.
\end{align}
$$

This cannot be right. If the left and right states were separated by a single discontinuity then it would have to travel to the right with speed 1, and points in the $x-t$ plane with $0 < x < t$ would therefore not be determined.

However, neither can we have a single rarefaction. The left and right characteristics speeds would give the boundaries of such a rarefaction, and here they are both $0$.

The alternative is a *compound wave*: one that is made up of more than one piece, separating rarefactions with discontinuities satisfying the Rankine-Hugoniot conditions.

To construct a solution, let us start from the left state $q_l = 1$. We know that we want to connect to the state at $q_r = 0$, so we want $q$ to decrease as we move along the wave curve.

Recall the behaviour of the flux and the characteristic speed $\xi = f'(q)$:

```python
a = 0.5
q = np.linspace(0.0, 1.0)
f = q**2 / (q**2 + a * (1.0 - q)**2)
df = 2.0 * a * q * (1.0 - q) / (q**2 + a * (1.0 - q)**2)
fig = plt.figure(figsize=(12,6))
ax1 = fig.add_subplot(121)
ax1.plot(q, f)
ax1.set_xlabel(r"$q$")
ax1.set_ylabel(r"$f$")
ax2 = fig.add_subplot(122)
ax2.plot(q, df)
ax2.set_xlabel(r"$q$")
ax2.set_ylabel(r"$\partial f / \partial q$")
fig.tight_layout()
plt.show()
```

As we move from the left state $q_l = 1$ to *smaller* values of $q$ we see that the characteristic coordinate $\xi = f'(q)$ increases, as it should. Therefore the first section of the compound wave could be a rarefaction. However, we cannot get all the way to $q_r = 0$, as after the maximum in $f'$ the characteristic coordinate would start to decrease again.

So, at some point $q_*$ the wave must stop being a rarefaction and join to a discontinuity. In order for the characteristics to make sense, they must *join*, not intersect (as at a shock). That is, the shock speed must match the characteristic speed at $q_*$.

So our first attempt at a solution must try to find a state $q_*$ such that, when connected to the right state $q_r = 0$ using a discontinuity satisfying the Rankine-Hugoniot conditions, the shock speed is $s = f'(q_*)$. This requires that
$$
\begin{align}
  && \frac{q_r^2}{q_r^2 + a (1 - q_r)^2} - \frac{q_*^2}{q_*^2 + a (1 - q_*)^2} & = \frac{2 a q_* (1 - q_*)}{\left[ q_*^2 + a (1 - q_*)^2 \right]^2} \left( q_r - q_* \right) \\
  \implies && \frac{q_*^2}{q_*^2 + a (1 - q_*)^2} & = \frac{2 a q_*^2 (1 - q_*)}{\left[ q_*^2 + a (1 - q_*)^2 \right]^2} \\
  \implies && q_*^2 + a (1 - q_*)^2 & = 2 a  (1 - q_*) \\
  \implies && q_* & = \pm \sqrt{\frac{a}{1+a}}.
\end{align}
$$

The characteristic picture for this shows a rarefaction attached to a *non-classical* shock. The shock is not the standard shock, as the characteristics only converge on it from one side.

```python
def flux_buckley_leverett(q):
    return q**2 / (q**2 + a * (1.0 - q)**2)

def df_buckley_leverett(q):
    return 2.0 * a * q * (1.0 - q) / (q**2 + a * (1.0 - q)**2)

fig = plt.figure(figsize=(12,6))
ax = fig.add_subplot(111)

a = 0.5
ql = 1.0
qr = 0.0
qs = np.sqrt(a / (1.0 + a))
s = df_buckley_leverett(qs) # Shock speed matches characteristic speed on left!

slope_l = df_buckley_leverett(ql)
slope_r = df_buckley_leverett(qr)

left_edge = np.min([-1.0, -1.0 - slope_l])
right_edge = np.max([1.0, 1.0 - slope_r])
x_start_points_l = np.linspace(left_edge, 0.0, 20)
x_start_points_r = np.linspace(0.0, right_edge, 20)
x_end_points_l = x_start_points_l + slope_l
x_end_points_r = x_start_points_r + slope_r

# Rarefaction wave
xi_l = df_buckley_leverett(ql)
xi_r = df_buckley_leverett(qs)
xi = np.linspace(xi_l, xi_r, 7)
x_end_rarefaction = xi

# Shock intersections
t_end_points_r = np.ones_like(x_start_points_r)
t_end_points_r = np.minimum(t_end_points_r, x_start_points_r / (s - slope_r))

for xs, xe in zip(x_start_points_l, x_end_points_l):
    ax.plot([xs, xe], [0.0, 1.0], 'b-')
for xs, xe, te in zip(x_start_points_r, x_end_points_r, t_end_points_r):
    ax.plot([xs, xe], [0.0, te], 'g-')
for xe in x_end_rarefaction:
    ax.plot([0.0, xe], [0.0, 1.0], 'r--')
# Shock
ax.plot([0.0, s], [0.0, 1.0], 'k-', linewidth=3)

ax.set_xbound(-1.0, 1.0)
ax.set_ybound(0.0, 1.0)
ax.set_xlabel(r"$x$")
ax.set_ylabel(r"$t$")

fig.tight_layout()
```

Whilst this approach can be generalised (see e.g. the [thesis by Voss](http://134.130.184.8/opus/volltexte/2005/1210/)), we usually hope that the system is simple enough (i.e., $f''(q) \ne 0$, or the equivalent for a system) that we only have to deal with shocks or rarefactions.


```python
from qiskit import IBMQ
import qiskit
qiskit.__qiskit_version__
```

    {'qiskit-terra': '0.13.0',
     'qiskit-aer': '0.5.0',
     'qiskit-ignis': '0.3.0',
     'qiskit-ibmq-provider': '0.6.0',
     'qiskit-aqua': '0.6.5',
     'qiskit': '0.18.0'}

```python
provider = IBMQ.load_account()
provider = IBMQ.get_provider(hub='ibm-q-ncsu', group='nc-state', project='on-boarding')
#provider = IBMQ.get_provider(hub='ibm-q', group='open', project='main')
provider.backends()
```

    [<IBMQSimulator('ibmq_qasm_simulator') from IBMQ(hub='ibm-q-ncsu', group='nc-state', project='on-boarding')>,
     <IBMQBackend('ibmq_poughkeepsie') from IBMQ(hub='ibm-q-ncsu', group='nc-state', project='on-boarding')>,
     <IBMQBackend('ibmq_vigo') from IBMQ(hub='ibm-q-ncsu', group='nc-state', project='on-boarding')>,
     <IBMQBackend('ibmq_ourense') from IBMQ(hub='ibm-q-ncsu', group='nc-state', project='on-boarding')>,
     <IBMQBackend('ibmq_valencia') from IBMQ(hub='ibm-q-ncsu', group='nc-state', project='on-boarding')>,
     <IBMQBackend('ibmq_london') from IBMQ(hub='ibm-q-ncsu', group='nc-state', project='on-boarding')>,
     <IBMQBackend('ibmq_burlington') from IBMQ(hub='ibm-q-ncsu', group='nc-state', project='on-boarding')>,
     <IBMQBackend('ibmq_almaden') from IBMQ(hub='ibm-q-ncsu', group='nc-state', project='on-boarding')>,
     <IBMQBackend('ibmq_rochester') from IBMQ(hub='ibm-q-ncsu', group='nc-state', project='on-boarding')>,
     <IBMQBackend('ibmq_essex') from IBMQ(hub='ibm-q-ncsu', group='nc-state', project='on-boarding')>,
     <IBMQBackend('ibmq_paris') from IBMQ(hub='ibm-q-ncsu', group='nc-state', project='on-boarding')>]

```python
import math
import numpy as np
import random
import re
import qiskit.tools.jupyter

from qiskit import *
from qiskit.compiler import transpile
from qiskit.visualization import plot_histogram
from qiskit.tools.monitor import job_monitor

from qiskit.transpiler import PassManager
from qiskit.transpiler import CouplingMap, Layout
from qiskit.transpiler.passes import StochasticSwap, BasicSwap

from qiskit.tools.monitor import job_monitor, backend_monitor, backend_overview
%matplotlib inline
```

```python
from qiskit.transpiler import PassManagerConfig
from qiskit.transpiler.coupling import CouplingMap
from statistics import mean
from passmanager import level_3_with_contant_pure
from benchmark.suites.qpe import QPE
```

```python
device = provider.get_backend('ibmq_almaden')
backend_monitor(device)
device
```

    ibmq_almaden
    ============
    Configuration
    -------------
        n_qubits: 20
        operational: True
        status_msg: active
        pending_jobs: 1
        backend_version: 1.4.6
        basis_gates: ['u1', 'u2', 'u3', 'cx', 'id']
        local: False
        simulator: False
        dt: 3.5555555555555554e-09
        conditional: False
        coupling_map: [[0, 1], [1, 0], [1, 2], [1, 6], [2, 1], [2, 3], [3, 2], [3, 4], [3, 8], [4, 3], [5, 6], [5, 10], [6, 1], [6, 5], [6, 7], [7, 6], [7, 8], [7, 12], [8, 3], [8, 7], [8, 9], [9, 8], [9, 14], [10, 5], [10, 11], [11, 10], [11, 12], [11, 16], [12, 7], [12, 11], [12, 13], [13, 12], [13, 14], [13, 18], [14, 9], [14, 13], [15, 16], [16, 11], [16, 15], [16, 17], [17, 16], [17, 18], [18, 13], [18, 17], [18, 19], [19, 18]]
        max_shots: 8192
        online_date: 2019-09-13T04:00:00+00:00
        u_channel_lo: [[{'q': 1, 'scale': [1.0, 0.0]}], [{'q': 0, 'scale': [1.0, 0.0]}], [{'q': 2, 'scale': [1.0, 0.0]}], [{'q': 6, 'scale': [1.0, 0.0]}], [{'q': 1, 'scale': [1.0, 0.0]}], [{'q': 3, 'scale': [1.0, 0.0]}], [{'q': 2, 'scale': [1.0, 0.0]}], [{'q': 4, 'scale': [1.0, 0.0]}], [{'q': 8, 'scale': [1.0, 0.0]}], [{'q': 3, 'scale': [1.0, 0.0]}], [{'q': 6, 'scale': [1.0, 0.0]}], [{'q': 10, 'scale': [1.0, 0.0]}], [{'q': 1, 'scale': [1.0, 0.0]}], [{'q': 5, 'scale': [1.0, 0.0]}], [{'q': 7, 'scale': [1.0, 0.0]}], [{'q': 6, 'scale': [1.0, 0.0]}], [{'q': 8, 'scale': [1.0, 0.0]}], [{'q': 12, 'scale': [1.0, 0.0]}], [{'q': 3, 'scale': [1.0, 0.0]}], [{'q': 7, 'scale': [1.0, 0.0]}], [{'q': 9, 'scale': [1.0, 0.0]}], [{'q': 8, 'scale': [1.0, 0.0]}], [{'q': 14, 'scale': [1.0, 0.0]}], [{'q': 5, 'scale': [1.0, 0.0]}], [{'q': 11, 'scale': [1.0, 0.0]}], [{'q': 10, 'scale': [1.0, 0.0]}], [{'q': 12, 'scale': [1.0, 0.0]}], [{'q': 16, 'scale': [1.0, 0.0]}], [{'q': 7, 'scale': [1.0, 0.0]}], [{'q': 11, 'scale': [1.0, 0.0]}], [{'q': 13, 'scale': [1.0, 0.0]}], [{'q': 12, 'scale': [1.0, 0.0]}], [{'q': 14, 'scale': [1.0, 0.0]}], [{'q': 18, 'scale': [1.0, 0.0]}], [{'q': 9, 'scale': [1.0, 0.0]}], [{'q': 13, 'scale': [1.0, 0.0]}], [{'q': 16, 'scale': [1.0, 0.0]}], [{'q': 11, 'scale': [1.0, 0.0]}], [{'q': 15, 'scale': [1.0, 0.0]}], [{'q': 17, 'scale': [1.0, 0.0]}], [{'q': 16, 'scale': [1.0, 0.0]}], [{'q': 18, 'scale': [1.0, 0.0]}], [{'q': 13, 'scale': [1.0, 0.0]}], [{'q': 17, 'scale': [1.0, 0.0]}], [{'q': 19, 'scale': [1.0, 0.0]}], [{'q': 18, 'scale': [1.0, 0.0]}]]
        url: None
        uchannels_enabled: True
        dtm: 3.5555555555555554e-09
        n_registers: 1
        rep_times: [0]
        allow_q_circuit: False
        meas_lo_range: [[6764856891.0, 7764856891.0], [6570226896.0, 7570226896.0], [6722414155.0, 7722414155.0], [6630569974.0, 7630569974.0], [6715856761.0, 7715856761.0], [6479389665.0, 7479389665.0], [6640246272.0, 7640246272.0], [6534825806.0, 7534825806.0], [6797209412.0, 7797209412.0], [6597757684.0, 7597757684.0], [6786899984.0, 7786899984.0], [6554657786.0, 7554657786.0], [6703657122.0, 7703657122.0], [6653318307.0, 7653318307.0], [6682410566.0, 7682410566.0], [6475843032.0, 7475843032.0], [6633222076.0, 7633222076.0], [6518386448.0, 7518386448.0], [6807007960.0, 7807007960.0], [6604129811.0, 7604129811.0]]
        n_uchannels: 46
        allow_object_storage: True
        open_pulse: True
        meas_map: [[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19]]
        credits_required: True
        qubit_lo_range: [[4357152165.037528, 5357152165.037528], [4471766555.388986, 5471766555.388986], [4571111439.9388, 5571111439.9388], [4426983341.092592, 5426983341.092592], [4219733314.5663195, 5219733314.566319], [4275034867.93258, 5275034867.93258], [4413664104.357497, 5413664104.357497], [4289295570.149339, 5289295570.14934], [4357610541.397088, 5357610541.397088], [4159205763.5126843, 5159205763.512684], [4479225971.88113, 5479225971.88113], [4422131122.694569, 5422131122.694569], [4334019900.794435, 5334019900.794435], [4443847604.774473, 5443847604.774473], [4250449580.438109, 5250449580.438108], [4156964331.284013, 5156964331.284013], [4349296297.171849, 5349296297.171849], [4285018344.010058, 5285018344.010057], [4326325747.532302, 5326325747.532302], [4213149144.2146716, 5213149144.214672]]
        backend_name: ibmq_almaden
        hamiltonian: {'description': 'Qubits are modelled as a two level system. Qubits are coupled through resonator buses. The provided Hamiltonian has been projected into the zero excitation subspace of the resonator buses leading to an effective qubit-qubit flip-flop interaction. The qubit resonance frequencies in the Hamiltonian are the cavity dressed frequencies and not exactly what is returned by the backend defaults, which also includes the dressing due to the qubit-qubit interactions.\nWarning: Currently not all system Hamiltonian information is available to the public, missing values have been replaced with 0.\n', 'h_latex': '\\begin{align} \\mathcal{H}/\\hbar = & \\sum_{i=0}^{19}\\left(\\frac{\\omega_{q,i}}{2} (\\mathbb{1}_i-\\sigma_i^{z})+ \\Omega_{d,i}D_i(t)\\sigma_i^{X}\\right) \\\\ & + J_{11,16}(\\sigma_{11}^{+}\\sigma_{16}^{-}+\\sigma_{11}^{-}\\sigma_{16}^{+}) + J_{10,11}(\\sigma_{10}^{+}\\sigma_{11}^{-}+\\sigma_{10}^{-}\\sigma_{11}^{+}) + J_{7,12}(\\sigma_{7}^{+}\\sigma_{12}^{-}+\\sigma_{7}^{-}\\sigma_{12}^{+}) + J_{5,6}(\\sigma_{5}^{+}\\sigma_{6}^{-}+\\sigma_{5}^{-}\\sigma_{6}^{+}) \\\\ & + J_{8,9}(\\sigma_{8}^{+}\\sigma_{9}^{-}+\\sigma_{8}^{-}\\sigma_{9}^{+}) + J_{15,16}(\\sigma_{15}^{+}\\sigma_{16}^{-}+\\sigma_{15}^{-}\\sigma_{16}^{+}) + J_{1,6}(\\sigma_{1}^{+}\\sigma_{6}^{-}+\\sigma_{1}^{-}\\sigma_{6}^{+}) + J_{18,19}(\\sigma_{18}^{+}\\sigma_{19}^{-}+\\sigma_{18}^{-}\\sigma_{19}^{+}) \\\\ & + J_{1,2}(\\sigma_{1}^{+}\\sigma_{2}^{-}+\\sigma_{1}^{-}\\sigma_{2}^{+}) + J_{16,17}(\\sigma_{16}^{+}\\sigma_{17}^{-}+\\sigma_{16}^{-}\\sigma_{17}^{+}) + J_{6,7}(\\sigma_{6}^{+}\\sigma_{7}^{-}+\\sigma_{6}^{-}\\sigma_{7}^{+}) + J_{12,13}(\\sigma_{12}^{+}\\sigma_{13}^{-}+\\sigma_{12}^{-}\\sigma_{13}^{+}) \\\\ & + J_{3,4}(\\sigma_{3}^{+}\\sigma_{4}^{-}+\\sigma_{3}^{-}\\sigma_{4}^{+}) + J_{9,14}(\\sigma_{9}^{+}\\sigma_{14}^{-}+\\sigma_{9}^{-}\\sigma_{14}^{+}) + J_{2,3}(\\sigma_{2}^{+}\\sigma_{3}^{-}+\\sigma_{2}^{-}\\sigma_{3}^{+}) + J_{11,12}(\\sigma_{11}^{+}\\sigma_{12}^{-}+\\sigma_{11}^{-}\\sigma_{12}^{+}) \\\\ & + J_{17,18}(\\sigma_{17}^{+}\\sigma_{18}^{-}+\\sigma_{17}^{-}\\sigma_{18}^{+}) + J_{0,1}(\\sigma_{0}^{+}\\sigma_{1}^{-}+\\sigma_{0}^{-}\\sigma_{1}^{+}) + J_{5,10}(\\sigma_{5}^{+}\\sigma_{10}^{-}+\\sigma_{5}^{-}\\sigma_{10}^{+}) + J_{13,14}(\\sigma_{13}^{+}\\sigma_{14}^{-}+\\sigma_{13}^{-}\\sigma_{14}^{+}) \\\\ & + J_{3,8}(\\sigma_{3}^{+}\\sigma_{8}^{-}+\\sigma_{3}^{-}\\sigma_{8}^{+}) + J_{13,18}(\\sigma_{13}^{+}\\sigma_{18}^{-}+\\sigma_{13}^{-}\\sigma_{18}^{+}) + J_{7,8}(\\sigma_{7}^{+}\\sigma_{8}^{-}+\\sigma_{7}^{-}\\sigma_{8}^{+}) \\\\ & + \\Omega_{d,0}(U_{0}^{(0,1)}(t))\\sigma_{0}^{X} + \\Omega_{d,1}(U_{1}^{(1,0)}(t)+U_{3}^{(1,6)}(t)+U_{2}^{(1,2)}(t))\\sigma_{1}^{X} \\\\ & + \\Omega_{d,2}(U_{4}^{(2,1)}(t)+U_{5}^{(2,3)}(t))\\sigma_{2}^{X} + \\Omega_{d,3}(U_{7}^{(3,4)}(t)+U_{6}^{(3,2)}(t)+U_{8}^{(3,8)}(t))\\sigma_{3}^{X} \\\\ & + \\Omega_{d,4}(U_{9}^{(4,3)}(t))\\sigma_{4}^{X} + \\Omega_{d,5}(U_{11}^{(5,10)}(t)+U_{10}^{(5,6)}(t))\\sigma_{5}^{X} \\\\ & + \\Omega_{d,6}(U_{14}^{(6,7)}(t)+U_{12}^{(6,1)}(t)+U_{13}^{(6,5)}(t))\\sigma_{6}^{X} + \\Omega_{d,7}(U_{15}^{(7,6)}(t)+U_{16}^{(7,8)}(t)+U_{17}^{(7,12)}(t))\\sigma_{7}^{X} \\\\ & + \\Omega_{d,8}(U_{19}^{(8,7)}(t)+U_{18}^{(8,3)}(t)+U_{20}^{(8,9)}(t))\\sigma_{8}^{X} + \\Omega_{d,9}(U_{21}^{(9,8)}(t)+U_{22}^{(9,14)}(t))\\sigma_{9}^{X} \\\\ & + \\Omega_{d,10}(U_{24}^{(10,11)}(t)+U_{23}^{(10,5)}(t))\\sigma_{10}^{X} + \\Omega_{d,11}(U_{25}^{(11,10)}(t)+U_{26}^{(11,12)}(t)+U_{27}^{(11,16)}(t))\\sigma_{11}^{X} \\\\ & + \\Omega_{d,12}(U_{28}^{(12,7)}(t)+U_{30}^{(12,13)}(t)+U_{29}^{(12,11)}(t))\\sigma_{12}^{X} + \\Omega_{d,13}(U_{31}^{(13,12)}(t)+U_{32}^{(13,14)}(t)+U_{33}^{(13,18)}(t))\\sigma_{13}^{X} \\\\ & + \\Omega_{d,14}(U_{34}^{(14,9)}(t)+U_{35}^{(14,13)}(t))\\sigma_{14}^{X} + \\Omega_{d,15}(U_{36}^{(15,16)}(t))\\sigma_{15}^{X} \\\\ & + \\Omega_{d,16}(U_{38}^{(16,15)}(t)+U_{39}^{(16,17)}(t)+U_{37}^{(16,11)}(t))\\sigma_{16}^{X} + \\Omega_{d,17}(U_{41}^{(17,18)}(t)+U_{40}^{(17,16)}(t))\\sigma_{17}^{X} \\\\ & + \\Omega_{d,18}(U_{42}^{(18,13)}(t)+U_{43}^{(18,17)}(t)+U_{44}^{(18,19)}(t))\\sigma_{18}^{X} + \\Omega_{d,19}(U_{45}^{(19,18)}(t))\\sigma_{19}^{X} \\\\ \\end{align}', 'h_str': ['_SUM[i,0,19,wq{i}/2*(I{i}-Z{i})]', '_SUM[i,0,19,omegad{i}*X{i}||D{i}]', 'jq11q16*Sp11*Sm16', 'jq11q16*Sm11*Sp16', 'jq10q11*Sp10*Sm11', 'jq10q11*Sm10*Sp11', 'jq7q12*Sp7*Sm12', 'jq7q12*Sm7*Sp12', 'jq5q6*Sp5*Sm6', 'jq5q6*Sm5*Sp6', 'jq8q9*Sp8*Sm9', 'jq8q9*Sm8*Sp9', 'jq15q16*Sp15*Sm16', 'jq15q16*Sm15*Sp16', 'jq1q6*Sp1*Sm6', 'jq1q6*Sm1*Sp6', 'jq18q19*Sp18*Sm19', 'jq18q19*Sm18*Sp19', 'jq1q2*Sp1*Sm2', 'jq1q2*Sm1*Sp2', 'jq16q17*Sp16*Sm17', 'jq16q17*Sm16*Sp17', 'jq6q7*Sp6*Sm7', 'jq6q7*Sm6*Sp7', 'jq12q13*Sp12*Sm13', 'jq12q13*Sm12*Sp13', 'jq3q4*Sp3*Sm4', 'jq3q4*Sm3*Sp4', 'jq9q14*Sp9*Sm14', 'jq9q14*Sm9*Sp14', 'jq2q3*Sp2*Sm3', 'jq2q3*Sm2*Sp3', 'jq11q12*Sp11*Sm12', 'jq11q12*Sm11*Sp12', 'jq17q18*Sp17*Sm18', 'jq17q18*Sm17*Sp18', 'jq0q1*Sp0*Sm1', 'jq0q1*Sm0*Sp1', 'jq5q10*Sp5*Sm10', 'jq5q10*Sm5*Sp10', 'jq13q14*Sp13*Sm14', 'jq13q14*Sm13*Sp14', 'jq3q8*Sp3*Sm8', 'jq3q8*Sm3*Sp8', 'jq13q18*Sp13*Sm18', 'jq13q18*Sm13*Sp18', 'jq7q8*Sp7*Sm8', 'jq7q8*Sm7*Sp8', 'omegad1*X0||U0', 'omegad0*X1||U1', 'omegad6*X1||U3', 'omegad2*X1||U2', 'omegad1*X2||U4', 'omegad3*X2||U5', 'omegad4*X3||U7', 'omegad2*X3||U6', 'omegad8*X3||U8', 'omegad3*X4||U9', 'omegad10*X5||U11', 'omegad6*X5||U10', 'omegad7*X6||U14', 'omegad1*X6||U12', 'omegad5*X6||U13', 'omegad6*X7||U15', 'omegad8*X7||U16', 'omegad12*X7||U17', 'omegad7*X8||U19', 'omegad3*X8||U18', 'omegad9*X8||U20', 'omegad8*X9||U21', 'omegad14*X9||U22', 'omegad11*X10||U24', 'omegad5*X10||U23', 'omegad10*X11||U25', 'omegad12*X11||U26', 'omegad16*X11||U27', 'omegad7*X12||U28', 'omegad13*X12||U30', 'omegad11*X12||U29', 'omegad12*X13||U31', 'omegad14*X13||U32', 'omegad18*X13||U33', 'omegad9*X14||U34', 'omegad13*X14||U35', 'omegad16*X15||U36', 'omegad15*X16||U38', 'omegad17*X16||U39', 'omegad11*X16||U37', 'omegad18*X17||U41', 'omegad16*X17||U40', 'omegad13*X18||U42', 'omegad17*X18||U43', 'omegad19*X18||U44', 'omegad18*X19||U45'], 'osc': {}, 'qub': {'0': 2, '1': 2, '2': 2, '3': 2, '4': 2, '5': 2, '6': 2, '7': 2, '8': 2, '9': 2, '10': 2, '11': 2, '12': 2, '13': 2, '14': 2, '15': 2, '16': 2, '17': 2, '18': 2, '19': 2}, 'vars': {'jq0q1': 0, 'jq10q11': 0, 'jq11q12': 0, 'jq11q16': 0, 'jq12q13': 0, 'jq13q14': 0, 'jq13q18': 0, 'jq15q16': 0, 'jq16q17': 0, 'jq17q18': 0, 'jq18q19': 0, 'jq1q2': 0, 'jq1q6': 0, 'jq2q3': 0, 'jq3q4': 0, 'jq3q8': 0, 'jq5q10': 0, 'jq5q6': 0, 'jq6q7': 0, 'jq7q12': 0, 'jq7q8': 0, 'jq8q9': 0, 'jq9q14': 0, 'omegad0': 0, 'omegad1': 0, 'omegad10': 0, 'omegad11': 0, 'omegad12': 0, 'omegad13': 0, 'omegad14': 0, 'omegad15': 0, 'omegad16': 0, 'omegad17': 0, 'omegad18': 0, 'omegad19': 0, 'omegad2': 0, 'omegad3': 0, 'omegad4': 0, 'omegad5': 0, 'omegad6': 0, 'omegad7': 0, 'omegad8': 0, 'omegad9': 0, 'wq0': 30.518387118099312, 'wq1': 31.23853057154694, 'wq10': 31.285399467650514, 'wq11': 30.926661950125876, 'wq12': 30.373042815285313, 'wq13': 31.063110631253963, 'wq14': 29.847955006306155, 'wq15': 29.260569862383118, 'wq16': 30.46902724455054, 'wq17': 30.06515695366879, 'wq18': 30.324699024557493, 'wq19': 29.613589453475672, 'wq2': 31.862732890493785, 'wq3': 30.957149337471563, 'wq4': 29.65495901588911, 'wq5': 30.002428923464205, 'wq6': 30.87346210491477, 'wq7': 30.09203155810261, 'wq8': 30.52126718170686, 'wq9': 29.27465319642934}}
        memory: True
        acquisition_latency: []
        allow_q_object: True
        description: 20 qubit device Almaden
        discriminators: ['linear_discriminator', 'quadratic_discriminator']
        meas_kernels: ['boxcar']
        quantum_volume: 8
        max_experiments: 900
        meas_levels: [1, 2]
        conditional_latency: []
        sample_name: HexV2

    Qubits [Name / Freq / T1 / T2 / U1 err / U2 err / U3 err / Readout err]
    -----------------------------------------------------------------------
        Q0 / 4.85715 GHz / 72.63238 µs / 102.04211 µs / 0.0 / 0.00068 / 0.00137 / 0.03667
        Q1 / 4.97177 GHz / 83.68511 µs / 20.15841 µs / 0.0 / 0.00055 / 0.0011 / 0.045
        Q2 / 5.07111 GHz / 96.78522 µs / 19.83045 µs / 0.0 / 0.00047 / 0.00094 / 0.12833
        Q3 / 4.92698 GHz / 93.15195 µs / 138.82031 µs / 0.0 / 0.00036 / 0.00073 / 0.015
        Q4 / 4.71973 GHz / 131.55839 µs / 162.34797 µs / 0.0 / 0.00026 / 0.00052 / 0.01667
        Q5 / 4.77503 GHz / 115.38173 µs / 119.84606 µs / 0.0 / 0.00066 / 0.00132 / 0.08167
        Q6 / 4.91366 GHz / 69.34857 µs / 35.02859 µs / 0.0 / 0.00061 / 0.00122 / 0.05667
        Q7 / 4.7893 GHz / 21.41658 µs / 14.68001 µs / 0.0 / 0.00095 / 0.00189 / 0.04833
        Q8 / 4.85761 GHz / 112.82705 µs / 96.60506 µs / 0.0 / 0.00028 / 0.00057 / 0.02833
        Q9 / 4.65921 GHz / 106.30635 µs / 127.34094 µs / 0.0 / 0.0004 / 0.00079 / 0.02833
        Q10 / 4.97923 GHz / 131.83902 µs / 13.37327 µs / 0.0 / 0.00049 / 0.00098 / 0.03667
        Q11 / 4.92213 GHz / 56.03556 µs / 84.2544 µs / 0.0 / 0.00066 / 0.00132 / 0.00833
        Q12 / 4.83402 GHz / 112.74639 µs / 37.51677 µs / 0.0 / 0.00084 / 0.00167 / 0.01667
        Q13 / 4.94385 GHz / 143.49342 µs / 13.9653 µs / 0.0 / 0.00088 / 0.00176 / 0.025
        Q14 / 4.75045 GHz / 124.95256 µs / 16.51211 µs / 0.0 / 0.0007 / 0.0014 / 0.01833
        Q15 / 4.65696 GHz / 117.64104 µs / 151.27719 µs / 0.0 / 0.00037 / 0.00075 / 0.015
        Q16 / 4.8493 GHz / 86.06115 µs / 111.01472 µs / 0.0 / 0.00037 / 0.00074 / 0.03
        Q17 / 4.78502 GHz / 115.56301 µs / 108.73886 µs / 0.0 / 0.00057 / 0.00114 / 0.025
        Q18 / 4.82633 GHz / 95.36256 µs / 12.55396 µs / 0.0 / 0.00164 / 0.00329 / 0.06
        Q19 / 4.71315 GHz / 143.72866 µs / 195.37614 µs / 0.0 / 0.00084 / 0.00169 / 0.06667

    Multi-Qubit Gates [Name / Type / Gate Error]
    --------------------------------------------
        cx0_1 / cx / 0.00997
        cx1_0 / cx / 0.00997
        cx1_2 / cx / 0.0123
        cx1_6 / cx / 0.01653
        cx2_1 / cx / 0.0123
        cx2_3 / cx / 0.0155
        cx3_2 / cx / 0.0155
        cx3_4 / cx / 0.01163
        cx3_8 / cx / 0.01085
        cx4_3 / cx / 0.01163
        cx5_6 / cx / 0.01164
        cx5_10 / cx / 0.00818
        cx6_1 / cx / 0.01653
        cx6_5 / cx / 0.01164
        cx6_7 / cx / 0.03835
        cx7_6 / cx / 0.03835
        cx7_8 / cx / 0.01683
        cx7_12 / cx / 0.02122
        cx8_3 / cx / 0.01085
        cx8_7 / cx / 0.01683
        cx8_9 / cx / 0.01919
        cx9_8 / cx / 0.01919
        cx9_14 / cx / 0.01158
        cx10_5 / cx / 0.00818
        cx10_11 / cx / 0.01402
        cx11_10 / cx / 0.01402
        cx11_12 / cx / 0.01117
        cx11_16 / cx / 0.01005
        cx12_7 / cx / 0.02122
        cx12_11 / cx / 0.01117
        cx12_13 / cx / 0.01909
        cx13_12 / cx / 0.01909
        cx13_14 / cx / 0.0105
        cx13_18 / cx / 0.02128
        cx14_9 / cx / 0.01158
        cx14_13 / cx / 0.0105
        cx15_16 / cx / 0.01165
        cx16_11 / cx / 0.01005
        cx16_15 / cx / 0.01165
        cx16_17 / cx / 0.0093
        cx17_16 / cx / 0.0093
        cx17_18 / cx / 0.01675
        cx18_13 / cx / 0.02128
        cx18_17 / cx / 0.01675
        cx18_19 / cx / 0.01621
        cx19_18 / cx / 0.01621

    VBox(children=(HTML(value="<h1 style='color:#ffffff;background-color:#000000;padding-top: 1%;padding-bottom: 1…

    <IBMQBackend('ibmq_almaden') from IBMQ(hub='ibm-q-ncsu', group='nc-state', project='on-boarding')>

```python
simulator = provider.get_backend('ibmq_qasm_simulator')
```

```python
seed = 0
pmconfig = PassManagerConfig(
            initial_layout=None,
            basis_gates=['u1', 'u2', 'u3', 'cx', 'id'],
            coupling_map=CouplingMap(device.configuration().coupling_map),
            backend_properties=device.properties(),
            seed_transpiler=seed)
```

```python
pm = level_3_with_contant_pure(pmconfig)
```

```python
QCList = []
for i in range(3, 7):
    QCList.append(transpile(QPE(nbits = i),device, optimization_level = 3))
    QCList.append(pm.run(QPE(nbits = i)))
```

```python
QCList[0].count_ops()
```

    OrderedDict([('cx', 28),
                 ('u2', 13),
                 ('u1', 11),
                 ('u3', 3),
                 ('measure', 3),
                 ('barrier', 1)])

```python
QCList[1].count_ops()
```

    OrderedDict([('cx', 20),
                 ('u1', 15),
                 ('u2', 8),
                 ('measure', 3),
                 ('barrier', 1)])

```python
from qiskit.qobj.utils import MeasLevel, MeasReturnType
from qiskit.pulse import Schedule
from qiskit.exceptions import QiskitError
```

```python
qobj = assemble(QCList,
            qobj_id=None, qobj_header=None,
            shots=8192,
            memory=False, max_credits=10, seed_simulator=None,
            default_qubit_los=None, default_meas_los=None,  # schedule run options
            schedule_los=None, meas_level=MeasLevel.CLASSIFIED,
            meas_return=MeasReturnType.AVERAGE,
            memory_slots=None, memory_slot_size=100, rep_time=None, parameter_binds=None,
            backend=device,
            )
```

```python
job2 = device.run(qobj)
job_monitor(job2)
res2 = job2.result()
```

    Job Status: job has successfully run

```python
plot_histogram(res2.get_counts()[0])
```

```python
plot_histogram(res2.get_counts()[1])
```

```python
plot_histogram(res2.get_counts()[1]).savefig('almadenour.pdf')
```

```python
plot_histogram(res2.get_counts()[0]).savefig('almadenlevel3.pdf')
```

```python
#level3 result
res2.get_counts()[0]
```

    {'111': 1089,
     '000': 655,
     '101': 621,
     '100': 550,
     '001': 960,
     '110': 699,
     '010': 1160,
     '011': 2458}

```python
#our result
res2.get_counts()[1]
```

    {'111': 2935,
     '000': 532,
     '101': 1138,
     '100': 746,
     '001': 683,
     '110': 1025,
     '010': 360,
     '011': 773}


# Summary on the Deep Learning Study, Week 1

- Date: January 15th, 2018
- Lectures Covered
    - 수업에 사용하는 코드 https://github.com/hunkim/DeepLearningZeroToAll
    - 수업의 개요
    - 머신러닝의 개념과 용어
    - Linear Regression 의 개념
    - Linear Regression cost 함수 최소화
    - 여러 개의 입력(feature)의 Linear Regression

```python
import sys
sys.version  # Current version of Python
```

    '3.6.4 |Anaconda custom (64-bit)| (default, Dec 21 2017, 15:39:08) \n[GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE_401/final)]'

```python
import os

# Set an environment variable TF_CPP_MIN_LOG_LEVEL as '2'.
# With the setting, you can ignore tensorflow logs except for error-level logs.
# https://stackoverflow.com/questions/35869137/avoid-tensorflow-print-on-standard-error
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '2'
```

# Environment Setup

## Anaconda

> With over 4.5 million users, the open source Anaconda Distribution is the easiest way to do Python data science and machine learning. It includes hundreds of popular data science packages and the conda package and virtual environment manager for Windows, Linux, and MacOS. Conda makes it quick and easy to install, run, and upgrade complex data science and machine learning environments like scikit-learn, TensorFlow, and SciPy. Anaconda Distribution is the foundation of millions of data science projects as well as Amazon Web Services' Machine Learning AMIs and Anaconda for Microsoft on Azure and Windows.
>
> — [What is Anaconda? :: Anaconda](https://www.anaconda.com/what-is-anaconda/)

- [Downloads :: Anaconda](https://www.anaconda.com/download/)
- [Installation — Conda documentation](https://conda.io/docs/user-guide/install/index.html)

### Install (Windows)

Just download graphical installer and run it! :)

### Install (macOS)

#### Install with the official bash file

```bash
wget https://repo.continuum.io/archive/Anaconda3-5.0.1-MacOSX-x86_64.sh
bash Anaconda3-5.0.1-MacOSX-x86_64.sh
rm Anaconda3-5.0.1-MacOSX-x86_64.sh

# You should add the bin directory of anaconda to the PATH environment variable.
# You can do it by adding a line as below to ~/.bashrc or ~/.zshrc
#
#   export PATH=/path/to/your/anaconda/bin:$PATH
#
# The exact path of the anaconda directory can vary.
```

#### Install with Homebrew

- [Homebrew — The missing package manager for macOS](https://brew.sh/)

```bash
# If you already installed homebrew, just running this line will work!
brew cask install anaconda

# You should add the bin directory of anaconda to the PATH environment variable.
# You can do it by adding a line as below to ~/.bashrc or ~/.zshrc
#
#   export PATH=/path/to/your/anaconda/bin:$PATH
#
# The exact path of the anaconda directory can vary.
```

### Install (Linux)

```bash
wget https://repo.continuum.io/archive/Anaconda3-5.0.1-Linux-x86_64.sh
bash Anaconda3-5.0.1-Linux-x86_64.sh
rm Anaconda3-5.0.1-Linux-x86_64.sh

# You should add the bin directory of anaconda to the PATH environment variable.
# You can do it by adding a line as below to ~/.bashrc or ~/.zshrc
#
#   export PATH=/path/to/your/anaconda/bin:$PATH
#
# The exact path of the anaconda directory can vary.
```

## Conda

> Conda is an open source package management system and environment management system that runs on Windows, macOS and Linux. Conda quickly installs, runs and updates packages and their dependencies. Conda easily creates, saves, loads and switches between environments on your local computer. It was created for Python programs, but it can package and distribute software for any language.
>
> — [Conda documentation page](https://conda.io/docs/)

- [Getting started — Conda documentation](https://conda.io/docs/user-guide/getting-started.html)
- [Cheat sheet — Conda documentation](https://conda.io/docs/user-guide/cheatsheet.html)

### Update all the packages
```bash
# Update all the packages in the base environment of anaconda
conda update --all
```

### Use a virtual environment for TensorFlow

- [Managing environments — Conda documentation](https://conda.io/docs/user-guide/tasks/manage-environments.html)

```bash
# Create new environment, tf, with python 3.6 and anaconda packages
conda create -n tf python=3.6 anaconda

# Activate tf environment
source activate tf

# Install Tensorflow from PyPI
pip install tensorflow

# Deactivate tf environment
source deactivate
```

### (Optional) Use conda-forge instead of PyPI

- [conda-forge | community driven packaging for conda](https://conda-forge.org/)
- [conda-forge :: Anaconda Cloud](https://anaconda.org/conda-forge)

```bash
# Install tensorflow distributed on conda-forge
conda install -c conda-forge tensorflow
```

## Jupyter Notebook

> The Jupyter Notebook is an open-source web application that allows you to create and share documents that contain live code, equations, visualizations and narrative text. Uses include: data cleaning and transformation, numerical simulation, statistical modeling, data visualization, machine learning, and much more.
>
> — [Project Jupyter | Home](http://jupyter.org/)

- [Project Jupyter | Install](http://jupyter.org/install)
- [nacyot의 프로그래밍 이야기 :: 주피터(Jupyter, IPython >= 3)의 다중 커널 개념 이해하기 - 파이썬2와 파이썬3 동시에 사용하기](http://blog.nacyot.com/articles/2015-05-08-jupyter-multiple-pythons/)
- [Jupyter notebook 이해하기](https://www.slideshare.net/dahlmoon/jupyter-notebok-20160815)

### Run

```bash
cd <working-directory>
jupyter notebook
```

## JupyterLab

> An extensible environment for interactive and reproducible computing, based on the Jupyter Notebook and Architecture.
>
> JupyterLab is the next generation user interface for Project Jupyter. It offers all the familiar building blocks of the classic Jupyter Notebook (notebook, terminal, text editor, file browser, rich outputs, etc.) in a flexible and powerful user inteface that can be extended through third party extensions that access our public APIs. Eventually, JupyterLab will replace the classic Jupyter Notebook.
>
> — [jupyterlab README](https://github.com/jupyterlab/jupyterlab)

- [jupyterlab/jupyterlab: JupyterLab computational environment.](https://github.com/jupyterlab/jupyterlab)

### Run

```bash
cd <working-directory>
jupyter lab
```

# Tensorflow

```python
import tensorflow as tf
tf.__version__
```

    /usr/local/anaconda3/envs/tf/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: compiletime version 3.5 of module 'tensorflow.python.framework.fast_tensor_util' does not match runtime version 3.6
      return f(*args, **kwds)
    /usr/local/anaconda3/envs/tf/lib/python3.6/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
      from ._conv import register_converters as _register_converters

    '1.4.1'

```python
hello = tf.constant("Hello, Tensorflow!")

sess = tf.Session()
print(sess.run(hello))
sess.close()
```

    b'Hello, Tensorflow!'

```python
a = tf.constant(5.)
b = tf.constant(6.)
c = a * b

with tf.Session() as sess:
    print(sess.run(c))
```

    30.0

# Datasets

- [Red Wine Quality | Kaggle](https://www.kaggle.com/uciml/red-wine-quality-cortez-et-al-2009)

> **Input variables (based on physicochemical tests)**:
1. fixed acidity
2. volatile acidity
3. citric acid
4. residual sugar
5. chlorides
6. free sulfur dioxide
7. total sulfur dioxide
8. density
9. pH
10. sulphates
11. alcohol

> **Output variable (based on sensory data)**:
1. quality (score between 0 and 10)

> **Relevant Papers**:
>
> - P. Cortez, A. Cerdeira, F. Almeida, T. Matos and J. Reis. Modeling wine preferences by data mining from physicochemical properties.
In Decision Support Systems, Elsevier, 47(4):547-553, 2009.

- Cf. [hunkim/KaggleZeroToAll: Kaggle problem solving](https://github.com/hunkim/KaggleZeroToAll)

```python
DIM_INPUTS = 11
DIM_OUTPUTS = 1
```

## Read datasets

### Using NumPy

- [numpy.loadtxt — NumPy v1.13 Manual](https://docs.scipy.org/doc/numpy-1.13.0/reference/generated/numpy.loadtxt.html)
- [numpy.genfromtxt — NumPy v1.13 Manual](https://docs.scipy.org/doc/numpy-1.13.0/reference/generated/numpy.genfromtxt.html)

```python
import numpy as np

dat = np.genfromtxt('datasets/winequality-red.csv', delimiter=',', skip_header=1)
```

```python
dat
```

    array([[ 7.4  ,  0.7  ,  0.   , ...,  0.56 ,  9.4  ,  5.   ],
           [ 7.8  ,  0.88 ,  0.   , ...,  0.68 ,  9.8  ,  5.   ],
           [ 7.8  ,  0.76 ,  0.04 , ...,  0.65 ,  9.8  ,  5.   ],
           ...,
           [ 6.3  ,  0.51 ,  0.13 , ...,  0.75 , 11.   ,  6.   ],
           [ 5.9  ,  0.645,  0.12 , ...,  0.71 , 10.2  ,  5.   ],
           [ 6.   ,  0.31 ,  0.47 , ...,  0.66 , 11.   ,  6.   ]])

```python
dat.shape
```

    (1599, 12)

### Using Pandas

- [pandas.read_csv — pandas 0.22.0 documentation](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.read_csv.html)

```python
import pandas as pd

dat = pd.read_csv('datasets/winequality-red.csv')
```

```python
type(dat)
```

    pandas.core.frame.DataFrame

```python
dat.head()
```

      fixed acidity
      volatile acidity
      citric acid
      residual sugar
      chlorides
      free sulfur dioxide
      total sulfur dioxide
      density
      pH
      sulphates
      alcohol
      quality

      0
      7.4
      0.70
      0.00
      1.9
      0.076
      11.0
      34.0
      0.9978
      3.51
      0.56
      9.4
      5

      1
      7.8
      0.88
      0.00
      2.6
      0.098
      25.0
      67.0
      0.9968
      3.20
      0.68
      9.8
      5

      2
      7.8
      0.76
      0.04
      2.3
      0.092
      15.0
      54.0
      0.9970
      3.26
      0.65
      9.8
      5

      3
      11.2
      0.28
      0.56
      1.9
      0.075
      17.0
      60.0
      0.9980
      3.16
      0.58
      9.8
      6

      4
      7.4
      0.70
      0.00
      1.9
      0.076
      11.0
      34.0
      0.9978
      3.51
      0.56
      9.4
      5

```python
dat.values
```

    array([[ 7.4  ,  0.7  ,  0.   , ...,  0.56 ,  9.4  ,  5.   ],
           [ 7.8  ,  0.88 ,  0.   , ...,  0.68 ,  9.8  ,  5.   ],
           [ 7.8  ,  0.76 ,  0.04 , ...,  0.65 ,  9.8  ,  5.   ],
           ...,
           [ 6.3  ,  0.51 ,  0.13 , ...,  0.75 , 11.   ,  6.   ],
           [ 5.9  ,  0.645,  0.12 , ...,  0.71 , 10.2  ,  5.   ],
           [ 6.   ,  0.31 ,  0.47 , ...,  0.66 , 11.   ,  6.   ]])

```python
dat.shape
```

    (1599, 12)

```python
dat.describe()
```

      fixed acidity
      volatile acidity
      citric acid
      residual sugar
      chlorides
      free sulfur dioxide
      total sulfur dioxide
      density
      pH
      sulphates
      alcohol
      quality

      count
      1599.000000
      1599.000000
      1599.000000
      1599.000000
      1599.000000
      1599.000000
      1599.000000
      1599.000000
      1599.000000
      1599.000000
      1599.000000
      1599.000000

      mean
      8.319637
      0.527821
      0.270976
      2.538806
      0.087467
      15.874922
      46.467792
      0.996747
      3.311113
      0.658149
      10.422983
      5.636023

      std
      1.741096
      0.179060
      0.194801
      1.409928
      0.047065
      10.460157
      32.895324
      0.001887
      0.154386
      0.169507
      1.065668
      0.807569

      min
      4.600000
      0.120000
      0.000000
      0.900000
      0.012000
      1.000000
      6.000000
      0.990070
      2.740000
      0.330000
      8.400000
      3.000000

      25%
      7.100000
      0.390000
      0.090000
      1.900000
      0.070000
      7.000000
      22.000000
      0.995600
      3.210000
      0.550000
      9.500000
      5.000000

      50%
      7.900000
      0.520000
      0.260000
      2.200000
      0.079000
      14.000000
      38.000000
      0.996750
      3.310000
      0.620000
      10.200000
      6.000000

      75%
      9.200000
      0.640000
      0.420000
      2.600000
      0.090000
      21.000000
      62.000000
      0.997835
      3.400000
      0.730000
      11.100000
      6.000000

      max
      15.900000
      1.580000
      1.000000
      15.500000
      0.611000
      72.000000
      289.000000
      1.003690
      4.010000
      2.000000
      14.900000
      8.000000

### Using TensorFlow

- [Importing Data  |  TensorFlow](https://www.tensorflow.org/programmers_guide/datasets)

## Make X & Y

```python
X_total = dat.values[:, 0:-1]
Y_total = dat.values[:, [-1]]
```

## Split Training/Test datasets

### Using NumPy

```python
# This way of choosing train datasets cannot make fixed rate of train-test split.
# The number of True on the mask vector is distributed around 0.8.
mask = np.random.rand(len(X_total)) < 0.8
```

```python
X_train = X_total[mask]
Y_train = Y_total[mask]
X_test = X_total[~mask]
Y_test = Y_total[~mask]
```

```python
# Formatted String on Python 3.6
print(f'X_train: {X_train.shape}\nX_test: {X_test.shape}')
print(f'Y_train: {Y_train.shape}\nY_test: {Y_test.shape}')
```

    X_train: (1261, 11)
    X_test: (338, 11)
    Y_train: (1261, 1)
    Y_test: (338, 1)

- Cf. [PyFormat: Using % and .format() for great good!](https://pyformat.info/)

### Using Pandas

```python
dat_train = dat.sample(frac=0.8)
dat_test = dat.drop(dat_train.index)
```

```python
X_train, Y_train = dat_train.values[:, 0:-1], dat_train.values[:, [-1]]
X_test, Y_test = dat_test.values[:, 0:-1], dat_test.values[:, [-1]]
```

```python
print(f'X_train: {X_train.shape}\nX_test: {X_test.shape}')
print(f'Y_train: {Y_train.shape}\nY_test: {Y_test.shape}')
```

    X_train: (1279, 11)
    X_test: (320, 11)
    Y_train: (1279, 1)
    Y_test: (320, 1)

### Using Scikit-Learn

- [sklearn.model_selection.train_test_split — scikit-learn 0.19.1 documentation](http://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html)

```python
from sklearn.model_selection import train_test_split
```

```python
X_train, X_test, Y_train, Y_test = train_test_split(X_total, Y_total, test_size=0.2)
```

```python
print(f'X_train: {X_train.shape}\nX_test: {X_test.shape}')
print(f'Y_train: {Y_train.shape}\nY_test: {Y_test.shape}')
```

    X_train: (1279, 11)
    X_test: (320, 11)
    Y_train: (1279, 1)
    Y_test: (320, 1)

# Linear Regression

\begin{align}
    H(X)
    &= X W + b \\
    &= \underbrace{\begin{bmatrix}
        x_{11} & \cdots & x_{1m} \\
        \vdots & \ddots & \vdots \\
        x_{n1} & \cdots & x_{nm}
    \end{bmatrix}}_{n \times m} \cdot \underbrace{\begin{bmatrix}
        w_{11} & \cdots & w_{1k} \\
        \vdots & \ddots & \vdots \\
        w_{m1} & \cdots & w_{mk}
    \end{bmatrix}}_{m \times k} + \underbrace{\begin{bmatrix}
        b_1 \\
        \vdots \\
        b_k \\
    \end{bmatrix}}_{k} \\
    &= \begin{bmatrix}
        (\sum_i x_{1i} w_{i1}) + b_1 & (\sum_i x_{1i} w_{i2}) + b_2 & \cdots & (\sum_i x_{1i} w_{ik}) + b_k \\
        (\sum_i x_{2i} w_{i1}) + b_1 & (\sum_i x_{2i} w_{i2}) + b_2 & \cdots & (\sum_i x_{2i} w_{ik}) + b_k \\
        \vdots & \vdots & \ddots & \vdots \\
        (\sum_i x_{ni} w_{i1}) + b_1 & (\sum_i x_{ni} w_{i2}) + b_2 & \cdots & (\sum_i x_{ni} w_{ik}) + b_k \\
    \end{bmatrix} \\
    &= \begin{bmatrix}
        h_{11} & h_{12} & \cdots & h_{1k} \\
        h_{21} & h_{22} & \cdots & h_{2k} \\
        \vdots & \vdots & \ddots & \vdots \\
        h_{n1} & h_{n2} & \cdots & h_{nk}
    \end{bmatrix} \\
\end{align}

where $n$ = the number of records, $m$ = the number of input variables (aka features, regressors, predictors), $k$ = the number of output variables.

## Model

```python
# Input & Target
X = tf.placeholder(tf.float32, shape=[None, DIM_INPUTS], name='Input')
Y = tf.placeholder(tf.float32, shape=[None, DIM_OUTPUTS], name='Target')

# Weight & Bias
W = tf.Variable(tf.random_normal([DIM_INPUTS, DIM_OUTPUTS]), name='Weight')
b = tf.Variable(tf.random_normal([DIM_OUTPUTS]), name='Bias')
```

```python
# Hypothesis with the linear regression model
H = tf.matmul(X, W) + b

# Cost/Loss function (Mean Squared Error)
cost = tf.reduce_mean(tf.square(H - Y))
```

## Optimization

```python
# Gradient Descent Optimizer provided by TensorFlow
optimizer = tf.train.GradientDescentOptimizer(learning_rate=1e-4)

# Train operation to minimize the cost function
train = optimizer.minimize(cost)
```

```python
# Manually computed gradient
gradient = tf.reduce_mean((H - Y) * X) * 2

# Gradients computed by the optimizer
gvs = optimizer.compute_gradients(cost, [W])

# Apply computed gradients back to optimizer
apply_gradients = optimizer.apply_gradients(gvs)
```

## Start a session

```python
# Launch the graph in a session.
sess = tf.Session()
# Initializes global variables in the graph.
sess.run(tf.global_variables_initializer())
```

## Train

```python
costs = []
for step in range(2001):
    val_cost, _ = sess.run([cost, train], feed_dict={X: X_train, Y: Y_train})
    # val_cost, _ = sess.run([cost, apply_gradients], feed_dict={X: X_train, Y: Y_train})
    costs.append(val_cost)
```

```python
for i in range(0, 2001, 100):
    print(f'Step {i:4d} {costs[i]:.4f}')
```

    Step    0 4731.5576
    Step  100 23.9495
    Step  200 12.0473
    Step  300 9.3886
    Step  400 8.3725
    Step  500 7.6884
    Step  600 7.1054
    Step  700 6.5798
    Step  800 6.1007
    Step  900 5.6630
    Step 1000 5.2628
    Step 1100 4.8969
    Step 1200 4.5622
    Step 1300 4.2561
    Step 1400 3.9761
    Step 1500 3.7200
    Step 1600 3.4855
    Step 1700 3.2710
    Step 1800 3.0746
    Step 1900 2.8949
    Step 2000 2.7302

```python
%matplotlib inline
from matplotlib import pyplot as plt

plt.plot(costs)
plt.xlabel('Steps')
plt.ylabel('MSE cost')
plt.show()
```

## Test

```python
val_cost = sess.run([cost], feed_dict={X: X_test, Y: Y_test})
val_cost
```

    [2.6323833]

## Close the session

```python
sess.close()
```

# Using multiple datasets

```python
filename_queue = tf.train.string_input_producer(
    ['datasets/winequality-red.csv'], shuffle=False, name='filename_queue')
reader = tf.TextLineReader(skip_header_lines=1)
key, value = reader.read(filename_queue)

record_defaults = [[0.]] * 12
xy = tf.decode_csv(value, record_defaults=record_defaults)

train_x_batch, train_y_batch = tf.train.batch([xy[0:-1], xy[-1:]], batch_size=10)
```

## Abstraction with functions

- [Reading data  |  TensorFlow](https://www.tensorflow.org/api_guides/python/reading_data)
- [Importing Data  |  TensorFlow](https://www.tensorflow.org/programmers_guide/datasets)
- [Threading and Queues  |  TensorFlow](https://www.tensorflow.org/versions/r0.12/how_tos/threading_and_queues/)
- [쓰레드와 큐 · 텐서플로우 문서 한글 번역본](https://tensorflowkorea.gitbooks.io/tensorflow-kr/content/g3doc/how_tos/threading_and_queues/)

```python
def read_data_format(filename_queue):
    reader = tf.TextLineReader(skip_header_lines=1)
    key, value = reader.read(filename_queue)

    record_defaults = [[0.]] * 12
    xy = tf.decode_csv(value, record_defaults=record_defaults)

    features = xy[0:-1]
    labels = xy[-1:]

    return features, labels
```

```python
def input_pipeline(filenames, batch_size, num_epochs=None):
    filename_queue = tf.train.string_input_producer(
        filenames, num_epochs=num_epochs, shuffle=True)

    features, labels = read_data_format(filename_queue)

    min_after_dequeue = 10000
    capacity = min_after_dequeue + 3 * batch_size

    feature_batch, label_batch = tf.train.shuffle_batch(
        [features, labels],
        batch_size=batch_size,
        capacity=capacity,
        min_after_dequeue=min_after_dequeue)

    return feature_batch, label_batch
```

```python
train_x_batch, train_y_batch = input_pipeline(['datasets/winequality-red.csv'], 100)
```

```python
# Launch the graph in a session.
sess = tf.Session()
# Initializes global variables in the graph.
sess.run(tf.global_variables_initializer())
```

```python
# Start populating the filename queue.
coord = tf.train.Coordinator()
threads = tf.train.start_queue_runners(sess=sess, coord=coord)

costs = []
for step in range(2001):
    # Evaluate X_batch & Y_batch
    X_batch, Y_batch = sess.run([train_x_batch, train_y_batch])

    # Train the regression model and evaluate the cost
    _, val_cost = sess.run([train, cost], feed_dict={X: X_batch, Y: Y_batch})

    # Append it to the list of costs
    costs.append(val_cost)

coord.request_stop()
coord.join(threads)
```

```python
%matplotlib inline
from matplotlib import pyplot as plt

plt.plot(costs)
plt.xlabel('Steps')
plt.ylabel('MSE cost')
plt.show()
```

```python
sess.close()
```

# Useful Resources

- [aicodes/tf-bestpractice: Tensorflow Best Practices](https://github.com/aicodes/tf-bestpractice)
- [aymericdamien/TensorFlow-Examples: TensorFlow Tutorial and Examples for Beginners with Latest APIs](https://github.com/aymericdamien/TensorFlow-Examples)
- [nlintz/TensorFlow-Tutorials: Simple tutorials using Google's TensorFlow Framework](https://github.com/nlintz/TensorFlow-Tutorials)
- [sjchoi86/Tensorflow-101: TensorFlow Tutorials](https://github.com/sjchoi86/Tensorflow-101): Useful tutorial for TensorFlow by Sungjoon Choi in SNU