#### _Speech Processing Labs 2020: Signals: Module 1_

```python
## Run this first!
%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np

import cmath
from math import floor
from matplotlib.animation import FuncAnimation
from IPython.display import HTML
plt.style.use('ggplot')

## some helper functions:
from dspMisc import *
```

# 4 The Discrete Fourier Transform

### Learning Outcomes
* Understand the DFT equation: what input does it take and what outputs does it produce.

### Need to know
* Topic Videos: Series Expansion, Fourier Analysis, Frequency Domain
* Vector operations: dot product
* [Digital Signals: Sampling sinusoids](./sp-m1-3-sampling-sinusoids.ipynb)

     Optional-ish:  This notebook goes through the DFT equation in quite a lot of detail - more than is strictly necessary for this course.  It's perfectly fine to just focus on the visualizations or to skip it for now.

That said, you might like to run the code anyway and look at the visualization of the different phasors (i.e. basis functions) that correspond to the different DFT outputs in the DFT work through section called [Generate Phasors](#genphasors), and the corresponding magnitude and phase response graphs for that example

Do you remember learning a bunch of math (trig, algebra, calculus) in high-school and thinking you'll never have to use this? While you might not use it directly, a lot of the technology you use everyday depends on it.  Pretty much all current speech technology depends on the Fourier Transform in some way! It has been (and continues to be) used to solve problems in many different fields from from image processing to quantum mechanics

In the previous notebook, you looked at how you can add 'pure' sinusoids together to form complicated waveforms. The Fourier Transform gives us a way of doing the opposite: taking sequence of measurements over time (like a speech wave) and decomposing it into sinusoids of different frequencies, magnitudes, and phases.

Since we are interested in digital signals (i.e. measuresment over discrete time steps), we'll need to use the **Discrete Fourier Transform**.  This makes it possible to actually do these calculations on a computer, but it does also lead to some unexpected side-effects.

Equation alert: If you're viewing this on github, please note that the equation rendering is not always perfect. You should view the notebooks through a jupyter notebook server for an accurate view.

## 4.1 The DFT equation

You can think of the DFT as a machine that takes $N$ amplitude samples over time and returns the frequency spectrum of the input signal (just like you when you view a spectral slice in praat).

You can treat this as a black box. But, now that we've gone through complex numbers, phasors, etc we can look at how the machine works and, more importantly, what it's limitations are.

* Input: $x[n]$, for $n=1..N$ (i.e. a time series of $N$ samples)
* Output: N complex numbers $\mapsto$ magnitude and phases of specific phasors:

$$
\begin{align}
DFT[k] &= \sum_{n=0}^{N-1} x[n] e^{-j \frac{2\pi n}{N} k} \\
DFT[k] &= \sum_{n=0}^{N-1} x[n]\big[\cos(\frac{2\pi n}{N} k) - j \sin(\frac{2\pi n}{N} k) \big]
\end{align}
$$

You'll usually see this written as $X[k]$ in signal processing textbooks, but we'll just say $DFT[k]$ to be a bit more transparent (and reduce the probability of typos!)

Notice the $-j$ in the phasor! This means as $n$ increases the 'clockhand' of the phasor is rotating clockwise!

## 4.2 The DFT Step-by-Step

But what does the DFT do for us? Very broadly, the DFT takes a series of $N$ values in time as input (e.g. waveform amplitudes over time for our sound waves) and outputs the **correlations** between the input and $N$ pure cosine waves with specific frequencies.  That is, it tells you how you would weight (and shift) these cosine waves so that you can add them all up and recover the original input.

_Let's break it down!_

### The DFT for $k=0$
Let's first look at the DFT for $k=0$:

$$
\begin{align}
DFT[0] = \sum_{n=0}^{N-1} x[n] e^{-j \frac{2\pi n}{N} 0}
\end{align}
$$

This is usually referred to as the **bias** term.  It tells us whether the input function is shifted up or down the amplitude axis (in a time v amplitude plot of the waveform).

### Exercise:

* Q: Why doesn't DFT[0] tell us anything about the frequency of the input?

```python

```

### The DFT for $k=1$:

Now we can start to look at the correlation between the input sequence and some phasors of different frequencies.  For $k=1$, we have:

$$ DFT[1] = \sum_{n=0}^{N-1} x[n] e^{-j \frac{2 \pi n}{N}}$$

Each of the $e^{j 2\pi n/N}$ terms in the sum is a step of $2\pi n/N$ radians around the circle.  That is, our $k=1$ phasor $e^{-j \frac{2 \pi}{N}n}$ makes completes 1 full circle every $N$ time steps.

Let's plot the the $k=1$ phasor for $N=16$:

```python
## plot complex numbers in the DFT[1] equation
N=16

## make an array of N steps: 0,...,N
nsteps = np.array(range(N))
tsteps = 2*np.pi*nsteps/N
Tmin = np.min(tsteps)
Tmax = np.max(tsteps)

## Corresponding complex numbers z[n] = e^{j 2 pi n/N}
zn_k1 = np.exp(-1j*2*np.pi*nsteps/N)

```

```python
## Plot the phasor corresponding to DFT[k=1]
X_k1, Y_k1, fig, phasor, iproj, rproj = plot_phasor_static(zn_k1, tsteps, plot_real=True, plot_imag=True)
```

```python
## Animate the phasor!
phasor1_vid = get_phasor_animation(X_k1, Y_k1, tsteps, phasor, iproj, rproj, fig)
phasor1_vid
```

What this animation shows is that for $k=1$, the $e^{-j\varphi}$ terms in the DFT represent $N$ evenly space samples around the unit circle aka one period of a sinusoid. So, The term $x[n] e^{-j2\pi n/N}$ represents multiplying the $n$th step around the unit circle with the $n$th input value.

### Exercise (optional)

We saw from Euler's formula previously that projecting the phasor (left fig) onto the imaginary (vertical) axis, would give us a sine wave, i.e. $\sin(t)$.  This isn't quite what we see above.  Instead the sinusoid on the right, looks like the mirror image of $\sin(t)$, i.e. $-\sin(t)$.  Why is this?

### Notes

```python

```

So, we can interpret the DFT as taking the dot product between the input and a cosine (and sine) wave of specific frequency.  That is, it tells us if the input sequence repeats itself  with a specific frequency.

### The dot product with the input $x[n]$
It's useful at this point to break the DFT equation into two parts using Euler's rule. This esssentially gives us two dot products (i.e. elementwise multiplication and summation). This allows us to calculate the real and imaginary components of the DFT output separately as a dot product with a cosine (the real part) and a dot product with a sine (the imaginary part):

$$
\begin{align}
DFT[1] &= \sum_{n=0}^{N-1} x[n] \big(\cos(2\pi n/N) -j\sin(2\pi n/N)\big)\\
&= \sum_{n=0}^{N-1} x[n] \cos(2\pi n/N) - j \sum_{n=0}^{N-1} x[n] \sin(2\pi n/N)
\end{align}
$$

### Example:
Let's look at what's happening in these sum terms using a simple sampled square wave with the same period as the $k=1$ phasor for input.  The following visualizes the second sum (i.e. the dot product with $\sin(2\pi n/N)$).

```python

## Make the background and plot the k=1 phasor
X_k1, Y_k1, fig, phasor, iproj, rproj = plot_phasor_static(zn_k1, tsteps, plot_real=True, plot_imag=True)

## Now, let's say our input x looks like this square wave:
xn = np.array([-1]*8 + [1]*8)

## Exercise: Try some other inputs later
#xn = np.array([-1, -1,-1,-1, 1, 1,1,1]*2)
#xn = np.array([-1]*4 + [1]*8 + [-1]*4)
#xn = np.array([1] + [-1, -1,-1, 1, 1,1,1]*2 + [-1])
#xn = zn_imag_k1
#xn = np.concatenate([Ys[np.arange(0, 16, 2)], Ys[np.arange(0, 16, 2)]])
#xn = np.array([1] + [-1, -1,-1, 1, 1,1,1]*2 + [-1])

print("x[n]:", xn)

## Plot the input x[n]:

# with respect to the imaginary component (top right)
iproj.scatter(tsteps, xn, color='C1')

# and the real component (bottom left)
rproj.scatter(xn, tsteps, color='C1')

## Do the elementwise multiplication of input (xn) and the k=1 phasor (zn_k1)
xn_times_zn = xn * zn_k1

## Get the real and imaginary parts
real_xn_zn = np.real(xn_times_zn)
imag_xn_zn = np.imag(xn_times_zn)

## Plot the imaginary parts = x[n]*sin(theta) (top right) in yellow
iproj.plot(tsteps, imag_xn_zn, color='C4')
iproj.scatter(tsteps, imag_xn_zn, color='C4', s=100, alpha=0.5)

## Plot the real parts = x[n]*cos(theta) (bottom left) in yello
rproj.plot(real_xn_zn, tsteps,  color='C4')
rproj.scatter(real_xn_zn, tsteps, color='C4', s=100, alpha=0.5)
```

In the plot above, you can see that:
* The grey dots show our samples from the $k=1$ phasor (top left), and projected imaginary component, i.e. the sine in the DFT sum (top right), and the projected real component, i.e. the cosine (bottom left)
* The blue dots show our input sequence (top right and bottom left)
* the yellow dots show the element wise multiplication of the phasor values and the input, projected on the imaginary and real axes.

Let's just look at the imaginary (sine) part of the $x[n] \cdot z_1[n]$ multiplication (i.e., `xn_times_zn`):
When we multiply the values in both sequences together, we can see that (1)  the values in the input and phasor don't exactly match, but (2) they are always the same sign. That is, the input and the sine component of the $k=1$ phasor are correlated to some extent. In this case, this means that the multiplication terms (in yellow) in this case are all positive. So, adding them all up will give us a positive term for the imaginary component of the DFT.

We can also see that, even though we basically chose this example to match the sine component, we do also get a positive correlation with the real (cos) component.

The following cell shows that you get the same result from doing the dot products with the real (cos) and imaginary (sin) parts of the phasor separately, or doing the dot product with the phasor and then projecting the real and imaginary parts.

```python
## The dot product: sum all the elementwise multiplications
dft_1_real = np.sum(real_xn_zn)
dft_1_imag = np.sum(imag_xn_zn)

print("* projection and then two separate products: DFT[k] = %f + i%f" %(dft_1_real, dft_1_imag))

## check these are the same!
dft_1 = np.sum(xn*zn_k1)
print("* one dot product and then projection: DFT[k] = %f + i%f" %(dft_1.real, dft_1.imag))
```

### Exercise: Change the input
What happens when we change the input `xn` so that:

* it has a different period
* exactly matches the sine component of the k=1 phasor
* is out of phase with the sine component of the k=1 phasor
* has a different magnitude
* something else ...

There are some commented out options that you can try in the cell above

### Notes

```python

```

### Example: Phase shifted input

Let's see what happens when our input matches the imaginary component of $k=1$ phasor but has it's phase shifted a bit.  Remember, we can shift the start point of our phasor by multiply the whole phasor sequence a complex number.

$$ \sin(\theta + \phi) = Imag(e^{j\phi}e^{j\theta}) = Imag(e^{j\theta+\phi})$$

```python
## Let's generate our DFT[1] phasor for an input of N=16 points:
N=16

## make an array of N steps: 0,...,N
nsteps = np.array(range(N))
tsteps = 2*np.pi*nsteps/N

## The k=1 phasor:
zn_k1 = np.exp(-1j*tsteps)
```

Now let's create an input that's the same as the sine component of the DFT[1] phasor, but shifted by $\pi/3$ radians.

```python
## Plot the DFT[1] phasor
X_k1, Y_k1, fig, phasor, iproj, rproj = plot_phasor_static(zn_k1, tsteps, plot_real=False, plot_imag=True)

## Now, for the input let's use a phase shifted version of the sine component of our phasor, zn (defined above)
# Remember that multiplying a complex number by e^{j theta} rotates it by theta degrees
# (anticlockwise if theta is positive)
# So to shift the whole sequence we just multiply everything in the sequence by the same complex number

## For this example, let's shift our -sin(2 pi n/N) by pi/3
zn_shift = np.exp(1j*np.pi/3) * zn_k1

## And take as input just the imaginary component of this shifted sine wave
xn = np.imag(zn_shift)

## Plot the phase shifted sine wave it in blue!
iproj.scatter(2*np.pi*nsteps/N, xn, color='C1')
iproj.plot(2*np.pi*nsteps/N, xn, color='C1')

```

In the plot above, you should see the input (in blue) is the same as the sine component of the $k=1$ phasor but *phase shifted*. By multiplying everything by $e^{j\pi/3}$ _delays_ the our $-sin(\theta)$ function by $\pi/3$ seconds (in this example we're assuming that it takes $\theta$ seconds to travel $\theta$ degrees around the phasor).

Now let's see how this effects the DFT output:

```python
 ## Elementwise multiplication of the input and the k=1 phasor
xn_times_zn = xn * zn_k1

## Add it all up to get the dot product
dft_1 = np.sum(xn_times_zn)

print("DFT[1] = %f + i%f" %(dft_1.real, dft_1.imag))
print("in polar coordinates: magnitude=%f, phase=%f" %cmath.polar(dft_1))

## Plot the sequence of multiplications (yellow)
iproj.plot(2*np.pi*nsteps/N, np.imag(xn_times_zn), color='C4')
iproj.scatter(2*np.pi*nsteps/N, np.imag(xn_times_zn), color='C4')
fig
```

The result of the DFT is 6.93 + j4, which in polar coordinates has a magnitude=8 and phase angle=0.52

* Non-zero magnitude means that the input has a frequency component that matches the frequency of the $k=1$ phasor
* Non-zero phase means that the input is like the $k=1$ phasor in frequency, but starting from a different starting angle.

### How does the detected phase angle relate to the phase we used to shift the input?

The magnitude of the DFT output is relatively straightforward to interpret.  The bigger the magnitude, the bigger the amplitude of this frequency component in the input.

When we convert the $DFT[1]$ output to polar coordinates we can, as before, interpret $DFT[1]$ as a scaling (magnitude) and rotation (phase) factor.  We can think of the phase as signalling a bigger amplitude of this frequency in the the input.   We can think of the phase angle as rotating the starting position of the DFT[1] phasor 'clock hand' by that angle. When we convert this to time vs amplitude, this essential means starting from a future point in the sinusoid for positive phase (or a past point for negative phase) relative to the direction the phasor is rotating.

###  DFT phase is relative to cosine!

An important thing to note is that the phase output of the DFT is relative to the **cosine** function with the same frequency as the phasor. This is why the phase value we calculated isn't actually the same as the angle we used to shift the input sequence ($\pi/3=1.047$ radians), since that input sequence was actually based on a sine function.

### Exercise (optional)

Show that our input above $-\sin(t-\pi/3)$ is the same as a cosine wave with the phase shift derived above $\pi/6$.
You'll need to use these trig identities:
$$ \cos(\alpha - \pi/2) = \sin(\alpha) $$
$$\sin(t+\pi) = -\sin(t)$$

**Note** We definitely won't be asking you do these sorts of calculations for assessment, but going through it will help consolidate your intutions about sinusoids and the relationship between cosine and sine waves.

### Notes

```python

```

Even if you don't do the exercise above, we can see that shifting a cosine function by the DFT output phase gives us the same sinusoid as our input.

That is:
$$ \cos(t + \pi/6) = -\sin(t-\pi/3)$$

just by plotting them:

```python
## Plot the input from above and the equivalent cos and sin function based on DFT phase output.
_, _, fig, phasor, iproj, rproj = plot_phasor_static(zn_k1, tsteps, plot_real=False, plot_imag=True)

#fig, phasor, iproj, rproj = create_phasor_iproj_bkg(Tmin, Tmax, ymax=1.5)
#phasor.scatter(zn_real_k1, zn_imag_k1)

## Our input (C1=blue)
tsteps = 2*np.pi*nsteps/N
iproj.scatter(tsteps, xn, color='C1', s=300)
iproj.plot(tsteps, xn, color='C1')

## cos(t + pi/6) (C0=red)
iproj.scatter(tsteps, np.cos(tsteps+np.pi/6), color='C0', s=200)
iproj.plot(tsteps, np.cos(tsteps+np.pi/6), color='C0')

## -sin(t-pi/3) (C5=green)
iproj.scatter(tsteps, -np.sin(tsteps-np.pi/3), color='C5', s=80)
iproj.plot(tsteps, -np.sin(tsteps-np.pi/3), color='C5')
```

In the plot above, you should see:
* the DFT[1] phasor in grey, i.e. $-\sin(t)$
* Our phase shifted input in blue
* The phase shift determined by DFT[1] applied as a cosine wave: $\cos(t + \pi/6)$ in red
* Our phase shifted input generated directly using the `np.sin` function: $-\sin(t-\pi/3)$ in green

You should see is that that the last three functions are all the same! We can always write a sine wave as a cosine wave with a phase shift.

### The DFT for k = 2 and beyond

We can think of DFT[2] as representing the contribution of a phasor that spins around the unit circle
twice as fast as the $k=1$ DFT phasor:

* For $k=2$, Each $e^{i \frac{2\pi n}{N}k}$ is a step of $\frac{2\pi}{N}\times 2$ radians around the unit circle
    * i.e. we only sample every second point compared to the $k=1$ case.

* This means it only takes half the time to make a full cycle. So, this phasor represents a sinusoid with twice the frequency of the one for $k=1$

### Exercise:
Plot sinusoids for different values of `k` using the code below
* What happens when $k=N/2$?
* What happens when $k > N/2$?
* What if you increase $N$?
* How many actual frequencies can the DFT actually tell us about?
* How does this relate to the aliasing problem we saw in the previously?

```python
## Plot the phasor and sinusoid for different values of k

## Number of samples
N=16

## DFT output we're looking at: a sinusoid with k times the freq of the k=1 phasor
k=15

## indices of points in sample
nsteps = np.array(range(N))

## times of samples
tsteps = 2*np.pi*nsteps/N

## N sample values of the kth DFT phasor
zn_k = np.exp(k*-1j*tsteps)

X_k, Y_k, fig, phasor, iproj, rproj = plot_phasor_static(zn_k, tsteps, plot_real=False, plot_imag=True)

```

```python
kphasor_anim= get_phasor_animation(X_k, Y_k, tsteps, phasor, iproj, rproj, fig)
kphasor_anim
```

### Notes

```python

```

## 4.3 The full DFT

Now that we've seen what happens when we calculate the individual DFT components, lets do the whole thing!

We need

1. **Input:** a sequence of $N$ samples
1. generate $N$ phasors, with $N$ sampled points
1. generate $N$ dot products between the input and the phasors
1. **Output:** $N$ complex numbers representing magnitude and phase

The magnitudes and phases give us the decomposition of the input into pure cosine waves

### Set the input $x[n]$

Let's use the same input as above $x[n] = -\sin(2\pi n/N -\pi/3)$

```python
## 1) Input: -sin(t-pi/3) as above
N=16
nsteps = np.array(range(N))
theta_step = 2*np.pi/N
theta_steps = theta_step * nsteps

## set the phase shift
phase_in = np.pi/3

## set the input as -sin with phase shift
x = -np.sin(theta_steps-phase_in)

## Plot the input x
fig, tdom = plt.subplots(figsize=(16, 4))
tdom.scatter(tsteps, x, color='magenta')
tdom.plot(tsteps, x, color='magenta')

```

You should see the input (16 points), `x`, plotted in magenta

### Generate phasors

Let's use some functions to generate all the DFT phasors in one go:

```python
## 2) Generate the phasors
# Given a sequence length N, return dictionary representing N DFT outputs
# the kth element of the return dictionary contains the complex numbers sampled around the unit circle after
# N steps, their real and imaginary parts, and the angles of those complex numbers (magnitude is always 1)
def get_dft_phasors(N, centered=False):

    ## N input steps, N phasors
    nsteps = np.array(range(N))

    ## DFT works for negatively indexed samples, i.e. x[-n], but we don't need this right now!
    if centered:
        nsteps = nsteps - floor(N/2)
        print(nsteps)

    # We know the smallest step around the unit circle we can take is 2pi/N
    theta_step = 2*np.pi/N
    # And we're going to take N steps
    theta_steps = theta_step * nsteps

    ## Generate N phasors
    phasors = {}
    for k in range(N):
        ## samples around the unit circle
        zs = np.exp(k*-1j*theta_steps)
        real = np.real(zs)
        imag = np.imag(zs)

        ## Since we're here, let's return all these things for convenience
        # zs: the phasor samples, real: the real component, imag: the imaginary component
        # theta_steps: the angles for each phasor sample, theta_step: the smallest angle step size

        phasors[k] = {'zs':zs, 'real':real, 'imag':imag, 'thetas':theta_steps, 'theta_step':theta_step}

    return phasors

```

```python
## get the list of phasors
phasors = get_dft_phasors(N)
```

```python
## plot the different phasors and sine (imaginary) components
## You should be able to see the frequency relationship between each DFT component
for k in range(N):
    X_k, Y_k, fig, phasor, iproj, rproj = plot_phasor_static(phasors[k]['zs'], tsteps, plot_real=False, plot_imag=True, color='C'+str(k))

```

You should see in the plots above that phasors $k > N/2$ repeat the frequencies we see for $k < N/2$. For example,
the $k=1$ phasor is equivalent in frequency to the $k=15$ phasor!

### Calculate dot products (i.e.  input and phasor correlations)

```python
## 3) Get the dot products for each for each phasor and the input

def get_dft_outputs(x, phasors):
    DFT = []

    ## Go through our list of N phasors
    for k, phasor in phasors.items():

        ## Do the dot product between the input and each phasor sequence
        DFT.append(np.sum(x * phasor['zs']))

    return DFT

```

```python
## do the DFT
dft = get_dft_outputs(x, phasors)
```

### Get the output  magnitudes and phases

Now we convert the results of the dot products we just calculated into polar form to get magnitudes and phases. We can then plot them!

```python
## convert to polar coordinates to get magnitude and phase
dft_polar = [cmath.polar(z) for z in dft]
mags = [m for m, _ in dft_polar]
phases = [ph if mag > 0.00001 else 0 for mag,ph   in dft_polar]

## we need to put a condition into the phase calculation because phase is calculated by
## a ratio of the imaginary component and the real component. Both of these values might be very, very small.
## This makes it susceptible to floating point errors.
## Then, the ratio of two very small things can actually end up to be quite (spuriously) big!

```

```python
## Plot the magnitudes
#print(mags)
fig, freqdom = plt.subplots(figsize=(16, 4))
freqdom.set(xlim=(-1, N), ylim=(-1, 10))
freqdom.scatter(range(N), mags)
```

```python
## plot the phase angle for each DFT component
#print(phases)
fig, fdom = plt.subplots(figsize=(16, 4))
fdom.set(xlim=(-1, N), ylim=(-2, 2))
fdom.scatter(range(N), phases)
```

### How do we interpret the magnitude and phase graphs?

The top plot shows the magnitude response of the DFT with respect to our input.  The bottom plot shows the phase respose.

The magnitude plot shows positive magnitudes for the 1st and the N-1th DFT components, and zero for all the rest.  This means that the input is a sinusoid with the same frequency as the 1st DFT component.

The positive phase for the first component indicates a phase shift.  We see the opposite phase shift for the 15th component because the phasor for that has the same frequency but rotates the opposite way.

So, this is what we expect since we generated the input as exactly the sine projection of the k=1 phasor, just shifted by $\pi/3$ radians!

### Plotting DFT outputs on the complex plane
So far, we've looked at the real and imaginary components of the DFT outputs separately.  But we can also think about their properties on the complex plane.

The following plots the DFT outputs for the example above on the complex plane.

```python
## Plot the actual DFT outputs as complex numbers on the complex plane
## For the example above, we can see that most of the outputs are at (0,0), i.e. not contribution
## but we get positive magnitudes for the 1st and 15th component, with the same
## magnitude by 'mirrored' phase angle:

ymax=(N/2) + 2
fig, ax = plt.subplots(figsize=(5, 5))
ax.set(xlim=(-10, 10), ylim=(-10, 10))
ax.plot([-ymax,ymax], [0,0], color='grey')
ax.plot([0,0],[-ymax,ymax], color='grey')
circle1 = plt.Circle((0, 0), 1, color='grey',fill=False)

ax.add_artist(circle1)

for k in range(N):
    zs = phasors[k]['zs']
    xz = x * zs
    dftk = sum(xz)
    #print(dftk, dftk.real, cmath.polar(dftk))
    ax.plot(dftk.real, dftk.imag, 'o')

    if dftk.real > 0.0001:
        print("k=%d, %f + j%f" % (k, dftk.real, dftk.imag))

```

You can see from this plot that all but two of the DFT outputs have zero magnitude.  The two that have non-zero magnitude are the 1st and 15th components.  You can see that they have the same magnitude, but mirror each other in phase (they are complex conjugates). This symmetry is something you see a lot of in digital signal processing.  We won't go into it in this class, but for one thing we can take advantage of to save some computations!

### Exercise (optional)

You can use the code below to visualize the dot products of the input with with cos (real) and sin (imaginary) parts of each DFT component by varying $k$:

* What's the relationship between the DFT phasors for $k=1$ and $k=N-1$ in terms of the dot product between the real (left plot) and imaginary (right plot) components.
* What's happen for components that don't have the same frequency as the input? e.g. $k=2$

```python
## 1) Input: -sin(t-pi/3) as above
nsteps = np.array(range(N))
theta_step = 2*np.pi/N
theta_steps = theta_step * nsteps

Tmin = np.min(theta_steps)
Tmax = np.max(theta_steps)

phase_in = np.pi/3
x = -np.sin(theta_steps-phase_in)

fig, tdom = plt.subplots(figsize=(16, 4))
tdom.scatter(tsteps, x, color='magenta')
tdom.plot(tsteps, x, color='magenta')

```

```python
## let's break it down into cos and sin components
k=15

## zcos is our sampling of the real part of the kth phasor
zcos = phasors[k]['real']
zsin = phasors[k]['imag']

## the elementwise multiplication between the input x and the real part of the phasor, zcos
xz_real = x * zcos
xz_imag = x * zsin

## This initializes our plots, makes them the appropriate size etc, but leave the plot empty
_, _, fig, phasor, iproj, rproj = plot_phasor_static(phasors[k]['zs'], tsteps, plot_phasor=False, plot_real=True, plot_imag=True, color='grey')

## plot the zero line: if the sequence we get after the elementwise multiplication is
## symmetric around zero, we know that the dot product will be zero since we're adding all the
## values together
#inusoid.plot([Tmin-1,Tmax+1], [0,0], color='grey')

## Plot the input in magenta
rproj.plot(tsteps, x, color='magenta')
rproj.scatter(tsteps, x, color='magenta')

## Plot the elementwise multiplication in orange
rproj.plot(tsteps, xz_real, color='orange')
rproj.scatter(tsteps, xz_real, color='orange')

## Plot the input in magenta
iproj.plot(tsteps, x, color='magenta')
iproj.scatter(tsteps, x, color='magenta')

## Plot the elementwise multiplication in orange
iproj.plot(tsteps, xz_imag, color='orange')
iproj.scatter(tsteps, xz_imag, color='orange')

## Add it all up to get the dot product
dftk_real, dftk_imag = sum(xz_real), sum(xz_imag)
print("dft[%d] = %f + j%f "  % (k, dftk_real, dftk_imag))
```

### Notes

```python

```

```python

```

```python

```

```python

```

```python

```


# Numpy array creation
## 创建均匀分布的值
### arange
 * arange([start,] stop[, step,], dtype=None)：生成一个线性增长的某一范围内的数组，和Python中的 range 类似，但是arange支持浮点参数
  - start，stop '[start, stop)';start 默认为0；
  - dtype默认从其他参数推断

```python
import numpy as np
print np.arange(1,10,2)
print np.arange(10.2, 0.8)
print np.arange(10.4)
x = xrange(10)
print x
list(x)
```

    [1 3 5 7 9]
    []
    [  0.   1.   2.   3.   4.   5.   6.   7.   8.   9.  10.]
    xrange(10)

    [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

x 是一个 generator, 因此不能直接打印

### linspace
 * linspace(start, stop, num=50, endpoint=True, retstep=False)：在闭区间或半开区间中生成等间隔的序列点，即均分区间；
  - start,stop '[start,stop]' 或者 '[start,stop)';
  - num 生成的点数，默认50；
  - endpoint，即[start,stop]为真，[start,stop）为假，就是带不带端点的意思；
  - retstep，为真的时候，会返回额外的间隔，即返回 (samples, step) 二元组；

```python
print np.linspace(1,10)
print np.linspace(1,10,7) #step = (10-1)/(7-1)
print np.linspace(1,10,7,False)
samples, step = np.linspace(1,10,7,True,True)
print samples, step
```

    [  1.           1.18367347   1.36734694   1.55102041   1.73469388
       1.91836735   2.10204082   2.28571429   2.46938776   2.65306122
       2.83673469   3.02040816   3.20408163   3.3877551    3.57142857
       3.75510204   3.93877551   4.12244898   4.30612245   4.48979592
       4.67346939   4.85714286   5.04081633   5.2244898    5.40816327
       5.59183673   5.7755102    5.95918367   6.14285714   6.32653061
       6.51020408   6.69387755   6.87755102   7.06122449   7.24489796
       7.42857143   7.6122449    7.79591837   7.97959184   8.16326531
       8.34693878   8.53061224   8.71428571   8.89795918   9.08163265
       9.26530612   9.44897959   9.63265306   9.81632653  10.        ]
    [  1.    2.5   4.    5.5   7.    8.5  10. ]
    [ 1.          2.28571429  3.57142857  4.85714286  6.14285714  7.42857143
      8.71428571]
    [  1.    2.5   4.    5.5   7.    8.5  10. ] 1.5

## 创建多维数组ndarray
比 Python list 更加高效的多维数组, 接受以 Python list 形式递归嵌套定义的多维数组；
### 0维数组
可以认为是一个标量scalar

```python
x = np.array(777)
print "x: %d" % x
print "type: %s" % type(x)
print "dimension: %d" % np.ndim(x)
print "shape: ", x.shape
```

    x: 777
    type: <type 'numpy.ndarray'>
    dimension: 0
    shape:  ()

### 1维数组

```python
V = np.array([1,2,3,4,5])
print "V: ", V
print "type: ", type(V)
print "dimension: ", np.ndim(V)
print "shape: ", V.shape
```

    V:  [1 2 3 4 5]
    type:  <type 'numpy.ndarray'>
    dimension:  1
    shape:  (5,)

### 多维数组

```python
A = np.array([[1,2,3],
              [4,5,6],
              [7,8,9]])
print A
print A.ndim
print A.shape

Q = np.array([
    [[1,2,3],[4,5,6],[7,8,9]],
    [[10,11,12],[13,14,15],[16,17,18]]
])
print Q
print Q.ndim
print Q.shape
```

    [[1 2 3]
     [4 5 6]
     [7 8 9]]
    2
    (3, 3)
    [[[ 1  2  3]
      [ 4  5  6]
      [ 7  8  9]]

     [[10 11 12]
      [13 14 15]
      [16 17 18]]]
    3
    (2, 3, 3)

### ndarray shape 和 axis
多维数组的 shape 就是一个各个维度长度组成的 tuple，每一个维度都有一个轴 axis

```python
x = np.array([ [67, 63, 87],
               [77, 69, 59],
               [85, 87, 99],
               [79, 72, 71],
               [63, 89, 93],
               [68, 92, 78]])
print(np.shape(x))
```

    (6, 3)

二维数组的轴，你可以理解为二维平面坐标轴，axis=0 表示当沿着y方向，axis=1 表示沿着x方向

三维数组的轴，可以理解为三维空间坐标轴，axis=0 表示当沿着y方向，axis=1 表示沿着x方向，axis=2 表示沿着z方向

**高维空间是无法映射到物理空间的，因此也不能画出图像来，但是还是一样，每一个维度对应一个轴，沿着某一个轴，剩下的维度构成一个超空间。**对于2维，沿着x轴，剩下的y维度构成一个直线向下平移；

>多维数组维度的理解，这里如何看出Q是三维且shape为(2,3,3)呢？

**递归定义，递归检查**
 - 先检查第一个中括号对[]里的元素，发现是两个list元素，那么第一维度的长度为2；
 - 进一步进入内部的list [[1,2,3],[4,5,6],[7,8,9]]，递归进行维度检查，发现内部有三个list元素，因此第二维度长度为3；
 - 再进行递归，进入第三层list [1,2,3],此时内部含有三个元素，因此第三维度长度为3；最后再次递归时，发现元素是scalar，停止；

因此是 3 维且 shape 为 (2,3,3)；axis 分别是 0,1,2

```python
def my_shape(l):
    shape = []
    shape.extend([len(l)])
    for e in l:
        if isinstance(e, list):
            shape.extend(my_shape(e))
            break
        else:
            continue
    return shape
l = [
    [[1,2,3],[4,5,6],[7,8,9]],
    [[10,11,12],[13,14,15],[16,17,18]]
]
my_shape(l)
```

    [2, 3, 3]

>维度和axis有什么用呢？

**整形，广播和聚合计算**
 - 我们先来看看聚合计算的例子，就容易知道axis的作用了！

```python
B = np.array([[1,2,3],
              [4,5,6],
              [7,8,9],
              [10,11,12]])
C = np.array([
    [[1,2,3],[4,5,6],[7,8,9]],
    [[10,11,12],[13,14,15],[16,17,18]]
])
print B
print B.shape
print np.sum(B, axis=0)
print np.sum(B, axis=1)
print '-------------------------'
print C
print C.shape
print "axis=0\n", np.sum(C, axis=0)
print "axis=1\n", np.sum(C, axis=1)
print "axis=2\n", np.sum(C, axis=2)

D = np.array([C,C+1,C+2])
print '-------------------------'
print repr(D)
print D.shape
print "axis=0\n", repr(np.sum(D, axis=0))
print "axis=1\n", repr(np.sum(D, axis=1))
print "axis=2\n", repr(np.sum(D, axis=2))
print "axis=3\n", repr(np.sum(D, axis=3))
```

    [[ 1  2  3]
     [ 4  5  6]
     [ 7  8  9]
     [10 11 12]]
    (4, 3)
    [22 26 30]
    [ 6 15 24 33]
    -------------------------
    [[[ 1  2  3]
      [ 4  5  6]
      [ 7  8  9]]

     [[10 11 12]
      [13 14 15]
      [16 17 18]]]
    (2, 3, 3)
    axis=0
    [[11 13 15]
     [17 19 21]
     [23 25 27]]
    axis=1
    [[12 15 18]
     [39 42 45]]
    axis=2
    [[ 6 15 24]
     [33 42 51]]
    -------------------------
    array([[[[ 1,  2,  3],
             [ 4,  5,  6],
             [ 7,  8,  9]],

            [[10, 11, 12],
             [13, 14, 15],
             [16, 17, 18]]],

           [[[ 2,  3,  4],
             [ 5,  6,  7],
             [ 8,  9, 10]],

            [[11, 12, 13],
             [14, 15, 16],
             [17, 18, 19]]],

           [[[ 3,  4,  5],
             [ 6,  7,  8],
             [ 9, 10, 11]],

            [[12, 13, 14],
             [15, 16, 17],
             [18, 19, 20]]]])
    (3, 2, 3, 3)
    axis=0
    array([[[ 6,  9, 12],
            [15, 18, 21],
            [24, 27, 30]],

           [[33, 36, 39],
            [42, 45, 48],
            [51, 54, 57]]])
    axis=1
    array([[[11, 13, 15],
            [17, 19, 21],
            [23, 25, 27]],

           [[13, 15, 17],
            [19, 21, 23],
            [25, 27, 29]],

           [[15, 17, 19],
            [21, 23, 25],
            [27, 29, 31]]])
    axis=2
    array([[[12, 15, 18],
            [39, 42, 45]],

           [[15, 18, 21],
            [42, 45, 48]],

           [[18, 21, 24],
            [45, 48, 51]]])
    axis=3
    array([[[ 6, 15, 24],
            [33, 42, 51]],

           [[ 9, 18, 27],
            [36, 45, 54]],

           [[12, 21, 30],
            [39, 48, 57]]])

> 以上聚合计算，axis如何改变 np.sum 的最终结果呢？

 * 坐标空间法
   - 根据坐标平面来理解，利用空间想象，axis=0就是沿着y轴移动，即每一条垂直于x轴的线，累加起来，reduce成为一个sum点，得到 [22 26 30]的结果；反之，axis=1就是沿着x轴移动，即每一条垂直与y轴的线，累加起来，reduce成为一个sum点，得到 [ 6 15 24 33]；
   - 利用空间只能想象到三维，因此不太通用，并不是聚合计算中采用axis的本质；
 * **抽象维度法**（我也称之为**降维攻击法** ^<^）
   - 更加通用的方法，其实 axis 的本质，采用抽象的方法来理解，更容易计算出最终结果；以上面的C为例:
   - axis=0，即把该维度进行降维，根据sum将该维度reduce成一个点。具体操作就是定位第0维度，即递归的第一层次，该维度长度为2，剩下维度构成一个超空间，第0维度的每一个元素都对应着一个超空间，即第一个list元素[[1,2,3],[4,5,6],[7,8,9]]超空间，第二个list元素[[10,11,12],[13,14,15],[16,17,18]]超空间，沿着第0维度将这些超空间的元素对应位置累加起来，最后reduce掉第0维度，即结果成为shape（3,3）的一个超空间（在这里就是一个平面）[[11,13,15],[17,19,21],[23,25,27]];
   - axis=1,即把第1维度进行降维，首先定位，即第二次递归，进入第二层次的那个维度，这个维度有第0维度的第一个元素中的三个list元素[1,2,3],[4,5,6],[7,8,9]以及第0维度的第二个元素中的三个list元素[10,11,12],[13,14,15],[16,17,18]，此时对其进行reduce降维，将三个元素累加起来，得到shape为(2,3)的超空间[[12,15,18],[39,42,45]],即在不改变另外两个维度的前提下，缩减axis维度。
   - axis=2，即把递归进入第三层，该维度有三个scalar元素，即子空间[0][0]的[1,2,3],[0][1]的[4,5,6],[0][2]的[7,8,9],[1][0]的[10,11,12], [1][1]的[13,14,15], [1][2]的[16,17,18]，将三个scalar直接相加起来，[[6,15,24],[33,42,51]]
   - 抽象维度法的步骤就是，**定位维度所在递归列表的层次，不改变另外两个维度的前提下，将所有处于该维度的元素reduce降维；**

>D的axis=1的例子？

定位到第二个`[`,内部两个元素，因为该维度长度2，
\begin{equation}
 \begin{bmatrix}
   1  &  2  &  3 \\
   4  &  5  &  6 \\
   7  &  8  &  9
 \end{bmatrix},
 \begin{bmatrix}
   10 & 11 & 12 \\
   13 & 14 & 15 \\
   16 & 17 & 18
 \end{bmatrix}
\end{equation}

将该维度降维，即
\begin{equation}
 \begin{bmatrix}
   1 & 2 & 3 \\
   4 & 5 & 6 \\
   7 & 8 & 9
  \end{bmatrix}+
 \begin{bmatrix}
   10 & 11 & 12 \\
   13 & 14 & 15 \\
   16 & 17 & 18
  \end{bmatrix}=
 \begin{bmatrix}
   11 & 13 & 15 \\
   17 & 19 & 21 \\
   23 & 25 & 27
  \end{bmatrix}
\end{equation}

### 索引和切片（indexing & slicing）
索引和 Python list tuple 一样的用法，可以采用 list[i][j][k]的写法或者 list[i,j,k]，并且还支持 tuple 索引，即索引是一个数组，直接取出一批数据；

```python
python_list = [1,2,3,4,5,6]
python_slice = python_list[1:4]
print python_list
print python_slice
python_slice[0] = 777
print python_slice
print python_list

X = np.arange(28)
print(X)
X = X.reshape(4,7)
print X
print X[1,1]
array_index_x = X[(0,1,2),(4,5,6)]
print array_index_x
array_index_x[0] = 999
print array_index_x
print X
```

    [1, 2, 3, 4, 5, 6]
    [2, 3, 4]
    [777, 3, 4]
    [1, 2, 3, 4, 5, 6]
    [ 0  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
     25 26 27]
    [[ 0  1  2  3  4  5  6]
     [ 7  8  9 10 11 12 13]
     [14 15 16 17 18 19 20]
     [21 22 23 24 25 26 27]]
    8
    [ 4 12 20]
    [999  12  20]
    [[ 0  1  2  3  4  5  6]
     [ 7  8  9 10 11 12 13]
     [14 15 16 17 18 19 20]
     [21 22 23 24 25 26 27]]

切片和 Python list tuple 也类似，但是有一点不同：** Python list 和 tuple 中切片会创建一个新的对象，改变这个对象不影响原来的 list，但是 numpy 中的 ndarray，切片创建的是原来 ndarray 的视图，修改后会改变原始 ndarray；但是通过数组索引的形式获取的，就不是视图了，而是新的对象。**
 - X[start:stop:step], [start, stop)左闭右开

```python
X = np.arange(28)
print(X)
print X[::-1]
X = np.reshape(X,(4,7))
print X
view = X[::2, ::3]
print(view)
view[0][0] = 777
print X
view = X[1:-1, 1:-1]
print view
view = X[::-1, ::-1]
print view
```

    [ 0  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
     25 26 27]
    [27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10  9  8  7  6  5  4  3
      2  1  0]
    [[ 0  1  2  3  4  5  6]
     [ 7  8  9 10 11 12 13]
     [14 15 16 17 18 19 20]
     [21 22 23 24 25 26 27]]
    [[ 0  3  6]
     [14 17 20]]
    [[777   1   2   3   4   5   6]
     [  7   8   9  10  11  12  13]
     [ 14  15  16  17  18  19  20]
     [ 21  22  23  24  25  26  27]]
    [[ 8  9 10 11 12]
     [15 16 17 18 19]]
    [[ 27  26  25  24  23  22  21]
     [ 20  19  18  17  16  15  14]
     [ 13  12  11  10   9   8   7]
     [  6   5   4   3   2   1 777]]

### 内存共享
我们可以使用 `np.may_share_memory(A, B)` 来查看两个array是否共享了内存；如果A B 内存有共享的部分，就返回真；

```python
np.may_share_memory(X, view)
```

    True

`ndarray.data` 用来得到array的数组数据的指针地址，我们会发现**reshape 或者非索引形式的切片存在某种内存共享；**

```python
A = np.arange(12)
B = A.reshape(3,4)
print repr(A.data)
print repr(B.data)
print (A.data == B.data)
print np.may_share_memory(A,B)
A[7] = 777
print A
print B
#print (view.data) #AttributeError: cannot get single-segment buffer for discontiguous array
```

    <read-write buffer for 0x7fc0ac1c2120, size 96, offset 0 at 0x7fc0ac2bb830>
    <read-write buffer for 0x7fc0ac1c20d0, size 96, offset 0 at 0x7fc0ac2bb7b0>
    True
    True
    [  0   1   2   3   4   5   6 777   8   9  10  11]
    [[  0   1   2   3]
     [  4   5   6 777]
     [  8   9  10  11]]

## 创建特殊array & 数组copy
创建特殊的ndarray的方法，numpy 提供了很多[api](https://docs.scipy.org/doc/numpy/reference/routines.array-creation.html)来创建特殊的ndarray；ones & zeros & eye & identity

```python
zeros = np.zeros((4,4))
ones = np.ones((3,3))
identity = np.identity(4)
eye = np.eye(4,5, k=1)
print zeros
print ones
print identity
print eye
copy = ones.copy()
print copy
```

    [[ 0.  0.  0.  0.]
     [ 0.  0.  0.  0.]
     [ 0.  0.  0.  0.]
     [ 0.  0.  0.  0.]]
    [[ 1.  1.  1.]
     [ 1.  1.  1.]
     [ 1.  1.  1.]]
    [[ 1.  0.  0.  0.]
     [ 0.  1.  0.  0.]
     [ 0.  0.  1.  0.]
     [ 0.  0.  0.  1.]]
    [[ 0.  1.  0.  0.  0.]
     [ 0.  0.  1.  0.  0.]
     [ 0.  0.  0.  1.  0.]
     [ 0.  0.  0.  0.  1.]]
    [[ 1.  1.  1.]
     [ 1.  1.  1.]
     [ 1.  1.  1.]]

# Numpy data type
## dtype
>假如要使用numpy装入一个二维的下列表格，表中的每一列数据类型都不一样，如何做呢？

|Country | Population Density| Area  | Population|
|:-------|:-----------|:-------|:------|:-----------|
|Netherlands|	393	| 41526	| 16,928,800 |
|Belgium	|   337	| 30510	| 11,007,020 |
|United Kingdom	|256|243610	| 62,262,000 |
|Germany	|233	|357021	| 81,799,600 |
|Liechtenstein|	205	|160	| 32,842     |
|Italy	    |192	|301230	| 59,715,625 |
|Switzerland|	177	|41290	|7,301,994|
|Luxembourg	|173	|2586	|512,000|
|France	|111	|547030	|63,601,002|
|Austria	|97	|83858	|8,169,929|
|Greece	|81	|131940	|11,606,813|
|Ireland	|65	|70280	|4,581,269|
|Sweden	|20	|449964	|9,515,744|
|Finland|16	|338424	|5,410,233|
|Norway	|13	|385252	|5,033,675|

```python
dt = np.dtype([('density', np.int32)])
x = np.array([(393,), (337,), (256,)],
             dtype=dt)
print(x)
print("\nThe internal representation:")
print(repr(x))
print x['density']
```

    [(393,) (337,) (256,)]

    The internal representation:
    array([(393,), (337,), (256,)],
          dtype=[('density', '<i4')])
    [393 337 256]

我们可以采用 `np.dtype` 给某一列定义一个数据类型对象，这个对象可以通过复杂的组合，生成一个结构化的数据类型，就像结构体一样；对于 density 这一列，是 int32 数据类型。目前只有这一列，因此x 的构造写法是按列的写法；np.int32 的 内部表示其实是 `<i4`；`<>=`表示大端小端法和本地；

```python
dt = np.dtype('<d')
print(dt.name, dt.byteorder, dt.itemsize)
dt = np.dtype('>d')
print(dt.name, dt.byteorder, dt.itemsize)
dt = np.dtype('d')
print(dt.name, dt.byteorder, dt.itemsize)
```

    ('float64', '=', 8)
    ('float64', '>', 8)
    ('float64', '=', 8)

以上分别表示 8 字节的浮点类型，小端（即本地的），大端和本地的；
下面创建的就是一个结构体类型对象，每一个tuple对应一列；

```python
dt = np.dtype([('country', 'S20'), ('density', 'i4'), ('area', 'i4'), ('population', 'i4')])
x = np.array([('Netherlands', 393, 41526, 16928800),
('Belgium', 337, 30510, 11007020),
('United Kingdom', 256, 243610, 62262000),
('Germany', 233, 357021, 81799600),
('Liechtenstein', 205, 160, 32842),
('Italy', 192, 301230, 59715625),
('Switzerland', 177, 41290, 7301994),
('Luxembourg', 173, 2586, 512000),
('France', 111, 547030, 63601002),
('Austria', 97, 83858, 8169929),
('Greece', 81, 131940, 11606813),
('Ireland', 65, 70280, 4581269),
('Sweden', 20, 449964, 9515744),
('Finland', 16, 338424, 5410233),
('Norway', 13, 385252, 5033675)],
             dtype=dt)
print(x[:4])
```

    [('Netherlands', 393, 41526, 16928800) ('Belgium', 337, 30510, 11007020)
     ('United Kingdom', 256, 243610, 62262000)
     ('Germany', 233, 357021, 81799600)]

```python
print(x['density'])
print(x['country'])
print(x['area'][2:5])
```

    [393 337 256 233 205 192 177 173 111  97  81  65  20  16  13]
    ['Netherlands' 'Belgium' 'United Kingdom' 'Germany' 'Liechtenstein' 'Italy'
     'Switzerland' 'Luxembourg' 'France' 'Austria' 'Greece' 'Ireland' 'Sweden'
     'Finland' 'Norway']
    [243610 357021    160]

下面是一个嵌套定义的结构类型，每一行记录有两个大类型，time 和 temperature， time又是一个结构类型，分别由 h min sec 构成；

```python
time_type = np.dtype([('time', [('h', int), ('min', int), ('sec', int)]),('temperature', float)])
times = np.array( [((11, 42, 17), 20.8), ((13, 19, 3), 23.2) ], dtype=time_type)
print(times)
print(times['time'])
print(times['time']['h'])
print(times['temperature'])
```

    [((11, 42, 17), 20.8) ((13, 19, 3), 23.2)]
    [(11, 42, 17) (13, 19, 3)]
    [11 13]
    [ 20.8  23.2]

小结：
 - **类型可以嵌套定义组合**
 - **确定一个表的最终类型时，可以看每一列是什么数据类型，然后每一列是一个二元tuple，即(fieldname, datatype),然后组合np.dtype([(f1,d1),(f2,d2),...(fn,dn)]);**
 - **分解子datatype，如果dn不是基本datatype，则可以继续分解递归定义；**

# Numpy numerical operations
## Arithmetic Operations with two Arrays
对应位置元素 做某种操作运算

```python
A = np.array([ [11, 12, 13], [21, 22, 23], [31, 32, 33] ])
B = np.ones((3,3))
print("Adding to arrays: ")
print(A + B)
print("\nMultiplying two arrays: ")
print(A * (B + 1))
```

    Adding to arrays:
    [[ 12.  13.  14.]
     [ 22.  23.  24.]
     [ 32.  33.  34.]]

    Multiplying two arrays:
    [[ 22.  24.  26.]
     [ 42.  44.  46.]
     [ 62.  64.  66.]]

## Matrix Multiplication
矩阵运算，点积dot product

 - dot(a, b, out=None)
   - 对于 2-D arrays, 点积运算等价于矩阵运算；对于 1-D arrays, 点积等价于内积 inner product即向量积；对于 N 维数组，点积是 `a` 的最后一个axis 和 `b` 的 倒数第二个axis 的和积；**和积: 2-D 里就是'a'的每一个行向量和'b'的每一个列向量做 inner product**
   - 如果 'a' 的 最后一维长度 和 'b' 的倒数第二维长度不一样，抛出 ValueError； 例如， a.shape[-1] != b.shape[-2]

### 二维上的dot

```python
A = np.array([ [1, 2, 3],
               [3, 2, 1] ])
B = np.array([ [2, 3, 4, -2],
               [1, -1, 2, 3],
               [1, 2, 3, 0] ])

print(A.shape[-1] == B.shape[-2], A.shape[1])
print(np.dot(A, B))
```

    (True, 3)
    [[ 7  7 17  4]
     [ 9  9 19  0]]

### 三维上的dot

```python
X = np.array(
    [[[3, 1, 2],
      [4, 2, 2]],
     [[-1, 0, 1],
      [1, -1, -2]],

     [[3, 2, 2],
      [4, 4, 3]],
     [[2, 2, 1],
      [3, 1, 3]]])
Y = np.array(
    [[[2, 3, 1, 2, 1],
      [2, 2, 2, 0, 0],
      [3, 4, 0, 1, -1]],
     [[1, 4, 3, 2, 2],
      [4, 1, 1, 4, -3],
      [4, 1, 0, 3, 0]]])
R = np.dot(X, Y)
print("X.shape: ", X.shape, "   X.ndim: ", X.ndim)
print("Y.shape: ", Y.shape, "   Y.ndim: ", Y.ndim)
print("R.shape: ",     R.shape, "R.ndim: ", R.ndim)
print("\nThe result array R:\n")
print(R)
```

    ('X.shape: ', (4, 2, 3), '   X.ndim: ', 3)
    ('Y.shape: ', (2, 3, 5), '   Y.ndim: ', 3)
    ('R.shape: ', (4, 2, 2, 5), 'R.ndim: ', 4)

    The result array R:

    [[[[ 14  19   5   8   1]
       [ 15  15  10  16   3]]

      [[ 18  24   8  10   2]
       [ 20  20  14  22   2]]]

     [[[  1   1  -1  -1  -2]
       [  3  -3  -3   1  -2]]

      [[ -6  -7  -1   0   3]
       [-11   1   2  -8   5]]]

     [[[ 16  21   7   8   1]
       [ 19  16  11  20   0]]

      [[ 25  32  12  11   1]
       [ 32  23  16  33  -4]]]

     [[[ 11  14   6   5   1]
       [ 14  11   8  15  -2]]

      [[ 17  23   5   9   0]
       [ 19  16  10  19   3]]]]

按照 dot 对 n 维的定义，
 - 3 维应该是 X.shape[2] 和 Y.shape[1] 做 sum product；他们在各自维度上长度都是 3；该维度会被吞并成为一个scalar，最后结果为 shape 为 `(X.shape[0], X.shape[1], Y.shape[0], Y.shape[2])` 的array；
 - **如果是 n 维度的， 那么最后结果就是 `(X.shape[0], X.shape[1],..., X.shape[-2], Y.shape[0],..., Y.shape[-1])`**

以 R[0] 为例，它是 R 的 axis = 0 维度的第一个元素， 那么该元素列表应该是由剩余的维度做 sum product得到；

\begin{equation*}
 \begin{bmatrix}
   3 & 1 & 2
 \end{bmatrix}
 \begin{bmatrix}
  2 \\
  2 \\
  3
 \end{bmatrix}
 =
 14
\end{equation*}

```python
# 这里由于第一个维度 固定是第0个元素，因此 i = 0；不进行循环；
i = 0
for j in range(X.shape[1]):
    for k in range(Y.shape[0]):
        for m in range(Y.shape[2]):
            fmt = "    sum(X[{}, {}, :] * Y[{}, :, {}]) :  {}"
            arguments = (i, j, k, m, sum(X[i, j, :] * Y[k, :, m]))
            print(fmt.format(*arguments))

print R[0]
```

        sum(X[0, 0, :] * Y[0, :, 0]) :  14
        sum(X[0, 0, :] * Y[0, :, 1]) :  19
        sum(X[0, 0, :] * Y[0, :, 2]) :  5
        sum(X[0, 0, :] * Y[0, :, 3]) :  8
        sum(X[0, 0, :] * Y[0, :, 4]) :  1
        sum(X[0, 0, :] * Y[1, :, 0]) :  15
        sum(X[0, 0, :] * Y[1, :, 1]) :  15
        sum(X[0, 0, :] * Y[1, :, 2]) :  10
        sum(X[0, 0, :] * Y[1, :, 3]) :  16
        sum(X[0, 0, :] * Y[1, :, 4]) :  3
        sum(X[0, 1, :] * Y[0, :, 0]) :  18
        sum(X[0, 1, :] * Y[0, :, 1]) :  24
        sum(X[0, 1, :] * Y[0, :, 2]) :  8
        sum(X[0, 1, :] * Y[0, :, 3]) :  10
        sum(X[0, 1, :] * Y[0, :, 4]) :  2
        sum(X[0, 1, :] * Y[1, :, 0]) :  20
        sum(X[0, 1, :] * Y[1, :, 1]) :  20
        sum(X[0, 1, :] * Y[1, :, 2]) :  14
        sum(X[0, 1, :] * Y[1, :, 3]) :  22
        sum(X[0, 1, :] * Y[1, :, 4]) :  2
    [[[14 19  5  8  1]
      [15 15 10 16  3]]

     [[18 24  8 10  2]
      [20 20 14 22  2]]]

```python
R2 = np.zeros(R.shape, dtype=np.int)
for i in range(X.shape[0]):
    for j in range(X.shape[1]):
        for k in range(Y.shape[0]):
            for m in range(Y.shape[2]):
                R2[i, j, k, m] = np.dot(X[i, j, :],Y[k, :, m])
print( np.array_equal(R, R2) )
```

    True

以上就是三维 dot 的内部工作机制，**其实就是简单的循环累加，弄清楚我们需要做 sum product 的是哪一个维度，因为这个维度会被合并降维成一个scalar，剩下的维度做循环即可。**

## Comparison Operators
numpy中的 比较运算符返回的并不是 Python 中的 一个单一的 True 或 False, 而是一个array，里面每一个元素都是bool，是element-wise比较；因为 Python 中 True == 1 False == 0; 因此
 - 求两个array对应位置相等元素的位置索引
   - A[np.nonzero(A==B)]
 - 求两个array对应元素相等的个数，使用
   - len(np.flatnonzero(A==B))
 - 求相同元素百分比：
   - np.mean(A==B)

```python
A = np.array([ [11, 12, 13], [21, 22, 23], [31, 32, 33] ])
B = np.array([ [11, 102, 13], [201, 22, 203], [31, 32, 303] ])
common = A == B
print common
print A[np.nonzero(A==B)]
print len(np.flatnonzero(common))
print np.mean(A==B)
```

    [[ True False  True]
     [False  True False]
     [ True  True False]]
    [11 13 22 31 32]
    5
    0.555555555556

## Broadcasting
广播是numpy很重要的一个功能，这里介绍在numpy array shape 里面提到的与shape 和 axis有关的广播；

```python
B = np.array([[1,2,3]] * 3)
print B.shape
B
```

    (3, 3)

    array([[1, 2, 3],
           [1, 2, 3],
           [1, 2, 3]])

注意以上并不是广播，这是Python自带的循环重复元素的写法；我们先介绍一个升维的方法，np.newaxis。

```python
A = np.array([ [11, 12, 13], [21, 22, 23], [31, 32, 33], [41, 42, 43] ])
print repr(A)
print A.shape
A1 = A[np.newaxis,:,:]
A2 = A[:,np.newaxis,:]
A3 = A[:,:,np.newaxis]
print repr(A1)
print A1.shape
print repr(A2)
print A2.shape
print repr(A3)
print A3.shape

AA = A[np.newaxis, :, :, np.newaxis]
print repr(AA)
print AA.shape
```

    array([[11, 12, 13],
           [21, 22, 23],
           [31, 32, 33],
           [41, 42, 43]])
    (4, 3)
    array([[[11, 12, 13],
            [21, 22, 23],
            [31, 32, 33],
            [41, 42, 43]]])
    (1, 4, 3)
    array([[[11, 12, 13]],

           [[21, 22, 23]],

           [[31, 32, 33]],

           [[41, 42, 43]]])
    (4, 1, 3)
    array([[[11],
            [12],
            [13]],

           [[21],
            [22],
            [23]],

           [[31],
            [32],
            [33]],

           [[41],
            [42],
            [43]]])
    (4, 3, 1)
    array([[[[11],
             [12],
             [13]],

            [[21],
             [22],
             [23]],

            [[31],
             [32],
             [33]],

            [[41],
             [42],
             [43]]]])
    (1, 4, 3, 1)

可以看出，**np.newaxis 就是增加一个维度，并且增加的维度与其插入的位置有关，插入到第几个位置，就成为第几个维度，原始的维度值保持不变；**

>np.newaxis是如何改变原始array的呢？

**递归列表未完待续**
 - 目前有一种tricks，就是看插入位置的[]加在哪里。新轴在第2个维度，那么就给原本第1个维度的每一个元素加个[],新轴在第三维度，就在原本第2个维度的每一个元素加个[]

### 广播进行的方向

```python
B = np.array([1, 2, 3])
print B.shape
B1 = B[:, np.newaxis]
B1[0] = 7
print B
print repr(B1)
B1[0] = 1
A * B[:, np.newaxis]
```

```python
A = np.array([10, 20, 30])
B = np.array([1, 2, 3])
print repr(A[:, np.newaxis])
print repr(A[:, np.newaxis] * B)
```

以下是实现以上操作的另外两种实现，显式的使用了 concatenate 和 tile 方法升维；

```python
A = np.array([ [11, 12, 13], [21, 22, 23], [31, 32, 33] ])
B = np.array([1, 2, 3])
B = B[np.newaxis, :]
B = np.concatenate((B, B, B))
print("Multiplication: ")
print(A * B)
print("... and now addition again: ")
print(A + B)

A = np.array([ [11, 12, 13], [21, 22, 23], [31, 32, 33] ])
B = np.tile(np.array([1, 2, 3]), (3, 1))
print(B)
print("Multiplication: ")
print(A * B)
print("... and now addition again: ")
print(A + B)
```

### 三维空间broadcasting

```python
A = np.array([ [[3, 4, 7], [5, 0, -1] , [2, 1, 5]],
      [[1, 0, -1], [8, 2, 4], [5, 2, 1]],
      [[2, 1, 3], [1, 9, 4], [5, -2, 4]]])
B = np.array([ [[3, 4, 7], [1, 0, -1], [1, 2, 3]] ])
B * A

B = np.array([1, 2, 3])
B = B[np.newaxis, :]
print(B.shape)
B = np.concatenate((B, B, B)).transpose()
print(B.shape)
B = B[:, np.newaxis]
print(B.shape)
print(B)
print(A * B)
```

    (1, 3)
    (3, 3)
    (3, 1, 3)
    [[[1 1 1]]

     [[2 2 2]]

     [[3 3 3]]]
    [[[ 3  4  7]
      [ 5  0 -1]
      [ 2  1  5]]

     [[ 2  0 -2]
      [16  4  8]
      [10  4  2]]

     [[ 6  3  9]
      [ 3 27 12]
      [15 -6 12]]]

小结：
>Broadcasting provides a means of vectorizing array operations so that looping occurs in C instead of Python. It does this without making needless copies of data and usually leads to efficient algorithm implementations. There are, however, cases where broadcasting is a bad idea because it leads to inefficient use of memory that slows computation.

广播是numpy提供的向量化数组的操作，能够使得循环发生在 C 代码级别，而不是在 Python 级别，从而提高执行速度，但这种加速是有限制的，如果数组过于庞大，会进行大量的复制操作，反而会减慢计算速度，从而更加低效；[numpy broadcasting efficiency](http://scipy.github.io/old-wiki/pages/EricsBroadcastingDoc)

广播规则：
 1. **如果两个 array 的 rank 即 len(shape) 不同，那么给较低的array的 shape 在前面加上(prepend) 1 从而升维，直到低维array的shape和高维array的shape长度一样，即len(x.shape) == len(y.shape)；** 例如 x.shape = (1,2,3) y.shape = (3,) 则 y.shape = (1,1,3)
 2. **两个数组某个维度上兼容`compatible`,就是该维度长度要么相等，要么其中之一是1；**
 3. **只有所有维度`compatible`数组才能进行合法自动广播；**
 4. **广播之后，每一个数组，在每一个维度上就变成相同的了，且等于较大者；**
 5. **在某个维度上长度为1的数组会进行广播，就是沿着这个维度复制剩余维度的元素，使得该维度长度变长到两个数组中较大者；**

>numpy.transpose(a, axes=None)

axes : list of ints, optional By default, reverse the dimensions, otherwise permute the axes according to the values given.

第二个参数是坐标变换，即维度变换，如三维空间(x,y,z) 分别表示 (0,1,2) 维度，如果给出 (2,1,0) 则表示 (z,y,x) 即元素的取法变成了 先 z 后 y 最后 x。

# Numpy changing dimensions
## flatten
多维数组退化成一维
 - flatten()
 - ravel()

```python
A = np.array([[[ 0,  1],
               [ 2,  3],
               [ 4,  5],
               [ 6,  7]],
              [[ 8,  9],
               [10, 11],
               [12, 13],
               [14, 15]],
              [[16, 17],
               [18, 19],
               [20, 21],
               [22, 23]]])
Flattened_X = A.flatten()
print(Flattened_X)
print(A.flatten(order="C"))
print(A.flatten(order="F"))
print(A.flatten(order="A"))
```

    [ 0  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23]
    [ 0  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23]
    [ 0  8 16  2 10 18  4 12 20  6 14 22  1  9 17  3 11 19  5 13 21  7 15 23]
    [ 0  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23]

## reshape
整形，总数不变的情况下生成另一个shape的array
 - reshape(a, newshape, order='C')
   - order	'C', 'F', 'A', 代表展开顺序，C-style 首维最慢，尾维最快，F-style 按列展开，首维最快，尾维最慢
   - ‘C’ means to read / write the elements using C-like index order, with the last axis index changing fastest, back to the first axis index changing slowest.
   - ‘F’ means to read / write the elements using Fortran-like index order, with the first index changing fastest, and the last index changing slowest.
   - read/write就表示，读取原来的数组，按照order方式，然后生成新shape的数组，也按order方式写入；

```python
X = np.array(range(24))
Y = X.reshape((3,4,2))
print repr(Y)
Z = Y.reshape((4,3,2),order='F')
print repr(Z)
```

    array([[[ 0,  1],
            [ 2,  3],
            [ 4,  5],
            [ 6,  7]],

           [[ 8,  9],
            [10, 11],
            [12, 13],
            [14, 15]],

           [[16, 17],
            [18, 19],
            [20, 21],
            [22, 23]]])
    array([[[ 0,  1],
            [10, 11],
            [20, 21]],

           [[ 8,  9],
            [18, 19],
            [ 6,  7]],

           [[16, 17],
            [ 4,  5],
            [14, 15]],

           [[ 2,  3],
            [12, 13],
            [22, 23]]])

整形的真实过程其实像如下代码一样，最后验证我们的判断是正确的，即**先按照 order 读取原数组为一个一维的数组，然后再将一维数组中的元素按照 order 写入新shape的array中；**

```python
# F style read
temp = []
for i in range(Y.shape[2]):
    for j in range(Y.shape[1]):
        for k in range(Y.shape[0]):
            temp.append(Y[k, j, i])
print temp

# F style write
newshape = (4,3,2)
result = np.zeros(newshape, dtype=np.int32)
for i in range(newshape[2]):
    for j in range(newshape[1]):
        for k in range(newshape[0]):
            result[k, j, i] = temp[i * newshape[1] * newshape[0] + j * newshape[0] + k]
print result == Z

```

    [0, 8, 16, 2, 10, 18, 4, 12, 20, 6, 14, 22, 1, 9, 17, 3, 11, 19, 5, 13, 21, 7, 15, 23]
    [[[ True  True]
      [ True  True]
      [ True  True]]

     [[ True  True]
      [ True  True]
      [ True  True]]

     [[ True  True]
      [ True  True]
      [ True  True]]

     [[ True  True]
      [ True  True]
      [ True  True]]]

## Concatenating Arrays
numpy.concatenate((a1, a2, ...), axis=0)
 - axis : int, optional
   The axis along which the arrays will be joined. Default is 0.

沿着axis进行连接，使得该维度的元素成为一组；

### 对一维数组的连接

```python
A = np.array([7,77,777])
B = np.array([3,1,2])
C = np.concatenate((A,B))
print A,B,C
```

    [  7  77 777] [3 1 2] [  7  77 777   3   1   2]

### 对二维数组的连接

```python
x = np.array(range(24))
x = x.reshape((3,4,2))
y = np.array(range(100,124))
y = y.reshape((3,4,2))
z = np.concatenate((x,y))
print(z)
```

    [[[  0   1]
      [  2   3]
      [  4   5]
      [  6   7]]

     [[  8   9]
      [ 10  11]
      [ 12  13]
      [ 14  15]]

     [[ 16  17]
      [ 18  19]
      [ 20  21]
      [ 22  23]]

     [[100 101]
      [102 103]
      [104 105]
      [106 107]]

     [[108 109]
      [110 111]
      [112 113]
      [114 115]]

     [[116 117]
      [118 119]
      [120 121]
      [122 123]]]

```python
z = np.concatenate((x,y),axis = 1)
print(z)
```

    [[[  0   1]
      [  2   3]
      [  4   5]
      [  6   7]
      [100 101]
      [102 103]
      [104 105]
      [106 107]]

     [[  8   9]
      [ 10  11]
      [ 12  13]
      [ 14  15]
      [108 109]
      [110 111]
      [112 113]
      [114 115]]

     [[ 16  17]
      [ 18  19]
      [ 20  21]
      [ 22  23]
      [116 117]
      [118 119]
      [120 121]
      [122 123]]]

**万变不离其宗，与 axis 和 shape 有关的操作，都是递归确定在哪一个维度上进行，然后就是合并，降维，sum product等等;**这里就是合并；第二个维度即第二个括号里面的元素，同级的第二个括号里面的元素重新合并在一起了。

## Adding New Dimensions

```python
x = np.array([2,5,18,14,4])
y = x[:, np.newaxis]
print(y)
```

    [[ 2]
     [ 5]
     [18]
     [14]
     [ 4]]

## Vector Stacking

```python
A = np.array([3, 4, 5])
B = np.array([1,9,0])
print(np.row_stack((A, B)))
print(np.column_stack((A, B)))
np.shape(A)

```

    [[3 4 5]
     [1 9 0]]
    [[3 1]
     [4 9]
     [5 0]]

    (3,)

```python
A = np.array([[3, 4, 5],
              [1, 9, 0],
              [4, 6, 8]])
print repr(np.column_stack((A, A, A)))
print repr(np.column_stack((A[0], A[0], A[0])))
print repr(np.dstack((A, A, A)))
```

    array([[3, 4, 5, 3, 4, 5, 3, 4, 5],
           [1, 9, 0, 1, 9, 0, 1, 9, 0],
           [4, 6, 8, 4, 6, 8, 4, 6, 8]])
    array([[3, 3, 3],
           [4, 4, 4],
           [5, 5, 5]])
    array([[[3, 3, 3],
            [4, 4, 4],
            [5, 5, 5]],

           [[1, 1, 1],
            [9, 9, 9],
            [0, 0, 0]],

           [[4, 4, 4],
            [6, 6, 6],
            [8, 8, 8]]])

## Repeating Patterns, The "tile" Method

```python
import numpy as np
x = np.array([ [1, 2], [3, 4]])
print np.tile(x, (3,4))
x = np.array([[1, 2], [3, 4]])
print(np.tile(x, 2))
```

    [[1 2 1 2 1 2 1 2]
     [3 4 3 4 3 4 3 4]
     [1 2 1 2 1 2 1 2]
     [3 4 3 4 3 4 3 4]
     [1 2 1 2 1 2 1 2]
     [3 4 3 4 3 4 3 4]]
    [[1 2 1 2]
     [3 4 3 4]]


# Issue 5: Analytic Spherical Spheroidal Mixing

```python
# Setup the environment and import usefuls
%load_ext autoreload
%autoreload 2
%matplotlib inline
from positive import *
from numpy import *
from matplotlib.pyplot import *
from kerr import ysprod
```

    (positive)>> Applying custom matplotlib settings.

#### Briefly on the premise here

**Introduction**

It's well known that the spheroidal harmonics of spin weight $s$ are the eigenfunctions of Teukolsy's polar angular equation. This equation has the form

\begin{align}
    ( \hat{D}_0 + \lambda \hat{D}_1 ) |\alpha\rangle = E_\alpha |\alpha\rangle \; .
\end{align}

Here $\lambda = a \, (\omega^r_\alpha + i \omega^c_\alpha)$, $\alpha \leftrightarrow \{\ell,m,n\}$, $a=J/M$, and $\langle\theta|\alpha\rangle = {_{-2}}S_{\alpha}(\theta,\phi;\lambda)$.

**Premise**

We wish to find $|\alpha\rangle$ in the basis of spherical harmonics of spin weight -2:

\begin{align}
    |\alpha\rangle \; = \; \sum_{k} |k\rangle \langle k| \alpha \rangle \; = \; \sum_{k} \lambda^{|k|} \, c_{\alpha k} |k\rangle
\end{align}

where

\begin{align}
    k \leftrightarrow \{ \ell',m \}
\end{align}

In particular, with $\alpha \leftrightarrow \{\ell,m,n\}$, choosing

\begin{align}
    k = (\ell' - \ell) \ge (|s|-\ell) \text{ with, } k \in \mathcal{Z}
\end{align}

For example, if we are considering $\alpha=(3,2,0)$, then $k \in \{ -1,0,1,2,3,4,5,... \}$. Here $k=0$ will be the dominant term in tthe expansion of $|\alpha\rangle$.

## Tool Building

```python
def clebsch_gordon_coeff(l1,m1,l2,m2,L,M,output_number=True):
    '''
    Wrapper for sympy's clebsch gordon coefficient calculator.
    '''
    # Import usefuls
    from sympy.physics.quantum.cg import CG
    from sympy import S, N
    #
    cgf = CG( S(l1),S(m1), S(l2),S(m2), L, M ).doit()
    return complex(N(cgf) if output_number else cgf)
```

```python
l,m = 2,2
lp,mp = l+1,m
s = -2
theta = linspace(0,pi,2e3)
dth = theta[1]-theta[0]
Dlm = sYlm(s,l,m,theta,0)
Dlpmp = sYlm(s,lp,mp,theta,0)
```

    /Users/book/anaconda2/lib/python2.7/site-packages/ipykernel_launcher.py:4: DeprecationWarning: object of type <type 'float'> cannot be safely interpreted as an integer.
      after removing the cwd from sys.path.

```python
def prod(A,B,TH):
    INTGRND = A.conj()*B*sin(TH)
    RE_INTGRND = INTGRND.real
    IM_INTGRND = INTGRND.imag
    TH0,TH1 = lim(TH)
    return (2*pi)*(spline(TH,RE_INTGRND).integral(TH0,TH1) + 1j*spline(TH,IM_INTGRND).integral(TH0,TH1))
```

```python
print prod( Dlm,Dlm, theta ).real
```

    1.0

```python
def avgcos(S,L1,M1,L2,M2):
    '''
    Compute the average value of cos(theta) weighted by spherical harmonics with weight S, and indeces (L1,M1), and (L2,M2).
    See e.g. https://arxiv.org/pdf/1408.1860.pdf
    '''
    a = 1
    a = a * sqrt( (2*L1+1.0)/(2*L2+1.0) )
    a = a * clebsch_gordon_coeff( L1,M1, 1,0, L2,M2 )
    a = a * clebsch_gordon_coeff( L1,-S, 1,0, L2,-S )
    return a
```

```python
def avgcos2(S,L1,M1,L2,M2):
    '''
    Compute the average value of cos(theta)^2 weighted by spherical harmonics with weight S, and indeces (L1,M1), and (L2,M2).
    See e.g. https://arxiv.org/pdf/1408.1860.pdf
    '''
    a = (1.0/3) * ( 0 if L1!=L2 else 1 )
    b = (2.0/3) * sqrt( (2*L1+1.0)/(2*L2+1.0) )
    b = b * clebsch_gordon_coeff( L1,M1, 2,0, L2,M2 )
    b = b * clebsch_gordon_coeff( L1,-S, 2,0, L2,-S )
    return a+b
```

#### Sanity Check

```python
print avgcos( s,l,m,lp,mp )
print prod( Dlm,Dlpmp*cos(theta),theta ).real
```

    (0.2817180849095055+0j)
    0.281718084909

#### Sanity Check

```python
print avgcos2( s,l,m,lp,mp )
print prod( Dlm,Dlpmp*(cos(theta)**2),theta ).real
```

    (0.28171808490950556+0j)
    0.281718084909

```python
?leaver
```

```python
def kD1km1( AW, L, M, S=-2 ):

    #
    LP = L-1
    return complex(AW*avgcos2(S,L,M,LP,M) -2*S*avgcos(S,L,M,LP,M))

```

```python
def calphak( aw,l,lp,m,s=-2 ):

    #
    #kmin = abs(s)-l
    #if kmin<0: error('Cases with kmin<0 are not currrently handeled.')

    #
    kmax = 10
    e0 = l*(l+1)

    #
    c = 1.0

    #print range(l+1,lp+1)
    k = 0
    for lp_ in range(l+1,lp+1,1):

        #
        ek = lp_*(lp_+1)
        c_ = -1 * kD1km1(aw,lp_,m,s)/( ek-e0 )
        #print 'c_ = ',c_
        #print 'k = ',k
        k+=1
        c*=c_

    return c
```

```python
def local_ysprod(X,L,M,(LP,MP,N),TH=None):
    if TH==None:
        TH = linspace(0,pi,1e3)
    return prod( sYlm(-2,L,M,TH,0),slm(X,LP,MP,N,TH,0), TH ) / prod( sYlm(-2,LP,MP,TH,0),slm(X,LP,MP,N,TH,0), TH )
```

```python
def D0(THETA,STATE,M,S=-2):

    #
    SIN = sin(THETA)
    COS = cos(THETA)

    #
    D1TH = (spline( THETA, STATE ).derivative()(THETA)) * SIN
    D2TH = spline( THETA, D1TH ).derivative()(THETA)  / SIN

    #
    ans = D2TH - STATE*(M*M+S*S+2*M*S*COS)/(SIN*SIN)
    return ans

```

```python

th = linspace(0.01,pi*0.01,1e3)

Y = sYlm(-2,2,2,th,0)
state = Y*exp(lam*th)

# d0 = D0(th,state,0)
# plot( th,d0 )

# d1 = D0(th,state,1)
# plot( th,d1 )

# state = cos(th)
# d2 = D0(th,state,2)
# plot( th, d2/(-2*(2+1)) )

# state = cos(th)
d2 = D0(th,state,2)
# plot( th, d2.real )
# plot( th, state )

plot( th, D0(th,Y*exp(lam*th),2) )
plot( th, -Y*6*exp(lam*th) )

# plot( th, D0(th,Y,2) )
# plot( th, -Y*6 )

# plot( th, -2*cos(th) - (m**2 + s**2 + 2*m*s*cos(th))*(1.0/tan(th))*(1.0/sin(th)), ls='--',lw=2,alpha=0.5)
```

```python
def D1(AW,THETA,S):
    COS = cos( THETA )
    return AW*COS*COS + -2*S*COS
```

```python
M = 1.0
chi = 0.5
a = chi*M

l,m,n,s =ll,mm,0,-2
cw,sc = leaver( chi, l,m,n,s, Mf=M )
lam = a*cw

ll,mm = 3,2
llp,mmp = 2,mm

print prod( sYlm(s,ll,mm,theta,0), D1(lam,theta,s)*sYlm(s,llp,mm,theta,0), theta )
print kD1km1( lam, ll, mm )
```

    (1.20387481328-0.0129641937505j)
    (1.20387481329-0.0129641937505j)

```python
M = 1.0
chi = 0.5
a = chi*M

l,m,n,s =3,2,0,-2
cw,sc = leaver( chi, l,m,n,s, Mf=M )

ll = l
mm = m
llp = ll-1

cw *= -1
# cw = -cw.conj()
print cw
lam = a*cw

from sympy import N
print ll-llp

# print 'calphak = ',N( calphak( lam,llp,ll,mm,s=-2 ) )

e1 = ll*(ll+1)
e0 = llp*(llp+1)
ca0 = 1.0
ca1 = - ca0 * ( 2*s*prod( sYlm(-2,ll,mm,theta,0),sYlm(-2,llp,mm,theta,0)*cos(theta), theta ) - prod( sYlm(-2,ll,mm,theta,0),sYlm(-2,llp,mm,theta,0)*lam*(cos(theta)**2), theta ) )

print 'ca1*lambda = \t\t',ca1*lam**abs(ll-llp)
print 'calphak*lambda^k = \t',calphak( lam,llp,ll,mm,s=-2 )*lam**abs(ll-llp)

print ''
print 'ysprod = \t\t',local_ysprod( chi,ll,mm,(llp,mm,0) )
print 'ysprod = \t\t',local_ysprod( chi,llp,mm,(ll,mm,0) )
```

    (-0.5466633331145515+0.0920366454620499j)
    1
    ca1*lambda = 		(-0.28755927059089426+0.04476972563524529j)
    calphak*lambda^k = 	(0.04792654509856243-0.00746162093921984j)

    ysprod = 		(0.0437803646692-0.008100162268j)
    ysprod = 		(-0.0656836867829+0.00852585139852j)

    /Users/book/anaconda2/lib/python2.7/site-packages/ipykernel_launcher.py:3: DeprecationWarning: object of type <type 'float'> cannot be safely interpreted as an integer.
      This is separate from the ipykernel package so we can avoid doing imports until

```python

l = 4
lprange = arange(2,l+3,1)
jf = 0.7
m = 0
ys = array([ysprod(jf,lp,m,(l,m,0),500) for lp in lprange])

plot( lprange, abs(ys), '-o' )
yscale('log')
```

```python
from kerr.pttools import slm
?slm
```

```python
chi=0
J = 4
print prod( slm(chi,J,J,0,theta,0,norm=-1),slm(chi,J,J,0,theta,0,norm=-1), theta )
print prod( sYlm(-2,J,J,theta,0),sYlm(-2,J,J,theta,0), theta )
```

    (0.999999999999+0j)
    (1+0j)

```python

```


# Histogram of Oriented Gradients
We explore the method of 'histogram of oriented gradients' (HOG) as described in Navneet Dalal and Bill Triggs paper entitled Histograms of Oriented Gradients for Human Detection. We will be using the INRIA Dataset, specifically those images found in 160x96 folder to test our implementation. The code to process the data we will be written in `Python`. In addition we will show how to use the HOG found in `OpenCV`.

## Peope Detection
To be able to detect people in an image we follow the approach outline in Histograms of Oriented Gradients for Human Detection. The idea presented in the paper is to capture the objects appearance and shape by characterizing it using local intensity gradients and edge directions. The image is densely divided into small spacial regions called cells. Each cell is then grouped into a block consisting of 4 cell arrange in a 2x2 configuration. The 160x96 pixel image divided into 19x11 blocks such that each block has a 50% overlap with another block. Within each of the blocks each cell is used to generate a 1-D histogram of gradient directions/edge directions and later all cell data is combined to give a complete histogram of oriented gradients descriptor of the window, resulting in a 19x11x(4*9) = 4930-D vector for a histogram utilizing 9 bins. The entire process is represented by the image below.

(In our case we will not be normalizing gamma nor we will be normalizing the colours, we will also be converting the images into grayscale before processing them further.)

## Code
### Required Libraries

```python
# Libraries
# Standard Libraries
import os
from PIL import Image

# Third Party Libraries
import matplotlib.image as mpimg
import matplotlib.pyplot as plt
import numpy as np
from scipy import ndimage
from sklearn import svm
from sklearn.model_selection import KFold

%matplotlib inline
```

### Compute Gradients
The gradient is computed as a point discrete derivative mask. In their paper Dalal and Triggs describe various methods to compute the gradient and they discovered that using a 1-D $[−1, 0, 1]$ masks in both horizontal and vertical directions at $\sigma=0$ work best. (The mask that were tested include an uncentred $[−1, 1]$, centred $[−1, 0, 1]$ and cubic corrected $[1, −8, 0, 8, −1]$, in addition to a 3×3 Sobel masks and 2×2 diagonal ones $\left(\begin{array}{rr}
0 & 1 \\
-1 & 0 \end{array} \right)$ and $\left(\begin{array}{rr}
-1 & 0 \\
0 & 1 \end{array} \right)$).

Below we show how to generate the mask by defining a `grad` functions that takes the image as an argument.

```python
def grad(image):
    grad_hor, grad_ver = np.zeros_like(image), np.zeros_like(image)

    grad_hor = np.gradient(image, axis=1)
    grad_hor[:, 1:-1] = (2 * grad_hor[:, 1:-1]) % 256

    grad_ver = np.gradient(image, axis=0)
    grad_ver[1:-1, :] = (2 * grad_ver[1:-1, :]) % 256

    return grad_hor, grad_ver
```

In addition to computing the gradients we also need to compute the magnitudes and orientation. As per the findings in the paper we choose to go with an unsigned orientation thus limiting our angle to be between $0^{\circ}$ and $180^{\circ}$.

```python
def magnitude_orientation(gx, gy):
    magnitude = np.hypot(gx, gy)
    orientation = (arctan2(gy, gx) * 180 / np.pi) % 180

    return magnitude, orientation
```

### Weigthed Vote Into Spatial & Orientation Cells
This stage involves dividing the window into adjacent, non-overlapping regions called cells of size $C\times C$ pixel. For simplicity we choose to use rectangular cells, nevertheless, there is no restriction to the shape of a cell and Dalal and Triggs give a description of a radial (log-polar sectors) cell.  Within each cell "each pixel calculates a weighted vote for an edge orientation histogram channel based on the orientation of the gradient element centred on it, and the votes are accumulated into orientation bins over local spatial regions. The orientation bins are evenly spaced over $0^{\circ}$ to $180^{\circ}$. To reduce aliasing, votes are interpolated bilinearly between the neighbouring bin centres in both orientation and position. The vote is a function of the gradient magnitude at the pixel, either the magnitude itself, its square, its square root, or a clipped form of the magnitude representing soft presence/absence of an edge at the pixel".

Example:
To compute the weigthed votes let $B$ be the number of bins (in our case $B=9$), $m$ the magnitude, and $\theta$ the orientation. We number the bins from $0$ to $B−1$ where each bin has width $w = \frac{180}{B} \big(\frac{180}{9} = 20\big)$. Bin $i$ has boundaries $[wi, w(i + 1))$ (in our case the boundary is $[20i, 20(i+1))$). The vote for a pixel is computed to be:

$
\begin{align}
v_j &= m\frac{w(j+1) − \theta}{w} \text{ to bin number } j \\
v_{j+1} &= m\frac{w(j+2) − \theta}{w} \ \text{ to bin number } (j+1)
\end{align}
$

More concretely, if we let $\theta = 77^{\circ}$, $B=9$, $w=20$ then

$
\begin{align}
v_3 &= m\frac{w_4 − \theta}{20} \text{ to bin number } j = 3 \\
&= m\frac{80-77}{20} = m\frac{3}{20} = 0.15m\\
\text{and}&\\
v_{4} &= m\frac{w_5 - \theta}{20} \ \text{ to bin number } j =4 \\
&= m\frac{90 - 77}{20} = m\frac{17}{20} = 0.85m
\end{align}
$

To compute the "weigthed vote into spatial & orientation cells" we first need to build the cell which should take as arguments the orientation, the gradient, and the number of bins, this is done below in the `_build_cell` function. We can then define a `_build_hist` functions whih takes the same arguments as the `_build_cell` function in addition to cell size to build a histogram. We choose to compute the vote using the magnitude simalarly to the example above.

```python
def _build_cell(orientation, gradient, nbins):

    cell_histogram = np.zeros(shape=nbins)
    b_step = 180/nbins

    for orient, grad in zip(orientation, gradient):
        for o, g in zip(orient, grad):
            c_bin = np.int8(np.abs(o//20))
            prop = np.abs(o % b_step) / b_step

            if c_bin == (nbins -1):
                cell_histogram[c_bin] += (1-prop) * g
                cell_histogram[0] += prop * g
            else:
                cell_histogram[c_bin] += (1-prop) * g
                cell_histogram[c_bin+1] += prop * g

    return cell_histogram

def _build_hist(orientation, gradient, nbins, cell_size):

    cell_y, cell_x = cell_size
    overfill = tuple([s % c for s, c in zip(orientation.shape, cell_size)])

    # Gets the left side of the image, if the image cannot be covered by
    # equal sized cells
    shape = tuple([s_i - over_i
                   for s_i, over_i in zip(orientation.shape, overfill)])

    histograms = np.zeros((shape[0]//cell_y, shape[1]//cell_x, nbins))
    i = 0
    for y in range(0, shape[0], cell_y):
        j = 0
        for x in range(0, shape[1], cell_x):
            orient = orientation[y: y+cell_y, x: x+cell_x]
            grad = gradient[y: y+cell_y, x:x+cell_x]
            histograms[i, j, :] += _build_cell(orient, grad, nbins=nbins)
            j += 1
        i+= 1

    return histograms
```

### Contrast Normalize Over Overlapping Spatial Blocks

Gradient strengths vary over a wide range owing to local variations in illumination and foreground-background contrast, as such it is necessary to apply some form of normalization. Normalization is accomplished by grouping cells into overlapping blocks of $2\times2$ cells, so that each block is of size size $2C×2C$ pixels. Two horizontally or vertically consecutive blocks overlap by two cells, that is, the block stride is C pixels. As a consequence, each internal cell is included into four seperate blocks.

#### Normalization
Dalal and Triggs evaluate four different normalization schemes. If we let $\mathbf{v}$ be the unnormalized descriptor vector,
$||v||_k$ be its k-norm for $k=1, 2$, and $\varepsilon$ be a small constant then the normalizations are described as follows:

$
\begin{align}
L_1 &= \frac{v}{||v||_1 +\varepsilon}\\
L_{1,sqrt} &= \sqrt{\frac{v}{||v||_1 +\varepsilon}}\\
L_2 &= \frac{v}{||v||_2 +\varepsilon}\\
L_{2,clip} &= L_2 \text{ followed by clipping and renormalizing}
\end{align}
$

The authors claim that $L_{2,clip}, L_2$ and $L_{1,sqrt}$ all perform equally well, while simple $L_1$ reduces performance by 5%. As such we decide to use $L_2$ as a normalization scheme.

The `_build_block` function defined bellow describes how to compute this stage using $L_2$ normalization, it takes the same arguments as `_build_hist` in addition to block_size. The function works by sliding over the orientation and gradients to compute the histograms of the cells and grouping them into appropriate size.

```python
def _build_block(orientation, gradient, nbins, cell_size, block_size):

    histograms = _build_hist(orientation, gradient, nbins, cell_size)
    nbr_y, nbr_x, _ = histograms.shape
    dim = reduce(lambda x, y: x*y, block_size)
    hog_matrix = np.zeros(((nbr_y-1)*(nbr_x-1), dim*nbins))

    i = 0
    for y in range(0, nbr_y-1):
        j = 0

        for x in range(0, nbr_x-1):
            block = histograms[y: y+block_size[0], x: x+block_size[1]]
            norm_value = np.linalg.norm(block.ravel() + 1e-4, ord=2)
            block = block.ravel()/norm_value
            position = (nbr_x-1)*i + j
            hog_matrix[position] = block
            j += 1
        i+= 1

    return hog_matrix
```

## HOG Descriptor
Putting it all together we have the following function

```python
def hog(image, nbins=9, cell_size=(8, 8), block_size=(2, 2), flatten=True):
    gx, gy = grad(image)
    mag, orientation = magnitude_orientation(gx, gy)

    hog_matrix = _build_block(orientation=orientation, gradient=mag,
                              nbins=nbins, cell_size=cell_size,
                              block_size=block_size)

    if flatten:
        return hog_matrix.ravel()
    else:
        return hog_matrix
```

### Linear SVM
The article describes using a linear SVM for training. We train the SVM using the sklearn library and we use K-fold cross validation to reduce the change of overfitting.
def hog_svm(data, seed=35, save=False, save_loc='C:\\...\\Train\\model.pkl'):

	from sklearn import svm
	from sklearn.model_selection import KFold

	rng = np.random.RandomState(seed=seed)
	random_index = rng.randint(0, data.shape[0], 4930)
	data = data[random_index]

	kf = KFold(n_splits=10)
	clf = svm.SVC(kernel='linear')
	coefficients = np.zeros(shape=(1, 7524))

	for train, test in kf.split(data):
		train_data = data[train]
		test_data = data[test]

		clf.fit(train_data[:, :-1], train_data[:, -1])
		predictions = [int(a) for a in clf.predict(test_data[:, :-1])]
		num_correct = sum(int(a==y) for a, y in zip(predictions, test_data[:, -1]))

	if save:
		from sklearn.externals import joblib

		print("Saving Model")
		joblib.dump(clf, save_loc)

    return
        clf
### People Detection Using a Sliding Window
Finally, to be able to detect people in an image we use a sliding window approach which we define below.

```python
def sliding_window(image, window_size, step_size):

    for y in range(0, image.shape[0], step_size):
        for x in range(0, image.shape[1], step_size):
            # yield the current window
                img = image[y: y+window_size[1], x: x+window_size[0]]
                if img.shape == (int(window_size[1]), int(window_size[0])):

                    yield (x, y, img)
```

## Running The Code
To run the code simply load the positive and negative training images, run `hog()` on each image to get the hog descriptor. Once the images have been processed simply call
loc_p = 'C:\\..\\Train\\pos\\'
loc_n = 'C:\\...\\Train\\neg\\'

p_files = os.listdir(loc_p)
n_files = os.listdir(loc_n)
total_files = len(p_files) + len(n_files)
data = np.zeros(shape=(total_files, 7525))

print("Starting to process postive training data")
for i, files in enumerate(p_files):
    print("{}, {}".format(i, files))
    file = loc_p + str(files)
    img = ndimage.imread(file, mode='L')
    hog_descriptor = hog(img)

    data[i, :-1] += hog_descriptor
    data[i, -1] = -1

print("\nStarting to process negative training data")
for file in n_files:
    i += 1
    print("{}, {}".format(i, file))
    file = loc_n + str(file)
    img = ndimage.imread(file, mode='L')
    hog_descriptor = hog(img)

    data[i, :-1] += hog_descriptor
    data[i, -1] = 1

clf = hog_svm(data, save=False)

## Will run sliding window over test image
# test_im = 'C:\\...\\Train\\crop_000608.png'
# img = ndimage.imread(test_im, mode='L')

# rects = []
# sl_window = sliding_window(img, window_size=(96, 160), step_size=8)
# for window in sl_window:
#    wdw = window[-1]
#    hog_im = hog(wdw)
#    hog_im = hog_im.reshape(1, -1)
#    predict = clf.predict(hog_im)
#    print(predict)

#    if predict == -1:
#         x, y = windows[0], window[1]
#         rects.append([x, y, x+96, y+160])

# pick = non_max_suppression(rects, probs=None, overlapThresh=0.65)
To improve detection we can generate an image pyramid and apply a sliding window to each sub-image. We would then combine the overlapped boxes to produce our final detection. This can be done using a mean-shift algorithm to detect multiple modes in the bounding box space by utilizing the (x, y) coordinates of the bounding box as well as the logarithm of the current scale of the image as suggested by Dalal and Triggs. One can also use a non-maximum suppression algorithm to combine the overlapped boxes, we found that this yielded slightly better results  and can be found in the code utilizing `OpenCv`.

## Using OpenCV
The implimentation in OpenCV is more optimized then that presented above. Furthermore, `OpenCV` has a pretrained people detector, which facilitates the entire process. The code below shows how to use `OpenCV` to perform people detecting.

```python
import cv2
import imutils
from imutils.object_detection import non_max_suppression
import numpy as np
```

```python
hog = cv2.HOGDescriptor()
hog.setSVMDetector(cv2.HOGDescriptor_getDefaultPeopleDetector())
```
box = 0

loc = 'C:\\..\\Pedestrian_detection\\Datasets\\INRIA\\Train\\pos\\crop001001.png'
im = cv2.imread(loc)
im = imutils.resize(im, width=min(100, im.shape[1]))

# detect people in the image
rects, weights = hog.detectMultiScale(im, winStride=(1, 1),
                                      padding=(6, 6), scale=1.05)

# for (x, y, w, h) in rects:
#     cv2.rectangle(orig, (x, y), (x + w, y + h), (0, 0, 255), 2)

# apply non-maxima suppression to the bounding boxes
rects = np.array([[x, y, x + w, y + h] for (x, y, w, h) in rects])
pick = non_max_suppression(rects, probs=None, overlapThresh=0.65)

# draw the final bounding boxes
for (xA, yA, xB, yB) in pick:
    cv2.rectangle(im, (xA, yA), (xB, yB), (0, 255, 0), 2)

# show the output images
cv2.imshow("People Detect", im)
cv2.waitKey(0)

```python

```


```python
from estado import *
from sympy import *
import copy

```

```python
import numpy as np
from sympy import *
import copy
```

```python
# pot = 120000
# Tmin = 305 #kelvin
# Tmax = 1500 #kelvin
# razao_compr = 15/1
# Pmin = 100
# k = 1.4 #constante ar
# Cp = 1.004
#
#
#
#
#
# T1 = Tmin
# T3 = Tmax
# P1 = Pmin
# P2 = P1*razao_compr
# P3 = P2
# P4 = P1
#
# T4 = T3*(P4/P3)**((k-1)/k)
#
# T2 = T1*(P2/P1)**((k-1)/k)
#
# DeltaH43 = Cp*(T4-T3)
#
# DeltaH21 = Cp*(T2-T1)
#
#
# mponto = -pot/(DeltaH43+DeltaH21)
#
# Pot_turb = -mponto*DeltaH43
#
# Efic = 1 - 1/(razao_compr**((k-1)/k))
#
# print(Pot_turb/1000, 'Pot turb')
# print(Efic*100, 'Eficiencia')
#
#
```

```python

```

```python
# pot = 15000
# Tmin = 27 + 273.15 #kelvin
# # Tmax = 1500 #kelvin
# Calor = 950
# razao_compr = 14/1
# Pmin = 100
#
# k = 1.4 #constante ar
# Cp = 1.004
#
# T1 = Tmin
# # T3 = Tmax
# P1 = Pmin
# P2 = P1*razao_compr
# P3 = P2
# P4 = P1
# R = 0.287
# #
# arr = np.array([['300', str(T1), '320'],
#             ['300.47','','320.58']])
#
# h1 = interpolacao(arr)
#
#
# arr = np.array([['300', str(T1), '320'],
#             ['6.86926','','6.93413']])
#
# s1 = interpolacao(arr)
#
# s2 = s1 + R * log(razao_compr)
#
# arr = np.array([['7.61090', str(s2), '7.64448'],
#             ['620','','640']])
#
# T2 = interpolacao(arr)
#
# arr = np.array([['628.38', '', '649.53'],
#             ['620',str(T2),'640']])
#
# h2 = interpolacao(arr)
#
#
# h3 = h2 + Calor
#
# arr = np.array([['1450', '', '1500'],
#             ['1575.40',str(h3),'1635.80']])
#
# T3 = interpolacao(arr)
#
# arr = np.array([['1450', str(T3), '1500'],
#             ['8.57111','','8.61208']])
#
# s3 = interpolacao(arr)
#
# s4 = s3 + R * log(1/razao_compr)
#
# arr = np.array([['7.80008', str(s4), '7.82905'],
#             ['756.73','','778.46']])
#
# h4 = interpolacao(arr)
#
# mponto = -pot/(h4 - h3 + h2 - h1)
#
#
# Pot_turb = -mponto*DeltaH43
#
# Efic = 1 - 1/(razao_compr**((k-1)/k))
#
# print(mponto, 'Vaz mass')
# print(T3 , 'Tmax')
#
```

```python
# s4

```

```python
# Tfria = -10
# Tamb = 25
# material =
#
# _estado34 = estado('nh3','saturado',T=Tamb)
# _estado34.propriedade_dado_titulo(1)
# estado3 = copy.copy(_estado34)
# _estado34.propriedade_dado_titulo(0)
# estado4 = copy.copy(_estado34)
#
#
# _estado12 = estado('nh3','saturado',T=Tfria)
# _estado12.titulo_dada_propriedade('specific_entropy',estado4.specific_entropy)
#
#
# estado1 = copy.copy(_estado12)
# _estado12.titulo_dada_propriedade('specific_entropy',estado3.specific_entropy)
# estado2 = copy.copy(_estado12)
```

```python

```

    1.248