# Lab 1.2: Introduction to Radar

```python
%matplotlib widget
import rad.example as ex
import rad.quiz as qz
from rad.const import c, k
from rad.radar import to_db, from_db, deg2rad, rad2deg
from math import sqrt, sin, asin, cos, acos, tan, atan2, pi, log, log10
```

**Reminders**:

- Hit the *Run All* button  button above before continuing
- Useful formulae and definitions are available in [Reference](Reference.ipynb)

Alright, now we can start learning about radar! To begin, we will first start on the basics of ranging.

## Ranging

Many commercial and defense applications rely on the estimation of the location and signature properties of a **target**:
- Airplane in the sky
- Object under the ground
- Submarine in the water
- Person in a building

To interrogate this region, sensing can be done in two different modes: *active* or *passive*. In active sensing, energy is emitted from a controlled source, the targets perturb or scatter the transmitted energy and create an echo; the echo is then received back at the sensor (see figure below). The sensor then will extract target information from the received echo. In contrast, passive sensing relies on an external uncontrolled source of energy---it could even be the target itself. For this course, though, we will only be concerned with active sensing.

### Waves and Propagation

For active ranging to work, we need the ability emit a **wave** that travels through space and reach the target. The wave will move at a naturally defined speed called the **propagation velocity**, $v_p$ , that depends on the physical mechanism used to propagate the wave. For instance, in sonar, waves move at the speed of sound in water, i.e., $v_p = 1500~\mathrm{m/s}$, and in radar, waves will move at the speed of light[1](#foot_light), i.e., $v_p = 3\times 10^8~\mathrm{m/s}$.

Most ranging systems use sine waves (or variations thereof) as the main wave shape due to their ease in generation and well-understood behavior. In the interactive plot below, we see the transmission of a sine wave at different frequencies and propagation velocities. The pink dot on the left shows the instantaneous value of the transmitted wave and the red line is the actual wave. We can start the transmission using the *Play*  button, freeze time using the *Pause*  button, and reset the plot using the *Stop*  button. Additionally, the wave frequency and propagation velocity can be changed using the *Frequency* and *Propagation Velocity* sliders, respectively.

```python
ex.ex_1_2_1()
```

We can notice that changing the *Frequency* only affects how quickly the instantaneous value (i.e., pink dot) moves and altering the *Propagation Velocity* affects how quickly the wave moves outward.

***

#### Question 1

Using the *Frequency* and *Propagation Velocity* sliders and *Pause*  button, estimate the wavelength (i.e., distance from peak-to-peak or valley-to-valley) for the following cases:

**(a)** What is the wavelength, $\lambda$ , of a transmitted sine wave with frequency $f = 1000~\mathrm{Hz}$ and propagation velocity $v_p = 1000~\mathrm{m/s}$?

```python
qz.quiz_1_2_1a()
```

```python
# Scratch space
```

**(b)** What is the wavelength, $\lambda$ , of a transmitted with frequency $f = 1000~\mathrm{Hz}$ and propagation velocity $v_p = 2000~\mathrm{m/s}$?

```python
qz.quiz_1_2_1b()
```

```python
# Scratch space
```

**(c)** What is the wavelength, $\lambda$ , of a transmitted with frequency $f = 500~\mathrm{Hz}$ and propagation velocity $v_p = 1000~\mathrm{m/s}$?

```python
qz.quiz_1_2_1c()
```

```python
# Scratch space
```

***

We can see that wavelength, $\lambda$ , is related to both the transmit frequency, $f$ , and the propagation velocity, $v_p$. In fact, it can be summarized by:

$$
\lambda = \frac{v_p}{f}
$$

One way to think about this result is as follows: consider sending one period of a sine wave with period $T$. At time $t = 0$, the beginning of the wave is transmitted. By the definition of the period, the time it takes for the transmitted sine wave to progress through a single period is $T$. At the end of that period at which $t = T$, the front of the wave has travelled outward by a distance of $v_p \cdot T$. We also know that $T = 1/f$, so the distance travelled by the front of the wave after a period, i.e., the wavelength, is $\lambda = v_p \cdot T = v_p \cdot 1/f$.

We can verify this with the interactive plot below. Try seeing how far the front of the wave moves while the instantaneous value (i.e., pink dot) starts and returns to a peak value.

```python
ex.ex_1_2_1()
```

Note that you can also rearrange the wavelength equation to also get:

$$
f = \frac{v_p}{\lambda}
$$

So, if we know the wavelength and the propagation velocity, we can also calculate the frequency of the transmitted sine wave.

***

#### Question 2

**(a)** In radar ($v_p = 3\times 10^8~\mathrm{m/s}$), what is wavelength, $\lambda$ , of a transmitted sine wave with a frequency $f = 500~\mathrm{MHz}$?

```python
qz.quiz_1_2_2a()
```

```python
# Scratch space
```

**(b)** In sonar ($v_p = 1500~\mathrm{m/s}$), if a transmitted sine wave has a wavelength of $\lambda = 5~\mathrm{m}$, what is the transmit frequency, $f$?

```python
qz.quiz_1_2_2b()
```

```python
# Scratch space
```

***

As the emitted energy travels outward from the sensor, it weakens; this is called **propagation loss**. This is due to the fact that there is a fixed amount of energy in the emission, and it spreads out in space over time. This is the same reason that if you were to throw a rock in a still pond, the ripples on the water get shallower as they move out from where the rock hit the water.

To illustrate this, the following example shows the transmission of a source producing a time-limited sine wave propagating at $v_p = 1000~\mathrm{m/s}$ and how its strength changes as it emanates. We will be looking from the top down at the sensor on a patch of ground that is $200~\mathrm{km}$ by $200~\mathrm{km}$; the wave plotted is the value at ground level. The amplitude is shown by the transparency of the lines; sine wave peaks are thick lines, sine wave valleys are thin lines.

```python
ex.ex_1_2_2()
```

To garner more intuition, we can also view how the amplitude of a wave evolves along a line of propagation, i.e., along the ground. The interactive plot below shows the perspective of the sine wave as it travels away from the source at propagation velocity of $v_p = 1000~\mathrm{m/s}$. Try changing the *Transmit Energy* and *Transmit Frequency* to see how the wave changes.

```python
ex.ex_1_2_3()
```

It is important to note that there are often other factors that may cause propagation loss to be more severe (e.g., absorption in water), making the impulse lose amplitude at a greater rate than shown above. However, for this course, we will only consider the attenuation due to spherical emanation of the wave. A more detailed discussion of propagation loss is in [Lab 2.1: Radar Range Equation](2_1_Radar_Range_Equation.ipynb).

### Undirected Ranging

First, we will discuss undirected ranging sensors; this means a **pulse** (i.e., time-limited wave) of energy will be sent in all directions equally, and echoes from all targets will be received back at the sensor. For the sake of presentation, we will begin by considering a sensor that can produce an **impulse**, i.e., an extremely short and energetic pulse. (Practically, impulses are extremely difficult to generate, but they are a good way to introduce the basics of ranging.)

In the interactive plot below, we can see the transmitted impulse in **blue** and echo in **red**; you can use the *Play*  button to transmit the impulse, and the *Stop*  button to reset. To scrub through the scene, you can click on the *Frame* slider and use the left and right arrow keys to move it one step at a time. You can also adjust the location of the target using the *East* and *North* sliders. The propagation velocity for this example is $v_p = 1000~\mathrm{m/s}$.

**Note:** Once we press *Play* , we cannot change any parameters until we reset with *Stop* .

```python
ex.ex_1_2_4()
```

You may have also noticed that when an echo is received back at the sensor, the sensor will temporarily turn **orange**. This known as a **detection**--an acknowledgement that an echo has been received. We will discuss detections and how they are made in [Lab 3.2: Detection](3_2_Detection.ipynb).

***

#### Question 3

Using the interactive plot above to estimate some times of interest (Hint: Use the *Frame* slider). For a target at $75~\mathrm{m}$ East, $80~\mathrm{m}$ North:

**(a)** What is the time that the transmitted pulse reaches the target (i.e., center of target circle)?

```python
qz.quiz_1_2_3a()
```

```python
# Scratch space
```

**(b)** What is the time that the echo reaches the sensor (i.e., center of sensor circle)?

```python
qz.quiz_1_2_3b()
```

```python
# Scratch space
```

***

In typical ranging applications, we *cannot* measure the time that the pulse hits the target, but we *can* measure when the echo comes back to the sensor. The time between transmission of the impulse and the reception of the echo is called the **propagation delay**, $\tau$. (For this course, we will only be concerned with the two-way propagation delay (to and from the target); in some applications, e.g., wireless communications, one-way propagation is a key quantity of interest.)

Thus, to measure range, the sensor will measure the propagation delay from the transmission of the impulse to the reception of its echo. The range from the sensor, $r$ , is related to the propagation delay, $\tau$ , and propagation velocity, $v_p$ , by:

$$\tau = \frac{2r}{v_p}$$

The factor of two comes from the fact that the transmitted pulse travels the range to the target, and then the echo travels the same distance back.

For the target in Question 4, we can calculate the exact time that the echo should hit the sensor by:
$$
\begin{align*}
\tau &= \frac{2r}{v_p}\\
&= \frac{2 \cdot \sqrt{75^2 + 80^2}}{1000}\\
&= 219.3~\mathrm{ms}
\end{align*}
$$
where we used the fact that the range is calculated by $r = \sqrt{x^2 + y^2}$.

Rearranging terms, we can also solve for the range if we know the propagation delay as:

$$
r = \frac{v_p \tau}{2}
$$

***

#### Question 4

**(a)** What is the propagation delay, $\tau$ , to and from a target at a range $r = 1000~\mathrm{km}$ from a radar? (Propagation velocity for radar is $v_p = 3 \times 10^8~\mathrm{m/s}$)

```python
qz.quiz_1_2_4a()
```

```python
# Scratch space
```

**(b)** If the propagation delay for a radar return is $5.7~\mathrm{ms}$, what is the range to the target?

```python
qz.quiz_1_2_4b()
```

```python
# Scratch space
```

#### Question 5

Now, let us take one more step toward a real ranging system. In the interactive plot below, there is a hidden target, and the echo is no longer plotted. Also, there is a *Detections* box to keep track of any detected echoes and their time of reception. The propagation velocity is $v_p=1000~\mathrm{m/s}$.

```python
ex.ex_1_2_5()
```

What is the range of the hidden target (in meters)?

```python
qz.quiz_1_2_5()
```

```python
# Scratch space
```

***

So, from the last example, we know that by the finding the presence of echoes, we can locate the range of a hidden target. However, we still do not know *which direction* it is relative to the radar. To figure that out, we have to use **directed ranging**.

### Directed Ranging

There are two main forms of directed ranging:

- **Directed on transmit**: Transmitted energy is focused into a certain direction, so if an echo is received, you know the direction from which it originated
- **Directed on receive**: Only certain directions are listened for echoes, so if an echo is received, you know the direction from which it originated

These methods can be used separately but are most often used together. Note that it is not typically useful in ranging applications to transmit your energy in one region on transmit and listen to a different region for echoes on receive; for this course, we will assume that if you are using both methods, then their directions are identical.

To describe direction, we will use **azimuth** as defined in the picture below. It ranges from 0° (North), 90° (East), 180° (South), 270° (West), to 360° (North).

To begin, we will take a look at directed ranging on transmit. To see how this works, let us revisit the first example of this lab. In the interactive plot below, you will now notice a few things:

- The transmitted pulse is now focused to only a certain span of direction
- New *Transmit Azimuth* and *Transmit Beamwidth* sliders allow you to choose which direction and swath of angle to send a pulse
- An echo is only created when the target is hit with the focused energy

**Note:** Once we press *Play* , we cannot change any parameters until we reset with *Stop* .

```python
ex.ex_1_2_6()
```

As mentioned above, focusing energy around a certain direction creates a **beam**—just in the same way that the focused light from a flashlight is called a beam. One important property of a beam is called the **beamwidth**, which is the angular extent of the beam. The smaller the beamwidth, the narrower the energy becomes focused in space. If we denote the direction of interrogation as $\theta$ (defined using as an azimuth angle), then we will denote the beamwidth as $\Delta \theta$. In the following question, we will see how forming a beam can be used to localize a target in a certain direction.

***

#### Question 6

In the scene below, there is a hidden target. Find its position in terms of range and azimuth using directed ranging on transmit. The propagation velocity is $v_p = 1000~\mathrm{m/s}$.

**Note:** Once we press *Play* , we cannot change any parameters until we reset with *Stop* .

```python
ex.ex_1_2_7()
```

**(a)** What is the range of the target?

```python
qz.quiz_1_2_6a()
```

```python
# Scratch space
```

**(b)** What is the azimuth of the target?

```python
qz.quiz_1_2_6b()
```

```python
# Scratch space
```

***

From Question 6, you can note an interest tradeoff starting to appear:
- A *wider* beam makes it easier to get a detection with fewer transmissions, but it has higher uncertainty in the direction of the target
- A *smaller* beam requires more transmissions to find the target, yet it gives lower uncertainty of the direction of the target

The beamwidth can also affect the strength of transmitted energy into a region of space; we will discuss that in following labs.

For directed ranging on receive, we have a different example. Now, in the interactive plot, the azimuth changes the region in space that is listened to for echoes. You will notice that the detection (i.e., sensor turns **orange** when hit by echo) only happens when the target is in the receive beam of the sensor.

**Note:** Once we press *Play* , we cannot change any parameters until we reset with *Stop* .

```python
ex.ex_1_2_8()
```

***

#### Question 7

In the following, use directed ranging on receive to locate a hidden target in range and azimuth. The propagation velocity is $v_p = 1000~\mathrm{m/s}$.

**Note:** Once we press *Play* , we cannot change any parameters until we reset with *Stop* .

```python
ex.ex_1_2_9()
```

**(a)** What is the range of the target?

```python
qz.quiz_1_2_7a()
```

```python
# Scratch space
```

**(b)** What is the azimuth of the target?

```python
qz.quiz_1_2_7b()
```

```python
# Scratch space
```

***

While finding the target in Question 7, we can notice that the same beamwidth tradeoff shows up as in directed on transmit: a wider beam makes it easier to detect the presence of a target but makes the direction more uncertain.

Up until now, we have looked at undirected ranging for finding the range of targets and directed ranging for estimating both the range and direction of targets. A good question to answer now is: *how does a sensor perform directed ranging*? Directed ranging uses two key steps for concentrating energy into a particular region in angle:

- **Focusing**: Narrowing of emission to desired swath in angle
- **Steering**: Choosing direction of transmission

In the following sections, we will discuss how focusing and steering are typically done.

### Focusing

For most ranging applications, focusing is done by creating an **aperture**, i.e., a surface in which identical transmissions/receptions are made. Apertures can be made in two different fashions:

 - Reflect the wave off of a shaping device, such as a parabolic dish
 - Populate the chosen surface with discrete emitters that can be operated in concert; this is called an **array**, each emitter is called an **element** of the array

The figure below shows two common figurations used to create apertures. On the left is a parabolic reflector, whose shape is chosen to align all of the waves emanating from a source and create a uniform transmission surface. On the right is an example of an array, which uses multiple sources to fill the same surface and create a coordinated transmission.

It may seem counterintuitive at first, but *as the aperture gets larger, the more focused the energy becomes*. For a concrete example, it can be shown[[1]](#ref_orfanidis) that using a circular aperture (like a dish) of diameter $D$ at transmit wavelength $\lambda$, the beamwidth $\Delta\theta$ (in degrees) is going to be bounded by

$$
\Delta\theta \geq 70^\circ\frac{\lambda}{D}
$$

For instance, if we were using a dish radar with diameter $D = 5~\mathrm{m}$ and transmitting at a wavelength of $\lambda = 0.33~\mathrm{m}$, then

$$
\begin{align}
\Delta\theta &\geq 70^\circ\frac{0.33}{5}\nonumber\\
&= 4.62^\circ
\end{align}
$$

**Note:** As beamwidths for actual systems are designed to get close to this bound, we will use this formula throughout the course as an approximation to the true beamwidth of a circular aperture.

To get a smaller beam, you would either have to transmit at a higher frequency (i.e., shorter wavelength) or increase the size of the dish. This result can be extended to non-circular apertures also. In general, the lower bound on beamwidth of any aperture will be of the form

$$
\Delta\theta \geq \mathrm{Shape~Factor}~\cdot\frac{\lambda}{D}
$$

where the shape factor depends on the specific aperture shape, and $D$ is the extent of the aperture in the considered dimension. Thus, to *narrow the beamwidth* of any sensor, we can:

- Increase transmit frequency, $f$ (i.e., shorten wavelength, $\lambda$)
- Increase aperture extent, $D$

It is important to note that even though we have discussed the process of focusing in the context of transmission, the same principles apply for reception. For example, the receive beamwidth will also be dictated in the same by the wavelength and size of the receive aperture.

***

#### Question 8

**(a)** What is the minimum beamwidth of a circular aperture of with diameter $D = 3.8~\mathrm{m}$ operating at a wavelength of $\lambda = 0.12~\mathrm{m}$?

```python
qz.quiz_1_2_8a()
```

```python
# Scratch space
```

**(b)** What is the smallest diameter possible to obtain a $1^\circ$ beamwidth when operating at transmit frequency $f =10~\mathrm{GHz}$ with a propagation velocity of $v_p = 3 \times 10^8~\mathrm{m/s}$?

```python
qz.quiz_1_2_8b()
```

```python
# Scratch space
```

***

### Gain

Concentrating energy into a focused beam not only allows you to get more certain angle information from target echoes, it also allows you to sense objects further in range. This is due to the propagation loss incurred as the waves travel through space; from before, we know that the further the wave travels, its amplitude weakens. By squeezing the transmitted energy into a beam, you are increasing the initial strength of the wave in the beam, allowing it to travel further without getting too weak. This concept is known as the **transmit gain**, $G_t(\theta)$, of an aperture: the ratio of the transmitted energy[2](#foot_gain) in a direction versus the energy input into the aperture:

$$
G_t(\theta) = \frac{\mathrm{Transmitted~Energy~in~Direction~}\theta}{\mathrm{Input~Energy}}
$$

Below is an interactive figure showing the gain pattern of a dish radar (i.e., circular aperture) with variable *Dish Radius* and *Transmit Frequency*. Try changing the radius and frequency and see how each impact the pattern.

```python
ex.ex_1_2_10()
```

We can notice that there is not a perfect "beam", but the plot exhibits a primary region in the middle which has the highest gain. This is because any aperture of finite size cannot actually focus all energy perfectly into a beam; there will always be a little amount of energy transmitted outside of the desired region. The central portion of the pattern with the peak gain is called the **mainlobe** of the pattern; the other outlying lobes are called **sidelobes**. We will discuss these more in later labs.

By varying the dish radius and transmit frequency, we have noticed that the energy becomes more concentrated and the maximum gain increases with: (i) increasing radius, and (ii) increasing frequency. In fact, it can be shown that the maximum gain at the center of the beam can be shown to be bounded by:

$$ G_t(\theta) \leq \frac{4\pi A_t}{\lambda^2} $$

where $A_t$ is the area of the transmit aperture. Thus, the maximum gain can be raised by:

- Increasing aperture area, $A$
- Increasing transmit frequency, $f$ (i.e., shortening wavelength, $\lambda$)

Note that the same results can be applied to the receive gain, $G_r(\theta)$, defined as:

$$
G_r(\theta) = \frac{\mathrm{Received~Energy}}{\mathrm{Incident~Energy~in~Direction~}\theta}
$$

Thus, the receive gain: (i) can also be increased by increasing aperture area, increasing frequency, and (ii) can be upper bounded by

$$ G_r(\theta) \leq \frac{4\pi A_r}{\lambda^2} $$

For systems that use the same aperture for transmission and reception, the transmit gain, $G_t(\theta)$, and receive gain, $G_r(\theta)$, will be very similar.

**Note:** Just as with the beamwidth bound, actual systems are designed to get as close as possible to the gain bound, and, thus, we will use
$$
G = \frac{4\pi A}{\lambda^2}
$$
as an approximation for true gain values throughout the course.

***

#### Question 9

If the propagation velocity is $v_p = 2000~\mathrm{m/s}$, what is the maximum transmit gain of a rectangular aperture of height $h = 2.1~\mathrm{m}$ and width $w = 3.3~\mathrm{m}$, operating at frequency $f = 15~\mathrm{kHz}$? (Note that the answer is asked for in $\mathrm{dB}$, which is the typical form used to represent gain.)

```python
qz.quiz_1_2_9()
```

```python
# Scratch space
```

***

### Steering

We now have determined how to focus energy in space using an aperture. Now, the next step in directed ranging is to choose which direction to emit the energy; this step is called steering. The most straightforward method of steering is called **mechanical steering**, which simply uses a gimbal to move the aperture around and point at the desired direction. This is commonly done for reflector-type apertures like parabolic dishes. Mechanical steering can be quite slow depending on the size and weight of the sensing apparatus. For instance, large radar systems can weigh up to 10 tons, making it very difficult to move the direction of transmission quickly. Below is a picture of the Haystack[[2]](#ref_haystack) radar system and its elaborate pedestal used for movement.

For arrays, there is another type of steering that can be much faster than mechanical called **electronic steering**. It is called *electronic* because it uses electronic components to slightly change the transmission from each element in order to steer the signal. The most intuitive form of electronic steering is called **delay steering**, which follows this main premise: *delay the transmission of each element in the array appropriately so they all hit the target at the same time*.

The following interactive example shows how delay steering looks for an array with $7$ elements, spaced $4~\mathrm{m}$ apart. Try changing the transmit azimuth and see how the energy concentrates at different angles. You will notice that the strongest (i.e., darkest) echo occurs when the all of the impulses hit the target at the same time.

```python
ex.ex_1_2_11()
```

The mathematics behind delay steering (and its commonly-used variant called **phase steering**) will be discussed more in [Lab 3.1: Radar Transmissions and Receptions](3_1_Radar_Transmissions_and_Receptions.ipynb).

### Target Motion

In the discussions above, we have only considered slow-moving or stationary targets. If a target it moving with respect to the sensor, it can alter the characteristics of the transmitted pulse when reflecting it back. This is commonly known as the **Doppler effect**. In the interactive example below, we can see how the **range rate** of the target (speed at which the target moves directly toward/away from the sensor) causes the reflected wave to change. Try varying the *Initial Range*, *Range Rate*, and *Transmit Frequency* sliders to see the difference from the transmitted wave to the received echo. For this example, the propagation velocity is $v_p = 1000~\mathrm{m/s}$.

```python
ex.ex_1_2_12()
```

We can notice that as the target moves faster toward the sensor, the wave is compressed, which increases its frequency; when the target is moving away from the sensor, the wave dilates, lowering the frequency. We will discuss the Doppler effect in detail and how we can use it to measurement target velocity in [Lab 4.1: Target Parameter Estimation](4_1_Target_Parameter_Estimation.ipynb). For the next few labs, however, we will focus on slowly-moving targets, as it makes concepts much easier to introduce.

Up until now, we have discussed concepts that will apply to most ranging methods:

- Waves and propagation
- Estimating target range from propagation delay
- Directed ranging for estimation of target angle
- Focusing transmitted and received energy using apertures
- Steering transmitted and received energy using mechanical or electronic methods
- Target motion can alter the transmitted wave through compression/dilation

Now, it is time to narrow our scope down to a specific type of ranging system: **radar**.

## Radar

Radar is short for *radio detection and ranging*; the "radio" implies that it uses electromagnetic waves in the radio frequency portion of the electromagnetic spectrum to perform ranging. This tells us a couple of the special properties of radar versus other ranging mechanisms:

- Propagation velocity is the speed of light[1](#foot_light), $v_p = 3\times 10^8~\mathrm{m/s}$
- Radars typically operate at frequencies from $10~\mathrm{MHz}$ to $300~\mathrm{GHz}$; equivalent to transmit wavelengths from $30~\mathrm{m}$ down to $1~\mathrm{mm}$

We now revisit the very first interactive plot from this lab that shows the propagation of a wave into space; now, however, we use the speed of light as the propagation velocity. This can give you a rough picture of a transmission emanating from a radar. Notice that the timestamp is now in nanoseconds ($\mathrm{ns}$).

```python
ex.ex_1_2_13()
```

Some bands of frequency are so commonly used that they have special names. for instance, here are some prevalent ones for radar systems:

- L-band: $1-2~\mathrm{GHz}$
- S-band: $2-4~\mathrm{GHz}$
- C-band: $4-8~\mathrm{GHz}$
- X-band: $8-12~\mathrm{GHz}$

Another special property of electromagnetic waves are that they are **transverse**, meaning the direction of variation is perpendicular to the direction of travel; this is often easier to explain graphically. Below is an animation of a radar transmission in three dimensions. You will notice that the wave varies up and down as it travels outward, similar to making waves on a taut rope.

The orientation of the wave with respect to the direction of travel is called its **polarization**; the polarization shown in the animation is called **vertical polarization** as the wave varies up and down as it propagates. We will discussion polarization in more detail in [Lab 3.1: Radar Transmissions and Receptions](Lab_3_1_Radar_Transmissions_and_Receptions.ipynb).

### Basic Radar Structure

A basic radar system consists of five main components:

- **Transmitter**: Dictates how and where energy is to be emitted
- **Antenna**: Emits electromagnetic energy into space, converts electromagnetic energy into received signals
- **Receiver**: Chooses how and where to listen for target echoes
- **Detection**: Finds target detections, estimates target attributes
- **Tracking**: Organizes detections into target tracks, estimates target motion

We will dive more into transmitters, antennas, and receivers in the [Lab 3.1: Radar Transmissions and Receptions](Lab_3_1_Radar_Transmissions_and_Receptions.ipynb), and the latter half of the course will focus on detection in [Lab 3.2: Detection](Lab_3_2_Detection.ipynb) and tracking in [Lab 4.2: Target Tracking](Lab_3_1_Target_Tracking.ipynb).

### Missions

Radars often have multiple objectives to accomplish. Here are some of the most frequently used:

- **Search**: Persistent surveillance of region for detection of target presence
- **Track**: Following a detected target, taking further measurements, and estimating target motion
- **Discrimination**: Analyzing target signature information to classify type

This course will focus on the *search* and *track* missions; as we will see throughout the course, each of these missions will influence radar design differently. When building a radar system, these constraints have to be balanced with each other and the construction cost.

Now, to wrap things up, let us look at applying some of the general ranging concepts discussed in this lab to a notional radar system.

***

#### Question 10

For a dish radar with a circular aperture of diameter $D = 4.4~\mathrm{m}$, transmitting at $f = 5~\mathrm{GHz}$:

**(a)** What is the transmit wavelength?

```python
qz.quiz_1_2_10a()
```

```python
# Scratch space
```

**(b)** What is the narrowest beamwidth possible for this system?

```python
qz.quiz_1_2_10b()
```

```python
# Scratch space
```

**(c)** What is the maximum transmit gain possible for this system?

```python
qz.quiz_1_2_10c()
```

```python
# Scratch space
```

***

 ## Summary

In this lab, we introduced the basics of ranging that are common to many applications:

- Waves and propagation
- Estimating target range from propagation delay
- Directed ranging for estimation of target angle
- Focusing transmitted and received energy using apertures
- Steering transmitted and received energy using mechanical or electronic methods

After discussing fundamental ranging, we then narrowed our focus to radar, which uses transmissions of electromagnetic energy in radio frequency bands to perform ranging. Aspects unique to using electromagnetic waves are:

- Electromagnetic waves propagate at the speed of light, $v_p = 3\times 10^8~\mathrm{m/s}$
- Electromagnetic waves are transverse, i.e., have a polarization

A radar system will typically consist of a transmitter, antennas, receiver, a detector, and a tracker.

## Footnotes

1The speed of light in the Earth's atmosphere is actually slightly slower than $3 \times 10^8~\mathrm{m/s}$, but we will use this value throughout the course since it is easy to remember.

2Gain is also commonly defined as the ratio of transmitted power in a direction to the inputted power. This definition is more useful for the study of constant-power (i.e., infinite duration) signals. The definition using energy is chosen for the course as radar signals (with some notable exceptions) are typically short in duration.

## References

[1] S. Orfanidis, *Electromagnetic Waves and Antennas*, 2016.

[2] *MIT Haystack Observatory* https://www.haystack.mit.edu/


# What's the purpose of Jupyt-nb ?

Jupyt-nb is designed to study Mathematical and Physical Constants properties of ratios as detailed in http://vixra.org/pdf/1904.0218v4.pdf

-
It uses 2 Julia packages:

* MathPhysicalConstants.SI

* PhysicalConstant.CODATA2019

it can be linked to numpy using PyCall

## Reference for Continuous Improvement

[](https://travis-ci.org/LaGuer/Jupyt-Nb)

## Reference for offline visualization

[](https://nbviewer.jupyter.org/github/LaGuer/Jupyt-Nb/blob/master/Julia%20Jupiter-Notebook%20Constants%20in%20Cosmology.ipynb)

Pkg.clone("https://github.com/laguer/MathPhysicalConstants.jl")

```julia
versioninfo()
```

    Julia Version 0.6.4
    Commit 9d11f62bcb (2018-07-09 19:09 UTC)
    Platform Info:
      OS: Linux (x86_64-pc-linux-gnu)
      CPU: Intel(R) Xeon(R) CPU @ 2.20GHz
      WORD_SIZE: 64
      BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Haswell MAX_THREADS=16)
      LAPACK: libopenblas64_
      LIBM: libopenlibm
      LLVM: libLLVM-3.9.1 (ORCJIT, broadwell)

```julia
Pkg.init()
```

    INFO: Initializing package repository /srv/julia/pkg/v0.6
    INFO: Package directory /srv/julia/pkg/v0.6 is already initialized.

```julia
Pkg.add("Unitful")
```

    INFO: Package Unitful is already installed

```julia
#import Pkg;
Pkg.status()
```

```julia
Pkg.clone("https://github.com/laguer/PhysicalConstant.jl.git")
```

```julia
Pkg.add("PhysicalConstant")
```

    INFO: Package PhysicalConstant is already installed

Pkg.dir("PhysicalConstant")

```julia
using Unitful
using Measurements
using MathPhysicalConstants.SI
```

Let's experiment MathPhysConstants by looking at the reduced electron Compton wavelength
Basic formula is given by ƛ_e ≡ ħ/m_e.c

```julia
big(SI.PlanckConstantHbar)/(big(SI.MassElectron)*big(SI.SpeedOfLight))

```

    3.861592046068737688702740558503376825155847784672545991504355454736855198364958e-13

```julia
big(SI.SanchezElectrConstant)
```

    1.370359991390000118371972348541021347045898437500000000000000000000000000000000e+02

```julia
measurement(SI.PlanckConstantHbar)
```

$1.05457162825e-34 \pm 0.0$

```julia
big(measurement(SI.PlanckConstantHbar))
```

$1.054571628249999983241389640919049903556150033962677651299952084762545619284539e-34 \pm 0.000000000000000000000000000000000000000000000000000000000000000000000000000000$

```julia
using Unitful
using Measurements
using MathPhysicalConstants.SI
```

```julia
a1 = big(e^e^e)
```

    3.814279104760214220732450485229492187500000000000000000000000000000000000000000e+06

```julia
a1^e
```

    7.764865179158049315959156489518625436695868996929735868921423813247391069467881e+17

```julia
a = SI.SanchezElectrConstant
```

    137.035999139

# $ln137/ln(a/137) ≈ (2 + 135/d_e)^2$

```julia
using Unitful
using Measurements
using Cosmology
using PyCall
@pyimport numpy as np

np.log(137)/ np.log(a/137)
```

    18726.168436946326

# $exp(2^{e^{2+1/2}})$ ≈ $exp(e^{2e} + e^2)$ = 8.820166056085355e102

```julia
using Unitful
using Measurements
using Cosmology
using PyCall
@pyimport numpy as np
@pyimport sympy as sy
@pyimport decimal

#np.exp(2^e^2.5)
np.exp(2^e)
```

    721.178003510984

```julia
exp(2^(e^2.5))
```

    Inf

# $exp(e^{2e} + e^2)$

```julia
using Unitful
using Measurements
using Cosmology
using PyCall
@pyimport numpy as np
@pyimport sympy as sy
@pyimport decimal

# decimal.Decimal(np.exp(2^e^2.5))
decimal.Decimal(np.exp(e^(2*e)+e^2))
```

    8.820166056085355e102

# The economic number : $e^{e^e}$

```julia
using Unitful
using Measurements
using Cosmology
using PyCall
@pyimport numpy as np
@pyimport sympy as sy
@pyimport scipy as sci
@pyimport matplotlib.pyplot as plt
@pyimport decimal
println(np.e^np.e^np.e)

```

    3.8142791047601975e6

Wien Reduced Constant

* See https://en.wikipedia.org/wiki/Wien%27s_displacement_law

$\omega = hc/kTλ_{Wien} $

NIST tells b=2.897 7729 x 10-3

```julia
(SI.PlanckConstantH*SI.SpeedOfLight)/(SI.Boltzmann * 1 * 2.8977729e-3)
```

    4.9651126128756795

$f(26) ≈ 6(2π^2a^3)^5$, where $2π^2a^3$ is the area of a 4-sphere of radius a

```julia

```

```julia
using Unitful
using Measurements
using Cosmology
using PyCall
@pyimport math
println(math.pi)

```

    3.141592653589793

This notebook is a preview demo of **IJulia**: a [Julia-language](https://julialang.org/) backend combined with the [IPython](https://ipython.org/) interactive environment.   This combination allows you to interact with the Julia language using IPython's powerful [graphical notebook](http://ipython.org/notebook.html), which combines code, formatted text, math, and multimedia in a single document.

* Note: this is a *preview*, because it relies on pre-release bleeding-edge versions of Julia, IPython, and several Julia packages, as explained on the [IJulia github page](https://github.com/JuliaLang/IJulia.jl), and functionality is evolving rapidly.  We hope to have a more polished release soon.

## Basic Julia interaction

Basic mathematical expressions work like you expect:

```julia
i = e
exp(i*pi)
```

    5113.984999751927

$$ \exp(i \pi)$$

You can define variables, write loops, and execute arbitrary multiline code blocks.  Here is an example of an alternating harmonic series $\sum_{n=1}^\infty \frac{(-1)^n}{n}$ from a [Julia tutorial by Homer Reid](http://homerreid.ath.cx/teaching/18.330/JuliaProgramming.shtml#SimplePrograms):

```julia
Sum=0.0
for n = 1:2:10000
    Sum += 1/n - 1/(n+1)
end
Sum # an expression on the last line (if it doesn't end with ";") is printed as "Out"
```

    0.6930971830599458

Previous outputs can be referred to via `Out[`*n*`]`, following the IPython, for example `Out[2]` for the result above.  You can also use the shorthand `_2`, or `_` for the previous result, as in IPython.  Like in Matlab, `ans` can also be used to refer to the previous result, *even if it was not printed* (when the command ended with `;`).

For example, the harmonic series above should be converging (slowly) to $\ln 2$, and we can check this:

```julia
Out[26] - log(2)
```

    -4.9997499999454575e-5

```julia
println(math.e^128-1)

```

    3.8877084059945687e55

```julia
using PhysConsts
α.quantity

```

```julia
big((α.value)^-1)

```

```julia
big(log((α.value)^-1))

```

```julia
big(math.exp(699))

```

```julia
import Pkg; Pkg.status()
```

```julia
using PhysConsts.DATA
```

```julia
c.quantity
```

    Speed of light in vacuum (c)
    Value                         = 2.99792458e8 m s^-1
    Standard uncertainty          = (exact)
    Relative standard uncertainty = (exact)
    Reference                     = CODATA 2019

```julia
a0.value
```

    Bohr radius (a0)
    Value                         = 5.2917721067e-11 m
    Standard uncertainty          = 1.2e-20 m
    Relative standard uncertainty = 2.3e-10
    Reference                     = CODATA 2019

```julia
G.value
```

    Newtonian constant of gravitation (Gg)
    Value                         = 6.67408e-11 m^3 kg^-1 s^-2
    Standard uncertainty          = 3.1e-15 m^3 kg^-1 s^-2
    Relative standard uncertainty = 4.6e-5
    Reference                     = CODATA 2019

```julia
g.quantity
```

    Proton mass (m_p)
    Value                         = 1.672621898e-27 kg
    Standard uncertainty          = 2.1e-35 kg
    Relative standard uncertainty = 1.3e-8
    Reference                     = CODATA 2019

Like Matlab or Scipy + Numpy, Julia has lots of mathematical functions and linear algebra built in.  For example, we can define a $500\times500$ random matrix $R$ and form the positive-definite matrix $R^* R$:

```julia
R = rand(50,50)
R' * R
```

    50×50 Array{Float64,2}:
     14.8401  11.0159   11.0398  12.1371  12.9914  …  10.0666   10.2719   11.5613
     11.0159  14.5508   10.6115  12.8797  13.3489      9.09651  11.5845   12.8904
     11.0398  10.6115   14.664   12.1591  12.6995     11.3192   11.2392   12.7697
     12.1371  12.8797   12.1591  18.0848  15.3832     10.9588   12.952    12.7643
     12.9914  13.3489   12.6995  15.3832  18.8978     11.3417   13.766    13.7376
     10.3582  12.1922   10.8784  12.5355  14.0756  …   9.75837  11.2291   12.7569
     11.403   11.4861   12.1963  12.413   13.4622     11.4271   12.3969   12.9923
     11.9348  11.3753   11.5956  13.6172  13.1759     10.7935   11.2115   12.2721
     12.2734  12.3439   11.7942  14.8864  14.2113     11.7685   12.4404   12.9881
     12.611   12.6692   12.6645  14.3057  15.212      12.4616   12.4764   13.661
     12.46    10.9597   11.9727  12.0599  12.9428  …  11.0595   11.6294   13.2982
     10.7607  12.462    11.4551  12.9311  14.3571     10.629    12.2371   14.6529
     11.5589  12.314    12.3473  12.9978  13.8131     10.5067   12.1406   13.694
      ⋮                                            ⋱
     11.3496  11.8048   11.2844  13.7931  13.7687     11.7626   12.937    12.6045
     11.0356  12.8355   11.1324  13.4014  13.5355      9.83385  11.9688   12.6278
     12.3802  14.0015   12.4311  13.7834  14.4353  …  12.0277   12.6047   14.6398
     11.4193  12.8167   12.8603  12.5936  14.013      10.8855   12.3162   14.082
     11.9126  11.3802   10.747   12.2219  12.3567     10.434    10.8304   12.1167
     11.8378  12.2769   11.7172  14.8255  14.3169     11.4519   13.129    13.469
     12.6271  12.8352   12.74    13.0265  15.1861     12.46     13.2996   15.0121
     12.1822  13.1224   11.2227  13.8452  14.8181  …   9.94071  12.3738   13.4795
     11.6848  10.9105   10.2393  11.041   12.3348      9.44363   9.89868  10.8772
     10.0666   9.09651  11.3192  10.9588  11.3417     14.6154   10.5976   10.1157
     10.2719  11.5845   11.2392  12.952   13.766      10.5976   16.6939   12.2948
     11.5613  12.8904   12.7697  12.7643  13.7376     10.1157   12.2948   18.5953

(Notice that, by default, only a portion of a large matrix is shown.  You didn't really want to read $500^2 = 250,000$ numbers, did you?)

Standard output from Julia is also captured and sent to the IJulia notebook as you'd expect:

```julia
println("Hello world!\n")
println(STDERR, "Testing, some unicode output on stderr...\n")
```

    Hello world!

    Testing, some unicode output on stderr...

IJulia even captures output from external C libraries (and notice how easy it is using Julia's `ccall` intrinsic):

```julia
ccall(:printf, Cint, (Ptr{UInt8},), "Hello from C!!\n");
```

    Hello from C!!

We can define functions, of course, and use them in later input cells:

```julia
using Unitful
using Measurements
using Cosmology
using PyCall

@pyimport math
@pyimport numpy as np
f(x) = np.power(np.log(2), (x / 4 ))
```

    f (generic function with 1 method)

```julia
using Unitful
using Measurements
using Cosmology
using PyCall
@pyimport math
@pyimport numpy as np
println(f(2))
f([1.1,1.2,2,3,5,8])
```

    0.8325546111576977

    6-element Array{Float64,1}:
     0.904122
     0.895875
     0.832555
     0.75966
     0.632458
     0.480453

Notice that the input above both printed an scalar to `STDOUT` and also returned a vector, the latter using Julia's ability to write polymorphic functions and built-in array operations.

On the other hand adding a string to a number is not defined (there is no `+` method defined for those types, although we could easily add one), and attempting to do so will throw an exception:

# f("Hello?")

# Interactive Plotting in IJulia

Below we'll show off some plotting using the excellent Gadfly package. The plots are heavily inspired by [this blog post](http://int8.io/basic-visualization-in-julia-gadfly/).

✨✨✨✨✨

* **NOTE**: This will take a while to compile the first time you import packages. If you know a bit of Julia and have a good solution for how Binder can pre-compile packages *before* the user runs them, let us know in the repo2docker Julia issue here: https://github.com/jupyter/repo2docker/issues/23.

✨✨✨✨✨

```julia
using RDatasets
sleep = dataset("lme4","sleepstudy")
```

```julia
using Gadfly
plot(sleep, x = "Days", y = "Reaction", Geom.point)
```

```julia
plot(sleep, x = "Reaction", Geom.density, color = "Subject", Scale.x_continuous(minvalue= 0, maxvalue= 500))
```

```julia
plot(x = sleep[:Reaction], Geom.histogram(bincount = 30), Scale.x_continuous(minvalue = 200), color = sleep[:Days])
```

## Multimedia display in IJulia

Like most programming languages, Julia has a built-in `print(x)` function for outputting an object `x` as text, and you can override the resulting text representation of a user-defined type by overloading Julia's `show` function.  The next version of Julia, however, will extend this to a more general mechanism to display **arbitrary multimedia representations** of objects, as defined by standard [MIME types](https://en.wikipedia.org/wiki/Internet_media_type).   More specifically, the Julia [multimedia I/O API](http://docs.julialang.org/en/latest/stdlib/base/#multimedia-i-o) provides:

* A `display(x)` function requests the richest available multimedia display of a Julia object x (with a `text/plain` fallback).
* Overloading `writemime` allows one to indicate arbitrary multimedia representations (keyed by standard MIME types) of user-defined types.
* Multimedia-capable display backends may be registered by subclassing a generic `Display` type.  IJulia provides one such backend which, thanks to the IPython notebook, is capable of displaying HTML, LaTeX, SVG, PNG, and JPEG media formats.

The last two points are critical, because they separate **multimedia export** (which is defined by functions associated with the originating Julia data) from **multimedia display** (defined by backends which know nothing about the source of the data).

Precisely these mechanism were used to create the inline PyPlot plots above.   To start with, the simplest thing is to provide the MIME type of the data when you call `display`, which allows you to pass "raw" data in the corresponding format:

```julia
display("text/html", """Hello <b>world</b> in <font color="red">HTML</font>!""")
```

However, it will be more common to attach this information to types, so that they display correctly automatically.  For example, let's define a simple `HTML` type in Julia that contains a string and automatically displays as HTML (given an HTML-capable backend such as IJulia):

```julia
type HTML
   s::String
end
import Base.show
show(io::IO, MIME, x::HTML) = print(io, x.s)
```

    show (generic function with 428 methods)

Here, `show` is just a function that writes `x` in the corresponding format (`text/html`) to the I/O stream `io`.  The `@MIME` is a bit of magic to allow Julia's [multiple dispatch](http://en.wikipedia.org/wiki/Multiple_dispatch) to automatically select the correct `writemime` function for a given MIME type (here `"text/html"`) and object type (here `HTML`).  We also needed an `import` statement in order to add new methods to an existing function from another module.

This `show` definition is all that we need to make any object of type `HTML` display automatically as HTML text in IJulia:

```julia
x = HTML("<ul> <li> Hello from a bulleted list! </ul>")
```

<ul> <li> Hello from a bulleted list! </ul>

```julia
display(x)
#println(x)
```

```julia

```

Julia Jupiter-Notebook Constants in Cosmology.ipynb


# Computationally Visuallizing Crystals
### Christina Lee
### Category: General Audience
### Crystals Series
* [Fundamental Crystal Shapes](../General/Crystal-Shapes.ipynb)
* [MultiSite Unit Cells](../General/MultiSite-Unit-Cells.ipynb)

In condensed matter, we find ourselves in the interesting middle ground of dealing with large numbers $10^{23}$ of extremely small particles, ex. atoms, electrons.

Luckily, the particles don't each do their own thing, but often come in nice, structured, repeated units.  Lattices.  So as our first step into the field, we will look at the most basic type, a Bravais Lattice.

In a Bravais Lattice, every site looks like every other site. Mathematically, we use three vectors, $\vec{a},\vec{b},\vec{c}$ to express how we move from one site to a neighbor.

\begin{equation}
\mathbf{R}_{lmn}=l \vec{a} + m \vec{b} + n \vec{c}  \;\;\;\; \text{for } l,m,n \in \mathbb{N}
\end{equation}

To keep things working out right, we have to put a constraint on these vectors; that we can't get one from scaling and adding the other two.  If we could, then we couldn't put sites in an entire 3 dimensional space.

Stay tuned for a later post where we explore more elaborate lattices.

```julia
# importing our packages
using Plots
gr()
```

    Plots.GRBackend()

## Define The Relevant Variables
Choose the lattice you want to look at, and put that string into the lattice variable.
Current options:

 Simple Cubic = "sc"

 Plane triangular lattice = "pt"

 Body-Centered Cubic = "bcc"

 Face-Centered Cubic = "fcc"

Note: Square is Simple Cubic for Nz=1

14 distinct lattice types are possible, but these common four give the important ideas.

Also, input the size of lattice you want to look at.

```julia
lattice="fcc";

Nx=3;
Ny=2;
Nz=3;
```

## The Lattice vectors

```julia
# A cell to just evaluate
# This one sets the unit vectors (a,b,c) for the different unit cells
# Can you guess what a lattice will look like by looking at the vectors?
if(lattice=="sc")
    d=3;
    a=[1,0,0];
    b=[0,1,0];
    c=[0,0,1];
elseif(lattice=="pt")
    d=2;
    a=[1,0,0];
    b=[.5,sqrt(3)/2,0];
    c=[0,0,1];
elseif(lattice=="bcc")
    d=3;
    a=[.5,.5,.5];
    b=[.5,.5,-.5];
    c=[.5,-.5,.5];
elseif(lattice=="fcc")
    d=3;
    a=[.5,.5,0];
    b=[.5,0,.5];
    c=[0,.5,.5];
end
"Cell Finished"
```

    "Cell Finished"

# Creating the Position arrays

Now we have everything we need about the specific lattice we want to look at.  From here on out, everything is completely general for any Bravais Lattice.

## Some helpful variables
The variables created in the next cell just make computation a bit easier, get the above variables into nicer forms.

```julia
# Another cell to just evaluate
N=Nx*Ny*Nz;    #The total number of sites

#these allow us to copy an entire row or layer at once
aM=transpose(a);
bM=transpose(repeat(b,outer=[1,Nx]));
cM=transpose(repeat(c,outer=[1,Nx*Ny]));

X=Array{Float64}(undef,N,3);  #where we store the positions
"Cell Finished"
```

    "Cell Finished"

## Entering the Position Values
Here, we first initialize a first row, then tile the first plane with the first row.
Afterward, we can tile the entire lattice with that first plane.

```julia
# Another cell to just evaluate
# Here we are actually calculating the positions for every site
for i in 1:Nx    #for the first row
    X[i,:]=(i-1)*a;
end

for j in 2:Ny    #copying the first row into the first layer
    X[Nx*(j-1).+(1:Nx),:]=X[1:Nx,:].+(j-1)*bM;
end

for j in 2:Nz    #copying the first layer into the entire cube
    X[Ny*Nx*(j-1).+(1:Nx*Ny),:]=X[1:Nx*Ny,:].+(j-1)*cM;
end
```

#### Programming Tip:

In Julia, ranges, like `1:Nx`, are a special variable type that can be manipulated.  We can add numbers to them:
`3+(1:3)=4:6`,
or add a minus sign to force it to iterate in the opposite direction, though with different start/stop:
`-(1:3)=-1:-1:-3`

Danger! Make sure to use the parentheses around the range if you are performing these operations.

## Plotting

```julia
plot(legend=false,title=lattice,xlabel="X",ylabel="Y",zlabel="Z")

ls=2
v=collect(0:ls)
zed=zeros(length(v))
for ii in 0:ls
    for jj in 0:ls
        plot!(zed .+ii,v,zed .+jj,linecolor=:black)
        plot!(zed .+ii,zed .+jj,v,linecolor=:black)
        plot!(v, zed .+ii,zed .+jj,linecolor=:black)
    end
end

scatter!(X[:,1],X[:,2],X[:,3])
```

## Go Back and Fiddle!
As you might have noticed, this isn't just a blog where you just read through it. Interact with it. Change some lines, and see what happens. I choose body centered cubic to display first, but what do the other lattices look like?

Look at different lattice sizes.

Can you hand draw them on paper?

```julia

```


# Parameter Identification Technique (PIT) on a linear Nomoto model

# Purpose
Show the general workflow by applying PIT regression on a very simple model

# Methodology
* Load time series from a ZigZag test
* Determine yaw rate and acceleration (compare with accelerometers from test).
* Find the best parameter values in the Nomoto model using OLS linear regression.

# Setup

```python
# %load imports.py
%matplotlib inline
%load_ext autoreload
%autoreload 2

## External packages:
import pandas as pd
pd.options.display.max_rows = 999
pd.options.display.max_columns = 999
pd.set_option("display.max_columns", None)

import numpy as np
import os
import matplotlib.pyplot as plt

import plotly.express as px
import plotly.graph_objects as go

import seaborn as sns
import sympy as sp
from sympy.physics.mechanics import (dynamicsymbols, ReferenceFrame,
                                      Particle, Point)
from sympy.physics.vector.printing import vpprint, vlatex
from IPython.display import display, Math, Latex
from src.substitute_dynamic_symbols import run, lambdify

import pyro

import sklearn
import pykalman
from statsmodels.sandbox.regression.predstd import wls_prediction_std
import statsmodels.api as sm

from scipy.integrate import solve_ivp

## Local packages:
from src.data import mdl

```

## Load test

```python
df_runs = mdl.runs()
```

```python
id=22773
df, units, meta_data = mdl.load(id=id)
```

```python
meta_data.dropna()
```

```python
df.head()
```

```python
units = mdl.load_units()
```

```python
from src.visualization.plot import track_plot
fig,ax=plt.subplots()
fig.set_size_inches(10,10)
track_plot(df=df, lpp=meta_data.lpp, x_dataset='x0', y_dataset='y0',  psi_dataset='psi', beam=meta_data.beam, ax=ax);
```

## Yaw rate

```python
px.line(data_frame=df, y=['psi'], width=1400, height=300,)
```

```python
t = df.index.total_seconds()
df['r_numerical'] = np.gradient(df['psi'],t)
df['r1d_numerical'] = np.gradient(df['r_numerical'],t)

```

## Yaw rate from Kalman filter

```python
from src.data import kalman_filter
```

```python
df = kalman_filter.yaw(df=df)

display(px.line(data_frame=df, y=['psi','psi_filtered'], width=1400, height=300,))
display(px.line(data_frame=df, y=['r_numerical','r'], width=1400, height=300,))
display(px.line(data_frame=df, y=['r1d'], width=1400, height=300,))
```

```python
display(px.line(data_frame=df.resample('2S').mean(), y=['Hull/Acc/X1','Hull/Acc/Y1','Hull/Acc/Y2','Hull/Acc/Z1','Hull/Acc/Z2','Hull/Acc/Z3'], width=1400, height=300,))
```

# First order Nomoto model for ship maneuvering dynamics
The first order Nomoto model can be written as:

```python
K, delta, T_1, T_2 = sp.symbols('K delta T_1 T_2')
r = dynamicsymbols('r')

eq_nomoto = sp.Eq(-K*delta,
                 r + T_1*r1d + T_2*r1d.diff())
Math(vlatex(eq_nomoto))
```

where $r$ is yaw rate with its time derivatives and $\delta$ is the rudder angle. $K$, $T_{1}$
 and $T_{1}$ are the coefficients describing the hydrodynamics of the ship.

For slow manoeuvres this equation can be further simplified by removing the $\ddot{r}$ term:

```python
eq_nomoto_simple = eq_nomoto.subs(r1d.diff(),0)
Math(vlatex(eq_nomoto_simple))
```

```python
eq_r1 = sp.Eq(r1d,
             sp.solve(eq_nomoto_simple, r1d)[0])
r1d_lambda = lambdify(sp.solve(eq_nomoto_simple, r1d)[0])
Math(vlatex(eq_r1))

```

```python
def nomoto(t,states,df_control,params):
    """
    Simulation model for heading and yaw rate using Nomoto with only K and T_1
    """
    # states:
    psi = states[0]
    r = states[1]

    index = df_control.index.get_loc(t, method='nearest')
    delta = float(df_control['delta'].iloc[index])
    r1d_ = r1d_lambda(K=params['K'], T_1=params['T_1'], delta=delta, r=r)
    d_states_dt = np.array([r,r1d_])
    return d_states_dt
```

## Simulate a pull out manoeuvre
Simulating a simple pull out manoeuvre:

```python
params = {
    'K' :  0.1,
    'T_1' : 10,
}

states_0 = np.array([df.iloc[0]['psi'],df.iloc[0]['r']])

df_control = df.copy()
df_control.index = df_control.index.total_seconds()
t = df_control.index

sol = solve_ivp(fun = nomoto,t_span=[t[0],t[-1]],t_eval=t, y0 = states_0, args=(df_control,params))
psi = sol.y[0,:]
r = sol.y[1,:]

fig,ax=plt.subplots()
df_control['psi_deg'] = np.rad2deg(df_control['psi'])
df_control.plot(y='psi_deg', label='model test', ax=ax)
ax.plot(t,np.rad2deg(psi), label='simulation');
ax.grid(True)
ax.set_title('Resimulation with Nomoto model')
ax.set_ylabel('$\psi$ [deg]')
ax.set_xlabel('time [s]');
ax.legend()

```

## Regress nomoto parameters

```python
eq_r1
```

```python
import sympy.physics.mechanics as me

me.find_dynamicsymbols(eq_r1)
```

```python
eq_r1.rhs
```

```python
y = df_control['r1d']
X = df_control[['delta','r']]

#y = df_control['r1d_numerical']
#X = df_control[['delta','r_numerical']]
#X.rename(columns={'r_numerical': 'r'}, inplace=True)

model = sm.OLS(y,X)
results = model.fit()
results.summary()
```

```python
r1d_pred = results.predict(X)
X_ = X.iloc[::100, :]
prstd, iv_l, iv_u = wls_prediction_std(results, exog=X_, alpha=0.05)

fig,ax=plt.subplots()
ax.plot(t,y, label=r'$\dot{r}$')
ax.plot(t,r1d_pred, label=r'$\dot{r}_{pred}$')

ax.plot(X_.index,iv_l, 'k--')
ax.plot(X_.index,iv_u, 'k--')
ax.legend()
```

```python
params['T_1'] = -1/results.params['r']
params['K'] = -results.params['delta']*params['T_1']
```

```python
states_0 = np.array([df.iloc[0]['psi'],df.iloc[0]['r']])

df_control = df.copy()

df_control.index = df_control.index.total_seconds()
t = df_control.index

sol = solve_ivp(fun = nomoto,t_span=[t[0],t[-1]],t_eval=t, y0 = states_0, args=(df_control,params))
psi = sol.y[0,:]
r = sol.y[1,:]

fig,ax=plt.subplots()
df_control['psi_deg'] = np.rad2deg(df_control['psi'])
df_control.plot(y='psi_deg', label='model test', ax=ax)

df_control['-delta_deg'] = -np.rad2deg(df_control['delta'])
df_control.plot(y='-delta_deg', label=r'$-\delta$', ax=ax)

ax.plot(t,np.rad2deg(psi), label='simulation');
ax.grid(True)
ax.set_title('Resimulation with Nomoto model')
ax.set_ylabel('$\psi$ [deg]')
ax.set_xlabel('time [s]');
ax.legend();

```

```python
params
```

```python

```


# Vessel Manoeuvring Models
Many simulation model for ship manoeuvring have been developed in the field of ship hydrodynamics such as: the Abkowitz model {cite:p}`abkowitz_ship_1964` or the Norrbin model {cite:p}`norrbin_study_1960`.
This chapter will develop a general simulation model for ship manoeuvring, that can be further specified to become either the Abkowitz or Norbin model. Expressing the models on a general form is important in this research where many different models will be tested and compared.

```python
# %load imports.py
%load_ext autoreload
%autoreload 2

%config Completer.use_jedi = False  ## (To fix autocomplete)

import pandas as pd
from src.models.vmm import ModelSimulator
import matplotlib.pyplot as plt
import matplotlib
plt.style.use('presentation')
from src.visualization.plot import track_plots, plot, captive_plot
import kedro
import numpy as np
import os.path
import anyconfig

from myst_nb import glue
from src.symbols import *
import src.symbols as symbols
from src.system_equations import *

from IPython.display import display, Math, Latex, Markdown
from sympy.physics.vector.printing import vpprint, vlatex

from src.models.regression import MotionRegression

from src.parameters import df_parameters
p = df_parameters["symbol"]

# Read configs:
conf_path = os.path.join("../../conf/base/")
runs_globals_path = os.path.join(
    conf_path,
    "runs_globals.yml",
)

runs_globals = anyconfig.load(runs_globals_path)
model_test_ids = runs_globals["model_test_ids"]

join_globals_path = os.path.join(
    conf_path,
    "join_globals.yml",
)

joins = runs_globals["joins"]
join_runs_dict = anyconfig.load(join_globals_path)

globals_path = os.path.join(
    conf_path,
    "globals.yml",
)
global_variables = anyconfig.load(globals_path)

vmm_names = global_variables["vmms"]
only_joined = global_variables[
    "only_joined"
]  # (regress/predict with only models from joined runs)S

vmms = {}
for vmm_name in vmm_names:
    vmms[vmm_name] = catalog.load(vmm_name)

```

    2022-03-29 08:08:33,835 - kedro.io.data_catalog - INFO - Loading data from `vmm_abkowitz` (PickleDataSet)...
    2022-03-29 08:08:33,999 - kedro.io.data_catalog - INFO - Loading data from `vmm_abkowitz_simple` (PickleDataSet)...
    2022-03-29 08:08:34,058 - kedro.io.data_catalog - INFO - Loading data from `vmm_martin` (PickleDataSet)...
    2022-03-29 08:08:34,108 - kedro.io.data_catalog - INFO - Loading data from `vmm_linear` (PickleDataSet)...
    2022-03-29 08:08:34,146 - kedro.io.data_catalog - INFO - Loading data from `vmm_martins_simple` (PickleDataSet)...
    2022-03-29 08:08:34,185 - kedro.io.data_catalog - INFO - Loading data from `vmm_abkowitz_expanded` (PickleDataSet)...

3DOF system for manoeurving:

```python
eq_system
```

$\displaystyle \left[\begin{matrix}- X_{\dot{u}} + m & 0 & 0\\0 & - Y_{\dot{v}} + m & - Y_{\dot{r}} + m x_{G}\\0 & - N_{\dot{v}} + m x_{G} & I_{z} - N_{\dot{r}}\end{matrix}\right] \left[\begin{matrix}\dot{u}\\\dot{v}\\\dot{r}\end{matrix}\right] = \left[\begin{matrix}m r^{2} x_{G} + m r v + \operatorname{X_{D}}{\left(u,v,r,\delta,thrust \right)}\\- m r u + \operatorname{Y_{D}}{\left(u,v,r,\delta,thrust \right)}\\- m r u x_{G} + \operatorname{N_{D}}{\left(u,v,r,\delta,thrust \right)}\end{matrix}\right]$

The manoeuvring simulation can now be conducted by numerical integration of the above equation. The main difference between various vessel manoeuvring models such as the Abkowitz model {cite:p}`abkowitz_ship_1964` or the Norrbin model {cite:p}`norrbin_study_1960` lies in how the hydrodynamic functions $X_D(u,v,r,\delta,thrust)$, $Y_D(u,v,r,\delta,thrust)$, $N_D(u,v,r,\delta,thrust)$ are defined. These functions cane be found in [Appendix](appendix_vmms.md).

Note that a coefficient $X_{thrust}$ has been added to the Abkowitz X equation to allow for propeller thrust as an input to the model.

```python
vmms['vmm_abkowitz'].Y_qs_eq
```

$\displaystyle \operatorname{Y_{D}}{\left(u,v,r,\delta,thrust \right)} = Y_{0uu} u^{2} + Y_{0u} u + Y_{0} + Y_{deltadeltadelta} \delta^{3} + Y_{delta} \delta + Y_{rdeltadelta} \delta^{2} r + Y_{rrdelta} \delta r^{2} + Y_{rrr} r^{3} + Y_{r} r + Y_{udelta} \delta u + Y_{ur} r u + Y_{uudelta} \delta u^{2} + Y_{uur} r u^{2} + Y_{uuv} u^{2} v + Y_{uv} u v + Y_{vdeltadelta} \delta^{2} v + Y_{vrdelta} \delta r v + Y_{vrr} r^{2} v + Y_{vvdelta} \delta v^{2} + Y_{vvr} r v^{2} + Y_{vvv} v^{3} + Y_{v} v$

```python
vmms['vmm_linear'].Y_qs_eq
```

$\displaystyle \operatorname{Y_{D}}{\left(u,v,r,\delta,thrust \right)} = Y_{delta} \delta + Y_{r} r + Y_{u} u + Y_{v} v$

This equation can be rewritten to get the acceleration on the left hand side:

```python
eq_acceleration_matrix_clean
```

$\displaystyle \dot{\nu} = \left[\begin{matrix}\dot{u}\\\dot{v}\\\dot{r}\end{matrix}\right] = \left[\begin{matrix}\frac{1}{- X_{\dot{u}} + m} & 0 & 0\\0 & - \frac{- I_{z} + N_{\dot{r}}}{S} & - \frac{- Y_{\dot{r}} + m x_{G}}{S}\\0 & - \frac{- N_{\dot{v}} + m x_{G}}{S} & - \frac{Y_{\dot{v}} - m}{S}\end{matrix}\right] \left[\begin{matrix}m r^{2} x_{G} + m r v + \operatorname{X_{D}}{\left(u,v,r,\delta,thrust \right)}\\- m r u + \operatorname{Y_{D}}{\left(u,v,r,\delta,thrust \right)}\\- m r u x_{G} + \operatorname{N_{D}}{\left(u,v,r,\delta,thrust \right)}\end{matrix}\right]$

where $S$ is a helper variable:

```python
eq_S
```

$\displaystyle S = - I_{z} Y_{\dot{v}} + I_{z} m + N_{\dot{r}} Y_{\dot{v}} - N_{\dot{r}} m - N_{\dot{v}} Y_{\dot{r}} + N_{\dot{v}} m x_{G} + Y_{\dot{r}} m x_{G} - m^{2} x_{G}^{2}$

A state space model for manoeuvring can now be defined with six states:

```python
eq_x
```

$\displaystyle \vec{x} = \left[\begin{matrix}x_{0}\\y_{0}\\\Psi\\u\\v\\r\end{matrix}\right]$

An transition function $f$ defines how the states changes with time:

```python
eq_state_space
```

$\displaystyle \dot{\vec{x}} = f{\left(\vec{x},u_{input},w_{noise} \right)}$

Using geometrical relations for how $x_0$, $y_0$ and $\Psi$ depend on $u$, $v$, and $r$ and the time derivatives that was derived above: $\dot{u}$, $\dot{v}$, $\dot{r}$, the transition function can be written:

```python
eq_f
```

$\displaystyle f{\left(\vec{x},u_{input},w_{noise} \right)} = \left[\begin{matrix}u \cos{\left(\Psi \right)} - v \sin{\left(\Psi \right)}\\u \sin{\left(\Psi \right)} + v \cos{\left(\Psi \right)}\\r\\\dot{u}\\\dot{v}\\\dot{r}\end{matrix}\right]$

```python
%reload_kedro
vmm_name = 'vmm_martins_simple'
model = catalog.load(f"{ vmm_name}.motion_regression.joined.model")
vmm = catalog.load(f"{ vmm_name }")
initial_parameters = catalog.load("initial_parameters")
model.parameters=initial_parameters

id = 22773
ship_data = catalog.load("ship_data")
data = catalog.load(f"{ id }.data_ek_smooth")
```

    2022-03-29 08:08:36,025 - kedro.framework.session.store - INFO - `read()` not implemented for `SQLiteStore`. Assuming empty store.
    2022-03-29 08:08:38,077 - root - INFO - ** Kedro project wPCC_pipeline
    2022-03-29 08:08:38,080 - root - INFO - Defined global variable `context`, `session`, `catalog` and `pipelines`
    2022-03-29 08:08:38,089 - root - INFO - Registered line magic `run_viz`
    2022-03-29 08:08:38,090 - kedro.io.data_catalog - INFO - Loading data from `vmm_martins_simple.motion_regression.joined.model` (PickleDataSet)...
    2022-03-29 08:08:38,097 - kedro.io.data_catalog - INFO - Loading data from `vmm_martins_simple` (PickleDataSet)...
    2022-03-29 08:08:38,102 - kedro.io.data_catalog - INFO - Loading data from `initial_parameters` (YAMLDataSet)...
    2022-03-29 08:08:38,108 - kedro.io.data_catalog - INFO - Loading data from `ship_data` (YAMLDataSet)...
    2022-03-29 08:08:38,110 - kedro.io.data_catalog - INFO - Loading data from `22773.data_ek_smooth` (CSVDataSet)...

```python
#t = np.arange(0, 70, 0.01)
#input_columns = ['delta','U','thrust']
#state_columns = ['x0', 'y0', 'psi', 'u', 'v', 'r']
#data = pd.DataFrame(index=t, columns=state_columns + input_columns)
#data['u'] = 2
#data['delta'] = np.deg2rad(-35)
#data['thrust'] = 30
#data.fillna(0, inplace=True)
#data['U'] = np.sqrt(data['u']**2 + data['v']**2)
#

result = model.simulate(df_=data)

dataframes = {'simulation': result.result,
             'model test' : data}
styles = {}
styles['model test'] = {'style':'k-', 'alpha':1, 'lw':1.5}
styles['simulation'] = {'style':'r-', 'alpha':1, 'lw':1.5}

#dataframes['simulate'] = ek.simulate(data=data, input_columns=input_columns, solver='Radau')

fig,ax=plt.subplots()
fig.set_size_inches(matplotlib.rcParams["figure.figsize"][0]*0.4, matplotlib.rcParams["figure.figsize"][1])

track_plots(dataframes, lpp=model.ship_parameters['L'], beam=model.ship_parameters['B'], N=7,
            styles=styles, ax=ax);

result.result.to_csv('example.csv')
```


# Quiz -Week 6A

##Q1.

* The figure below shows two positive points (purple squares) and two negative points (green circles):

* That is, the training data set consists of:

    * (x1,y1) = ((5,4),+1)
    * (x2,y2) = ((8,3),+1)
    * (x3,y3) = ((7,2),-1)
    * (x4,y4) = ((3,3),-1)

* Our goal is to find the __maximum-margin linear classifier__ for this data. In easy cases, the shortest line between a positive and negative point has a __perpendicular bisector that separates the points__. If so, the perpendicular bisector is surely the maximum-margin separator. Alas, in this case, the closest pair of positive and negative points, x2 and x3, have a perpendicular bisector that misclassifies x1 as negative, so that won't work.

* The __next-best possibility__ is that we can find a pair of points on one side (i.e., either two positive or two negative points) such that __a line parallel to the line through these points__ is the maximum-margin separator. In these cases, the limit to how far from the two points the parallel line can get is determined by the closest (to the line between the two points) of the points on the other side. For our simple data set, this situation holds.

* Consider all possibilities for boundaries of this type, and express the boundary as w.x+b=0, such that w.x+b≥1 for positive points x and w.x+b≤-1 for negative points x. Assuming that w = (w1,w2), identify in the list below the true statement about one of w1, w2, and b.

```python
import numpy as np

p1 = (5, 4)
p2 = (8, 3)
p3 = (7, 2)
p4 = (3, 3)

def calc_wb(p1, p2):
    dx = ( p1[0] - p2[0] )
    dy = ( p1[1] - p2[1] )
    return ( ( float(dy) *2 / float(dy - dx), float(-dx)*2 / float(dy - dx) ),\
            (dx*p2[1] - dy * p2[0])*2 / float(dy - dx) + 1) # b = dx*y1 - dy*x1

def cal_margin(w, b, pt):
    return w[0] * pt[0] + w[1] * pt[1] + b

w, b = calc_wb(p1, p2)
print "w for p1, p2: " + str(w)
print "b for p1, p2: " + str(b)
print "==========================="
print cal_margin(w, b, p1)
print cal_margin(w, b, p2)
print cal_margin(w, b, p3)
print cal_margin(w, b, p4)
print

w, b = calc_wb(p4, p3)
print "w for p1, p2: " + str(w)
print "b for p1, p2: " + str(b)
print "==========================="
print cal_margin(w, b, p1)
print cal_margin(w, b, p2)
print cal_margin(w, b, p3)
print cal_margin(w, b, p4)

```

    w for p1, p2: (0.5, 1.5)
    b for p1, p2: -7.5
    ===========================
    1.0
    1.0
    -1.0
    -1.5

    w for p1, p2: (0.4, 1.6)
    b for p1, p2: -5.0
    ===========================
    3.4
    3.0
    1.0
    1.0

##Q2.

* Consider the following training set of 16 points. The eight purple squares are positive examples, and the eight green circles are negative examples.

* We propose to use the diagonal line with slope +1 and intercept +2 as a decision boundary, with positive examples above and negative examples below. However, like any linear boundary for this training set, some examples are misclassified. We can measure the goodness of the boundary by computing all the slack variables that exceed 0, and then using them in one of several objective functions. In this problem, we shall only concern ourselves with computing the slack variables, not an objective function.

* To be specific, suppose the boundary is written in the form w.x+b=0, where w = (-1,1) and b = -2. Note that we can scale the three numbers involved as we wish, and so doing changes the margin around the boundary. However, we want to consider this specific boundary and margin.

* Determine the slack for each of the 16 points. Then, identify the correct statement in the list below.

```python
w = (-1, 1)
b = -2

def cal_margin(w, b, pt):
    return w[0] * pt[0] + w[1] * pt[1] + b

print cal_margin(w, b, (7,  10) )
print cal_margin(w, b, (7,  8)  )
print cal_margin(w, b, (3,  4)  )
print cal_margin(w, b, (3,  4)  )
```

    1
    -1
    -1
    -1

##Q3.

* Below we see a set of 20 points and a decision tree for classifying the points.

* To be precise, the 20 points represent (Age,Salary) pairs of people who do or do not buy gold jewelry. Age (appreviated A in the decision tree) is the x-axis, and Salary (S in the tree) is the y-axis. Those that do are represented by gold points, and those that do not by green points. The 10 points of gold-jewelry buyers are:

* (28,145), (38,115), (43,83), (50,130), (50,90), (50,60), (50,30), (55,118), (63,88), and (65,140).

* The 10 points of those that do not buy gold jewelry are:

* (23,40), (25,125), (29,97), (33,22), (35,63), (42,57), (44, 105), (55,63), (55,20), and (64,37).

* Some of these points are correctly classified by the decision tree and some are not. Determine the classification of each point, and then indicate in the list below the point that is misclassified.

```python
def predict_by_tree(pt):
    if pt[0] < 45:
        if pt[1] < 110:
            print "Doesn't buy"
        else:
            print "Buy"
    else:
        if pt[1] < 75:
            print "Doesn't buy"
        else:
            print "Buy"

predict_by_tree((43, 83))
predict_by_tree((55, 118))
predict_by_tree((65, 140))
predict_by_tree((28, 145))

print "=============="
predict_by_tree((65, 140))
predict_by_tree((25, 125))
predict_by_tree((44, 105))
predict_by_tree((35, 63))
```

    Doesn't buy
    Buy
    Buy
    Buy
    ==============
    Buy
    Buy
    Doesn't buy
    Doesn't buy

# Quiz Week 6A.

## Q1.

* Using the matrix-vector multiplication described in Section 2.3.1, applied to the matrix and vector:

    | 1   2   3   4  |   | 1 |
    | 5   6   7   8  | * | 2 |
    | 9   10  11  12 |   | 3 |
    | 13  14  15  16 |   | 4 |

* Apply the Map function to this matrix and vector. Then, identify in the list below, one of the key-value pairs that are output of Map.

## Solution 1.

The matrix-vector product is the vector x of length n, whose ith element xi is given by

$$
\begin{equation}
    x_i =􏰅 \sum_{ j = 1}^n m_{ij} \cdot v_j
\end{equation}
$$

* From each matrix element mij it produces the key-value pair ( i, $m_{ij} \cdot v_j$ ).
* Thus, all terms of the sum that make up the component $x_i$ of the matrix-vector product will get the same key, i.

```python
import numpy as np

mat = np.array([ [1,  2, 3, 4],
                 [5,  6, 7, 8],
                 [9, 10,11,12],
                 [13,14,15,16] ])
vec = np.array([1, 2, 3, 4])

def key_val(mat, vec):
    pair = dict()
    for idx, row in enumerate(mat):
#         pair[idx + 1] = np.dot(row, vec)
        pair[idx + 1] = row * vec

    return pair

print key_val(mat, vec)
```

    {1: array([ 1,  4,  9, 16]), 2: array([ 5, 12, 21, 32]), 3: array([ 9, 20, 33, 48]), 4: array([13, 28, 45, 64])}

## Q2.
* Suppose we use the algorithm of __Section 2.3.10__ to compute the __product of matrices M and N__. Let M have x rows and y columns, while N has y rows and z columns. As a function of x, y, and z, express the answers to the following questions:
* The output of __the Map function__ has __how many different keys__? How many __key-value pairs__ are there with each key? How many __key-value pairs are there in all__?
* The input to the Reduce function has how many keys? What is the length of the value (a list) associated with each key?

## Solution 2.

1. Diffrent keys output of Map function => $x * z$
2. Key-value pairs with each key => $2 * y$
3. Key value pairs in all => $2 * x * y * z$
4. Key Input to Reduce function =>
5. Length of value list = $2 * y$

##Q3.
* Suppose we use the __two-stage algorithm of Section 2.3.9__ to compute the product of matrices M and N. Let M have x rows and y columns, while N has y rows and z columns. As a function of x, y, and z, express the answers to the following questions:
* The output of the first Map function has __how many different keys__? How many key-value pairs are there with each key? How many key-value pairs are there in all?
* The output of the __first Reduce function has how many keys__? What is __the length of the value (a list) associated with each key__?
* The output of the second Map function has how many different keys? How many key-value pairs are there with each key? How many key-value pairs are there in all?
* Then, identify the true statement in the list below.

## Solution 3.

1. Different keys of first map => y
2. Different key-value pairs of each key => x + z
3. Key-value pairs in all => $( y * x + y * z)$
4. Keys of first reduce output => $x * z$
5. Length of value list with each key => y
6. Different keys of second map function => $x * z$
7. Pairs for each key => y
8. Pairs in all => $x * y * z$

## Q4.

* Suppose we have the following relations:

       R               S

    __A__		__B__    __B__		__C__
    0		1    0		1
    1		2    1		2
    2		3    2		3

* and we take their __natural join__ by the algorithm of Section 2.3.7. Apply the Map function to the tuples of these relations. Then, construct the elements that are input to the Reduce function. Identify one of these elements in the list below.

* Map Results:

    (1, (R, 0))
    (2, (R, 1))
    (3, (R, 2))
    (0, (S, 1))
    (1, (S, 2))
    (2, (S, 3))

```python

```


# Trading Lists and Trading Trajectories

### Introduction

[Almgren and Chriss](https://cims.nyu.edu/~almgren/papers/optliq.pdf) provided a solution to the optimal liquidation problem by assuming the that stock prices follow a discrete arithmetic random walk, and that the permanent and temporary market impact functions are linear functions of the trading rate.

Almgren and Chriss showed that for each value of risk aversion there is a unique optimal execution strategy. This optimal execution strategy is determined by a trading trajectory and its associated trading list. The optimal trading trajectory is given by:

\begin{equation}
x_j = \frac{\sinh \left( \kappa \left( T-t_j\right)\right)}{ \sinh (\kappa T)}X, \hspace{1cm}\text{ for } j=0,...,N
\end{equation}

and the associated trading list is given by:

\begin{equation}
n_j = \frac{2 \sinh \left(\frac{1}{2} \kappa \tau \right)}{ \sinh \left(\kappa T\right) } \cosh \left(\kappa \left(T - t_{j-\frac{1}{2}}\right)\right) X, \hspace{1cm}\text{ for } j=1,...,N
\end{equation}

where $t_{j-1/2} = (j-\frac{1}{2}) \tau$.

Given some initial parameters, such as the number of shares, the liquidation time, the trader's risk aversion, etc..., the trading list will tell us how many shares we should sell at each trade to minimize our transaction costs.

In this notebook, we will see how the trading list varies according to some initial trading parameters.

## Visualizing Trading Lists and Trading Trajectories

Let's assume we have 1,000,000 shares that we wish to liquidate. In the code below, we will plot the optimal trading trajectory and its associated trading list for different trading parameters, such as trader's risk aversion, number of trades, and liquidation time.

```python
%matplotlib inline

import matplotlib.pyplot as plt

import utils

# We set the default figure size
plt.rcParams['figure.figsize'] = [17.0, 7.0]

# Set the number of days to sell all shares (i.e. the liquidation time)
l_time = 60

# Set the number of trades
n_trades = 60

# Set the trader's risk aversion
t_risk = 1e-6

# Plot the trading list and trading trajectory. If show_trl = True, the data frame containing the values of the
# trading list and trading trajectory is printed
utils.plot_trade_list(lq_time = l_time, nm_trades = n_trades, tr_risk = t_risk, show_trl = True)
```

# Implementing a Trading List

Once we have the trading list for a given set of initial parameters, we can actually implement it. That is, we can sell our shares in the stock market according to the trading list and see how much money we made or lost. To do this, we are going to simulate the stock market with a simple trading environment. This simulated trading environment uses the same price dynamics and market impact functions as the Almgren and Chriss model. That is, stock price movements evolve according to a discrete arithmetic random walk and the permanent and temporary market impact functions are linear functions of the trading rate. We are going to use the same environment to train our Deep Reinforcement Learning algorithm later on.

We will describe the details of the trading environment in another notebook, for now we will just take a look at its default parameters. We will distinguish between financial parameters, such the annual volatility in stock price, and the parameters needed to calculate the trade list using the Almgren and Criss model, such as the trader's risk aversion.

```python
import utils

# Get the default financial and AC Model parameters
financial_params, ac_params = utils.get_env_param()
```

### Default Financial Parameters

```python
financial_params
```

Financial Parameters

  Annual Volatility:	  12%	   Bid-Ask Spread:    	     0.125

  Daily Volatility:	  0.8%	   Daily Trading Volume:	 5,000,000

### Parameters for the Almgren and Chriss Model

```python
ac_params
```

Almgren and Chriss Model Parameters

  Total Number of Shares to Sell:	                  1,000,000	   Fixed Cost of Selling per Share:	 $0.062

  Starting Price per Share:	                         $50.00	     Trader's Risk Aversion:	           1e-06

  Price Impact for Each 1% of Daily Volume Traded:	 $2.5e-06	    Permanent Impact Constant:	       2.5e-07

  Number of Days to Sell All the Shares:	              60	       Single Step Variance:	             0.144

  Number of Trades:	                                   60	       Time Interval between trades:	      1.0

The code below implements the trading list resulting from different trading parameters, such as trader's risk aversion, number of trades, and liquidation time. All other parameters, such as total number shares to sell, are taken from the simulated trading environment (see above).

```python
%matplotlib inline

import matplotlib.pyplot as plt

import utils

# We set the default figure size
plt.rcParams['figure.figsize'] = [17.0, 7.0]

# Set the random seed
sd = 0

# Set the number of days to sell all shares (i.e. the liquidation time)
l_time = 60

# Set the number of trades
n_trades = 60

# Set the trader's risk aversion
t_risk = 1e-6

# Implement the trading list for the given parameters
utils.implement_trade_list(seed = sd, lq_time = l_time, nm_trades = n_trades, tr_risk = t_risk)
```

11. Deep RL for Finance - 1/.ipynb_checkpoints/02. Trading Lists-checkpoint.ipynb


```python
import sympy as sym
import numpy as np
```

```python
alpha
```

    (-0.4999999999999998+0.8660254037844387j)

```python
#alpha = np.exp(2.0/3*np.pi*1j)
a_r,a_i = sym.symbols('a_r,a_i',real=True)
alpha = a_r + sym.I*a_i
A_0a =  sym.Matrix([[1, 1, 1],
                  [1, alpha**2, alpha],
                  [1, alpha, alpha**2]])

A_a0 = 1/3* sym.Matrix([[1, 1, 1],
                                     [1, alpha, alpha**2],
                                     [1, alpha**2, alpha]])

```

```python
v_a_r,v_b_r,v_c_r = sym.symbols('v_a_r,v_b_r,v_c_r',real=True)
v_a_i,v_b_i,v_c_i = sym.symbols('v_a_i,v_b_i,v_c_i',real=True)
v_abc = sym.Matrix([[v_a_r+sym.I*v_a_i],[v_b_r+sym.I*v_b_i],[v_c_r+sym.I*v_c_i]])
i_a_r,i_b_r,i_c_r = sym.symbols('i_a_r,i_b_r,i_c_r',real=True)
i_a_i,i_b_i,i_c_i = sym.symbols('i_a_i,i_b_i,i_c_i',real=True)
i_abc = sym.Matrix([[i_a_r+sym.I*i_a_i],[i_b_r+sym.I*i_b_i],[i_c_r+sym.I*i_c_i]])
```

```python
v_opn = A_a0*v_abc
i_opn = A_a0*i_abc
```

```python
v_0 = v_opn[0]
v_p = v_opn[1]
v_n = v_opn[2]
i_0 = i_opn[0]
i_p = i_opn[1]
i_n = i_opn[2]
```

```python
sym.re(v_p)
```

$\displaystyle - 0.666666666666667 a_{i} a_{r} v_{c i} - 0.333333333333333 a_{i} v_{b i} + 0.333333333333333 a_{r} v_{b r} + 0.333333333333333 v_{a r} + v_{c r} \left(- 0.333333333333333 a_{i}^{2} + 0.333333333333333 a_{r}^{2}\right)$

```python
s_abc = sym.conjugate(i_abc.T)*(v_abc)
s_opn = 3*sym.conjugate(i_opn.T)*(v_opn)
```

```python
sym.simplify(sym.expand(s_abc[0]) - sym.expand(s_opn[0]))
```

$\displaystyle - 0.333333333333333 a_{i}^{4} i_{b i} v_{b i} + 0.333333333333333 i a_{i}^{4} i_{b i} v_{b r} - 0.333333333333333 i a_{i}^{4} i_{b r} v_{b i} - 0.333333333333333 a_{i}^{4} i_{b r} v_{b r} - 0.333333333333333 a_{i}^{4} i_{c i} v_{c i} + 0.333333333333333 i a_{i}^{4} i_{c i} v_{c r} - 0.333333333333333 i a_{i}^{4} i_{c r} v_{c i} - 0.333333333333333 a_{i}^{4} i_{c r} v_{c r} - 0.666666666666667 a_{i}^{2} a_{r}^{2} i_{b i} v_{b i} + 0.666666666666667 i a_{i}^{2} a_{r}^{2} i_{b i} v_{b r} - 0.666666666666667 i a_{i}^{2} a_{r}^{2} i_{b r} v_{b i} - 0.666666666666667 a_{i}^{2} a_{r}^{2} i_{b r} v_{b r} - 0.666666666666667 a_{i}^{2} a_{r}^{2} i_{c i} v_{c i} + 0.666666666666667 i a_{i}^{2} a_{r}^{2} i_{c i} v_{c r} - 0.666666666666667 i a_{i}^{2} a_{r}^{2} i_{c r} v_{c i} - 0.666666666666667 a_{i}^{2} a_{r}^{2} i_{c r} v_{c r} - 0.666666666666667 a_{i}^{2} a_{r} i_{b i} v_{c i} + 0.666666666666667 i a_{i}^{2} a_{r} i_{b i} v_{c r} - 0.666666666666667 i a_{i}^{2} a_{r} i_{b r} v_{c i} - 0.666666666666667 a_{i}^{2} a_{r} i_{b r} v_{c r} - 0.666666666666667 a_{i}^{2} a_{r} i_{c i} v_{b i} + 0.666666666666667 i a_{i}^{2} a_{r} i_{c i} v_{b r} - 0.666666666666667 i a_{i}^{2} a_{r} i_{c r} v_{b i} - 0.666666666666667 a_{i}^{2} a_{r} i_{c r} v_{b r} + 0.333333333333333 a_{i}^{2} i_{a i} v_{b i} - 0.333333333333333 i a_{i}^{2} i_{a i} v_{b r} + 0.333333333333333 a_{i}^{2} i_{a i} v_{c i} - 0.333333333333333 i a_{i}^{2} i_{a i} v_{c r} + 0.333333333333333 i a_{i}^{2} i_{a r} v_{b i} + 0.333333333333333 a_{i}^{2} i_{a r} v_{b r} + 0.333333333333333 i a_{i}^{2} i_{a r} v_{c i} + 0.333333333333333 a_{i}^{2} i_{a r} v_{c r} + 0.333333333333333 a_{i}^{2} i_{b i} v_{a i} - 0.333333333333333 i a_{i}^{2} i_{b i} v_{a r} - 0.333333333333333 a_{i}^{2} i_{b i} v_{b i} + 0.333333333333333 i a_{i}^{2} i_{b i} v_{b r} + 0.333333333333333 i a_{i}^{2} i_{b r} v_{a i} + 0.333333333333333 a_{i}^{2} i_{b r} v_{a r} - 0.333333333333333 i a_{i}^{2} i_{b r} v_{b i} - 0.333333333333333 a_{i}^{2} i_{b r} v_{b r} + 0.333333333333333 a_{i}^{2} i_{c i} v_{a i} - 0.333333333333333 i a_{i}^{2} i_{c i} v_{a r} - 0.333333333333333 a_{i}^{2} i_{c i} v_{c i} + 0.333333333333333 i a_{i}^{2} i_{c i} v_{c r} + 0.333333333333333 i a_{i}^{2} i_{c r} v_{a i} + 0.333333333333333 a_{i}^{2} i_{c r} v_{a r} - 0.333333333333333 i a_{i}^{2} i_{c r} v_{c i} - 0.333333333333333 a_{i}^{2} i_{c r} v_{c r} - 0.666666666666667 i a_{i} a_{r} i_{a i} v_{b i} - 0.666666666666667 a_{i} a_{r} i_{a i} v_{b r} - 0.666666666666667 i a_{i} a_{r} i_{a i} v_{c i} - 0.666666666666667 a_{i} a_{r} i_{a i} v_{c r} + 0.666666666666667 a_{i} a_{r} i_{a r} v_{b i} - 0.666666666666667 i a_{i} a_{r} i_{a r} v_{b r} + 0.666666666666667 a_{i} a_{r} i_{a r} v_{c i} - 0.666666666666667 i a_{i} a_{r} i_{a r} v_{c r} + 0.666666666666667 i a_{i} a_{r} i_{b i} v_{a i} + 0.666666666666667 a_{i} a_{r} i_{b i} v_{a r} - 0.666666666666667 a_{i} a_{r} i_{b r} v_{a i} + 0.666666666666667 i a_{i} a_{r} i_{b r} v_{a r} + 0.666666666666667 i a_{i} a_{r} i_{c i} v_{a i} + 0.666666666666667 a_{i} a_{r} i_{c i} v_{a r} - 0.666666666666667 a_{i} a_{r} i_{c r} v_{a i} + 0.666666666666667 i a_{i} a_{r} i_{c r} v_{a r} - 0.333333333333333 i a_{i} i_{a i} v_{b i} - 0.333333333333333 a_{i} i_{a i} v_{b r} - 0.333333333333333 i a_{i} i_{a i} v_{c i} - 0.333333333333333 a_{i} i_{a i} v_{c r} + 0.333333333333333 a_{i} i_{a r} v_{b i} - 0.333333333333333 i a_{i} i_{a r} v_{b r} + 0.333333333333333 a_{i} i_{a r} v_{c i} - 0.333333333333333 i a_{i} i_{a r} v_{c r} + 0.333333333333333 i a_{i} i_{b i} v_{a i} + 0.333333333333333 a_{i} i_{b i} v_{a r} - 0.333333333333333 a_{i} i_{b r} v_{a i} + 0.333333333333333 i a_{i} i_{b r} v_{a r} + 0.333333333333333 i a_{i} i_{c i} v_{a i} + 0.333333333333333 a_{i} i_{c i} v_{a r} - 0.333333333333333 a_{i} i_{c r} v_{a i} + 0.333333333333333 i a_{i} i_{c r} v_{a r} - 0.333333333333333 a_{r}^{4} i_{b i} v_{b i} + 0.333333333333333 i a_{r}^{4} i_{b i} v_{b r} - 0.333333333333333 i a_{r}^{4} i_{b r} v_{b i} - 0.333333333333333 a_{r}^{4} i_{b r} v_{b r} - 0.333333333333333 a_{r}^{4} i_{c i} v_{c i} + 0.333333333333333 i a_{r}^{4} i_{c i} v_{c r} - 0.333333333333333 i a_{r}^{4} i_{c r} v_{c i} - 0.333333333333333 a_{r}^{4} i_{c r} v_{c r} - 0.666666666666667 a_{r}^{3} i_{b i} v_{c i} + 0.666666666666667 i a_{r}^{3} i_{b i} v_{c r} - 0.666666666666667 i a_{r}^{3} i_{b r} v_{c i} - 0.666666666666667 a_{r}^{3} i_{b r} v_{c r} - 0.666666666666667 a_{r}^{3} i_{c i} v_{b i} + 0.666666666666667 i a_{r}^{3} i_{c i} v_{b r} - 0.666666666666667 i a_{r}^{3} i_{c r} v_{b i} - 0.666666666666667 a_{r}^{3} i_{c r} v_{b r} - 0.333333333333333 a_{r}^{2} i_{a i} v_{b i} + 0.333333333333333 i a_{r}^{2} i_{a i} v_{b r} - 0.333333333333333 a_{r}^{2} i_{a i} v_{c i} + 0.333333333333333 i a_{r}^{2} i_{a i} v_{c r} - 0.333333333333333 i a_{r}^{2} i_{a r} v_{b i} - 0.333333333333333 a_{r}^{2} i_{a r} v_{b r} - 0.333333333333333 i a_{r}^{2} i_{a r} v_{c i} - 0.333333333333333 a_{r}^{2} i_{a r} v_{c r} - 0.333333333333333 a_{r}^{2} i_{b i} v_{a i} + 0.333333333333333 i a_{r}^{2} i_{b i} v_{a r} - 0.333333333333333 a_{r}^{2} i_{b i} v_{b i} + 0.333333333333333 i a_{r}^{2} i_{b i} v_{b r} - 0.333333333333333 i a_{r}^{2} i_{b r} v_{a i} - 0.333333333333333 a_{r}^{2} i_{b r} v_{a r} - 0.333333333333333 i a_{r}^{2} i_{b r} v_{b i} - 0.333333333333333 a_{r}^{2} i_{b r} v_{b r} - 0.333333333333333 a_{r}^{2} i_{c i} v_{a i} + 0.333333333333333 i a_{r}^{2} i_{c i} v_{a r} - 0.333333333333333 a_{r}^{2} i_{c i} v_{c i} + 0.333333333333333 i a_{r}^{2} i_{c i} v_{c r} - 0.333333333333333 i a_{r}^{2} i_{c r} v_{a i} - 0.333333333333333 a_{r}^{2} i_{c r} v_{a r} - 0.333333333333333 i a_{r}^{2} i_{c r} v_{c i} - 0.333333333333333 a_{r}^{2} i_{c r} v_{c r} - 0.333333333333333 a_{r} i_{a i} v_{b i} + 0.333333333333333 i a_{r} i_{a i} v_{b r} - 0.333333333333333 a_{r} i_{a i} v_{c i} + 0.333333333333333 i a_{r} i_{a i} v_{c r} - 0.333333333333333 i a_{r} i_{a r} v_{b i} - 0.333333333333333 a_{r} i_{a r} v_{b r} - 0.333333333333333 i a_{r} i_{a r} v_{c i} - 0.333333333333333 a_{r} i_{a r} v_{c r} - 0.333333333333333 a_{r} i_{b i} v_{a i} + 0.333333333333333 i a_{r} i_{b i} v_{a r} - 0.333333333333333 i a_{r} i_{b r} v_{a i} - 0.333333333333333 a_{r} i_{b r} v_{a r} - 0.333333333333333 a_{r} i_{c i} v_{a i} + 0.333333333333333 i a_{r} i_{c i} v_{a r} - 0.333333333333333 i a_{r} i_{c r} v_{a i} - 0.333333333333333 a_{r} i_{c r} v_{a r} - 0.333333333333333 i_{a i} v_{b i} + 0.333333333333333 i i_{a i} v_{b r} - 0.333333333333333 i_{a i} v_{c i} + 0.333333333333333 i i_{a i} v_{c r} - 0.333333333333333 i i_{a r} v_{b i} - 0.333333333333333 i_{a r} v_{b r} - 0.333333333333333 i i_{a r} v_{c i} - 0.333333333333333 i_{a r} v_{c r} - 0.333333333333333 i_{b i} v_{a i} + 0.333333333333333 i i_{b i} v_{a r} + 0.666666666666667 i_{b i} v_{b i} - 0.666666666666667 i i_{b i} v_{b r} - 0.333333333333333 i_{b i} v_{c i} + 0.333333333333333 i i_{b i} v_{c r} - 0.333333333333333 i i_{b r} v_{a i} - 0.333333333333333 i_{b r} v_{a r} + 0.666666666666667 i i_{b r} v_{b i} + 0.666666666666667 i_{b r} v_{b r} - 0.333333333333333 i i_{b r} v_{c i} - 0.333333333333333 i_{b r} v_{c r} - 0.333333333333333 i_{c i} v_{a i} + 0.333333333333333 i i_{c i} v_{a r} - 0.333333333333333 i_{c i} v_{b i} + 0.333333333333333 i i_{c i} v_{b r} + 0.666666666666667 i_{c i} v_{c i} - 0.666666666666667 i i_{c i} v_{c r} - 0.333333333333333 i i_{c r} v_{a i} - 0.333333333333333 i_{c r} v_{a r} - 0.333333333333333 i i_{c r} v_{b i} - 0.333333333333333 i_{c r} v_{b r} + 0.666666666666667 i i_{c r} v_{c i} + 0.666666666666667 i_{c r} v_{c r}$

```python
sym.simplify(sym.re(sym.expand(s_abc)))
```

$\displaystyle \left[\begin{matrix}i_{a i} v_{a i} + i_{a r} v_{a r} + i_{b i} v_{b i} + i_{b r} v_{b r} + i_{c i} v_{c i} + i_{c r} v_{c r}\end{matrix}\right]$

```python
 sym.simplify(sym.re(s_opn))
```

$\displaystyle \left[\begin{matrix}- \frac{108172851219475563584786333870763 a_{i} a_{r} i_{a i} v_{b r}}{162259276829213363391578010288128} - \frac{108172851219475563584786333870763 a_{i} a_{r} i_{a i} v_{c r}}{162259276829213363391578010288128} - \frac{108172851219475563584786333870763 a_{i} a_{r} i_{a r} v_{b i}}{162259276829213363391578010288128} - \frac{108172851219475563584786333870763 a_{i} a_{r} i_{a r} v_{c i}}{162259276829213363391578010288128} - \frac{108172851219475563584786333870763 a_{i} a_{r} i_{b i} v_{a r}}{162259276829213363391578010288128} - \frac{108172851219475563584786333870763 a_{i} a_{r} i_{b r} v_{a i}}{162259276829213363391578010288128} - \frac{108172851219475563584786333870763 a_{i} a_{r} i_{c i} v_{a r}}{162259276829213363391578010288128} - \frac{108172851219475563584786333870763 a_{i} a_{r} i_{c r} v_{a i}}{162259276829213363391578010288128} + \frac{108172851219475563584786333870763 a_{i} a_{r} \left(a_{i}^{2} - a_{r}^{2}\right) \left(i_{b i} v_{b r} + i_{b r} v_{b i}\right)}{81129638414606681695789005144064} + \frac{108172851219475563584786333870763 a_{i} a_{r} \left(a_{i}^{2} - a_{r}^{2}\right) \left(i_{c i} v_{c r} + i_{c r} v_{c i}\right)}{81129638414606681695789005144064} - \frac{108172851219475563584786333870763 a_{i} a_{r} \left(i_{b i} v_{b r} + i_{b r} v_{b i}\right)}{162259276829213363391578010288128} - \frac{108172851219475563584786333870763 a_{i} a_{r} \left(i_{c i} v_{c r} + i_{c r} v_{c i}\right)}{162259276829213363391578010288128} - \frac{108172851219475563584786333870763 a_{i} a_{r} \left(- a_{i} i_{b i} v_{c i} + a_{i} i_{b r} v_{c r} + a_{r} i_{b i} v_{c r} + a_{r} i_{b r} v_{c i}\right)}{81129638414606681695789005144064} - \frac{108172851219475563584786333870763 a_{i} a_{r} \left(- a_{i} i_{c i} v_{b i} + a_{i} i_{c r} v_{b r} + a_{r} i_{c i} v_{b r} + a_{r} i_{c r} v_{b i}\right)}{81129638414606681695789005144064} - \frac{324518553658426690754359001612289 i_{a i} v_{a i}}{324518553658426726783156020576256} + \frac{108172851219475563584786333870763 i_{a i} v_{b i} \left(a_{i}^{2} - a_{r}^{2}\right)}{324518553658426726783156020576256} - \frac{108172851219475563584786333870763 i_{a i} v_{b i}}{324518553658426726783156020576256} + \frac{108172851219475563584786333870763 i_{a i} v_{c i} \left(a_{i}^{2} - a_{r}^{2}\right)}{324518553658426726783156020576256} - \frac{108172851219475563584786333870763 i_{a i} v_{c i}}{324518553658426726783156020576256} - \frac{108172851219475563584786333870763 i_{a i} \left(a_{i} v_{b r} + a_{r} v_{b i}\right)}{324518553658426726783156020576256} - \frac{108172851219475563584786333870763 i_{a i} \left(a_{i} v_{c r} + a_{r} v_{c i}\right)}{324518553658426726783156020576256} + \frac{324518553658426690754359001612289 i_{a r} v_{a r}}{324518553658426726783156020576256} - \frac{108172851219475563584786333870763 i_{a r} v_{b r} \left(a_{i}^{2} - a_{r}^{2}\right)}{324518553658426726783156020576256} + \frac{108172851219475563584786333870763 i_{a r} v_{b r}}{324518553658426726783156020576256} - \frac{108172851219475563584786333870763 i_{a r} v_{c r} \left(a_{i}^{2} - a_{r}^{2}\right)}{324518553658426726783156020576256} + \frac{108172851219475563584786333870763 i_{a r} v_{c r}}{324518553658426726783156020576256} - \frac{108172851219475563584786333870763 i_{a r} \left(a_{i} v_{b i} - a_{r} v_{b r}\right)}{324518553658426726783156020576256} - \frac{108172851219475563584786333870763 i_{a r} \left(a_{i} v_{c i} - a_{r} v_{c r}\right)}{324518553658426726783156020576256} + \frac{108172851219475563584786333870763 i_{b i} v_{a i} \left(a_{i}^{2} - a_{r}^{2}\right)}{324518553658426726783156020576256} - \frac{108172851219475563584786333870763 i_{b i} v_{a i}}{324518553658426726783156020576256} - \frac{108172851219475563584786333870763 i_{b i} v_{b i}}{324518553658426726783156020576256} - \frac{108172851219475563584786333870763 i_{b i} v_{c i}}{324518553658426726783156020576256} - \frac{108172851219475563584786333870763 i_{b r} v_{a r} \left(a_{i}^{2} - a_{r}^{2}\right)}{324518553658426726783156020576256} + \frac{108172851219475563584786333870763 i_{b r} v_{a r}}{324518553658426726783156020576256} + \frac{108172851219475563584786333870763 i_{b r} v_{b r}}{324518553658426726783156020576256} + \frac{108172851219475563584786333870763 i_{b r} v_{c r}}{324518553658426726783156020576256} + \frac{108172851219475563584786333870763 i_{c i} v_{a i} \left(a_{i}^{2} - a_{r}^{2}\right)}{324518553658426726783156020576256} - \frac{108172851219475563584786333870763 i_{c i} v_{a i}}{324518553658426726783156020576256} - \frac{108172851219475563584786333870763 i_{c i} v_{b i}}{324518553658426726783156020576256} - \frac{108172851219475563584786333870763 i_{c i} v_{c i}}{324518553658426726783156020576256} - \frac{108172851219475563584786333870763 i_{c r} v_{a r} \left(a_{i}^{2} - a_{r}^{2}\right)}{324518553658426726783156020576256} + \frac{108172851219475563584786333870763 i_{c r} v_{a r}}{324518553658426726783156020576256} + \frac{108172851219475563584786333870763 i_{c r} v_{b r}}{324518553658426726783156020576256} + \frac{108172851219475563584786333870763 i_{c r} v_{c r}}{324518553658426726783156020576256} - \frac{108172851219475563584786333870763 v_{a i} \left(a_{i} i_{b r} + a_{r} i_{b i}\right)}{324518553658426726783156020576256} - \frac{108172851219475563584786333870763 v_{a i} \left(a_{i} i_{c r} + a_{r} i_{c i}\right)}{324518553658426726783156020576256} - \frac{108172851219475563584786333870763 v_{a r} \left(a_{i} i_{b i} - a_{r} i_{b r}\right)}{324518553658426726783156020576256} - \frac{108172851219475563584786333870763 v_{a r} \left(a_{i} i_{c i} - a_{r} i_{c r}\right)}{324518553658426726783156020576256} + \frac{108172851219475563584786333870763 \left(a_{i}^{2} - a_{r}^{2}\right) \left(i_{b i} v_{b i} - i_{b r} v_{b r}\right)}{324518553658426726783156020576256} + \frac{108172851219475563584786333870763 \left(a_{i}^{2} - a_{r}^{2}\right) \left(i_{c i} v_{c i} - i_{c r} v_{c r}\right)}{324518553658426726783156020576256} + \frac{108172851219475563584786333870763 \left(a_{i}^{2} - a_{r}^{2}\right) \left(a_{i} i_{b i} v_{c r} + a_{i} i_{b r} v_{c i} + a_{r} i_{b i} v_{c i} - a_{r} i_{b r} v_{c r}\right)}{162259276829213363391578010288128} + \frac{108172851219475563584786333870763 \left(a_{i}^{2} - a_{r}^{2}\right) \left(a_{i} i_{c i} v_{b r} + a_{i} i_{c r} v_{b i} + a_{r} i_{c i} v_{b i} - a_{r} i_{c r} v_{b r}\right)}{162259276829213363391578010288128} - \frac{108172851219475563584786333870763 \left(i_{b i} v_{b i} - i_{b r} v_{b r}\right) \left(a_{i}^{4} - 6 a_{i}^{2} a_{r}^{2} + a_{r}^{4}\right)}{324518553658426726783156020576256} - \frac{108172851219475563584786333870763 \left(i_{c i} v_{c i} - i_{c r} v_{c r}\right) \left(a_{i}^{4} - 6 a_{i}^{2} a_{r}^{2} + a_{r}^{4}\right)}{324518553658426726783156020576256}\end{matrix}\right]$

```python

```


```python
from IPython.display import Image
from IPython.core.display import HTML
Image(url= "https://i.imgur.com/ehWy2jK.png")
```

```python
from sympy import *; x = symbols('x')
```

```python
# a good first step is to write down the expressions
expr0 = (x**2)*(1-(x/2))
T = lambdify(x,expr0, 'numpy')
# f = lambdify(x, expr, 'numpy')
print("T(x) =");T(x)
```

    T(x) =

$\displaystyle x^{2} \left(1 - 0.5 x\right)$

```python
lambdify??
```

```python
#Question wants derivatives
#let us try diff() and if that fails we will try limit(difference_quotient)
diff(T(x))
```

$\displaystyle - 0.5 x^{2} + 2 x \left(1 - 0.5 x\right)$

```python
# What if we took our T(x) and just plugged in 1
```

```python
#T(1)
```

    0.5

```python
h = symbols('h')
expr1  = ( T(x+h)-T(x) ) / h
G = lambdify(x,expr1, 'numpy')
```

```python
limit(G(x),h,0)
```

$\displaystyle x \left(2.0 - 1.5 x\right)$

```python
limit(G(x),h,0).subs(x,1)
```

$\displaystyle 0.5$

```python
limit(G(x),h,0).subs(x,3)
```

$\displaystyle -7.5$

```python
limit(G(x),h,0).subs(x,6)
```

$\displaystyle -42.0$

```python
Image(url= "https://i.imgur.com/5Sw09wi.png")
```

```python

```