# Building Chemical Reaction Networks (CRNs) Directly with BioCRNpyler

**Overview:**  This tutorial shows how to use [BioCRNpyler](https://github.com/BuildACell/BioCRNPyler) to represent simple CRNs

## What is a CRN?
A CRN is a widely established model of chemistry and biochemistry.
* A set of species $S$
* A set of reactions $R$ interconvert species $I_r$ to $O_r$

\begin{align}
\\
I \xrightarrow[]{\rho(s)} O
\\
\end{align}

 * $I$ and $O$ are multisets of species $S$.
 * $\rho(s): S \to \mathbb{R}$ is a function that determines how fast the reaction occurs.

```python
#Import everything from biocrnpyler
from biocrnpyler import *
```

## Combining Species and Reactions into a CRN

The following code defines a species called 'S' made out of material 'material'. Species can also have attributes to help identify them. Note that Species with the same name, but different materials or attributes are considered different species in terms of the reactions they participate in.

    S = Species('name', material_type = 'material', attributes = [])

The collowing code produces a reaction R

    R = Reaction(Inputs, Outputs, k)

here Inputs and Outputs must both be a list of Species. the parameter k is the rate constant of the reaction. By default, propensities in BioCRNpyler are massaction:

### $\rho(S) = k \Pi_{s} s^{I_s}$

Note: for stochastic models mass action propensities are $\rho(S) = k \Pi_{s} s!/(s - I_s)!$.

Massaction reactions can be made reversible with the k_rev keyword:

    R_reversible = Reaction(Inputs, Outputs, k, k_rev = krev)

is the same as two reactions:

    R = Reaction(Inputs, Outputs, k)
    Rrev = Reaction(Outputs, Inputs, krev)

Finally, a CRN can be made by combining species and reactions:

    CRN = ChemicalReactionNetwork(species = species, reactions = reactions)

```python
#Example: Model the CRN consisting of: A --> 2B, 2B <--> B + C where C has the same name as B but a new material
A = Species("A", material_type = "m1", attributes = ["attribute"])
B = Species("B", material_type = "m1")
C = Species("B", material_type = "m2")
D = Species("D")

print("Species can be printed to show their string representation:", A, B, C, D)

#Reaction Rates
k1 = 3.
k2 = 1.4
k2rev = 0.15

#Reaciton Objects
R1 = Reaction.from_massaction([A], [B, B], k_forward = k1)
R2 = Reaction.from_massaction([B], [C, D], k_forward = k2)

print("\nReactions can be printed as well:\n", R1,"\n", R2)

#Make a CRN
CRN = ChemicalReactionNetwork(species = [A, B, C, D], reactions = [R1, R2])

#CRNs can be printed in two different ways
print("\nDirectly printing a CRN shows the string representation of the species used in BioCRNpyler:")
print(CRN)

print("\nCRN.pretty_print(...) is a function that prints a more customizable version of the CRN, but doesn't show the proper string representation of species.")
print(CRN.pretty_print(show_materials = True, show_rates = True, show_attributes = True))
```

    Species can be printed to show their string representation: m1_A_attribute m1_B m2_B D

    Reactions can be printed as well:
     m1[A(attribute)] --> 2m1[B]
     m1[B] --> m2[B]+D

    Directly printing a CRN shows the string representation of the species used in BioCRNpyler:
    Species = m1_A_attribute, m1_B, m2_B, D
    Reactions = [
    	m1[A(attribute)] --> 2m1[B]
    	m1[B] --> m2[B]+D
    ]

    CRN.pretty_print(...) is a function that prints a more customizable version of the CRN, but doesn't show the proper string representation of species.
    Species (4) = {0. m1[A(attribute)] init_conc = 0, 1. m1[B] init_conc = 0, 2. m2[B] init_conc = 0, 3. D init_conc = 0}

    Reactions (2) = [
    0. m1[A(attribute)] --> 2m1[B]
     Kf=k_forward * m1_A_attribute
      k_forward=3.0

    1. m1[B] --> m2[B]+D
     Kf=k_forward * m1_B
      k_forward=1.4

    ]

## CRNs can be saved as SBML and simulated

To save a CRN as SBML:

    CRN.write_sbml_file("file_name.xml")

To simulate a CRN with biosrape:

    Results, Model = CRN_expression.simulate_with_bioscrape(timepoints, initial_condition_dict = x0)

Where x0 is a dictionary: x0 = {str(species_name):initial_value}

```python
#Saving and simulating a CRN
CRN.write_sbml_file("build_crns_directly.xml")

try:
    import bioscrape
    import numpy as np
    import pylab as plt
    import pandas as pd

    #Initial conditions can be set with a dictionary:
    x0 = {str(A):120}
    timepoints = np.linspace(0, 1, 100)#Timepoints to simulate over

    #This function can also take a filename keyword to save the file at the same time
    R = CRN.simulate_with_bioscrape_via_sbml(timepoints = timepoints, initial_condition_dict = x0)

    #Check to ensure simulation worked
    #Results are in a Pandas Dictionary and can be accessed via string-names of species
    plt.plot(R['time'], R[str(A)], label = "A")
    plt.plot(R['time'], R[str(B)], label = "B")
    plt.plot(R['time'], R[str(C)], "--", label = "C")
    plt.plot(R['time'], R[str(D)],":", label = "D")
    plt.legend()

except ModuleNotFoundError:
    print("Plotting Modules not installed.")
```

## ComplexSpecies and OrderedComplexSpecies

When Species bind together to form complexes, it is recommended to use the Complex function:

    Complex([list of species], ordered = True/False)

This function returns the classes ComplexSpecies or OrderedComplexSpecies subclasses which contain information about the species inside of them. ComplexSpecies treats its internal species as an unordered multiset. OrderedComplexSpecies treats its internal species as an ordered list. It is recommended to always use the function Complex to create these types of Species for compatability reasons discussed in the OrderedPolymerSpecies example notebook.

_Note: These objects do not automatically generate binding reactions. To do that, use the Component wrappers ChemicalComplex and OrderedChemicalComplex._

```python
A = Species("A", material_type = "m1")
B = Species("B", material_type = "m1")
B2 = Species("B", material_type = "m2")

print("\nFor ComplexSpecies, the order of the elements does not matter:")
C1 = Complex([A, B, B2, A])
C2 = Complex([B, A, B2, A])
print("C1=ComplexSpecies([A, B, B2, A])=", C1)
print("C2=ComplexSpecies([B, A, B2, A])=", C2)
print("C1==C2 ==>", C1==C2)

print("\nFor OrderedComplexSpecies, the Order Does Matter:")
C3 = Complex([A, B, B2, A], ordered = True)
C4 = Complex([B, A, B2, A], ordered = True)
print("C3=OrderedComplexSpecies([A, B, B2, A])=", C3)
print("C4=OrderedComplexSpecies([B, A, B2, A])=", C4)
print("C3==C4 ==>", C3==C4)

print("\nComplexSpecies (and the OrderedComplexSpecies and Multimers) are Species and can be used in reactions:")
R = Reaction.from_massaction([A, B, B2, A], [C1], k_forward=10)
print("Reaction.from_massaction([A, B, B2, A], [C1], k_forward = 10)=")
print(R)
```

    For ComplexSpecies, the order of the elements does not matter:
    C1=ComplexSpecies([A, B, B2, A])= complex_m1_A_2x_m1_B_m2_B
    C2=ComplexSpecies([B, A, B2, A])= complex_m1_A_2x_m1_B_m2_B
    C1==C2 ==> True

    For OrderedComplexSpecies, the Order Does Matter:
    C3=OrderedComplexSpecies([A, B, B2, A])= ordered_complex_m1_A_m1_B_m2_B_m1_A
    C4=OrderedComplexSpecies([B, A, B2, A])= ordered_complex_m1_B_m1_A_m2_B_m1_A
    C3==C4 ==> False

    ComplexSpecies (and the OrderedComplexSpecies and Multimers) are Species and can be used in reactions:
    Reaction.from_massaction([A, B, B2, A], [C1], k_forward = 10)=
    2m1[A]+m1[B]+m2[B] --> complex[2x_m1[A]:m1[B]:m2[B]]

## Non-massaction propensities in BioCRNpyler
By default, BioCRNpyler assumes that propensities are massaction with only one parameter, the rate constant $k$. However, non-massaction propensities are also supported. However, additional propensity types are also supported. When creating a reaction with non-massaction propensities, a propensity_params dictionary must be passed into the chemical_reaction_network.reaction constructor with the appropriate parameters for that particular propensity type. The supported propensity types and their relevant parameters are:

1. "massaction": $\rho(S) = k \Pi_{s} s^{I_s}$. Required parameters: "k" the rate constant. Note: for stochastic models mass action propensities are $\rho(S) = k \Pi_{s} s!/(s - I_s)!$.
2. "positivehill": $\rho(s) = k \frac{s^n}{(K^n+s^n)}$. Requried parameters: rate constant "k", offset "K", hill coefficient "n", hill species "s1".
3. "negativehill": $\rho(s) = k \frac{1}{(K^n+s^n)}$. Requried parameters: rate constant "k", offset "K", hill coefficient "n", hill species "s1".
4. "proportionalpositivehill": $\rho(s) = k d \frac{s^n}{(K^n+s^n)}$. Requried parameters: rate constant "k", offset "K", hill coefficient "n", hill species "s1", propritional species "d".
5. "proportionalnegativehill": $\rho(s) = k d \frac{1}{(K^n+s^n)}$. Requried parameters: rate constant "k", offset "K", hill coefficient "n", hill species "s1", propritional species "d".
6. "general": $\rho(s) = f(s)$ where $f$ can be any algebraic function typed as a string. Required parameters: "rate" an algebraic expression including species and model parameters written as a string.

The following notebook contains examples of the propensity types support by BioCRNpyler applied to a very simple model of genetic activation and repression.

```python
from biocrnpyler.chemical_reaction_network import ChemicalReactionNetwork
from biocrnpyler.species import Species
from biocrnpyler.reaction import Reaction

#Parameter Values
kb = 100
ku = 10
kex = 1.
kd = .1

#Species
G = Species(name = "G", material_type = "dna") #DNA
A = Species(name = "A", material_type = "protein") #Activator
GA = Complex([G, A, A]) #Activated Gene
X = Species(name = "X", material_type = "protein")

rxnd = Reaction.from_massaction([X], [], k_forward=kd)

#Massaction Activation
species1 = [G, A, GA, X]
mak = MassAction(k_forward=kb, k_reverse=ku)
rxn0_1 = Reaction([G, A, A], [GA], propensity_type=mak)
rxn0_2 = Reaction.from_massaction([GA], [GA, X], k_forward=kex)
CRN0 = ChemicalReactionNetwork(species1, [rxn0_1, rxn0_2, rxnd])
print("\nMassaction Activation", repr(CRN0))

#Massaction Repression
rxn1_1 = Reaction.from_massaction([G, A, A], [GA], k_forward=kb, k_reverse=ku)
rxn1_2 = Reaction.from_massaction([G], [G, X], k_forward=kex)
CRN1 = ChemicalReactionNetwork(species1, [rxn1_1, rxn1_2, rxnd])
print("\nMassaction Repression", repr(CRN1))

#Hill positive
species2 = [G, A, X]
hill_positive = HillPositive(k=kex, K=float(kb/ku), s1=A, n=2)
rxn2_1 = Reaction([G], [G, X], propensity_type=hill_positive)
CRN2 = ChemicalReactionNetwork(species2, [rxn2_1, rxnd])
print("\nHill Positive CRN", repr(CRN2))

#proportional Hill positive
prop_hill_positive = ProportionalHillPositive(k=kex, K=float(kb/ku), s1=A, d=G, n=2)
rxn3_1 = Reaction([G], [G, X], propensity_type=prop_hill_positive)
CRN3 = ChemicalReactionNetwork(species2, [rxn3_1, rxnd])
print("\nProportional Hill Positive CRN", repr(CRN3))

#Hill Negative
hill_negative = HillNegative(k=kex, K=float(kb/ku), n=2, s1=A)
rxn4_1 = Reaction([G], [G, X], propensity_type=hill_negative)
CRN4 = ChemicalReactionNetwork(species2, [rxn4_1, rxnd])
print("\nHill Negative CRN", repr(CRN4))

#proportional hill negative
prop_hill_negative = ProportionalHillNegative(k=kex, K=float(kb/ku), s1=A, d=G, n=2)
rxn5_1 = Reaction([G], [G, X], propensity_type=prop_hill_negative)
CRN5 = ChemicalReactionNetwork(species2, [rxn5_1, rxnd])
print("\nProportional Hill Negative CRN", repr(CRN5))
```

    Massaction Activation Species = dna_G, protein_A, complex_dna_G_protein_A_2x, protein_X
    Reactions = [
    	dna[G]+2protein[A] <--> complex[dna[G]:2x_protein[A]]
    	complex[dna[G]:2x_protein[A]] --> complex[dna[G]:2x_protein[A]]+protein[X]
    	protein[X] -->
    ]

    Massaction Repression Species = dna_G, protein_A, complex_dna_G_protein_A_2x, protein_X
    Reactions = [
    	dna[G]+2protein[A] <--> complex[dna[G]:2x_protein[A]]
    	dna[G] --> dna[G]+protein[X]
    	protein[X] -->
    ]

    Hill Positive CRN Species = dna_G, protein_A, protein_X
    Reactions = [
    	dna[G] --> dna[G]+protein[X]
    	protein[X] -->
    ]

    Proportional Hill Positive CRN Species = dna_G, protein_A, protein_X
    Reactions = [
    	dna[G] --> dna[G]+protein[X]
    	protein[X] -->
    ]

    Hill Negative CRN Species = dna_G, protein_A, protein_X
    Reactions = [
    	dna[G] --> dna[G]+protein[X]
    	protein[X] -->
    ]

    Proportional Hill Negative CRN Species = dna_G, protein_A, protein_X
    Reactions = [
    	dna[G] --> dna[G]+protein[X]
    	protein[X] -->
    ]

examples/1. Building CRNs Directly.ipynb


Continuous Data and the Gaussian Distribution
=======

### Preliminaries

- Goal
  - Review of information processing with Gaussian distributions in linear systems
- Materials
  - Mandatory
    - These lecture notes
  - Optional
    - Bishop pp. 85-93
    - [MacKay - 2006 - The Humble Gaussian Distribution](https://github.com/bertdv/BMLIP/blob/master/lessons/notebooks/files/Mackay-2006-The-humble-Gaussian-distribution.pdf) (highly recommended!)
    - [Ariel Caticha - 2012 - Entropic Inference and the Foundations of Physics](https://github.com/bertdv/BMLIP/blob/master/lessons/notebooks/files/Caticha-2012-Entropic-Inference-and-the-Foundations-of-Physics.pdf), pp.30-34, section 2.8, the Gaussian distribution

### Example Problem

Consider a set of observations $D=\{x_1,…,x_N\}$ in the 2-dimensional plane (see Figure). All observations were generated by the same process. We now draw an extra observation $x_\bullet = (a,b)$ from the same data generating process. What is the probability that $x_\bullet$ lies within the shaded rectangle $S$?

```julia
using Pkg;Pkg.activate("probprog/workspace/");Pkg.instantiate();
IJulia.clear_output();
```

```julia
using Distributions, PyPlot
N = 100
generative_dist = MvNormal([0,1.], [0.8 0.5; 0.5 1.0])
function plotObservations(obs::Matrix)
    plot(obs[1,:], obs[2,:], "kx", zorder=3)
    fill_between([0., 2.], 1., 2., color="k", alpha=0.4, zorder=2) # Shaded area
    text(2.05, 1.8, "S", fontsize=12)
    xlim([-3,3]); ylim([-2,4]); xlabel("a"); ylabel("b")
end
D = rand(generative_dist, N) # Generate observations from generative_dist
plotObservations(D)
x_dot = rand(generative_dist) # Generate x∙
plot(x_dot[1], x_dot[2], "ro");
```

### The Gaussian Distribution

- Consider a random (vector) variable $x \in \mathbb{R}^M$ that is "normally" (i.e., Gaussian) distributed. The _moment_ parameterization of the Gaussian distribution is completely specified by its _mean_ $\mu$ and _variance_ $\Sigma$ and given by
$$
p(x | \mu, \Sigma) = \mathcal{N}(x|\mu,\Sigma) \triangleq \frac{1}{\sqrt{(2\pi)^M |\Sigma|}} \,\exp\left\{-\frac{1}{2}(x-\mu)^T \Sigma^{-1} (x-\mu) \right\}\,.
$$
where $|\Sigma| \triangleq \mathrm{det}(\Sigma)$ is the determinant of $\Sigma$.

- Alternatively, the _canonical_ (a.k.a. _natural_  or _information_ ) parameterization of the Gaussian distribution is given by
$$\begin{equation*}
p(x | \eta, \Lambda) = \mathcal{N}_c(x|\eta,\Lambda)  = \exp\left\{ a + \eta^T x - \frac{1}{2}x^T \Lambda x \right\}\,.
\end{equation*}$$
  - $a = -\frac{1}{2} \left( M \log(2 \pi) - \log |\Lambda| + \eta^T \Lambda \eta\right)$ is the normalizing constant that ensures that $\int p(x)\mathrm{d}x = 1$.
  - $\Lambda = \Sigma^{-1}$ is called the *precision matrix*.
  - $\eta = \Sigma^{-1} \mu$ is the _natural_ mean or for clarity often called the *precision-weighted* mean.

### Why the Gaussian?

- Why is the Gaussian distribution so ubiquitously used in science and engineering? (see [Jaynes, pg.220](https://archive.org/details/ProbabilityTheoryTheLogicOfScience/page/n249)).

- (1) Operations on probability distributions tend lead to Gaussian distributions:
  - Any smooth function with single rounded maximum, if raised to higher and higher powers, goes into a Gaussian function. (Example: sequential Bayesian inference).
  - The [Gaussian distribution has higher entropy](https://en.wikipedia.org/wiki/Differential_entropy#Maximization_in_the_normal_distribution) than any other with the same variance.
    - Therefore any operation on a probability distribution that discards information but preserves variance gets us closer to a Gaussian.
    - As an example, see [Jaynes (pg.220)](https://archive.org/details/ProbabilityTheoryTheLogicOfScience/page/n249) for how this leads to the [Central Limit Theorem](https://en.wikipedia.org/wiki/Central_limit_theorem), which results from performing convolution operations on distributions.

- (2) Once the Gaussian has been attained, this form tends to be preserved. e.g.,
  - The convolution of two Gaussian functions is another Gaussian function (useful in sum of 2 variables and linear transformations)
  - The product of two Gaussian functions is another Gaussian function (useful in Bayes rule).
  - The Fourier transform of a Gaussian function is another Gaussian function.

###  Transformations and Sums of Gaussian Variables

- A **linear transformation** $z=Ax+b$ of a Gaussian variable $\mathcal{N}(x|\mu_x,\Sigma_x)$ is Gaussian distributed as
$$
p(z) = \mathcal{N} \left(z \,|\, A\mu_x+b, A\Sigma_x A^T \right) \tag{SRG-4a}
$$
  - In fact, after a linear transformation $z=Ax+b$, no matter how $x$ is distributed, the mean and variance of $z$ are always given by $\mu_z = A\mu_x + b$  and $\Sigma_z = A\Sigma_x A^T$, respectively (see   [probability theory review lesson](https://nbviewer.jupyter.org/github/bertdv/BMLIP/blob/master/lessons/notebooks/Probability-Theory-Review.ipynb#linear-transformation)). In case $x$ is not Gaussian, higher order moments may be needed to specify the distribution for $z$.

- The **sum of two independent Gaussian variables** is also Gaussian distributed. Specifically, if $x \sim \mathcal{N} \left(\mu_x, \Sigma_x \right)$ and $y \sim \mathcal{N} \left(\mu_y, \Sigma_y \right)$, then the PDF for $z=x+y$ is given by
$$\begin{align}
p(z) &= \mathcal{N}(x\,|\,\mu_x,\Sigma_x) \ast \mathcal{N}(y\,|\,\mu_y,\Sigma_y) \notag\\
  &= \mathcal{N} \left(z\,|\,\mu_x+\mu_y, \Sigma_x +\Sigma_y \right) \tag{SRG-8}
\end{align}$$

  - The sum of two Gaussian _distributions_ is NOT a Gaussian distribution. Why not?

### Example: Gaussian Signals in a Linear System

- Given independent variables
$x \sim \mathcal{N}(\mu_x,\sigma_x^2)$ and $y \sim \mathcal{N}(\mu_y,\sigma_y^2)$, what is the PDF for $z = A\cdot(x -y) + b$ ? (see [Exercises](https://nbviewer.jupyter.org/github/bertdv/BMLIP/blob/master/lessons/exercises/Exercises.ipynb))

- Think about the role of the Gaussian distribution for stochastic linear systems in relation to what sinusoidals mean for deterministic linear system analysis.

### Bayesian Inference for the Gaussian

- Let's estimate a constant $\theta$ from one 'noisy' measurement $x$ about that constant.

- We assume the following measurement equations (the tilde $\sim$ means: 'is distributed as'):
$$\begin{align*}
x &= \theta + \epsilon \\
\epsilon &\sim \mathcal{N}(0,\sigma^2)
\end{align*}$$

- Also, let's assume a Gaussian prior for $\theta$
$$\begin{align*}
\theta &\sim \mathcal{N}(\mu_0,\sigma_0^2) \\
\end{align*}$$

##### Model specification

- Note that you can rewrite these specifications in probabilistic notation as follows:

$$\begin{align}
    p(x|\theta) &=  \mathcal{N}(x|\theta,\sigma^2) \tag{likelihood}\\
    p(\theta) &=\mathcal{N}(\theta|\mu_0,\sigma_0^2) \tag{prior}
\end{align}$$

- (**Notational convention**). Note that we write $\epsilon \sim \mathcal{N}(0,\sigma^2)$ but not $\epsilon \sim \mathcal{N}(\epsilon | 0,\sigma^2)$, and we write  $p(\theta) =\mathcal{N}(\theta|\mu_0,\sigma_0^2)$ but not $p(\theta) =\mathcal{N}(\mu_0,\sigma_0^2)$.

##### Inference

- For simplicity, we assume that the variance $\sigma^2$ is given and will proceed to derive a Bayesian posterior for the mean $\theta$. The case for Bayesian inference for $\sigma^2$ with a given mean is [discussed in the optional slides](#inference-for-precision).

- Let's do Bayes rule for the posterior PDF $p(\theta|x)$.
$$\begin{align*}
p(\theta|x)  &= \frac{p(x|\theta) p(\theta)}{p(x)} \propto p(x|\theta) p(\theta) \notag \\
    &= \mathcal{N}(x|\theta,\sigma^2) \mathcal{N}(\theta|\mu_0,\sigma_0^2)   \notag \\
    &\propto \exp \left\{   -\frac{(x-\theta)^2}{2\sigma^2} - \frac{(\theta-\mu_0)^2}{2\sigma_0^2} \right\} \notag \\
    &\propto \exp \left\{ \theta^2 \cdot \left( -\frac{1}{2 \sigma_0^2} - \frac{1}{2\sigma^2}  \right)  + \theta \cdot  \left( \frac{\mu_0}{\sigma_0^2} + \frac{x}{\sigma^2}\right)   \right\} \notag \\
    &= \exp\left\{ -\frac{\sigma_0^2 + \sigma^2}{2 \sigma_0^2 \sigma^2} \left( \theta - \frac{\sigma_0^2 x +  \sigma^2 \mu_0}{\sigma^2 + \sigma_0^2}\right)^2  \right\}
\end{align*}$$
which we recognize as a Gaussian distribution w.r.t. $\theta$.

- (Just as an aside,) this computational 'trick' for multiplying two Gaussians is called **completing the square**. The procedure makes use of the equality $$ax^2+bx+c_1 = a\left(x+\frac{b}{2a}\right)^2+c_2$$

- In particular, it follows that the posterior for $\theta$ is
$$\begin{equation*}
    p(\theta|x) = \mathcal{N} (\theta |\, \mu_1, \sigma_1^2)
\end{equation*}$$
where
$$\begin{align*}
  \frac{1}{\sigma_1^2}  &= \frac{\sigma_0^2 + \sigma^2}{\sigma^2 \sigma_0^2} = \frac{1}{\sigma_0^2} + \frac{1}{\sigma^2}  \\
  \mu_1   &= \frac{\sigma_0^2 x +  \sigma^2 \mu_0}{\sigma^2 + \sigma_0^2} = \sigma_1^2 \, \left(  \frac{1}{\sigma_0^2} \mu_0 + \frac{1}{\sigma^2} x \right)
\end{align*}$$

### (Multivariate) Gaussian Multiplication

- So, multiplication of two Gaussian distributions yields another (unnormalized) Gaussian with
  - posterior precision equals **sum of prior precisions**
  - posterior precision-weighted mean equals **sum of prior precision-weighted means**

- As we just saw, great application to Bayesian inference!

$$\begin{equation*}
\underbrace{\text{Gaussian}}_{\text{posterior}}
 \propto \underbrace{\text{Gaussian}}_{\text{likelihood}} \times \underbrace{\text{Gaussian}}_{\text{prior}}
\end{equation*}$$

- In general, the multiplication of two multi-variate Gaussians yields an (unnormalized) Gaussian:
$$\begin{equation*}
\boxed{\mathcal{N}(x|\mu_a,\Sigma_a) \cdot \mathcal{N}(x|\mu_b,\Sigma_b) = \underbrace{\mathcal{N}(\mu_a|\, \mu_b, \Sigma_a + \Sigma_b)}_{\text{normalization constant}} \cdot \mathcal{N}(x|\mu_c,\Sigma_c)} \tag{SRG-6}
\end{equation*}$$
where
$$\begin{align*}
\Sigma_c^{-1} &= \Sigma_a^{-1} + \Sigma_b^{-1} \\
\Sigma_c^{-1} \mu_c &= \Sigma_a^{-1}\mu_a + \Sigma_b^{-1}\mu_b
\end{align*}$$

- $\Rightarrow$ Note that Bayesian inference is trivial in the [_canonical_ parameterization of the Gaussian](#natural-parameterization), where we would get
$$\begin{align*}
 \Lambda_c &= \Lambda_a + \Lambda_b  \quad &&\text{(precisions add)}\\
 \eta_c &= \eta_a + \eta_b \quad &&\text{(precision-weighted means add)}
\end{align*}$$

### CODE EXAMPLE

Let's plot the exact product of two Gaussian PDFs as well as the normalized product according to the above derivation.

```julia
using PyPlot, Distributions
d1 = Normal(0, 1) # μ=0, σ^2=1
d2 = Normal(3, 2) # μ=3, σ^2=4

# Calculate the parameters of the product d1*d2
s2_prod = (d1.σ^-2 + d2.σ^-2)^-1
m_prod = s2_prod * ((d1.σ^-2)*d1.μ + (d2.σ^-2)*d2.μ)
d_prod = Normal(m_prod, sqrt(s2_prod)) # Note that we neglect the normalization constant.

# Plot stuff
x = range(-4, stop=8, length=100)
plot(x, pdf.(d1,x), "k")
plot(x, pdf.(d2,x), "b")
plot(x, pdf.(d1,x) .* pdf.(d2,x), "r-") # Plot the exact product
plot(x, pdf.(d_prod,x), "r:")          # Plot the normalized Gaussian product
legend([L"\mathcal{N}(0,1)",
        L"\mathcal{N}(3,4)",
        L"\mathcal{N}(0,1) \mathcal{N}(3,4)",
        L"Z^{-1} \mathcal{N}(0,1) \mathcal{N}(3,4)"]);
```

The solid and dotted red curves are identical up to a scaling factor $Z$.

### Bayesian Inference with multiple Observations

- Now consider that we measure a data set $D = \{x_1, x_2, \ldots, x_N\}$, with measurements
  $$\begin{align*}
x_n &= \theta + \epsilon_n \\
\epsilon_n &\sim \mathcal{N}(0,\sigma^2)
\end{align*}$$
and the same prior for $\theta$:
$$\begin{align*}
\theta &\sim \mathcal{N}(\mu_0,\sigma_0^2) \\
\end{align*}$$

- Let's derive a distribution for the next sample $x_{N+1}$.

##### inference

- Clearly, the posterior for $\theta$ is now
$$\begin{align*}
p(\theta|D) \propto  \underbrace{\mathcal{N}(\theta|\mu_0,\sigma_0^2)}_{\text{prior}} \cdot \underbrace{\prod_{n=1}^N \mathcal{N}(x_n|\theta,\sigma^2)}_{\text{likelihood}}
\end{align*}$$
which is a multiplication of $N+1$ Gaussians and is therefore also Gaussian distributed.

- Using the property that precisions and precision-weighted means add when Gaussians are multiplied, we can immediately write the posterior $$p(\theta|D) = \mathcal{N} (\theta |\, \mu_N, \sigma_N^2)$$ as
$$\begin{align*}
  \frac{1}{\sigma_N^2}  &= \frac{1}{\sigma_0^2} + \sum_n  \frac{1}{\sigma^2} \tag{B-2.142}\\
  \mu_N   &= \sigma_N^2 \, \left( \frac{1}{\sigma_0^2} \mu_0 + \sum_n \frac{1}{\sigma^2} x_n  \right) \tag{B-2.141}
\end{align*}$$

##### application: prediction of future sample

- We now have a posterior for the model parameters. Let's write down what we know about the next sample $x_{N+1}$.

$$\begin{align*}
p(x_{N+1}|D_N) &= \int p(x_{N+1}|\theta) p(\theta|D_{N})\mathrm{d}\theta \\
  &= \int \mathcal{N}(x_{N+1}|\theta,\sigma^2) \mathcal{N}(\theta|\mu_N,\sigma^2_N) \mathrm{d}\theta \\
  &= \mathcal{N}(x_{N+1}|\mu_N, \sigma^2_N +\sigma^2 ) \tag{use SRG-6}
\end{align*}$$

- Uncertainty about $x_{N+1}$ comprises both uncertainty about the parameter ($\sigma_N^2$) and observation noise $\sigma^2$.

### Maximum Likelihood Estimation for the Gaussian

- In order to determine the _maximum likelihood_ estimate of $\theta$, we let $\sigma_0^2 \rightarrow \infty$ (leads to uniform prior for $\theta$), yielding $ \frac{1}{\sigma_N^2} = \frac{N}{\sigma^2}$ and consequently
$$\begin{align*}
  \mu_{\text{ML}}  = \left.\mu_N\right\vert_{\sigma_0^2 \rightarrow \infty} = \sigma_N^2 \, \left(   \frac{1}{\sigma^2}\sum_n  x_n  \right) = \frac{1}{N} \sum_{n=1}^N x_n
  \end{align*}$$

- As expected, having an expression for the maximum likelihood estimate, it is now possible to rewrite the (Bayesian) posterior mean for $\theta$ as (exercise)
$$\begin{align*}
  \underbrace{\mu_N}_{\text{posterior}}   &= \frac{\sigma^2}{N} \, \left( \frac{1}{\sigma_0^2} \mu_0 + \sum_n \frac{1}{\sigma^2} x_n  \right) \\
  &= \underbrace{\mu_0}_{\text{prior}} + \underbrace{\underbrace{\frac{N \sigma_0^2}{N \sigma_0^2 + \sigma^2}}_{\text{gain}}\cdot \underbrace{\left(\mu_{\text{ML}} - \mu_0 \right)}_{\text{prediction error}}}_{\text{correction}}\tag{B-2.141}
\end{align*}$$

- Hence, the posterior mean always lies somewhere between the prior mean $\mu_0$ and the maximum likelihood estimate (the "data" mean) $\mu_{\text{ML}}$.

### Conditioning and Marginalization of a Gaussian

- Let $z = \begin{bmatrix} x \\ y \end{bmatrix}$ be jointly normal distributed as

$$\begin{align*}
p(z) &= \mathcal{N}(z | \mu, \Sigma)
  =\mathcal{N} \left( \begin{bmatrix} x \\ y \end{bmatrix} \left| \begin{bmatrix} \mu_x \\ \mu_y \end{bmatrix},
  \begin{bmatrix} \Sigma_x & \Sigma_{xy} \\ \Sigma_{yx} & \Sigma_y \end{bmatrix} \right. \right)
\end{align*}$$

- Since covariance matrices are by definition symmetric, it follows that $\Sigma_x$ and $\Sigma_y$ are symmetric and $\Sigma_{xy} = \Sigma_{yx}^T$.

- Let's factorize $p(z) = p(x,y)$ into $p(y|x)\cdot p(x)$ through conditioning and marginalization.

$$\begin{equation*}
\text{conditioning: }\boxed{ p(y|x) = \mathcal{N}\left(y\,|\,\mu_y + \Sigma_{yx}\Sigma_x^{-1}(x-\mu_x),\, \Sigma_y - \Sigma_{yx}\Sigma_x^{-1}\Sigma_{xy} \right)}
\end{equation*}$$

$$\begin{equation*}
\text{marginalization: } \boxed{ p(x) = \mathcal{N}\left( x|\mu_x, \Sigma_x \right)}
\end{equation*}$$

- **proof**: in Bishop pp.87-89

- Hence, conditioning and marginalization in Gaussians leads to Gaussians again. This is very useful for applications to Bayesian inference in jointly Gaussian systems.

- With a natural parameterization of the Gaussian $p(z) = \mathcal{N}_c(z|\eta,\Lambda)$ with precision matrix $\Lambda = \Sigma^{-1} = \begin{bmatrix} \Lambda_x & \Lambda_{xy} \\ \Lambda_{yx} & \Lambda_y \end{bmatrix}$,  the conditioning operation results in a simpler result, see Bishop pg.90, eqs. 2.96 and 2.97.

- As an exercise, interpret the formula for the conditional mean ($\mathbb{E}[y|x]=\mu_y + \Sigma_{yx}\Sigma_x^{-1}(x-\mu_x)$) as a prediction-correction operation.

#### CODE EXAMPLE

Plot of the joint, marginal, and conditional distributions.

```julia
using PyPlot, Distributions

μ = [1.0; 2.0]
Σ = [0.3 0.7;
     0.7 2.0]
joint = MvNormal(μ,Σ)
marginal_x = Normal(μ[1], sqrt(Σ[1,1]))

#Plot p(x,y)
subplot(221)
x_range = y_range = range(-2,stop=5,length=1000)
joint_pdf = [ pdf(joint, [x_range[i];y_range[j]]) for  j=1:length(y_range), i=1:length(x_range)]
imshow(joint_pdf, origin="lower", extent=[x_range[1], x_range[end], y_range[1], y_range[end]])
grid(); xlabel("x"); ylabel("y"); title("p(x,y)"); tight_layout()

# Plot p(x)
subplot(222)
plot(range(-2,stop=5,length=1000), pdf.(marginal_x, range(-2,stop=5,length=1000)))
grid(); xlabel("x"); ylabel("p(x)"); title("Marginal distribution p(x)"); tight_layout()

# Plot p(y|x)
x = 0.1
conditional_y_m = μ[2]+Σ[2,1]*inv(Σ[1,1])*(x-μ[1])
conditional_y_s2 = Σ[2,2] - Σ[2,1]*inv(Σ[1,1])*Σ[1,2]
conditional_y = Normal(conditional_y_m, sqrt.(conditional_y_s2))
subplot(223)
plot(range(-2,stop=5,length=1000), pdf.(conditional_y, range(-2,stop=5,length=1000)))
grid(); xlabel("y"); ylabel("p(y|x)"); title("Conditional distribution p(y|x)"); tight_layout()
```

As is clear from the plots, the conditional distribution is a renormalized slice from the joint distribution.

### Example: Conditioning of Gaussian

- Consider (again) the system
$$\begin{align*}
p(x\,|\,\theta) &= \mathcal{N}(x\,|\,\theta,\sigma^2)
\end{align*}$$
with a Gaussian prior for $\theta$:
$$
p(\theta) = \mathcal{N}(\theta\,|\,\mu_0,\sigma_0^2)
$$

- Let $z = \begin{bmatrix} x \\ \theta \end{bmatrix}$. The distribution for $z$ is then given by (Exercise)

$$
p(z) = p\left(\begin{bmatrix} x \\ \theta \end{bmatrix}\right) = \mathcal{N} \left( \begin{bmatrix} x\\
  \theta  \end{bmatrix}
  \,\left|\, \begin{bmatrix} \mu_0\\
  \mu_0\end{bmatrix},
         \begin{bmatrix} \sigma_0^2+\sigma^2  & \sigma_0^2\\
         \sigma_0^2 &\sigma_0^2
  \end{bmatrix}
  \right. \right)
$$

- Direct substitution of the rule for Gaussian conditioning leads to the posterior (derivation as an Exercise):
$$\begin{align*}
p(\theta|x) &= \mathcal{N} \left( \theta\,|\,\mu_1, \sigma_1^2 \right)\,,
\end{align*}$$
with
$$\begin{align*}
K &= \frac{\sigma_0^2}{\sigma_0^2+\sigma^2} \qquad \text{($K$ is called: Kalman gain)}\\
\mu_1 &= \mu_0 + K \cdot (x-\mu_0)\\
\sigma_1^2 &= \left( 1-K \right) \sigma_0^2
\end{align*}$$

- $\Rightarrow$ Moral: For jointly Gaussian systems, we can do inference simply in one step by using the formulas for conditioning and marginalization.

### Recursive Bayesian Estimation

- Consider the signal $x_t=\theta+\epsilon_t$, where $D_t= \left\{x_1,\ldots,x_t\right\}$ is observed _sequentially_ (over time).

- **Problem**: Derive a recursive algorithm for $p(\theta|D_t)$, i.e., an update rule for (posterior) $p(\theta|D_t)$ based on (prior) $p(\theta|D_{t-1})$ and (new observation) $x_t$.

##### Model specification
- Let's define the estimate after $t$ observations (i.e., our _solution_ ) as $p(\theta|D_t) = \mathcal{N}(\theta\,|\,\mu_t,\sigma_t^2)$.

- We define the joint distribution for $\theta$ and $x_t$, given background $D_{t-1}$, by

$$\begin{align*} p(x_t,\theta \,|\, D_{t-1}) &= p(x_t|\theta) \, p(\theta|D_{t-1}) \\
  &= \underbrace{\mathcal{N}(x_t\,|\, \theta,\sigma^2)}_{\text{likelihood}} \, \underbrace{\mathcal{N}(\theta\,|\,\mu_{t-1},\sigma_{t-1}^2)}_{\text{prior}}
\end{align*}$$

##### Inference

- Use Bayes rule,
$$\begin{align*}
p(\theta|D_t) &= p(\theta|x_t,D_{t-1}) \\
  &\propto p(x_t,\theta | D_{t-1}) \\
  &= p(x_t|\theta) \, p(\theta|D_{t-1}) \\
  &= \mathcal{N}(x_t|\theta,\sigma^2) \, \mathcal{N}(\theta\,|\,\mu_{t-1},\sigma_{t-1}^2) \\
  &= \mathcal{N}(\theta|x_t,\sigma^2) \, \mathcal{N}(\theta\,|\,\mu_{t-1},\sigma_{t-1}^2) \;\;\text{(note this trick)}\\
  &= \mathcal{N}(\theta|\mu_t,\sigma_t^2) \;\;\text{(use Gaussian multiplication formula SRG-6)}
\end{align*}$$
with
$$\begin{align*}
K_t &= \frac{\sigma_{t-1}^2}{\sigma_{t-1}^2+\sigma^2} \qquad \text{(Kalman gain)}\\
\mu_t &= \mu_{t-1} + K_t \cdot (x_t-\mu_{t-1})\\
\sigma_t^2 &= \left( 1-K_t \right) \sigma_{t-1}^2
\end{align*}$$

- This linear _sequential_ estimator of mean and variance in Gaussian observations is called a **Kalman Filter**.

- Note that the uncertainty about $\theta$ decreases over time (since $0<(1-K_t)<1$). Since we assume that the statistics of the system do not change (stationarity), each new sample provides new information.

- Recursive Bayesian estimation is the basis for **adaptive signal processing** algorithms such as Least Mean Squares (LMS) and Recursive Least Squares (RLS).

#### CODE EXAMPLE

Let's implement the Kalman filter described above. We'll use it to recursively estimate the value of $\theta$ based on noisy observations.

```julia
using PyPlot

N = 50                         # Number of observations
θ = 2.0                        # True value of the variable we want to estimate
σ_ϵ2 = 0.25                    # Observation noise variance
x = sqrt(σ_ϵ2) * randn(N) .+ θ # Generate N noisy observations of θ

t = 0
μ = fill!(Vector{Float64}(undef,N), NaN)    # Means of p(θ|D) over time
σ_μ2 = fill!(Vector{Float64}(undef,N), NaN) # Variances of p(θ|D) over time

function performKalmanStep()
    # Perform a Kalman filter step, update t, μ, σ_μ2
    global t += 1
    if t>1 # Use posterior from prev. step as prior
        K = σ_μ2[t-1] / (σ_ϵ2 + σ_μ2[t-1]) # Kalman gain
        μ[t] = μ[t-1] + K*(x[t] - μ[t-1])  # Update mean using (1)
        σ_μ2[t] = σ_μ2[t-1] * (1.0-K)      # Update variance using (2)
    elseif t==1 # Use prior
        # Prior p(θ) = N(0,1000)
        K = 1000.0 / (σ_ϵ2 + 1000.0) # Kalman gain
        μ[t] = 0 + K*(x[t] - 0)      # Update mean using (1)
        σ_μ2[t] = 1000 * (1.0-K)     # Update variance using (2)
    end
end

while t<N
    performKalmanStep()
end

# Plot the 'true' value of θ, noisy observations x, and the recursively updated posterior p(θ|D)
t = collect(1:N)
plot(t, θ*ones(N), "k--")
plot(t, x, "rx")
plot(t, μ, "b-")
fill_between(t, μ-sqrt.(σ_μ2), μ+sqrt.(σ_μ2), color="b", alpha=0.3)
legend([L"\theta", L"x[t]", L"\mu[t]"])
xlim((1, N)); xlabel(L"t"); grid()
```

The shaded area represents 2 standard deviations of posterior $p(\theta|D)$. The variance of the posterior is guaranteed to decrease monotonically for the standard Kalman filter.

###  Product of Normally Distributed Variables
- (We've seen that) the sum of two Gausssian distributed variables is also Gaussian distributed.

- Has the _product_ of two Gaussian distributed variables also a Gaussian distribution?

- **No**! In general this is a difficult computation. As an example, let's compute $p(z)$ for $Z=XY$ for the special case that $X\sim \mathcal{N}(0,1)$ and $Y\sim \mathcal{N}(0,1)$.
$$\begin{align*}
p(z) &= \int_{X,Y} p(z|x,y)\,p(x,y)\,\mathrm{d}x\mathrm{d}y \\
  &= \frac{1}{2 \pi}\int  \delta(z-xy) \, e^{-(x^2+y^2)/2} \, \mathrm{d}x\mathrm{d}y \\
  &=  \frac{1}{\pi} \int_0^\infty \frac{1}{x} e^{-(x^2+z^2/x^2)/2} \, \mathrm{d}x \\
  &= \frac{1}{\pi} \mathrm{K}_0( \lvert z\rvert )\,.
\end{align*}$$
where  $\mathrm{K}_n(z)$ is a [modified Bessel function of the second kind](http://mathworld.wolfram.com/ModifiedBesselFunctionoftheSecondKind.html).

### CODE EXAMPLE

- We plot $p(Z=XY)$ and $p(X)p(Y)$ for $X\sim\mathcal{N}(0,1)$ and $Y \sim \mathcal{N}(0,1)$ to give an idea of how these distributions differ.

```julia
using PyPlot, Distributions, SpecialFunctions
X = Normal(0,1)
Y = Normal(0,1)
pdf_product_std_normals(z::Vector) = (besselk.(0, abs.(z))./π)
range1 = collect(range(-4,stop=4,length=100))
plot(range1, pdf.(X, range1))
plot(range1, pdf.(X,range1).*pdf.(Y,range1))
plot(range1, pdf_product_std_normals(range1))
legend([L"p(X)=p(Y)=\mathcal{N}(0,1)", L"p(X)*p(Y)",L"p(Z=X*Y)"]); grid()
```

- In short, Gaussian-distributed variables remain Gaussian in linear systems, but this is not the case in non-linear systems.

### Solution to Example Problem

We apply maximum likelihood estimation to fit a 2-dimensional Gaussian model ($m$) to data set $D$. Next, we evaluate $p(x_\bullet \in S | m)$ by (numerical) integration of the Gaussian pdf over $S$: $p(x_\bullet \in S | m) = \int_S p(x|m) \mathrm{d}x$.

```julia
using HCubature, LinearAlgebra# Numerical integration package
# Maximum likelihood estimation of 2D Gaussian
N = length(sum(D,dims=1))
μ = 1/N * sum(D,dims=2)[:,1]
D_min_μ = D - repeat(μ, 1, N)
Σ = Hermitian(1/N * D_min_μ*D_min_μ')
m = MvNormal(μ, convert(Matrix, Σ));

# # Contour plot of estimated Gaussian density
A = Matrix{Float64}(undef,100,100); B = Matrix{Float64}(undef,100,100)
density = Matrix{Float64}(undef,100,100)
for i=1:100
    for j=1:100
        A[i,j] = a = (i-1)*6/100 .- 2
        B[i,j] = b = (j-1)*6/100 .- 3
        density[i,j] = pdf(m, [a,b])
    end
end
c = contour(A, B, density, 6, zorder=1)
PyPlot.set_cmap("cool")
clabel(c, inline=1, fontsize=10)

# Plot observations, x∙, and the countours of the estimated Gausian density
plotObservations(D)
plot(x_dot[1], x_dot[2], "ro")

# Numerical integration of p(x|m) over S:
(val,err) = hcubature((x)->pdf(m,x), [0., 1.], [2., 2.])
println("p(x⋅∈S|m) ≈ $(val)")
```

### Summary

- A linear transformation of a Gaussian-distributed (potentially multivariate) variable remains Gaussian.

- Bayesian inference with Gaussian prior and likelihood leads to an analytically computable Gaussian posterior.

- Here's a nice [summary of Gaussian calculations](https://github.com/bertdv/AIP-5SSB0/raw/master/lessons/notebooks/files/RoweisS-gaussian_formulas.pdf).

##  OPTIONAL SLIDES

### Inference for the Precision Parameter of the Gaussian

- Again, we consider an observed data set $D = \{x_1, x_2, \ldots, x_N\}$ and try to explain these data by a Gaussian distribution.

- We discussed earlier Bayesian inference for the mean with a given variance. Now we will derive a posterior for the variance if the mean is given. (Technically, we will do the derivation for a precision parameter $\lambda = \sigma^{-2}$, since the discussion is a bit more straightforward for the precision parameter).

##### model specification
- The likelihood for the precision parameter is
$$\begin{align*}
p(D|\lambda) &= \prod_{n=1}^N \mathcal{N}\left(x_n \,|\, \mu, \lambda^{-1} \right) \\
  &\propto \lambda^{N/2} \exp\left\{ -\frac{\lambda}{2}\sum_{n=1}^N \left(x_n - \mu \right)^2\right\} \tag{B-2.145}
\end{align*}$$

- The conjugate distribution for this function of $\lambda$ is the [_Gamma_ distribution](https://en.wikipedia.org/wiki/Gamma_distribution), given by
$$
p(\lambda\,|\,a,b) = \mathrm{Gam}\left( \lambda\,|\,a,b \right) \triangleq \frac{1}{\Gamma(a)} b^{a} \lambda^{a-1} \exp\left\{ -b \lambda\right\}\,, \tag{B-2.146}
$$
where $a>0$ and $b>0$ are known as the _shape_ and _rate_ parameters, respectively.

- (Bishop fig.2.13). Plots of the Gamma distribution $\mathrm{Gam}\left( \lambda\,|\,a,b \right) $ for different values of $a$ and $b$.

- The mean and variance of the Gamma distribution evaluate to $\mathrm{E}\left( \lambda\right) = \frac{a}{b}$ and $\mathrm{var}\left[\lambda\right] = \frac{a}{b^2}$.

##### inference

- We will consider a prior $p(\lambda) = \mathrm{Gam}\left( \lambda\,|\,a_0, b_0\right)$, which leads by Bayes rule to the posterior
$$\begin{align*}
p(\lambda\,|\,D) &\propto \underbrace{\lambda^{N/2} \exp\left\{ -\frac{\lambda}{2}\sum_{n=1}^N \left(x_n - \mu \right)^2\right\} }_{\text{likelihood}} \cdot \underbrace{\frac{1}{\Gamma(a_0)} b_0^{a_0} \lambda^{a_0-1} \exp\left\{ -b_0 \lambda\right\}}_{\text{prior}} \\
  &\propto \mathrm{Gam}\left( \lambda\,|\,a_N,b_N \right)
\end{align*}$$
with
$$\begin{align*}
a_N &= a_0 + \frac{N}{2} \tag{B-2.150} \\
b_N &= b_0 + \frac{1}{2}\sum_n \left( x_n-\mu\right)^2 \tag{B-2.151}
\end{align*}$$

- Hence the **posterior is again a Gamma distribution**. By inspection of B-2.150 and B-2.151, we deduce that we can interpret $2a_0$ as the number of a priori (pseudo-)observations.

- Since the most uninformative prior is given by $a_0=b_0 \rightarrow 0$, we can derive the **maximum likelihood estimate** for the precision as
$$
\lambda_{\text{ML}} = \left.\mathrm{E}\left[ \lambda\right]\right\vert_{a_0=b_0\rightarrow 0} = \left. \frac{a_N}{b_N}\right\vert_{a_0=b_0\rightarrow 0} = \frac{N}{\sum_{n=1}^N \left(x_n-\mu \right)^2}
$$

- In short, if we do density estimation with a Gaussian distribution $\mathcal{N}\left(x_n\,|\,\mu,\sigma^2 \right)$ for an observed data set $D = \{x_1, x_2, \ldots, x_N\}$, the maximum likelihood estimates for $\mu$ and $\sigma^2$ are given by
$$\begin{align*}
\mu_{\text{ML}} &= \frac{1}{N} \sum_{n=1}^N x_n \tag{B-2.121} \\
\sigma^2_{\text{ML}} &= \frac{1}{N} \sum_{n=1}^N \left(x_n - \mu_{\text{ML}} \right)^2 \tag{B-2.122}
\end{align*}$$

- These estimates are also known as the _sample mean_ and _sample variance_ respectively.

```julia
open("../../styles/aipstyle.html") do f
    display("text/html", read(f, String))
end
```

<link href='http://fonts.googleapis.com/css?family=Alegreya+Sans:100,300,400,500,700,800,900,100italic,300italic,400italic,500italic,700italic,800italic,900italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Arvo:400,700,400italic' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=PT+Mono' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Shadows+Into+Light' rel='stylesheet' type='text/css'>
<link href='http://fonts.googleapis.com/css?family=Nixie+One' rel='stylesheet' type='text/css'>

```julia

```


```python
from sympy import Eq, Symbol, Matrix
```

## 1.4 Модель корректировки размера дивидендов Линтера

Задача 4. Модель корректировки размера дивидендов Линтнера.
Пусть $\pi_{t}$ - текущая прибыль фирмы после уплаты налогов,
$D_{t}$ - дивиденды, которые фирма выплачивает своим акционерам в текущем периоде. Известный американский экономист Дж. Линтнер сформулировал в 1956г. следующие утверждения:
1) у фирмы имеется долгосрочная целевая доля $g$ текущей прибыли и соответствующий этой доле уровень дивидендов $D_{t}^{*}$, которые фирма хотела бы выплачивать своим акционерам (желаемый уровень);

2) Изменение $\left(D_{t}-D_{t-1}\right)$ уровня реальных дивидендов за период прямо пропорционально отличию желаемого объёма $D_{t}^{*}$ текущих дивидендов от их реального уровня в предшествующем периоде $D_{t-1} .$
Требуется:
a) составить спецификацию модели корректировки размера дивидендов, позволяющую объяснить текущий уровень $D_{t}$ дивидендов их лаговым значением и текущей прибылью фирмы;

б) Уточнить спецификацию включением случайного возмущения.

в) Представить структурную форму спецификации в матричном виде.

г) Составить приведенную форму модели.

д) Записать приведенную форму в матричном виде.

#### Дано

```python
endos = 'D_t D^*_t'
endos = utils.get_symbols_dict(endos)

predeters = 'P_t D_t-1'
predeters = utils.get_symbols_dict(predeters)

params = 'g a b'
params = utils.get_symbols_dict(params)

devs = 'e_0'
devs = utils.get_symbols_dict(devs)
```

```python

```

```python
from my_utils.task_1_4 import Y, X, A, B, U, reduced_form_matrix, reduced_form_system
from my_utils import utils
```

```python
# Коэффы при эндогенных переменных
A = Matrix([
    [0, 1],
    [1, -params['b']],
])
```

```python
# матрица коэффициентов при ПРЕДОПРЕДЕЛЕННЫХ (и мнимых) переменных
B = Matrix([
    [0, -params['g'], 0],
    [-params['a'], 0, params['b'] - 1],
])
```

```python
U = Matrix([
    [0],
    [devs['e_0']],
])
```

```python
endos = 'D_t D^*_t'
endos = utils.get_symbols_dict(endos)
```

```python
Eq(Symbol('Y'), Y, evaluate=False)
```

$\displaystyle Y = \left[\begin{matrix}D_{t}\\D^{*}_{t}\end{matrix}\right]$

```python
Eq(Symbol('X'), X, evaluate=False)
```

$\displaystyle X = \left[\begin{matrix}1\\P_{t}\\D_{t-1}\end{matrix}\right]$

```python
Eq(Symbol('A'), A, evaluate=False)
```

$\displaystyle A = \left[\begin{matrix}0 & 1\\1 & - b\end{matrix}\right]$

```python
Eq(Symbol('B'), B, evaluate=False)
```

$\displaystyle B = \left[\begin{matrix}0 & - g & 0\\- a & 0 & b - 1\end{matrix}\right]$

```python
Eq(Symbol('U'), U, evaluate=False)
```

$\displaystyle U = \left[\begin{matrix}0\\e_{0}\end{matrix}\right]$

### Приведенная форма (матричная)

```python
reduced_form_matrix
```

$\displaystyle \left[\begin{matrix}D_{t}\\D^{*}_{t}\end{matrix}\right] = \left[\begin{matrix}b & 1\\1 & 0\end{matrix}\right] \left[\begin{matrix}0\\e_{0}\end{matrix}\right] + \left[\begin{matrix}a & b g & 1 - b\\0 & g & 0\end{matrix}\right] \left[\begin{matrix}1\\P_{t}\\D_{t-1}\end{matrix}\right]$

### Приведенная форма (система)

```python
Eq(Symbol('D_t'), reduced_form_system[Symbol('D_t')], evaluate=False)
```

$\displaystyle D_{t} = - D_{t-1} b + D_{t-1} + P_{t} b g + a + e_{0}$

```python
Eq(Symbol('D^*_t'), reduced_form_system[Symbol('D^*_t')], evaluate=False)
```

$\displaystyle D^{*}_{t} = P_{t} g$

## 2.1 Модель Манделла - Флеминга (IS-LM-BP)

2.1. Необходимо составить спецификацию модификации модели IS - $L M$ - модели Манделла-Флеминга открытой экономики в краткосрочном периоде с плавающим обменным курсом.

Состояние национальной экономики в заданном периоде времени описывается количественными характеристиками $Y, C, I, N X, E, R, L$, $C F, M, P, G, T$

где Y - объем (величина) выпуска (совокупный спрос, валовый внутренний продукт);

С - уровень потребления;

I - объем инвестиций в экономику;

R - ставка процента (рефинансирования);

L - спрос на деньги в реальном выражении (не путать с уровнем затрат труда!);

М - уровень предложения денег;

Р - уровень цен;

G - объем государственных расходов;

Т - уровень налогов.

Используя приведенные ниже утверждения, составьте спецификацию макромодели Манделла—Флеминга, предназначенной для объяснения переменных Y, R ,L, I, С, NX, Е и CF значениями переменных М,Р, G и Т.

При составлении спецификации модели следует использовать следующие утверждения:
1.	Объем выпуска Y является суммой частного потребления С, инвестиций I, государственного потребления (расходов) G и чистого экспорта
NX;
2.	Уровень потребления С объясняется располагаемым доходом (Y — Т), и возрастает в ответ на увеличение этого дохода, при этом каждая дополнительная единица располагаемого дохода потребляется не полностью;
3.	Объем инвестиций в экономику / объясняется реальной ставкой процента R и снижается в ответ на рост этой ставки;
4.	Уровень спроса на деньги в реальном выражении L объясняется реальной ставкой процента R и величиной дохода Y; L снижается при увеличении R и возрастает в ответ на увеличение Y;
5.	На денежном рынке существует равновесие спроса и предложения денег в реальном выражении;
6.	Уровень чистого экспорта NX объясняется номинальным обменным курсом Е, ив ответ на увеличение номинального обменного курса уровень чистого экспорта снижается;
7.	Приток иностранного капитала CF (счет движения капитала платежного баланса) объясняется реальной ставкой процента R и увеличивается в ответ на рост этой ставки;
8.	Счет движения капитала платежного баланса CF и счет текущих операций NX уравновешивают друг друга, т.е. в сумме равны нулю.
После составления спецификации модели Манделла-Флеминга преобразуйте ее к приведенной форме. Запишите модель в матричном виде, указав вектора эндогенных и экзогенных векторов, матрицы коэффициентов при эндогенных и экзогенных переменных. Получив приведенную модель запишите результирующую матрицу \begin{equation}
\mathrm{M}=-\mathrm{A}^{-1 *} \mathrm{~B}
\end{equation}

```python
from src.task_2_1 import Y, X, A, B, U, reduced_form_matrix, reduced_form_system
```

```python
Eq(Symbol('Y'), Y, evaluate=False)
```

$\displaystyle Y = \left[\begin{matrix}Y\\C\\I\\NX\\E\\R\\L\\CF\end{matrix}\right]$

```python
Eq(Symbol('X'), X, evaluate=False)
```

$\displaystyle X = \left[\begin{matrix}1\\M\\P\\G\\T\end{matrix}\right]$

```python
Eq(Symbol('A'), A, evaluate=False)
```

$\displaystyle A = \left[\begin{matrix}1 & -1 & -1 & -1 & 0 & 0 & 0 & 0\\- b & 1 & 0 & 0 & 0 & 0 & 0 & 0\\0 & 0 & 1 & 0 & 0 & - d & 0 & 0\\- g & 0 & 0 & 0 & 0 & - h & 1 & 0\\0 & 0 & 0 & 0 & 0 & 0 & -1 & 0\\0 & 0 & 0 & 1 & - q & 0 & 0 & 0\\0 & 0 & 0 & 0 & 0 & - s & 0 & 1\\0 & 0 & 0 & 1 & 0 & 0 & 0 & 1\end{matrix}\right]$

```python
Eq(Symbol('B'), B, evaluate=False)
```

$\displaystyle B = \left[\begin{matrix}0 & 0 & 0 & -1 & 0\\- a & 0 & 0 & 0 & b\\- c & 0 & 0 & 0 & 0\\- f & 0 & 0 & 0 & 0\\0 & 1 & 0 & 0 & 0\\- p & 0 & 0 & 0 & 0\\- r & 0 & 0 & 0 & 0\\0 & 0 & 0 & 0 & 0\end{matrix}\right]$

```python
Eq(Symbol('U'), U, evaluate=False)
```

$\displaystyle U = \left[\begin{matrix}0\\e_{0}\\e_{1}\\e_{2}\\0\\e_{3}\\e_{4}\\0\end{matrix}\right]$

### Приведенная форма (матричная)

```python
reduced_form_matrix
```

$\displaystyle \left[\begin{matrix}Y\\C\\I\\NX\\E\\R\\L\\CF\end{matrix}\right] = \left[\begin{matrix}- \frac{a h}{b h - d g + g s - h} - \frac{c h}{b h - d g + g s - h} + \frac{f \left(d - s\right)}{b h - d g + g s - h} + \frac{h r}{b h - d g + g s - h} & - \frac{d - s}{b h - d g + g s - h} & 0 & - \frac{h}{b h - d g + g s - h} & \frac{b h}{b h - d g + g s - h}\\\frac{a \left(- d g + g s - h\right)}{b h - d g + g s - h} - \frac{b c h}{b h - d g + g s - h} + \frac{b h r}{b h - d g + g s - h} + \frac{f \left(b d - b s\right)}{b h - d g + g s - h} & - \frac{b d - b s}{b h - d g + g s - h} & 0 & - \frac{b h}{b h - d g + g s - h} & - \frac{b \left(- d g + g s - h\right)}{b h - d g + g s - h}\\\frac{a d g}{b h - d g + g s - h} + \frac{c \left(b h + g s - h\right)}{b h - d g + g s - h} - \frac{d g r}{b h - d g + g s - h} + \frac{f \left(- b d + d\right)}{b h - d g + g s - h} & - \frac{- b d + d}{b h - d g + g s - h} & 0 & \frac{d g}{b h - d g + g s - h} & - \frac{b d g}{b h - d g + g s - h}\\- \frac{a g s}{b h - d g + g s - h} - \frac{c g s}{b h - d g + g s - h} + \frac{f \left(b s - s\right)}{b h - d g + g s - h} + \frac{r \left(- b h + d g + h\right)}{b h - d g + g s - h} & - \frac{b s - s}{b h - d g + g s - h} & 0 & - \frac{g s}{b h - d g + g s - h} & \frac{b g s}{b h - d g + g s - h}\\- \frac{a g s}{b h q - d g q + g q s - h q} - \frac{c g s}{b h q - d g q + g q s - h q} + \frac{f \left(b s - s\right)}{b h q - d g q + g q s - h q} - \frac{p}{q} + \frac{r \left(- b h + d g + h\right)}{b h q - d g q + g q s - h q} & - \frac{b s - s}{b h q - d g q + g q s - h q} & 0 & - \frac{g s}{b h q - d g q + g q s - h q} & \frac{b g s}{b h q - d g q + g q s - h q}\\\frac{a g}{b h - d g + g s - h} + \frac{c g}{b h - d g + g s - h} + \frac{f \left(1 - b\right)}{b h - d g + g s - h} - \frac{g r}{b h - d g + g s - h} & - \frac{1 - b}{b h - d g + g s - h} & 0 & \frac{g}{b h - d g + g s - h} & - \frac{b g}{b h - d g + g s - h}\\0 & 1 & 0 & 0 & 0\\\frac{a g s}{b h - d g + g s - h} + \frac{c g s}{b h - d g + g s - h} + \frac{f \left(- b s + s\right)}{b h - d g + g s - h} + \frac{r \left(b h - d g - h\right)}{b h - d g + g s - h} & - \frac{- b s + s}{b h - d g + g s - h} & 0 & \frac{g s}{b h - d g + g s - h} & - \frac{b g s}{b h - d g + g s - h}\end{matrix}\right] \left[\begin{matrix}1\\M\\P\\G\\T\end{matrix}\right] + \left[\begin{matrix}- \frac{h}{b h - d g + g s - h} & - \frac{h}{b h - d g + g s - h} & - \frac{h}{b h - d g + g s - h} & \frac{d - s}{b h - d g + g s - h} & \frac{d - s}{b h - d g + g s - h} & 0 & \frac{h}{b h - d g + g s - h} & - \frac{h}{b h - d g + g s - h}\\- \frac{b h}{b h - d g + g s - h} & \frac{- d g + g s - h}{b h - d g + g s - h} & - \frac{b h}{b h - d g + g s - h} & \frac{b d - b s}{b h - d g + g s - h} & \frac{b d - b s}{b h - d g + g s - h} & 0 & \frac{b h}{b h - d g + g s - h} & - \frac{b h}{b h - d g + g s - h}\\\frac{d g}{b h - d g + g s - h} & \frac{d g}{b h - d g + g s - h} & \frac{b h + g s - h}{b h - d g + g s - h} & \frac{- b d + d}{b h - d g + g s - h} & \frac{- b d + d}{b h - d g + g s - h} & 0 & - \frac{d g}{b h - d g + g s - h} & \frac{d g}{b h - d g + g s - h}\\- \frac{g s}{b h - d g + g s - h} & - \frac{g s}{b h - d g + g s - h} & - \frac{g s}{b h - d g + g s - h} & \frac{b s - s}{b h - d g + g s - h} & \frac{b s - s}{b h - d g + g s - h} & 0 & \frac{- b h + d g + h}{b h - d g + g s - h} & \frac{b h - d g - h}{b h - d g + g s - h}\\- \frac{g s}{b h q - d g q + g q s - h q} & - \frac{g s}{b h q - d g q + g q s - h q} & - \frac{g s}{b h q - d g q + g q s - h q} & \frac{b s - s}{b h q - d g q + g q s - h q} & \frac{b s - s}{b h q - d g q + g q s - h q} & - \frac{1}{q} & \frac{- b h + d g + h}{b h q - d g q + g q s - h q} & \frac{b h - d g - h}{b h q - d g q + g q s - h q}\\\frac{g}{b h - d g + g s - h} & \frac{g}{b h - d g + g s - h} & \frac{g}{b h - d g + g s - h} & \frac{1 - b}{b h - d g + g s - h} & \frac{1 - b}{b h - d g + g s - h} & 0 & - \frac{g}{b h - d g + g s - h} & \frac{g}{b h - d g + g s - h}\\0 & 0 & 0 & 0 & -1 & 0 & 0 & 0\\\frac{g s}{b h - d g + g s - h} & \frac{g s}{b h - d g + g s - h} & \frac{g s}{b h - d g + g s - h} & \frac{- b s + s}{b h - d g + g s - h} & \frac{- b s + s}{b h - d g + g s - h} & 0 & \frac{b h - d g - h}{b h - d g + g s - h} & \frac{g s}{b h - d g + g s - h}\end{matrix}\right] \left[\begin{matrix}0\\e_{0}\\e_{1}\\e_{2}\\0\\e_{3}\\e_{4}\\0\end{matrix}\right]$

### Приведенная форма (система)

```python
Eq(Symbol('Y'), reduced_form_system[Symbol('Y')], evaluate=False)
```

$\displaystyle Y = \frac{- G h - M d + M s + T b h - a h - c h + d e_{2} + d f - e_{0} h - e_{1} h - e_{2} s + e_{4} h - f s + h r}{b h - d g + g s - h}$

```python
Eq(Symbol('С'), reduced_form_system[Symbol('C')], evaluate=False)
```

$\displaystyle С = \frac{- G b h - M b d + M b s + T b d g - T b g s + T b h - a d g + a g s - a h - b c h + b d e_{2} + b d f - b e_{1} h - b e_{2} s + b e_{4} h - b f s + b h r - d e_{0} g + e_{0} g s - e_{0} h}{b h - d g + g s - h}$

```python
Eq(Symbol('I'), reduced_form_system[Symbol('I')], evaluate=False)
```

$\displaystyle I = \frac{G d g + M b d - M d - T b d g + a d g + b c h - b d e_{2} - b d f + b e_{1} h + c g s - c h + d e_{0} g + d e_{2} - d e_{4} g + d f - d g r + e_{1} g s - e_{1} h}{b h - d g + g s - h}$

```python
Eq(Symbol('NX'), reduced_form_system[Symbol('NX')], evaluate=False)
```

$\displaystyle NX = \frac{- G g s - M b s + M s + T b g s - a g s + b e_{2} s - b e_{4} h + b f s - b h r - c g s + d e_{4} g + d g r - e_{0} g s - e_{1} g s - e_{2} s + e_{4} h - f s + h r}{b h - d g + g s - h}$

```python
Eq(Symbol('E'), reduced_form_system[Symbol('E')], evaluate=False)
```

$\displaystyle E = \frac{- G g s - M b s + M s + T b g s - a g s + b e_{2} s - b e_{3} h - b e_{4} h + b f s - b h p - b h r - c g s + d e_{3} g + d e_{4} g + d g p + d g r - e_{0} g s - e_{1} g s - e_{2} s - e_{3} g s + e_{3} h + e_{4} h - f s - g p s + h p + h r}{b h q - d g q + g q s - h q}$

```python
Eq(Symbol('R'), reduced_form_system[Symbol('R')], evaluate=False)
```

$\displaystyle R = \frac{G g + M b - M - T b g + a g - b e_{2} - b f + c g + e_{0} g + e_{1} g + e_{2} - e_{4} g + f - g r}{b h - d g + g s - h}$

```python
Eq(Symbol('L'), reduced_form_system[Symbol('L')], evaluate=False)
```

$\displaystyle L = M$

```python
Eq(Symbol('CF'), reduced_form_system[Symbol('CF')], evaluate=False)
```

$\displaystyle CF = \frac{G g s + M b s - M s - T b g s + a g s - b e_{2} s + b e_{4} h - b f s + b h r + c g s - d e_{4} g - d g r + e_{0} g s + e_{1} g s + e_{2} s - e_{4} h + f s - h r}{b h - d g + g s - h}$

## 2.2 Модель денежного и товарного рынков

Представленную ниже структурную форму модели преобразуйте к приведенной. Запишите приведенную форму задачи в матричном виде.
Модель денежного и товарного рынков

\begin{array}{ll}
R t=a 1+b 12 \cdot Y t+b 14 \cdot M t & \text { (функция денежного рынка) } \\
Y t=a 2+b 21 \cdot R t+b 23 \cdot I t+b 25 \cdot G t & \text { (функция товарного рынка) } \\
I t=a 3+b 31 \cdot R t & \text { (функция инвестиций) }
\end{array}

где:

\begin{aligned}
&R t - процентные\ ставки \\
&Y t - реальный\ ВВП\\
&M t - денежная\ масса\\
&I t - внутренние\ инвестиции\\
&G t - реальные\ государственные\ расходы
\end{aligned}

```python
from src.task_2_2 import Y, X, A, B, U, reduced_form_matrix, reduced_form_system
```

```python
Eq(Symbol('Y'), Y, evaluate=False)
```

$\displaystyle Y = \left[\begin{matrix}R\\Y\\I\end{matrix}\right]$

```python
Eq(Symbol('X'), X, evaluate=False)
```

$\displaystyle X = \left[\begin{matrix}1\\M\\G\end{matrix}\right]$

```python
Eq(Symbol('A'), A, evaluate=False)
```

$\displaystyle A = \left[\begin{matrix}1 & - b_{12} & 0\\- b_{21} & 1 & - b_{23}\\- b_{31} & 0 & 1\end{matrix}\right]$

```python
Eq(Symbol('B'), B, evaluate=False)
```

$\displaystyle B = \left[\begin{matrix}- a_{1} & - b_{14} & 0\\- a_{2} & 0 & - b_{25}\\- a_{3} & 0 & 0\end{matrix}\right]$

```python
Eq(Symbol('U'), U, evaluate=False)
```

$\displaystyle U = \left[\begin{matrix}0\\0\\0\end{matrix}\right]$

### Приведенная форма (матричная)

```python
reduced_form_matrix
```

$\displaystyle \left[\begin{matrix}R\\Y\\I\end{matrix}\right] = \left[\begin{matrix}- \frac{a_{1}}{b_{12} b_{21} + b_{12} b_{23} b_{31} - 1} - \frac{a_{2} b_{12}}{b_{12} b_{21} + b_{12} b_{23} b_{31} - 1} - \frac{a_{3} b_{12} b_{23}}{b_{12} b_{21} + b_{12} b_{23} b_{31} - 1} & - \frac{b_{14}}{b_{12} b_{21} + b_{12} b_{23} b_{31} - 1} & - \frac{b_{12} b_{25}}{b_{12} b_{21} + b_{12} b_{23} b_{31} - 1}\\\frac{a_{1} \left(- b_{21} - b_{23} b_{31}\right)}{b_{12} b_{21} + b_{12} b_{23} b_{31} - 1} - \frac{a_{2}}{b_{12} b_{21} + b_{12} b_{23} b_{31} - 1} - \frac{a_{3} b_{23}}{b_{12} b_{21} + b_{12} b_{23} b_{31} - 1} & \frac{b_{14} \left(- b_{21} - b_{23} b_{31}\right)}{b_{12} b_{21} + b_{12} b_{23} b_{31} - 1} & - \frac{b_{25}}{b_{12} b_{21} + b_{12} b_{23} b_{31} - 1}\\- \frac{a_{1} b_{31}}{b_{12} b_{21} + b_{12} b_{23} b_{31} - 1} - \frac{a_{2} b_{12} b_{31}}{b_{12} b_{21} + b_{12} b_{23} b_{31} - 1} + \frac{a_{3} \left(b_{12} b_{21} - 1\right)}{b_{12} b_{21} + b_{12} b_{23} b_{31} - 1} & - \frac{b_{14} b_{31}}{b_{12} b_{21} + b_{12} b_{23} b_{31} - 1} & - \frac{b_{12} b_{25} b_{31}}{b_{12} b_{21} + b_{12} b_{23} b_{31} - 1}\end{matrix}\right] \left[\begin{matrix}1\\M\\G\end{matrix}\right]$

### Приведенная форма (система)

```python
Eq(Symbol('R'), reduced_form_system[Symbol('R')], evaluate=False)
```

$\displaystyle R = \frac{- G b_{12} b_{25} - M b_{14} - a_{1} - a_{2} b_{12} - a_{3} b_{12} b_{23}}{b_{12} b_{21} + b_{12} b_{23} b_{31} - 1}$

```python
Eq(Symbol('I'), reduced_form_system[Symbol('I')], evaluate=False)
```

$\displaystyle I = \frac{- G b_{12} b_{25} b_{31} - M b_{14} b_{31} - a_{1} b_{31} - a_{2} b_{12} b_{31} + a_{3} b_{12} b_{21} - a_{3}}{b_{12} b_{21} + b_{12} b_{23} b_{31} - 1}$

```python
Eq(Symbol('Y'), reduced_form_system[Symbol('Y')], evaluate=False)
```

$\displaystyle Y = \frac{- G b_{25} - M b_{14} b_{21} - M b_{14} b_{23} b_{31} - a_{1} b_{21} - a_{1} b_{23} b_{31} - a_{2} - a_{3} b_{23}}{b_{12} b_{21} + b_{12} b_{23} b_{31} - 1}$

## 2.3 Гипотетическая модель экономики

Представленную ниже структурную форму модели преобразуйте к приведенной. Запишите приведенную форму задачи в матричном виде.
Гипотетическая модель экономики:
\begin{aligned}
&C t=a 1+b 11 \cdot Y t+b 12 \cdot J t \\
&J t=a 2+b 21 \cdot Y t-1 \\
&T t=a 3+b 31 \cdot Y t \\
&Y t=C t+J t+G t
\end{aligned}

где:

\begin{aligned}
&C t - совокупное\ потребление\ в\ период t\\
&Y t - совокупный\ доход\ в\ период\ t\\
&J t - инвестиции\ в\ период\ t\\
&T t - налоги\ в\ период\ t\\
&G t - государственные\ доходы\ в\ период\ t
\end{aligned}

```python
from src.task_2_3 import Y, X, A, B, U, reduced_form_matrix, reduced_form_system
```

```python
Eq(Symbol('Y'), Y, evaluate=False)
```

$\displaystyle Y = \left[\begin{matrix}C\\J\\T\\Y\end{matrix}\right]$

```python
Eq(Symbol('X'), X, evaluate=False)
```

$\displaystyle X = \left[\begin{matrix}1\\Y_{t-1}\\G\end{matrix}\right]$

```python
Eq(Symbol('A'), A, evaluate=False)
```

$\displaystyle A = \left[\begin{matrix}1 & - b_{12} & 0 & - b_{11}\\0 & 1 & 0 & 0\\0 & 0 & 1 & - b_{31}\\-1 & -1 & 0 & 1\end{matrix}\right]$

```python
Eq(Symbol('B'), B, evaluate=False)
```

$\displaystyle B = \left[\begin{matrix}- a_{1} & 0 & 0\\- a_{2} & - b_{21} & 0\\- a_{3} & 0 & 0\\0 & 0 & -1\end{matrix}\right]$

```python
Eq(Symbol('U'), U, evaluate=False)
```

$\displaystyle U = \left[\begin{matrix}0\\0\\0\\0\end{matrix}\right]$

### Приведенная форма (матричная)

```python
reduced_form_matrix
```

$\displaystyle \left[\begin{matrix}C\\J\\T\\Y\end{matrix}\right] = \left[\begin{matrix}\frac{a_{1}}{1 - b_{11}} + \frac{a_{2} \left(b_{11} + b_{12}\right)}{1 - b_{11}} & \frac{b_{21} \left(b_{11} + b_{12}\right)}{1 - b_{11}} & \frac{b_{11}}{1 - b_{11}}\\a_{2} & b_{21} & 0\\\frac{a_{1} b_{31}}{1 - b_{11}} + \frac{a_{2} \left(- b_{12} b_{31} - b_{31}\right)}{b_{11} - 1} + a_{3} & \frac{b_{21} \left(- b_{12} b_{31} - b_{31}\right)}{b_{11} - 1} & \frac{b_{31}}{1 - b_{11}}\\\frac{a_{1}}{1 - b_{11}} - a_{2} \left(- \frac{b_{12}}{1 - b_{11}} - \frac{1}{1 - b_{11}}\right) & - b_{21} \left(- \frac{b_{12}}{1 - b_{11}} - \frac{1}{1 - b_{11}}\right) & \frac{1}{1 - b_{11}}\end{matrix}\right] \left[\begin{matrix}1\\Y_{t-1}\\G\end{matrix}\right]$

### Приведенная форма (система)

```python
Eq(Symbol('C'), reduced_form_system[Symbol('C')], evaluate=False)
```

$\displaystyle C = \frac{- G b_{11} - Y_{t-1} b_{11} b_{21} - Y_{t-1} b_{12} b_{21} - a_{1} - a_{2} b_{11} - a_{2} b_{12}}{b_{11} - 1}$

```python
Eq(Symbol('J'), reduced_form_system[Symbol('J')], evaluate=False)
```

$\displaystyle J = Y_{t-1} b_{21} + a_{2}$

```python
Eq(Symbol('T'), reduced_form_system[Symbol('T')], evaluate=False)
```

$\displaystyle T = \frac{- G b_{31} - Y_{t-1} b_{12} b_{21} b_{31} - Y_{t-1} b_{21} b_{31} - a_{1} b_{31} - a_{2} b_{12} b_{31} - a_{2} b_{31} + a_{3} b_{11} - a_{3}}{b_{11} - 1}$

```python
Eq(Symbol('Y'), reduced_form_system[Symbol('Y')], evaluate=False)
```

$\displaystyle Y = \frac{- G - Y_{t-1} b_{12} b_{21} - Y_{t-1} b_{21} - a_{1} - a_{2} b_{12} - a_{2}}{b_{11} - 1}$

## 2.4. Форма отчета по разделу 2 «Составление спецификации эконометрической модели»

По проведенным исследованиям моделей в п. 2.1-2.3 представить:
-	запись исходной спецификации модели, т.е. структурной ее формы;
-	запись приведенной формы модели;
-	матричная запись моделей: вектор эндогенных переменных, вектор экзогенных переменных, вектор случайных возмущений, матрицу А, матрицу В, матрицу М - приведенной формы модели \begin{equation}
\mathrm{M}=-\mathrm{A}^{-1 *} \mathrm{~B}
\end{equation}

```python

```

```python

```