```julia
using BenchmarkTools, SpecialFunctions
using LinearAlgebra, Statistics
using Parameters
using Plots
using Optim #https://github.com/JuliaNLSolvers/Optim.jl
using Random, Distributions
gr(fmt=:png); # setting for easier display in jupyter notebooks
```

**Introduction**

I'm teaching myself ``julia``. The simplest way for me todo so is to work with a model that I know extremely well---the Ricardian model of Eaton and Kortum (2002). So below I will outline the economic environment and then use the properties of ``julia`` to map the fundamental, mathematical environment into a computational/quantitative outcome. Outside of performance properties, the more I reflect on this, this is a huge appeal of the ``julia`` language. We can express the code and design it in a way that most closely corresponds with the math/model. In contrast, in for example MATLAB, where the map from white board to computation is big.

### EK (2002) Ricardian Model

**Countries.** To keep things simple, let's first work with a simple two country set up---a home and a foreign country, $h$, $f$.

**Individual Goods Production.**
Within a country, there is a continuum of intermediate goods indexed by $\omega \in [0, 1]$. As in the Ricardian model of \citet{dornbusch1977comparative} and \citet{eaton2002technology}, intermediate goods are not nationally differentiated. What this means is that intermediate $\omega$ produced in one country is a perfect substitute for the same intermediate $\omega$ produced by another country.

Competitive firms produce intermediate goods with linear production technologies:

\begin{align}
q(\omega) = z(\omega) \ell,
\end{align}

where $z$ is the productivity level of firms and $\ell$ is the number of efficiency units of labor.

To ship a technology to another country, iceberg costs are incurred. I will write this as $\tau_{hf}$ which is the cost the home country must incur to buy from the foreign country. The notation here will correspond with the row, column format of a matrix, so for a given row all the trade costs across the different columns are the ones that they face to buy from different countries. Then the usual normalization here is that $\tau_{hh} = 1$. Again, on the computer, this means that the diagonal entries are one.

**The Aggregator.** In this model, intermediate goods are aggregated by a competitive final-goods producer. In Julia, what would be interesting is to make the aggregator generic and let the user specify how this works. But for know, we will follow the canonical situation with a standard CES production function:

\begin{align}
Q = \left[ \int_0^1 q(\omega)^{\frac{\sigma - 1}{\sigma}} d\omega \right]^{\frac{\sigma}{\sigma - 1}},
\label{eq:ces}
\end{align}

where $q(\omega)$ is the quantity of individual intermediate goods $\omega$ demanded by the final-goods firm, and $\sigma$ is elasticity of substitution across variety. I want to talk through some properties/definitions related to this aggregator.

- The aggregator in (\ref{eq:ces}) gives rise to the following the demand curve (in quantities) for an individual variety:

\begin{align}
q(\omega) & = \left(\frac{p_h(\omega)}{P_h}\right)^{-\sigma}Q.
\label{eq:demand_curve}
\end{align}

where $Q$ is the aggregate demand for the final good; $P_h$ is the CES price index, defined below; And $p_h(\omega)$ is the price paid by the home country for good $\omega$ (which is not the same necessarily as the price offered by a home country supplier).

- The demand curve can also be expressed in shares, not quantities. So

\begin{align}
\frac{p_h(\omega)q(\omega)}{P_h Q} & = \left(\frac{p_h(\omega)}{P_h}\right)^{1-\sigma}.
\label{eq:demand_curve_share}
\end{align}

which says that the value of of good $\omega$ relative to total expenditures $P_h Q$ is given by the formula above. This helps as we can compute expenditure patterns while being agnostic about the actual size of the economy, i.e. $Q$. One useful relationship that builds on this is to condition on the source and aggregate to arrive at the share of goods purchased from another country. How does this work?

\begin{align}
\pi_{h,f} = \int_{p_h = p_{h,f}} \left(\frac{p_h(\omega)}{P_h}\right)^{1-\sigma} d\omega
\end{align}

Where the notation here $\pi_{h,f}$ is the share of goods that the home country sources from the foreign country. This is equal do the integral over all goods purchased from them, hence, the conditioning where $p_h = p_{h,f}$ and then the inside of the brackets is the same as above. In the computer, the code will be setup with the same row, column notation. So fix a row, then as you go across the different columns you'll view the share of expenditures the, say home country, coming from other countries.

A small side note: If you stare at this enough (and know the liturature), than you can start to see a gravity like relationship here without even imposing the Frechet assumption. The key issue is the conditional distribution of goods prices and how it relates to the aggregate, home CES price index discussed next.

- The **CES Price Index** is given by

\begin{align}
P_h  = \left [ \int_{0}^{1} p_h(\omega)^{1-\sigma} d\omega \right]^{\frac{1}{1-\sigma}}
\label{eq:ces_price_index}
\end{align}

which takes all the prices at the micro-level and is like a harmonic mean of these prices. Also note how one can substitute \ref{eq:ces_price_index} into the share formula to get an expression of expenditures shares purely in terms of micro-level prices.

**The Technology Distribution**

This is the key component that makes Eaton and Kortum fly. First, let's be a bit more general here. One could simply think of each countries **individual productivity** draws being characterized by some distribution:

\begin{align}
Z_h \sim F_{h}(z)
\end{align}

where $F_{h}$ is a well defined cumulative probability distribution with the associated pdf $f_h(z)$. What this means is for each $\omega$ variety, in say the home country, the technology/productivity to produce that good is a random $z$ which is characterized by the distribution above. As I'll talk about below, what this allows one to think about is (i) if you hand me a distribution from which I can simulate from (ii) then with assumptions on market structure I can construct prices and (iii) with prices I can construct trade flows.

Now Eaton and Kortum make a very specific distributional assumption. They use a Type II extreme value distribution also known as a Frechet:

\begin{align}
Z_i \sim \exp\left(-T_i z^{-\theta}\right)
\end{align}

where the distribution is characterized by a centering parameter $T_i$ which is allowed to vary across countries and a shape parameter $\theta$ which is assumed to be constant across countries. These assumptions are critical to the closed form characterizations in Eaton and Kortum. However, return to the discussion above. On the computer there are only two limiting factors (i) can you simulate from the distribution and (ii) the empirical question of how to discipline the distribution from which you are simulating.

**Optimization**

I'm going to walk through a couple of observations about how this economy works.

The first observation is that within a country, the key equilibrium price is the wage rate $w$. This wage is the same independent of the labor market / goods-producing sector / etc. This derives from the (within-country) free mobility of labor assumption (which I did not talk about). In some other notebooks, I'll relax this. But for now, labor is freely mobile and now we just need to find the wage so that labor demand equals labor supply.

The second observation is that given competitive firms and that they face the wage rate $w$, we know that their marginal cost (and hence price at which they are willing to sell the good domestically) must be:

\begin{align}
p_{h,h}(\omega) = \frac{w_{h}}{z_{h}(\omega)},
\label{eq:ek_wage}
\end{align}

and then the price at which they would be willing to export the good, say from home to foreign is just adjusted by the ice-berg trade costs so

\begin{align}
p_{f,h}(\omega) = \frac{\tau_{f,h} w_{h}}{z_{h}(\omega)}.
\end{align}

The third question is who buys what from whom? This aspect of the model is that really is just an application of the min operator. So buy the good from the country with the lowest price:

\begin{align}
p_{h}(\omega) = \min\left\{ \frac{w_{h}}{z_{h}(\omega)}, \ \frac{\tau_{h,f} w_{f}}{z_{f}(\omega)} \right\}
\end{align}

where the thing on the left is the domestic price, and then the thing on the right is on the world price (inclusive of the trade cost).

### Equilibrium

I'm going to present this in a way slightly different than usual. So I want to think of the wage as clearing the labor market. This implies that goods demand will equal goods supply which will reduce to balanced trade. The idea here is to build up into some later notebooks where we can think of more sophisticated labor market settings with elastic labor supply, mobility/migration frictions similar to my work with Spencer.

**Labor Demand** so the key idea here is the the CES demand curve also is a labor demand curve when combined with the production function. First, consider the simple case of a closed economy. Here we have:

\begin{align}
\ell(\omega) & = \frac{1}{z(\omega)}\left(\frac{w / z(\omega)}{P_h}\right)^{-\sigma}Q.
\label{eq:labor_demand_curve}
\end{align}

where I substituted in the price equaling marginal costs, which after pulling things out we have that

\begin{align}
\ell_h(\omega) & = z_h(\omega)^{\sigma - 1}\left(\frac{w_h}{P_h} \right)^{-\sigma}Q.
\label{eq:demand_curve2}
\end{align}

which has the inuitive idea that (i) if productivity is high or aggregate demand is high you need **more workers** and (ii) if the wage is high you demand less workers. Now let's extend this to the open economy setting. So labor demand is

\begin{align}
\ell_h(\omega) & = \underbrace{\frac{1}{z_h(\omega)}\left(\frac{w_h / z_h(\omega)}{P_h}\right)^{-\sigma}Q_h}_{\mbox{domestic demand}} \ + \ \underbrace{\frac{1}{z_h(\omega)}\left(\frac{\tau_{f,h} w_h / z_h(\omega)}{P_f}\right)^{-\sigma}Q_f}_{\mbox{foreign demand}}
\end{align}

where again, I substituted in the price for good $\omega$ in the domestic demand curve and (if purchased) into the foreign demand curve. Note that this is a generic representation in the sense that domestic demand and foreign demand might be zero. In this case, the good is imported from abroad. Or it might be only domesticly demanded (which in this cases the good is non traded) and then it's both bought at home and sent abroad (so that good is exported).

Then one way to think about an equilibrium is we need to find a wage vector $\mathbb{w}$ such labor supply equals labor demand. So

\begin{align}
L_h^{D} = \int \ell_h(\omega) d\omega = \int_{p_h = p_{h,h}} \frac{1}{z(\omega)}\left(\frac{w_h / z(\omega)}{P_h}\right)^{-\sigma}Q_h + \int_{p_f = p_{f,h}} \frac{\tau_{f,h}}{z(\omega)}\left(\frac{\tau_{f,h} w_h / z(\omega)}{P_f}\right)^{-\sigma}Q_f
\end{align}

and we would require that $w$ be such that $L_i^{D}(\mathbb{w}) = L^{S}$.

**Goods Demand and Goods Supply** This is an alternative way to find the equillibrium wage vector via the goods market. Now there are in principal two ways to go about this. One is to work from the income side. We know that total income (and spending) in the country is

\begin{align}
L_h w_h
\end{align}

which because all this income is only spent on goods, this is the same as goods demand. Then we know that income recived is related to spending and the expenditure shares. So working from the perspective of the home country, we have:

\begin{align}
\sum_{i} L_i w_i \pi_{i,h} (\mathbb{w})
\end{align}

which is the sum of each country's expenditure on goods from home. In other words, these are the payments for all the production in the home country. Now this sum has a nice matrix representation.If you define $L_i w_i = Y_i$ which is the typical element in the vector $\mathbb{Y}$, the vector of income for the production of goods across all countries is:

\begin{align}
\mathbb{Y}'(\mathbb{w}) \times \mathbb{\pi}(\mathbb{w})
\end{align}

which must equall total spending which is $\mathbb{Y}'(\mathbb{w})$ in equillibrium.

### Computing

Given everything laid out, some wages $w_i$ and then primitives regarding technologies $T$, $\theta$, $\sigma$ and $\tau$ we can compute an equillibrium and the objects associated with it, i.e. the pattern of trade, the price index, quantities, etc. Now to compute the equillibrium, we do need to computationally compute the integrals above. I'm going to explore two approaches:

##### Simmulation
One approach is to evaluate the integrals by simulation: the idea will be to think of the product space $\omega$ as an integer on the real line which indexes the goods location. That is $\omega = 1,2,...$ Then for each of those products, each country is randomly assigned a productivity from the distribution. Then the number of goods, so to speak, is set to be large and thus will approximate the integrals above. So what this means is, for example, the expenditure share of goods that country $h$ buys from country $f$ is:

\begin{align}
\pi_{h,f} =  (1 / N) \sum_{\omega = 1}^N  \mathbb{1}\{p_h = p_{h,f}\} \left(\frac{p_h(\omega)}{P_h}\right)^{1-\sigma}
\end{align}

Note that the probability country $h$ sources from country $f$ is
\begin{align}
\hat \pi_{h,f} =  (1 / N) \sum_{\omega = 1}^N  \mathbb{1}\{p_h = p_{h,f}\}
\end{align}

where it turns out that the sourcing probability happens to correspond with the expenditure share, so $\pi_{h,f} = \hat \pi_{h,f}$. This latter point is very subtle, special to the Frechet distribution + CES. One way to see what is going on is to note that if (this is super loose, but in the discretized case I think you can see this easier) the expected valye of imported $p_{h,f}$ goods is the same as the aggregate price index, then things cancel and one could see the correspondence.

Now, to compute the expenditure shares, we must compute the CES price index. This is computed as:

\begin{align}
P_h  = \left [ (1 / N) \sum_{\omega = 1}^N p_h(\omega)^{1-\sigma} \right]^{\frac{1}{1-\sigma}}
\end{align}

And labor demand is computed as:

\begin{align}
L^D = (1 / N) \sum_{\omega = 1}^N  \mathbb{1}\{p_h = p_{h,h}\}  \frac{1}{z(\omega)}\left(\frac{w_h / z_h(\omega)}{P_h}\right)^{-\sigma}Q_h +
(1 / N) \sum_{\omega = 1}^N  \mathbb{1}\{p_f = p_{f,h}\} \frac{\tau_{f,h}}{z(\omega)}\left(\frac{\tau_{f,h} w_h / z_f(\omega)}{P_f}\right)^{-\sigma}Q_f
\end{align}

- The CES function ``ces_shares`` that takes prices and sourcing decisions to compute trade shares.

- A function that computes potential prices based on marginal costs, ``marginal_costs``

- A function that constructs a vector of productivity for each good, across all countries, given the Frechet distribution, this is the ``make_productivity`` function. Note, future, it would be nice if the distribution type could be directly passed into this and also appropriately indexed. Maybe with a dictionary?

```julia
# this creates a tuple with the named parameters
# then with the @unpack command (from parameters)
# you can simply grab from it what you need...
# so pp = ekparams(θ = 5.0) creates the parameter structure
# but with theta = 5. if not specified, below are the defalut values.
# for example @unpack θ, T, σ = pp; only pulls out
# the theta, T, sigma from the

ekparams = @with_kw (θ = 4.0,
    T = [1.0 2.5],
    τ_matrix = [1.0 1.5; 1.5 1.0],
    L = [1.0 1.0],
    ngoods = 10000,
    ncntry = ndims(τ_matrix),
    σ = 2.0,
    seed = 3281978
    )
```

    ##NamedTuple_kw#253 (generic function with 2 methods)

```julia
# This is me trying multiple dispatch
# so the ces function will do different things depending upon
# the arguments passed. The frist one deliivers the shares
# but if aggregate demand is passed (Q), then you get out the
# real quantities.

function ces(p, σ)
    ngoods = size(p)[1]

    return ( sum( p.^(one(σ)-σ) ) / ngoods ).^( one(σ) ./ (one(σ) - σ) )

    # So if it's just small p, then kick back the price index

    # I'm using the one(sigma) to make sure type stable. maybe
    # this might be doing ints on floats here...need to think more
    # about this, use promotions?

end

function ces(p, P, σ)

    return (p ./ P).^(one(σ) - σ)
    # So if it has the big P, it return the expenditure share

end

function ces(p, P, Q, σ)

    return ((p ./ P).^(- σ) ).*Q
    # return the quantity if big Q is included

end
```

    ces (generic function with 3 methods)

```julia
function ces_shares(p, source_country::Array{CartesianIndex{2},2}, σ = 2.0)
    # this will take in (i) prices **purchased at** and
    # a cartesian index that can determine country supplier

    ngoods = length(p) # number of goods

    ncntry = ndims(source_country) # number of coutnries

    price_index = ces(p, σ) # CES price index

    # now construct the shares...
    trade_shares = zeros(ncntry)

    source_prob = zeros(ncntry)

    for cntry = 1:ncntry
        # This is the loop that conditions on the source country
        # first fix a country...

        supplier = [ zzz[2] == cntry for zzz in source_country ]
        # source_country is a 2dim caartesian index which
        # says for each good (first dimension), which country is the low cost
        # supplier (second country).
        #
        # this is then a simple inline expression saying, for each
        # good (so every entry along the first dimension), which values in the second
        # dimension zzz[2] are equall to the country in question "cntry"
        # the result is supplier which is a vector of logicals, with true (one)
        # values indicating that "cntry: is the low cost supplier
        #
        # To map to the math above, this is compute the indicator functions
        # for each good, from the perspective of each country.

        source_prob[cntry] = (1 / ngoods) * sum(supplier)
        # This will compute the simple probability of being the low cost
        # supplier, by summing across the indicator/bools
        # In the Frechet model = trade share. Alternative distributional
        # assumptions will deliver something different.

        trade_shares[cntry] = (1 / ngoods) * sum(ces(p[supplier], price_index, σ))
        # then this is just the ces demand curve described above.
        # sum across p's, conditional on it being supplied,
        # then divide by number of goods

    end

    return price_index, trade_shares, source_prob

end
```

    ces_shares (generic function with 2 methods)

```julia
function marginal_cost(z, wage, τ)

    return (τ.*wage)./z

end
```

    marginal_cost (generic function with 1 method)

```julia
function make_productity(model_params)

    @unpack θ, T, ncntry, ngoods, seed = model_params;

    z = zeros(ngoods, ncntry)

    for cntry = 1:ncntry

        dist = Frechet(θ, T[cntry]^(one(θ)/θ))
        # Ideally, it would be cool to potentially be able to pass as an argument
        # the distibution for each country or something like that.

        z[:,cntry] = rand(MersenneTwister(seed + cntry), dist, ngoods, 1);
        #I tried the rand! command (if modifies the argument),
        # it did not work...seemed to rewrite everytime
        # this thing looped through.
        # MersenneTwister is the seed setter for the random number generator...
        # works simmilar to matlab...takes an integer and then specifies

    end

    return z

end
```

    make_productity (generic function with 1 method)

```julia
function low_cost_supplier(z, w, τ)

    return p, source_country = findmin(marginal_cost(z, w, τ); dims = 2);

end
```

    low_cost_supplier (generic function with 1 method)

```julia
function equillibrium_evaluate(w, z, model_params)

     @unpack ncntry, ngoods, τ_matrix, L = model_params;

    price_index = similar(w)
    trade_shares = zeros(ncntry,ncntry)
    source_prob = similar(trade_shares)
    source_country = Array{CartesianIndex{2}}(undef, size(z))
    # this I thought I knew what I was doing, not sure now

    for cntry = 1:ncntry

        p, sc = low_cost_supplier(z, w, τ_matrix[cntry,:]');
            # find the min over marginal costs
            # note that the tau_matrix returns a vector, not a row vector as I was anticipating
            # so take the transpose to make it work.

        (price_index[cntry], trade_shares[cntry,:], source_prob[cntry,:]) = ces_shares(p, sc);
            # then this is set up so the row is the buyer...then you go across the columns
            # and that is share of goods the home country buys from that country.

        source_country[:, cntry] = sc ;
    end

    value_expenditure = (L.*w)*trade_shares ;
    # Need to spell this out in markdown
    # Note this is a matrix operation. Lw is N x 1, trades shares is N x N
    # so this gives a N x 1 vector of expendeture.

    value_production = L.*w ;

    return norm(value_production - value_expenditure), trade_shares, price_index, source_country, source_prob

end
##########################################################################################################
# here again, I'm going to try and explit multiple dispatch. So if I pass an additional variable then it
# will compute the equillibrium via labor demand

##########################################################################################################

function equillibrium_evaluate(w, z, model_params, ld_flag)

     @unpack ncntry, ngoods, τ_matrix, L = model_params;

    price_index = similar(w)
    agg_l_demand = similar(w)
    trade_shares = zeros(ncntry,ncntry)
    source_prob = similar(trade_shares)
    source_country = Array{CartesianIndex{2}}(undef, size(z))
    # this I thought I knew what I was doing, not sure now

    for cntry = 1:ncntry

        p, sc = low_cost_supplier(z, w, τ_matrix[cntry,:]');
            # find the min over marginal costs
            # note that the tau_matrix returns a vector, not a row vector as I was anticipating
            # so take the transpose to make it work.

        (price_index[cntry], trade_shares[cntry,:], source_prob[cntry,:]) = ces_shares(p, sc);
            # then this is set up so the row is the buyer...then you go across the columns
            # and that is share of goods the home country buys from that country.

        source_country[:, cntry] = sc ;
    end

    endogenous_var = @with_kw (source_country = source_country,
    z = z,
    w = w,
    price_index = price_index,
    Q =  L.*w ./ price_index)

    for cntry = 1:ncntry

        agg_l_demand[cntry] = labor_demand(endogenous_var(), cntry, model_params)[1];

    end

    return norm(agg_l_demand - L), trade_shares, price_index, source_country, source_prob

end
```

    equillibrium_evaluate (generic function with 2 methods)

```julia
function labor_demand(endogenous_var, country, model_params)

    @unpack source_country, z,  w, price_index, Q = endogenous_var;

    @unpack τ_matrix, σ = model_params;

    destinations = Array{Bool}(undef, size(z))
    ncntry = size(z)[2]
    ngoods = size(z)[1]

    for cntry = 1:ncntry

        destinations[:, cntry] = [ zzz[2] == country for zzz in source_country[:, cntry] ]
        # this is different than the ces demand line. it says for a given country, go through
        # each country low cost supplier and find the places where that given country is the
        # low cost supplier.
        # Then the destination country is a matrix of bools so indicating if country
        # is the low cost supplier to those different (columns) destiation.
        # really this is like from the exporter side....

    end

    mc = marginal_cost(z[:, country], w[country], τ_matrix[:, country]')

    q_demand = ces(mc, price_index, Q, σ)
    # here I'm using the multiple dispatch of ces...when I pass
    # the z argument to it, it returns labor demand

    l_demand = q_demand ./ ( z[:, country] ./  τ_matrix[:, country]' ) ./ ngoods

    agg_l_demand = sum(sum(l_demand.*destinations, dims = 2))

    return agg_l_demand, l_demand.*destinations, destinations

end
```

    labor_demand (generic function with 1 method)

```julia
model_params = ekparams(ngoods = 100000, θ = 4.0)

z = make_productity(model_params)

ld_flag = true

f(x) = equillibrium_evaluate(([1.0 x]), z, model_params)[1]
# this is like creating an inline function in matlab. much easier
# just create f(x) where x is the variable of interest.
# again, the ld_flag if present uses the labor demand condition

# This is an example of univarite, between bounds optimization

result = optimize(f, (0.05), (5.0),  show_trace = true,
                             show_every = 10 )

```

    Iter     Function value      Lower bound       Upper bound       Best bound
         0     1.403193e+00      5.000000e-02      5.000000e+00      initial
        10     8.637413e-04      1.217818e+00      1.219812e+00      upper
        20     5.582012e-07      1.219045e+00      1.219051e+00      lower

    Results of Optimization Algorithm
     * Algorithm: Brent's Method
     * Search Interval: [0.050000, 5.000000]
     * Minimizer: 1.219048e+00
     * Minimum: 2.141659e-09
     * Iterations: 27
     * Convergence: max(|x - x_upper|, |x - x_lower|) <= 2*(1.5e-08*|x|+2.2e-16): true
     * Objective Function Calls: 28

```julia
println(result.minimizer)
```

    1.2190478905339983

```julia
println(result.minimizer)
```

    1.2190478905339983

```julia
w = [1.0 result.minimizer]

obj = equillibrium_evaluate(w, z, model_params)

println("expenditure shares: ", obj[2])

println("sourcing probabilities: ", obj[5])
```

    expenditure shares: [0.819257083393353 0.18074291660664696; 0.14826564200185333 0.8517343579981468]
    sourcing probabilities: [0.8189400000000001 0.18106000000000003; 0.14831000000000003 0.8516900000000001]

```julia
@unpack τ_matrix = model_params;

p, sc = low_cost_supplier(z, w, τ_matrix[1,:]');

mc = marginal_cost(z, w, τ_matrix[1,:]');

histogram(p, bins = 100, alpha = 0.5, color = "dark blue", label = "traded prices", xlabel = "prices/marginal costs")
histogram!(mc[:,1], bins = 100, alpha = 0.5, color = "red", label = "domestic marginal costs")

# Note there is something up with the bins here. Eg. it is overriding my stuff...
# not sure why.
```

```julia
obj = equillibrium_evaluate(w, z, model_params)

@unpack L = model_params;

endogenous_var = @with_kw (source_country = obj[4],
    z = z,
    w = w,
    price_index = obj[3],
    Q =  L.*w ./ price_index,
    )

agg_l_demand, ld, srcprb = labor_demand(endogenous_var(), 1, model_params);

lall = sum(ld[ld[:,1] .> 0.0, :], dims = 2)

lexp = sum(ld[ld[:,2] .> 0.0, :], dims = 2)

histogram(log.(lall), bins = 100, alpha = 0.25, color = "dark blue", xlabel = "log labor demand", label = "all goods")

histogram!(log.(lexp), bins = 100, alpha = 0.5, color = "red", label = "exported goods")

# same deal and then the histogram looks messed up as the binning is
# different
```

```julia

```

```julia
ql = sum(sum(ld, dims = 2));

print("Labor Demand ", string(ql)[1:5])
```

    Labor Demand 1.000

```julia
agg_l_demand
```

    0.9999999995753414

```julia
@btime f(result.minimizer)
```

      6.658 ms (83 allocations: 15.64 MiB)

    2.1416585547327893e-9

```julia
@btime f(result.minimizer)
```

      6.756 ms (83 allocations: 15.64 MiB)

    2.1416585547327893e-9

```julia
@btime labor_demand(endogenous_var(), 1, model_params)
```

      17.141 ms (39 allocations: 13.35 MiB)

    (1.000000001514381, [1.1583083039934047e-5 0.0; 9.29867201075947e-6 0.0; … ; 0.0 0.0; 6.846828387465427e-6 0.0], Bool[1 0; 1 0; … ; 0 0; 1 0])

```julia
@btime labor_demand(endogenous_var(), 1, model_params)
```

      17.185 ms (39 allocations: 13.35 MiB)

    (1.000000001514381, [1.1583083039934047e-5 0.0; 9.29867201075947e-6 0.0; … ; 0.0 0.0; 6.846828387465427e-6 0.0], Bool[1 0; 1 0; … ; 0 0; 1 0])

```julia
@code_warntype equillibrium_evaluate(w, z, model_params)
```

    Variables
      #self#::Core.Compiler.Const(equillibrium_evaluate, false)
      w::Array{Float64,2}
      z::Array{Float64,2}
      model_params::NamedTuple{(:θ, :T, :τ_matrix, :L, :ngoods, :ncntry, :σ, :seed),Tuple{Float64,Array{Float64,2},Array{Float64,2},Array{Float64,2},Int64,Int64,Float64,Int64}}
      255::NamedTuple{(:θ, :T, :τ_matrix, :L, :ngoods, :ncntry, :σ, :seed),Tuple{Float64,Array{Float64,2},Array{Float64,2},Array{Float64,2},Int64,Int64,Float64,Int64}}
      ncntry::Int64
      ngoods::Int64
      τ_matrix::Array{Float64,2}
      L::Array{Float64,2}
      price_index::Array{Float64,2}
      trade_shares::Array{Float64,2}
      source_prob::Array{Float64,2}
      source_country::Array{CartesianIndex{2},2}
      @_14::Union{Nothing, Tuple{Int64,Int64}}
      value_expenditure::Array{Float64,2}
      value_production::Array{Float64,2}
      cntry::Int64
      @_18::Int64
      @_19::Int64
      p::Array{Float64,2}
      sc::Array{CartesianIndex{2},2}

    Body::Tuple{Float64,Array{Float64,2},Array{Float64,2},Array{CartesianIndex{2},2},Array{Float64,2}}
    1 ─       Core.NewvarNode(:(value_expenditure))
    │         Core.NewvarNode(:(value_production))
    │         (255 = model_params)
    │   %4  = Base.getproperty(UnPack, :unpack)::Core.Compiler.Const(UnPack.unpack, false)
    │   %5  = 255::NamedTuple{(:θ, :T, :τ_matrix, :L, :ngoods, :ncntry, :σ, :seed),Tuple{Float64,Array{Float64,2},Array{Float64,2},Array{Float64,2},Int64,Int64,Float64,Int64}}
    │   %6  = Core.apply_type(Main.Val, :ncntry)::Core.Compiler.Const(Val{:ncntry}, false)
    │   %7  = (%6)()::Core.Compiler.Const(Val{:ncntry}(), false)
    │         (ncntry = (%4)(%5, %7))
    │   %9  = Base.getproperty(UnPack, :unpack)::Core.Compiler.Const(UnPack.unpack, false)
    │   %10 = 255::NamedTuple{(:θ, :T, :τ_matrix, :L, :ngoods, :ncntry, :σ, :seed),Tuple{Float64,Array{Float64,2},Array{Float64,2},Array{Float64,2},Int64,Int64,Float64,Int64}}
    │   %11 = Core.apply_type(Main.Val, :ngoods)::Core.Compiler.Const(Val{:ngoods}, false)
    │   %12 = (%11)()::Core.Compiler.Const(Val{:ngoods}(), false)
    │         (ngoods = (%9)(%10, %12))
    │   %14 = Base.getproperty(UnPack, :unpack)::Core.Compiler.Const(UnPack.unpack, false)
    │   %15 = 255::NamedTuple{(:θ, :T, :τ_matrix, :L, :ngoods, :ncntry, :σ, :seed),Tuple{Float64,Array{Float64,2},Array{Float64,2},Array{Float64,2},Int64,Int64,Float64,Int64}}
    │   %16 = Core.apply_type(Main.Val, :τ_matrix)::Core.Compiler.Const(Val{:τ_matrix}, false)
    │   %17 = (%16)()::Core.Compiler.Const(Val{:τ_matrix}(), false)
    │         (τ_matrix = (%14)(%15, %17))
    │   %19 = Base.getproperty(UnPack, :unpack)::Core.Compiler.Const(UnPack.unpack, false)
    │   %20 = 255::NamedTuple{(:θ, :T, :τ_matrix, :L, :ngoods, :ncntry, :σ, :seed),Tuple{Float64,Array{Float64,2},Array{Float64,2},Array{Float64,2},Int64,Int64,Float64,Int64}}
    │   %21 = Core.apply_type(Main.Val, :L)::Core.Compiler.Const(Val{:L}, false)
    │   %22 = (%21)()::Core.Compiler.Const(Val{:L}(), false)
    │         (L = (%19)(%20, %22))
    │         255
    │         (price_index = Main.similar(w))
    │         (trade_shares = Main.zeros(ncntry, ncntry))
    │         (source_prob = Main.similar(trade_shares))
    │   %28 = Core.apply_type(Main.CartesianIndex, 2)::Core.Compiler.Const(CartesianIndex{2}, false)
    │   %29 = Core.apply_type(Main.Array, %28)::Core.Compiler.Const(Array{CartesianIndex{2},N} where N, false)
    │   %30 = Main.size(z)::Tuple{Int64,Int64}
    │         (source_country = (%29)(Main.undef, %30))
    │   %32 = (1:ncntry)::Core.Compiler.PartialStruct(UnitRange{Int64}, Any[Core.Compiler.Const(1, false), Int64])
    │         (@_14 = Base.iterate(%32))
    │   %34 = (@_14 === nothing)::Bool
    │   %35 = Base.not_int(%34)::Bool
    └──       goto #4 if not %35
    2 ┄ %37 = @_14::Tuple{Int64,Int64}::Tuple{Int64,Int64}
    │         (cntry = Core.getfield(%37, 1))
    │   %39 = Core.getfield(%37, 2)::Int64
    │   %40 = Base.getindex(τ_matrix, cntry, Main.:(:))::Array{Float64,1}
    │   %41 = Main.:var"'"(%40)::Adjoint{Float64,Array{Float64,1}}
    │   %42 = Main.low_cost_supplier(z, w, %41)::Tuple{Array{Float64,2},Array{CartesianIndex{2},2}}
    │   %43 = Base.indexed_iterate(%42, 1)::Core.Compiler.PartialStruct(Tuple{Array{Float64,2},Int64}, Any[Array{Float64,2}, Core.Compiler.Const(2, false)])
    │         (p = Core.getfield(%43, 1))
    │         (@_18 = Core.getfield(%43, 2))
    │   %46 = Base.indexed_iterate(%42, 2, @_18::Core.Compiler.Const(2, false))::Core.Compiler.PartialStruct(Tuple{Array{CartesianIndex{2},2},Int64}, Any[Array{CartesianIndex{2},2}, Core.Compiler.Const(3, false)])
    │         (sc = Core.getfield(%46, 1))
    │   %48 = Main.ces_shares(p, sc)::Tuple{Float64,Array{Float64,1},Array{Float64,1}}
    │   %49 = Base.indexed_iterate(%48, 1)::Core.Compiler.PartialStruct(Tuple{Float64,Int64}, Any[Float64, Core.Compiler.Const(2, false)])
    │   %50 = Core.getfield(%49, 1)::Float64
    │         Base.setindex!(price_index, %50, cntry)
    │         (@_19 = Core.getfield(%49, 2))
    │   %53 = Base.indexed_iterate(%48, 2, @_19::Core.Compiler.Const(2, false))::Core.Compiler.PartialStruct(Tuple{Array{Float64,1},Int64}, Any[Array{Float64,1}, Core.Compiler.Const(3, false)])
    │   %54 = Core.getfield(%53, 1)::Array{Float64,1}
    │         Base.setindex!(trade_shares, %54, cntry, Main.:(:))
    │         (@_19 = Core.getfield(%53, 2))
    │   %57 = Base.indexed_iterate(%48, 3, @_19::Core.Compiler.Const(3, false))::Core.Compiler.PartialStruct(Tuple{Array{Float64,1},Int64}, Any[Array{Float64,1}, Core.Compiler.Const(4, false)])
    │   %58 = Core.getfield(%57, 1)::Array{Float64,1}
    │         Base.setindex!(source_prob, %58, cntry, Main.:(:))
    │         Base.setindex!(source_country, sc, Main.:(:), cntry)
    │         (@_14 = Base.iterate(%32, %39))
    │   %62 = (@_14 === nothing)::Bool
    │   %63 = Base.not_int(%62)::Bool
    └──       goto #4 if not %63
    3 ─       goto #2
    4 ┄ %66 = Base.broadcasted(Main.:*, L, w)::Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{2},Nothing,typeof(*),Tuple{Array{Float64,2},Array{Float64,2}}}
    │   %67 = Base.materialize(%66)::Array{Float64,2}
    │         (value_expenditure = %67 * trade_shares)
    │   %69 = Base.broadcasted(Main.:*, L, w)::Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{2},Nothing,typeof(*),Tuple{Array{Float64,2},Array{Float64,2}}}
    │         (value_production = Base.materialize(%69))
    │   %71 = (value_production - value_expenditure)::Array{Float64,2}
    │   %72 = Main.norm(%71)::Float64
    │   %73 = trade_shares::Array{Float64,2}
    │   %74 = price_index::Array{Float64,2}
    │   %75 = source_country::Array{CartesianIndex{2},2}
    │   %76 = Core.tuple(%72, %73, %74, %75, source_prob)::Tuple{Float64,Array{Float64,2},Array{Float64,2},Array{CartesianIndex{2},2},Array{Float64,2}}
    └──       return %76

```julia
@btime equillibrium_evaluate(([1.0 result.minimizer]), z, L, tau_matrix)
```

      6.709 ms (82 allocations: 15.64 MiB)

    (2.1416585547327893e-9, [0.8192570833933529 0.1807429166066469; 0.1482656420018533 0.8517343579981468], [0.7762230565878647 0.7603160725099696], CartesianIndex{2}[CartesianIndex(1, 1) CartesianIndex(1, 2); CartesianIndex(2, 1) CartesianIndex(2, 2); … ; CartesianIndex(99999, 2) CartesianIndex(99999, 2); CartesianIndex(100000, 1) CartesianIndex(100000, 2)])

```julia

```


This is a demo of sampling-based minimum Bayes risk decoding for NMT [(Eikema and Aziz, 2020)](https://www.aclweb.org/anthology/2020.coling-main.398/) for educational purposes. For a scalable implementation check [mbr_nmt](https://github.com/Roxot/mbr-nmt) and [(Eikema and Aziz, 2021)](https://arxiv.org/abs/2108.04718).

```python
from demo import SampleFromNMT, SampleFromBuffer, Utility, MBR, load_de_en
```

Let's start with a quick recap of concepts.

# Neural machine translation

Given an input $x$, a trained NMT model predicts a conditional distribution $Y|X=x, \theta$ over all possible translations of $x$.

The sample space $\mathcal Y$ of the model is the set of all sequences of the form $(y_1, \ldots, y_N)$ where $y_n$ belongs to a vocabulary of known target-language symbols, $N \ge 1$ is the sequence length, and $y_N$ is a special end-of-sequence (EOS) symbol.

An outcome $y \in \mathcal Y$ is assigned probability mass:

\begin{align}
P_{Y|X}(y|x,\theta) &= \prod_{n=1}^{N} \mathrm{Cat}(y_n|f(x, y_{<n}; \theta))
\end{align}

where $y_{<n}$ is the sequence of tokens before the $n$th token, and $f(\cdot; \theta)$ is a neural network architecture with parameters $\theta$.

# Maximum-a-posterior decoding

MAP decoding picks the most probable translation.
\begin{align}
y^{\mathrm{mode}} &= \arg\max_{h \in \mathcal Y} ~ P_{Y|X}(h|x, \theta)
\end{align}

As the space $\mathcal Y$ is unbounded and the NMT model makes no Markov assumptions, this search is intractable. In practice, we approximate the decision rule searching through a beam of probable translations.

\begin{align}
y^{\mathrm{beam}} &= \arg\max_{h \in \mathrm{beam}(x)} ~  P_{Y|X}(h|x, \theta)
\end{align}

# Minimum Bayes risk decoding

MBR decoding picks the translation that has highest expected utility:

\begin{align}
y^{\text{mbr}} &= \arg\max_{h \in \mathcal Y} ~ \mathbb E[u(Y, h; x)|x, \theta] \\
&= \arg\max_{h \in \mathcal Y} ~ \sum_{y \in \mathcal Y} u(y, h; x) P_{Y|X}(y|x, \theta)
\end{align}

where a utility function $u(y, h; x)$ quantifies the benefit of choosing $h$ as the translation of $x$, when $y$ is the correct (or preferred) translation.

There are two sources of intractability in MBR decoding. First,  just like in MAP decoding, the search space (i.e., the sample space $\mathcal Y$) is unbounded. Second, for any given candidate translation $h$, the expected utility $\mathbb E[u(Y, h; x)|x, \theta]$ is intractable to compute.

[Eikema and Aziz (2020)](https://www.aclweb.org/anthology/2020.coling-main.398/) propose to

1. approximate the hypothesis space by a tractable subset of hypotheses $\mathcal H(x)$ obtained by sampling from the model;
2. approximate the expected value using Monte Carlo (MC).

The decision rule becomes

\begin{align}
y^{\text{smbr}} &= \arg\max_{h \in \mathcal H(x)} ~ \frac{1}{S} \sum_{s=1}^S u(y^{(s)}, h; x)
\end{align}

where $y^{(s)} \sim Y|X=x, \theta$ is a sample from the NMT model (samples can be drawn efficiently via ancestral sampling).

For much more on sampling-based MBR see also [Eikema and Aziz (2021)](https://arxiv.org/abs/2108.04718).

# Load pre-trained models

Load pre-processing pipelines and fairseq models. This will take a moment.

Make sure you've downloaded the models (run `bash download-data.sh`).

```python
models = load_de_en(np_seed=10, torch_seed=10)
```

Here, `x` is German and `y` is English.

```python
example_x = 'Es war noch nie leicht, ein rationales Gespräch über den Wert von Gold zu führen.'
example_y = 'It has never been easy to have a rational conversation about the value of gold.'
```

```python
models.pipeline_x.decode(models.pipeline_x.encode(example_x))
```

    'Es war noch nie leicht, ein rationales Gespräch über den Wert von Gold zu führen.'

```python
models.pipeline_y.decode(models.pipeline_y.encode(example_y))
```

    'It has never been easy to have a rational conversation about the value of gold.'

# Demo

## Approximate MAP decoding

```python
models.x2y.beam_search(example_x)['output'][0]
```

    'It has never been easy to talk rationally about the value of gold.'

## Approximate MBR decoding

Let's start by creating a hypothesis space. For example, we can use the unique translations found in a large sample.

```python
hyp_space = list(set(models.x2y.ancestral_sampling(example_x, num_samples=100)['output']))
len(hyp_space)
```

    84

As source sentences get longer or away from the training domain, it's common to find very few duplicates in samples from NMT. Note that duplicates in the hypothesis space do not affect the results, they only waste some computation, that's why we keep only the unique translations.

Now we need to choose a utility function. For most of this demo we will assign utility using ChrF (but do check the last section for an example using COMET, an NN-based metric).

```python
from sacrebleu import sentence_chrf
```

```python
class ChrF(Utility):

    def __call__(self, src: str, hyp: str, ref: str) -> float:
        return sentence_chrf(hyp, ref).score  # note that chrf does not make use of the source sentence

chrf = ChrF()
```

```python
chrf('This is cool!', 'Das ist cool!', 'Cool!')
```

    0.40623147802266224

```python
chrf.batch('This is cool!', 'Das ist cool!', ['Cool!', 'Das ist cool!'])
```

    array([0.40623148, 1.        ])

MBR decoding requires approximating the expected utility of each candidate in the hypothesis space, the mechanism of choice is MC estimation. Thus, MBR decoding requires access to samples from the model.

Ideally, for each hypothesis, we would draw samples completely independently:

```python
nmt_sampler = SampleFromNMT(models.x2y, sample_size=30)
```

```python
nmt_sampler(example_x)
```

    ['It has never been easy to talk rationally about the value of gold.',
     'It has never been easy to talk rationally about the value of gold.',
     'It has never been easy to discuss the value of gold rationally.',
     'It has never been easy to hold a rational conversation about the value of gold.',
     "It has never been easy to engage in a rational conversation about gold's value.",
     'It was never easy to have a rational conversation about the value of gold.',
     'It was never easy to talk rationally about the value of gold.',
     'It has never been easy to discuss the value of gold in a rational manner.',
     'It has never been easy to engage rationally in a conversation about the value of gold.',
     "It has never been easy to have a rational conversation on gold's value.",
     'It has never been easy to enter a rational conversation about the value of gold.',
     'It has never been easy to enter into a rational conversation on the value of gold.',
     'It has never been simple to talk rationally about the value of gold.',
     'It has never been easy to engage in rational talking about the value of gold.',
     "It has never been easy to have a rational conversation over gold's value.",
     'There has never been an easy time having a rational conversation about the value of gold.',
     'It has never been easy to have any rational discussion on the value of gold.',
     'It has never been easy discussing the value of gold rationally.',
     "It has never been easy to generate a rational discussion of gold's value.",
     "It has never been easy to produce a rational argument about gold's value.",
     'It was never easy to be rationalistic about the value of gold.',
     "Rational consideration of gold's value has never been easy.",
     'It was never easy to negotiate rationally on gold values.',
     "They have never been easy to talk rationally about the company's value.",
     'Trading with gold has never been easy.',
     'Discussion of the sum of gold on budget has never been easy.',
     'Choosing gold like its value has never been easy.',
     'It has never been worth remembering rationally about what gold represents.',
     'Granting the gold value a Rational Talkship Zylinder is never as easy as avoiding an Inaccurate Job.',
     'Knowing how you grasp the prospect of a conclusion - mind to be quiet - is crucial for us today.']

Depending on the size of your sample space, this will quickly become too costly. A good alternative is to obtain a large collection of samples (e.g., 100 to 1000) and draw samples with replacement from this collection:

```python
buffered_sampler = SampleFromBuffer(
    {example_x: models.x2y.ancestral_sampling(example_x, num_samples=100)['output']},
    sample_size=30
)
```

```python
buffered_sampler(example_x)
```

    ['It has never been easy to have a rational discussion about the value of gold.',
     'It has never been easy to enter into a rational discussion about the value of gold.',
     "It had never been easy to conduct a rational conversation about gold's value.",
     'It was never easy to negotiate rationally about the value of gold.',
     "It's never been easy to make rational talks about the value of gold.",
     'It has never been easy to hold a rational conversation about the value of gold.',
     'It has never before been easy to discuss the value of gold rationally.',
     'It has never been easy to ask about the value of gold, as a rational conversation is simply not in itself acknowledging it.',
     'It has never been easy to command a rational conversation about the value of gold.',
     'It has never been easy to negotiate on the value of gold rationally.',
     "It has never been easy to listen rationally to gold's value.",
     'It has never been easy to discuss the value of gold in a rational way.',
     "However, it was never easy to talk rationally about gold's value.",
     'Seeking a reasonable exchange on the value of money has never been easy.',
     'It has never been easy to talk rationally about the gold value.',
     'It has never been easy to engage in rational dialog communication about the value of gold.',
     'It is never easy to make rational conversations about the value of gold.',
     'It has never been easy to negotiate on the value of gold rationally.',
     'It has never been easy to have a rational discussion on the value of gold.',
     'Establishing a rational dialogue on the value of gold has never been easy;',
     "It has never been easy to have a rational conversation on gold's value.",
     'It has never been easy to engage in a rational conversation on the value of gold.',
     'It has never been easy to talk rationally about the value of gold.',
     'It has never been easy to find a rational conversation about the value of gold.',
     'It has never been easy to talk rationally about the gold value.',
     'It has never been easy to talk rationally about the gold value.',
     'It has never been easy to discuss the value of gold rationally.',
     'It has never been easy to discuss the value of goldspaths rationally.',
     "It has never been easy to talk rationally about gold's value.",
     'It has never been easy to have a rational conversation about what gold should do as its value does.']

Next, we estimate expected utilities and rank our candidates.

```python
mbr = MBR(chrf, buffered_sampler)  # This version of MBR is what Eikema and Aziz (2021) call MBR N-by-S
```

```python
y_mbr = mbr.decode(example_x, hyp_space)
y_mbr
```

    'It has never been easy to make a rational conversation about the value of gold.'

```python
mbr.mu(example_x, y_mbr)
```

    0.677204700537251

You can also use inspect the expected utility of all candidates (note that estimates of expected utility are random variables, thus some variability across runs is expected, esp for small sample sizes):

```python
for h, mu in sorted(zip(hyp_space, mbr.mus(example_x, hyp_space)), key=lambda pair: pair[1], reverse=True):
    print(f"{mu:.4f}\t{h}")
```

    0.7119	It has never been easy to carry out rational conversation about the value of gold.
    0.7051	It has never been easy to hold a rational conversation about the value of gold.
    0.6855	It has never been easy to enter a rational conversation about the value of gold.
    0.6703	It has never been easy to make a rational conversation about the value of gold.
    0.6673	It has never been easy to engage in rational conversations about the value of gold.
    0.6662	Getting a rational conversation about the value of gold has never been easy.
    0.6653	It has never been easy to have a rational conversation about the value of gold.
    0.6558	It has never been easy to engage in a rational conversation about the value of gold.
    0.6506	Raising a rational conversation about the value of gold has never been easy.
    0.6490	It has never been easy to proceed with a rational conversation about the value of gold.
    0.6456	It has never been easy to make a rational discussion about the value of gold.
    0.6456	It has never been easy to talk rationally about the value of gold.
    0.6365	It has never been easy to negotiate a rational conversation about the value of gold.
    0.6310	It has never been easy to conduct a rational discussion about the value of gold.
    0.6280	It has never been easy to have a rational conversation on the value of gold previously.
    0.6212	Having a rational conversation about the value of gold has never been easy.
    0.6199	It has never been easy to talk proportionally about the value of gold.
    0.6134	It has never been easy to involve a rational conversation on the value of gold.
    0.6108	It has never been easy to engage in a rational exchange about the value of gold.
    0.6089	It has never been easy to do rational talk about the value of gold.
    0.5950	It has never been easy to entertain rational talk about the value of gold ever.
    0.5936	It has never been easy to engage in rational conversation about gold's value.
    0.5925	It has never been easy to hold rational conversations about gold's value.
    0.5924	It has never been easy to initiate a rational dialogue of the value of gold.
    0.5914	You have never been easy to start a rational discussion about the value of gold.
    0.5855	It has never been easy to have a rational conversation about gold's value.
    0.5842	It has never been easy to discuss the value of gold rationally.
    0.5778	Having a rational exchange about the value of gold has never been easy.
    0.5735	It has never been easy to engage in rational talk about the value of gold.
    0.5686	It has never been easy to talk about the value of gold rationally.
    0.5669	Raising a rational conversation about gold's value has never been easy.
    0.5646	It has never been easier to discuss the value of gold in rational terms.
    0.5607	It was never easy to have a rational discussion about the value of gold.
    0.5596	It has never been easy to hold a rational conversation on gold's value.
    0.5573	It has never been easy to discuss the value of gold in a rational way.
    0.5525	It has never been easy to discuss the value of gold head-on with rationality.
    0.5504	It's never been easy to establish a rational exchange of views about the value of gold.
    0.5494	It has never been easy to make a coherent conversation on the value of gold.
    0.5490	Any rational conversation about gold value has never been easy.
    0.5453	It has never been easy to make a rational conversation about gold.
    0.5424	It has never been easy to engage in rational talk about gold's value.
    0.5412	It has not been easy to hold a rational discussion about the value of gold.
    0.5382	It has never been easy to get a rational conversation about Gold's worth.
    0.5346	It has never been easy to actually talk about the value of gold.
    0.5344	It has never been easy to discuss and negotiate properly the value of gold.
    0.5173	It was never easy to conduct a rational conversation about the value of gold.
    0.5156	It has never been easy to engage in rational discussion about gold's worth.
    0.5142	The solution has never been easy to achieve rational discussion of gold's value.
    0.5115	It has never been simple to talk rationally about the value of gold.
    0.5106	Discussion about the value of gold has never been easy.
    0.5101	It has never been easy to discuss the value of gold interactively.
    0.5008	It has never been easy to discuss the value of gold efficiently.
    0.4914	It has never been easy to calculate the value of gold.
    0.4908	Economically speaking, it has never always been easy to talk about the value of gold.
    0.4897	It has never been easy to know about gold as a country to talk about rational wise terms.
    0.4875	Raising a rational conversation on the value of gold was never easy.
    0.4835	A rational price conversation about gold has never been easy.
    0.4827	"It has never been easy for a rational discussion on gold's value to be undertaken.
    0.4815	It was never easy to talk rationally about the value of gold.
    0.4806	It has never been easy to talk rationally about Gold's value.
    0.4758	The rational conversation of gold value has never been easy.
    0.4690	It has never been easy to return to the subject of gold as one rational matter.
    0.4665	It has never been readily possible to anchor a rational discussion about gold's value.
    0.4647	It have never been easy to talk on a rational basis about how value should be abundantly gold.
    0.4611	It has never been easier to trump the value of gold.
    0.4553	It has never been much easier to talk rationally about gold in value.
    0.4543	It has never been easy to engage in rational exchange on gold.
    0.4453	The price of gold has never been easy to ask amidst rational talk.
    0.4428	It has not been easy to discuss the value of gold with one voice.
    0.4245	To talk rationally about gold has never been easy.
    0.4244	There has never been an easy time discussing the value of gold.
    0.4074	It was never easy to talk rationally about gold's value.
    0.3766	No rational talk about the value of gold ever seemed easy.
    0.3572	A rational discussion of gold's value was never easy.
    0.3568	It never was easy to pursue a rational debate about gold values.
    0.3446	It was never a straightforward discussion of the value of gold.
    0.3407	Talk about gold value has never been easy.
    0.3394	More than ever, the rational example of the value of gold was uncomplicated.
    0.3384	It was never easy to consider Nielsen's values rationally.
    0.3173	You never have a way to talk rationally about gold's value.
    0.1931	To achieve what, far from being warranted, we must fail to sort out what the taxpayers think with capital penalties.
    0.1906	True enough nobody has ever known about gold.
    0.1541	Encouraging their choices was the result of an interrogation of money before, confusing the mathematical-plowing combines of the Ayurvedic work with the financial somethinglectionism of ancient estates and natural intellectual and intellectual property. O'Tébailes was appointed the first teacher, Professor, and President in Architecture, to approach the financial crisis with an argument about economics, economics, and economy.
    0.1430	It took no time to deal sensibly with gold well.

## COMET as Utility

The utility function is anything we trust for assessing the merits of a translation candidate. In this part of the demo, we will employ a modern NN-based metric: [COMET](https://github.com/Unbabel/COMET). Some of COMET's dependencies are in conflict with our trained NMT models (e.g., fairseq version), thus we have prepared a flask that abstracts away from all the mess behind maintaining different versions of python packages: git clone Probabll's [mteval-flask](https://github.com/probabll/mteval-flask) and follow the instructions there to start an automatic evaluation server.

The reason we pick COMET is not only that it's modern, unlike most MT evaluation metrics, COMET makes use of the source sentence, which we think is the right way towards evaluating adequacy.

```python
import requests
```

```python
class COMET(Utility):

    def __call__(self, src: str, hyp: str, ref: str) -> float:
        jobs = {'hyps': [hyp], 'refs': [ref], 'srcs': [src]}
        # Unlike ChrF, COMET actually makes use of the source sentence! :D
        results = requests.post("http://localhost:4000/score", json=jobs, headers={'Content-Type': 'application/json'}).json()
        return results['comet'][0]

    def batch(self, src: str, hyp: str, refs: list):
        jobs = {'hyps': [hyp] * len(refs), 'refs': refs, 'srcs': [src] * len(refs)}
        results = requests.post("http://localhost:4000/score", json=jobs, headers={'Content-Type': 'application/json'}).json()
        return results['comet']
```

```python
comet = COMET()
```

```python
mbr_comet = MBR(comet, buffered_sampler)  # This version of MBR is what Eikema and Aziz (2021) call MBR N-by-S
```

```python
for h, mu in sorted(zip(hyp_space, mbr_comet.mus(example_x, hyp_space)), key=lambda pair: pair[1], reverse=True):
    print(f"{mu:.4f}\t{h}")
```

    0.8370	It was never easy to conduct a rational conversation about the value of gold.
    0.8291	It was never easy to have a rational discussion about the value of gold.
    0.8069	You have never been easy to start a rational discussion about the value of gold.
    0.7906	It has never been easy to carry out rational conversation about the value of gold.
    0.7861	It has not been easy to hold a rational discussion about the value of gold.
    0.7856	It has never been easy to have a rational conversation about the value of gold.
    0.7838	It has never been easy to engage in rational conversations about the value of gold.
    0.7836	It has never been easy to make a rational conversation about the value of gold.
    0.7776	It has never been easy to enter a rational conversation about the value of gold.
    0.7589	It has never been easy to make a rational discussion about the value of gold.
    0.7583	It has never been easy to conduct a rational discussion about the value of gold.
    0.7473	It has never been easy to hold a rational conversation about the value of gold.
    0.7433	It has never been easy to engage in a rational conversation about the value of gold.
    0.7422	It has never been easy to talk rationally about the value of gold.
    0.7387	It has never been easy to have a rational conversation about gold's value.
    0.7218	It has never been easy to involve a rational conversation on the value of gold.
    0.7158	It has never been easy to engage in rational conversation about gold's value.
    0.7142	It has never been easy to discuss the value of gold in a rational way.
    0.7141	It has never been easy to engage in rational talk about gold's value.
    0.7075	It has never been easy to proceed with a rational conversation about the value of gold.
    0.7070	It has never been easy to negotiate a rational conversation about the value of gold.
    0.6990	Getting a rational conversation about the value of gold has never been easy.
    0.6962	It has never been easy to hold rational conversations about gold's value.
    0.6765	It has never been easy to engage in a rational exchange about the value of gold.
    0.6750	It has never been easy to engage in rational talk about the value of gold.
    0.6653	It was never easy to talk rationally about the value of gold.
    0.6538	It has never been easy to talk about the value of gold rationally.
    0.6403	It has never been easy to make a coherent conversation on the value of gold.
    0.6330	Having a rational conversation about the value of gold has never been easy.
    0.6291	It has never been simple to talk rationally about the value of gold.
    0.6242	It has never been easy to do rational talk about the value of gold.
    0.6121	It has never been easy to discuss the value of gold rationally.
    0.6081	Raising a rational conversation about the value of gold has never been easy.
    0.6066	It has never been easy to hold a rational conversation on gold's value.
    0.6047	It's never been easy to establish a rational exchange of views about the value of gold.
    0.5969	It has never been easy to make a rational conversation about gold.
    0.5942	It has never been easy to talk proportionally about the value of gold.
    0.5905	It has never been easy to initiate a rational dialogue of the value of gold.
    0.5706	It has never been easy to talk rationally about Gold's value.
    0.5703	It has never been easier to discuss the value of gold in rational terms.
    0.5679	It has never been easy to have a rational conversation on the value of gold previously.
    0.5590	It never was easy to pursue a rational debate about gold values.
    0.5587	It was never easy to talk rationally about gold's value.
    0.5107	Raising a rational conversation about gold's value has never been easy.
    0.5014	It has never been easy to engage in rational discussion about gold's worth.
    0.4975	Having a rational exchange about the value of gold has never been easy.
    0.4760	Economically speaking, it has never always been easy to talk about the value of gold.
    0.4721	The solution has never been easy to achieve rational discussion of gold's value.
    0.4686	It has never been easy to discuss the value of gold interactively.
    0.4250	You never have a way to talk rationally about gold's value.
    0.4187	Any rational conversation about gold value has never been easy.
    0.4113	Discussion about the value of gold has never been easy.
    0.4085	To talk rationally about gold has never been easy.
    0.4024	It has never been readily possible to anchor a rational discussion about gold's value.
    0.3997	It has never been easy to discuss the value of gold efficiently.
    0.3977	It has never been easy to entertain rational talk about the value of gold ever.
    0.3955	It has never been easy to actually talk about the value of gold.
    0.3876	There has never been an easy time discussing the value of gold.
    0.3722	It has never been much easier to talk rationally about gold in value.
    0.3552	The rational conversation of gold value has never been easy.
    0.3464	It has never been easy to engage in rational exchange on gold.
    0.3445	It has never been easy to discuss and negotiate properly the value of gold.
    0.3408	It has never been easy to calculate the value of gold.
    0.3366	Raising a rational conversation on the value of gold was never easy.
    0.3213	No rational talk about the value of gold ever seemed easy.
    0.2928	It has never been easy to discuss the value of gold head-on with rationality.
    0.2808	It have never been easy to talk on a rational basis about how value should be abundantly gold.
    0.2603	A rational price conversation about gold has never been easy.
    0.2501	It was never a straightforward discussion of the value of gold.
    0.2406	It has never been easy to get a rational conversation about Gold's worth.
    0.2356	The price of gold has never been easy to ask amidst rational talk.
    0.2079	"It has never been easy for a rational discussion on gold's value to be undertaken.
    0.2048	A rational discussion of gold's value was never easy.
    0.2028	It has not been easy to discuss the value of gold with one voice.
    0.1554	Talk about gold value has never been easy.
    0.0992	It has never been easy to know about gold as a country to talk about rational wise terms.
    0.0609	It was never easy to consider Nielsen's values rationally.
    0.0232	It has never been easy to return to the subject of gold as one rational matter.
    -0.1140	It has never been easier to trump the value of gold.
    -0.1696	More than ever, the rational example of the value of gold was uncomplicated.
    -0.5259	To achieve what, far from being warranted, we must fail to sort out what the taxpayers think with capital penalties.
    -0.5481	True enough nobody has ever known about gold.
    -0.5577	It took no time to deal sensibly with gold well.
    -0.7896	Encouraging their choices was the result of an interrogation of money before, confusing the mathematical-plowing combines of the Ayurvedic work with the financial somethinglectionism of ancient estates and natural intellectual and intellectual property. O'Tébailes was appointed the first teacher, Professor, and President in Architecture, to approach the financial crisis with an argument about economics, economics, and economy.

```python

```


# Allen Cahn equation

* Physical space
\begin{align}
u_{t} = \epsilon u_{xx} + u - u^{3}
\end{align}
* Discretized with Chebyshev differentiation matrix (D)
\begin{align}
u_t = (\epsilon D^2 + I)u - u^{3}
\end{align}

# Imports

```python
import numpy as np
import matplotlib.pyplot as plt
from rkstiff.grids import construct_x_Dx_cheb
from rkstiff.etd35 import ETD35
from rkstiff.etd34 import ETD34
from rkstiff.if34 import IF34
import time
```

# Linear operator, nonlinear function

```python
N = 20
epsilon = 0.01
x,D = construct_x_Dx_cheb(N,-1,1)
D2 = D.dot(D)
L = epsilon*D2 + np.eye(*D2.shape)
L = L[1:-1,1:-1] # Interior points

def NL(u):
    return x[1:-1] - np.power(u+x[1:-1],3)
```

# Set initial field

```python
u0 = 0.53*x + 0.47*np.sin(-1.5*np.pi*x)
w0 = u0 - x

plt.plot(x,u0)
plt.xlabel('x')
plt.ylabel('$u_0$')
plt.title('Initial field')
```

# Apply nondiagonal and diagonalized solvers for comparison

```python
nondiag_params = {'epsilon' : 1e-4, 'contour_points' : 64, 'contour_radius' : 20}
solver34 = ETD34(linop=L,NLfunc=NL,**nondiag_params)
solver35 = ETD35(linop=L,NLfunc=NL,**nondiag_params)
solverIF = IF34(linop=L,NLfunc=NL,**nondiag_params)
diag_params = {'epsilon' : 1e-4, 'contour_points' : 32, 'contour_radius' : 1,'diagonalize' : True}
solverDiag34 = ETD34(linop=L,NLfunc=NL,**diag_params)
solverDiag35 = ETD35(linop=L,NLfunc=NL,**diag_params)
solverDiagIF34 = IF34(linop=L,NLfunc=NL,**diag_params)
solvers = [solver34,solver35,solverIF,solverDiag34,solverDiag35,solverDiagIF34]
titles = ['ETD34','ETD35','IF34','ETD34 Diagonalized','ETD35 Diagonalized','IF34 Diagonalized']
```

# Run simulations

```python
Xvec,Tvec,Uvec = [],[],[]
for solver in solvers:
    _ = solver.evolve(w0[1:-1],t0=0,tf=100)
    U = []
    for wint in solver.u:
        w = np.r_[0,wint.real,0]
        u = w + x
        U.append(u)
    U = np.array(U)
    t = np.array(solver.t)
    T,X = np.meshgrid(t,x,indexing='ij')
    Xvec.append(X); Tvec.append(T); Uvec.append(U)
```

# Plot results

```python
fig = plt.figure(figsize=(16,12))
for i in range(6):
    ax = fig.add_subplot(2,3,i+1,projection='3d')
    ax.plot_wireframe(Xvec[i],Tvec[i],Uvec[i],color='black')
    ax.set_xlabel('x')
    ax.set_ylabel('t')
    ax.set_zlabel('z')
    ax.set_facecolor('white')
    ax.grid(False)
    ax.set_title(titles[i])
    ax.view_init(elev=36,azim=-131)
# fig.tight_layout()
```

# Time simulations

```python
start = time.time()
solver = ETD34(linop=L,NLfunc=NL,**nondiag_params)
_ = solver.evolve(w0[1:-1],t0=0,tf=100,store_data=False)
end = time.time()
print(titles[0],'-> {:.2e}'.format(end-start))

start = time.time()
solver = ETD35(linop=L,NLfunc=NL,**nondiag_params)
_ = solver.evolve(w0[1:-1],t0=0,tf=100,store_data=False)
end = time.time()
print(titles[1],'-> {:.2e}'.format(end-start))

start = time.time()
solver = IF34(linop=L,NLfunc=NL,**nondiag_params)
_ = solver.evolve(w0[1:-1],t0=0,tf=100,store_data=False)
end = time.time()
print(titles[2],'-> {:.2e}'.format(end-start))

start = time.time()
solverDiag = ETD34(linop=L,NLfunc=NL,**diag_params)
_ = solver.evolve(w0[1:-1],t0=0,tf=100,store_data=False)
end = time.time()
print(titles[3],'-> {:.2e}'.format(end-start))

start = time.time()
solver = ETD35(linop=L,NLfunc=NL,**diag_params)
_ = solver.evolve(w0[1:-1],t0=0,tf=100,store_data=False)
end = time.time()
print(titles[4],'-> {:.2e}'.format(end-start))

start = time.time()
solver = IF34(linop=L,NLfunc=NL,**diag_params)
_ = solver.evolve(w0[1:-1],t0=0,tf=100,store_data=False)
end = time.time()
print(titles[5],'-> {:.2e}'.format(end-start))
```

    ETD34 -> 7.02e-01
    ETD35 -> 1.45e+00
    IF34 -> 1.59e-01
    ETD34 Diagonalized -> 1.65e-01
    ETD35 Diagonalized -> 1.59e-01
    IF34 Diagonalized -> 1.75e-01

# Diagonalizing IF method has little impact on performance

```python
%%timeit
solver = IF34(linop=L,NLfunc=NL,epsilon=1e-4,contour_points=32,contour_radius=1,diagonalize=True)
_ = solver.evolve(w0[1:-1],t0=0,tf=100,store_data=False)
```

```python
%%timeit
solver = IF34(linop=L,NLfunc=NL,epsilon=1e-4,contour_points=64,contour_radius=20)
_ = solver.evolve(w0[1:-1],t0=0,tf=100,store_data=False)
```

    155 ms ± 1.77 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)


```python
import sympy as sm
import sympy.vector
import matplotlib.pyplot as plt

A = sm.Matrix([1, 2, 3])
C = sm.vector.CoordSys3D('')
v = sm.vector.matrix_to_vector(A,C)
a = v.to_matrix(C)
```

# [Jacobian](https://suhak.tistory.com/944)
> ## chain rule = pipe line rule
>> $ z = f(x,y),\:where\:
\begin{cases}
x & = g(u,v) & = r\:cos(\theta), \\ y & = h(u,v) & = r\:sin(\theta) \\
\end{cases}\\
z \iff f(x,y) \iff f(g(r,\theta),\:h(r,\theta)) \iff f(\:r\:cos(\theta),\:r\:sin(\theta))
$
>>> $ df(x,y) =
\begin{cases}
dx  & = f(x+dx,\:y) \\
    & = g(r+dr,\:\theta) + g(r,\:\theta + d\theta) \\
    & = \frac{\partial g(r,\theta)}{\partial r}dr +
        \frac{\partial g(r,\theta)}{\partial \theta}d\theta \\
\therefore & \text{in x, scale factor r} =
    \frac{\partial g(r,\theta)}{\partial r} \\
\therefore & \text{in x, scale factor } \theta =
    \frac{\partial g(r,\theta)}{\partial \theta} \\
\\
dy & = f(x,\:y+dy) \\
    & = h(r+dr,\:\theta) + h(r,\:\theta + d\theta) \\
    & = \frac{\partial h(r,\theta)}{\partial r}dr +
        \frac{\partial h(r,\theta)}{\partial \theta}d\theta \\
\therefore & \text{in y, scale factor r} =
    \frac{\partial h(r,\theta)}{\partial r} \\
\therefore & \text{in y, scale factor } \theta =
    \frac{\partial h(r,\theta)}{\partial \theta} \\
\end{cases} \\
\text{Jacobian matrix }\iff
\begin{bmatrix}
dx \\ dy
\end{bmatrix} =
\begin{bmatrix}
\frac{\partial g}{\partial r} & \frac{\partial g}{\partial \theta} \\
\frac{\partial h}{\partial r} & \frac{\partial h}{\partial \theta} \\
\end{bmatrix}\:
\begin{bmatrix}
dr \\ d\theta
\end{bmatrix} \\
\therefore dxdy\text{는 x-y 좌표계에서 $drd\theta$ 는 $r-\theta$ 좌표계에서 넓이의 변화량이므로, 면적의 크기를 곱해준다. 각각의 } \frac{\partial g}{\partial r},
\frac{\partial g}{\partial \theta} \text{ 편미분 크기를 곱하면 면적이 된다.}\\
\text{Jacobian }\iff J(r,\theta) = \frac{\partial(f,g)}{\partial(r,\theta)} =
\begin{vmatrix}
\frac{\partial g}{\partial r} & \frac{\partial g}{\partial \theta} \\
\frac{\partial h}{\partial r} & \frac{\partial h}{\partial \theta} \\
\end{vmatrix} \\
\therefore \text{면적의 크기로 나누어준다. 각각의 } \frac{\partial g}{\partial r},
\frac{\partial g}{\partial \theta} \text{ 행열식(면적) 나누어 주면 된다.}
$

# Jacobian Matrix
> $
\int_{a}^{b}f(g(x))\:g'(x)\:dx = \int_{g(a)}^{g(b)}\:f(u)du \\
(u = g(x) \iff du = g'(x)dx)
$

```python
x, y = sm.symbols('x y')
B = sm.vector.CoordSys3D('B')
S = B.create_new('S',transformation='spherical')

A = sm.Matrix([[S.r*sm.sin(S.theta)*sm.cos(S.phi)],
              [S.r*sm.sin(S.theta)*sm.sin(S.phi)],
              [S.r*sm.cos(S.theta)]])
J = A.diff(S.r).row_join(A.diff(S.theta)).row_join(A.diff(S.phi))

pv = sm.vector.matrix_to_vector(A,S)
jr = pv.diff(S.r)
jr
#A.jacobian(sm.Matrix([x,y]))
```

$\displaystyle (\sin{\left(\mathbf{{theta}_{S}} \right)} \cos{\left(\mathbf{{phi}_{S}} \right)})\mathbf{\hat{i}_{S}} + (\sin{\left(\mathbf{{phi}_{S}} \right)} \sin{\left(\mathbf{{theta}_{S}} \right)})\mathbf{\hat{j}_{S}} + (\cos{\left(\mathbf{{theta}_{S}} \right)})\mathbf{\hat{k}_{S}}$

```python
sm.Matrix(S.transformation_to_parent()).jacobian([S.r,S.theta,S.phi]).det()
```

$\displaystyle \mathbf{{r}_{S}}^{2} \sin^{2}{\left(\mathbf{{phi}_{S}} \right)} \sin^{3}{\left(\mathbf{{theta}_{S}} \right)} + \mathbf{{r}_{S}}^{2} \sin^{2}{\left(\mathbf{{phi}_{S}} \right)} \sin{\left(\mathbf{{theta}_{S}} \right)} \cos^{2}{\left(\mathbf{{theta}_{S}} \right)} + \mathbf{{r}_{S}}^{2} \sin^{3}{\left(\mathbf{{theta}_{S}} \right)} \cos^{2}{\left(\mathbf{{phi}_{S}} \right)} + \mathbf{{r}_{S}}^{2} \sin{\left(\mathbf{{theta}_{S}} \right)} \cos^{2}{\left(\mathbf{{phi}_{S}} \right)} \cos^{2}{\left(\mathbf{{theta}_{S}} \right)}$

# Total Derivative( $df  = \nabla \cdot f$ )
cf. Taylor Series => sympy.sin(x).series()
> ### $$
\begin{align}
f(x,y,z)|_{a,b,c} & =
f(a,b,c) \\ & +
 \frac{\partial f}{\partial x}(x-a) +
 \frac{\partial f}{\partial y}(y-a) +
 \frac{\partial f}{\partial z}(z-a) \\ & +
\frac{(x-a)^2}{2!}\frac{\partial ^2f}{\partial x^2} +
\frac{(y-a)^2}{2!}\frac{\partial ^2f}{\partial y^2} +
\frac{(z-a)^2}{2!}\frac{\partial ^2f}{\partial z^2} + ...
\end{align}
$$

> ## $df = \lim \limits_{x\to a\\ y\to b \\ z \to c}(f(x,y,z) - f(a,b,c))$
>> ### $
df =
\begin{cases}
\frac{\partial f}{\partial x}(x-a) +
\frac{\partial f}{\partial y}(y-a) +
\frac{\partial f}{\partial z}(z-a)
\\
\quad \therefore
\frac{\partial f}{\partial x}(dx) +
\frac{\partial f}{\partial y}(dy) +
\frac{\partial f}{\partial z}(dz)
\\
(\frac{\partial f}{\partial x} ,
\frac{\partial f}{\partial y} ,
\frac{\partial f}{\partial z}) \cdot (dx,dy,dz)
\end{cases}
\\
\quad \therefore \nabla f =
(\frac{\partial f}{\partial x} ,
\frac{\partial f}{\partial y} ,
\frac{\partial f}{\partial z})
\\
\quad \because d\vec{r} = (dx, dy, dz)
= (\frac{\partial f}{\partial x} ,
\frac{\partial f}{\partial y} ,
\frac{\partial f}{\partial z})
\cdot
(dx, dy, dz)
\\
= (\frac{\partial}{\partial x}f ,
\frac{\partial}{\partial y}f ,
\frac{\partial}{\partial z}f)
\cdot
(dx, dy, dz)
\\
= (\frac{\partial}{\partial x} ,
\frac{\partial}{\partial y} ,
\frac{\partial}{\partial z})f
\cdot
(dx, dy, dz)
\\
= \nabla f \cdot (dx, dy, dz)
\\
= \nabla f \cdot d \, \vec{r}
$

 ```
     import sympy.vector
     C = sympy.vector.CoordSys3D('')
     nabla = sympy.vector.Del()
     f v1 v2 v3 = sympy.symbols('f v_1 v_2 v_3',cls=sympy.Funciton)

     sFeild = f(C.x,C.y,C.z)
     vField = v1(C.x,C.y,C.z)*C.i + v2(C.x,C.y,C.z)*C.j + v3(C.x,C.y,C.z)*C.k

     nabla(f)

     nabla.dot(vF)
     nabla.cross(vF)
```
> # $
\because
df = (
\frac{\partial}{\partial x},
\frac{\partial}{\partial y},
\frac{\partial}{\partial z})
\,f \cdot d\vec{r} \\
\therefore
\mbox{ Gradient } \Rightarrow
\nabla f = (
\frac{\partial}{\partial x},
\frac{\partial}{\partial y},
\frac{\partial}{\partial z})
\,f
$
---
# Directional derivative
> ### $
\begin{align}
\because
df & = (
\frac{\partial}{\partial x},
\frac{\partial}{\partial y},
\frac{\partial}{\partial z})
\,f \cdot d\vec{r} \\
& = (
\frac{\partial}{\partial x},
\frac{\partial}{\partial y},
\frac{\partial}{\partial z})
\,f \cdot dl\,\hat r \\
\frac{df}{dl}  & = (
\frac{\partial}{\partial x},
\frac{\partial}{\partial y},
\frac{\partial}{\partial z})
\,f \cdot \hat r \\
& =
\nabla f \cdot \hat r \\
\end{align}
\\
\qquad \mbox{Directional derivative } \overset {def}{\iff}
\begin{cases}
\nabla f \cdot \vec{\hat r} \\
  |\nabla f| |\hat r| \cos(\theta) \\
  |\nabla f| \cos(\theta)
  \end{cases}
$

---
#  Gradient of $f$
> ### apply to scalar function
>> scalar function that returns the value of a real number
> Potential Energy
> ## $
\nabla f =
(\frac{\partial}{\partial x},\frac{\partial}{\partial y},\frac{\partial}{\partial z})f \\
\quad \because \mbox{gradient of } f \overset{def}{\iff}
(\frac{\partial}{\partial x},\frac{\partial}{\partial y},\frac{\partial}{\partial z})f \\
\nabla =
(\frac{\partial}{\partial x},\frac{\partial}{\partial y},\frac{\partial}{\partial z}) \\
\quad \because \mbox{nabla or del } \overset{ def}{\iff}
(\frac{\partial}{\partial x},\frac{\partial}{\partial y},\frac{\partial}{\partial z})
$
---
# Divergence
> ### apply dot product
>> ## $\nabla \cdot $

# Curl
> ### apply cross product
>> ## $\nabla \, \times $

```python
# Nabla (Del)operator(mathmatical notaion)
nabla  = sm.vector.Del()
nabla , nabla(C.x * C.y * C.z)
```

    (delop,
     (Derivative(C.x*C.y*C.z, C.x))*C.i + (Derivative(C.x*C.y*C.z, C.y))*C.j + (Derivative(C.x*C.y*C.z, C.z))*C.k)

```python
# Curl
nabla = sm.vector.Del()
nabla , nabla.cross(C.x * C.y * C.z * C.i)
```

    (delop,
     (Derivative(0, C.y) - Derivative(0, C.z))*C.i + (-Derivative(0, C.x) + Derivative(C.x*C.y*C.z, C.z))*C.j + (Derivative(0, C.x) - Derivative(C.x*C.y*C.z, C.y))*C.k)

```python
sm.poly(sm.sin(theta))
```

$\displaystyle \operatorname{Poly}{\left( \sin{\left(\theta \right)}, \sin{\left(\theta \right)}, domain=\mathbb{Z} \right)}$

```python
f = sm.symbols('f', cls=sm.Function)
```

# Problem 1
## find the unit normal $\hat{n}$ to the surface $x^2 + y^2 -z = 0$ at the point (1,1,2)

```python
import numpy as np
import matplotlib.pyplot as plt
%matplotlib widget
nabla = sm.vector.Del()
sol = nabla(C.x**2+C.y**2-C.z).subs([(C.x,1),(C.y,1),(C.z,2)]).normalize().doit()
sol = sol.to_matrix(C)
print(sol.col(0))

fig = plt.figure()
ax = fig.add_subplot(projection = '3d')

xi = np.linspace(-3,3,1000)
xi, yi  = np.meshgrid(xi,xi)
ax.plot_surface(xi,yi,xi**2 + yi**2,alpha=0.5)
ax.quiver(1,1,2,sol[0],sol[1],sol[2],color='r')
ax.scatter3D(1,1,2,color='red',s=30)
```

    Matrix([[2/3], [2/3], [-1/3]])

    <mpl_toolkits.mplot3d.art3d.Path3DCollection at 0x7fe239840e80>

        Figure

```python

```

$\displaystyle 2 \mathbf{{x}_{C}}^{2} \mathbf{{y}_{C}}$

```python
eV.diff(C.x)+ eV.diff(C.y) + eV.diff(C.z)
```

$\displaystyle 2 \mathbf{{x}_{C}}^{2} + 4 \mathbf{{x}_{C}} \mathbf{{y}_{C}}$

```python
nabla = sm.vector.Del()
nabla(eV)
```

$\displaystyle (\frac{\partial}{\partial \mathbf{{x}_{C}}} 2 \mathbf{{x}_{C}}^{2} \mathbf{{y}_{C}})\mathbf{\hat{i}_{C}} + (\frac{\partial}{\partial \mathbf{{y}_{C}}} 2 \mathbf{{x}_{C}}^{2} \mathbf{{y}_{C}})\mathbf{\hat{j}_{C}} + (\frac{\partial}{\partial \mathbf{{z}_{C}}} 2 \mathbf{{x}_{C}}^{2} \mathbf{{y}_{C}})\mathbf{\hat{k}_{C}}$

```python
nabla.dot(eV*(C.i+C.j+C.k))
```

$\displaystyle \frac{\partial}{\partial \mathbf{{x}_{C}}} 2 \mathbf{{x}_{C}}^{2} \mathbf{{y}_{C}} + \frac{\partial}{\partial \mathbf{{y}_{C}}} 2 \mathbf{{x}_{C}}^{2} \mathbf{{y}_{C}}$

```python
nabla.dot(C.i+C.j+C.k)
```

$\displaystyle 0$

```python
nabla(eV).cross(C.i+C.j+C.k)
nabla.cross(C.i + C.j + C.k)
```

$\displaystyle (\frac{d}{d \mathbf{{y}_{C}}} 1 - \frac{d}{d \mathbf{{z}_{C}}} 1)\mathbf{\hat{i}_{C}} + (- \frac{d}{d \mathbf{{x}_{C}}} 1 + \frac{d}{d \mathbf{{z}_{C}}} 1)\mathbf{\hat{j}_{C}} + (\frac{d}{d \mathbf{{x}_{C}}} 1 - \frac{d}{d \mathbf{{y}_{C}}} 1)\mathbf{\hat{k}_{C}}$

```python
nabla.gradient(eV)
```

$\displaystyle (\frac{\partial}{\partial \mathbf{{x}_{C}}} 2 \mathbf{{x}_{C}}^{2} \mathbf{{y}_{C}})\mathbf{\hat{i}_{C}} + (\frac{\partial}{\partial \mathbf{{y}_{C}}} 2 \mathbf{{x}_{C}}^{2} \mathbf{{y}_{C}})\mathbf{\hat{j}_{C}} + (\frac{\partial}{\partial \mathbf{{z}_{C}}} 2 \mathbf{{x}_{C}}^{2} \mathbf{{y}_{C}})\mathbf{\hat{k}_{C}}$

```python
v = (sm.sin(x)**2 + sm.cos(x)**2)*C.i - (2*sm.cos(y)**2) -1*C.k
sm.Integral(v,x)
```

$\displaystyle (\int \left(\sin^{2}{\left(x \right)} + \cos^{2}{\left(x \right)}\right)\, dx)\mathbf{\hat{i}_{C}} + (\int \left(-1\right)\, dx)\mathbf{\hat{k}_{C}}$

```python
sm.Derivative(v,x)
```

$\displaystyle \frac{\partial}{\partial x} \left((\sin^{2}{\left(x \right)} + \cos^{2}{\left(x \right)})\mathbf{\hat{i}_{C}} - \mathbf{\hat{k}_{C}}\right)$

# Prove the third rule
> ## $ \nabla \cdot (f\,\vec{v}) = f(\nabla \cdot \vec{v}) + \vec{v}\cdot (\nabla f)$

```python
nabla = sm.vector.Del()
v1, v2, v3, f = sm.symbols('v_1 v_2 v_3 f',cls=sm.Function)
vF = v1(C.x,C.y,C.z)*C.i + v2(C.x,C.y,C.z)*C.j + v3(C.x,C.y,C.z)*C.k
sF = f(C.x,C.y,C.z)
```

```python
lhs = nabla.dot(sF*vF)
lhs
```

$\displaystyle \frac{\partial}{\partial \mathbf{{x}_{C}}} f{\left(\mathbf{{x}_{C}},\mathbf{{y}_{C}},\mathbf{{z}_{C}} \right)} \operatorname{v_{1}}{\left(\mathbf{{x}_{C}},\mathbf{{y}_{C}},\mathbf{{z}_{C}} \right)} + \frac{\partial}{\partial \mathbf{{y}_{C}}} f{\left(\mathbf{{x}_{C}},\mathbf{{y}_{C}},\mathbf{{z}_{C}} \right)} \operatorname{v_{2}}{\left(\mathbf{{x}_{C}},\mathbf{{y}_{C}},\mathbf{{z}_{C}} \right)} + \frac{\partial}{\partial \mathbf{{z}_{C}}} f{\left(\mathbf{{x}_{C}},\mathbf{{y}_{C}},\mathbf{{z}_{C}} \right)} \operatorname{v_{3}}{\left(\mathbf{{x}_{C}},\mathbf{{y}_{C}},\mathbf{{z}_{C}} \right)}$

```python
rhs = sF*(nabla.dot(vF)) + vF.dot(nabla(sF))
rhs
```

$\displaystyle \left(\frac{\partial}{\partial \mathbf{{x}_{C}}} \operatorname{v_{1}}{\left(\mathbf{{x}_{C}},\mathbf{{y}_{C}},\mathbf{{z}_{C}} \right)} + \frac{\partial}{\partial \mathbf{{y}_{C}}} \operatorname{v_{2}}{\left(\mathbf{{x}_{C}},\mathbf{{y}_{C}},\mathbf{{z}_{C}} \right)} + \frac{\partial}{\partial \mathbf{{z}_{C}}} \operatorname{v_{3}}{\left(\mathbf{{x}_{C}},\mathbf{{y}_{C}},\mathbf{{z}_{C}} \right)}\right) f{\left(\mathbf{{x}_{C}},\mathbf{{y}_{C}},\mathbf{{z}_{C}} \right)} + \operatorname{v_{1}}{\left(\mathbf{{x}_{C}},\mathbf{{y}_{C}},\mathbf{{z}_{C}} \right)} \frac{\partial}{\partial \mathbf{{x}_{C}}} f{\left(\mathbf{{x}_{C}},\mathbf{{y}_{C}},\mathbf{{z}_{C}} \right)} + \operatorname{v_{2}}{\left(\mathbf{{x}_{C}},\mathbf{{y}_{C}},\mathbf{{z}_{C}} \right)} \frac{\partial}{\partial \mathbf{{y}_{C}}} f{\left(\mathbf{{x}_{C}},\mathbf{{y}_{C}},\mathbf{{z}_{C}} \right)} + \operatorname{v_{3}}{\left(\mathbf{{x}_{C}},\mathbf{{y}_{C}},\mathbf{{z}_{C}} \right)} \frac{\partial}{\partial \mathbf{{z}_{C}}} f{\left(\mathbf{{x}_{C}},\mathbf{{y}_{C}},\mathbf{{z}_{C}} \right)}$

```python
lhs.doit().expand().simplify() == rhs.doit().expand().simplify()
```

    True

```python
sm.srepr(x**2 + x*y)
```

    "Add(Pow(Symbol('x'), Integer(2)), Mul(Symbol('x'), Symbol('y')))"

```python

```

```python

```


```python
from miscpy.utils.sympyhelpers import *
init_printing()
```

```python
s,p,q,sd,pd,qd,u = symbols('sigma,p,q,sigmadot,pdot,qdot,u')
diffmap = {s:sd,p:pd,q:qd}

def genfgseries(N):
    fn = s/s
    gn = 0*s
    fs = [fn]
    gs = [gn]
    for j in range(N):
        print(j)
        fnp1 = difftotal(fn,t,diffmap) - s*gn
        gnp1 = fn + difftotal(gn,t,diffmap)
        fn = fnp1
        gn = gnp1
        fn = simplify(fn.subs(([sd,-3*s*p],[pd,q-2*p**2],[qd,-p*(s+2*q)])))
        gn = simplify(gn.subs(([sd,-3*s*p],[pd,q-2*p**2],[qd,-p*(s+2*q)])))
        fs.append(fn)
        gs.append(gn)
    return fs,gs
```

```python
fs,gs = genfgseries(8)
```

    0
    1
    2
    3
    4
    5
    6
    7

```python
for f in fs:
    display(f)
```

```python
for g in gs:
    display(g)
```

```python
dt = symbols("\Delta{t}")
```

```python
fseries = 0
for j in range(len(fs)):
    fseries = fseries + dt**j/factorial(j)*fs[j]
```

```python
fseries
```

```python
gseries = 0
for j in range(len(gs)):
    gseries = gseries + dt**j/factorial(j)*gs[j]
```

```python
gseries
```

```python
fdotseries = 0
for j in range(1,len(fs)):
    fdotseries = fdotseries + dt**(j-1)/factorial(j-1)*fs[j]
```

```python
fdotseries
```

```python
gdotseries = 0
for j in range(1,len(gs)):
    gdotseries = gdotseries + dt**(j-1)/factorial(j-1)*gs[j]
```

```python
gdotseries
```

```python

```

output for MATLAB

```python
from sympy.utilities.codegen import codegen
```

```python
dtboring = symbols("dt")
```

```python
print(codegen([("fseries", fseries.subs(dt,dtboring).as_ordered_terms()[::-1]),
               ("gseries", gseries.subs(dt,dtboring).as_ordered_terms()[::-1]),
               ("fdotseries", fdotseries.subs(dt,dtboring).as_ordered_terms()[::-1]),
               ("gdotseries", gdotseries.subs(dt,dtboring).as_ordered_terms()[::-1])],
               language="Octave")[0][1])
```

    function [out1, out2, out3, out4, out5, out6, out7, out8] = fseries(dt, p, q, sigma)
      %FSERIES  Autogenerated by sympy
      %   Code generated with sympy 1.5.1
      %
      %   See http://www.sympy.org/ for more information.
      %
      %   This file is part of 'project'

      out1 = 1;
      out2 = -dt.^2.*sigma/2;
      out3 = dt.^3.*p.*sigma/2;
      out4 = dt.^4.*sigma.*(-15*p.^2 + 3*q + sigma)/24;
      out5 = dt.^5.*p.*sigma.*(7*p.^2 - 3*q - sigma)/8;
      out6 = dt.^6.*sigma.*(-945*p.^4 + 630*p.^2.*q + 210*p.^2.*sigma - 45*q.^2 - 24*q.*sigma - sigma.^2)/720;
      out7 = dt.^7.*p.*sigma.*(165*p.^4 - 150*p.^2.*q - 50*p.^2.*sigma + 25*q.^2 + 14*q.*sigma + sigma.^2)/80;
      out8 = dt.^8.*sigma.*(-135135*p.^6 + 155925*p.^4.*q + 51975*p.^4.*sigma - 42525*p.^2.*q.^2 - 24570*p.^2.*q.*sigma - 2205*p.^2.*sigma.^2 + 1575*q.^3 + 1107*q.^2.*sigma + 117*q.*sigma.^2 + sigma.^3)/40320;

    end

    function [out1, out2, out3, out4, out5, out6, out7] = gseries(dt, p, q, sigma)

      out1 = dt;
      out2 = -dt.^3.*sigma/6;
      out3 = dt.^4.*p.*sigma/4;
      out4 = dt.^5.*sigma.*(-45*p.^2 + 9*q + sigma)/120;
      out5 = dt.^6.*p.*sigma.*(14*p.^2 - 6*q - sigma)/24;
      out6 = dt.^7.*sigma.*(-4725*p.^4 + 3150*p.^2.*q + 630*p.^2.*sigma - 225*q.^2 - 54*q.*sigma - sigma.^2)/5040;
      out7 = dt.^8.*p.*sigma.*(495*p.^4 - 450*p.^2.*q - 100*p.^2.*sigma + 75*q.^2 + 24*q.*sigma + sigma.^2)/320;

    end

    function [out1, out2, out3, out4, out5, out6, out7] = fdotseries(dt, p, q, sigma)

      out1 = -dt.*sigma;
      out2 = 3*dt.^2.*p.*sigma/2;
      out3 = dt.^3.*sigma.*(-15*p.^2 + 3*q + sigma)/6;
      out4 = 5*dt.^4.*p.*sigma.*(7*p.^2 - 3*q - sigma)/8;
      out5 = dt.^5.*sigma.*(-945*p.^4 + 630*p.^2.*q + 210*p.^2.*sigma - 45*q.^2 - 24*q.*sigma - sigma.^2)/120;
      out6 = 7*dt.^6.*p.*sigma.*(165*p.^4 - 150*p.^2.*q - 50*p.^2.*sigma + 25*q.^2 + 14*q.*sigma + sigma.^2)/80;
      out7 = dt.^7.*sigma.*(-135135*p.^6 + 155925*p.^4.*q + 51975*p.^4.*sigma - 42525*p.^2.*q.^2 - 24570*p.^2.*q.*sigma - 2205*p.^2.*sigma.^2 + 1575*q.^3 + 1107*q.^2.*sigma + 117*q.*sigma.^2 + sigma.^3)/5040;

    end

    function [out1, out2, out3, out4, out5, out6, out7] = gdotseries(dt, p, q, sigma)

      out1 = 1;
      out2 = -dt.^2.*sigma/2;
      out3 = dt.^3.*p.*sigma;
      out4 = dt.^4.*sigma.*(-45*p.^2 + 9*q + sigma)/24;
      out5 = dt.^5.*p.*sigma.*(14*p.^2 - 6*q - sigma)/4;
      out6 = dt.^6.*sigma.*(-4725*p.^4 + 3150*p.^2.*q + 630*p.^2.*sigma - 225*q.^2 - 54*q.*sigma - sigma.^2)/720;
      out7 = dt.^7.*p.*sigma.*(495*p.^4 - 450*p.^2.*q - 100*p.^2.*sigma + 75*q.^2 + 24*q.*sigma + sigma.^2)/40;

    end

```python

```


```python
import volpriceinference as vl
import numpy as np
from hypothesis import given, strategies as st
import sympy as sym
```

```python

```

```python

```

```python

```

```python
vl.A_func
```

    <function libvolpriceinference.PyCapsule.A_func>

```python

```

```python
sym.N(vl.volprice._A_func.replace(vl.logit_rho, 1).replace(vl.log_scale, 0).replace(vl.volprice._x, 1))
```

    0.365529289315002

```python
def test_link_total(phi, pi, theta, log_both, log_scale, logit_rho, psi):

    args = {vl.volprice.phi: phi, vl.volprice.pi: pi, vl.volprice.theta: theta, vl.volprice.log_both:log_both,
            vl.volprice.log_scale: log_scale, vl.volprice.logit_rho: logit_rho, vl.volprice.psi : psi}

    val1 = np.real(np.array(sym.N(vl.volprice._link_grad_sym.xreplace(args))).astype(np.complex))
    val2 = vl.link_jacobian(phi=phi, pi=pi, theta=theta, logit_rho=logit_rho,
                         log_both=log_both, log_scale=log_scale, psi=psi)

    return val1 - val2

```

```python
test_link_total(phi = -0.1, pi = -0.1, theta = 0.1, log_both = 0.0, log_scale = 0.02, logit_rho = 0.0, psi = 0.0
)
```

    array([[ 0.00000000e+00,  0.00000000e+00,  0.00000000e+00,
             0.00000000e+00,  0.00000000e+00, -2.77555756e-16,
             0.00000000e+00],
           [ 0.00000000e+00,  0.00000000e+00,  1.11022302e-16,
            -3.33066907e-16, -4.44089210e-16,  0.00000000e+00,
             0.00000000e+00],
           [ 0.00000000e+00,  0.00000000e+00,  0.00000000e+00,
             0.00000000e+00,  0.00000000e+00,  0.00000000e+00,
             0.00000000e+00],
           [ 0.00000000e+00,  0.00000000e+00,  0.00000000e+00,
             0.00000000e+00,  0.00000000e+00,  0.00000000e+00,
             0.00000000e+00]])

```python
B_func =  vl.sym.exp(vl.log_both - vl.log_scale) * (sym.log(vl.volprice._B_func_in))

```

```python
args = {vl.volprice.phi: -.1, vl.volprice.pi: -.1, vl.volprice.theta: .1, vl.volprice.log_both:1,
            vl.volprice.log_scale: 1, vl.volprice.logit_rho: 0, vl.volprice.psi : .5}
```

```python
vl.sym.N(B_func.diff(vl.log_scale).xreplace(args).replace(vl.volprice._x, 2))
```

    -1.01735720755521

```python
B_func
```

    exp(log_both - log_scale)*log(_x*exp(log_scale) + 1)

```python
B_func.diff(vl.log_scale)
```

    _x*exp(log_scale)*exp(log_both - log_scale)/(_x*exp(log_scale) + 1) - exp(log_both - log_scale)*log(_x*exp(log_scale) + 1)

```python
vl.B_diff3(2, log_both=1, log_scale=1)
```

    -1.0173572075552146

```python
(arg1 - arg2).xreplace(args)
```

```python
lvl.C_func(1.3, 0, 0)
```

```python
np.real(5 + 3j)
```

```python
lvl.A_func(-1, 1, 0)
```

```python
vl.volprice._B_func_in.replace(vl.log_scale, 0)
```

```python
sym.N(vl.volprice._A_func.replace(vl.logit_rho, 1).replace(vl.log_scale, 0).replace(vl.volprice._x, -1)))
```

```python
type(np.real(0j))
```

```python
test_link(1,1,1, 1, 1, 1)
```

```python

```

```python
val2
```

```python
lvl.A_func(logit_rho=1, log_scale=0, x=1)
```

```python
gamma_val = vl.compute_gamma(logit_rho=0, log_scale=0, log_both=0, pi=0, theta=0, phi=0)

```

```python

```


$\newcommand{\RR}{\mathbb{R}}$
$\newcommand{\R}[1]{\RR^{#1}}$
$\newcommand{\SO}[1]{\mathit{SO}(#1)}$
# The $\SO{3} \subset \SO{5}$ Subgroup Chain

## Background

ACM uses the subgroup chain $\SO{3} \subset \SO{5}$ where $\SO{5}$ acts on the
5-dimensional space of quadrapole moments $q_M, M \in \{-2, -1, 0, 1, 2\}$ which
transform as an $L=2$ irrep of $\SO{3}$.
The inner product on this space is $|q|^2 = \sum_M |q_M|^2$.

## Problem

Show that the action of $\SO{3}$ on the quadrapole moments leaves the norm invariant
so that we have a subgroup inclusion $\SO{3} \subset \SO{5}$.

## The Fundamental Representation of $\SO{3}$

The fundamental representation of $\SO{3}$ acts on 3-dimensional space
$\R{3}$
endowed with the usual Euclidean inner product.
First we need to decide how to represent $\R{3}$.

## Real Numbers and Symbols

SymPy supports real and complex numeric types.

```python
from sympy import *

mlue = Integer(42)
mlue, mlue.is_real, type(mlue.is_real)
```

    (42, True, bool)

```python
alpha = Rational(1, 137)
alpha, alpha.is_real, type(alpha.is_real)
```

    (1/137, True, bool)

By default, the real number status of symbols is undefined.

```python
z = symbols('z')
z.is_real, type(z.is_real)
```

    (None, NoneType)

SymPy also allows symbols to be explicitly declared as real.

```python
w = symbols('w', real=False)
w.is_real, type(w.is_real)
```

    (False, bool)

```python
a = symbols('a', real=True)
a, a.is_real, type(a.is_real)
```

    (a, True, bool)

## Vectors

There are several ways to represent vectors in Python and SymPy.
The simplest way might be to represent vectors in $\R{3}$ as SymPy matrices
whose shape is (3, 1), i.e. as column vectors.

```python
def is_real_matrix(A: Matrix) -> bool:
    return isinstance(A, Matrix) and all([a.is_real for a in A])

A1 = Matrix([[mlue, alpha], [a, 0]])
assert is_real_matrix(A1)

A1
```

$\displaystyle \left[\begin{matrix}42 & \frac{1}{137}\\a & 0\end{matrix}\right]$

```python
W = Matrix([w])
assert not is_real_matrix(Matrix([w]))

W
```

$\displaystyle \left[\begin{matrix}w\end{matrix}\right]$

```python
def is_R3_vector(v: Matrix) -> bool:
    return is_real_matrix(v) and v.shape == (3, 1)

e1 = Matrix([1, 0, 0])
assert is_R3_vector(e1)

e1
```

$\displaystyle \left[\begin{matrix}1\\0\\0\end{matrix}\right]$

```python
e1.shape
```

    (3, 1)

```python
type(e1)
```

    sympy.matrices.dense.MutableDenseMatrix

In SymPy, `Matrix` objects are mutable. Mathematically it is more natural to
regard objects as immutable. Therefore, `ImmutableMatrix` should probably be used for the basis vectors of $\R{3}$.
However, I'll never mutate the vectors so, for simplicity,
I'll leave them as `Matrix` for now.

```python
e2 = Matrix([0, 1, 0])
e2
```

$\displaystyle \left[\begin{matrix}0\\1\\0\end{matrix}\right]$

```python
e3 = Matrix([0, 0, 1])
e3
```

$\displaystyle \left[\begin{matrix}0\\0\\1\end{matrix}\right]$

SymPy implements scalar multiplication and vector addition naturally.

```python
v1 = 3*e1 + 4*e2 - 5*e3
v1
```

$\displaystyle \left[\begin{matrix}3\\4\\-5\end{matrix}\right]$

SymPy `Matrix` objects are iterable.

```python
[(c, type(c)) for c in v1]
```

    [(3, sympy.core.numbers.Integer),
     (4, sympy.core.numbers.Integer),
     (-5, sympy.core.numbers.Integer)]

```python
[(i, c) for i, c in enumerate(v1)]
```

    [(0, 3), (1, 4), (2, -5)]

## Dual Vectors

Dual vectors are created using the transpose operator.

```python
def is_R3_dual_vector(f: Matrix) -> bool:
    return is_real_matrix(f) and f.shape == (1, 3)

f1 = e1.T
assert(is_R3_dual_vector(f1))

f1
```

$\displaystyle \left[\begin{matrix}1 & 0 & 0\end{matrix}\right]$

```python
[e for e in enumerate(f1)]
```

    [(0, 1), (1, 0), (2, 0)]

```python
f2 = e2.T
f3 = e3.T
f1 * v1
```

$\displaystyle \left[\begin{matrix}3\end{matrix}\right]$

```python
v1.T * v1
```

$\displaystyle \left[\begin{matrix}50\end{matrix}\right]$

```python
(v1.T * v1).shape
```

    (1, 1)

```python
[e for e in enumerate(v1.T * v1)]
```

    [(0, 50)]

## Inner Product

Note that multiplying a dual vector times a vector results
in a matrix with shape (1, 1), not a scalar.

```python
def inner_product(u: Matrix, v: Matrix) -> Basic:
    return (u.T * v)[0]

norm_v1_squared = inner_product(v1, v1)
norm_v1_squared, type(norm_v1_squared)
```

    (50, sympy.core.numbers.Integer)

## Linear Transformations

A linear transformation is naturally represented by a matrix
with shape (3, 3).
$\newcommand{\glR}[1]{\mathit{gl}({#1},\RR)}$
The set of linear transformations is denoted $\glR{3}$.

```python
def is_gl3R(M: Matrix) -> bool:
    return is_real_matrix(M) and M.shape == (3, 3)

is_gl3R(zeros(3, 3))
```

    True

```python
from sympy import eye
I3 = eye(3)
I3
```

$\displaystyle \left[\begin{matrix}1 & 0 & 0\\0 & 1 & 0\\0 & 0 & 1\end{matrix}\right]$

```python
I3 * v1
```

$\displaystyle \left[\begin{matrix}3\\4\\-5\end{matrix}\right]$

```python
two_I = 2 * I3
two_I
```

$\displaystyle \left[\begin{matrix}2 & 0 & 0\\0 & 2 & 0\\0 & 0 & 2\end{matrix}\right]$

```python
two_I * v1
```

$\displaystyle \left[\begin{matrix}6\\8\\-10\end{matrix}\right]$

```python
zero_M = zeros(3, 3)
zero_M
```

$\displaystyle \left[\begin{matrix}0 & 0 & 0\\0 & 0 & 0\\0 & 0 & 0\end{matrix}\right]$

```python
zero_M * v1
```

$\displaystyle \left[\begin{matrix}0\\0\\0\end{matrix}\right]$

```python
diag_123 = diag(1, 2, 3)
diag_123
```

$\displaystyle \left[\begin{matrix}1 & 0 & 0\\0 & 2 & 0\\0 & 0 & 3\end{matrix}\right]$

```python
diag_123 * v1
```

$\displaystyle \left[\begin{matrix}3\\8\\-15\end{matrix}\right]$

## Bases

We can express any vector $v$ as a unique linear combination of any three given linearly independent vectors, e.g. $e_1, e_2, e_3$.

```python
a, b, c = symbols('a b c')
lc = a * e1 + b * e2 + c * e3
lc
```

$\displaystyle \left[\begin{matrix}a\\b\\c\end{matrix}\right]$

```python
eqns = lc - v1
eqns
```

$\displaystyle \left[\begin{matrix}a - 3\\b - 4\\c + 5\end{matrix}\right]$

```python
A, B = linear_eq_to_matrix(eqns, [a, b, c])
A
```

$\displaystyle \left[\begin{matrix}1 & 0 & 0\\0 & 1 & 0\\0 & 0 & 1\end{matrix}\right]$

```python
B
```

$\displaystyle \left[\begin{matrix}3\\4\\-5\end{matrix}\right]$

```python
linsolve((A, B), [a, b, c])
```

$\displaystyle \left\{\left( 3, \  4, \  -5\right)\right\}$

```python
system = [eq for eq in eqns]
system
```

    [a - 3, b - 4, c + 5]

```python
linsolve(system, [a, b, c])
```

$\displaystyle \left\{\left( 3, \  4, \  -5\right)\right\}$

```python
solution = { [a, b, c][i]: value for i, value in enumerate((3, 4, -5))}
solution
```

    {a: 3, b: 4, c: -5}

```python
lc.subs(solution)
```

$\displaystyle \left[\begin{matrix}3\\4\\-5\end{matrix}\right]$

```python
lc.subs(solution) == v1
```

    True

## Nonsingular Linear Transformations

$\newcommand{\GLR}[1]{\mathit{GL}({#1},\RR)}$
The nonsingular linear transformations are denoted $\GLR{3})$.

```python
def is_GL3R(A: Matrix) -> bool:
    return is_gl3R(A) and simplify(det(A)) != 0

det(I3), type(det(I3))
```

    (1, sympy.core.numbers.One)

```python
assert is_GL3R(I3)
```

## Special Linear Transformations

$\newcommand{\SLR}[1]{\mathit{SL}({#1},\RR)}$
The set of special linear transformations is denoted $\SLR{3}$.

```python
def is_SL3R(A: Matrix) -> bool:
    return is_gl3R(A) and simplify(det(A)) == 1

assert is_SL3R(I3)
```

## Orthogonal Transformations

$\newcommand{\O}[1]{\mathit{O}({#1})}$
The set of orthogonal transformations is denoted $\O{3}$. They preserve the inner product.

```python
def is_O3(M: Matrix) -> bool:
    return is_gl3R(M) and simplify(M.T * M) == eye(3)

assert is_O3(I3)

reflect_z = Matrix([[1, 0, 0], [0, 1, 0], [0, 0, -1]])
assert is_O3(reflect_z)

reflect_z
```

$\displaystyle \left[\begin{matrix}1 & 0 & 0\\0 & 1 & 0\\0 & 0 & -1\end{matrix}\right]$

## Special Orthogonal Transformations

The set of special orthogonal transformations is denoted $\SO{3}$.
They preserve orientation.

```python
def is_SO3(M: Matrix) -> bool:
    return is_O3(M) and is_SL3R(M)

assert is_SO3(I3)
assert not is_SO3(reflect_z)
```

## Rotations about the z-axis

Let $R_z(\theta)$ denote a counter-clockwise rotation about the z-axis by the
angle $\theta$.

```python
def rotate_z(theta: Basic) -> Matrix:
    assert isinstance(theta, Basic)
    assert theta.is_real
    return Matrix([[cos(theta), -sin(theta), 0],
                   [sin(theta), cos(theta), 0],
                   [0, 0, 1]])

R_0 = rotate_z(S.Zero)
assert is_SO3(R_0)

R_0
```

$\displaystyle \left[\begin{matrix}1 & 0 & 0\\0 & 1 & 0\\0 & 0 & 1\end{matrix}\right]$

```python
R_45 = rotate_z(pi/4)
assert is_SO3(R_45)

R_45
```

$\displaystyle \left[\begin{matrix}\frac{\sqrt{2}}{2} & - \frac{\sqrt{2}}{2} & 0\\\frac{\sqrt{2}}{2} & \frac{\sqrt{2}}{2} & 0\\0 & 0 & 1\end{matrix}\right]$

```python
theta = symbols('theta', real=True)
theta, theta.is_real
```

    (theta, True)

```python
R_theta = rotate_z(theta)
assert is_SO3(R_theta)

R_theta
```

$\displaystyle \left[\begin{matrix}\cos{\left(\theta \right)} & - \sin{\left(\theta \right)} & 0\\\sin{\left(\theta \right)} & \cos{\left(\theta \right)} & 0\\0 & 0 & 1\end{matrix}\right]$

```python
v1
```

$\displaystyle \left[\begin{matrix}3\\4\\-5\end{matrix}\right]$

```python
R_theta * v1
```

$\displaystyle \left[\begin{matrix}- 4 \sin{\left(\theta \right)} + 3 \cos{\left(\theta \right)}\\3 \sin{\left(\theta \right)} + 4 \cos{\left(\theta \right)}\\-5\end{matrix}\right]$

## Bivectors

Next consider the action $\rho$ of $\SO{3}$ on the tensor product
$\R{3} \otimes \R{3}$.

$$
\rho(R) (u \otimes v) = (R~u) \otimes (R~v)
$$

Clearly $\R{3} \otimes \R{3}$ has dimension 9.
We can represent its vectors as `Matrix` objects that have shape (9,1).
We need to define a standard basis and the tensor product operation on vectors.