#Optimize API

Suppose you've written an algorithm that's developed a signal (for example, a [Pipeline Factor](https://www.quantopian.com/tutorials/pipeline#lesson3)) that's predictive of future asset returns. You might think that the hard part is over, but you're still left with the daunting task of translating your signal into an algorithm that can turn a profit while also managing risk exposures.

Traditionally in quantitative finance, the solution to the problem of maximizing returns while constraining risk has been to employ some form of [Portfolio Optimization](https://en.wikipedia.org/wiki/Portfolio_optimization), but performing sophisticated optimizations is challenging on today's Quantopian.

Python libraries like [scipy.optimize](http://docs.scipy.org/doc/scipy/reference/optimize.html), [CVXOPT](http://cvxopt.org/), and [CVXPY](http://www.cvxpy.org/en/latest/) (all available on Quantopian today) provide generic tools for solving optimization problems. These libraries are powerful and flexible, but it takes significant expertise to convert the data structures available on Quantopian into the specific formats understood by these libraries.

The [**Optimize API**](https://www.quantopian.com/help#optimize-title) is a suite of tools for defining and solving portfolio optimization problems directly in terms of financial domain concepts. Users interact with the Optimize API by providing an `Objective` and a list of `Constraint` objects to an API function that runs a portfolio optimization. The Optimize API hides most of the complex mathematics of portfolio optimization, allowing users to think in terms of high-level concepts like “maximize expected returns” and “constrain sector exposure” instead of abstract matrix products.

### What is Portfolio Optimization?

Many algorithms written on Quantopian reach a point where they need to solve an **optimization problem** of the following form:

Given a list of stocks, choose a vector,

$$w$$

of weights in each stock that maximizes (or minimizes) an **objective** function,

$$F(w)$$

subject to a set of inequality **constraints**,

\begin{align}
C_1(w) &\leq h_1 \\
C_2(w) &\leq h_2    \\
&\dots           \\
C_N(w) &\leq h_N    \\
\end{align}

####Example:

An algorithm builds a model that predicts expected returns for a list of stocks. The algorithm wants to allocate a limited amount of capital to those stocks in a way that gives it the greatest possible expected return without placing too big a bet on any single stock.

We can frame this as an optimization problem as follows:

Given an expected returns vector $r = [r_0, r_1, \dots, r_n]$, a maximum single-stock weight $w_{max}$, and a maximum total weight $W_{max}$, find a portfolio weight vector $w = [w_0, w_1, \dots, w_n]$ that solves the following system:

\begin{align}
\text{maximize}&& \\
&&r \cdot w \\
\text{subject to}&& \\
&&w_i &\leq w_{max} && 0 \leq i \leq n\\
&& \sum_{i=0}^{n}{|w_i|} &\leq W_{max}
\end{align}

A more sophisticated algorithm might introduce a more complex objective (e.g. "maximize return with a penalty applied for expected volatility"), or it might introduce more complex constraints (e.g. limit exposure to particular market sectors).

## API Overview

The Optimize API is defined in the [**`quantopian.optimize`**](https://www.quantopian.com/help#module-quantopian.optimize) module.

The `optimize` module has three important concepts:

1. [**`Objectives`**](https://www.quantopian.com/help#objectives): classes that define the function to be minimized or maximized by the optimizer.
2. [**`Constraints`**](https://www.quantopian.com/help#objectives): classes that define constraints that the optimizer must respect when finding a new portfolio.
3. [**`Entry Points`**](https://www.quantopian.com/help#running-optimizations), functions that actually run optimizations. The Optimize API has three entrypoints:
  - `calculate_optimal_portfolio`, the primary entrypoint in research notebooks.
  - `order_optimal_portfolio`, the primary entrypoint in trading algorithms.
  - `run_optimization`, a lower-level entrypoint useful for debugging.

To run a portfolio optimization **in a notebook**, you call `calculate_optimal_portfolio` and provide  up to three inputs:

- An `Objective` to optimize.
- A list of `Constraint`s to enforce.
- (optionally) A [pandas Series](http://pandas.pydata.org/pandas-docs/stable/generated/pandas.Series.html) containing weights for the current portfolio. If no current weights are provided, the optimization proceeds assuming that the current portfolio is empty.

To run a portfolio optimization **in a trading algorithm**, you call `order_optimal_portfolio` instead.
`order_optimal_portfolio` accepts two inputs:

- An `Objective` to optimize (same as above).
- A list of `Constraints` to enforce (same as above).

It is also possible to call `calculate_optimal_portfolio` from a trading algorithm to compute the portfolio that `order_optimal_portfolio` would attempt to achieve if invoked.

```python
import quantopian.optimize as opt
print opt.calculate_optimal_portfolio.__doc__
```

        Calculate optimal portfolio weights given ``objective`` and ``constraints``.

        Parameters
        ----------
        objective : Objective
            The objective to be minimized or maximized.
        constraints : list[Constraint]
            List of constraints that must be satisfied by the new portfolio.
        current_portfolio : pd.Series, optional
            A Series containing the current portfolio weights, expressed as
            percentages of the portfolio's liquidation value.

            When called from a trading algorithm, the default value of
            ``current_portfolio`` is the algorithm's current portfolio.

            When called interactively, the default value of ``current_portfolio``
            is an empty portfolio.

        Returns
        -------
        optimal_portfolio : pd.Series
            A Series containing portfolio weights that maximize (or minimize)
            ``objective`` without violating any ``constraints``. Weights should be
            interpreted in the same way as ``current_portfolio``.

        Raises
        ------

        InfeasibleConstraints
            Raised when there is no possible portfolio that satisfies the received
            constraints.
        UnboundedObjective
            Raised when the received constraints are not sufficient to put an upper
            (or lower) bound on the calculated portfolio weights.

        Notes
        -----
        This function is a shorthand for calling ``run_optimization``, checking for
        an error, and extracting the result's ``new_weights`` attribute.

        If an optimization problem is feasible, the following are equivalent::

            # Using calculate_optimal_portfolio.
            >>> weights = calculate_optimal_portfolio(objective, constraints, portfolio)

            # Using run_optimization.
            >>> result = run_optimization(objective, constraints, portfolio)
            >>> result.raise_for_status()  # Raises if the optimization failed.
            >>> weights = result.new_weights

        See Also
        --------
        :func:`quantopian.optimize.run_optimization`

Lists of the currently available objectives and constraints can be found under **`quantopian.optimize.objectives`** and **`quantopian.optimize.constraints`**, respectively.

```python
opt.objectives
```

    [quantopian.optimize.objectives.MaximizeAlpha,
     quantopian.optimize.objectives.Objective,
     quantopian.optimize.objectives.TargetWeights]

```python
opt.constraints
```

    [quantopian.optimize.constraints.Basket,
     quantopian.optimize.constraints.CannotHold,
     quantopian.optimize.constraints.Constraint,
     quantopian.optimize.constraints.DollarNeutral,
     quantopian.optimize.constraints.FactorExposure,
     quantopian.optimize.constraints.FixedWeight,
     quantopian.optimize.constraints.Frozen,
     quantopian.optimize.constraints.LongOnly,
     quantopian.optimize.constraints.MaxGrossExposure,
     quantopian.optimize.constraints.MaxTurnover,
     quantopian.optimize.constraints.NetExposure,
     quantopian.optimize.constraints.NetGroupExposure,
     quantopian.optimize.constraints.Pair,
     quantopian.optimize.constraints.PositionConcentration,
     quantopian.optimize.constraints.ReduceOnly,
     quantopian.optimize.constraints.ShortOnly]

### Example: Maximize Expected Returns with Exposure and Concentration Constraints

Let's look at how we can use the `optimize` API to solve the example outlined above:

> An algorithm builds a model that predicts expected returns for a list of stocks. The algorithm wants to allocate a limited amount of capital to those stocks in a way that gives it the greatest possible expected return without placing too big a bet on any single stock.

Since we're assuming we already have a returns model, we'll just use random numbers for this example.

```python
import numpy as np
import pandas as pd

def fancy_returns_model(assets):
    """Not actually fancy."""
    # Generate a random number for each asset.
    rng = np.random.RandomState(5)
    return pd.Series(index=assets, data=rng.randn(len(assets)))
```

We'll also add a few utility functions for plotting portfolio weights.

```python
def draw_asset_barplot(weights, title, plot_kwargs=None):
    """Draw a bar plot from a Series with Asset labels."""
    return draw_barplot(
        weights,
        title,
        xticks=[asset.symbol for asset in weights.index],
        plot_kwargs=plot_kwargs,
    )

def draw_barplot(weights, title, xticks=None, plot_kwargs=None):
    """Draw a bar plot from a pd.Series."""

    # Draw the plot. Forward plot_kwargs (if provided) as keywords to ``plot``.
    axes = weights.plot(kind='bar', rot=0, fontsize=12, **(plot_kwargs or {}))
    axes.grid(False, axis='x')

    # Set a title.
    axes.set_title(title, {'fontsize': 14})

    # Set xtick labels, if provided.
    if xticks is not None:
        axes.set_xticklabels(xticks)

    return axes
```

```python
# Choose a small universe to make the graphs manageable.
universe = symbols(
    ['AAPL', 'MSFT', 'TWTR', 'BP', 'XOM', 'MCD', 'QSR'],
    symbol_reference_date='2016-10-07',
)

empty_portfolio = pd.Series(index=universe, data=0.0)
expected_returns = fancy_returns_model(universe)
```

```python
draw_asset_barplot(expected_returns, 'Expected Returns');
```

Since our goal is to maximize a function directly proportional to asset returns, we'll use **`opt.MaximizeAlpha`** as our objective function.

```python
print opt.MaximizeAlpha.__doc__
```

        Objective that maximizes ``weights.dot(alphas)`` for an alpha vector.

        Ideally, ``alphas`` should contain coefficients such that ``alphas[asset]``
        is proportional to the expected return of ``asset`` for the time horizon
        over which the target portfolio will be held.

        In the special case that ``alphas`` is an estimate of expected returns for
        each asset, this objective simply maximizes the expected return of the
        total portfolio.

        Parameters
        ----------
        alphas : pd.Series[Asset -> float] or dict[Asset -> float]
            Map from assets to alpha coefficients for those assets.

        Notes
        -----
        This objective should almost always be used with a `MaxGrossExposure`
        constraint, and should usually be used with a `PositionConcentration`
        constraint.

        Without a constraint on gross exposure, this objective will raise an error
        attempting to allocate an unbounded amount of capital to every asset with a
        nonzero alpha.

        Without a constraint on individual position size, this objective will
        allocate all of its capital in the single asset with the largest expected
        return.

```python
maximize_returns = opt.MaximizeAlpha(expected_returns)
maximize_returns
```

    <quantopian.optimize.objectives.MaximizeAlpha at 0x7f3256f1cad0>

`Objective` classes can render themselves in mathematical notation via the `.math` property:

```python
maximize_returns.math
```

\begin{align*}
&\underset{\boldsymbol{w}}{\text{maximize}} && \boldsymbol{\alpha} \cdot \boldsymbol{w} \\
&\text{where} &&\boldsymbol{\alpha} = [0.441, -0.331, \ldots, -0.909]
\end{align*}

The documentation for `MaximizeAlpha` warns us that we need to put a constraint on the gross exposure of our portfolio or else the optimizer will try to allocate an unbounded amount of capital, so we'll add a **`MaxGrossExposure`** constraint as well.

```python
print opt.MaxGrossExposure.__doc__
```

        Constraint on the maximum gross exposure for the portfolio.

        Requires that the sum of the absolute values of the portfolio weights be
        less than ``max``.

        Parameters
        ----------
        max : float
            The maximum gross exposure of the portfolio.

        Examples
        --------
        ``MaxGrossExposure(1.5)`` constrains the total value of the
        portfolios longs and shorts to be no more than 1.5x the current portfolio
        value.

```python
def optimal_portfolio_constrained_exposure_only():
    """Calculate the optimal portfolio if we're only constrained by gross exposure."""

    # Our objective function.
    objective = opt.MaximizeAlpha(expected_returns)

    # A list containing our constraints.
    constraints = [opt.MaxGrossExposure(1.0)]

    return opt.calculate_optimal_portfolio(objective, constraints, empty_portfolio)
```

```python
constrained_exposure_pf = optimal_portfolio_constrained_exposure_only()
draw_asset_barplot(constrained_exposure_pf, 'Optimal Portfolio (Constrained Exposure Only)');
```

With no constraints other than maximizing expected return, the optimizer decided to allocate all of our capital to a single long position in `TWTR`. While this might be an acceptable portfolio for someone of a gambling disposition, we'd probably rather not put all our eggs in one basket.

We can force the optimizer to diversify our portfolio by adding a **`PositionConcentration`** constraint:

```python
print opt.PositionConcentration.__doc__
```

        Constraint enforcing minimum/maximum position weights.

        Parameters
        ----------
        min_weights : pd.Series[Asset -> float] or dict[Asset -> float]
            Map from asset to minimum position weight for that asset.
        max_weights : pd.Series[Asset -> float] or dict[Asset -> float]
            Map from asset to maximum position weight for that asset.
        default_min_weight : float, optional
            Value to use as a lower bound for assets not found in ``min_weights``.
            Default is 0.0.
        default_max_weight : float, optional
            Value to use as a lower bound for assets not found in ``max_weights``.
            Default is 0.0.
        etf_lookthru : pd.DataFrame, optional
            Indexed by constituent assets x ETFs, expresses the weight of each
            constituent in each ETF.

            A DataFrame containing ETF constituents data. Each column of the frame
            should contain weights (from 0.0 to 1.0) representing the holdings for
            an ETF. Each row should contain weights for a single stock in each
            ETF. Columns should sum approximately to 1.0. If supplied, ETF
            holdings in the current and target portfolio will be decomposed into
            their constituents before constraints are applied.

        Notes
        -----
        Negative weight values are interpreted as bounds on the magnitude of short
        positions. A minimum weight of 0.0 constrains an asset to be long-only. A
        maximum weight of 0.0 constrains an asset to be short-only.

        A common special case is to create a PositionConcentration constraint that
        applies a shared lower/upper bound to all assets. An alternate constructor,
        :meth:`PositionConcentration.with_equal_bounds`, provides a simpler API
        supporting this use-case.

```python
print opt.PositionConcentration.with_equal_bounds.__doc__
```

            Special case constructor that applies static lower and upper bounds to
            all assets.

            ::

                PositionConcentration.with_equal_bounds(min, max)

            is equivalent to::

                PositionConcentration(pd.Series(), pd.Series(), min, max)

            Parameters
            ----------
            min : float
                Minimum position weight for all assets.
            max : float
                Maximum position weight for all assets.

```python
# Allow 15% of our liquidation value to be in any single short position.
# Allow 30% of our liquidation value to be in any single long position.
# Try changing these parameters to see how they change the results!
MIN_POSITION_WEIGHT = -0.15
MAX_POSITION_WEIGHT = 0.30

def optimal_portfolio_constrained_exposure_and_concentration():
    """Calculate the optimal portfolio if we're constrained by exposure and concentration."""
    objective = opt.MaximizeAlpha(expected_returns)
    constraints = [
        opt.MaxGrossExposure(1.0),
        opt.PositionConcentration.with_equal_bounds(
            min=MIN_POSITION_WEIGHT,
            max=MAX_POSITION_WEIGHT,
        )
    ]
    return opt.calculate_optimal_portfolio(objective, constraints, empty_portfolio)
```

```python
constrained_concentration_pf = optimal_portfolio_constrained_exposure_and_concentration()
plot = draw_asset_barplot(
    constrained_concentration_pf,
    "Optimal Portfolio (Constrained Exposure and Concentration)",
    {'ylim': (-MAX_POSITION_WEIGHT - 0.1, MAX_POSITION_WEIGHT + 0.1)},
)
plot.axhline(MIN_POSITION_WEIGHT, color='r', linestyle='dashed')
plot.axhline(MAX_POSITION_WEIGHT, color='r', linestyle='dashed');
```

With position size constraints, our portfolio is significantly more diverse than before.  The optimizer was forced to cap the long allocations to `TWTR` and `GM` at 30% of NLV (Net Liquidation Value), and it was forced to cap the short allocations to `MCD` and `BK` at 15% NLV.  The remaining 10% under the gross exposure cap went to `AAPL`.

### Adding Sector Constraints

Eyeballing these distributions a bit, we might notice that we're heavily exposed to the technology and food service industries.  We can confirm that concern by using `Pipeline` to grab sector codes for our portfolio.

```python
from quantopian.pipeline import Pipeline
from quantopian.pipeline.classifiers.morningstar import Sector
from quantopian.pipeline.filters import StaticAssets
from quantopian.research import run_pipeline

def get_sector_names(stocks, date):
    """Get sector codes for a set of stocks for a particular date."""
    result = run_pipeline(
        Pipeline({'sector': Sector()}, screen=StaticAssets(stocks)),
        start_date=date,
        end_date=date
    )
    # result will have a pd.MultiIndex whose first level just contains `date`.
    # Drop the first level so that we just get assets as our index.
    sector_codes = result['sector'].reset_index(level=0, drop=True)

    # Convert integral sector codes to human-readable strings by doing a lookup
    # from Sector.SECTOR_NAMES, which is a dict mapping codes to strings.
    #
    # This is purely for readability of the charts below.
    # We could proceed with the rest of the notebook using sector codes and it would work just fine.
    return sector_codes.map(Sector.SECTOR_NAMES)
```

```python
sector_names = get_sector_names(universe, '2016-10-07')
sector_names
```

    Equity(24 [AAPL])              TECHNOLOGY
    Equity(4707 [MCD])      CONSUMER_CYCLICAL
    Equity(5061 [MSFT])            TECHNOLOGY
    Equity(8347 [XOM])                 ENERGY
    Equity(19675 [BP])                 ENERGY
    Equity(45815 [TWTR])           TECHNOLOGY
    Equity(48215 [QSR])     CONSUMER_CYCLICAL
    Name: sector, dtype: object

```python
def plot_sector_exposure(pf, sectors):
    return draw_barplot(
        pf.groupby(sectors).sum(),
        'Portfolio Concentration by Sector',
    );

plot_sector_exposure(constrained_concentration_pf, sector_names);
```

As we suspected, we're heavily exposed to the technology sector, and we're moderately exposed to cyclical consumer goods.

We can tell the optimizer to try to build a sector neutral portfolio by adding a `NetGroupExposure` constraint:

```python
print opt.NetGroupExposure.__doc__
```

        Constraint requiring bounded net exposure to groups of assets.

        Groups are defined by map from (asset -> label). Each unique label
        generates a constraint specifying that the sum of the weights of assets
        mapped to that label should fall between a lower and upper bounds.

        Min/Max group exposures are specified as maps from (label -> float).

        Examples of common group labels are sector, industry, and country.

        Parameters
        ----------
        labels : pd.Series[Asset -> object] or dict[Asset -> object]
            Map from asset -> group label.
        min_weights : pd.Series[object -> float] or dict[object -> float]
            Map from group label to minimum net exposure to assets in that group.
        max_weights : pd.Series[object -> float] or dict[object -> float]
            Map from group label to maximum net exposure to assets in that group.
        etf_lookthru : pd.DataFrame, optional
            Indexed by constituent assets x ETFs, expresses the weight of each
            constituent in each ETF.

            A DataFrame containing ETF constituents data. Each column of the frame
            should contain weights (from 0.0 to 1.0) representing the holdings for
            an ETF. Each row should contain weights for a single stock in each
            ETF. Columns should sum approximately to 1.0. If supplied, ETF
            holdings in the current and target portfolio will be decomposed into
            their constituents before constraints are applied.

        Examples
        --------
        Suppose we're interested in four stocks: AAPL, MSFT, TSLA, and GM.

        AAPL and MSFT are in the technology sector. TSLA and GM are in the consumer
        cyclical sector. We want no more than 50% of our portfolio to be in the
        technology sector, and we want no more than 25% of our portfolio to be in
        the consumer cyclical sector.

        We can construct a constraint expressing the above preferences as follows:

        .. code-block:: python

            # Map from each asset to its sector.
            labels = {AAPL: 'TECH', MSFT: 'TECH', TSLA: 'CC', GM: 'CC'}

            # Maps from each sector to its min/max exposure.
            min_exposures = {'TECH': -0.5, 'CC': -0.25}
            max_exposures = {'TECH': 0.5, 'CC': 0.5}

            constraint = NetGroupExposure(labels, min_exposures, max_exposures)

        Notes
        -----
        For a group that should not have a lower exposure bound, set::

            min_weights[group_label] = opt.NotConstrained

        For a group that should not have an upper exposure bound, set::

            max_weights[group_label] = opt.NotConstrained

```python
ALL_SECTOR_NAMES = Sector.SECTOR_NAMES.values()
MIN_SECTOR_EXPOSURE = -0.0001
MAX_SECTOR_EXPOSURE = 0.0001

def optimal_portfolio_sector_neutral():
    """
    Calculate the optimal portfolio if we're constrained by exposure, concentration,
    and sector neutrality.
    """
    objective = opt.MaximizeAlpha(expected_returns)
    constraints = [
        opt.MaxGrossExposure(1.0),
        opt.PositionConcentration.with_equal_bounds(
            min=MIN_POSITION_WEIGHT,
            max=MAX_POSITION_WEIGHT,
        ),
        opt.NetPartitionExposure(
            labels=sector_names,
            min_weights={sector: MIN_SECTOR_EXPOSURE for sector in ALL_SECTOR_NAMES},
            max_weights={sector: MAX_SECTOR_EXPOSURE for sector in ALL_SECTOR_NAMES},
        )
    ]
    return opt.calculate_optimal_portfolio(objective, constraints, empty_portfolio)
```

```python
optimal_sector_neutral_pf = optimal_portfolio_sector_neutral()
plot = draw_asset_barplot(
    optimal_sector_neutral_pf,
    'Optimal Portfolio (Sector Neutral)',
    {'ylim': (-MAX_POSITION_WEIGHT - 0.1, MAX_POSITION_WEIGHT + 0.1)},
)
plot.axhline(MIN_POSITION_WEIGHT, color='r', linestyle='dashed')
plot.axhline(MAX_POSITION_WEIGHT, color='r', linestyle='dashed');
```

```python
plot = plot_sector_exposure(optimal_sector_neutral_pf, sector_names);
plot.axhline(MIN_SECTOR_EXPOSURE, color='r', linestyle='dashed')
plot.axhline(MAX_SECTOR_EXPOSURE, color='r', linestyle='dashed');
```

With our partition exposure constraint in place, the optimizer chooses to hedge our large `TWTR` position with shorts in `AAPL` in `MSFT`. The optimizer also reigns in the large long in `MCD` from 30% to 15%, because our 15% short exposure constraint prevents us from getting a equal-sized hedge in `QSR`, which is the only other `CONSUMER_CYCLICAL` stock in our universe. That leaves 10% of our capital unallocated, so the optimizer divides it evenly into a pair on `BP` and `XOM`.

### Constraining Risk Factor Exposures

So far we've looked at constraining our portfolio by restricting how much value we allow the optimizer to allocate to groupings of assets at various levels of granularity:

- We used `MaxGrossExposure` to limit the amount of value the optimizer allocated to the largest possible group: the group of **all assets**.
- On the other end of the spectrum, we used `PositionConcentration` to limit how much value the optimizer allocated to the smallest possible groups: the groups containing individual assets.
- Between these two extremes, we used `NetGroupExposure` to control how much value the optimizer allocated to any single sector.

While these are all reasonable applications of the Optimize API, sometimes we want control our portfolio in ways that are more sophisticated than just controlling the value allocated to particular groups of assets.

A different technique for controlling risk is to associate each asset in our portfolio with a set of **risk factors** that describe how an asset's price has moved historically with respect to some common source of risk. For each risk factor, we can then set upper and lower bounds on the **factor-weighted** exposure of our portfolio.

The most commonly-used risk factor is market beta, which in the simplest case is calculated as the slope of a linear regression between an asset's returns and the returns of "the market". Other common risk factors are "size" (measured as a function of market cap, for example) and "value".

```python
from quantopian.pipeline.factors import RollingLinearRegressionOfReturns
from quantopian.pipeline.data import Fundamentals

def get_risk_factors(stocks, date):
    # Measure beta as the slope of a linear regression of 60 days of daily close-to-close returns.
    # Each asset's returns are regressed against the returns of SPY.
    empirical_regression_coeffs = RollingLinearRegressionOfReturns(
        target=symbols('SPY'),
        returns_length=2,
        regression_length=60,
        mask=StaticAssets(stocks),
    )
    # RollingLinearRegressionOfReturns calculates a bunch of coefficients,
    # but we only care about beta for this example.
    empirical_beta = empirical_regression_coeffs.beta

    # Measure size by taking the z-score of the log of market cap.
    # Taking the log of market cap is common because company sizes are
    # distributed exponentially.
    log_mcap_zscore = Fundamentals.market_cap.latest.log().zscore()

    # Measure value using book value yield.
    book_value_yield = Fundamentals.book_value_yield.latest.zscore()

    factor_terms = {
        'market': empirical_beta,
        'size': log_mcap_zscore,
        'value': book_value_yield,
    }

    result = run_pipeline(
        Pipeline(factor_terms, screen=StaticAssets(stocks)),
        start_date=date,
        end_date=date
    )
    # Drop dates from the index of the returned frame.
    return result.reset_index(level=0, drop=True)
```

```python
risk_factors = get_risk_factors(universe, '2016-10-07')
risk_factors.head()
```

<div>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>market</th>
      <th>size</th>
      <th>value</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>Equity(24 [AAPL])</th>
      <td>0.881295</td>
      <td>3.049261</td>
      <td>-0.035366</td>
    </tr>
    <tr>
      <th>Equity(4707 [MCD])</th>
      <td>0.566868</td>
      <td>2.210673</td>
      <td>-0.062235</td>
    </tr>
    <tr>
      <th>Equity(5061 [MSFT])</th>
      <td>1.088901</td>
      <td>2.908003</td>
      <td>-0.041585</td>
    </tr>
    <tr>
      <th>Equity(8347 [XOM])</th>
      <td>0.933150</td>
      <td>2.807673</td>
      <td>0.000507</td>
    </tr>
    <tr>
      <th>Equity(19675 [BP])</th>
      <td>1.039710</td>
      <td>2.277613</td>
      <td>0.047715</td>
    </tr>
  </tbody>
</table>
</div>

Lets look at how our how heavily our previous sector-neutral portfolio is exposed to these risk factors:

```python
def plot_factor_exposures(pf, factor_loadings):
    return draw_barplot(
        pf.dot(factor_loadings),
        'Portfolio Factor Exposures',
    );
plot_factor_exposures(optimal_sector_neutral_pf, risk_factors);
```

Our sector neutral portfolio appears to have a strong negative exposure to our `market` and `size` factors. We can force the optimizer to control for these effects using a `FactorExposure` constraint.

```python
print opt.FactorExposure.__doc__
```

        Constraint requiring bounded net exposure to a set of risk factors.

        Factor loadings are specified as a DataFrame of floats whose columns are
        factor labels and whose index contains Assets. Minimum and maximum factor
        exposures are specified as maps from factor label to min/max net exposure.

        For each column in the loadings frame, we constrain::

            (new_weights * loadings[column]).sum() >= min_exposure[column]
            (new_weights * loadings[column]).sum() <= max_exposure[column]

        Parameters
        ----------
        loadings : pd.DataFrame
             An (assets x labels) frame of weights for each (asset, factor) pair.
        min_exposures : dict or pd.Series
             Minimum net exposure values for each factor.
        max_exposures : dict or pd.Series
             Maximum net exposure values for each factor.

```python
MIN_FACTOR_EXPOSURE = -0.01
MAX_FACTOR_EXPOSURE = 0.01

def optimal_portfolio_factor_neutral():
    """
    Calculate the optimal portfolio if we're constrained by exposure, concentration,
    sector, and market/size/value exposures.
    """
    objective = opt.MaximizeAlpha(expected_returns)
    constraints = [
        opt.MaxGrossExposure(1.0),
        opt.PositionConcentration.with_equal_bounds(
            min=MIN_POSITION_WEIGHT,
            max=MAX_POSITION_WEIGHT,
        ),
        opt.NetPartitionExposure(
            labels=sector_names,
            min_weights={sector: MIN_SECTOR_EXPOSURE for sector in ALL_SECTOR_NAMES},
            max_weights={sector: MAX_SECTOR_EXPOSURE for sector in ALL_SECTOR_NAMES},
        ),
        opt.FactorExposure(
            loadings=risk_factors,
            min_exposures={'market': MIN_FACTOR_EXPOSURE,
                           'size': MIN_FACTOR_EXPOSURE,
                           'value': MIN_FACTOR_EXPOSURE},
            max_exposures={'market': MAX_FACTOR_EXPOSURE,
                           'size': MAX_FACTOR_EXPOSURE,
                           'value': MAX_FACTOR_EXPOSURE}
        )
    ]
    return opt.calculate_optimal_portfolio(objective, constraints, empty_portfolio)
```

```python
optimal_factor_neutral_pf = optimal_portfolio_factor_neutral()
plot = draw_asset_barplot(
    optimal_factor_neutral_pf,
    'Optimal Portfolio (Factor Neutral)',
    {'ylim': (-MAX_POSITION_WEIGHT - 0.1, MAX_POSITION_WEIGHT + 0.1)},
)
```

```python
plot = plot_factor_exposures(optimal_factor_neutral_pf, risk_factors);
plot.axhline(MIN_FACTOR_EXPOSURE, color='r', linestyle='dashed')
plot.axhline(MAX_FACTOR_EXPOSURE, color='r', linestyle='dashed');
```

### Example: Minimize Distance from Target Portfolio with Turnover Constraints

Many algorithms written on Quantopian today perform stock selection via a process other than explicit returns/alpha-modeling. Pair-trading algorithms, for example, work by identifying pairs of [cointegrated](https://www.quantopian.com/posts/quantopian-lecture-series-integration-cointegration-and-stationarity) stocks and entering into a long position in one half of the pair and a short position in the other half of the pair.

For these existing algorithms (or for new algorithms written in a style other than alpha-modeling), it may not be natural to express portfolio construction in terms of an explicit alpha-maximization problem.
Nevertheless, it may still be useful to frame the portfolio construction process as an optimization problem using different objectives and constraints.

Suppose we have an algorithm that trades in four pairs and periodically decides which of the pairs it should allocate capital toward. Via its own internal logic, the algorithm arrives at a "target portfolio" that expresses its ideal holdings. The algorithm wants to trade from its current holdings into its ideal holdings, but it doesn't turn over its entire portfolio at once because doing so may incur excessive transaction costs.  The algorithm also wants to ensure that it maintains a net exposure of 0 in all of its pairs.

We can frame this as an optimization problem as follows:

Minimize the distance between the new portfolio and the ideal portfolio, subject to the constraints that:

1. The long components of our pairs have a weights $\geq$ 0.
2. The short components of our pairs have weights $\leq$ 0.
3. The sum of the weights in each of our pairs is 0.
4. The sum of the absolute values of the changes in weight across all stocks is less than our max turnover.

In fancy math language, we can say the following:

Given a vector $W_{current}$ of current portfolio weights, a vector $W_{ideal}$ of ideal portfolio weights, a maximum turnover rate $T_{max}$, and a list of pairs of long/short stocks $P = [(l_1, s_1), (l_2, s_2), \dots, (l_n, s_n)]$, choose a vector $w$ of weights that solves the following system:

\begin{align}
\text{minimize}&& \\
&&||w - W_{ideal}||_2 \\
\text{subject to}&& \\
&&w_l &\geq 0&& l \in [l_1, l_2, \dots, l_n] \\
&&w_s &\leq 0&& s \in [s_1, s_2, \dots, s_n] \\
&&w_l + w_s &= 0&& (l, s) \in P \\
&&||w - W_{current}||_1 &\leq T_{max}&& \\
\end{align}

(In the above, $||v||_1$ is the $L^1$ norm of $v$, or the ["Taxicab Norm"](https://en.wikipedia.org/wiki/Norm_%28mathematics%29#Taxicab_norm_or_Manhattan_norm), and $||v||_2$ is the $L^2$ norm of $v$, i.e, the standard [Euclidean Norm](https://en.wikipedia.org/wiki/Norm_%28mathematics%29#Euclidean_norm).)

While the mathematical definition of the problem here is a bit hairy, we can express the problem succintly using the `TargetWeights` objective and the `Pair` and `MaxTurnover` constraints.

```python
from itertools import chain

pairs = [
    symbols(['AAPL', 'MSFT']),
    symbols(['GM', 'F']),
    symbols(['MCD', 'QSR']),
    symbols(['BP', 'XOM']),
]

# Flatten the list of pairs into a single list for use as an index.
all_pair_stocks = list(chain.from_iterable(pairs))

# Initially allocate half of our capital in the AAPL/MSFT pair and half in the GM/F pair.
initial_pairs_portfolio = pd.Series({
    pairs[0][0]:  0.25,  # AAPL
    pairs[0][1]: -0.25,  # MSFT
    pairs[1][0]:  0.25,  # GM
    pairs[1][1]: -0.25,  # F
}).reindex(
    all_pair_stocks,
    fill_value=0.0,
)

draw_asset_barplot(initial_pairs_portfolio, 'Initial Pairs Portfolio');
```

Suppose that after some period of time, our algorithm decides that it wants to exit the `AAPL/MSFT` pair and enter the `MCD/QSR` and `BP/XOM` pairs.  It might calculate a new target portfolio that's equally-weighted between the three new pairs.

```python
target_pairs_portfolio = pd.Series({
    pairs[0][0]:  0.16666,
    pairs[0][1]: -0.16666,
    pairs[2][0]:  0.16666,
    pairs[2][1]: -0.16666,
    pairs[3][0]:  0.16666,
    pairs[3][1]: -0.16666,
}).reindex(
    all_pair_stocks,
    fill_value=0.0,
)

draw_asset_barplot(target_pairs_portfolio, 'Target Pairs Portfolio');
```

If we naively run `calculate_target_weights` using an unconstrained `TargetWeights` objective, we'll get back our target.  But if we supply turnover and pairs constraints, we'll move our capital incrementally out of the current pairs and into the new ones:

```python
print opt.TargetWeights.__doc__
```

        Objective that minimizes the distance from an already-computed portfolio.

        Parameters
        ----------
        weights : pd.Series[Asset -> float] or dict[Asset -> float]
            Map from asset to target percentage of holdings.

        Notes
        -----
        A target value of 1.0 indicates that 100% of the portfolio's current net
        liquidation value should be held in a long position in the corresponding
        asset.

        A target value of -1.0 indicates that -100% of the portfolio's current net
        liquidation value should be held in a short position in the corresponding
        asset.

        Assets with target values of exactly 0.0 are ignored unless an algorithm
        has an existing position in the given asset.

        If an algorithm has an existing position in an asset and no target weight
        is provided, the target weight is assumed to be zero.

```python
opt.TargetWeights(target_pairs_portfolio).math
```

\begin{align*}
&\underset{\boldsymbol{w}}{\text{minimize}} && ||\boldsymbol{w} - \boldsymbol{t}||_2 \\
&\text{where} &&\boldsymbol{t} = [0.167, -0.167, \ldots, -0.167]
\end{align*}

```python
print opt.Pair.__doc__
```

        A constraint representing a pair of inverse-weighted stocks.

        Parameters
        ----------
        long : Asset
            The asset to long.
        short : Asset
            The asset to short.
        hedge_ratio : float, optional
            The ratio between the respective absolute values of the long and short
            weights. Required to be greater than 0. Default is 1.0, signifying
            equal weighting.
        tolerance : float, optional
            The amount by which the hedge ratio of the calculated weights is
            allowed to differ from the given hedge ratio, in either direction.
            Required to be greater than or equal to 0. Default is 0.0.

```python
print opt.MaxTurnover.__doc__
```

        A constraint enforcing a maximum turnover for the optimal portfolio.

        Turnover is computed as the sum of the magnitude of the difference in
        weights for each asset in the portfolio, i.e., as::

            sum(abs(new_weights - old_weights))

        Parameters
        ----------
        max : float
            The maximum allowable turnover across the whole portfolio.

```python
def migrate_pairs_constrain_turnover(current_pf, target_pf, max_turnover):
    """Calculate a new target portfolio while respecting turnover and pair constraints."""
    objective = opt.TargetWeights(target_pf)

    constraints = [opt.Pair(long, short) for long, short in pairs]
    constraints.append(opt.MaxTurnover(max_turnover))

    return opt.calculate_optimal_portfolio(
        objective=objective,
        constraints=constraints,
        current_portfolio=current_pf,
    )
```

```python
# Try changing with the `MAX_TURNOVER` parameter here to see how it affects
# the resulting portfolio!
MAX_TURNOVER = 0.75

migrated_pairs_portfolio = migrate_pairs_constrain_turnover(
    current_pf=initial_pairs_portfolio,
    target_pf=target_pairs_portfolio,
    max_turnover=MAX_TURNOVER,
)
```

```python
before_and_after_frame = pd.DataFrame(
    {
        'Initial Weight': initial_pairs_portfolio,
        'Migrated Weight': migrated_pairs_portfolio,
        'Target Weight': target_pairs_portfolio,
    },
    # Explicitly set a column order so that the plot below is
    # created in the order we want.
    columns=['Initial Weight', 'Migrated Weight', 'Target Weight']
)

ax = draw_asset_barplot(
    before_and_after_frame,
    # The fancy brace syntax here is a format specification.
    # See https://docs.python.org/2/library/string.html#formatspec for more info.
    'Optimal Portfolio: {0:.0f}% Max Turnover '.format(MAX_TURNOVER * 100),
);
```

Since the optimizer can't reach its target portfolio without violating the `MaxTurnover` constraint, it attempts to equalize the size of the difference between each stock's output weight and it's target weight.

```python
print "Total Turnover: %f\n" % (migrated_pairs_portfolio - initial_pairs_portfolio).abs().sum()
print "Absolute Difference from Target Weight:\n"
print (migrated_pairs_portfolio - target_pairs_portfolio).abs()
```

    Total Turnover: 0.750000

    Absolute Difference from Target Weight:

    Equity(24 [AAPL])      0.072915
    Equity(5061 [MSFT])    0.072915
    Equity(40430 [GM])     0.072915
    Equity(2673 [F])       0.072915
    Equity(4707 [MCD])     0.072915
    Equity(48215 [QSR])    0.072915
    Equity(19675 [BP])     0.072915
    Equity(8347 [XOM])     0.072915
    dtype: float64

# Debugging and Diagnostics

One issue users may encounter when using the Optimize API is that it’s possible to accidentally ask the optimizer to solve problems for which no solution exists. This can happen, for example, if we provide the optimizer with two mutually-exclusive constraints.

If the optimizer determines that no portfolio can satisfy all the constraints we've provided, it raises an `InfeasibleConstraints` error. Debugging `InfeasibleConstraints` can be tricky: since different portfolios might violate different constraints, there isn't necessarily a single constraint we can point to as being the source of infeasibility.

One observation that can help us when we encounter infeasible constraints is to note that there are certain "special" portfolios that we might have expected to pass our constraints. We can often figure out the problem with our optimization by examining the constraints that were violated by these special portfolios.

When an `InfeasibleConstraints` error is raised, the optimizer examines a small number of these portfolios and produces an error message detailing the constraints that were violated by each tested portfolio. The portfolios current exampled by the optimizer for diagnostics are:

1. The current portfolio.
2. An empty portfolio.
3. The target portfolio (only applies when using `TargetWeights`).

### Example:

Suppose we have a portfolio that currently has long and short positions worth 10% of our portfolio value in `AAPL` and `MSFT`, respectively.

```python
AAPL, MSFT = symbols(['AAPL', 'MSFT'])
current_weights = pd.Series({AAPL: 0.1, MSFT: -0.1})
```

We use `TargetWeights` as our objective to try to close the `MSFT` position.

```python
objective = opt.TargetWeights({AAPL: 0.1, MSFT: 0.0})
```

We also define three mutually-exclusive constraints:

- Our portfolio must be dollar-neutral (i.e., it must have the same amount invested long and short). Our target portfolio violates this constraint.
- Fixed weight in `AAPL`. This simulates what might happen if `AAPL` were subject to a trading halt during the day. An empty portfolio would violate this constraint.
- Cannot short `MSFT`. The current portfolio violates this constraint.

```python
impossible_constraints = [
    opt.DollarNeutral(tolerance=0.01),
    opt.FixedWeight(AAPL, 0.1),
    opt.LongOnly(MSFT),
]
```

```python
opt.calculate_optimal_portfolio(objective, impossible_constraints)
```

Occasionally it can useful to interrogate the results of an optimize error programatically.  The `run_optimization` entrypoint makes this possible.

`run_optimization` performs the same calculation as `calculate_optimal_portfolio`, but it returns an `OptimizationResult` which contains additional information besides the computed portfolio.

```python
# Run an optimization with no constraints to get a successful result.
success_result = opt.run_optimization(objective, [])
print "Success:", success_result

# Re-run the impossible optimization to see a failed result.
fail_result = opt.run_optimization(objective, impossible_constraints)
print "Failure:", fail_result
```

    Success: OptimizationResult(success=True, status='optimal')
    Failure: OptimizationResult(success=False, status='infeasible')

When an optimization succeeds, the result's `success` attribute is set to `True`, and the new optimal portfolio is stored as `result.new_weights`.  If the result failed, `success` is `False`, and `result.new_weights` is `None`.

```python
print "Success:", success_result.success
print "New Portfolio:", success_result.new_weights
```

    Success: True
    New Portfolio: Equity(24 [AAPL])    0.1
    dtype: float64

```python
print "Success:", fail_result.success
print "New Portfolio:", fail_result.new_weights
```

    Success: False
    New Portfolio: None

The result object also has a `print_diagnostics` method, which can be used to print the same debugging information displayed by `InfeasibleConstraints`. This can be useful for debugging cases where an optimization is **succeeding**, but not producing the results expected.

```python
success_result.print_diagnostics()
```

    Target Portfolio (as provided to TargetWeights):
      All Constraints Satisfied

```python
fail_result.print_diagnostics()
```

    Target Portfolio (as provided to TargetWeights):

       Would violate DollarNeutral() because:
          Net exposure (0.1) would be greater than max net exposure (0.01).

    Current Portfolio (at the time of the optimization):

       Would violate FixedWeight(Equity(24 [AAPL])) because:
          New weight (0.0) would not equal required weight (0.1).

    Empty Portfolio (no positions):

       Would violate FixedWeight(Equity(24 [AAPL])) because:
          New weight (0.0) would not equal required weight (0.1).

# Review

- The **Optimize API** is a suite of tools for solving portfolio optimization problems on Quantopian. It provides a high-level, declarative, finance-oriented interface to Python's excellent ecosystem of numerical optimization libraries.

- Users interact with the Optimize API by passing an `Objective` and a list of `Constraint`s to one of three optimize entrypoints:
  - The primary entrypoint in research is `calculate_optimal_portfolio`.
  - The primary entrypoint in algorithms is `order_optimal_portfolio`.
  - `run_optimization` is a lower-level entrypoint primarily useful for debugging.

- When an optimization fails because of `InfeasibleConstraints`, the optimizer provides information about the failed optimization by testing interesting portfolios for constraint information.

Quantopian Contest Resources/Optimize API.ipynb


# Model evaluation

```python
import numpy as np
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
```

We will start up by creating a dataframe from the data imported from the given website. Column names were extracted from the file with a describtion of the dataset. We can give a look at our data by displaying first five rows with `head()` method.

```python
url = 'https://archive.ics.uci.edu/ml/machine-learning-databases/wine/wine.data'
df = pd.read_csv(url, delimiter=',', names=[
    'Class',
    'Alcohol',
    'Malic acid',
    'Ash',
    'Alcalanity of ash',
    'Magnesium',
    'Total phenols',
    'Flavanoids',
    'Nonflavanoid phenols',
    'Proanthocyanins',
    'Color intensity',
    'Hue',
    'OD280/OD315 of diluted wines',
    'Proline'])

df.head()
```

      Class
      Alcohol
      Malic acid
      Ash
      Alcalanity of ash
      Magnesium
      Total phenols
      Flavanoids
      Nonflavanoid phenols
      Proanthocyanins
      Color intensity
      Hue
      OD280/OD315 of diluted wines
      Proline

      0
      1
      14.23
      1.71
      2.43
      15.6
      127
      2.80
      3.06
      0.28
      2.29
      5.64
      1.04
      3.92
      1065

      1
      1
      13.20
      1.78
      2.14
      11.2
      100
      2.65
      2.76
      0.26
      1.28
      4.38
      1.05
      3.40
      1050

      2
      1
      13.16
      2.36
      2.67
      18.6
      101
      2.80
      3.24
      0.30
      2.81
      5.68
      1.03
      3.17
      1185

      3
      1
      14.37
      1.95
      2.50
      16.8
      113
      3.85
      3.49
      0.24
      2.18
      7.80
      0.86
      3.45
      1480

      4
      1
      13.24
      2.59
      2.87
      21.0
      118
      2.80
      2.69
      0.39
      1.82
      4.32
      1.04
      2.93
      735

Next we can define predictor and target variables by selecting appropriate columns.

```python
X = df.iloc[:, 1:].values
y = df.iloc[:, 0].values
```

To simplify the task and modularize the code we will define three functions. The first function fits a classifying model (taken as an argument) to the training dataset and predicts the target variable based on test features.

```python
def predict_clf(classifier, X_train, X_test, y_train):
    classifier.fit(X_train, y_train)
    y_pred = classifier.predict(X_test)
    return y_pred
```

The second function prints classification metrics comparing predicted output with the test target variable. We chose accuracy, precision and recall to check a performance of the classifier. Accuracy is a most common evaluation metric – it's just the number of correct predictions made as a ratio of all predictions made:

\begin{equation}
\text{Accuracy} = \frac{TP + TN}{TP + FP + FN + TN}.
\end{equation}

Precision measures the portion of positive identifications in a classification set that were actually correct:

\begin{equation}
\text{Precision} = \frac{TP}{TP + FP}.
\end{equation}

On the other hand recall represents the proportion of actual positives that were identified correctly:

\begin{equation}
\text{Recall} = \frac{TP}{TP + FN},
\end{equation}

where $TP$ is the number of true positives, $TN$ – true negatives, $FP$ – false positives and $FN$ – false negatives.

```python
from sklearn.metrics import accuracy_score, precision_score, recall_score

def print_metrics(name, classifier, X_train, X_test, y_train, y_test):
    y_pred = predict_clf(classifier, X_train, X_test, y_train)

    accuracy = accuracy_score(y_test, y_pred)
    precision = precision_score(y_test, y_pred, average='macro')
    recall = recall_score(y_test, y_pred, average='macro')

    print('Classifier: {}'.format(name))
    print('Accuracy: {}%'.format(str(round(accuracy*100, 2))))
    print('Precision: {}%'.format(str(round(precision*100, 2))))
    print('Recall: {}%'.format(str(round(recall*100, 2))))
    print('------------------')
```

Finally the third function prints and compares the metrics of all the classifiers in a list taken as an argument.

```python
def compare_models(names, classifiers, X_train, X_test, y_train, y_test):
    for name, clf in zip(names, classifiers):
        print_metrics(name, clf, X_train, X_test, y_train, y_test)
```

The last step before evaluation of the models is creating lists with names of the classifiers and and the actual models.

```python
from sklearn.discriminant_analysis import LinearDiscriminantAnalysis, QuadraticDiscriminantAnalysis
from sklearn.naive_bayes import GaussianNB

names = ['LDA', 'QDA', 'NB']
classifiers = [LinearDiscriminantAnalysis(), QuadraticDiscriminantAnalysis(), GaussianNB()]
```

## All the features – training dataset as test dataset

```python
compare_models(names, classifiers, X, X, y, y)
```

    Classifier: LDA
    Accuracy: 100.0%
    Precision: 100.0%
    Recall: 100.0%
    ------------------
    Classifier: QDA
    Accuracy: 99.44%
    Precision: 99.44%
    Recall: 99.53%
    ------------------
    Classifier: NB
    Accuracy: 98.88%
    Precision: 98.85%
    Recall: 98.97%
    ------------------

Although the classifiers have almost perfect accuracy it's not a desirable method of model evaluation. Taking every feature into consideration and testing the classifier with training dataset lead to overfitting of our model. It means that the model doesn't generalize well and would have much lower accuracy when testing with independent dataset.

## First 2/5/10 features – training dataset as test dataset

```python
X_two_features = df.iloc[:, 1:3].values
compare_models(names, classifiers, X_two_features, X_two_features, y, y)
```

    Classifier: LDA
    Accuracy: 80.9%
    Precision: 79.87%
    Recall: 79.67%
    ------------------
    Classifier: QDA
    Accuracy: 81.46%
    Precision: 80.43%
    Recall: 80.01%
    ------------------
    Classifier: NB
    Accuracy: 80.9%
    Precision: 79.76%
    Recall: 79.45%
    ------------------

```python
X_five_features = df.iloc[:, 1:6]
compare_models(names, classifiers, X_five_features, X_five_features, y, y)
```

    Classifier: LDA
    Accuracy: 87.64%
    Precision: 87.13%
    Recall: 86.72%
    ------------------
    Classifier: QDA
    Accuracy: 88.76%
    Precision: 88.31%
    Recall: 88.24%
    ------------------
    Classifier: NB
    Accuracy: 85.39%
    Precision: 84.96%
    Recall: 84.88%
    ------------------

```python
X_ten_features = df.iloc[:, 1:11]
compare_models(names, classifiers, X_ten_features, X_ten_features, y, y)
```

    Classifier: LDA
    Accuracy: 98.88%
    Precision: 98.97%
    Recall: 98.84%
    ------------------
    Classifier: QDA
    Accuracy: 99.44%
    Precision: 99.44%
    Recall: 99.53%
    ------------------
    Classifier: NB
    Accuracy: 96.07%
    Precision: 96.3%
    Recall: 96.2%
    ------------------

We can see the models have worse accuracy with decreasing number of features. However they generalize much better. Optimally we should have applied feature selection to choose the features that are the most impactful or correlate with the target variable the most. To do so we could have used regularization techniques or XGBoost algorithm.

## First 2 features – splitting the data into training, validation and test datasets

```python
from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X_two_features, y, test_size=0.25)
X_train, X_val, y_train, y_val = train_test_split(X_train, y_train, test_size=0.33)

compare_models(names, classifiers, X_train, X_val, y_train, y_val)
```

    Classifier: LDA
    Accuracy: 77.27%
    Precision: 79.45%
    Recall: 76.49%
    ------------------
    Classifier: QDA
    Accuracy: 79.55%
    Precision: 79.44%
    Recall: 78.87%
    ------------------
    Classifier: NB
    Accuracy: 79.55%
    Precision: 79.44%
    Recall: 78.87%
    ------------------

Splitting the data into training, validation and test datasets provides enhanced generalization because the validation set can be employed to tuning of model's hyperparameters without overfitting. However this method has one major drawback – performance of the model varies greatly with a way the data is split. Changing which observations happens to be in test dataset can significantly change test accuracy. That's where cross validation comes into play.

## First 2 features – cross validation

Cross validation is a techique widely used to test an effectiveness of the model when there is a limited amount of data. Basically it splits the dataset into $k$ folds. One of the folds becomes test dataset and the remaining ones are used for training. The process is being repeated $k$ times with a change of partition employed for testing every time. At the end we can use average of the metrics to check performance of the classifier.

```python
from sklearn.model_selection import KFold, cross_validate

lda = classifiers[0]
kfold = KFold(n_splits=10, shuffle=True)
scoring = {'accuracy': 'accuracy',
           'recall': 'recall_macro',
           'precision': 'precision_macro'}
cross_val_scores = cross_validate(lda, X_two_features, y, cv=kfold, scoring=scoring)
cross_val_scores = {key: score.mean() for key, score in cross_val_scores.items()}

print('Accuracy: {}%'.format(str(round(cross_val_scores['test_accuracy']*100, 2))))
print('Precision: {}%'.format(str(round(cross_val_scores['test_precision']*100, 2))))
print('Recall: {}%'.format(str(round(cross_val_scores['test_recall']*100, 2))))
```

    Accuracy: 78.07%
    Precision: 80.12%
    Recall: 78.89%

The metrics of our model have been improved slightly in comparison with splitting technique. We could have increased the accuracy a little if we had applied nonlinear model instead of LDA. For example we can see QDA outperforms LDA in almost every case.


# Characterization of One-Dimensional Systems by Differential Equations

*This Jupyter notebook is part of a [collection of notebooks](../index.ipynb) in the masters course Selected Topics in Audio Signal Processing, Communications Engineering, Universität Rostock. Please direct questions and suggestions to [Sascha.Spors@uni-rostock.de](mailto:Sascha.Spors@uni-rostock.de).*

## Analysis of a Passive Electrical Network

[Electrical networks](https://en.wikipedia.org/wiki/Electrical_network) composed of linear passive elements, like resistors, capacitors and inductors can be described by linear ordinary differential equations (ODEs) with constant coefficients. Hence, in view of the theory of signals and systems they can be interpreted as a linear time-invariant (LTI) system. The different ways to characterize the properties of an LTI system introduced before are illustrated at the example of a second-order analog [low-pass filter](https://en.wikipedia.org/wiki/Low-pass_filter).

It is assumed that no energy is stored in the capacitor and inductor for $t<0$. It is furthermore assumed that $x(t) = 0$ for $t<0$. Hence $y(t) = 0$ and $\frac{d y(t)}{dt} = 0$ for $t<0$. For illustration, the normalized values $L = .5$, $R = 1$, $C = .4$ are used for the elements of the electrical network.

### Differential Equation

The differential equation describing the input/output relation of the electrical network is derived by applying [Kirchhoff's circuit laws](https://en.wikipedia.org/wiki/Kirchhoff's_circuit_laws) to the network. This results in the following ODE

\begin{equation}
C L \frac{d^2 y(t)}{dt^2} + C R \frac{d y(t)}{dt} + y(t) = x(t)
\end{equation}

This ODE is defined in `SymPy`

```python
import sympy as sym
sym.init_printing()

t, L, R, C = sym.symbols('t L R C', real=True)
x = sym.Function('x')(t)
y = sym.Function('y')(t)

ode = sym.Eq(L*C*y.diff(t, 2) + R*C*y.diff(t) + y, x)
ode
```

The normalized values of the network elements are stored in a dictionary for substitution

```python
RLC = {R: 1, L: sym.Rational('.5'), C: sym.Rational('.4')}
RLC
```

### Impulse Response

The passive electrical network and the ODE describing its input/output relation can be interpreted as an LTI system. Hence, the system can be characterized by its [impulse response](impulse_response.ipynb) $h(t)$ which is defined as the output of the system for a Dirac impulse $x(t) = \delta(t)$ at the input. For the given system, the impulse response is calculated by explicit solution of the ODE

```python
solution_h = sym.dsolve(ode.subs(x, sym.DiracDelta(t)).subs(y, sym.Function('h')(t)))
solution_h
```

The integration constants $C_1$ and $C_2$ have to be determined from the initial conditions $y(t) = 0$ and $\frac{d y(t)}{dt} = 0$ for $t<0$.

```python
integration_constants = sym.solve( (solution_h.rhs.limit(t, 0, '-'), solution_h.rhs.diff(t).limit(t, 0, '-')), ['C1', 'C2'] )
integration_constants
```

Substitution of the values for the integration constants $C_1$ and $C_2$ into the result from above yields the impulse response of the low-pass

```python
h = solution_h.subs(integration_constants).rhs
h
```

The impulse response is plotted for the values of $R$, $L$ and $C$ given above

```python
sym.plot(h.subs(RLC), (t,0,10), ylabel=r'h(t)');
```

### Step Response

The [step response](step_response.ipynb) is derived by integrating over the impulse response $h(t)$. For ease of illustration this is performed for the specific values of the elements given above

```python
tau = sym.symbols('tau')

he = sym.integrate(h.subs(RLC).subs(t,tau).subs(sym.Heaviside(tau),1), (tau,0,t))
he
```

Let's plot the step response

```python
sym.plot(he, (t,0,10), ylabel=r'$h_\epsilon(t)$');
```

### Transfer Function

For an exponential input signal $x(t) = e^{s t}$, the [transfer function](eigenfunctions.ipynb#Transfer-Function) $H(s)$ represents the weight of the exponential output signal $y(t) = H(s) \cdot e^{s t}$. The transfer function is derived by introducing $x(t)$ and $y(t)$ into the ODE and solving for $H(s)$

```python
s = sym.symbols('s')
H = sym.Function('H')(s)

H, = sym.solve(ode.subs(x, sym.exp(s*t)).subs(y, H*sym.exp(s*t)).doit(), H)
H
```

The transfer characteristics of an LTI system for harmonic exponential signals $e^{j \omega} = \cos(\omega t) + j \sin(\omega t)$ are of special interest in the analysis of electrical circuits. It can be derived from $H(s)$ by substituting the complex frequency $s$ with $s = j \omega$. The resulting transfer function $H(j \omega)$ provides the attenuation and phase of an harmonic input signal by the system.

```python
w = sym.symbols('omega')

Hjw = H.subs(s, sym.I * w)
Hjw
```

The magnitude of the transfer function $|H(j \omega)|$ is plotted for illustration

```python
sym.plot(abs(Hjw.subs(RLC)), (w, -10, 10), ylabel=r'$|H(j \omega)|$', xlabel=r'$\omega$');
```

It is obvious, that the low frequencies pass through the system and that high frequencies are attenuated. This motivates the term 'low-pass' for such systems.

As alternative to the solution of the ODE, the transfer function $H(s)$ is [computed from the impulse response](eigenfunctions.ipynb#Link-between-Transfer-Function-and-Impulse-Response) and plotted.

```python
H2 = sym.integrate(h.subs(RLC)*sym.exp(-s*t), (t,0,sym.oo), conds='none')
H2 = H2.simplify()
H2
```

```python
sym.plot(abs(H2.subs(s, sym.I*w)), (w, -10, 10), ylabel=r'$|H(j \omega)|$', xlabel=r'$\omega$');
```

**Copyright**

This notebook is provided as [Open Educational Resources](https://en.wikipedia.org/wiki/Open_educational_resources). Feel free to use the notebook for your own purposes. The text/images/data are licensed under [Creative Commons Attribution 4.0](https://creativecommons.org/licenses/by/4.0/), the code of the IPython examples under the [MIT license](https://opensource.org/licenses/MIT). Please attribute the work as follows: *Sascha Spors, Selected Topics in Audio Signal Processing - Supplementary Material*.


# Занятие 9
# Прикладная алгебра и численные методы
## Численное интегрирование

```
import numpy as np
import sympy
import scipy.integrate
import pandas as pd
from google.colab import files
```

```
#!python -m pip install scipy --upgrade
```

```
sympy.__version__, scipy.__version__, np.__version__
```

    ('1.7.1', '1.6.1', '1.19.5')

## Численное интегрирование

Определенный интеграл функции $f(x)$ есть предел интегральных сумм
при стремлении к нулю диаметра разбиения (если этот предел
существует и конечен):
\begin{align}
&\int_a^b f(x)\,dx=\lim_{d\to0}\sum_{k=1}^n f(\xi_k)\Delta_k,
\quad a=x_0<x_1<\ldots<x_n=b,\\
&\xi_k\in[x_{k-1};x_k],\Delta_k=x_k-x_{k-1},\ d=\max_{1\le k\le n}
\Delta_k.
\end{align}

## Метод  трапеций

Приближенным значением интеграла $\int_a^b f(x)\,dx$ в методе
трапеций считается сумма площадей прямоугольных трапеций высоты
$h_k=x_k-x_{k-1}$ с основаниями длины $f(x_{k-1})$ и $f(x_k)$.

Площадь $S_k$ одной элементарной трапеции
\begin{align*}
S_k=\frac{f(x_{k-1})+f(x_k)}{2}h_k,
\end{align*}
сложив площади всех элементарных трапеций, получим
\begin{align*}
\int_a^b f(x)\,dx\approx\sum_{k=1}^n \frac{f(x_{k-1})+f(x_k)}{2}h_k.
\end{align*}

Пусть задана равномерная сетка $x_k=a+hk$, $k=0,\ldots,n$,
$h=(b-a)/n$, тогда
\begin{align*}
&\int_a^b f(x)\,dx\approx \frac{h }{2}\sum_{k=1}^n
(f(x_{k-1})+f(x_k))=h\Biggl(\frac{f(x_0)+ f(x_n)}{2}+\sum_{k=1}^{n-1}
f(x_k)\Biggr),
\end{align*}
погрешность этого приближения
\begin{align*}
R=-h^2\frac{b-a}{12}f''(\xi),\quad\xi\in[a;b].
\end{align*}
Если функция $f(x)$ имеет ограниченную на $[a;b]$ вторую
производную, то
\begin{align*}
|R|\le h^2M\frac{b-a}{12},\quad M=\max_{[a;b]}f''(x).
\end{align*}

## Метод Симпсона

Приближенным значением интеграла $\int_a^b f(x)\,dx$ в методе
Симпсона считается сумма площадей криволинейных трапеций ширины
$h_k=x_k-x_{k-1}$, ограниченных сверху параболой, проходящей через
точки $A_k(x_{k-1};f(x_{k-1}))$, $B_k(x_{k};f(x_k))$ и
$C_k(x_{k+1};f(x_{k+1}))$.

Обозначим $f_{-1}=f(-h)$,
$f_{0}=f(0)$,$f_{1}=f(h)$.
Вычислим вначале площадь одной элементарной криволинейной трапеции
при $x_{k-1}=-h$, $x_{k}=0$, $x_{k+1}=h$
\begin{align*}
S=\frac{h}{3}(f_{-1}+4f_0+f_1).
\end{align*}

Пусть задана равномерная сетка $x_k=a+hk$, $k=0,\ldots,2n$,
$h=(b-a)/2n$, тогда
\begin{align*}
&\int_a^b f(x)\,dx\approx \frac{h }{3}\sum_{k=1}^n
(f_{k-1}+4f_k+f_{k+1})= \frac{h }{3}\Biggl(f_0+f_{2n}+4\sum_{k=1}^n
f_{2k-1}+2\sum_{k=1}^{n-1} f_{2k}\Biggr),
\end{align*}
погрешность этого приближения
\begin{align*}
R=-h^4\frac{b-a}{180}f^{(4)}(\xi),\quad\xi\in[a;b].
\end{align*}
Если функция $f(x)$ имеет ограниченную на $[a;b]$  производную
четвертого порядка, то
\begin{align*}
|R|\le h^4M\frac{b-a}{180},\quad M=\max_{[a;b]}f^{(4)}(x).
\end{align*}

## Уточнение результата численного интегрирования.

В случае, если ничего не известно о производных функции $f(x)$,
можно уточнять значение интеграла $\int_a^b f(x)\,dx$, сравнивая
результаты численного интегрирования $I_h$ и $I_{h/2}$ (шаг
разбиения соответственно $h$ и $h/2$).

Пусть требуется вычислить  $\int_a^b f(x)\,dx$ с точностью $\varepsilon$,
тогда
\begin{align*}
\int_a^b f(x)\,dx\approx I_{h/2},\quad\mbox{ если }
|I_h-I_{h/2}|<\varepsilon,
\end{align*}
если же условие не выполняется, уменьшаем шаг разбиения и вычисляем
$I_{h/4}$. Процесс продолжаем до тех пор, пока не выполнится условие
\begin{align*}
|I_{h_k}-I_{h_{k+1}}|<\varepsilon,
\end{align*}
тогда считаем
\begin{align*}
\int_a^b f(x)\,dx\approx I_{h_{k+1}}.
\end{align*}

## Задание 1
Найти методом трапеций интеграл таблично заданной функции из файла f.xlsx

```
uploaded = files.upload()

for fn in uploaded.keys():
  print('User uploaded file "{name}" with length {length} bytes'.format(name=fn, length=len(uploaded[fn])))
```

```

```

## Задание 2
Найти интеграл задания 1 методом Симпсона.

```

```

## Задание 3
Вычислить приближенно интеграл
$$
\int_2^5\sin^3(2x - 1)\,dx
$$
используя scipy.integrate

```

```

## Задание 4
Вычислить аналитически интеграл
$$
\int_0^{\pi/3}\sin^5(2x)\,dx
$$
с помощью sympy, затем вычислить этот интеграл приближенно,
используя scipy.integrate (scipy.integrate.quad  и  метод Ромберга).

```

```

## Задание 5
Вычислить интеграл задания 4 с помощью метода Симпсона из scipy, использовать равномерную сетку.

```

```

## Задание 6
Написать функцию, которая возвращает определенный интеграл, вычисленный методом трапеций с заданной точностью. Аргументы функции - имя интегрируемой функции, пределы интегрирования и точность. Для достижения заданной точности измельчать сетку X.

```

```


## 1-1. 量子ビット
古典コンピュータ（量子コンピュータではない既存のコンピュータのこと）内部では情報は 0 と 1 の２つの状態で表現されている。例えば、スイッチのオン・オフの状態や、電荷がたまった状態とそうでない状態、電圧の高・低などでその２状態を表現している。一方、量子力学では異なる２つの状態の**重ね合わせ状態**というのが許されているので、量子の世界の情報の最小単位である**“量子”ビット**は $\alpha$と$\beta$という二つの複素数を用いた複素ベクトルを用いて

$$
\left(
\begin{array}{c}
\alpha
\\
\beta
\end{array}
\right)
$$

のようにその量子状態が記述される。

$\alpha$や$\beta$はどの程度の重みで0状態と1状態が重ね合わさっているかを表しており、**複素確率振幅**と呼ばれる。
$\alpha$や$\beta$が複素数になっているのは、量子の世界では0や1といった離散的な量も波の性質をもち干渉するためである。

古典ビットの0に対応する状態は

$$
\left(
\begin{array}{c}
1
\\
0
\end{array}
\right)
$$

1に対応する状態は

$$
\left(
\begin{array}{c}
0
\\
1
\end{array}
\right)
$$

となる。

列ベクトルを毎回書いているとスペースが無駄なので、**ディラックのブラケット表記**という簡略化した表記を導入する。これは列ベクトルである、という量子状態の型宣言のようなもので、この記号がついていると量子状態をあらわす複素ベクトルであることが一目でわかるようになっている。

$$
|0\rangle = \left(
\begin{array}{c}
1
\\
0
\end{array}
\right)
$$
$$
|1\rangle = \left(
\begin{array}{c}
0
\\
1
\end{array}
\right)
$$

この表記を用いると量子ビットは

$$
|\psi \rangle = \alpha |0\rangle + \beta |1\rangle
$$

と書かれる（スペースが省略できた！）。

### 複素確率振幅の意味

複素確率振幅はいったいどのような物理的実体に対応するだろうか。実は、量子力学では観測者（人間）は直接複素確率振幅にはアクセスすることができず、**測定という操作をした時に初めて0か1かが確率的にきまる**。測定結果の確率分布に影響するのが複素確率振幅である。測定結果が0になる確率$p_0$, 1になる確率$p_1$は複素確率振幅の絶対値の2乗で表される:

$$
p_0 = |\alpha |^2,
\\
p_1 = |\beta |^2
$$

確率の和が1になるように、規格化条件 $|\alpha |^2 + |\beta |^2 =1$を課す。

**測定を行うと、量子状態は測定結果に対応する状態に変化する**。具体的には、測定結果が0の場合は$|0\rangle$、1の場合は$|1\rangle$に変化する。この測定を、正規直交基底$|0\rangle$, $|1\rangle$での射影測定と呼ぶ。$|0\rangle$, $|1\rangle$以外の正規直交基底での射影測定や、より一般の測定もあるが、ここでは扱わない。

まとめると、

- 量子状態は、大きさが１に規格化された複素ベクトルによって記述される。
- 各成分の絶対値の２乗が、測定をしたときにその成分に対応する状態を得る確率である。
- 測定後の量子状態は、測定結果に応じて$|0\rangle$または$|1\rangle$となる。

したがって、

$$
|0\rangle = \left(
\begin{array}{c}
1
\\
0
\end{array}
\right), |1\rangle = \left(
\begin{array}{c}
0
\\
1
\end{array}
\right)
$$

は、確実に0や1が得られる古典的な状態に対応し、

$$
\frac{1}{\sqrt{2}}
(|0\rangle + |1\rangle) = \left(
\begin{array}{c}
\frac{1}{\sqrt{2}}
\\
\frac{1}{\sqrt{2}}
\end{array}
\right)
$$

は0と1が同じおもみで重ね合わさった状態であり、測定をすると0と1が確率1/2で完全にランダムに得られる。

複素確率振幅は複素数なので、

$$
\frac{1}{\sqrt{2}}(|0\rangle + i|1\rangle ) =
\left(
\begin{array}{c}
\frac{1}{\sqrt{2}}
\\
\frac{i}{\sqrt{2}}
\end{array}
\right)
$$

といった状態も許されている。より一般に、

$$
\frac{1}{\sqrt{2}} ( |0\rangle + e^{ i \phi } |1\rangle )
$$

なども許されている。この状態の場合、状態0に対する確率振幅 ($1/\sqrt{2}$) が正の実数であるのに対して、状態1に対する確率振幅 ($e^{i\phi}/\sqrt{2}$) は、複素平面上で$\phi$回転している。このような、重ね合わせ状態における確率振幅間の相対的な偏角のことを**位相**と呼び、量子力学全般において重要な役割を果たす。

(詳細は Nielsen-Chuang の `1.2 Quantum bits` を参照)

### SymPyを用いて量子ビットを表示してみる
SymPyでは、量子状態を扱うことができる。初期化された量子ビットを準備する場合は ` Qubit() ` 関数を用いる。

```python
from IPython.display import Image, display_png
from sympy import *
from sympy.physics.quantum import *
from sympy.physics.quantum.qubit import Qubit,QubitBra
init_printing() # ベクトルや行列を綺麗に表示するため
```

```python
# Google Colaboratory上でのみ実行してください
from IPython.display import HTML
def setup_mathjax():
    display(HTML('''

    '''))
get_ipython().events.register('pre_run_cell', setup_mathjax)
```

```python
psi = Qubit('0')
```

```python
psi #ブラケット表示
```

```python
represent(psi) #列ベクトル表示
```

SymPyでは文字をシンボルとして扱うことができるので、一般的な量子ビットも簡単に書ける。

```python
a, b = symbols('alpha, beta')  #a, bをシンボルとして、alpha, betaとして表示
ket0 = Qubit('0')
ket1 = Qubit('1')
psi = a * ket0 + b* ket1
psi # 状態をそのまま書くとケットで表示してくれる
```

```python
represent(psi)
```

もちろん具体的な数値を代入することもできる。

```python
psi.subs([([a,1/sqrt(2)]),([b,1/sqrt(2)])]) # alpha, betaに具体的な数字を代入
```


# 2 DOF System

The data that describes our system and its loading:

```python
k0, m0 = 1.0, 1.0 # ideally, dimensional units...
w20 = k0/m0
w0 = np.sqrt(w20)

k1, k2 = 2*k0, 3*k0
m1, m2 = 2*m0, 4*m0

M = np.array(((m1, 0), ( 0, m2)))
K = np.array(((k1+k2, -k2), (-k2, k2)))

p = np.array(( 0.0, 1.0)); w = 2.0*w0

print_mat(M, pre='\\boldsymbol{M}=m\\,', fmt='%d')
print_mat(K, pre='\\boldsymbol{K}=k\\,', fmt='%d')
print_mat(p[:,None], pre=r'\boldsymbol{p}(t) = p_0\,', fmt='%d',
          post='\\sin(%d\\omega_0t)'%w, mt='B')
```

$$\boldsymbol{M}=m\,\begin{bmatrix}2&0\\0&4\end{bmatrix}$$

$$\boldsymbol{K}=k\,\begin{bmatrix}5&-3\\-3&3\end{bmatrix}$$

$$\boldsymbol{p}(t) = p_0\,\begin{Bmatrix}0\\1\end{Bmatrix}\sin(2\omega_0t)$$

## Computing the eigenvalues and the eigenvectors

Using the `eigh` library function, specialized for Hermitian (symmetric) matrices.

We compute also the _starred_ matrices.  Note that the eigenvectors returned by `eigh` are normalized with respect to `M`, i.e., $M_i=m$ for every $i$.

```python
evals, Psi = eigh(K, M)
Psi[:,0] /= Psi[0,0] ; Psi[:,1] /= Psi[1,1]
Mstar = Psi.T@M@Psi
Kstar = Psi.T@K@Psi
pstar = Psi.T@p
```

The `@` operator stands, in this context, for matrix multiplication.

We print the computed values.

```python
print_mat(evals[None,:], mt='p', pre=r'\omega^2_i=\omega^2_o\,')
print_mat(Psi, pre=r'\boldsymbol{\Psi}=')
print_mat(Mstar, pre=r'\boldsymbol{M}^\star=m\,')
print_mat(Kstar, pre=r'\boldsymbol{K}^\star=k\,')
print_mat(pstar[:,None], pre=r'\boldsymbol{p}^\star=p_o\,', mt='B')
```

$$\omega^2_i=\omega^2_o\,\begin{pmatrix}0.250000&3.000000\end{pmatrix}$$

$$\boldsymbol{\Psi}=\begin{bmatrix}1.000000&-3.000000\\1.500000&1.000000\end{bmatrix}$$

$$\boldsymbol{M}^\star=m\,\begin{bmatrix}11.000000&0.000000\\0.000000&22.000000\end{bmatrix}$$

$$\boldsymbol{K}^\star=k\,\begin{bmatrix}2.750000&-0.000000\\-0.000000&66.000000\end{bmatrix}$$

$$\boldsymbol{p}^\star=p_o\,\begin{Bmatrix}1.500000\\1.000000\end{Bmatrix}$$

## Modal Response

The modal equation of motion, divided by $M_i$
$$ \ddot q_i + \Lambda^2_i\omega_0^2 q_i = \frac{p_0}{k}\frac{k}{m}\frac{m}{M_i}\,p^\star_i \sin2\omega_0t.$$

The particular integral $\xi_i(t)$ is given by

$$\xi_i = \Delta_{st} \frac{\omega_0^2}{(\Lambda^2_i-4)\omega_0^2}\frac{m}{M_i}p^\star_i\sin2\omega_0t.$$

Eventually the modal response, computed in the hypotesis of initial rest conditions is

$$q_i = \frac{\Delta_{st}}{\Lambda^2_i-4}\frac{m}{M_i}p^\star_i(\sin2\omega_0t-\beta\sin\Lambda_i\omega_0t).$$

```python
L  = np.sqrt(evals)
DAF = np.linalg.inv(Mstar)@(1.0/(L**2-w**2))
beta = w/L
t = np.linspace(0,50,1001)[:,None]
q = pstar*DAF*(np.sin(w*t)-beta*np.sin(L*t))
```

The definition of time vector is a bit complicated...

```python
plt.style.use(['fivethirtyeight', '00_mplrc'])

```

```python
curves = plt.plot(t,q)
plt.legend(curves,['q1', 'q2'])
plt.title('Modal Response')
plt.xlabel('$\omega_0t$')
plt.ylabel('$q_i/\Delta_{st}$');
plt.tight_layout()
plt.savefig('../figures/modal_response.pdf')
```

```python
x = (Psi@q.T).T

curves = plt.plot(t, x)
plt.legend(curves,['x1', 'x2'])
plt.title('Structural Response')
plt.xlabel('$\omega_0t$')
plt.ylabel('$X_i/\Delta_{st}$')
plt.tight_layout()
plt.savefig('../figures/structural_response.pdf')
```

```python
plt.plot(t, (2*x[:,0]+4*x[:,1])/6, label='$(4x_2+2x_1)/6$')
plt.plot(t, (x[:,1]+x[:,0])  , label='$(x_2-x_1)$')
plt.title('Structural Response Tweaked')
plt.xlabel('$\omega_0t$')
plt.ylabel('$X_i/\Delta_{st}$')
plt.legend()
plt.tight_layout()
plt.savefig('../figures/tweaked.pdf')

```

The following code cell (that is executed _before_ any other code cell by the notebook) loads libraries (or functions from libraries) and determines the style of plots and of the notebook itself.  Besides the cell defines a function to format conveniently matrices and vectors.

```python
%matplotlib inline

import matplotlib.pyplot as plt

import numpy as np ; from scipy.linalg import eigh
np.set_printoptions(suppress=False, linewidth=120)

from IPython.display import Latex

def print_mat(mat, pre='', post='', fmt='%.6f', mt='b'):
    display(Latex(
        r'$$' + pre + r'\begin{%smatrix}'%mt +
        r'\\'.join('&'.join(fmt%x for x in row) for row in mat) +
        r'\end{%smatrix}'%mt + post + r'$$'))

from IPython.display import HTML
HTML(open('00_custom.css').read())
```

<style>
@font-face {
    font-family: 'Charis SIL';
    src: url('fonts/CharisSILEur-R.woff') format('woff');
}
@font-face {
    font-family: 'Architects Daughter';
    font-style: normal;
    src: url(https://fonts.gstatic.com/s/architectsdaughter/v6/RXTgOOQ9AAtaVOHxx0IUBM3t7GjCYufj5TXV5VnA2p8.woff2) format('woff2');
    unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02C6, U+02DA, U+02DC, U+2000-206F, U+2074, U+20AC, U+2212, U+2215, U+E0FF, U+EFFD, U+F000;
}
.prompt{display: None;}
div.cell{margin: auto;width:1000px;}

div #notebook { /* centre the content */
    background: #fff;
    margin: auto;
    padding: 0em;
    padding-top: 1em;
}
div #notebook_container {width: 100px!important;}
#notebook li { /* More space between bullet points */
    margin-top:0.2em;
}

div.text_cell_render{
    font-family: "Charis SIL", Cambria,  serif;
    line-height: 155%;
    font-size: 130%;
}
.CodeMirror{
    font-family: Consolas, monospace;
    font-size: 120%;
    background-color:#fafcff!important;
}

.output_area {
    font-family: Consolas,monospace;font-size: 110%;
    background-color:#fafcff!important;
    margin-top:0.8em;
}

div.output_latex{overflow:hidden}

h1 {
    font-family: 'Architects Daughter', serif;
    font-size:   48pt!important;
    text-align:  center;
    text-shadow: 4px 4px 4px #aaa;
    padding-bottom: 48pt;
}

.warning{color: rgb( 240, 20, 20 )}
</style>

```python
import sympy as sy
sy.init_printing(use_latex=1)
o, m0, k0 = sy.symbols('omega^2 m k')
k1, k2, m1, m2 = 2*k0, 3*k0, 2*m0, 4*m0
sM = sy.Matrix(((m1,0,),(0,m2)))
sK = sy.Matrix(((k1+k2, -k2),(-k2,k2)))
KooM = sK - o*sM
display(KooM)
display(KooM.det())
display(KooM.det().expand())
iKooM = KooM.inv()
sp = sy.Matrix(((0,),(1,)))
a,b=(iKooM*sp)
display(sy.Eq(sy.symbols('D_1'),a),sy.Eq(sy.symbols('D_2'),b))
a = a.expand().simplify().subs({m0:1,k0:1})
b = b.expand().simplify().subs({m0:1,k0:1});
```

```python
with plt.style.context(['classic', '00_mplrc']):
    plot = sy.plot(a, b, (o, 0, 5), ylim=(-5,5), show=False)
    plot[0].line_color = 'black'; plot[0].label = '$D_1$'
    plot[1].line_color = 'red'  ; plot[1].label = '$D_2$'
    plot.xlabel = r'$\beta^2$'
    plot.ylabel = r'$D_{i}$'
    plot.legend = True
    plot.show()
```