# Python's Next Top Model
By The Good, The Bad and the Ugly aka. The Three Musketeers.

```python
# Importing packages
import matplotlib.pyplot as plt
import pandas as pd
import ipywidgets as widgets
import numpy as np
import itertools
from ipywidgets import Layout
import math

import time
from scipy import linalg
import scipy.optimize as optimize
import sympy as sm
#from data_gen import gen_df
from IPython.display import clear_output
```

Welcome back to the wonderful Python universe of Credible Threats. We will continue the exploration of movies, and we're sure that our data project notebook will look at this notebook and think, *you complete me*. You might want to ask, *Magic mirror on the wall, who's the fairest one of all*, and the answer will be this notebook. This notebook will try to estimate the IMDb ratings of movies. And as a bonus, we even have a live-updating figure, which will leave you thinking, *it's alive, it's alive*.

We will calculate the utility of each movie, given genres, awards, duration and so forth, and we will try to calculate the IMDb ratings for each movie, based on this utility. Finally, we will minimize the distance between our estimated ratings and the actual IMDb ratings. Now, we're sure you're thinking, *show me the money*, so this is it.

Our utility function is given by:
$$ U_i = \sum_{k=1}^{23}(\alpha_k G_{ik}) + \sum_{k=1920}^{2010} (\beta_k D_{ik}) + \gamma N_i + \delta W_i + \rho L_i $$.

$$\text{Where, } G_{ik} \text{ is genre } k \text{, } D_{ik} \text{ is decade } k \text{, } N_i \text{ is award nominations, } W_i \text{ is award wins, and } L_i \text{ is duration. } $$

I think we can agree, it is *beauty kills the beast*.
Now, based on this utility function, we will estimate the ratings of each movies,
$$ R_i^{model} = \frac{\exp(\omega x_i')}{1 + \exp(\omega x_i')} $$

where,
$$ x_i = \big[G_1, G_2, ..., G_{23}, D_{1920}, D_{1930}, ..., D_{2010}, N_i, W_i, L_i \big] $$
$$ \omega = \big[\alpha_1, \alpha_2, ..., \alpha_{23}, \beta_{1920}, \beta_{1930}, ..., \beta_{2010}, \gamma, \delta, \rho_1 \big] $$

We then use optimize methods to solve the following:
$$ \min_{\omega} \Big\{ \sum_{i=1}^{n} \left( R_i^{model} - R_i^{data} \right)^2 \Big\} $$

$$ \text{where, } R_i^{data} \text{ are the true IMDb ratings from the dataset, and} R_i^{model} \text{are the ratings calculated from the utility function.}$$

Throughout our notebook, you might *feel the need - the need for speed*. But, *patience you must have, my young Padawan*, because optimizing takes time.

*Of all the python notebooks in all the towns in all the world, you walk into ours*. How lucky you are, you'll soon see why.

So, *say hello to my little friend*, Python's next top model.
Let's *get busy coding, or get busy dying*.

```python
# List for later use
variables = ['Action','Adventure','Animation','Biography','Comedy','Crime','Documentary',
            'Drama','Family','Fantasy','FilmNoir','History','Horror','Music','Musical',
            'Mystery','Romance','SciFi','Short','Sport','Thriller','War','Western',
            'decade_1920','decade_1930','decade_1940','decade_1950','decade_1960',
            'decade_1970','decade_1980','decade_1990','decade_2000','decade_2010',
            'nrOfNominations','nrOfWins','duration']

vars_dec = ['Action','Adventure','Animation','Biography','Comedy','Crime','Documentary',
        'Drama','Family','Fantasy','FilmNoir','History','Horror','Music','Musical',
        'Mystery','Romance','SciFi','Short','Sport','Thriller','War','Western',
        'nrOfNominations','nrOfWins','duration']

decade_list = [1920, 1930, 1940, 1950, 1960, 1970, 1980, 1990, 2000, 2010]

# Function to calculate squared mean differences
def sqr_diff_sum(df_X,pars):
    """ Generates a float of the sum of squared differences between the ratings from the data and the model.

    Args:
        df_X (DataFrame): DataFrame containing the variable in x for all observations.
        pars (List): List of parameters in omega

    Returns:
        A float.
    """
    # Calculate the matrix product between omega and X
    util1 = df_X@pars[1:1+len(variables)]
    util2 = df_X@pars[1+len(variables):]
    # Scale the product so it's between 0 and 10. This is the R_model
    df_Y['rat_model1'] = 10*np.exp(util1)/(1+np.exp(util1))
    df_Y['rat_model2'] = 10*np.exp(util2)/(1+np.exp(util2))
    Y = lambda pi: pi*df_Y['rat_model1']+(1-pi)*df_Y['rat_model2']
    # Calculate the squared difference between R_data and R_model
    df_Y['sqr_diff'] =  (Y(pars[0])-df_Y['rat_data'])**2

    return df_Y['sqr_diff'].sum() # Returns the sum of the squared differences
```

```python
def gen_df(filename):

    # Get .csv.file
    data = pd.read_csv(filename, sep=';', encoding='latin-1', escapechar='\\')

    # Read file into pandas dataframe
    df = pd.DataFrame(data)

    # Drop unwanted columns
    for i in range(44,48):
        df.drop(columns=[f'Unnamed: {i}'], inplace=True)

    # Filters out movies with less than 5000 ratings and drops some genres and other columns
    df = df.loc[(df['ratingCount']>=5000)]
    df.drop(columns=['fn','wordsInTitle','url','Adult','GameShow','News','RealityTV','TalkShow'], inplace=True)

    # Keep only observations of movie-type
    I = df['type'] == 'video.movie'
    df = df.loc[I]
    df.drop(columns=['type'], inplace=True)

    # Drop observations with missing data
    df.dropna(inplace=True)

    # Replace 0's in imdb-ratings
    df['imdbRating'] = df['imdbRating'].astype(str)
    df['imdbRating'].replace(regex=True, inplace=True,to_replace='0',value='')
    df['imdbRating'] = df['imdbRating'].astype(float)

    # Transform duration from seconds to hours
    df['duration'] = df['duration']/60**2

    # Drop years before 1920 and 2014 because of few obervations
    I = (df['year']>=1920) & (df['year']<=2013)
    df = df.loc[I]

    # Change the type of 'year' to integer
    df['year'] = df['year'].astype(int)

    # Sort observations and reset index
    df.sort_values('year', inplace=True)
    df.reset_index(inplace=True)

    # Generating variable that shows the decade (as a string)
    year_list = [1920, 1930, 1940, 1950, 1960, 1970, 1980, 1990, 2000, 2010]

    df['decade'] = ''
    for i,start in enumerate(year_list):
        end = start+10
        df.loc[(df['year'] >= start) & (df['year'] < end), 'decade'] = f'{year_list[i]}s'

    return df
```

```python
def df_s(keep_top=None, decade=None):
    """ Prepares data for optimization by creating dummmy-variables for decades, dropping movies with less than 5000 ratings,
        dropping some variables that we don't wish to use in the model, and splitting the dataset, so we have a data set
        consisting only of the variables used in the model.

    Args:
        keep_top (optional, type: int): When a number (n) is put in this option, only the top n movies a kept in the dataset. This is used to remove movies for a better fit of the model

    Returns:
        df_X (type: Pandas dataframe): A dataframe consisting only of the varaibles that are used to calculate the rating.
        df_Y (type: Pandas dataframe): A dataframe consisting of the true ratings from the dataset and an index-variable, used for merging with original dataset, later in the process.

    Notice:
        The function includes global for 'df', which means that the dataframe 'df' generated, can be called outside the function. This is used so that we won't have to filter the original dataframe everytime we use it.
    """
    global df

    # Calls the file 'imdb.csv' which is located in the repository, and contains the dataset used
    filename = 'imdb.csv'

    # A function generated earlier, which cleans the dataset [Should we include this, and do some of the filtering below in that funciton?]
    df = gen_df(filename)

    # Keeps only top n movies, if this is specified when calling the function
    if keep_top != None:
        df = df.sort_values('imdbRating', ascending=False)
        df = df.iloc[:keep_top]

    if decade == None:
        # Decade dummies
        for i in decade_list:
            df[f'decade_{i}'] = 0
            df.loc[(df['decade'] == f'{i}s'),f'decade_{i}'] = 1

        # Splits the dataset into two datasets
        df_X = df.copy()
        df_Y = pd.DataFrame(df[['imdbRating', 'index']].copy())
        df_Y = df_Y.rename(columns = {'imdbRating':'rat_data'})

        # Rearrange and keep given columns
        df_X = df_X.reindex(['Action','Adventure','Animation','Biography','Comedy','Crime','Documentary',
                              'Drama','Family','Fantasy','FilmNoir','History','Horror','Music','Musical',
                              'Mystery','Romance','SciFi','Short','Sport','Thriller','War','Western',
                              'decade_1920','decade_1930','decade_1940','decade_1950','decade_1960',
                              'decade_1970','decade_1980','decade_1990','decade_2000','decade_2010',
                              'nrOfNominations','nrOfWins','duration'], axis=1)

    if decade != None:
        # Keeps movies from the specified decade
        df = df.loc[df['decade'] == f'{decade}s']

        # Splits the dataset into two datasets
        df_X = df.copy()
        df_Y = pd.DataFrame(df[['imdbRating', 'index']].copy())
        df_Y = df_Y.rename(columns = {'imdbRating':'rat_data'})

        # Rearrange columns and keep the specified variables
        df_X = df_X.reindex(['Action','Adventure','Animation','Biography','Comedy','Crime','Documentary',
                                  'Drama','Family','Fantasy','FilmNoir','History','Horror','Music','Musical',
                                  'Mystery','Romance','SciFi','Short','Sport','Thriller','War','Western',
                                  'nrOfNominations','nrOfWins','duration'], axis=1)

    return df_X, df_Y
```

## Optimization
We analyze preferences for movies by minimizing the sum of the squared differences from our model prediction and the true rating, i.e.:
$$
\min_{\omega} \Big\{ \sum_{i=1}^{n} \left( R_i^{model} - R_i^{data} \right)^2 \Big\}
$$
By doing so, we'll find parameters that best describe movie preferences given our model. As we will discuss later, our model is not necessarily the best model to describe preferences, which might lead to these preferences not being accurate.

If the model could predict all movie ratings correctly the parameters would perfectly describe movie preferences. For example if the parameter for action-movies were negative, the prediction would be precise, if action-movies generally had a negative impact on ratings. Of course people have much different preferences, and an interesting analysis would be, to predict preferences for different people, and by this look at a distribution of these parameters. This is not included in this project, though, mainly because of we don't have the data for it.

To shorten the length of the optimizing process, and to sort out movies that makes it harder for the model to predict preferences, the optimization can be based on the top n movies, based on IMDb rating. When all movies are included in the optimizer the lowest ranking movies will have ratings around 2. Our rating predictor will in this case not rank any movies under 5. Thereby it is clear that the predictor model is not able to predict movies with very low rating. But it is able to predict movies with higher ratings.

```python
def optimizer(keep_top=None, live_graph=True):
    """ Creates and optimizes the function which calculates rating based on variables in the dataset.

    Args:
        keep_top (optional, type: int): When a number (n) is put in this option, only the top n movies a kept in the dataset. This is used to sort out movies for a better fit of the model
        live_graph(optional, type: boolean): Produces a live graph of the optimization proces, plotting the function values against the iterations. Makes the optimizer slower, but is a good visualization of the process.

    Returns:
        result (type: scipy optimizer object): An object containing results from the optimizer among other information such as status etc.
        timer (type: int): Time of running optimizer in seconds.

    Notice:
        Uses the package scipy.optimize.minimize which optimizes functions based on a given method. Here 'Nelder-Mead' is used.
        This function defines a set of functions used in the optimizer which are not documented by docstring but comments in code.
    """
    # "Initializes" a timer for printing time of optimization
    start = time.time()

    # Set of globals, so variables can be called inside the functions defined in this function
    global fs
    global evals
    global x0
    global df
    global df_Y

    # Plots a live graph of optimization process, if chosen.
    if live_graph:
        # The live plot is created by taking the function value for each iteration and saving in a list
        def live_plot(evals, fs, ymax=10000, figsize=(7,5)):
            # Clears plot every time a new plot is created
            clear_output(wait=True)
            plt.figure(figsize=figsize)

            # Plots values and sets title, grid, labels, etc.
            plt.plot(evals, fs)
            plt.title('Figure 1: Optimizing path')
            plt.grid(True)
            plt.xlabel('Iterations')
            plt.ylabel('Function value')
            plt.xlim(0,125_000)
            plt.ylim(0,ymax)

            plt.show();

    # Function to be called by optimizer each iteration. This returns function values for each iteration and plots live graph
    def collect(x):
        # Set of globals to be called in live graph-function
        global fs
        global evals
        global x0
        global fig
        global ax
        global ymax
        global x_1, x_2, x_3, x_4, x_5, x_6, x_7, x_8, x_9, x_10, x_11, x_12, x_13, x_14, x_15, x_16, x_17, x_18, x_19, x_20
        global x_21, x_22, x_23, x_24, x_25, x_26, x_27, x_28, x_29, x_30, x_31, x_32, x_33, x_34, x_35, x_36

        # Calculates function value for initial guess, for first iteration
        if evals == 0:
            fs = []
            fs.append(obj_fun(x0))
            # Generates vector of parameter estimates
            for i in range(1,37): # 1 to 36
                globals()[f'x_{i}'] = [x0[i-1]]

        # Calculates function values for current parameter values
        if evals != 0:
            fs.append(obj_fun(x))

        # Plots live graph if chosen
        if live_graph:
            # Calculates max y-value for axis for first iteration
            if evals == 1:
                ymax = math.ceil(obj_fun(x)/2000)*2000

            # Updates plot every 100th iteration for the first 1000 iterations
            if evals < 1000:
                if evals > 0 and evals%100 == 0:
                    live_plot(range(evals+1), fs, ymax)
            # Updates plot every 500th iteration for the rest of the process
            if evals >=1000:
                if evals%500 == 0:
                    live_plot(range(evals+1), fs, ymax)

        # Appends esimtates to x_vectors
        for i in range(1,37):
            globals()[f'x_{i}'].append(x[i-1])

        # Adds one to the number of iterations each iteration
        evals += 1

    # Define datasets to be used using function defined earlier
    df_X, df_Y = df_s(keep_top=keep_top, decade=None)

    # Intiial guess
    x0 = np.zeros(len(variables)*2+1)

    # Start iteration number
    evals = 0

    # Defines function to be minimized
    #obj1 = lambda x: sqr_diff_sum(df_X,x)
    #obj2 = lambda xx: sqr_diff_sum(df_X,xx)
    #obj_fun = lambda x: x[0]*obj1(x[1:1+len(variables)])+(1-x[0])*obj2(x[2+len(variables):-1])

    obj_fun = lambda x: sqr_diff_sum(df_X,x)

    # Run optimizer
    result = optimize.minimize(obj_fun,x0,
                               method="Nelder-Mead",
                               options={"disp":True, "maxiter":1000000}, # display the results
                               callback=collect
                               )

    # End timer
    end = time.time()-start

    # Returns
    return result, end
```

By runing the code below you can see a live-updating graph that shows the value of the sum of the squared difference for each iteration. As you will see, the sqaured difference reaches close to its minimum very quickly so the last 10,000 iterations almost don't change the value of the squard difference. The live-graph slows the optimizer down, so choose option live_graph=False, if you are impacient and don't care about an awesome live-updating graph.

```python
# Run optimizer function and print results and time
result, timer = optimizer()
print(f'{" ":9s}Time: {timer:.4f} seconds')

# Keep copy of df and df_Y for later purpose
df_Y_all = df_Y.copy()
df_all = df.copy()
```

The following cell will give a list of the estimated parameters/preferences, and the cell after that produce a figure showing the path of the estimated parameters, i.e. the parameter estimate for each iteration.

```python
# List of Parameter Estimates
variables_all = ['pi','Action1','Adventure1','Animation1','Biography1','Comedy1','Crime1','Documentary1',
            'Drama1','Family1','Fantasy1','FilmNoir1','History1','Horror1','Music1','Musical1',
            'Mystery1','Romance1','SciFi1','Short1','Sport1','Thriller1','War1','Western1',
            'decade_1920_1','decade_1930_1','decade_1940_1','decade_1950_1','decade_1960_1',
            'decade_1970_1','decade_1980_1','decade_1990_1','decade_2000_1','decade_2010_1',
            'nrOfNominations1','nrOfWins1','duration1',
            'Action2','Adventure2','Animation2','Biography2','Comedy2','Crime2','Documentary2',
            'Drama2','Family2','Fantasy2','FilmNoir2','History2','Horror2','Music2','Musical2',
            'Mystery2','Romance2','SciFi2','Short2','Sport2','Thriller2','War2','Western2',
            'decade_1920_2','decade_1930_2','decade_1940_2','decade_1950_2','decade_1960_2',
            'decade_1970_2','decade_1980_2','decade_1990_2','decade_2000_2','decade_2010_2',
            'nrOfNominations2','nrOfWins2','duration2']

# Storing the results in a scalar (pi) and two vectors (omega1 and omega2)
pi = result.x[0]
beta1 = result.x[1:1+len(variables)]
beta2 = result.x[1+len(variables):]
```

    pi : 0.21
    Action1 : 0.51
    Adventure1 : 0.32
    Animation1 : 0.68
    Biography1 : 1.05
    Comedy1 : 0.48
    Crime1 : 1.98
    Documentary1 : -3.72
    Drama1 : -0.99
    Family1 : 0.14
    Fantasy1 : -1.38
    FilmNoir1 : 2.38
    History1 : 1.19
    Horror1 : 0.02
    Music1 : 1.51
    Musical1 : 0.27
    Mystery1 : 0.75
    Romance1 : -2.25
    SciFi1 : 1.23
    Short1 : 1.29
    Sport1 : -0.37
    Thriller1 : 1.82
    War1 : 0.51
    Western1 : 2.15
    decade_1920_1 : 0.75
    decade_1930_1 : 0.71
    decade_1940_1 : -2.32
    decade_1950_1 : -4.85
    decade_1960_1 : 0.01
    decade_1970_1 : -2.79
    decade_1980_1 : 0.90
    decade_1990_1 : -1.08
    decade_2000_1 : -1.67
    decade_2010_1 : 1.17
    nrOfNominations1 : 0.03
    nrOfWins1 : -0.10
    duration1 : -3.29
    Action2 : -0.18
    Adventure2 : -0.19
    Animation2 : 0.73
    Biography2 : -0.11
    Comedy2 : -0.14
    Crime2 : -0.07
    Documentary2 : 0.43
    Drama2 : 0.27
    Family2 : -0.35
    Fantasy2 : -0.04
    FilmNoir2 : -0.24
    History2 : -0.24
    Horror2 : -0.17
    Music2 : 0.50
    Musical2 : 0.84
    Mystery2 : -0.14
    Romance2 : -0.12
    SciFi2 : -0.10
    Short2 : 1.57
    Sport2 : -0.08
    Thriller2 : -0.15
    War2 : 0.12
    Western2 : 0.60
    decade_1920_2 : 0.98
    decade_1930_2 : 0.80
    decade_1940_2 : 1.25
    decade_1950_2 : 0.78
    decade_1960_2 : 0.69
    decade_1970_2 : 0.36
    decade_1980_2 : -0.23
    decade_1990_2 : -0.40
    decade_2000_2 : -0.42
    decade_2010_2 : -0.48
    nrOfNominations2 : 0.00
    nrOfWins2 : 0.09
    duration2 : 1.05

```python
pi = result.x[0]
beta1 = result.x[1:1+len(variables)]
beta2 = result.x[1+len(variables):]
print(pi, beta1, beta2)
```

    0.20749148508587262 [ 0.5126855   0.31644409  0.67935101  1.05073511  0.48115292  1.97509937
     -3.71633105 -0.9893191   0.14164573 -1.38221837  2.38230374  1.1908366
      0.01928083  1.51208177  0.26679186  0.74835108 -2.25248631  1.22973842
      1.29037034 -0.36850403  1.82412992  0.508087    2.15338994  0.75314066
      0.7092667  -2.31802709 -4.85177525  0.00936921 -2.79444306  0.90370945
     -1.07599533 -1.67043156  1.17369347  0.02901874 -0.09936469 -3.28799401] [-0.17642366 -0.19240625  0.73052715 -0.10667021 -0.13588986 -0.06734203
      0.43275351  0.27354848 -0.34688792 -0.04464825 -0.23591994 -0.23934028
     -0.16938705  0.5022226   0.8408503  -0.13913251 -0.11599286 -0.10046638
      1.56991504 -0.07814778 -0.14956947  0.11715331  0.59972736  0.98125051
      0.79668914  1.24526378  0.77637431  0.69156417  0.36329475 -0.2253367
     -0.40360875 -0.42462343 -0.48467226  0.00264565  0.09471785  1.05445557]

```python
fig = plt.figure(figsize=(16,6))
ax1 = fig.add_subplot(1,1,1)

barWidth = 0.5
r1 = np.arange(len(beta1))
r2 = [x + barWidth for x in r1]

# Plots the estimated parameters for the chosen sample of movies
ax1.bar(r1, beta1, width=barWidth, label=f'Estimates for type 1', edgecolor='white')
ax1.bar(r2, beta2,  width=barWidth, label=f'Estimates for type 2', edgecolor='white')

# Legends and labels
ax1.legend(loc="lower right")
ax1.set_ylabel('Parameter estimates')
ax1.set_title(f'Parameter estimates for the two types of individuals, Pi={pi:.2f}')
plt.xticks([r + barWidth for r in range(len(beta1))], variables)
#ax1.set_ylim([-0.7,0.7])
ax1.axhline(y=0,color='black',linewidth=1)
for tick in ax1.get_xticklabels():
    tick.set_rotation(90)

```

```python
## Convergence of Parameter Estimates

# Constructing dataframe which contains all parameter vectors as variables
par_dict = {"Action": x_1,
             "Adventure": x_2,
             "Animation": x_3,
             "Biography": x_4,
             "Comedy": x_5,
             "Crime": x_6,
             "Documentary": x_7,
             "Drama": x_8,
             "Family": x_9,
             "Fantasy": x_10,
             "Film Noir": x_11,
             "History": x_12,
             "Horror": x_13,
             "Music": x_14,
             "Musical": x_15,
             "Mystery": x_16,
             "Romance": x_17,
             "SciFi": x_18,
             "Short": x_19,
             "Sport": x_20,
             "Thriller": x_21,
             "War": x_22,
             "Western": x_23,
             "1920's": x_24,
             "1930's": x_25,
             "1940's": x_26,
             "1950's": x_27,
             "1960's": x_28,
             "1970's": x_29,
             "1980's": x_30,
             "1990's": x_31,
             "2000's": x_32,
             "2010's": x_33,
             "Nr. of Nominations": x_34,
             "Nr. of Wins": x_35,
             "Duration": x_36
            }

df_par = pd.DataFrame(par_dict)

##  Interactive graph of parametervalues for each iteration in optimizer
# Chose which parameter to show

# Interactive graph of parameter estimates
def graph(par):
    """
    Constructs a graph which shows the progress in parameter estimates foreach iteration in
    the optimizing process.

    Args:
        par (type: str): Variable name, for which the graphs shows the parameter estimates.

    Returns:
        Graph of progress in parameter estimates.

    Notice:
        The function is meant to be called within the graph_int function.
        The graph_int function turns this graph into an interactive function, where the
        user can choose between variables.
        Thus, the graph is not meant to be run on its own.
    """
    # Generating figure
    plt.plot(df_par[par])

    # Grid and axes
    plt.grid(True)
    plt.axhline(df_par[par].iloc[-1], linestyle="dashed", color="orange", label="Optimum")

    # Labels and Titles
    plt.title("Figure 2: Progress in Parameter Estimation")
    plt.ylabel("Parameter Estimate")
    plt.xlabel("Nr. of Iterations")
    plt.legend()

# Interactive part: Choose between parameters
def graph_int():
    """
    Constructs interactive graph from the graph function.
    """
    widgets.interact(graph,
                     par = widgets.Dropdown(
                     description="Variable",
                     options=df_par.columns.values,
                     value="Action"));
graph_int()

```

    interactive(children=(Dropdown(description='Variable', options=('Action', 'Adventure', 'Animation', 'Biography…

From the above figure, we notice that most parameter estimates are very volatile in the first 10,000-15,000 iterations, following which they converge towards the optimum value.

However, this does not appear to be the case for the variables, Nr. of nominations and Nr. of awards, which get quite close to the optimum value at around the 7500th iteration.
Additionally, we notice that the variable Duration appears to have the largest effect on ratings, which the optimzer seems to realize after approximately 2500 iterations.

In the next part of the project we will plot the optimal estimates in a figure. Furthermore, we show how the estimates changes a lot when the sample size is restricted to only the best movies.

```python
## This piece of code solves the model for three different sample size

#### OBS: The code will take a couple of minutes to run because the model is solved three times!! ####

# List to store result for the optimization with the different sample sizes
results_list = [result.x]

# This loop solve the model for the top 500, 1000, and 2000 movies
for i in [500, 1000, 2000]:
    res_temp, timer_temp = optimizer(keep_top=i, live_graph=False)
    temp = res_temp.x
    print(f'{" ":9s}Time: {timer_temp:.4f} seconds')
    results_list.append(temp) # Store the results in the 'results_list'
```

```python
## The code creates an interactive plot of the estimated parameters for all variables.
## In the interactive part you can choose between estimates when the model is solved
## for all movies or just top top 500, 1000, or 2000 rated movies.

def fig_2(val):
    """ Generates a figure which plots estimated parameters for all variables.

    Args:
        val (string or int): Should be one of the elements in the options-list

    Returns:
        One interactive plot.

    Notice:
        The function is generated so that it can be called using widgets.interact.
        Thus, it is not intended to be used on its own.
    """
    # Initiates figure
    fig = plt.figure(figsize=(12,6))
    ax1 = fig.add_subplot(1,1,1)

    # Plots the estimated parameters for the chosen sample of movies
    ax1.bar(variables, results_list[options.index(val)], label=f'Estimates with {options[options.index(val)]} movies')

    # Scatter plot with the estimated paramters for the entire sample
    ax1.scatter(variables, results_list[0], marker='D', s=15, zorder=2, label='Estimates with all movies')

    # Legends and labels
    ax1.legend(loc="lower right")
    ax1.set_ylabel('Parameter estimates')
    ax1.set_title(f'Figure 3: Parameter estimates for {options[options.index(val)]} movies')
    ax1.set_ylim([-0.7,0.7])
    ax1.axhline(y=0,color='black',linewidth=1)
    for tick in ax1.get_xticklabels():
        tick.set_rotation(90)

options = ['All', 500, 1000, 2000] # Option list

# Making the figure interactive so the estimates are shown for the chosen sample size
widgets.interact(fig_2,
    val = widgets.Dropdown(description='Nr. of movies', value='All', options=options,
                ),
);
```

As also showed in the list above, duration is the varaible that effects the ratings the most. This is also the case when the model is solved for top 500, 1000, and 2000 rated movies, cf. figure 3. However, it seems like duration is the only estimate which is fairly stable. The sign of the decade dummies is an example. When the model is solved for the entire sample, the estimates are positive for the older decade-dummies, i.e. the 1920s and 1930s, and negative for the newer decade-dummies, the 2000s and 2010s. However, when the sample only contains the top 2000 rated movies we see the opposite picture. We know from figure 10 in our dataproject that the ratings of the younger movies have a larger variance with more high-rated movies but also a lot of poorly rated movies. By restricting the sample to only the best movies we drop all the bad new movies, and this is probably the reason to the positive estimates of the newer movies.

## Robustness checks

### Prediction precision
Ideally our model would predict movie ratings perfectly, and thereby our parameters would perfectly describe the general preferences for movies. Whether this is possible, even with an advanced model, is not sure. People have different preferences, and two movies that would seem identical in our dataset, based on genres, duration, time of release, etc. could have very different ratings. Therefore some devitaion in our prediction from the true ratings are expected. For that reason, this section will explore how well our model describe the actual ratings, and why some estimates are more reliable than others.

```python
def merge_df(df_org):
    """ Merges the original dataset with the optimal solution from the optimizer, i.e. the last dataset of calculated raitngs.

    Args:
        df_org (type: Pandas dataframe): The orignal dataset of movies, containing true ratings.

    Returns:
        df_merge (type: Pandas dataframe): Original dataset combined with calculated ratings from optimal parameters.
    """

    # Merges original dataset with optimal ratings based on model
    df_merge = df_org.merge(df_Y_all, how='right', left_on='index', right_on='index')

    # Calculates deviations both in normal and absolute form
    df_merge['abs_diff'] = abs(df_merge['rat_model']-df_merge['rat_data'])
    df_merge['diff'] = df_merge['rat_model']-df_merge['rat_data']

    return df_merge

def _mean_genre(df,group, diff):
    """ Calculates the mean of deviations from true ratings and ratings based on optimal parameters from optimizer.

    Args:
        df (type: Pandas dataframe): Dataframe consisting information on groups and deivations
        group (type: string): Defines which group mean of deviations are presented for. Chosen by fixed list in widget.

    Returns:
        None

    Notice:
        Only to be run through widget.interact()
    """
    # List of values used for each group
    genre_list = ['Action','Adventure','Animation','Biography','Comedy','Crime','Documentary',
                    'Drama','Family','Fantasy','FilmNoir','History','Horror','Music','Musical',
                    'Mystery','Romance','SciFi','Short','Sport','Thriller','War','Western']

    decade_list = ['1920s', '1930s', '1940s', '1950s', '1960s', '1970s', '1980s', '1990s', '2000s', '2010s']

    dur_list = [0, .5, 1, 1.5, 2, 2.5, 3, 3.5]

    rat_list = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

    # Dictionaries for plotting
    mean_dict = dict()
    count_dict = dict()

    # Choose which type of difference to plot
    if diff == 'Absolute':
        diff_col = 'abs_diff'
    else:
        diff_col = 'diff'

    # Creates dictionaries based on chosen group
    if group == 'Genres':
        for i in genre_list:
            I = df[i] == 1
            mean_dict[i] = df.loc[I][diff_col].mean()
            count_dict[i] = df.loc[I]['index'].count()

    if group == 'Decades':
        for i in decade_list:
            I = df['decade'] == i
            mean_dict[i] = df.loc[I][diff_col].mean()
            count_dict[i] = df.loc[I]['index'].count()

    if group == 'Duration':
        for j,i in enumerate(dur_list):
            if j != len(dur_list)-1:
                # Uses values from dict, to find movies between to items in the lists.
                I = ((df['duration'] >= dur_list[j]) & (df['duration'] < dur_list[j+1]))
                mean_dict[f'{dur_list[j]:2.1f} - {dur_list[j+1]:2.1f}'] = df.loc[I][diff_col].mean()
                count_dict[f'{dur_list[j]:2.1f} - {dur_list[j+1]:2.1f}'] = df.loc[I]['index'].count()
            else:
                I = df['duration'] >= dur_list[j]
                mean_dict[f'{dur_list[j]:2.1f} {"+":5s}'] = df.loc[I][diff_col].mean()
                count_dict[f'{dur_list[j]:2.1f} {"+":5s}'] = df.loc[I]['index'].count()

    if group == 'True rating':
        for j,i in enumerate(rat_list):
            if j != len(rat_list)-1:
                # Uses values from dict, to find movies between to items in the lists.
                I = ((df['rat_data'] >= rat_list[j]) & (df['rat_data'] < rat_list[j+1]))
                mean_dict[f'{rat_list[j]:4.1f} - {rat_list[j+1]:3.1f}'] = df.loc[I][diff_col].mean()
                count_dict[f'{rat_list[j]:4.1f} - {rat_list[j+1]:3.1f}'] = df.loc[I]['index'].count()
            else:
                pass

    # Creates figure to hold two subplots
    fig1, (ax1,ax2) = plt.subplots(nrows=1,ncols=2,sharey=True,figsize=(12,5))
    #plt.title("Figure 4")

    # Plots means of deviations
    ax1.barh(*zip(*mean_dict.items()))
    ax1.set_ylabel(group)
    ax1.set_xlabel('Model deviation')
    ax1.grid(axis='x')
    ax1.set_title('Figure 4: Deviation between the model and the actual ratings')

    # Plots count of movies
    ax2.barh(*zip(*count_dict.items()))
    ax2.set_xlabel('Number of observations')
    ax2.grid(axis='x')
    ax2.set_title('Figure 5: Number of observation')

# Calls merge-function
df_merge = merge_df(df_all)

# Runs interactive figure
mean_genre = widgets.interact(_mean_genre,
                             df = widgets.fixed(df_merge),
                             group = widgets.Dropdown(
                                 options = ['Genres','Decades','Duration','True rating'],
                                 description = 'Group',
                                 value = 'Genres'),
                             diff = widgets.RadioButtons(
                                 options = ['Average', 'Absolute'],
                                 description = 'Difference',
                                 value = 'Average'))

```

Figure 4 plot the mean of deviations based on genres, duration, decades, and true ratings. Figure 5 plot the number of movies in the dataset based on the same groups. From these graphs we see a clear correlation between precision and number of movies in the dataset. The more movies of a given genre, duration, etc. the more precise the prediction of movie-ratings in these groups. Take drama-movies as an example. The mean of the deviation is close to zero, and movies of this genre is also quite overrepresentated in the dataset. The complete opposite case is short-movies. But we also see that sci-fi-movies are quite well predicted, even though there aren't relatively many of these movies in the dataset. The same is seen in other different genres. This general image is seen when we group movies on decades, duration, and true rating as well.

When looking at absolute differences instead, the correlation is not as strong, especially not for the genres.

The next section will look at the changes in the estimates when the model is solved for one decade at a time.

```python
## The following code produces the parameter estimates where the sample is restricted to all the decades,
## one at the time. The function return a list which contains a list of the estimates for every decade.

#### OBS: The code will take about 3-4 minutes to run, since the model is estimated 10 times!! ####

def optimizer_dec():
    """ Generates a list containing 10 list with estimates of the model for every decade.

    Args:
        No arguments are needed.

    Returns:
        A list.

    Notice:
        It will raise an error if the function is given an argument!
    """

    result = [] # Empty list to store the estimated parameters

    # Make a loop so the model is estimated for all ten decades
    for j,decade in enumerate(decade_list,1):
        df_X, df_Y = df_s(keep_top=None, decade=decade) # Call the function to generate the two dataframes
        x0 = np.zeros(len(vars_dec)) # Starting values

        obj_fun = lambda x: sqr_diff_sum(df_X,x) # The objective function -> sum of squared differences
        start = time.time()

        # Use Scipy optimizer to solve the model
        print(f'Opmization {j} out of {len(decade_list)}: {decade}')

        result_i = optimize.minimize(obj_fun,x0,
                               method='Nelder-Mead',
                               options={"disp":True, "maxiter":50000}, # display the results
                               );

        end_time = time.time()-start
        print(f'{" ":9s}Time: {end_time:.4f} seconds')

        # Add the result for each deacde to the result-list
        result.append(list(result_i.x))

    return result # Returns the result-list

# Call the optimize_dec function
result_dec = optimizer_dec()
```

```python
## The following code produces a figure with the estimated parameters for each deacde for a chosen variable

result_dec_mod = [] # Empty list to storage

# The loop changes the order of the result-list so the
# estimates are ordered by the variables and subordered by decade
# insted of being ordered by decade and subordered by variables
for j,var in enumerate(vars_dec):
    temp = []
    for i,dec in enumerate(decade_list):
        temp.append(result_dec[i][j])

    result_dec_mod.append(temp)

# Defining a figure to plot the estimates
def fig(var):
    """ Generates a figure which plots estimated parameters for each decade for one variable

    Args:
        var (type: string): Should be one of the variables in the X-vector

    Returns:
        One interactive plot.

    Notice:
        The function is generated so that it can be called using widgets.interact()
        Thus, it is not intended to be used on its own.
        """
    # Initalize figure
    fig = plt.figure(figsize=(8,6))
    ax1 = fig.add_subplot(1,1,1)

    # Plotting variables for chosen decade
    ax1.bar(decade_list, result_dec_mod[vars_dec.index(var)],width=6)

    # Setting labels, ticks etc.
    ax1.set_ylabel('Parameter estimates')
    ax1.set_title(f'Figure 6: Parameter estimates for {var} per decade')
    ax1.set_xticks(decade_list)
    ax1.axhline(y=0,color='black',linewidth=1)

# Making the figure interactive so the estimates are shown for the chosen variable
widgets.interact(fig,
    var = widgets.Dropdown(description='Variable', value='Action', options=vars_dec,
                ),
);
```

Figure 6 shows the final robustness check, how the estimates vary when the sample only contains movies from one deacde. Going through the different variables, it is hard to find a generel trend. E.g. the estimates of action movies are negative in some decades and positive in other decades. On the other hand, the estimates of war movies are negative for all but two decades, and duration has positive estimates in all decades.

The messy picture could be caused by too small samples whereby the estimates will be very volatile. Another reason could be that the preferences change. By looking at the estimates of drama movies you could make the arguement that the the genre of drama affect the ratings positively for old movies, but the opposite after the 1950s. This could be an indication that people have less preferences for newer drama movies compared to older drama movies.

## Conclusion

Based on the above results, it appears that duration increases ratings by quite a lot, while Nr. of nominations and Nr. of wins doesn't appear to play a role. Meanwhile, the effect of different genres and decades are quite different. We see that genres like Action and Horror have very little effect on ratings, while the Fantasy movies appear to have a signifiant positive marginal effect on ratings.

Our estimates are not precise though, but this is to be expected. It is very difficult to create a model that predicts ratings like these 100 pct. right (if it is even possible with the given variables). Our robustness checks shows, that estimates are volatile to whether only top movies are included and how many of these are included, as shown in figure 4. This volatility is probably a combination of model misspecification and changing preferences. Additionally, genres could have very different distributions (e.g. the best movies are often action-movies, but there are A LOT of bad action movies as well).

A way to deal with this volatility could be to consider another functional form of the model or maybe find other variables that could improve how well the model predicts ratings.

We hope you enjoyed our movie model project, and don't think we are completely idiots. Because as our model we conclude that *much to learn, we still have*.


# simbMoments

*simbMoments* determines a system of equations corresponding to the first and second moments of the population observations. The process to find each moment is quite similar to the way it was done to find the system of differential equations using *simbODE*. The equations are sympy objects which one can manipulate to compute some statistics from the whole population. The implemented function just determines upon the second moment. The first moment is equivalent to the mean behavior of the system output, while the second expresses the cross dependence of the species and is equivalent to the variance of the system output.

```
#libraries required
import numpy as np
import sympy as sp
```

**Defines System Properties**

```
#molecular species
species = ['x1', 'x2']
species = sp.var(species)

#system input
inp = ['u']
uh = sp.var(inp)
ruh = 0  #reaction affected by input (0 = 1st reaction)

#reagent and product matrices
reactants = np.array([[0, 1, 1, 0],
                      [0, 0, 0, 1]])
products = np.array([[1, 0, 1, 0],
                     [0, 0, 1, 0]])

#kinetic parameters
pars = ['c1', 'c2', 'c3', 'c4']
parsValues = sp.var(pars)
#to replace kinetic parameters for numeric values, uncomment the next line
#and comment the two previous ones
#parsValues = [4.0, 0.010, 1.0, 0.006]
```

**Pre-processing of the System**. From previous defined information determines stoichiometric matrix and propensity vector. These arrays are used to compute
the moments of the system.

```
#stoichiometric matrix
V = products - reactants

#pre-propensity function
aPro = parsValues
aPro[ruh] = aPro[ruh]*uh[0]  #Attaches system input

#system dimentions
Sn, Rm = V.shape

#propensity function vector
for r in range(0,Rm):
    for s in range(0, Sn):
        #determines propensity vector expressions
        for a in range(0, reactants[s,r]):
            aPro[r] *= species[s]
        #end for a
    #end for s
#end for r
print("Stoichimotric Matrix:\n", V)
print("Propensity Function Vector:", aPro)
```

    Stoichimotric Matrix:
     [[ 1 -1  0  0]
     [ 0  0  1 -1]]
    Propensity Function Vector: [c1*u, c2*x1, c3*x1, c4*x2]

**Computes Moments Equations**. Each species has its own first moment equation and second orden moment equation. The second moment is found for itself and crossed with other species.

*First Moment*

```
#System of First Moment Equations
odeX = []
for s in range(0,Sn):
    temp = 0
    #Determines Defferential Equations
    for r in range(0, Rm):
        temp += V[s,r]*aPro[r]
    #end for r
    #Set of Differential Equations
    odeX.append(temp)
#end for s
```

*Second Order Moment*

```
#System of Second Order Equations
ode2m = []
name2m = []
nameODE = species
odeTotal = odeX
for s1 in range(0, Sn):
    for s2 in range(0, Sn):
        #Determines second order expression
        temp = 0
        for r in range(0,Rm):
            temp += (V[s1,r]*aPro[r]*species[s2] + V[s2,r]*aPro[r]*species[s1] \
                     + V[s1,r]*V[s2,r]*aPro[r])
        #end for r

        #set of second order moment equations
        if temp not in ode2m:
            ode2m.append(temp)
            odeTotal.append(temp)
        #end if temp

        #variable names of second order species
        if (species[s1]*species[s2]) not in name2m:
            name2m.append(species[s1]*species[s2])
            nameODE.append(species[s1]*species[s2])
        #end if species s1*s2
    #end for s2
#end for s1
```

Some processing of the determined data to make easy posterior manipulation

```
#replaces variable names
dxName = []
dxODE = []
for exp in odeTotal:
    #at each expression searches for the variable names to replaces them
    #for a nickname "dx#"
    for j in range(0,len(nameODE)):
        name = sp.var('dx' + str(len(nameODE)-j))
        exp = exp.subs(nameODE[len(nameODE)-j-1],name)

        #stores nicknames
        if name not in dxName:
            dxName.append(name)
        #end if name
    #end for j
    dxODE.append(exp)
#end for exp
dxName.reverse()
```

```
#Shows sistem of differential equations determined
for k in range(0,len(odeTotal)):
    print(f'd{nameODE[k]}/dt:', odeTotal[k])
 #     print(f'd({dxName[k]})/dt:', dxODE[k])
 #     print('\n')
 #end for k
```

    dx1/dt: c1*u - c2*x1
    dx2/dt: c3*x1 - c4*x2
    dx1**2/dt: 2*c1*u*x1 + c1*u - 2*c2*x1**2 + c2*x1
    dx1*x2/dt: c1*u*x2 - c2*x1*x2 + c3*x1**2 - c4*x1*x2
    dx2**2/dt: 2*c3*x1*x2 + c3*x1 - 2*c4*x2**2 + c4*x2


## Imports

```python
import numpy as np
import pylab as plt
from matplotlib import cm
import seaborn as sns
from tqdm import tqdm
```

## System

> NLSE differential equation

\begin{equation}
\partial_z A + \frac{\alpha}{2}A+\frac{i}{2}\beta_2 \partial_T^2 A = i \gamma \| A\|^2 A
\end{equation}

For now, let's suppose $\alpha=0$

\begin{equation}
\partial_z A = (\hat{D}+\hat{N})A
\end{equation}

with $\hat{D}=-\frac{i}{2}\beta_2 \partial_T^2$ and $\hat{N}=i \gamma \| A\|^2$

The operator $\hat{D}$ will be computed in the fourier domain such that $\hat{D}(\omega)=\frac{i}{2}\omega^2\beta_2$

Finally we have at first glance :

\begin{equation}
A(z+h, T)=e^{h\hat{N}} F^{-1} \cdot e^{h\hat{D}(\omega)} \cdot F \cdot A(z,t)
\end{equation}

Where $F$ is the Fourier operator, $h$ is the propagation step (of a few meters) and $\beta_2=-D\lambda^2/2\pi c$ wich is related to the optical fiber chromatic dispersion.

> Source : doi:10.1109/ICEE.2007.4287333

```python
# Constants

c=3e8 # light celerity
l0 = 1.55e-6 # wavelength
nm = 1e-9 # nanometer
ns = 1e-9
km = 5e3 #kilometer
ps = 1e-12 #picosecond
D = 17*ps/nm/km # Dispersion
b2 = -D*l0**2/(2*np.pi*c) # group velocity dispersion (2nd order)
w0 = 2*np.pi*1e10
# Parameters

alpha=0#0.00005 # Losses
gamma=0.78e-3 # [1]/m/W Nonlinear factor https://ieeexplore.ieee.org/document/7764544

L = 5e3 # Fiber length

Nt= 10000 # Time sampling
Nl = 5000 # Length sampling

# Calculated factors and vectors

h = L/Nl # Lengthstep
dT = 0.1*ps # FWHM
T = np.linspace(-50, 50, Nt)*ps # Pulse local time vector
z = np.arange(0, Nl, 1)*h # Propagation distance vector
P0=120 # W
n = 0.03

dt = T[1]-T[0] # timestep
w =  np.fft.fftshift(np.fft.fftfreq(Nt, d=dt)) # Pulsation vector # ?? 2*np.pi*c/l0 +
Dw = 0.5*1j*w**2*b2 # Calculated dispersion operator

Noise = np.random.randn(1,Nt) # Amplitude noise vector

A =np.asarray(np.zeros((Nl, Nt), dtype=complex)) # System matrix
B =np.asarray(np.zeros((Nl, Nt), dtype=complex))
A[0,:]= n*np.sqrt(P0)*Noise + 0.5*np.sqrt(P0)*(np.exp(-T**2/(10*(dT)**2))) # Initial state
A[0, :] +=  np.sqrt(P0)*(np.exp(-(T-10*ps)**2/(2*(dT)**2)))
A[0, :] +=  np.sqrt(P0)*(np.exp(-(T+10*ps)**2/(2*(dT)**2)))
#A[0,:]= n*np.sqrt(P0)*Noise + np.sqrt(P0)*np.sin(w0*T) # Initial state
B[0,:]=np.fft.fftshift(np.fft.fft(A[0,:]))

```

```python
plt.plot(T,np.abs(A[0,:]))
plt.title("Initial pulse")
plt.show()

```

```python
for i in tqdm(range(1,Nl)):

    N = 1j*gamma*np.abs(A[i-1,:])**2-0.5*alpha
    Ai = np.exp(0.5*h*N)*A[i-1,:] # half Nonlinearity 1
    Ai = np.fft.fftshift(np.fft.fft(Ai)) # Fourier domain
    Ai = np.exp(h*Dw)*Ai # Dispersion in Fourier domain
    Ai = np.fft.ifft(np.fft.ifftshift(Ai)) # Temporal domain
    Ai = np.exp(0.5*h*N)*Ai # half Nonlinearity 2
    A[i,:] = Ai
    B[i,:] = np.fft.fftshift(np.fft.fft(Ai))
```

    100%|█████████████████████████████████████████████████████████████████████████████| 4999/4999 [00:30<00:00, 164.10it/s]

```python
f = w/(2*np.pi)
extent = [f[0], f[-1], z[-1], z[0]]
plt.imshow(np.abs(A), aspect=1, cmap='jet')
plt.xlabel(r"f (Hz)")
plt.ylabel(r"L (m)")

plt.tight_layout()
plt.show()
```

```python
plt.plot(T/ps,np.abs(A[-1,:]), label='output')
plt.plot(T/ps,np.abs(A[0,:]), label='input')

plt.grid()
plt.legend()
plt.xlabel("Pulse time (ps)")
plt.ylabel("Pulse Power (W)")
plt.show()

plt.figure()
plt.plot(T/ps,np.real(A[-1,:]), label='output')
plt.plot(T/ps,np.real(A[0,:]), label='input')

plt.grid()
plt.legend()
plt.xlabel("Pulse time (ps)")
plt.ylabel("Pulse Power (W)")

plt.show()

```

```python
w[0]
```

    -4999499999998.172

```python
Dw
```

    array([-0.-0.05415821j, -0.-0.05413655j, -0.-0.0541149j , ...,
           -0.-0.05409324j, -0.-0.0541149j , -0.-0.05413655j])

```python
N
```

    array([0.+4.25759166e-08j, 0.+1.08248965e-08j, 0.+7.47240172e-08j, ...,
           0.+1.06654191e-07j, 0.+4.60674140e-09j, 0.+9.80428558e-08j])

```python
h
```

    50.0

```python
c/l0
```

    193548387096774.2

```python
z[-1]
```

    4997.5

```python

```


```python
from logicqubit.logic import *
from cmath import *
import numpy as np
import sympy as sp
from scipy.linalg import expm, sinm, cosm, logm, sqrtm
```

    Cuda is not available!
    logicqubit version 1.5.5

```python
def qft(qr):
    for i in range(len(qr)):
        for j in range(i):
            qr[i].CU1(qr[j], pi/float(2**(i-j)))
        qr[i].H()

def iqft(qr):
    for i in range(len(qr)):
        for j in range(i):
            qr[i].CU1(qr[j], -pi/float(2**(i-j)))
        qr[i].H()

def swap(s1, s2):
    s2.CX(s1)
    s1.CX(s2)
    s2.CX(s1)
```

```python
logicQuBit  = LogicQuBit(2, symbolic=False)

# RY = [[cos(theta/2), -sin(theta/2)],
#       [sin(theta/2), cos(theta/2)]]

x = Qubit()
a = Qubit()

x.RY(pi/4)
x.H()
a.CRY(x,pi/2)
a.CX(x)

x.getDictPsi()
```

    {'00': (0.9238795325112866+0j),
     '01': 0j,
     '10': (0.27059805007309845+0j),
     '11': (0.2705980500730985+0j)}

```python
logicQuBit.setSymbolValuesForAll(1,0)
logicQuBit.PrintState()
```

    This session is not symbolic!

$\displaystyle \begin{pmatrix}0.92388 \\ 0 \\ 0.270598 \\ 0.270598 \end{pmatrix}$

```python

#res_a = logicQuBit.Measure_One(a)
#print(res_a)
res = logicQuBit.Measure([a]) # x = medir(b)
logicQuBit.Plot()
print(res)
```

```python
logicQuBit.PlotDensityMatrix()
```

```python

```


# メモ

いまはsympyのtutorialを読んでいるところ。

次はcalculus微積分

sympyのmanualはむずかしそうなので考える。
そんな意味ではtutorialも難しい。
ほんとは数学と平行してやらないとあまり意味がないが、とりあえず写経をしておいて、数学はあとから考える。
数学をやっていて道具がないと勉強できない！！
とおもったからsympyをやっているのだよ。
だからある程度の行ったり来たりせざるを得ない。

1. 本家`SymPy Tutorial` (https://docs.sympy.org/latest/tutorial/index.html#tutorial)
1. 入力例で学ぶPython(SymPy)の使い方(入門) (https://pianofisica.hatenablog.com/entry/2019/04/04/183712)
1. `Doing Math With Python` PDF ( http://index-of.es/Varios-2/Doing%20Math%20with%20Python.pdf)
1. `Doing Math With Python` site (http://www.nostarch.com/doingmathwithpython/ )

# calculus 微積分

```python
from sympy import *
init_printing()
x, y, z = symbols('x y z')
display(diff(sin(x)))
display(diff(cos(x)))
```

```python
display(Eq(Derivative(sin(x)), diff(sin(x))))
print()
display(Eq(Derivative(cos(x)), diff(cos(x))))
```

# 偏微分 partial differentiation

```latex
%%latex
\displaystyle
\frac{\partial ^7}{\partial x \partial y^2 \partial z^2} e ^{xyz}
```

\displaystyle
\frac{\partial ^7}{\partial x \partial y^2 \partial z^2} e ^{xyz}

```python
from sympy import *
init_printing()
x, y, z = symbols('x y z')
expr = exp(x)
display (expr)
display (diff(expr))
display(Eq(Derivative(expr), diff(expr)))
```

```python
from sympy import *
init_printing()
x, y, z = symbols('x y z')
expr = exp(x**2)
display(Eq(Derivative(expr), diff(expr)))
```

```python
expr = exp(x*y*z)
display(diff(expr,x,y,z))
display(Eq(Derivative(expr,x,y,y,z,z,z,z), diff(expr,x,y,y,z,z,z,z)))
```

```python
diff(expr,x,y,z)
```

```python
diff(expr,x,y,2,z,4)
```

```python
diff(expr,x,y,y,z,4)
```

### 未評価の導関数を作る

導関数クラスを使う。

```python
from sympy import *
init_printing()
x, y, z = symbols('x y z')
expr = exp(x*y*z)

deriv = Derivative(expr, x, y, y, z, 4)
deriv
```

###  未評価の導関数を評価する

`doit`メソッドを使う。

```python
deriv.doit()
```

```python
Derivative((x**2),x).doit()
```

###  Derivatives of unspecified order

Derivatives of unspecified order can be created using tuple (x, n) where n is the order of the derivative with respect to x

```python
from sympy import *
init_printing()
m, n, a, b = symbols('m n a b')
expr = (a*x + b)**m
expr.diff((x,n))
```

### Integrals 積分

```python
integrate(cos(x),x)
```

```python
Integral(cos(x),(x,0,oo))
```

```python
integrate(cos(x),(x,0,oo))
```

```python
integrate(exp(-x),(x,0,oo))
```

```python
Integral(exp(-x**2 -y**2),(x, -oo, oo), (y,-oo,oo))
```

```python
integrate(exp(-x**2 - y**2), (x, -oo, oo), (y, -oo, oo))
```

```python
expr = integrate(x**x, x)
print(expr)
display(expr)
```

```python
expr
```

```python
expr = Integral(log(x)**2,x)
expr
```

```python
expr.doit()
```

```python
integ = Integral((x**4 + x**2*exp(x) - x**2 - 2*x*exp(x) - 2*x - exp(x))*exp(x)/((x - 1)**2*(x + 1)**2*(exp(x) + 1)), x)
integ
```

```python
integ.doit()
```

```python
integ = Integral(sin(x**2),x)
integ
```

```python
integ.doit()
```

```python
integ = Integral(x**y*exp(-x), (x,0,oo))
integ
```

```python
integ.doit()
```

$\displaystyle \begin{cases} \Gamma\left(y + 1\right) & \text{for}\: \operatorname{re}{\left(y\right)} > -1 \\\int\limits_{0}^{\infty} x^{y} e^{- x}\, dx & \text{otherwise} \end{cases}$

# limits 極限

$$ \lim_{x \to x_0} f (x) $$

```latex
%%latex
\displaystyle

\lim_{x \to x_0} f (x)
```

\displaystyle

\lim_{x \to x_0} f (x)

```python

Limit(sin(x)/x, x, 0)
```

```python
Limit(sin(x)/x, x, 0).doit()
```

```python
expr = x**2/exp(x)
expr.subs(x,oo)
```

$\displaystyle \text{NaN}$

```python
limit(expr,x,oo)
```

```python
expr = Limit((cos(x) - 1)/x, x, 0)
expr
```

```python
expr.doit()
```

```python
limit(1/x,x,0,'+')
```

```python
limit(1/x,x,0,'-')
```

# series expansion 数列展開

asymptotic series -> 漸近数列

`SymPy`はある点における関数の漸近数列を計算できる。

点`x=x0`の近辺でタームズ・オブ・オーダーを `xn`とする関数 `f(x)` について、`f(x).serries(x, x0, n)` を使う。 `x0` と `n` は省略可能で、その場合 `x0=0`、 `n=6` が使われる。

SymPy can compute asymptotic series expansions of functions around a point. To compute the expansion of f(x) around the point x=x0 terms of order xn, use f(x).series(x, x0, n). x0 and n can be omitted, in which case the defaults x0=0 and n=6 will be used.

```python
expr = exp(sin(x))
expr.series(x, 0, 4)
```

#  Landau order ランダウ・オーダー

終点における $ O(x^4) $ タームは $x=0$ におけるランダウ・ターム・オーダーをあらwしている。コンピューターサイエンスにおけるビッグオー記法と一緒にしてはいけない。ビッグオーは一般的に `x=∞` におけるランダウ・ターム・オーダーを表す。
 $ O(x^4) $ タームは `x4` もしくはそれ以上の冪乗の `x` タームは無視するという意味で、オーダー・タームは数列の外で作られ、操作される。高次のタームは自動的に吸収される。

The $ O(x^4) $ term at the end represents the Landau order term at $x=0$ (not to be confused with big O notation used in computer science, which generally represents the Landau order term at x=∞). It means that all x terms with power greater than or equal to x4 are omitted. Order terms can be created and manipulated outside of series. They automatically absorb higher order terms.

```python
x + x**3 + x**6 + O(x**4)
```

```python
x*O(1)
```

```python
expr.series(x,0,4).removeO()
```

### オー記法は、ゼロ以外の任意の極限点をサポートする

The O notation supports arbitrary limit points (other than 0):

```python
exp(x - 6).series(x, x0=6)
```

# Finite differences 有限差分

以上見てきたのは解析微分 `analytic derivative` と原始関数 `primitive function` であったが、閉鎖的な表現を持たない曲線の微分をしたいときにはどうするか。また、そもそも関数的な値を持っていない場合はどうするか。

そういう場合のアプローチのひとつが有限差分アプローチである。

有限差分を使用して微分する最も簡単な方法は、`differentiate_finite` 関数を使う。

```python
f, g = symbols('f g', cls=Function)
differentiate_finite(f(x)*g(x))
```

中間導関数を拡張する場合は、フラグevaluate = Trueを渡す。

```python
differentiate_finite(f(x)*g(x), evaluate=True)
```

ただし、このフォームは `product` ルールを無視する。

すでに導関数インスタンスがある場合は、as_finite_differenceメソッドを使用して、導関数の任意のオーダーの近似を生成できる。

```python
f = Function('f')
dfdx =f(x).diff(x)
dfdx.as_finite_difference()
```

ここで、1次導関数は、ステップサイズ1を使用して等距離で評価された最小数の点（1次導関数の場合は2）を使用してxの周りで近似された。

記号式を含む任意のステップを使用できる。

```python
f = Function('f')
dfdx2 =f(x).diff(x, 2)
h = Symbol('h')
dfdx2.as_finite_difference([-3*h, -h, 2*h])
```

重みの評価だけに関心がある場合は、手動で行うこともできる。

```python
finite_diff_weights(2, [-3,-1,2], 0)[-1][-1]
```

finite_diff_weightsから返される最後のサブリストの最後の要素のみが必要であることに注意。 これは、関数が低次の導関数の重みを生成し、使用するポイントが少ないため。

finite_diff_weightsが複雑に見え、導関数インスタンスのas_finite_differenceメソッドの柔軟性が十分でない場合は、order、x_list、y_list、x0をパラメーターとして取るapply_finite_diffを使用しよう。

```python
x_list = [-3, 1, 2]
y_list = symbols('a b c')
apply_finite_diff(1, x_list, y_list, 0)
```

# solvers ソルバー

```python
from sympy import *
x, y, z = symbols('x y z')
init_printing()

Eq(x , y)
```

```python
solveset(Eq(x**2, 1))
```

```python
solveset(Eq(x**2 -1, 0))
```

```python
solveset(x**2 -1)
```

### 代数的に方程式を解く

`solveset(eqation, variable=None, domain=S.Complexes)`

`solve(equations, variables)` の代わりに `solveset`を使う。

`solveset` の返り値は `FiniteSet`、もしくは解答の `Interval` か `ImageSet` である。

```python
solveset(x**2 - x, x)
```

```python

```

```python
from sympy import *
s = solveset(x**2 - x, x)
type(s)
```

    sympy.sets.sets.FiniteSet

```python
solveset(x - x, x , domain=S.Reals)
```

```python
type (solveset(x - x, x , domain=S.Reals))
```

    sympy.sets.fancysets.Reals

```python
from sympy import *
x = symbols('x')
solveset(sin(x) - 1, x, domain=S.Reals)
```

$\displaystyle \left\{2 n \pi + \frac{\pi}{2}\; |\; n \in \mathbb{Z}\right\}$

```python
solveset(exp(x),x) # No solution exists -> EmptySet
```

```python
solveset(cos(x) -x, x) # Not able to find solution -> ConditonSet
```

# 線形方程式 linear equations

`linsolve`を使う。

```python
linsolve([x+y+z-1, x+y+2*z-3],(x,y,z))
```

```python
linsolve?
```

```python
from sympy import *
x, y, z = symbols('x y z')
init_printing()
linsolve(Matrix(([1,1,1,1],[1,1,2,3])),(x,y,z))
```

```python
# A*x = b Form
M = Matrix(((1,1,1,1),(1,1,2,3)))
system = A, b = M[:, :-1], M[:, -1]
linsolve(system, x, y, z)
```

### 非線形解 nonlinsolve

solvesetモジュールでは、非線形システムの方程式は`nonlinsolve`を使用して解く。

```python
# when only real solution is present
from sympy import*
init_printing()
x, y, z = symbols('x y z')
a,b,c,d = symbols('a b c d', real=True)

nonlinsolve([a**2 + a, a - b], [a, b])
```

```python
nonlinsolve([x*y - 1, x - 2], x, y)
```

```python
# when only complex solution is present
nonlinsolve([x**2 + 1, y**2 + 1], [x, y])
```

```python
# when both real and complex solution is present
from sympy import sqrt
system = [x**2 - 2*y**2 - 2, x*y - 2]
vars = [x, y]
nonlinsolve(system, vars)
```

```python
n = Dummy('n')
system = [exp(x) - sin(y), 1/y - 3]
real_soln = (log(sin(S(1)/3)), S(1)/3)
img_lamda = Lambda(n, 2*n*I*pi + Mod(log(sin(S(1)/3)), 2*I*pi))
complex_soln = (ImageSet(img_lamda, S.Integers), S(1)/3)
soln = FiniteSet(real_soln, complex_soln)
nonlinsolve(system,[x,y]) == soln
```

```python
real_soln
```

```python
complex_soln
```

方程式の非線形システムが正の次元システムである場合（無限に多くの解を持つシステムは正の次元であると言われる）

```python
nonlinsolve([x*y, x*y - x],[x, y])
```

```python
system = [a**2 + a*c , a - b]
nonlinsolve(system , [a, b])
```

解のオーダーが、与えられたシンボルのオーダーに対応しているとき。

ランベルトのWの形での解が存在しても、現在の`nonlinsolve`はランベルトのWの形での解を返さない。

このような時は`solve`を使う。

```python
solve([x**2 - y**2/exp(x)],[x,y], dict=True)
```

現在`nonlinsolve`は、三角関数をもつ連立方程式を適切に解くことができない。

このような時は必ずしも全部ではないが`solve`が使える。

```python
solve([sin(x+y), cos(x-y)], [x,y])
```

重複解の場合

`solveset` は解を1回しか返さない。重複解を持つ、多項式の解を得るには、`roots` を使用する。

```python
solveset(x**3 - 6*x**2 + 9*x, x)
```

```python
roots(x**3 - 6*x**2 + 9*x, x)
```

出力 `{0：1、3：2}` の意味は、`0` が多重度1の解であり、`3` が多重度2の解である、ということ。

現在 `solveset` は次のタイプの方程式を解くことができない。

LambertW（超越方程式ソルバー）で解ける方程式

このような時には `solve` が使える。

```python
solve(x*exp(x) - 1, x)
```

# 微分方程式を解く
differential equations

```python
from sympy import*
init_printing()
x, y, z = symbols('x y z')
f, g = symbols('f g' , cls=Function)

f(x)
```

```python
f(x).diff(x)
```

したがって、次のような微分方程式は `sympy`では次のセルのように書くことになる。

$$f′′(x)−2f′(x)+f(x)=sin(x)$$

```python
diffreq = Eq(f(x).diff(x,x) - 2*f(x).diff(x) + f(x), sin(x))
diffreq
```

# 常微分方程式 ordinary differential equation, ODE

常微分方程式を解くには、`dsolve` を使う。

```python
dsolve(diffreq, f(x))
```

```python
dsolve(f(x).diff(x)*(1 - sin(f(x))), f(x))
```

# 行列 matrices

```python
from sympy import *
init_printing()
Matrix([[1,-1],[3,4],[0,2]])
```

$\displaystyle \left[\begin{matrix}1 & -1\\3 & 4\\0 & 2\end{matrix}\right]$

# いまここ
行列式の３つの定義と意味
https://mathtrain.jp/determinant

## 行列式

行列式とは，正方行列に対して決まる量 (スカラー) である。行列 A の行列式を $det A$ や $|A|$ と表す。

```python
# 行列式
a11, a12, a13, a21, a22, a23, a31, a32, a33 = symbols('a11, a12, a13, a21, a22, a23, a31, a32, a33')
display(Eq(A, (Matrix([[a11, a12],[a21, a22]])), evaluate=False))
display(det((Matrix([[a11, a12],[a21, a22]]))))
display(Eq(B, (Matrix([[a11, a12,a13],[a21, a22,a23],[a31,a32,a33]])), evaluate=False))
display(det((Matrix([[a11, a12,a13],[a21, a22,a23],[a31,a32,a33]]))))
```

```python
# 行列式
a11,a12,a13,a14,a21,a22,a23,a24,a31,a32,a33,a34,a41,a42,a43,a44 = symbols('a11,a12,a13,a14,a21,a22,a23,a24,a31,a32,a33,a34,a41,a42,a43,a44')
A=Matrix([[a11,a12],[a21,a22]])
display(det(A))
B=Matrix([[a11,a12,a13],[a21,a22,a23],[a31,a32,a33]])
display(det(B))
print()
C=Matrix([[a11,a12,a13,a14],[a21,a22,a23,a24],[a31,a32,a33,a34],[a41,a42,a43,a44]])
display(det(C))
```

```python
# 行ベクトル
Matrix([[1,2,3]])
```

$\displaystyle \left[\begin{matrix}1 & 2 & 3\end{matrix}\right]$

```python
# 列ベクトル
Matrix([1,2,3])
```

$\displaystyle \left[\begin{matrix}1\\2\\3\end{matrix}\right]$

```python
M = Matrix([[1,2,3],[3,2,1]])
N = Matrix([0,1,1])
M*N
```

SymPy` の行列は `immutable` ではない!!!!!

不変にしたい場合はImmutableMatrixを使う

```python
M = Matrix([[1,2,3],[-2,0,4]])
M
```

```python
M.shape
```

```python
M.row(0)
```

```python
M.col(-1)
```

```python
M.col_del(0)
M
```

```python
M.row_del(1)
M
```

```python
M = M.row_insert(1,Matrix([[0,4]]))
M
```

```python
M = M.col_insert(0,Matrix([1,-2]))
M
```

###  逆行列

逆行列は$-1$乗する

```python
M = Matrix([[1,3],[-2,3]])
N = Matrix([[0,3],[0,7]])
M + N
```

```python
M * N
```

```python
3 * M
```

```python
M * M
```

```python
M**2
```

```python
M**-1
```

```python
M * M**-1
```

```python
%xmode plain
```

```python
# N ** -1
# ValueError: Matrix det == 0; not invertible.
```

###  転置行列

```python
M = Matrix([[1,2,3],[4,5,6]])
M
```

```python
M.T
```

```python
eye(3)
```

```python
zeros(2,3)
```

```python
ones(3,2)
```

```python
diag(1,2,3)
```

```python
diag(-1, ones(2,2), Matrix([5,7,5]))
```

```python
M = Matrix([[1,0,1],[2,-1,3],[4,3,2]])
M
```

```python
M.det()
```

### 階段型(梯形)への変換

行列を階段型に変換するには `rrf` を使う。

```python
M = Matrix([[1, 0, 1, 3],[2, 3, 4, 7],[-1, -3, -3, -4]])
M
```

```python
M.rref()
```

### 零空間 nullspace

```python
M = Matrix([[1,2,3,0,0],[4,10,0,0,1]])
M
```

```python
M.nullspace()
```

### 列空間 columnspace

```python
M = Matrix([[1,1,2],[2,1,3],[3,1,4]])
M
```

```python
M.columnspace()
```

###  固有値 eigenvalue

enganvalues, eigenvectors, and diagonalization

```python
from sympy import *
init_printing()
M = Matrix([[3,-2,4,-2],[5,3,-3,-2],[5,-2,2,-2],[5,-2,-3,3]])
M
```

```python
M.eigenvals()
```

`M.eigenvals()` `{−2:1,3:1,5:2}`

この意味は行列 `M` は固有値、`-2`、`3`、`5` をもち、このうち `2`、`3` は代数的多重度が `1` で、固有値 `5` は代数的多重度が `2` である。

```python
M.eigenvects()
```

`M.eigenvect()` の結果は、固有値 `5` には2つの固有ベクトルがあるため、幾何多重度も `2` あることを示している。 代数的および幾何学的多重度はすべての固有値で同じであるため、Mは対角化可能。

行列を対角化するには、`diagonalize` を使用します。 `diagonalize` は、タプル`(P, D)` を返す。ここで、`D` は対角で、`M = PDP-1` である。

```python
P, D = M.diagonalize()
P
```

```python
D
```

```python
P*D*P**-1
```

```python
P*D*P**-1 == M
```

固有多項式 `characteristic polynomial` だけが必要な場合は、charpolyを使用しする。 シンボリックルートは計算にコストがかかる場合がかかるため、`eigenval` よりも効率的。

```python
lamda = symbols('lamda')
p = M.charpoly(lamda)
factor(p)
```

# 高度な式操作

advanced expression manipulation

式ツリー `expression tree`

```python
from sympy import *
init_printing()
x, y, z = symbols('x y z')

expr = 2**x + x*y
srepr(expr)
```

# graphviz

```python
import graphviz
g = graphviz.Digraph('G')
g.edge('Hello', 'World')
graphviz.Source(g)
```

# matplotlib

```python
% matplotlib inline

import numpy as np
import matplotlib.pyplot as plt

def sigmoid(x):
    return 1 / (1 + np.exp(x))

x = np.arange(-20.0, 20.0, 0.1)
y = sigmoid(x)
plt.plot(x, y)
plt.ylim(-0.1, 1.1)
plt.show()
```

```python
### symbol -> Symbol
from sympy import *
init_printing()
# x = Symbol('x')
x = symbols('x')
srepr(x)
```

```python
srepr(1)
```

```python
expr = 2.3**x
srepr(expr)
```

```python
srepr(Pow(2.3,x))
```

```python
type(simplify(2))
```

```python
x, y, z = symbols('x y z')
Mul(x,y)
```

```python
expr = sin(x*y)/2 - x**2 + 1/y
srepr(expr)
```

```python
srepr(x - y)
```

```python
expr = x / y
srepr(expr)
```

```python
1 + x
```

```python
(1 + x).func
```

```python
expr = Add(x, x)
expr.func # Mul
```

```python
expr
```

# クラス・コンストラクター

`Add(x, x)` は `x + x` という意味であるが、自動的に `Mul(2, x)` に変換される。つまり `2*x` である。 `SymPy` は `__new__` クラス・コンストラクターをヘビーに使う。`__new__` クラス・コンストラクターは `__new__` とは違い、コンストラークターから違うクラスを返すことができる。

```python
Integer(2).func
```

```python
Integer(0).func # Zero
```

```python
Integer(-1).func # NegativeOne
```

```python
expr = 3 * y**2 * x
expr.func # Mul
```

```python
expr.args # (3, x, y**2)
```

```python
expr.func(*expr.args)
```

```python
expr == expr.func(*expr.args)
```

```python
expr.func(Mul(expr.args))
```

```python
Mul(3, 8)
```

```python
# * (3, 8) # SyntaxError: can't use starred expression here
print(*(3,8)) # -> print (3, 8) -> 3 8
```

```python
expr.args[2]
```

```python
expr.args[2].args
```

```python
y.args
```

```python
Integer(2).args
```

# キー不変 `key invariant`

`SymPy` の式はすべて、空の引数を持つか、`expr == expr.func（* expr.args`）を満たす必要がある。

Pythonでは、`a` がタプルの場合、`f(* a)` は `a` の要素からの引数を使用して `f` を呼び出すことを意味する。たとえば、`f(*(1, 2, 3))` は `f(1, 2, 3)と同値である。

```python
56 * 35 * 21
```

```python
def pre(expr):
    print(expr)
    for arg in expr.args:
        pre(arg)

expr = x*y + 1
pre(expr)
```

# 前順探索走査 pre-order traversal

次に示すのは式ツリーの `pre-order traversal` 関数である。

`post-order traversal` が書けるかどうか考えること。

Can you guess why we called our function pre? We just wrote a pre-order traversal function for our expression tree. See if you can write a post-order traversal function.

```python
from sympy import *
init_printing()
x, y, z = symbols('x y z')
expr = x*y + 1

for arg in preorder_traversal(expr):
    print(arg)
```

```python
for arg in postorder_traversal(expr):
    print(arg)
```

```python
from sympy import Add
from sympy.abc import x, y, z
x + x
```

```python
Add(x, x, evaluate=False)
```

```python
sympify("x + x", evaluate=False)
```

```python
expr = Add(x, x, evaluate=False)
expr
```

```python
expr + x
```

```python
expr = x + UnevaluatedExpr(x)
expr
```

```python
x + expr
```

```python
(expr + x).doit()
```

```python
from sympy import *
from sympy.abc import x, y, z
uexpr = UnevaluatedExpr(S.One*5/7)*UnevaluatedExpr(S.One*3/4)
uexpr
```

```python
x*UnevaluatedExpr(1/x)
```

```python
UnevaluatedExpr(x + x)
```

```python
UnevaluatedExpr(sympify("x + x", evaluate=False)) + y
```

# いまここ  コピー中

# メモ

* mathwithpython01
* mathwithpython02
* mathwithpython03

は `SymPy Tutorial` をトレースしているが、いまの `SymPy Tutorial`  では最後にはない、`gotchas and pitfalls` が最後になっている。

```python
from sympy import *
from sympy.abc import *
init_printing()

x - y
```

```python
a = x - y
a
```

```python
(x+1)**2 -(x**2 + 2*x + 1)
```

```python
simplify ((x+1)**2 -(x**2 + 2*x + 1))
```

```python
eq = sin(2*x) - 2*sin(x)*cos(x)
display (eq)
display(simplify(eq))
```

```python
expand(eq, trig=True)
```

```python
trigsimp(eq)
```

```python
from sympy import Symbol
a = Symbol('a')

b = a + 1
b
```

```python
a = 4
b
```

```python
a + 1
```

```python
import sympy
sympy.var('z') # This is the easiest way to define z as a standard symbol
z**2
```

# sympyの対話型環境 isympy

ターミナルで `isympy` と打つと `sympy` の対話型環境が立ち上がる。Colabでは動かない。必要もない。

```
~$ isympy
IPython console for SymPy 1.1.1 (Python 3.6.3-64-bit) (ground types: gmpy)

These commands were executed:
>>> from __future__ import division
>>> from sympy import *
>>> x, y, z, t = symbols('x y z t')
>>> k, m, n = symbols('k m n', integer=True)
>>> f, g, h = symbols('f g h', cls=Function)
>>> init_printing()

Documentation can be found at http://docs.sympy.org/1.1.1/
```

# OSINEQ

ユーザーが使う変数やシンボルで、$I, E, S, N, O, Q$ は使わない。なぜなら、これらは複素数 ($i$)、自然対数 ($e$)、簡素化関数 ($S$)、数値評価関数 ($N$)、ビッグオーオーダーシンボル ($O(nlogn)$)、キーをサポートするオブジェクト (Q.real など)に使われているから。`OSINEQ`と覚える。

Lastly, it is recommended that you not use $I, E, S, N, C, O,$ or $Q$ for variable or symbol names, as those are used for the imaginary unit ($i$), the base of the natural logarithm ($e$), the sympify() function (see Symbolic Expressions below), numeric evaluation (N() is equivalent to evalf() ), the big O order symbol (as in $O(nlogn)$), and the assumptions object that holds a list of supported ask keys (such as Q.real), respectively. You can use the mnemonic OSINEQ to remember what Symbols are defined by default in SymPy.
###### C is deprecated

```python
from sympy import *
cos(pi)
```

```python
pi = 3
from sympy import pi
cos(pi)
```

```python
import sympy
# dir(sympy)
# dir(__builtins__)
# vars()
```

# 数値 numbers

数値を含む式は自動的にPythonの式にになってしまう。`SymPy`の式であることを確実にするためには、`symplify()`、もしくは `S()` とする必要がある。

```python
type(6.2)
```

```python
type(S(6.2))
```

```python
x = symbols('x')
x**(1/2)
```

```python
x**(S(1)/2)
```

```python
x**Rational(1,2)
```

```python
Rational(0.5)
```

```python
sqrt(x) == x**Rational(1,2)
```

```python
x**(2*x/3)
```

```python
x = symbols('x')
solve(7*x - 22 , x)
```

```python
22/7
```

```python
N(pi)
```

```python
a = solve(7*x - 22 , x)[0]
a
```

```python
22/S(7)
```

```python
S("22/7") # sympify it
```

```python
from __future__ import division
1/2 # integer division -> float division
```

```python
from sympy import var
r, t , d = var('rate time short_life')
d = r*t
d
```

```python
r = 80
t = 2
d # rate*time
```

```python
d = r *  t
d # 160
```

# いまここ

```python
c,d = var('c d')
def ctimesd():
    return c*d
ctimesd() # c*D
```

```python
c=2
d=8
ctimesd() # 16
```

```python
from sympy import *
init_printing()
x,y,z = symbols('x y z')
# Rational(2, x) -> error!!!
2/x
```

#  数学演算子 mathematical operators

```python
3 - x**(2*x)/(x + 1)
```

# 逆三角関数 nverse trig functions

`arcsin` 、 `arccos` ではなく、 `asin()`, `acos()` を使う。

# リストはイミュータブルではない

lists are mutable

```python
a = [x, 1]
a
```

```python
a[0] = 2
a
```

```python
solve(x**2 +2*x -1, x)
```

# 辞書 dictionaries, dict, {}

```python
d = {'a': 1, 'b': 2}  # A dictionary.
d
```

```python
from sympy import *
init_printing()
x = symbols('x')
roots((x - 1)**2*(x - 2), x)  # Some functions return dicts
```

```python
solve((x - 1)**2*(x - 2), x)
```

```python
(x - 1)**2*(x - 2)
```

```python
roots((x - 1)**2*(x - 2))
```

```python
roots((x - 5)**2*(x + 3), x)
```

# タプル

タプルはイミュータブルである。

`SymPy` では基本的にタプルは使わない。

```python
t = (1, 2, x)
t
```

```python
(x,)
```

```python
type ((x,))
```

```python
(x)
```

```python
integrate(x**2, (x, 0, 1))
```

```python
integrate?
```

```python
Integral(x**2, [x, 0, 1])
```

# キーワード引数 keyword argumebnts

```python
sqrt(x**2)
```

```python
x = symbols('x', positive=True)
sqrt(x**2)
```

```python
x, y, z, m, n = symbols('x y z m n')
x**n*x**m*y**n*y**m
```

```python
powsimp(x**n*x**m*y**n*y**m)
```

```python
simplify(x**n*x**m*y**n*y**m)
```

```python
powsimp(x**n*x**m*y**n*y**m, combine='all', force=True)
```

```python
powsimp(x**n*x**m*y**n*y**m, combine='exp')
```

```python
powsimp(x**n*x**m*y**n*y**m, combine='base', force=True)
```

# 指数の簡素化 powsimp

powsimp?

powsimp??

を読む。

```python
powsimp(x**y*x**z*y**z, combine='all')
```

```python
powsimp?
```

######  next
ここまででsympy documentationのtutorialとgotchas and pitfallsを読み終わった。

nextボタンを押したらsympy user's guideになった。
これはこれで面白そうというか勉強になるのだろうが、計画としてはmath with pythonに戻る予定である。
math with pythonがすばらしい本というわけではない。
jupyterでsympyをやりたいのだから、むしろネットでいろりろ検索してなでた方がいいと思う。

というわけですこし寄り道をします。

あと、jupyterの本を買ったのだが、bokehの本なので、とりあえずsympyではない。

```python
# 実験 https://docs.sympy.org/latest/modules/concrete.html より
from sympy.abc import i, k, m, n, x
from sympy import Sum, factorial, oo, IndexedBase, Function
# Sum(k, (k, 1, m))
# Sum(k, (k, 1, m))
# Sum(k, (k, 1, m)).doit()
# m**2/2 + m/2
# Sum(k**2, (k, 1, m))
# Sum(k**2, (k, 1, m))
# Sum(k**2, (k, 1, m)).doit()
# m**3/3 + m**2/2 + m/6
# Sum(x**k, (k, 0, oo))
# Sum(x**k, (k, 0, oo))
# Sum(x**k, (k, 0, oo)).doit()
# Piecewise((1/(1 - x), Abs(x) < 1), (Sum(x**k, (k, 0, oo)), True))
# Sum(x**k/factorial(k), (k, 0, oo)).doit()
# exp(x)

Sum(k, (k, 1, m))
```

```python
# 実験
from sympy import *
Sum(k, (k, 1, m))
```

# いまここ


```python
%matplotlib inline
```

Linear Regression with Regularization
=====================================

Regularization is a way to prevent overfitting and allows the model to
generalize better. We’ll cover the *Ridge* and *Lasso* regression here.

The Need for Regularization
---------------------------

Unlike polynomial fitting, it’s hard to imagine how linear regression
can overfit the data, since it’s just a single line (or a hyperplane).
One situation is that features are **correlated** or redundant.

Suppose there are two features, both are exactly the same, our predicted
hyperplane will be in this format:

\begin{align}\hat{y} = w_0 + w_1x_1 + w_2x_2\end{align}

and the true values of $x_2$ is almost the same as $x_1$ (or
with some multiplicative factor and noise). Then, it’s best to just drop
$w_2x_2$ term and use:

\begin{align}\hat{y} = w_0 + w_1x_1\end{align}

to fit the data. This is a simpler model.

But we don’t know whether $x_1$ and $x_2$ is **actually**
redundant or not, at least with bare eyes, and we don’t want to manually
drop a parameter just because we feel like it. We want to model to learn
to do this itself, that is, to *prefer a simpler model that fits the
data well enough*.

To do this, we add a *penalty term* to our loss function. Two common
penalty terms are L2 and L1 norm of $w$.

L2 and L1 Penalty
-----------------

0. No Penalty (or Linear)
~~~~~~~~~~~~~~~~~~~~~~~~~

This is linear regression without any regularization (from `previous
article </blog_content/linear_regression/linear_regression_tutorial.html#writing-sse-loss-in-matrix-notation>`__):

\begin{align}L(w) = \sum_{i=1}^{n} \left( y^i - wx^i \right)^2\end{align}

1. L2 Penalty (or Ridge)
~~~~~~~~~~~~~~~~~~~~~~~~

We can add the **L2 penalty term** to it, and this is called **L2
regularization**.:

\begin{align}L(w) = \sum_{i=1}^{n} \left( y^i - wx^i \right)^2 + \lambda\sum_{j=0}^{d}w_j^2\end{align}

This is called L2 penalty just because it’s a L2-norm of $w$. In
fancy term, this whole loss function is also known as **Ridge
regression**.

Let’s see what’s going on. Loss function is something we **minimize**.
Any terms that we add to it, we also want it to be minimized (that’s why
it’s called *penalty term*). The above means we want $w$ that fits
the data well (first term), but we also want the values of $w$ to
be small as possible (second term). The lambda ($\lambda$) is
there to adjust how much to penalize $w$. Note that ``sklearn``
refers to this as alpha ($\alpha$) instead, but whatever.

It’s tricky to know the appropriate value for lambda. You just have to
try them out, in exponential range (0.01, 0.1, 1, 10, etc), then select
the one that has the lowest loss on validation set, or doing k-fold
cross validation.

Setting $\lambda$ to be very low means we don’t penalize the
complex model much. Setting it to $0$ is the original linear
regression. Setting it high means we strongly prefer simpler model, at
the cost of how well it fits the data.

Closed-form solution of Ridge
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

It’s not hard to find a closed-form solution for Ridge, first write the
loss function in matrix notation:

\begin{align}L(w) = {\left\lVert y - Xw \right\rVert}^2 + \lambda{\left\lVert w \right\rVert}_2^2\end{align}

Then the gradient is:

\begin{align}\nabla L_w = -2X^T(y-Xw) + 2\lambda w\end{align}

Setting to zero and solve:

\begin{align}\begin{align}
   0 &= -2X^T(y-Xw) + 2\lambda w \\
   &= X^T(y-Xw) - \lambda w    \\
   &= X^Ty - X^TXw - \lambda w \\
   &= X^Ty - (X^TX + \lambda I_d) w
   \end{align}\end{align}

Move that to other side and we get a closed-form solution:

\begin{align}\begin{align}
   (X^TX + \lambda I_d) w &= X^Ty    \\
   w &= (X^TX + \lambda I_d)^{-1}X^Ty
   \end{align}\end{align}

which is almost the same as linear regression without regularization.

2. L1 Penalty (or Lasso)
~~~~~~~~~~~~~~~~~~~~~~~~

As you might guess, you can also use L1-norm for **L1 regularization**:

\begin{align}L(w) = \sum_{i=1}^{n} \left( y^i - wx^i \right)^2 + \lambda\sum_{j=0}^{d}\left|w_j\right|\end{align}

Again, in fancy term, this loss function is also known as **Lasso
regression**. Using matrix notation:

\begin{align}L(w) = {\left\lVert y - Xw \right\rVert}^2 + \lambda{\left\lVert w \right\rVert}_1\end{align}

It’s more complex to get a closed-form solution for this, so we’ll leave
it here.

Visualizing the Loss Surface with Regularization
------------------------------------------------

Let’s see what these penalty terms mean geometrically.

L2 loss surface
~~~~~~~~~~~~~~~

.. raw:: html

| |l2 surface|

.. raw:: html

This simply follows the 3D equation:

\begin{align}L(w) = {\left\lVert w \right\rVert}_2^2 = w_0^2 + w_1^2\end{align}

The center of the bowl is lowest, since ``w = [0,0]``, but that is not
even a line and it won’t predict anything useful.

L2 loss surface under different lambdas
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

When you multiply the L2 norm function with lambda,
$L(w) = \lambda(w_0^2 + w_1^2)$, the width of the bowl changes.
The lowest (and flattest) one has lambda of 0.25, which you can see it
penalizes The two subsequent ones has lambdas of 0.5 and 1.0.

.. raw:: html

| |l2 surface many lambdas|

.. raw:: html

L1 loss surface
~~~~~~~~~~~~~~~

Below is the loss surface of L1 penalty:

.. raw:: html

| |l1 surface|

.. raw:: html

Similarly the equation is
$L(w) = \lambda(\left| w_0 \right| + \left| w_1 \right|)$.

Contour of different penalty terms
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

If the L2 norm is 1, you get a unit circle ($w_0^2 + w_1^2 = 1$).
In the same manner, you get “unit” shapes in other norms:

.. raw:: html

| |norm contours|

.. raw:: html

**When you walk along these lines, you get the same loss, which is 1**

These shapes can hint us different behaviors of each norm, which brings
us to the next question.

.. |l2 surface| image:: imgs/img_l2_surface.png
.. |l2 surface many lambdas| image:: imgs/img_l2_surface_lambdas.png
.. |l1 surface| image:: imgs/img_l1_surface.png
.. |norm contours| image:: imgs/img_penalty_contours.png

Which one to use, L1 or L2?
---------------------------

What’s the point of using different penalty terms, as it seems like both
try to push down the size of $w$.

**Turns out L1 penalty tends to produce sparse solutions**. This means
many entries in $w$ are zeros. This is good if you want the model
to be simple and compact. Why is that?

Geometrical Explanation
~~~~~~~~~~~~~~~~~~~~~~~

*Note: these figures are generated with unusually high lambda to
exaggerate the plot*

First let’s bring both linear regression and penalty loss surface
together (left), and recall that we want to find the **minimum loss when
both surfaces are summed up** (right):

.. raw:: html

| |ridge regression|

.. raw:: html

Ridge regression is like finding the middle point where the loss of a
sum between linear regression and L2 penalty loss is lowest:

.. raw:: html

| |ridge regression sol 30|

.. raw:: html

You can imagine starting with the linear regression solution (red point)
where the loss is the lowest, then you move towards the origin (blue
point), where the penalty loss is lowest. **The more lambda you set, the
more you’ll be drawn towards the origin, since you penalize the values
of $w_i$ more** so it wants to get to where they’re all zeros:

.. raw:: html

| |ridge regression sol 60|

.. raw:: html

Since the loss surfaces of linear regression and L2 norm are both
ellipsoid, the solution found for Ridge regression **tends to be
directly between both solutions**. Notice how the summed ellipsoid is
still right in the middle.

--------------

For Lasso:

.. raw:: html

| |lasso regression|

.. raw:: html

And this is the Lasso solution for lambda = 30 and 60:

.. raw:: html

| |lasso regression sol 30|

| |lasso regression sol 60|

.. raw:: html

Notice that the ellipsoid of linear regression **approaches, and finally
hits a corner of L1 loss**, and will always stay at that corner. What
does a corner of L1 norm means in this situation? It means
$w_1 = 0$.

Again, this is because the contour lines **at the same loss value** of
L2 norm reaches out much farther than L1 norm:

.. raw:: html

| |l1 vs l2|

.. raw:: html

If the linear regression finds an optimal contact point along the L2
circle, then it will stop since there’s no use to move sideways where
the loss is usually higher. However, with L1 penalty, it can drift
toward a corner, because it’s **the same loss along the line** anyway (I
mean, why not?) and thus is exploited, if the opportunity arises.

.. |ridge regression| image:: imgs/img_ridge_regression.png
.. |ridge regression sol 30| image:: imgs/img_ridge_sol_30.png
.. |ridge regression sol 60| image:: imgs/img_ridge_sol_60.png
.. |lasso regression| image:: imgs/img_lasso_regression.png
.. |lasso regression sol 30| image:: imgs/img_lasso_sol_30.png
.. |lasso regression sol 60| image:: imgs/img_lasso_sol_60.png
.. |l1 vs l2| image:: imgs/img_l1_vs_l2_contour.png


# Estimation of exchange interactions
Author: [Takuya Nomoto](mailto:nomoto@ap.t.u-tokyo.ac.jp)

## Liechtenstein method

The Liechtenstein method is a method for deriving the classical spin models from the itinerant electron models. Initially developed by Liechtenstein et al.[1] in the framework of the spin density functional theory, the method has been applied to various systems and is successful in understanding many magnetic behaviors, including finite temperature effects and long-period magnetic structures, which are not accessible from the spin density functional theory alone. Here, as the simplest case, we will see the formulation for the single-orbital model, where we can use the sparse_ir library to accelerate the calculations.

#### Liechtenstein formula in single-orbital model
Let us consider to derive exchange interactions from the following one-body Hamiltonian:

$$
\begin{align}
H=-t\sum_{<i,j>}\sum_s c^\dagger_{is} c_{js} - \sum_i \sum_{ss'} c^\dagger_{is} (\boldsymbol{B_i} \cdot \boldsymbol{\sigma}_{ss'}) c_{is'}
\end{align}
$$

If we set the effective magnetic field $\boldsymbol{B}_i$ to $\boldsymbol{B}_i=JS\boldsymbol{e}_i/2$, this is a classical Kondo lattice model, while if $\boldsymbol{B}_i=U\boldsymbol{m}_i/2$, this is a Hubbard model with the mean-field approximation. The basic idea of the Liechtenstein method is to impose the equivalence of the spin-angle-derivatives of the total energies in the itinerant systems and the classical spin systems. Note that the derivatives generally depend on a reference magnetic structure. Here, let us consider the classical Heisenberg model $E=-2\sum_{<i,j>}J_{ij}\boldsymbol{e}_i\cdot \boldsymbol{e}_j$ as a spin model and the $\hat{z}$-polarized ferromagnetic state (namely, $\boldsymbol{B_i}=B\hat{z}$) as a reference magnetic structure. Then, we will get

$$
\begin{align}
J_{ij} = -B^2T\sum_{\omega_n} G_{ij,+}(i\omega_n)G_{ji ,-}(i\omega_n)
\end{align}
$$

where the Green function $G_{ij,\sigma}(i\omega_n)$ is a Fourier transform of $G_{\boldsymbol{k},\sigma}(i\omega_n) = [i\omega_n - (\varepsilon_{\boldsymbol{k}}-B\sigma-\mu)]^{-1}$ with the single-particle dispersion $\varepsilon_{\boldsymbol{k}}$ and chemical potential $\mu$. $i\omega_n=(2n+1)\pi T$ is the Matsubara frequency and $\boldsymbol{k}$ is the crystal momentum. An important quantity here is defined by $J_0=\sum_{0\neq i}J_{0i}$, which is related to the mean-filed transition temperature $T_c^{\rm mf}$ in the classical Heisenberg model by $T_c^{\rm mf}=2J_0/3$. This can be evaluated from the above $J_{ij}$, or the direct comparison of the second derivatives of the total energy, which gives

$$
\begin{align}
J_0 = \frac{B}{2} (n_{0,+}- n_{0,-})+ B^2T\sum_{\omega_n} G_{00,+}(i\omega_n)G_{00,-}(i\omega_n)
\end{align}
$$

In practice, $G_{00,\sigma}(i\omega_n)$ can be obtained by the $\boldsymbol{k}$ summation of $G_{\boldsymbol{k},\sigma}(i\omega_n)$ since all sites are equivalent in this model. Thus, the number of operations is $\mathcal{O}(N_M N_k^{tot})$ where $N_M$ is the number of Matsubara frequency and $N_k^{tot}$ is the total number of k-points. Since the typical value of $N_M$ is $N_M\sim \beta W$, the total cost is given by $\mathcal{O}(\beta W  N_k)$. On the other hand, we need $G_{ij,+}(i\omega_n)$ to evaluate $J_{ij}$, and in this case, the number of operations becomes $\mathcal{O}(N_M (N_k\ln N_k)^d)$ since the Fourier transform requires $N_k\ln N_k$ operations for a single axis, where $N_k$ is the number of k-points for a single axis and $d$ is the dimension. Thus, we need $\mathcal{O}(\beta W (N_k\ln N_k)^d)$ in total.

#### Sparse_ir
Since $G_{ij,\sigma}(i\omega_n)$ as well as $G_{00,\sigma}(i\omega_n)$ are just linear conbinations of $G_{\boldsymbol{k},\sigma}(i\omega_n)$, all terms we need take the following form,

$$
\begin{align}
\frac{1}{i\omega_n - \varepsilon_a}\frac{1}{i\omega_n - \varepsilon_b}=\frac{1}{\varepsilon_a-\varepsilon_b}\left(\frac{1}{i\omega_n - \varepsilon_a}-\frac{1}{i\omega_n - \varepsilon_b} \right)
\end{align}
$$

Thus, we can expand $G_{ij,+}(i\omega_n)G_{ji ,-}(i\omega_n)$ in the expression for $J_{ij}$ and $G_{00,+}(i\omega_n)G_{00,-}(i\omega_n)$ for $J_{0}$ by the intermadiate representation. Since the summation over $i\omega_n$ is equivalent to evaluating the Fourier transform at $\tau=0$, we need only $\mathcal{O}(\ln (\beta W))$ operations for the Matsubara frequancy axis. Thus, the computational cost becomes $\mathcal{O}(\ln(\beta W)  N_k^{tot})$ for $J_0$ and $\mathcal{O}(\ln(\beta W) (N_k\ln N_k)^d)$ for $J_{ij}$.

## Code implementation
Here, let us see an example code in a square lattice tight-binding model with dispersion $\varepsilon_{\boldsymbol{k}} = -2t\,[\cos(k_x) + \cos(k_y)]$. First, we load all necessary modules that we need:

```
import numpy as np
%matplotlib inline
import matplotlib.pyplot as plt
import matplotlib.cm as cm
import sparse_ir
```

#### Parameter setting
The parameters to define the system are as follows; $t$: the nearest-neighbor hopping amplitude, $\beta$:  the inverse temperature, $N_k$: the numer of k-poitns for a single-axis and $B$: the strengh of the effective magnetic field.

```
t = 1 # means the bandwidth W is W=8t in a square lattice model
beta = 50 # inverse temperature
nk = 36 # number of k-points for a signle-axis
b_eff = 3 # strength of the effective magnetic field
```

#### Liechtenstein solver
This class contains two solvers, j0_calc for calculation of $J_{0}$ and jij_calc for $J_{ij}$. Both can be called after instantiating the class.

```
class liechtenstein:
  def __init__(self, basis, *args):
    # constructor of liechtenstein class
    # the class requires five arguments: sparse_ir basis object, t, beta, nk, and b_eff

    self.basis, self.M, self.T = basis, sparse_ir.MatsubaraSampling(basis), sparse_ir.TauSampling(basis)
    t, self.beta, self.nk, self.b_eff= args[0], args[1], args[2], args[3]
    k1, k2 = np.meshgrid(np.arange(self.nk)/self.nk, np.arange(self.nk)/self.nk, indexing='ij')
    self.ek = -2* t * (np.cos(2*np.pi*k1) + np.cos(2*np.pi*k2))
    self.nkt = self.nk**2

  def j0_calc(self, e_range=np.zeros(1), NM=None):
    # solver to evaluate J_0
    # e_range: numpy ndarray object to specify the chemical potentials at which we evaluate J_0
    # NM: maximum number of Matsubara frequency. if None, sparse sampling will be used

    # set Matsubara frequency grid
    if NM is None: iw = 1j*np.pi*self.M.sampling_points/self.beta
    else         : iw = 1j*np.pi*(2*np.arange(-NM,NM)+1)/self.beta

    # calculate the first term in the expresson of J_0
    n_up = .5 * (1-np.tanh(.5*self.beta*(self.ek[None,:,:]-e_range[:,None,None]-self.b_eff)))
    n_dn = .5 * (1-np.tanh(.5*self.beta*(self.ek[None,:,:]-e_range[:,None,None]+self.b_eff)))
    j0_1 = .5 * self.b_eff * np.mean(n_up - n_dn, axis=(1,2))

    # calculate the second term in the expression of J_0 as a function of iw
    gkio_up = 1/(iw[:,None,None,None] - (self.ek[None,None,:,:]-self.b_eff-e_range[None,:,None,None]))
    gkio_dn = 1/(iw[:,None,None,None] - (self.ek[None,None,:,:]+self.b_eff-e_range[None,:,None,None]))
    gio_up = np.mean(gkio_up, axis=(2,3))
    gio_dn = np.mean(gkio_dn, axis=(2,3))
    j0_2_w = self.b_eff**2 * gio_up * gio_dn

    # evaluate a summation over iw
    if NM is None:
      j0_2_l = self.M.fit(j0_2_w.reshape(len(iw),len(e_range))) # transform to the intermediate representation
      j0_2 = self.basis.u(0)@j0_2_l # evaluate the function at \tau=0
    else:
      j0_2 = np.sum(j0_2_w,axis=0)/self.beta

    # sum up the two terms
    self.j0 = np.real(j0_1 + j0_2)
    self.j0_e_range = e_range

  def jij_calc(self, mu=0, NM=None):
    # solver to evaluate J_ij
    # mu: the chemical potential at which we evaluate J_ij
    # NM: maximum number of Matsubara frequency. if None, sparse sampling will be used

    # set Matsubara frequency grid
    if NM is None: iw = 1j*np.pi*self.M.sampling_points/self.beta
    else         : iw = 1j*np.pi*(2*np.arange(-NM,NM)+1)/self.beta

    # calculate J_ij as a function of iw
    gkio_up = 1/(iw[:,None,None] - (self.ek[None,:,:]-self.b_eff-mu))
    gkio_dn = 1/(iw[:,None,None] - (self.ek[None,:,:]+self.b_eff-mu))
    grio_up = np.fft.fftn(gkio_up,axes=(1,2))/self.nkt
    grio_dn = np.fft.ifftn(gkio_dn,axes=(1,2))
    jij_w = - self.b_eff**2 * grio_up * grio_dn

    # evaluate a summation of J_ij over iw
    if NM is None:
      jij_l = self.M.fit(jij_w.reshape(len(iw),self.nkt))
      jij = self.basis.u(0)@jij_l
    else:
      jij = np.sum(jij_w,axis=0)/self.beta

    # arrange the format to ouput the data
    self.jij = np.real(jij)
    self.jij[0] = 0
    self.jij_r = np.linalg.norm(np.array(np.meshgrid(np.arange(self.nk),np.arange(self.nk), indexing='ij')), axis=0).flatten()
```

#### Instantiation
In the instantation, we have to pass a sparse_ir.basis object as the first argument of the liechtenstein class.

```
Lambda = 2*max(8*t, b_eff) * beta
basis = sparse_ir.FiniteTempBasis(statistics='F', beta=beta, wmax=Lambda, eps=1e-7)
l = liechtenstein(basis, t, beta, nk, b_eff)
```

#### Executing the solver for J_0
Here, let us evaluate J_0 as a function of the chemmical potential $\mu$. To check the validity of the sparse sampleing method, we will compare it with that evaluated by the conventional Matsubara frequency grid. This can be done as follows:

```
plt.xlabel("mu")
plt.ylabel("J_0")
for i, nm in enumerate([100, 200, 400, 800, 1600]):
      l.j0_calc(e_range=np.linspace(-10,10,41), NM=nm)
      plt.plot(l.j0_e_range, l.j0, marker='.', label=str(nm), color=cm.cool(i/5.))
l.j0_calc(e_range=np.linspace(-10,10,41))
plt.plot(l.j0_e_range, l.j0, marker='.', label='Sparse_IR', color=cm.cool(1.0))
plt.legend()
plt.show()
```

As is expected, the system favors the antiferromagnetic state at the half-filling while it favors the ferromagnetic state at the low-carrier regime. This feature is consistent with the super-exchange and the doule-exchange interactions.

#### Executing the solver for J_ij
We can also evaluate J_ij by using jij_calc function as follows.

```
plt.clf
plt.xlabel("distance")
plt.ylabel("J_ij")
plt.xlim([0,5])
l.jij_calc(0)
plt.scatter(l.jij_r, l.jij, marker='.')
plt.show()
```

We can easily check $J_0=\sum_j J_{0j}$ as follows.

```
J0_1 = l.j0[20]   # J0 at mu=0 evaluated by j0_calc
J0_2 = sum(l.jij)
print(J0_1,J0_2)
```

[1] Liechtenstein et al., J. Phys. F: Met. Phys. 14 L125 (1984).