# Random Number Generators and Seeding

This week, we will turn to a new topic: generating and using random numbers. In the first lecture on this topic, we will look mainly at generating randomness in our programs. In later lectures we will turn to some examples on how we can use this randomness for different algorithms and scientific problems, and also how we can do testing on stochastic systems.

Randomness is important for many scientific applications, and many different problems and algorithms need access to be able to draw elements or samples at random. Also in other parts of programming, randomness can be an important topic. Most games for example, have a considerable component of randomness. Another major field that needs randomness is *cryptography*, which is needed to communicate securely, a crucial component of everything on the internet for example.

In the coming lectures, we will show some examples of using randomness, but for now, we simply state it: randomness is a [valuable commoditiy](https://en.wikipedia.org/wiki/Applications_of_randomness). We therefore need to know how to generate it.

### Some History

People have been generating random numbers by mechanical means for millenia, for example by throwing dice, spinning roluette wheels, shuffling cards, and so on. While these techniques might mostly be connected to games and gambling, we can use them to produce randomness for use in scientific studies, statistics and so on as well.

However, sitting at producing your own random numbers by rolling a die is cumbersome and time consuming. To get around this and make the task of producing randomness easier, random number tables would be distributed. These were simply random numbers that were generated, and then tabulated into large tables. As the tables already are random, it doesn't matter how you read these tables, you can go by row, by column, or diagonally for example. But having access to a random number table makes life easier if you need a few random samples.

To emphasise the importance of random numbers, you can take a look at the book *A Million Random Digits with 100,000 Normal Deviates*, first published in 1955 by the RAND company. As the name implies, this is a book of a millon random digits. The interesting thing however, is that this book was considered quite groundbreaking, as it enabled more people access to a large number of random numbers than ever before.

The RAND's coorperation's book is an historically important example of the importance of randomness, but it has also seen a new print edition in 2001 with a new foreword. The book has a bunch of humerous [reviews on Amazon](https://www.amazon.com/Million-Random-Digits-Normal-Deviates/product-reviews/0833030477/).

## Randomness on Computers

With the advent of computers and computing, it was clear that having access to random numbers in a computer program would also be a valuable asset. However, there is an issue, because computers were created to be entirely stable and *deterministic*. There was simply no way for a computer to produce a random number from nothingness.

The solution was to "import" random numbers from outside sources. The RAND coorperations book *A million random digits* for example, also came in the format of punch cards, so that they could be read into the computers memory and used by a program. This process was however, very slow, and having access to a sufficient number of sufficiently random numbers was challening.

Where did this randomness that was imported actually come from? These were taken from some physical setup that measured some seemingly random process. *A million random digits* for example, were taken from an electric roulette wheel connected to a large computer that kept spinning the wheel and noting the results. In modern days, many other physical systems for randomness have been proposed and used. Lava lamps have for example been popular, as the movement of the "lava" as it moves around and breaks up and merges together can be measured by an electric sensor and seemling produces random outputs. Other options are looking at radioactive decay with a geiger counter, looking at thermal fluctations or quantum mechanical noise. These are just some examples, and there are countless more.

Physical means of producing random numbers are still used today, the website [random.org](https://www.random.org/) for example, uses *atmospheric noise* to generate random numbers, the noise is measured by radio and tabulated into random number tables, which is then used when randomness is needed.

### Pseudorandomness

While there are many techniques to measuring physical randomness and using it, there are some problems with these techniques, which we will explain more later. An alternative is to let the computer produce its own random numbers.

Now, earlier, we said that this is problematic, because computers are fully deterministic machines, and this is still true. There is no way for a computer to produce non-deterministic randomness, if it is not connected to some physical RNG. However, in scientific programming, what we care about in our random numbers is the *statistics*. That means, even though they might be deterministic, if we can get the computer to produce numbers that look and behave like random numbers, from a statistical sense, then it will be sufficient for our purposes. We call this *statistically* random numbers. This concept is called [statistical randomness](https://en.wikipedia.org/wiki/Statistical_randomness).

In the rest of this lecture we will look at examples on how a computer can produce *statistical randomness* that is sufficient for scientific applications. However, it is important to stress that these are completely deterministic processes. To make this clear, another popular term is calling it *pseudorandomness*.

Any system that produces random numbers is called a *random number generator* or RNG for short. To emphasise that a software-based RNG produces pseudorandom numbers, it is often designated a PRNG/pRNG (pseudorandom number generator).

## Pseudorandom Number Generators

We will noe start looking at how to implement a pRNG on a computer. Now, recall that we cannot generate random numbers out of *nothing*, and so instead, to implement a pRNG, we will implement some rule for taking a number, and producing a new one from the first one. We can then take the second number, and produce a third one, and so one. A pRNG is effectively a *difference* equation that produces a sequence of numbers.

Now, because each number in the sequence i supposed to be statistically random, whatever rule or algorithm we do to find the next number in the sequence should produce a number that should be *uncorrelated* from the previous one. Put simply, each new number in the sequence should "look" random.

Because a pRNG effectively just keeps iterating to produce new numbers in a sequence, it needs some starting point. This is called the *seed* of the pRNG, and it is up to us, the user, to supply this seed. We will turn to discuss the seed a bit later, but for now, we simply pick some number.

### The Middle-Square Method

One of the first examples of a computer implemented pRNG was proposed by John von Neumann and is called the *middle-square method*. In this method, you find the next number in the sequence by first squaring your number, and then extracting the middle from that result.

Now, to start generating such a sequence, we need to start somewhere. Let us say we start with the number

$$X_0 = 4096.$$
We can now find the next number in the sequence by first squaring:
$$X_0^2 = 16777216,$$

and then extracting the middle digits. Because we started with a 4-digit number, we extract the middlemost 4 digits. Meaning the next number in the sequence becomes
$$X_1 = 7772.$$

We can now perform another iteration, note that we now use the previous result as the point to continue from:

$$X_1^2 = 60403984,$$
Extracting the middlemost 4 digits now gives
$$X_2 = 4039.$$

Let us implement this process in Python:

```python
import numpy as np
import matplotlib.pyplot as plt
```

```python
def next_number(current):
    current **= 2
    return int(str(current)[2:6])

X = np.zeros(21)
X[0] = 4096

for i in range(20):
    X[i+1] = next_number(X[i])

print(X)
```

    [4096. 7772. 4039. 3135. 2822. 6368. 5514. 4041. 3296. 8636. 5804. 6864.
     1144.  873. 2129. 3264. 6536. 7192. 7248. 5335. 4622.]

The middle-square method has some glaring faults, which means it does not work too well as a pRNG in practice. But that is not something we will go into now.

The important points is that von Neumann proposed a method for generating sequences of numbers that *looked* random, in a statistical sense, and his method was both easy to implement and very fast. For his purposes, the technique worked well. Compared to reading in random numbers from punch cards, he found that his method was more than a hundred times as fast.

After von Neumann, many other pRNGs have been proposed, but they all behave in the same manner: they produce statistically random numbers in a sequence, by performing a set of operations on the previous number in the sequence.

## Linear Congrugential Generators

While the middle-square method was a very original idea, it did not stand the test of time. Another pRNG that did however, which is perhaps even simpler than the middle-square method, are the [linear congrugential generators](https://en.wikipedia.org/wiki/Linear_congruential_generator) (LCGs).

LCGs are a whole family of pRGNs that are easy to implement and fast. If they are designed, implemented and used correctly they produce fairly good random numbers, and they have historically been very important and still see some use today. And while they are by no means considered the *best* option out there today, they are still considered good for certain use cases.

As we mentioned, LCGs are a *family* of generators. They all produce random integers by following the governing  difference equation:

$$X_{n+1} = aX_n + c \mod m.$$

From this formula, you can see that there are three steps to producing a new number with a LCG:
1. Multiply the current number by $a$,  the multiplier
2. Add the constant $c$, the increment
3. Take the modulo with $m$, the modulus

An LCG thus has three adjustable parameters: the multiplier, the increment and the modulus. By defining these three you are deciding on a specific LCG. All three parameters must be integers, and $m$ must be positive, $a$ must be positivive and smaller than $m$ and $c$ must be non-negative and smaller than $m$.

We call these pRNGs *linear* generator, because this difference equation is a linear equation. Strictly speaking, because of the modulo operator it is a discontinious, piecewise linear equation. If we set $c=0$ we get a sub-family of LCGs called *multiplicative congruential generator* (MCG).

### Choice of parameters

Now, it's important to stress that the parameters $a$, $m$ and $c$ should not be chosen freely, but must be rather carefully selected, if we want to end up with a good pRNG. We'll return to this a bit later and explain what makes a "good" choice. For now, we take the parameters given in the book *Numerical Recipes*

In numerical recipes, they propose a LCG with the parameters:
$$\begin{align}
m &= 2^{32}, \\
a &= 1664525, \\
c &= 1013904223.
\end{align}$$

```python
def next_number(current):
    m = 2**32
    a = 1664525
    c = 1013904223
    return (a*current + c) % m

X = np.empty(101, dtype=np.int)
X[0] = 10981

for i in range(100):
    X[i+1] = next_number(X[i])

print(X)
```

    [     10981 2112184064 1273939551 2581585970  620970473 3650953780
     1008625667 2411097478 3932940077 3043663528 1819800039  678905370
     2834685617 2263795804 3412051979 1125350894 3034000501 2122306384
      971935343  589523202 1648628857 1507668868 4145359123  968530774
     2956139197  673422584 3405822967  830204138 3714122561 1929446316
     3602819867 3521996222 1804915205 2166922144  740334207 2383177170
     1428409097 2785466580 2447404579 3525410086 2025157197 3535105864
      402934279 1691647930 2135395281 3694088956 1619169835 1870692750
     3228703637  717535216 3223710351   29873826 3907741081 3335459364
     2612082483  610800374 1733029341 4138014104 2772671511 3477998218
     3809638497 3788420684 3724311867 1458164062 1105725733 3984140352
     2145353375  616741746 4280542249 3324698484  871924803 3477822662
     3650063725 1253543912  952087079 3841296730  459983089 3430165916
     1555917899 4270216494   11508405 1487596688 4050540207 3970569282
     4288038585 4283384004  442979667  348674710 4149825789 1257365048
      481922103 1360526378 2302246273 1576319212 2459328347 3891465470
      532795461 1761543392 2805166783 1852498194 1749781833]

We see that our *Numerical Recipes* LCG produces large integers, that at least to the naked eye, seem fairly random. We'll return to the topic on how to check wether they are statistically random a bit later. We will also come to the topic of what good these huge integers are shortly.

First, we should improve our implementation somewhat, using object-oriented programming.

### Encapsulation

Both examples we have seen of a pRNG so far, has used the previous number in the sequence to produce the next one. This is a hazzle for the user to deal with, and it is error-prone. What if the user sends in the wrong number to the function? Suddenly we have no guarantee what the pRNG produces.

To fix this, we turn to object oriented programming, and use the technique of encapsulation to create a pRNG that only spits numbers out. We now introduce some terminology, we call the "current" number of the pRNG the *state* of the pRNG. The pRNG still needs some place to start, and so the starting state is called the *seed*, and the user still has to supply this, but only once.

```python
class NumRecipesLCG:
    def __init__(self, seed):
        self.state = seed

    def __call__(self):
        self.state = (1664525*self.state + 1013904223) % 2**32
        return(self.state)
```

Our class is fairly simple. We have a constructor that takes in a given seed, and sets the current state to this seed. We also implement a call special method that takes the state of the pRNG, and advances it one step, then we return it.

From the outside, we can now initialize a specific pRNG object by seeding it, and then call it to produce random integers. What happens to the internal state as we call the object is of no concern from the outside, it is an example of *encapsulation*.

```python
random = NumRecipesLCG(2401293)

for i in range(10):
    print(random())
```

    3706549768
    2555191751
    4060282
    3471243665
    616719804
    2117273067
    2371150414
    1434944853
    2562561712
    1061796431

### Possible Outcomes

Now, the LCG we have created spits out a bunch of large integers. And it is at least hard to predict what number comes next. However, to have any use of the numbers that are outputted, we need to at least know the range of possible outcomes.

To compute the next number in the sequence, we multiply the current state by $a=1664525$, and then increment it by $c=1013904223$. Both of these are massive numbers, to we would expect the state of the system to grow by quite a large margin. However, it does not grow beyond all bounds, because the last step is to take the modulo.
$$X_{n+1} = aX_n + c \mod m.$$

Taking the [modulo](https://en.wikipedia.org/wiki/Modulo_operation) of a number means finding the remainder after division. It is an importat operation in number theory. We won't cover it in detail here, but a nice analogy for the modulo operation is a clock. A normal clock face shows numbers in the range $[0, 12)$, once it goves beyond 12, it starts back at 0. This is effectively a modulo operation. For example, 17:00 on a 24-hour clock would be $17 \ \text{mod}\ 12 = 5$ on a 12-hour clock.

In our LCG, we always take the modulo with $m$. This means, just like for the clock, every time our state grows beyond $m$, it "loops back around. Thus, we are guaranteed that any number output by our LCG will be a number in the range:

$$[0, m),$$
or for our given choice of parameters:
$$[0, 2^{32}) = [0, 4294967296).$$

Now. This gives a range for possible values our LCG potentially can output. Are we guaranteed that every single integer in this range can, and will, be output? No, not in general no, but let us assume that this is the case.  The range $[0, 2^{32})$ includes exactly $2^{32}$ integers, so if all possible values in this range are possible, or LCG can then output $2^{32}$ different outcomes. For our LCG to be considered statistically random, we would want all of these outcomes to be equally likely.

Now, because we are doing the modulo operation at the end, we can envision our LCG working by spinning the handle of a clock around. Because we multiply $a$ by the previous state $X$, the amount the handle of the clock moves in the next iteration is not a constant length, but dependent on the pseudorandom number $X$. Moving the handle a random amount around the clock gives the next number in the sequence.

Now, an interesting question is why we have chosen exactly $m=2^{32}$ as our range. Well, recall that an `int` in C++, and most other programming languages, are precisely 32 bits. As a bit can be either 0 or 1, that means a normal integer in most programming languages can have $2^{32}$ possible values. As we draw non-negative numbers, our LCG produces unsigned integers, but these can be stored in an unsigned int variable, which we define in C++ as `usigned int`. In other words, by choosing $m=2^{32}$, we are producing exactly one possible 32-bits integer value.

Based on this, it shouldn't come as a surprise that another popular choice for the modulus is $m=2^{64}$, which is the size of a `long` integer in C++ and other programming languages. A benefit of sticking to modulus that are strictly powers of $2^n$ is that computing the modulus is as simple as throwing away the unimportant bits, a process known as *truncating*. This is extremely fast on a computer, much faster than computing the actual modulo, and choosing $m=2^n$ will therefore give a fast LCG.

## Transforming numbers

Now, let us look at how to actually use these huge integers output by our LCG. As we are rarely interested in huge integers in our algorithms, we need some way to *translate* our randomness to whatever format we require.

An important form of randomness we often require are real numbers in the range $[0, 1)$. For our LCG, computing such values is actually not to tricky. We now that an LCG will produce integers in the range $[0, m)$, and so we can convert these to random floating points on the range $[0, 1)$ by dividing by $m$:

```python
random = NumRecipesLCG(2401293)

for i in range(10):
    print(random()/2**32)
```

    0.8629983682185411
    0.5949269400443882
    0.0009453580714762211
    0.8082118968013674
    0.14359126891940832
    0.49296605098061264
    0.5520764770917594
    0.33409913373179734
    0.5966428928077221
    0.24721874645911157

This transformation from random integers to random floats is just a mapping of values. In theory, all the $m$ different possible integers should map to $m$ different decimal values evenly distributed over the range $[0, 1)$. However, with the limited precision of floating point numbers, this is not guaranteed. In fact, if we wanted to ensure our pRNG was doing a good job of this task, we would need to really analyze the floating point behavior. However, let us ignore such details for now.

Now, what if we want random integers in a different, interval than $[0, m)$? Say we want to have 100 different outcomes? Or we want to roll a die and have 6 outcomes? One way to shift a number from the interval $[0, m)$ to a smaller interval $[0, 100)$ is to use modulo:

```python
random = NumRecipesLCG(240141556)

for i in range(5):
    r = random()
    print(f"{r:10} -> {r % 100:4}")
```

    2916068291 ->   91
    2490719814 ->   14
    3200950509 ->    9
    1610386792 ->   92
    2344718759 ->   59

Now, while this output might seem reasonable. We need to be a bit careful when transforming the numbers in this way, because the number of possible values might not be divisible by 100. Indeed, for our pRNG:
$$2^{32} = 4294967296$$
which does not evenly divide 100, but leaves 96. This means when transforming from all possible outcomes to the range $[0, m)$, there will be a very, very slightly lowered chance of getting the numbers 97, 98, or 99.

Now, we might try to get smart and first convert our ints to floats on the interval $[0, 1)$, and then convert the back into integers somehow, like for example

```python
random = NumRecipesLCG(240141556)

for i in range(5):
    r = random()
    r /= 2**32
    print(f"{r:5.5f} -> {r*100:4}")
```

    0.67895 -> 67.89500571321696
    0.57992 -> 57.991589745506644
    0.74528 -> 74.52793673146516
    0.37495 -> 37.49473933130503
    0.54592 -> 54.59223778452724

However, this also cannot be a good solution, and we haven't actually solved our problems, but just hidden them from sight. Our pRNG outputs $2^{32}$ possible values. Because this number is not divisible by 100, there is simply no way to divide these outputs into 100 values cleanly. It doesn't matter if we use fancy math, or clever tricks.

If we want to have numbers uniformly spread on the range $[0, 100)$, the only proper solution would be to throw out some of our pRNG's output, so that we did get an equal number of values into each wanted outcome.

Note that this is *not* a problem with our pRNG, which might be outputting perfectly statistically random samples, but it is a problem of trying to divvy up numbers into allotments, that is not possible. This is an example of something called the [Pigeonhole Principle](https://en.wikipedia.org/wiki/Pigeonhole_principle) in mathematics. If you try to divide up
$$2^{32} =4294967296,$$
Pigeons into 100 pigeonholes, there is no way to get an equal number of pigeons into each hole.

<center>Picture by <a href=https://en.wikipedia.org/wiki/Pigeonhole_principle#/media/File:TooManyPigeons.jpg>Ben Frantz Dale</a> and used under a <a href=https://creativecommons.org/licenses/by-sa/3.0/>CC BY-SA 3.0</a> license.</center>

### Random numbers following a given distribution

We have now seen how we can use a pRNG to get random outputs of either integers on a range $[0, m)$, or uniformly distributed random floats on the range $[0, 1)$.

However, often we want random numbers following some other distribution. Perhaps we want standard normally distributed numbers. Or numbers following a Poisson distribution, or Bernoulli-distributed numbers.

The thing is, our randomly generated numbers can be translated to any such distribution, using differenc techniques, algorithms or methods. The important thing is that the *randomness* itself is already generated, the remaining steps is just converting how it is understood or interpreted. Because of this, one sometimes talk about producing random numbers simply as producing random *bits*, to make it clear that the randomness produced itself is just information.

How uniform random numbers on the interval $[0, 1)$ can be transformed to other distributions, say randomly disributed numbers with a standard deviation of 3.6 and a mean of 4.7, is not a big topic of this course. This is because this is actually a statistical problem, aka, a mathematical problem. However, we do cover one example, just to give you some idea of how it can be done.

We do not assume students of IN1910 have taken, or will take, statistics courses. But if you do know some statistics, we can mention that you can generate random numbers from any probability distribution if you know its cumulative distribution function through a process called [inverse transform sampling](https://en.wikipedia.org/wiki/Inverse_transform_sampling).

#### Uniform distribution on the range $[a, b)$.

If we have uniformly distributed numbers on the range $[0, 1)$, we can easily shift them to any other uniform distribution simply by multiplying and shifting.

Say for example we are interested in the range $[-2, 2)$, we can then first scale the range $[0, 1)$ to $[0, 4)$ by multiplying any sample $X$ by 4: $4x$. Then we shift the whole thing down by 2, so if $X$ is a random number on the range $[0, 1)$, then $4X-2$ is a random number on $[-2, 2)$.

This holds for any $a$ and $b$. If we want random numbers on the range $[a, b)$, we can scale the sample outputted by our pRNG according to the formula:

$$(b-a)X + a.$$

#### The Box-Muller Algorithm for finding Normally distributed numbers

Just to show an example of a much used transformation algorithm, we cover the *Box-Muller* algorithm, which produces normally distributed numbers with a standard deviation of 1, and a mean of 0, i.e., the standard normal distribution.

The algorithm itself is based on a clever geometric idea, which you can read more about here [here](https://en.wikipedia.org/wiki/Box%E2%80%93Muller_transform).

In Box-Muller, we use two randomly generated independent points on the range $[0, 1)$: $U_1$ and $U_2$, to produce two randomly generated independent normally distributed points: $Z_1$ and $Z_2$.

This is done through the formula:
$$\begin{align}
Z_1 = \sqrt{-2 \ln U_1}\cos (2\pi U_2), \\
Z_2 = \sqrt{-2 \ln U_1}\sin (2\pi U_2).
\end{align}$$

Let us look at an example:

```python
nr_samples = 5000
assert nr_samples % 2 == 0

# Generate uniform samples
seed = 240156
random = NumRecipesLCG(seed)
uniform_samples = np.empty(nr_samples)
for i in range(nr_samples):
    uniform_samples[i] = random()/2**32

# Use Box-Muller to transform to normal distributed numbers
normal_samples = np.empty(nr_samples)
for i in range(0, nr_samples, 2):
    U1 = uniform_samples[i]
    U2 = uniform_samples[i+1]
    Z1 = np.sqrt(-2*np.log(U1))*np.cos(2*np.pi*U2)
    Z2 = np.sqrt(-2*np.log(U1))*np.sin(2*np.pi*U2)
    normal_samples[i] = Z1
    normal_samples[i+1] = Z2

plt.hist(uniform_samples, bins=50)
plt.title('Uniform distribution')
plt.show()

plt.hist(normal_samples, bins=50)
plt.title('Normal distribution')
plt.show()
```

If we want to shift the standard deviation of the normal distribution or its mean, we can simply multiply each sample by $\sigma$, and if we want to move the mean we simply add $\mu$ to each sample.

## Seeding and Reproducability

We have seen that a pRNG needs to start at a given *seed state* to be able to produce a sequence of pseudorandom numbers. It is up to the user of the system to select a seed, and feed it into the pRNG. The choice of the seed will impact what sequence of numbers we get. If we choose a different seed, we get a different sequence out.

Now, because each iteration of the pRNG is a deterministic computation, if we choose the *same seed state*, we get the same exact output. You might have noticed this already, if you execute the same examples in this notebook so far, we get the exact same output, the "random" numbers do not change whatsowever, untill you change the seed.

While the fact that "random" numbers do not change on execution, but are 100% determined by the seed might feel like a major flaw of the system. However, recall that the goal is not to produce *true* random numbers, but *statistically* random numbers. The fact that the seed perfectly predicts the outcome is actually a major benefit!

Because the same seed produces the same "random" sequence, it means that our code is *reproducible* and much easier to test. In fact, this is one of the big reasons pRNGs exist in the first place. Von Neumann developed the middle-square method not only for speed, but also because he found it impossible to test and reproduce his code run with punch cards supplied random numbers. Of course, he could simply write down the sequence the random numbers where supplied into the program in, but the computer did not have close to enough memory to remember all these numbers. With his method however, to reproduce and test the program, you would simply need to remember a single number, the *seed state*.

A somewhat silly example of this is the game Minecraft. In Minecraft, the whole world is *procedurally generated*, meaning the world you start in is randomly generated following a set of rules. This means that every game of minecraft should have a completely unique world. However, perhaps you find a really cool world some time, and want to share it with a friend. Well, you would be in luck, because the Minecraft RNG used to randomly generated these worlds is a pRNG, and so if you simply feed in the correct *seed state* to this pRNG, you would end up with the same world. Minecraft uses this intentionally, and for any new world created, you can find the *seed* state of the world, which is basically a long number.

## Periodicity of a pRNG

There is an interesting consequence of a pRNG being completely deterministic and seedable. If you when iterating through and generating a sequence of random numbers, end up at the *same* number, you have effectively returned to your seed state. From this point onwards, your sequence will just repeat the exact same values!

Let us show an example of this with a very simple LCG, with $m=9$, $a=2$, $c=1$:

```python
class SimpleLCG:
    def __init__(self, seed):
        self.state = seed

    def __call__(self):
        self.state = (2*self.state + 1) % 9
        return self.state

random = SimpleLCG(6)
for i in range(13):
    print(random())
```

    4
    0
    1
    3
    7
    6
    4
    0
    1
    3
    7
    6
    4

Look closely at this sequence. We have seeded our pRNG with the seed value of $6$, when we iterate through and produce new numbers, we eventually end back up at the value 6. From that point onwards, the sequence just repeats itself in exactly the same manner. This is a huge problem, because an important quality of *statistically random* is that numbers are uncorrelated, and we are now getting big correlations in our datasets.

<center>
Source: <a href=https://xkcd.com/221/>XKCD #221</a>
</center>

The number of values a pRNG produces before it repeats itself is called its *period*. For our example, the period was 6, because it produced six numbers before repeating itself. It's important when we use a pRNG that we do not exhaust its period.

Now, our example LCG was purposfully very simple, to illustrate a short period. But note that while this LCG technically has 9 possible outcomes, because $m=9$, the period is shorter than 9. This is true for any LCG, if our LCG has $m=2^{32}$, then we know that the period is *at most* $2^{32}$. This is again because of the Pigeonhole princicple, we simply have to hit a repeated number if we reach $2^{32}+1$ numbers, as we have exhausted all options.

Thus, for an LCG, the period is at *most* $2^{32}$, but it can be much shorter. And this is were good choices of $a$ and $c$ comes in, as we want to choose values that maximized the period. Note also that the seed also impacts the period. Normally, when we talk about the *period* of a pRNG, we refer to the longest possible period, if we choose the right seed. Seeds that produce cycles of shorter periods than the ideal are often called *bad seed states*. A good pRNG should not really have *bad seed states*, because it is unreasonable to assume our user knows what seeds to avoid.

Based on this, we see that RANDU is a LCG with $a=65539$, $m=2^{31}$ and $c=0$. As we have seen, these choices of parameters turned out to be bad. To get a better LCG you shouldn't try to choose these parameters yourself, but you should look them up. The [Wikipedia page on LCGs](https://en.wikipedia.org/wiki/Linear_congruential_generator#Parameters_in_common_use) lists the parameters used by the most commonly used generators today.

## The Infamous RANDU pRNG

Perhaps the most notorious pRNG is RANDU, a linear congrugential generator made by IBM in the 1960s.

RANDU was created to be especially fast on certain hardware that was commonplace at the time. It chose the parameters $c=0$, $a=65539$ and $m=2^{31}$. Note also that it has to be seeded with an odd number:

```python
class RANDU:
    def __init__(self, seed):
        assert seed % 2 == 1
        self.state = seed

    def __call__(self):
        self.state = 65539*self.state % 2**31
        return self.state

rand = RANDU(41451)
for i in range(5):
    print(rand())
```

    569173441
    1267183939
    333059017
    1331116891
    694202897

Now, looking at these numbers. They look to be the same as the ones produced by our Numerical Recipes LCG shown earlier. However, it turns out that IBM choose parameters that were quite bad, and RANDU happens to produce numbers that are very correlated. Quoting [Wikipedia](https://en.wikipedia.org/wiki/RANDU):
> IBM's RANDU is widely considered to be one of the most ill-conceived random number generators ever designed[2], and was described as "truly horrible" by Donald Knuth.[3] It fails the spectral test badly for dimensions greater than 2, and every integer result is odd.

The "spectral test" in this regard refers to test LCGs, such as randu. If we use our pRNG to draw random points in a unit cube, if our pRNG is behaving as it should, these points will spread out uniformly in the cube, but if there is correlation between the different samples produced by the pRNG, we will see patterns forming in these random points. For most pRNGs we have to move to high dimensions before we can see these patterns, but for RANDU they show up even in 3D.

<center><b>Figure 2:</b> (Left) For a "good" pRNG, like the one used by `np.random`, we can see no discernible patterns in the random samples, i.e., they are uncorrelated. (Right) RANDU however, fails this test, as we can see the points line up into 15 parallell planes. </center>

At the time of RANDU's creation, not enough diligence was put on verifying its properties. And because of its simplicty and speed, it was quite popular in the 60s and 70s. Because of this, a lot of scientific resultst that rely on stochastic simulations from that time are seen with skepticism.

## State vs Ouput

In all our examples so far, the *state* of the pRNG is simply the previously outputted number. However, there is a problem with this approach, because it means we cannot get *repeated* outcomes. For any truly random sequence, we sometimes expect repeated outcomes.

The solution to this problem is to create a pRNG that has a larger internal state, and then returns only parts o f that state. For example, an LCG might use a modulus of $m=2^{64}$, meaning the state has 64 bits of information, however, when actually returning the ouput to the user, they might only use certain parts of this number, such as extracting for example $48$ bits of information. For an LCG, it is common to return the most significant bits in this way, because it can be shown that these bits have a longer period than the lower bits.

By only returning parts of the state, we ensure that we can actually get the same output multiple times, without the state being the same and producing a low period.

A test of a pRNG testing this is called *the birthday test*, because it is based on the [Birthday problem](https://en.wikipedia.org/wiki/Birthday_problem). The Birthday problem is as follows: In a room of $n$ people, what is the chance that at least two people share the same birthday? Assuming that the birthdays of people in the room are random, it effectively asks the chance of drawing the same random sample from a distribution.

The Birthday problem has the surprising answer that with only 23 people in the room, there is a bigger than 50% chance that two people share a birthday, and that increases to a staggering 99.9% with just 70 people! By generalizing the problem, we can test wether an RNG gives us the expected number of repeated samples for a given distributions. Many simpler pRNGs fail the birthday test.

## Mersenne Twister

In this lecture, we have talked much about LCGs, and as we mentioned in the beginning, LCGs are historically important and much used. They are still seen quite a bit, but they are considered a bit out-dated. Wikipedia's article on LCGs has a [table](https://en.wikipedia.org/wiki/Linear_congruential_generator#Parameters_in_common_use) showing commonly used LCGs today, and you can see that it is for example, still used in Java.

In Python however, there is a different pRNG that is used, which is considered the *gold standard* for pRNGs. This gold standard is called the *Mersenne twister*. Let us first explain the name:

### Mersenne Primes

Mersenne primes are primes that can be written on the form $2^n-1$. Note that this expression is *not* prime for all $n$, but for some $n$, it is, and those are the Mersenne primes. Mersenne primes are interesting in mathematics, because it turns out that they have an important connection to *perfect* numbers. Because of this, people are searching hard to find every larger Mersenne primes.

The currently know largest prime is a Mersenne prime, specifically it is

$$2^{77,232,917} − 1,$$

which is a number with 23,249,425 digits. This happens to also be the largest prime known altogether. In fact, out of the 10 largest primes known, 9 of them are Mersenne primes.

Most of the largest Mersenne primes are found by a collaboration known as the [Great Internet Mersenne Prime Search](https://www.mersenne.org/) (GIMPS), which is a group of volunteers running the GIMPS software on their normal desktop computers.

The Electronic Frontier Foundation has traditionally given out large cash prizes to whoever breaks a big milestone, such as the first prime to pass one million, and ten million digits. The next prize is $150,000 to whoever finds the first prime with 100 million digits or more. It wouldn't be unreasonable to assume the GIMPS will be the first to find this number.

<center>The number of digits in the largest prime known over time. Note that the vertical axis is logarithmic, meaning the growth is exponential. The dashed red line is the best fit exponential growth. Figure created by <a href="https://commons.wikimedia.org/wiki/File:Digits_in_largest_prime_2016.svg">Nicoguaro</a>, used under a <a href=https://creativecommons.org/licenses/by/4.0/>CC BY 4.0</a> license.</center>

## Mersenne Twister 19937

The Mersenne twister algorithm is called *Mersenne* because it uses a specific Mersenne prime as its basis. This could in theory by any Mersenne prime, but the most commenly used version uses the Mersenne prime

$$2^{19937}-1,$$

and it is sometimes referred to as Mersenne Twister 19937 for this reason.

The "twister" part of the name comes from the fact that this algorithm is *blazingly* fast. And this name is not really an exaggeration, it is one of the fastest algorithms for drawing random numbers out there.

We won't go through this algorithm in detail, because it is a bit complicated, but for one thing, we can mention that it has a *period* of precisely

$$2^{19937}-1,$$

which is an amazingly large number. This means that there is no chance of hitting repeated cycles for any real application of the Mersenne Twister.

The Mersenne Twister algorithm was first published in 1997 by Makoto Matsumoto Takuji Nishimura. At that time it was definitely the best pRNG available. Over time the algorithm has been slightly been improved, but it is still  considered the gold standard. While some other options have become available, the Mersenne Twister is still a very good choice for most scientific programming applications.

Precisely because of its great qualities, the Mersenne twister is the pRNG of choice for most software purposes, and it is for example the one you use when you use `random` or `np.random` in Python. It is also available from the standard library in C++. While we won't cover how the Mersenne twister works, you should know about it, because it is what you will most likely end up using a lot in the future.

### A note on Cryptographic Security

While the Mersenne twister is a very good pRNG, it is not at all suited for *cryptographic* use cases. While you can freely use Mersenne twister for scientific programming or games, you should *not* use it for security purposes. In Python for example, starting with 3.6, there is a fully seperate package used to produce more secure random numbers called `secrets`, which should be used instead of random.

## Generating Random Numbers in C++

When we start to use random numbers in the coming lecutres, we will mostly stick with Python. However, we would like to take a few minutes to show you how to generate random numbers in your C++ programs as well, because there are some easy traps you can walk straight into. We first cover the "wrong" way to do things, this is the same way as in C, and the "old" C++ way. Then we cover the "proper" way to do things, which is available since C++11.

### The "Old" or "Wrong" way to do things

In C and C++, there is a built in function called simply `rand()`. This underlying pRNG that is used relies on your compiler, but in most cases, it is a LCG, though with different parameters for different compilers.

You can call it as a function with no input. The following complete C++ program prints out 5 consecutive integers from the pRNG:

***
```C++
#include <iostream>

int main() {
    for (int i=0; i<5; i++) {
        cout << rand() << endl;
    }
    return 0;
}
```
***

However, if you run the program several times, you will notice that the output is exactly the same. This is because if no seed is explicity given, it will seed with the value `1`. And so each run of the program will therfore be identical.

You can explicitly seed your program with the built-in function `srand` (for "seed rand"). If you *do* want your program to behave differently each run,  a common way to do it is to use `time`, which gives you the current time in seconds. Running a program a few seconds apart will then give a different seed, and thus a different sequence:
***
```C++
srand(time(nullptr));

for (int i=0; i<5; i++) {
    cout << rand() << endl;
}
```
***

To translate the numbers made by `rand()` to useable numbers, we need to know the range of the pRNG. This is defined in the built-in variable `RAND_MAX`, which specifies the biggest integer we can get out, so to get random floats in the range $[0, 1)$, we need to divide by `RAND_MAX + 1`. we should also cast it to a double, otherwise we get integer division and most likely an overflow error.

Script that prints out 5 random floats:
***
```C++
srand(time(nullptr));

for (int i=0; i<5; i++) {
    cout << rand()/((double) RAND_MAX + 1) << endl;
}
```
***

Now, why do we call this the "wrong" way of doing things? Well, the C++ standard gives few guarantees for how good the underlying pRNG is, or about it's period. The only requirement from the C++ standard is that it should output at least 32 thousand numbers. A scientific application can easily exhaust this, and so using the built in `rand` would completely ruin the program. In addition, the `rand` function is an LCG, which is sort of outdated and not considered great.

### The "proper" way of generating random numbers in C++

With C++11, things were much improved for random numbers in C++, with a new standard header called `<random>`, that works much better than the `rand()` function.

First of, the `<random>` header splits up the underyling pRNG *engine*, and the random number distribution itself. So if we want to create random numbers, we should first specify what pRNG to use. We recommend sticking with the `mt19937` engine, its constructor only takes the seed as the input:
***
```C++
#include <random>

using namespace std;

int main() {
    int seed = 1424;
    mt19937 engine(seed);

    return 0;
}
```
***
Here we create an object of the class `mt19937`, which stands for Mersenne Twister ($2^{19937}-1$). Here we just select the seed manually, but you could also use `time` to do this. We called our Mersenne Twister object our *engine*, as it is what drives our randomness.

To actually use our engine object, we also need to create a *distribution* object, which converts the randomness made by our engine into whatever distribution we actually want. Let us start of with random floats between 0 and 1:

***
```C++
#include <iostream>
#include <random>

using namespace std;

int main(int argc, char const *argv[])
{
    // Generate engine
	int seed = 1424;
	mt19937 engine(seed);

    // Generate distribution
	uniform_real_distribution<float> uniform(0, 1);

    // Print random numbers
	for (int i=0; i<10; i++) {
		cout << uniform(engine) << endl;
	}

	return 0;
}
```
***

Here we again make our engine object. Next we make our distribution object. The constructor for the `uniform_real_distribution` takes the ends of the range, so we send in 0 and 1. We also have to template the distribution to tell it what data type we want as output. Here we select `float`, but `double` would be another natural type.

Note that we are making two distrinct object types here: the engine (of type `mt19937`) and the distribution (of type `uniform_real_distribution`). The engine produces the actuall randomness, the distribution "filters" it, so we get it out on the form we want. We need both to produce our samples, so we call it with `unfiorm(engine)` (these are the two names I chose for my two objects, you can choose other names).

Note that for any given program we should only ever create and use a single engine (unless we are parallell programming, in which case each thread should have its own engine). But we can make several distribution objects, and use them with the same engine. So say your program needs floats on the range 0 to 1, integers between 0 and 1999 and normally distributed numbers with a standard deviation of 3, we would set up our engine and distributions as follows

***
```C++
// Generate engine
int seed = 1424;
mt19937 engine(seed);

// Generate distributions
uniform_real_distribution<float> rand_float(0, 1);
uniform_int_distribution<int> rand_int(1000, 1999);
normal_distribution<float> rand_normal(0, 3);
```
***

Note that we are making two disitinct types of objects here, the first is the Mersenne Twister engine, it is the one that is producing the randomness in the form of large integers. When we want to use that randomness, we filter it through a *distribution* object, which makes sure we get output that follows the distribution we want.

The standard `<random>` header has quite a few engines in addition to mersenne twister. Most of these exist for backward compatability, so unless you have a really good reason to use them, just stick with MT19937 (or MT19937_64 if you need double float precision).

The header also has a lot of different distributions, and these can be very helpful, to produce different types of randomness for different programs. See the documentation to see what is available.

https://en.cppreference.com/w/cpp/numeric/random

#### Comparing the "old" and "new" C++ way

The following talk by one of the Visual C++ developers is a good introduction to the new C++11 `<random>` header an why it exists:

* https://channel9.msdn.com/Events/GoingNative/2013/rand-Considered-Harmful

```python
from IPython.display import HTML

HTML('')
```

```python

```


```python
import numpy as np
import scipy.misc
from scipy.fftpack import dct, idct
import sys
from PIL import Image
import matplotlib
import matplotlib.pyplot as plt
import random
from tqdm._tqdm_notebook import tqdm_notebook
from scipy.fftpack import dct, idct
import seaborn as sns
from skimage.metrics import structural_similarity as ssim
import pandas as pd
import sympy

%matplotlib inline

class ImageLoader:
    def __init__(self, FILE_PATH):
        self.img = np.array(Image.open(FILE_PATH))
        # 行数
        self.row_blocks_count = self.img.shape[0] // 8
        # 列数
        self.col_blocks_count = self.img.shape[1] // 8

    def get_points(self, POINT):
        Row = random.randint(0, len(self.img) - POINT - 1)
        Col = random.randint(0, len(self.img) - 1)
        return self.img[Row : Row + POINT, Col]

    def get_block(self, col, row):
        return self.img[col * 8 : (col + 1) * 8, row * 8 : (row + 1) * 8]

# plt.rcParams['font.family'] ='sans-serif'#使用するフォント
# plt.rcParams["font.sans-serif"] = "Source Han Sans"
plt.rcParams["font.family"] = "Source Han Sans JP"  # 使用するフォント
plt.rcParams["xtick.direction"] = "in"  # x軸の目盛線が内向き('in')か外向き('out')か双方向か('inout')
plt.rcParams["ytick.direction"] = "in"  # y軸の目盛線が内向き('in')か外向き('out')か双方向か('inout')
plt.rcParams["xtick.major.width"] = 1.0  # x軸主目盛り線の線幅
plt.rcParams["ytick.major.width"] = 1.0  # y軸主目盛り線の線幅
plt.rcParams["font.size"] = 12  # フォントの大きさ
plt.rcParams["axes.linewidth"] = 1.0  # 軸の線幅edge linewidth。囲みの太さ
matplotlib.font_manager._rebuild()

MONO_DIR_PATH = "../../Mono/"
AIRPLANE = ImageLoader(MONO_DIR_PATH + "airplane512.bmp")
BARBARA = ImageLoader(MONO_DIR_PATH + "barbara512.bmp")
BOAT = ImageLoader(MONO_DIR_PATH + "boat512.bmp")
GOLDHILL = ImageLoader(MONO_DIR_PATH + "goldhill512.bmp")
LENNA = ImageLoader(MONO_DIR_PATH + "lenna512.bmp")
MANDRILL = ImageLoader(MONO_DIR_PATH + "mandrill512.bmp")
MILKDROP = ImageLoader(MONO_DIR_PATH + "milkdrop512.bmp")
SAILBOAT = ImageLoader(MONO_DIR_PATH + "sailboat512.bmp")
```

```python
n_bar = 5
N = 32
```

# MSDS

```python
def msds(N,arr):
    w_e = 0
    e_e = 0
    n_e = 0
    s_e = 0
    nw_e = 0
    ne_e = 0
    sw_e = 0
    se_e = 0
    for row in range(arr.shape[0] // N):
        for col in range(arr.shape[1] // N):
            f_block = arr[row * N : (row + 1) * N, col * N : (col + 1) * N]
            # w
            if col == 0:
                w_block = np.fliplr(f_block)
            else:
                w_block = arr[row * N : (row + 1) * N, (col - 1) * N : col * N]
            # e
            if col == arr.shape[1] // N - 1:
                e_block = np.fliplr(f_block)
            else:
                e_block = arr[row * N : (row + 1) * N, (col + 1) * N : (col + 2) * N]
            # n
            if row == 0:
                n_block = np.flipud(f_block)
            else:
                n_block = arr[(row - 1) * N : row * N, col * N : (col + 1) * N]
            # s
            if row == arr.shape[0] // N - 1:
                s_block = np.flipud(f_block)
            else:
                s_block = arr[(row + 1) * N : (row + 2) * N, col * N : (col + 1) * N]

            w_d1 = f_block[:, 0] - w_block[:, N-1]
            e_d1 = f_block[:, N-1] - e_block[:, 0]
            n_d1 = f_block[0, :] - n_block[N-1, :]
            s_d1 = f_block[N-1, :] - s_block[0, :]

            w_d2 = (w_block[:, N-1] - w_block[:, N-2] + f_block[:, 1] - f_block[:, 0]) / 2
            e_d2 = (e_block[:, 1] - e_block[:, 0] + f_block[:, N-1] - f_block[:, N-2]) / 2
            n_d2 = (n_block[N-1, :] - n_block[N-2, :] + f_block[1, :] - f_block[0, :]) / 2
            s_d2 = (s_block[1, :] - s_block[0, :] + f_block[N-1, :] - f_block[N-2, :]) / 2

            w_e += np.sum((w_d1 - w_d2) ** 2 )
            e_e += np.sum((e_d1 - e_d2) ** 2 )
            n_e += np.sum((n_d1 - n_d2) ** 2)
            s_e += np.sum((s_d1 - s_d2) ** 2)

            # nw
            if row == 0 or col == 0:
                nw_block = np.flipud(np.fliplr(f_block))
            else:
                nw_block = arr[(row - 1) * N : row * N, (col - 1) * N : col * N]
            # ne
            if row == 0 or col == arr.shape[1] // N - 1:
                ne_block = np.flipud(np.fliplr(f_block))
            else:
                ne_block = arr[(row-1) * N : row * N, (col + 1) * N : (col + 2) * N]
            # sw
            if row == arr.shape[0] // N -1 or col == 0:
                sw_block = np.flipud(np.fliplr(f_block))
            else:
                sw_block = arr[row * N : (row+1) * N, (col-1) * N : col * N]
            # se
            if  row == arr.shape[0]//N-1 or col == arr.shape[0] // N -1:
                se_block = np.flipud(np.fliplr(f_block))
            else:
                se_block = arr[(row + 1) * N : (row + 2) * N, (col+1) * N : (col + 2) * N]

            nw_g1 = f_block[0, 0] - nw_block[N-1, N-1]
            ne_g1 = f_block[0, N-1] - ne_block[N-1, 0]
            sw_g1 = f_block[N-1, 0] - sw_block[0, N-1]
            se_g1 = f_block[N-1, N-1] - se_block[0, 0]

            nw_g2 = (nw_block[N-1,N-1] - nw_block[N-2,N-2] + f_block[1,1] - f_block[0,0])/2
            ne_g2 = (ne_block[N-1,0] - ne_block[N-2,1] + f_block[1,N-2] - f_block[0,N-1])/2
            sw_g2 = (sw_block[0,N-1] - nw_block[1,N-2] + f_block[N-2,1] - f_block[N-1,0])/2
            se_g2 = (nw_block[0,0] - nw_block[1,1] + f_block[N-2,N-2] - f_block[N-1,N-1])/2

            nw_e += (nw_g1 - nw_g2) ** 2
            ne_e += (ne_g1 - ne_g2) ** 2
            sw_e += (sw_g1 - sw_g2) ** 2
            se_e += (se_g1 - se_g2) ** 2

    MSDSt = (w_e + e_e + n_e + s_e + nw_e + ne_e + sw_e + se_e)/ ((arr.shape[0]/N)**2)
    MSDS1 = (w_e + e_e + n_e + s_e)/ ((arr.shape[0]/N)**2)
    MSDS2 = (nw_e + ne_e + sw_e + se_e)/ ((arr.shape[0]/N)**2)
    return MSDSt, MSDS1, MSDS2
```

```python
class DMLCT:
    def __init__(self, n_bar, N):
        self.n_bar = n_bar
        self.N = N

        self.x_l = (2 * np.arange(N) + 1) / (2 * N)
        self.s_l = np.arange(n_bar) / (n_bar - 1)
        self.xi = (np.arange(n_bar + 1) - 0.5) / (n_bar - 1)

        self.lambda_kh = self.get_lambda_kh(self.n_bar)

        self.w_k_j = self.get_w_k_j(self.n_bar, self.N)
        self.W_L_k_kh = self.get_W_L_k_kh(self.n_bar, self.N)
        self.W_k_kh = self.get_W_k_kh(self.n_bar, self.N)
        self.W_R_k_kh = self.get_W_R_k_kh(self.n_bar, self.N)

    def Lagrange_j(self, j):
        x = sympy.Symbol("x")
        L_x = 1.0
        for l in range(self.n_bar):
            if l != j:
                L_x *= (x - self.s_l[l]) / (self.s_l[j] - self.s_l[l])
        return sympy.integrate(L_x)

    def get_lambda_kh(self, n_bar):
        lambda_kh = np.ones(n_bar)
        lambda_kh[0] = np.sqrt(1 / 2)
        return lambda_kh

    def get_w_k_j(self, n_bar, N):
        L_j = np.zeros((n_bar, N))
        x = sympy.Symbol("x")
        for j in range(n_bar):
            temp = []
            Lj = self.Lagrange_j(j)
            for k in range(N):
                temp.append(Lj.subs(x, self.x_l[k]))
            L_j[j] = np.array(temp)

        w_k_j = np.zeros((n_bar, N))
        for j in range(n_bar):
            w_k_j[j] = scipy.fftpack.dct(L_j[j], norm="ortho")
        return w_k_j

    def get_W_L_k_kh(self, n_bar, N):
        W_L_k_kh = np.zeros((n_bar - 1, N))
        lambda_kh = self.get_lambda_kh(n_bar)

        for kh in range(n_bar - 1):
            W_L_k_kh[kh] = (
                (1 - n_bar)
                * np.sqrt(2 / N)
                * lambda_kh[kh]
                * np.cos(np.pi * kh * (self.xi[0] + 1))
                * self.w_k_j[0]
            )
        return W_L_k_kh

    def get_W_k_kh(self, n_bar, N):
        W_k_kh = np.zeros((n_bar - 1, N))
        for kh in range(n_bar - 1):
            sum_sin = np.zeros(N)
            for j in range(1, n_bar - 2 + 1):
                sum_sin += np.sin(np.pi * kh * self.s_l[j]) * self.w_k_j[j]

            W_k_kh[kh] = (
                (n_bar - 1)
                * np.sqrt(2 / N)
                * self.lambda_kh[kh]
                * (
                    np.cos(np.pi * kh * self.xi[1])
                    * (self.w_k_j[0] - (-1) ** (kh) * self.w_k_j[n_bar - 1])
                    - 2 * np.sin((np.pi * kh) / (2 * (n_bar - 1))) * sum_sin
                )
            )
        return W_k_kh

    def get_W_R_k_kh(self, n_bar, N):
        W_R_k_kh = np.zeros((n_bar - 1, N))
        for kh in range(n_bar - 1):
            W_R_k_kh[kh] = (
                (n_bar - 1)
                * np.sqrt(2 / N)
                * self.lambda_kh[kh]
                * np.cos(np.pi * kh * (self.xi[n_bar] - 1))
                * self.w_k_j[n_bar - 1]
            )
        return W_R_k_kh
```

```python
def get_F_L_k_horizontal(arr, N, row, col):
    # w
    if col == 0:
#         w_block = np.zeros(N)
        w_block = arr[row, col * N : (col + 1) * N]
    else:
        w_block = arr[row, (col - 1) * N : col * N]

    return w_block
```

```python
def get_F_R_k_horizontal(arr, N, row, col):
    # e
    if col == arr.shape[1] // N - 1:
#         e_block = np.zeros(N)
        e_block = arr[row, col * N : (col + 1) * N]
    else:
        e_block = arr[row, (col + 1) * N : (col + 2) * N]

    return e_block
```

```python
def get_F_L_k_vertical(arr, N, row, col):
    # n
    if row == 0:
#         n_block = np.zeros(N)
        n_block = arr[row * N : (row + 1) * N, col]
    else:
        n_block = arr[(row - 1) * N : row * N, col]

    return n_block
```

```python
def get_F_R_k_vertical(arr, N, row, col):
    # s
    if row == arr.shape[0] // N - 1:
#         s_block = np.zeros(N)
        s_block = arr[row * N : (row + 1) * N, col]
    else:
        s_block = arr[(row + 1) * N : (row + 2) * N, col]

    return s_block
```

```python
# dmlct = DMLCT(n_bar, N)
```

```python
IMG = LENNA
# IMG = ImageLoader(MONO_DIR_PATH + "LENNA.bmp")
```

```python
Fk = np.zeros(IMG.img.shape)
```

# 順変換

## 縦方向

### DCT

```python
for row in range(IMG.img.shape[0] // N):
    for col in range(IMG.img.shape[1]):
        eight_points = IMG.img[N * row : N * (row + 1), col]
        c = scipy.fftpack.dct(eight_points, norm="ortho")
        Fk[N * row : N * (row + 1), col] = c
```

### 残差

```python
dmlct = DMLCT(n_bar, N)
for row in range(IMG.img.shape[0] // N):
    for col in range(IMG.img.shape[1]):
        # ビューなら直接いじっちゃう
        F = Fk[N * row : N * (row + 1), col]
        F_L = get_F_L_k_vertical(Fk, N, row, col)
        F_R = get_F_R_k_vertical(Fk, N, row, col)

        U_k_n_bar = np.zeros(N)
        for kh in range(n_bar - 2 + 1):
            U_k_n_bar += (
                F_L[kh] * dmlct.W_L_k_kh[kh]
                + F[kh] * dmlct.W_k_kh[kh]
                + F_R[kh] * dmlct.W_R_k_kh[kh]
            )

        # n_bar = 4 なら 0,1,2は残す 3,4,5,6,7を書き換える
        F[n_bar - 2 + 1 :] -= U_k_n_bar[n_bar - 2 + 1 :]
```

```python
# 0を残す
for k in reversed(range(1, n_bar - 2 + 1)):
    dmlct = DMLCT(k+1, N)
    for row in range(IMG.img.shape[0] // N):
        for col in range(IMG.img.shape[1]):
            # ビューなら直接いじっちゃう
            F = Fk[N * row : N * (row + 1), col]
            F_L = get_F_L_k_vertical(Fk, N, row, col)
            F_R = get_F_R_k_vertical(Fk, N, row, col)

            U_k_n_bar = np.zeros(N)

            for kh in range((k + 1) - 2 + 1):
                U_k_n_bar += (
                    F_L[kh] * dmlct.W_L_k_kh[kh]
                    + F[kh] * dmlct.W_k_kh[kh]
                    + F_R[kh] * dmlct.W_R_k_kh[kh]
                )

            F[k] -= U_k_n_bar[k]
```

## 横方向

### DCT

```python
for row in range(Fk.shape[0]):
    for col in range(Fk.shape[1] // N):
        eight_points = Fk[row, N * col : N * (col + 1)]
        c = scipy.fftpack.dct(eight_points, norm="ortho")
        Fk[row, N * col : N * (col + 1)] = c
```

### 残差

```python
dmlct = DMLCT(n_bar, N)
for row in range(IMG.img.shape[0]):
    for col in range(IMG.img.shape[1] // N):
        F = Fk[row, N * col : N * (col + 1)]
        F_L = get_F_L_k_horizontal(Fk, N, row, col)
        F_R = get_F_R_k_horizontal(Fk, N, row, col)

        U_k_n_bar = np.zeros(N)
        for kh in range(n_bar - 2 + 1):
            U_k_n_bar += (
                F_L[kh] * dmlct.W_L_k_kh[kh]
                + F[kh] * dmlct.W_k_kh[kh]
                + F_R[kh] * dmlct.W_R_k_kh[kh]
            )

        # n_bar = 4 なら 0,1,2は残す 3,4,5,6,7を書き換える
        F[n_bar - 2 + 1 :] -= U_k_n_bar[n_bar - 2 + 1 :]
```

```python
# 0を残す
for k in reversed(range(1, n_bar - 2 + 1)):
    dmlct = DMLCT(k+1, N)
    for row in range(IMG.img.shape[0]):
        for col in range(IMG.img.shape[1] // N):
            F = Fk[row, N * col : N * (col + 1)]
            F_L = get_F_L_k_horizontal(Fk, N, row, col)
            F_R = get_F_R_k_horizontal(Fk, N, row, col)

            U_k_n_bar = np.zeros(N)

            for kh in range((k + 1) - 2 + 1):
                U_k_n_bar += (
                    F_L[kh] * dmlct.W_L_k_kh[kh]
                    + F[kh] * dmlct.W_k_kh[kh]
                    + F_R[kh] * dmlct.W_R_k_kh[kh]
                )

            F[k] -= U_k_n_bar[k]
```

```python
plt.imshow(Fk)
# pd.DataFrame(Fk).to_csv("DMLCT_lenna_hiroya_coef.csv",header=False,index=False)
```

# 係数の確保

```python
Fk_Ori = np.copy(Fk)
```

# 逆変換

```python
recover = np.zeros(IMG.img.shape)
```

## 横方向

### 残差

```python
for k in range(1, n_bar - 2 + 1):
    dmlct = DMLCT(k+1, N)
    for row in range(IMG.img.shape[0]):
        for col in range(IMG.img.shape[1] // N):
            F = Fk[row, N * col : N * (col + 1)]
            F_L = get_F_L_k_horizontal(Fk, N, row, col)
            F_R = get_F_R_k_horizontal(Fk, N, row, col)

            U_k_n_bar = np.zeros(N)
            for kh in range((k + 1) - 2 + 1):
                U_k_n_bar += (
                    F_L[kh] * dmlct.W_L_k_kh[kh]
                    + F[kh] * dmlct.W_k_kh[kh]
                    + F_R[kh] * dmlct.W_R_k_kh[kh]
                )

            F[k] += U_k_n_bar[k]
```

```python
dmlct = DMLCT(n_bar, N)
for row in range(IMG.img.shape[0]):
    for col in range(IMG.img.shape[1] // N):
        F = Fk[row, N * col : N * (col + 1)]
        F_L = get_F_L_k_horizontal(Fk, N, row, col)
        F_R = get_F_R_k_horizontal(Fk, N, row, col)

        U_k_n_bar = np.zeros(N)
        for kh in range(n_bar - 2 + 1):
            U_k_n_bar += (
                F_L[kh] * dmlct.W_L_k_kh[kh]
                + F[kh] * dmlct.W_k_kh[kh]
                + F_R[kh] * dmlct.W_R_k_kh[kh]
            )

        # n_bar = 4 なら 0,1,2は残す 3,4,5,6,7を書き換える
        F[n_bar - 2 + 1 :] += U_k_n_bar[n_bar - 2 + 1 :]
```

### IDCT

```python
for row in range(Fk.shape[0]):
    for col in range(Fk.shape[1] // N):
        F = Fk[row, N * col : N * col + N]
        data = scipy.fftpack.idct(F, norm="ortho")
        # Fkに代入した後、縦方向に対して処理
        Fk[row, N * col : N * col + N] = data
```

## 縦方向

### 残差

```python
for k in range(1, n_bar - 2 + 1):
    dmlct = DMLCT(k+1, N)
    for row in range(IMG.img.shape[0] // N):
        for col in range(IMG.img.shape[1]):
            # ビューなら直接いじっちゃう
            F = Fk[N * row : N * (row + 1), col]
            F_L = get_F_L_k_vertical(Fk, N, row, col)
            F_R = get_F_R_k_vertical(Fk, N, row, col)

            U_k_n_bar = np.zeros(N)

            for kh in range((k + 1) - 2 + 1):
                U_k_n_bar += (
                    F_L[kh] * dmlct.W_L_k_kh[kh]
                    + F[kh] * dmlct.W_k_kh[kh]
                    + F_R[kh] * dmlct.W_R_k_kh[kh]
                )

            F[k] += U_k_n_bar[k]
```

```python
dmlct = DMLCT(n_bar, N)
for row in range(IMG.img.shape[0] // N):
    for col in range(IMG.img.shape[1]):
        # ビューなら直接いじっちゃう
        F = Fk[N * row : N * (row + 1), col]
        F_L = get_F_L_k_vertical(Fk, N, row, col)
        F_R = get_F_R_k_vertical(Fk, N, row, col)

        U_k_n_bar = np.zeros(N)
        for kh in range(n_bar - 2 + 1):
            U_k_n_bar += (
                F_L[kh] * dmlct.W_L_k_kh[kh]
                + F[kh] * dmlct.W_k_kh[kh]
                + F_R[kh] * dmlct.W_R_k_kh[kh]
            )

        # n_bar = 4 なら 0,1,2は残す 3,4,5,6,7を書き換える
        F[n_bar - 2 + 1 :] += U_k_n_bar[n_bar - 2 + 1 :]
```

### IDCT

```python
for row in range(Fk.shape[0] // N):
    for col in range(Fk.shape[1]):
        F = Fk[N * row : N * (row + 1), col]
        data = scipy.fftpack.idct(F, norm="ortho")
        # 復元画像
        recover[N * row : N * (row + 1), col] = data
```

```python
plt.imshow(recover.astype("u8"), cmap="gray")
```

もどった...！

# 量子化テーブル

```python
Q = 108
Q_Luminance = np.ones((N,N)) * Q
```

# 量子化

```python
Fk = np.copy(Fk_Ori)
Q_Fk = np.zeros(Fk.shape)
for row in range(IMG.img.shape[0] // N):
    for col in range(IMG.img.shape[1] // N):
        block = Fk[row * N : (row + 1) * N, col * N : (col + 1) * N]
        # 量子化
        block = np.round(block / Q_Luminance)
        # 逆量子化
        block = block * Q_Luminance
        Q_Fk[row * N : (row+1)*N, col * N : (col+1)*N] = block
```

```python
Fk = np.copy(Q_Fk)
Q_recover = np.zeros(Q_Fk.shape)
```

## 横方向

### 残差

```python
for k in range(1, n_bar - 2 + 1):
    dmlct = DMLCT(k+1, N)
    for row in range(IMG.img.shape[0]):
        for col in range(IMG.img.shape[1] // N):
            F = Fk[row, N * col : N * (col + 1)]
            F_L = get_F_L_k_horizontal(Fk, N, row, col)
            F_R = get_F_R_k_horizontal(Fk, N, row, col)

            U_k_n_bar = np.zeros(N)
            for kh in range((k + 1) - 2 + 1):
                U_k_n_bar += (
                    F_L[kh] * dmlct.W_L_k_kh[kh]
                    + F[kh] * dmlct.W_k_kh[kh]
                    + F_R[kh] * dmlct.W_R_k_kh[kh]
                )

            F[k] += U_k_n_bar[k]
```

```python
dmlct = DMLCT(n_bar, N)
for row in range(IMG.img.shape[0]):
    for col in range(IMG.img.shape[1] // N):
        F = Fk[row, N * col : N * (col + 1)]
        F_L = get_F_L_k_horizontal(Fk, N, row, col)
        F_R = get_F_R_k_horizontal(Fk, N, row, col)

        U_k_n_bar = np.zeros(N)
        for kh in range(n_bar - 2 + 1):
            U_k_n_bar += (
                F_L[kh] * dmlct.W_L_k_kh[kh]
                + F[kh] * dmlct.W_k_kh[kh]
                + F_R[kh] * dmlct.W_R_k_kh[kh]
            )

        # n_bar = 4 なら 0,1,2は残す 3,4,5,6,7を書き換える
        F[n_bar - 2 + 1 :] += U_k_n_bar[n_bar - 2 + 1 :]
```

### IDCT

```python
for row in range(Fk.shape[0]):
    for col in range(Fk.shape[1] // N):
        F = Fk[row, N * col : N * col + N]
        data = scipy.fftpack.idct(F, norm="ortho")
        # Fkに代入した後、縦方向に対して処理
        Fk[row, N * col : N * col + N] = data
```

## 縦方向

### 残差

```python
for k in range(1, n_bar - 2 + 1):
    dmlct = DMLCT(k+1, N)
    for row in range(IMG.img.shape[0] // N):
        for col in range(IMG.img.shape[1]):
            # ビューなら直接いじっちゃう
            F = Fk[N * row : N * (row + 1), col]
            F_L = get_F_L_k_vertical(Fk, N, row, col)
            F_R = get_F_R_k_vertical(Fk, N, row, col)

            U_k_n_bar = np.zeros(N)

            for kh in range((k + 1) - 2 + 1):
                U_k_n_bar += (
                    F_L[kh] * dmlct.W_L_k_kh[kh]
                    + F[kh] * dmlct.W_k_kh[kh]
                    + F_R[kh] * dmlct.W_R_k_kh[kh]
                )

            F[k] += U_k_n_bar[k]
```

```python
dmlct = DMLCT(n_bar, N)
for row in range(IMG.img.shape[0] // N):
    for col in range(IMG.img.shape[1]):
        # ビューなら直接いじっちゃう
        F = Fk[N * row : N * (row + 1), col]
        F_L = get_F_L_k_vertical(Fk, N, row, col)
        F_R = get_F_R_k_vertical(Fk, N, row, col)

        U_k_n_bar = np.zeros(N)
        for kh in range(n_bar - 2 + 1):
            U_k_n_bar += (
                F_L[kh] * dmlct.W_L_k_kh[kh]
                + F[kh] * dmlct.W_k_kh[kh]
                + F_R[kh] * dmlct.W_R_k_kh[kh]
            )

        # n_bar = 4 なら 0,1,2は残す 3,4,5,6,7を書き換える
        F[n_bar - 2 + 1 :] += U_k_n_bar[n_bar - 2 + 1 :]
```

### IDCT

```python
for row in range(Fk.shape[0] // N):
    for col in range(Fk.shape[1]):
        F = Fk[N * row : N * (row + 1), col]
        data = scipy.fftpack.idct(F, norm="ortho")
        # 復元画像
        Q_recover[N * row : N * (row + 1), col] = data
```

```python
Q_recover = np.round(Q_recover)
```

```python
plt.imshow(Q_recover, cmap="gray")
plt.imsave("DMLCT_16x16_n"+str(n_bar)+ "_LENNA.png",Q_recover,cmap="gray")
```

# 情報量

```python
qfk = pd.Series(Q_Fk.flatten())
pro = qfk.value_counts() / qfk.value_counts().sum()
pro.head()
```

    -0.0      0.981468
     108.0    0.007042
    -108.0    0.007008
     216.0    0.001083
    -216.0    0.000996
    dtype: float64

```python
S = 0
for pi in pro:
    S -= pi * np.log2(pi)
S
```

    0.1806067711578881

# PSNR

```python
MSE = np.sum(np.sum(np.power((IMG.img - Q_recover),2)))/(Q_recover.shape[0] * Q_recover.shape[1])
PSNR = 10 * np.log10(255 * 255 / MSE)
PSNR
```

    28.991609974679005

# MSSIM

```python
MSSIM = ssim(IMG.img,Q_recover.astype(IMG.img.dtype),gaussian_weights=True,sigma=1.5,K1=0.01,K2=0.03)
MSSIM
```

    0.7870765007405786

```python
dmlct = DMLCT(n_bar, N)
```

# MSDS

```python
MSDSt, MSDS1, MSDS2 = msds(N,Q_recover)
```

```python
MSDS1
```

    39983.40234375

```python
MSDS2
```

    6007.1181640625

```python

```


```python
import numpy as np
from matplotlib import pyplot as plt
from pprint import pprint
np.set_printoptions(precision=4, linewidth=120, floatmode='maxprec_equal')
```

```python
import pygsti
import pygsti.extras.interpygate as interp
from pygsti.tools.basistools import change_basis
from pygsti.modelpacks import smq1Q_XY
```

```python
from pathlib import Path
working_dir = Path.cwd()
```

## Build model gate

```python
sigI = np.array([[1.,0],[0, 1]], dtype='complex')
sigX = np.array([[0, 1],[1, 0]], dtype='complex')
sigY = np.array([[0,-1],[1, 0]], dtype='complex') * 1.j
sigZ = np.array([[1, 0],[0,-1]], dtype='complex')
sigM = (sigX - 1.j*sigY)/2.
sigP = (sigX + 1.j*sigY)/2.
```

```python
class SingleQubitTargetOp(pygsti.modelmembers.operations.OpFactory):

    def __init__(self):
        self.process = self.create_target_gate
        pygsti.modelmembers.operations.OpFactory.__init__(self, 4, evotype="densitymx")
        self.dim = 4

    def create_target_gate(self, v):

        phi, theta = v
        target_unitary = (np.cos(theta/2) * sigI +
                          1.j * np.sin(theta/2) * (np.cos(phi) * sigX + np.sin(phi) * sigY))
        superop = change_basis(np.kron(target_unitary.conj(), target_unitary), 'col', 'pp')

        return superop

    def create_object(self, args=None, sslbls=None):
        assert(sslbls is None)
        mx = self.process([*args])
        return pygsti.modelmembers.operations.StaticArbitraryOp(mx)
```

```python
class SingleQubitGate(interp.PhysicalProcess):
    def __init__(self,
                 verbose=False,
                 cont_param_gate = False,
                 num_params = None,
#                  process_shape = (4, 4),
                 item_shape = (4,4),
                 aux_shape = None,
                 num_params_evaluated_as_group = 0,
                 ):

        self.verbose = verbose

        self.cont_param_gate = cont_param_gate

        self.num_params = num_params
        self.item_shape = item_shape

        self.aux_shape = aux_shape
        self.num_params_evaluated_as_group = num_params_evaluated_as_group

    def create_process_matrix(self, v, comm=None, return_generator=False):

        processes = []
        phi, theta, t = v
        theta = theta * t
        target_unitary = (np.cos(theta/2) * sigI +
                          1.j * np.sin(theta/2) * (np.cos(phi) * sigX + np.sin(phi) * sigY))
        superop = change_basis(np.kron(target_unitary.conj(), target_unitary), 'col', 'pp')
        processes += [superop]
        return np.array(processes) if (processes is not None) else None

    def create_aux_info(self, v, comm=None):
        return []  # matches aux_shape=() above

    def create_process_matrices(self, v, grouped_v, comm=None):
        assert(len(grouped_v) == 1)  # we expect a single "grouped" parameter

        processes = []
        times = grouped_v[0]
        phi_in, theta_in = v
        for t in times:
            phi = phi_in
            theta = theta_in * t
            target_unitary = (np.cos(theta/2) * sigI +
                              1.j * np.sin(theta/2) * (np.cos(phi) * sigX + np.sin(phi) * sigY))
            superop = change_basis(np.kron(target_unitary.conj(), target_unitary), 'col', 'pp')
            processes += [superop]
        return np.array(processes) if (processes is not None) else None

    def create_aux_infos(self, v, grouped_v, comm=None):
        import numpy as np
        times = grouped_v[0]
        return [ [] for t in times] # list elements must match aux_shape=() above
```

```python
param_ranges = [(0.9,1.1,3)]

arg_ranges = [2*np.pi*(1+np.cos(np.linspace(np.pi,0, 7)))/2,
              (0, np.pi, 3)]
arg_indices = [0,1]

target_op = SingleQubitTargetOp()
gate_process = SingleQubitGate(num_params = 3,num_params_evaluated_as_group = 1)
```

```python
opfactory_linear = interp.InterpolatedOpFactory.create_by_interpolating_physical_process(
    target_op, gate_process, argument_ranges=arg_ranges, parameter_ranges=param_ranges,
    argument_indices=arg_indices, interpolator_and_args='linear')

opfactory_spline = interp.InterpolatedOpFactory.create_by_interpolating_physical_process(
    target_op, gate_process, argument_ranges=arg_ranges, parameter_ranges=param_ranges,
    argument_indices=arg_indices, interpolator_and_args='spline')

```

### Check that the interpolator is working

```python
if False:
    indices = (2,3)
    nparams = 30

    x = np.linspace(0,2*np.pi, nparams)
    y = np.linspace(0, np.pi, nparams)
    for p in np.linspace(.9,1.1,5):

        def interp_linear(x, y):
            op = opfactory_linear.create_op([x, y])
            return op.base_interpolator([x,y,p])[indices]

        def interp_spline(x, y):
            op = opfactory_spline.create_op([x, y])
            return op.base_interpolator([x,y,p])[indices]

        def truth(x, y):
            tmp_gate = gate_process.create_process_matrix([x,y,p])[0]
            tar_gate = target_op.create_target_gate([x,y])
            return pygsti.error_generator(tmp_gate, tar_gate, 'pp', 'logGTi')[indices]

        X, Y = np.meshgrid(x, y, indexing='ij')
        Z_linear = np.zeros([nparams, nparams])
        Z_spline = np.zeros([nparams, nparams])
        Z_truth  = np.zeros([nparams, nparams])
        for idx, xx in enumerate(x):
            for idy, yy in enumerate(y):
                Z_linear[idx,idy] = interp_linear(xx,yy)
                Z_spline[idx,idy] = interp_spline(xx,yy)
                Z_truth[idx,idy]  = truth(xx,yy)

        fig = plt.figure(figsize=(10,10))
        ax = plt.axes(projection='3d')

        ax.plot_surface(X, Y, Z_linear-Z_truth, rstride=1, cstride=1,
                        edgecolor='none', alpha=.8)
        ax.plot_surface(X, Y, Z_spline-Z_truth, rstride=1, cstride=1,
                        cmap='viridis', edgecolor='none', alpha=.8)

        ax.set_xlabel('x')
        ax.set_ylabel('y')
        ax.set_zlabel('z')
    #     ax.set_zlim(-1,1)
        plt.show()
```

# Build a model from this gate

```python
x_gate = opfactory_spline.create_op([0,np.pi/4])
y_gate = opfactory_spline.create_op([np.pi/2,np.pi/4])
```

```python
x_gate.from_vector([1.03])
y_gate.from_vector([1.0])
print(np.round_(x_gate,4))
print()
print(np.round_(y_gate,4))
```

    [[ 1.0000  0.0000 -0.0000  0.0000]
     [ 0.0000  1.0000  0.0000  0.0000]
     [ 0.0000  0.0000  0.6903  0.7236]
     [ 0.0000  0.0000 -0.7236  0.6903]]

    [[ 1.0000  0.0000 -0.0000  0.0000]
     [ 0.0000  0.7071  0.0000 -0.7071]
     [-0.0000 -0.0000  1.0000 -0.0000]
     [ 0.0000  0.7071 -0.0000  0.7071]]

## Make a fake dataset

```python
#Model only has Gx and Gy gates.  Let's rename them.

model = pygsti.models.ExplicitOpModel([0],'pp')

model['rho0'] = [ 1/np.sqrt(2), 0, 0, 1/np.sqrt(2) ] # density matrix [[1, 0], [0, 0]] in Pauli basis
model['Mdefault'] = pygsti.modelmembers.povms.UnconstrainedPOVM(
    {'0': [ 1/np.sqrt(2), 0, 0, 1/np.sqrt(2) ],   # projector onto [[1, 0], [0, 0]] in Pauli basis
     '1': [ 1/np.sqrt(2), 0, 0, -1/np.sqrt(2) ] }, evotype="default") # projector onto [[0, 0], [0, 1]] in Pauli basis
model['Gxpi2',0] = x_gate
model['Gypi2',0] = y_gate
```

```python
model.num_params
```

    14

```python
# Define the error model used to generate data
datagen_model = model.copy()
datagen_params = datagen_model.to_vector()
datagen_params[-2:] = [1.03,1.00]
datagen_model.from_vector(datagen_params)
datagen_model.probabilities( (('Gxpi2',0),('Gypi2',0),))
```

    OutcomeLabelDict([(('0',), 0.7440406663460976), (('1',), 0.2559593336539022)])

```python
model._print_gpindices()
```

    PRINTING MODEL GPINDICES!!!
    >>> rho0 [FullState]: 4 params, indices=slice(0, 4, None)
    >>> Mdefault [UnconstrainedPOVM]: 8 params, indices=slice(4, 12, None)
    >>> Gxpi2:0 [InterpolatedDenseOp]: 1 params, indices=slice(12, 13, None)
    >>> Gypi2:0 [InterpolatedDenseOp]: 1 params, indices=slice(13, 14, None)

```python
# # Link the over-rotation errors on Gx and Gy
# model['rho0'].set_gpindices(slice(0,4),model)
# model['Mdefault'].set_gpindices(slice(4,12),model)
# model['Gxpi2',0].set_gpindices(slice(12,13),model)
# model['Gypi2',0].set_gpindices(slice(12,13),model)
# model._rebuild_paramvec()
# model._print_gpindices()
print(model.num_params)

# # Define the error model used to generate data
# datagen_model = model.copy()
# datagen_params = datagen_model.to_vector()
# datagen_params[-1:] = [1.02]
# datagen_model.from_vector(datagen_params)
# datagen_model.probabilities( (('Gxpi2',0),('Gypi2',0),))
```

    14

```python
# Define the perfect target model
target_model = model.copy()
target_params = target_model.to_vector()
target_params[-2:] = [1,1]
# target_model.from_vector(target_params)
target_model.probabilities( (('Gxpi2',0),('Gypi2',0),))
```

    OutcomeLabelDict([(('0',), 0.7440406663460976), (('1',), 0.2559593336539022)])

### Germ and fiducial selection

```python
final_germs = pygsti.algorithms.germselection.find_germs(
                target_model, randomize=False, force=None, algorithm='greedy',
                verbosity=4, num_nongauge_params=2)
```

    Using greedy algorithm.
    Complete initial germ set succeeds on all input models.
    Now searching for best germ set.
    Starting germ set optimization. Lower score is better.
      Outer iteration: 1 of 2 amplified, 0 germs
        Inner iter over candidate germs [##################################################] 100.0% Gxpi2:0Gypi2:0Gypi2:0Gypi2:0Gypi2:0Gypi2:0@(0)
        Added Gypi2:0@(0) to final germs (Score: major=-1.0 minor=0.8105694654717684, N: 1)
      Outer iteration: 1 of 2 amplified, 1 germs
        Inner iter over candidate germs [##################################################] 100.0% Gxpi2:0Gypi2:0Gypi2:0Gypi2:0Gypi2:0Gypi2:0@(0)
        Added Gxpi2:0@(0) to final germs (Score: major=-2.0 minor=1.621138935590229, N: 2)
      Outer iteration: 2 of 2 amplified, 2 germs
    Constructed germ set:
    ['Gypi2:0@(0)', 'Gxpi2:0@(0)']
    Score: Score: major=0.0 minor=inf, N: 0

```python
fiducial_pairs = pygsti.algorithms.fiducialpairreduction.find_sufficient_fiducial_pairs_per_germ(
                                model,
                                smq1Q_XY.prep_fiducials([0]),
                                smq1Q_XY.meas_fiducials([0]),
                                final_germs)
```

```python
# # Reduce the number of fiducial pairs by hand, if you want

# fiducial_pairs2 = fiducial_pairs.copy()
# for key in fiducial_pairs2.keys():
#     fiducial_pairs2[key] = fiducial_pairs2[key][0:2]
# fiducial_pairs = fiducial_pairs2

# print(fiducial_pairs)
```

    {Circuit(Gypi2:0@(0)): [(0, 0), (0, 1)], Circuit(Gxpi2:0@(0)): [(0, 0), (0, 1)]}

```python
working_dir = Path.cwd() / 'example_files'

# get experiment design from smq1Q_XY
# exp_design = smq1Q_XY.get_gst_experiment_design(max_max_length=32,qubit_labels=('Q0',))

# Use fiducial pair reductions
exp_design = pygsti.protocols.StandardGSTDesign(model,
                                                smq1Q_XY.prep_fiducials([0]),
                                                smq1Q_XY.meas_fiducials([0]),
                                                final_germs,
                                                max_lengths=[1,2,4,8,16,32,64,128,256],
                                                fiducial_pairs=fiducial_pairs,
                                                include_lgst=False)

# write an empty data object (creates a template to fill in)
pygsti.io.write_empty_protocol_data(exp_design, working_dir, clobber_ok=True)

# fill in the template with simulated data (you would run the experiment and use actual data)
pygsti.io.fill_in_empty_dataset_with_fake_data(
    datagen_model,
    working_dir / 'data/dataset.txt', num_samples=1000, seed=1234)

# load the data object back in, now with the experimental data
data = pygsti.io.load_data_from_dir(working_dir)
```

```python
len(data.dataset)
```

    35

## Fisher information matrix

What is the expected covariance in the estimated parameters given the specified experiment? Assuming the GST estimator is unbiased,

$$ \mathrm{cov}\left(\left\langle \vec \theta \right\rangle\right) \le I(\theta)^{-1}$$

Where $I(\vec\theta)$ is the Fisher information matrix, with entries:

$$ I_{m,n} = - \left\langle \frac{\partial^2}{\partial \theta_m \partial \theta_n} \mathrm{log} f(x;\vec\theta) \right\rangle $$

Here $f(x;\vec\theta)$ is the likelihood function. For a single quantum circuit, it is equal to:

$$ f(x;\vec\theta) = \prod_i p_i(\theta)^{x_i} $$

Where $p_i(theta)$ is the probability of measuring the $i$th possible outcome, and $x_i$ is the number of times that outcome was observed. The logarithmic derivatives can be evaluated as:

$$
\begin{align}
I_{m,n} &= - \sum_i \left\langle x_i \frac{\partial^2}{\partial \theta_m \partial \theta_n}  \log p_i(\vec\theta)\right\rangle  \\
        &= - \sum_i \left\langle x_i \frac{\partial}{\partial \theta_m }  \frac{\partial \theta_n p_i(\vec\theta)}{p_i(\vec\theta)}\right\rangle  \\
        &= - \sum_i \left\langle x_i \left(\frac{\partial\theta_m \partial \theta_n p_i(\vec\theta)}{p_i(\vec\theta)} -
        \frac{(\partial\theta_m p_i(\vec\theta) )(\partial\theta_n p_i(\vec\theta))}{p_i(\vec\theta)^2} \right)
        \right\rangle  \\
I &= N \sum_i \left(
        \frac{(\nabla p_i(\vec\theta) )(\nabla p_i(\vec\theta))^\mathsf{T}}{p_i(\vec\theta)} -
        \nabla\nabla p_i(\vec\theta) \right) \vert_{\theta = \theta_0}\\
&= N \sum_i \left(
        \frac{J_i J_i^\mathsf{T}}{p_i} - H_i \right) \vert_{\theta = \theta_0}
\end{align}
$$

```python
def fisher_information_matrix(model, circuit_list):
    n_params = model.num_params
    outcomes = model.sim.probs(()).keys()
    fisher_information = np.zeros([n_params, n_params])
    for circuit in circuit_list:
        p = model.sim.probs (circuit)
        j = model.sim.dprobs(circuit)
        h = model.sim.hprobs(circuit)
        for outcome in outcomes:
            fisher_information += np.outer(j[outcome], j[outcome])/p[outcome] - h[outcome]
    return fisher_information
```

```python
fim = fisher_information_matrix(model, exp_design.all_circuits_needing_data[0:2])
```

```python
np.log(np.linalg.inv(fisher_information_matrix(model, exp_design.all_circuits_needing_data[0:6])))
```

    <ipython-input-99-7b56a9d3cdc6>:1: RuntimeWarning: invalid value encountered in log
      np.log(np.linalg.inv(fisher_information_matrix(model, exp_design.all_circuits_needing_data[0:6])))

    array([[ -2.4849,      nan, -35.1551,  -2.4849,      nan,      nan, -34.3222, -35.3782, -34.5673, -34.6851,      nan,
            -36.0714,      nan,      nan],
           [     nan,   1.3541,      nan,   0.9474,   1.3862,   0.7837,      nan,      nan,      nan,      nan,   0.5966,
              1.8351,   1.1788,   1.7805],
           [-35.2635,      nan,  -0.1295,      nan,      nan,      nan,  -0.9581,   1.0328,   0.5676,   0.3140,      nan,
                 nan,      nan,      nan],
           [ -2.4849,   0.9474,      nan,   0.5923,   1.0365,   0.5696,      nan,      nan,      nan,      nan,   0.0500,
              1.4359,   0.8999,   1.3422],
           [     nan,   1.3862,      nan,   1.0365,   1.9922,   1.6898,      nan,      nan,      nan,      nan,   1.4629,
              2.0662,   1.3962,   1.7764],
           [     nan,   0.7837,      nan,   0.5696,   1.6898,   1.5024,      nan,      nan,      nan,      nan,   1.1590,
              1.6392,   1.1709,   1.1678],
           [-34.8032,      nan,  -0.9581,      nan,      nan,      nan,   1.3086,   1.4033,   1.5180,   1.2039,      nan,
                 nan,      nan,      nan],
           [-33.8549,      nan,   1.0328,      nan,      nan,      nan,   1.4033,   2.4262,   2.0509,   1.6648,      nan,
                 nan,      nan,      nan],
           [-34.1413,      nan,   0.5676,      nan,      nan,      nan,   1.5180,   2.0509,   1.9922,   1.7073,      nan,
                 nan,      nan,      nan],
           [-34.4705,      nan,   0.3140,      nan,      nan,      nan,   1.2039,   1.6648,   1.7073,   1.5759,      nan,
                 nan,      nan,      nan],
           [     nan,   0.5966,      nan,   0.0500,   1.4629,   1.1590,      nan,      nan,      nan,      nan,   1.3967,
              1.2242,  -0.6442,   1.0996],
           [     nan,   1.8351,      nan,   1.4359,   2.0662,   1.6392,      nan,      nan,      nan,      nan,   1.2242,
              2.4309,   1.8259,   2.2329],
           [     nan,   1.1788,      nan,   0.8999,   1.3962,   1.1709,      nan,      nan,      nan,      nan,  -0.6442,
              1.8259,   1.7238,   1.4482],
           [     nan,   1.7805,      nan,   1.3422,   1.7764,   1.1678,      nan,      nan,      nan,      nan,   1.0996,
              2.2329,   1.4482,   2.2118]])

```python
np.linalg.eig(np.linalg.inv(fim))
```

    (array([ 2.5000e-01+0.0000e+00j, -3.9005e+15+1.1457e+15j, -3.9005e+15-1.1457e+15j,  5.3603e+14+0.0000e+00j,
             1.4888e+14+0.0000e+00j,  2.8210e+00+0.0000e+00j,  1.1441e+00+0.0000e+00j, -4.9274e-01+0.0000e+00j,
            -3.7677e-01+4.9547e-02j, -3.7677e-01-4.9547e-02j,  4.3818e-01+1.0126e-01j,  4.3818e-01-1.0126e-01j,
             2.4698e-01+3.9999e-02j,  2.4698e-01-3.9999e-02j]),
     array([[ 1.0000e+00+0.0000e+00j,  2.6254e-17-2.8793e-17j,  2.6254e-17+2.8793e-17j, -1.5293e-16+0.0000e+00j,
              7.8796e-18+0.0000e+00j, -6.6313e-02+0.0000e+00j, -2.5171e-01+0.0000e+00j,  1.6226e-01+0.0000e+00j,
              1.6653e-02-1.2934e-01j,  1.6653e-02+1.2934e-01j,  4.6007e-01+0.0000e+00j,  4.6007e-01-0.0000e+00j,
              9.1612e-01+0.0000e+00j,  9.1612e-01-0.0000e+00j],
            [ 0.0000e+00+0.0000e+00j,  5.4595e-03-1.4286e-03j,  5.4595e-03+1.4286e-03j,  6.0662e-02+0.0000e+00j,
             -4.8244e-01+0.0000e+00j, -2.5079e-02+0.0000e+00j, -1.3608e-01+0.0000e+00j,  1.1794e-01+0.0000e+00j,
              3.1224e-01+5.5154e-02j,  3.1224e-01-5.5154e-02j,  8.4144e-03-4.1106e-01j,  8.4144e-03+4.1106e-01j,
             -1.8597e-03-9.5862e-02j, -1.8597e-03+9.5862e-02j],
            [ 0.0000e+00+0.0000e+00j,  1.0742e-03-7.1716e-04j,  1.0742e-03+7.1716e-04j, -1.7696e-01+0.0000e+00j,
             -9.1593e-03+0.0000e+00j,  4.9275e-02+0.0000e+00j,  4.0996e-02+0.0000e+00j,  6.1700e-01+0.0000e+00j,
              5.1216e-01+0.0000e+00j,  5.1216e-01-0.0000e+00j,  1.4029e-01+1.2222e-01j,  1.4029e-01-1.2222e-01j,
             -2.7354e-02-1.4018e-01j, -2.7354e-02+1.4018e-01j],
            [ 0.0000e+00+0.0000e+00j, -3.2515e-04+2.1709e-04j, -3.2515e-04-2.1709e-04j,  5.3565e-02+0.0000e+00j,
              2.7725e-03+0.0000e+00j, -7.3487e-02+0.0000e+00j, -8.0255e-01+0.0000e+00j, -3.2937e-02+0.0000e+00j,
              1.0429e-02+7.7806e-02j,  1.0429e-02-7.7806e-02j, -1.9685e-01+1.2352e-01j, -1.9685e-01-1.2352e-01j,
             -8.9057e-02-3.9972e-02j, -8.9057e-02+3.9972e-02j],
            [ 0.0000e+00+0.0000e+00j,  5.6111e-02+8.2978e-02j,  5.6111e-02-8.2978e-02j, -1.4039e-02+0.0000e+00j,
              8.0565e-03+0.0000e+00j, -8.6914e-04+0.0000e+00j, -2.5048e-01+0.0000e+00j,  1.3138e-01+0.0000e+00j,
             -2.0173e-02-1.0589e-01j, -2.0173e-02+1.0589e-01j,  2.3022e-01-2.4351e-01j,  2.3022e-01+2.4351e-01j,
              6.5127e-02+9.2371e-02j,  6.5127e-02-9.2371e-02j],
            [ 0.0000e+00+0.0000e+00j,  5.0706e-01+0.0000e+00j,  5.0706e-01-0.0000e+00j, -2.3587e-02+0.0000e+00j,
              2.1322e-02+0.0000e+00j, -3.4552e-02+0.0000e+00j, -1.7008e-01+0.0000e+00j,  4.6777e-02+0.0000e+00j,
              2.4101e-01+4.4141e-02j,  2.4101e-01-4.4141e-02j, -1.5430e-01+2.5313e-01j, -1.5430e-01-2.5313e-01j,
              2.2442e-02+6.9485e-03j,  2.2442e-02-6.9485e-03j],
            [ 0.0000e+00+0.0000e+00j,  1.8158e-01-4.9221e-02j,  1.8158e-01+4.9221e-02j,  2.4866e-01+0.0000e+00j,
              1.7203e-02+0.0000e+00j,  3.9975e-01+0.0000e+00j,  8.7746e-02+0.0000e+00j,  5.8934e-01+0.0000e+00j,
              4.2298e-01-3.5674e-02j,  4.2298e-01+3.5674e-02j, -1.6961e-02-1.1032e-01j, -1.6961e-02+1.1032e-01j,
              3.4126e-02+1.1343e-01j,  3.4126e-02-1.1343e-01j],
            [ 0.0000e+00+0.0000e+00j,  4.2836e-01-1.1778e-01j,  4.2836e-01+1.1778e-01j, -1.1086e-01+0.0000e+00j,
              4.3833e-03+0.0000e+00j, -2.1374e-01+0.0000e+00j,  1.5286e-01+0.0000e+00j, -1.8174e-01+0.0000e+00j,
              9.9550e-02+2.0668e-01j,  9.9550e-02-2.0668e-01j,  2.7644e-01+1.4411e-01j,  2.7644e-01-1.4411e-01j,
              6.1154e-03+1.2143e-01j,  6.1154e-03-1.2143e-01j],
            [ 0.0000e+00+0.0000e+00j, -5.6111e-02-8.2978e-02j, -5.6111e-02+8.2978e-02j,  1.4039e-02+0.0000e+00j,
             -8.0565e-03+0.0000e+00j, -2.5264e-02+0.0000e+00j, -3.4056e-01+0.0000e+00j, -2.0239e-02+0.0000e+00j,
              1.0205e-01+5.2742e-02j,  1.0205e-01-5.2742e-02j, -3.7080e-02-2.3843e-01j, -3.7080e-02+2.3843e-01j,
              2.0292e-01+2.3261e-02j,  2.0292e-01-2.3261e-02j],
            [ 0.0000e+00+0.0000e+00j, -5.0706e-01-7.6328e-17j, -5.0706e-01+7.6328e-17j,  2.3587e-02+0.0000e+00j,
             -2.1322e-02+0.0000e+00j, -1.7302e-02+0.0000e+00j, -1.0638e-01+0.0000e+00j,  1.5399e-01+0.0000e+00j,
              1.5459e-01-6.8028e-02j,  1.5459e-01+6.8028e-02j,  3.4715e-02+2.4953e-01j,  3.4715e-02-2.4953e-01j,
             -7.4991e-02+5.5816e-02j, -7.4991e-02-5.5816e-02j],
            [ 0.0000e+00+0.0000e+00j, -1.8158e-01+4.9221e-02j, -1.8158e-01-4.9221e-02j, -2.4866e-01+0.0000e+00j,
             -1.7203e-02+0.0000e+00j, -7.5957e-01+0.0000e+00j,  3.0804e-02+0.0000e+00j,  3.3859e-01+0.0000e+00j,
              3.0128e-01+1.5700e-02j,  3.0128e-01-1.5700e-02j, -5.3246e-02-9.5386e-02j, -5.3246e-02+9.5386e-02j,
              4.8362e-02+7.5766e-02j,  4.8362e-02-7.5766e-02j],
            [ 0.0000e+00+0.0000e+00j, -4.2836e-01+1.1778e-01j, -4.2836e-01-1.1778e-01j,  1.1086e-01+0.0000e+00j,
             -4.3833e-03+0.0000e+00j,  2.6530e-01+0.0000e+00j,  1.1354e-01+0.0000e+00j, -1.8161e-01+0.0000e+00j,
              2.3807e-01+2.9685e-01j,  2.3807e-01-2.9685e-01j,  1.0296e-01+1.4131e-01j,  1.0296e-01-1.4131e-01j,
              9.7454e-02+8.8680e-02j,  9.7454e-02-8.8680e-02j],
            [ 0.0000e+00+0.0000e+00j, -5.4706e-03+3.6525e-03j, -5.4706e-03-3.6525e-03j,  9.0124e-01+0.0000e+00j,
              4.6648e-02+0.0000e+00j, -3.6333e-01+0.0000e+00j,  5.2699e-02+0.0000e+00j,  4.6452e-02+0.0000e+00j,
              3.1126e-02-4.3564e-03j,  3.1126e-02+4.3564e-03j,  4.9290e-02+4.5567e-02j,  4.9290e-02-4.5567e-02j,
             -6.8719e-03-2.6016e-02j, -6.8719e-03+2.6016e-02j],
            [ 0.0000e+00+0.0000e+00j,  1.0416e-02-2.9633e-03j,  1.0416e-02+2.9633e-03j,  1.2778e-02+0.0000e+00j,
             -8.7368e-01+0.0000e+00j,  1.3928e-02+0.0000e+00j,  7.5572e-02+0.0000e+00j, -6.5498e-02+0.0000e+00j,
             -1.7340e-01-3.0630e-02j, -1.7340e-01+3.0630e-02j, -4.6730e-03+2.2828e-01j, -4.6730e-03-2.2828e-01j,
              1.0328e-03+5.3238e-02j,  1.0328e-03-5.3238e-02j]]))

```python
plt.matshow(np.linalg.inv(fim))
```

# Run GST on the dataset

```python
proto = pygsti.protocols.GateSetTomography(target_model)
results = proto.run(data)
```

    --- Iterative GST: Iter 1 of 9  4 circuits ---:
      MatrixLayout: 1 processors divided into 1 x 1 (= 1) grid along circuit and parameter directions.
         1 atoms, parameter block size limits (None,)
      *** Distributing 1 atoms to 1 atom-processing groups (1 cores) ***
          More atom-processors than hosts: each host gets ~1 atom-processors
          Atom-processors already occupy a single node, dividing atom-processor into 1 param-processors.
      *** Divided 1-host atom-processor (~1 procs) into 1 param-processing groups ***
      --- chi2 GST ---
      Sum of Chi^2 = 0.0112986 (4 data params - 14 (approx) model params = expected mean of -10; p-value = nan)
      Completed in 0.1s
      Iteration 1 took 0.1s

    --- Iterative GST: Iter 2 of 9  7 circuits ---:
      MatrixLayout: 1 processors divided into 1 x 1 (= 1) grid along circuit and parameter directions.
         1 atoms, parameter block size limits (None,)
      *** Distributing 1 atoms to 1 atom-processing groups (1 cores) ***
          More atom-processors than hosts: each host gets ~1 atom-processors
          Atom-processors already occupy a single node, dividing atom-processor into 1 param-processors.
      *** Divided 1-host atom-processor (~1 procs) into 1 param-processing groups ***
      --- chi2 GST ---
      Sum of Chi^2 = 0.241037 (7 data params - 14 (approx) model params = expected mean of -7; p-value = nan)
      Completed in 0.0s
      Iteration 2 took 0.1s

    --- Iterative GST: Iter 3 of 9  11 circuits ---:
      MatrixLayout: 1 processors divided into 1 x 1 (= 1) grid along circuit and parameter directions.
         1 atoms, parameter block size limits (None,)
      *** Distributing 1 atoms to 1 atom-processing groups (1 cores) ***
          More atom-processors than hosts: each host gets ~1 atom-processors
          Atom-processors already occupy a single node, dividing atom-processor into 1 param-processors.
      *** Divided 1-host atom-processor (~1 procs) into 1 param-processing groups ***
      --- chi2 GST ---
      Sum of Chi^2 = 1.31238 (11 data params - 14 (approx) model params = expected mean of -3; p-value = nan)
      Completed in 0.4s
      Iteration 3 took 0.4s

    --- Iterative GST: Iter 4 of 9  15 circuits ---:
      MatrixLayout: 1 processors divided into 1 x 1 (= 1) grid along circuit and parameter directions.
         1 atoms, parameter block size limits (None,)
      *** Distributing 1 atoms to 1 atom-processing groups (1 cores) ***
          More atom-processors than hosts: each host gets ~1 atom-processors
          Atom-processors already occupy a single node, dividing atom-processor into 1 param-processors.
      *** Divided 1-host atom-processor (~1 procs) into 1 param-processing groups ***
      --- chi2 GST ---
      Sum of Chi^2 = 9.93515 (15 data params - 14 (approx) model params = expected mean of 1; p-value = 0.00162152)
      Completed in 0.4s
      Iteration 4 took 0.4s

    --- Iterative GST: Iter 5 of 9  19 circuits ---:
      MatrixLayout: 1 processors divided into 1 x 1 (= 1) grid along circuit and parameter directions.
         1 atoms, parameter block size limits (None,)
      *** Distributing 1 atoms to 1 atom-processing groups (1 cores) ***
          More atom-processors than hosts: each host gets ~1 atom-processors
          Atom-processors already occupy a single node, dividing atom-processor into 1 param-processors.
      *** Divided 1-host atom-processor (~1 procs) into 1 param-processing groups ***
      --- chi2 GST ---
      Sum of Chi^2 = 12.6447 (19 data params - 14 (approx) model params = expected mean of 5; p-value = 0.0269459)
      Completed in 0.2s
      Iteration 5 took 0.2s

    --- Iterative GST: Iter 6 of 9  23 circuits ---:
      MatrixLayout: 1 processors divided into 1 x 1 (= 1) grid along circuit and parameter directions.
         1 atoms, parameter block size limits (None,)
      *** Distributing 1 atoms to 1 atom-processing groups (1 cores) ***
          More atom-processors than hosts: each host gets ~1 atom-processors
          Atom-processors already occupy a single node, dividing atom-processor into 1 param-processors.
      *** Divided 1-host atom-processor (~1 procs) into 1 param-processing groups ***
      --- chi2 GST ---
      Sum of Chi^2 = 14.7439 (23 data params - 14 (approx) model params = expected mean of 9; p-value = 0.0982157)
      Completed in 0.2s
      Iteration 6 took 0.2s

    --- Iterative GST: Iter 7 of 9  27 circuits ---:
      MatrixLayout: 1 processors divided into 1 x 1 (= 1) grid along circuit and parameter directions.
         1 atoms, parameter block size limits (None,)
      *** Distributing 1 atoms to 1 atom-processing groups (1 cores) ***
          More atom-processors than hosts: each host gets ~1 atom-processors
          Atom-processors already occupy a single node, dividing atom-processor into 1 param-processors.
      *** Divided 1-host atom-processor (~1 procs) into 1 param-processing groups ***
      --- chi2 GST ---
      Sum of Chi^2 = 14.9305 (27 data params - 14 (approx) model params = expected mean of 13; p-value = 0.311713)
      Completed in 0.2s
      Iteration 7 took 0.2s

    --- Iterative GST: Iter 8 of 9  31 circuits ---:
      MatrixLayout: 1 processors divided into 1 x 1 (= 1) grid along circuit and parameter directions.
         1 atoms, parameter block size limits (None,)
      *** Distributing 1 atoms to 1 atom-processing groups (1 cores) ***
          More atom-processors than hosts: each host gets ~1 atom-processors
          Atom-processors already occupy a single node, dividing atom-processor into 1 param-processors.
      *** Divided 1-host atom-processor (~1 procs) into 1 param-processing groups ***
      --- chi2 GST ---
      Sum of Chi^2 = 18.7044 (31 data params - 14 (approx) model params = expected mean of 17; p-value = 0.345761)
      Completed in 0.2s
      Iteration 8 took 0.2s

    --- Iterative GST: Iter 9 of 9  35 circuits ---:
      MatrixLayout: 1 processors divided into 1 x 1 (= 1) grid along circuit and parameter directions.
         1 atoms, parameter block size limits (None,)
      *** Distributing 1 atoms to 1 atom-processing groups (1 cores) ***
          More atom-processors than hosts: each host gets ~1 atom-processors
          Atom-processors already occupy a single node, dividing atom-processor into 1 param-processors.
      *** Divided 1-host atom-processor (~1 procs) into 1 param-processing groups ***
      --- chi2 GST ---
      Sum of Chi^2 = 22.3196 (35 data params - 14 (approx) model params = expected mean of 21; p-value = 0.381298)
      Completed in 0.2s
      Iteration 9 took 0.3s

      Last iteration:
      --- dlogl GST ---
      2*Delta(log(L)) = 22.9198 (35 data params - 14 (approx) model params = expected mean of 21; p-value = 0.348267)
      Completed in 0.4s
      Final optimization took 0.4s

    Iterative GST Total Time: 2.5s

```python
# What is the estimated value of the error parameter?

final_model = results.estimates['GateSetTomography'].models['final iteration estimate']
print('Actual: ', datagen_model.to_vector()[-2:])
print('Estimated: ', final_model.to_vector()[-2:])
```

    Actual:  [1.03 1.00]
    Estimated:  [1.0304 1.0000]

```python
pprint(np.sqrt(2)*final_model.to_vector()[0:4])
pprint(np.sqrt(2)*final_model.to_vector()[4:8])
pprint(np.sqrt(2)*final_model.to_vector()[8:12])
```

    array([ 1.0001, -0.0730, -0.0239,  0.9976])
    array([ 1.0064, -0.0219, -0.0209,  1.0079])
    array([ 0.9969,  0.0461,  0.0113, -0.9958])

```python

```