# Asynchronous communication

In general speaking, asynchronous communication is when you send a message without expecting an immediate response. For example, you send an email. I open and respond to the email several hours later. In contrast, synchronous communication is when you send a message and the recipient processes the information immediately. All previous communications are synchronous communications because all agents have to actively involved with the communication operation in the same time. It is common in cluster and HPC since the computers are dedicated to do one thing at one time. Then, why do we care about the asynchronous communication?

The main advantages of asynchronous communication over the synchronous communication is it can avoid the idel times.
In the synchronous communication pattern, all agents has to start and end the communication at the same time. If some agents is delayed, all others have to wait for that one. On the other hand, the asynchronous communication eventible bought more complexity into the code and you may also face some data racing problem.
Bluefog provided the convenient functionality for asynchronous communication. We will use the examples to show the traits of asynchronous communication.

There are two main goals of this notebook. The first is to introduce the asynchronous communication in BlueFog. In order to achieve that, we have to introduce the concept the `window`, whihc is an area of memory, which follows some protocols that allow others to modify it. The modification operation, i.e. data transfer, is called one-sided communication. Second, we will revisit the average consensus and push-sum algorithm and exam them under the asynchronous situation. You see the behavior of the same algorithm under synchronous and asynchronous scenario may differ.

```
import torch
import networkx as nx
import ipyparallel as ipp
import matplotlib.pyplot as plt

%matplotlib inline
```

```
rc = ipp.Client(profile="bluefog")
rc.block = True
rc.ids
```

    [0, 1, 2, 3]

```
%%px
import torch
import bluefog.torch as bf

bf.init()
```

# Asynchronous Operation in BlueFog

Asynchronous operation may have different meaning in different context.
> In BlueFog, *asynchronous operation* means the operation executing in agent $k$ is independent from the operation running in  other agent $k'$.

*Note the difference between the asynchronous operation and nonblocking operation introduced before.*

For example, `neighbor_allreduce()` is NOT asynchronous operation because it will be finished in agent $k$ only when the other agents $k'$ finished the call as well. In other words, all collective communication operation in BlueFog is classified as *synchronous* operation. Similarly, the basic send-recieve communication pair can be the asynchronous operation since the send operation in agent $k$ is dependent in the recieve operation of other agent.

BlueFog built the asynchronous operation and algorithm is based upon the *One-sided communication ops*, which is introduced in MPI-2. The most notable feature of one-sided communication is indicated by the name that allows the communication ops of one process to be decoupled from the behavior of another process. [Here](https://pages.tacc.utexas.edu/~eijkhout/pcse/html/mpi-onesided.html) is a nice introduction for the MPI one-sided communication ops.

This decoupled behavior can bring us several of benefits. For instance, under heterogenous environment, some agent may have better hardware and some get worse one. For example, you may have several GPUs but some were bought with latest model but some were bought years ago. Hence, the computation time can vary among agents. Under the one-sided communications, every one can immediately continue the process without awaiting.

Now, we formally introduce the asynchronous operation in BlueFog. The first concept is called **window**. A window in each processor (agent) is an area of memory, available to one-sided transfers.

Therefore, creating a window is always the first step to use the one-sided communication.
In Bluefog, each window must be associated with a unique name, which will be the identity to operate the window in the future. The following example shows how you can create a window named `x`. The memory size of that window is exactly the same size as the the shape of tensor you provided.

```
%%px
topo = bf.load_topology()
x = torch.FloatTensor([bf.rank()])
bf.win_create(x, name="x")
print(x)
```

    [stdout:0] tensor([0.])
    [stdout:1] tensor([2.])
    [stdout:2] tensor([1.])
    [stdout:3] tensor([3.])

The shape and the name of one window is immutable, i.e., as long as you create it, you cannot modify it.
If you want to change it, you have to manually free it through `bf.win_free` then create a new one through `bf.win_create` again:

```
%%px
x = torch.FloatTensor([bf.rank(), bf.rank()])
# If you uncomment the following code, you will encouter
# the assertion failure. It is because we require each
# window has a unique name.
# assert bf.win_create(x, name="x")

# However, you can free previous window named "x".
# and create a new one
assert bf.win_free(name="x")
assert bf.win_create(x, name="x")
print(x)
```

    [stdout:0] tensor([0., 0.])
    [stdout:1] tensor([2., 2.])
    [stdout:2] tensor([1., 1.])
    [stdout:3] tensor([3., 3.])

A unique property of `win_create` in BlueFog is that it is related with the underlying topology actually. Let's plot the default topology first.

```
topo = rc[0].pull("topo", targets=0)
nx.draw_circular(topo)
```

We assume there are four agents and the topology used is Exponential 2 graph. What happened after `win_create` called is illustrated:

After this call, each process will allocate the number of incoming neighbor’s windows as buffer, which is illustrated in the figure as red square. Each buffer is dedicated to one neighbor. You don’t need to know which one is dedicated to which neighbor because these buffers are invisible to the python frontend. The only way to interact with them is through the win_update.

## Interacte with window object in BlueFog

Two basic operation to interact the data in the window is `win_get` and `win_put` operation. As their name indicate, they allow to put the data in local memory to window of the remote windows or get the data in remote memory into local window.

win_get is one of three main methods to exchange the information between the processes in window. By default, it will get (fetch) the incoming neighbor’s local value into the its own buffer. Note it doesn’t need the sender to do anything.

```
%%px

if bf.rank() == 0:
    bf.win_get(name="x")
    print(x)  # the value of x is not changed.
```

    [stdout:0] tensor([0., 0.])

As you see before, the value of `x` is still 0 afte the `win_get` operations from the neighbors. Why it don't update the value? It is because when you get the value from your neighbors, BlueFog do not update that value into the main window memory. Instead, it just put that value into the buffer of that window for future usage as shown in the following figure.

The main reason to design like this style is like non-blocking function:

> ```python
handle = bf.win_get_nonblocking(name="x")
SomeOperation(x)
bf.win_wait(handle)
> ```

Now, you should not worry about data racing problem -- whether the value `x` is applied some operation first or replaced by win_get value or in the mixing situation.

In order to interact with the value in the buffer, you have to explicitly call the `win_update` function, which is the bridge to connect the value of buffers (corresponding to the neighbor value) with the local value. It has two functionalities. One is to update the buffer to make sure that the neighbor value, which may be changed through win_put or win_get, is synchronized and visible to local memory. Another is it updates the local value to the average of self and neighbor’s value.

```
%%px
bf.win_update(name="x")
print(f"{bf.rank()}: {x}")
```

    [stdout:0] 0: tensor([1.6667, 1.6667])
    [stdout:1] 2: tensor([2., 2.])
    [stdout:2] 1: tensor([1., 1.])
    [stdout:3] 3: tensor([3., 3.])

Now you see that the value of `x` in rank 0 is updated. (In four neighbor and exp2 graph case, it's value is 1.66 because $(0+2+3)/3$. Meanwhile, the value of `x` in other ranks are unchanged.

Similar as the `neighbor_allreduce` function, `win_get()` has `src_weights` argument to control the weight of receiving information and which neighbor to recieve the information.

```
%%px
y = torch.FloatTensor([bf.rank()])
assert bf.win_create(y, name="y")
if bf.rank() == 0:
    bf.win_get(src_weights={bf.size() - 1: 0.5}, name="y")
    bf.win_update(name="y", self_weight=1.0, neighbor_weights={bf.size() - 1: 0.4})
print(f"Rank {bf.rank()}: {y}")
assert bf.win_free(name="y")
```

    [stdout:0] Rank 0: tensor([0.6000])
    [stdout:1] Rank 2: tensor([2.])
    [stdout:2] Rank 1: tensor([1.])
    [stdout:3] Rank 3: tensor([3.])

Let's see the status of window object in above code step-by-step (assuming 4 nodes under exponential-2 graph):
1. All ranks create create a window with initial value: \[rank\]
2. For rank 0:
    The window intialized \[0\] for self and \[0,0\] for neighbors.
    It called `win_get` the value from agent 3 only with weight 0.5. Hence, after this step the window object became:
    \[0\] for self and \[0, 1.5\].
3. For rank 0:
    It updates local value with self value 1.0 and neighbor weights 0.4 for rank 3, this produces $(0 \times 1.0 + 1.5  \times 0.4)=0.6$.
4. Last, all agents printed their local value. You saw the value of `x` is 0.4 as we calculated and freed the window.

There are another operations `win_put` shown in the next figure, which is very similar as `win_get`. So we leave the user to explore this function.

# Asynchronous "Average Consensus" algorithm

Now you can use these window operation to build simple asynchonous Average Consensus.

```
%%px
bf.win_free()  # free all windows in case "x" is used.
# Set up the average consensus problem.
x = torch.randn(1, dtype=torch.double)
x_bar = bf.allreduce(x, average=True)

assert bf.win_create(x, name="x", zero_init=True)
```

```
%%px
def async_consensus_step(x):
    bf.win_put(x, name="x")
    bf.win_update(name="x")  # notice it is inplace update
```

```
x_bar = rc[0].pull("x_bar", block=True)
x_list = []
for i in range(15):
    %px async_consensus_step(x)
    x_list.append(rc[:].pull("x", block=True))
```

```
plt.plot(x_list)
plt.axhline(x_bar, linestyle="--", color="k")
plt.legend([f"agent {i}" for i in range(len(x_list[0]))])
plt.xlabel("relative time")
plt.ylabel("value")
```

You see from the above the figure that all agents reached some consensus value but that value is not the same as the mean of initial value. That is mainly because the asynchronous essense breaks the unbias property of average consensus algorithm. Fortunately, it is easy to correct its bias and the algorithm is the same push-sum algorithm we introduced in previous section. But before we present the push-sum algorithm in asynchronous style, let's exam the key feature of asynchronous communication -- time.

# Asynchronous versus synchronous communication in time

As mentioned before, in the case that some agents work fast and some agents work slow, the synchronous algorithm has to wait until the slowerest one finished meanwhile asynchronous one do not. The speed of agents usually depends on lots of factors including hardward, system scheduling, network congestion, etc. For this tutorial, we use `sleep` function to illustrate the slow worker behavior.

```
%%px
import time

def async_consensus_step_with_sleep(x):
    if bf.rank() == 0:
        time.sleep(0.5)
    bf.win_put(x, name="x")
    bf.win_update(name="x")  # notice it is inplace update

def sync_consensus_step_with_sleep(x):
    if bf.rank() == 0:
        time.sleep(0.5)
    y = bf.neighbor_allreduce(x)
    return y
```

```
%%px

x = torch.randn(1, dtype=torch.double)
t_start = time.time()
for _ in range(20):
    x = sync_consensus_step_with_sleep(x)
duration = time.time() - t_start
print(f"Rank {bf.rank()} took {duration} seconds")
```

    [stdout:0] Rank 3 took 10.731921195983887 seconds
    [stdout:1] Rank 0 took 10.72654104232788 seconds
    [stdout:2] Rank 1 took 10.72762393951416 seconds
    [stdout:3] Rank 2 took 10.725753784179688 seconds

```
%%px

x = torch.randn(1, dtype=torch.double)
x_bar = bf.allreduce(x, average=True)
bf.win_free()
assert bf.win_create(x, name="x", zero_init=True)
t_start = time.time()
for _ in range(20):
    async_consensus_step_with_sleep(x)
duration = time.time() - t_start
print(f"Rank {bf.rank()} took {duration} seconds")
```

    [stdout:0] Rank 3 took 0.42594003677368164 seconds
    [stdout:1] Rank 0 took 10.355370044708252 seconds
    [stdout:2] Rank 1 took 0.5432538986206055 seconds
    [stdout:3] Rank 2 took 0.451002836227417 seconds

# Asynchronous Push-Sum Consensus Algorithm

Recall the Push-Sum consensus algorithm in synchronized behavior is

For each agent $i$, run it in parallel:
$$
    x_{i, k+1} = \sum_{j\in \mathcal{N}_i} w_{ij}^{(k)} x_{j,k}\\
    p_{i, k+1} = \sum_{j\in \mathcal{N}_i} w_{ij}^{(k)} p_{j,k}\\
    y_{i, k+1} = x_{i, k+1} / p_{i, k+1}
$$

In asynchronous style, the mathametical equation becomes tricky to represent it. we are no longer able to use the global iteration/time $k$. Instead, we will use the logical event counter $e$, which acts similar as iteration $k$ but with different definition.

[TODO] Add a figure of global counter

There are two ways to descibe it -- with *push* view or *pull* view, one is based on the events that the data is pushed to the neighbors and another is based on the events that the data is recieved from neighbors. It is easy to describe the `push` view in words but not easy to write in words. So we will first use words to describte the push mode in words then give the equation description in equations:

- Push view:

    When at agent $i$ is ready to push a information: Split the data $x_i$ into $|\mathcal{N}_i|$ pieces. Each piece only holds the partial of $x_i$ according to the weights, i.e. $a_{ij} x_i$ and do the same thing for $p_i$ as well. Then push this information to the corresponding neighbors for adding.

- Pull view:

Let's use $e$ to denote the global event counter and assume that $j$ is the agent rank that triggers the event $e$.
\begin{align}
    x_{i, e+1} =&\; x_{i, e} + w_{ij}^{(k)} x_{j,k}\;\; {\rm if}\; i\neq j\;\; {\rm else}\; w_{jj} x_{j, e}\\
    p_{i, e+1} =&\; x_{i, e} + w_{ij}^{(k)} p_{j,k}\;\; {\rm if}\; i\neq j\;\; {\rm else}\; w_{jj} p_{j, e}\\
    y_{i, e+1} =&\; x_{i, e+1} / p_{i, e_i+1}
\end{align}

If you do not get the intuition of equations, it will be much clear if we re-write above equations over all agents by introducing the stacking matrix:

$$
    X_{e} = \left[ \begin{array}{c}
    -x^T_{1,e}- \\
    -x^T_{2,e}- \\
   \cdots \\
   -x^T_{N, e}-
  \end{array} \right]
$$

Under this notation, the first equation of **synchronous** mode is equivalent to
$$
    X_{k+1} = W^{(k)} X_k
$$
where matrix $W^{(k)} = [w_{ij}]$ (it is doubly stochastic matrix).

The first equation of **asynchronous** mode is
$$
    X_{k+1} = W^{(e)} X_k
$$
where matrix $W^{(e)}$ is (again assuming $i'$ is the agent that trigger the event $e$.)
$$
    W^{(e)} = \left[ \begin{array}{ccccc}
    1 &      & w_{1i'} &   &\\
      &\ddots& \vdots &   &\\
      &      & w_{i'i'}   &   &\\
      &      & \vdots & \ddots&\\
      &      & w_{Ni'} &   & 1
  \end{array} \right]
$$
Note now that $W^{(e)}$ is no longer a doubly stochastic matrix but a column stochastic matrix, i.e. the summation of each column is 1 but not the sum of rows. One key property that you should verify, we call it mass conservation property, is
$$
    \sum_{i} x_{i, e} =  \sum_{i} x_{i, e'},\;\; \forall e, e'
$$

It should be clear now that we need a new function called `win_accumulate`, which is similar to `win_put` that sends data to remote window object. But it accumulate (add) the value onto the remove window object instead of overwriting the data.

Now let's use a simple example to illustrate the behavior of `win_accunulate`.

```
%%px
y = torch.DoubleTensor([bf.rank() + 1])
# zero_init will create the buffer with 0.
bf.win_create(y, name="y", zero_init=True)
if bf.rank() == 0:
    bf.win_accumulate(y, name="y")
if bf.rank() == 1:
    # neighbor_weights: Dict[int: float] -- rank to weight.
    bf.win_update(name="y", self_weight=1.0, neighbor_weights={0: 1.0})
    print(y)
bf.barrier()
if bf.rank() == 1:
    bf.win_update(name="y", self_weight=1.0, neighbor_weights={0: 1.0})
    print(y)
```

    [stdout:2]
    tensor([2.], dtype=torch.float64)
    tensor([3.], dtype=torch.float64)

First, the argument `self_weight` and `neighbor_weights` in `win_update` is the same as previous `neighbot_allreduce` argument. It controls the weights to do the update. In this case, we meant update the main memory of window `y` equals to its previous value times $1.0$ plus the buffer memory value for agent `0` with weights 1.0.

Hence, the first print you saw is $2.0$ because self value in main memory is initialized as $2$ and buffer value for neighbor is $0$ due to zero initialization. You may curious that isn't `bf.accumulate` of agent $0$ will accumulate value $1$ into the buffer? Actually it is not likely because note in agent `0` and `1` these two function is executed almost simultaneously. Hence, the buffer hasn't been updated yet. However, after we added `barrier` function, it is guaranteed that `win_update` in agent $0$ is finished. Hence, the second print funciton, you will see the value is 3 because the buffer value is updated into 1 at that time.

Now, let see how to build asynchronous push-sum algorithm:

```
%%px
# Set up the average consensus problem.
x = torch.randn(10, dtype=torch.double)
x_bar = bf.allreduce(x, average=True)
mse = [torch.norm(x - x_bar, p=2) / torch.norm(x_bar, p=2)]
p = torch.DoubleTensor([1.0])
x_ext = torch.cat([x, p], 0)
# Instead of initalize the buffer with same value as x_ext,
# we initialize it as zero for accumulation.
bf.win_create(x_ext, name="x_ext", zero_init=True)
outdegree = len(bf.out_neighbor_ranks())
indegree = len(bf.in_neighbor_ranks())

dst_weights = {rank: 1.0 / (outdegree + 1) for rank in bf.out_neighbor_ranks()}
self_weight = 1 / (1 + outdegree)
```

But it is not enough with accumulate only.
We designate the local buffer of one agent for all incoming neighbors. This means the `win_accumulate` from different agents sends to common destination agent is independent. This property is very crucial to algorithm implementation because you no longer need to consider the data racing problem between the two agents sends to common destinations.
On the other hand, it is possible that the `win_update` and the corresponding `win_accumulate` from other agents operate on the same window object *simulatenously*.
Since agent $k$ cannot know when the incoming neighbor agent $k'$ will execute the `win_accumulate`, it is impossible to write the code completely avoid that. Consider the following example, the `win_update` at Node 1 may read the old value or the updated value from Node 2, but you cannot control.

This data racing can be problematic for the push-sum algorithm. It can easily break the key mass conservation property  of push-sum algorithm, i.e., the sum of $p$ and $x$ over all agent is always constant.

Fortunately, it is very easy to avoid this situation through the **distributed mutex**. It is same usage as normal mutext that when agent $k$ want to update its local window object it has to acquire the mutex first. If it cannot acquire the mutex. it has to wait until the mutex is available. The difference from the normal mutex is that this mutex can also be acquired by neighbor agents. Hence, when an agent $k'$ wants to accumulate some value to the remote window object at agent $k$, it has to acquire the mutex of agent $k'$ first as well.
To use this distributed mutex, you just need to set the `require_mutex` argument in `win_accumulate` to be True. (`win_put` and `win_get` have the same argument as well.)

Besides the `win_accumulate` which is the writing side of function, we also need to change the behavior of `win_update` for asynchronous push-sum as well, which is the reading side. One obvious thing is that we need to set `require_mutex` as true. A less obvious thing is that we need clean the buffer in the window. Note by default `win_update` only compute the average value between self and neighbors and leave the value in the buffer untouched. In this case, `win_accumulate` will continue adding the values. Instead, what we want is move the value from buffer to local value, i.e. let self value to be the sum of self and neighbor, and  let the neighbor value to be zero. It is doable though `win_update` with the proper `self_weight`, `neighbor_weigth` and `reset` argument. But it is boilerplate, so we introduce a new function `win_update_then_collect`.

Now we are ready to present the asynchronous push-sum algorithm:

```
%%px
def push_sum_alg(x, self_weight, dst_weights, name):
    global mse
    bf.win_accumulate(
        x,
        self_weight=self_weight,
        dst_weights=dst_weights,
        name=name,
        require_mutex=True,
    )
    bf.win_update_then_collect(name="x_ext")
    x, associated_p = x_ext[:-1], x_ext[-1]
    mse.append(torch.norm(x / associated_p - x_bar, p=2) / torch.norm(x_bar, p=2))
    return x_ext
```

```
for i in range(100):
    if i % 5 == 0:
        print(f"iteration {i}", end="\r")
    %px x_ext = push_sum_alg(x_ext, self_weight, dst_weights, name="x_ext")
```

    iteration 95

```
%%px
# Do not forget to sync at last!
bf.barrier()
bf.win_update_then_collect(name="x_ext")
x, associated_p = x_ext[:-1], x_ext[-1]
print(f"associated p at {bf.rank()} is {associated_p}")
mse.append(torch.norm(x / associated_p - x_bar, p=2) / torch.norm(x_bar, p=2))
assert bf.win_free(name="x_ext")
```

    [stdout:0] associated p at 0 is 1.00427521234524
    [stdout:1] associated p at 2 is 0.991455734193527
    [stdout:2] associated p at 1 is 1.0000002021018388
    [stdout:3] associated p at 3 is 1.004276265183896

```
mse_list = rc[:].pull("mse")
for i, mse in enumerate(mse_list):
    plt.semilogy(mse, label="agent " + str(i))
plt.legend()
plt.xlabel("Local update counts")
```

# Asynchronous push-sum algorithm over the dynamic topology

Last, let's present the push-sum combining the asynchronous operation over dynamic topology.
The nice convergence property will gurantee that under very mild condition, push-sum algorithm will converge to the mean of all agents no matter what the order of asynchronous operations between agents is and how different the dynamic topologies change with time.

```
%%px
bf.win_free()

# Set up the average consensus problem.
x = torch.randn(10, dtype=torch.double)
x_bar = bf.allreduce(x, average=True)
mse = [torch.norm(x - x_bar, p=2) / torch.norm(x_bar, p=2)]
p = torch.DoubleTensor([1.0])
x_ext = torch.cat([x, p], 0)
# Instead of initalize the buffer with same value as x_ext,
# we initialize it as zero for accumulation.
bf.win_create(x_ext, name="x_ext", zero_init=True)
outdegree = len(bf.out_neighbor_ranks())
indegree = len(bf.in_neighbor_ranks())
```

```
%%px
def dynamic_push_sum_alg(i, x, self_weight, dst_weights, name):
    global mse
    num_out_neighbors = len(bf.out_neighbor_ranks())
    sent_neighbor = bf.out_neighbor_ranks()[i % num_out_neighbors]
    dst_weights = {sent_neighbor: 0.5}
    self_weight = 0.5
    bf.win_accumulate(
        x,
        self_weight=self_weight,
        dst_weights=dst_weights,
        name=name,
        require_mutex=True,
    )
    bf.win_update_then_collect(name="x_ext")
    x, associated_p = x_ext[:-1], x_ext[-1]
    mse.append(torch.norm(x / associated_p - x_bar, p=2) / torch.norm(x_bar, p=2))
    return x_ext
```

```
for i in range(100):
    if i % 5 == 0:
        print(f"iteration {i}", end="\r")
    rc[:].push({"i": i})
    %px x_ext = dynamic_push_sum_alg(i, x_ext, self_weight, dst_weights, name="x_ext")
```

    iteration 95

```
%%px
# Do not forget to sync at last!
bf.barrier()
bf.win_update_then_collect(name="x_ext")
x, associated_p = x_ext[:-1], x_ext[-1]
print(f"associated p at {bf.rank()} is {associated_p}")
mse.append(torch.norm(x / associated_p - x_bar, p=2) / torch.norm(x_bar, p=2))
assert bf.win_free(name="x_ext")
```

    [stdout:0] associated p at 0 is 0.832190677523613
    [stdout:1] associated p at 2 is 1.5
    [stdout:2] associated p at 1 is 0.667809322476387
    [stdout:3] associated p at 3 is 1.0

```
mse_list = rc[:].pull("mse")
for i, mse in enumerate(mse_list):
    plt.semilogy(mse, label="agent " + str(i))
plt.legend()
plt.xlabel("Local update counts")
```

# Other features

Similar as the `neighbor_allreduce`, we also provided the nonblocking version of all window functions, such as `win_put_nonblocking`, etc. We didn't illustrated in this notebook. The reader can try to use the nonblocking version of window operation to build an asynchronous Adapt-With-Combination algorithm, which combines nonblocking, asynchrnous, gradient, consensus communication features.


```python
from math import *
import numpy as  np
import scipy as sp
from pylab import *
import matplotlib.pyplot as plt
```

Auteurs :  AL AMMIRI Anass, WANG Ruiqi

# Exercice 1 (Tirage d'importance auto-normalisé)

En cours vous avez vu la notion de tirage d'importance afin de tenter d'améliorer les performances de l'estimateur de Monte Carlo naïf pour l'estimation d'une espérance. Nous allons revenir sur cette problématique en généralisant son cadre d'application : la densité "naturelle" $f$ est connue à une constante multiplicative près (et même parfois la densité d'importance $p$ également). Nous sommes donc dans la situtation où

$$f(x) = c_f f_0(x), \qquad p(x) = c_p p_0(x), \qquad x \in \mathbb{R},$$

où les constantes de normalisation $c_f$ et $c_p$ sont évidemment données par
$$c_f^{-1} = \int_\mathbb{R} f_0(x) \mbox{d$x$}, \qquad c_p^{-1} = \int_\mathbb{R} p_0(x) \mbox{d$x$}.$$

Dans la suite, notre objectif sera d'estimer la quantité (supposée finie)
$$I = \int_\mathbb{R} h(x) f(x) \mbox{d$x$},$$
pour une fonction donnée $h$.

1. Montrez que l'on peut écrire

$$I = \frac{\mathbb{E}[h(X) w(X)]}{\mathbb{E}[w(X)]}, \qquad X \sim p,$$
où $w: x \mapsto w(x)$ est une fonction connue que vous déterminerez et qui fera intervenir les fonctions $p_0$ et $f_0$.

$$I = \int_\mathbb{R} h(x) f(x) \mbox{d$x$} = \frac{\int_\mathbb{R} h(x) \frac{f_0(x)}{p_0(x)} p(x) \mbox{d$x$}}{\int_\mathbb{R} \frac{f_0(x)}{p_0(x)} p(x) \mbox{d$x$}} = \frac{\int_\mathbb{R} h(x) w(x) p(x) \mbox{d$x$}}{\int_\mathbb{R} w(x) p(x) \mbox{d$x$}} = \frac{\mathbb{E}[h(X) w(X)]}{\mathbb{E}[w(X)]},\qquad w(x) = \frac{f_0(x)}{p_0(x)}$$

2. Donnez deux estimateurs de Monte Carlo "naïfs", notés $\hat{N}_n $ et $\hat{D}_n$, pour le numérateur et le dénominateur de l'expression précédente, utilisant $n$ variables aléatoires indépendantes $X_1,\dots,X_n$ de même loi que $X$. En déduire un estimateur $\hat{I}_n$ pour $I$, où $\hat{N}_n$ et $\hat D_n$ sont fonction des mêmes $n$ variables aléatoires.

$$\hat{N}_n = \frac{1}{n} {\sum_{i=1}^n h(X_i) w(X_i)}, \qquad \hat{D}_n = \frac{1}{n} {\sum_{i=1}^n w(X_i)}, \qquad \hat{I}_n = \frac{\frac{1}{n} {\sum_{i=1}^n h(X_i) w(X_i)}}{\frac{1}{n} {\sum_{i=1}^n w(X_i)}}$$

3. En quoi ce nouvel estimateur se distingue de l'estimateur par tirage d'importance que vous avez vu en cours ? En quoi répond-il au cadre de travail posé en introduction ?

L'estimateur par tirage d'importance est sous la forme :
$ \hat{I}_n = \frac{1}{n} \sum_{i=1}^n h(Z_i) \frac{f_0(Z_{i})}{p_0(Z_{i})}$. Avec Z de densité $p_0$. \\
Sauf que dans notre cas $f_0$ est connue à une constante multiplicative près. D'où l'intérêt du terme ${\mathbb{E}[w(X)]}$.

4. Montrez que, sous des hypothèses que vous préciserez, $\hat I_n$ est un estimateur fortement convergent, i.e., $\hat I_n \to I$ p.s. lorsque $n \to \infty$.

Pour $n$ variables aléatoires indépendantes $X_1,\dots,X_n$ de même loi que $X$, et $h$ et $w$ deux fonctions réelles, on a selon la loi forte des grands nombres : $\hat{N}_n$ converge ps vers $\mathbb{E}[w(X)]$, de même $\hat{D}_n$ vers $\mathbb{E}[w(X)]$ (qui est strictement postive, sinon $f_0$ est nulle). D'où la convergence presque sûre de $\hat{I}_n$.

$$\hat{N}_n = \frac{1}{n} {\sum_{i=1}^n h(X_i) w(X_i)} \to \mathbb{E}[h(X) w(X)], \qquad \hat{D}_n = \frac{1}{n} {\sum_{i=1}^n w(X_i)}\to \mathbb{E}[w(X)]$$

$$\hat{I}_n = \frac{\frac{1}{n} {\sum_{i=1}^n h(X_i) w(X_i)}}{\frac{1}{n} {\sum_{i=1}^n w(X_i)}}\to I $$

5. Selon vous $\hat I_n$ est-il sans bians pour $I$ ?

Oui
🟥

6. Calculez $\mbox{Var}(\hat N_n)$, $\mbox{Var}(\hat D_n)$ et $\mbox{Cov}(\hat N_n, \hat D_n)$. En déduire que l'expression suivante
$$\sum_{i=1}^n \omega_i^2 \{h(X_i) - \hat I_n\}^2, \qquad X_1, \ldots, X_n \stackrel{iid}{\sim} p,$$
est un estimateur de la variance (approchée) de $\hat I_n$ avec
$$\omega_i = \frac{w(X_i)}{\sum_{j=1}^n w(X_j)}.$$

Astuce : La méthode delta est votre meilleur ami pour obtenir la dernière expression !

* Pour $\mbox{Var}(\hat N_n) $ \begin{align}
\mbox{Var}(\hat N_n)    &  = & Var(\frac{1}{n} {\sum_{i=1}^n h(X_i) w(X_i)}) \\
  &  = & \frac{1}{n^2} Var( {\sum_{i=1}^n h(X_i) w(X_i)}) \\
   &  = & \frac{1}{n^2} \sum_{i=1}^n Var( { h(X_i) w(X_i)}) \text{    (par indépendance)}\\
   &  = & \frac{1}{n^2} \sum_{i=1}^n Var( { h(X_i) w(X_i)}) \\
   &  = & \frac{1}{n} Var( { h(X) w(X)}) \\
   &  = & \frac{1}{n} Var( { h(X) \frac{f_0(X)}{p_0(X)}}) \\
\end{align}

* Pour $\mbox{Var}(\hat D_n) $ \begin{align*}
\mbox{Var}(\hat D_n)    &  = & Var(\frac{1}{n} {\sum_{i=1}^n w(X_i)}) \\
  &  = & \frac{1}{n^2} Var( {\sum_{i=1}^n w(X_i)}) \\
   &  = & \frac{1}{n^2} \sum_{i=1}^n Var( { w(X_i)}) \text{    (par indépendance)}\\
   &  = & \frac{1}{n^2} \sum_{i=1}^n Var( {w(X_i)}) \\
   &  = & \frac{1}{n} Var({w(X)}) \\
   &  = & \frac{1}{n} Var( { \frac{f_0(X)}{p_0(X)}}) \\
\end{align*}

* Pour $\mbox{Cov}(\hat N_n, \hat D_n)$ \begin{align*}
\mbox{Cov}(\hat N_n, \hat D_n)   &  = & Cov(\frac{1}{n} {\sum_{i=1}^n h(X_i) w(X_i)}, \frac{1}{n} {\sum_{i=1}^n w(X_i)}) \\
  &  = & \frac{1}{n^2}Cov( {\sum_{i=1}^n h(X_i) w(X_i)}, {\sum_{i=1}^n w(X_i)}) \\
   &  = & \frac{1}{n^2} \sum_{i=1}^n \sum_{j=1}^n Cov( h(X_i) w(X_i), w(X_j)) \\
   &  = & \frac{1}{n^2} \sum_{i=1}^n \sum_{j=1}^n Cov( h(X_i) w(X_i), w(X_i))   \\ \text{    ( pour $ i \neq j$ la covariance est nulle par indépendance)}\\
   &  = & \frac{1}{n} Cov( h(X) w(X), w(X))   \\
\end{align*}

Pour trouver l'expression de l'estimateur on utilise la délta méthode. \\
On note :

*   $\Psi = ( w(X),h(X)w(X))$
*   $\hat{\Psi}_n = \mbox{Var} (\hat D_n, \hat N_n )$
*   $\mathbb{E}_n = \mathbb{E} (h(X)w(X))$
*   $\mathbb{E}_d = \mathbb{E} (w(X))$
*   $V_n = \mbox{Var} (h(X)w(X))$
*   $V_d = \mbox{Var} (w(X))$
*   $\theta = (\mathbb{E}_d,\mathbb{E}_n)$
*   $Cov= Cov( h(X) w(X), w(X)) $

La suite des variables aléatoires $\hat{\Psi}_1, \ldots, \hat{\Psi}_n$. Par le théorème de central limite: $\sqrt{n}(\hat{\Psi}_n - \mathbb{E}(\Psi)) \xrightarrow{L} \mathcal{N}(0,\Sigma)$. Avec $\Sigma $ la matrice de covariance de $Ψ$. On pose la fonction $g$, telle que $ g(x,y)= \frac{x}{y}$. Par la delta méthode :
${\sqrt{n}[g(\hat{\Psi}_n )-g(\Psi)]\,\xrightarrow{L}\,\mathcal{N}\left(0,\nabla g(\theta)\Sigma \nabla g(\theta)^T\right)}$ avec $\nabla g(\theta)$ la matrice jacobienne de $g$ en $\theta$.

Donc : ${\sqrt{n}[\hat{I}_n -I]\,\xrightarrow{L}\,\mathcal{N}\left(0,\nabla g(\theta)\Sigma \nabla g(\theta)^T\right)}$
Alors :  $\hat{I}_n \,\xrightarrow{L}\,\mathcal{N}\left(I,\frac{1}{n}\nabla g(\theta)\Sigma \nabla g(\theta)^T\right)$

Donc l'expression de la variance (approchée ) de $\hat{I}_n$ est : $\frac{1}{n}\nabla g(\theta)\Sigma \nabla g(\theta)^T = \frac{1}{n \mathbb{E}_d^2} \left( V_n - 2 \frac{\mathbb{E}_n}{\mathbb{E}_d}Cov+  \frac{\mathbb{E}_n^2}{\mathbb{E}_d^2}V_d \right) =  \frac{1}{n \mathbb{E}(w(X))^2} \left( V(h(X)w(X)) - 2 \frac{\mathbb{E}(h(X)w(X))}{\mathbb{E}(w(X))}Cov(h(X)w(X),w(X))+  \left( \frac{\mathbb{E}(h(X)w(X))}{\mathbb{E}(w(X))} \right)^2 V(w(X)) \right) = \frac{1}{n \mathbb{E}(w(X))^2} \left( V(h(X)w(X)) - 2 \frac{\mathbb{E}(h(X)w(X))}{\mathbb{E}(w(X))}Cov(h(X)w(X),w(X))+  \left( \frac{\mathbb{E}(h(X)w(X))}{\mathbb{E}(w(X))} \right)^2 V(w(X)) \right) = \frac{1}{n \mathbb{E}(w(X))^2} \left( V(h(X)w(X)) - 2 I.Cov(h(X)w(X),w(X))+  I^2 V(w(X)) \right) = \frac{1}{n \mathbb{E}(w(X))^2} \left( V(h(X)w(X)) - 2 Cov(h(X)w(X),w(X)I)+ V(w(X) I) \right) = \frac{1}{n \mathbb{E}(w(X))^2} \left( V(h(X)w(X)) - 2 Cov(h(X)w(X),w(X)I)+ V(w(X) I) \right)=  \frac{ Var(h(X)w(X)- w(X)I) }{n \mathbb{E}(w(X))^2} = \frac{ Var(h(X)w(X)- w(X)I) + \stackrel{\mathbb{E}(h(X)w(X)- w(X)I)=0}{\mathbb{E}(h(X)w(X)- w(X)I)^2} }{n \mathbb{E}(w(X))^2} =  \frac{ \mathbb{E}\left( \left( h(X)w(X)- w(X)I \right)^2 \right) }{n \mathbb{E}(w(X))^2} \stackrel{empiriquement}{=}  \frac{1}{n} \frac{\frac{1}{n}\sum_{i=1}^n  \{h(X_i).w(X_i) - \hat I_n.w(X_i)\}^2}{\left( \frac{1}{n} \sum_{i=1}^n w(X_i) \right) ^2} = \frac{\sum_{i=1}^n w(X_i) ^2 \{h(X_i) - \hat I_n\}^2}{\left(\sum_{i=1}^n w(X_i) \right) ^2}= \sum_{i=1}^n \omega_i^2 \{h(X_i) - \hat I_n\}^2$

Pour $\omega_i = \frac{w(X_i)}{\sum_{j=1}^n w(X_j)}$ . Cqfd

7. En déduire un intervalle de confiance (symétrique) pour $I$ de niveau asymptotique $1 - \alpha$.

Par le théorèm centrale limite (déjà appliquée) et par le théorème de Slutsky on a comme IC asymptotique $Ξ_n= \left[\hat I_n  - z_{1-α/2} \sqrt{\frac{\sum_{i=1}^n \omega_i^2 \{h(X_i) - \hat I_n\}^2}{n}} , \hat I_n  + z_{1-α/2} \sqrt{\frac{\sum_{i=1}^n \omega_i^2 \{h(X_i) - \hat I_n\}^2}{n}} \right]$

Avec $\omega_i = \frac{w(X_i)}{\sum_{j=1}^n w(X_j)}$, et $z_{1-α/2}$ le quantile $1-α/2$ de la loi normale.

8. Application numérique. On considère le cas où $h(x) = 1_{\{x \in A\}}$, $A \subset \mathbb{R}$, de sorte que $I$ est la probabilité d'appartenir à $A$ sous la densité $f$. Pour cette application numérique nous allons considérer le cas où $A = (4, \infty)$ et $f$ est la densité d'une $N(0,1)$.

  a) Définissez un estimateur de Monte Carlo naïf pour $I$.

  b) Définissez un estimateur par tirage d'importance pour $I$ avec une loi d'importance que vous choisirez.

  c) Définissez un estimateur par tirage d'importance auto-normalisé pour $I$ basé sur celui de b).

  d) Commentez les résultats obtenus

```python
from scipy.stats import norm

def f(x):
  return (1/sqrt(2*np.pi)*np.exp(-x**2/2))

def h(x):
  return(x>4).astype(int)

def q(x):
  return np.exp(-(x-4))

def w(x):
  return np.divide(f(x),q(x))

def hw(x):
  return np.multiply(h(x),w(x))
```

  a) Définissez un estimateur de Monte Carlo naïf pour $I$.

 $\hat I_n = \frac{1}{n} \sum_{i=1}^n  𝟙_{\{X_n \geq 4 \}} f(X_n)$

```python
n=50
def MC1(n):
  X=np.random.normal(size=n)
  I=mean(np.multiply((X>4).astype(int)  ,X))
  return I
MC1(n)
```

    0.0

```python

n_values=list(range(0,1000,100))
I_values = [MC1(n) for n in n_values]
print(I_values)
#plt.plot(n_values,I_values)
#plt.title("L'estimation de I pour différentes valeurs de n ")
#plt.show()
```

    [nan, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]

    /usr/local/lib/python3.7/dist-packages/numpy/core/fromnumeric.py:3373: RuntimeWarning: Mean of empty slice.
      out=out, **kwargs)
    /usr/local/lib/python3.7/dist-packages/numpy/core/_methods.py:170: RuntimeWarning: invalid value encountered in double_scalars
      ret = ret.dtype.type(ret / rcount)

Nous pouvons voir qu'il y a un problème ici, car presque tous les échantillons sont plus petits que 4. Et pour la variance :

```python
n=10000
X=np.random.normal(size=n)
np.var(np.multiply((X>4).astype(int)  ,X))
```

    0.0021943521469248617

 b) Définissez un estimateur par tirage d'importance pour $I$ avec une loi d'importance que vous choisirez.

On prend la loi d'importance $ \begin{equation}
    q(x)=
    \begin{cases}
      exp(-(x-4)) = exp(-x).exp(4) , & \text{if}\ x \geq 4 \\
      0  & \text{otherwise}
    \end{cases}
  \end{equation} $ , pour ce choix on a utilisé cette [référence](https://www.math.arizona.edu/~tgk/mc/book_chap6.pdf) où la variance est aussi calculé.

 $\hat I_n = \frac{1}{n} \sum_{i=1}^n   𝟙_{\{Z_n \geq 4 \}} \frac{ f(Z_n)}{q(Z_n)}$

```python
def MC2(n):
  Z= np.random.exponential(scale=1.0, size=n)
  I=mean(np.multiply((Z>4).astype(int)  ,np.divide(f(Z),q(Z))))
  return I

n=10000
MC2(n)
```

    5.483400921877393e-07

La variance:

```python
n=10000
Z=np.random.exponential(scale=1.0, size=n)
np.var(np.multiply((Z>4).astype(int)  ,np.divide(f(Z),q(Z))))
```

    5.070188557023406e-11

La variance est plus petite par rapport à la première méthode

c) Définissez un estimateur par tirage d'importance auto-normalisé pour  I  basé sur celui de b)

On utilise les résultats des questions précédentes.

```python
def MC3(n):
  X=np.random.normal(scale=1.0, size=n)
  return np.mean(hw(X))/np.mean(w(X))
n=10000
MC3(n)
```

    0.0

Pour la variance :

```python
n=10000
Z=np.random.normal(scale=1.0, size=n)
f_z=f(Z)
q_z=q(Z)
np.var([np.mean(f_z[0:n])/np.mean(q_z[0:n]) for n in range(2,len(f_z)) ])
```

    7.45230411377179e-09

d) Commentez les résultats obtenus

```python
n=1000000
I_exact=1-norm.cdf(4)
print(f"Valeur exacte: I={Iexact}")
I=MC1(n)
print(f"Estimation naïve: I={I}")
erreur = np.linalg.norm(I_exact-I)
print(f"Erreur: {I}")
I=MC2(n)
print(f"Estimation par tirage d'importance: I={I}")
erreur = np.linalg.norm(I_exact-I)
print(f"Erreur: {I}")
I=MC3(n)
print(f"Estimation par tirage d'importance auto-normalisé: I={I}")
erreur = np.linalg.norm(I_exact-I)
print(f"Erreur: {I}")
```

    Valeur exacte: I=3.167124183311998e-05
    Estimation naïve: I=0.00014548439116039618
    Erreur: 0.00014548439116039618
    Estimation par tirage d'importance: I=5.88678965280222e-07
    Erreur: 5.88678965280222e-07
    Estimation par tirage d'importance auto-normalisé: I=2.1298953023364217e-07
    Erreur: 2.1298953023364217e-07

On remarque que l'erreur la plus petite est celle du tirage auto-normalisé (ceci pour plusieurs essais)

Pour la variance (empirique):

```python
n=1000000

n_values=list(range(0,1000000,100000))
I_values_1 =[]
I_values_2 =[]
I_values_3 = []
for n in n_values :
  X=np.random.normal(size=n)
  I_values_1.append(np.var(np.multiply((X>4).astype(int)  ,X)))
  Z=np.random.exponential(scale=1.0, size=n)
  I_values_2.append(np.var(np.multiply((Z>4).astype(int)  ,np.divide(f(Z),q(Z)))))
  Z=np.random.normal(scale=1.0, size=n)
  f_z=f(X)
  q_z=q(X)
  I_values_3.append(np.var([np.mean(f_z[0:n])/np.mean(q_z[0:n]) for n in range(2,len(f_z)) ]))

plt.plot(n_values,I_values_1, label = "Monte Carlo naif")
plt.plot(n_values,I_values_2, label = "Echantillonnage d’importance")
plt.plot(n_values,I_values_3, label = "Tirage d'importance auto-normalisé ")
plt.title("Variance empirique pour différentes valeurs de n pour les trois méthodes")
plt.show()

```

```python
plt.plot(n_values,I_values_1, label = "Monte Carlo naif")
plt.plot(n_values,I_values_2, label = "Echantillonnage d’importance")
plt.plot(n_values,I_values_3, label = "Tirage d'importance auto-normalisé ")
plt.title("Variance empirique pour différentes valeurs de n pour les trois méthodes")
plt.legend()
```

```python

plt.plot(n_values,I_values_2, label = "Echantillonnage d’importance")
plt.plot(n_values,I_values_3, label = "Tirage d'importance auto-normalisé ")
plt.title("Variance empirique pour différentes valeurs de n pour les trois méthodes")
plt.legend()
```

La troisième méthode est meilleure en termes de précision.
Dans ce cas, la variance empirique est plus petite pour la secondes méthode, alors qu'elle décroît avec n pour la troisième méthode (qui est celle qui est prouvée). Il existe d'autres méthodes pour diminuer la variance asymptotiquement à chaque fois (mise à jour des poids, estimations cumulées..).
Bien que l'exemple que nous avons soit particulier, si nous n'ayons pas choisi une bonne loi d'importance ou si nous ne connaissions pas la constante multiplicative on aurait pas le même résultat. Il est préférable d'utiliser la troisième méthode (mem.

Nous avons également confirmé que la méthode du tirage est meilleure que la méthode naive.

# Exercice 2 (Décomposition en valeurs singulières randomisée)

La **décomposition en valeurs singulières (SVD)** d'une matrice $A \in \mathbb{R}^{m\times n}$, avec $m \ge n$, s'écrit sous la forme
$$
\tag{1}
A = U S V^T
$$
où $U = [u_1, \dots, u_n]\in \mathbb{R}^{m\times m}$ et  $V= [v_1, \dots, v_n]\in \mathbb{R}^{n\times n}$ sont des matrices orthonormées. Les $u_1, \dots, u_m$ sont appelés vecteurs singuliers à gauche, ils forment une base pour l'espace engendré par les colonnes de $A$. De même les $v_1, \dots, v_n$ sont les vecteurs singuliers à droite et forment une base pour l'espace engendré par les lignes de $A$.
La matrice $S = \mathrm{diag}(\sigma_1, \dots, \sigma_n)$ de $\mathbb{R}^{m \times n}$ est une matrice rectangulaire formée des valeurs singulières de la matrice $A$ notées $\sigma_1\ge \dots \ge \sigma_n \ge 0$.

Dans certaines applications (e.g. compression d'image, réduction de modèles), il peut être intéressant de considérer la SVD tronquée à $r$ termes, avec $r\le n$, de la matrice $A$. On la note

$$
\tag{2}
A_r = \sum_{i=1}^r [u_1, \dots, u_r]\mathrm{diag}(\sigma_1, \dots, \sigma_{r}) [v_1, \dots, v_r]^T = \sum_{i=1}^r\sigma_i u_iv_i^T,
$$
qui est une matrice de rang au plus $r$.

Le theorème d'Eckart-Young assure que $A_r$ est solution du problème de meilleure approximation de $A$ par une matrice de rang au plus $r$,
$$
\tag{3}
 \| A-A_r \| := \min_{Z \in R^{m \times n}, \mathrm{rang}(Z)\le r}\| A-Z \|,
$$
avec comme norme matricielle la norme spectrale ou la norme de Frobenius.
- Pour la norme spectrale $\|\cdot\|:=\|\cdot\|_2$ l'erreur d'approximation est donnée par
$$
\| A-A_r \|_2  = \sigma_{r+1},
$$
- Pour la norme de Frobenius $\|\cdot\|:=\|\cdot\|_F$ l'erreur d'approximation est donnée par
$$
\| A-A_r \|_F  = \sqrt{\sum_{i=r+1}^n\sigma^2_{i}}.
$$

**Question 1.**

 Considérer une matrice dont les entrées sont des variables aléatoires indépendantes qui suivent une loi uniforme discrete dans $\{0,\dots,9\}$.
- En utilisant la commande `numpy.linalg.svd()` de Python, tester la décomposition en valeurs singulières de cette matrice. Afficher les valeurs singulières de $A$.
- Calculer $A_r$ une approximation de rang $r$ de $A$ et donner les erreurs d'approximation en norme spectrale et de Frobenius.

On prendra par la suite $m=20, n=15$ et $r=5$ pour les tests numériques.

```python
#definition de la matrice A
n=15
m=20
entries = np.random.randint(0, 9+1, size= n*m)
A= np.reshape(entries, (m, n))
A
```

    array([[8, 5, 7, 7, 0, 2, 0, 6, 0, 8, 6, 3, 5, 2, 5],
           [0, 2, 8, 3, 6, 3, 5, 2, 3, 4, 2, 6, 3, 9, 3],
           [8, 0, 2, 0, 1, 4, 3, 2, 0, 9, 6, 1, 1, 0, 8],
           [8, 0, 8, 8, 7, 0, 1, 5, 0, 7, 3, 5, 5, 4, 8],
           [3, 2, 7, 3, 8, 7, 8, 6, 8, 8, 8, 9, 8, 2, 9],
           [4, 7, 0, 6, 8, 5, 2, 2, 7, 2, 9, 5, 7, 9, 5],
           [9, 8, 7, 7, 0, 2, 9, 7, 5, 5, 9, 5, 4, 2, 2],
           [5, 2, 2, 2, 6, 5, 7, 1, 0, 9, 9, 8, 0, 5, 7],
           [2, 3, 2, 1, 3, 9, 0, 5, 4, 0, 8, 1, 2, 6, 7],
           [9, 7, 8, 2, 8, 5, 6, 4, 6, 5, 1, 4, 7, 9, 3],
           [6, 0, 7, 5, 5, 4, 3, 7, 1, 2, 2, 0, 2, 4, 3],
           [7, 9, 5, 6, 0, 0, 8, 7, 2, 3, 6, 0, 4, 2, 7],
           [2, 5, 0, 4, 4, 2, 1, 7, 2, 4, 9, 7, 1, 1, 5],
           [6, 0, 2, 4, 0, 2, 7, 1, 5, 9, 0, 6, 3, 7, 3],
           [3, 8, 4, 2, 0, 4, 6, 7, 7, 6, 3, 0, 4, 2, 7],
           [2, 3, 9, 1, 9, 9, 6, 6, 8, 0, 6, 2, 2, 9, 7],
           [9, 7, 2, 8, 3, 3, 9, 8, 9, 2, 9, 2, 3, 3, 8],
           [7, 4, 3, 1, 8, 2, 1, 1, 1, 7, 3, 9, 9, 9, 5],
           [7, 7, 9, 6, 9, 1, 2, 9, 1, 0, 5, 8, 2, 6, 4],
           [8, 6, 4, 8, 3, 4, 6, 1, 3, 2, 9, 6, 4, 9, 1]])

```python
# décomposition en valeurs singulières
U,S,Vh = np.linalg.svd(A)
print(U, "\n", S, "\n", Vh)
```

    [[-2.08774734e-01  2.34620877e-01 -2.88850371e-01  1.22543532e-01
       1.39100581e-02  1.96469208e-01  2.51839557e-01 -3.13431996e-02
      -1.23290778e-01  5.25220081e-01  3.48876746e-03 -1.86897703e-02
      -1.53495321e-01 -1.64456482e-01  6.04106200e-02  1.85410790e-01
       4.21311501e-02 -9.33343593e-02  5.37674578e-01  1.41716736e-01]
     [-1.85281262e-01 -3.20948591e-01  5.08064583e-03  9.78451304e-02
       1.37814296e-01 -1.09698642e-01 -2.96477995e-01  2.88796061e-02
      -1.04774215e-01  2.58784707e-01 -3.79439936e-01  2.59434647e-01
      -5.44962730e-02 -5.00791726e-02 -2.40801107e-01 -5.62360601e-01
       1.94764940e-01 -5.15255854e-03  1.00580076e-01  1.27619504e-01]
     [-1.51647230e-01  1.55934833e-01 -2.80537532e-01 -3.93367623e-01
       1.52181971e-01  1.49431353e-01  1.96130270e-01 -3.29336193e-01
      -1.37232307e-01 -2.02247777e-01  1.72400583e-01 -1.45791413e-02
      -2.42884733e-02 -2.53683680e-01 -2.09944047e-01 -4.23462180e-01
      -3.58553519e-01 -4.87044675e-02 -1.08861310e-01  7.76696942e-02]
     [-2.25825414e-01 -5.70258713e-02 -3.76039027e-01  1.79168995e-01
       1.47445902e-01  3.23833570e-01  7.04739542e-02 -1.72894901e-02
       4.55763782e-01 -1.59646844e-02 -1.39319097e-01  2.30635009e-01
      -5.21671853e-02 -2.22182836e-01  6.01547604e-02  1.57761674e-01
       2.32198669e-01  7.86705958e-02 -4.62698588e-01 -7.07320501e-02]
     [-3.00823578e-01 -1.14811681e-01  9.29402798e-03 -3.57556218e-01
       2.02983748e-01  6.53211077e-02 -2.42618638e-01  4.60828889e-01
       2.07326562e-01  1.71930064e-01  4.42244065e-01  1.62574530e-01
       1.69937377e-01  1.18861040e-01  2.76431304e-01 -5.68723183e-02
      -1.34637229e-01 -8.95594479e-02  6.50825300e-02  6.40356737e-02]
     [-2.46023329e-01 -1.79230079e-01  2.21858535e-01 -5.10828878e-02
      -4.48206774e-01 -9.38793566e-02  3.25458411e-01  9.52830629e-02
       2.75312133e-01  4.57579841e-02 -2.81961918e-02  1.58007012e-01
      -4.10119244e-01 -2.11953910e-03 -1.04218699e-01 -9.08160699e-02
      -2.39101852e-01 -2.35127320e-01  3.35089406e-02 -3.55349343e-01]
     [-2.61669373e-01  3.64656393e-01 -1.89118583e-02  1.36862564e-01
      -6.62392096e-02 -1.97580806e-01 -2.81503273e-01  2.52737514e-02
      -1.80778943e-01  1.98362589e-01  3.64192164e-01 -8.44211207e-02
       3.71950112e-02 -8.69176025e-02 -3.94703790e-01  2.54771111e-02
       2.16303951e-01 -1.91153685e-03 -2.20057520e-01 -4.28685084e-01]
     [-2.23168062e-01 -1.02302589e-01 -2.17298036e-01 -4.04623068e-01
      -1.67533753e-01  5.11956390e-02 -3.35707160e-01 -2.50558893e-01
      -2.65398844e-01 -2.67252148e-01 -1.29451617e-01  1.22528330e-01
      -2.71021005e-01  1.34100048e-01  2.39760511e-01  2.22843794e-01
       2.56664752e-01  4.84260831e-02  1.75036380e-01 -2.27054779e-01]
     [-1.69587929e-01 -7.17453753e-02  3.47288621e-01 -2.25326526e-01
      -6.18381541e-02  2.47204903e-01  3.00155530e-01 -2.72758261e-01
      -1.03002592e-01  2.66758965e-01 -9.04246207e-02 -1.65456108e-01
       4.25943469e-01  8.95510895e-02  2.04934227e-01 -1.30632943e-01
       3.44132451e-01 -1.63983211e-01 -1.75293215e-01 -1.52998920e-01]
     [-2.64545236e-01 -2.06316440e-01  1.18501417e-02  2.33980287e-01
       2.21216067e-01 -2.96631002e-01  2.47861272e-01  2.96757304e-02
      -2.29562973e-01 -2.43070484e-01  2.76727018e-01 -1.82579311e-01
      -3.05134352e-01 -1.12659016e-02  1.92041221e-01 -2.26355536e-02
       2.91085772e-01 -3.09557368e-01 -1.49253726e-01  2.79579795e-01]
     [-1.64778167e-01 -6.33818807e-03  1.01497051e-02  2.22131564e-01
       2.84125697e-01  2.02666094e-01 -1.70279068e-02 -3.02650559e-01
       2.05613929e-01  2.56064148e-02  1.07790728e-01 -1.73773990e-01
      -1.84052454e-01  7.15103427e-01 -1.24724114e-01 -9.74593971e-02
      -1.18387897e-01  1.57512538e-01  9.11600251e-02 -8.65248518e-02]
     [-2.16519339e-01  4.09560634e-01  4.19753315e-02  1.41047945e-01
       5.66471563e-02 -5.70319786e-02  7.25844628e-02  4.43130523e-02
      -1.69499731e-01 -2.15488213e-01 -2.54822887e-01  5.18726491e-01
       2.52852577e-01  2.97503618e-01 -2.13647079e-02  1.18264174e-01
      -1.46021844e-01 -3.77583049e-01 -4.59379444e-02  8.92456244e-02]
     [-1.75196771e-01  1.05972121e-01  2.03749254e-02 -1.63793315e-01
      -3.40014342e-01  3.18425915e-01 -1.56507937e-01  2.88384573e-01
      -7.10158493e-02  2.91734049e-02 -1.92864194e-01 -3.60306247e-01
      -1.66770357e-01  1.66380387e-01 -2.69584227e-01  8.56371268e-02
      -3.19483749e-04 -8.37201346e-02 -2.72580947e-01  4.65233469e-01]
     [-1.73074698e-01 -5.90561287e-02 -2.86275563e-01 -1.21123344e-01
       1.24357531e-01 -4.78369517e-01 -1.07695575e-01 -9.79472123e-02
       2.61250727e-01  1.02464589e-01 -3.57043727e-01 -4.46462410e-01
       2.13337216e-01  5.58436989e-03  2.93847145e-02  1.49558418e-01
      -2.14802709e-01 -2.76021666e-01 -4.83355369e-03 -8.80469935e-02]
     [-1.96998641e-01  2.43637862e-01  1.61237092e-01 -1.32502065e-01
       2.91146808e-01 -1.57440389e-01  2.29449890e-01  2.99559224e-01
      -2.21910617e-01  8.33639099e-02 -3.21408480e-01 -7.53939162e-02
      -2.17796750e-01 -5.03646536e-03  2.23219181e-01 -2.71092998e-02
      -1.27516139e-01  5.21270809e-01 -1.77200186e-01 -1.54121566e-01]
     [-2.49747124e-01 -2.69832259e-01  4.63476131e-01 -4.44368597e-02
       3.06813388e-01  9.46965709e-02 -1.01252642e-01 -2.07265602e-01
      -5.06488378e-02 -1.82165357e-02 -8.68339584e-03  6.50363425e-02
       9.41718579e-03 -2.84873717e-01 -3.21565990e-01  4.95238781e-01
      -1.97690123e-01  4.89981054e-02  6.04359599e-02  9.73360744e-02]
     [-2.74478087e-01  3.58149231e-01  2.41821194e-01 -4.29968233e-02
      -7.34786894e-02 -1.04843127e-01 -7.53480126e-03 -3.02867780e-02
       4.54985712e-01 -3.63276204e-01 -2.73257878e-02 -1.04840193e-01
       8.87154936e-02 -1.62735998e-01 -3.04329513e-02 -1.62125436e-01
       3.19019651e-01  1.89498539e-01  3.70662743e-01  1.77034737e-01]
     [-2.23940276e-01 -3.58423947e-01 -3.06916546e-01  2.05353984e-02
      -1.80752675e-01 -7.82550970e-02  3.58673432e-01  2.22421731e-01
      -1.56569556e-01 -1.83032254e-01  6.01136161e-02  1.22824837e-02
       4.05648181e-01  1.59975367e-01 -3.18641860e-01  1.03439599e-01
       5.76130935e-02  3.46588561e-01  1.47479087e-01 -2.24241449e-02]
     [-2.47919618e-01 -8.01553924e-02  2.89721655e-02  4.58330983e-01
      -1.04496576e-01  3.50211126e-01 -2.24406901e-01  1.20422937e-01
      -1.85531259e-01 -2.54620667e-01 -7.40961127e-02 -2.67255374e-01
       1.59354378e-01 -2.19213534e-01  2.85981803e-01 -1.72120078e-01
      -2.98062657e-01 -4.79378515e-02  1.17559292e-01 -2.38303828e-01]
     [-2.39235543e-01 -9.38940215e-03  1.56061483e-02  1.65947763e-01
      -4.07068437e-01 -2.65049510e-01 -1.15638726e-01 -3.81285285e-01
       1.06742045e-03  2.17536484e-01  1.47982051e-01  1.54227045e-01
       8.50715175e-02  2.49790946e-03  2.90194554e-01  2.08710875e-03
      -2.36519152e-01  3.33614444e-01 -2.11341386e-01  3.51874392e-01]]
     [81.14251504 22.33329813 19.92933606 17.94414652 15.83212029 15.65264764
     11.61541834 11.2170188   8.72086849  7.99833555  6.96043009  6.09387527
      3.63729034  3.12147252  1.7207727 ]
     [[-0.31350191 -0.24372383 -0.27013956 -0.23613004 -0.25288055 -0.20053127
      -0.25793457 -0.26065103 -0.21085107 -0.24536732 -0.31606018 -0.24679151
      -0.21833357 -0.27657486 -0.29210497]
     [ 0.23878531  0.26469916 -0.08981383  0.24396718 -0.51849299 -0.18692987
       0.1924538   0.28564061  0.00874485  0.04254616  0.21281435 -0.29247458
      -0.1155718  -0.48050606  0.07721237]
     [-0.33415327  0.23071191 -0.04277932 -0.09537178  0.08434827  0.30499434
       0.0932124   0.19088353  0.42618107 -0.59888631  0.17371963 -0.29320517
      -0.11316316  0.09822258 -0.00548057]
     [ 0.21877162  0.23548388  0.40161433  0.32650863  0.09254078 -0.33312413
      -0.1339512   0.17850909 -0.15937526 -0.38718168 -0.31087109 -0.05903465
       0.07936016  0.1931191  -0.37471508]
     [-0.031724   -0.26782042  0.51419471 -0.23200777 -0.02386439  0.13425985
       0.22165485  0.22480197  0.14606787  0.19433309 -0.49900185 -0.35226083
      -0.00376818 -0.1347955   0.20100809]
     [-0.08810087 -0.16987254  0.17451895  0.03308471  0.30415517  0.05009905
      -0.49881647  0.38365681 -0.36544855 -0.12524468  0.26070935  0.00934138
      -0.18166355 -0.28353645  0.33182725]
     [ 0.27722177  0.23168339 -0.24074605 -0.12964543 -0.06024995  0.12437601
      -0.51425241 -0.05240606  0.04480352  0.07207315 -0.14568477 -0.43502768
       0.45206464  0.16218758  0.24507271]
     [-0.40289921  0.30565658 -0.07012055 -0.1155291   0.13291834 -0.32517406
      -0.07364287  0.21482896  0.25780893  0.08981442 -0.20661549  0.34666549
       0.41079543 -0.37871259  0.03777692]
     [ 0.05428482 -0.56400838 -0.24899899  0.57797391  0.14696497 -0.15083099
      -0.02016742 -0.01607608  0.39060744 -0.14065713 -0.09696082 -0.07690468
       0.14982558 -0.07486426  0.14883826]
     [-0.38895357 -0.12183006  0.33010045  0.2980194  -0.48508519  0.29892305
      -0.32157564  0.00986814  0.07765515  0.1893101   0.18673209  0.05597537
       0.23607912  0.08231073 -0.25600903]
     [ 0.37930455 -0.16816826  0.1568249  -0.22667472  0.18161261  0.24245224
       0.02300469 -0.16286992  0.08818231 -0.17132089  0.23834561  0.01759871
       0.35063808 -0.4957636  -0.41240277]
     [-0.29935043  0.0788777   0.257303    0.12903502  0.10102726 -0.21541665
       0.2696949  -0.48783359 -0.29981289 -0.12513405  0.23001787 -0.29996943
       0.3311563  -0.05792381  0.30418602]
     [ 0.14479589 -0.25095912  0.03500787 -0.26480817 -0.47538546 -0.06982907
       0.09038322  0.12444344 -0.10768094 -0.47991833 -0.01011793  0.37787997
       0.26873577  0.15269858  0.33141187]
     [-0.1644352  -0.10928829 -0.36647903  0.09867367  0.09962025  0.26163756
       0.34122578  0.42729173 -0.48630419  0.03711768 -0.09096173 -0.13217603
       0.34858568  0.07323263 -0.22156958]
     [ 0.03888073  0.26793681 -0.02841155  0.33642402 -0.03390119  0.53710828
       0.01690946 -0.26946926 -0.14393739 -0.17425854 -0.42982195  0.2662602
      -0.09599482 -0.29528041  0.21689784]]

```python
print("les valeurs singulières: ", S)
```

    les valeurs singulières:  [81.14251504 22.33329813 19.92933606 17.94414652 15.83212029 15.65264764
     11.61541834 11.2170188   8.72086849  7.99833555  6.96043009  6.09387527
      3.63729034  3.12147252  1.7207727 ]

Vérification :

```python
import scipy.linalg as la
U@la.diagsvd(S,*(U.shape[1],Vh.shape[0]))@Vh
```

    array([[ 8.00000000e+00,  5.00000000e+00,  7.00000000e+00,
             7.00000000e+00, -1.47846521e-14,  2.00000000e+00,
             5.26029070e-15,  6.00000000e+00, -2.73483985e-15,
             8.00000000e+00,  6.00000000e+00,  3.00000000e+00,
             5.00000000e+00,  2.00000000e+00,  5.00000000e+00],
           [ 2.97930644e-15,  2.00000000e+00,  8.00000000e+00,
             3.00000000e+00,  6.00000000e+00,  3.00000000e+00,
             5.00000000e+00,  2.00000000e+00,  3.00000000e+00,
             4.00000000e+00,  2.00000000e+00,  6.00000000e+00,
             3.00000000e+00,  9.00000000e+00,  3.00000000e+00],
           [ 8.00000000e+00,  1.13878268e-14,  2.00000000e+00,
             5.87809901e-15,  1.00000000e+00,  4.00000000e+00,
             3.00000000e+00,  2.00000000e+00, -2.80585842e-15,
             9.00000000e+00,  6.00000000e+00,  1.00000000e+00,
             1.00000000e+00, -1.08158142e-14,  8.00000000e+00],
           [ 8.00000000e+00,  1.13674421e-14,  8.00000000e+00,
             8.00000000e+00,  7.00000000e+00, -3.36414170e-15,
             1.00000000e+00,  5.00000000e+00,  8.41067758e-16,
             7.00000000e+00,  3.00000000e+00,  5.00000000e+00,
             5.00000000e+00,  4.00000000e+00,  8.00000000e+00],
           [ 3.00000000e+00,  2.00000000e+00,  7.00000000e+00,
             3.00000000e+00,  8.00000000e+00,  7.00000000e+00,
             8.00000000e+00,  6.00000000e+00,  8.00000000e+00,
             8.00000000e+00,  8.00000000e+00,  9.00000000e+00,
             8.00000000e+00,  2.00000000e+00,  9.00000000e+00],
           [ 4.00000000e+00,  7.00000000e+00,  2.43271144e-15,
             6.00000000e+00,  8.00000000e+00,  5.00000000e+00,
             2.00000000e+00,  2.00000000e+00,  7.00000000e+00,
             2.00000000e+00,  9.00000000e+00,  5.00000000e+00,
             7.00000000e+00,  9.00000000e+00,  5.00000000e+00],
           [ 9.00000000e+00,  8.00000000e+00,  7.00000000e+00,
             7.00000000e+00, -1.47039110e-14,  2.00000000e+00,
             9.00000000e+00,  7.00000000e+00,  5.00000000e+00,
             5.00000000e+00,  9.00000000e+00,  5.00000000e+00,
             4.00000000e+00,  2.00000000e+00,  2.00000000e+00],
           [ 5.00000000e+00,  2.00000000e+00,  2.00000000e+00,
             2.00000000e+00,  6.00000000e+00,  5.00000000e+00,
             7.00000000e+00,  1.00000000e+00,  9.03503125e-16,
             9.00000000e+00,  9.00000000e+00,  8.00000000e+00,
             5.85860544e-16,  5.00000000e+00,  7.00000000e+00],
           [ 2.00000000e+00,  3.00000000e+00,  2.00000000e+00,
             1.00000000e+00,  3.00000000e+00,  9.00000000e+00,
             5.78036559e-15,  5.00000000e+00,  4.00000000e+00,
             4.12327313e-15,  8.00000000e+00,  1.00000000e+00,
             2.00000000e+00,  6.00000000e+00,  7.00000000e+00],
           [ 9.00000000e+00,  7.00000000e+00,  8.00000000e+00,
             2.00000000e+00,  8.00000000e+00,  5.00000000e+00,
             6.00000000e+00,  4.00000000e+00,  6.00000000e+00,
             5.00000000e+00,  1.00000000e+00,  4.00000000e+00,
             7.00000000e+00,  9.00000000e+00,  3.00000000e+00],
           [ 6.00000000e+00,  1.11727574e-14,  7.00000000e+00,
             5.00000000e+00,  5.00000000e+00,  4.00000000e+00,
             3.00000000e+00,  7.00000000e+00,  1.00000000e+00,
             2.00000000e+00,  2.00000000e+00, -5.03092605e-15,
             2.00000000e+00,  4.00000000e+00,  3.00000000e+00],
           [ 7.00000000e+00,  9.00000000e+00,  5.00000000e+00,
             6.00000000e+00, -1.34637851e-14, -4.20529909e-15,
             8.00000000e+00,  7.00000000e+00,  2.00000000e+00,
             3.00000000e+00,  6.00000000e+00, -7.24328196e-15,
             4.00000000e+00,  2.00000000e+00,  7.00000000e+00],
           [ 2.00000000e+00,  5.00000000e+00, -2.70453764e-17,
             4.00000000e+00,  4.00000000e+00,  2.00000000e+00,
             1.00000000e+00,  7.00000000e+00,  2.00000000e+00,
             4.00000000e+00,  9.00000000e+00,  7.00000000e+00,
             1.00000000e+00,  1.00000000e+00,  5.00000000e+00],
           [ 6.00000000e+00,  7.72801445e-15,  2.00000000e+00,
             4.00000000e+00, -8.71558572e-15,  2.00000000e+00,
             7.00000000e+00,  1.00000000e+00,  5.00000000e+00,
             9.00000000e+00,  2.19603744e-15,  6.00000000e+00,
             3.00000000e+00,  7.00000000e+00,  3.00000000e+00],
           [ 3.00000000e+00,  8.00000000e+00,  4.00000000e+00,
             2.00000000e+00, -8.98472074e-15,  4.00000000e+00,
             6.00000000e+00,  7.00000000e+00,  7.00000000e+00,
             6.00000000e+00,  3.00000000e+00, -3.34535897e-15,
             4.00000000e+00,  2.00000000e+00,  7.00000000e+00],
           [ 2.00000000e+00,  3.00000000e+00,  9.00000000e+00,
             1.00000000e+00,  9.00000000e+00,  9.00000000e+00,
             6.00000000e+00,  6.00000000e+00,  8.00000000e+00,
             3.69514014e-15,  6.00000000e+00,  2.00000000e+00,
             2.00000000e+00,  9.00000000e+00,  7.00000000e+00],
           [ 9.00000000e+00,  7.00000000e+00,  2.00000000e+00,
             8.00000000e+00,  3.00000000e+00,  3.00000000e+00,
             9.00000000e+00,  8.00000000e+00,  9.00000000e+00,
             2.00000000e+00,  9.00000000e+00,  2.00000000e+00,
             3.00000000e+00,  3.00000000e+00,  8.00000000e+00],
           [ 7.00000000e+00,  4.00000000e+00,  3.00000000e+00,
             1.00000000e+00,  8.00000000e+00,  2.00000000e+00,
             1.00000000e+00,  1.00000000e+00,  1.00000000e+00,
             7.00000000e+00,  3.00000000e+00,  9.00000000e+00,
             9.00000000e+00,  9.00000000e+00,  5.00000000e+00],
           [ 7.00000000e+00,  7.00000000e+00,  9.00000000e+00,
             6.00000000e+00,  9.00000000e+00,  1.00000000e+00,
             2.00000000e+00,  9.00000000e+00,  1.00000000e+00,
             1.98316377e-15,  5.00000000e+00,  8.00000000e+00,
             2.00000000e+00,  6.00000000e+00,  4.00000000e+00],
           [ 8.00000000e+00,  6.00000000e+00,  4.00000000e+00,
             8.00000000e+00,  3.00000000e+00,  4.00000000e+00,
             6.00000000e+00,  1.00000000e+00,  3.00000000e+00,
             2.00000000e+00,  9.00000000e+00,  6.00000000e+00,
             4.00000000e+00,  9.00000000e+00,  1.00000000e+00]])

```python
np.allclose(A,U@la.diagsvd(S,*(U.shape[1],Vh.shape[0]))@Vh)# np.dot(U[:, :len(S)] * S, Vh))
```

    True

```python
r=5

```

Calculons la matrice $A_r$ :

On a $A_r = \sum_{i=1}^r [u_1, \dots, u_r]\mathrm{diag}(\sigma_1, \dots, \sigma_{r}) [v_1, \dots, v_r]^T = \sum_{i=1}^r\sigma_i u_iv_i^T$ :

```python
#Ur
Ur=U[np.ix_(list(range(len(U))),list(range(r)))]
#Sr
Sr=S[:r]
#Vr
Vhr=Vh[np.ix_(list(range(r)),list(range(Vh.shape[1])))]
Sr
```

    array([81.14251504, 22.33329813, 19.92933606, 17.94414652, 15.83212029])

```python
#Vr exemple
Vhr=Vh[np.ix_(list(range(r)),list(range(Vh.shape[1])))]
Vhr
```

    array([[-0.31350191, -0.24372383, -0.27013956, -0.23613004, -0.25288055,
            -0.20053127, -0.25793457, -0.26065103, -0.21085107, -0.24536732,
            -0.31606018, -0.24679151, -0.21833357, -0.27657486, -0.29210497],
           [ 0.23878531,  0.26469916, -0.08981383,  0.24396718, -0.51849299,
            -0.18692987,  0.1924538 ,  0.28564061,  0.00874485,  0.04254616,
             0.21281435, -0.29247458, -0.1155718 , -0.48050606,  0.07721237],
           [-0.33415327,  0.23071191, -0.04277932, -0.09537178,  0.08434827,
             0.30499434,  0.0932124 ,  0.19088353,  0.42618107, -0.59888631,
             0.17371963, -0.29320517, -0.11316316,  0.09822258, -0.00548057],
           [ 0.21877162,  0.23548388,  0.40161433,  0.32650863,  0.09254078,
            -0.33312413, -0.1339512 ,  0.17850909, -0.15937526, -0.38718168,
            -0.31087109, -0.05903465,  0.07936016,  0.1931191 , -0.37471508],
           [-0.031724  , -0.26782042,  0.51419471, -0.23200777, -0.02386439,
             0.13425985,  0.22165485,  0.22480197,  0.14606787,  0.19433309,
            -0.49900185, -0.35226083, -0.00376818, -0.1347955 ,  0.20100809]])

On définit la décomposition en une fonction :

```python
def rSVD_1(A):
  U,S,Vh = np.linalg.svd(A)
  #Ur
  Ur=U[np.ix_(list(range(len(U))),list(range(r)))]
  #Sr
  Sr=S[:r]
  #Vr
  Vhr=Vh[np.ix_(list(range(r)),list(range(Vh.shape[1])))]
  return Ur,Sr,Vhr

Ur,Sr,Vhr=rSVD_1(A)
Ar=Ur@la.diagsvd(Sr,*(Ur.shape[1],Vhr.shape[0]))@Vhr
Ar
```

    array([[ 8.95974685,  4.64650981,  5.34831695,  6.49441131,  1.27977213,
            -0.04106605,  4.59565044,  5.2554752 ,  0.84610491,  7.01853742,
             4.67582335,  4.1287213 ,  3.91821448,  1.99707888,  4.60482914],
           [ 3.28272648,  1.61932382,  6.52781938,  1.85869933,  7.63727795,
             4.09364879,  2.7562393 ,  2.69448837,  3.18932714,  3.06751539,
             1.60929908,  4.90478764,  4.23052459,  7.65717373,  3.61823309],
           [ 4.93679228,  0.32348869,  1.65450161,  1.42473104,  0.12373696,
             2.78623937,  4.80253997,  2.41645655,  1.71914168,  9.71694979,
             4.65106531,  3.22549369,  2.34755221, -0.50720302,  7.02318192],
           [ 8.57402756,  2.53178219,  7.87656098,  5.23902404,  4.90382365,
             0.86932971,  3.86951549,  4.08056095,  0.48719814,  8.13895897,
             2.05425192,  6.07993759,  5.2423606 ,  5.25005042,  4.55978806],
           [ 5.4726843 ,  2.94165002,  5.89203813,  2.28012079,  6.84737421,
             7.9994919 ,  7.39162526,  5.24243701,  6.69527372,  8.87798676,
             7.59232947,  5.966431  ,  5.08352333,  6.32909938,  9.98132263],
           [ 3.84973845,  6.51061785,  1.54624756,  4.66266668,  7.58112736,
             5.45260711,  3.34082755,  3.14515471,  5.16814532,  1.0558863 ,
            10.05164673,  7.35477785,  4.2748377 ,  8.65840717,  4.41509175],
           [ 9.29758464,  8.10280391,  5.46750037,  8.08161804,  1.3671976 ,
             1.66156387,  6.4473874 ,  7.99122889,  3.84289836,  4.62730913,
             8.13827622,  3.19304197,  3.93605772,  2.53776567,  5.70194912],
           [ 5.07427237,  1.8101711 ,  1.00243806,  2.37628107,  4.79001184,
             4.80015606,  4.21205884,  1.34836389,  2.72230735,  9.23527234,
             8.06548542,  7.76996187,  4.14158487,  4.6361782 ,  7.32438713],
           [ 0.76517494,  3.83658684,  1.43790247,  1.10530725,  4.54361177,
             6.38539899,  4.21075486,  3.50837627,  6.33855961,  0.53847634,
             6.95607722,  2.41890865,  2.08920903,  4.60676198,  5.17623476],
           [ 6.35783885,  4.11727243,  9.68959675,  4.48038432,  8.14225278,
             4.30950183,  4.88592969,  5.86083861,  4.42888334,  3.98454927,
             2.79206184,  5.09439998,  5.51251958,  8.5128847 ,  5.04394291],
           [ 4.81960052,  3.00180175,  7.52979266,  3.36116189,  3.73311336,
             2.04548285,  3.90347882,  5.20597637,  2.9259514 ,  2.48440371,
             0.74711618,  1.46193248,  3.21207759,  3.94924643,  3.30415302],
           [ 7.93774557,  7.25193893,  5.36639451,  6.91861053, -0.0163558 ,
             1.34571917,  6.22971434,  8.00516497,  3.86855562,  3.39334551,
             6.41040236,  0.95002613,  2.88158346,  0.91406883,  5.06550926],
           [ 4.41393936,  4.9345021 , -0.33803866,  4.18475529,  2.25853171,
             2.78852604,  3.36060456,  2.72412425,  2.87331003,  3.43748421,
             8.66718338,  4.76688766,  2.57136891,  2.99245125,  4.35233113],
           [ 5.45628231,  0.71828533,  4.29576665,  2.37205467,  3.50588336,
             2.31103768,  3.56426671,  2.24934654,  1.15209549,  8.03070276,
             2.86006321,  4.95920087,  3.68436753,  3.27237269,  5.24185057],
           [ 4.57046353,  4.28315997,  5.10728113,  2.94979904,  1.16205333,
             4.57932366,  6.80998473,  6.94589962,  5.83973015,  4.04561164,
             5.20740976, -0.0720133 ,  2.29150944,  1.04161256,  6.88928493],
           [ 1.49913057,  3.98627176,  7.79796944,  1.0467368 ,  8.83861141,
             8.92523135,  6.11178053,  6.27356628,  8.99336734,  0.43693187,
             4.55110977,  2.39147502,  3.99417123,  8.59896681,  6.67880492],
           [ 7.1499432 ,  8.78717005,  4.18391121,  6.76881652,  1.84773554,
             4.54171506,  7.57876389,  8.61060615,  6.77293899,  2.99151441,
            10.39903455,  2.19938558,  3.33605478,  2.79760924,  7.15216895],
           [ 6.00053735,  1.75187608,  4.56585429,  3.70543793,  8.33200367,
             2.76769761,  1.89258104,  0.70471419,  0.67786263,  7.08239401,
             4.29047881,  9.6054066 ,  5.62468957,  8.72812687,  4.01001263],
           [ 7.53800585,  6.94210046,  8.02276271,  7.32754264,  6.86459706,
             1.58293172,  3.4297571 ,  5.93856029,  2.91965987,  1.00822665,
             4.34629967,  5.41619303,  5.19264706,  8.29197751,  2.32049927],
           [ 6.78766233,  7.17471815,  3.13158827,  6.97048374,  5.47328849,
             2.16955958,  3.16831462,  4.14204261,  2.80783669,  2.16255001,
             8.43505542,  6.85534017,  4.48797113,  6.94402132,  3.24122537]])

Vérification des normes :

```python
#norme spectrale
np.linalg.norm(A-Ar,ord=2)
```

    15.652647638313617

```python
S[r]
```

    15.652647638313615

```python
np.isclose(S[r],np.linalg.norm(A-Ar,ord=2))
```

    True

```python
#norme de Frobenus
sqrt(sum(S[r:]**2))
```

    27.51888777204114

```python
linalg.norm(A-Ar, ord='fro')
```

    27.51888777204115

```python
np.isclose(sqrt(sum(S[r:]**2)),linalg.norm(A-Ar, ord='fro'))
```

    True

On peut aussi vérifer avec la ```TruncatedSVD``` de Sklearn :

```python
from sklearn.decomposition import TruncatedSVD

svd = TruncatedSVD(n_components=r, n_iter=10, random_state=42)
svd.fit(A)
Ar_sk = svd.transform(A)

print(svd.singular_values_)

```

    [81.14251504 22.33329813 19.92933606 17.94414652 15.83212029]

```python
S[:r]
```

    array([81.14251504, 22.33329813, 19.92933606, 17.94414652, 15.83212029])

**Question 2.** On considère désormais une version randomisée de la SVD.
1. On génère une matrice gaussienne $\Omega$ de taille $n \times k$ dont les entrées sont i.i.d. et suivent la loi $\cal{N}(0,1/k)$.
2. On en déduit $Y = A \Omega$.
3. On calcule la décomposition $QR$ de $Y$.
4. On forme la matrice $B =Q^T A$ de taille $k \times n$.
5. On calcule la SVD de la matrice $B$ notée $\tilde USV^T$
6. Alors la SVD randomisée de la matrice $A$ est donnée par
$$\hat A_k = U S V^T$$ avec $U =Q \tilde U \in  \mathbb{R}^{m\times k}$, $S \in  \mathbb{R}^{k \times k}$ et $V \in \mathbb{R}^{n\times k}$.

*Remarque : Ici SVD randomisée retourne une décomposition tronquée à $k$ termes.*

*Question 2.a*

Définir une fonction  `rSVD(A,Omega)` qui prend en argument une matrice $A \in \mathbb{R}^{m \times n}$ et une matrice aléatoire gaussienne $\Omega \in  \mathbb{R}^{n \times k}$ et retourne $U,S,V^T$. Pour la factorisation $QR$ de la matrice $Y$ utiliser `np.linalg.qr()`.

```python

#definition de la matrice A
n=15
m=20
entries = np.random.randint(0, 9+1, size= n*m)
A= np.reshape(entries, (m, n))
```

```python
k=10
entries = np.random.normal(0, 1/k, size= n*k)
Omega= np.reshape(entries, (n, k))
#Omega

def rSVD(A,Omega):
  Y=A@Omega
  Q,R = np.linalg.qr(Y)
  B = np.transpose(Q)@A
  U_ , S , Vh = np.linalg.svd(B)
  U = Q @ U_
  return U,S,Vh
```

```python
rSVD(A,Omega)
```

    (array([[-0.24295666,  0.20911214, -0.24293062,  0.05276083, -0.31596681,
              0.30214748,  0.16522719,  0.08847456, -0.1399176 , -0.07546655],
            [-0.16848859,  0.18517951, -0.047516  , -0.15466959, -0.35063234,
             -0.05480719,  0.16554326, -0.27024564,  0.04138071, -0.04251701],
            [-0.17548436,  0.0928677 ,  0.26968528, -0.34463618,  0.48983575,
              0.26704732,  0.06048475, -0.22841078, -0.07647665, -0.33760688],
            [-0.2968976 ,  0.08205571, -0.27648518, -0.36144564,  0.19105236,
              0.01605749, -0.00316294,  0.19875134, -0.08575054,  0.56527614],
            [-0.20902202, -0.37712235,  0.09696073, -0.23110745, -0.08425176,
             -0.05900037, -0.41447744,  0.19981562, -0.08020377,  0.08899209],
            [-0.22436721, -0.36921733, -0.11799808, -0.11087539,  0.22197215,
              0.29083374,  0.02287996, -0.02094831,  0.3066016 , -0.02220439],
            [-0.18689386,  0.40479101,  0.01683913, -0.04302421, -0.09224616,
              0.08114749, -0.09034164,  0.11882648, -0.14240549, -0.37934202],
            [-0.14230039,  0.18873909, -0.13036354,  0.17969913,  0.0569992 ,
             -0.32739414, -0.16790684, -0.3217911 , -0.17486032,  0.17275734],
            [-0.26179171, -0.41174869, -0.0999043 ,  0.36893015,  0.04010438,
             -0.21916585,  0.03279012, -0.18012159,  0.09510267, -0.16676408],
            [-0.19856871,  0.10808812, -0.3159143 , -0.26095169,  0.15532888,
             -0.3942888 ,  0.3479804 , -0.0676543 ,  0.38489553, -0.09952426],
            [-0.23791391, -0.1419586 ,  0.21077888, -0.03649549, -0.39590134,
              0.24418668,  0.07248369,  0.25287732,  0.21735436, -0.0467456 ],
            [-0.20260179,  0.0623463 ,  0.46208712, -0.21030492, -0.18286413,
             -0.25465885, -0.07148665, -0.01896655,  0.26976586,  0.04581718],
            [-0.27791119,  0.23899929, -0.00132358,  0.08518556, -0.1267616 ,
              0.15156236, -0.48517319, -0.34975753,  0.19666836,  0.23101747],
            [-0.24919191, -0.1731812 , -0.22774647, -0.03864886, -0.13778046,
             -0.19280983,  0.08252423,  0.27239413, -0.42686884, -0.1133825 ],
            [-0.22184961, -0.23142576, -0.02511646, -0.11693985, -0.12572896,
              0.14963216,  0.19555557, -0.44102734, -0.3291617 , -0.06942797],
            [-0.14573509, -0.03348083,  0.22664695,  0.30406075,  0.1611378 ,
              0.2730162 ,  0.2752955 , -0.12671524, -0.1953835 ,  0.39118034],
            [-0.24041097,  0.11870321,  0.45774606,  0.02628102,  0.18982602,
             -0.31267192,  0.09608713,  0.20470138, -0.29104032,  0.03551051],
            [-0.26887731, -0.09125861,  0.11879997,  0.28772346, -0.01943043,
             -0.16910973,  0.00130477, -0.04768803,  0.07929625, -0.12891901],
            [-0.20256003,  0.20545566,  0.05199864,  0.34608409,  0.09772623,
              0.09496305,  0.30579857,  0.26993282,  0.28011614,  0.14067005],
            [-0.24063663,  0.13008079, -0.22582298,  0.23693143,  0.30651923,
              0.10892196, -0.37609545,  0.20969682,  0.01164535, -0.27034729]]),
     array([79.86161996, 21.2792864 , 20.15592485, 16.08372274, 15.21831086,
            13.77627942, 10.99120537,  9.23170503,  8.49802838,  6.75765844]),
     array([[-2.65586335e-01, -2.60742118e-01, -2.43437639e-01,
             -2.70152489e-01, -2.68031708e-01, -3.12649838e-01,
             -2.16589485e-01, -2.37874220e-01, -2.58504949e-01,
             -3.03146493e-01, -2.78807116e-01, -2.49214041e-01,
             -2.35036355e-01, -2.61347673e-01, -1.84496020e-01],
            [ 4.51585180e-01,  1.53111506e-01, -2.86639331e-01,
              4.08242765e-01, -1.91582538e-01,  2.65808809e-02,
             -2.09351449e-01,  1.43661834e-01,  2.43800476e-01,
              1.41816454e-01, -3.37589760e-01, -2.85631795e-01,
             -3.82027890e-01,  2.37237028e-02, -1.77538051e-02],
            [-4.24801713e-03,  3.36226889e-01,  2.83071004e-01,
             -1.31092351e-02, -2.65128536e-01,  1.58219200e-01,
             -2.34589533e-01,  8.74959305e-02,  2.76464787e-02,
             -3.68543557e-01, -3.44548064e-01, -9.24442895e-02,
              4.32375863e-01, -1.97988465e-01,  3.98273561e-01],
            [ 2.04932466e-01, -5.74864681e-02, -1.78802392e-01,
              2.46604718e-02, -8.31185952e-02, -1.79099895e-01,
             -4.66195758e-01,  4.03900853e-01, -3.64347481e-01,
              4.92857314e-02,  7.04795637e-03,  5.50349572e-01,
              1.93270834e-01,  2.09440823e-02, -1.63569351e-01],
            [-3.76303982e-02,  7.43564971e-03, -1.56917762e-01,
              4.79772036e-01,  1.72259041e-01, -3.13401574e-02,
             -9.66617804e-02, -5.27860888e-01, -4.51221475e-01,
              2.66271753e-01, -7.55386426e-02, -1.23483762e-01,
              3.19551335e-01,  3.66289630e-02,  1.61795382e-01],
            [-4.98475759e-01,  1.62442330e-01, -3.98960238e-01,
              3.74587432e-01, -9.70254520e-02,  5.96436922e-02,
              2.99769224e-01,  1.33547963e-01,  1.04984954e-01,
             -9.42607158e-02, -5.67142626e-02,  3.91083208e-01,
             -1.21587566e-01, -3.08189649e-01,  1.38374618e-01],
            [ 3.85925629e-01, -8.29449880e-02,  2.82174568e-01,
              1.73280043e-01,  1.21656563e-01, -5.61618134e-01,
              4.42359949e-01, -2.18216209e-02, -1.29007294e-01,
             -2.49825909e-01, -1.96423866e-01,  1.90561401e-01,
             -1.13353551e-01, -1.19855670e-01,  1.64295659e-01],
            [ 6.40072418e-02, -5.78707192e-02, -1.83660840e-01,
             -1.90951954e-01,  6.05172578e-01,  3.27366512e-01,
             -7.08640048e-02, -1.40827820e-01,  5.80814322e-02,
             -2.50058452e-01, -5.10923301e-01,  2.52215334e-01,
             -9.13773042e-02,  1.58651743e-01, -5.01717871e-03],
            [-7.82972234e-02, -3.89554050e-02,  2.15704099e-01,
              1.48342081e-01,  3.56093603e-01, -2.08886857e-01,
             -2.80994808e-01, -4.87461567e-02,  4.22489473e-01,
              1.66572079e-01, -3.76738393e-02, -3.63256088e-03,
              1.78972911e-01, -5.65636394e-01, -3.35254810e-01],
            [ 2.42212938e-01,  7.12328600e-01, -1.81603123e-01,
             -2.37169834e-01, -5.24658171e-02, -9.37278090e-02,
              1.81481794e-02, -4.20387487e-01,  8.33410039e-03,
             -9.83748779e-02,  2.13951716e-01,  1.82601021e-01,
             -4.55597584e-02, -9.81680089e-02, -2.35628693e-01],
            [-1.56646829e-02,  2.87819942e-01, -5.63354632e-03,
             -2.80623128e-01,  2.97066056e-01,  5.67260108e-02,
              1.27780038e-01,  3.65983235e-01, -4.38685104e-01,
              3.64754343e-01, -9.78888149e-02, -2.50685490e-01,
             -1.77448817e-01, -3.79880844e-01,  1.50606606e-01],
            [-2.50780191e-02, -3.97796524e-02,  5.01214867e-01,
              5.11869697e-02, -3.25690371e-01,  3.58916032e-01,
              2.23560408e-02, -2.40680008e-01, -1.76333441e-01,
              2.84209165e-01, -2.62532457e-01,  3.39157102e-01,
             -3.07572275e-01, -1.20509381e-01, -2.13748103e-01],
            [ 2.09090247e-01, -1.53990774e-01, -2.81365933e-01,
             -2.00874088e-01, -2.39388084e-01,  8.64320180e-02,
              4.65863633e-01,  2.69016572e-02,  8.91568137e-02,
              2.30117352e-01, -3.20401939e-01, -3.25596549e-02,
              5.19885937e-01, -1.01071749e-01, -2.92239280e-01],
            [-4.05895278e-01,  3.31339442e-01,  1.36863231e-01,
             -6.04977547e-02,  1.60315386e-04, -4.40929695e-01,
             -5.70151407e-03,  8.68031713e-02,  1.09922037e-01,
              2.75404604e-01, -3.90280884e-01,  8.05682559e-03,
              3.80132435e-02,  4.96429792e-01, -1.13504097e-01],
            [ 8.03004082e-02, -1.69393703e-01, -1.43791203e-01,
             -3.44898807e-01, -1.13787312e-01, -1.75964322e-01,
             -1.54287756e-01, -2.36588148e-01,  2.88872640e-01,
              4.06156093e-01, -1.76753425e-02,  2.53778599e-01,
             -6.70260677e-02, -1.02794474e-01,  6.11024955e-01]]))

*Question 2.b*

Reprendre les différents points de la Question 1. pour la fonction la SVD randomisée `rSVD()`.
On considèrera une matrice aléatoire de taille $k=l+r$ avec $l=4$.

```python
l =  4
n=15
k=r + l
entries = np.random.randint(0, 9+1, size= n*m)
A= np.reshape(entries, (m, n))

entries = np.random.normal(0, 1/k, size= n*k)
Omega= np.reshape(entries, (n, k))

```

```python
# Valeurs sing. S
#par la méthode de la question 1
U_1,S_1,Vh_1 = rSVD_1(A)
print(S_1)
U_2,S_2,Vh_2 = rSVD(A,Omega)
print(S_2)
U_3,S_3,Vh_3 = np.linalg.svd(A)
print(S_3)
```

    [78.2028406  21.80831211 19.88334755 17.5464853  16.91267418]
    [77.68442211 21.57704051 19.31971216 16.79758938 14.67452894 12.65156491
     11.51871842 10.66549451  6.51861085]
    [78.2028406  21.80831211 19.88334755 17.5464853  16.91267418 14.07579029
     12.90014707 11.56092787 10.34131357  8.70682192  7.56925519  6.75349791
      5.45240487  4.01244576  3.12538132]

Nous avons extrait avec cette méthode les valeurs singulières sans utiliser la méthode svd implémentée, avec une précision acceptable. Il s'agit donc d'un moyen rapide de réduire la dimensionnalité.

**Question 3**

On va essayer d'améliorer la SVD randomisée en effectuant des puissances itérées de la matrice $A$. Ainsi l'étape 2. de l'algorithme devient :

2. Calculer $Y = A \Omega$\
   Pour $i=1,\dots,p$ faire \
       $Y = A (A^T Y)$\
   Fin pour

*Question 3.a*

Ecrire une fonction `power_iteration(A,Omega,p=3)` renvoyant une matrice $Y$ obtenue par des puissances itérées de $A$. Cette fonction prendra en argument $A, \Omega$ et $p=3$ un paramètre correspondant au nombre de puissances itérés. Ecrire une nouvelle fonction `rSVD2(A,Omega)` où la matrice $Y$ est obtenue par puissances itérées.

```python
def power_iteration(A,Omega,p=3):
  Y= A@Omega
  for i in range(p):
    Y =  A @ (np.transpose(A)@Y)
  return Y
```

*Question 3.b* Reprendre les différents points de la Question 2.c. Comparer les résultats obtenus avec `np.linalg.svd()`et `rSVD()`.

```python
def rSVD_power(A,Omega,p):
  Y=power_iteration(A,Omega,p=3)
  Q,R = np.linalg.qr(Y)
  B = np.transpose(Q)@A
  U_ , S , Vh = np.linalg.svd(B)
  U = Q @ U_
  return U,S,Vh
```

```python
U_1,S_1,Vh_1 = rSVD_power(A,Omega,3)
print("rSVD power :" , S_1)
U_2,S_2,Vh_2 = rSVD(A,Omega)
print("rSVD :" , S_2)
U_3,S_3,Vh_3 = np.linalg.svd(A)
print("SVD :" , S_3)
```

    rSVD power : [78.2028406  21.80830967 19.8832032  17.54415591 16.9085791  14.05367681
     12.89877685 11.55815385  9.7562184 ]
    rSVD : [77.68442211 21.57704051 19.31971216 16.79758938 14.67452894 12.65156491
     11.51871842 10.66549451  6.51861085]
    SVD : [78.2028406  21.80831211 19.88334755 17.5464853  16.91267418 14.07579029
     12.90014707 11.56092787 10.34131357  8.70682192  7.56925519  6.75349791
      5.45240487  4.01244576  3.12538132]

Nous pouvons voir clairement que rSVD_power est bien meilleur que rSVD .
On peut comparer les distances :

```python
linalg.norm(A-U_1@la.diagsvd(S_1,*(U_1.shape[1],Vh_1.shape[0]))@Vh_1, ord='fro')

```

    15.716715113498228

```python
linalg.norm(A-U_2@la.diagsvd(S_2,*(U_2.shape[1],Vh_2.shape[0]))@Vh_2, ord='fro')

```

    24.491259024169523

**Question 4.**

On s'intéresse dans cette dernière question à la compression d'une image de taille $256 \times 256$ : `lena256x256.png`

- Charger l'image en utilisant `imageio.imread()`. Celle-ci est alors stockée sous forme d'une matrice $A$ de taille $256\times 256$.
- Afin de compresser cette image, il suffit d'appliquer une SVD tronquée de rang $r$  à la matrice $A$. Calculer les approximations de rang $r=50$ pour les trois méthodes étudiées précédemment. Afficher les images compressées obtenues.
- Comparer les erreurs associées (en norme $\|\cdot\|_2$ et $\|\cdot\|_F$) et les temps d'exécution de chacune d'elle.

Commmenter.

```python
import os
from google.colab import drive

drive.mount("/content/gdrive")
```

    Mounted at /content/gdrive

```python
image_path = "/content/gdrive/My Drive/Google Collab/lena256x256.png"
```

```python
import imageio as iio

im = iio.imread(image_path)
```

```python
r = 50
```

```python
np.shape(im)
```

    (256, 256)

Méthode 1 (rSVD):

```python
import time
n=256
start = time.time()

k=r
entries = np.random.normal(0, 1/k, size= n*k)
Omega= np.reshape(entries, (n, k))
U,S,Vh = rSVD(im,Omega)
A_k = U@la.diagsvd(S,*(U.shape[1],Vh.shape[0]))@Vh

end = time.time()
print(end - start)
```

    0.018517017364501953

Méthode 2 avec k=r+l

```python

start = time.time()
k= r + l
entries = np.random.normal(0, 1/k, size= n*k)
Omega= np.reshape(entries, (n, k))
U_2,S_2,Vh_2 = rSVD(im,Omega)
A_k_2 = U_2@la.diagsvd(S_2,*(U_2.shape[1],Vh_2.shape[0]))@Vh_2
end = time.time()
print(end - start)
```

    0.02601337432861328

Méthode 3 (power):

```python
import scipy.linalg as la

start = time.time()
p =3
U_3,S_3,Vh_3 = rSVD_power(im,Omega,p)

A_k_3 =  U_3@la.diagsvd(S_3,*(U_3.shape[1],Vh_3.shape[0]))@Vh_3
end = time.time()
print(end - start)
```

    0.021928071975708008

SVD originale :

```python
start = time.time()
U_4,S_4,Vh_4 = np.linalg.svd(im)
A_k_4 =  U_4@la.diagsvd(S_4,*(U_4.shape[1],Vh_4.shape[0]))@Vh_4
end = time.time()
print(end - start)
```

    0.051738739013671875

Comme prévu, moins nous utilisons d'informations et plus la dimension est petite, plus l'algorithme est rapide, même si nous avons gagné beaucoup de temps par rapport aux informations perdues.

Comparaison :

```python
import matplotlib.pyplot as plt
imgplot = plt.imshow(im)
```

```python
imgplot = plt.imshow(A_k_4)
```

La SVD complète donne les mêmes résultats que l'original, comme prévu.

```python
imgplot = plt.imshow(A_k)
```

rSVD, est rapide  et donne un bon résultat

```python
imgplot = plt.imshow(A_k_2)
```

également bon résultat pour k>r

```python
imgplot = plt.imshow(A_k_3)
```

Voici également une bonne image presque identique à l'original.
Nous concluons que les méthodes rSVD pourraient nous donner un gain énorme en termes de temps de calcul (et aussi un peu de gain de mémoire).