# Understanding Word Embeddings
> "Deep dive into the details of word2vec and Global Vectors (GloVe)"
- toc: true
- branch: master
- badges: true
- image: images/w2v.PNG
- comments: true
- author: Kushal Jain
- categories: [word2vec, GloVe, word-embeddings, NLP]

## Introduction

Words are an inseparable part of human lives. We communicate verbally, write letters and mail, and even think using words. We can do all of this because we have an understanding of the intrinsic meaning that each word has. Computers on the other hand are not good at understanding words and hence natural language. But they are good at processing large matrices/tensors of floating point values. Hence, humans have always tried to encode the words using numbers in some way that the computers can comprehend and work on challenging tasks. This encoding is usually called a word embedding. Its basically a vector that represents various features or characteristics of the word across its dimensions.
In this post, I intend to explain in detail the working of two techniques that have been used widely to calculate such word embeddings viz. word2vec and GloVe. There are multiple reasons to write this post.
1. Most of us know how to use these word embeddings in code and build complex architectures using them. However, we often ignore the details of the model and how these vectors were trained in the first place. It is equally important to understand and appreciate them.
2. For a long time I had the misconception that word2vec and GloVe are somewhat _similar_ with some tweaks here and there. But there are a lot of fundamental differences in their approaches to calculate word-vectors.

In this post I'll briefly explain word2vec and then move on to GloVe dive deep into the model details.

## Word2Vec
word2vec uses a _*pseudo*_ neural network to calculate continuous vector representations for words. Why pseudo? Because we don't actually use the outputs of the trained neural network. The neural network is trained to perform a pseudo or fake task. The task is formulated as done below.
- The network is trained using word pairs from large text corpus. For each word in a sentence, we generate word pairs by looking at a fixed number of words before and after the current word (or the input word). This fixed number of words is also known as the window size. This can be 2, 3, 4 or 5. Window size of 3 means that we look at 3 words before and 3 words after the current word $w_{t}$. Which means the context words are ($w_{t-3}$, $w_{t-2}$, $w_{t-1}$, $w_{t+1}$, $w_{t+2}$, $w_{t+3}$). The word-pairs hence generated would be [($w_{t}$,$w_{t-3}$), ($w_{t}$,$w_{t-2}$), ...  ($w_{t}$,$w_{t+3}$)]. The word-pairs generated are used to train the network. That is, $w_{t}$ is given as input to the model and the output is one of the words from the context.

- Assume that the vocabulary size of the corpus is 10,000. This means that there are 10,000 unique words in the corpus. The input to the network is a one-hot encoded vector representing the input word. The network has one hidden layer with 300 neurons. The output layer has 10,000 neurons, one for each word, with a softmax activation. No activation is used in the hidden layer. The presence of softmax means that the model will actually output probabilities for 10,000 words. __This probability is the probability of the word at that index being the nearbuy or a context word for the input/current word__. Intuitively, words that occur near the input word multiple times in the corpus will have a larger probability than others.
- For example, consider the word "Obama". It is more likely to be surrounded by words like "Barack", "USA", "President" etc. rather than words like "juice", "rabbit" etc. The network hence captures various features of the word statistically by looking at the local context of the word.
- Now coming back to the pseudo task. Once this network is trained for all the word pairs in the corpus, we simply remove the output layer. The hidden layer is associated with a matrix of size (10,000 X 300), one 300 dimensional vector for each word in the corpus. After backpropagation, the values of this matrix represent our word vectors. So the proposed task was fake because we never used the output layer. We just wanted to learn representations of the words.

The model discussed above is the skip gram model in which we are predicting the context given the input. The authors also proposed another model in the same paper called continuous bag-of-words (CBOW) model where we predict the input word from the context. A schematic diagram for both these models as given in the paper is shown below

### Subsampling

word2vec has been trained on a large corpus extracted from Google News that contains around 100 billion tokens. In such a large corpus, there are bound to exist very high frequency words that contribute very little to the training process. For exxample words like "the", "and", "a" etc. might occur in many context windows and hence be a part of many word-pairs. Thus the authors introduced subsampling that deletes words with high frequency from the corpus. Basically, each word is assigned a probability of whether it will be kept or dropped from the text. For example, if "the" is deleted from a sentence, it will generate fewer word-pairs for training and hence reduce training time.

### Negative Sampling

word2vec model has 300 X 10,000 weight values in total. For each training sample, all these weights will get tweaked very  slightly. This will happen for all the word-pairs generated in our text. That is, only for one word (ground truth) the output should be 1 and for the rest of the thousand words it should be 0. This would make the training very slow and is not even required since each training sample would not affect the a large fraction of weights significantly. Hence, for each training sample, we choose 5 negative words that we are not present in the input word's context. Weights are tweaked only for the label or the ground truth present in the word pair and these 5 negative words.

## Global Vectors (GloVe)

GloVe follows a more *principled* approach in calculating word-embeddings. The major difference between word2vec and GloVe is that the latter does not use a neural net for the task. The authors develop a strong mathematical model to learn the embeddings. GloVe also overcomes the drawbacks of previous techniques used to calculate word-embeddings. Before word2vec, statistical methods like Latent Semantic Analysis (LSA) were used to approximate embeddings for terms in a document. These methods took into account __global count statistics__ of the dataset. However, the vectors derived from such methods did not capture the meaning of the words like word2vec does and hence performed poorly in tasks like syntactic and semantic similarity and word analogies like "king - queen + woman = man". Meanwhile word2vec took into consideration the local context of words but failed to account for the global count satistics of the dataset. The main aim of GloVe was to combine the two approaches to learn word-vectors. Quoting from the paper,
>The statistics of word occurrences in a corpus is the primary source of information available to all unsupervised methods for learning word representations, and although many such methods now exist, the question still remains as to how meaning is generated from these statistics, and how the resulting word vectors might represent that meaning.

>Our model efﬁciently leverages statistical information by training only on the non-zero elements in a word-word co-occurrence matrix, rather than on the entire sparse matrix or on individual context windows in a large corpus.

As highlighted above, GloVe derives its training data by calculating a word-word co-occurence matrix. We'll show how this is calculated below with a toy example. But its important to understand the difference between this method and that used by word2vec. Word2vec generated training samples by forming word-pairs for all the words in a local context window completely ignoring the global statistics of the words that occur in the context window.

### Co-occurence matrix
Consider the sentence - "Winter is coming and it is here.". We construct a co-occurence matrix for this sentence with a context window size of 2. For "winter", the context words are "is" and "coming". Hence, we put 1 in the respective box. We also assume that the word itself is part of its context and hence increment the count in the diagonal of the matrix.

### Derivation
We'll now get into the derivation of the GloVe model. The following section contains equations taken from the paper directly. Although I'll try my best to explain the significance and meaning of each equation, the derivation can get a bit daunting. Let's start by defining some notations.
 - Let the word-word co-occurrence table be denoted by $X$. Each entry in the table $X_{ij}$ denotes the number of times word $j$ occurs in the context of word $i$.

 - Let $X_{i}$ be the number of times any word appears in the context of word $i$. That is the total number of distinct words throughout the corpus that appear in word $i$'s context. Therefore, $X_{i}$ = $\sum_{k}$$X_{ik}$ where $k$ are the different words that appear in $i$'s context throughout the dataset.

 - Let $P_{ij}$ = $P (j | i)$ = $X_{ij}$ / $X_{i}$. This defines the probability that word $j$ appears in the context of word $i$. $P_{ij}$ is also called as the co-occurrence probability.

 By calculating this probability, we are actually incorporating the global statistics of the dataset. Word2vec ignores the denominator of $P_{ij}$ calculated above.

Consider the following calculations for a particular dataset. The following example will help us understand how can we derive meaning of words by the ratio of probabilities calculated above.

The above matrix shows raw probabilities and ratios for two words viz. ice and steam. These words are topically concerned with the thermodynamic phases of water. The first 2 rows show calculations of the the probability $P_{ik}$ where $k$ are some words from the dataset called probe words. The last row shows the ratio of probabilities calculated in the first two rows.
The hypothesis is as follows:
> The relationship of these words can be examined by studying the ratio of their co-occurrence probabilities with various probe words, $k$.

Among the probe words, "solid" is closely related to the word "ice" (word $i$) and "gas" is closely related to "steam" (word $j$). As can be seen in the table above, the ratio $P_{ik}$/$P_{jk}$ is much greater than 1 (8.9) for words that are similar to "ice" ($i$) and much smaller than 1 (0.085) for words that are related to "steam" ($j$).
Moreover, for words that are equally related to both "ice" and "steam" like "water" and words that not related to any off the two words (like "fashion") have the ratio around 1.
Therefore, the hypothesis stated above is true in the sense that ratios of co-occurrence probabilities gives a better sense about the meaning of a word than raw probabilities.

- Therefore, this seems to be a good starting point for learning word vectors. We need to learn a function $F$, parameterized by 3 word vectors $w_{i}$, $w_{j}$ and $\tilde w_{k}$ such that,

$w_{i}$, $w_{j}$ and $\tilde w_{k}$ belong to $R^{ d}$. $\tilde w_{k}$ represent separate context words or the probe words as discussed above.
- There are a large number of possibilities for the function $F$. So lets impose some constraints on the model. The purpose of this model is to learn embeddings or feature vectors for words. Once learned, these vectors can be projected into a vector space just like we project 2 and 3 dimensional vectors in a cartesian plane. These vector spaces are inherently linear in nature because a vector is ultimately just a _line_.
The above discussion leads us to our first constraint. The most common way to compare two linear structures is to calculate their difference. Thus we can restrict our function as,

- In the above equation, the right hand side is a ratio which is scalar and the arguments are vectors. While, learning a complex function $F$ that does the above mapping is possible but that would introduce non-linearities in our model because that's what neural nets do. We don't wish to obscure the linear structure that our model captures. Therefore, we take a dot product of the arguments on the left hand side to make it a scalar.

- The next constraint is that of symmetry. The distinction between a word and the context words is arbitary and hence they can be exchanged with each other. This is easy to understand. If "water" and "gas" can be used as context words for "steam", then "steam" can also be used as a context word for "water" and "gas". This symmetry is also evident in our co-occurrence matrix $X$ which is symmetric about the diagonal. Symmetry implies that $w$ $<->$ $\tilde w$. To restore this symmetry, we require that the function $F$ is a homomorphism between groups $(R, +)$ and $(R_{>0}, *)$.

Let's define a group and homomorphism.
A group $(G, *)$ is set of elements, that is closed under a particular operation $*$. That is if $x$ and $y$ belong to $G$ then, $z$ = $x$ * $y$ also belongs to $G$. Each group has an inverse such that for all $x$,  $x^{-1}*x$ = $e$. Also, for all elements in a group, there exists an identity such that $x*e = e*x$ = $x$.
Function $F$ is a homomorphism between 2 groups $(G, *)$ and $(H, @)$ such that $F:G ->H$ if $F(x*y) = F(x) @ F(y)$.

Coming back to the equation of the model, homomorphism for $F$ can be explained as

\begin{equation}
F((w^{T}_{i} - w^{T}_{j})\tilde w_{k}) = F(w^{T}_{i}\tilde w_{k} + (-w^{T}_{j}\tilde w_{k}))
\end{equation}

\begin{equation}
 F(w^{T}_{i}\tilde w_{k} + (-w^{T}_{j}\tilde w_{k})) = F(w_{i}^{T} \tilde w_{k}) *  F(-w_{j}^{T} \tilde w_{k})
\end{equation}

\begin{equation}
 F(w_{i}^{T} \tilde w_{k}) *  F(-w_{j}^{T} \tilde w_{k}) = F(w_{i}^{T} \tilde w_{k}) *  F(w_{j}^{T} \tilde w_{k})^{-1}
\end{equation}

The above equation gives the following result,

The above equation is solved by equation $(3)$,

And the exponential function is the solution for equation $(4)$, therefore we have the following,

he symmetry is still not restored because of the $log(X_{i})$ term on the right hand side. Since, this term is independent of $k$, we can absorb it into bias $b_{i}$. Also introducing a bias $\tilde b_{k}$ for $\tilde w_{k}$, we finally have the following equation,

The above equation gives us a cost function for a new weighted least squares regression model. The weighting function $f(X_{ij})$ takes into consideration the co-occurrences of words.

## References and Acknowledgements
This post is heavily derived from the respective research papers of the techniques. Figures and equations are taken from the papers and blogs referenced below. Chris McCormick's posts on word2vec are still the best explanations for the topic and I strongly recommend the reader to read those.

1. https://nlp.stanford.edu/pubs/glove.pdf
2. https://arxiv.org/abs/1301.3781
3. https://papers.nips.cc/paper/5021-distributed-representations-of-words-and-phrases-and-their-compositionality.pdf
4. http://mccormickml.com/2016/04/19/word2vec-tutorial-the-skip-gram-model/
5. http://mccormickml.com/2017/01/11/word2vec-tutorial-part-2-negative-sampling/
6. https://math.stackexchange.com/questions/2580647/what-does-homomorphism-mean-in-the-glove-paper/2580648
7. https://datascience.stackexchange.com/questions/27042/glove-vector-representation-homomorphism-question
8. https://www.youtube.com/watch?v=g7L_r6zw4-c
9. https://towardsdatascience.com/word-embedding-part-ii-intuition-and-some-maths-to-understand-end-to-end-glove-model-9b08e6bf5c06

```python

```


# The Variational Quantum Eigensolver (VQE)

This notebook assumes users have installed:

- the Microsoft .NET Core SDK, IQ# and the qsharp Python package
- the openqemist Python package

A simple way to set up your environment is to use the docker image provided [here](https://github.com/1QB-Information-Technologies/openqemist/tree/master/docker_images). Otherwise, installation instructions can be found at https://github.com/1QB-Information-Technologies/openqemist.

## Table of Contents

1. [A brief overview of the VQE algorithm](#intro)
2. [Computing the ground–state energy of H$_{\text{2}}$ with UCCSD-VQE](#vqeuccsd)
    1. [The qsharp Python package](#qsharp)
    2. [Input data](#inputdata)
    3. [Qubit Hamiltonian, UCCSD ansatz, and initial variational parameters](#uccsd)
    4. [Energy evaluation using the Q# quantum algorithms](#energyeval)
    5. [Classical optimization](#optimization)
3. [Potential energy surface of H$_\text{2}$  with VQE, using the 1QBit OpenQEMIST package](#vqeopenqemist)

## 1. A brief overview of the VQE algorithm

The Variational Quantum Eigensolver (VQE) \[[Peruzzo_et_al.,_2014](https://arxiv.org/abs/1304.3061),  [McClean_et_al.,_2015](https://arxiv.org/abs/1509.04279)\] has been introduced as a hybrid quantum–classical algorithm for simulating quantum systems. Some examples of quantum simulation using VQE include solving the molecular electronic Schrödinger equation and model systems in condensed matter physics (e.g., Fermi– and Bose–Hubbard models). In this notebook, we focus on VQE within the context of solving the molecular electronic structure problem for the ground-state energy of a molecular system. The second-quantized Hamiltonian of such a system, within the Born-Oppenheimer approximation, assumes the following form:

\begin{equation}
\hat{H} = h_{\text{nuc}} + \sum_{p,q} h^{p}_{q} \hat{a}^{\dagger}_p \hat{a}_q + \sum_{p,q,r,s} h^{pq}_{rs} \hat{a}^{\dagger}_p \hat{a}^{\dagger}_q \hat{a}_s \hat{a}_r\nonumber
\end{equation}

Here, $h_{\text{nuc}}$ denotes the nuclear repulsion energy. The coefficients $h^{p}_{q}$ and $h^{pq}_{rs}$ are obtained by solving a mean-field problem. The Hamiltonian is then transformed into the qubit basis (e.g., Jordan–Wigner, Bravyi–Kitaev). This means that it is expressed entirely in terms of operators acting on qubits:

\begin{equation}
\hat{H} = h_{\text{nuc}} + \sum_{\substack{p \\ \alpha}} h_{p}^{\alpha} \sigma_p^{\alpha} + \sum_{\substack{p,q \\ \alpha,\beta}} h_{pq}^{\alpha\beta}\sigma_p^{\alpha}\otimes\sigma_{q}^{\beta} + \sum_{\substack{p,q,r \\ \alpha,\beta,\gamma}}h_{pqr}^{\alpha\beta\gamma}\sigma_p^{\alpha}\otimes\sigma_{q}^{\beta}\otimes\sigma_r^{\gamma} + \ldots \nonumber
\end{equation}

In this expression, the $\sigma_p^\alpha$ are Pauli matrices ($\alpha \in \{x,y,z\}$), acting on the $p$-th qubit. We now consider a trial wavefunction ansatz $\vert \Psi(\vec{\theta}) \rangle = U(\vec{\theta}) \vert 0 \rangle$ that depends on $m$ parameters defining $\vec{\theta}=(\theta_1, \theta_2, \ldots, \theta_m)$, which enter a unitary operator that acts on the reference (i.e., mean-field) state $\vert 0 \rangle$. The variational principle dictates that we can minimize the expectation value of the Hamiltonian,

\begin{equation}
E = \min_{\vec{\theta}} \frac{\langle \Psi(\vec{\theta}) \vert \hat{H} \vert \Psi(\vec{\theta}) \rangle}{\langle \Psi(\vec{\theta}) \vert \Psi(\vec{\theta}) \rangle} \geq E_{\text{gs}}\nonumber
\end{equation}

to determine the optimal set of variational parameters. The energy thus computed will be an upper bound to the true ground-state energy $E_{\text{gs}}$. Once a suitable variational trial ansatz has been chosen (e.g., a unitary coupled-cluster ansatz, a heuristic ansatz), we must provide a suitable set of initial guess parameters. If our ansatz is written in according to the second quantization picture, we must also transform it into the qubit basis before proceeding. We must also apply other approximations (e.g., Trotter–Suzuki) to render it amenable for translation into a quantum circuit. The resulting qubit form of the ansatz can then be translated into a quantum circuit and, thus, able to be implemented on quantum hardware. Once the initial state has been prepared using a quantum circuit, energy measurements are performed using quantum hardware or an appropriate simulation tool. The energy value obtained is the sum of the measurements of the expectation values of each of the terms that contribute to the Hamiltonian (assuming the wavefunction has been normalized to unity):

\begin{equation}
E = \langle \Psi(\vec{\theta}) \vert \hat{H} \vert \Psi(\vec{\theta}) \rangle =\langle\hat{H}\rangle = h_{\text{nuc}} + \sum_{\substack{p \\ \alpha}} h_{p}^{\alpha} \langle\sigma_p^{\alpha}\rangle + \sum_{\substack{p,q \\ \alpha,\beta}} h_{pq}^{\alpha\beta}\langle\sigma_p^{\alpha}\otimes\sigma_{q}^{\beta}\rangle + \sum_{\substack{p,q,r \\ \alpha,\beta,\gamma}}h_{pqr}^{\alpha\beta\gamma}\langle\sigma_p^{\alpha}\otimes\sigma_{q}^{\beta}\otimes\sigma_r^{\gamma}\rangle + \ldots \nonumber
\end{equation}

The computed energy is then input to a classical optimizer in order to find a new set of variational parameters, which are then used to prepare a new state (i.e., a quantum circuit) on the quantum hardware. The process is repeated until convergence. The algorithm is illustrated below.

## 2. Computing the ground–state energy of H$_{\text{2}}$ with UCCSD-VQE

The **Microsoft Quantum Development Kit (QDK)** provides a way to simulate quantum circuits on classical hardware and quantum processors. It uses the Microsoft Q# language, which was developed specifically to handle hybrid quantum–classical workflows.

The [**Quantum Development Kit chemistry library**](https://docs.microsoft.com/quantum/libraries/chemistry/) provides key functionalities for tackling problems in quantum chemistry. It is written in C#, and relies on Q# operations to implement various quantum algorithms. This is an open source [GitHub repository](https://github.com/Microsoft/QuantumLibraries) that accepts suggestions and contributions.

Although users are able to write and call their own code in Q# and C#, this is not a requirement. This notebook uses Python exclusively. All the functionality needed to execute the example that follows can be accessed through the Quantum Development Kit Python interoperability package for python, **qsharp**, available on pip. Further details about this pip package are available at https://docs.microsoft.com/quantum/install-guide/python .

This section shows how these functionalities can be used to compute the ground state energy of H$_{\mathrm{2}}$ (the simplest molecule) in a minimal basis set, using the unitary coupled-cluster ansatz with single and double excitations (UCCSD) and compare the results with the exact results obtained in this basis. A ball-stick model for H$_\text{2}$ is shown below. The distance between the two hydrogen atoms is called the bond length, and its value is set to approximately 0.7414$~$Å in this section.

### 2-A The qsharp Python package

The cell below prepares the Q# environment and loads the useful functionalities of the chemistry library through `qsharp.chemistry`. This notebook later details how each of these play a role in this implementation of VQE.

```python
from qsharp.chemistry import load_broombridge, load_fermion_hamiltonian, load_input_state, encode
```

### 2-B Input data

Users need to provide quantities defining the target molecular system, such as the following:

- one- and two-electron integrals
- nuclear repulsion energy

The use of VQE requires to specify extra input, such as the following:

- the type of ansatz desired (UCCSD, for example)
- the values for initial variational parameters
- an initial state (a reference wavefunction, such as the Hartree–Fock wavefunction)

The [**Broombridge format**](https://docs.microsoft.com/quantum/libraries/chemistry/schema/broombridge), created by Microsoft and PNNL, provides a way to store all the input information in a human-readable .yaml file. Loading a pre-existing Broombridge file containing the information of interest for the target molecular system is the shortest way to get started with running VQE. The `qsharp.chemistry` package allows Python packages to load and work with the data stored in Broombridge files.

The following code snippet shows how to load existing data from a Broombridge file (here for H$_\text{2}$ at a bond length of 0.7414), and explores the resulting data structure.

```python
# C# Chemistry library :: Loading molecular data (electronic integrals, etc.) from Broombridge
filename = 'data/hydrogen_0.2.yaml'
broombridge_data =  load_broombridge(filename)
```

The data structure is easier to navigate when using a pretty-print application or a proper IDE.

It is worth mentioning that users do not need a Broombridge file describing the molecular system of interest in order to get started. They could, for example, compute and provide their own data at runtime using third-party libraries such as PySCF, and then be free to extract and overwrite the information in the data structures produced by reading any Broombridge file.

The instructions below show how users can read information stored in a data structure (writing to the data structure is just as straightforward).

```python
# Retrieve basis set and geometry used to generate the input data
basis_set = broombridge_data.problem_description[0].basis_set
geometry = broombridge_data.problem_description[0].geometry

# Retrieve the nuclear repulsion and the one-electron integrals (Mulliken convention)
nuclear_repulsion = broombridge_data.problem_description[0].coulomb_repulsion['Value']
one_electron_integrals =  broombridge_data.problem_description[0].hamiltonian['OneElectronIntegrals']['Values']

print("nuclear_repulsion = ", nuclear_repulsion)
print("one_electron_integrals = ", one_electron_integrals)
```

Note that users who are directly writing to the data structures should be aware that the Python interop relies on JSON serialization, and should use fundamental data types. They should make sure to pass lists instead of NumPy arrays, or to cast their integer and floating point values with the built-in **int** and **float** Python functions to avoid JSON serialization errors at runtime.

### 2-C Qubit Hamiltonian, UCCSD ansatz, and initial variational parameters

The following section shows how to prepare the qubit Hamiltonian (also referred to as the Pauli Hamiltonian) and access the information related to one of the available ansatz for VQE: UCCSD.

Note that the underlying data structures may change in the future. The code cells below encourage users to print their content by directly accessing the available fields, exposed by the `dir` built-in Python function.

The fermionic Hamiltonian can be built using the chemistry library, and is returned to the Python context:

```python
ferm_hamiltonian = broombridge_data.problem_description[0].load_fermion_hamiltonian()
print("ferm_hamiltonian ::", ferm_hamiltonian)
print(dir(ferm_hamiltonian))
```

A Broombridge file can contain suggestions of initial states to use to carry electronic computations of a molecule. In particular, they can be used by the UCCSD ansatz to store information about the initial state (i.e., the reference wavefunction) as well as initial values for the variational parameters and the spin-orbital excitations to whic they correspond.

Several initial states can be available and stored in a Broombridge file as a result of classical computations from libraries such as NWChem, for example. The user can specify which initial state to load with the following code snippet:

```python
input_state = load_input_state(filename, "UCCSD |G>")
print("input_state ::", input_state)
print(dir(input_state))
```

Users can decide what excitations should be included in the ansatz and how the values of variational parameters can be tied to specific excitations, or enforce that a unique value should be tied to several terms during the classical optimization later. The last entry in `inputstate[Superposition]` is the initial state, here showing a Hartree-Fock state, with the two lower orbitals filled with one electron each.

The chemistry library can now build the qubit Hamiltonian with a transformation such as the Jordan–Wigner transformation.

```python
jw_hamiltonian = encode(ferm_hamiltonian, input_state)
print("jw_hamiltonian :: \n", jw_hamiltonian)
```

Please note that, currently, the underlying `JordanWignerEncodingData` data structure from the chemistry library is also used to store the initial state for UCCSD as well as the variational parameters representing the one- and two-body amplitudes (specified as the third entry of the resulting `jw_hamiltonian` tuple object). In the future, the objects may be kept separate and thus the `inputState` field may not be required to compute the qubit Hamiltonian. Users can, however, retrieve the values of the variational parameters directly from the data structure, with a function such as the following:

```python
def get_var_params(jw_hamiltonian):
    """ Retrieve the values of variational parameters from the jw_hamiltonian object """
    _, _, input_state, _ = jw_hamiltonian
    _, var_params = input_state
    params = [param for ((param, _), _) in var_params]
    return params[:-1]

var_params = get_var_params(jw_hamiltonian)
print(var_params)
```

### 2-D Energy evaluation using the Q# quantum algorithms

The qsharp package can be used to directly call quantum algorithms written in Q#. These can be user defined, or come from one of the available Q# libraries.

The energy is computed as an expectation value $E(\theta) = \langle \Psi(\vec{\theta}) \vert \hat{H} \vert \Psi(\vec{\theta}) \rangle =\langle\hat{H}\rangle$, which can be estimated by drawing many samples of the underlying distribution (e.g., running the quantum circuit and measuring for each sample). This approach is the one used on  quantum hardware, and relies on sampling to approach the expectation value, using the `simulate` function. The accuracy of the expectation value, and therefore the result of the energy evaluation, directly correlates with the number of samples used. The fast frequency estimator provided in the QDK allows for the approximation of the result for a very large number of samples without incurring longer runtimes.

```python
import qsharp

# It is possible to create a Python object to represent a
# Q# callable from the chemistry library
estimate_energy = qsharp.QSharpCallable("Microsoft.Quantum.Chemistry.JordanWigner.VQE.EstimateEnergy", "")

# The Q# operation can then be called through the simulate method
# A large number of samples is selected for high accuracy
energy = estimate_energy.simulate(jwHamiltonian=jw_hamiltonian, nSamples=1e18)

print("Energy evaluated at {0} : {1} \n".format(var_params, energy))
```

### 2-E Classical optimization

VQE is a quantum–classical hybrid algorithm that aims to compute $E = \min_{\vec{\theta}} \: \langle \Psi(\vec{\theta}) \vert \hat{H} \vert \Psi(\vec{\theta}) \rangle$. This approach relies on solving an optimization problem, using a classical optimizer to tune the values of the variational parameters $\{\theta_i\}_{i=1}^{m}$.

There are several Python libraries that provide implementations of optimizers based on different heuristics, and SciPy is one that is widely used. The optimizers in `scipy.optimize` have a common interface that require users to provide  the following:

- A handle to a Python function to perform energy evaluations. It takes the variational parameters as its first input, leaving other parameters that are to be left out of the optimization process afterwards
- Values for the initial parameters
- Optional parameters used by our energy evaluation function, that should not be optimized
- Optional parameters defining the behaviour and termination criteria for the chosen optimizer

The first item requires the user to provide a Python wrapper (here named `energy_eval_wrapper`) with the expected signature, in order to call the `energy_evaluation` operation available in the Quantum Development Kit chemistry library. This wrapper requires the variational parameters to be passed as a list or a NumPy array and, currently, an extra step is needed to modify the data structure passed to the Q# context in order to use the correct values (defined in `set_var_params` below).

```python
def set_var_params(var_params, jw_hamiltonian):
    """ Set variational parameters stored in the JW data-structure to the desired values"""
    # Unpack data structure
    a1, a2, input_state, a3 = jw_hamiltonian
    b1, amps = input_state
    # Unpack and overwrite variational parameters
    new_amps = [((var_params[i], 0.0), amps[i][1]) for i in range(len(var_params))]
    new_amps.append(amps[-1])
    # Re-pack the data structure
    input_state = (b1, new_amps)
    jw_hamiltonian = (a1, a2, input_state, a3)
    return jw_hamiltonian
```

```python
def energy_eval_wrapper(var_params, jw_hamiltonian, n_samples):
    """
        A wrapper whose signature is compatible with the use of scipy optimizers,
        calling the Q# energy_evalaution from the Microsoft Chemistry library
    """

    # NumPy arrays are currently not supported by the Python interops
    # This ensures that neither the user nor SciPy call the energy evaluation function with a NumPy array
    var_params = list(var_params)

    # Set the varational parameters to the right values in the jw_hamiltonian object
    jw_hamiltonian = set_var_params(var_params, jw_hamiltonian)

    # Estimate the energy
    energy = estimate_energy.simulate(jwHamiltonian=jw_hamiltonian, nSamples=1e18)

    print("Energy evaluated at {0} : {1} \n".format(var_params, energy))
    return energy
```

These two functions can then be used to run VQE. For simplicity, a specific optimizer from the SciPy library is used, with given hyperparameters such as tolerance or step size. Since accuracy of energy evaluation is correlated to the number of samples drawn, it is important to set it to a number large enough to guarantee that it is consistent with the optimizers convergence criteria, and to ensure the correct approximation of derivatives used by some optimizers. Setting a very large number of samples would solve this issue.

```python
from scipy.optimize import minimize

def VQE(initial_var_params, jw_hamiltonian, n_samples):
    """ Run VQE Optimization to find the optimal energy and the associated variational parameters """

    opt_result = minimize(energy_eval_wrapper,
                          initial_var_params,
                          args=(jw_hamiltonian, n_samples),
                          method="COBYLA",
                          tol=0.000001,
                          options={'disp': True, 'maxiter': 200,'rhobeg' : 0.05})

    return opt_result
```

```python
# Run VQE and print the results of the optimization process
# A large number of samples is selected for higher accuracy
opt_result = VQE(var_params, jw_hamiltonian, n_samples=1e18)
print(opt_result)
```

```python
# Print difference with exact FCI value known for this bond length
fci_value = -1.1372704220924401
print("Difference with exact FCI value :: ", abs(opt_result.fun - fci_value))
```

## 3 Potential energy surface of H$_\text{2}$  with VQE, using the 1QBit OpenQEMIST package

The potential energy surface of this molecule can be obtained by plotting the energy of the system as a function of the distance between the hydrogen atoms.

This section shows how the 1QBit OpenQEMIST package allows users to run VQE without relying on an input Broombridge file, or worrying about modifying the data structures returned by the Python interop in the previous section. Users can directly provide the geometry and basis set of the target molecular system: OpenQEMIST computes the mean field and electronic integrals using PySCF, generates the UCCSD one- and two-body excitations, and provides good initial variational parameters using MP2 amplitudes.

OpenQEMIST provides several electronic structure solvers, such as VQE, FCI, and CCSD. This package can be used to compute the H$_\text{2}$ bond dissociation curve using VQE, with Microsoft libraries, and compare it to the exact FCI values, computed on-the-fly. Running the code cells in this section should yield a plot that closely resembles the one below:

```python
# Import the OpenQEMIST package from 1QBit and PySCF
import openqemist
import pyscf
import numpy as np
```

```python
from pyscf import gto, scf
from openqemist.electronic_structure_solvers import VQESolver, FCISolver
from openqemist.quantum_solvers.parametric_quantum_solver import ParametricQuantumSolver
from openqemist.quantum_solvers import MicrosoftQSharpParametricSolver

# Iterate over different bond lengths
bond_lengths = np.arange(0.4, 1.7, 0.1)
energies_FCI, energies_VQE = [], []

for bond_length in bond_lengths:

    # Create molecule object with PySCF
    H2 = [['H',[ 0, 0, 0]], ['H',[0,0, bond_length]]]
    mol = gto.Mole()
    mol.atom = H2
    mol.basis = "sto-3g"
    mol.charge = 0
    mol.spin = 0
    mol.build()

    # Compute FCI energy with PySCF, for reference
    solver = FCISolver()
    energy = solver.simulate(mol)
    energies_FCI += [energy]

    # Compute energy with VQE, instantiating a VQESolver object using the UCCSD ansatz
    solver = VQESolver()
    solver.hardware_backend_type = MicrosoftQSharpParametricSolver
    solver.ansatz_type = MicrosoftQSharpParametricSolver.Ansatze.UCCSD
    energy = solver.simulate(mol)
    energies_VQE += [energy]
```

```python
import matplotlib.pyplot as plt
%matplotlib inline

plt.plot(bond_lengths, energies_FCI, color = 'black', label='Full CI')
plt.plot(bond_lengths, energies_VQE, 'ro', label='UCCSD-VQE')
plt.title("Potential energy surface of H2")
plt.xlabel("Distance between hydrogen atoms (angstroms)")
plt.ylabel("Energy (hartrees)")
plt.legend()
```

Copyright 1QBit 2019. This software is released under the Apache Software License version 2.0.


```
import numpy

#Parche porque las versiones de Numpy y Keras son parcialemente incompatibles
#Numpy cambió allow_pickle a False y eso hace que falle.
#En la nueva versión de keras el problema está solucionado, pero no viene por defecto en colab.
old_load = numpy.load
def load(file, mmap_mode=None, allow_pickle=True, fix_imports=True, encoding='ASCII'):
  return old_load(file, mmap_mode, allow_pickle, fix_imports, encoding)

numpy.load = load
```

# Trabajando con lenguaje natural

El lenguaje natural presenta muchos desafios a la hora de ser procesados por técnicas de Machine Learning, incluidas las de Deep Learning. En partícular, las características más importantes son:

* Si bien existen muchas palabras, un texto generalmente utiliza solo una pequeña cantidad de ellas.
* El orden de las palabras hacen al sentido del texto, las mismas palabras en distinto orden pueden tener distintos significados.
* El largo del texto no es fijo, es decir distintos textos pueden tener distinta cantidad de palabras.

Tradicionalmente, una de las formas de representar texto para machine learning es utilizar "bag-of-words", o alguna variación que cuente la frecuencia de las palabras en el texto y corpus, como [TF-IDF](https://en.wikipedia.org/wiki/Tf%E2%80%93idf), ver documentación de [sk-learn](http://scikit-learn.org/stable/modules/feature_extraction.html#text-feature-extraction). Estas técnicas representan el texto como un vector donde cada elemento es una palabra del vocabulario, y su valor representa el peso de esa palabra en el texto.

$$ Vector=(pal_1, pal_2, ..., pal_m) $$

Por ejemplo, una representación con pesado binario, es decir, el elemento del vector vale 1 si la palabra está en el texto y 0 sino. Si asumimos un vocabulario de 100 palabras y el texto "Hola, mi nombre es Pedro", la representación del texto tendrá cinco unos y 95 ceros.

## Large Movie Review Dataset

[Large Movie Review Dataset](https://ai.stanford.edu/~amaas/data/sentiment/) es un dataset de reviewes de películas extraido de IMDB. El objetivo es determinar si el review es positivo o negativo.

Keras lo provee preprocesado, donde todas las palabras han sido asignadas a un número.

```
from itertools import chain
from keras.datasets import imdb
(x_train, y_train), (x_test, y_test) = imdb.load_data()

print('Instancias de entrenamiento: {}'.format(len(x_train)))
print('Ejemplo de una instancia: {}'.format(x_train[0]))

words = set(chain.from_iterable(x_train))
print('Palabras en el vocabulario: {}'.format(len(words)))

overlap = [len(set(x))/len(words) * 100 for x in x_train]
print('En promedio, una review utiliza {} palabras'.format(numpy.average([len(set(x)) for x in x_train])))
print('En promedio, una review tiene el {}% del vocabulario'.format(numpy.average(overlap)))
print('La review que menos palabras tiene utiliza el {}% del vocabulario'.format(numpy.min(overlap)))
print('La review que más palabras tiene utiliza el {}% del vocabulario'.format(numpy.max(overlap)))
```

## Ejemplo clásico

Utilizando una representación binaria, podemos implementar una regresión lógistica para clasificar si un review es positivo o no.

Debido a que keras tiene limitado soporte de matrices sparse, los requerimientos de memoria hacen difícil procesar todo el dataset ya que se requieren 25000 * 88585 * 32 bits de memoria (8 Gigas aproximadamente). Por este motivo, no procesaremos todo el dataset sino que utilizaremos un generador. Un generador permite procesar datos a medida que se necesitan.

A continuación se muestra un ejemplo de un generador para la serie de Fibonacci. Cada vez que se ejecuta la instrucción yield, la función queda suspendida retornando el valor indicado. Cuando se pide un siguiente valor, la función continúa ejecutando desde el punto que se detuvo.

```
def fib(x):
    a, b = 1, 1
    i = 1
    while i <= x:
        yield a
        a, b = b, a + b
        i += 1
    return

for i in fib(10):
    print(i)
```

```
from keras.models import Model
from keras.layers import Dense, Input
(x_train, y_train), (x_test, y_test) = imdb.load_data()

#Mapeando palabras
words = list(words)
words.sort()
words = {w: i for i, w in enumerate(words)}

#Generador infinito que cicla por todos los elementos
def generator(x, y, batch=128):
    #Llevo un contador de posicion
    i = 0
    while(True):
        #Genero los x e y del batch actual
        res = numpy.zeros((batch, len(words)))
        res_i = numpy.zeros((batch,))
        #Cuento cuantos elementos voy generando
        count = 0
        while(True):
            #Genero la representación para la review i
            s = x[i]
            for w in set(s):
                if w in words:
                    res[count, words[w]] = 1
            res_i[count] = y[i]
            #Incremento i y el countador de batch en 1. Si i es mayor que
            #la cantidad de elementos, retorno a zero
            i = (i + 1) % len(x)
            count += 1
            #si llegue a la cantidad maxima
            #hago el yield del estado actual
            if count == batch:
                yield res, res_i
                break
    return

print('Representación de las dos primeras instancias de entrenamiento')
x, y = next(generator(x_train, y_train))
print(x[0, :].tolist())
print(x[1, :].tolist())

#Regresión logística
i = Input((len(words),))
o = Dense(1, activation='sigmoid')(i)

model = Model(inputs=i, outputs=o)

model.compile(loss='binary_crossentropy',
              optimizer='adam',
              metrics=['accuracy'])

#Entrenando con generador
print('Entrenando...')
model.fit_generator(generator(x_train, y_train),
          steps_per_epoch=int(len(x_train)/128 + 1),
          epochs=1)
print('Evaluando...')
score, acc = model.evaluate_generator(generator(x_test, y_test),
          steps=int(len(x_test)/128 + 1))
print('Test score:', score)
print('Test accuracy:', acc)
```

## Word Embeddings

Word embeddings es una técnica donde a cada palabra se le asigna un vector de características. Este vector tiene por objetivo representar la semantica de la palabra. Al utilizar esta técnica, un texto queda representado por una matriz de tamaño cantidad de palabras por cantidad de características.

Ejemplo, supongamos un lenguaje con 10 palabras, y un texto que contenga las palabras ```[1, 3, 5, 5] ```. Si utilizamos un embedding de 20 características, el texto quedará representado como una matriz de 4x20.

```
from keras.layers import Input, Embedding
from keras.models import Model

text = [1, 3, 5, 5]

print('El texto es {}'.format(text))
#Le agrego una dimensión más que representa todas las instancias
#ya que Keras espera intancias x características
x_input = numpy.asarray([text])

#None indica que no se cuantas palabras tiene el texto
i = Input((None,))
o = Embedding(10, 20)(i)
#Este modelo retorna la matriz que representa al texto
model = Model(inputs=i, outputs=o)

model.compile(loss='binary_crossentropy',
              optimizer='adam',
              metrics=['accuracy'])

print('Ejemplo de como se representa el texto')
print(model.predict(x_input)[0, :, :])

```

Obviamente, el embedding está sin entrenar, por lo que el resultado es de poca utilidad. En la práctica se puede utilizar modelos preentrenados como los disponibles para:

* Word2Vec
* FastText
* GloVe

O entrenar el embedding ad-hoc. En nuestro caso entrenaremos el embedding.

Una técnica muy popular para clasificar texto, es utilizar CNN. En vez de utilizar las 2D como en el caso de las imagenes, se utiliza las CNN 1D que consideran número de palabras X caractéristicas.

Para facilitar el entrenamiento:

* Haremos un padding del texto para que todos tengan el mismo largo. El padding es un proceso que ajusta la longitud de las oraciones. Si la oración es más larga que el maxímo largo, solo considera los últimos elementos. Si la oración es más corta, añade ceros al principio.
* El elemento 0 está reservado, por lo que el indice de palabras comienza en 1.

Una vez pasado por la CNN1D, utilizaremos una capa GlobalMaxPooling, que para cada uno de los filtros se queda con el máximo valor. Finalmente utilizaremos una capa densa para la predicción. Además, utilizaremos capas dropout que arbitrariamente cambian valores de los resultados por ceros durante el entrenamiento. Estas capas se utilizan para mejorar la calidad del entrenamiento ya que introducen ruido.

Basado en los ejemplos presentados en el [Github de Keras](https://github.com/keras-team/keras)

```
from keras.preprocessing import sequence
from keras.models import Model
from keras.layers import Dense, Dropout, Input
from keras.layers import Embedding
from keras.layers import Conv1D, GlobalMaxPooling1D
from keras.datasets import imdb

# set parameters:
max_features = 88585
maxlen = 400
batch_size = 32
embedding_dims = 50
filters = 250
kernel_size = 3
hidden_dims = 250
epochs = 5

(x_train, y_train), (x_test, y_test) = imdb.load_data(num_words=max_features)

print('Pad sequences (ejemplos x largo)')
x_train = sequence.pad_sequences(x_train, maxlen=maxlen)
x_test = sequence.pad_sequences(x_test, maxlen=maxlen)
print('x_train shape:', x_train.shape)
print('x_test shape:', x_test.shape)

print('Creando modelo...')
i = Input((maxlen,))
o = Embedding(max_features, embedding_dims)(i)
o = Dropout(0.2)(o)
o = Conv1D(filters,
                 kernel_size,
                 padding='valid',
                 activation='relu',
                 strides=1)(o)
o = GlobalMaxPooling1D()(o)
o = Dense(hidden_dims, activation='relu')(o)
o = Dropout(0.2)(o)
o = Dense(1, activation='sigmoid')(o)

model = Model(inputs=i, outputs=o)

model.compile(loss='binary_crossentropy',
              optimizer='adam',
              metrics=['accuracy'])

model.summary()
model.fit(x_train, y_train,
          batch_size=batch_size,
          epochs=epochs,
          validation_data=(x_test, y_test))

score, acc = model.evaluate(x_test, y_test)
print('Test score:', score)
print('Test accuracy:', acc)
```

## Redes LSTM

Otra opción es utilizar redes de tipo recurrentes, como las LSTM. Estas redes suelen ser utilizadas en texto ya que consideran el orden de los elementos y propagan el calculo de manera secuencias. Esto en principio es una "ventaja" con respecto a las convolucionales que solo consideran localidad.

> Red Neuronal Recurrente. Imagen: [Wikipedia](https://en.wikipedia.org/wiki/Recurrent_neural_network) <br>

En particular, las LSTM tiene una estructura que no solo consideran la salida de la última capa, sino que además el estado interno.

>Capa LSTM del artículo [LSTM de Wipipedia](https://upload.wikimedia.org/wikipedia/commons/thumb/3/3b/The_LSTM_cell.png/1920px-The_LSTM_cell.png)

$$
\begin{align}
f_t &= \sigma_g(W_{f} x_t + U_{f} h_{t-1} + b_f) \\
i_t &= \sigma_g(W_{i} x_t + U_{i} h_{t-1} + b_i) \\
o_t &= \sigma_g(W_{o} x_t + U_{o} h_{t-1} + b_o) \\
c_t &= f_t \circ c_{t-1} + i_t \circ \sigma_c(W_{c} x_t + U_{c} h_{t-1} + b_c) \\
h_t &= o_t \circ \sigma_h(c_t)
\end{align}
$$

```
from keras.preprocessing import sequence
from keras.models import Model
from keras.layers import Dense, Embedding, Input
from keras.layers import LSTM
from keras.datasets import imdb

max_features = 88585
# cut texts after this number of words (among top max_features most common words)
maxlen = 80
batch_size = 32

(x_train, y_train), (x_test, y_test) = imdb.load_data(num_words=max_features)

print('Pad sequences (ejemplos x largo)')
x_train = sequence.pad_sequences(x_train, maxlen=maxlen)
x_test = sequence.pad_sequences(x_test, maxlen=maxlen)
print('x_train shape:', x_train.shape)
print('x_test shape:', x_test.shape)

print('Creando modelo...')

i = Input((maxlen,))
o = Embedding(max_features, embedding_dims)(i)
o = LSTM(embedding_dims, dropout=0.2, recurrent_dropout=0.2)(o)
o = Dense(1, activation='sigmoid')(o)

model = Model(inputs=i, outputs=o)

model.compile(loss='binary_crossentropy',
              optimizer='adam',
              metrics=['accuracy'])

model.summary()
model.fit(x_train, y_train,
          batch_size=batch_size,
          epochs=1,
          validation_data=(x_test, y_test))
score, acc = model.evaluate(x_test, y_test,
                            batch_size=batch_size)
print('Test score:', score)
print('Test accuracy:', acc)
```

## Consideraciones

1. Las CNN son más rápidas pero no consideran el orden de las palabras, solo su localidad.
1. Las LSTM consideran el orden, pero cuando las secuencias son largas tiende a no darle la misma importancia al comienzo de la oración.
1. Las LSTM son lentas de entrenar ya que tienen un componente secuencial que no puede aprovechar el soporte de computación paralela ofrecida por las GPUs y TPUs.
1. El ejemplo es muy sencillo y no se puede apreciar las ventajas de un modelo sobre el otro.

## Otro ejemplo de LSTM: Generación de texto

En este caso, el objetivo es generar texto que "parezca" ingles. Para esto, utilizaremos textos de Nietzsche en ingles. Para esto:

1. Dado una secuencia de carácteres (40 en este ejemplo), cual es el siguiente. Esto puede verse como clasificación, donde la clase es el siguiente carácter.
1. Los carácteres se representan utilizando one-hot. Es decir, un vector de largo cantidad de carácteres, donde todos los valores son cero, salvo el elemento que representa al carácter que es uno.
1. Para la generación se utiliza muestreo y no la clase de mayor probabilidad (esta última opción no funciona bien).

```
from keras.callbacks import LambdaCallback
from keras.models import Sequential
from keras.layers import Dense
from keras.layers import LSTM
from keras.optimizers import RMSprop
from keras.utils.data_utils import get_file
import numpy as np
import random
import sys
import io

path = get_file(
    'nietzsche.txt',
    origin='https://s3.amazonaws.com/text-datasets/nietzsche.txt')
with io.open(path, encoding='utf-8') as f:
    text = f.read().lower()
print('corpus length:', len(text))

chars = sorted(list(set(text)))
print('total chars:', len(chars))
char_indices = dict((c, i) for i, c in enumerate(chars))
indices_char = dict((i, c) for i, c in enumerate(chars))

# cut the text in semi-redundant sequences of maxlen characters
maxlen = 40
step = 3
sentences = []
next_chars = []
for i in range(0, len(text) - maxlen, step):
    sentences.append(text[i: i + maxlen])
    next_chars.append(text[i + maxlen])
print('nb sequences:', len(sentences))

print('Vectorization...')
x = np.zeros((len(sentences), maxlen, len(chars)), dtype=np.bool)
y = np.zeros((len(sentences), len(chars)), dtype=np.bool)
for i, sentence in enumerate(sentences):
    for t, char in enumerate(sentence):
        x[i, t, char_indices[char]] = 1
    y[i, char_indices[next_chars[i]]] = 1

print('Ejemplo de la representación:')
print('Primera oración: ', sentences[0])
print('Siguente carácter: ')
print(next_chars[0])
print('Mapa de indices: ', char_indices)
print('Representación oración: ')
print('[')
for s in x[0, :, :].astype(numpy.int8).tolist():
    print(s)
print(']')
print('Representación clase: ', y[0, :].astype(numpy.int8))

# build the model: a single LSTM
print('Construyendo modelo...')
model = Sequential()
model.add(LSTM(128, input_shape=(maxlen, len(chars))))
model.add(Dense(len(chars), activation='softmax'))

optimizer = RMSprop(lr=0.01)
model.compile(loss='categorical_crossentropy', optimizer=optimizer)

#Sample aplicando una temperatura que hace que el resultado
#sea más o menos aleatoreo
def sample(preds, temperature=1.0):
    # helper function to sample an index from a probability array
    preds = np.asarray(preds).astype('float64')
    preds = np.log(preds) / temperature
    exp_preds = np.exp(preds)
    preds = exp_preds / np.sum(exp_preds)
    probas = np.random.multinomial(1, preds, 1)
    return np.argmax(probas)

def on_epoch_end(epoch, _):
    # Function invoked at end of each epoch. Prints generated text.
    print()
    print('----- Generating text after Epoch: %d' % epoch)

    start_index = random.randint(0, len(text) - maxlen - 1)
    for diversity in [0.2, 0.5, 1]:
        print('----- diversity:', diversity)

        generated = ''
        #Texto random para iniciar la predicción
        sentence = text[start_index: start_index + maxlen]
        generated += sentence
        print('----- Generating with seed: "' + sentence + '"')
        sys.stdout.write(generated)
        #Predice 100 carácteres
        for i in range(100):
            #Fromatea la entrada a la red
            x_pred = np.zeros((1, maxlen, len(chars)))
            for t, char in enumerate(sentence):
                x_pred[0, t, char_indices[char]] = 1.
            #Predice las probabilidades del siguiente carácter y samplea
            preds = model.predict(x_pred, verbose=0)[0]
            next_index = sample(preds, diversity)
            next_char = indices_char[next_index]
            #Elimina el primer carácter de la sentencia raiz y agrega el
            #carácter predicho al final
            sentence = sentence[1:] + next_char

            #Imprime el resultado.
            sys.stdout.write(next_char)
            sys.stdout.flush()
        print()

print_callback = LambdaCallback(on_epoch_end=on_epoch_end)
print('Generando sin entrenamiento')
on_epoch_end(0, None)

print('Entrenando y generando')
model.fit(x, y,
          batch_size=128,
          epochs=2,
          callbacks=[print_callback])
```

## Sunspring

Utilizaron está técnica para generar [Sunspring](https://www.youtube.com/watch?v=LY7x2Ihqjmc), un corto de SciFi cuyo libreto fue generado por una red neuronal.

# Autoencoders

Un autoencoder son dos redes neuronales, una cuyo objetivo es reducir la representación de las instancias a un espacio dimensional más bajo. Por ejemplo, en el ejemplo de la MNIST existen 784 caractéristicas y se podría desear reducirlas a un número menor. Esto se llama reducción de dimensionalidad y tiene diversas aplicaciones, como entrenar modelos que no se comportan bien cuando hay muchas caractéristicas, eliminar caractéristicas redundantes, o reducir el nivel de ruido. Un ejemplo de utilización de autoencoders puede ser para comprimir imagenes, de hecho se ha probado que son competitivos cuando se comparan con estandares de la industria como JPEG2000 [1]. La arquitectura de un autoencoder es:

> [Autoencoder](https://en.wikipedia.org/wiki/Autoencoder)

[1] Theis, L., Shi, W., Cunningham, A., & Huszár, F. (2017). Lossy image compression with compressive autoencoders. arXiv preprint arXiv:1703.00395.

## Ejemplo de autoencoder.

El siguiente ejemplo, basado en los [ejemplos de Keras](https://github.com/keras-team/keras/blob/master/examples/mnist_denoising_autoencoder.py), utilizaremos un autoencoder para sacar ruido del MNIST. En el caso del ejemplo, se agregará ruido artificialmente. En particular a cada pixel se le agregará un ruido de media 0.5 y desviación estandard de 0.5. Notese que los pixeles están normalizados a valores entre 0 y 1, por lo que el ruido es significativo.

El encoder tiene las siguiente arquitectura:

1. Entrada de 28 x 28 x 1
1.  Convolucional de 32 filtros y kernel de 3x3
1.  Convolucional de 64 filtros y kernel de 3x3
1. Capa de aplanado. Cada imagen resulta en vectores de 3136 elementos
1. Densa con 16 neuronas

Es decir, al final del encoder cada imagen queda representada por un vector de 16 caractéristicas en lugar de 784 pixeles.

El decoder, quien es el encargado de regenerar la imagen tiene la siguiente arquitectura:

1. Entrada de 16
1. Una capa densa con 3136 salidas
1. Deconvolución de 64 filtros
1. Deconvolución de 32 filtros
1. Deconvolución de 1 filtro. Reconstruendo la imagen original.

Las deconvoluciones son operaciones que permiten reconstruir imagenes a las que se le aplicaron filtros convolucionales. Ver: [Deconvolutional Networks](https://www.matthewzeiler.com/mattzeiler/deconvolutionalnetworks.pdf).

```
%matplotlib inline
import keras
from keras.layers import Activation, Dense, Input
from keras.layers import Conv2D, Flatten
from keras.layers import Reshape, Conv2DTranspose
from keras.models import Model
from keras import backend as K
from keras.datasets import mnist
import numpy as np
import matplotlib.pyplot as plt
from PIL import Image

np.random.seed(1337)

# MNIST dataset
(x_train, _), (x_test, _) = mnist.load_data()

image_size = x_train.shape[1]
x_train = np.reshape(x_train, [-1, image_size, image_size, 1])
x_test = np.reshape(x_test, [-1, image_size, image_size, 1])
x_train = x_train.astype('float32') / 255
x_test = x_test.astype('float32') / 255

# Generate corrupted MNIST images by adding noise with normal dist
# centered at 0.5 and std=0.5
noise = np.random.normal(loc=0.5, scale=0.5, size=x_train.shape)
x_train_noisy = x_train + noise
noise = np.random.normal(loc=0.5, scale=0.5, size=x_test.shape)
x_test_noisy = x_test + noise

x_train_noisy = np.clip(x_train_noisy, 0., 1.)
x_test_noisy = np.clip(x_test_noisy, 0., 1.)

# Network parameters
input_shape = (image_size, image_size, 1)
batch_size = 128
kernel_size = 3
latent_dim = 16
# Encoder/Decoder number of CNN layers and filters per layer
layer_filters = [32, 64]

# Build the Autoencoder Model
# First build the Encoder Model
inputs = Input(shape=input_shape, name='encoder_input')
x = inputs
# Stack of Conv2D blocks
# Notes:
# 1) Use Batch Normalization before ReLU on deep networks
# 2) Use MaxPooling2D as alternative to strides>1
# - faster but not as good as strides>1
for filters in layer_filters:
    x = Conv2D(filters=filters,
               kernel_size=kernel_size,
               strides=2,
               activation='relu',
               padding='same')(x)

# Shape info needed to build Decoder Model
shape = K.int_shape(x)

# Generate the latent vector
x = Flatten()(x)
latent = Dense(latent_dim, name='latent_vector')(x)

# Instantiate Encoder Model
encoder = Model(inputs, latent, name='encoder')
print('Encoder')
encoder.summary()

# Build the Decoder Model
latent_inputs = Input(shape=(latent_dim,), name='decoder_input')
x = Dense(shape[1] * shape[2] * shape[3])(latent_inputs)
x = Reshape((shape[1], shape[2], shape[3]))(x)

# Stack of Transposed Conv2D blocks
# Notes:
# 1) Use Batch Normalization before ReLU on deep networks
# 2) Use UpSampling2D as alternative to strides>1
# - faster but not as good as strides>1
for filters in layer_filters[::-1]:
    x = Conv2DTranspose(filters=filters,
                        kernel_size=kernel_size,
                        strides=2,
                        activation='relu',
                        padding='same')(x)

x = Conv2DTranspose(filters=1,
                    kernel_size=kernel_size,
                    padding='same')(x)

outputs = Activation('sigmoid', name='decoder_output')(x)

# Instantiate Decoder Model
decoder = Model(latent_inputs, outputs, name='decoder')
print('Decoder')
decoder.summary()

# Autoencoder = Encoder + Decoder
# Instantiate Autoencoder Model
print('Encoder-decoder apliado para entrenamiento')
autoencoder = Model(inputs, decoder(encoder(inputs)), name='autoencoder')
autoencoder.summary()

autoencoder.compile(loss='mse', optimizer='adam')

# Train the autoencoder
autoencoder.fit(x_train_noisy,
                x_train,
                validation_data=(x_test_noisy, x_test),
                epochs=30,
                batch_size=batch_size)

# Predict the Autoencoder output from corrupted test images
x_decoded = autoencoder.predict(x_test_noisy)

# Display the 1st 8 corrupted and denoised images
rows, cols = 10, 30
num = rows * cols
imgs = np.concatenate([x_test[:num], x_test_noisy[:num], x_decoded[:num]])
imgs = imgs.reshape((rows * 3, cols, image_size, image_size))
imgs = np.vstack(np.split(imgs, rows, axis=1))
imgs = imgs.reshape((rows * 3, -1, image_size, image_size))
imgs = np.vstack([np.hstack(i) for i in imgs])
imgs = (imgs * 255).astype(np.uint8)
plt.rcParams['figure.figsize'] = [25, 25]
plt.figure()
plt.axis('off')
plt.title('Original images: top rows, '
          'Corrupted Input: middle rows, '
          'Denoised Input:  third rows')
plt.imshow(imgs, interpolation='none', cmap='gray')
Image.fromarray(imgs).save('corrupted_and_denoised.png')
plt.show()
```

## Visualizando los resultado

```
plt.rcParams['figure.figsize'] = [25, 25]
plt.figure()
plt.axis('off')
plt.title('Original images: top rows, '
          'Corrupted Input: middle rows, '
          'Denoised Input:  third rows')
plt.imshow(imgs, interpolation='none', cmap='gray')
Image.fromarray(imgs).save('corrupted_and_denoised.png')
plt.show()
```

## Variational autoencoder

Los variational autoencoders son considerados modelos generativos, es decir, que se pueden construir nuevas intancias utilizando este modelo. Al igual que el autoencoder presentado anteriormente, reduce la representación a un espacio dimensional menor. Sin embargo, este espacio representan poblaciones de distribución normal representado por su media y varianza (generalmente se utiliza el logaritmo de la varianza). Por este motivo, a la hora de reconstruir el decoder deber realizar un sampling de este espacio normal. Al ser un sampling, la reconstrucción no es deterministica por lo que se generan nuevas instancias similares a las anteriores.

```
%matplotlib inline
from keras.layers import Lambda, Input, Dense
from keras.models import Model
from keras.datasets import mnist
from keras.losses import mse, binary_crossentropy
from keras.utils import plot_model
from keras import backend as K

import numpy as np
import matplotlib.pyplot as plt
import argparse
import os

# reparameterization trick
# instead of sampling from Q(z|X), sample epsilon = N(0,I)
# z = z_mean + sqrt(var) * epsilon
def sampling(args):
    """Reparameterization trick by sampling from an isotropic unit Gaussian.
    # Arguments
        args (tensor): mean and log of variance of Q(z|X)
    # Returns
        z (tensor): sampled latent vector
    """

    z_mean, z_log_var = args
    batch = K.shape(z_mean)[0]
    dim = K.int_shape(z_mean)[1]
    # by default, random_normal has mean = 0 and std = 1.0
    epsilon = K.random_normal(shape=(batch, dim))
    return z_mean + K.exp(0.5 * z_log_var) * epsilon

def plot_results(models,
                 data,
                 batch_size=128,
                 model_name="vae_mnist"):
    """Plots labels and MNIST digits as a function of the 2D latent vector
    # Arguments
        models (tuple): encoder and decoder models
        data (tuple): test data and label
        batch_size (int): prediction batch size
        model_name (string): which model is using this function
    """

    encoder, decoder = models
    x_test, y_test = data
    os.makedirs(model_name, exist_ok=True)

    filename = os.path.join(model_name, "vae_mean.png")
    # display a 2D plot of the digit classes in the latent space
    z_mean, z_log_var, _ = encoder.predict(x_test,
                                   batch_size=batch_size)
    plt.figure(figsize=(12, 10))
    plt.scatter(z_mean[:, 0], z_mean[:, 1], c=y_test)
    plt.colorbar()
    plt.xlabel("z[0]")
    plt.ylabel("z[1]")
    plt.savefig(filename)
    plt.show()

    plt.figure(figsize=(12, 10))
    plt.scatter(z_log_var[:, 0], z_log_var[:, 1], c=y_test)
    plt.colorbar()
    plt.xlabel("z[0]")
    plt.ylabel("z[1]")
    plt.savefig(filename)
    plt.show()

    filename = os.path.join(model_name, "digits_over_latent.png")
    # display a 30x30 2D manifold of digits
    n = 30
    digit_size = 28
    figure = np.zeros((digit_size * n, digit_size * n))
    # linearly spaced coordinates corresponding to the 2D plot
    # of digit classes in the latent space
    grid_x = np.linspace(-4, 4, n)
    grid_y = np.linspace(-4, 4, n)[::-1]

    for i, yi in enumerate(grid_y):
        for j, xi in enumerate(grid_x):
            z_sample = np.array([[xi, yi]])
            x_decoded = decoder.predict(z_sample)
            digit = x_decoded[0].reshape(digit_size, digit_size)
            figure[i * digit_size: (i + 1) * digit_size,
                   j * digit_size: (j + 1) * digit_size] = digit

    plt.figure(figsize=(10, 10))
    start_range = digit_size // 2
    end_range = (n - 1) * digit_size + start_range + 1
    pixel_range = np.arange(start_range, end_range, digit_size)
    sample_range_x = np.round(grid_x, 1)
    sample_range_y = np.round(grid_y, 1)
    plt.xticks(pixel_range, sample_range_x)
    plt.yticks(pixel_range, sample_range_y)
    plt.xlabel("z[0]")
    plt.ylabel("z[1]")
    plt.imshow(figure, cmap='Greys_r')
    plt.savefig(filename)
    plt.show()

# MNIST dataset
(x_train, y_train), (x_test, y_test) = mnist.load_data()

image_size = x_train.shape[1]
original_dim = image_size * image_size
x_train = np.reshape(x_train, [-1, original_dim])
x_test = np.reshape(x_test, [-1, original_dim])
x_train = x_train.astype('float32') / 255
x_test = x_test.astype('float32') / 255

# network parameters
input_shape = (original_dim, )
intermediate_dim = 512
batch_size = 128
latent_dim = 2
epochs = 50

# VAE model = encoder + decoder
# build encoder model
inputs = Input(shape=input_shape, name='encoder_input')
x = Dense(intermediate_dim, activation='relu')(inputs)
z_mean = Dense(latent_dim, name='z_mean')(x)
z_log_var = Dense(latent_dim, name='z_log_var')(x)

# use reparameterization trick to push the sampling out as input
# note that "output_shape" isn't necessary with the TensorFlow backend
z = Lambda(sampling, output_shape=(latent_dim,), name='z')([z_mean, z_log_var])

# instantiate encoder model
encoder = Model(inputs, [z_mean, z_log_var, z], name='encoder')
encoder.summary()
plot_model(encoder, to_file='vae_mlp_encoder.png', show_shapes=True)

# build decoder model
latent_inputs = Input(shape=(latent_dim,), name='z_sampling')
x = Dense(intermediate_dim, activation='relu')(latent_inputs)
outputs = Dense(original_dim, activation='sigmoid')(x)

# instantiate decoder model
decoder = Model(latent_inputs, outputs, name='decoder')
decoder.summary()
plot_model(decoder, to_file='vae_mlp_decoder.png', show_shapes=True)

# instantiate VAE model
outputs = decoder(encoder(inputs)[2])
vae = Model(inputs, outputs, name='vae_mlp')

def run():
    models = (encoder, decoder)
    data = (x_test, y_test)

    # VAE loss = mse_loss or xent_loss + kl_loss
    #reconstruction_loss = mse(inputs, outputs)
    reconstruction_loss = binary_crossentropy(inputs, outputs)

    reconstruction_loss *= original_dim
    kl_loss = 1 + z_log_var - K.square(z_mean) - K.exp(z_log_var)
    kl_loss = K.sum(kl_loss, axis=-1)
    kl_loss *= -0.5
    vae_loss = K.mean(reconstruction_loss + kl_loss)
    vae.add_loss(vae_loss)
    vae.compile(optimizer='adam')
    vae.summary()
    plot_model(vae,
               to_file='vae_mlp.png',
               show_shapes=True)

    # train the autoencoder
    vae.fit(x_train,
            epochs=epochs,
            batch_size=batch_size,
            validation_data=(x_test, None))
    #vae.save_weights('vae_mlp_mnist.h5')

    plot_results(models,
                 data,
                 batch_size=batch_size,
                 model_name="vae_mlp")

run()
```