## Summary

Author: Trenton Bricken

1. Plotting the text inputs that are put into GPT2 in order to try and infer it's empirical $\beta$ values used.
2. Running these texts through GPT2 to compute their Key and Query values and retrieve their dot product to infer the interval of input values that go into the Attention Softmax functions for all Attention heads across all layers.

cd to this directory then:
pip install -e .

### Sources used for the text inputs:
* Alice and wonderland: https://gist.github.com/phillipj/4944029
* reddit story 1: https://www.reddit.com/r/stories/comments/50yi2d/prettier_pt_3/
* reddit story 2: https://www.reddit.com/r/stories/comments/ermiqv/i_will_never_take_a_shit_at_school_or_someones/
* attention is all you need paper: https://arxiv.org/pdf/1706.03762.pdf
* SDM book chapter: https://redwood.berkeley.edu/wp-content/uploads/2020/08/KanervaP_SDMrelated_models1993.pdf

## 1. Pretty Plot of Text Inputs

```python
%load_ext autoreload
%autoreload 2
import numpy as np
import matplotlib.pyplot as plt
from scipy.stats import binom, norm
from scipy.sparse import csc_matrix, coo_matrix, csr_matrix
import pandas as pd
from scipy.integrate import quad
#from sympy import *
import time
from SDM_Circ_Inter_Funcs import *
import pickle
directory = 'GPT2Outputs/'

texts = []
with open('text_inputs.txt') as f:
    lines = f.readlines()
    for l in lines:
        texts.append(l.strip('\r\n'))

lines[0]
```

    'We’ve trained a large language model called GPT-2 that generates realistic paragraphs of text, while also exhibiting zero shot generalization on tasks like machine translation, question answering, reading comprehension, and summarization - problems usually approached by using training datasets and models designed explicitly for these tasks. A typical approach to language modeling is to learn the following task: predict the next word, given all of the previous words within some text. Last year, OpenAI’s Generative Pre-trained Transformer (GPT) showed that language models trained on large amounts of data can be fine-tuned to specific tasks to achieve high performance. GPT-2 shows that much larger language models trained on a more diverse dataset derived from the internet begin to learn these NLP tasks without needing task-specific training data, instead learning from examples the system derives from the raw text. These systems also display a substantial qualitative jump in the realism and coherence of generated text.\n'

```python
from transformers import GPT2Tokenizer, GPT2Model, GPT2LMHeadModel
tokenizer = GPT2Tokenizer.from_pretrained('gpt2')
```

```python
lens = []
for t in texts:
    lens.append(tokenizer(t, return_tensors='pt')['input_ids'].shape[1])
```

```python
df = pd.DataFrame([texts, lens]).T
pd.set_option('display.max_colwidth', -1)
df.columns = ['Text', 'Token Lengths']
df
```

    /Users/trentonbricken/anaconda3/envs/ed2/lib/python3.7/site-packages/ipykernel_launcher.py:2: FutureWarning: Passing a negative integer is deprecated in version 1.0 and will not be supported in future version. Instead, use None to not limit the column width.

      Text
      Token Lengths

      0
      We’ve trained a large language model called GPT-2 that generates realistic paragraphs of text, while also exhibiting zero shot generalization on tasks like machine translation, question answering, reading comprehension, and summarization - problems usually approached by using training datasets and models designed explicitly for these tasks. A typical approach to language modeling is to learn the following task: predict the next word, given all of the previous words within some text. Last year, OpenAI’s Generative Pre-trained Transformer (GPT) showed that language models trained on large amounts of data can be fine-tuned to specific tasks to achieve high performance. GPT-2 shows that much larger language models trained on a more diverse dataset derived from the internet begin to learn these NLP tasks without needing task-specific training data, instead learning from examples the system derives from the raw text. These systems also display a substantial qualitative jump in the realism and coherence of generated text.
      195

      1
      Used most notably in the Transformer, Attention has enabled deep learning to arguably approach human level performance across modalities with larger models continuing to boost performance. However, the heuristic motivations that produced Attention leave open the question of why it is so good at the same tasks humans are. Insights into why Attention is so effective would not only make it more interpretable but also guiding future improvements. Much work has been done to try and explain the Attention's success, including work showing that Transformers representations map more closely to human brain recordings and inductive biases than other models. Our work goes further in showing the potential relationship between Attention and the brain at the level of the connectome, giving a new and exciting perspective behind Attention's success. This potential relationship to the brain is created by showing mathematically that Attention closely approximates Sparse Distributed Memory (SDM) and is summarized in Figure A. SDM is an associative memory model developed in 1988 to solve the ``Best Match Problem'', where we have a set of memories and want to quickly find the ``best'' match to any given query. In the development of its solution, SDM respected fundamental biological constraints, such as Dale's law that synapses are fixed to be either excitatory or inhibitory and cannot dynamically switch (see Section 1 for an overview of SDM). While being developed independently of brain regions, surprisingly, SDM's biologically plausible solution maps strikingly well onto the cerebellum. At a high level, the relationship between SDM and Attention exists because SDM's update operation approximates the Attention softmax rule. To give intuition for why this is the case, imagine two identical, intersecting spheres with their centers distance $c$ apart. As the distance $c$ between the spheres increases, under certain conditions the volume of their intersection will decay approximately exponentially. Mapping this onto SDM, one sphere defines the neurons that have written a pattern into memory and the other sphere defines the neurons being read from by a query. The volume of the sphere intersection defines the weight or ``attention'' assigned to a pattern. The approximately exponential decay as the distance between the pattern and query spheres increases can be mapped onto the exponential function used in Attention's softmax. Section 2 provides more details and a formal analysis of this approximate exponential and how Attention closely approximates itEstablishing that Attention approximates SDM in theory, we then test it in pre-trained GPT2 Transformer models (Section 3). We use the Query-Key Normalized Transformer variant to directly show that the relationship to SDM holds well. We then use original GPT2 models to help confirm this result and make it more general. Given these results, we put forward the hypothesis that due to the exponential function's rapid ``take-off'' with increasingly large positive inputs, for Attention to work well it may need to exist in a regime where it always closely approximates SDM.Using the SDM framework, we are able to go beyond Attention and interpret the Transformer architecture as a whole, providing its heuristic structure with deeper intuition (Section 4). The fact that SDM maps well onto the whole Transformer suggests that extensions to SDM that already exist could motivate future Transformer improvements.Finally, in related work (Section 5), we explain how SDM is a generalization of Hopfield Networks and, in turn, how our results are also a generalization of work relating Hopfield Networks to Attention. We also discuss Vector Symbolic Architectures that are compatible with SDM and relations to other memory models such as the Neural Turing and Kanerva Machines.To what extent will our most successful deep learning systems converge with the brain? Some convergence seems inevitable, given that we want our models to excel at the same tasks that brains do. However, the brain and machine learning models have different optimizers, objective functions, and constraints. In this light, it is surprising that the connectome of the cerebellum, if it implements Sparse Distributed Memory, closely approximates Attention and many aspects of the whole Transformer architecture. We believe this work provides a small piece of evidence that the brain and the best deep learning models may be more convergent than we might otherwise think.
      855

      2
      Alice was beginning to get very tired of sitting by her sister on the bank, and of having nothing to do:  once or twice she had peeped into the book her sister was reading, but it had no pictures or conversations in it, `and what is the use of a book,' thought Alice `without pictures or conversation?'   So she was considering in her own mind (as well as she could, for the hot day made her feel very sleepy and stupid), whether the pleasure of making a daisy-chain would be worth the trouble of getting up and picking the daisies, when suddenly a White Rabbit with pink eyes ran close by her.   There was nothing so VERY remarkable in that; nor did Alice think it so VERY much out of the way to hear the Rabbit say to itself, `Oh dear!  Oh dear!  I shall be late!'  (when she thought it over afterwards, it occurred to her that she ought to have wondered at this, but at the time it all seemed quite natural); but when the Rabbit actually TOOK A WATCH OUT OF ITS WAISTCOAT- POCKET, and looked at it, and then hurried on, Alice started to her feet, for it flashed across her mind that she had never before seen a rabbit with either a waistcoat-pocket, or a watch to take out of it, and burning with curiosity, she ran across the field after it, and fortunately was just in time to see it pop down a large rabbit-hole under the hedge.   In another moment down went Alice after it, never once considering how in the world she was to get out again.   The rabbit-hole went straight on like a tunnel for some way, and then dipped suddenly down, so suddenly that Alice had not a moment to think about stopping herself before she found herself falling down a very deep well.   Either the well was very deep, or she fell very slowly, for she had plenty of time as she went down to look about her and to wonder what was going to happen next.  First, she tried to look down and make out what she was coming to, but it was too dark to see anything; then she looked at the sides of the well, and noticed that they were filled with cupboards and book-shelves; here and there she saw maps and pictures hung upon pegs.  She took down a jar from one of the shelves as she passed; it was labelled `ORANGE MARMALADE', but to her great disappointment it was empty:  she did not like to drop the jar for fear of killing somebody, so managed to put it into one of the cupboards as she fell past it.   `Well!' thought Alice to herself, `after such a fall as this, I shall think nothing of tumbling down stairs!  How brave they'll all think me at home!  Why, I wouldn't say anything about it, even if I fell off the top of the house!' (Which was very likely true.)   Down, down, down.  Would the fall NEVER come to an end!  `I wonder how many miles I've fallen by this time?' she said aloud. `I must be getting somewhere near the centre of the earth.  Let me see:  that would be four thousand miles down, I think--' (for, you see, Alice had learnt several things of this sort in her lessons in the schoolroom, and though this was not a VERY good opportunity for showing off her knowledge, as there was no one to listen to her, still it was good practice to say it over) `--yes, that's about the right distance--but then I wonder what Latitude or Longitude I've got to?'  (Alice had no idea what Latitude was, or Longitude either, but thought they were nice grand words to say.) Presently she began again.  `I wonder if I shall fall right THROUGH the earth!  How funny it'll seem to come out among the people that walk with their heads downward!  The Antipathies, I think--' (she was rather glad there WAS no one listening, this time, as it didn't sound at all the right word) `--but I shall have to ask them what the name of the country is, you know. Please, Ma'am, is this New Zealand or Australia?' (and she tried to curtsey as she spoke--fancy CURTSEYING as you're falling through the air!  Do you think you could manage it?)  `And what an ignorant little girl she'll think me for asking!  No, it'll never do to ask:  perhaps I shall see it written up somewhere.
      978

      3
      Previous work showed that the modern Hopfield Network, when made continuous and optimized differently, becomes Attention. This result was one motivation for this work because Hopfield Networks are another form of Associative Memory. However, Hopfield Networks are biologically implausible, lack SDM's geometric framework, are not part of the powerful family of Vector Symbolic Architectures that will be outlined next, and in their original form are less appropriate for handling heteroassociative memory and correlated patterns and the new energy functions of the modern discrete Hopfield network make it a closer approximation to SDM. That SDM is significantly more biologicially plausible than any Hopfield Network variant and has an immediate relationship to Attention that explains the route taken by the Hopfield Network makes this work a generalization of previous insights.Since the publication of SDM there have been a number of advancements not only to SDM specifically, but also through the creation of related associative memory algorithms under the name of ``Vector Symbolic Architectures''. Advancements to SDM specifically include: using integer rather than binary vectors; approaches to handle correlated patterns; hierarchical data storage; and more sophisticated convergence analysis. Vector Symbolic Architectures, most notably Holographic Reduced Representations, have ideas that can be related back to SDM and the Transformer in ways that may be fruitful and will be explored in future work. The use of external memory modules in neural networks has been explored most notably with the Neural Turing Machine (NTM) and its followup, the Differential Neural Computer (DNC). In order to have read and write operations to the external memory be differentiable, they use the softmax function, this combined with their use of cosine similarity between the query and memory locations makes both models closely related to SDM. However, subtle yet arguably important differences exist. In both the NTM and DNC, the memory vectors do not have addresses, only pointers meaning the cosine similarity is between the query and memory contents (analogous to the neuron value vector rather than address). We believe this lack of variable binding between addresses and pointers makes the memory system weaker because one memory cannot point to others. One indication of this weakness is the need for a location based addressing implementation to learn sequence transitions, in addition to the content based addressing outlined. A more recent improvement to the NTM and DNC directly inspired by SDM is the Kanerva Machine. The Kanerva Machine augments a VAE with external memory in the form of SDM and treats it as Bayesian, allowing for Bayesian inference to determine what memories should be stored. However, the Kanerva Machine remains distinct from SDM and Attention because it lacks the softmax function on the cosine similarities between its query and patterns. The model is also restricted to the autoassociative setting with training difficulties noted when the model was used for a heteroassociative reinforcement learning problem. Independent of these discrepancies with SDM, the Kanerva Machine presents a number of interesting ideas that we believe can be related to our work and developed further. The result that Attention approximates SDM should enable more cross pollination of ideas between neuroscience, theoretical models of associative learning, and deep learning. Considering avenues for future deep learning research, SDM's relationship to Vector Symbolic Architectures is particularly compelling because they can apply logical and symbolic operations on memories, making SDM more powerful. Analogizing to computers, SDM is to RAM as VSA is to the CPU. Beyond symbolic approaches, there are extensions to SDM that leverage sparsity and discrete representations that may improve Transformers, similarly to how vector quantization has improved VAEs. SDM and its relation to the brain can inspire new research in not only deep learning research but also neuroscience, because of the empirical success of Transformers and its relation to the cerebellum via SDM. While SDM fits unique features of the cerebellum in striking ways, this connection and its evidence relies on neuroscientific evidence available in 1988. Although we are unaware of any results that disprove this connection and the Marr-Albus-Ito model of the cerebellum that fits with SDM remains popular, in future work we will review modern neuroscience to update the state of SDM's biological plausibility. There are two reasons to be initially excited about this investigation. First, there has been growing evidence that the cerebellum is important for many more cognitive functions than just precise motor control, long believed to be its only role. Second, improvements to our understanding of the computational capabilities of Pyramidal neurons means that they may have parallels to Purkinje cells and that the core algorithmic requirements of SDM could be implemented in additional brain regions such as the hippocampus and cortical columns. These remain early hypotheses but the success of Transformers and their link to SDM should catalyze novel investigations.One limitation of our analysis in this paper is that we cannot use SDM to predict the $\beta$ value that Attention should use.
      1005

      4
      So, we left off in my senior year of high school. It was 2007. Chris Brown was still a good guy in public eye, Britney Spears and K-Fed had recently split, Michael Jackson was still alive, and my high school boyfriend and I had reached a massive fork in the road. We were accepted into two different universities on opposite sides of the country. He was moving to the other side while I was staying close to home. The good news? We were accepted into our first picks. The bad news? Our relationship was toast. We decided we would enjoy whatever time we had left together, and then go our separate ways. I wasn’t very keen on long distance as I wanted to focus more on my studies than making sure my boyfriend wasn’t cheating on me. Dean agreed. We were exactly on the same page in that regard. When time came around for us to say our goodbyes, I made sure to let him know he was as close to perfect as I had ever met. Not only was he patient, supportive and trusting, he was all of those things and so effortlessly understanding at the same time. Of course we had our moments, but it was all worth it. I can safely say I was the lucky one in our relationship. I’ve heard so many horror stories of crazy, manipulative boyfriends but Dean wasn’t like that at all. He was good. So a couple of months into university, when I heard he was dating another girl at his school, I wasn’t at all surprised. Dean was a charmer. Of course there was another girl lol. She was cute. I had only a few pictures to go off of on social media, but yeah. She very cute, very bubbly looking, and exactly the type of girl I had always imagined for Dean. In other words I was happy for them. I mean, don’t get me wrong, I had my jealous moments … but I was mostly happy for them. The fact that I was suddenly thrust into university and forced to make new friends at school, helped to distract me from those tiny, unwelcome sparks of jealousy. I eventually wasn’t jealous anymore, and I eventually met this girl when Dean brought her home for Christmas. Our tight knit group of friends from high school had all met up at one of our houses the day before Christmas Eve. Dean politely introduced his girlfriend to me, and we got on really well. She was a nice person. Still is, as a matter of fact. (They’re married now and pregnant with their first child. Crazy, huh?) The next night, when I was with my family and a bunch of our relatives at my parents’ Christmas Eve party, I realized how much I missed home. School wasn’t too far away. Maybe two hours on a particularly busy day. But during those first few months, I tried my best not to go home and fall into old, comfortable habits. I wanted to grow up and separate myself from my family a little bit. That said, I absolutely cried when I saw them again. Such a cornball lol. My brother made fun of me for years over that. In fact he still does. He’s kind of an asshole like that, but in a good way. Suffice to say I stuffed my face and immediately fell into food coma. It was close to midnight, I would say. I dragged myself into my old bedroom, where everything had been left the same since the last time I was there. The murmur of music and laughter down below kept me somewhat alert of my surroundings. I knew it wouldn’t be long before my mother or father came knocking to see if I wanted some extra dessert or something, so I told myself I would rest for only a few minutes. The second I flopped down in bed and sunk into the comfy mattress I been deprived of at school (seriously, the mattress in my dorm was like a slab of concrete) I felt my pocket vibrate. I groaned into my pillow, lazily retrieving the flip phone from my pocket to find a text message. From: Nicky Merry Christmas However tired I felt, I smiled when I saw the message. There were times when we would go months without speaking, and times when we would speak nearly everyday. No matter how long we had gone without talking, we’d always pick up exactly where we left off as though no time had passed. He was going to school in the same city as me. But, for some reason, we had yet to hang out. We tried a few times in the beginning, organizing lunch or coffee somewhere in the city, but the plan would always change. We’d run into scheduling problems or papers that were due. For the most part, he was the one to cancel on me. I figured he wanted to focus on school and his new friends
      1006

      5
      So this takes place back in high school, long time ago. I was in class and I wasn't feeling too good that day. I wish I was kidding, I really do but I'm not. I have a bowel disorder with the intestines, not gonna get into that. But I was in class, didn't shit for about a week and a half if I remember correctly. I had to take a shit. Really fucking bad. My stomach was aching so bad. I would NEVER shit at someones house, or at school. Except for that day. I thought to myself "Fuck it, I'll take a shit in the bathroom. It's gonna be the biggest shit anyone has seen, but most likely they'll have no idea who it was. So I asked the teacher to use the bathroom, took a hall pass and was on my way. I get into that bathroom, nobody was in it a the time (thank god) and immediately run to the stall, close it, lock it, drop my pants and sit on the toilet. About 5-7 seconds later, there it goes. It was thirty seconds of pushing, one came out, and then another one. I could smell the defecation as soon as it hit the water in the toilet, and it was awful. This wasn't going to flush, there's no way. But then a student walked in. My heart sank from my chest all the way to my ballsack probably.I was genuinely terrified. The student walked right in. He said "oh god what the fuck is that" he took a piss, and left as quick as possible. I sat on the toilet for the next few minutes. My stomach was aching, but I also felt relief. No, I didn't flush it. Didn't even try, I knew it would clog. I didn't want anyone to suspect it was me in even the slightest. I eventually left the bathroom, and walked back to class. I was in there for a bit, so the teacher said I was late on the way back. With some quick thinking I tell her "I'm sorry miss, Won't lie to you, my friends walked in and we got caught up in silly conversation" She replies to me "Well, at least you're telling the truth, I appreciate that. But you're behind on work" I went back to my desk, finished my stuff. At the end of the day, nobody had an idea who it was that took this giant shit in the toilet and left it there. I went home that day, did things a teenager does at home after school, usual shenanigans. Took a piss, video games, half-assed my homework, busted a nut all that stuff. Went back to school the next day. My first thought is "I'm going to that bathroom to see if that shit is still there, if the janitors did anything about it. So I walked in, and that entire stall had a sign that read "out of order" the toilet was closed up and everything. I was dying of laughter, it was absolutely hilarious. I tried so hard to contain my laughter which I did. I thought it was fucking hysterical coming back the next day and seeing that same toilet out of order the next day. It stayed like that for maybe another month. And to this very day, nobody had any idea who did it. This was definitely one of my weirdest, but also funniest HS experiences I've ever had.
      711

      6
      The dominant sequence transduction models are based on complex recurrent or convolutional neural networks that include an encoder and a decoder. The best performing models also connect the encoder and decoder through an attention mechanism. We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely. Experiments on two machine translation tasks show these models to be superior in quality while being more parallelizable and requiring significantly less time to train. Our model achieves 28.4 BLEU on the WMT 2014 Englishto-German translation task, improving over the existing best results, including ensembles, by over 2 BLEU. On the WMT 2014 English-to-French translation task, our model establishes a new single-model state-of-the-art BLEU score of 41.8 after training for 3.5 days on eight GPUs, a small fraction of the training costs of the best models from the literature. We show that the Transformer generalizes well to other tasks by applying it successfully to English constituency parsing both with large and limited training data. Recurrent models typically factor computation along the symbol positions of the input and output sequences. Aligning the positions to steps in computation time, they generate a sequence of hidden states ht, as a function of the previous hidden state ht−1 and the input for position t. This inherently sequential nature precludes parallelization within training examples, which becomes critical at longer sequence lengths, as memory constraints limit batching across examples. Recent work has achieved significant improvements in computational efficiency through factorization tricks [21] and conditional computation [32], while also improving model performance in case of the latter. The fundamental constraint of sequential computation, however, remains. Attention mechanisms have become an integral part of compelling sequence modeling and transduction models in various tasks, allowing modeling of dependencies without regard to their distance in the input or output sequences [2, 19]. In all but a few cases [27], however, such attention mechanisms are used in conjunction with a recurrent network. In this work we propose the Transformer, a model architecture eschewing recurrence and instead relying entirely on an attention mechanism to draw global dependencies between input and output. The Transformer allows for significantly more parallelization and can reach a new state of the art in translation quality after being trained for as little as twelve hours on eight P100 GPUs. The goal of reducing sequential computation also forms the foundation of the Extended Neural GPU [16], ByteNet [18] and ConvS2S [9], all of which use convolutional neural networks as basic building block, computing hidden representations in parallel for all input and output positions. In these models, the number of operations required to relate signals from two arbitrary input or output positions grows in the distance between positions, linearly for ConvS2S and logarithmically for ByteNet. This makes it more difficult to learn dependencies between distant positions [12]. In the Transformer this is reduced to a constant number of operations, albeit at the cost of reduced effective resolution due to averaging attention-weighted positions, an effect we counteract with Multi-Head Attention as described in section 3.2. Self-attention, sometimes called intra-attention is an attention mechanism relating different positions of a single sequence in order to compute a representation of the sequence. Self-attention has been used successfully in a variety of tasks including reading comprehension, abstractive summarization, textual entailment and learning task-independent sentence representations [4, 27, 28, 22]. End-to-end memory networks are based on a recurrent attention mechanism instead of sequencealigned recurrence and have been shown to perform well on simple-language question answering and language modeling tasks [34]. To the best of our knowledge, however, the Transformer is the first transduction model relying entirely on self-attention to compute representations of its input and output without using sequencealigned RNNs or convolution. In the following sections, we will describe the Transformer, motivate self-attention and discuss its advantages over models such as [17, 18] and [9].
      843

      7
      This chapter describes one basic model of associative memory, called the sparse distributed memory, and relates it to other models and circuits: to ordinary computer memory, to correlation-matrix memories, to feed-forward artificial neural nets, to neural circuits in the brain, and to associative-memory models of the cerebellum. Presenting the various designs within one framework will hopefully help the reader see the similarities and the differences in designs that are often described in different ways. 3.1.1. Sparse Distributed Memory as a Model of Human Long-Term Memory Sparse Distributed Memory (SDM) was developed as a mathematical model of human long-term memory (Kanerva 1988). The pursuit of a simple idea led to the discovery of the model, namely, that the distances between concepts in our minds correspond to the distances between points of a high-dimensional space. In what follows, ‘high-dimensional’ means that the number of dimensions is at least in the hundreds, although smaller numbers of dimensions are often found in examples. If a concept, or a percept, or a moment of experience, or a piece of information in memory—a point of interest—is represented by a high-dimensional (or “long”) vector, the representation need not be exact. This follows from the distribution of points of a high-dimensional space: Any point of the space that might be a point of interest is relatively far from most of the space and from other points of interest. Therefore, a point of interest can be represented with considerable slop before it is confused with other points of interest. In this sense, long vectors are fault-tolerant or robust, and a device based on them can take advantage of the robustness. This corresponds beautifully to how humans and animals with advanced sensory systems and brains work. The signals received by us at two different times are hardly ever identical, and yet we can identify the source of the signal as a specific individual, object, place, scene, thing. The representations used by the brain must allow for such identification, in fact, they must make the identification nearly automatic, and high-dimensional vectors as internal representations of things do that. Another property of high-dimensional spaces also has to do with the distances between points. If we take two points (of interest) at random, they are relatively far from each other, on the average: they are uncorrelated. However, there are many points between the two that are close to both, in the sense that the amount of space around an intermediate point—in a hypersphere—that contains both of the two original points is very small. This corresponds to the relative ease with which we can find a concept that links two unrelated concepts. Strictly speaking, a mathematical space need not be a high-dimensional vector space to have the desired properties; it needs to be a huge space, with an appropriate similarity measure for pairs of points, but the measure need not define a metric on the space. The important properties of high-dimensional spaces are evident even with the simplest of such spaces—that is, when the dimensions are binary. Therefore, the sparse distributed memory model was developed using long (i.e., highdimensional) binary vectors or words. The memory is addressed by such words, and such words are stored and retrieved as data. The following two examples demonstrate the memory’s robustness in dealing with approximate data. The memory works with 256-bit words: it is addressed by them, and it stores and retrieves them. On top of Figure 3.1 are nine similar (20% noisy) 256-bit words. To help us compare long words, their 256 bits are laid on a 16-by-16 grid, with 1s shown in black. The noise-free prototype word was designed in the shape of a circle within the grid. (This example is confusing in that it might be taken to imply that humans recognize circles based on stored retinal images of circles. No such claim is intended.) The nine noisy words were stored in a sparse distributed memory autoassociatively, meaning that each word was stored with itself as the address. When a tenth noisy word (bottom left), different from the nine, was used as the address, a relatively noise-free 11th word was retrieved (bottom middle), and with that as the address, a nearly noise-free 12th word was retrieved (bottom right), which in turn retrieved itself. This example demonstrates the memory’s tendency to construct a prototype from noisy data.
      926

      8
      Biological neurons are cells that process signals in animals and humans, allowing them to respond rapidly to the environment. To achieve speed, neurons use electrochemical mechanisms to generate a signal (a voltage level or electrical pulses) and to transmit it to nearby and distant sites. Biological neurons come in many varieties. The peripheral neurons couple the organism to the world. They include the sensory neurons that convert an external stimulus into an electrical signal, the motor neurons whose electrical pulses cause muscle fibers to contract, and other effector neurons that regulate the secretion of glands. However, most neurons in highly evolved animals are interneurons that connect directly to other neurons rather than to sensors or to effectors. Interneurons also come in many varieties and they are organized into a multitude of neural circuits. A typical interneuron has a cell body and two kinds of arborizations: a dendrite tree that receives signals from other neurons, and an axon tree that transmits the neuron’s signal to other neurons. Transmission-contact points between neurons are called synapses. They are either excitatory (positive synaptic weight) or inhibitory (negative synaptic weight) according to whether a signal received through the synapse facilitates or hinders the activation of the receiving neuron. The axon of one neuron can make synaptic contact with the dendrites and cell bodies of many other neurons. Thus, a neuron receives multiple inputs, it integrates them, and it transmits the result to other neurons. Artificial neural networks are networks of simple, interconnected processing units, called (artificial) neurons. The most common artificial neuron in the literature has multiple (N) inputs and one output and is defined by a set of input coefficients—a vector of N reals, standing for the synaptic weights—and a nonlinear scalar activation function. The value of this function is the neuron’s output, and it serves as input to other neurons. A linear threshold function is an example of an artificial neuron, and the simplest kind—one with binary inputs and output—is used in the sparse distributed memory. It may seem strange to model brain activity with binary neurons when real neurons are very complex in comparison. However, the brain is organized in large circuits of neurons working in parallel, and the mathematical study of neural nets is aimed more at understanding the behavior of circuits than of individual neurons. An important fact—perhaps the most important—is that the states of a large circuit can be mapped onto the points of a high-dimensional space, so that although a binary neuron is a grossly simplified model of a biological neuron, a large circuit of binary neurons, by virtue of its high dimension, can be a useful model of a circuit of biological neurons. The sparse distributed memory’s connection to biology is made in the standard way. Each row through A, d, y, and C in Figure 3.9—each hidden unit—is an artificial neuron that represents a biological neuron. Vector x represents the N signals coming to these neurons as inputs from N other neurons (along their axons), vector Am represents the weights of the synapses through which the input signals enter the mth neuron (at its dendrites), dm represents the integration of the input signals by the mth neuron, and ym represents the output signal, which is passed along the neuron’s axon to U other neurons through synapses with strengths Cm. We will call these (the hidden units) the address-decoder neurons because they are like the address-decoder circuit of a random-access memory: they select locations for reading and writing. The address that the mth address-decoder neuron decodes is given by the input coefficients Am; location Am is activated by inputs x that equal or are sufficiently similar to Am. How similar, depends on the radius of activation H. It is interesting that a linear threshold function with N inputs, which is perhaps the oldest mathematical model of a neuron, is ideal for address decoding in the sparse distributed memory, and that a proper choice of a single parameter, the threshold, makes it into an address decoder for a location of an ordinary randomaccess memory.
      844

```python
import dataframe_image as dfi

# will save a high res screenshot of the dataframe.
#dfi.export(df, directory+"TransformerTextInputsTable.png")
```

## 2. Computing Softmax Input Intervals

Have custom modified the Hugging Face codebase to return the key value and query matrices. (Using its implementation we could not get out the query matrix that is crucial for computing the size of the vector dot products that go into the softmax)

```python
import numpy as np
import matplotlib.pyplot as plt
import torch
import torch.optim as optim
import pandas as pd
from IPython.display import display
import pickle

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

############ Parameters:

modelname = 'gpt2-small'#'bert-base-uncased' #'gpt2-xl' #gpt2-small is in fact the small model.

if 'lm-' in modelname:
    modelname_toload = modelname.split('lm-')[-1]
elif modelname=='gpt2-small':
    modelname_toload = 'gpt2' # this is the Hugging Face name for the small model
else:
    modelname_toload = modelname

if 'gpt2-small' in modelname:
    layers=12
    nheads= 12
elif 'gpt2-large' in modelname:
    layers=36
    nheads= 20
elif 'gpt2-xl' in modelname:
    layers=48
    nheads= 25
```

```python

if 'gpt' in modelname:

    from transformers import GPT2Tokenizer, GPT2Model, GPT2LMHeadModel
    tokenizer = GPT2Tokenizer.from_pretrained(modelname_toload)
    if 'lm-' in modelname:
        model = GPT2LMHeadModel.from_pretrained(modelname_toload, output_attentions=True, output_hidden_states=True).to(device)
    else:
        model = GPT2Model.from_pretrained(modelname_toload, output_attentions=True, output_hidden_states=True).to(device)

elif 'bert' in modelname:

    from transformers import BertTokenizer, BertModel, BertLMHeadModel
    # Download model and configuration from huggingface.co and cache.
    tokenizer = BertTokenizer.from_pretrained(modelname_toload)
    model = BertLMHeadModel.from_pretrained(modelname_toload, output_attentions=True, output_hidden_states=True).to(device)

```

    Init GPT2 hello world and custom code!!!

```python
tokenizer(texts[0], return_tensors='pt')['input_ids'].shape
```

    torch.Size([1, 195])

```python
encoded_input = tokenizer(texts[0], return_tensors='pt')
output = model(**encoded_input)
output.keys()
```

    odict_keys(['last_hidden_state', 'past_key_values', 'hidden_states', 'kqvs', 'attentions'])

```python
len(output['past_key_values'])
```

    12

```python
len(output['kqvs'])
```

    12

## Checking that I customized the code to get out the keys, queries, and values correctly and evaluating tensor shapes

```python
output['past_key_values'][5][0].shape
```

    torch.Size([1, 12, 195, 64])

```python
output['kqvs'][5][0].shape
```

    torch.Size([1, 12, 195, 64])

```python
# output['past_key_values'][LAYER][0=KEY, 1=VALUE][batch_size, num_heads, sequence_length, embed_size_per_head]
# output['kqvs'][LAYER][KEY=0, QUERY=1, VALUE=2][batch_size, num_heads, sequence_length, embed_size_per_head]

output['past_key_values'][5][0][0,3,:,:] == output['kqvs'][5][0][0,3,:,:]
```

    tensor([[True, True, True,  ..., True, True, True],
            [True, True, True,  ..., True, True, True],
            [True, True, True,  ..., True, True, True],
            ...,
            [True, True, True,  ..., True, True, True],
            [True, True, True,  ..., True, True, True],
            [True, True, True,  ..., True, True, True]])

```python
key, query, value = output['kqvs'][2] #[0][0,3,:,:]
```

```python
query[0,:,-1,:].unsqueeze(-1).shape
```

    torch.Size([12, 64, 1])

```python
key[0,:,:,:].shape
```

    torch.Size([12, 195, 64])

```python
torch.matmul(key[0,:,:,:], query[0,:,-1,:].unsqueeze(-1)).squeeze().shape
```

    torch.Size([12, 195])

```python
output['attentions'][3][0,:,-1,:].shape
```

    torch.Size([12, 195])

```python
np.linalg.norm(value[0,:,:,:].detach().numpy(), axis=2).shape
```

    (12, 195)

```python
value[0,:,:,:].shape
```

    torch.Size([12, 195, 64])

## Collecting Keys, Queries, Values, and Attention for every text input

```python
verbose = False
beta = 1/np.sqrt(64)
over_texts = dict()

for text_ind, text in enumerate(texts):
    print('text index is:', text_ind)
    encoded_input = tokenizer(text, return_tensors='pt')
    output = model(**encoded_input)
    input_len = encoded_input['input_ids'].shape[1]

    # storing the processed results:
    h_summary = dict()
    l_summary = dict()

    for l_ind in range(layers):
        if verbose:
            print('layer is:', l_ind)

        key, query, value = output['kqvs'][l_ind] #[0][0,3,:,:]

        # if not working with just the last input need to do masking. Not doing this here.
        # code should look something like this.
        #K_masked = K[:, :, :(inp_ind+1),:]

        sm_inputs = beta*torch.matmul(key[0,:,:,:], query[0,:,-1,:].unsqueeze(-1)).squeeze().detach().numpy() # [NHEADS, SEQLEN]

        if verbose:
            print('softmax inputs shpae', sm_inputs.shape)
        V_norms = np.linalg.norm(value[0,:,:,:].detach().numpy(), axis=2)
        A = output['attentions'][l_ind][0,:,-1,:].detach() # # [NHEADS, SEQLEN]
        layer_dtypes = np.stack([sm_inputs, V_norms, A],axis=0)[np.newaxis,:]
        if l_ind==0:
            all_layers_res = layer_dtypes
        else:
            all_layers_res = np.concatenate([all_layers_res, layer_dtypes],axis=0)

        # max for each of the heads.
        maxs = sm_inputs.max(axis=1)
        mins = sm_inputs.min(axis=1)
        intervals = pd.DataFrame([mins,maxs])
        if verbose:
            print('observed interval is:')
            display(intervals.T)
            print('mean value of maxes for each head in layer:', np.mean(maxs))
            print('median:', np.median(maxs))
            print('max of maxes:', np.max(maxs))

        # max of each head logging each layer
        h_summary[l_ind] = maxs
        # summary across heads for each layer
        l_summary[l_ind] = [np.mean(maxs), np.median(maxs), np.max(maxs) ]

    # all_layers_res: Layers x Dtypes x Heads x SeqLen

    # input to softmax for all layers.
    plt.figure()
    plt.hist(all_layers_res[:,0,:,:].flatten(), bins=50)
    plt.gcf().savefig(directory + modelname+'AllLayerPreSMs|text_ind='+str(text_ind)+'.png', dpi=250)
    plt.show()

    plt.figure()
    plt.scatter(all_layers_res[:,2,:,:].flatten(), all_layers_res[:,1,:,:].flatten())
    plt.xlabel("Attention Weight")
    plt.ylabel("Value L2 Norm")
    plt.title("Value Magnitude as a function of Attention weight")
    plt.gcf().savefig(directory + modelname+'AllLayerValMagAttWeight|text_ind='+str(text_ind)+'.png', dpi=250)
    plt.show()

    l_summary = pd.DataFrame(l_summary).T
    l_summary.columns = ['mean', 'median', 'max']
    # rows are layers and columns are heads
    h_summary = pd.DataFrame(h_summary).T

    if verbose:
        display(l_summary)
        display(h_summary)

    # saving out these summaries:
    over_texts[text_ind] = (l_summary, h_summary, all_layers_res)

with open(directory+modelname+"results_over_all_texts.pkl", 'wb') as fname:
    pickle.dump(over_texts, fname)
```

```python
all_layers_res.shape # Layers x Dtypes x Heads x SeqLen
```

    (12, 3, 12, 844)

## Plotting various aggregations of the heads, layers, and texts to get the effective $\beta$ and also investigate the relationship between attention weights and the value vector's L2 norm

```python
import pickle
with open(directory+modelname+"results_over_all_texts.pkl", 'rb') as fname:
    over_texts = pickle.load( fname)
```

```python
# replotting the results from before but also aggregating across them.
for text_ind, text in enumerate(texts):
    print('text ind', text_ind)
    #print(text)
    # saving out these summaries:
    l_summary, h_summary, all_layers_res = over_texts[text_ind]

    # input to softmax for all layers.
    #all_layers_res = all_layers_res[np.newaxis, :]
    if text_ind==0:
        all_text_vals = all_layers_res
    else:
        #print(all_text_vals.shape, all_layers_res.shape)
        all_text_vals = np.concatenate([all_text_vals, all_layers_res], axis=-1)
    #plt.hist(all_layers_res[:,0,:,:].flatten(), bins=50)
    #plt.gcf().savefig(directory+modelname+'AllLayerPreSMs|text_ind='+str(text_ind)+'.png', dpi=250)
    #plt.show()

    #display(l_summary)
    #display(h_summary)

# all_text_vals:  Layers x Dtypes x Heads x (SeqLen * Texts, all combined!)
print("Summary of all:")
plt.hist(all_text_vals[:,0,:,:].flatten(), bins=50)
plt.gcf().savefig(directory+modelname+'SMValuesForAllTextsLayersHeads.png', dpi=250)
plt.show()

plt.figure()
number_to_plot = 10000
rand_inds = np.random.choice(np.arange(len(all_text_vals[:,2,:,:].flatten())),number_to_plot, replace =False)
plt.scatter(all_text_vals[:,2,:,:].flatten()[rand_inds], all_text_vals[:,1,:,:].flatten()[rand_inds])
plt.xlabel("Attention Weight")
plt.ylabel("Value L2 Norm")
plt.title("Value Magnitude as a function of Attention weight")
plt.gcf().savefig(directory + modelname+'ValuesvsAttentionAllTextsLayersHeads.png', dpi=250)
plt.show()

############

plt.figure()

r_min = np.min(all_text_vals[:,0,:,:], axis=-1)
r_max = np.max(all_text_vals[:,0,:,:], axis=-1)
ranges = np.abs(r_max-r_min)
mu = (r_max+r_min)/2
size_of_addition_components = mu + r_min

plt.hist(ranges.flatten()/2, bins =50)
plt.title("Ranges of values for each head across texts")
plt.show()

plt.figure()
plt.hist(size_of_addition_components.flatten(), bins =50)
plt.title("size of addition components")
plt.show()

plt.figure()
plt.hist(mu.flatten(), bins =50)
plt.title("means of the ranges")
plt.show()

plt.figure()
#rescaling_betas = ((2*np.abs(r_min))/ranges) -2
#print(ranges*rescaling_betas)
plt.title("Rescaling Beta calc")
plt.hist(rescaling_betas.flatten())
plt.show()
```

```python
### How do results differ between the models?!
import pickle
models = ['gpt2-small', 'gpt2-large'] # did not make a big difference 'lm-gpt2-large']

valsforall = {k:{kk:[] for kk in ['sm', 'vals', 'att']} for k in models}

for m in models:

    print(m)

    with open(directory+m+"results_over_all_texts.pkl", 'rb') as fname:
            model_over_texts = pickle.load( fname)

    for text_ind, text in enumerate(texts):

        #print(text)

        l_summary, h_summary, all_layer_vals = model_over_texts[text_ind]

        '''sm_inputs, val_norms, attns = all_layer_vals[:,0], all_layer_vals[:,1], all_layer_vals[:,2]

        # input to softmax for all layers.
        print(sm_inputs.shape)
        valsforall[m]['sm'] += list(sm_inputs)
        valsforall[m]['vals'] += list(val_norms)
        valsforall[m]['att'] += list(attns)
        plt.hist(sm_inputs, bins=50, alpha=0.5, label=m, density=True)'''
        #print('l and then h summary. these are the max sm inputs.')
        #display(l_summary)
        #display(h_summary)
        plt.plot(l_summary.index, l_summary['mean'], label = 'Text '+str(text_ind))
    plt.legend()
    plt.ylabel('Mean of Head maxes')
    plt.xlabel('Layer Number')
    #plt.xlim(-max(sm_inputs),max(sm_inputs))
    plt.title(r"Mean effective $\beta$ by Layer"+ "\n"+r"(Larger $\beta$ Means Smaller Hamming Distance)")
    plt.gcf().savefig(directory+'PerModelLayerAverageBetas_'+str(m)+'.png', dpi=250)
    plt.show()

    print('======================')

```

```python
models = ['gpt2-small', 'gpt2-large'] # did not make a big difference 'lm-gpt2-large']

valsforall = {k:{kk:[] for kk in ['sm', 'vals', 'att']} for k in models}

across_models, heads_across_models = [], []
for m in models:

    print(m)
    if 'small' in m:
        nlay = 12
    else:
        nlay = 36

    with open(directory+m+"results_over_all_texts.pkl", 'rb') as fname:
            model_over_texts = pickle.load( fname)

    max_seen = np.zeros((nlay,1))

    heads_max_seen = 0

    plt.figure(figsize=(6,5))

    for text_ind, text in enumerate(texts):

        #print(text)

        l_summary, h_summary, all_layer_vals = model_over_texts[text_ind]

        max_seen = np.concatenate([max_seen,
                                   np.asarray(l_summary['max']).reshape(-1,1)], axis=1)

        if text_ind==0:
            heads_max_seen = np.asarray(h_summary).reshape(h_summary.shape[0], h_summary.shape[1], 1)
        else:
            heads_max_seen = np.concatenate([heads_max_seen,
                                   np.asarray(h_summary).reshape(h_summary.shape[0], h_summary.shape[1], 1)], axis=2)

        '''sm_inputs, val_norms, attns = all_layer_vals[:,0], all_layer_vals[:,1], all_layer_vals[:,2]

        # input to softmax for all layers.
        print(sm_inputs.shape)
        valsforall[m]['sm'] += list(sm_inputs)
        valsforall[m]['vals'] += list(val_norms)
        valsforall[m]['att'] += list(attns)
        plt.hist(sm_inputs, bins=50, alpha=0.5, label=m, density=True)'''
        #print('l and then h summary. these are the max sm inputs.')
        #display(l_summary)
        #display(h_summary)
        plt.plot(l_summary.index, l_summary['mean'], label = 'Text '+str(text_ind))
    plt.axhline(10.1,  linestyle='dashed',label=r"$\beta_{CD}=10.1$", color='red')
    plt.axhline(15.9,  linestyle='dashed',label=r"$\beta_{SNR}=15.9$", color='red')
    #plt.axhline(19.3,  linestyle='dashed',label=r"$\beta=19.3$", color='red')
    plt.legend()
    plt.ylabel(r'Mean $\beta$ across heads for each layer')
    plt.xlabel('Layer Number')
    #plt.xlim(-max(sm_inputs),max(sm_inputs))
    plt.title(r"Mean effective $\beta$ by Layer by Text Input")
    plt.gcf().savefig(directory+'PerTextPerModelLayerMeanBetas_'+str(m)+'.png', dpi=250)
    plt.show()

    max_seen = np.nan_to_num(max_seen)
    heads_max_seen =  np.nan_to_num(heads_max_seen)
    plt.figure()
    plt.plot(l_summary.index, max_seen.max(axis=1))
    across_models.append((l_summary.index, max_seen.max(axis=1)))

    #plt.axhline(19.3,  linestyle='dashed',label=r"$\beta=19.3$", color='red')
    plt.axhline(10.1,  linestyle='dashed',label=r"$\beta_{CD}=10.1$", color='red')
    plt.axhline(15.9,  linestyle='dashed',label=r"$\beta_{SNR}=15.9$", color='red')
    #plt.axvline(35.5,  linestyle='dashed',label=r"$\beta_{Mem}=35.5$", color='red')
    plt.ylabel('Max across all heads and texts')
    plt.legend()
    plt.xlabel('Layer Number')
    #plt.xlim(-max(sm_inputs),max(sm_inputs))
    plt.title(r"Max effective $\beta$ by Layer"+ "\n"+r"(Larger $\beta$ Means Smaller Hamming Distance)")
    plt.gcf().savefig(directory+'PerModelLayerMaxBetas_'+str(m)+'.png', dpi=250)
    plt.show()

    # ========

    plt.figure()
    plt.hist(heads_max_seen.max(axis=2).flatten(), bins=50)
    print("dim of heads being plotted", heads_max_seen.max(axis=2).shape)
    heads_across_models.append(heads_max_seen.max(axis=2))

    #plt.axvline(19.3,  linestyle='dashed',label=r"$\beta=19.3$", color='red')
    plt.ylabel('Counts')
    plt.legend()
    plt.xlabel(r'Effective $\beta$s')
    #plt.xlim(-max(sm_inputs),max(sm_inputs))
    plt.title(r"Max effective $\beta$ for every head across texts"+ "\n"+r"(Larger $\beta$ Means Smaller Hamming Distance)")
    plt.gcf().savefig(directory+'PerModelLayerMaxHeadBetas_'+str(m)+'.png', dpi=250)
    plt.show()

    print('======================')
max_seen.shape

for v in across_models:
    x, y = v

    plt.plot(x,y)

plt.axhline(19.3,  linestyle='dashed',label=r"$\beta=19.3$", color='red')
plt.ylabel('Max across all heads and texts')
plt.legend()
plt.xlabel('Layer Number')
#plt.xlim(-max(sm_inputs),max(sm_inputs))
plt.title(r"Max effective $\beta$ by Layer"+ "\n"+r"(Larger $\beta$ Means Smaller Hamming Distance)")
plt.gcf().savefig(directory+'BothModelsLayerMaxBetas_'+str(m)+'.png', dpi=250)
plt.show()

# =======
plt.figure()
for ind, v in enumerate(heads_across_models):
    plt.hist(v.flatten(), bins=50, alpha=0.5, density=True, label=models[ind])

#plt.axvline(19.3,  linestyle='dashed',label=r"$\beta=19.3$", color='red')
plt.axvline(10.1,  linestyle='dashed',label=r"$\beta_{CD}=10.1$", color='red')
plt.axvline(15.9,  linestyle='dashed',label=r"$\beta_{SNR}=15.9$", color='red')
plt.axvline(35.5,  linestyle='dashed',label=r"$\beta_{Mem}=35.5$", color='red')
plt.ylabel('Normalized Counts')
plt.legend()
plt.xlabel(r'Effective $\beta$s')
#plt.xlim(-max(sm_inputs),max(sm_inputs))
plt.title(r"Max effective $\beta$ for every Head across Texts")#+ "\n"+r"(Larger $\beta$ Means Smaller Hamming Distance)")
plt.gcf().savefig(directory+'BothModelsLayerMaxHeadBetas_'+str(m)+'.png', dpi=250)
plt.show()

```

```python
### How do results differ between the models?!

models = ['gpt2-small', 'gpt2-large'] # did not make a big difference 'lm-gpt2-large']

# aggregate over all texts for each model.
second_plot = [] # storing values for second plot here so I can save them easily as separate figures.

for m in models:

    #print(m)

    with open(directory+m+"results_over_all_texts.pkl", 'rb') as fname:
        model_over_texts = pickle.load( fname)

    for text_ind, text in enumerate(texts):
        #print("text ind", text_ind)
        #print(text)

        l_summary, h_summary, all_layers_res = model_over_texts[text_ind]

        if text_ind==0:
            all_text_vals = all_layers_res
        else:
            all_text_vals = np.concatenate([all_text_vals, all_layers_res], axis=-1)

        #display(l_summary)
        #display(h_summary)

    plt.hist(all_text_vals[:,0,:,:].flatten(), bins=500, alpha=0.5, label=m, density=True)

    second_plot.append((all_text_vals[:,2,:,:].flatten(), all_text_vals[:,1,:,:].flatten()))

#plt.axvline(19.3,  linestyle='dashed',label=r"$\beta=19.3$", color='red')
plt.legend()
plt.title("Softmax Inputs Aggregated for all Inputs, Layers, Heads.")
plt.ylabel('Normalized Counts')
plt.xlabel('Inputs to softmax')
#plt.xlim(left=0)
max_x_val = np.max(all_text_vals[:,0,:,:].flatten()) # ignores a few outlier negative values that we don't care about.
plt.xlim(-max_x_val,max_x_val)
plt.gcf().savefig(directory+'AllModels|ValuesForAllTextsLayersHeads.png', dpi=250)
plt.show()

#------------------
plt.figure()
number_to_plot = 500000

for ind, els in enumerate(second_plot):
    x, y = els
    rand_inds = np.random.choice(np.arange(len(x)),number_to_plot, replace =False)
    plt.scatter(x[rand_inds],y[rand_inds] ,alpha=0.2, label=models[ind])
plt.axhline(1.0,  linestyle='dashed',label="$y=1.0$", color='red')

lgnd = plt.legend()
for lh in lgnd.legendHandles:
    lh.set_alpha(1)
plt.title("Value Vector Norms as a Function of Attention Weight \n for all Inputs, Layers, Heads.")
plt.ylabel('Value Vector $L^2$ Norm')
plt.xlabel('Attention weight')
plt.gcf().savefig(directory+'AllModels|ValueVectorAttentionScatterForAllTextsLayersHeads.png', dpi=250)
plt.show()

#------------------ Multiplying value vector by attention
plt.figure()
number_to_plot = 500000

for ind, els in enumerate(second_plot):
    x, y = els
    rand_inds = np.random.choice(np.arange(len(x)),number_to_plot, replace =False)
    plt.scatter(x[rand_inds],x[rand_inds]*y[rand_inds] ,alpha=0.2, label=models[ind])
xrang = np.linspace(0,1,100)
plt.plot(xrang, xrang, linestyle='dashed',label="$L^2=1$", color='red')
lgnd = plt.legend()
for lh in lgnd.legendHandles:
    lh.set_alpha(1)
plt.title("Value Vector $L^2$ Norms multiplied by Attention Weight \n for all Inputs, Layers, Heads.")
plt.ylabel('Value Vector $L^2$ Norm * Attention')
plt.xlabel('Attention weight')
plt.gcf().savefig(directory+'AllModels|ValueVectorTimesAttentionScatterForAllTextsLayersHeads.png', dpi=250)
plt.show()
```

```python

```


# Jupyter Notebooks Notes

## Table Of Content

- [Overview](#Overview)
- [Create Table of Content](#Create-Table-of-Content)
- [Jupyter Notebook Getting Started](#Jupyter-Notebook-Getting-Started)
    - [Installing Jupyter Notebooks](#Installing-Jupyter-Notebooks)
        - [Running Jutpiter Notebook](#Running-Jutpiter-Notebook)
        - [Uploading Notebook to Anconda Cloud](#Uploading-Notebook-to-Anconda-Cloud)
- [Keyboard Shortcuts](#Keyboard-Shortcuts)
- [Notebook Syntax](#Notebook-Syntax)
    - [Tables](#Tables)
    - [Images and Videos](#Images-and-Videos)
    - [LaTeX Equations](#LaTeX-Equations)
    - [Maxwell's Equations](#Maxwell%E2%80%99s-Equations)
- [Basic Magics](#Basic-Magics)
- [Local Custom Styles](#Local-Custom-Styles)
- [Global Custom Styles](#Global-Custom-Styles)

## Overview
The Jupyter Notebooks is an open-source web application that allows to create and share documents that contain live code, equations, visualizations and narrative text. This includes: data cleaning and transformation, numerical simulation, statistical modeling, data visualization, machine learning, and much more. The Jupyter Notebook app supports [Python](https://www.python.org/) and several other languages. For more information see [Jupyter](http://jupyter.org/).
Jupyter Notebooks is a brillant mechanism to keep notes and experiment with Python language. You can build snippets of code, annotate them, and execute them.

### Create Table of Content
To create a table of content for a Notebook, you can use the predifined header links and include them in the standard link construct such as [Maxwell's Equations](#Maxwell%E2%80%99s-Equations).
You can get the link value by mousing over the header for which you want to get the link and then right-click on the pilcrow ¶, also called the paragraph mark. From the dropdown menu select **Copy Link Address**.

From the link just copy the last part prefixed by the # sign and used it as the value in the standard link construct.

## Jupyter Notebook Getting Started
Jupyter Notebook is a powerful tool for interactively developing and presenting data science projects. A notebook integrates code and its output into a single document that combines visualizations, narrative text, mathematical equations, and other rich media. The intuitive workflow promotes iterative and rapid development, making notebooks an increasingly popular choice at the heart of contemporary data science, analysis, and increasingly science at large. Best of all, as part of the open source Project Jupyter, they are completely free, For more information, see [Jupyter Notebook for Beginners: A Tutorial](https://www.dataquest.io/blog/jupyter-notebook-tutorial/).

### What is an ipynb File?
It will be useful to understand what this file really is. Each .ipynb file is a text file that describes the contents of your notebook in a format called JSON. Each cell and its contents, including image attachments that have been converted into strings of text, is listed therein along with some metadata. You can edit this yourself — if you know what you are doing! — by selecting “Edit > Edit Notebook Metadata” from the menu bar in the notebook.

You can also view the contents of your notebook files by selecting “Edit” from the controls on the dashboard, but the keyword here is “can“; there’s no reason other than curiosity to do so unless you really know what you are doing.

### Installing Jupyter Notebooks
You can run Jupyter Notebook online at . In these notes we run Jupyter on a Mac laptop in the integrated enviroment called **Anaconda** or **Conda** for short. It is an open source package management system and environment management system that runs on Windows, macOS and Linux. Conda quickly installs, runs and updates packages and their dependencies. Conda easily creates, saves, loads and switches between environments on your local computer. It was created for Python programs, but it can package and distribute software for any language. It also comes with integrated Jupyter Notebook.
A suggested alternative to "stand-alone" Python is a package and environment mangement system such as [Conda](https://conda.io/docs/index.html). One big advantadge is the creation of **virtual enviroments** which do not intefer with the uderlying OS environment. The following are some of the Conda main advantadges:
- Quickly installs, runs and updates packages and their dependencies.
- Easily creates, saves, loads and switches between environments on your local computer. It was created for Python programs, but it can package and distribute software for any language.
- Conda as a package manager helps you find and install packages.
- If you need a package that requires a different version of Python, with just a few commands, you can set up a totally separate environment to run that different version of Python, while continuing to run your usual version of Python in your normal environment.
- That said, let's install Conda following the instructions found at [Downloading conda](https://conda.io/docs/user-guide/install/download.html). Install Anaconda Navigator, not Miniconda.
- I also signed for the [Anaconda Cloud](http://anaconda.org). Where packages, notebooks, projects and environments are shared.
- Please, see [notebooks-cheatsheet](https://www.cheatography.com/weidadeyue/cheat-sheets/jupyter-notebook/pdf_bw/) and [Getting started with Conda](https://conda.io/docs/user-guide/getting-started.html).

#### Running Jutpiter Notebook

1. Activate the **Anaconda Navigator**. The following UI is displayed:

1. Click the **Launch** button  Jupyter Notebook from the related button
The Jupyter Noteboook app will run on the local host on tha available port.
For example [http://localhost:8892/tree](http://localhost:8892/tree).
1. Select the Notebook.  Notice that Jupyter Notebook runs from the direct

#### Uploading Notebook to Anconda Cloud
To upload a notebbok file the cloud follow these steps:
1. Activate your terminal console.
1. Change the directory where the notebook file is.
1. Login into the [Anaconda Cloud](http://anaconda.org) cloud by executing this command: `anaconda login`
1. Upload the file by executing this command: `anaconda upload --force "filename.ipynb"`

## Keyboard Shortcuts
You can get a list of keybord shortcuts under the menu at the top: **Help > Keyboard Shortcuts**, or by pressing **H** in command mode.
Another way to access keyboard shortcuts, and a handy way to learn them is to use the command palette: **Cmd + Shift + P** (or **Ctrl + Shift + P** on Linux and Windows).

```python
# Example of code snippet
greetings = "hello world!"
print(greetings)
```

    hello world!

### Jupyter Notebook Keyboard Input Modes
The Jupyter Notebook has two different keyboard input modes.

1. **Edit mode** allows you to type code or text into a cell and is indicated by a **green cell border**.
2. **Command mode** binds the keyboard to notebook level commands and is indicated by a grey cell border with a blue left margin.

See also [28 Jupyter Notebook tips, tricks, and shortcuts](https://www.dataquest.io/blog/jupyter-notebook-tips-tricks-shortcuts/).

#### Jupyter Notebook Input Modes Keyboard Shortcuts
The following are the Mac OS X modifier keys you must be familiar with when using the shorcuts. See also [Making Sense of Mac Keyboard Symbols](http://osxdaily.com/2012/03/27/making-sense-of-mac-keyboard-symbols/).

| Symbol| Meaning|
|:----- |:-------|
| ⌘    | Command|
| ⌃    | Control|
| ⌥    | Option|
| ⇧    | Shift|
| ↩    | Return|
| ␣     | Space|
| ⇥    | Tab|

See
[Notebooks-Cheatsheet](https://www.cheatography.com/weidadeyue/cheat-sheets/jupyter-notebook/pdf_bw/)

## Notebook Syntax
As you have noticed this document is a **Jupyter Notebook**. You can find more information at this location: [The Jupyter Notebook](https://jupyter-notebook.readthedocs.io/en/stable/notebook.html) and plenty of examples at [Notebook Examples](https://jupyter-notebook.readthedocs.io/en/stable/examples/Notebook/examples_index.html).
Refer akso to [Markdown Cells](https://jupyter-notebook.readthedocs.io/en/stable/examples/Notebook/Working%20With%20Markdown%20Cells.html#Markdown-Cells).

### Tables
You can create tables with pipes | and hyphens -. Hyphens are used to create each column's header, while pipes separate each column. You must include a blank line before your table in order for it to correctly render.

| First Header  | Second Header |
| ------------- | ------------- |
|   Content Cell| Content Cell  |
| Content Cell  | Content Cell  |

The pipes on either end of the table are optional.

Cells can vary in width and do not need to be perfectly aligned within columns. There must be at least three hyphens in each column of the header row.

| Command | Description |
| --- | --- |
| git status | List all new or modified files |
| git diff | Show file differences that haven't been staged |

For more information, see [Organizing information with tables](https://help.github.com/articles/organizing-information-with-tables/).

### Images and Videos
The following are images and videos which are located in a directory or subdirectory on the machine where the notebook file resides.

|What|Media|
|----|-----|
|Vesuviana ||
|Facendo un 'cazze|<video controls src="https://docs.google.com/uc?export=download&id=151_iDpkc4TM8yoFoqnek0bdKIMGK_S65" width="300px" height="300px"/>|

If you want to share your notebook file then you must have the resources on a public server. The following is an example of an image located on the Google Drive.

#### Using Images Stored in Google Drive
These are the steps to make an image sharable so it can be used in a notebook.

1. In your Google Drive, create a public folder as follows:
    1. Create a new folder and name it *Image*, for instance.
    1. Right click the folder, just created and select **Share** from the drop-down menu.
    1. In the popup dialog window, click the **Advanced** link in the bottom right.
    1. In the section *Who has access* select *Public on the web - Anyone on the Internet can find an view*.
    1. Click **Done** button.
1. Store the image in the folder you just created.
1. Right click on the image, and from the drop down menu select **Share**.
1. Click the **Copy link** button. A link is copied in your clipboard.
1. From the link copy the long image ID made of numbers and letters.
1. Replace *imageID* in the following URL: https://docs.google.com/uc?export=download&id=<imageID> with the ID you copied in the previous step.
1. Copy this URL in the markdown image tag such as:
1. Or in the image tag and changing the width, for example:

#### Using Images Stored in GitHub

#### Using Images Stored Locally

### LaTeX Equations
Courtesy of MathJax, you can include mathematical expressions both inline and displayed.
See also [LaTex Equations](https://jupyter-notebook.readthedocs.io/en/stable/examples/Notebook/Working%20With%20Markdown%20Cells.html#LaTeX-equations).

$e^{i\pi} + 1 = 0$
$$e^x=\sum_{i=0}^\infty \frac{1}{i!}x^i$$

### Maxwell’s Equations
See, also [Motivating Examples](https://jupyter-notebook.readthedocs.io/en/stable/examples/Notebook/Typesetting%20Equations.html#The-Lorenz-Equations).
\begin{align}
\nabla \times \vec{\mathbf{B}} -\, \frac1c\, \frac{\partial\vec{\mathbf{E}}}{\partial t} & = \frac{4\pi}{c}\vec{\mathbf{j}} \\   \nabla \cdot \vec{\mathbf{E}} & = 4 \pi \rho \\
\nabla \times \vec{\mathbf{E}}\, +\, \frac1c\, \frac{\partial\vec{\mathbf{B}}}{\partial t} & = \vec{\mathbf{0}} \\
\nabla \cdot \vec{\mathbf{B}} & = 0
\end{align}

## Basic Magics
Magics are handy commands built into the IPython kernel that make it easier to perform particular tasks. Although they often resemble unix commands, under the hood they are all implemented in Python. There exist far more magics than it would make sense to cover, but it’s worth highlighting a variety of examples. We will start with a few basics before moving on to more interesting cases.

There are two categories of magic: **line magics** and **cell magics**. Respectively, they act on a single line or can be spread across multiple lines or entire cells. Quote and more borrowed from [Advanced Jupyter Notebooks Tutorial](https://www.dataquest.io/blog/advanced-jupyter-notebooks-tutorial/).

```python
# To see the available magics, you can execute the following command:
%lsmagic
```

    Available line magics:
    %alias  %alias_magic  %autocall  %automagic  %autosave  %bookmark  %cat  %cd  %clear  %colors  %config  %connect_info  %cp  %debug  %dhist  %dirs  %doctest_mode  %ed  %edit  %env  %gui  %hist  %history  %killbgscripts  %ldir  %less  %lf  %lk  %ll  %load  %load_ext  %loadpy  %logoff  %logon  %logstart  %logstate  %logstop  %ls  %lsmagic  %lx  %macro  %magic  %man  %matplotlib  %mkdir  %more  %mv  %notebook  %page  %pastebin  %pdb  %pdef  %pdoc  %pfile  %pinfo  %pinfo2  %popd  %pprint  %precision  %profile  %prun  %psearch  %psource  %pushd  %pwd  %pycat  %pylab  %qtconsole  %quickref  %recall  %rehashx  %reload_ext  %rep  %rerun  %reset  %reset_selective  %rm  %rmdir  %run  %save  %sc  %set_env  %store  %sx  %system  %tb  %time  %timeit  %unalias  %unload_ext  %who  %who_ls  %whos  %xdel  %xmode

    Available cell magics:
    %%!  %%HTML  %%SVG  %%bash  %%capture  %%debug  %%file  %%html  %%javascript  %%js  %%latex  %%markdown  %%perl  %%prun  %%pypy  %%python  %%python2  %%python3  %%ruby  %%script  %%sh  %%svg  %%sx  %%system  %%time  %%timeit  %%writefile

    Automagic is ON, % prefix IS NOT needed for line magics.

```python
%%HTML
<h5>Using HTML</h5>
This is <em>really</em> neat!
```

Using HTML

This is really neat!

```latex
%%latex
Some important equations:
$E = mc^2$
$e^{i pi} = -1$
```

Some important equations:
$E = mc^2$
$e^{i pi} = -1$

```python
# Autosave every 60 sec
%autosave 60
```

    Autosaving every 60 seconds

## Enhancing Charts with Seaborn
One of the most common exercises Jupyter Notebook users undertake is producing plots. But Matplotlib, Python’s most popular charting library, isn’t renowned for attractive results despite it’s customisability. Seaborn instantly prettifies Matplotlib plots and even adds some additional features pertinent to data science, making your reports prettier and your job easier. It’s included in the default Anaconda installation or easily installed via pip install seaborn.
Quote and more borrowed from [Advanced Jupyter Notebooks Tutorial](https://www.dataquest.io/blog/advanced-jupyter-notebooks-tutorial/).

```python
# First, we’ll import our libraries and load some data.
import matplotlib.pyplot as plt
import seaborn as sns
data = sns.load_dataset("tips")

data.head()
```

      total_bill
      tip
      sex
      smoker
      day
      time
      size

      0
      16.99
      1.01
      Female
      No
      Sun
      Dinner
      2

      1
      10.34
      1.66
      Male
      No
      Sun
      Dinner
      3

      2
      21.01
      3.50
      Male
      No
      Sun
      Dinner
      3

      3
      23.68
      3.31
      Male
      No
      Sun
      Dinner
      2

      4
      24.59
      3.61
      Female
      No
      Sun
      Dinner
      4

```python
# We can easily plot total_bill vs tip in Matplotlib.
plt.scatter(data.total_bill, data.tip);
```

```python
# Plotting in Seaborn is just as easy!
# Seaborn is closely integrated with pandas data structures,
# its own scatter plot function unlocks additional features.
sns.scatterplot(x="total_bill", y="tip", data=data);

```

```python
# Let’s color by the size of the party paying the bill while also
# discriminating between smokers and non-smokers.
sns.scatterplot(x="total_bill", y="tip", hue="size", style="smoker", data=data);
```

## Custom Styles
The styles shown in this section work on your local machine, 'cause the style *custom.css* file is local, in the case of Jupyter Notebooks. The file is located at
/Users/user name/.jupyter/custom/custom.css on the Mac computer.  But they do not work if the file is on a public server such as Anaconda cloud or GitHub. To solve the problem we make the *custom.css* file available globally on GitHub and run a script similar to the following in the markdown file:

```
from IPython.core.display import HTML

def set_css_style(css_file_path):
    """
    Read the custom CSS file and load it into Jupyter.
    Pass the file path to the CSS file.
    """
    styles = open(css_file_path, "r").read()
    return HTML(styles)
set_css_style('../Css/custom.css')

```

### Style Examples

#### Markers
***
Danger
Warning
Success
Info
Other

#### Notes
***

Danger

Warning

Success

Info

Other

from IPython.core.display import HTML

```python
from IPython.core.display import HTML

def set_css_style(css_file_path):
    """
    Read the custom CSS file and load it into Jupyter.
    Pass the file path to the CSS file.
    """
    styles = open(css_file_path, "r").read()
    return HTML(styles)
set_css_style('../Css/custom.css')
```

# Mac Notes
[Making Sense of Mac Keyboard Symbols](http://osxdaily.com/2012/03/27/making-sense-of-mac-keyboard-symbols/)

## Password Protected File
1. Open the file.
1. In the File menu select **Save As**.
1. In the popup dialog window, check **Save with password** box.
1. Click the **Replace** button.
1. In the popup dialog enteer the password you want to use.

Write down the password and keep it an a safe place.

# References
- [Jupyter Notebook Documentation](https://jupyter-notebook.readthedocs.io/en/stable/)
- [Markdown Guide](http://markdown-guide.readthedocs.io/en/latest/index.html)
- [28 Jupyter Notebook tips, tricks and shortcuts](https://www.dataquest.io/blog/jupyter-notebook-tips-tricks-shortcuts/)
- [Jupyter Notebook Tutorial: The Definitve Guide](https://www.datacamp.com/community/tutorials/tutorial-jupyter-notebook)
- [Making Sense of Mac Keyboard Symbols](http://osxdaily.com/2012/03/27/making-sense-of-mac-keyboard-symbols/)
- [3d surface plot in Plotly](https://nbviewer.jupyter.org/gist/jackparmer/8d7e979bc4cec23db057)

more .....


**This is not under MIT license.** It is intended for testing and might contain proprietary code.

```python
import exercises
from qa_display_widgets import FillGapsBox

exerc = exercises.SolveForXLinear(2)
FillGapsBox(exerc).box()
```

```python
import main
main.SolveForXLinear(3).display()
```

```python
import ipywidgets as widgets
import traitlets
from IPython.display import display
```

```python
import main
import sympy
from IPython.display import Math
sympy.init_printing()
```

```python
from ipywidgets import Layout, Button, Box

items_layout = Layout(flex='1 1 auto',
                      width='auto')     # override the default width of the button to 'auto' to let the button grow

box_layout = Layout(display='flex',
                    flex_flow='column',
                    align_items='stretch',
                    border='solid',
                    width='50%')

words = ['correct', 'horse', 'battery', 'staple']
items = [Button(description=w, layout=items_layout, button_style='danger') for w in words]
box = Box(children=items, layout=box_layout)
box
```

```python
from ipywidgets import Layout, Button, Box

items_layout = Layout(width='auto')     # override the default width of the button to 'auto' to let the button grow

box_layout = Layout(display='flex',
                    flex_flow='column',
                    align_items='stretch',
                    border='solid',
                    width='50%')

words = ['correct', 'horse', 'battery', 'staple']
items = [Button(description=w, layout=items_layout, button_style='danger') for w in words]
box = Box(children=items, layout=box_layout)
box
```

```python
b1 = Button(description='My button')
b1.style.background = '/home/blue/PycharmProjects/ipy_student_exercises/own_images/simple_grey_button.png'
b1
```

```python
from IPython.display import HTML
HTML(""" <style>
.widget-button {
  background-image: ;
  background-position: 50% 50%;
  background-size: cover;
}
</style> """)
```

 <style>
.widget-button {
  background-image: ;
  background-position: 50% 50%;
  background-size: cover;
}
</style>

```python
from ipywidgets import Layout, Button, Box

items = [
    Button(description='1'),
    Button(description='2', layout=Layout(flex='0.1 1 auto')),
    Button(description='1'),
 ]

box_layout = Layout(display='flex',
                    flex_flow='row',
                    align_items='stretch',
                    border='solid',
                    width='100%')
box = Box(children=items, layout=box_layout)
box
```