```python
%load_ext notexbook
%texify
```

<style type="text/css">
    /*
Author: Valerio Maggio < @leriomaggio >
Code: https://github.com/leriomaggio/notexbook-jupyter-theme
License: Apache License 2.0
*/

/*
Author: Valerio Maggio < @leriomaggio >
Code: https://github.com/leriomaggio/notexbook-jupyter-theme
License: Apache License 2.0

 ======================================================
                         FONTS

    FONT WEIGHT MAP:
    ----------------
    100: Ultra Light
    200: Thin
    300: Light
    400: Regular (normal)
    500: Medium
    600: Semi-bold
    700: Bold
    800: Heavy
    900: Black

    Font faces:
        - ====================================|=======================
        -              Scope                  | Font Family Name
        - ====================================| ======================
        - Markdown Display (Computer Modern)    (CMU Serif)
        - Markdown Display monospace            (CMU Typewriter Text)
        - Markdown Edit    monospace            (Hack)
        - Source Code      monospace            (Fira Code)
        - ====================================| =======================

 ======================================================  */

/* Roboto Slab */
@import url("https://fonts.googleapis.com/css2?family=Roboto+Slab:wght@100;200;300;400;500;600;700;800;900&display=swap");
/*md-display-computer-modern --> CMU Serif*/
@import url("https://fonts.cdnfonts.com/css/cmu-serif?styles=30038,30039,30037,30036");
/*md-display-monospace --> CMU Typewriter Text*/
@import url("https://fonts.cdnfonts.com/css/cmu-typewriter-text?styles=24833,24829,24830,24831,24832,24834");
/*code-monospace --> Fira Code*/
@import url("https://fonts.googleapis.com/css2?family=Fira+Code:wght@300;400;500&display=swap");
/*md-edit-monospace --> Hack*/
@import url("https://fonts.cdnfonts.com/css/hack?styles=20708,20707,20705,20706");
/*
Author: Valerio Maggio < @leriomaggio >
Code: https://github.com/leriomaggio/notexbook-jupyter-theme
License: Apache License 2.0

======================================================

    ---------------------------------
    Code Mirror - Code Cell Highlight
    ---------------------------------

    Define the Style for Code **and** Markdown editors (Themes)
    Theme files are all located in the "themes" folder, and the
    different styles for the Code and Markdown editors are
    distinguished by a common prefix in the file names:

        --> Code editors Themes: "themes/code_*"
        --> Markdown Editor Themes: "themes/md_*"

    Styles can be imported as separate CSS modules.

    Current stylesheet (editor.css) defines the CSS rules
    for notebook tags and classes. These rules are
    all based to colors defined in external Theme files.

    This would ease the definition of other custom CSS Editor themes
*/

/* Code Editor theme */

    /*
Author: Valerio Maggio < @leriomaggio >
Code: https://github.com/leriomaggio/notexbook-jupyter-theme
License: Apache License 2.0

 ======================================================

The color palette is inspired by Light
 Material Design theme
Repo: https://github.com/JonaDuran/Material-Light-Theme/
*/

:root {
    /* Color Palette */
    --ml-white: #ffffff;
    --ml-light-white: #FAFAFA;
    --ml-black: #24292E;
    --ml-dark-blue : #01579B11;
    --ml-dark-blue-2: #01579B22;
    --ml-blue: #1565C0;
    --ml-green: #2E7D32;
    --ml-yellow: #A8601A;
    --ml-cyan: #00838f;
    --ml-magenta: #9C00B0;
    --ml-red: #C0392B;
    --ml-grey: #9E9E9E;
    --ml-light-blue: #78909c;

    /* Editor Theme */
    --code-background-color: var(--ml-light-white);
    --gutter-background: var(--ml-light-white);
    --selection-background-color: var(--ml-dark-blue-2);
    --line-numbers: var(--ml-grey);
    --cursor: var(--ml-black);
    --bracket: var(--ml-black);
    --matching-bracket: var(--ml-blue);
    --code-text-color: var(--ml-black);
    --keywords: var(--ml-magenta);
    --types: var(--ml-magenta);
    --variables: var(--code-text-color);
    --variables2: var(--ml-green);
    --def: var(--ml-blue);
    --property: var(--ml-blue);
    --meta: var(--ml-light-blue);
    --builtin: var(--ml-blue);
    --attribute: var(--ml-blue);
    --strings: var(--ml-yellow);
    --strings2: var(--ml-grey);
    --comments: var(--ml-grey);
    --operator: var(--ml-magenta);
    --numbers: var(--ml-red);
    /* Dataframe */
    --dataframe: var(--code-text-color);
    --df-bg: var(--ml-light-white);
    --df-thead: var(--ml-blue);
    --df-thead-border: var(--ml-black);
    --df-tr-hover: var(--selection-background-color);
    --df-border: var(--ml-white);
    --df-border-right: var(--ml-grey);
    --df-th-bg: var(--ml-white);
    /*ANSI Colours*/
    --ansi-red: var(--ml-red);
    --ansi-green: var(--ml-green);
    --ansi-green-intense: var(--ml-grey);
    --ansi-cyan: var(--ml-cyan);
    --ansi-blue: var(--ml-blue);
}

/* Markdown Editor theme */

    /*
Author: Valerio Maggio < @leriomaggio >
Code: https://github.com/leriomaggio/notexbook-jupyter-theme
License: Apache License 2.0

 ======================================================

    Custom Theme for Markdown Editor
    based on Material Clear Theme.

 */

:root {
    /* Color Palette */
    --md-ml-red: #C0392B;
    --md-ml-yellow: #A8601A;
    --md-ml-black: #24292E;
    --md-ml-grey: #9E9E9E;
    --md-ml-blue: #1565C0;
    --md-ml-dark-blue: #01579B;
    --md-ml-light-blue: #78909c;

    /*  Editor Theme  */
    --editor-text: var(--md-ml-black);
    --header: var(--md-ml-dark-blue);
    --quote: var(--md-ml-grey);
    --link: var(--md-ml-blue);
    --attribute: var(--md-ml-red);
    --tag: var(--md-ml-yellow);
    --string: var(--md-ml-yellow);
    --delimiter: var(--md-ml-black);
    --monospace: var(--md-ml-light-blue);
}

/*
Author: Valerio Maggio < @leriomaggio >
Code: https://github.com/leriomaggio/notexbook-jupyter-theme
License: Apache License 2.0
*/

/*
    =================
    Code Editor Theme
    =================
*/
div.output_error pre,
div.output_result pre,
div.output_stream pre {
    color: var(--code-text-color) !important;
}

.cm-s-ipython.CodeMirror,
.cm-s-jupyter.CodeMirror {
    background: var(--code-background-color);
    color: var(--code-text-color);
}

.cm-s-ipython div.CodeMirror-selected,
.cm-s-jupyter div.CodeMirror-selected {
    cursor: pointer;
    background: var(--selection-background-color);
}

.cm-s-ipython .CodeMirror-gutters,
.cm-s-jupyter .CodeMirror-gutters {
    background: var(--gutter-background);
    border-right: 0;
}

.cm-s-ipython .CodeMirror-linenumber,
.cm-s-jupyter .CodeMirror-linenumber {
    color: var(--line-numbers);
}

.cm-s-ipython .CodeMirror-cursor,
.cm-s-jupyter .CodeMirror-cursor {
    border-left: 1px solid var(--cursor) !important;
}

.cm-s-ipython span.cm-bracket,
.cm-s-jupyter span.cm-bracket {
    /*color: #828282;*/
    color: var(--bracket);
}

span.CodeMirror-matchingbracket {
    text-decoration: underline !important;
    text-decoration-color: var(--matching-bracket) !important;
    color: var(--matching-bracket) !important;
}

.cm-s-ipython span.cm-keyword,
.cm-s-jupyter span.cm-keyword {
    color: var(--keywords);
    font-weight: 400;
}

.cm-s-ipython span.cm-variable,
.cm-s-jupyter span.cm-variable {
    color: var(--variables);
}

.cm-s-ipython span.cm-variable-2,
.cm-s-jupyter span.cm-variable-2 {
    color: var(--variables2);
}

.cm-s-ipython span.cm-def,
.cm-s-jupyter span.cm-def {
    color: var(--def);
}

.cm-s-ipython span.cm-property,
.cm-s-jupyter span.cm-property {
    color: var(--property);
}

.cm-s-ipython span.cm-meta,
.cm-s-jupyter span.cm-meta {
    color: var(--meta);
}

.cm-s-ipython span.cm-attribute,
.cm-s-jupyter span.cm-attribute {
    color: var(--attribute);
}

.cm-s-ipython span.cm-builtin,
.cm-s-jupyter span.cm-builtin {
    color: var(--builtin)
}

.cm-s-ipython span.cm-string,
.cm-s-jupyter span.cm-string {
    color: var(--strings);
}

.cm-s-ipython span.cm-string-2,
.cm-s-jupyter span.cm-string-2 {
    color: var(--strings2);
}

.cm-s-ipython span.cm-comment,
.cm-s-jupyter span.cm-comment {
    color: var(--comments);
    font-style: normal;
}

.cm-s-ipython span.cm-operator,
.cm-s-jupyter span.cm-operator {
    color: var(--operator);
    font-weight: 400;
}

.cm-s-ipython span.cm-number,
.cm-s-jupyter span.cm-number {
    color: var(--numbers);
}

.cm-s-ipython span.cm-type,
.cm-s-jupyter span.cm-type {
    color: var(--types);
}

/*
==============================
Markdown MathJax Customisation
==============================
*/

/*This customisation only applies to Math Display showing the font bigger than normal*/
.MathJax_Display {
    font-size: 2rem;
}

.MathJax_Display .mjx-char {
    font-family: var(--txbk-content-font-family) !important;
}

.MathJax_Display, .MathJax span {
    color: var(--code-text-color);
}

a .MathJax span {
    color: var(--link-color);
}

a:hover .MathJax span {
    text-decoration: underline;
}

.MathJax span[style*="STIXMathJax_Normal"],
.MathJax span[style*="STIXMathJax_Normal-italic"],
.MathJax span[style*="STIXMathJax_Main"],
.MathJax span[style*="STIXMathJax_Variants"] {
    font-family: var(--txbk-content-font-family) !important;
}

/*
    Pygments CSS (replacement) for HTML export
    ===========================================

    Original Pygments CSS rewrite thanks to @rubik
    https://github.com/jupyter/nbconvert/issues/447#issuecomment-270766965
*/

.highlight .hll, div.highlight > pre {
    background-color: var(--code-background-color);
    color: var(--code-text-color);
    font-family: var(--txbk-code-font-family);
    font-size: var(--txbk-code-font-size);
    line-height: var(--txbk-code-line-height);
}

/* Comment */
.highlight .c {
    color: var(--comments);
}

/* Error */
.highlight .err {
    color: #960050;
    background-color: #1e0010;
}

/* Keyword */
.highlight .k {
    color: var(--keywords);
    font-weight: 400;
}

/* Literal */
.highlight .l {
    color: var(--numbers);
}

/* Name */
.highlight .n {
    color: var(--code-text-color);
}

/* HACK:
fix Interpreter mismatch pygments vs codemirror
mpl.rcParams <- rcParams will be highlighted as property (as in codemirror)
*/
.highlight .o + .n {
    color: var(--property);
}

/* Operator */
.highlight .o {
    color: var(--code-text-color);
    font-weight: 400;
}

/* Punctuation */
.highlight .p {
    color: var(--code-text-color);
}

/* Comment.Multiline */
.highlight .cm {
    color: var(--comments);
    font-style: normal;
}

/* Comment.Preproc */
.highlight .cp {
    color: var(--meta);
    font-style: normal;
}

.highlight .cpf {
    color: var(--meta);
    font-style: normal;
}

/* Comment.Single */
.highlight .c1 {
    color: var(--comments);
    font-style: normal;
}

/* Comment.Special */
.highlight .cs {
    color: var(--comments);
    font-style: normal;
}

/* @ Generic.Deleted */
.highlight .gd {
    color: var(--red);
}

/* Generic.Emph */
.highlight .ge {
    font-style: italic
}

/* @ Generic.Inserted */
.highlight .gi {
    color: #a6e22e
}

/* Generic.Strong */
.highlight .gs {
    font-weight: 500 !important;
}

/* @ Generic.Subheading */
.highlight .gu {
    color: #75715e
}

/* */

/* Keyword.Constant */
.highlight .kc {
    color: var(--keywords);
    font-weight: 400;
}

/* Keyword.Declaration */
.highlight .kd {
    color: var(--keywords);
    font-weight: 400;
}

/* Keyword.Namespace */
.highlight .kn {
    color: var(--keywords);
    font-weight: 400;
}

/* Keyword.Pseudo */
.highlight .kp {
    color: var(--keywords);
    font-weight: 400;
}

/* Keyword.Reserved */
.highlight .kr {
    color: var(--keywords);
    font-weight: 400;
}

/* Keyword.Type */
.highlight .kt {
    color: var(--types);
    font-weight: 400;
}

/* Literal.Date */
.highlight .ld {
    color: var(--numbers)
}

/* Literal.Number */
.highlight .m {
    color: var(--numbers);
}

/* Literal.String */
.highlight .s {
    color: var(--string);
}

/* Name.Attribute */
.highlight .na {
    color: var(--property);
}

/* Name.Builtin */
.highlight .nb {
    color: var(--builtin);
}

/* Name.Class */
.highlight .nc {
    color: var(--def);
    font-weight: normal !important;
}

/* @ Name.Constant */
.highlight .no {
    color: var(--variables2);
}

/* Name.Decorator */
.highlight .nd {
    color: var(--builtin);
}

/* @ Name.Entity */
.highlight .ni {
    color: var(--variables2);
}

/* Name.Exception */
.highlight .ne {
    color: var(--code-text-color);
    font-weight: normal !important;
}

/* Name.Function */
.highlight .nf, .highlight .fm {
    color: var(--def);
}

/* @ Name.Label */
.highlight .nl {
    color: var(--comments);
}

/* Name.Namespace */
.highlight .nn {
    color: var(--code-text-color);
    font-weight: normal !important;
}

/* @ Name.Other */
.highlight .nx {
    color: var(--code-text-color);
}

/* @ Name.Property */
.highlight .py {
    color: var(--property);
}

/* @ Name.Tag */
.highlight .nt {
    color: var(--tag);
}

/* @ Name.Variable */
.highlight .nv {
    color:  var(--variables);
}

/* Operator.Word */
.highlight .ow {
    color: var(--operator);
    font-weight: 400;
}

/* @ Text.Whitespace */
.highlight .w {
    color: var(--code-background-color);

}

/* Literal.Number.Bin */
/* Literal.Number.Float */
/* Literal.Number.Hex */
/* Literal.Number.Integer */
/* Literal.Number.Oct */
.highlight .mb,
.highlight .mf,
.highlight .mh,
.highlight .mi,
.highlight .mo {
    color: var(--numbers);
}

/* Literal.String.Backtick */
.highlight .sb {
    color: var(--strings2);
}

.highlight .sc,  /* Literal.String.Char */
.highlight .sd,  /* Literal.String.Doc */
.highlight .s2,  /* Literal.String.Double */
.highlight .sh,  /* Literal.String.Heredoc */
.highlight .si,  /* Literal.String.Interpol */
.highlight .sx,  /* Literal.String.Other */
.highlight .sr,  /* Literal.String.Regex */
.highlight .s1,  /* Literal.String.Single */
.highlight .ss  /* Literal.String.Symbol */
{
    color: var(--strings);
    font-style: normal;
}

/* Literal.String.Escape */
.highlight .se {
    color: var(--red);
}

/* Name.Builtin.Pseudo */
.highlight .bp {
    color: var(--variables2);
}

/* Name.Variable.Class */
.highlight .vc {
    color: var(--variables);
}

/* Name.Variable.Global */
.highlight .vg {
    color: var(--variables2);
}

/* Name.Variable.Instance */
.highlight .vi {
    color: var(--code-text-color);
}

/* Literal.Number.Integer.Long */
.highlight .il {
    color: var(--numbers);
}

/*
    Dataframe Colors
    ----------------
    Adapt Dataframe table to comply with the theme
*/

table.dataframe {
    color: var(--dataframe) !important;
}

table.dataframe tbody tr th {
    background-color: var(--df-bg);
}

table.dataframe tbody tr:hover {
    background-color: var(--df-tr-hover);
}

table.dataframe td, table.dataframe th {
    border: 1px solid var(--df-border);
}

table.dataframe > th:not(:empty) {
    background-color: var(--df-th-bg);
}

table.dataframe tr:nth-child(2) th:empty,
table.dataframe tr:nth-child(2) th:empty {
    border-right: 1px dotted var(--df-border-right);
}

table.dataframe thead tr th:not(:empty) {
    color: var(--df-thead);
    border-bottom:  1px solid var(--df-thead-border);
}

/* =======================================
              ANSI colors
    (stdout / stderr color customisation)
   =======================================
*/

span.ansi-green-fg {
    color: var(--ansi-green) !important;
}

span.ansi-green-intense-fg{
    color: var(--ansi-green-intense) !important;
}

span.ansi-red-fg {
    color: var(--ansi-red) !important;
}

span.ansi-cyan-fg {
    color: var(--ansi-cyan) !important;
}

span.ansi-blue-fg {
    color: var(--ansi-blue) !important;
}

span.ansi-bold {
    font-weight: 500 !important;
}

/* ----------------------------------- */

/*
    ======================
    Markdown Editor Theme
    ======================
*/

div.text_cell.unrendered pre {
    color: var(--editor-text) !important;
}

span.cm-header {
    color: var(--header) !important;
    font-weight: 500;
}

span.cm-quote {
    color: var(--quote) !important;
}

span.cm-string.cm-url {
    color: var(--link) !important;
}

span.cm-string {
   color: var(--strings) !important;
}
/* Sometimes useful in Mono code Highlighting*/
span.cm-string-2 {
    color: var(--strings2) !important;
}

span.cm-link {
    color: var(--link) !important;
}

span.cm-attribute {
    color: var(--attribute) !important;
}

span.cm-tag {
    color: var(--tag) !important;
}

span.cm-delimiter {
    color: var(--delimiter) !important;
}

span.cm-comment {
    color: var(--comments) !important;
}
/*
Author: Valerio Maggio < @leriomaggio >
Code: https://github.com/leriomaggio/notexbook-jupyter-theme
License: Apache License 2.0

   ---------------------------------------
    CODE CELL OUTPUT (DISPLAY MODE)

    Pandas DataFrame - as HTML Table
        (includes also HTML tables)

   Note: This stylesheet does NOT include
   any color nor background color for tables,
   as those are assumed to be part of the
   Coding colourisation theme.

   ---------------------------------------
*/
table.dataframe {
    border-collapse: collapse;
    border: none;
    margin-left: 20px !important;
    font-size: var(--txbk-code-font-size) !important;
}

table.dataframe thead {
    padding-bottom: 10px;
}

table.dataframe thead tr {
    font-style: normal;
    padding: 5px 10px;
    border-bottom:  1px solid;
    vertical-align: middle;
    text-align: center;
    empty-cells: hide;
}

table.dataframe thead tr th,
table.dataframe thead tr:only-child th {
    vertical-align: middle;
    text-align: center;
}

table.dataframe tbody {
    padding-top:  5px;
}

table.dataframe tbody tr {
}

table.dataframe tbody tr th {
    text-align: left !important;
    font-style: italic;
    font-weight: normal;
    margin-right: 5px;
}

table.dataframe tbody tr td {
    padding-left: 1.0rem;
    padding-right: 1.0rem;
}

table.dataframe tr {
    border: none;
}

table.dataframe td,
table.dataframe th {
    padding-left: 0.25em;
    padding-right: 0.25em;
}

table.dataframe > th:not(:empty) {
    text-align: left;
    padding: 0 10px;
    font-style: italic
}

table.dataframe tr:nth-child(2) th:empty,
table.dataframe tr:nth-child(2) th:empty {
    border-left: none;
}

table.dataframe td {
    padding: 0.375em 1em;
}

table.dataframe thead {
	padding: 10px 0;
	font-weight: bold;
}

table.dataframe thead tr th:not(:empty) {
    text-align: left;
    font-style: normal;
    padding: 5px 10px;
}

/* Hacking Font-weight bold capped at 500 */
.rendered_html th {
    font-weight: 500 !important;
}
:root {
    /* Jupyter Lab Integration (Light Theme) */

    --jp-ui-font-family: var(--txbk-ui-font-family);
    --jp-ui-font-color1: var(--txbk-ui-color);
    --jp-ui-font-size1: var(--txbk-ui-font-size);

    --jp-content-font-color1: var(--txbk-content-mono-color);
    --jp-content-font-family: var(--txbk-content-font-family);
    --jp-content-line-height: var(--txbk-content-line-height);
    --jp-content-link-color: var(--link-color);
    --jp-content-font-size: var(--txbk-content-font-size);

    --jp-layout-color3: #efefef;
    --jp-notebook-multiselected-color: var(--txbk-multiselect-bgcolor);

    --jp-code-font-size: var(--txbk-code-font-size);
    --jp-code-line-height: var(--txbk-ui-mono-line-height);
    --jp-code-padding: 5px;
    --jp-code-font-family-default: var(--txbk-code-font-family);
    --jp-code-font-family: var(--jp-code-font-family-default);
    --jp-code-presentation-font-size: var(--txbk-ui-mono-font-size);
    --jp-content-heading-font-weight: 500;
    --jp-content-heading-line-height: 1;

    --jp-cell-prompt-font-family: var(--txbk-code-font-family);
    --jp-cell-prompt-width: 80px;

    --jp-cell-editor-border-color: transparent;

    --jp-layout-color0: var(--code-background-color);
    --jp-cell-prompt-not-active-font-color: #aba9a9;

    --jp-cell-editor-background: var(--code-background-color);
    --jp-cell-editor-active-background: var(--code-background-color);
    --jp-cell-editor-active-border-color: var(--txbk-cell-edit-border-color);

}

/* ================
    Jupyter Lab UI
   ================
*/

.jp-DirListing-item.jp-mod-running .jp-DirListing-itemIcon:before {
    color: var(--btn-danger);
}

.jp-DirListing-item.jp-mod-selected {
    background-color: var(--texbook-blue);
    color: white;
}

button.jp-RunningSessions-shutdownAll.jp-mod-styled {
    color: var(--btn-warning)
}

button.jp-RunningSessions-shutdownAll.jp-mod-styled {
    color: var(--btn-warning);
}

.jp-icon-warn0[fill] {
    fill: var(--texbook-turquoise);
}

#jp-MainLogo .jp-icon-warn0[fill] {
    /* Set default not to override logo colour */
    fill: var(--jp-warn-color0);
}

/* Change Colour only of Icons on the left-hand side toolbar */
.jp-SideBar.lm-TabBar.jp-mod-left .jp-icon3[fill] {
    fill: var(--texbook-yellow);
}

.jp-SideBar.lm-TabBar.jp-mod-left.lm-TabBar-tab.lm-mod-current .jp-icon3[fill] {
    fill: var(--texbook-orangex);
}

/*Kernel*/
div.f14jk5uf[title="Kernel Busy"] .jp-icon3[fill] {
    fill: var(--kernel-busy);
}

div.f14jk5uf[title="Kernel Idle"] .jp-icon3[fill] {
    fill: var(--kernel-idle);
}

div.f14jk5uf[title="Kernel Unknown"] .jp-icon3[fill],
div.f14jk5uf[title="Kernel Restarting"] .jp-icon3[fill] {
    fill: var(--kernel-disconnected);
}

/* body class*/

body[data-jp-theme-light="true"]:not(.jp-Notebook) {
    font-family: var(--jp-ui-font-family);
    background: var(--jp-layout-color3) !important;
    margin: 0;
    padding: 0;
    overflow: hidden;
}

.jp-Notebook {
    background-color: var(--background-color) !important;
    font-weight: 300;
    color: var(--txbk-ui-color);
    font-size: var(--txbk-ui-font-size);
}

/* -------------------------------------------------
    HTML Cells output generics
    (overriding default JupyterLab theme settings)
   -------------------------------------------------
*/

.jp-RenderedHTMLCommon :not(pre) > code {
    background-color: transparent !important;
    padding: 0 0 !important;
}

.jp-RenderedHTMLCommon pre, .jp-RenderedHTMLCommon code {
    background-color: transparent !important;
}

.jp-RenderedHTMLCommon p {
    margin-bottom: 0;
}

.jp-RenderedHTMLCommon * + p {
    margin-top: 1em;
}

/* ======================================
    MARKDOWN CELLS DISPLAY MODE
    (text_cell_render in Notebook HTML)
   ======================================
*/

.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon {
    font-family: var(--txbk-content-font-family) !important;
    font-size: var(--txbk-content-font-size) !important;
    color: var(--txbk-content-color) !important;
    font-kerning: auto;
    text-align: justify !important;
    display: block;
    word-break: normal !important;
    word-wrap: break-word !important;
}

.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon a {
    color: var(--link-color) !important;
}

.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon h1,
.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon h2,
.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon h3,
.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon h4,
.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon h5,
.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon h6 {
    font-family:  var(--txbk-content-font-family) !important;
    font-style: normal !important;
}

/*  Font-sizes for Headers
    ----------------------*/
.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon h1 {
    font-size: 2.2em !important;
    text-align: center !important;
    padding-top:  3rem;
    padding-bottom: 3rem;
}

.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon h2 {
    font-size: 1.9em !important;
}

.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon h3 {
    font-size: 1.7em !important;
}

.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon h4 {
    font-size: 1.5em !important;
}

.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon h5 {
    font-size: 1.3em !important;
    border-bottom: 1px solid rgb(204, 204, 204);
}

.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon h6 {
    font-size: 1.1em !important;
}

.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon p,
.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon pre,
.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon code {
    font-size: 1em !important;
}

.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon pre,
.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon code {
    color: var(--txbk-content-mono-color);
    font-family: var(--txbk-content-mono-font-family);
}

/* Monospace Code - no code syntax highlight*/
.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon pre {
    /* hack: using code background colour here to allow for compatibility */
    background-color: var(--code-background-color) !important;
    font-family: var(--txbk-content-mono-font-family);
}

.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon p {
    line-height: var(--txbk-content-line-height);
    text-align: justify !important;
}

/* Monospace inline rendered markdown */
.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon p code {
    /*Re-defining variables for Mono using Code Editor's settings*/
    font-size: var(--txbk-content-mono-font-size);
    color: var(--txbk-content-mono-color);
    font-family: var(--txbk-content-mono-font-family);
}

.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon p strong code {
    font-weight: bold !important;
    color: var(--txbk-content-mono-color);
}

.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon p em code {
    font-style: italic !important;
    color: var(--txbk-content-mono-color);
}

.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon pre code {
    padding: 1rem 1.5rem;
    display: block;
    color: var(--txbk-content-mono-color);
    font-family: var(--txbk-content-mono-font-family);
    word-wrap: break-word;
    word-break: normal;
    /* Important Override rules */
    background-color: var(--txbk-content-mono-bgcolor) !important;
    font-size: var(--txbk-content-mono-font-size) !important;
}

.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon blockquote {
    --txbk-ui-color: #6f6f6f;
    margin: .2rem .2rem;
    padding: .3rem .5rem;
    border-left: .3rem solid #a7a7a7;
    background-color: var(--txbk-content-mono-bgcolor);
}

.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon blockquote:not(:first-child) {
    margin: 2rem .2rem;
}

.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon blockquote p {
    padding: .2rem 1.5rem;
}

.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon table {
    font-size: var(--txbk-content-font-size) !important;
    border-collapse: collapse;
    border-spacing: 0;
    overflow: auto;
    break-inside: auto;
    margin-top: 25px;
    margin-bottom: 15px;
}

.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon table thead {
    display: table-header-group;
    vertical-align: middle;
    border-bottom: none;
}

.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon table thead tr,
.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon table thead tr td,
.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon table tbody tr,
.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon table tbody tr th,
.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon table tbody tr td {
    break-inside: avoid;
    break-after: auto;
    vertical-align: middle;
    padding: 5px 5px;
}

.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon table tr th {
    border-bottom-width: 0;
}

.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon ul,
.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon ol {
    position: relative;
    display: block;
    margin-bottom: 0 !important;
    line-height: var(--txbk-content-line-height);
}

.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon ul {
    list-style-type: disc;
}

.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon ul li,
.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon ol li {
    /*white-space: pre-wrap;*/
    padding: .6rem !important;
}

.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon ul ul {
    list-style-type: circle !important;
}

.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon ul ul,
.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon ul ol,
.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon ol ul,
.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon ol ol {
    padding-top: .3rem !important;
}

.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon ul li > ul li,
.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon ul li > ol li,
.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon ol li > ul li,
.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon ol li > ol li {
    padding: .3rem !important;
}

.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon ul li > ul li a code,
.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon ul li > ol li a code,
.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon ol li > ul li a code,
.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon ol li > ol li a code {
    /*font-size: .7em !important;*/
    color: var(--link-color);
}

.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon ul ul ul {
    list-style-type: square !important;
}

.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon p > img:only-child,
.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon img:only-child {
    display: block;
    margin: auto;
    max-width: 40%;
}

.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon img {
    max-width: 40%;
}

/* Img MaxW classes */
.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon img.maxw10,
.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon p > img.maxw10 {
    max-width: 10% !important;
}

.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon img.maxw15,
.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon p > img.maxw15 {
    max-width: 15% !important;
}

.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon img.maxw20,
.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon p > img.maxw20 {
    max-width: 20% !important;
}

.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon img.maxw25,
.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon p > img.maxw25 {
    max-width: 25% !important;
}

.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon img.maxw30,
.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon p > img.maxw30 {
    max-width: 30% !important;
}

.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon img.maxw35,
.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon p > img.maxw35 {
    max-width: 35% !important;
}

.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon img.maxw40,
.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon p > img.maxw40 {
    max-width: 40% !important;
}

.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon img.maxw45,
.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon p > img.maxw45 {
    max-width: 45% !important;
}

.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon img.maxw50,
.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon p > img.maxw50 {
    max-width: 50% !important;
}

.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon img.maxw55,
.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon p > img.maxw55 {
    max-width: 55% !important;
}

.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon img.maxw60,
.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon p > img.maxw60 {
    max-width: 60% !important;
}

.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon img.maxw65,
.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon p > img.maxw65 {
    max-width: 65% !important;
}

.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon img.maxw70,
.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon p > img.maxw70 {
    max-width: 70% !important;
}

.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon img.maxw75,
.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon p > img.maxw75 {
    max-width: 75% !important;
}

.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon img.maxw80,
.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon p > img.maxw80 {
    max-width: 80% !important;
}

.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon img.maxw85,
.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon p > img.maxw85 {
    max-width: 85% !important;
}

.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon img.maxw90,
.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon p > img.maxw90 {
    max-width: 90% !important;
}

.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon img.maxw95,
.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon p > img.maxw95 {
    max-width: 95% !important;
}

.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon img.maxw100,
.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon p > img.maxw100 {
    max-width: 100% !important;
}

/* FIX Forcing monospace code to adhere with footnotes rules!*/
.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon p [id^='fn'] > code,
.jp-Notebook .jp-MarkdownCell .jp-RenderedHTMLCommon p [class^='fn'] > code {
    font-size: 1em !important;
}

/* Code Cell Render (HTML output) */
.jp-CodeCell .jp-RenderedHTMLCommon {
    color: var(--code-text-color);
}

.jp-Notebook .jp-CodeCell .jp-RenderedHTMLCommon p {
    font-family: var(--txbk-code-font-family);
    font-size: 1.4em;
    font-weight: normal;
}

.jp-Notebook .jp-CodeCell .jp-RenderedHTMLCommon .nomono {
    font-family: var(--txbk-content-font-family);
    font-size: 1em;
    margin-top: 0;
}

.jp-Notebook .jp-Cell:not(.jp-mod-dropTarget) {
    background: transparent;
    border: none;
}

/* Code Cell Editor */

.jp-InputArea-editor {
    background-color: var(--jp-cell-editor-active-background);
}

.jp-Notebook .jp-Cell .jp-InputArea-editor,
.jp-Notebook .jp-MarkdownCell .jp-MarkdownOutput {
    /* Set default LEFT border as transparent - to avoid change in paddings when selected */
    border-left: 0.2rem dotted var(--txbk-cell-border-color);
    box-shadow: none;
    border-right: none;
    border-top: none;
    border-bottom: none;
    overflow: hidden;
}

.jp-Notebook .jp-Cell .jp-Cell-inputWrapper {
    /* Set default RIGHT border as transparent - to avoid change in paddings when selected */
    border-right: 5px solid var(--txbk-cell-border-color);
}

.jp-Notebook .jp-Cell.jp-mod-active.jp-mod-selected:not(.jp-mod-multiSelected) .jp-InputArea-editor,
.jp-Notebook.jp-mod-commandMode .jp-MarkdownCell.jp-mod-active.jp-mod-selected:not(.jp-mod-multiSelected) .jp-MarkdownOutput {
    /* Force border redefinition to override default Jupyter theme rules */
    border-left: 0.2rem dotted var(--txbk-cell-selected-border-left-color);
    box-shadow: none;
    border-right: none;
    border-top: none;
    border-bottom: none;
    overflow: hidden;
}

.jp-Notebook.jp-mod-commandMode .jp-Cell.jp-mod-active.jp-mod-selected:not(.jp-mod-multiSelected) .jp-Cell-inputWrapper {
    border-right-color: var(--txbk-cell-display-border-color);
}

.jp-Notebook.jp-mod-editMode .jp-Cell.jp-mod-active.jp-mod-selected:not(.jp-mod-multiSelected) .jp-Cell-inputWrapper {
    border-right-color: var(--txbk-cell-edit-border-color);
}

/* Cell Multiple Selection */
.jp-Notebook.jp-mod-commandMode .jp-Cell.jp-mod-selected:not(.jp-mod-active),
.jp-Notebook.jp-mod-commandMode .jp-Cell.jp-mod-multiSelected {
    border: 2px dashed var(--texbook-azure);
    background: var(--txbk-multiselect-bgcolor);
    margin-top: 10px;
}

/*
    Overlay with Shadow for Code Cell Selected !
    ============================================
*/
/* Overlay limited to CodeCells */
.jp-Notebook .jp-CodeCell.jp-mod-selected {
    border: none;
    background: transparent;
    box-shadow: 0 6px 18px #aaa;
    z-index: 10;
    top: -10px;
    padding-top: 12px;
    padding-bottom: 12px;
    margin-top: 18px;
    margin-bottom: 2px;
}

.jp-Notebook .jp-CodeCell.jp-mod-active.jp-mod-selected:before {
    position: absolute;
    display: none;
    top: -1px;
    left: -1px;
    width: 0;
    height: calc(100% + 2px);
    content: '';
    background: none;
}

.jp-Notebook .jp-Cell .jp-Collapser {
    display: none !important;
}

.jp-Notebook .jp-CodeCell .jp-InputPrompt {
    color: var(--texbook-light-grey) !important;
    font-style: normal !important;
    font-size: 1em;
    opacity: 1 !important;
    font-weight: normal;
    min-width: 9.5ex;
}

.jp-OutputPrompt {
    color: transparent !important;
    min-width: 9.5ex;  /* Aligns with Input Prompt */
}

.jp-Notebook .jp-CodeCell .jp-InputPrompt bdi,
.jp-Notebook .jp-CodeCell .jp-OutputPrompt  bdi {
    line-height: 0;
    font-size:0;
    color: transparent;
    left: -10000px;
    content: "\21E2";
}

/*.jp-Notebook .jp-CodeCell.jp-mod-active.jp-mod-selected:not(.jp-mod-multiSelected) .jp-InputPrompt:before {*/
.jp-Notebook .jp-CodeCell.jp-mod-active.jp-mod-selected:not(.jp-mod-multiSelected) .jp-InputArea:before {
    display: inline-block;
    position: absolute;
    content: "\279E";
    font-size: 1.75rem;
    top: 4px;
    width: 10px;
    left: 0;
    font-weight: bold !important;
}

.jp-Notebook.jp-mod-commandMode .jp-CodeCell.jp-mod-active.jp-mod-selected:not(.jp-mod-multiSelected) .jp-InputArea:before {
    color: var(--txbk-cell-display-border-color);
}

.jp-Notebook.jp-mod-editMode .jp-CodeCell.jp-mod-active.jp-mod-selected:not(.jp-mod-multiSelected) .jp-InputArea:before {
    color: var(--txbk-cell-edit-border-color);
}

/* =============
   Code Editors
   ============= */

.jp-Notebook .jp-Cell .CodeMirror * {
    line-height: var(--txbk-code-line-height);
}

.jp-Notebook .jp-CodeCell .CodeMirror * {
    font-family: var(--txbk-code-font-family) !important;
    font-size: var(--txbk-code-font-size) !important;
}

.jp-Notebook .jp-MarkdownCell .CodeMirror * {
    font-family: var(--txbk-md-font-family) !important;
    font-size: var(--txbk-md-font-size) !important;
}

/* Cell Output rendering (.output_area in Notebook HTML) */

.jp-OutputArea-output.jp-RenderedText pre,
.jp-OutputArea-output.jp-RenderedMarkdown p {
    padding-bottom: 15px;
    padding-top: 10px;
}

.jp-OutputArea-output.jp-RenderedMarkdown p {
    font-size: var(--txbk-content-font-size);
}

.jp-OutputArea-output.jp-RenderedMarkdown pre code {
    font-size: var(--txbk-content-mono-font-size);
    font-family: var(--txbk-content-mono-font-family);
    color: var(--txbk-content-mono-color);
}

.jp-OutputArea-output.jp-RenderedText pre {
    --txbk-content-mono-font-family: var(--txbk-code-font-family);
    font-size: var(--txbk-code-font-size);
    font-weight: normal;
    word-break: normal;
    word-wrap: break-word;
}

/* Standard Error */
.jp-RenderedText[data-mime-type='application/vnd.jupyter.stderr'] {
    background: none !important;
}
/*
Author: Valerio Maggio < @leriomaggio >
Code: https://github.com/leriomaggio/notexbook-jupyter-theme
License: Apache License 2.0
*/

/* HTML Export (Jupyter Light Theme)
   ================================ */

body.jp-Notebook .jp-CodeCell .jp-InputPrompt {
    margin-top: 7px;
    overflow: initial;
}

body.jp-Notebook .CodeMirror-linenumber {
    --txbk-ui-mono-font-size: 10px !important;
}

body.jp-Notebook .jp-RenderedHTMLCommon.jp-MarkdownOutput p > img:only-child,
body.jp-Notebook .jp-RenderedHTMLCommon.jp-MarkdownOutput img:only-child {
    max-width: 30%;
}

body.jp-Notebook .jp-CodeCell .jp-InputPrompt {
    font-size: var(--txbk-code-font-size);
    padding-top: 4px;
}

/* ==================================================
   IMPORTANT NOTE:
   ALL markdown rules cells needs adapting because of
   a different structure in exported HTML
   ===================================================
*/
body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput {
    font-family: var(--txbk-content-font-family) !important;
    font-size: var(--txbk-content-font-size) !important;
    color: var(--txbk-content-color) !important;
    font-kerning: auto;
    text-align: justify !important;
    display: block;
    word-break: normal !important;
    word-wrap: break-word !important;
}

body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  a {
    color: var(--link-color) !important;
}

body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  h1,
body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  h2,
body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  h3,
body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  h4,
body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  h5,
body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  h6 {
    font-family:  var(--txbk-content-font-family) !important;
    font-style: normal !important;
}

/*  Font-sizes for Headers
    ----------------------*/
body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  h1 {
    font-size: 2.2em !important;
    text-align: center !important;
    padding-top:  3rem;
    padding-bottom: 3rem;
}

body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  h2 {
    font-size: 1.9em !important;
}

body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  h3 {
    font-size: 1.7em !important;
}

body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  h4 {
    font-size: 1.5em !important;
}

body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  h5 {
    font-size: 1.3em !important;
    border-bottom: 1px solid rgb(204, 204, 204);
}

body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  h6 {
    font-size: 1.1em !important;
}

body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  p,
body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  pre,
body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  code {
    font-size: 1em !important;
}

body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  pre,
body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  code {
    color: var(--txbk-content-mono-color);
}

/* Monospace Code - no code syntax highlight*/
body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  pre {
    /* hack: using code background colour here to allow for compatibility */
    background-color: var(--code-background-color) !important;
    font-family: var(--txbk-content-mono-font-family) !important;
}

body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  p {
    line-height: var(--txbk-content-line-height);
    text-align: justify !important;
}

/* Monospace inline rendered markdown */
body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  p code {

    /*Re-defining variables for Mono using Code Editor's settings*/
    --txbk-content-mono-font-family: var(--txbk-code-font-family);
    --txbk-content-mono-font-size: var(--txbk-code-font-size);

    background-color: var(--txbk-content-mono-bgcolor) !important;
    font-size: var(--txbk-content-mono-font-size) !important;
    color: var(--txbk-content-mono-color);
}

body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  p strong code {
    font-weight: bold !important;
    color: var(--txbk-content-mono-color);
}

body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  p em code {
    font-style: italic !important;
    color: var(--txbk-content-mono-color);
}

body.jp-Notebook .jp-CodeCell .jp-RenderedHTMLCommon p.nomono {
    font-family: var(--txbk-content-font-family);
    font-size: 1em;
}

body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  pre code {
    padding: 1rem 1.5rem;
    display: block;
    color: var(--txbk-content-mono-color);
    background-color: var(--txbk-content-mono-bgcolor) !important;
    font-family: var(--txbk-content-mono-font-family);
    font-size: var(--txbk-content-mono-font-size) !important;
    word-wrap: break-word;
    word-break: normal;
}

body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  blockquote {
    --txbk-ui-color: #6f6f6f !important;

    margin: 2rem .2rem !important;
    padding: .3rem .5rem !important;
    border-left: .3rem solid #a7a7a7 !important;
}

body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  blockquote p {
    padding: .2rem 1.5rem;
}

body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  table {
    margin-top: 25px;
}

body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  table {
    font-size: var(--txbk-content-font-size) !important;
    border-collapse: collapse;
    border-spacing: 0;
    width: 100%;
    overflow: auto;
    break-inside: auto;
    text-align: left;
    margin-top: 25px;
}

body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  table thead {
    display: table-header-group;
    vertical-align: middle !important;
    border-bottom: none !important;
}

body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  table thead tr,
body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  table thead tr td,
body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  table tbody tr,
body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  table tbody tr th,
body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  table tbody tr td {
    break-inside: avoid;
    break-after: auto;
    text-align: left !important;
    vertical-align: middle !important;
    padding: 0 !important;
}

body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  table tr th {
    border-bottom-width: 0;
}

body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  ul,
body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  ol {
    position: relative;
    display: block;
    margin-bottom: 0 !important;
    line-height: var(--txbk-content-line-height);
}

body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  ul {
    list-style-type: disc;
}

body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  ul li,
body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  ol li {
    /*white-space: pre-wrap;*/
    padding: .8rem !important;
}

body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  ul li > code,
body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  ol li > code {
    font-size: .9em !important
}

body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  ul li a code,
body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  ol li a code {
    font-size: .9em !important
}

body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  ul ul {
    list-style-type: circle !important;
}

body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  ul ul,
body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  ul ol,
body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  ol ul,
body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  ol ol {
    padding-top: .5rem !important;
}

body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  ul li > ul li,
body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  ul li > ol li,
body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  ol li > ul li,
body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  ol li > ol li {
    padding: .5rem !important;
}

body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  ul li > ul li code,
body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  ul li > ol li code,
body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  ol li > ul li code,
body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  ol li > ol li code {
    font-size: .9em !important;
}

body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  ul li > ul li a code,
body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  ul li > ol li a code,
body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  ol li > ul li a code,
body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  ol li > ol li a code {
    font-size: .9em !important;
    color: var(--link-color);
}

body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  ul ul ul {
    list-style-type: square !important;
}

body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  p > img:only-child,
body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  img:only-child {
    display: block;
    margin: auto;
    max-width: 40%;
}

body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  img {
    max-width: 40%;
}

/* Img MaxW classes */
body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  img.maxw10,
body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  p > img.maxw10 {
    max-width: 5% !important;
}

body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput img.maxw15,
body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput p > img.maxw15 {
    max-width: 5% !important;
}

body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput img.maxw20,
body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput p > img.maxw20 {
    max-width: 10% !important;
}

body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput img.maxw25,
body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput p > img.maxw25 {
    max-width: 15% !important;
}

body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput img.maxw30,
body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput p > img.maxw30 {
    max-width: 20% !important;
}

body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput img.maxw35,
body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput p > img.maxw35 {
    max-width: 25% !important;
}

body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput img.maxw40,
body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput p > img.maxw40 {
    max-width: 30% !important;
}

body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput img.maxw45,
body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput p > img.maxw45 {
    max-width: 35% !important;
}

body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput img.maxw50,
body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput p > img.maxw50 {
    max-width: 40% !important;
}

body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput img.maxw55,
body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput p > img.maxw55 {
    max-width: 45% !important;
}

body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput img.maxw60,
body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput p > img.maxw60 {
    max-width: 50% !important;
}

body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput img.maxw65,
body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput p > img.maxw65 {
    max-width: 55% !important;
}

body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput img.maxw70,
body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput p > img.maxw70 {
    max-width: 60% !important;
}

body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput img.maxw75,
body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput p > img.maxw75 {
    max-width: 65% !important;
}

body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput img.maxw80,
body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput p > img.maxw80 {
    max-width: 70% !important;
}

body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput img.maxw85,
body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput p > img.maxw85 {
    max-width: 75% !important;
}

body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput img.maxw90,
body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput p > img.maxw90 {
    max-width: 80% !important;
}

body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput img.maxw95,
body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput p > img.maxw95 {
    max-width: 85% !important;
}

body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput img.maxw100,
body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput p > img.maxw100 {
    max-width: 90% !important;
}

/* FIX Forcing monospace code to adhere with footnotes rules!*/
body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  p [id^='fn'] > code,
body.jp-Notebook .jp-Cell-inputWrapper .jp-RenderedHTMLCommon.jp-RenderedMarkdown.jp-MarkdownOutput  p [class^='fn'] > code {
    font-size: 1em !important;
}

/* ======================================================
              GLOBALS (with vars integration)
   ======================================================
*/

:root {
    --template-txbk-content-font-size: 16px;
    --template-txbk-content-line-height: 1.4;
    --template-txbk-content-mono-font-family: "CMU Typewriter Text", "Fira Code", monospace;
    --template-txbk-code-font-family: "Fira Code", "Fira Code", monospace;
    --template-txbk-md-font-family: "Hack", "Hack", monospace;
    --template-txbk-code-font-size: 14px;
    --template-txbk-md-font-size: 14px;
}

/* ======================================================
                         GLOBALS
   ======================================================
*/

:root {
    /* Base colors */
    --texbook-red: #d43133;
    --texbook-pink: #D14187;
    --texbook-turquoise: #009489;
    --texbook-azure: #00afde;
    --texbook-blue: #2875d9;
    --texbook-light-grey: #828282;
    --texbook-dark-grey: rgb(56, 56, 56);
    --texbook-yellow: #f9ab00;
    --texbook-orange: #e8710a;

    /* Font formatting */
    --background-color: #ffffff;
    --txbk-ui-font-family: "Roboto Slab", "Helvetica Neue", Helvetica, Arial, sans-serif;
    --txbk-ui-color: rgb(51, 51, 51);
    --txbk-ui-font-size: 14px;
    --txbk-ui-line-height: 1;

    --txbk-ui-header-font-family: "Roboto Slab", "Helvetica Neue", Helvetica, Arial, sans-serif;
    --txbk-ui-header-color: rgb(56, 56, 56);

    --txbk-ui-mono-color: #828282;
    --txbk-ui-mono-font-size: 15px;
    --txbk-ui-mono-line-height: 1.4;

    /* Cells */
    --txbk-cell-selected-border-left-color: lightgrey;
    --txbk-cell-border-color: var(--background-color);
    --txbk-multiselect-bgcolor: var(--background-color);
    --txbk-cell-display-border-color: #009489;
    --txbk-cell-edit-border-color: #d43133;

    /* Content Formatting */
    --txbk-content-font-family: "CMU Serif", "Times New Roman", serif;
    --txbk-content-font-size: var(--template-txbk-content-font-size);
    --txbk-content-color: rgb(51, 51, 51);
    --txbk-content-line-height: var(--template-txbk-content-line-height);

    --txbk-content-mono-font-family: var(--template-txbk-content-mono-font-family);
    --txbk-content-mono-font-size: var(--txbk-content-font-size);
    --txbk-content-mono-color: var(--texbook-dark-grey);
    --txbk-content-mono-bgcolor: var(--code-background-color);

    --txbk-code-font-family: var(--template-txbk-code-font-family);
    --txbk-code-font-size: var(--template-txbk-code-font-size);
    --txbk-md-font-family: var(--template-txbk-md-font-family);
    --txbk-md-font-size: var(--template-txbk-md-font-size);
    --txbk-code-line-height: 1.6;
    --txbk-code-color: var(--code-text-color);
    --txbk-code-bgcolor: var(--code-background-color);

    --link-color: var(--texbook-blue);
    --del-color: var(--texbook-red);
    --drop-cap-color: var(--texbook-red);;

    --kernel-name-color: var(--texbook-azure);
    --kernel-idle: var(--texbook-turquoise);
    --kernel-busy: var(--texbook-pink);
    --kernel-disconnected: var(--texbook-red);
    --btn-warning: var(--texbook-pink);
    --btn-danger: var(--texbook-red);
    --running-notebook: var(--texbook-turquoise);
}

html {
    font-size: 10px;
}

body {

    background-color: var(--background-color) !important;
    font-weight: 300;
    font-family: var(--txbk-ui-font-family);
    color: var(--txbk-ui-color);
    overflow: inherit;
    vertical-align: middle;
    font-size: var(--txbk-ui-font-size);
}

h1, h2, h3, h4, h5, h6, label > strong {
    font-family: var(--txbk-ui-header-font-family);
    color: var(--txbk-ui-header-color);
}

h1 img, h2 img , h3 img, h4 img, h5 img, h6 img {
    display: inline !important;
}

h1 {
    font-weight: bold;
    font-style: normal;
}

h2 {
    font-weight: bold;
    font-style: italic;
}

h3 {
    font-weight: 600;
    font-style: normal;
}

h4 {
    font-weight: 500;
    font-style: normal;
}

h5, h6 {
    font-weight: 500;
    font-style: italic;
}

code, kbd, pre, samp {
    font-family: var(--txbk-code-font-family);
    color: var(--txbk-ui-mono-color);
    font-weight: 400;
    font-size: var(--txbk-ui-mono-font-size);
    line-height: var(--txbk-ui-mono-line-height);
}

table > thead > tr > td.info,
table > tbody > tr > td.info,
table > tfoot > tr > td.info,
table > thead > tr > th.info,
table > tbody > tr > th.info,
table > tfoot > tr > th.info,
table > thead > tr.info > td,
table > tbody > tr.info > td,
table > tfoot > tr.info > td,
table > thead > tr.info > th,
table > tbody > tr.info > th,
table > tfoot > tr.info > th {
    background-color: #d9edf7;
}

del {
    color: var(--del-color) !important;
}

/* ======================================================
                 Intro/Home Page Server
   ======================================================
*/

#ipython-main-app p {
    text-align: justify !important;
}

.toolbar_info,
.list-container {
    color: #828282;
}

.list_placeholder {
    font-family: "Roboto Slab", serif !important;
    font-weight: 300;
    font-style: normal;
}

a.item_link {
    color: var(--link-color);
    font-weight: 300;
}

.item_buttons .kernel-name {
    color: var(--kernel-name-color);
    font-weight: 400;
}

.btn-warning {
    background-color: var(--btn-warning) !important;
    border-color: var(--btn-warning) !important;
}

.btn-warning:focus {
    border-color: var(--btn-warning) !important;
}

.btn-danger {
    background-color: var(--btn-danger) !important;
    border-color: var(--btn-danger) !important;
}

.btn-danger:focus {
    border-color: var(--btn-danger) !important;
}

.running-indicator,
.running_notebook_icon:before {
    color: var(--running-notebook);
}

/* ======================================================
                         Notebook
   ======================================================
*/

/* Modal Dialog */

.modal-header > h4 {
    font-style: normal !important;
    text-decoration: none !important;
    font-family: var(--txbk-ui-font-family) !important;
    margin: 0 !important;
}

.notebook_app {
    background-color: var(--background-color) !important;
    font-family: "Helvetica Neue", Helvetica, Arial, sans-serif !important;
    font-weight: normal;
}

div#notebook_name {
    font-family: var(--txbk-ui-font-family) !important;
    font-weight: normal;
    font-size: 24px;
    padding: 1px 7px 1px 1px;
}

div#site {
	overflow: inherit;
    top: 112px;
    position: absolute;
}

#header {
    display: block;
    position: fixed !important;
    top: 0;
    width: 100%;
    max-width: 100%;
    height: fit-content;
    background-color: var(--background-color);
}

#notebook-container {
    /* Remove shadow so that it looks like a page of a PDF doc */
    box-shadow: none !important;
    -webkit-box-shadow: none !important;
    padding: 0;
}

div#notebook {
    border-top: none;
    font-size: 1rem;
    padding: 0 !important;
    padding-top: 30px !important;
    font-family: var(--txbk-content-font-family) !important;
}

.kernel_idle_icon:before {
    color: var(--kernel-idle);
}

.kernel_busy_icon:before {
    color: var(--kernel-busy);
}

.kernel_disconnected_icon:before {
    color: var(--kernel-disconnected);
}

#notification_area {
    font-family: var(--txbk-ui-font-family);
}

#notification_kernel {
    color: #fff;
    background-color: var(--kernel-idle);
    border-color: var(--kernel-idle);
}

#notification_kernel.warning {
    color: #fff;
    background-color: var(--kernel-disconnected);
    border-color: var(--kernel-disconnected);
}

#notification_trusted {
    border-color: var(--kernel-disconnected);
}

#notification_trusted[disabled="disabled"] {
    border-color: var(--kernel-idle);
}

#notification_notebook {
    border-color: var(--kernel-idle);
}

/* -----------------------------------
    CELLS
   -----------------------------------
*/

div.cell {
    right: 2px;
    border:  none;
    margin-top: 5px;
}

div.cell.selected {
    /* Reset all - ready for customisation */
    border-radius: 0;
    background: none;
}

div.cell .inner_cell {
    border-right: 5px solid var(--txbk-cell-border-color);
    border-left:  0.2rem dotted var(--txbk-cell-border-color);
}

div.cell.selected:not(.jupyter-soft-selected) .inner_cell {
    border-left-color:  var(--txbk-cell-selected-border-left-color);
}

.command_mode div.cell.selected:not(.jupyter-soft-selected) .inner_cell {
    border-right-color: var(--txbk-cell-display-border-color);
}

.edit_mode div.cell.selected:not(.jupyter-soft-selected) .inner_cell {
    border-right-color: var(--txbk-cell-edit-border-color);
}

/* Multiple Selection */
.command_mode div.cell.jupyter-soft-selected {
    border: 2px dashed var(--texbook-azure);
    background: var(--txbk-multiselect-bgcolor);
    margin-top: 10px;
}

/*
Overlay with Shadow for Code Cell Selected !
*/
div.code_cell.selected,
.command_mode div.code_cell.jupyter-soft-selected {
    box-shadow: 0 6px 18px #aaa;
    z-index: 10;
    top: -10px;
}

div.code_cell.selected {
    border: none;
    background: transparent;
}

div.prompt {
    font-family: var(--txbk-code-font-family) !important;
    font-style: normal;
    font-size: 1rem;
    text-align: right;
    line-height: 1rem;
    min-width: 9ex;
}

div.cell.selected:before,
div.cell.jupyter-soft-selected:before {
    /* Important Needed to Override */
    width: 0 !important;
    background: none !important;
}

div.cell.selected:not(.jupyter-soft-selected) div.input:before {
    display: inline-block;
    position: relative;
    content: "\279E";
    font-size: 1.8rem;
    top: 11px;
    width: 10px;
    left: 3px;
    color: var(--txbk-cell-display-border-color);
    font-weight: bold !important;
}

.edit_mode div.cell.selected div.input:before,
.edit_mode div.cell.selected.unrendered div.input:before {
    color: var(--txbk-cell-edit-border-color);
}

.code_cell div.input_prompt:after,
.code_cell div.output_prompt:after {
    display: inline-block;
    content: '';
    font-size:  0.75rem;
    font-style: normal !important;
}

.edit_mode div.cell.selected:before{
    width: 0;
    background: none;
}

/* ==========================
    Cell Input and Rendering
   ==========================
*/

div.input_area {
    border-radius: 0;
    border: none;
    padding: 2px 5px 0 5px;
}

div.input_prompt {
    color: #aba9a9;
    font-style: normal;
    font-size: 1.4rem;
    padding-top: 14px;
}

/* FIX:
   Fixing Empty cell output in HTML export
*/
div.output_prompt {
    color: transparent !important;
}

div.input_prompt bdi,
div.output_prompt bdi {
    line-height: 0;
    font-size:0;
    color: transparent;
    left: -10000px;
    content: "\21E2";

}

div.output_wrapper {
    margin-top: 8px;
}

div.output_area div.output_text pre,
div.output_area div.output_markdown p {
    padding-bottom: 15px;
    padding-top: 10px;
}

div.output_area div.output_markdown p {
    font-size: var(--txbk-content-font-size);
}

div.output_area div.output_markdown pre code {
    font-size: var(--txbk-content-mono-font-size);
    font-family: var(--txbk-content-mono-font-family);
    color: var(--txbk-content-mono-color);
}

div.output_area div.output_text pre {
    --txbk-content-mono-font-family: var(--txbk-code-font-family);

    font-weight: normal;
    word-break: normal;
    word-wrap: break-word;
}

/* Output Standard Error  */
div.output_stderr {
    background-color: #FFFFFF;
}

div.out_prompt_overlay:hover {
    box-shadow: none;
    background: none;
}

/* Output HTML after cell code - e.g. Pandas DataFrame */
div.output_html {
    color: var(--code-text-color);
}

div.output_html {
    font-size: 1.4em;
    font-weight: normal;
    font-family: var(--txbk-code-font-family);
}

div.output_html .nomono {
    font-family: var(--txbk-content-font-family);
    margin-top: 0;
    font-size: 1em;
}

div.output_html > iframe, div.output_area {
    margin-left: 2rem;
}

div.output_subarea {
    margin-left: 0;
}

div.output_svg div {
    max-width: 98%;
    margin-left: 0 !important;
}

/* ----------------------------
    Override default CSS rules
   ---------------------------- */
.rendered_html code {
    background-color: transparent !important;
}

.rendered_html h1:first-child,
.rendered_html h2:first-child,
.rendered_html h3:first-child,
.rendered_html h4:first-child,
.rendered_html h5:first-child,
.rendered_html h6:first-child {
    margin-top: 0.25em;
}

.rendered_html h1 {
    margin-top: 0.48em;
}

.rendered_html h2 {
    margin-top: 0.57em;
}

.rendered_html h3 {
    margin-top: 0.85em;
}

.rendered_html h4 {
    margin-top: 1.3em;
}

.rendered_html h5 {
    margin-top: 1.2em;
}

.rendered_html h6 {
    margin-top: 1.1em;
}

/* ===============
    CELLS Render
   =============== */

/*  -------------------------
    (1) CODE CELLS
    Display + Edit modes
    -------------------------
*/
div.code_cell pre, div.CodeMirror, div.CodeMirror-linenumber {
    font-family: var(--txbk-code-font-family);
    font-size: var(--txbk-code-font-size);
    line-height: var(--txbk-code-line-height);
}

div.CodeMirror-linenumber {
    --txbk-ui-mono-font-size: 12px !important;
}

/* -- MARKDOWN CELLS (Edit) -- */

/* md cell */
div.text_cell.unrendered pre {
    /* general font rule */
    font-family: var(--txbk-md-font-family);
    font-size: var(--txbk-md-font-size);
    line-height: var(--txbk-code-line-height);
}

/* md cell headers (edit) */
.cm-header-1, .cm-header-2, .cm-header-3,
.cm-header-4, .cm-header-5, .cm-header-6 {
    /* general font rule */
    font-family: var(--txbk-md-font-family);
}

/* ------------------------------------------------------------------
    (2) MARKDOWN CELLS (DISPLAY MODE)

    Notes: Font-families to use here:
    computer-modern --> general text (Computer Modern Typeface, CMU)
    md-display-monospace --> Monospace output (CMU Typewriter)
   ------------------------------------------------------------------
 */

div.text_cell_render {
    font-family: var(--txbk-content-font-family);
    font-size: var(--txbk-content-font-size);
    color: var(--txbk-content-color);
    font-kerning: auto;
    text-align: justify;
    display: block;
    word-break: normal;
    word-wrap: break-word;
}

div.text_cell_render a {
    color: var(--link-color);
}

div.text_cell_render h1, div.text_cell_render h2,
div.text_cell_render h3, div.text_cell_render h4,
div.text_cell_render h5, div.text_cell_render h6 {
    font-family:  var(--txbk-content-font-family);
    font-style: normal;
}

/* Font-sizes for headers */
div.text_cell_render h1 {
    font-size: 2.2em;
    text-align: center;
    padding-top:  3rem;
    padding-bottom: 3rem;
}

div.text_cell_render h2 {
    font-size: 1.9em;
}

div.text_cell_render h3 {
    font-size: 1.7em;
}

div.text_cell_render h4 {
    font-size: 1.5em;
}

div.text_cell_render h5 {
    font-size: 1.3em;
    border-bottom: 1px solid rgb(204, 204, 204);
}

div.text_cell_render h6 {
    font-size: 1.1em;
}

div.text_cell_render p,
div.text_cell_render pre,
div.text_cell_render code {
    font-size: 1em !important;
}

div.text_cell_render pre,
div.text_cell_render code {
    color: var(--txbk-content-mono-color);
}

/* Monospace Code - no code syntax highlight*/
div.text_cell_render pre {
    /* hack: using code background colour here to allow for compatibility */
    background-color: var(--code-background-color);
    font-family: var(--txbk-content-mono-font-family);
}

div.text_cell_render p {
    line-height: var(--txbk-content-line-height);
    text-align: justify !important;
}

div.text_cell_render code {
    font-family: var(--txbk-content-mono-font-family);
}

/* Monospace inline rendered markdown */
div.text_cell_render p code {
    /*Re-defining variables for Mono using Code Editor's settings*/
    /*--txbk-content-mono-font-family: var(--txbk-code-font-family);*/
    /*--txbk-content-mono-font-size: var(--txbk-code-font-size);*/
    font-size: var(--txbk-content-mono-font-size) !important;
    color: var(--txbk-content-mono-color);
    font-family: var(--txbk-content-mono-font-family);
}

div.text_cell_render p strong code {
    font-weight: bold !important;
    color: var(--txbk-content-mono-color);
}

div.text_cell_render p em code {
    font-style: italic !important;
    color: var(--txbk-content-mono-color);
}

div.text_cell_render pre code {
    padding: 1rem 1.5rem;
    display: block !important;
    color: var(--txbk-content-mono-color);
    background-color: var(--txbk-content-mono-bgcolor) !important;
    font-family: var(--txbk-content-mono-font-family);
    font-size: var(--txbk-content-mono-font-size) !important;
    word-wrap: break-word;
    word-break: normal;
}

div.text_cell_render blockquote {
    --txbk-ui-color: #6f6f6f;
    margin: .2rem .2rem;
    padding: .3rem .5rem;
    border-left: .3rem solid #a7a7a7;
    background-color: var(--txbk-content-mono-bgcolor);
}

div.text_cell_render blockquote:not(:first-child) {
    margin: 2rem .2rem;
}

div.text_cell_render blockquote p {
    padding: .2rem 1.5rem;
}

div.text_cell_render table {
    font-size: var(--txbk-content-font-size) !important;
    border-collapse: collapse;
    border-spacing: 0;
    overflow: auto;
    break-inside: auto;
    margin-top: 25px;
    margin-bottom: 15px;
}

div.text_cell_render table thead {
    display: table-header-group;
    vertical-align: middle;
    border-bottom: none;
}

div.text_cell_render table thead tr,
div.text_cell_render table thead tr th,
div.text_cell_render table thead tr td,
div.text_cell_render table tbody tr,
div.text_cell_render table tbody tr th,
div.text_cell_render table tbody tr td {
    break-inside: avoid;
    break-after: auto;
    vertical-align: middle;
    padding: 5px 5px;
}

div.text_cell_render table tr th {
    border-bottom-width: 0;
}

div.text_cell_render ul, div.text_cell_render ol {
    position: relative;
    display: block;
    margin-bottom: 0 !important;
    line-height: var(--txbk-content-line-height);
}

div.text_cell_render ul {
    list-style-type: disc;
}

div.text_cell_render ul li, div.text_cell_render ol li {
    /*white-space: pre-wrap;*/
    padding: .6rem !important;
}

div.text_cell_render ul ul {
    list-style-type: circle !important;
}

div.text_cell_render ul ul,
div.text_cell_render ul ol,
div.text_cell_render ol ul,
div.text_cell_render ol ol {
    padding-top: .3rem !important;
}

div.text_cell_render ul li > ul li,
div.text_cell_render ul li > ol li,
div.text_cell_render ol li > ul li,
div.text_cell_render ol li > ol li {
    padding: .3rem !important;
}

div.text_cell_render ul li > ul li a code,
div.text_cell_render ul li > ol li a code,
div.text_cell_render ol li > ul li a code,
div.text_cell_render ol li > ol li a code {
    color: var(--link-color);
}

div.text_cell_render ul ul ul {
    list-style-type: square !important;
}

div.text_cell_render p > img:only-child {
    display: block;
    margin: auto;
    max-width: 70% !important;
}

div.text_cell_render img {
    max-width: 70%;
}

/*
    ===================================
    New CSS Classes and Styles
    ===================================

    --------------
    1. Footnotes
    --------------
*/
[id^='fn'], [class^='fn'] {
    font-size: small !important;
    font-weight: normal !important;
    font-style: normal !important;
    border-top: 1px solid var(--txbk-ui-mono-color);
    padding-top: .8rem;
    display: block;
}

p > [id^='fn']:not(:first-child),
p > [class^='fn']:not(:first-child) {
    border-top: 0 !important;
}

[id^='fn'] i, [class^='fn'] i,
[id^='fn'] b, [class^='fn'] b,
[id^='fn'] strong, [class^='fn'] strong,
p > strong > strong {
    background-color: var(--txbk-content-mono-bgcolor);
    font-style: normal !important;
    font-weight: bold !important;
}

/* FIX Forcing monospace code to adhere with footnotes rules!*/
div.text_cell_render p [id^='fn'] > code,
div.text_cell_render p [class^='fn'] > code {
    font-size: 1em !important;
}

/* CSS Anchors link as superscript (like footnotes) */
a[target="_self"][href^="#fn"], a[target="_self"][href*="fn"] {
    font-size: 60%;
    vertical-align: top;
}
/* Surround Footnotes by Square brackets */
a[target="_self"][href^="#fn"]:before, a[target="_self"][href*="fn"]:before {
    content: "[";
}

a[target="_self"][href^="#fn"]:after, a[target="_self"][href*="fn"]:after {
    content: "]";
}

/* --------------
    2. Drop cap
   --------------
*/
.drop {
    color: var(--drop-cap-color);
    font-size: 75px;
    line-height: 60px;
    padding-top: 4px;
    margin-top: 0.2rem;
}

/* -----------------------------
    3. noTeXbook Text colours
   -----------------------------
*/
.texbook-red {
    color: var(--texbook-red);
}

.texbook-pink {
    color: var(--texbook-pink);
}

.texbook-turquoise {
    color: var(--texbook-turquoise);
}

.texbook-azure {
    color: var(--texbook-azure);
}

.texbook-blue {
    color: var(--texbook-blue);
}

.texbook-light-grey {
    color: var(--texbook-light-grey);
}

.texbook-dark-grey {
    color: var(--texbook-dark-grey);
}

.texbook-orange {
    color: var(--texbook-orange);
}

.texbook-yellow {
    color: var(--texbook-yellow);
}

/* ---------------------------------------------------------
    4. IMG MaxWidthXX

    CSS classes to control the max-width property of images
    when the default 70% is not enough.
    This is when images need to be enlarged or shrunk.
   ---------------------------------------------------------
*/

div.text_cell_render img.maxw10,
div.text_cell_render p > img.maxw10 {
    max-width: 10% !important;
}

div.text_cell_render img.maxw15,
div.text_cell_render p > img.maxw15 {
    max-width: 15% !important;
}

div.text_cell_render img.maxw20,
div.text_cell_render p > img.maxw20 {
    max-width: 20% !important;
}

div.text_cell_render img.maxw25,
div.text_cell_render p > img.maxw25 {
    max-width: 25% !important;
}

div.text_cell_render img.maxw30,
div.text_cell_render p > img.maxw30 {
    max-width: 30% !important;
}

div.text_cell_render img.maxw35,
div.text_cell_render p > img.maxw35 {
    max-width: 35% !important;
}

div.text_cell_render img.maxw40,
div.text_cell_render p > img.maxw40 {
    max-width: 40% !important;
}

div.text_cell_render img.maxw45,
div.text_cell_render p > img.maxw45 {
    max-width: 45% !important;
}

div.text_cell_render img.maxw50,
div.text_cell_render p > img.maxw50 {
    max-width: 50% !important;
}

div.text_cell_render img.maxw55,
div.text_cell_render p > img.maxw55 {
    max-width: 55% !important;
}

div.text_cell_render img.maxw60,
div.text_cell_render p > img.maxw60 {
    max-width: 60% !important;
}

div.text_cell_render img.maxw65,
div.text_cell_render p > img.maxw65 {
    max-width: 65% !important;
}

div.text_cell_render img.maxw70,
div.text_cell_render p > img.maxw70 {
    max-width: 70% !important;
}

div.text_cell_render img.maxw75,
div.text_cell_render p > img.maxw75 {
    max-width: 75% !important;
}

div.text_cell_render img.maxw80,
div.text_cell_render p > img.maxw80 {
    max-width: 80% !important;
}

div.text_cell_render img.maxw85,
div.text_cell_render p > img.maxw85 {
    max-width: 85% !important;
}

div.text_cell_render img.maxw90,
div.text_cell_render p > img.maxw90 {
    max-width: 90% !important;
}

div.text_cell_render img.maxw95,
div.text_cell_render p > img.maxw95 {
    max-width: 95% !important;
}

div.text_cell_render img.maxw100,
div.text_cell_render p > img.maxw100 {
    max-width: 100% !important;
}

/*  ------------
     4. Badges
    ------------
*/
.badges img, .badges p img {
    margin: 0 !important;  /* Override margins */
    display: inline !important;
    padding-right: 7px;
}

.badges p {
    display: inline !important;
}

.badges {
    display: block;
    text-align: center;
}

/*  ----------------
     5. Inline Math
    ----------------
*/
.inline-math {
    display: inline-block;
}

/*  --------------------
     6. Inline Mono Font
    --------------------
*/
.codemono code, .codemono pre, .codemono pre code {
    font-family: var(--txbk-code-font-family) !important;
}

.mdmono code, .mdmono pre, .mdmono pre code {
    font-family: var(--txbk-md-font-family) !important;
}
</style>

<div class="nomono">
   The notebook is using
    <a href="https://github.com/leriomaggio/notexbook-jupyter-theme" title="noTeXbook Theme">
        <b>no$\TeX$book</b></a> Jupyter Theme (release <code>2.0.0</code>).

</div>

# Automatic Differentiation

---

### Perceptron and MLP short recap

<span id="fn1"> source: [3]</span>

__**source:**_This example has been extracted from Reference [3](#ref3)___

Perceptron (Adaline algorithm) - **ADAptive LInear Neuron** main features:

- Unit Step function
- Binary Classification
- Linear Model
- Weight Update rule with `learning rate` and Gradient Descent Optimisation

$
    {\bf w} = {\bf w} + \Delta {\bf w} \text{, where } \Delta {\bf w} = -\eta \nabla J({\bf w})
$ [2](#fn2)

<span id="fn2">2: ${\bf w}$ (in bold) refers to the **weights vector**. </span>

In other words, we computed the gradient based on the whole training dataset and updated the weights of the model by taking a step into the opposite direction of the gradient $\nabla J(w)$.

In order to find the optimal weights of the model, we optimized an objective function
that we defined as the **sum of squared errors** (SSE) cost function $J(w)$.

Furthermore, we multiplied the gradient by a factor, the learning rate $\eta$, which we had to choose carefully to balance the speed of learning against the risk of overshooting the global minimum of the cost function.

##### Choosing $\eta$ at a glance

In  descent optimization, we updated all weights simultaneously after each epoch, and we defined the partial derivative for each weight $w_j$ in the weight vector ${\bf w}$ as follows:

$$
\begin{equation}
    \frac{\partial}{\partial w_j} J({\bf w}) = - \sum_{i} (y^{(i)} - a^{(i)})x_j^{(i)}
\end{equation}
$$

Here $x_j^{(i)}$ represents the feature $j$ of the $i-th$ sample, whereas $y^{(i)}$ and $a^{(i)}$ are the corresponding target and prediction (i.e. *activation of the neuron*), respectively.

In the special case of Adaline, we simply defined:

$$
a = \phi(z) = z = \displaystyle{\sum_{j} w_jx_j}
$$

##### As good as a **Linear Classifier**

---

<span id="fn3"> source: [3] </span>

###### Calculating Gradients?

**This is more or less where we left**.

Now the question is: How do we *really* calculate those gradients ?

In addition, two very compulsory requirements about this calculation:
- **computationally exact**;
- **computationally efficient**.

## Introducing Automatic Differentiation

> Methods for the computation of derivatives in computer programs can be classified into four categories:
>
> (1) **manually** working out derivatives and coding them;
>
> (2) **numerical** differentiation using finite difference approximations;
>
> (3) **symbolic** differentiation using expression manipulation in computer algebra > systems;
>
> (4) **automatic** differentiation, also called *algorithmic differentiation*

<span id="fn0">Source: [4]</span>

<ins>**Automatic Differentiation** is the subject matter of this notebook (and section!)</ins>

### Automatic Differentiation in a Nutshell

**TL; DR**: Automatic Differentiation lets you compute **exact** derivatives in **constant time**

_Teaser_

Automatic Differentiation is the secret ingredient that powers most of the existing Deep Learning frameworks (e.g. Pytorch or TensorFlow).

In a nutshell, Deep learnin frameworks provide the (technical) infrastructure in which computing the derivative of a function takes as much time as evaluating the function. In particular, the design idea is: "you define a network with a loss function, and you get a gradient *for free*".

**Differentiation** in general is becoming a **first class citizen** in programming languages with early work started by Chris Lattner of LLVM famework — see the [Differentiable Programming Manifesto](https://github.com/apple/swift/blob/master/docs/DifferentiableProgramming.md) for more detail.

However, if you're still wondering whether any of this is just boring math and how this relates to computer programming, I definitely suggest to have a look at **this** vide0 on YouTube:

[GOTO 2018 - Machine Learning: Alchemy for the Modern Computer Scientist by Erik Meijer](https://www.youtube.com/watch?v=Rs0uRQJdIcg)

### Automatic Differentiation is NOT $\ldots$

In the above list, we included automatic differentiation as the fourth item [3](#fn3 "Manual really?"), so separated from other differentiation methods such as **symbolic** or **numerical** methods.

<span id="fn3">3: The first was indeed manual, hard coded implementation - not really meant though!</span>

#### Automatic Differentiation $\neq$ Symbolic Differentiation

This is the **most** obvious, I presume.

Symbolic differentiation works by breaking apart a complex expression into a bunch of simpler expressions by using various rules — very similar to a compiler.

(Mathematically and) Technically speaking this is all possible thanks to the [**compositionality**](https://en.wikipedia.org/wiki/Principle_of_compositionality) of derivatives calculation.

Examples of some rules:

**Sum rule**:

$$
\begin{equation}
\frac{d}{dx} (f(x) + g(x)) = \frac{d}{dx} f(x) + \frac{d}{dx} g(x)
\end{equation}
$$

**Derivatives of powers rule**:

$$
\begin{equation}
\frac{d}{dx} x^r = rx^{r-1}
\end{equation}
$$

**Multiplication rule**:

$$
\begin{equation}
\frac{d(fg)}{dx} = \frac{df}{dx} g + f\frac{dg}{dx}
\end{equation}
$$

**Chain rule (!!)**:

$$
\begin{equation}
\frac{d}{dx}[f(g(x))] = \frac{df}{dg} \cdot \frac{dg}{dx}
\end{equation}
$$

**Issue**:

- For complicated functions, the resultant expression can be exponentially large - $O(2^n)$;
- Could be wasteful to keep around intermediate symbolic expressions if we only need a **numeric value** of the gradient in the end;

**Real world examples**: `theano` (see [doc](http://deeplearning.net/software/theano/tutorial/gradients.html))

#### Automatic Differentiation $\neq$ Numerical Differentiation

Numerical Differentiation is far more intuitive and effectively the (most) natural way to compute derivatives, building on the very definition of derivatives as the **limit of the different quotient**:

$$
\begin{equation}
f'(x) = \lim_{\epsilon \leftarrow 0} \frac{f(x + \epsilon) - f(x)}{\epsilon}
\end{equation}
$$

Recap on (other) notations of derivatives:

- f'(x): **Lagrange's Notation**
- $\frac{df}{dx}$: **Leibniz's Notation**

For **multivariate** functions $f: \mathbb{R} \mapsto \mathbb{R}^n $, we can approximate the gradient
$\nabla f = ( \frac{\partial f}{\partial x_1}, \ldots, \frac{\partial f}{\partial x_n} )$ using:

$$
\frac{\partial f({\bf x})}{\partial x_i} \approx \frac{f({\bf x} + h{\bf e_i}) - f({\bf x})}{h}
$$

where $\bf{e_i}$ is the $i-th$ unit vector and $ h > 0$ is a small step size.

This has the **advantage** of being uncomplicated to implement, but the **disadvantages** of performin $O(n)$
evaluations of $f$ for a gradient in $n$ dimensions, and requiring **careful considerations** in selecting the step size $h$.

Indeed, Numerical approximations of derivatives are inherently ill-conditioned and unstable[4](#fn4).

<span id="fn4">**[4]**: Using the limit definition of the derivative for finite difference approximation commits both cardinal sins of numerical analysis: **“thou shalt not add small numbers to big numbers”, and “thou shalt not subtract numbers which are approximately equal”**.</span>

### Automatic Differentiation $=$ Automatic Differentiation

1. Automatic Differentiation (`AD`) gives exact answers in constant time.

2. AD can be thought of as performing a non-standard interpretation of a computer program where this interpretation involves **augmenting** the standard computation with the calculation of various derivatives.

3. All numerical computations are ultimately compositions of a finite set of elementary operations for which derivatives are known, and combining the derivatives of the constituent operations through the **chain rule** gives the derivative of the overall composition.

The chain rule above told us what we **wanted**. What it does not say is **how** to compute it efficiently.

AD is **a way** to get these gradients efficiently without having to do anything but write the objective function in <ins>computer code</ins>.

AD is a very broadly applicable technique and it has been studied for decades. Curiously however, it has traditionally only been used in a limited way in machine learning despite the ubiquity of gradient-based optimization problems in ML.

It has only been recently that serious automatic differentiation (versus naïve hand-coded backprop rules) have started to make their way into mainstream deep learning toolchains.

**TensorFlow**, for example, has the ability to compute gradients of the computational graphs it supports with its limited domain-specific language, but it is not close to a full AD system.

Automatic differentiation can be implemented in a variety of ways, via **run-time abstractions** and also via **source code transformation**.

##### Example

Rather than talking about large neural networks, we will seek to understand automatic differentiation via a small problem borrowed from the book of *Griewank and Walther (2008)*.

In the following we will adopt their very same **three-part** notation (also used in [4]).

A function $f: \mathbb{R^n} \mapsto \mathbb{R^m}$ is constructed using intermediate variables $v_i$ such that:

- variables $v_{i-n} = x_i$, $i = 1,\ldots,n$ are the input variables;
- variables $v_i$, $i = 1,\ldots,l$ are the working **intermediate** variables;
- variables $y_{m-i} = v_{l-i}$, $i = m-1,\ldots,0$ are the output variables.

The **traversal** of the graph and the **direction** in which gradients are actually computed defines the two modalities of AD:

* **forward mode** AD;
* **backward mode** AD.

##### Forward (Tangent) Mode

The idea of forward mode automatic differentiation is that we can compute derivatives as we go and
that the chain rule says the overall derivative that we want is a composition of these incremental
computations.

Let’s imagine that our overall goal is to compute $\frac{\partial y}{\partial x_1}$ for the example above.

We denote all the intermediate *partial derivatives* (**with respect to $x_1$**) as:
$$
\dot{v}_{i} = \frac{\partial v_i}{\partial x_1}
$$

The **very same** applies when we want to compute $\frac{\partial y}{\partial x_2}$

**Dual Numbers**

Mathematically, **forward mode AD** can be viewed as evaluating a function using **dual numbers** which can be defined as truncated Taylor series of the form:
$$
v + \dot{v}\epsilon
$$

where $v$ and $\dot{v} \in \mathbb{R}$, and $\epsilon$ is a *nilpotent* number such as $\epsilon^2 = 0$, and
$\epsilon \neq 0$.

Therefore:
$$
f(v + \dot{v}\epsilon) = f(v) + f'(v)\dot{v}\epsilon
$$

and using dual numbers as data structures for carrying the **tangent** value together with the primal.
In fact, **forward mode AD** is also known as **tangent mode**.

The interesting bit is that we can implement this bookkeeping during the *execution trace* just via abstraction.

We can replace our floating point numbers with `tuples`, and replace primitive functions with the following Python implementation (just using `numpy`)

```python
import numpy as np

def add(atuple, btuple):
    (a, adot) = atuple
    (b, bdot) = btuple
    return ( a + b, adot + bdot)

def subtract(atuple, btuple):
    (a, adot) = atuple
    (b, bdot) = btuple
    return (a - b, adot - bdot)

def multiply(atuple, btuple):
    (a, adot) = atuple
    (b, bdot) = btuple
    return (a * b, adot * b + bdot * a)

def divide(atuple, btuple):
    (a, adot) = atuple
    (b, bdot) = btuple
    return (a / b, (adot * b - bdot * a) / (b*b))

def ln(atuple):
    (a, adot) = atuple
    return (np.log(a), (1/a)*adot)

def sin(atuple):
    (a, adot) = atuple
    return (np.sin(a), np.cos(a)*adot)
```

```python
def f(x1: tuple, x2: tuple):
    # ln(x1) + x1x2 - sin(x2)
    v1 = ln(x1)
    v2 = multiply(x1, x2)
    v3 = add(v1, v2)
    v4 = sin(x2)
    v5 = subtract(v3, v4)
    return v5
```

```python
f(x1=(2, 1), x2=(5, 0))
```

    (11.652071455223084, 5.5)

```python
f(x1=(2, 0), x2=(5, 1))
```

    (11.652071455223084, 1.7163378145367738)

##### Reverse (Co-Tangent) Mode

AD in the reverse accumulation mode corresponds to a generalized backpropagation algorithm, in that it propagates derivatives backward from a given output. This is done by complementing each intermediate variable $v_i$ with an **adjoint**:
$$
\bar{v}_{i} = \frac{\partial y_i}{\partial v_i} = \displaystyle{\sum_{j:\text{child of i}} \bar{vj} \frac{\partial v_j}{\partial v_i}}
$$

There are various ways to implement this abstraction in its full generality, but an implementation requires more code than can easily appear here. The three major approaches are:

**source code transformation**: The adjoint backward pass code is generated a priori from the forward computation. A clean Python example of such a system is [**Tangent**](https://colab.research.google.com/drive/1cjoX9GteBymbnqcikNMZP1uenMcwAGDe).

**graph-based**: This approach uses an embedded mini-language to specify a graph of computations that can then be manipulated for function evaluations and gradients.

$\rightarrow$ The advantage of this approach is that it is amenable to intelligent graph optimizations and use of compilers. The embedded mini-language also makes it possible to build specialized hardware that targets the differentiable primitives.

$\rightarrow$ The downside of this approach is that you are not coding in the host language (e.g., Python) and so you can’t take advantage of its imperative design and control flow. Generally the mini-language is less expressive than the host language. Also, the lazy execution of the function represented by the graph can make it difficult to debug. TensorFlow 1.x is an example of this kind of automatic differentiation.

**tape-based**: This approach tracks the actual composed functions as they are called during execution of the forward pass. One name for this data structure is the *Wengert list*.

$\rightarrow$ With the ordered sequence of computations in hand, it is then possible to walk backward through the list to compute the gradient.

$\rightarrow$ The advantage of this is that it can more easily use all the features of the host language and the imperative execution is easier to understand.

$\rightarrow$ The downside is that it can be more difficult to optimize the code and reuse computations across executions.

[Autograd](https://github.com/HIPS/autograd) is an example of this.
The automatic differentiation in [PyTorch](https://pytorch.org/) also roughly follows this model.

### Re-inventing the Wheel: Introducing `micrograd`

[`micrograd`](https://github.com/karpathy/micrograd) is:

> a tiny Autograd engine (with a bite! :)).
> Implements backpropagation (reverse-mode autodiff) over a dynamically built DAG and a small neural networks library on top of it with a PyTorch-like API.
>
> Both are tiny, with about 100 and 50 lines of code respectively.
>
> The DAG only operates over scalar values, so e.g. we chop up each neuron into all of its individual tiny adds and multiplies. However, this is enough to build up entire deep neural nets doing binary classification. Potentially useful for educational purposes.

```python
def example(this):
    for i in range(190):
        print(map(lambda x: x+2,
                 iter(range(32))))
```

(*from the doc*)

```python
from micrograd.engine import Value

a = Value(-4.0)
b = Value(2.0)
c = a + b
d = a * b + b**3
c += c + 1
c += 1 + c + (-a)
d += d * 2 + (b + a).relu()
d += 3 * d + (b - a).relu()
e = c - d
f = e**2
g = f / 2.0
g += 10.0 / f
print(f'{g.data:.4f}') # prints 24.7041, the outcome of this forward pass
g.backward()
print(f'{a.grad:.4f}') # prints 138.8338, i.e. the numerical value of dg/da
print(f'{b.grad:.4f}') # prints 645.5773, i.e. the numerical value of dg/db
```

```python
from micrograd import nn
n = nn.Neuron(2)
x = [Value(1.0), Value(-2.0)]
y = n(x)
dot = draw_dot(y)
```

### References and Futher Reading:

1. [Automatic Differentiation Step by Step](https://medium.com/@marksaroufim/automatic-differentiation-step-by-step-24240f97a6e6)

2. [Deep Learning with PyTorch (**free sample**) - Luca Antiga et. al.](https://pytorch.org/deep-learning-with-pytorch)

3. [Python Machine Learning, 3rd ed. - Sebastian Raschka](https://sebastianraschka.com/books.html)

4. [(*Paper*) Automatic Differentiation in Machine Learning: a Survey](https://arxiv.org/abs/1502.05767)

```python

```