[astroautomata.com/symbolic_tutorial.html](https://astroautomata.com/symbolic_tutorial.html)

In this tutorial, we will explore some symbolic regression problems with [PySR](https://github.com/MilesCranmer/PySR) and [PySINDy](https://github.com/dynamicslab/pysindy).

# Setup

## Instructions
1. Work on a copy of this notebook: _File_ > _Save a copy in Drive_ (you will need a Google account). Alternatively, you can download the notebook using _File_ > _Download .ipynb_, then upload it to [Colab](https://colab.research.google.com/).
2. Turn on the GPU with Edit->Notebook settings->Hardware accelerator->GPU
3. Execute the following cell (click on it and press Ctrl+Enter) to install Julia, IJulia and other packages (if needed, update `JULIA_VERSION` and the other parameters). This takes a couple of minutes.
4. Continue to the next section.

_Notes_:
* If your Colab Runtime gets reset (e.g., due to inactivity), repeat steps 3, 4.
* After installation, if you want to change the Julia version or activate/deactivate the GPU, you will need to reset the Runtime: _Runtime_ > _Factory reset runtime_ and repeat steps 2-4.

**Run the following code if you need to install Julia**

**(select all lines -> Option-/)**

```python
%%shell
set -e

#---------------------------------------------------#
JULIA_VERSION="1.6.1"
# Which Julia packages to install:
JULIA_PACKAGES="SymbolicRegression"
# In this case, we install github.com/MilesCranmer/SymbolicRegression.jl
JULIA_NUM_THREADS=4
#---------------------------------------------------#

if [ -z `which julia` ]; then
  # Install Julia
  JULIA_VER=`cut -d '.' -f -2 <<< "$JULIA_VERSION"`
  echo "Installing Julia $JULIA_VERSION on the current Colab Runtime..."
  BASE_URL="https://julialang-s3.julialang.org/bin/linux/x64"
  URL="$BASE_URL/$JULIA_VER/julia-$JULIA_VERSION-linux-x86_64.tar.gz"
  wget -nv $URL -O /tmp/julia.tar.gz # -nv means "not verbose"
  tar -x -f /tmp/julia.tar.gz -C /usr/local --strip-components 1
  rm /tmp/julia.tar.gz

  for PKG in `echo $JULIA_PACKAGES`; do
    echo "Installing Julia package $PKG..."
    julia -e 'using Pkg; pkg"add '$PKG'; precompile;"'
  done

fi
```

    Installing Julia 1.6.1 on the current Colab Runtime...
    2021-12-06 19:01:09 URL:https://storage.googleapis.com/julialang2/bin/linux/x64/1.6/julia-1.6.1-linux-x86_64.tar.gz [112784227/112784227] -> "/tmp/julia.tar.gz" [1]
    Installing Julia package SymbolicRegression...
    ┌ Warning: The Pkg REPL interface is intended for interactive use, use with caution from scripts.
    └ @ Pkg.REPLMode /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Pkg/src/REPLMode/REPLMode.jl:378
      Installing known registries into `~/.julia`
           Added registry `General` to `~/.julia/registries/General`
       Resolving package versions...
       Installed SymbolicRegression ────────── v0.6.16
       Installed Referenceables ────────────── v0.1.2
       Installed DefineSingletons ──────────── v0.1.1
       Installed Parameters ────────────────── v0.12.3
       Installed Parsers ───────────────────── v2.1.2
       Installed RecipesBase ───────────────── v1.2.1
       Installed AutoHashEquals ────────────── v0.2.0
       Installed Compat ────────────────────── v3.40.0
       Installed MicroCollections ──────────── v0.1.1
       Installed Preferences ───────────────── v1.2.2
       Installed Reexport ──────────────────── v1.2.2
       Installed DiffResults ───────────────── v1.0.3
       Installed IteratorInterfaceExtensions ─ v1.0.0
       Installed LabelledArrays ────────────── v1.6.7
       Installed Transducers ───────────────── v0.4.67
       Installed LearnBase ─────────────────── v0.4.1
       Installed Requires ──────────────────── v1.1.3
       Installed ChainRulesCore ────────────── v1.11.2
       Installed DiffRules ─────────────────── v1.5.0
       Installed ForwardDiff ───────────────── v0.10.23
       Installed Baselet ───────────────────── v0.1.1
       Installed IrrationalConstants ───────── v0.1.1
       Installed CompositionsBase ──────────── v0.1.1
       Installed DataAPI ───────────────────── v1.9.0
       Installed FiniteDiff ────────────────── v2.8.1
       Installed MacroTools ────────────────── v0.5.9
       Installed DocStringExtensions ───────── v0.8.6
       Installed BangBang ──────────────────── v0.3.32
       Installed ArgCheck ──────────────────── v2.1.0
       Installed DataValueInterfaces ───────── v1.0.0
       Installed ThreadsX ──────────────────── v0.1.8
       Installed MutableArithmetics ────────── v0.2.22
       Installed LineSearches ──────────────── v7.1.1
       Installed TermInterface ─────────────── v0.2.3
       Installed SortingAlgorithms ─────────── v1.0.1
       Installed AbstractTrees ─────────────── v0.3.4
       Installed SymbolicUtils ─────────────── v0.18.1
       Installed StatsBase ─────────────────── v0.33.13
       Installed TableTraits ───────────────── v1.0.1
       Installed Setfield ──────────────────── v0.8.1
       Installed Metatheory ────────────────── v1.3.2
       Installed PositiveFactorizations ────── v0.2.4
       Installed StatsAPI ──────────────────── v1.1.0
       Installed LogExpFunctions ───────────── v0.3.5
       Installed JSON3 ─────────────────────── v1.9.2
       Installed DataStructures ────────────── v0.18.10
       Installed ConstructionBase ──────────── v1.3.0
       Installed MultivariatePolynomials ───── v0.3.18
       Installed Combinatorics ─────────────── v1.0.2
       Installed StaticArrays ──────────────── v1.2.13
       Installed Missings ──────────────────── v1.0.2
       Installed NLSolversBase ─────────────── v7.8.2
       Installed Static ────────────────────── v0.4.0
       Installed Tables ────────────────────── v1.6.0
       Installed ChangesOfVariables ────────── v0.1.1
       Installed SpecialFunctions ──────────── v1.8.1
       Installed ExprTools ─────────────────── v0.1.6
       Installed UnPack ────────────────────── v1.0.2
       Installed FromFile ──────────────────── v0.1.2
       Installed Bijections ────────────────── v0.1.3
       Installed JLLWrappers ───────────────── v1.3.0
       Installed StructTypes ───────────────── v1.8.1
       Installed DynamicPolynomials ────────── v0.3.21
       Installed Optim ─────────────────────── v1.5.0
       Installed TimerOutputs ──────────────── v0.5.13
       Installed IfElse ────────────────────── v0.1.1
       Installed ZygoteRules ───────────────── v0.2.2
       Installed OpenSpecFun_jll ───────────── v0.5.5+0
       Installed NaNMath ───────────────────── v0.3.5
       Installed CommonSubexpressions ──────── v0.3.0
       Installed OrderedCollections ────────── v1.4.1
       Installed Adapt ─────────────────────── v3.3.1
       Installed InverseFunctions ──────────── v0.1.2
       Installed ArrayInterface ────────────── v3.2.1
       Installed SplittablesBase ───────────── v0.1.14
       Installed InitialValues ─────────────── v0.2.11
       Installed FillArrays ────────────────── v0.12.7
       Installed LossFunctions ─────────────── v0.7.2
     Downloading artifact: OpenSpecFun
      Downloaded artifact: OpenSpecFun
        Updating `~/.julia/environments/v1.6/Project.toml`
      [8254be44] + SymbolicRegression v0.6.16
        Updating `~/.julia/environments/v1.6/Manifest.toml`
      [1520ce14] + AbstractTrees v0.3.4
      [79e6a3ab] + Adapt v3.3.1
      [dce04be8] + ArgCheck v2.1.0
      [4fba245c] + ArrayInterface v3.2.1
      [15f4f7f2] + AutoHashEquals v0.2.0
      [198e06fe] + BangBang v0.3.32
      [9718e550] + Baselet v0.1.1
      [e2ed5e7c] + Bijections v0.1.3
      [d360d2e6] + ChainRulesCore v1.11.2
      [9e997f8a] + ChangesOfVariables v0.1.1
      [861a8166] + Combinatorics v1.0.2
      [bbf7d656] + CommonSubexpressions v0.3.0
      [34da2185] + Compat v3.40.0
      [a33af91c] + CompositionsBase v0.1.1
      [187b0558] + ConstructionBase v1.3.0
      [9a962f9c] + DataAPI v1.9.0
      [864edb3b] + DataStructures v0.18.10
      [e2d170a0] + DataValueInterfaces v1.0.0
      [244e2a9f] + DefineSingletons v0.1.1
      [163ba53b] + DiffResults v1.0.3
      [b552c78f] + DiffRules v1.5.0
      [ffbed154] + DocStringExtensions v0.8.6
      [7c1d4256] + DynamicPolynomials v0.3.21
      [e2ba6199] + ExprTools v0.1.6
      [1a297f60] + FillArrays v0.12.7
      [6a86dc24] + FiniteDiff v2.8.1
      [f6369f11] + ForwardDiff v0.10.23
      [ff7dd447] + FromFile v0.1.2
      [615f187c] + IfElse v0.1.1
      [22cec73e] + InitialValues v0.2.11
      [3587e190] + InverseFunctions v0.1.2
      [92d709cd] + IrrationalConstants v0.1.1
      [82899510] + IteratorInterfaceExtensions v1.0.0
      [692b3bcd] + JLLWrappers v1.3.0
      [0f8b85d8] + JSON3 v1.9.2
      [2ee39098] + LabelledArrays v1.6.7
      [7f8f8fb0] + LearnBase v0.4.1
      [d3d80556] + LineSearches v7.1.1
      [2ab3a3ac] + LogExpFunctions v0.3.5
      [30fc2ffe] + LossFunctions v0.7.2
      [1914dd2f] + MacroTools v0.5.9
      [e9d8d322] + Metatheory v1.3.2
      [128add7d] + MicroCollections v0.1.1
      [e1d29d7a] + Missings v1.0.2
      [102ac46a] + MultivariatePolynomials v0.3.18
      [d8a4904e] + MutableArithmetics v0.2.22
      [d41bc354] + NLSolversBase v7.8.2
      [77ba4419] + NaNMath v0.3.5
      [429524aa] + Optim v1.5.0
      [bac558e1] + OrderedCollections v1.4.1
      [d96e819e] + Parameters v0.12.3
      [69de0a69] + Parsers v2.1.2
      [85a6dd25] + PositiveFactorizations v0.2.4
      [21216c6a] + Preferences v1.2.2
      [3cdcf5f2] + RecipesBase v1.2.1
      [189a3867] + Reexport v1.2.2
      [42d2dcc6] + Referenceables v0.1.2
      [ae029012] + Requires v1.1.3
      [efcf1570] + Setfield v0.8.1
      [a2af1166] + SortingAlgorithms v1.0.1
      [276daf66] + SpecialFunctions v1.8.1
      [171d559e] + SplittablesBase v0.1.14
      [aedffcd0] + Static v0.4.0
      [90137ffa] + StaticArrays v1.2.13
      [82ae8749] + StatsAPI v1.1.0
      [2913bbd2] + StatsBase v0.33.13
      [856f2bd8] + StructTypes v1.8.1
      [8254be44] + SymbolicRegression v0.6.16
      [d1185830] + SymbolicUtils v0.18.1
      [3783bdb8] + TableTraits v1.0.1
      [bd369af6] + Tables v1.6.0
      [8ea1fca8] + TermInterface v0.2.3
      [ac1d9e8a] + ThreadsX v0.1.8
      [a759f4b9] + TimerOutputs v0.5.13
      [28d57a85] + Transducers v0.4.67
      [3a884ed6] + UnPack v1.0.2
      [700de1a5] + ZygoteRules v0.2.2
      [efe28fd5] + OpenSpecFun_jll v0.5.5+0
      [0dad84c5] + ArgTools
      [56f22d72] + Artifacts
      [2a0f44e3] + Base64
      [ade2ca70] + Dates
      [8bb1440f] + DelimitedFiles
      [8ba89e20] + Distributed
      [f43a241f] + Downloads
      [9fa8497b] + Future
      [b77e0a4c] + InteractiveUtils
      [b27032c2] + LibCURL
      [76f85450] + LibGit2
      [8f399da3] + Libdl
      [37e2e46d] + LinearAlgebra
      [56ddb016] + Logging
      [d6f4376e] + Markdown
      [a63ad114] + Mmap
      [ca575930] + NetworkOptions
      [44cfe95a] + Pkg
      [de0858da] + Printf
      [3fa0cd96] + REPL
      [9a3f8284] + Random
      [ea8e919c] + SHA
      [9e88b42a] + Serialization
      [1a1011a3] + SharedArrays
      [6462fe0b] + Sockets
      [2f01184e] + SparseArrays
      [10745b16] + Statistics
      [fa267f1f] + TOML
      [a4e569a6] + Tar
      [8dfed614] + Test
      [cf7118a7] + UUIDs
      [4ec0a83e] + Unicode
      [e66e0078] + CompilerSupportLibraries_jll
      [deac9b47] + LibCURL_jll
      [29816b5a] + LibSSH2_jll
      [c8ffd9c3] + MbedTLS_jll
      [14a3606d] + MozillaCACerts_jll
      [05823500] + OpenLibm_jll
      [83775a58] + Zlib_jll
      [8e850ede] + nghttp2_jll
      [3f19e933] + p7zip_jll
    Precompiling project...
      Progress [>                                        ]  0/83
      ◒ Requires
      ◓ TermInterface
      ◓ DataAPI

      ◐ Requires
      ◑ TermInterface
      ◑ DataAPI

      ◓ Requires
      ◒ TermInterface
      ◒ DataAPI

      ◑ Requires
      ◐ TermInterface
      ◐ DataAPI

      ◒ Requires
      ◓ TermInterface
      ◓ DataAPI

      ◐ Requires
      ✓ TermInterface
      ◑ DataAPI
      ◓ Combinatorics

      ✓ Requires
      ✓ TermInterface
      ✓ DataAPI
      ◑ Combinatorics
      ◐ InverseFunctions
      ◓ RecipesBase

      ✓ Requires
      ✓ TermInterface
      ✓ DataAPI
      ◒ Combinatorics
      ◓ InverseFunctions
      ◑ RecipesBase

      ✓ Requires
      ✓ TermInterface
      ✓ DataAPI
      ◐ Combinatorics
      ◑ InverseFunctions
      ◒ RecipesBase

      ✓ Requires
      ✓ TermInterface
      ✓ DataAPI
      ◓ Combinatorics
      ◒ InverseFunctions
      ◐ RecipesBase

      ✓ Requires
      ✓ TermInterface
      ✓ DataAPI
      ◑ Combinatorics
      ◐ InverseFunctions
      ◓ RecipesBase

      ✓ Requires
      ✓ TermInterface
      ✓ DataAPI
      ◒ Combinatorics
      ◓ InverseFunctions
      ◑ RecipesBase

      ✓ Requires
      ✓ TermInterface
      ✓ DataAPI
      ◐ Combinatorics
      ◑ InverseFunctions
      ◒ RecipesBase

      ✓ Requires
      ✓ TermInterface
      ✓ DataAPI
      ◓ Combinatorics
      ◒ InverseFunctions
      ◐ RecipesBase

      ✓ Requires
      ✓ TermInterface
      ✓ DataAPI
      ◑ Combinatorics
      ◐ InverseFunctions
      ◓ RecipesBase

      ✓ Requires
      ✓ TermInterface
      ✓ DataAPI
      ◒ Combinatorics
      ✓ InverseFunctions
      ◑ RecipesBase
      ◐ DataValueInterfaces

      ◐ Combinatorics
      ✓ InverseFunctions
      ◒ RecipesBase
      ◓ DataValueInterfaces

      ◓ Combinatorics
      ✓ InverseFunctions
      ◐ RecipesBase
      ◑ DataValueInterfaces

      ◑ Combinatorics
      ✓ InverseFunctions
      ◓ RecipesBase
      ◒ DataValueInterfaces

      ◒ Combinatorics
      ✓ InverseFunctions
      ✓ RecipesBase
      ◐ DataValueInterfaces
      ◓ ExprTools

      ◐ Combinatorics
      ✓ InverseFunctions
      ✓ RecipesBase
      ◓ DataValueInterfaces
      ◑ ExprTools
      Progress [===>                                     ]  5/83
      ◓ Combinatorics
      ✓ InverseFunctions
      ✓ RecipesBase
      ◑ DataValueInterfaces
      ◒ ExprTools

      ✓ Combinatorics
      ✓ InverseFunctions
      ✓ RecipesBase
      ✓ DataValueInterfaces
      ◐ ExprTools
      ◑ StatsAPI
      ◒ Parsers

      ✓ Combinatorics
      ✓ InverseFunctions
      ✓ RecipesBase
      ✓ DataValueInterfaces
      ◓ ExprTools
      ◒ StatsAPI
      ◐ Parsers

      ✓ Combinatorics
      ✓ InverseFunctions
      ✓ RecipesBase
      ✓ DataValueInterfaces
      ◑ ExprTools
      ◐ StatsAPI
      ◓ Parsers

      ✓ Combinatorics
      ✓ RecipesBase
      ✓ DataValueInterfaces
      ◒ ExprTools
      ◓ StatsAPI
      ◑ Parsers

      ✓ Combinatorics
      ✓ RecipesBase
      ✓ DataValueInterfaces
      ◐ ExprTools
      ◑ StatsAPI
      ◒ Parsers

      ✓ Combinatorics
      ✓ RecipesBase
      ✓ DataValueInterfaces
      ◓ ExprTools
      ✓ StatsAPI
      ◐ Parsers
      ◒ Compat

      ✓ Combinatorics
      ✓ RecipesBase
      ✓ DataValueInterfaces
      ✓ ExprTools
      ✓ StatsAPI
      ◓ Parsers
      ◐ Compat
      ◐ StructTypes

      ✓ Combinatorics
      ✓ RecipesBase
      ✓ DataValueInterfaces
      ✓ ExprTools
      ✓ StatsAPI
      ◑ Parsers
      ◓ Compat
      ◓ StructTypes

      ✓ Combinatorics
      ✓ DataValueInterfaces
      ✓ ExprTools
      ✓ StatsAPI
      ◒ Parsers
      ◑ Compat
      ◑ StructTypes

      ✓ Combinatorics
      ✓ DataValueInterfaces
      ✓ ExprTools
      ✓ StatsAPI
      ◐ Parsers
      ◒ Compat
      ◒ StructTypes

      ✓ Combinatorics
      ✓ DataValueInterfaces
      ✓ ExprTools
      ✓ StatsAPI
      ◓ Parsers
      ◐ Compat
      ◐ StructTypes

      ✓ ExprTools
      ✓ StatsAPI
      ◑ Parsers
      ◓ Compat
      ◓ StructTypes

      ✓ ExprTools
      ✓ StatsAPI
      ◒ Parsers
      ◑ Compat
      ◑ StructTypes

      ✓ ExprTools
      ✓ StatsAPI
      ◐ Parsers
      ◒ Compat
      ◒ StructTypes

      ✓ ExprTools
      ✓ StatsAPI
      ◓ Parsers
      ◐ Compat
      ◐ StructTypes

      ✓ ExprTools
      ◑ Parsers
      ◓ Compat
      ◓ StructTypes

      ◒ Parsers
      ◑ Compat
      ◑ StructTypes

      ◐ Parsers
      ◒ Compat
      ◒ StructTypes

      ◓ Parsers
      ◐ Compat
      ◐ StructTypes
      Progress [=====>                                   ]  9/83
      ◑ Parsers
      ◓ Compat
      ◓ StructTypes

      ◒ Parsers
      ✓ Compat
      ◑ StructTypes
      ◐ UnPack

      ◐ Parsers
      ✓ Compat
      ✓ StructTypes
      ◓ UnPack
      ◓ IteratorInterfaceExtensions

      ◓ Parsers
      ✓ Compat
      ✓ StructTypes
      ◑ UnPack
      ◑ IteratorInterfaceExtensions

      ◑ Parsers
      ✓ Compat
      ✓ StructTypes
      ◒ UnPack
      ◒ IteratorInterfaceExtensions

      ◒ Parsers
      ✓ Compat
      ✓ StructTypes
      ◐ UnPack
      ◐ IteratorInterfaceExtensions

      ◐ Parsers
      ✓ Compat
      ✓ StructTypes
      ◓ UnPack
      ◓ IteratorInterfaceExtensions

      ◓ Parsers
      ✓ Compat
      ✓ StructTypes
      ◑ UnPack
      ◑ IteratorInterfaceExtensions

      ◑ Parsers
      ✓ Compat
      ✓ StructTypes
      ✓ UnPack
      ✓ IteratorInterfaceExtensions
      ◓ OrderedCollections
      ◓ OpenLibm_jll

      ◒ Parsers
      ✓ Compat
      ✓ StructTypes
      ✓ UnPack
      ✓ IteratorInterfaceExtensions
      ◑ OrderedCollections
      ◑ OpenLibm_jll

      ◐ Parsers
      ✓ Compat
      ✓ StructTypes
      ✓ UnPack
      ✓ IteratorInterfaceExtensions
      ◒ OrderedCollections
      ◒ OpenLibm_jll

      ◓ Parsers
      ✓ StructTypes
      ✓ UnPack
      ✓ IteratorInterfaceExtensions
      ◐ OrderedCollections
      ◐ OpenLibm_jll

      ◑ Parsers
      ✓ UnPack
      ✓ IteratorInterfaceExtensions
      ◓ OrderedCollections
      ◓ OpenLibm_jll

      ◒ Parsers
      ✓ UnPack
      ✓ IteratorInterfaceExtensions
      ◑ OrderedCollections
      ◑ OpenLibm_jll

      ◐ Parsers
      ✓ UnPack
      ✓ IteratorInterfaceExtensions
      ◒ OrderedCollections
      ◒ OpenLibm_jll

      ◓ Parsers
      ✓ UnPack
      ✓ IteratorInterfaceExtensions
      ◐ OrderedCollections
      ✓ OpenLibm_jll
      ◐ CompilerSupportLibraries_jll

      ◑ Parsers
      ✓ UnPack
      ✓ IteratorInterfaceExtensions
      ◓ OrderedCollections
      ✓ OpenLibm_jll
      ◓ CompilerSupportLibraries_jll

      ◒ Parsers
      ✓ UnPack
      ✓ IteratorInterfaceExtensions
      ◑ OrderedCollections
      ✓ OpenLibm_jll
      ◑ CompilerSupportLibraries_jll

      ◐ Parsers
      ✓ UnPack
      ✓ IteratorInterfaceExtensions
      ◒ OrderedCollections
      ✓ OpenLibm_jll
      ◒ CompilerSupportLibraries_jll

      ◓ Parsers
      ◐ OrderedCollections
      ✓ OpenLibm_jll
      ◐ CompilerSupportLibraries_jll
      Progress [========>                                ]  15/83
      ◑ Parsers
      ✓ OrderedCollections
      ✓ OpenLibm_jll
      ◓ CompilerSupportLibraries_jll
      ◐ NaNMath

      ◒ Parsers
      ✓ OrderedCollections
      ✓ OpenLibm_jll
      ◑ CompilerSupportLibraries_jll
      ◓ NaNMath

      ◐ Parsers
      ✓ OrderedCollections
      ✓ OpenLibm_jll
      ✓ CompilerSupportLibraries_jll
      ◑ NaNMath
      ◓ IfElse

      ◓ Parsers
      ✓ OrderedCollections
      ✓ OpenLibm_jll
      ✓ CompilerSupportLibraries_jll
      ◒ NaNMath
      ◑ IfElse

      ◑ Parsers
      ✓ OrderedCollections
      ✓ OpenLibm_jll
      ✓ CompilerSupportLibraries_jll
      ◐ NaNMath
      ◒ IfElse

      ◒ Parsers
      ✓ OrderedCollections
      ✓ CompilerSupportLibraries_jll
      ◓ NaNMath
      ◐ IfElse

      ◐ Parsers
      ✓ OrderedCollections
      ✓ CompilerSupportLibraries_jll
      ✓ NaNMath
      ◓ IfElse
      ◑ FillArrays

      ◓ Parsers
      ✓ OrderedCollections
      ✓ CompilerSupportLibraries_jll
      ✓ NaNMath
      ◑ IfElse
      ◒ FillArrays

      ◑ Parsers
      ✓ OrderedCollections
      ✓ CompilerSupportLibraries_jll
      ✓ NaNMath
      ✓ IfElse
      ◐ FillArrays
      ◓ ConstructionBase

      ◒ Parsers
      ✓ OrderedCollections
      ✓ CompilerSupportLibraries_jll
      ✓ NaNMath
      ✓ IfElse
      ◓ FillArrays
      ◑ ConstructionBase

      ◐ Parsers
      ✓ CompilerSupportLibraries_jll
      ✓ NaNMath
      ✓ IfElse
      ◑ FillArrays
      ◒ ConstructionBase

      ◓ Parsers
      ✓ CompilerSupportLibraries_jll
      ✓ NaNMath
      ✓ IfElse
      ◒ FillArrays
      ◐ ConstructionBase

      ◑ Parsers
      ✓ NaNMath
      ✓ IfElse
      ◐ FillArrays
      ◓ ConstructionBase

      ◒ Parsers
      ✓ NaNMath
      ✓ IfElse
      ◓ FillArrays
      ◑ ConstructionBase

      ◐ Parsers
      ✓ NaNMath
      ✓ IfElse
      ◑ FillArrays
      ◒ ConstructionBase

      ◓ Parsers
      ✓ NaNMath
      ✓ IfElse
      ◒ FillArrays
      ◐ ConstructionBase

      ◑ Parsers
      ✓ IfElse
      ◐ FillArrays
      ✓ ConstructionBase
      ◐ Bijections

      ◒ Parsers
      ✓ IfElse
      ◓ FillArrays
      ✓ ConstructionBase
      ◓ Bijections

      ◐ Parsers
      ◑ FillArrays
      ✓ ConstructionBase
      ◑ Bijections

      ◓ Parsers
      ◒ FillArrays
      ✓ ConstructionBase
      ◒ Bijections
      Progress [==========>                              ]  19/83
      ◑ Parsers
      ◐ FillArrays
      ✓ ConstructionBase
      ◐ Bijections

      ◒ Parsers
      ◓ FillArrays
      ✓ ConstructionBase
      ✓ Bijections
      ◑ StaticArrays

      ◐ Parsers
      ◑ FillArrays
      ✓ ConstructionBase
      ✓ Bijections
      ◒ StaticArrays

      ◓ Parsers
      ◒ FillArrays
      ✓ ConstructionBase
      ✓ Bijections
      ◐ StaticArrays

      ◑ Parsers
      ◐ FillArrays
      ✓ ConstructionBase
      ✓ Bijections
      ◓ StaticArrays

      ◒ Parsers
      ◓ FillArrays
      ✓ ConstructionBase
      ✓ Bijections
      ◑ StaticArrays

      ◐ Parsers
      ◑ FillArrays
      ✓ ConstructionBase
      ✓ Bijections
      ◒ StaticArrays

      ◓ Parsers
      ✓ FillArrays
      ✓ Bijections
      ◐ StaticArrays
      ◑ Adapt

      ◑ Parsers
      ✓ FillArrays
      ✓ Bijections
      ◓ StaticArrays
      ◒ Adapt

      ◒ Parsers
      ✓ FillArrays
      ✓ Bijections
      ◑ StaticArrays
      ◐ Adapt

      ◐ Parsers
      ✓ FillArrays
      ✓ Bijections
      ◒ StaticArrays
      ◓ Adapt

      ✓ Parsers
      ✓ FillArrays
      ◐ StaticArrays
      ◑ Adapt
      ◓ DefineSingletons

      ✓ Parsers
      ✓ FillArrays
      ◓ StaticArrays
      ◒ Adapt
      ◑ DefineSingletons

      ✓ Parsers
      ✓ FillArrays
      ◑ StaticArrays
      ◐ Adapt
      ◒ DefineSingletons

      ✓ Parsers
      ✓ FillArrays
      ◒ StaticArrays
      ✓ Adapt
      ◐ DefineSingletons
      ◑ Reexport

      ✓ Parsers
      ✓ FillArrays
      ◐ StaticArrays
      ✓ Adapt
      ◓ DefineSingletons
      ◒ Reexport

      ✓ Parsers
      ✓ FillArrays
      ◓ StaticArrays
      ✓ Adapt
      ◑ DefineSingletons
      ◐ Reexport

      ✓ Parsers
      ✓ FillArrays
      ◑ StaticArrays
      ✓ Adapt
      ◒ DefineSingletons
      ◓ Reexport

      ✓ Parsers
      ◒ StaticArrays
      ✓ Adapt
      ◐ DefineSingletons
      ◑ Reexport

      ✓ Parsers
      ◐ StaticArrays
      ✓ Adapt
      ✓ DefineSingletons
      ◒ Reexport
      ◓ Preferences
      Progress [=============>                           ]  25/83
      ✓ Parsers
      ◓ StaticArrays
      ✓ Adapt
      ✓ DefineSingletons
      ✓ Reexport
      ◑ Preferences
      ◐ Zlib_jll

      ✓ Parsers
      ◑ StaticArrays
      ✓ Adapt
      ✓ DefineSingletons
      ✓ Reexport
      ◒ Preferences
      ◓ Zlib_jll

      ◒ StaticArrays
      ✓ Adapt
      ✓ DefineSingletons
      ✓ Reexport
      ◐ Preferences
      ◑ Zlib_jll

      ◐ StaticArrays
      ✓ Adapt
      ✓ DefineSingletons
      ✓ Reexport
      ◓ Preferences
      ◒ Zlib_jll

      ◓ StaticArrays
      ✓ DefineSingletons
      ✓ Reexport
      ◑ Preferences
      ◐ Zlib_jll

      ◑ StaticArrays
      ✓ DefineSingletons
      ✓ Reexport
      ◒ Preferences
      ◓ Zlib_jll

      ◒ StaticArrays
      ✓ DefineSingletons
      ✓ Reexport
      ✓ Preferences
      ✓ Zlib_jll
      ◓ InitialValues
      ◒ CompositionsBase

      ◐ StaticArrays
      ✓ DefineSingletons
      ✓ Reexport
      ✓ Preferences
      ✓ Zlib_jll
      ◑ InitialValues
      ◐ CompositionsBase

      ◓ StaticArrays
      ✓ DefineSingletons
      ✓ Reexport
      ✓ Preferences
      ✓ Zlib_jll
      ◒ InitialValues
      ◓ CompositionsBase

      ◑ StaticArrays
      ✓ Reexport
      ✓ Preferences
      ✓ Zlib_jll
      ◐ InitialValues
      ◑ CompositionsBase

      ◒ StaticArrays
      ✓ Preferences
      ✓ Zlib_jll
      ◓ InitialValues
      ◒ CompositionsBase

      ◐ StaticArrays
      ✓ Preferences
      ✓ Zlib_jll
      ◑ InitialValues
      ◐ CompositionsBase

      ◓ StaticArrays
      ✓ Preferences
      ✓ Zlib_jll
      ◒ InitialValues
      ◓ CompositionsBase

      ◑ StaticArrays
      ✓ Preferences
      ✓ Zlib_jll
      ◐ InitialValues
      ✓ CompositionsBase
      ◒ DocStringExtensions

      ◒ StaticArrays
      ✓ Preferences
      ✓ Zlib_jll
      ◓ InitialValues
      ✓ CompositionsBase
      ◐ DocStringExtensions

      ◐ StaticArrays
      ✓ Preferences
      ✓ Zlib_jll
      ◑ InitialValues
      ✓ CompositionsBase
      ◓ DocStringExtensions

      ◓ StaticArrays
      ◒ InitialValues
      ✓ CompositionsBase
      ◑ DocStringExtensions

      ◑ StaticArrays
      ◐ InitialValues
      ✓ CompositionsBase
      ◒ DocStringExtensions

      ◒ StaticArrays
      ✓ InitialValues
      ✓ CompositionsBase
      ◐ DocStringExtensions
      ◓ MutableArithmetics

      ◐ StaticArrays
      ✓ InitialValues
      ✓ CompositionsBase
      ◓ DocStringExtensions
      ◑ MutableArithmetics
      Progress [==============>                          ]  29/83
      ◓ StaticArrays
      ✓ InitialValues
      ✓ CompositionsBase
      ◑ DocStringExtensions
      ◒ MutableArithmetics

      ◑ StaticArrays
      ✓ InitialValues
      ✓ CompositionsBase
      ◒ DocStringExtensions
      ◐ MutableArithmetics

      ◒ StaticArrays
      ✓ InitialValues
      ✓ CompositionsBase
      ✓ DocStringExtensions
      ◓ MutableArithmetics
      ◓ ArgCheck

      ◐ StaticArrays
      ✓ InitialValues
      ✓ DocStringExtensions
      ◑ MutableArithmetics
      ◑ ArgCheck

      ◓ StaticArrays
      ✓ InitialValues
      ✓ DocStringExtensions
      ◒ MutableArithmetics
      ◒ ArgCheck

      ◑ StaticArrays
      ✓ InitialValues
      ✓ DocStringExtensions
      ◐ MutableArithmetics
      ◐ ArgCheck

      ◒ StaticArrays
      ✓ InitialValues
      ✓ DocStringExtensions
      ◓ MutableArithmetics
      ◓ ArgCheck

      ◐ StaticArrays
      ✓ InitialValues
      ✓ DocStringExtensions
      ◑ MutableArithmetics
      ◑ ArgCheck

      ◓ StaticArrays
      ✓ InitialValues
      ✓ DocStringExtensions
      ◒ MutableArithmetics
      ◒ ArgCheck

      ◑ StaticArrays
      ✓ DocStringExtensions
      ◐ MutableArithmetics
      ✓ ArgCheck
      ◐ MacroTools

      ◒ StaticArrays
      ✓ DocStringExtensions
      ◓ MutableArithmetics
      ✓ ArgCheck
      ◓ MacroTools

      ◐ StaticArrays
      ✓ DocStringExtensions
      ◑ MutableArithmetics
      ✓ ArgCheck
      ◑ MacroTools

      ◓ StaticArrays
      ◒ MutableArithmetics
      ✓ ArgCheck
      ◒ MacroTools

      ◑ StaticArrays
      ◐ MutableArithmetics
      ✓ ArgCheck
      ◐ MacroTools

      ◒ StaticArrays
      ◓ MutableArithmetics
      ✓ ArgCheck
      ◓ MacroTools

      ◐ StaticArrays
      ◑ MutableArithmetics
      ✓ ArgCheck
      ◑ MacroTools

      ◓ StaticArrays
      ◒ MutableArithmetics
      ✓ ArgCheck
      ◒ MacroTools

      ◑ StaticArrays
      ◐ MutableArithmetics
      ✓ ArgCheck
      ◐ MacroTools

      ◒ StaticArrays
      ◓ MutableArithmetics
      ✓ ArgCheck
      ◓ MacroTools

      ◐ StaticArrays
      ◑ MutableArithmetics
      ◑ MacroTools
      Progress [===============>                         ]  31/83
      ◓ StaticArrays
      ◒ MutableArithmetics
      ◒ MacroTools

      ◑ StaticArrays
      ◐ MutableArithmetics
      ◐ MacroTools

      ◒ StaticArrays
      ◓ MutableArithmetics
      ◓ MacroTools

      ◐ StaticArrays
      ◑ MutableArithmetics
      ◑ MacroTools

      ◓ StaticArrays
      ◒ MutableArithmetics
      ◒ MacroTools

      ◑ StaticArrays
      ◐ MutableArithmetics
      ◐ MacroTools

      ◒ StaticArrays
      ◓ MutableArithmetics
      ◓ MacroTools

      ◐ StaticArrays
      ◑ MutableArithmetics
      ◑ MacroTools

      ◓ StaticArrays
      ◒ MutableArithmetics
      ◒ MacroTools

      ◑ StaticArrays
      ◐ MutableArithmetics
      ◐ MacroTools

      ◒ StaticArrays
      ◓ MutableArithmetics
      ◓ MacroTools

      ◐ StaticArrays
      ◑ MutableArithmetics
      ◑ MacroTools

      ◓ StaticArrays
      ◒ MutableArithmetics
      ◒ MacroTools

      ◑ StaticArrays
      ◐ MutableArithmetics
      ◐ MacroTools

      ◒ StaticArrays
      ◓ MutableArithmetics
      ◓ MacroTools

      ◐ StaticArrays
      ◑ MutableArithmetics
      ◑ MacroTools

      ◓ StaticArrays
      ◒ MutableArithmetics
      ◒ MacroTools

      ◑ StaticArrays
      ◐ MutableArithmetics
      ◐ MacroTools

      ◒ StaticArrays
      ◓ MutableArithmetics
      ◓ MacroTools

      ◐ StaticArrays
      ◑ MutableArithmetics
      ◑ MacroTools
      Progress [===============>                         ]  31/83
      ◓ StaticArrays
      ◒ MutableArithmetics
      ◒ MacroTools

      ◑ StaticArrays
      ◐ MutableArithmetics
      ◐ MacroTools

      ◒ StaticArrays
      ◓ MutableArithmetics
      ◓ MacroTools

      ◐ StaticArrays
      ◑ MutableArithmetics
      ◑ MacroTools

      ◓ StaticArrays
      ◒ MutableArithmetics
      ◒ MacroTools

      ◑ StaticArrays
      ◐ MutableArithmetics
      ✓ MacroTools
      ◐ AutoHashEquals

      ◒ StaticArrays
      ◓ MutableArithmetics
      ✓ MacroTools
      ◓ AutoHashEquals

      ◐ StaticArrays
      ◑ MutableArithmetics
      ✓ MacroTools
      ◑ AutoHashEquals

      ◓ StaticArrays
      ◒ MutableArithmetics
      ✓ MacroTools
      ◒ AutoHashEquals

      ◑ StaticArrays
      ◐ MutableArithmetics
      ✓ MacroTools
      ◐ AutoHashEquals

      ◒ StaticArrays
      ◓ MutableArithmetics
      ✓ MacroTools
      ◓ AutoHashEquals

      ◐ StaticArrays
      ◑ MutableArithmetics
      ✓ MacroTools
      ✓ AutoHashEquals
      ◒ FromFile

      ◓ StaticArrays
      ◒ MutableArithmetics
      ✓ MacroTools
      ✓ AutoHashEquals
      ◐ FromFile

      ◑ StaticArrays
      ◐ MutableArithmetics
      ✓ MacroTools
      ✓ AutoHashEquals
      ◓ FromFile

      ◒ StaticArrays
      ◓ MutableArithmetics
      ✓ MacroTools
      ✓ AutoHashEquals
      ◑ FromFile

      ◐ StaticArrays
      ◑ MutableArithmetics
      ✓ AutoHashEquals
      ◒ FromFile

      ◓ StaticArrays
      ◒ MutableArithmetics
      ✓ AutoHashEquals
      ◐ FromFile

      ◑ StaticArrays
      ◐ MutableArithmetics
      ✓ AutoHashEquals
      ✓ FromFile
      ◐ IrrationalConstants

      ◒ StaticArrays
      ◓ MutableArithmetics
      ✓ AutoHashEquals
      ✓ FromFile
      ◓ IrrationalConstants

      ◐ StaticArrays
      ◑ MutableArithmetics
      ✓ AutoHashEquals
      ✓ FromFile
      ◑ IrrationalConstants
      Progress [=================>                       ]  34/83
      ◓ StaticArrays
      ◒ MutableArithmetics
      ✓ AutoHashEquals
      ✓ FromFile
      ◒ IrrationalConstants

      ◑ StaticArrays
      ◐ MutableArithmetics
      ✓ FromFile
      ◐ IrrationalConstants

      ◒ StaticArrays
      ◓ MutableArithmetics
      ✓ FromFile
      ◓ IrrationalConstants

      ◐ StaticArrays
      ◑ MutableArithmetics
      ✓ FromFile
      ◑ IrrationalConstants

      ◓ StaticArrays
      ◒ MutableArithmetics
      ✓ FromFile
      ◒ IrrationalConstants

      ◑ StaticArrays
      ◐ MutableArithmetics
      ✓ FromFile
      ◐ IrrationalConstants

      ◒ StaticArrays
      ◓ MutableArithmetics
      ✓ FromFile
      ◓ IrrationalConstants

      ◐ StaticArrays
      ◑ MutableArithmetics
      ◑ IrrationalConstants

      ◓ StaticArrays
      ◒ MutableArithmetics
      ◒ IrrationalConstants

      ◑ StaticArrays
      ◐ MutableArithmetics
      ◐ IrrationalConstants

      ◒ StaticArrays
      ◓ MutableArithmetics
      ✓ IrrationalConstants
      ◐ LearnBase

      ◐ StaticArrays
      ◑ MutableArithmetics
      ✓ IrrationalConstants
      ◓ LearnBase

      ◓ StaticArrays
      ◒ MutableArithmetics
      ✓ IrrationalConstants
      ◑ LearnBase

      ◑ StaticArrays
      ◐ MutableArithmetics
      ✓ IrrationalConstants
      ◒ LearnBase

      ◒ StaticArrays
      ◓ MutableArithmetics
      ✓ IrrationalConstants
      ◐ LearnBase

      ◐ StaticArrays
      ◑ MutableArithmetics
      ✓ IrrationalConstants
      ◓ LearnBase

      ◓ StaticArrays
      ◒ MutableArithmetics
      ✓ IrrationalConstants
      ◑ LearnBase

      ◑ StaticArrays
      ◐ MutableArithmetics
      ✓ IrrationalConstants
      ◒ LearnBase

      ◒ StaticArrays
      ◓ MutableArithmetics
      ✓ IrrationalConstants
      ✓ LearnBase
      ◓ MbedTLS_jll

      ◐ StaticArrays
      ◑ MutableArithmetics
      ✓ IrrationalConstants
      ✓ LearnBase
      ◑ MbedTLS_jll
      Progress [==================>                      ]  36/83
      ◓ StaticArrays
      ◒ MutableArithmetics
      ✓ LearnBase
      ◒ MbedTLS_jll

      ◑ StaticArrays
      ◐ MutableArithmetics
      ✓ LearnBase
      ◐ MbedTLS_jll

      ◒ StaticArrays
      ◓ MutableArithmetics
      ✓ LearnBase
      ◓ MbedTLS_jll

      ◐ StaticArrays
      ◑ MutableArithmetics
      ✓ LearnBase
      ✓ MbedTLS_jll
      ◑ AbstractTrees

      ◓ StaticArrays
      ◒ MutableArithmetics
      ✓ LearnBase
      ✓ MbedTLS_jll
      ◒ AbstractTrees

      ◑ StaticArrays
      ◐ MutableArithmetics
      ✓ LearnBase
      ✓ MbedTLS_jll
      ◐ AbstractTrees

      ◒ StaticArrays
      ◓ MutableArithmetics
      ✓ LearnBase
      ✓ MbedTLS_jll
      ◓ AbstractTrees

      ◐ StaticArrays
      ◑ MutableArithmetics
      ✓ LearnBase
      ✓ MbedTLS_jll
      ◑ AbstractTrees

      ◓ StaticArrays
      ◒ MutableArithmetics
      ✓ MbedTLS_jll
      ◒ AbstractTrees

      ◑ StaticArrays
      ◐ MutableArithmetics
      ✓ MbedTLS_jll
      ◐ AbstractTrees

      ◒ StaticArrays
      ◓ MutableArithmetics
      ✓ MbedTLS_jll
      ◓ AbstractTrees

      ◐ StaticArrays
      ◑ MutableArithmetics
      ✓ MbedTLS_jll
      ◑ AbstractTrees

      ✓ StaticArrays
      ◒ MutableArithmetics
      ✓ MbedTLS_jll
      ◒ AbstractTrees
      ◐ Baselet

      ✓ StaticArrays
      ◐ MutableArithmetics
      ◐ AbstractTrees
      ◓ Baselet

      ✓ StaticArrays
      ◓ MutableArithmetics
      ◓ AbstractTrees
      ◑ Baselet

      ✓ StaticArrays
      ✓ MutableArithmetics
      ✓ AbstractTrees
      ◒ Baselet
      ◓ PositiveFactorizations
      ◐ ChangesOfVariables

      ✓ StaticArrays
      ✓ MutableArithmetics
      ✓ AbstractTrees
      ◐ Baselet
      ◑ PositiveFactorizations
      ◓ ChangesOfVariables

      ✓ StaticArrays
      ✓ MutableArithmetics
      ✓ AbstractTrees
      ◓ Baselet
      ◒ PositiveFactorizations
      ◑ ChangesOfVariables

      ✓ StaticArrays
      ✓ MutableArithmetics
      ✓ AbstractTrees
      ◑ Baselet
      ◐ PositiveFactorizations
      ◒ ChangesOfVariables

      ✓ StaticArrays
      ✓ MutableArithmetics
      ✓ AbstractTrees
      ◒ Baselet
      ◓ PositiveFactorizations
      ◐ ChangesOfVariables
      Progress [====================>                    ]  40/83
      ✓ StaticArrays
      ✓ MutableArithmetics
      ✓ AbstractTrees
      ◐ Baselet
      ◑ PositiveFactorizations
      ◓ ChangesOfVariables

      ✓ StaticArrays
      ✓ MutableArithmetics
      ✓ AbstractTrees
      ◓ Baselet
      ◒ PositiveFactorizations
      ◑ ChangesOfVariables

      ✓ MutableArithmetics
      ✓ AbstractTrees
      ◑ Baselet
      ◐ PositiveFactorizations
      ◒ ChangesOfVariables

      ✓ MutableArithmetics
      ✓ AbstractTrees
      ◒ Baselet
      ◓ PositiveFactorizations
      ◐ ChangesOfVariables

      ✓ MutableArithmetics
      ✓ AbstractTrees
      ◐ Baselet
      ◑ PositiveFactorizations
      ◓ ChangesOfVariables

      ◓ Baselet
      ✓ PositiveFactorizations
      ◑ ChangesOfVariables
      ◐ Missings

      ◑ Baselet
      ✓ PositiveFactorizations
      ◒ ChangesOfVariables
      ◓ Missings

      ◒ Baselet
      ✓ PositiveFactorizations
      ◐ ChangesOfVariables
      ◑ Missings

      ✓ Baselet
      ✓ PositiveFactorizations
      ◓ ChangesOfVariables
      ◒ Missings
      ◑ TimerOutputs

      ✓ Baselet
      ✓ PositiveFactorizations
      ◑ ChangesOfVariables
      ◐ Missings
      ◒ TimerOutputs

      ✓ Baselet
      ✓ PositiveFactorizations
      ◒ ChangesOfVariables
      ◓ Missings
      ◐ TimerOutputs

      ✓ Baselet
      ✓ PositiveFactorizations
      ◐ ChangesOfVariables
      ◑ Missings
      ◓ TimerOutputs

      ✓ Baselet
      ✓ PositiveFactorizations
      ◓ ChangesOfVariables
      ◒ Missings
      ◑ TimerOutputs

      ✓ Baselet
      ✓ PositiveFactorizations
      ◑ ChangesOfVariables
      ✓ Missings
      ◒ TimerOutputs
      ◑ ChainRulesCore

      ✓ Baselet
      ✓ PositiveFactorizations
      ✓ ChangesOfVariables
      ✓ Missings
      ◐ TimerOutputs
      ◒ ChainRulesCore
      ◐ TableTraits

      ✓ Baselet
      ✓ ChangesOfVariables
      ✓ Missings
      ◓ TimerOutputs
      ◐ ChainRulesCore
      ◓ TableTraits

      ✓ Baselet
      ✓ ChangesOfVariables
      ✓ Missings
      ◑ TimerOutputs
      ◓ ChainRulesCore
      ◑ TableTraits

      ✓ Baselet
      ✓ ChangesOfVariables
      ✓ Missings
      ◒ TimerOutputs
      ◑ ChainRulesCore
      ◒ TableTraits

      ✓ Baselet
      ✓ ChangesOfVariables
      ✓ Missings
      ◐ TimerOutputs
      ◒ ChainRulesCore
      ◐ TableTraits

      ✓ ChangesOfVariables
      ✓ Missings
      ◓ TimerOutputs
      ◐ ChainRulesCore
      ✓ TableTraits
      ◓ Parameters
      Progress [======================>                  ]  45/83
      ✓ ChangesOfVariables
      ✓ Missings
      ◑ TimerOutputs
      ◓ ChainRulesCore
      ✓ TableTraits
      ◑ Parameters

      ✓ ChangesOfVariables
      ✓ Missings
      ◒ TimerOutputs
      ◑ ChainRulesCore
      ✓ TableTraits
      ◒ Parameters

      ✓ ChangesOfVariables
      ✓ Missings
      ◐ TimerOutputs
      ◒ ChainRulesCore
      ✓ TableTraits
      ◐ Parameters

      ✓ ChangesOfVariables
      ✓ Missings
      ◓ TimerOutputs
      ◐ ChainRulesCore
      ✓ TableTraits
      ◓ Parameters

      ◑ TimerOutputs
      ◓ ChainRulesCore
      ✓ TableTraits
      ◑ Parameters

      ◒ TimerOutputs
      ◑ ChainRulesCore
      ✓ TableTraits
      ◒ Parameters

      ◐ TimerOutputs
      ◒ ChainRulesCore
      ✓ TableTraits
      ◐ Parameters

      ◓ TimerOutputs
      ◐ ChainRulesCore
      ✓ TableTraits
      ◓ Parameters

      ◑ TimerOutputs
      ◓ ChainRulesCore
      ✓ TableTraits
      ✓ Parameters
      ◑ DataStructures

      ✓ TimerOutputs
      ◑ ChainRulesCore
      ✓ Parameters
      ◒ DataStructures
      ◑ Static

      ✓ TimerOutputs
      ◒ ChainRulesCore
      ✓ Parameters
      ◐ DataStructures
      ◒ Static

      ✓ TimerOutputs
      ◐ ChainRulesCore
      ✓ Parameters
      ◓ DataStructures
      ◐ Static

      ✓ TimerOutputs
      ◓ ChainRulesCore
      ✓ Parameters
      ◑ DataStructures
      ◓ Static

      ✓ TimerOutputs
      ◑ ChainRulesCore
      ✓ Parameters
      ◒ DataStructures
      ◑ Static

      ✓ TimerOutputs
      ✓ ChainRulesCore
      ✓ Parameters
      ◐ DataStructures
      ◒ Static
      ◑ JSON3

      ✓ TimerOutputs
      ✓ ChainRulesCore
      ✓ Parameters
      ◓ DataStructures
      ◐ Static
      ◒ JSON3

      ✓ TimerOutputs
      ✓ ChainRulesCore
      ✓ Parameters
      ◑ DataStructures
      ◓ Static
      ◐ JSON3

      ✓ TimerOutputs
      ✓ ChainRulesCore
      ✓ Parameters
      ◒ DataStructures
      ◑ Static
      ◓ JSON3

      ✓ TimerOutputs
      ✓ ChainRulesCore
      ◐ DataStructures
      ◒ Static
      ◑ JSON3

      ✓ TimerOutputs
      ✓ ChainRulesCore
      ◓ DataStructures
      ◐ Static
      ◒ JSON3
      Progress [========================>                ]  48/83
      ✓ ChainRulesCore
      ◑ DataStructures
      ◓ Static
      ◐ JSON3

      ✓ ChainRulesCore
      ◒ DataStructures
      ◑ Static
      ◓ JSON3

      ✓ ChainRulesCore
      ◐ DataStructures
      ◒ Static
      ◑ JSON3

      ✓ ChainRulesCore
      ◓ DataStructures
      ◐ Static
      ◒ JSON3

      ◑ DataStructures
      ◓ Static
      ◐ JSON3

      ◒ DataStructures
      ✓ Static
      ◓ JSON3
      ◓ Referenceables

      ◐ DataStructures
      ✓ Static
      ◑ JSON3
      ◑ Referenceables

      ◓ DataStructures
      ✓ Static
      ◒ JSON3
      ◒ Referenceables

      ◑ DataStructures
      ✓ Static
      ◐ JSON3
      ◐ Referenceables

      ◒ DataStructures
      ✓ Static
      ◓ JSON3
      ◓ Referenceables

      ◐ DataStructures
      ✓ Static
      ◑ JSON3
      ◑ Referenceables

      ◓ DataStructures
      ✓ Static
      ◒ JSON3
      ✓ Referenceables
      ◒ JLLWrappers

      ◑ DataStructures
      ✓ Static
      ◐ JSON3
      ✓ Referenceables
      ◐ JLLWrappers

      ◒ DataStructures
      ✓ Static
      ◓ JSON3
      ✓ Referenceables
      ◓ JLLWrappers

      ◐ DataStructures
      ✓ Static
      ◑ JSON3
      ✓ Referenceables
      ◑ JLLWrappers

      ◓ DataStructures
      ◒ JSON3
      ✓ Referenceables
      ◒ JLLWrappers

      ◑ DataStructures
      ◐ JSON3
      ✓ Referenceables
      ◐ JLLWrappers

      ◒ DataStructures
      ◓ JSON3
      ✓ Referenceables
      ◓ JLLWrappers

      ◐ DataStructures
      ◑ JSON3
      ✓ Referenceables
      ◑ JLLWrappers

      ◓ DataStructures
      ◒ JSON3
      ✓ Referenceables
      ✓ JLLWrappers
      ◒ ZygoteRules
      Progress [=========================>               ]  51/83
      ◑ DataStructures
      ◐ JSON3
      ✓ Referenceables
      ✓ JLLWrappers
      ◐ ZygoteRules

      ◒ DataStructures
      ◓ JSON3
      ✓ JLLWrappers
      ◓ ZygoteRules

      ◐ DataStructures
      ◑ JSON3
      ✓ JLLWrappers
      ◑ ZygoteRules

      ◓ DataStructures
      ◒ JSON3
      ✓ JLLWrappers
      ◒ ZygoteRules

      ◑ DataStructures
      ◐ JSON3
      ✓ JLLWrappers
      ◐ ZygoteRules

      ◒ DataStructures
      ◓ JSON3
      ✓ JLLWrappers
      ◓ ZygoteRules

      ◐ DataStructures
      ◑ JSON3
      ✓ JLLWrappers
      ◑ ZygoteRules

      ◓ DataStructures
      ◒ JSON3
      ✓ JLLWrappers
      ◒ ZygoteRules

      ◑ DataStructures
      ◐ JSON3
      ✓ JLLWrappers
      ◐ ZygoteRules

      ◒ DataStructures
      ◓ JSON3
      ◓ ZygoteRules

      ✓ DataStructures
      ◑ JSON3
      ◑ ZygoteRules
      ◒ CommonSubexpressions

      ✓ DataStructures
      ◒ JSON3
      ◒ ZygoteRules
      ◐ CommonSubexpressions

      ✓ DataStructures
      ◐ JSON3
      ◐ ZygoteRules
      ◓ CommonSubexpressions

      ✓ DataStructures
      ◓ JSON3
      ◓ ZygoteRules
      ◑ CommonSubexpressions

      ✓ DataStructures
      ✓ JSON3
      ◑ ZygoteRules
      ◒ CommonSubexpressions
      ◒ Setfield

      ✓ DataStructures
      ✓ JSON3
      ◒ ZygoteRules
      ◐ CommonSubexpressions
      ◐ Setfield

      ✓ DataStructures
      ✓ JSON3
      ◐ ZygoteRules
      ◓ CommonSubexpressions
      ◓ Setfield

      ✓ DataStructures
      ✓ JSON3
      ◓ ZygoteRules
      ◑ CommonSubexpressions
      ◑ Setfield

      ✓ DataStructures
      ✓ JSON3
      ◑ ZygoteRules
      ◒ CommonSubexpressions
      ◒ Setfield

      ✓ DataStructures
      ✓ JSON3
      ✓ ZygoteRules
      ◐ CommonSubexpressions
      ◐ Setfield
      ◓ LibSSH2_jll
      Progress [===========================>             ]  54/83
      ✓ JSON3
      ✓ ZygoteRules
      ◓ CommonSubexpressions
      ◓ Setfield
      ◑ LibSSH2_jll

      ✓ JSON3
      ✓ ZygoteRules
      ◑ CommonSubexpressions
      ◑ Setfield
      ◒ LibSSH2_jll

      ✓ JSON3
      ✓ ZygoteRules
      ◒ CommonSubexpressions
      ◒ Setfield
      ◐ LibSSH2_jll

      ✓ JSON3
      ✓ ZygoteRules
      ◐ CommonSubexpressions
      ◐ Setfield
      ◓ LibSSH2_jll

      ✓ ZygoteRules
      ✓ CommonSubexpressions
      ◓ Setfield
      ◑ LibSSH2_jll
      ◑ DiffResults

      ✓ ZygoteRules
      ✓ CommonSubexpressions
      ◑ Setfield
      ✓ LibSSH2_jll
      ◒ DiffResults
      ◒ Tables

      ✓ ZygoteRules
      ✓ CommonSubexpressions
      ◒ Setfield
      ✓ LibSSH2_jll
      ◐ DiffResults
      ◐ Tables

      ✓ ZygoteRules
      ✓ CommonSubexpressions
      ◐ Setfield
      ✓ LibSSH2_jll
      ◓ DiffResults
      ◓ Tables

      ✓ ZygoteRules
      ✓ CommonSubexpressions
      ◓ Setfield
      ✓ LibSSH2_jll
      ◑ DiffResults
      ◑ Tables

      ✓ CommonSubexpressions
      ◑ Setfield
      ✓ LibSSH2_jll
      ◒ DiffResults
      ◒ Tables

      ✓ CommonSubexpressions
      ◒ Setfield
      ✓ LibSSH2_jll
      ◐ DiffResults
      ◐ Tables

      ✓ CommonSubexpressions
      ◐ Setfield
      ✓ LibSSH2_jll
      ◓ DiffResults
      ◓ Tables

      ✓ CommonSubexpressions
      ◓ Setfield
      ✓ LibSSH2_jll
      ◑ DiffResults
      ◑ Tables

      ✓ CommonSubexpressions
      ◑ Setfield
      ✓ LibSSH2_jll
      ◒ DiffResults
      ◒ Tables

      ✓ CommonSubexpressions
      ◒ Setfield
      ✓ LibSSH2_jll
      ◐ DiffResults
      ◐ Tables

      ◐ Setfield
      ◓ DiffResults
      ◓ Tables

      ◓ Setfield
      ◑ DiffResults
      ◑ Tables

      ◑ Setfield
      ◒ DiffResults
      ◒ Tables

      ◒ Setfield
      ◐ DiffResults
      ✓ Tables
      ◐ LogExpFunctions

      ✓ Setfield
      ◓ DiffResults
      ✓ Tables
      ◓ LogExpFunctions
      ◑ ArrayInterface
      Progress [============================>            ]  58/83
      ✓ Setfield
      ◑ DiffResults
      ✓ Tables
      ◑ LogExpFunctions
      ◒ ArrayInterface

      ✓ Setfield
      ◒ DiffResults
      ✓ Tables
      ◒ LogExpFunctions
      ◐ ArrayInterface

      ✓ Setfield
      ◐ DiffResults
      ✓ Tables
      ◐ LogExpFunctions
      ◓ ArrayInterface

      ✓ Setfield
      ◓ DiffResults
      ✓ Tables
      ◓ LogExpFunctions
      ◑ ArrayInterface

      ✓ Setfield
      ◑ DiffResults
      ✓ Tables
      ◑ LogExpFunctions
      ◒ ArrayInterface

      ✓ Setfield
      ◒ DiffResults
      ✓ Tables
      ◒ LogExpFunctions
      ◐ ArrayInterface

      ✓ Setfield
      ◐ DiffResults
      ✓ Tables
      ◐ LogExpFunctions
      ◓ ArrayInterface

      ✓ Setfield
      ◓ DiffResults
      ✓ Tables
      ◓ LogExpFunctions
      ◑ ArrayInterface

      ✓ Setfield
      ◑ DiffResults
      ◑ LogExpFunctions
      ◒ ArrayInterface

      ◒ DiffResults
      ◒ LogExpFunctions
      ◐ ArrayInterface

      ◐ DiffResults
      ◐ LogExpFunctions
      ◓ ArrayInterface

      ◓ DiffResults
      ◓ LogExpFunctions
      ◑ ArrayInterface

      ◑ DiffResults
      ◑ LogExpFunctions
      ◒ ArrayInterface

      ✓ DiffResults
      ◒ LogExpFunctions
      ◐ ArrayInterface
      ◑ OpenSpecFun_jll

      ✓ DiffResults
      ◐ LogExpFunctions
      ◓ ArrayInterface
      ◒ OpenSpecFun_jll

      ✓ DiffResults
      ◓ LogExpFunctions
      ◑ ArrayInterface
      ◐ OpenSpecFun_jll

      ✓ DiffResults
      ◑ LogExpFunctions
      ◒ ArrayInterface
      ◓ OpenSpecFun_jll

      ✓ DiffResults
      ◒ LogExpFunctions
      ◐ ArrayInterface
      ◑ OpenSpecFun_jll

      ✓ DiffResults
      ◐ LogExpFunctions
      ◓ ArrayInterface
      ◒ OpenSpecFun_jll

      ✓ DiffResults
      ◓ LogExpFunctions
      ◑ ArrayInterface
      ◐ OpenSpecFun_jll
      Progress [=============================>           ]  59/83
      ✓ DiffResults
      ◑ LogExpFunctions
      ◒ ArrayInterface
      ◓ OpenSpecFun_jll

      ✓ DiffResults
      ◒ LogExpFunctions
      ◐ ArrayInterface
      ◑ OpenSpecFun_jll

      ✓ DiffResults
      ◐ LogExpFunctions
      ✓ ArrayInterface
      ◒ OpenSpecFun_jll
      ◒ SortingAlgorithms

      ✓ DiffResults
      ◓ LogExpFunctions
      ✓ ArrayInterface
      ◐ OpenSpecFun_jll
      ◐ SortingAlgorithms

      ◑ LogExpFunctions
      ✓ ArrayInterface
      ◓ OpenSpecFun_jll
      ◓ SortingAlgorithms

      ◒ LogExpFunctions
      ✓ ArrayInterface
      ◑ OpenSpecFun_jll
      ◑ SortingAlgorithms

      ◐ LogExpFunctions
      ✓ ArrayInterface
      ◒ OpenSpecFun_jll
      ◒ SortingAlgorithms

      ◓ LogExpFunctions
      ✓ ArrayInterface
      ◐ OpenSpecFun_jll
      ◐ SortingAlgorithms

      ◑ LogExpFunctions
      ✓ ArrayInterface
      ◓ OpenSpecFun_jll
      ◓ SortingAlgorithms

      ◒ LogExpFunctions
      ✓ ArrayInterface
      ◑ OpenSpecFun_jll
      ◑ SortingAlgorithms

      ◐ LogExpFunctions
      ✓ ArrayInterface
      ◒ OpenSpecFun_jll
      ◒ SortingAlgorithms

      ◓ LogExpFunctions
      ✓ ArrayInterface
      ◐ OpenSpecFun_jll
      ◐ SortingAlgorithms

      ◑ LogExpFunctions
      ◓ OpenSpecFun_jll
      ✓ SortingAlgorithms
      ◒ MultivariatePolynomials

      ◒ LogExpFunctions
      ✓ OpenSpecFun_jll
      ✓ SortingAlgorithms
      ◐ MultivariatePolynomials
      ◑ SplittablesBase

      ◐ LogExpFunctions
      ✓ OpenSpecFun_jll
      ✓ SortingAlgorithms
      ◓ MultivariatePolynomials
      ◒ SplittablesBase

      ✓ LogExpFunctions
      ✓ OpenSpecFun_jll
      ✓ SortingAlgorithms
      ◑ MultivariatePolynomials
      ◐ SplittablesBase
      ◒ BangBang

      ✓ LogExpFunctions
      ✓ OpenSpecFun_jll
      ✓ SortingAlgorithms
      ◒ MultivariatePolynomials
      ◓ SplittablesBase
      ◐ BangBang

      ✓ LogExpFunctions
      ✓ OpenSpecFun_jll
      ✓ SortingAlgorithms
      ◐ MultivariatePolynomials
      ◑ SplittablesBase
      ◓ BangBang

      ✓ LogExpFunctions
      ✓ OpenSpecFun_jll
      ✓ SortingAlgorithms
      ◓ MultivariatePolynomials
      ◒ SplittablesBase
      ◑ BangBang

      ✓ LogExpFunctions
      ✓ OpenSpecFun_jll
      ✓ SortingAlgorithms
      ◑ MultivariatePolynomials
      ◐ SplittablesBase
      ◒ BangBang
      Progress [===============================>         ]  63/83
      ✓ LogExpFunctions
      ✓ OpenSpecFun_jll
      ✓ SortingAlgorithms
      ◒ MultivariatePolynomials
      ◓ SplittablesBase
      ◐ BangBang

      ✓ LogExpFunctions
      ✓ OpenSpecFun_jll
      ✓ SortingAlgorithms
      ◐ MultivariatePolynomials
      ◑ SplittablesBase
      ◓ BangBang

      ✓ LogExpFunctions
      ✓ OpenSpecFun_jll
      ◓ MultivariatePolynomials
      ◒ SplittablesBase
      ◑ BangBang

      ✓ LogExpFunctions
      ◑ MultivariatePolynomials
      ◐ SplittablesBase
      ◒ BangBang

      ✓ LogExpFunctions
      ◒ MultivariatePolynomials
      ◓ SplittablesBase
      ◐ BangBang

      ◐ MultivariatePolynomials
      ◑ SplittablesBase
      ◓ BangBang

      ◓ MultivariatePolynomials
      ◒ SplittablesBase
      ◑ BangBang

      ◑ MultivariatePolynomials
      ◐ SplittablesBase
      ◒ BangBang

      ◒ MultivariatePolynomials
      ◓ SplittablesBase
      ◐ BangBang

      ◐ MultivariatePolynomials
      ◑ SplittablesBase
      ◓ BangBang

      ◓ MultivariatePolynomials
      ◒ SplittablesBase
      ◑ BangBang

      ◑ MultivariatePolynomials
      ◐ SplittablesBase
      ◒ BangBang

      ◒ MultivariatePolynomials
      ◓ SplittablesBase
      ◐ BangBang

      ◐ MultivariatePolynomials
      ◑ SplittablesBase
      ◓ BangBang

      ◓ MultivariatePolynomials
      ◒ SplittablesBase
      ◑ BangBang

      ◑ MultivariatePolynomials
      ◐ SplittablesBase
      ◒ BangBang

      ◒ MultivariatePolynomials
      ◓ SplittablesBase
      ◐ BangBang

      ◐ MultivariatePolynomials
      ✓ SplittablesBase
      ◓ BangBang
      ◓ FiniteDiff

      ◓ MultivariatePolynomials
      ✓ SplittablesBase
      ◑ BangBang
      ◑ FiniteDiff

      ◑ MultivariatePolynomials
      ✓ SplittablesBase
      ◒ BangBang
      ◒ FiniteDiff
      Progress [===============================>         ]  64/83
      ◒ MultivariatePolynomials
      ✓ SplittablesBase
      ◐ BangBang
      ◐ FiniteDiff

      ◐ MultivariatePolynomials
      ✓ SplittablesBase
      ◓ BangBang
      ◓ FiniteDiff

      ◓ MultivariatePolynomials
      ✓ SplittablesBase
      ◑ BangBang
      ◑ FiniteDiff

      ◑ MultivariatePolynomials
      ✓ SplittablesBase
      ◒ BangBang
      ◒ FiniteDiff

      ◒ MultivariatePolynomials
      ✓ SplittablesBase
      ◐ BangBang
      ◐ FiniteDiff

      ◐ MultivariatePolynomials
      ✓ SplittablesBase
      ◓ BangBang
      ◓ FiniteDiff

      ◓ MultivariatePolynomials
      ✓ SplittablesBase
      ◑ BangBang
      ◑ FiniteDiff

      ◑ MultivariatePolynomials
      ◒ BangBang
      ◒ FiniteDiff

      ✓ MultivariatePolynomials
      ◐ BangBang
      ◐ FiniteDiff
      ◑ LabelledArrays

      ✓ MultivariatePolynomials
      ◓ BangBang
      ◓ FiniteDiff
      ◒ LabelledArrays

      ✓ MultivariatePolynomials
      ◑ BangBang
      ◑ FiniteDiff
      ◐ LabelledArrays

      ✓ MultivariatePolynomials
      ◒ BangBang
      ◒ FiniteDiff
      ◓ LabelledArrays

      ✓ MultivariatePolynomials
      ◐ BangBang
      ◐ FiniteDiff
      ◑ LabelledArrays

      ✓ MultivariatePolynomials
      ◓ BangBang
      ◓ FiniteDiff
      ◒ LabelledArrays

      ✓ MultivariatePolynomials
      ✓ BangBang
      ◑ FiniteDiff
      ◐ LabelledArrays
      ◒ StatsBase

      ✓ MultivariatePolynomials
      ✓ BangBang
      ◒ FiniteDiff
      ◓ LabelledArrays
      ◐ StatsBase

      ✓ MultivariatePolynomials
      ✓ BangBang
      ◐ FiniteDiff
      ◑ LabelledArrays
      ◓ StatsBase

      ✓ MultivariatePolynomials
      ✓ BangBang
      ◓ FiniteDiff
      ◒ LabelledArrays
      ◑ StatsBase

      ✓ BangBang
      ◑ FiniteDiff
      ◐ LabelledArrays
      ◒ StatsBase

      ✓ BangBang
      ◒ FiniteDiff
      ◓ LabelledArrays
      ◐ StatsBase
      Progress [================================>        ]  66/83
      ✓ BangBang
      ◐ FiniteDiff
      ◑ LabelledArrays
      ◓ StatsBase

      ✓ BangBang
      ◓ FiniteDiff
      ◒ LabelledArrays
      ◑ StatsBase

      ✓ BangBang
      ◑ FiniteDiff
      ◐ LabelledArrays
      ◒ StatsBase

      ✓ BangBang
      ◒ FiniteDiff
      ◓ LabelledArrays
      ◐ StatsBase

      ◐ FiniteDiff
      ◑ LabelledArrays
      ◓ StatsBase

      ◓ FiniteDiff
      ◒ LabelledArrays
      ◑ StatsBase

      ◑ FiniteDiff
      ◐ LabelledArrays
      ◒ StatsBase

      ◒ FiniteDiff
      ◓ LabelledArrays
      ◐ StatsBase

      ◐ FiniteDiff
      ◑ LabelledArrays
      ◓ StatsBase

      ◓ FiniteDiff
      ◒ LabelledArrays
      ◑ StatsBase

      ◑ FiniteDiff
      ◐ LabelledArrays
      ◒ StatsBase

      ◒ FiniteDiff
      ◓ LabelledArrays
      ◐ StatsBase

      ◐ FiniteDiff
      ◑ LabelledArrays
      ◓ StatsBase

      ✓ FiniteDiff
      ◒ LabelledArrays
      ◑ StatsBase
      ◑ SpecialFunctions

      ✓ FiniteDiff
      ◐ LabelledArrays
      ◒ StatsBase
      ◒ SpecialFunctions

      ✓ FiniteDiff
      ◓ LabelledArrays
      ◐ StatsBase
      ◐ SpecialFunctions

      ✓ FiniteDiff
      ◑ LabelledArrays
      ◓ StatsBase
      ◓ SpecialFunctions

      ✓ FiniteDiff
      ◒ LabelledArrays
      ◑ StatsBase
      ◑ SpecialFunctions

      ✓ FiniteDiff
      ◐ LabelledArrays
      ◒ StatsBase
      ◒ SpecialFunctions

      ✓ FiniteDiff
      ◓ LabelledArrays
      ◐ StatsBase
      ◐ SpecialFunctions
      Progress [=================================>       ]  67/83
      ✓ FiniteDiff
      ◑ LabelledArrays
      ◓ StatsBase
      ◓ SpecialFunctions

      ✓ FiniteDiff
      ◒ LabelledArrays
      ◑ StatsBase
      ◑ SpecialFunctions

      ✓ FiniteDiff
      ◐ LabelledArrays
      ◒ StatsBase
      ◒ SpecialFunctions

      ◓ LabelledArrays
      ◐ StatsBase
      ◐ SpecialFunctions

      ◑ LabelledArrays
      ◓ StatsBase
      ◓ SpecialFunctions

      ◒ LabelledArrays
      ◑ StatsBase
      ◑ SpecialFunctions

      ◐ LabelledArrays
      ◒ StatsBase
      ◒ SpecialFunctions

      ◓ LabelledArrays
      ◐ StatsBase
      ◐ SpecialFunctions

      ✓ LabelledArrays
      ◓ StatsBase
      ◓ SpecialFunctions
      ◑ DynamicPolynomials

      ✓ LabelledArrays
      ◑ StatsBase
      ◑ SpecialFunctions
      ◒ DynamicPolynomials

      ✓ LabelledArrays
      ◒ StatsBase
      ◒ SpecialFunctions
      ◐ DynamicPolynomials

      ✓ LabelledArrays
      ◐ StatsBase
      ◐ SpecialFunctions
      ◓ DynamicPolynomials

      ✓ LabelledArrays
      ◓ StatsBase
      ◓ SpecialFunctions
      ◑ DynamicPolynomials

      ✓ LabelledArrays
      ◑ StatsBase
      ◑ SpecialFunctions
      ◒ DynamicPolynomials

      ✓ LabelledArrays
      ◒ StatsBase
      ◒ SpecialFunctions
      ◐ DynamicPolynomials

      ✓ LabelledArrays
      ✓ StatsBase
      ◐ SpecialFunctions
      ◓ DynamicPolynomials
      ◑ MicroCollections

      ✓ LabelledArrays
      ✓ StatsBase
      ◓ SpecialFunctions
      ◑ DynamicPolynomials
      ◒ MicroCollections

      ✓ LabelledArrays
      ✓ StatsBase
      ◑ SpecialFunctions
      ◒ DynamicPolynomials
      ◐ MicroCollections

      ✓ StatsBase
      ◒ SpecialFunctions
      ◐ DynamicPolynomials
      ◓ MicroCollections

      ✓ StatsBase
      ◐ SpecialFunctions
      ◓ DynamicPolynomials
      ◑ MicroCollections
      Progress [==================================>      ]  69/83
      ✓ StatsBase
      ◓ SpecialFunctions
      ◑ DynamicPolynomials
      ◒ MicroCollections

      ✓ StatsBase
      ◑ SpecialFunctions
      ◒ DynamicPolynomials
      ◐ MicroCollections

      ✓ StatsBase
      ◒ SpecialFunctions
      ◐ DynamicPolynomials
      ◓ MicroCollections

      ✓ StatsBase
      ◐ SpecialFunctions
      ◓ DynamicPolynomials
      ◑ MicroCollections

      ✓ StatsBase
      ◓ SpecialFunctions
      ◑ DynamicPolynomials
      ◒ MicroCollections

      ◑ SpecialFunctions
      ◒ DynamicPolynomials
      ◐ MicroCollections

      ◒ SpecialFunctions
      ◐ DynamicPolynomials
      ◓ MicroCollections

      ◐ SpecialFunctions
      ◓ DynamicPolynomials
      ◑ MicroCollections

      ◓ SpecialFunctions
      ◑ DynamicPolynomials
      ◒ MicroCollections

      ◑ SpecialFunctions
      ◒ DynamicPolynomials
      ◐ MicroCollections

      ◒ SpecialFunctions
      ◐ DynamicPolynomials
      ◓ MicroCollections

      ◐ SpecialFunctions
      ◓ DynamicPolynomials
      ◑ MicroCollections

      ◓ SpecialFunctions
      ◑ DynamicPolynomials
      ◒ MicroCollections

      ◑ SpecialFunctions
      ◒ DynamicPolynomials
      ◐ MicroCollections

      ◒ SpecialFunctions
      ◐ DynamicPolynomials
      ✓ MicroCollections
      ◐ LossFunctions

      ◐ SpecialFunctions
      ◓ DynamicPolynomials
      ✓ MicroCollections
      ◓ LossFunctions

      ◓ SpecialFunctions
      ◑ DynamicPolynomials
      ✓ MicroCollections
      ◑ LossFunctions

      ◑ SpecialFunctions
      ◒ DynamicPolynomials
      ✓ MicroCollections
      ◒ LossFunctions

      ◒ SpecialFunctions
      ◐ DynamicPolynomials
      ✓ MicroCollections
      ◐ LossFunctions

      ◐ SpecialFunctions
      ✓ DynamicPolynomials
      ✓ MicroCollections
      ◓ LossFunctions
      ◓ Transducers
      Progress [===================================>     ]  71/83
      ◓ SpecialFunctions
      ✓ DynamicPolynomials
      ✓ MicroCollections
      ◑ LossFunctions
      ◑ Transducers

      ◑ SpecialFunctions
      ✓ DynamicPolynomials
      ✓ MicroCollections
      ◒ LossFunctions
      ◒ Transducers

      ◒ SpecialFunctions
      ✓ DynamicPolynomials
      ✓ MicroCollections
      ◐ LossFunctions
      ◐ Transducers

      ◐ SpecialFunctions
      ✓ DynamicPolynomials
      ✓ MicroCollections
      ◓ LossFunctions
      ◓ Transducers

      ◓ SpecialFunctions
      ✓ DynamicPolynomials
      ◑ LossFunctions
      ◑ Transducers

      ◑ SpecialFunctions
      ✓ DynamicPolynomials
      ◒ LossFunctions
      ◒ Transducers

      ◒ SpecialFunctions
      ✓ DynamicPolynomials
      ◐ LossFunctions
      ◐ Transducers

      ◐ SpecialFunctions
      ✓ DynamicPolynomials
      ◓ LossFunctions
      ◓ Transducers

      ◓ SpecialFunctions
      ✓ DynamicPolynomials
      ◑ LossFunctions
      ◑ Transducers

      ◑ SpecialFunctions
      ◒ LossFunctions
      ◒ Transducers

      ◒ SpecialFunctions
      ◐ LossFunctions
      ◐ Transducers

      ◐ SpecialFunctions
      ◓ LossFunctions
      ◓ Transducers

      ◓ SpecialFunctions
      ◑ LossFunctions
      ◑ Transducers

      ◑ SpecialFunctions
      ◒ LossFunctions
      ◒ Transducers

      ◒ SpecialFunctions
      ◐ LossFunctions
      ◐ Transducers

      ◐ SpecialFunctions
      ◓ LossFunctions
      ◓ Transducers

      ◓ SpecialFunctions
      ◑ LossFunctions
      ◑ Transducers

      ◑ SpecialFunctions
      ◒ LossFunctions
      ◒ Transducers

      ◒ SpecialFunctions
      ◐ LossFunctions
      ◐ Transducers

      ◐ SpecialFunctions
      ◓ LossFunctions
      ◓ Transducers
      Progress [===================================>     ]  71/83
      ◓ SpecialFunctions
      ◑ LossFunctions
      ◑ Transducers

      ◑ SpecialFunctions
      ✓ LossFunctions
      ◒ Transducers

      ◒ SpecialFunctions
      ✓ LossFunctions
      ◐ Transducers

      ◐ SpecialFunctions
      ✓ LossFunctions
      ◓ Transducers

      ◓ SpecialFunctions
      ✓ LossFunctions
      ◑ Transducers

      ◑ SpecialFunctions
      ✓ LossFunctions
      ◒ Transducers

      ◒ SpecialFunctions
      ✓ LossFunctions
      ◐ Transducers

      ◐ SpecialFunctions
      ✓ LossFunctions
      ◓ Transducers

      ◓ SpecialFunctions
      ✓ LossFunctions
      ◑ Transducers

      ◑ SpecialFunctions
      ✓ LossFunctions
      ◒ Transducers

      ◒ SpecialFunctions
      ✓ LossFunctions
      ◐ Transducers

      ◐ SpecialFunctions
      ✓ LossFunctions
      ◓ Transducers

      ◓ SpecialFunctions
      ◑ Transducers

      ◑ SpecialFunctions
      ◒ Transducers

      ◒ SpecialFunctions
      ◐ Transducers

      ◐ SpecialFunctions
      ◓ Transducers

      ✓ SpecialFunctions
      ◑ Transducers
      ◑ DiffRules

      ✓ SpecialFunctions
      ◒ Transducers
      ◒ DiffRules

      ✓ SpecialFunctions
      ◐ Transducers
      ◐ DiffRules

      ✓ SpecialFunctions
      ◓ Transducers
      ◓ DiffRules
      Progress [====================================>    ]  73/83
      ✓ SpecialFunctions
      ◑ Transducers
      ◑ DiffRules

      ✓ SpecialFunctions
      ◒ Transducers
      ◒ DiffRules

      ✓ SpecialFunctions
      ◐ Transducers
      ◐ DiffRules

      ✓ SpecialFunctions
      ◓ Transducers
      ◓ DiffRules

      ✓ SpecialFunctions
      ◑ Transducers
      ◑ DiffRules

      ✓ SpecialFunctions
      ✓ Transducers
      ◒ DiffRules
      ◒ ThreadsX

      ✓ Transducers
      ✓ DiffRules
      ◐ ThreadsX
      ◑ ForwardDiff

      ✓ Transducers
      ✓ DiffRules
      ◓ ThreadsX
      ◒ ForwardDiff

      ✓ Transducers
      ✓ DiffRules
      ◑ ThreadsX
      ◐ ForwardDiff

      ✓ Transducers
      ✓ DiffRules
      ◒ ThreadsX
      ◓ ForwardDiff

      ✓ Transducers
      ✓ DiffRules
      ◐ ThreadsX
      ◑ ForwardDiff

      ✓ Transducers
      ✓ DiffRules
      ◓ ThreadsX
      ◒ ForwardDiff

      ✓ Transducers
      ✓ DiffRules
      ◑ ThreadsX
      ◐ ForwardDiff

      ✓ Transducers
      ✓ DiffRules
      ◒ ThreadsX
      ◓ ForwardDiff

      ✓ Transducers
      ✓ DiffRules
      ◐ ThreadsX
      ◑ ForwardDiff

      ✓ DiffRules
      ◓ ThreadsX
      ◒ ForwardDiff

      ◑ ThreadsX
      ◐ ForwardDiff

      ◒ ThreadsX
      ◓ ForwardDiff

      ◐ ThreadsX
      ◑ ForwardDiff

      ◓ ThreadsX
      ◒ ForwardDiff
      Progress [=====================================>   ]  75/83
      ◑ ThreadsX
      ◐ ForwardDiff

      ◒ ThreadsX
      ◓ ForwardDiff

      ◐ ThreadsX
      ◑ ForwardDiff

      ◓ ThreadsX
      ◒ ForwardDiff

      ◑ ThreadsX
      ◐ ForwardDiff

      ◒ ThreadsX
      ◓ ForwardDiff

      ◐ ThreadsX
      ◑ ForwardDiff

      ✓ ThreadsX
      ◒ ForwardDiff
      ◑ Metatheory

      ✓ ThreadsX
      ◐ ForwardDiff
      ◒ Metatheory

      ✓ ThreadsX
      ◓ ForwardDiff
      ◐ Metatheory

      ✓ ThreadsX
      ◑ ForwardDiff
      ◓ Metatheory

      ✓ ThreadsX
      ◒ ForwardDiff
      ◑ Metatheory

      ✓ ThreadsX
      ◐ ForwardDiff
      ◒ Metatheory

      ✓ ThreadsX
      ◓ ForwardDiff
      ◐ Metatheory

      ✓ ThreadsX
      ◑ ForwardDiff
      ◓ Metatheory

      ✓ ThreadsX
      ◒ ForwardDiff
      ◑ Metatheory

      ✓ ThreadsX
      ◐ ForwardDiff
      ◒ Metatheory

      ◓ ForwardDiff
      ◐ Metatheory

      ◑ ForwardDiff
      ◓ Metatheory

      ◒ ForwardDiff
      ◑ Metatheory
      Progress [=====================================>   ]  76/83
      ◐ ForwardDiff
      ◒ Metatheory

      ◓ ForwardDiff
      ◐ Metatheory

      ◑ ForwardDiff
      ◓ Metatheory

      ◒ ForwardDiff
      ◑ Metatheory

      ◐ ForwardDiff
      ◒ Metatheory

      ◓ ForwardDiff
      ◐ Metatheory

      ◑ ForwardDiff
      ◓ Metatheory

      ◒ ForwardDiff
      ◑ Metatheory

      ◐ ForwardDiff
      ◒ Metatheory

      ◓ ForwardDiff
      ◐ Metatheory

      ◑ ForwardDiff
      ◓ Metatheory

      ◒ ForwardDiff
      ◑ Metatheory

      ◐ ForwardDiff
      ◒ Metatheory

      ◓ ForwardDiff
      ◐ Metatheory

      ◑ ForwardDiff
      ◓ Metatheory

      ◒ ForwardDiff
      ◑ Metatheory

      ◐ ForwardDiff
      ◒ Metatheory

      ◓ ForwardDiff
      ◐ Metatheory

      ◑ ForwardDiff
      ◓ Metatheory

      ◒ ForwardDiff
      ◑ Metatheory
      Progress [=====================================>   ]  76/83
      ◐ ForwardDiff
      ◒ Metatheory

      ✓ ForwardDiff
      ◐ Metatheory
      ◓ NLSolversBase

      ✓ ForwardDiff
      ◓ Metatheory
      ◑ NLSolversBase

      ✓ ForwardDiff
      ◑ Metatheory
      ◒ NLSolversBase

      ✓ ForwardDiff
      ◒ Metatheory
      ◐ NLSolversBase

      ✓ ForwardDiff
      ◐ Metatheory
      ◓ NLSolversBase

      ✓ ForwardDiff
      ◓ Metatheory
      ◑ NLSolversBase

      ✓ ForwardDiff
      ◑ Metatheory
      ◒ NLSolversBase

      ✓ ForwardDiff
      ◒ Metatheory
      ◐ NLSolversBase

      ✓ ForwardDiff
      ◐ Metatheory
      ◓ NLSolversBase

      ✓ ForwardDiff
      ✓ Metatheory
      ◑ NLSolversBase
      ◒ SymbolicUtils

      ✓ Metatheory
      ◒ NLSolversBase
      ◐ SymbolicUtils

      ✓ Metatheory
      ◐ NLSolversBase
      ◓ SymbolicUtils

      ✓ Metatheory
      ◓ NLSolversBase
      ◑ SymbolicUtils

      ✓ Metatheory
      ◑ NLSolversBase
      ◒ SymbolicUtils

      ✓ Metatheory
      ◒ NLSolversBase
      ◐ SymbolicUtils

      ✓ Metatheory
      ◐ NLSolversBase
      ◓ SymbolicUtils

      ✓ Metatheory
      ◓ NLSolversBase
      ◑ SymbolicUtils

      ✓ Metatheory
      ◑ NLSolversBase
      ◒ SymbolicUtils

      ✓ Metatheory
      ◒ NLSolversBase
      ◐ SymbolicUtils
      Progress [======================================>  ]  78/83
      ✓ Metatheory
      ◐ NLSolversBase
      ◓ SymbolicUtils

      ◓ NLSolversBase
      ◑ SymbolicUtils

      ◑ NLSolversBase
      ◒ SymbolicUtils

      ◒ NLSolversBase
      ◐ SymbolicUtils

      ◐ NLSolversBase
      ◓ SymbolicUtils

      ◓ NLSolversBase
      ◑ SymbolicUtils

      ◑ NLSolversBase
      ◒ SymbolicUtils

      ◒ NLSolversBase
      ◐ SymbolicUtils

      ◐ NLSolversBase
      ◓ SymbolicUtils

      ◓ NLSolversBase
      ◑ SymbolicUtils

      ✓ NLSolversBase
      ◒ SymbolicUtils
      ◐ LineSearches

      ✓ NLSolversBase
      ◐ SymbolicUtils
      ◓ LineSearches

      ✓ NLSolversBase
      ◓ SymbolicUtils
      ◑ LineSearches

      ✓ NLSolversBase
      ◑ SymbolicUtils
      ◒ LineSearches

      ✓ NLSolversBase
      ◒ SymbolicUtils
      ◐ LineSearches

      ✓ NLSolversBase
      ◐ SymbolicUtils
      ◓ LineSearches

      ✓ NLSolversBase
      ◓ SymbolicUtils
      ◑ LineSearches

      ✓ NLSolversBase
      ◑ SymbolicUtils
      ◒ LineSearches

      ✓ NLSolversBase
      ◒ SymbolicUtils
      ◐ LineSearches

      ✓ NLSolversBase
      ◐ SymbolicUtils
      ◓ LineSearches
      Progress [=======================================> ]  79/83
      ◓ SymbolicUtils
      ◑ LineSearches

      ◑ SymbolicUtils
      ◒ LineSearches

      ◒ SymbolicUtils
      ◐ LineSearches

      ◐ SymbolicUtils
      ◓ LineSearches

      ◓ SymbolicUtils
      ◑ LineSearches

      ◑ SymbolicUtils
      ◒ LineSearches

      ◒ SymbolicUtils
      ◐ LineSearches

      ◐ SymbolicUtils
      ◓ LineSearches

      ◓ SymbolicUtils
      ◑ LineSearches

      ◑ SymbolicUtils
      ◒ LineSearches

      ◒ SymbolicUtils
      ◐ LineSearches

      ◐ SymbolicUtils
      ◓ LineSearches

      ◓ SymbolicUtils
      ◑ LineSearches

      ◑ SymbolicUtils
      ◒ LineSearches

      ◒ SymbolicUtils
      ◐ LineSearches

      ◐ SymbolicUtils
      ◓ LineSearches

      ◓ SymbolicUtils
      ◑ LineSearches

      ◑ SymbolicUtils
      ◒ LineSearches

      ◒ SymbolicUtils
      ◐ LineSearches

      ◐ SymbolicUtils
      ◓ LineSearches
      Progress [=======================================> ]  79/83
      ◓ SymbolicUtils
      ◑ LineSearches

      ◑ SymbolicUtils
      ◒ LineSearches

      ◒ SymbolicUtils
      ◐ LineSearches

      ◐ SymbolicUtils
      ◓ LineSearches

      ◓ SymbolicUtils
      ◑ LineSearches

      ◑ SymbolicUtils
      ◒ LineSearches

      ◒ SymbolicUtils
      ◐ LineSearches

      ◐ SymbolicUtils
      ◓ LineSearches

      ◓ SymbolicUtils
      ◑ LineSearches

      ◑ SymbolicUtils
      ◒ LineSearches

      ◒ SymbolicUtils
      ◐ LineSearches

      ◐ SymbolicUtils
      ◓ LineSearches

      ◓ SymbolicUtils
      ✓ LineSearches
      ◓ Optim

      ◑ SymbolicUtils
      ✓ LineSearches
      ◑ Optim

      ◒ SymbolicUtils
      ✓ LineSearches
      ◒ Optim

      ◐ SymbolicUtils
      ✓ LineSearches
      ◐ Optim

      ◓ SymbolicUtils
      ✓ LineSearches
      ◓ Optim

      ◑ SymbolicUtils
      ✓ LineSearches
      ◑ Optim

      ◒ SymbolicUtils
      ✓ LineSearches
      ◒ Optim

      ◐ SymbolicUtils
      ✓ LineSearches
      ◐ Optim
      Progress [=======================================> ]  80/83
      ◓ SymbolicUtils
      ✓ LineSearches
      ◓ Optim

      ◑ SymbolicUtils
      ✓ LineSearches
      ◑ Optim

      ◒ SymbolicUtils
      ◒ Optim

      ◐ SymbolicUtils
      ◐ Optim

      ◓ SymbolicUtils
      ◓ Optim

      ◑ SymbolicUtils
      ◑ Optim

      ◒ SymbolicUtils
      ◒ Optim

      ◐ SymbolicUtils
      ◐ Optim

      ◓ SymbolicUtils
      ◓ Optim

      ◑ SymbolicUtils
      ◑ Optim

      ◒ SymbolicUtils
      ◒ Optim

      ◐ SymbolicUtils
      ◐ Optim

      ◓ SymbolicUtils
      ◓ Optim

      ◑ SymbolicUtils
      ◑ Optim

      ◒ SymbolicUtils
      ◒ Optim

      ◐ SymbolicUtils
      ◐ Optim

      ◓ SymbolicUtils
      ◓ Optim

      ◑ SymbolicUtils
      ◑ Optim

      ◒ SymbolicUtils
      ◒ Optim

      ◐ SymbolicUtils
      ◐ Optim
      Progress [=======================================> ]  80/83
      ◓ SymbolicUtils
      ◓ Optim

      ◑ SymbolicUtils
      ◑ Optim

      ◒ SymbolicUtils
      ◒ Optim

      ◐ SymbolicUtils
      ◐ Optim

      ◓ SymbolicUtils
      ◓ Optim

      ◑ SymbolicUtils
      ◑ Optim

      ◒ SymbolicUtils
      ◒ Optim

      ◐ SymbolicUtils
      ◐ Optim

      ◓ SymbolicUtils
      ◓ Optim

      ◑ SymbolicUtils
      ◑ Optim

      ◒ SymbolicUtils
      ◒ Optim

      ◐ SymbolicUtils
      ◐ Optim

      ◓ SymbolicUtils
      ◓ Optim

      ◑ SymbolicUtils
      ◑ Optim

      ◒ SymbolicUtils
      ◒ Optim

      ◐ SymbolicUtils
      ◐ Optim

      ◓ SymbolicUtils
      ◓ Optim

      ◑ SymbolicUtils
      ◑ Optim

      ◒ SymbolicUtils
      ◒ Optim

      ◐ SymbolicUtils
      ◐ Optim
      Progress [=======================================> ]  80/83
      ◓ SymbolicUtils
      ◓ Optim

      ◑ SymbolicUtils
      ◑ Optim

      ◒ SymbolicUtils
      ◒ Optim

      ◐ SymbolicUtils
      ◐ Optim

      ◓ SymbolicUtils
      ◓ Optim

      ◑ SymbolicUtils
      ◑ Optim

      ◒ SymbolicUtils
      ◒ Optim

      ◐ SymbolicUtils
      ◐ Optim

      ◓ SymbolicUtils
      ◓ Optim

      ◑ SymbolicUtils
      ◑ Optim

      ◒ SymbolicUtils
      ◒ Optim

      ◐ SymbolicUtils
      ◐ Optim

      ◓ SymbolicUtils
      ◓ Optim

      ◑ SymbolicUtils
      ◑ Optim

      ◒ SymbolicUtils
      ◒ Optim

      ◐ SymbolicUtils
      ✓ Optim

      ◓ SymbolicUtils
      ✓ Optim

      ◑ SymbolicUtils
      ✓ Optim

      ◒ SymbolicUtils
      ✓ Optim

      ◐ SymbolicUtils
      ✓ Optim
      Progress [========================================>]  81/83
      ◓ SymbolicUtils
      ✓ Optim

      ◑ SymbolicUtils
      ✓ Optim

      ◒ SymbolicUtils
      ✓ Optim

      ◐ SymbolicUtils
      ✓ Optim

      ◓ SymbolicUtils
      ✓ Optim

      ◑ SymbolicUtils

      ◒ SymbolicUtils

      ◐ SymbolicUtils

      ◓ SymbolicUtils

      ◑ SymbolicUtils

      ◒ SymbolicUtils

      ◐ SymbolicUtils

      ◓ SymbolicUtils

      ◑ SymbolicUtils

      ◒ SymbolicUtils

      ◐ SymbolicUtils

      ◓ SymbolicUtils

      ◑ SymbolicUtils

      ◒ SymbolicUtils

      ◐ SymbolicUtils
      Progress [========================================>]  81/83
      ◓ SymbolicUtils

      ◑ SymbolicUtils

      ✓ SymbolicUtils
      ◒ SymbolicRegression

      ✓ SymbolicUtils
      ◐ SymbolicRegression

      ✓ SymbolicUtils
      ◓ SymbolicRegression

      ✓ SymbolicUtils
      ◑ SymbolicRegression

      ✓ SymbolicUtils
      ◒ SymbolicRegression

      ✓ SymbolicUtils
      ◐ SymbolicRegression

      ✓ SymbolicUtils
      ◓ SymbolicRegression

      ✓ SymbolicUtils
      ◑ SymbolicRegression

      ✓ SymbolicUtils
      ◒ SymbolicRegression

      ✓ SymbolicUtils
      ◐ SymbolicRegression

      ◓ SymbolicRegression

      ◑ SymbolicRegression

      ◒ SymbolicRegression

      ◐ SymbolicRegression

      ◓ SymbolicRegression

      ◑ SymbolicRegression

      ◒ SymbolicRegression

      ◐ SymbolicRegression
      Progress [========================================>]  82/83
      ◓ SymbolicRegression

      ◑ SymbolicRegression

      ◒ SymbolicRegression

      ◐ SymbolicRegression

      ◓ SymbolicRegression

      ◑ SymbolicRegression

      ◒ SymbolicRegression

      ◐ SymbolicRegression

      ◓ SymbolicRegression

      ◑ SymbolicRegression

      ◒ SymbolicRegression

      ◐ SymbolicRegression

      ◓ SymbolicRegression

      ◑ SymbolicRegression

      ◒ SymbolicRegression

      ◐ SymbolicRegression

      ◓ SymbolicRegression

      ◑ SymbolicRegression

      ◒ SymbolicRegression

      ◐ SymbolicRegression
      Progress [========================================>]  82/83
      ◓ SymbolicRegression

      ◑ SymbolicRegression

      ◒ SymbolicRegression

      ◐ SymbolicRegression

      ◓ SymbolicRegression

      ◑ SymbolicRegression

      ◒ SymbolicRegression

      ◐ SymbolicRegression

      ◓ SymbolicRegression

      ◑ SymbolicRegression

      ◒ SymbolicRegression

      ◐ SymbolicRegression

      ◓ SymbolicRegression

      ◑ SymbolicRegression

      ◒ SymbolicRegression

      ◐ SymbolicRegression

      ◓ SymbolicRegression

      ◑ SymbolicRegression

      ◒ SymbolicRegression

      ◐ SymbolicRegression
      Progress [========================================>]  82/83
      ◓ SymbolicRegression

      ◑ SymbolicRegression

      ◒ SymbolicRegression

      ◐ SymbolicRegression

      ◓ SymbolicRegression

      ◑ SymbolicRegression

      ◒ SymbolicRegression

      ◐ SymbolicRegression

      ◓ SymbolicRegression

      ◑ SymbolicRegression

      83 dependencies successfully precompiled in 81 seconds

```python
!julia -e 'println("Success")'
```

    Success

Install PySINDy, PySR, and accelerate:

```python
!pip install -q pysr accelerate
```

Imports:

```python
import sympy
import numpy as np
from pysr import pysr, best, get_hof
import torch
from torch import nn, optim
from torch.nn import functional as F
from torch.utils.data import DataLoader, TensorDataset
import accelerate
from sklearn.model_selection import train_test_split

# import pysindy as ps
```

# Simple PySR example:

First, let's learn a simple function

$$2.5382 \cos(x3) + x0^2 - 2$$

$$y \approx f(x_1, x_2, ...)$$ given that $f$ is analytic formula.

```python
# Dataset
np.random.seed(0)
X = 2*np.random.randn(100, 5)
y = 2.5382*np.cos(X[:, 3]) + X[:, 0]**2 - 2
```

```python
default_pysr_params = dict(
    populations=8,
    procs=4,
)
```

```python
# Learn equations
equations = pysr(
        X, y,
        niterations=5,
        binary_operators=["plus", "mult"],
        unary_operators=["cos", "exp", "sin"],
        **default_pysr_params)
```

    Running on julia -O3 --threads 4 /tmp/tmp56zab0c8/runfile.jl
    Started!

    Cycles per second: 4.750e+03
    Head worker occupation: 11.2%
    Progress: 3 / 40 total iterations (7.500%)
    ==============================
    Hall of Fame:
    -----------------------------------------
    Complexity  Loss       Score     Equation
    1           3.360e+01  1.192e-07  2.3551402
    3           3.319e+01  6.114e-03  exp(cos(x3))
    4           2.050e+01  4.822e-01  exp(-0.68546265 * x0)
    6           1.240e+01  2.511e-01  (exp(-0.68546265 * x0) + x0)
    8           1.161e+01  3.294e-02  (exp(-0.68546265 * x0) + (x0 * 1.2790542))
    9           1.141e+01  1.751e-02  (exp(-0.68546265 * x0) + (cos(x3) + x0))
    10          1.870e-12  2.944e+01  (((cos(x3) * 2.5382018) + -2.0) + (x0 * x0))

    ==============================

    Cycles per second: 5.240e+03
    Head worker occupation: 5.3%
    Progress: 7 / 40 total iterations (17.500%)
    ==============================
    Hall of Fame:
    -----------------------------------------
    Complexity  Loss       Score     Equation
    1           3.360e+01  1.192e-07  2.3551364
    3           6.128e+00  8.509e-01  (x0 * x0)
    5           3.045e+00  3.496e-01  ((x0 * x0) + -1.7557635)
    7           3.030e+00  2.620e-03  (((x0 * x0) * 0.97769064) + -1.6640471)
    8           1.118e+00  9.964e-01  (cos(x3) + ((x0 * x0) + -1.8519877))
    10          1.870e-12  1.356e+01  (((cos(x3) * 2.5382018) + -2.0) + (x0 * x0))

    ==============================

    Cycles per second: 5.170e+03
    Head worker occupation: 3.9%
    Progress: 10 / 40 total iterations (25.000%)
    ==============================
    Hall of Fame:
    -----------------------------------------
    Complexity  Loss       Score     Equation
    1           3.360e+01  1.192e-07  2.3551288
    3           6.128e+00  8.509e-01  (x0 * x0)
    5           3.045e+00  3.496e-01  ((x0 * x0) + -1.7557507)
    7           3.035e+00  1.770e-03  (((x0 + -0.051442638) * x0) + -1.7663805)
    8           1.118e+00  9.981e-01  ((cos(x3) + -1.8520532) + (x0 * x0))
    10          1.870e-12  1.356e+01  (((cos(x3) * 2.5382018) + -2.0) + (x0 * x0))

    ==============================

    Cycles per second: 5.490e+03
    Head worker occupation: 3.2%
    Progress: 13 / 40 total iterations (32.500%)
    ==============================
    Hall of Fame:
    -----------------------------------------
    Complexity  Loss       Score     Equation
    1           3.360e+01  1.192e-07  2.3551002
    3           6.128e+00  8.509e-01  (x0 * x0)
    5           3.045e+00  3.496e-01  ((x0 * x0) + -1.7557467)
    7           3.030e+00  2.620e-03  (((x0 * x0) * 0.9777001) + -1.6640841)
    8           1.118e+00  9.964e-01  ((cos(x3) + -1.8519826) + (x0 * x0))
    10          1.870e-12  1.356e+01  (((cos(x3) * 2.5382018) + -2.0) + (x0 * x0))

    ==============================

    Cycles per second: 5.490e+03
    Head worker occupation: 2.8%
    Progress: 15 / 40 total iterations (37.500%)
    ==============================
    Hall of Fame:
    -----------------------------------------
    Complexity  Loss       Score     Equation
    1           3.360e+01  1.192e-07  2.355266
    3           6.128e+00  8.509e-01  (x0 * x0)
    5           3.045e+00  3.496e-01  ((x0 * x0) + -1.7557179)
    7           3.030e+00  2.620e-03  (((x0 * x0) + -1.7019966) * 0.977695)
    8           1.118e+00  9.964e-01  (((x0 * x0) + -1.8519857) + cos(x3))
    10          1.870e-12  1.356e+01  (((cos(x3) * 2.5382018) + -2.0) + (x0 * x0))

    ==============================

    Cycles per second: 5.620e+03
    Head worker occupation: 2.3%
    Progress: 19 / 40 total iterations (47.500%)
    ==============================
    Hall of Fame:
    -----------------------------------------
    Complexity  Loss       Score     Equation
    1           3.360e+01  1.192e-07  2.3550944
    3           6.128e+00  8.509e-01  (x0 * x0)
    5           3.045e+00  3.496e-01  ((x0 * x0) + -1.7557507)
    7           3.035e+00  1.770e-03  ((x0 * (x0 + -0.05142775)) + -1.7663476)
    8           1.118e+00  9.981e-01  ((cos(x3) + -1.851954) + (x0 * x0))
    10          1.870e-12  1.356e+01  (((cos(x3) * 2.5382018) + -2.0) + (x0 * x0))

    ==============================

    Cycles per second: 5.630e+03
    Head worker occupation: 2.1%
    Progress: 21 / 40 total iterations (52.500%)
    ==============================
    Hall of Fame:
    -----------------------------------------
    Complexity  Loss       Score     Equation
    1           3.360e+01  1.192e-07  2.355119
    3           6.128e+00  8.509e-01  (x0 * x0)
    5           3.045e+00  3.496e-01  ((x0 * x0) + -1.7557536)
    7           3.035e+00  1.770e-03  ((x0 + -1.3550031) * (x0 + 1.3035738))
    8           1.118e+00  9.981e-01  ((cos(x3) + -1.8519826) + (x0 * x0))
    9           9.811e-01  1.311e-01  (((x0 * x0) + -2.9473426) + exp(cos(x3)))
    10          1.870e-12  2.699e+01  (((cos(x3) * 2.5382018) + -2.0) + (x0 * x0))

    ==============================
    Killing process... will return when done.

```python
equations
```

      Complexity
      MSE
      score
      Equation
      sympy_format
      lambda_format

      0
      1
      3.360272e+01
      0.000000
      2.355163
      2.35516300000000
      PySRFunction(X=>2.35516300000000)

      1
      3
      6.128181e+00
      0.850855
      (x0 * x0)
      x0**2
      PySRFunction(X=>x0**2)

      2
      5
      3.045492e+00
      0.349618
      ((x0 * x0) + -1.7557584)
      x0**2 - 1.7557584
      PySRFunction(X=>x0**2 - 1.7557584)

      3
      7
      3.029573e+00
      0.002620
      (((x0 * x0) + -1.7020214) * 0.9776912)
      0.9776912*x0**2 - 1.66405134499168
      PySRFunction(X=>0.9776912*x0**2 - 1.6640513449...

      4
      8
      1.118488e+00
      0.996444
      ((cos(x3) + -1.8519883) + (x0 * x0))
      x0**2 + cos(x3) - 1.8519883
      PySRFunction(X=>x0**2 + cos(x3) - 1.8519883)

      5
      10
      1.869537e-12
      13.558654
      (((cos(x3) * 2.5382018) + -2.0) + (x0 * x0))
      x0**2 + 2.5382018*cos(x3) - 2.0
      PySRFunction(X=>x0**2 + 2.5382018*cos(x3) - 2.0)

```python
best(equations.query('MSE < 0.01'))
```

$\displaystyle x_{0}^{2} + 2.5382001 \cos{\left(x_{3} \right)} - 2.0$

## Output

The output is a Pandas DataFrame:

```python
from pysr import *
```

```python
best_tex()
```

    'x_{0}^{2} + 2.5382018 \\cos{\\left(x_{3} \\right)} - 2.0'

These are the columns. Let's discuss each of them:

```python
equations.columns
```

    Index(['Complexity', 'MSE', 'score', 'Equation', 'sympy_format',
           'lambda_format'],
          dtype='object')

```python
print(equations[['MSE', 'score', 'sympy_format']])
```

                MSE      score                          sympy_format
    0  3.360272e+01   0.000000                      2.35515170000000
    1  6.128181e+00   0.850855                                 x0**2
    2  3.045492e+00   0.349618                     x0**2 - 1.7557602
    3  3.029573e+00   0.002620          0.97770476*x0**2 - 1.6641104
    4  1.118488e+00   0.996444           x0**2 + cos(x3) - 1.8519799
    5  2.512124e-10  11.108350  x0**2 + 2.5381956*cos(x3) - 2.000015

# PySINDY Simple Example

The following simple SINDy example follows this [tutorial](https://pysindy.readthedocs.io/en/latest/examples/2_introduction_to_sindy.html),
and solves the differential equation:

$$\dot{X} \approx \Theta(X)\Xi,$$

for $\Theta(X)$ a library of expressions in $X$, for example,
$\Theta(X)_{i,j} = x_1(t_5)^2 x_3(t_5)$, for some $i, j$.

For $\Xi$ a matrix of coefficients, which SINDy seeks to find a sparse
solution for.

Consider the following dynamical system:
\begin{split}\frac{d}{dt} \begin{bmatrix} x \\ y \end{bmatrix}
= \begin{bmatrix} -2x \\ y \end{bmatrix}\end{split}

This can also be written as:

\begin{split}
= \begin{bmatrix} -2 & 0 \\ 0 & 1 \end{bmatrix}
\begin{bmatrix} x \\ y \end{bmatrix}
\end{split}

First, let's generate some data:

```python
t = np.linspace(0, 1, 100)
x = 3 * np.exp(-2 * t)
y = 0.5 * np.exp(t)
X = np.stack((x, y), axis=-1)  # First column is x, second is y
```

```python
#! pip install pysindy

#^Unfortunately this is incompatible with scikit-learn 1.0 (built-in for colab);
# monitor https://github.com/dynamicslab/pysindy/issues for a fix, then this should all work:
```

```python
# # Compute derivatives with finite difference:
# differentiation_method = ps.FiniteDifference(order=2)
# # Define polynomial as library we search for:
# feature_library = ps.PolynomialLibrary(degree=3)
## Define Optimizer:
# optimizer = ps.STLSQ(threshold=0.2)

## Create SINDy object:
# model = ps.SINDy(
#     differentiation_method=differentiation_method,
#     feature_library=feature_library,
#     optimizer=optimizer,
#     feature_names=["x", "y"]
# )

## Fit:
# model.fit(X, t=t);
# model.print()
```

This will print out:

```
(x)' = -2.000 x
(y)' = 1.000 y
```

so it has successfully found the coefficients in this differential equation.

See [docs](https://pysindy.readthedocs.io/en/latest/index.html) for full list of features, including
other algorithms such as SINDy-Net.

# Custom operators in GA-based symbolic regression; example with PySR:

A full list of operators is given here: https://pysr.readthedocs.io/en/latest/docs/operators/,
but we can also define our own.

Say that we want a command to do quartic powers:

$$ y = x_0^4 - 2 $$

```python
y = X[:, 0]**4 - 2
```

We can do this by passing a string in Julia syntax:

```python
equations = pysr(X, y, niterations=5, populations=40,
        binary_operators=["plus", "mult"],
        unary_operators=["cos", "exp", "sin",
                         "quart(x) = x^4"])
```

    Running on julia -O3 --threads 4 /tmp/tmpm34llamv/runfile.jl
    Started!

    Cycles per second: 6.340e+02
    Head worker occupation: 4.7%
    Progress: 1 / 200 total iterations (0.500%)
    ==============================
    Hall of Fame:
    -----------------------------------------
    Complexity  Loss       Score     Equation
    1           3.308e+02  -0.000e+00  8.430541
    2           4.000e+00  4.415e+00  quart(x0)
    4           3.259e-11  1.277e+01  (-2.000005 + quart(x0))

    ==============================

    Cycles per second: 1.580e+03
    Head worker occupation: 3.4%
    Progress: 2 / 200 total iterations (1.000%)
    ==============================
    Hall of Fame:
    -----------------------------------------
    Complexity  Loss       Score     Equation
    1           3.308e+02  -0.000e+00  8.431212
    2           4.000e+00  4.415e+00  quart(x0)
    4           3.259e-11  1.277e+01  (-2.000005 + quart(x0))

    ==============================
    Killing process... will return when done.

```python
best(equations)
```

$\displaystyle \operatorname{quart}{\left(x_{0} \right)} - 2.000005$

## Notes

One can also add a binary operator, with, e.g., `"myoperator(x, y) = x^2 * y"`. All Julia operators that work on scalar 32-bit floating point values are available.

Make sure that any operator you add is valid over the real line. So, e.g., you will need to define `"mysqrt(x) = sqrt(abs(x))"`.

## Scoring

The `best` function selects the equation with the max score and prints it. But in practice it is best to look through all the equations manually, select an equation above some MSE threshold, and then use the score to select among that MSE threshold:

$$ \text{score} = - \log(\text{MSE}_i/\text{MSE}_{i-1})/
(\text{complexity}_i - \text{complexity}_{i-1})$$

This scoring is motivated by the common strategy of looking for drops in the loss-complexity curve.

From Schmidt & Lipson (2009) -

One can get the most-recently calculated equations with `get_hof`, or pass the name of the equations_file:

```python
get_hof()
```

      Complexity
      MSE
      score
      Equation
      sympy_format
      lambda_format

      0
      1
      3.308448e+02
      0.000000
      8.430945
      8.43094500000000
      PySRFunction(X=>8.43094500000000)

      1
      2
      4.000000e+00
      4.415355
      quart(x0)
      quart(x0)
      PySRFunction(X=>quart(x0))

      2
      4
      3.258751e-11
      12.766693
      (-2.000005 + quart(x0))
      quart(x0) - 2.000005
      PySRFunction(X=>quart(x0) - 2.000005)

## LaTeX output

SymPy lets us get LaTeX versions:

```python
sympy.latex(get_hof().iloc[-1]['sympy_format'].simplify())
```

    '\\operatorname{quart}{\\left(x_{0} \\right)} - 2.000005'

Since we have defined our own op, we need to define it in SymPy as well to get the LaTeX version:

```python
sympy.latex(
    (get_hof(extra_sympy_mappings={'quart': lambda x: x**4})
    .iloc[-1]['sympy_format'].simplify())
)
```

    'x_{0}^{4} - 2.000005'

# Noise example

Here is an example with noise. Known Gaussian noise with $\sigma$ between 0.1 and 5.0. We record samples of $y$:

$$ \sigma \sim U(0.1, 5.0) $$
$$ \epsilon \sim \mathcal{N}(0, \sigma^2)$$
$$ y = 5\;\cos(3.5 x_0) - 1.3 + \epsilon.$$
We have 5 features, say. The weights change the loss function to be:
$$MSE = \sum [(y - f(x))^2*w],$$

so in this example, we can set:
$$w = 1/\sigma^2.$$

```python
np.random.seed(0)
N = 3000
upper_sigma = 5
X = 2*np.random.rand(N, 5)
sigma = np.random.rand(N)*(5-0.1) + 0.1
eps = sigma*np.random.randn(N)
y = 5*np.cos(3.5*X[:, 0]) - 1.3 + eps
```

Define some weights to use:

```python
weights = 1/sigma**2
```

```python
weights[:5]
```

    array([0.25199296, 0.0775419 , 0.11515701, 0.1781281 , 0.0448131 ])

Let's run PySR again:

```python
# eq = pysr(
#     X=X, y=y,
#     weights=weights,
#     # loss='myloss(x, y, w) = w * abs(x - y)',
#     niterations=20,
#     procs=4,
#     populations=20,
#     batching=True,
#     # useFrequency=True,
#     # annealing=False,
#     # optimizer_algorithm='BFGS',
#     # optimize_probability=1.0,
#     # optimizer_iterations=10,
#     binary_operators=["+", "*", "/", "-"],
#     unary_operators=["cos"]
# )
```

Let's see if we get similar results to the true equation:

```python
# best()
```

# High-dimensional input: Neural Nets + Symbolic Regression

In this example, let's learn a high-dimensional problem. **This will use the method proposed in our NeurIPS paper: https://arxiv.org/abs/2006.11287.**

Let's consider a time series problem:

$$ z = y^2,\quad y = \frac{1}{100} \sum(y_i),\quad y_i = x_{i0}^2 + 6 \cos(2*x_{i2})$$

Imagine our time series is 100 timesteps. That is very hard for symbolic regression, even if we impose the inductive bias of $$z=f(\sum g(x_i))$$ - it is the square of the number of possible equations!

But, as in our paper, **we can break this problem down into parts with a neural network. Then approximate the neural network with the symbolic regression!**

Then, instead of, say, $(10^9)^2=10^{18}$ equations, we only have to consider $2\times 10^9$ equations.

```python
###### np.random.seed(0)
N = 100000
Nt = 100
X = 6 * np.random.rand(N, Nt, 5) - 3
y_i = X[..., 0]**2 + 6*np.cos(2*X[..., 2])
y = np.sum(y_i, axis=1)/y_i.shape[1]
z = y**2
X.shape, y.shape
```

    ((100000, 100, 5), (100000,))

## Neural Network definition

So, as described above, let's first use a neural network with the sum inductive bias to solve this problem.

Essentially, we will learn two neural networks:
- `f`
- `g`

each defined as a multi-layer perceptron. We will sum over `g` the same way as in our equation, but we won't define the summed part beforehand.

Then, we will fit `g` and `f` **separately** using symbolic regression.

```python
hidden = 100

def mlp(size_in, size_out):
    return nn.Sequential(
        nn.Linear(size_in, hidden),
        nn.ReLU(),
        nn.Linear(hidden, hidden),
        nn.ReLU(),
        nn.Linear(hidden, hidden),
        nn.ReLU(),
        nn.Linear(hidden, size_out))

max_epochs = 20

class SumNet(nn.Module):
    def __init__(self):
        super().__init__()

    ########################################################
    # The same inductive bias as above!
        self.g = mlp(5, 1)
        self.f = mlp(1, 1)

    def forward(self, x):
        y_i = self.g(x)[:, :, 0]
        y = torch.sum(y_i, dim=1, keepdim=True)/y_i.shape[1]
        z = self.f(y)
        return z[:, 0]
    ########################################################

```

## Data bookkeeping

Put everything into PyTorch and do a train/validation split:

```python
Xt = torch.tensor(X).float()
zt = torch.tensor(z).float()
X_train, X_test, z_train, z_test = train_test_split(Xt, zt, random_state=0)
train_set = TensorDataset(X_train, z_train)
train = DataLoader(train_set, batch_size=128, num_workers=2)
test_set = TensorDataset(X_test, z_test)
test = DataLoader(test_set, batch_size=256, num_workers=2)
```

## Train the model with Accelerate on GPUs:

Start the model:

```python
torch.random.manual_seed(0)
model = SumNet()
```

Here, we fit the neural network:

```python
opt = optim.Adam(model.parameters(), lr=1e-3)
# Fast at start, cosine decay at end:
total_steps = 30000
sched = optim.lr_scheduler.OneCycleLR(opt, 1e-3, total_steps=total_steps)
acc = accelerate.Accelerator()
model, opt, sched, train, test = acc.prepare(model, opt, sched, train, test)
```

```python
losses = []
from tqdm.auto import trange
print_every = 300
step_iter = iter(trange(total_steps))
trainer = iter(train)
tester = iter(test)
step = 0

while step < total_steps - 1:

    for _ in range(print_every):
        opt.zero_grad()
        step = next(step_iter)

        try:
            xt, zt = next(trainer)
        except StopIteration:
            trainer = iter(train)
            xt, zt = next(trainer)

        predicted_zt = model(xt)
        loss = F.mse_loss(zt, predicted_zt)

        acc.backward(loss)
        opt.step()
        sched.step()

        losses.append(loss.item())

        if step % print_every == 0 and step > 0:
            # Show median loss:
            print('Train loss:', np.median(losses[-print_every:]))

        if step == total_steps - 1:
            break

```

      0%|          | 0/30000 [00:00<?, ?it/s]

    Train loss: 58.2508602142334
    Train loss: 7.377145767211914
    Train loss: 7.096177101135254
    Train loss: 7.296555280685425
    Train loss: 7.016629695892334
    Train loss: 7.209305763244629
    Train loss: 6.905282020568848
    Train loss: 7.0219621658325195
    Train loss: 6.568258285522461
    Train loss: 6.205851078033447
    Train loss: 5.548778533935547
    Train loss: 5.443612813949585
    Train loss: 5.290122032165527
    Train loss: 5.291875600814819
    Train loss: 4.994513273239136
    Train loss: 4.856929779052734
    Train loss: 4.389889478683472
    Train loss: 3.604907989501953
    Train loss: 2.1381300687789917
    Train loss: 1.1684916615486145
    Train loss: 0.7781404852867126
    Train loss: 0.5741005539894104
    Train loss: 0.47189703583717346
    Train loss: 0.3085206598043442
    Train loss: 0.27826203405857086
    Train loss: 0.26010963320732117
    Train loss: 0.22478465735912323
    Train loss: 0.17911610752344131
    Train loss: 0.18241025507450104
    Train loss: 0.17096580564975739
    Train loss: 0.17434779554605484
    Train loss: 0.14066744595766068
    Train loss: 0.11515192314982414
    Train loss: 0.12701362371444702
    Train loss: 0.13233984261751175
    Train loss: 0.12628118693828583
    Train loss: 0.11507120728492737
    Train loss: 0.10921511799097061
    Train loss: 0.09057427570223808
    Train loss: 0.09971822053194046
    Train loss: 0.08444985374808311
    Train loss: 0.09359220787882805
    Train loss: 0.0737948827445507
    Train loss: 0.09136428311467171
    Train loss: 0.06827393919229507
    Train loss: 0.08307631313800812
    Train loss: 0.06429216265678406
    Train loss: 0.06875896826386452
    Train loss: 0.06106966733932495
    Train loss: 0.06437951326370239
    Train loss: 0.05438915640115738
    Train loss: 0.06396336480975151
    Train loss: 0.04968982748687267
    Train loss: 0.05144074559211731
    Train loss: 0.04847731068730354
    Train loss: 0.0409537348896265
    Train loss: 0.043969644233584404
    Train loss: 0.03251940757036209
    Train loss: 0.044321563094854355
    Train loss: 0.030740883201360703
    Train loss: 0.03766464442014694
    Train loss: 0.03135195001959801
    Train loss: 0.03713810257613659
    Train loss: 0.02691495604813099
    Train loss: 0.03336961567401886
    Train loss: 0.02579842507839203
    Train loss: 0.024732105433940887
    Train loss: 0.024090292863547802
    Train loss: 0.023561792448163033
    Train loss: 0.02256203629076481
    Train loss: 0.022116520442068577
    Train loss: 0.023215980269014835
    Train loss: 0.01871446706354618
    Train loss: 0.021130948327481747
    Train loss: 0.017827631905674934
    Train loss: 0.01754436269402504
    Train loss: 0.017074208706617355
    Train loss: 0.017179724760353565
    Train loss: 0.016813533380627632
    Train loss: 0.015713135711848736
    Train loss: 0.015811187215149403
    Train loss: 0.014929334633052349
    Train loss: 0.014717343263328075
    Train loss: 0.013881377875804901
    Train loss: 0.014092985540628433
    Train loss: 0.014067093841731548
    Train loss: 0.012993591837584972
    Train loss: 0.01319288183003664
    Train loss: 0.012784810736775398
    Train loss: 0.012867916841059923
    Train loss: 0.012361138593405485
    Train loss: 0.01219808217138052
    Train loss: 0.012224954087287188
    Train loss: 0.011984719429165125
    Train loss: 0.011812381912022829
    Train loss: 0.011856984347105026
    Train loss: 0.011725838296115398
    Train loss: 0.01150071993470192
    Train loss: 0.011563369538635015

## Latent vectors of network

Let's get the input and output of the learned `g` function from the network over some random data:

```python
np.random.seed(0)
idx = np.random.randint(0, 10000, size=1000)

model = model.cpu()

X_for_pysr = Xt[idx]
y_i_for_pysr = model.g(X_for_pysr)[:, :, 0]
y_for_pysr = torch.sum(y_i_for_pysr, dim=1)/y_i_for_pysr.shape[1]
z_for_pysr = zt[idx] #Use true values.

X_for_pysr.shape, y_i_for_pysr.shape
```

    (torch.Size([1000, 100, 5]), torch.Size([1000, 100]))

## Learning over the network:

Now, let's fit `g` using PySR:

```python
np.random.seed(1)
tmpX = X_for_pysr.reshape(-1, 5).cpu().detach().numpy()
tmpy = y_i_for_pysr.reshape(-1).cpu().detach().numpy()
idx2 = np.random.randint(0, tmpy.shape[0], size=500)

y_i_equations = pysr(
    X=tmpX[idx2], y=tmpy[idx2],
    niterations=20,
    populations=20,
    useFrequency=True,
    # multithreading=False,
    binary_operators=["plus", "sub", "mult"],
    unary_operators=["cos", "square"]
)
```

    Running on julia -O3 /tmp/tmptz558hqg/runfile.jl
    Activating environment on workers.
    Importing installed module on workers...Finished!
    Testing module on workers...Finished!
    Testing entire pipeline on workers...Finished!
    Started!

    Cycles per second: 2.610e+02
    Head worker occupation: 4.0%
    Progress: 1 / 400 total iterations (0.250%)
    ==============================
    Hall of Fame:
    -----------------------------------------
    Complexity  Loss       Score     Equation
    1           3.043e+00  5.960e-08  0.7525229
    3           2.203e+00  1.615e-01  square(cos(x2))
    5           1.477e+00  1.997e-01  square(-1.5491782 * cos(x2))
    6           1.246e+00  1.699e-01  square(square(cos(x2)) * 1.6928658)
    7           8.753e-01  3.536e-01  (-1.3549564 + square(-2.0556479 * cos(x2)))
    8           7.979e-01  9.252e-02  (square(cos(x2) * -1.4166769) - cos(x0))
    10          2.347e-01  6.118e-01  (square(cos(x2) * -2.0671382) - (cos(x0) + 0.9960016))
    11          2.213e-01  5.909e-02  (square(cos(x2) * 2.0501647) - (cos(x0) + cos(-0.18744937)))
    12          5.722e-02  1.352e+00  (-1.365318 + ((-1.3436862 * cos(x0)) + square(-2.0805726 * cos(x2))))
    14          4.951e-02  7.237e-02  (-1.3183383 + ((-1.3236307 * cos(0.93770593 * x0)) + square(2.0788052 * cos(x2))))
    16          4.944e-02  7.766e-04  (-1.2881377 + ((-1.3236307 * cos(-0.015837993 + (0.949994 * x0))) + square(-2.0671382 * cos(x2))))
    17          4.756e-02  3.878e-02  (square(cos(x2) * -2.0671382) - ((cos((x0 - 0.013357178) * 0.93770593) + cos(-0.36752132)) * 1.3236307))

    ==============================

## Validation

Recall we are searching for $y_i$ above:

$$ z = y^2,\quad y = \frac{1}{100} \sum(y_i),\quad y_i = x_{i0}^2 + 6 \cos(2 x_{i2})$$

```python
best(y_i_equations)
```

$\displaystyle - 1.3436862 \cos{\left(x_{0} \right)} + 4.32878234387076 \cos^{2}{\left(x_{2} \right)} - 1.365318$

A neural network can easily undo a linear transform, so this is fine: the network for $f$ will learn to undo the linear transform.

Then, we can learn another analytic equation for $z$.

**Now, we can compose these together to get the time series model!**

Think about what we just did: we found an analytical equation for $z$ in terms of $500$ datapoints, under the assumption that $z$ is a function of a sum of another function over an axis:

$$ z = f(\sum_i g(x_i)) $$

And we pulled out analytical copies for $g$ using symbolic regression.

# Other PySR Options

See the page: https://pysr.readthedocs.io/en/latest/?badge=latest