# Constraint-based modelling using CobraPy

## Installing CobraPy

```python
!pip install cobra
```

    Collecting cobra
      Downloading cobra-0.22.1-py2.py3-none-any.whl (2.4 MB)
         |████████████████████████████████| 2.4 MB 5.1 MB/s
    Requirement already satisfied: pandas~=1.0 in /usr/local/lib/python3.7/dist-packages (from cobra) (1.1.5)
    Collecting ruamel.yaml~=0.16
      Downloading ruamel.yaml-0.17.17-py3-none-any.whl (109 kB)
         |████████████████████████████████| 109 kB 51.0 MB/s
    Collecting optlang~=1.5
      Downloading optlang-1.5.2-py2.py3-none-any.whl (147 kB)
         |████████████████████████████████| 147 kB 55.8 MB/s
    Collecting depinfo
      Downloading depinfo-1.7.0-py2.py3-none-any.whl (8.6 kB)
    Collecting pydantic~=1.6
      Downloading pydantic-1.8.2-cp37-cp37m-manylinux2014_x86_64.whl (10.1 MB)
         |████████████████████████████████| 10.1 MB 56.2 MB/s
    Collecting httpx~=0.14
      Downloading httpx-0.21.1-py3-none-any.whl (83 kB)
         |████████████████████████████████| 83 kB 2.0 MB/s
    Requirement already satisfied: future in /usr/local/lib/python3.7/dist-packages (from cobra) (0.16.0)
    Collecting diskcache~=5.0
      Downloading diskcache-5.3.0-py3-none-any.whl (44 kB)
         |████████████████████████████████| 44 kB 2.8 MB/s
    Collecting swiglpk
      Downloading swiglpk-5.0.3-cp37-cp37m-manylinux2010_x86_64.whl (2.1 MB)
         |████████████████████████████████| 2.1 MB 54.3 MB/s
    Collecting rich>=8.0
      Downloading rich-10.15.2-py3-none-any.whl (214 kB)
         |████████████████████████████████| 214 kB 68.1 MB/s
    Requirement already satisfied: importlib-resources in /usr/local/lib/python3.7/dist-packages (from cobra) (5.4.0)
    Requirement already satisfied: numpy~=1.13 in /usr/local/lib/python3.7/dist-packages (from cobra) (1.19.5)
    Collecting python-libsbml==5.19.0
      Downloading python_libsbml-5.19.0-cp37-cp37m-manylinux1_x86_64.whl (6.2 MB)
         |████████████████████████████████| 6.2 MB 11.3 MB/s
    Requirement already satisfied: appdirs~=1.4 in /usr/local/lib/python3.7/dist-packages (from cobra) (1.4.4)
    Collecting httpcore<0.15.0,>=0.14.0
      Downloading httpcore-0.14.3-py3-none-any.whl (56 kB)
         |████████████████████████████████| 56 kB 4.1 MB/s
    Collecting rfc3986[idna2008]<2,>=1.3
      Downloading rfc3986-1.5.0-py2.py3-none-any.whl (31 kB)
    Collecting sniffio
      Downloading sniffio-1.2.0-py3-none-any.whl (10 kB)
    Requirement already satisfied: charset-normalizer in /usr/local/lib/python3.7/dist-packages (from httpx~=0.14->cobra) (2.0.8)
    Requirement already satisfied: certifi in /usr/local/lib/python3.7/dist-packages (from httpx~=0.14->cobra) (2021.10.8)
    Collecting anyio==3.*
      Downloading anyio-3.4.0-py3-none-any.whl (78 kB)
         |████████████████████████████████| 78 kB 7.6 MB/s
    Collecting h11<0.13,>=0.11
      Downloading h11-0.12.0-py3-none-any.whl (54 kB)
         |████████████████████████████████| 54 kB 3.3 MB/s
    Requirement already satisfied: idna>=2.8 in /usr/local/lib/python3.7/dist-packages (from anyio==3.*->httpcore<0.15.0,>=0.14.0->httpx~=0.14->cobra) (2.10)
    Requirement already satisfied: typing-extensions in /usr/local/lib/python3.7/dist-packages (from anyio==3.*->httpcore<0.15.0,>=0.14.0->httpx~=0.14->cobra) (3.10.0.2)
    Requirement already satisfied: six>=1.9 in /usr/local/lib/python3.7/dist-packages (from optlang~=1.5->cobra) (1.15.0)
    Requirement already satisfied: sympy>=1.0 in /usr/local/lib/python3.7/dist-packages (from optlang~=1.5->cobra) (1.7.1)
    Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.7/dist-packages (from pandas~=1.0->cobra) (2.8.2)
    Requirement already satisfied: pytz>=2017.2 in /usr/local/lib/python3.7/dist-packages (from pandas~=1.0->cobra) (2018.9)
    Collecting commonmark<0.10.0,>=0.9.0
      Downloading commonmark-0.9.1-py2.py3-none-any.whl (51 kB)
         |████████████████████████████████| 51 kB 7.0 MB/s
    Collecting colorama<0.5.0,>=0.4.0
      Downloading colorama-0.4.4-py2.py3-none-any.whl (16 kB)
    Requirement already satisfied: pygments<3.0.0,>=2.6.0 in /usr/local/lib/python3.7/dist-packages (from rich>=8.0->cobra) (2.6.1)
    Collecting ruamel.yaml.clib>=0.1.2
      Downloading ruamel.yaml.clib-0.2.6-cp37-cp37m-manylinux1_x86_64.whl (546 kB)
         |████████████████████████████████| 546 kB 41.8 MB/s
    Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.7/dist-packages (from sympy>=1.0->optlang~=1.5->cobra) (1.2.1)
    Requirement already satisfied: importlib-metadata in /usr/local/lib/python3.7/dist-packages (from depinfo->cobra) (4.8.2)
    Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata->depinfo->cobra) (3.6.0)
    Installing collected packages: sniffio, rfc3986, h11, anyio, swiglpk, ruamel.yaml.clib, httpcore, commonmark, colorama, ruamel.yaml, rich, python-libsbml, pydantic, optlang, httpx, diskcache, depinfo, cobra
    Successfully installed anyio-3.4.0 cobra-0.22.1 colorama-0.4.4 commonmark-0.9.1 depinfo-1.7.0 diskcache-5.3.0 h11-0.12.0 httpcore-0.14.3 httpx-0.21.1 optlang-1.5.2 pydantic-1.8.2 python-libsbml-5.19.0 rfc3986-1.5.0 rich-10.15.2 ruamel.yaml-0.17.17 ruamel.yaml.clib-0.2.6 sniffio-1.2.0 swiglpk-5.0.3

```python
!pip install pytest_benchmark
```

    Collecting pytest_benchmark
      Downloading pytest_benchmark-3.4.1-py2.py3-none-any.whl (50 kB)
         |████████████████████████████████| 50 kB 2.7 MB/s
    Collecting py-cpuinfo
      Downloading py-cpuinfo-8.0.0.tar.gz (99 kB)
         |████████████████████████████████| 99 kB 5.5 MB/s
    Collecting pytest>=3.8
      Downloading pytest-6.2.5-py3-none-any.whl (280 kB)
         |████████████████████████████████| 280 kB 32.2 MB/s
    Requirement already satisfied: attrs>=19.2.0 in /usr/local/lib/python3.7/dist-packages (from pytest>=3.8->pytest_benchmark) (21.2.0)
    Requirement already satisfied: py>=1.8.2 in /usr/local/lib/python3.7/dist-packages (from pytest>=3.8->pytest_benchmark) (1.11.0)
    Requirement already satisfied: importlib-metadata>=0.12 in /usr/local/lib/python3.7/dist-packages (from pytest>=3.8->pytest_benchmark) (4.8.2)
    Requirement already satisfied: packaging in /usr/local/lib/python3.7/dist-packages (from pytest>=3.8->pytest_benchmark) (21.3)
    Collecting pluggy<2.0,>=0.12
      Downloading pluggy-1.0.0-py2.py3-none-any.whl (13 kB)
    Requirement already satisfied: toml in /usr/local/lib/python3.7/dist-packages (from pytest>=3.8->pytest_benchmark) (0.10.2)
    Requirement already satisfied: iniconfig in /usr/local/lib/python3.7/dist-packages (from pytest>=3.8->pytest_benchmark) (1.1.1)
    Requirement already satisfied: typing-extensions>=3.6.4 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata>=0.12->pytest>=3.8->pytest_benchmark) (3.10.0.2)
    Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata>=0.12->pytest>=3.8->pytest_benchmark) (3.6.0)
    Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from packaging->pytest>=3.8->pytest_benchmark) (3.0.6)
    Building wheels for collected packages: py-cpuinfo
      Building wheel for py-cpuinfo (setup.py) ... done
      Created wheel for py-cpuinfo: filename=py_cpuinfo-8.0.0-py3-none-any.whl size=22258 sha256=80d722800ae4468be413d6b14e309641237d7dd2cbe6e617bda71df7d589706f
      Stored in directory: /root/.cache/pip/wheels/d2/f1/1f/041add21dc9c4220157f1bd2bd6afe1f1a49524c3396b94401
    Successfully built py-cpuinfo
    Installing collected packages: pluggy, pytest, py-cpuinfo, pytest-benchmark
      Attempting uninstall: pluggy
        Found existing installation: pluggy 0.7.1
        Uninstalling pluggy-0.7.1:
          Successfully uninstalled pluggy-0.7.1
      Attempting uninstall: pytest
        Found existing installation: pytest 3.6.4
        Uninstalling pytest-3.6.4:
          Successfully uninstalled pytest-3.6.4
    ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
    datascience 0.10.6 requires folium==0.2.1, but you have folium 0.8.3 which is incompatible.
    Successfully installed pluggy-1.0.0 py-cpuinfo-8.0.0 pytest-6.2.5 pytest-benchmark-3.4.1

```python
import cobra
```

### Testing all the requirements are satisified

```python
import cobra.test
cobra.test.test_all()
```

    ============================= test session starts ==============================
    platform linux -- Python 3.7.12, pytest-6.2.5, py-1.11.0, pluggy-1.0.0 -- /usr/bin/python3
    cachedir: .pytest_cache
    benchmark: 3.4.1 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
    rootdir: /content
    plugins: benchmark-3.4.1, anyio-3.4.0, typeguard-2.7.1
    collecting ... collected 11 items

    test/test_manipulation/test_annotate.py::test_sbo_annotation PASSED      [  9%]
    test/test_manipulation/test_delete.py::test_prune_unused_metabolites_output_type PASSED [ 18%]
    test/test_manipulation/test_delete.py::test_prune_unused_metabolites_sanity PASSED [ 27%]
    test/test_manipulation/test_delete.py::test_prune_unused_reactions_output_type PASSED [ 36%]
    test/test_manipulation/test_delete.py::test_prune_unused_rxns_functionality PASSED [ 45%]
    test/test_manipulation/test_delete.py::test_gene_knockout PASSED         [ 54%]
    test/test_manipulation/test_delete.py::test_remove_genes PASSED          [ 63%]
    test/test_manipulation/test_modify.py::test_escape_ids PASSED            [ 72%]
    test/test_manipulation/test_modify.py::test_rename_genes PASSED          [ 81%]
    test/test_manipulation/test_validate.py::test_validate_mass_balance PASSED [ 90%]
    test/test_manipulation/test_validate.py::test_validate_formula_compartment PASSED [100%]

    =============================== warnings summary ===============================
    ../usr/local/lib/python3.7/dist-packages/_pytest/config/__init__.py:1114
      /usr/local/lib/python3.7/dist-packages/_pytest/config/__init__.py:1114: PytestAssertRewriteWarning: Module already imported so cannot be rewritten: anyio
        self._mark_plugins_for_rewrite(hook)

    -- Docs: https://docs.pytest.org/en/stable/warnings.html
    ======================== 11 passed, 1 warning in 2.82s =========================

    <ExitCode.OK: 0>

### Reading a constraint-based model

If you have a SBML model

```python
ecoli = cobra.io.read_sbml_model("e_coli_core.xml")
```

If you have a JSON model

```python
ecoli = cobra.io.load_json_model(filename)
```

If you have a MAT model

```python
ecoli =  cobra.io.load_matlab_model("e_coli_core.mat")
```

### How a model object looks like

```python
ecoli
```

        Name
        e_coli_core

        Memory address
        0x07fbfebe36f50

        Number of metabolites
        72

        Number of reactions
        95

        Number of groups
        0

        Objective expression
        1.0*BIOMASS_Ecoli_core_w_GAM - 1.0*BIOMASS_Ecoli_core_w_GAM_reverse_712e5

        Compartments
        extracellular space, cytosol

### Different attributes of this model object

```python
vars(ecoli).keys()
```

    dict_keys(['_id', 'name', 'notes', '_annotation', '_trimmed', '_trimmed_genes', '_trimmed_reactions', 'genes', 'reactions', 'metabolites', 'groups', '_compartments', '_contexts', '_solver', '_tolerance', '_sbml'])

```python
ecoli.genes[:10]
```

    [<Gene b1241 at 0x7fbfe876eb10>,
     <Gene b0351 at 0x7fbfe8773050>,
     <Gene s0001 at 0x7fbfe8773410>,
     <Gene b1849 at 0x7fbfe8773990>,
     <Gene b3115 at 0x7fbfe8773ed0>,
     <Gene b2296 at 0x7fbfe8775050>,
     <Gene b1276 at 0x7fbfe87758d0>,
     <Gene b0118 at 0x7fbfe8775a10>,
     <Gene b0474 at 0x7fbfe8775f10>,
     <Gene b0116 at 0x7fbfe8775510>]

Looking at active exchange reactions

```python
ecoli.medium
```

    {'EX_co2_e': 1000.0,
     'EX_glc__D_e': 10.0,
     'EX_h2o_e': 1000.0,
     'EX_h_e': 1000.0,
     'EX_nh4_e': 1000.0,
     'EX_o2_e': 1000.0,
     'EX_pi_e': 1000.0}

Looking at all echange reactions

```python
ecoli.exchanges
```

    [<Reaction EX_ac_e at 0x7fbfe86a9d90>,
     <Reaction EX_acald_e at 0x7fbfe86acc90>,
     <Reaction EX_akg_e at 0x7fbfe86b22d0>,
     <Reaction EX_co2_e at 0x7fbfe86b7e10>,
     <Reaction EX_etoh_e at 0x7fbfe86b2b50>,
     <Reaction EX_for_e at 0x7fbfe86bb5d0>,
     <Reaction EX_fru_e at 0x7fbfe86bb790>,
     <Reaction EX_fum_e at 0x7fbfe86becd0>,
     <Reaction EX_glc__D_e at 0x7fbfe86bed50>,
     <Reaction EX_gln__L_e at 0x7fbfe86bb2d0>,
     <Reaction EX_glu__L_e at 0x7fbfe86c0a50>,
     <Reaction EX_h_e at 0x7fbfe86c0d10>,
     <Reaction EX_h2o_e at 0x7fbfe86c0f10>,
     <Reaction EX_lac__D_e at 0x7fbfe86c8dd0>,
     <Reaction EX_mal__L_e at 0x7fbfe86c8b50>,
     <Reaction EX_nh4_e at 0x7fbfe86c4650>,
     <Reaction EX_o2_e at 0x7fbfe86c8e10>,
     <Reaction EX_pi_e at 0x7fbfe86caf10>,
     <Reaction EX_pyr_e at 0x7fbfe86cbc50>,
     <Reaction EX_succ_e at 0x7fbfe86cb150>]

### Accessing the metabolites and reactions in the model

Metabolites

```python
ecoli.metabolites[:10]
```

    [<Metabolite glc__D_e at 0x7fbfe87b9cd0>,
     <Metabolite gln__L_c at 0x7fbfe87b9d50>,
     <Metabolite gln__L_e at 0x7fbfe87c05d0>,
     <Metabolite glu__L_c at 0x7fbfe87c0690>,
     <Metabolite glu__L_e at 0x7fbfe87c21d0>,
     <Metabolite glx_c at 0x7fbfe87c3810>,
     <Metabolite h2o_c at 0x7fbfe87c42d0>,
     <Metabolite h2o_e at 0x7fbfe87c4c10>,
     <Metabolite h_c at 0x7fbfe87c6c90>,
     <Metabolite h_e at 0x7fbfe87c38d0>]

```python
ecoli.metabolites.glc__D_e.summary()
```

glc__D_e

C6H12O6

Producing Reactions

      Percent
      Flux
      Reaction
      Definition

      100.00%
      10
      EX_glc__D_e
      glc__D_e <=>

Consuming Reactions

      Percent
      Flux
      Reaction
      Definition

      100.00%
      -10
      GLCpts
      glc__D_e + pep_c --> g6p_c + pyr_c

```python
vars(ecoli.metabolites.glc__D_e).keys()
```

    dict_keys(['_id', 'name', 'notes', '_annotation', '_model', '_reaction', 'formula', 'compartment', 'charge', '_bound'])

Reactions

```python
ecoli.reactions[:10]
```

    [<Reaction PFK at 0x7fbfe873e410>,
     <Reaction PFL at 0x7fbfe873e390>,
     <Reaction PGI at 0x7fbfe8740310>,
     <Reaction PGK at 0x7fbfe8740f10>,
     <Reaction PGL at 0x7fbfe8740550>,
     <Reaction ACALD at 0x7fbfe8749590>,
     <Reaction AKGt2r at 0x7fbfe874b3d0>,
     <Reaction PGM at 0x7fbfe873eb90>,
     <Reaction PIt2r at 0x7fbfe874e890>,
     <Reaction ALCD2x at 0x7fbfe874e490>]

```python
ecoli.reactions.ATPM.summary()
```

ATPM

atp_c + h2o_c --> adp_c + h_c + pi_c

Bounds: 8.39, 1000.0

Flux: 8.39

```python
vars(ecoli.reactions.ATPM).keys()
```

    dict_keys(['_id', 'name', 'notes', '_annotation', '_gene_reaction_rule', 'subsystem', '_genes', '_metabolites', '_model', '_lower_bound', '_upper_bound'])

```python
ecoli.reactions.ATPM.name
```

    'ATP maintenance requirement'

### Simulating the model using flux balance analysis

```python
solution = ecoli.optimize()
vars(solution).keys()
```

    dict_keys(['objective_value', 'status', 'fluxes', 'reduced_costs', 'shadow_prices'])

```python
print('Satus: ',solution.status)
print('Growth rate: %.2f' % solution.objective_value)
```

    Satus:  optimal
    Growth rate: 0.87

Using the `model.summary()` function for FBA

```python
ecoli.summary()
```

Objective

1.0 BIOMASS_Ecoli_core_w_GAM = 0.8739215069684302

Uptake

      Metabolite
      Reaction
      Flux
      C-Number
      C-Flux

      glc__D_e
      EX_glc__D_e
      10
      6
      100.00%

      nh4_e
      EX_nh4_e
      4.765
      0
      0.00%

      o2_e
      EX_o2_e
      21.8
      0
      0.00%

      pi_e
      EX_pi_e
      3.215
      0
      0.00%

Secretion

      Metabolite
      Reaction
      Flux
      C-Number
      C-Flux

      co2_e
      EX_co2_e
      -22.81
      1
      100.00%

      h2o_e
      EX_h2o_e
      -29.18
      0
      0.00%

      h_e
      EX_h_e
      -17.53
      0
      0.00%

### Flux variability analysis (FVA)

```python
from cobra.flux_analysis import flux_variability_analysis
flux_variability_analysis(ecoli, ecoli.reactions[:10])
```

      minimum
      maximum

      PFK
      7.477382e+00
      7.477382e+00

      PFL
      0.000000e+00
      -2.906827e-14

      PGI
      4.860861e+00
      4.860861e+00

      PGK
      -1.602353e+01
      -1.602353e+01

      PGL
      4.959985e+00
      4.959985e+00

      ACALD
      6.459615e-15
      0.000000e+00

      AKGt2r
      3.633533e-15
      0.000000e+00

      PGM
      -1.471614e+01
      -1.471614e+01

      PIt2r
      3.214895e+00
      3.214895e+00

      ALCD2x
      5.626116e-15
      0.000000e+00

Using `model.summary()` function for FVA

```python
ecoli.summary(fva=0.95)
```

Objective

1.0 BIOMASS_Ecoli_core_w_GAM = 0.7040369478590289

Uptake

      Metabolite
      Reaction
      Flux
      Range
      C-Number
      C-Flux

      glc__D_e
      EX_glc__D_e
      10
      [9.5;  10]
      6
      100.00%

      nh4_e
      EX_nh4_e
      3.839
      [3.647;  3.897]
      0
      0.00%

      o2_e
      EX_o2_e
      27.53
      [26.15;  30.76]
      0
      0.00%

      pi_e
      EX_pi_e
      2.59
      [2.46;  2.59]
      0
      0.00%

Secretion

      Metabolite
      Reaction
      Flux
      Range
      C-Number
      C-Flux

      ac_e
      EX_ac_e
      0
      [-0.5;  0]
      2
      0.00%

      acald_e
      EX_acald_e
      0
      [-0.5;  0]
      2
      0.00%

      akg_e
      EX_akg_e
      0
      [-0.25;  0]
      5
      0.00%

      co2_e
      EX_co2_e
      -26.64
      [-31.54;  -25.31]
      1
      88.69%

      etoh_e
      EX_etoh_e
      0
      [-0.5;  0]
      2
      0.00%

      for_e
      EX_for_e
      -3.398
      [-4.228;  0]
      1
      11.31%

      glu__L_e
      EX_glu__L_e
      0
      [-0.25;  0]
      5
      0.00%

      h2o_e
      EX_h2o_e
      -31.77
      [-36.41;  -30.18]
      0
      0.00%

      h_e
      EX_h_e
      -17.52
      [-17.65;  -13.42]
      0
      0.00%

      lac__D_e
      EX_lac__D_e
      0
      [-0.5;  0]
      3
      0.00%

      pyr_e
      EX_pyr_e
      0
      [-0.5;  0]
      3
      0.00%

      succ_e
      EX_succ_e
      0
      [-0.5;  0]
      4
      0.00%

### Simulating deletions

```python
from cobra.flux_analysis import (
    single_gene_deletion, single_reaction_deletion, double_gene_deletion,
    double_reaction_deletion)
```

Reaction deletions

```python
print('complete model: ', ecoli.optimize())
with ecoli:
    ecoli.reactions.PFK.knock_out()
    print('pfk knocked out: ', ecoli.optimize())
```

    complete model:  <Solution 0.874 at 0x7fbfe8663a50>
    pfk knocked out:  <Solution 0.704 at 0x7fbfe8663fd0>

```python
single_reaction_deletion(ecoli, ecoli.reactions[:20])
```

      ids
      growth
      status

      0
      {ACONTa}
      1.511990e-15
      optimal

      1
      {PPCK}
      8.739215e-01
      optimal

      2
      {PGI}
      8.631596e-01
      optimal

      3
      {AKGt2r}
      8.739215e-01
      optimal

      4
      {PGL}
      8.638133e-01
      optimal

      5
      {ACALD}
      8.739215e-01
      optimal

      6
      {PPS}
      8.739215e-01
      optimal

      7
      {PGK}
      1.354023e-15
      optimal

      8
      {PPC}
      8.707448e-01
      optimal

      9
      {ACKr}
      8.739215e-01
      optimal

      10
      {ACONTb}
      8.307127e-15
      optimal

      11
      {ALCD2x}
      8.739215e-01
      optimal

      12
      {PIt2r}
      -1.140194e-15
      optimal

      13
      {ACALDt}
      8.739215e-01
      optimal

      14
      {PGM}
      -7.865014e-32
      optimal

      15
      {ATPM}
      9.166475e-01
      optimal

      16
      {PFL}
      8.739215e-01
      optimal

      17
      {PFK}
      7.040369e-01
      optimal

      18
      {ACt2r}
      8.739215e-01
      optimal

      19
      {ADK1}
      8.739215e-01
      optimal

Gene deletions

```python
print('complete model: ', ecoli.optimize())
with ecoli:
    ecoli.genes.b1723.knock_out()
    print('pfkA knocked out: ', ecoli.optimize())
    ecoli.genes.b3916.knock_out()
    print('pfkB knocked out: ', ecoli.optimize())
```

    complete model:  <Solution 0.874 at 0x7fbfe87b9750>
    pfkA knocked out:  <Solution 0.874 at 0x7fbfe87b9790>
    pfkB knocked out:  <Solution 0.704 at 0x7fbfe87b9710>

```python
sin_gene_deletion_results = single_gene_deletion(ecoli)
```

```python
print(sin_gene_deletion_results.knockout["b1723"])
```

            ids    growth   status
    58  {b1723}  0.873922  optimal

```python
sin_deletion_results
```

      ids
      growth
      status

      0
      {b0904}
      0.873922
      optimal

      1
      {b2284}
      0.211663
      optimal

      2
      {b3916}
      0.873922
      optimal

      3
      {b0721}
      0.814298
      optimal

      4
      {b2285}
      0.211663
      optimal

      ...
      ...
      ...
      ...

      132
      {b0729}
      0.858307
      optimal

      133
      {b3737}
      0.374230
      optimal

      134
      {b3739}
      0.873922
      optimal

      135
      {b1611}
      0.873922
      optimal

      136
      {b2925}
      0.873922
      optimal

137 rows × 3 columns

Double deletions

```python
double_gene_deletion(ecoli, ecoli.genes[-5:]).round(4)
```

      ids
      growth
      status

      0
      {b3919, b2464}
      0.7040
      optimal

      1
      {b3919}
      0.7040
      optimal

      2
      {b2935}
      0.8739
      optimal

      3
      {b3919, b2465}
      0.7040
      optimal

      4
      {b0008, b2935}
      0.8739
      optimal

      5
      {b2935, b2464}
      0.8739
      optimal

      6
      {b0008, b2464}
      0.8648
      optimal

      7
      {b2465}
      0.8739
      optimal

      8
      {b0008}
      0.8739
      optimal

      9
      {b2464}
      0.8739
      optimal

      10
      {b0008, b2465}
      0.8739
      optimal

      11
      {b2935, b2465}
      0.0000
      optimal

      12
      {b3919, b2935}
      0.7040
      optimal

      13
      {b2464, b2465}
      0.8739
      optimal

      14
      {b0008, b3919}
      0.7040
      optimal

```python
double_reaction_deletion(ecoli, ecoli.reactions[-5:]).round(4)
```

      ids
      growth
      status

      0
      {PDH, NADTRHD}
      0.7925
      optimal

      1
      {O2t}
      0.2117
      optimal

      2
      {O2t, NH4t}
      -0.0000
      optimal

      3
      {O2t, PDH}
      0.2117
      optimal

      4
      {NADTRHD}
      0.8739
      optimal

      5
      {PDH}
      0.7967
      optimal

      6
      {PDH, NH4t}
      -0.0000
      optimal

      7
      {NADH16, PDH}
      0.2117
      optimal

      8
      {NADH16, NH4t}
      0.0000
      optimal

      9
      {O2t, NADTRHD}
      0.2117
      optimal

      10
      {NADH16, O2t}
      0.2117
      optimal

      11
      {NH4t}
      0.0000
      optimal

      12
      {NH4t, NADTRHD}
      0.0000
      optimal

      13
      {NADH16, NADTRHD}
      0.2117
      optimal

      14
      {NADH16}
      0.2117
      optimal

Constraint based modelling using CobraPy.ipynb


# CBMPy Tutorial 01 Getting Started

## Checking dependencies and loading CBMPy

PySCeS CBMPy is a flexible framework for contraint based modelling. With this tutorial we setup a CBMPy session and load a model. To begin with let's see if we have all the basic dependencies installed. For more information please see the CBMPy reference guide (available from http://cbmpy.sourceforge.net).

No additional files are required for this tutorial.

```python
# Required for basic CBMPy functionality
import numpy

# extended functionaliy
import libsbml
print('\nYou are using libSBML version {}'.format(libsbml.LIBSBML_VERSION_STRING))
if libsbml.LIBSBML_VERSION > 51100 and libsbml.LIBSBML_VERSION < 51106:
    print('Your version of libSBML supports SBML 3 FBC version 1, please consider upgrading to the latest version for FBC version 2 support.\n')
elif libsbml.LIBSBML_VERSION >= 51106:
    print('Your version of libSBML supports SBML 3 FBC versions 1 and 2.\n')
else:
    print('You are using an extrememly old version of libSBML please upgrade now!\n')

# optional, but highly recommended for rerading/writing excel spreadsheets
import xlrd
import xlwt
# sympy is only required for methods utilizing exact arithmetic
try:
    import sympy
except ImportError:
    print('\nSympy is only required for selected methods utilizing exact arithmetic and is not required for normal functionality.')

```

    You are using libSBML version 51700
    Your version of libSBML supports SBML 3 FBC versions 1 and 2.

If any of of the above imports failed consider installing them. Please see the online documentation for comprehensive installation [instructions](http://cbmpy.sourceforge.net/reference/cbmpy.html). Next we need to import CBMPy, at this point it is convenient to alias the imported module. In all tutorials this will be `cbm`.

```python
import cbmpy
```

You should some status text and a version of the welcome screen:
```
***********************************************************************
* Welcome to CBMPy (0.7.20) - PySCeS Constraint Based Modelling       *
*                http://cbmpy.sourceforge.net                         *
* Copyright(C) Brett G. Olivier 2014 - 2017                           *
* Dept. of Systems Bioinformatics                                     *
* Vrije Universiteit Amsterdam, Amsterdam, The Netherlands            *
* CBMPy is developed as part of the BeBasic MetaToolKit Project       *
* Distributed under the GNU GPL v 3.0 licence, see                    *
* LICENCE (supplied with this release) for details                    *
***********************************************************************
```

## Testing the installation, loading the default models

CBMPy now includes a growing set of Nose based unittests, try run them to test your installation (this may take a few minutes) unfortunately the test runner does not seem to like the Notebook environment but try the following in a console:

```import cbmpy
cbmpy.test()```

In addition CBMPy is also shipped with two models that can be used to test your installation and get going quickly. In order to use these models at least SBML FBC version 1 support is required and both models can be loaded using the *special* filenames: `cbmpy_test_core` and  `cbmpy_test_ecoli`.

In CBMPy models are *read* from files and *instantiated* as Python objects. Here I will instantiated two model objectes from the two test files: `cmod` and `ecoli` but you can use any Python name. Note it is possible to instantiate multiple *independent* instances from the same input file.

```python
cmod = cbmpy.readSBML3FBC('cbmpy_test_core')
ecoli = cbmpy.readSBML3FBC('cbmpy_test_ecoli')
```

    core_memesa_model.l3.xml
    FBC version: 1
    M.getNumReactions: 26
    M.getNumSpecies: 22
    FBC.getNumObjectives: 1
    FBC.getNumGeneAssociations: 0
    FBC.getNumFluxBounds: 52
    Zero dimension compartment detected: Cell
    INFO: Active objective: objMaxJ25
    Adding objective: objMaxJ25

    SBML3 load time: 0.018

    INFO: no standard gene encoding detected, attempting to load from annotations.
    INFO: used key(s) '[]'
    INFO: Added 0 new genes and 0 associations to model
    Ecoli_iJR904.glc.l3.xml
    FBC version: 2
    M.getNumReactions: 1075
    M.getNumSpecies: 761
    FBC.getNumObjectives: 1
    FBC.getNumParameters: 7
    FBC.getNumGeneProducts: 904
    Zero dimension compartment detected: c
    Zero dimension compartment detected: e
    INFO: Active objective: obj
    Adding objective: obj
    Groups support: <GroupsModelPlugin>
    Group.getNumGroups: 31

    SBML3 load time: 2.082

Once model objects have been instantiated they can be modified and analysed, here we will perform FBA on each model and test that CBMPy is calculating the correct result:

```python
print('\n*****\nCalculating FBA for model: core\n*****')
optcm = cbmpy.doFBA(cmod)

print('\n*****\nCalculating FBA for model: ecoli\n*****')
optec = cbmpy.doFBA(ecoli)

if 1.0 - abs(round(optcm, 2)) < 1.2e-10:
    print('\nCore model test FBA: PASSED.\n Value expected: 1.0, calculated: {}'.format(round(optcm, 2)))
else:
    print('\nCore model test FBA: FAILED.\n Value expected: 1.0, calculated: {}'.format(round(optcm, 2)))
if 0.92 - abs(round(optec, 2)) < 1.2e-10:
    print('\nE. Coli model test FBA: PASSED.\n Value expected: 0.92, calculated: {}'.format(round(optec, 2)))
else:
    print('\nE. Coli model test FBA: FAILED.\n Value expected: 0.92, calculated: {}'.format(round(optec, 2)))

```

    *****
    Calculating FBA for model: core
    *****

    cplx_constructLPfromFBA time: 0.00600004196167

    cplx_analyzeModel FBA --> LP time: 0.00600004196167

    CPXPARAM_Read_DataCheck                          1
    Tried aggregator 1 time.
    LP Presolve eliminated 0 rows and 4 columns.
    Aggregator did 11 substitutions.
    Reduced LP has 8 rows, 11 columns, and 20 nonzeros.
    Presolve time = 0.02 sec. (0.02 ticks)
    Initializing dual steep norms . . .

    Iteration log . . .
    Iteration:     1   Dual objective     =          1000.000000
    INFO: Model is optimal: 1
    Solution status =  1 : optimal
    Solution method =  2 : dual
    Objective value =  1.0
    Model is optimal
    Status: LPS_OPT
    Model is optimal
    Model is optimal

    analyzeModel objective value: 1.0

    *****
    Calculating FBA for model: ecoli
    *****

    cplx_constructLPfromFBA time: 0.0649998188019

    cplx_analyzeModel FBA --> LP time: 0.0649998188019

    CPXPARAM_Read_DataCheck                          1
    Tried aggregator 1 time.
    LP Presolve eliminated 274 rows and 363 columns.
    Aggregator did 231 substitutions.
    Reduced LP has 256 rows, 481 columns, and 2430 nonzeros.
    Presolve time = 0.01 sec. (1.12 ticks)

    Iteration log . . .
    Iteration:     1   Dual objective     =        999999.000000
    Iteration:    62   Dual objective     =        552515.995359
    Iteration:   124   Dual objective     =        362603.457641
    Iteration:   186   Dual objective     =        130350.090149
    Iteration:   248   Dual objective     =         99850.178645
    Iteration:   310   Dual objective     =         49824.030589
    Iteration:   372   Dual objective     =             1.401622
    INFO: Model is optimal: 1
    Solution status =  1 : optimal
    Solution method =  2 : dual
    Objective value =  0.921948095023
    Model is optimal
    Status: LPS_OPT
    Model is optimal
    Model is optimal

    analyzeModel objective value: 0.921948095023

    Core model test FBA: PASSED.
     Value expected: 1.0, calculated: 1.0

    E. Coli model test FBA: PASSED.
     Value expected: 0.92, calculated: 0.92

If both of the above tests passed, congratulations you are ready to model with CBMPy. In tutorial 2 we will look in more detail at model ecoding formats that can be used with CBMPy.


```python
# Copyright 2019 NVIDIA Corporation. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ==============================================================================
```

# Torch-TensorRT Getting Started - CitriNet

## Overview

[Citrinet](https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/main/asr/models.html#citrinet) is an acoustic model used for the speech to text recognition task. It is a version of [QuartzNet](https://arxiv.org/pdf/1910.10261.pdf) that extends [ContextNet](https://arxiv.org/pdf/2005.03191.pdf), utilizing subword encoding (via Word Piece tokenization) and Squeeze-and-Excitation(SE) mechanism and are therefore smaller than QuartzNet models.

CitriNet models take in audio segments and transcribe them to letter, byte pair, or word piece sequences.

### Learning objectives

This notebook demonstrates the steps for optimizing a pretrained CitriNet model with Torch-TensorRT, and running it to test the speedup obtained.

## Content
1. [Requirements](#1)
1. [Download Citrinet model](#2)
1. [Create Torch-TensorRT modules](#3)
1. [Benchmark Torch-TensorRT models](#4)
1. [Conclusion](#5)

## 1. Requirements

Follow the steps in [README](README.md) to prepare a Docker container, within which you can run this notebook.
This notebook assumes that you are within a Jupyter environment in a docker container with Torch-TensorRT installed, such as an NGC monthly release of `nvcr.io/nvidia/pytorch:<yy.mm>-py3` (where `yy` indicates the last two numbers of a calendar year, and `mm` indicates the month in two-digit numerical form)

Now that you are in the docker, the next step is to install the required dependencies.

```python
# Install dependencies
!pip install wget
!apt-get update && DEBIAN_FRONTEND=noninteractive  apt-get install -y libsndfile1 ffmpeg
!pip install Cython

## Install NeMo
!pip install nemo_toolkit[all]==1.5.1
```

    Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
    Requirement already satisfied: wget in /opt/conda/lib/python3.8/site-packages (3.2)
    WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
    Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease
    Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease
    Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease
    Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease
    Reading package lists... Done
    Reading package lists... Done
    Building dependency tree
    Reading state information... Done
    libsndfile1 is already the newest version (1.0.28-7ubuntu0.1).
    ffmpeg is already the newest version (7:4.2.4-1ubuntu0.1).
    0 upgraded, 0 newly installed, 0 to remove and 22 not upgraded.
    Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
    Requirement already satisfied: Cython in /opt/conda/lib/python3.8/site-packages (0.29.28)
    WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
    Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
    Requirement already satisfied: nemo_toolkit[all]==1.5.1 in /opt/conda/lib/python3.8/site-packages (1.5.1)
    Requirement already satisfied: numpy>=1.18.2 in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (1.22.3)
    Requirement already satisfied: onnx>=1.7.0 in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (1.10.1)
    Requirement already satisfied: python-dateutil in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (2.8.2)
    Requirement already satisfied: tqdm>=4.41.0 in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (4.63.0)
    Requirement already satisfied: sentencepiece<1.0.0 in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (0.1.96)
    Requirement already satisfied: wget in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (3.2)
    Requirement already satisfied: numba in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (0.53.1)
    Requirement already satisfied: torch in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (1.12.0a0+2c916ef)
    Requirement already satisfied: unidecode in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (1.3.4)
    Requirement already satisfied: frozendict in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (2.3.2)
    Requirement already satisfied: wrapt in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (1.14.0)
    Requirement already satisfied: scikit-learn in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (0.24.2)
    Requirement already satisfied: ruamel.yaml in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (0.17.21)
    Requirement already satisfied: pesq in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (0.0.3)
    Requirement already satisfied: torchvision in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (0.13.0a0)
    Requirement already satisfied: gdown in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (4.4.0)
    Requirement already satisfied: editdistance in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (0.6.0)
    Requirement already satisfied: boto3 in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (1.21.45)
    Requirement already satisfied: isort[requirements]<5 in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (4.3.21)
    Requirement already satisfied: hydra-core>=1.1.0 in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (1.1.2)
    Requirement already satisfied: youtokentome>=1.0.5 in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (1.0.6)
    Requirement already satisfied: pytorch-lightning>=1.5.0 in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (1.6.1)
    Requirement already satisfied: jieba in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (0.42.1)
    Requirement already satisfied: fasttext in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (0.9.2)
    Requirement already satisfied: soundfile in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (0.10.3.post1)
    Requirement already satisfied: kaldiio in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (2.17.2)
    Requirement already satisfied: pangu in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (4.0.6.1)
    Requirement already satisfied: kaldi-python-io in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (1.2.2)
    Requirement already satisfied: parameterized in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (0.8.1)
    Requirement already satisfied: h5py in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (3.6.0)
    Requirement already satisfied: rapidfuzz in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (2.0.10)
    Requirement already satisfied: marshmallow in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (3.15.0)
    Requirement already satisfied: opencc in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (1.1.3)
    Requirement already satisfied: braceexpand in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (0.1.7)
    Requirement already satisfied: omegaconf>=2.1.0 in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (2.1.2)
    Requirement already satisfied: sphinx in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (4.4.0)
    Requirement already satisfied: pillow in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (9.0.0)
    Requirement already satisfied: wordninja==2.0.0 in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (2.0.0)
    Requirement already satisfied: torch-stft in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (0.1.4)
    Requirement already satisfied: sox in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (1.4.1)
    Requirement already satisfied: librosa in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (0.9.1)
    Requirement already satisfied: regex in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (2022.3.15)
    Requirement already satisfied: sacrebleu[ja] in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (2.0.0)
    Requirement already satisfied: black==19.10b0 in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (19.10b0)
    Requirement already satisfied: pydub in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (0.25.1)
    Requirement already satisfied: sphinxcontrib-bibtex in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (2.4.2)
    Requirement already satisfied: inflect in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (5.5.2)
    Requirement already satisfied: pyannote.core in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (4.4)
    Requirement already satisfied: packaging in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (21.3)
    Requirement already satisfied: kaldi-io in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (0.9.4)
    Requirement already satisfied: pyannote.metrics in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (3.2)
    Requirement already satisfied: g2p-en in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (2.1.0)
    Requirement already satisfied: matplotlib in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (3.5.1)
    Requirement already satisfied: torchmetrics>=0.4.1rc0 in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (0.8.0)
    Requirement already satisfied: nltk>=3.6.5 in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (3.7)
    Requirement already satisfied: pyyaml<6 in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (5.4.1)
    Requirement already satisfied: scipy in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (1.6.3)
    Requirement already satisfied: ipywidgets in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (7.7.0)
    Requirement already satisfied: pytest in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (6.2.5)
    Requirement already satisfied: pandas in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (1.3.5)
    Requirement already satisfied: pytest-runner in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (6.0.0)
    Requirement already satisfied: transformers>=4.0.1 in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (4.18.0)
    Requirement already satisfied: sacremoses>=0.0.43 in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (0.0.49)
    Requirement already satisfied: pystoi in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (0.3.3)
    Requirement already satisfied: attrdict in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (2.0.1)
    Requirement already satisfied: webdataset<=0.1.62,>=0.1.48 in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (0.1.62)
    Requirement already satisfied: wandb in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (0.12.15)
    Requirement already satisfied: pypinyin in /opt/conda/lib/python3.8/site-packages (from nemo_toolkit[all]==1.5.1) (0.46.0)
    Requirement already satisfied: attrs>=18.1.0 in /opt/conda/lib/python3.8/site-packages (from black==19.10b0->nemo_toolkit[all]==1.5.1) (21.4.0)
    Requirement already satisfied: appdirs in /opt/conda/lib/python3.8/site-packages (from black==19.10b0->nemo_toolkit[all]==1.5.1) (1.4.4)
    Requirement already satisfied: typed-ast>=1.4.0 in /opt/conda/lib/python3.8/site-packages (from black==19.10b0->nemo_toolkit[all]==1.5.1) (1.5.3)
    Requirement already satisfied: pathspec<1,>=0.6 in /opt/conda/lib/python3.8/site-packages (from black==19.10b0->nemo_toolkit[all]==1.5.1) (0.9.0)
    Requirement already satisfied: click>=6.5 in /opt/conda/lib/python3.8/site-packages (from black==19.10b0->nemo_toolkit[all]==1.5.1) (8.0.4)
    Requirement already satisfied: toml>=0.9.4 in /opt/conda/lib/python3.8/site-packages (from black==19.10b0->nemo_toolkit[all]==1.5.1) (0.10.2)
    Requirement already satisfied: antlr4-python3-runtime==4.8 in /opt/conda/lib/python3.8/site-packages (from hydra-core>=1.1.0->nemo_toolkit[all]==1.5.1) (4.8)
    Requirement already satisfied: importlib-resources<5.3 in /opt/conda/lib/python3.8/site-packages (from hydra-core>=1.1.0->nemo_toolkit[all]==1.5.1) (5.2.3)
    Requirement already satisfied: zipp>=3.1.0 in /opt/conda/lib/python3.8/site-packages (from importlib-resources<5.3->hydra-core>=1.1.0->nemo_toolkit[all]==1.5.1) (3.7.0)
    Requirement already satisfied: pip-api in /opt/conda/lib/python3.8/site-packages (from isort[requirements]<5->nemo_toolkit[all]==1.5.1) (0.0.29)
    Requirement already satisfied: pipreqs in /opt/conda/lib/python3.8/site-packages (from isort[requirements]<5->nemo_toolkit[all]==1.5.1) (0.4.11)
    Requirement already satisfied: fonttools>=4.22.0 in /opt/conda/lib/python3.8/site-packages (from matplotlib->nemo_toolkit[all]==1.5.1) (4.31.2)
    Requirement already satisfied: kiwisolver>=1.0.1 in /opt/conda/lib/python3.8/site-packages (from matplotlib->nemo_toolkit[all]==1.5.1) (1.4.0)
    Requirement already satisfied: pyparsing>=2.2.1 in /opt/conda/lib/python3.8/site-packages (from matplotlib->nemo_toolkit[all]==1.5.1) (3.0.7)
    Requirement already satisfied: cycler>=0.10 in /opt/conda/lib/python3.8/site-packages (from matplotlib->nemo_toolkit[all]==1.5.1) (0.11.0)
    Requirement already satisfied: joblib in /opt/conda/lib/python3.8/site-packages (from nltk>=3.6.5->nemo_toolkit[all]==1.5.1) (1.1.0)
    Requirement already satisfied: typing-extensions>=3.6.2.1 in /opt/conda/lib/python3.8/site-packages (from onnx>=1.7.0->nemo_toolkit[all]==1.5.1) (4.1.1)
    Requirement already satisfied: six in /opt/conda/lib/python3.8/site-packages (from onnx>=1.7.0->nemo_toolkit[all]==1.5.1) (1.16.0)
    Requirement already satisfied: protobuf>=3.12.2 in /opt/conda/lib/python3.8/site-packages (from onnx>=1.7.0->nemo_toolkit[all]==1.5.1) (3.19.4)
    Requirement already satisfied: pyDeprecate<0.4.0,>=0.3.1 in /opt/conda/lib/python3.8/site-packages (from pytorch-lightning>=1.5.0->nemo_toolkit[all]==1.5.1) (0.3.2)
    Requirement already satisfied: tensorboard>=2.2.0 in /opt/conda/lib/python3.8/site-packages (from pytorch-lightning>=1.5.0->nemo_toolkit[all]==1.5.1) (2.8.0)
    Requirement already satisfied: fsspec[http]!=2021.06.0,>=2021.05.0 in /opt/conda/lib/python3.8/site-packages (from pytorch-lightning>=1.5.0->nemo_toolkit[all]==1.5.1) (2022.2.0)
    Requirement already satisfied: requests in /opt/conda/lib/python3.8/site-packages (from fsspec[http]!=2021.06.0,>=2021.05.0->pytorch-lightning>=1.5.0->nemo_toolkit[all]==1.5.1) (2.27.1)
    Requirement already satisfied: aiohttp in /opt/conda/lib/python3.8/site-packages (from fsspec[http]!=2021.06.0,>=2021.05.0->pytorch-lightning>=1.5.0->nemo_toolkit[all]==1.5.1) (3.8.1)
    Requirement already satisfied: werkzeug>=0.11.15 in /opt/conda/lib/python3.8/site-packages (from tensorboard>=2.2.0->pytorch-lightning>=1.5.0->nemo_toolkit[all]==1.5.1) (2.0.3)
    Requirement already satisfied: markdown>=2.6.8 in /opt/conda/lib/python3.8/site-packages (from tensorboard>=2.2.0->pytorch-lightning>=1.5.0->nemo_toolkit[all]==1.5.1) (3.3.6)
    Requirement already satisfied: setuptools>=41.0.0 in /opt/conda/lib/python3.8/site-packages (from tensorboard>=2.2.0->pytorch-lightning>=1.5.0->nemo_toolkit[all]==1.5.1) (59.5.0)
    Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in /opt/conda/lib/python3.8/site-packages (from tensorboard>=2.2.0->pytorch-lightning>=1.5.0->nemo_toolkit[all]==1.5.1) (0.4.6)
    Requirement already satisfied: google-auth<3,>=1.6.3 in /opt/conda/lib/python3.8/site-packages (from tensorboard>=2.2.0->pytorch-lightning>=1.5.0->nemo_toolkit[all]==1.5.1) (2.6.2)
    Requirement already satisfied: wheel>=0.26 in /opt/conda/lib/python3.8/site-packages (from tensorboard>=2.2.0->pytorch-lightning>=1.5.0->nemo_toolkit[all]==1.5.1) (0.37.1)
    Requirement already satisfied: grpcio>=1.24.3 in /opt/conda/lib/python3.8/site-packages (from tensorboard>=2.2.0->pytorch-lightning>=1.5.0->nemo_toolkit[all]==1.5.1) (1.44.0)
    Requirement already satisfied: tensorboard-data-server<0.7.0,>=0.6.0 in /opt/conda/lib/python3.8/site-packages (from tensorboard>=2.2.0->pytorch-lightning>=1.5.0->nemo_toolkit[all]==1.5.1) (0.6.1)
    Requirement already satisfied: tensorboard-plugin-wit>=1.6.0 in /opt/conda/lib/python3.8/site-packages (from tensorboard>=2.2.0->pytorch-lightning>=1.5.0->nemo_toolkit[all]==1.5.1) (1.8.1)
    Requirement already satisfied: absl-py>=0.4 in /opt/conda/lib/python3.8/site-packages (from tensorboard>=2.2.0->pytorch-lightning>=1.5.0->nemo_toolkit[all]==1.5.1) (1.0.0)
    Requirement already satisfied: cachetools<6.0,>=2.0.0 in /opt/conda/lib/python3.8/site-packages (from google-auth<3,>=1.6.3->tensorboard>=2.2.0->pytorch-lightning>=1.5.0->nemo_toolkit[all]==1.5.1) (5.0.0)
    Requirement already satisfied: pyasn1-modules>=0.2.1 in /opt/conda/lib/python3.8/site-packages (from google-auth<3,>=1.6.3->tensorboard>=2.2.0->pytorch-lightning>=1.5.0->nemo_toolkit[all]==1.5.1) (0.2.8)
    Requirement already satisfied: rsa<5,>=3.1.4 in /opt/conda/lib/python3.8/site-packages (from google-auth<3,>=1.6.3->tensorboard>=2.2.0->pytorch-lightning>=1.5.0->nemo_toolkit[all]==1.5.1) (4.8)
    Requirement already satisfied: requests-oauthlib>=0.7.0 in /opt/conda/lib/python3.8/site-packages (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard>=2.2.0->pytorch-lightning>=1.5.0->nemo_toolkit[all]==1.5.1) (1.3.1)
    Requirement already satisfied: importlib-metadata>=4.4 in /opt/conda/lib/python3.8/site-packages (from markdown>=2.6.8->tensorboard>=2.2.0->pytorch-lightning>=1.5.0->nemo_toolkit[all]==1.5.1) (4.11.3)
    Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /opt/conda/lib/python3.8/site-packages (from pyasn1-modules>=0.2.1->google-auth<3,>=1.6.3->tensorboard>=2.2.0->pytorch-lightning>=1.5.0->nemo_toolkit[all]==1.5.1) (0.4.8)
    Requirement already satisfied: charset-normalizer~=2.0.0 in /opt/conda/lib/python3.8/site-packages (from requests->fsspec[http]!=2021.06.0,>=2021.05.0->pytorch-lightning>=1.5.0->nemo_toolkit[all]==1.5.1) (2.0.12)
    Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.8/site-packages (from requests->fsspec[http]!=2021.06.0,>=2021.05.0->pytorch-lightning>=1.5.0->nemo_toolkit[all]==1.5.1) (2021.10.8)
    Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.8/site-packages (from requests->fsspec[http]!=2021.06.0,>=2021.05.0->pytorch-lightning>=1.5.0->nemo_toolkit[all]==1.5.1) (3.3)
    Requirement already satisfied: urllib3<1.27,>=1.21.1 in /opt/conda/lib/python3.8/site-packages (from requests->fsspec[http]!=2021.06.0,>=2021.05.0->pytorch-lightning>=1.5.0->nemo_toolkit[all]==1.5.1) (1.26.8)
    Requirement already satisfied: oauthlib>=3.0.0 in /opt/conda/lib/python3.8/site-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard>=2.2.0->pytorch-lightning>=1.5.0->nemo_toolkit[all]==1.5.1) (3.2.0)
    Requirement already satisfied: huggingface-hub<1.0,>=0.1.0 in /opt/conda/lib/python3.8/site-packages (from transformers>=4.0.1->nemo_toolkit[all]==1.5.1) (0.5.1)
    Requirement already satisfied: tokenizers!=0.11.3,<0.13,>=0.11.1 in /opt/conda/lib/python3.8/site-packages (from transformers>=4.0.1->nemo_toolkit[all]==1.5.1) (0.12.1)
    Requirement already satisfied: filelock in /opt/conda/lib/python3.8/site-packages (from transformers>=4.0.1->nemo_toolkit[all]==1.5.1) (3.6.0)
    Requirement already satisfied: frozenlist>=1.1.1 in /opt/conda/lib/python3.8/site-packages (from aiohttp->fsspec[http]!=2021.06.0,>=2021.05.0->pytorch-lightning>=1.5.0->nemo_toolkit[all]==1.5.1) (1.3.0)
    Requirement already satisfied: yarl<2.0,>=1.0 in /opt/conda/lib/python3.8/site-packages (from aiohttp->fsspec[http]!=2021.06.0,>=2021.05.0->pytorch-lightning>=1.5.0->nemo_toolkit[all]==1.5.1) (1.7.2)
    Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /opt/conda/lib/python3.8/site-packages (from aiohttp->fsspec[http]!=2021.06.0,>=2021.05.0->pytorch-lightning>=1.5.0->nemo_toolkit[all]==1.5.1) (4.0.2)
    Requirement already satisfied: multidict<7.0,>=4.5 in /opt/conda/lib/python3.8/site-packages (from aiohttp->fsspec[http]!=2021.06.0,>=2021.05.0->pytorch-lightning>=1.5.0->nemo_toolkit[all]==1.5.1) (6.0.2)
    Requirement already satisfied: aiosignal>=1.1.2 in /opt/conda/lib/python3.8/site-packages (from aiohttp->fsspec[http]!=2021.06.0,>=2021.05.0->pytorch-lightning>=1.5.0->nemo_toolkit[all]==1.5.1) (1.2.0)
    Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in /opt/conda/lib/python3.8/site-packages (from boto3->nemo_toolkit[all]==1.5.1) (0.5.2)
    Requirement already satisfied: botocore<1.25.0,>=1.24.45 in /opt/conda/lib/python3.8/site-packages (from boto3->nemo_toolkit[all]==1.5.1) (1.24.45)
    Requirement already satisfied: jmespath<2.0.0,>=0.7.1 in /opt/conda/lib/python3.8/site-packages (from boto3->nemo_toolkit[all]==1.5.1) (1.0.0)
    Requirement already satisfied: pybind11>=2.2 in /opt/conda/lib/python3.8/site-packages (from fasttext->nemo_toolkit[all]==1.5.1) (2.9.1)
    Requirement already satisfied: distance>=0.1.3 in /opt/conda/lib/python3.8/site-packages (from g2p-en->nemo_toolkit[all]==1.5.1) (0.1.3)
    Requirement already satisfied: beautifulsoup4 in /opt/conda/lib/python3.8/site-packages (from gdown->nemo_toolkit[all]==1.5.1) (4.10.0)
    Requirement already satisfied: soupsieve>1.2 in /opt/conda/lib/python3.8/site-packages (from beautifulsoup4->gdown->nemo_toolkit[all]==1.5.1) (2.3.1)
    Requirement already satisfied: ipython-genutils~=0.2.0 in /opt/conda/lib/python3.8/site-packages (from ipywidgets->nemo_toolkit[all]==1.5.1) (0.2.0)
    Requirement already satisfied: ipython>=4.0.0 in /opt/conda/lib/python3.8/site-packages (from ipywidgets->nemo_toolkit[all]==1.5.1) (8.1.1)
    Requirement already satisfied: ipykernel>=4.5.1 in /opt/conda/lib/python3.8/site-packages (from ipywidgets->nemo_toolkit[all]==1.5.1) (6.9.2)
    Requirement already satisfied: jupyterlab-widgets>=1.0.0 in /opt/conda/lib/python3.8/site-packages (from ipywidgets->nemo_toolkit[all]==1.5.1) (1.1.0)
    Requirement already satisfied: widgetsnbextension~=3.6.0 in /opt/conda/lib/python3.8/site-packages (from ipywidgets->nemo_toolkit[all]==1.5.1) (3.6.0)
    Requirement already satisfied: traitlets>=4.3.1 in /opt/conda/lib/python3.8/site-packages (from ipywidgets->nemo_toolkit[all]==1.5.1) (5.1.1)
    Requirement already satisfied: nbformat>=4.2.0 in /opt/conda/lib/python3.8/site-packages (from ipywidgets->nemo_toolkit[all]==1.5.1) (5.2.0)
    Requirement already satisfied: jupyter-client<8.0 in /opt/conda/lib/python3.8/site-packages (from ipykernel>=4.5.1->ipywidgets->nemo_toolkit[all]==1.5.1) (7.1.2)
    Requirement already satisfied: psutil in /opt/conda/lib/python3.8/site-packages (from ipykernel>=4.5.1->ipywidgets->nemo_toolkit[all]==1.5.1) (5.9.0)
    Requirement already satisfied: tornado<7.0,>=4.2 in /opt/conda/lib/python3.8/site-packages (from ipykernel>=4.5.1->ipywidgets->nemo_toolkit[all]==1.5.1) (6.1)
    Requirement already satisfied: debugpy<2.0,>=1.0.0 in /opt/conda/lib/python3.8/site-packages (from ipykernel>=4.5.1->ipywidgets->nemo_toolkit[all]==1.5.1) (1.5.1)
    Requirement already satisfied: nest-asyncio in /opt/conda/lib/python3.8/site-packages (from ipykernel>=4.5.1->ipywidgets->nemo_toolkit[all]==1.5.1) (1.5.4)
    Requirement already satisfied: matplotlib-inline<0.2.0,>=0.1.0 in /opt/conda/lib/python3.8/site-packages (from ipykernel>=4.5.1->ipywidgets->nemo_toolkit[all]==1.5.1) (0.1.3)
    Requirement already satisfied: pickleshare in /opt/conda/lib/python3.8/site-packages (from ipython>=4.0.0->ipywidgets->nemo_toolkit[all]==1.5.1) (0.7.5)
    Requirement already satisfied: decorator in /opt/conda/lib/python3.8/site-packages (from ipython>=4.0.0->ipywidgets->nemo_toolkit[all]==1.5.1) (5.1.1)
    Requirement already satisfied: pygments>=2.4.0 in /opt/conda/lib/python3.8/site-packages (from ipython>=4.0.0->ipywidgets->nemo_toolkit[all]==1.5.1) (2.11.2)
    Requirement already satisfied: stack-data in /opt/conda/lib/python3.8/site-packages (from ipython>=4.0.0->ipywidgets->nemo_toolkit[all]==1.5.1) (0.2.0)
    Requirement already satisfied: prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0 in /opt/conda/lib/python3.8/site-packages (from ipython>=4.0.0->ipywidgets->nemo_toolkit[all]==1.5.1) (3.0.27)
    Requirement already satisfied: backcall in /opt/conda/lib/python3.8/site-packages (from ipython>=4.0.0->ipywidgets->nemo_toolkit[all]==1.5.1) (0.2.0)
    Requirement already satisfied: jedi>=0.16 in /opt/conda/lib/python3.8/site-packages (from ipython>=4.0.0->ipywidgets->nemo_toolkit[all]==1.5.1) (0.18.1)
    Requirement already satisfied: pexpect>4.3 in /opt/conda/lib/python3.8/site-packages (from ipython>=4.0.0->ipywidgets->nemo_toolkit[all]==1.5.1) (4.8.0)
    Requirement already satisfied: parso<0.9.0,>=0.8.0 in /opt/conda/lib/python3.8/site-packages (from jedi>=0.16->ipython>=4.0.0->ipywidgets->nemo_toolkit[all]==1.5.1) (0.8.3)
    Requirement already satisfied: entrypoints in /opt/conda/lib/python3.8/site-packages (from jupyter-client<8.0->ipykernel>=4.5.1->ipywidgets->nemo_toolkit[all]==1.5.1) (0.4)
    Requirement already satisfied: pyzmq>=13 in /opt/conda/lib/python3.8/site-packages (from jupyter-client<8.0->ipykernel>=4.5.1->ipywidgets->nemo_toolkit[all]==1.5.1) (22.3.0)
    Requirement already satisfied: jupyter-core>=4.6.0 in /opt/conda/lib/python3.8/site-packages (from jupyter-client<8.0->ipykernel>=4.5.1->ipywidgets->nemo_toolkit[all]==1.5.1) (4.9.2)
    Requirement already satisfied: jsonschema!=2.5.0,>=2.4 in /opt/conda/lib/python3.8/site-packages (from nbformat>=4.2.0->ipywidgets->nemo_toolkit[all]==1.5.1) (4.4.0)
    Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /opt/conda/lib/python3.8/site-packages (from jsonschema!=2.5.0,>=2.4->nbformat>=4.2.0->ipywidgets->nemo_toolkit[all]==1.5.1) (0.18.1)
    Requirement already satisfied: ptyprocess>=0.5 in /opt/conda/lib/python3.8/site-packages (from pexpect>4.3->ipython>=4.0.0->ipywidgets->nemo_toolkit[all]==1.5.1) (0.7.0)
    Requirement already satisfied: wcwidth in /opt/conda/lib/python3.8/site-packages (from prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0->ipython>=4.0.0->ipywidgets->nemo_toolkit[all]==1.5.1) (0.2.5)
    Requirement already satisfied: notebook>=4.4.1 in /opt/conda/lib/python3.8/site-packages (from widgetsnbextension~=3.6.0->ipywidgets->nemo_toolkit[all]==1.5.1) (6.4.1)
    Requirement already satisfied: Send2Trash>=1.5.0 in /opt/conda/lib/python3.8/site-packages (from notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets->nemo_toolkit[all]==1.5.1) (1.8.0)
    Requirement already satisfied: prometheus-client in /opt/conda/lib/python3.8/site-packages (from notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets->nemo_toolkit[all]==1.5.1) (0.13.1)
    Requirement already satisfied: terminado>=0.8.3 in /opt/conda/lib/python3.8/site-packages (from notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets->nemo_toolkit[all]==1.5.1) (0.13.3)
    Requirement already satisfied: jinja2 in /opt/conda/lib/python3.8/site-packages (from notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets->nemo_toolkit[all]==1.5.1) (3.0.3)
    Requirement already satisfied: nbconvert in /opt/conda/lib/python3.8/site-packages (from notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets->nemo_toolkit[all]==1.5.1) (6.4.4)
    Requirement already satisfied: argon2-cffi in /opt/conda/lib/python3.8/site-packages (from notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets->nemo_toolkit[all]==1.5.1) (21.3.0)
    Requirement already satisfied: argon2-cffi-bindings in /opt/conda/lib/python3.8/site-packages (from argon2-cffi->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets->nemo_toolkit[all]==1.5.1) (21.2.0)
    Requirement already satisfied: cffi>=1.0.1 in /opt/conda/lib/python3.8/site-packages (from argon2-cffi-bindings->argon2-cffi->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets->nemo_toolkit[all]==1.5.1) (1.15.0)
    Requirement already satisfied: pycparser in /opt/conda/lib/python3.8/site-packages (from cffi>=1.0.1->argon2-cffi-bindings->argon2-cffi->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets->nemo_toolkit[all]==1.5.1) (2.21)
    Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/lib/python3.8/site-packages (from jinja2->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets->nemo_toolkit[all]==1.5.1) (2.1.1)
    Requirement already satisfied: resampy>=0.2.2 in /opt/conda/lib/python3.8/site-packages (from librosa->nemo_toolkit[all]==1.5.1) (0.2.2)
    Requirement already satisfied: pooch>=1.0 in /opt/conda/lib/python3.8/site-packages (from librosa->nemo_toolkit[all]==1.5.1) (1.6.0)
    Requirement already satisfied: audioread>=2.1.5 in /opt/conda/lib/python3.8/site-packages (from librosa->nemo_toolkit[all]==1.5.1) (2.1.9)
    Requirement already satisfied: llvmlite<0.37,>=0.36.0rc1 in /opt/conda/lib/python3.8/site-packages (from numba->nemo_toolkit[all]==1.5.1) (0.36.0)
    Requirement already satisfied: threadpoolctl>=2.0.0 in /opt/conda/lib/python3.8/site-packages (from scikit-learn->nemo_toolkit[all]==1.5.1) (3.1.0)
    Requirement already satisfied: defusedxml in /opt/conda/lib/python3.8/site-packages (from nbconvert->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets->nemo_toolkit[all]==1.5.1) (0.7.1)
    Requirement already satisfied: nbclient<0.6.0,>=0.5.0 in /opt/conda/lib/python3.8/site-packages (from nbconvert->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets->nemo_toolkit[all]==1.5.1) (0.5.13)
    Requirement already satisfied: bleach in /opt/conda/lib/python3.8/site-packages (from nbconvert->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets->nemo_toolkit[all]==1.5.1) (4.1.0)
    Requirement already satisfied: mistune<2,>=0.8.1 in /opt/conda/lib/python3.8/site-packages (from nbconvert->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets->nemo_toolkit[all]==1.5.1) (0.8.4)
    Requirement already satisfied: pandocfilters>=1.4.1 in /opt/conda/lib/python3.8/site-packages (from nbconvert->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets->nemo_toolkit[all]==1.5.1) (1.5.0)
    Requirement already satisfied: testpath in /opt/conda/lib/python3.8/site-packages (from nbconvert->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets->nemo_toolkit[all]==1.5.1) (0.6.0)
    Requirement already satisfied: jupyterlab-pygments in /opt/conda/lib/python3.8/site-packages (from nbconvert->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets->nemo_toolkit[all]==1.5.1) (0.1.2)
    Requirement already satisfied: webencodings in /opt/conda/lib/python3.8/site-packages (from bleach->nbconvert->notebook>=4.4.1->widgetsnbextension~=3.6.0->ipywidgets->nemo_toolkit[all]==1.5.1) (0.5.1)
    Requirement already satisfied: pytz>=2017.3 in /opt/conda/lib/python3.8/site-packages (from pandas->nemo_toolkit[all]==1.5.1) (2021.3)
    Requirement already satisfied: pip in /opt/conda/lib/python3.8/site-packages (from pip-api->isort[requirements]<5->nemo_toolkit[all]==1.5.1) (21.2.4)
    Requirement already satisfied: yarg in /opt/conda/lib/python3.8/site-packages (from pipreqs->isort[requirements]<5->nemo_toolkit[all]==1.5.1) (0.1.9)
    Requirement already satisfied: docopt in /opt/conda/lib/python3.8/site-packages (from pipreqs->isort[requirements]<5->nemo_toolkit[all]==1.5.1) (0.6.2)
    Requirement already satisfied: simplejson>=3.8.1 in /opt/conda/lib/python3.8/site-packages (from pyannote.core->nemo_toolkit[all]==1.5.1) (3.17.6)
    Requirement already satisfied: sortedcontainers>=2.0.4 in /opt/conda/lib/python3.8/site-packages (from pyannote.core->nemo_toolkit[all]==1.5.1) (2.4.0)
    Requirement already satisfied: tabulate>=0.7.7 in /opt/conda/lib/python3.8/site-packages (from pyannote.metrics->nemo_toolkit[all]==1.5.1) (0.8.9)
    Requirement already satisfied: pyannote.database>=4.0.1 in /opt/conda/lib/python3.8/site-packages (from pyannote.metrics->nemo_toolkit[all]==1.5.1) (4.1.3)
    Requirement already satisfied: sympy>=1.1 in /opt/conda/lib/python3.8/site-packages (from pyannote.metrics->nemo_toolkit[all]==1.5.1) (1.10.1)
    Requirement already satisfied: typer[all]>=0.2.1 in /opt/conda/lib/python3.8/site-packages (from pyannote.database>=4.0.1->pyannote.metrics->nemo_toolkit[all]==1.5.1) (0.4.0)
    Requirement already satisfied: mpmath>=0.19 in /opt/conda/lib/python3.8/site-packages (from sympy>=1.1->pyannote.metrics->nemo_toolkit[all]==1.5.1) (1.2.1)
    Requirement already satisfied: colorama<0.5.0,>=0.4.3 in /opt/conda/lib/python3.8/site-packages (from typer[all]>=0.2.1->pyannote.database>=4.0.1->pyannote.metrics->nemo_toolkit[all]==1.5.1) (0.4.4)
    Requirement already satisfied: shellingham<2.0.0,>=1.3.0 in /opt/conda/lib/python3.8/site-packages (from typer[all]>=0.2.1->pyannote.database>=4.0.1->pyannote.metrics->nemo_toolkit[all]==1.5.1) (1.4.0)
    Requirement already satisfied: py>=1.8.2 in /opt/conda/lib/python3.8/site-packages (from pytest->nemo_toolkit[all]==1.5.1) (1.11.0)
    Requirement already satisfied: iniconfig in /opt/conda/lib/python3.8/site-packages (from pytest->nemo_toolkit[all]==1.5.1) (1.1.1)
    Requirement already satisfied: pluggy<2.0,>=0.12 in /opt/conda/lib/python3.8/site-packages (from pytest->nemo_toolkit[all]==1.5.1) (1.0.0)
    Requirement already satisfied: jarowinkler<1.1.0,>=1.0.2 in /opt/conda/lib/python3.8/site-packages (from rapidfuzz->nemo_toolkit[all]==1.5.1) (1.0.2)
    Requirement already satisfied: PySocks!=1.5.7,>=1.5.6 in /opt/conda/lib/python3.8/site-packages (from requests->fsspec[http]!=2021.06.0,>=2021.05.0->pytorch-lightning>=1.5.0->nemo_toolkit[all]==1.5.1) (1.7.1)
    Requirement already satisfied: ruamel.yaml.clib>=0.2.6 in /opt/conda/lib/python3.8/site-packages (from ruamel.yaml->nemo_toolkit[all]==1.5.1) (0.2.6)
    Requirement already satisfied: portalocker in /opt/conda/lib/python3.8/site-packages (from sacrebleu[ja]->nemo_toolkit[all]==1.5.1) (2.4.0)
    Requirement already satisfied: ipadic<2.0,>=1.0 in /opt/conda/lib/python3.8/site-packages (from sacrebleu[ja]->nemo_toolkit[all]==1.5.1) (1.0.0)
    Requirement already satisfied: mecab-python3==1.0.3 in /opt/conda/lib/python3.8/site-packages (from sacrebleu[ja]->nemo_toolkit[all]==1.5.1) (1.0.3)
    Requirement already satisfied: sphinxcontrib-htmlhelp>=2.0.0 in /opt/conda/lib/python3.8/site-packages (from sphinx->nemo_toolkit[all]==1.5.1) (2.0.0)
    Requirement already satisfied: alabaster<0.8,>=0.7 in /opt/conda/lib/python3.8/site-packages (from sphinx->nemo_toolkit[all]==1.5.1) (0.7.12)
    Requirement already satisfied: babel>=1.3 in /opt/conda/lib/python3.8/site-packages (from sphinx->nemo_toolkit[all]==1.5.1) (2.9.1)
    Requirement already satisfied: sphinxcontrib-serializinghtml>=1.1.5 in /opt/conda/lib/python3.8/site-packages (from sphinx->nemo_toolkit[all]==1.5.1) (1.1.5)
    Requirement already satisfied: sphinxcontrib-devhelp in /opt/conda/lib/python3.8/site-packages (from sphinx->nemo_toolkit[all]==1.5.1) (1.0.2)
    Requirement already satisfied: sphinxcontrib-jsmath in /opt/conda/lib/python3.8/site-packages (from sphinx->nemo_toolkit[all]==1.5.1) (1.0.1)
    Requirement already satisfied: sphinxcontrib-qthelp in /opt/conda/lib/python3.8/site-packages (from sphinx->nemo_toolkit[all]==1.5.1) (1.0.3)
    Requirement already satisfied: snowballstemmer>=1.1 in /opt/conda/lib/python3.8/site-packages (from sphinx->nemo_toolkit[all]==1.5.1) (2.2.0)
    Requirement already satisfied: imagesize in /opt/conda/lib/python3.8/site-packages (from sphinx->nemo_toolkit[all]==1.5.1) (1.3.0)
    Requirement already satisfied: sphinxcontrib-applehelp in /opt/conda/lib/python3.8/site-packages (from sphinx->nemo_toolkit[all]==1.5.1) (1.0.2)
    Requirement already satisfied: docutils<0.18,>=0.14 in /opt/conda/lib/python3.8/site-packages (from sphinx->nemo_toolkit[all]==1.5.1) (0.17.1)
    Requirement already satisfied: pybtex-docutils>=1.0.0 in /opt/conda/lib/python3.8/site-packages (from sphinxcontrib-bibtex->nemo_toolkit[all]==1.5.1) (1.0.1)
    Requirement already satisfied: pybtex>=0.24 in /opt/conda/lib/python3.8/site-packages (from sphinxcontrib-bibtex->nemo_toolkit[all]==1.5.1) (0.24.0)
    Requirement already satisfied: latexcodec>=1.0.4 in /opt/conda/lib/python3.8/site-packages (from pybtex>=0.24->sphinxcontrib-bibtex->nemo_toolkit[all]==1.5.1) (2.0.1)
    Requirement already satisfied: pure-eval in /opt/conda/lib/python3.8/site-packages (from stack-data->ipython>=4.0.0->ipywidgets->nemo_toolkit[all]==1.5.1) (0.2.2)
    Requirement already satisfied: asttokens in /opt/conda/lib/python3.8/site-packages (from stack-data->ipython>=4.0.0->ipywidgets->nemo_toolkit[all]==1.5.1) (2.0.5)
    Requirement already satisfied: executing in /opt/conda/lib/python3.8/site-packages (from stack-data->ipython>=4.0.0->ipywidgets->nemo_toolkit[all]==1.5.1) (0.8.3)
    Requirement already satisfied: pathtools in /opt/conda/lib/python3.8/site-packages (from wandb->nemo_toolkit[all]==1.5.1) (0.1.2)
    Requirement already satisfied: setproctitle in /opt/conda/lib/python3.8/site-packages (from wandb->nemo_toolkit[all]==1.5.1) (1.2.3)
    Requirement already satisfied: GitPython>=1.0.0 in /opt/conda/lib/python3.8/site-packages (from wandb->nemo_toolkit[all]==1.5.1) (3.1.27)
    Requirement already satisfied: sentry-sdk>=1.0.0 in /opt/conda/lib/python3.8/site-packages (from wandb->nemo_toolkit[all]==1.5.1) (1.5.10)
    Requirement already satisfied: shortuuid>=0.5.0 in /opt/conda/lib/python3.8/site-packages (from wandb->nemo_toolkit[all]==1.5.1) (1.0.8)
    Requirement already satisfied: docker-pycreds>=0.4.0 in /opt/conda/lib/python3.8/site-packages (from wandb->nemo_toolkit[all]==1.5.1) (0.4.0)
    Requirement already satisfied: promise<3,>=2.0 in /opt/conda/lib/python3.8/site-packages (from wandb->nemo_toolkit[all]==1.5.1) (2.3)
    Requirement already satisfied: gitdb<5,>=4.0.1 in /opt/conda/lib/python3.8/site-packages (from GitPython>=1.0.0->wandb->nemo_toolkit[all]==1.5.1) (4.0.9)
    Requirement already satisfied: smmap<6,>=3.0.1 in /opt/conda/lib/python3.8/site-packages (from gitdb<5,>=4.0.1->GitPython>=1.0.0->wandb->nemo_toolkit[all]==1.5.1) (5.0.0)
    WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

## 2. Download Citrinet model

Next, we download a pretrained Nemo Citrinet model and convert it to a Torchscript module:

```python
import nemo
import torch

import nemo.collections.asr as nemo_asr
from nemo.core import typecheck
typecheck.set_typecheck_enabled(False)
```

```python
variant = 'stt_en_citrinet_256'

print(f"Downloading and saving {variant}...")
asr_model = nemo_asr.models.EncDecCTCModelBPE.from_pretrained(model_name=variant)
asr_model.export(f"{variant}.ts")
```

    Downloading and saving stt_en_citrinet_256...
    [NeMo I 2022-04-21 23:12:45 cloud:56] Found existing object /root/.cache/torch/NeMo/NeMo_1.5.1/stt_en_citrinet_256/91a9cc5850784b2065e8a0aa3d526fd9/stt_en_citrinet_256.nemo.
    [NeMo I 2022-04-21 23:12:45 cloud:62] Re-using file from: /root/.cache/torch/NeMo/NeMo_1.5.1/stt_en_citrinet_256/91a9cc5850784b2065e8a0aa3d526fd9/stt_en_citrinet_256.nemo
    [NeMo I 2022-04-21 23:12:45 common:728] Instantiating model from pre-trained checkpoint
    [NeMo I 2022-04-21 23:12:46 mixins:146] Tokenizer SentencePieceTokenizer initialized with 1024 tokens

    [NeMo W 2022-04-21 23:12:47 modelPT:130] If you intend to do training or fine-tuning, please call the ModelPT.setup_training_data() method and provide a valid configuration file to setup the train data loader.
        Train config :
        manifest_filepath: null
        sample_rate: 16000
        batch_size: 32
        trim_silence: true
        max_duration: 16.7
        shuffle: true
        is_tarred: false
        tarred_audio_filepaths: null
        use_start_end_token: false

    [NeMo W 2022-04-21 23:12:47 modelPT:137] If you intend to do validation, please call the ModelPT.setup_validation_data() or ModelPT.setup_multiple_validation_data() method and provide a valid configuration file to setup the validation data loader(s).
        Validation config :
        manifest_filepath: null
        sample_rate: 16000
        batch_size: 32
        shuffle: false
        use_start_end_token: false

    [NeMo W 2022-04-21 23:12:47 modelPT:143] Please call the ModelPT.setup_test_data() or ModelPT.setup_multiple_test_data() method and provide a valid configuration file to setup the test data loader(s).
        Test config :
        manifest_filepath: null
        sample_rate: 16000
        batch_size: 32
        shuffle: false
        use_start_end_token: false

    [NeMo I 2022-04-21 23:12:47 features:265] PADDING: 16
    [NeMo I 2022-04-21 23:12:47 features:282] STFT using torch

    [NeMo W 2022-04-21 23:12:47 nemo_logging:349] /opt/conda/lib/python3.8/site-packages/nemo/collections/asr/parts/preprocessing/features.py:315: FutureWarning: Pass sr=16000, n_fft=512 as keyword args. From version 0.10 passing these as positional arguments will result in an error
          librosa.filters.mel(sample_rate, self.n_fft, n_mels=nfilt, fmin=lowfreq, fmax=highfreq), dtype=torch.float

    [NeMo I 2022-04-21 23:12:49 save_restore_connector:149] Model EncDecCTCModelBPE was successfully restored from /root/.cache/torch/NeMo/NeMo_1.5.1/stt_en_citrinet_256/91a9cc5850784b2065e8a0aa3d526fd9/stt_en_citrinet_256.nemo.

    [NeMo W 2022-04-21 23:12:49 export_utils:198] Swapped 0 modules
    [NeMo W 2022-04-21 23:12:49 conv_asr:73] Turned off 235 masked convolutions
    [NeMo W 2022-04-21 23:12:49 export_utils:198] Swapped 0 modules
    [NeMo W 2022-04-21 23:12:50 nemo_logging:349] /opt/conda/lib/python3.8/site-packages/torch/jit/_trace.py:916: UserWarning: `optimize` is deprecated and has no effect. Use `with torch.jit.optimized_execution() instead
          warnings.warn(

    [NeMo W 2022-04-21 23:12:50 nemo_logging:349] /opt/conda/lib/python3.8/site-packages/torch/_jit_internal.py:668: LightningDeprecationWarning: The `LightningModule.model_size` property was deprecated in v1.5 and will be removed in v1.7. Please use the `pytorch_lightning.utilities.memory.get_model_size_mb`.
          if hasattr(mod, name):

    [NeMo W 2022-04-21 23:12:50 nemo_logging:349] /opt/conda/lib/python3.8/site-packages/torch/_jit_internal.py:669: LightningDeprecationWarning: The `LightningModule.model_size` property was deprecated in v1.5 and will be removed in v1.7. Please use the `pytorch_lightning.utilities.memory.get_model_size_mb`.
          item = getattr(mod, name)

    [NeMo W 2022-04-21 23:12:50 nemo_logging:349] /opt/conda/lib/python3.8/site-packages/torch/_jit_internal.py:668: LightningDeprecationWarning: `LightningModule.use_amp` was deprecated in v1.6 and will be removed in v1.8. Please use `Trainer.amp_backend`.
          if hasattr(mod, name):

    [NeMo W 2022-04-21 23:12:50 nemo_logging:349] /opt/conda/lib/python3.8/site-packages/torch/_jit_internal.py:669: LightningDeprecationWarning: `LightningModule.use_amp` was deprecated in v1.6 and will be removed in v1.8. Please use `Trainer.amp_backend`.
          item = getattr(mod, name)

    (['stt_en_citrinet_256.ts'],
     ['nemo.collections.asr.models.ctc_bpe_models.EncDecCTCModelBPE exported to ONNX'])

### Benchmark utility

Let us define a helper benchmarking function, then benchmark the original Pytorch model.

```python
from __future__ import print_function
from __future__ import absolute_import
from __future__ import division

import argparse
import timeit
import numpy as np
import torch
import torch_tensorrt as trtorch
import torch.backends.cudnn as cudnn

def benchmark(model, input_tensor, num_loops, model_name, batch_size):
    def timeGraph(model, input_tensor, num_loops):
        print("Warm up ...")
        with torch.no_grad():
            for _ in range(20):
                features = model(input_tensor)

        torch.cuda.synchronize()
        print("Start timing ...")
        timings = []
        with torch.no_grad():
            for i in range(num_loops):
                start_time = timeit.default_timer()
                features = model(input_tensor)
                torch.cuda.synchronize()
                end_time = timeit.default_timer()
                timings.append(end_time - start_time)
                # print("Iteration {}: {:.6f} s".format(i, end_time - start_time))
        return timings
    def printStats(graphName, timings, batch_size):
        times = np.array(timings)
        steps = len(times)
        speeds = batch_size / times
        time_mean = np.mean(times)
        time_med = np.median(times)
        time_99th = np.percentile(times, 99)
        time_std = np.std(times, ddof=0)
        speed_mean = np.mean(speeds)
        speed_med = np.median(speeds)
        msg = ("\n%s =================================\n"
                "batch size=%d, num iterations=%d\n"
                "  Median samples/s: %.1f, mean: %.1f\n"
                "  Median latency (s): %.6f, mean: %.6f, 99th_p: %.6f, std_dev: %.6f\n"
                ) % (graphName,
                    batch_size, steps,
                    speed_med, speed_mean,
                    time_med, time_mean, time_99th, time_std)
        print(msg)
    timings = timeGraph(model, input_tensor, num_loops)
    printStats(model_name, timings, batch_size)

precisions_str = 'fp32' # Precision (default=fp32, fp16)
variant = 'stt_en_citrinet_256' # Nemo Citrinet variant
batch_sizes = [1, 8, 32, 128] # Batch sizes (default=1,8,32,128)
trt = False # If True, infer with Torch-TensorRT engine. Else, infer with Pytorch model.
precision = torch.float32 if precisions_str =='fp32' else torch.float16

for batch_size in batch_sizes:
    if trt:
        model_name = f"{variant}_bs{batch_size}_{precision}.torch-tensorrt"
    else:
        model_name = f"{variant}.ts"

    print(f"Loading model: {model_name}")
    # Load traced model to CPU first
    model = torch.jit.load(model_name).cuda()
    cudnn.benchmark = True
    # Create random input tensor of certain size
    torch.manual_seed(12345)
    input_shape=(batch_size, 80, 1488)
    input_tensor = torch.randn(input_shape).cuda()

    # Timing graph inference
    benchmark(model, input_tensor, 50, model_name, batch_size)
```

    Loading model: stt_en_citrinet_256.ts
    Warm up ...
    Start timing ...

    stt_en_citrinet_256.ts =================================
    batch size=1, num iterations=50
      Median samples/s: 102.0, mean: 102.0
      Median latency (s): 0.009802, mean: 0.009803, 99th_p: 0.009836, std_dev: 0.000014

    Loading model: stt_en_citrinet_256.ts
    Warm up ...
    Start timing ...

    stt_en_citrinet_256.ts =================================
    batch size=8, num iterations=50
      Median samples/s: 429.1, mean: 429.1
      Median latency (s): 0.018642, mean: 0.018643, 99th_p: 0.018670, std_dev: 0.000014

    Loading model: stt_en_citrinet_256.ts
    Warm up ...
    Start timing ...

    stt_en_citrinet_256.ts =================================
    batch size=32, num iterations=50
      Median samples/s: 551.3, mean: 551.2
      Median latency (s): 0.058047, mean: 0.058053, 99th_p: 0.058375, std_dev: 0.000106

    Loading model: stt_en_citrinet_256.ts
    Warm up ...
    Start timing ...

    stt_en_citrinet_256.ts =================================
    batch size=128, num iterations=50
      Median samples/s: 594.1, mean: 594.1
      Median latency (s): 0.215434, mean: 0.215446, 99th_p: 0.215806, std_dev: 0.000116

Confirming the GPU we are using here:

```python
!nvidia-smi
```

    Thu Apr 21 23:13:32 2022
    +-----------------------------------------------------------------------------+
    | NVIDIA-SMI 510.47.03    Driver Version: 510.47.03    CUDA Version: 11.6     |
    |-------------------------------+----------------------+----------------------+
    | GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
    | Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
    |                               |                      |               MIG M. |
    |===============================+======================+======================|
    |   0  NVIDIA TITAN V      On   | 00000000:17:00.0 Off |                  N/A |
    | 38%   55C    P2    42W / 250W |   2462MiB / 12288MiB |      0%      Default |
    |                               |                      |                  N/A |
    +-------------------------------+----------------------+----------------------+
    |   1  NVIDIA TITAN V      On   | 00000000:65:00.0 Off |                  N/A |
    | 28%   39C    P8    26W / 250W |    112MiB / 12288MiB |      0%      Default |
    |                               |                      |                  N/A |
    +-------------------------------+----------------------+----------------------+

    +-----------------------------------------------------------------------------+
    | Processes:                                                                  |
    |  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
    |        ID   ID                                                   Usage      |
    |=============================================================================|
    |    0   N/A  N/A      3909      G                                       4MiB |
    |    0   N/A  N/A      6047      C                                    2453MiB |
    |    1   N/A  N/A      3909      G                                      39MiB |
    |    1   N/A  N/A      4161      G                                      67MiB |
    +-----------------------------------------------------------------------------+

## 3. Create Torch-TensorRT modules

In this step, we optimize the Citrinet Torchscript module with Torch-TensorRT with various precisions and batch sizes.

```python
import torch
import torch.nn as nn
import torch_tensorrt as torchtrt
import argparse

variant = "stt_en_citrinet_256"
precisions = [torch.float, torch.half]
batch_sizes = [1,8,32,128]

model = torch.jit.load(f"{variant}.ts")

for precision in precisions:
    for batch_size in batch_sizes:
        compile_settings = {
            "inputs": [torchtrt.Input(shape=[batch_size, 80, 1488])],
            "enabled_precisions": {precision},
            "workspace_size": 2000000000,
            "truncate_long_and_double": True,
        }
        print(f"Generating Torchscript-TensorRT module for batchsize {batch_size} precision {precision}")
        trt_ts_module = torchtrt.compile(model, **compile_settings)
        torch.jit.save(trt_ts_module, f"{variant}_bs{batch_size}_{precision}.torch-tensorrt")
```

    Generating Torchscript-TensorRT module for batchsize 1 precision torch.float32
    Generating Torchscript-TensorRT module for batchsize 8 precision torch.float32
    Generating Torchscript-TensorRT module for batchsize 32 precision torch.float32
    Generating Torchscript-TensorRT module for batchsize 128 precision torch.float32
    Generating Torchscript-TensorRT module for batchsize 1 precision torch.float16
    Generating Torchscript-TensorRT module for batchsize 8 precision torch.float16
    Generating Torchscript-TensorRT module for batchsize 32 precision torch.float16
    Generating Torchscript-TensorRT module for batchsize 128 precision torch.float16

## 4. Benchmark Torch-TensorRT models

Finally, we are ready to benchmark the Torch-TensorRT optimized Citrinet models.

### FP32 (single precision)

```python
precisions_str = 'fp32' # Precision (default=fp32, fp16)
batch_sizes = [1, 8, 32, 128] # Batch sizes (default=1,8,32,128)
precision = torch.float32 if precisions_str =='fp32' else torch.float16
trt = True

for batch_size in batch_sizes:
    if trt:
        model_name = f"{variant}_bs{batch_size}_{precision}.torch-tensorrt"
    else:
        model_name = f"{variant}.ts"

    print(f"Loading model: {model_name}")
    # Load traced model to CPU first
    model = torch.jit.load(model_name).cuda()
    cudnn.benchmark = True
    # Create random input tensor of certain size
    torch.manual_seed(12345)
    input_shape=(batch_size, 80, 1488)
    input_tensor = torch.randn(input_shape).cuda()

    # Timing graph inference
    benchmark(model, input_tensor, 50, model_name, batch_size)
```

    Loading model: stt_en_citrinet_256_bs1_torch.float32.torch-tensorrt
    Warm up ...
    Start timing ...

    stt_en_citrinet_256_bs1_torch.float32.torch-tensorrt =================================
    batch size=1, num iterations=50
      Median samples/s: 242.2, mean: 218.0
      Median latency (s): 0.004128, mean: 0.004825, 99th_p: 0.008071, std_dev: 0.001270

    Loading model: stt_en_citrinet_256_bs8_torch.float32.torch-tensorrt
    Warm up ...
    Start timing ...

    stt_en_citrinet_256_bs8_torch.float32.torch-tensorrt =================================
    batch size=8, num iterations=50
      Median samples/s: 729.9, mean: 709.0
      Median latency (s): 0.010961, mean: 0.011388, 99th_p: 0.016114, std_dev: 0.001256

    Loading model: stt_en_citrinet_256_bs32_torch.float32.torch-tensorrt
    Warm up ...
    Start timing ...

    stt_en_citrinet_256_bs32_torch.float32.torch-tensorrt =================================
    batch size=32, num iterations=50
      Median samples/s: 955.6, mean: 953.4
      Median latency (s): 0.033488, mean: 0.033572, 99th_p: 0.035722, std_dev: 0.000545

    Loading model: stt_en_citrinet_256_bs128_torch.float32.torch-tensorrt
    Warm up ...
    Start timing ...

    stt_en_citrinet_256_bs128_torch.float32.torch-tensorrt =================================
    batch size=128, num iterations=50
      Median samples/s: 1065.8, mean: 1069.4
      Median latency (s): 0.120097, mean: 0.119708, 99th_p: 0.121618, std_dev: 0.001260

### FP16 (half precision)

```python
precisions_str = 'fp16' # Precision (default=fp32, fp16)
batch_sizes = [1, 8, 32, 128] # Batch sizes (default=1,8,32,128)
precision = torch.float32 if precisions_str =='fp32' else torch.float16

for batch_size in batch_sizes:
    if trt:
        model_name = f"{variant}_bs{batch_size}_{precision}.torch-tensorrt"
    else:
        model_name = f"{variant}.ts"

    print(f"Loading model: {model_name}")
    # Load traced model to CPU first
    model = torch.jit.load(model_name).cuda()
    cudnn.benchmark = True
    # Create random input tensor of certain size
    torch.manual_seed(12345)
    input_shape=(batch_size, 80, 1488)
    input_tensor = torch.randn(input_shape).cuda()

    # Timing graph inference
    benchmark(model, input_tensor, 50, model_name, batch_size)
```

    Loading model: stt_en_citrinet_256_bs1_torch.float16.torch-tensorrt
    Warm up ...
    Start timing ...

    stt_en_citrinet_256_bs1_torch.float16.torch-tensorrt =================================
    batch size=1, num iterations=50
      Median samples/s: 288.9, mean: 272.9
      Median latency (s): 0.003462, mean: 0.003774, 99th_p: 0.006846, std_dev: 0.000820

    Loading model: stt_en_citrinet_256_bs8_torch.float16.torch-tensorrt
    Warm up ...
    Start timing ...

    stt_en_citrinet_256_bs8_torch.float16.torch-tensorrt =================================
    batch size=8, num iterations=50
      Median samples/s: 1201.0, mean: 1190.9
      Median latency (s): 0.006661, mean: 0.006733, 99th_p: 0.008453, std_dev: 0.000368

    Loading model: stt_en_citrinet_256_bs32_torch.float16.torch-tensorrt
    Warm up ...
    Start timing ...

    stt_en_citrinet_256_bs32_torch.float16.torch-tensorrt =================================
    batch size=32, num iterations=50
      Median samples/s: 1538.2, mean: 1516.4
      Median latency (s): 0.020804, mean: 0.021143, 99th_p: 0.024492, std_dev: 0.000973

    Loading model: stt_en_citrinet_256_bs128_torch.float16.torch-tensorrt
    Warm up ...
    Start timing ...

    stt_en_citrinet_256_bs128_torch.float16.torch-tensorrt =================================
    batch size=128, num iterations=50
      Median samples/s: 1792.0, mean: 1777.0
      Median latency (s): 0.071428, mean: 0.072057, 99th_p: 0.076796, std_dev: 0.001351

## 5. Conclusion

In this notebook, we have walked through the complete process of optimizing the Citrinet model with Torch-TensorRT. On an A100 GPU, with Torch-TensorRT, we observe a speedup of ~**2.4X** with FP32, and ~**2.9X** with FP16 at batchsize of 128.

### What's next
Now it's time to try Torch-TensorRT on your own model. Fill out issues at https://github.com/NVIDIA/Torch-TensorRT. Your involvement will help future development of Torch-TensorRT.

```python

```