```python
!pip install pyannote.metrics
!pip install torchaudio
```

    Collecting pyannote.metrics
      Downloading https://files.pythonhosted.org/packages/76/44/39e79594474eb99a23b26cb0104d78f6f20e4f8de3a26d67f24d2ea9ef28/pyannote.metrics-3.0.1-py3-none-any.whl (49kB)
         |████████████████████████████████| 51kB 2.7MB/s
    Collecting pyannote.core>=4.1
      Downloading https://files.pythonhosted.org/packages/ba/09/64e64cedccf764be513405d1ff3d3d2bbcbea60d18fb6a6d9e815aedcd79/pyannote.core-4.1-py3-none-any.whl (56kB)
         |████████████████████████████████| 61kB 4.1MB/s
    Requirement already satisfied: pandas>=0.19 in /usr/local/lib/python3.7/dist-packages (from pyannote.metrics) (1.1.5)
    Requirement already satisfied: scipy>=1.1.0 in /usr/local/lib/python3.7/dist-packages (from pyannote.metrics) (1.4.1)
    Requirement already satisfied: matplotlib>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from pyannote.metrics) (3.2.2)
    Requirement already satisfied: sympy>=1.1 in /usr/local/lib/python3.7/dist-packages (from pyannote.metrics) (1.7.1)
    Requirement already satisfied: scikit-learn>=0.17.1 in /usr/local/lib/python3.7/dist-packages (from pyannote.metrics) (0.22.2.post1)
    Requirement already satisfied: docopt>=0.6.2 in /usr/local/lib/python3.7/dist-packages (from pyannote.metrics) (0.6.2)
    Requirement already satisfied: tabulate>=0.7.7 in /usr/local/lib/python3.7/dist-packages (from pyannote.metrics) (0.8.9)
    Collecting pyannote.database>=4.0.1
      Downloading https://files.pythonhosted.org/packages/f8/cb/82d8035dbc3dc9aac80b5475fcac63125fc3669e4d8d07261ee3d8507f57/pyannote.database-4.1-py3-none-any.whl (41kB)
         |████████████████████████████████| 51kB 4.4MB/s
    Requirement already satisfied: numpy>=1.10.4 in /usr/local/lib/python3.7/dist-packages (from pyannote.core>=4.1->pyannote.metrics) (1.19.5)
    Requirement already satisfied: sortedcontainers>=2.0.4 in /usr/local/lib/python3.7/dist-packages (from pyannote.core>=4.1->pyannote.metrics) (2.3.0)
    Collecting simplejson>=3.8.1
      Downloading https://files.pythonhosted.org/packages/a8/04/377418ac1e530ce2a196b54c6552c018fdf1fe776718053efb1f216bffcd/simplejson-3.17.2-cp37-cp37m-manylinux2010_x86_64.whl (128kB)
         |████████████████████████████████| 133kB 8.7MB/s
    Requirement already satisfied: typing-extensions>=3.7.4.1 in /usr/local/lib/python3.7/dist-packages (from pyannote.core>=4.1->pyannote.metrics) (3.7.4.3)
    Requirement already satisfied: pytz>=2017.2 in /usr/local/lib/python3.7/dist-packages (from pandas>=0.19->pyannote.metrics) (2018.9)
    Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.7/dist-packages (from pandas>=0.19->pyannote.metrics) (2.8.1)
    Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib>=2.0.0->pyannote.metrics) (1.3.1)
    Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.7/dist-packages (from matplotlib>=2.0.0->pyannote.metrics) (0.10.0)
    Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib>=2.0.0->pyannote.metrics) (2.4.7)
    Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.7/dist-packages (from sympy>=1.1->pyannote.metrics) (1.2.1)
    Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.7/dist-packages (from scikit-learn>=0.17.1->pyannote.metrics) (1.0.1)
    Requirement already satisfied: pyYAML>=3.12 in /usr/local/lib/python3.7/dist-packages (from pyannote.database>=4.0.1->pyannote.metrics) (3.13)
    Collecting typer[all]>=0.2.1
      Downloading https://files.pythonhosted.org/packages/90/34/d138832f6945432c638f32137e6c79a3b682f06a63c488dcfaca6b166c64/typer-0.3.2-py3-none-any.whl
    Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/dist-packages (from python-dateutil>=2.7.3->pandas>=0.19->pyannote.metrics) (1.15.0)
    Requirement already satisfied: click<7.2.0,>=7.1.1 in /usr/local/lib/python3.7/dist-packages (from typer[all]>=0.2.1->pyannote.database>=4.0.1->pyannote.metrics) (7.1.2)
    Collecting colorama<0.5.0,>=0.4.3; extra == "all"
      Downloading https://files.pythonhosted.org/packages/44/98/5b86278fbbf250d239ae0ecb724f8572af1c91f4a11edf4d36a206189440/colorama-0.4.4-py2.py3-none-any.whl
    Collecting shellingham<2.0.0,>=1.3.0; extra == "all"
      Downloading https://files.pythonhosted.org/packages/76/94/7a764d57d0f46534e0022e651da6547bc5cfe7b6372e7e0ed1dde6f5cb67/shellingham-1.4.0-py2.py3-none-any.whl
    Installing collected packages: simplejson, pyannote.core, colorama, shellingham, typer, pyannote.database, pyannote.metrics
    Successfully installed colorama-0.4.4 pyannote.core-4.1 pyannote.database-4.1 pyannote.metrics-3.0.1 shellingham-1.4.0 simplejson-3.17.2 typer-0.3.2
    Collecting torchaudio
      Downloading https://files.pythonhosted.org/packages/aa/55/01ad9244bcd595e39cea5ce30726a7fe02fd963d07daeb136bfe7e23f0a5/torchaudio-0.8.1-cp37-cp37m-manylinux1_x86_64.whl (1.9MB)
         |████████████████████████████████| 1.9MB 5.1MB/s
    Requirement already satisfied: torch==1.8.1 in /usr/local/lib/python3.7/dist-packages (from torchaudio) (1.8.1+cu101)
    Requirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from torch==1.8.1->torchaudio) (1.19.5)
    Requirement already satisfied: typing-extensions in /usr/local/lib/python3.7/dist-packages (from torch==1.8.1->torchaudio) (3.7.4.3)
    Installing collected packages: torchaudio
    Successfully installed torchaudio-0.8.1

```python
# load_data.py
import numpy as np
import pandas as pd
import IPython.display as ipd
import soundfile as sf

%load_ext autoreload
%autoreload 2

# ! git clone https://github.com/yistLin/dvector.git
%cd dvector

from google.colab import drive
drive.mount('/content/drive')
DATA_BASE_DIR = "/content/drive/MyDrive"
```

    [Errno 2] No such file or directory: 'dvector'
    /content
    Mounted at /content/drive

```python
import glob
from scipy.io import wavfile
from pyannote.database.util import load_rttm

class DataLoader():
    """docstring for DataLoader"""
    def __init__(self, audio_folder, labels_folder, names_only=False):
        # Audio files are assumed to have .wav extension,
        # Label files are assumed to have .rttm extension
        self.audio_files = sorted(glob.glob(audio_folder + "/*.wav"))
        self.label_files = sorted(glob.glob(labels_folder + "/*.rttm"))
        self.names_only = names_only

        assert len(self.audio_files) == len(self.label_files)

    def __len__(self):
        return len(self.label_files)

    def __getitem__(self, idx):
        # Extract labels from rttm file
        label_dict = load_rttm(self.label_files[idx])
        label = list(label_dict.values())
        label = label[0]

        if self.names_only:
          return (self.audio_files[idx], label)

        # Extract audio and sample rate from wav file
        samplerate, wav_data = wavfile.read(self.audio_files[idx])

        return ((samplerate, wav_data), label)

file_loader = DataLoader(audio_folder = f"{DATA_BASE_DIR}/voxconverse_dev_wav", labels_folder = f"{DATA_BASE_DIR}/voxconverse_dev_labels/dev", names_only=True)
```

## Trying to split

```python
## Mel-filterbank
mel_window_length = 25  # In milliseconds
mel_window_step = 10    # In milliseconds
mel_n_channels = 40

## Audio
sampling_rate = 16000
# Number of spectrogram frames in a partial utterance
partials_n_frames = 40     # 400 ms
```

```python
def compute_partial_slices(n_samples: int, rate, min_coverage):
        """
        Computes where to split an utterance waveform and its corresponding mel spectrogram to
        obtain partial utterances of <partials_n_frames> each. Both the waveform and the
        mel spectrogram slices are returned, so as to make each partial utterance waveform
        correspond to its spectrogram.
        The returned ranges may be indexing further than the length of the waveform. It is
        recommended that you pad the waveform with zeros up to wav_slices[-1].stop.
        :param n_samples: the number of samples in the waveform
        :param rate: how many partial utterances should occur per second. Partial utterances must
        cover the span of the entire utterance, thus the rate should not be lower than the inverse
        of the duration of a partial utterance. By default, partial utterances are 1.6s long and
        the minimum rate is thus 0.625.
        :param min_coverage: when reaching the last partial utterance, it may or may not have
        enough frames. If at least <min_pad_coverage> of <partials_n_frames> are present,
        then the last partial utterance will be considered by zero-padding the audio. Otherwise,
        it will be discarded. If there aren't enough frames for one partial utterance,
        this parameter is ignored so that the function always returns at least one slice.
        :return: the waveform slices and mel spectrogram slices as lists of array slices. Index
        respectively the waveform and the mel spectrogram with these slices to obtain the partial
        utterances.
        """
        assert 0 < min_coverage <= 1

        # Compute how many frames separate two partial utterances
        samples_per_frame = int((sampling_rate * mel_window_step / 1000))
        n_frames = int(np.ceil((n_samples + 1) / samples_per_frame))
        frame_step = int(np.round((sampling_rate / rate) / samples_per_frame))
        assert 0 < frame_step, "The rate is too high"
        assert frame_step <= partials_n_frames, "The rate is too low, it should be %f at least" % \
            (sampling_rate / (samples_per_frame * partials_n_frames))

        # Compute the slices
        wav_slices, mel_slices = [], []
        steps = max(1, n_frames - partials_n_frames + frame_step + 1)
        for i in range(0, steps, frame_step):
            mel_range = np.array([i, i + partials_n_frames])
            wav_range = mel_range * samples_per_frame
            mel_slices.append(slice(*mel_range))
            wav_slices.append(slice(*wav_range))

        # Evaluate whether extra padding is warranted or not
        last_wav_range = wav_slices[-1]
        coverage = (n_samples - last_wav_range.start) / (last_wav_range.stop - last_wav_range.start)
        if coverage < min_coverage and len(mel_slices) > 1:
            mel_slices = mel_slices[:-1]
            wav_slices = wav_slices[:-1]

        return wav_slices, mel_slices
```

```python
import torch
import torchaudio

wav2mel = torch.jit.load("wav2mel.pt")
dvector = torch.jit.load("dvector.pt").eval()

wav_file_name, reference = file_loader[1]
wav_tensor, sample_rate = torchaudio.load(wav_file_name)
mel_tensor = wav2mel(wav_tensor, sample_rate)  # shape: (frames, mel_dim)
emb_tensor = dvector.embed_utterance(mel_tensor)  # shape: (emb_dim)
```

```python
print(mel_tensor.shape)
print(emb_tensor.shape)
```

    torch.Size([10201, 40])
    torch.Size([256])

## Loading model

```python
% cd /content
```

    /content

```python
import torch
import torchaudio

# from data.wav2mel import Wav2Mel
torchaudio.set_audio_backend("sox_io")

# wav2mel = Wav2Mel()
# wav2mel = torch.jit.script(wav2mel)
# wav2mel.save("wav2mel.pt")
# wav2mel = torch.jit.load("log_melspectrogram.pt")
dvector = torch.jit.load("dvector.pt").eval()

# audio_file_name, reference = file_loader[1]

# wav_tensor, sample_rate = torchaudio.load(audio_file_name)
# mel_tensor = wav2mel(wav_tensor, sample_rate)  # shape: (frames, mel_dim)
# emb_tensor = dvector.embed_utterance(mel_tensor)  # shape: (emb_dim)a
# emb_tensor.shape
```

```python
# wav_tensor.shape, mel_tensor.shape, emb_tensor.shape
```

```python
import os
if os.path.exists("/content/Speaker-Diarization-System"):
   % cd /content/Speaker-Diarization-System
   ! git pull
else:
  % cd /content
  ! git clone https://gitlab.com/vaithak/Speaker-Diarization-System.git
  % cd /content/Speaker-Diarization-System

!pip install -r requirements.txt
```

    /content/Speaker-Diarization-System
    Already up to date.
    Requirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from -r requirements.txt (line 1)) (1.19.5)
    Requirement already satisfied: matplotlib in /usr/local/lib/python3.7/dist-packages (from -r requirements.txt (line 2)) (3.2.2)
    Requirement already satisfied: pandas in /usr/local/lib/python3.7/dist-packages (from -r requirements.txt (line 3)) (1.1.5)
    Requirement already satisfied: scipy in /usr/local/lib/python3.7/dist-packages (from -r requirements.txt (line 4)) (1.4.1)
    Requirement already satisfied: pyannote.metrics in /usr/local/lib/python3.7/dist-packages (from -r requirements.txt (line 5)) (3.0.1)
    Collecting resemblyzer
      Downloading https://files.pythonhosted.org/packages/e0/21/f0a22ee4afd9e5d9790b04329accdb71d2cf89ffaf5bb0611fb37cd91782/Resemblyzer-0.1.1.dev0-py3-none-any.whl (15.7MB)
         |████████████████████████████████| 15.7MB 254kB/s
    Collecting spectralcluster
      Downloading https://files.pythonhosted.org/packages/24/a7/12c567ed32165ccaa0a40bd55384958751a2049891c109fb9a1d3996e7a0/spectralcluster-0.1.0-py3-none-any.whl
    Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->-r requirements.txt (line 2)) (1.3.1)
    Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->-r requirements.txt (line 2)) (2.4.7)
    Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->-r requirements.txt (line 2)) (2.8.1)
    Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.7/dist-packages (from matplotlib->-r requirements.txt (line 2)) (0.10.0)
    Requirement already satisfied: pytz>=2017.2 in /usr/local/lib/python3.7/dist-packages (from pandas->-r requirements.txt (line 3)) (2018.9)
    Requirement already satisfied: sympy>=1.1 in /usr/local/lib/python3.7/dist-packages (from pyannote.metrics->-r requirements.txt (line 5)) (1.7.1)
    Requirement already satisfied: tabulate>=0.7.7 in /usr/local/lib/python3.7/dist-packages (from pyannote.metrics->-r requirements.txt (line 5)) (0.8.9)
    Requirement already satisfied: pyannote.database>=4.0.1 in /usr/local/lib/python3.7/dist-packages (from pyannote.metrics->-r requirements.txt (line 5)) (4.1)
    Requirement already satisfied: docopt>=0.6.2 in /usr/local/lib/python3.7/dist-packages (from pyannote.metrics->-r requirements.txt (line 5)) (0.6.2)
    Requirement already satisfied: scikit-learn>=0.17.1 in /usr/local/lib/python3.7/dist-packages (from pyannote.metrics->-r requirements.txt (line 5)) (0.22.2.post1)
    Requirement already satisfied: pyannote.core>=4.1 in /usr/local/lib/python3.7/dist-packages (from pyannote.metrics->-r requirements.txt (line 5)) (4.1)
    Requirement already satisfied: torch>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from resemblyzer->-r requirements.txt (line 6)) (1.8.1+cu101)
    Collecting typing
      Downloading https://files.pythonhosted.org/packages/05/d9/6eebe19d46bd05360c9a9aae822e67a80f9242aabbfc58b641b957546607/typing-3.7.4.3.tar.gz (78kB)
         |████████████████████████████████| 81kB 8.2MB/s
    Requirement already satisfied: librosa>=0.6.1 in /usr/local/lib/python3.7/dist-packages (from resemblyzer->-r requirements.txt (line 6)) (0.8.0)
    Collecting webrtcvad>=2.0.10
      Downloading https://files.pythonhosted.org/packages/89/34/e2de2d97f3288512b9ea56f92e7452f8207eb5a0096500badf9dfd48f5e6/webrtcvad-2.0.10.tar.gz (66kB)
         |████████████████████████████████| 71kB 6.8MB/s
    Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/dist-packages (from python-dateutil>=2.1->matplotlib->-r requirements.txt (line 2)) (1.15.0)
    Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.7/dist-packages (from sympy>=1.1->pyannote.metrics->-r requirements.txt (line 5)) (1.2.1)
    Requirement already satisfied: typing-extensions>=3.7.4; python_version < "3.8" in /usr/local/lib/python3.7/dist-packages (from pyannote.database>=4.0.1->pyannote.metrics->-r requirements.txt (line 5)) (3.7.4.3)
    Requirement already satisfied: typer[all]>=0.2.1 in /usr/local/lib/python3.7/dist-packages (from pyannote.database>=4.0.1->pyannote.metrics->-r requirements.txt (line 5)) (0.3.2)
    Requirement already satisfied: pyYAML>=3.12 in /usr/local/lib/python3.7/dist-packages (from pyannote.database>=4.0.1->pyannote.metrics->-r requirements.txt (line 5)) (3.13)
    Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.7/dist-packages (from scikit-learn>=0.17.1->pyannote.metrics->-r requirements.txt (line 5)) (1.0.1)
    Requirement already satisfied: simplejson>=3.8.1 in /usr/local/lib/python3.7/dist-packages (from pyannote.core>=4.1->pyannote.metrics->-r requirements.txt (line 5)) (3.17.2)
    Requirement already satisfied: sortedcontainers>=2.0.4 in /usr/local/lib/python3.7/dist-packages (from pyannote.core>=4.1->pyannote.metrics->-r requirements.txt (line 5)) (2.3.0)
    Requirement already satisfied: audioread>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from librosa>=0.6.1->resemblyzer->-r requirements.txt (line 6)) (2.1.9)
    Requirement already satisfied: soundfile>=0.9.0 in /usr/local/lib/python3.7/dist-packages (from librosa>=0.6.1->resemblyzer->-r requirements.txt (line 6)) (0.10.3.post1)
    Requirement already satisfied: decorator>=3.0.0 in /usr/local/lib/python3.7/dist-packages (from librosa>=0.6.1->resemblyzer->-r requirements.txt (line 6)) (4.4.2)
    Requirement already satisfied: resampy>=0.2.2 in /usr/local/lib/python3.7/dist-packages (from librosa>=0.6.1->resemblyzer->-r requirements.txt (line 6)) (0.2.2)
    Requirement already satisfied: pooch>=1.0 in /usr/local/lib/python3.7/dist-packages (from librosa>=0.6.1->resemblyzer->-r requirements.txt (line 6)) (1.3.0)
    Requirement already satisfied: numba>=0.43.0 in /usr/local/lib/python3.7/dist-packages (from librosa>=0.6.1->resemblyzer->-r requirements.txt (line 6)) (0.51.2)
    Requirement already satisfied: click<7.2.0,>=7.1.1 in /usr/local/lib/python3.7/dist-packages (from typer[all]>=0.2.1->pyannote.database>=4.0.1->pyannote.metrics->-r requirements.txt (line 5)) (7.1.2)
    Requirement already satisfied: colorama<0.5.0,>=0.4.3; extra == "all" in /usr/local/lib/python3.7/dist-packages (from typer[all]>=0.2.1->pyannote.database>=4.0.1->pyannote.metrics->-r requirements.txt (line 5)) (0.4.4)
    Requirement already satisfied: shellingham<2.0.0,>=1.3.0; extra == "all" in /usr/local/lib/python3.7/dist-packages (from typer[all]>=0.2.1->pyannote.database>=4.0.1->pyannote.metrics->-r requirements.txt (line 5)) (1.4.0)
    Requirement already satisfied: cffi>=1.0 in /usr/local/lib/python3.7/dist-packages (from soundfile>=0.9.0->librosa>=0.6.1->resemblyzer->-r requirements.txt (line 6)) (1.14.5)
    Requirement already satisfied: appdirs in /usr/local/lib/python3.7/dist-packages (from pooch>=1.0->librosa>=0.6.1->resemblyzer->-r requirements.txt (line 6)) (1.4.4)
    Requirement already satisfied: packaging in /usr/local/lib/python3.7/dist-packages (from pooch>=1.0->librosa>=0.6.1->resemblyzer->-r requirements.txt (line 6)) (20.9)
    Requirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from pooch>=1.0->librosa>=0.6.1->resemblyzer->-r requirements.txt (line 6)) (2.23.0)
    Requirement already satisfied: setuptools in /usr/local/lib/python3.7/dist-packages (from numba>=0.43.0->librosa>=0.6.1->resemblyzer->-r requirements.txt (line 6)) (56.1.0)
    Requirement already satisfied: llvmlite<0.35,>=0.34.0.dev0 in /usr/local/lib/python3.7/dist-packages (from numba>=0.43.0->librosa>=0.6.1->resemblyzer->-r requirements.txt (line 6)) (0.34.0)
    Requirement already satisfied: pycparser in /usr/local/lib/python3.7/dist-packages (from cffi>=1.0->soundfile>=0.9.0->librosa>=0.6.1->resemblyzer->-r requirements.txt (line 6)) (2.20)
    Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests->pooch>=1.0->librosa>=0.6.1->resemblyzer->-r requirements.txt (line 6)) (1.24.3)
    Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->pooch>=1.0->librosa>=0.6.1->resemblyzer->-r requirements.txt (line 6)) (2.10)
    Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests->pooch>=1.0->librosa>=0.6.1->resemblyzer->-r requirements.txt (line 6)) (2020.12.5)
    Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests->pooch>=1.0->librosa>=0.6.1->resemblyzer->-r requirements.txt (line 6)) (3.0.4)
    Building wheels for collected packages: typing, webrtcvad
      Building wheel for typing (setup.py) ... done
      Created wheel for typing: filename=typing-3.7.4.3-cp37-none-any.whl size=26308 sha256=e1895b732b00493d7c1529c501589bbcb8fa5a670fceed91c0eb0dc5c6674698
      Stored in directory: /root/.cache/pip/wheels/2d/04/41/8e1836e79581989c22eebac3f4e70aaac9af07b0908da173be
      Building wheel for webrtcvad (setup.py) ... done
      Created wheel for webrtcvad: filename=webrtcvad-2.0.10-cp37-cp37m-linux_x86_64.whl size=72295 sha256=e5172e95daec0131758c3346c60407df183710942c1cc1a5183604d97b4671e6
      Stored in directory: /root/.cache/pip/wheels/44/2a/18/bd1aec41cac7c3051fe95d92a6ed446122ea31dc713c432fa1
    Successfully built typing webrtcvad
    Installing collected packages: typing, webrtcvad, resemblyzer, spectralcluster
    Successfully installed resemblyzer-0.1.1.dev0 spectralcluster-0.1.0 typing-3.7.4.3 webrtcvad-2.0.10

```python
ls
```

    Clustering/  Embedding/  __init__.py  model.model     README.md         Utils/
    config/      hparam.py   LICENSE      Preprocessing/  requirements.txt

```python
# !pip install resemblyzer
# !pip install spectralcluster
from Utils import DataLoader
from Preprocessing import VAD_chunk
from Clustering import SpectralClustering
from Embedding import concat_segs, get_STFTs, align_embeddings, SpeechEmbedder
import torch
```

```python
def create_labelling(labels, continuos_times, seg_length_ms = 0.4):
    labelling = []

    time_idx = 0
    start_time = continuos_times[0][0]
    end_time = start_time + seg_length_ms

    for i in range(len(labels)):
        if end_time >= continuos_times[time_idx][1]:
            temp = [str(labels[i]), start_time, continuos_times[time_idx][1]]
            labelling.append(tuple(temp))
            time_idx += 1
            start_time = continuos_times[time_idx][0]
            end_time = start_time + seg_length_ms

        elif i==len(labels)-1:
            temp = [str(labels[i]), start_time, end_time]
            labelling.append(tuple(temp))

        elif labels[i] != labels[i+1]:
            temp = [str(labels[i]), start_time, end_time]
            labelling.append(tuple(temp))
            start_time = end_time
            end_time = start_time + seg_length_ms

        else:
            end_time += seg_length_ms

    return labelling
```

```python
# from sklearn.cluster import SpectralClustering, KMeans
from Clustering import SpectralClustering

def get_hypothesis(audio_file_name, embedder_net):
  times, segs = VAD_chunk(2, audio_file_name)
  concat_seg, continuos_times = concat_segs(times, segs)

  STFT_frames = get_STFTs(concat_seg)
  STFT_frames = np.stack(STFT_frames, axis=2)
  STFT_frames = torch.tensor(np.transpose(STFT_frames, axes=(2,1,0)))
  print(STFT_frames.shape)
  embeddings = []
  for STFT_frame in STFT_frames:
    embeddings.append((dvector.embed_utterance(STFT_frame.reshape(-1,40))).detach().numpy())
  print(len(embeddings), embeddings[0].shape)
  # embeddings = torch.tensor(embeddings)
  # embeddings = embedder_net(STFT_frames)
  aligned_embeddings = align_embeddings(embeddings)

  # Ger cluster ids for embedding
  clusterer = SpectralClustering(
    min_clusters=4,
    max_clusters=20,
    p_percentile=0.90,
    gaussian_blur_sigma=1
  )
  # labels = clusterer.predict(aligned_embeddings)
  labels = clusterer.predict(aligned_embeddings)

  # Get labelling from cluster assignment
  labelling = create_labelling(labels, continuos_times)
  return labelling, labels, aligned_embeddings
```

```python
(wav_data), reference = file_loader[2]
labelling, labels, embeddings = get_hypothesis(wav_data, 16000)
```

    torch.Size([5513, 24, 40])
    5513 (256,)

```python
len(labelling), len(labels), len(embeddings)
```

    (368, 1651, 1651)

```python
from __future__ import print_function
import time
import numpy as np
import pandas as pd
# from sklearn.datasets import fetch_mldata
from sklearn.decomposition import PCA
from sklearn.manifold import TSNE
%matplotlib inline
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
import seaborn as sns

tsne = TSNE(n_components=2, verbose=1, perplexity=40, n_iter=300)
tsne_results = tsne.fit_transform(embeddings)

df = pd.DataFrame(tsne_results, columns = {"one", "two"})
df["y"] = (labels)
df.head()
```

    [t-SNE] Computing 121 nearest neighbors...
    [t-SNE] Indexed 1651 samples in 0.032s...
    [t-SNE] Computed neighbors for 1651 samples in 1.532s...
    [t-SNE] Computed conditional probabilities for sample 1000 / 1651
    [t-SNE] Computed conditional probabilities for sample 1651 / 1651
    [t-SNE] Mean sigma: 0.184096
    [t-SNE] KL divergence after 250 iterations with early exaggeration: 69.869026
    [t-SNE] KL divergence after 300 iterations: 1.771355

<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>two</th>
      <th>one</th>
      <th>y</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>5.514346</td>
      <td>-3.630972</td>
      <td>2</td>
    </tr>
    <tr>
      <th>1</th>
      <td>14.042770</td>
      <td>6.848162</td>
      <td>2</td>
    </tr>
    <tr>
      <th>2</th>
      <td>9.613286</td>
      <td>-3.144570</td>
      <td>2</td>
    </tr>
    <tr>
      <th>3</th>
      <td>11.382989</td>
      <td>-5.070907</td>
      <td>2</td>
    </tr>
    <tr>
      <th>4</th>
      <td>6.550267</td>
      <td>-5.768169</td>
      <td>2</td>
    </tr>
  </tbody>
</table>
</div>

```python
np.unique(labels)
```

    array([0, 1, 2, 3], dtype=int32)

```python
# df_subset['tsne-2d-one'] = tsne_results[:,0]
# df_subset['tsne-2d-two'] = tsne_results[:,1]

plt.figure(figsize=(16,10))
sns.scatterplot(
    x="one", y="two",
    hue="y",
    palette=sns.color_palette("hls", len(np.unique(labels))),
    data=df,
    legend="full",
    alpha=0.3
)
```

```python
from pyannote.core import Segment, Timeline, Annotation
from pyannote.metrics.diarization import DiarizationErrorRate
import webrtcvad
import warnings
def Annotation_from_tuple_arr(labels_arr):
  annotate = Annotation()
  for label in labels_arr:
    annotate[Segment(label[1], label[2])] = str(int(label[0]))
  return annotate

hypothesis = Annotation_from_tuple_arr(labelling)
metric = DiarizationErrorRate()
abs(metric(reference, hypothesis))
```

    /usr/local/lib/python3.7/dist-packages/pyannote/metrics/utils.py:184: UserWarning: 'uem' was approximated by the union of 'reference' and 'hypothesis' extents.
      "'uem' was approximated by the union of 'reference' "

    0.511478674558818

```python
print(reference)
```

    [ 00:00:00.040 -->  00:00:19.040] 39 spk03
    [ 00:00:19.040 -->  00:00:19.520] 2 spk01
    [ 00:00:20.440 -->  00:00:34.800] 40 spk03
    [ 00:00:34.840 -->  00:01:04.840] 3 spk01
    [ 00:01:04.840 -->  00:01:05.640] 41 spk03
    [ 00:01:05.440 -->  00:01:07.840] 4 spk01
    [ 00:01:07.480 -->  00:01:11.760] 42 spk03
    [ 00:01:08.800 -->  00:01:10.720] 5 spk01
    [ 00:01:11.720 -->  00:01:44.400] 6 spk01
    [ 00:01:22.480 -->  00:01:22.920] 43 spk03
    [ 00:01:30.160 -->  00:01:32.160] 45 spk03
    [ 00:01:32.920 -->  00:01:33.360] 46 spk03
    [ 00:01:43.520 -->  00:02:12.440] 47 spk03
    [ 00:02:12.320 -->  00:02:27.320] 16 spk02
    [ 00:02:20.120 -->  00:02:21.320] 48 spk03
    [ 00:02:27.000 -->  00:02:34.160] 49 spk03
    [ 00:02:34.320 -->  00:02:53.680] 17 spk02
    [ 00:02:52.040 -->  00:03:03.040] 50 spk03
    [ 00:02:57.480 -->  00:02:59.000] 18 spk02
    [ 00:02:59.640 -->  00:03:00.440] 19 spk02
    [ 00:03:02.520 -->  00:03:17.800] 20 spk02
    [ 00:03:17.200 -->  00:03:21.520] 51 spk03
    [ 00:03:20.560 -->  00:03:34.200] 21 spk02
    [ 00:03:32.320 -->  00:03:40.800] 52 spk03
    [ 00:03:39.360 -->  00:03:43.160] 23 spk02
    [ 00:03:41.960 -->  00:03:42.480] 53 spk03
    [ 00:03:42.920 -->  00:04:12.120] 54 spk03
    [ 00:03:44.360 -->  00:03:45.560] 24 spk02
    [ 00:04:12.320 -->  00:04:59.000] 0 spk00
    [ 00:04:56.920 -->  00:05:06.400] 56 spk03
    [ 00:05:06.400 -->  00:05:06.800] 11 spk00
    [ 00:05:06.800 -->  00:06:15.080] 57 spk03
    [ 00:06:15.080 -->  00:06:33.960] 7 spk01
    [ 00:06:31.440 -->  00:06:32.000] 58 spk03
    [ 00:06:32.680 -->  00:06:35.680] 59 spk03
    [ 00:06:35.680 -->  00:06:36.680] 25 spk02
    [ 00:06:36.640 -->  00:07:03.080] 8 spk01
    [ 00:06:41.760 -->  00:06:42.240] 60 spk03
    [ 00:07:01.400 -->  00:07:28.920] 61 spk03
    [ 00:07:28.120 -->  00:07:42.440] 26 spk02
    [ 00:07:42.160 -->  00:07:43.240] 62 spk03
    [ 00:07:43.320 -->  00:07:44.240] 27 spk02
    [ 00:07:43.880 -->  00:07:44.680] 63 spk03
    [ 00:07:44.760 -->  00:07:45.520] 28 spk02
    [ 00:07:45.200 -->  00:07:45.720] 64 spk03
    [ 00:07:45.920 -->  00:08:03.440] 29 spk02
    [ 00:08:03.120 -->  00:08:04.800] 65 spk03
    [ 00:08:04.800 -->  00:08:09.240] 30 spk02
    [ 00:08:04.920 -->  00:08:07.080] 9 spk01
    [ 00:08:05.520 -->  00:08:07.080] 67 spk03
    [ 00:08:09.040 -->  00:08:09.480] 68 spk03
    [ 00:08:09.240 -->  00:08:16.800] 22 spk00
    [ 00:08:15.240 -->  00:08:18.120] 31 spk02
    [ 00:08:18.440 -->  00:08:19.760] 10 spk01
    [ 00:08:19.120 -->  00:08:20.600] 69 spk03
    [ 00:08:19.400 -->  00:08:21.080] 32 spk02
    [ 00:08:20.600 -->  00:08:27.640] 33 spk00
    [ 00:08:25.600 -->  00:08:31.960] 70 spk03
    [ 00:08:30.000 -->  00:08:33.480] 44 spk00
    [ 00:08:32.880 -->  00:08:33.640] 34 spk02
    [ 00:08:34.960 -->  00:08:35.920] 71 spk03
    [ 00:08:35.240 -->  00:08:59.120] 55 spk00
    [ 00:08:44.360 -->  00:08:45.480] 72 spk03
    [ 00:08:59.000 -->  00:09:06.080] 73 spk03
    [ 00:09:06.360 -->  00:09:36.840] 66 spk00
    [ 00:09:34.120 -->  00:09:34.760] 74 spk03
    [ 00:09:34.800 -->  00:09:36.440] 12 spk01
    [ 00:09:35.360 -->  00:09:40.760] 75 spk03
    [ 00:09:37.600 -->  00:09:47.480] 13 spk01
    [ 00:09:41.600 -->  00:09:42.160] 76 spk03
    [ 00:09:44.280 -->  00:09:44.800] 78 spk03
    [ 00:09:45.520 -->  00:09:46.280] 35 spk02
    [ 00:09:46.040 -->  00:09:53.120] 79 spk03
    [ 00:09:49.120 -->  00:09:49.680] 14 spk01
    [ 00:09:52.640 -->  00:10:03.160] 15 spk01
    [ 00:09:57.120 -->  00:09:58.120] 36 spk02
    [ 00:10:01.200 -->  00:10:01.640] 80 spk03
    [ 00:10:01.440 -->  00:10:01.880] 77 spk00
    [ 00:10:02.360 -->  00:10:03.160] 81 spk03
    [ 00:10:03.000 -->  00:10:17.640] 37 spk02
    [ 00:10:16.360 -->  00:10:34.240] 82 spk03
    [ 00:10:30.360 -->  00:10:30.880] 86 spk00
    [ 00:10:34.200 -->  00:10:46.120] 38 spk02
    [ 00:10:39.080 -->  00:10:40.320] 83 spk03
    [ 00:10:44.280 -->  00:10:45.880] 84 spk03
    [ 00:10:46.280 -->  00:10:56.520] 1 spk00
    [ 00:10:54.680 -->  00:11:04.280] 85 spk03

```python
print(hypothesis)
```

    [ 00:00:00.140 -->  00:00:03.740] _ 0
    [ 00:00:03.740 -->  00:00:03.880] _ 1
    [ 00:00:05.220 -->  00:00:08.820] _ 1
    [ 00:00:08.820 -->  00:00:12.520] _ 3
    [ 00:00:12.660 -->  00:00:13.860] _ 3
    [ 00:00:13.860 -->  00:00:14.660] _ 1
    [ 00:00:14.660 -->  00:00:15.060] _ 2
    [ 00:00:15.060 -->  00:00:18.660] _ 3
    [ 00:00:18.660 -->  00:00:19.060] _ 2
    [ 00:00:19.060 -->  00:00:21.420] _ 3
    [ 00:00:21.440 -->  00:00:25.040] _ 3
    [ 00:00:25.040 -->  00:00:25.060] _ 2
    [ 00:00:25.180 -->  00:00:25.980] _ 1
    [ 00:00:25.980 -->  00:00:29.640] _ 3
    [ 00:00:29.800 -->  00:00:31.800] _ 3
    [ 00:00:31.840 -->  00:00:33.440] _ 1
    [ 00:00:33.440 -->  00:00:34.400] _ 3
    [ 00:00:34.700 -->  00:00:35.900] _ 3
    [ 00:00:35.900 -->  00:00:37.500] _ 2
    [ 00:00:37.500 -->  00:00:38.300] _ 1
    [ 00:00:38.300 -->  00:00:39.920] _ 2
    [ 00:00:40.420 -->  00:00:40.820] _ 2
    [ 00:00:40.820 -->  00:00:41.080] _ 1
    [ 00:00:41.100 -->  00:00:43.900] _ 1
    [ 00:00:43.900 -->  00:00:45.100] _ 2
    [ 00:00:45.100 -->  00:00:45.520] _ 1
    [ 00:00:45.600 -->  00:00:47.600] _ 1
    [ 00:00:47.600 -->  00:00:49.180] _ 2
    [ 00:00:49.360 -->  00:00:50.560] _ 1
    [ 00:00:50.560 -->  00:00:51.720] _ 2
    [ 00:00:51.780 -->  00:00:52.980] _ 1
    [ 00:00:52.980 -->  00:00:53.380] _ 2
    [ 00:00:53.380 -->  00:00:53.820] _ 3
    [ 00:00:53.900 -->  00:00:57.100] _ 1
    [ 00:00:57.100 -->  00:00:57.900] _ 2
    [ 00:00:57.900 -->  00:00:58.600] _ 1
    [ 00:00:58.760 -->  00:01:03.960] _ 1
    [ 00:01:03.960 -->  00:01:05.160] _ 2
    [ 00:01:05.160 -->  00:01:05.420] _ 1
    [ 00:01:05.480 -->  00:01:06.860] _ 1
    [ 00:01:06.980 -->  00:01:07.880] _ 1
    [ 00:01:07.960 -->  00:01:09.160] _ 2
    [ 00:01:09.160 -->  00:01:09.960] _ 3
    [ 00:01:09.960 -->  00:01:12.500] _ 1
    [ 00:01:12.560 -->  00:01:14.560] _ 2
    [ 00:01:14.560 -->  00:01:14.960] _ 1
    [ 00:01:14.960 -->  00:01:15.760] _ 3
    [ 00:01:15.760 -->  00:01:16.560] _ 2
    [ 00:01:16.560 -->  00:01:16.680] _ 1
    [ 00:01:16.740 -->  00:01:17.540] _ 1
    [ 00:01:17.540 -->  00:01:20.340] _ 2
    [ 00:01:20.340 -->  00:01:20.700] _ 3
    [ 00:01:20.820 -->  00:01:23.820] _ 3
    [ 00:01:24.860 -->  00:01:26.240] _ 3
    [ 00:01:27.040 -->  00:01:28.180] _ 3
    [ 00:01:28.680 -->  00:01:29.420] _ 3
    [ 00:01:29.700 -->  00:01:32.900] _ 3
    [ 00:01:32.900 -->  00:01:32.960] _ 2
    [ 00:01:33.100 -->  00:01:33.900] _ 2
    [ 00:01:33.900 -->  00:01:34.600] _ 3
    [ 00:01:35.060 -->  00:01:37.860] _ 3
    [ 00:01:37.860 -->  00:01:38.220] _ 2
    [ 00:01:38.640 -->  00:01:39.440] _ 3
    [ 00:01:39.440 -->  00:01:40.660] _ 2
    [ 00:01:41.120 -->  00:01:42.720] _ 2
    [ 00:01:42.720 -->  00:01:44.820] _ 3
    [ 00:01:45.400 -->  00:01:45.800] _ 3
    [ 00:01:45.800 -->  00:01:46.940] _ 2
    [ 00:01:47.160 -->  00:01:47.820] _ 2
    [ 00:01:48.160 -->  00:01:50.940] _ 2
    [ 00:01:51.040 -->  00:01:51.840] _ 2
    [ 00:01:51.840 -->  00:01:52.100] _ 1
    [ 00:01:52.200 -->  00:01:52.600] _ 1
    [ 00:01:52.600 -->  00:01:53.440] _ 2
    [ 00:01:53.800 -->  00:01:56.760] _ 2
    [ 00:01:56.840 -->  00:02:00.240] _ 2
    [ 00:02:00.320 -->  00:02:02.600] _ 2
    [ 00:02:02.940 -->  00:02:03.740] _ 1
    [ 00:02:03.740 -->  00:02:04.440] _ 2
    [ 00:02:05.200 -->  00:02:06.000] _ 2
    [ 00:02:06.000 -->  00:02:07.200] _ 3
    [ 00:02:07.200 -->  00:02:08.000] _ 2
    [ 00:02:08.000 -->  00:02:08.300] _ 3
    [ 00:02:08.500 -->  00:02:10.300] _ 3
    [ 00:02:10.900 -->  00:02:11.300] _ 3

```python

```


# Text-to-Speech

# Installing and Importing the Dependencies

```python
# run the following from the terminal
#sudo apt-get install sox libsndfile1 ffmpeg
```

```python
# Installing the Dependencies
!pip install wget unidecode
!pip3.8 install pytorch_lightning
```

    Defaulting to user installation because normal site-packages is not writeable
    Requirement already satisfied: wget in /home/ubuntu/.local/lib/python3.8/site-packages (3.2)
    Requirement already satisfied: unidecode in /home/ubuntu/.local/lib/python3.8/site-packages (1.3.2)
    Defaulting to user installation because normal site-packages is not writeable
    Requirement already satisfied: pytorch_lightning in /home/ubuntu/.local/lib/python3.8/site-packages (1.1.5)
    Requirement already satisfied: future>=0.17.1 in /home/ubuntu/.local/lib/python3.8/site-packages (from pytorch_lightning) (0.18.2)
    Requirement already satisfied: torch>=1.3 in /home/ubuntu/.local/lib/python3.8/site-packages (from pytorch_lightning) (1.7.1)
    Requirement already satisfied: tqdm>=4.41.0 in /home/ubuntu/.local/lib/python3.8/site-packages (from pytorch_lightning) (4.62.3)
    Requirement already satisfied: tensorboard>=2.2.0 in /home/ubuntu/.local/lib/python3.8/site-packages (from pytorch_lightning) (2.7.0)
    Requirement already satisfied: PyYAML>=5.1 in /home/ubuntu/.local/lib/python3.8/site-packages (from pytorch_lightning) (6.0)
    Requirement already satisfied: fsspec[http]>=0.8.1 in /home/ubuntu/.local/lib/python3.8/site-packages (from pytorch_lightning) (2021.11.1)
    Requirement already satisfied: numpy>=1.16.6 in /home/ubuntu/.local/lib/python3.8/site-packages (from pytorch_lightning) (1.20.3)
    Requirement already satisfied: aiohttp in /home/ubuntu/.local/lib/python3.8/site-packages (from fsspec[http]>=0.8.1->pytorch_lightning) (3.8.1)
    Requirement already satisfied: requests in /home/ubuntu/.local/lib/python3.8/site-packages (from fsspec[http]>=0.8.1->pytorch_lightning) (2.26.0)
    Requirement already satisfied: grpcio>=1.24.3 in /home/ubuntu/.local/lib/python3.8/site-packages (from tensorboard>=2.2.0->pytorch_lightning) (1.43.0)
    Requirement already satisfied: absl-py>=0.4 in /home/ubuntu/.local/lib/python3.8/site-packages (from tensorboard>=2.2.0->pytorch_lightning) (1.0.0)
    Requirement already satisfied: werkzeug>=0.11.15 in /home/ubuntu/.local/lib/python3.8/site-packages (from tensorboard>=2.2.0->pytorch_lightning) (2.0.2)
    Requirement already satisfied: wheel>=0.26 in /home/ubuntu/.local/lib/python3.8/site-packages (from tensorboard>=2.2.0->pytorch_lightning) (0.36.2)
    Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in /home/ubuntu/.local/lib/python3.8/site-packages (from tensorboard>=2.2.0->pytorch_lightning) (0.4.6)
    Requirement already satisfied: markdown>=2.6.8 in /home/ubuntu/.local/lib/python3.8/site-packages (from tensorboard>=2.2.0->pytorch_lightning) (3.3.6)
    Requirement already satisfied: tensorboard-plugin-wit>=1.6.0 in /home/ubuntu/.local/lib/python3.8/site-packages (from tensorboard>=2.2.0->pytorch_lightning) (1.8.0)
    Requirement already satisfied: protobuf>=3.6.0 in /home/ubuntu/.local/lib/python3.8/site-packages (from tensorboard>=2.2.0->pytorch_lightning) (3.17.3)
    Requirement already satisfied: setuptools>=41.0.0 in /home/ubuntu/.local/lib/python3.8/site-packages (from tensorboard>=2.2.0->pytorch_lightning) (59.6.0)
    Requirement already satisfied: tensorboard-data-server<0.7.0,>=0.6.0 in /home/ubuntu/.local/lib/python3.8/site-packages (from tensorboard>=2.2.0->pytorch_lightning) (0.6.1)
    Requirement already satisfied: google-auth<3,>=1.6.3 in /home/ubuntu/.local/lib/python3.8/site-packages (from tensorboard>=2.2.0->pytorch_lightning) (2.3.3)
    Requirement already satisfied: typing-extensions in /home/ubuntu/.local/lib/python3.8/site-packages (from torch>=1.3->pytorch_lightning) (4.0.1)
    Requirement already satisfied: six in /usr/lib/python3/dist-packages (from absl-py>=0.4->tensorboard>=2.2.0->pytorch_lightning) (1.11.0)
    Requirement already satisfied: rsa<5,>=3.1.4 in /home/ubuntu/.local/lib/python3.8/site-packages (from google-auth<3,>=1.6.3->tensorboard>=2.2.0->pytorch_lightning) (4.8)
    Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/lib/python3/dist-packages (from google-auth<3,>=1.6.3->tensorboard>=2.2.0->pytorch_lightning) (0.2.1)
    Requirement already satisfied: cachetools<5.0,>=2.0.0 in /home/ubuntu/.local/lib/python3.8/site-packages (from google-auth<3,>=1.6.3->tensorboard>=2.2.0->pytorch_lightning) (4.2.2)
    Requirement already satisfied: requests-oauthlib>=0.7.0 in /home/ubuntu/.local/lib/python3.8/site-packages (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard>=2.2.0->pytorch_lightning) (1.3.0)
    Requirement already satisfied: importlib-metadata>=4.4 in /home/ubuntu/.local/lib/python3.8/site-packages (from markdown>=2.6.8->tensorboard>=2.2.0->pytorch_lightning) (4.9.0)
    Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3/dist-packages (from requests->fsspec[http]>=0.8.1->pytorch_lightning) (2.6)
    Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/lib/python3/dist-packages (from requests->fsspec[http]>=0.8.1->pytorch_lightning) (1.22)
    Requirement already satisfied: charset-normalizer~=2.0.0 in /home/ubuntu/.local/lib/python3.8/site-packages (from requests->fsspec[http]>=0.8.1->pytorch_lightning) (2.0.9)
    Requirement already satisfied: certifi>=2017.4.17 in /usr/lib/python3/dist-packages (from requests->fsspec[http]>=0.8.1->pytorch_lightning) (2018.1.18)
    Requirement already satisfied: aiosignal>=1.1.2 in /home/ubuntu/.local/lib/python3.8/site-packages (from aiohttp->fsspec[http]>=0.8.1->pytorch_lightning) (1.2.0)
    Requirement already satisfied: frozenlist>=1.1.1 in /home/ubuntu/.local/lib/python3.8/site-packages (from aiohttp->fsspec[http]>=0.8.1->pytorch_lightning) (1.2.0)
    Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /home/ubuntu/.local/lib/python3.8/site-packages (from aiohttp->fsspec[http]>=0.8.1->pytorch_lightning) (4.0.1)
    Requirement already satisfied: yarl<2.0,>=1.0 in /home/ubuntu/.local/lib/python3.8/site-packages (from aiohttp->fsspec[http]>=0.8.1->pytorch_lightning) (1.7.2)
    Requirement already satisfied: multidict<7.0,>=4.5 in /home/ubuntu/.local/lib/python3.8/site-packages (from aiohttp->fsspec[http]>=0.8.1->pytorch_lightning) (5.2.0)
    Requirement already satisfied: attrs>=17.3.0 in /usr/lib/python3/dist-packages (from aiohttp->fsspec[http]>=0.8.1->pytorch_lightning) (17.4.0)
    Requirement already satisfied: zipp>=0.5 in /home/ubuntu/.local/lib/python3.8/site-packages (from importlib-metadata>=4.4->markdown>=2.6.8->tensorboard>=2.2.0->pytorch_lightning) (3.6.0)
    Requirement already satisfied: oauthlib>=3.0.0 in /home/ubuntu/.local/lib/python3.8/site-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard>=2.2.0->pytorch_lightning) (3.1.1)
    Requirement already satisfied: pyasn1>=0.1.3 in /usr/lib/python3/dist-packages (from rsa<5,>=3.1.4->google-auth<3,>=1.6.3->tensorboard>=2.2.0->pytorch_lightning) (0.4.2)

```python
# Upgrading pip and cloning the NVIDIA
!pip3.8 install --upgrade pip
!pip3.8 install torch
BRANCH = 'r1.0.0rc1'
!python3.8 -m pip install git+https://github.com/NVIDIA/NeMo.git@$BRANCH#egg=nemo_toolkit[tts]
```

    Defaulting to user installation because normal site-packages is not writeable
    Requirement already satisfied: pip in /home/ubuntu/.local/lib/python3.8/site-packages (21.3.1)
    Defaulting to user installation because normal site-packages is not writeable
    Requirement already satisfied: torch in /home/ubuntu/.local/lib/python3.8/site-packages (1.7.1)
    Requirement already satisfied: numpy in /home/ubuntu/.local/lib/python3.8/site-packages (from torch) (1.20.3)
    Requirement already satisfied: typing-extensions in /home/ubuntu/.local/lib/python3.8/site-packages (from torch) (4.0.1)
    Defaulting to user installation because normal site-packages is not writeable
    Collecting nemo_toolkit[tts]
      Cloning https://github.com/NVIDIA/NeMo.git (to revision r1.0.0rc1) to /tmp/pip-install-2uc9vwai/nemo-toolkit_6f1b61c1834f467aabb86c7f87e07660
      Running command git clone --filter=blob:none -q https://github.com/NVIDIA/NeMo.git /tmp/pip-install-2uc9vwai/nemo-toolkit_6f1b61c1834f467aabb86c7f87e07660
      Running command git checkout -b r1.0.0rc1 --track origin/r1.0.0rc1
      Switched to a new branch 'r1.0.0rc1'
      Branch 'r1.0.0rc1' set up to track remote branch 'r1.0.0rc1' from 'origin'.
      Resolved https://github.com/NVIDIA/NeMo.git to commit 977b9a2c8bee5f0575935cde9527b9fe2b7badfe
      Preparing metadata (setup.py) ... done
    Requirement already satisfied: numpy>=1.18.2 in /home/ubuntu/.local/lib/python3.8/site-packages (from nemo_toolkit[tts]) (1.20.3)
    Requirement already satisfied: onnx>=1.7.0 in /home/ubuntu/.local/lib/python3.8/site-packages (from nemo_toolkit[tts]) (1.10.2)
    Requirement already satisfied: pytorch-lightning==1.1.5 in /home/ubuntu/.local/lib/python3.8/site-packages (from nemo_toolkit[tts]) (1.1.5)
    Requirement already satisfied: python-dateutil in /home/ubuntu/.local/lib/python3.8/site-packages (from nemo_toolkit[tts]) (2.8.2)
    Requirement already satisfied: torch<=1.7.1 in /home/ubuntu/.local/lib/python3.8/site-packages (from nemo_toolkit[tts]) (1.7.1)
    Requirement already satisfied: torchvision==0.8.2 in /home/ubuntu/.local/lib/python3.8/site-packages (from nemo_toolkit[tts]) (0.8.2)
    Requirement already satisfied: torchaudio==0.7.2 in /home/ubuntu/.local/lib/python3.8/site-packages (from nemo_toolkit[tts]) (0.7.2)
    Requirement already satisfied: torchtext==0.8.0 in /home/ubuntu/.local/lib/python3.8/site-packages (from nemo_toolkit[tts]) (0.8.0)
    Requirement already satisfied: wget in /home/ubuntu/.local/lib/python3.8/site-packages (from nemo_toolkit[tts]) (3.2)
    Requirement already satisfied: wrapt in /home/ubuntu/.local/lib/python3.8/site-packages (from nemo_toolkit[tts]) (1.13.3)
    Requirement already satisfied: ruamel.yaml in /home/ubuntu/.local/lib/python3.8/site-packages (from nemo_toolkit[tts]) (0.17.17)
    Requirement already satisfied: scikit-learn in /home/ubuntu/.local/lib/python3.8/site-packages (from nemo_toolkit[tts]) (0.24.2)
    Requirement already satisfied: omegaconf>=2.0.5 in /home/ubuntu/.local/lib/python3.8/site-packages (from nemo_toolkit[tts]) (2.1.1)
    Requirement already satisfied: hydra-core>=1.0.4 in /home/ubuntu/.local/lib/python3.8/site-packages (from nemo_toolkit[tts]) (1.1.1)
    Requirement already satisfied: transformers>=4.0.1 in /home/ubuntu/.local/lib/python3.8/site-packages (from nemo_toolkit[tts]) (4.14.1)
    Requirement already satisfied: sentencepiece<1.0.0 in /home/ubuntu/.local/lib/python3.8/site-packages (from nemo_toolkit[tts]) (0.1.96)
    Requirement already satisfied: webdataset>=0.1.48 in /home/ubuntu/.local/lib/python3.8/site-packages (from nemo_toolkit[tts]) (0.1.103)
    Requirement already satisfied: tqdm>=4.41.0 in /home/ubuntu/.local/lib/python3.8/site-packages (from nemo_toolkit[tts]) (4.62.3)
    Requirement already satisfied: opencc in /home/ubuntu/.local/lib/python3.8/site-packages (from nemo_toolkit[tts]) (1.1.3)
    Requirement already satisfied: pangu in /home/ubuntu/.local/lib/python3.8/site-packages (from nemo_toolkit[tts]) (4.0.6.1)
    Requirement already satisfied: jieba in /home/ubuntu/.local/lib/python3.8/site-packages (from nemo_toolkit[tts]) (0.42.1)
    Requirement already satisfied: matplotlib in /home/ubuntu/.local/lib/python3.8/site-packages (from nemo_toolkit[tts]) (3.5.0)
    Requirement already satisfied: pypinyin in /home/ubuntu/.local/lib/python3.8/site-packages (from nemo_toolkit[tts]) (0.44.0)
    Requirement already satisfied: attrdict in /home/ubuntu/.local/lib/python3.8/site-packages (from nemo_toolkit[tts]) (2.0.1)
    Requirement already satisfied: pystoi in /home/ubuntu/.local/lib/python3.8/site-packages (from nemo_toolkit[tts]) (0.3.3)
    Requirement already satisfied: pesq in /home/ubuntu/.local/lib/python3.8/site-packages (from nemo_toolkit[tts]) (0.0.3)
    Requirement already satisfied: g2p_en in /home/ubuntu/.local/lib/python3.8/site-packages (from nemo_toolkit[tts]) (2.1.0)
    Requirement already satisfied: braceexpand in /home/ubuntu/.local/lib/python3.8/site-packages (from nemo_toolkit[tts]) (0.1.7)
    Requirement already satisfied: editdistance in /home/ubuntu/.local/lib/python3.8/site-packages (from nemo_toolkit[tts]) (0.6.0)
    Requirement already satisfied: frozendict in /home/ubuntu/.local/lib/python3.8/site-packages (from nemo_toolkit[tts]) (2.1.1)
    Requirement already satisfied: inflect in /home/ubuntu/.local/lib/python3.8/site-packages (from nemo_toolkit[tts]) (5.3.0)
    Requirement already satisfied: kaldi-io in /home/ubuntu/.local/lib/python3.8/site-packages (from nemo_toolkit[tts]) (0.9.4)
    Requirement already satisfied: librosa in /home/ubuntu/.local/lib/python3.8/site-packages (from nemo_toolkit[tts]) (0.8.1)
    Requirement already satisfied: marshmallow in /home/ubuntu/.local/lib/python3.8/site-packages (from nemo_toolkit[tts]) (3.14.1)
    Requirement already satisfied: packaging in /home/ubuntu/.local/lib/python3.8/site-packages (from nemo_toolkit[tts]) (21.0)
    Requirement already satisfied: soundfile in /home/ubuntu/.local/lib/python3.8/site-packages (from nemo_toolkit[tts]) (0.10.3.post1)
    Requirement already satisfied: sox in /home/ubuntu/.local/lib/python3.8/site-packages (from nemo_toolkit[tts]) (1.4.1)
    Requirement already satisfied: torch-stft in /home/ubuntu/.local/lib/python3.8/site-packages (from nemo_toolkit[tts]) (0.1.4)
    Requirement already satisfied: unidecode in /home/ubuntu/.local/lib/python3.8/site-packages (from nemo_toolkit[tts]) (1.3.2)
    Requirement already satisfied: kaldi-python-io in /home/ubuntu/.local/lib/python3.8/site-packages (from nemo_toolkit[tts]) (1.2.2)
    Requirement already satisfied: kaldiio in /home/ubuntu/.local/lib/python3.8/site-packages (from nemo_toolkit[tts]) (2.17.2)
    Requirement already satisfied: scipy in /home/ubuntu/.local/lib/python3.8/site-packages (from nemo_toolkit[tts]) (1.7.1)
    Requirement already satisfied: pandas in /home/ubuntu/.local/lib/python3.8/site-packages (from nemo_toolkit[tts]) (1.3.1)
    Requirement already satisfied: pydub in /home/ubuntu/.local/lib/python3.8/site-packages (from nemo_toolkit[tts]) (0.25.1)
    Requirement already satisfied: pyannote.core in /home/ubuntu/.local/lib/python3.8/site-packages (from nemo_toolkit[tts]) (4.3)
    Requirement already satisfied: pyannote.metrics in /home/ubuntu/.local/lib/python3.8/site-packages (from nemo_toolkit[tts]) (3.1)
    Requirement already satisfied: spectralcluster in /home/ubuntu/.local/lib/python3.8/site-packages (from nemo_toolkit[tts]) (0.2.4)
    Requirement already satisfied: fsspec[http]>=0.8.1 in /home/ubuntu/.local/lib/python3.8/site-packages (from pytorch-lightning==1.1.5->nemo_toolkit[tts]) (2021.11.1)
    Requirement already satisfied: tensorboard>=2.2.0 in /home/ubuntu/.local/lib/python3.8/site-packages (from pytorch-lightning==1.1.5->nemo_toolkit[tts]) (2.7.0)
    Requirement already satisfied: PyYAML>=5.1 in /home/ubuntu/.local/lib/python3.8/site-packages (from pytorch-lightning==1.1.5->nemo_toolkit[tts]) (6.0)
    Requirement already satisfied: future>=0.17.1 in /home/ubuntu/.local/lib/python3.8/site-packages (from pytorch-lightning==1.1.5->nemo_toolkit[tts]) (0.18.2)
    Requirement already satisfied: typing-extensions in /home/ubuntu/.local/lib/python3.8/site-packages (from torch<=1.7.1->nemo_toolkit[tts]) (4.0.1)
    Requirement already satisfied: requests in /home/ubuntu/.local/lib/python3.8/site-packages (from torchtext==0.8.0->nemo_toolkit[tts]) (2.26.0)
    Requirement already satisfied: pillow>=4.1.1 in /home/ubuntu/.local/lib/python3.8/site-packages (from torchvision==0.8.2->nemo_toolkit[tts]) (8.3.1)
    Requirement already satisfied: antlr4-python3-runtime==4.8 in /home/ubuntu/.local/lib/python3.8/site-packages (from hydra-core>=1.0.4->nemo_toolkit[tts]) (4.8)
    Requirement already satisfied: importlib-resources in /home/ubuntu/.local/lib/python3.8/site-packages (from hydra-core>=1.0.4->nemo_toolkit[tts]) (5.4.0)
    Requirement already satisfied: six in /usr/lib/python3/dist-packages (from onnx>=1.7.0->nemo_toolkit[tts]) (1.11.0)
    Requirement already satisfied: protobuf in /home/ubuntu/.local/lib/python3.8/site-packages (from onnx>=1.7.0->nemo_toolkit[tts]) (3.17.3)
    Requirement already satisfied: tokenizers<0.11,>=0.10.1 in /home/ubuntu/.local/lib/python3.8/site-packages (from transformers>=4.0.1->nemo_toolkit[tts]) (0.10.3)
    Requirement already satisfied: sacremoses in /home/ubuntu/.local/lib/python3.8/site-packages (from transformers>=4.0.1->nemo_toolkit[tts]) (0.0.46)
    Requirement already satisfied: huggingface-hub<1.0,>=0.1.0 in /home/ubuntu/.local/lib/python3.8/site-packages (from transformers>=4.0.1->nemo_toolkit[tts]) (0.2.1)
    Requirement already satisfied: filelock in /home/ubuntu/.local/lib/python3.8/site-packages (from transformers>=4.0.1->nemo_toolkit[tts]) (3.0.12)
    Requirement already satisfied: regex!=2019.12.17 in /home/ubuntu/.local/lib/python3.8/site-packages (from transformers>=4.0.1->nemo_toolkit[tts]) (2021.11.10)
    Requirement already satisfied: pyparsing>=2.0.2 in /home/ubuntu/.local/lib/python3.8/site-packages (from packaging->nemo_toolkit[tts]) (2.4.7)
    Requirement already satisfied: nltk>=3.2.4 in /home/ubuntu/.local/lib/python3.8/site-packages (from g2p_en->nemo_toolkit[tts]) (3.6.5)
    Requirement already satisfied: distance>=0.1.3 in /home/ubuntu/.local/lib/python3.8/site-packages (from g2p_en->nemo_toolkit[tts]) (0.1.3)
    Requirement already satisfied: numba>=0.43.0 in /home/ubuntu/.local/lib/python3.8/site-packages (from librosa->nemo_toolkit[tts]) (0.54.0)
    Requirement already satisfied: pooch>=1.0 in /home/ubuntu/.local/lib/python3.8/site-packages (from librosa->nemo_toolkit[tts]) (1.4.0)
    Requirement already satisfied: decorator>=3.0.0 in /home/ubuntu/.local/lib/python3.8/site-packages (from librosa->nemo_toolkit[tts]) (5.0.9)
    Requirement already satisfied: resampy>=0.2.2 in /home/ubuntu/.local/lib/python3.8/site-packages (from librosa->nemo_toolkit[tts]) (0.2.2)
    Requirement already satisfied: joblib>=0.14 in /home/ubuntu/.local/lib/python3.8/site-packages (from librosa->nemo_toolkit[tts]) (1.0.1)
    Requirement already satisfied: audioread>=2.0.0 in /home/ubuntu/.local/lib/python3.8/site-packages (from librosa->nemo_toolkit[tts]) (2.1.9)
    Requirement already satisfied: threadpoolctl>=2.0.0 in /home/ubuntu/.local/lib/python3.8/site-packages (from scikit-learn->nemo_toolkit[tts]) (2.2.0)
    Requirement already satisfied: cffi>=1.0 in /home/ubuntu/.local/lib/python3.8/site-packages (from soundfile->nemo_toolkit[tts]) (1.14.6)
    Requirement already satisfied: fonttools>=4.22.0 in /home/ubuntu/.local/lib/python3.8/site-packages (from matplotlib->nemo_toolkit[tts]) (4.28.2)
    Requirement already satisfied: setuptools-scm>=4 in /home/ubuntu/.local/lib/python3.8/site-packages (from matplotlib->nemo_toolkit[tts]) (6.3.2)
    Requirement already satisfied: cycler>=0.10 in /home/ubuntu/.local/lib/python3.8/site-packages (from matplotlib->nemo_toolkit[tts]) (0.11.0)
    Requirement already satisfied: kiwisolver>=1.0.1 in /home/ubuntu/.local/lib/python3.8/site-packages (from matplotlib->nemo_toolkit[tts]) (1.3.2)
    Requirement already satisfied: pytz>=2017.3 in /home/ubuntu/.local/lib/python3.8/site-packages (from pandas->nemo_toolkit[tts]) (2021.1)
    Requirement already satisfied: simplejson>=3.8.1 in /home/ubuntu/.local/lib/python3.8/site-packages (from pyannote.core->nemo_toolkit[tts]) (3.17.6)
    Requirement already satisfied: sortedcontainers>=2.0.4 in /home/ubuntu/.local/lib/python3.8/site-packages (from pyannote.core->nemo_toolkit[tts]) (2.4.0)
    Requirement already satisfied: tabulate>=0.7.7 in /home/ubuntu/.local/lib/python3.8/site-packages (from pyannote.metrics->nemo_toolkit[tts]) (0.8.9)
    Requirement already satisfied: pyannote.database>=4.0.1 in /home/ubuntu/.local/lib/python3.8/site-packages (from pyannote.metrics->nemo_toolkit[tts]) (4.1.1)
    Requirement already satisfied: docopt>=0.6.2 in /home/ubuntu/.local/lib/python3.8/site-packages (from pyannote.metrics->nemo_toolkit[tts]) (0.6.2)
    Requirement already satisfied: sympy>=1.1 in /home/ubuntu/.local/lib/python3.8/site-packages (from pyannote.metrics->nemo_toolkit[tts]) (1.9)
    Requirement already satisfied: ruamel.yaml.clib>=0.1.2 in /home/ubuntu/.local/lib/python3.8/site-packages (from ruamel.yaml->nemo_toolkit[tts]) (0.2.6)
    Requirement already satisfied: pycparser in /home/ubuntu/.local/lib/python3.8/site-packages (from cffi>=1.0->soundfile->nemo_toolkit[tts]) (2.20)
    Requirement already satisfied: aiohttp in /home/ubuntu/.local/lib/python3.8/site-packages (from fsspec[http]>=0.8.1->pytorch-lightning==1.1.5->nemo_toolkit[tts]) (3.8.1)
    Requirement already satisfied: click in /home/ubuntu/.local/lib/python3.8/site-packages (from nltk>=3.2.4->g2p_en->nemo_toolkit[tts]) (7.1.2)
    Requirement already satisfied: setuptools in /home/ubuntu/.local/lib/python3.8/site-packages (from numba>=0.43.0->librosa->nemo_toolkit[tts]) (59.6.0)
    Requirement already satisfied: llvmlite<0.38,>=0.37.0rc1 in /home/ubuntu/.local/lib/python3.8/site-packages (from numba>=0.43.0->librosa->nemo_toolkit[tts]) (0.37.0)
    Requirement already satisfied: appdirs in /home/ubuntu/.local/lib/python3.8/site-packages (from pooch>=1.0->librosa->nemo_toolkit[tts]) (1.4.4)
    Requirement already satisfied: typer[all]>=0.2.1 in /home/ubuntu/.local/lib/python3.8/site-packages (from pyannote.database>=4.0.1->pyannote.metrics->nemo_toolkit[tts]) (0.4.0)
    Requirement already satisfied: tomli>=1.0.0 in /home/ubuntu/.local/lib/python3.8/site-packages (from setuptools-scm>=4->matplotlib->nemo_toolkit[tts]) (1.2.2)
    Requirement already satisfied: mpmath>=0.19 in /home/ubuntu/.local/lib/python3.8/site-packages (from sympy>=1.1->pyannote.metrics->nemo_toolkit[tts]) (1.2.1)
    Requirement already satisfied: grpcio>=1.24.3 in /home/ubuntu/.local/lib/python3.8/site-packages (from tensorboard>=2.2.0->pytorch-lightning==1.1.5->nemo_toolkit[tts]) (1.43.0)
    Requirement already satisfied: werkzeug>=0.11.15 in /home/ubuntu/.local/lib/python3.8/site-packages (from tensorboard>=2.2.0->pytorch-lightning==1.1.5->nemo_toolkit[tts]) (2.0.2)
    Requirement already satisfied: markdown>=2.6.8 in /home/ubuntu/.local/lib/python3.8/site-packages (from tensorboard>=2.2.0->pytorch-lightning==1.1.5->nemo_toolkit[tts]) (3.3.6)
    Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in /home/ubuntu/.local/lib/python3.8/site-packages (from tensorboard>=2.2.0->pytorch-lightning==1.1.5->nemo_toolkit[tts]) (0.4.6)
    Requirement already satisfied: absl-py>=0.4 in /home/ubuntu/.local/lib/python3.8/site-packages (from tensorboard>=2.2.0->pytorch-lightning==1.1.5->nemo_toolkit[tts]) (1.0.0)
    Requirement already satisfied: google-auth<3,>=1.6.3 in /home/ubuntu/.local/lib/python3.8/site-packages (from tensorboard>=2.2.0->pytorch-lightning==1.1.5->nemo_toolkit[tts]) (2.3.3)
    Requirement already satisfied: tensorboard-plugin-wit>=1.6.0 in /home/ubuntu/.local/lib/python3.8/site-packages (from tensorboard>=2.2.0->pytorch-lightning==1.1.5->nemo_toolkit[tts]) (1.8.0)
    Requirement already satisfied: wheel>=0.26 in /home/ubuntu/.local/lib/python3.8/site-packages (from tensorboard>=2.2.0->pytorch-lightning==1.1.5->nemo_toolkit[tts]) (0.36.2)
    Requirement already satisfied: tensorboard-data-server<0.7.0,>=0.6.0 in /home/ubuntu/.local/lib/python3.8/site-packages (from tensorboard>=2.2.0->pytorch-lightning==1.1.5->nemo_toolkit[tts]) (0.6.1)
    Requirement already satisfied: certifi>=2017.4.17 in /usr/lib/python3/dist-packages (from requests->torchtext==0.8.0->nemo_toolkit[tts]) (2018.1.18)
    Requirement already satisfied: charset-normalizer~=2.0.0 in /home/ubuntu/.local/lib/python3.8/site-packages (from requests->torchtext==0.8.0->nemo_toolkit[tts]) (2.0.9)
    Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/lib/python3/dist-packages (from requests->torchtext==0.8.0->nemo_toolkit[tts]) (1.22)
    Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3/dist-packages (from requests->torchtext==0.8.0->nemo_toolkit[tts]) (2.6)
    Requirement already satisfied: zipp>=3.1.0 in /home/ubuntu/.local/lib/python3.8/site-packages (from importlib-resources->hydra-core>=1.0.4->nemo_toolkit[tts]) (3.6.0)
    Requirement already satisfied: cachetools<5.0,>=2.0.0 in /home/ubuntu/.local/lib/python3.8/site-packages (from google-auth<3,>=1.6.3->tensorboard>=2.2.0->pytorch-lightning==1.1.5->nemo_toolkit[tts]) (4.2.2)
    Requirement already satisfied: rsa<5,>=3.1.4 in /home/ubuntu/.local/lib/python3.8/site-packages (from google-auth<3,>=1.6.3->tensorboard>=2.2.0->pytorch-lightning==1.1.5->nemo_toolkit[tts]) (4.8)
    Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/lib/python3/dist-packages (from google-auth<3,>=1.6.3->tensorboard>=2.2.0->pytorch-lightning==1.1.5->nemo_toolkit[tts]) (0.2.1)
    Requirement already satisfied: requests-oauthlib>=0.7.0 in /home/ubuntu/.local/lib/python3.8/site-packages (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard>=2.2.0->pytorch-lightning==1.1.5->nemo_toolkit[tts]) (1.3.0)
    Requirement already satisfied: importlib-metadata>=4.4 in /home/ubuntu/.local/lib/python3.8/site-packages (from markdown>=2.6.8->tensorboard>=2.2.0->pytorch-lightning==1.1.5->nemo_toolkit[tts]) (4.9.0)
    Requirement already satisfied: shellingham<2.0.0,>=1.3.0 in /home/ubuntu/.local/lib/python3.8/site-packages (from typer[all]>=0.2.1->pyannote.database>=4.0.1->pyannote.metrics->nemo_toolkit[tts]) (1.4.0)
    Requirement already satisfied: colorama<0.5.0,>=0.4.3 in /home/ubuntu/.local/lib/python3.8/site-packages (from typer[all]>=0.2.1->pyannote.database>=4.0.1->pyannote.metrics->nemo_toolkit[tts]) (0.4.4)
    Requirement already satisfied: aiosignal>=1.1.2 in /home/ubuntu/.local/lib/python3.8/site-packages (from aiohttp->fsspec[http]>=0.8.1->pytorch-lightning==1.1.5->nemo_toolkit[tts]) (1.2.0)
    Requirement already satisfied: frozenlist>=1.1.1 in /home/ubuntu/.local/lib/python3.8/site-packages (from aiohttp->fsspec[http]>=0.8.1->pytorch-lightning==1.1.5->nemo_toolkit[tts]) (1.2.0)
    Requirement already satisfied: attrs>=17.3.0 in /usr/lib/python3/dist-packages (from aiohttp->fsspec[http]>=0.8.1->pytorch-lightning==1.1.5->nemo_toolkit[tts]) (17.4.0)
    Requirement already satisfied: multidict<7.0,>=4.5 in /home/ubuntu/.local/lib/python3.8/site-packages (from aiohttp->fsspec[http]>=0.8.1->pytorch-lightning==1.1.5->nemo_toolkit[tts]) (5.2.0)
    Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /home/ubuntu/.local/lib/python3.8/site-packages (from aiohttp->fsspec[http]>=0.8.1->pytorch-lightning==1.1.5->nemo_toolkit[tts]) (4.0.1)
    Requirement already satisfied: yarl<2.0,>=1.0 in /home/ubuntu/.local/lib/python3.8/site-packages (from aiohttp->fsspec[http]>=0.8.1->pytorch-lightning==1.1.5->nemo_toolkit[tts]) (1.7.2)
    Requirement already satisfied: oauthlib>=3.0.0 in /home/ubuntu/.local/lib/python3.8/site-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard>=2.2.0->pytorch-lightning==1.1.5->nemo_toolkit[tts]) (3.1.1)
    Requirement already satisfied: pyasn1>=0.1.3 in /usr/lib/python3/dist-packages (from rsa<5,>=3.1.4->google-auth<3,>=1.6.3->tensorboard>=2.2.0->pytorch-lightning==1.1.5->nemo_toolkit[tts]) (0.4.2)

```python
# Imorting the libraries
import os
import pandas as pd
import re
import librosa
from datasets import Dataset
```

# Loading the Dataset

```python
# defining the root directory
data_directory = "../Badaga_Corpus-v.0.1.0/"
tagged_file = "Badaga-v0.1.0.xlsx"
```

```python
# loading the data
tagged_file_path = os.path.join(data_directory, tagged_file)
data_frame = pd.read_excel(tagged_file_path)

# droping the missing values
data_frame.dropna(inplace=True)

gender = "F" # "M", "F"

# loading the audio files
data_frame["audio_file_name"] = data_frame["audio_file_name"].apply(lambda x: os.path.join(data_directory, "clips", x))
data_frame = data_frame[data_frame["gender"]==gender]

# splitting the data to train and test using the aplit_index from transcription
train_df = data_frame[data_frame["split_label"]!="test"]
test_df = data_frame[data_frame["split_label"]=="test"]
```

```python
# printing the data
train_df
```

<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

    .dataframe tbody tr th {
        vertical-align: top;
    }

    .dataframe thead th {
        text-align: right;
    }
</style>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>items</th>
      <th>translated_transcript</th>
      <th>audio_file_name</th>
      <th>translterated_script</th>
      <th>user_id</th>
      <th>gender</th>
      <th>locale</th>
      <th>split_label</th>
      <th>duration</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <th>0</th>
      <td>1.0</td>
      <td>what is the recipe in home</td>
      <td>../Badaga_Corpus-v.0.1.0/clips/F002_1_1.mp3</td>
      <td>manaya aena udhaka</td>
      <td>F002</td>
      <td>F</td>
      <td>ba</td>
      <td>train</td>
      <td>2.377187</td>
    </tr>
    <tr>
      <th>1</th>
      <td>2.0</td>
      <td>who is there near you</td>
      <td>../Badaga_Corpus-v.0.1.0/clips/F002_1_2.mp3</td>
      <td>pakka dhara edhdharae</td>
      <td>F002</td>
      <td>F</td>
      <td>ba</td>
      <td>train</td>
      <td>2.377187</td>
    </tr>
    <tr>
      <th>2</th>
      <td>3.0</td>
      <td>what did you prepare for lunch</td>
      <td>../Badaga_Corpus-v.0.1.0/clips/F002_1_3.mp3</td>
      <td>hagulu hasuga aena maditha</td>
      <td>F002</td>
      <td>F</td>
      <td>ba</td>
      <td>train</td>
      <td>2.351062</td>
    </tr>
    <tr>
      <th>3</th>
      <td>4.0</td>
      <td>did you brush</td>
      <td>../Badaga_Corpus-v.0.1.0/clips/F002_1_4.mp3</td>
      <td>hallu ujjithaya</td>
      <td>F002</td>
      <td>F</td>
      <td>ba</td>
      <td>train</td>
      <td>2.194313</td>
    </tr>
    <tr>
      <th>4</th>
      <td>5.0</td>
      <td>did you eat</td>
      <td>../Badaga_Corpus-v.0.1.0/clips/F002_1_5.mp3</td>
      <td>nee thindhubutaya</td>
      <td>F002</td>
      <td>F</td>
      <td>ba</td>
      <td>train</td>
      <td>2.272687</td>
    </tr>
    <tr>
      <th>...</th>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
      <td>...</td>
    </tr>
    <tr>
      <th>9830</th>
      <td>593.0</td>
      <td>i dont cheat anybody</td>
      <td>../Badaga_Corpus-v.0.1.0/clips/F004_593.mp3</td>
      <td>na tharuvavu aemaathathu ellae</td>
      <td>F004</td>
      <td>F</td>
      <td>ba</td>
      <td>train</td>
      <td>2.011437</td>
    </tr>
    <tr>
      <th>9831</th>
      <td>594.0</td>
      <td>today it is so windy</td>
      <td>../Badaga_Corpus-v.0.1.0/clips/F004_594.mp3</td>
      <td>gaai jaasthi endhdhu</td>
      <td>F004</td>
      <td>F</td>
      <td>ba</td>
      <td>train</td>
      <td>1.358375</td>
    </tr>
    <tr>
      <th>9834</th>
      <td>597.0</td>
      <td>today is my birthday</td>
      <td>../Badaga_Corpus-v.0.1.0/clips/F004_597.mp3</td>
      <td>endhdhu aenna utti jaenna</td>
      <td>F004</td>
      <td>F</td>
      <td>ba</td>
      <td>train</td>
      <td>1.541250</td>
    </tr>
    <tr>
      <th>9835</th>
      <td>598.0</td>
      <td>we are coming tomorrow.</td>
      <td>../Badaga_Corpus-v.0.1.0/clips/F004_598.mp3</td>
      <td>enga naaiga bannaeyo</td>
      <td>F004</td>
      <td>F</td>
      <td>ba</td>
      <td>train</td>
      <td>1.724125</td>
    </tr>
    <tr>
      <th>9836</th>
      <td>599.0</td>
      <td>two o clock we will go</td>
      <td>../Badaga_Corpus-v.0.1.0/clips/F004_599.mp3</td>
      <td>aeradu manniga oppa</td>
      <td>F004</td>
      <td>F</td>
      <td>ba</td>
      <td>train</td>
      <td>1.306125</td>
    </tr>
  </tbody>
</table>
<p>4193 rows × 9 columns</p>
</div>

```python
# creating the dictionary for both train and test and storing it as a json file
import json

audio_paths = list(train_df["audio_file_name"])
durations = list(train_df["duration"])
labels = list(train_df["translterated_script"])

my_dict = dict()
i = 0
with open("files/tts_train.json", "w") as outfile:
    for path, dur, label in zip(audio_paths, durations, labels):
        my_dict = {"audio_filepath": path,
                "duration": dur,
                "text": label}
        json_object = json.dumps(my_dict)
        outfile.write(json_object + "\n")

audio_paths = list(test_df["audio_file_name"])
durations = list(test_df["duration"])
labels = list(test_df["translterated_script"])

my_dict = dict()
i = 0
with open("files/tts_test.json", "w") as outfile:
    for path, dur, label in zip(audio_paths, durations, labels):
        my_dict = {"audio_filepath": path,
                "duration": dur,
                "text": label}
        json_object = json.dumps(my_dict)
        outfile.write(json_object + "\n")
```

# Loading the Model

```python
# loading the Tactron model and training
directory = "tactron2-rbg-badaga-tts-"+gender.lower()

if os.path.exists(directory):
    pass
else:
    os.makedirs(directory)

!python3.8 tacotron2.py sample_rate=16000 train_dataset=files/tts_train.json validation_datasets=files/tts_test.json exp_manager.exp_dir=$directory trainer.max_epochs=20 trainer.accelerator=null trainer.check_val_every_n_epoch=1

```

    [NeMo W 2021-12-18 15:16:08 experimental:27] Module <class 'nemo.collections.asr.data.audio_to_text_dali.AudioToCharDALIDataset'> is experimental, not ready for production and is not fully supported. Use at your own risk.
    ################################################################################
    ### WARNING, path does not exist: KALDI_ROOT=/mnt/matylda5/iveselyk/Tools/kaldi-trunk
    ###          (please add 'export KALDI_ROOT=<your_path>' in your $HOME/.profile)
    ###          (or run as: KALDI_ROOT=<your_path> python <your_script>.py)
    ################################################################################

    [NeMo W 2021-12-18 15:16:08 nemo_logging:349] /home/ubuntu/.local/lib/python3.8/site-packages/torchaudio/backend/utils.py:53: UserWarning: "sox" backend is being deprecated. The default backend will be changed to "sox_io" backend in 0.8.0 and "sox" backend will be removed in 0.9.0. Please migrate to "sox_io" backend. Please refer to https://github.com/pytorch/audio/issues/903 for the detail.
          warnings.warn(

    GPU available: True, used: True
    GPU available: True, used: True
    TPU available: None, using: 0 TPU cores
    TPU available: None, using: 0 TPU cores
    LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
    LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
    [NeMo I 2021-12-18 15:16:09 exp_manager:208] Experiments will be logged at tactron2-rbg-badaga-tts-f/Tacotron2/2021-12-18_15-16-09
    [NeMo I 2021-12-18 15:16:09 exp_manager:548] TensorboardLogger has been set up
    [NeMo I 2021-12-18 15:16:09 collections:173] Dataset loaded with 4193 files totalling 2.81 hours
    [NeMo I 2021-12-18 15:16:09 collections:174] 0 files were filtered totalling 0.00 hours
    [NeMo I 2021-12-18 15:16:09 collections:173] Dataset loaded with 735 files totalling 0.48 hours
    [NeMo I 2021-12-18 15:16:09 collections:174] 0 files were filtered totalling 0.00 hours
    [NeMo I 2021-12-18 15:16:09 features:240] PADDING: 16
    [NeMo I 2021-12-18 15:16:09 features:249] STFT using conv
    {'labels': '${labels}', 'train_ds': {'dataset': {'_target_': 'nemo.collections.asr.data.audio_to_text.AudioToCharDataset', 'manifest_filepath': '${train_dataset}', 'max_duration': None, 'min_duration': 0.1, 'trim': False, 'int_values': False, 'load_audio': True, 'normalize': False, 'sample_rate': '${sample_rate}'}, 'dataloader_params': {'drop_last': False, 'shuffle': True, 'batch_size': 8, 'num_workers': 4}}, 'validation_ds': {'dataset': {'_target_': 'nemo.collections.asr.data.audio_to_text.AudioToCharDataset', 'manifest_filepath': '${validation_datasets}', 'max_duration': None, 'min_duration': 0.1, 'int_values': False, 'load_audio': True, 'normalize': False, 'sample_rate': '${sample_rate}', 'trim': False}, 'dataloader_params': {'drop_last': False, 'shuffle': False, 'batch_size': 8, 'num_workers': 8}}, 'preprocessor': {'_target_': 'nemo.collections.asr.parts.features.FilterbankFeatures', 'dither': 0.0, 'nfilt': '${n_mels}', 'frame_splicing': 1, 'highfreq': '${fmax}', 'log': True, 'log_zero_guard_type': 'clamp', 'log_zero_guard_value': 1e-05, 'lowfreq': 0, 'mag_power': 1.0, 'n_fft': '${n_fft}', 'n_window_size': 1024, 'n_window_stride': '${n_stride}', 'normalize': None, 'pad_to': 16, 'pad_value': '${pad_value}', 'preemph': None, 'sample_rate': '${sample_rate}', 'stft_conv': True, 'window': 'hann'}, 'encoder': {'_target_': 'nemo.collections.tts.modules.tacotron2.Encoder', 'encoder_kernel_size': 5, 'encoder_n_convolutions': 3, 'encoder_embedding_dim': 512}, 'decoder': {'_target_': 'nemo.collections.tts.modules.tacotron2.Decoder', 'decoder_rnn_dim': 1024, 'encoder_embedding_dim': '${model.encoder.encoder_embedding_dim}', 'gate_threshold': 0.5, 'max_decoder_steps': 1000, 'n_frames_per_step': 1, 'n_mel_channels': '${n_mels}', 'p_attention_dropout': 0.1, 'p_decoder_dropout': 0.1, 'prenet_dim': 256, 'prenet_p_dropout': 0.5, 'attention_dim': 128, 'attention_rnn_dim': 1024, 'attention_location_kernel_size': 31, 'attention_location_n_filters': 32, 'early_stopping': True}, 'postnet': {'_target_': 'nemo.collections.tts.modules.tacotron2.Postnet', 'n_mel_channels': '${n_mels}', 'p_dropout': 0.5, 'postnet_embedding_dim': 512, 'postnet_kernel_size': 5, 'postnet_n_convolutions': 5}, 'optim': {'name': 'adam', 'lr': 0.001, 'weight_decay': 1e-06}}
    [NeMo I 2021-12-18 15:16:13 modelPT:685] Optimizer config = Adam (
        Parameter Group 0
            amsgrad: False
            betas: (0.9, 0.999)
            eps: 1e-08
            lr: 0.001
            weight_decay: 1e-06
        )
    [NeMo I 2021-12-18 15:16:13 lr_scheduler:492] Scheduler not initialized as no `sched` config supplied to setup_optimizer()

      | Name                       | Type               | Params
    ------------------------------------------------------------------
    0 | audio_to_melspec_precessor | FilterbankFeatures | 0
    1 | text_embedding             | Embedding          | 35.3 K
    2 | encoder                    | Encoder            | 5.5 M
    3 | decoder                    | Decoder            | 18.3 M
    4 | postnet                    | Postnet            | 4.3 M
    5 | loss                       | Tacotron2Loss      | 0
    ------------------------------------------------------------------
    28.2 M    Trainable params
    0         Non-trainable params
    28.2 M    Total params

      | Name                       | Type               | Params
    ------------------------------------------------------------------
    0 | audio_to_melspec_precessor | FilterbankFeatures | 0
    1 | text_embedding             | Embedding          | 35.3 K
    2 | encoder                    | Encoder            | 5.5 M
    3 | decoder                    | Decoder            | 18.3 M
    4 | postnet                    | Postnet            | 4.3 M
    5 | loss                       | Tacotron2Loss      | 0
    ------------------------------------------------------------------
    28.2 M    Trainable params
    0         Non-trainable params
    28.2 M    Total params
    Validation sanity check:   0%|                            | 0/2 [00:00<?, ?it/s][NeMo W 2021-12-18 15:16:21 tacotron2:366] Reached max decoder steps 1000.
    Validation sanity check:  50%|██████████          | 1/2 [00:07<00:07,  7.96s/it][NeMo W 2021-12-18 15:16:22 tacotron2:366] Reached max decoder steps 1000.
    Training: 0it [00:00, ?it/s][NeMo W 2021-12-18 15:16:25 nemo_logging:349] /home/ubuntu/.local/lib/python3.8/site-packages/pytorch_lightning/utilities/distributed.py:49: RuntimeWarning: You are using `LearningRateMonitor` callback with models that have no learning rate schedulers. Please see documentation for `configure_optimizers` method.
          warnings.warn(*args, **kwargs)

    Epoch 0:   0%|                                          | 0/617 [00:00<?, ?it/s][NeMo W 2021-12-18 15:16:29 nemo_logging:349] /home/ubuntu/.local/lib/python3.8/site-packages/pytorch_lightning/utilities/distributed.py:49: UserWarning: The {progress_bar:dict keyword} was deprecated in 0.9.1 and will be removed in 1.0.0
        Please use self.log(...) inside the lightningModule instead.

        # log on a step or aggregate epoch metric to the logger and/or progress bar
        # (inside LightningModule)
        self.log('train_loss', loss, on_step=True, on_epoch=True, prog_bar=True)
          warnings.warn(*args, **kwargs)

    Epoch 0:  85%|▊| 525/617 [07:47<01:21,  1.12it/s, loss=2, v_num=6-09, training_l
    Validating: 0it [00:00, ?it/s]
    Validating:   0%|                                        | 0/92 [00:00<?, ?it/s][NeMo W 2021-12-18 15:24:20 tacotron2:366] Reached max decoder steps 1000.

    Epoch 0:  85%|▊| 527/617 [07:55<01:21,  1.11it/s, loss=2, v_num=6-09, training_l[NeMo W 2021-12-18 15:24:22 tacotron2:366] Reached max decoder steps 1000.

    Validating:   2%|▋                               | 2/92 [00:09<06:34,  4.39s/it][NeMo W 2021-12-18 15:24:24 tacotron2:366] Reached max decoder steps 1000.

    Epoch 0:  86%|▊| 529/617 [07:59<01:19,  1.10it/s, loss=2, v_num=6-09, training_l[NeMo W 2021-12-18 15:24:25 tacotron2:366] Reached max decoder steps 1000.

    Validating:   4%|█▍                              | 4/92 [00:13<03:48,  2.59s/it][NeMo W 2021-12-18 15:24:27 tacotron2:366] Reached max decoder steps 1000.

    Epoch 0:  86%|▊| 531/617 [08:02<01:18,  1.10it/s, loss=2, v_num=6-09, training_l[NeMo W 2021-12-18 15:24:29 tacotron2:366] Reached max decoder steps 1000.

    Validating:   7%|██                              | 6/92 [00:16<02:53,  2.02s/it][NeMo W 2021-12-18 15:24:30 tacotron2:366] Reached max decoder steps 1000.

    Epoch 0:  86%|▊| 533/617 [08:05<01:16,  1.10it/s, loss=2, v_num=6-09, training_l[NeMo W 2021-12-18 15:24:31 tacotron2:366] Reached max decoder steps 1000.

    Validating:   9%|██▊                             | 8/92 [00:19<02:23,  1.71s/it][NeMo W 2021-12-18 15:24:33 tacotron2:366] Reached max decoder steps 1000.

    Epoch 0:  87%|▊| 535/617 [08:08<01:14,  1.10it/s, loss=2, v_num=6-09, training_l[NeMo W 2021-12-18 15:24:34 tacotron2:366] Reached max decoder steps 1000.

    Validating:  11%|███▎                           | 10/92 [00:22<02:09,  1.57s/it][NeMo W 2021-12-18 15:24:36 tacotron2:366] Reached max decoder steps 1000.

    Epoch 0:  87%|▊| 537/617 [08:11<01:13,  1.09it/s, loss=2, v_num=6-09, training_l[NeMo W 2021-12-18 15:24:37 tacotron2:366] Reached max decoder steps 1000.

    Validating:  13%|████                           | 12/92 [00:25<02:00,  1.50s/it][NeMo W 2021-12-18 15:24:39 tacotron2:366] Reached max decoder steps 1000.

    Epoch 0:  87%|▊| 539/617 [08:14<01:11,  1.09it/s, loss=2, v_num=6-09, training_l[NeMo W 2021-12-18 15:24:40 tacotron2:366] Reached max decoder steps 1000.

    Validating:  15%|████▋                          | 14/92 [00:28<01:54,  1.47s/it][NeMo W 2021-12-18 15:24:42 tacotron2:366] Reached max decoder steps 1000.

    Epoch 0:  88%|▉| 541/617 [08:16<01:09,  1.09it/s, loss=2, v_num=6-09, training_l[NeMo W 2021-12-18 15:24:43 tacotron2:366] Reached max decoder steps 1000.

    Validating:  17%|█████▍                         | 16/92 [00:31<01:50,  1.46s/it][NeMo W 2021-12-18 15:24:44 tacotron2:366] Reached max decoder steps 1000.

    Epoch 0:  88%|▉| 543/617 [08:19<01:08,  1.09it/s, loss=2, v_num=6-09, training_l[NeMo W 2021-12-18 15:24:46 tacotron2:366] Reached max decoder steps 1000.

    Validating:  20%|██████                         | 18/92 [00:33<01:47,  1.45s/it][NeMo W 2021-12-18 15:24:47 tacotron2:366] Reached max decoder steps 1000.

    Epoch 0:  88%|▉| 545/617 [08:22<01:06,  1.08it/s, loss=2, v_num=6-09, training_l[NeMo W 2021-12-18 15:24:49 tacotron2:366] Reached max decoder steps 1000.

    Validating:  22%|██████▋                        | 20/92 [00:36<01:43,  1.44s/it][NeMo W 2021-12-18 15:24:50 tacotron2:366] Reached max decoder steps 1000.

    Epoch 0:  89%|▉| 547/617 [08:25<01:04,  1.08it/s, loss=2, v_num=6-09, training_l[NeMo W 2021-12-18 15:24:52 tacotron2:366] Reached max decoder steps 1000.

    Validating:  24%|███████▍                       | 22/92 [00:39<01:40,  1.44s/it][NeMo W 2021-12-18 15:24:53 tacotron2:366] Reached max decoder steps 1000.

    Epoch 0:  89%|▉| 549/617 [08:28<01:02,  1.08it/s, loss=2, v_num=6-09, training_l[NeMo W 2021-12-18 15:24:54 tacotron2:366] Reached max decoder steps 1000.

    Validating:  26%|████████                       | 24/92 [00:42<01:37,  1.44s/it][NeMo W 2021-12-18 15:24:56 tacotron2:366] Reached max decoder steps 1000.

    Epoch 0:  89%|▉| 551/617 [08:31<01:01,  1.08it/s, loss=2, v_num=6-09, training_l[NeMo W 2021-12-18 15:24:57 tacotron2:366] Reached max decoder steps 1000.

    Validating:  28%|████████▊                      | 26/92 [00:45<01:35,  1.44s/it][NeMo W 2021-12-18 15:24:59 tacotron2:366] Reached max decoder steps 1000.

    Epoch 0:  90%|▉| 553/617 [08:34<00:59,  1.08it/s, loss=2, v_num=6-09, training_l[NeMo W 2021-12-18 15:25:00 tacotron2:366] Reached max decoder steps 1000.

    Validating:  30%|█████████▍                     | 28/92 [00:48<01:32,  1.44s/it][NeMo W 2021-12-18 15:25:02 tacotron2:366] Reached max decoder steps 1000.

    Epoch 0:  90%|▉| 555/617 [08:37<00:57,  1.07it/s, loss=2, v_num=6-09, training_l[NeMo W 2021-12-18 15:25:03 tacotron2:366] Reached max decoder steps 1000.

    Validating:  33%|██████████                     | 30/92 [00:51<01:29,  1.44s/it][NeMo W 2021-12-18 15:25:05 tacotron2:366] Reached max decoder steps 1000.

    Epoch 0:  90%|▉| 557/617 [08:39<00:56,  1.07it/s, loss=2, v_num=6-09, training_l[NeMo W 2021-12-18 15:25:06 tacotron2:366] Reached max decoder steps 1000.

    Validating:  35%|██████████▊                    | 32/92 [00:54<01:26,  1.44s/it][NeMo W 2021-12-18 15:25:07 tacotron2:366] Reached max decoder steps 1000.

    Epoch 0:  91%|▉| 559/617 [08:42<00:54,  1.07it/s, loss=2, v_num=6-09, training_l[NeMo W 2021-12-18 15:25:09 tacotron2:366] Reached max decoder steps 1000.

    Validating:  37%|███████████▍                   | 34/92 [00:56<01:23,  1.44s/it][NeMo W 2021-12-18 15:25:10 tacotron2:366] Reached max decoder steps 1000.

    Epoch 0:  91%|▉| 561/617 [08:45<00:52,  1.07it/s, loss=2, v_num=6-09, training_l[NeMo W 2021-12-18 15:25:12 tacotron2:366] Reached max decoder steps 1000.

    Validating:  39%|████████████▏                  | 36/92 [00:59<01:22,  1.47s/it][NeMo W 2021-12-18 15:25:13 tacotron2:366] Reached max decoder steps 1000.

    Epoch 0:  91%|▉| 563/617 [08:48<00:50,  1.06it/s, loss=2, v_num=6-09, training_l[NeMo W 2021-12-18 15:25:15 tacotron2:366] Reached max decoder steps 1000.

    Validating:  41%|████████████▊                  | 38/92 [01:02<01:18,  1.46s/it][NeMo W 2021-12-18 15:25:16 tacotron2:366] Reached max decoder steps 1000.

    Epoch 0:  92%|▉| 565/617 [08:51<00:48,  1.06it/s, loss=2, v_num=6-09, training_l[NeMo W 2021-12-18 15:25:18 tacotron2:366] Reached max decoder steps 1000.

    Validating:  43%|█████████████▍                 | 40/92 [01:05<01:15,  1.45s/it][NeMo W 2021-12-18 15:25:19 tacotron2:366] Reached max decoder steps 1000.

    Epoch 0:  92%|▉| 567/617 [08:54<00:47,  1.06it/s, loss=2, v_num=6-09, training_l[NeMo W 2021-12-18 15:25:21 tacotron2:366] Reached max decoder steps 1000.

    Validating:  46%|██████████████▏                | 42/92 [01:08<01:12,  1.44s/it][NeMo W 2021-12-18 15:25:22 tacotron2:366] Reached max decoder steps 1000.

    Epoch 0:  92%|▉| 569/617 [08:57<00:45,  1.06it/s, loss=2, v_num=6-09, training_l[NeMo W 2021-12-18 15:25:23 tacotron2:366] Reached max decoder steps 1000.

    Validating:  48%|██████████████▊                | 44/92 [01:11<01:09,  1.44s/it][NeMo W 2021-12-18 15:25:25 tacotron2:366] Reached max decoder steps 1000.

    Epoch 0:  93%|▉| 571/617 [09:00<00:43,  1.06it/s, loss=2, v_num=6-09, training_l[NeMo W 2021-12-18 15:25:26 tacotron2:366] Reached max decoder steps 1000.

    Validating:  50%|███████████████▌               | 46/92 [01:14<01:06,  1.44s/it][NeMo W 2021-12-18 15:25:28 tacotron2:366] Reached max decoder steps 1000.

    Epoch 0:  93%|▉| 573/617 [09:03<00:41,  1.05it/s, loss=2, v_num=6-09, training_l[NeMo W 2021-12-18 15:25:29 tacotron2:366] Reached max decoder steps 1000.

    Validating:  52%|████████████████▏              | 48/92 [01:17<01:03,  1.44s/it][NeMo W 2021-12-18 15:25:31 tacotron2:366] Reached max decoder steps 1000.

    Epoch 0:  93%|▉| 575/617 [09:06<00:39,  1.05it/s, loss=2, v_num=6-09, training_l[NeMo W 2021-12-18 15:25:32 tacotron2:366] Reached max decoder steps 1000.

    Validating:  54%|████████████████▊              | 50/92 [01:20<01:00,  1.44s/it][NeMo W 2021-12-18 15:25:33 tacotron2:366] Reached max decoder steps 1000.

    Epoch 0:  94%|▉| 577/617 [09:08<00:38,  1.05it/s, loss=2, v_num=6-09, training_l[NeMo W 2021-12-18 15:25:35 tacotron2:366] Reached max decoder steps 1000.

    Validating:  57%|█████████████████▌             | 52/92 [01:23<00:57,  1.44s/it][NeMo W 2021-12-18 15:25:36 tacotron2:366] Reached max decoder steps 1000.

    Epoch 0:  94%|▉| 579/617 [09:11<00:36,  1.05it/s, loss=2, v_num=6-09, training_l[NeMo W 2021-12-18 15:25:38 tacotron2:366] Reached max decoder steps 1000.

    Validating:  59%|██████████████████▏            | 54/92 [01:25<00:54,  1.44s/it][NeMo W 2021-12-18 15:25:39 tacotron2:366] Reached max decoder steps 1000.

    Epoch 0:  94%|▉| 581/617 [09:14<00:34,  1.05it/s, loss=2, v_num=6-09, training_l[NeMo W 2021-12-18 15:25:41 tacotron2:366] Reached max decoder steps 1000.

    Validating:  61%|██████████████████▊            | 56/92 [01:28<00:52,  1.45s/it][NeMo W 2021-12-18 15:25:42 tacotron2:366] Reached max decoder steps 1000.

    Epoch 0:  94%|▉| 583/617 [09:17<00:32,  1.05it/s, loss=2, v_num=6-09, training_l[NeMo W 2021-12-18 15:25:44 tacotron2:366] Reached max decoder steps 1000.

    Validating:  63%|███████████████████▌           | 58/92 [01:31<00:49,  1.45s/it][NeMo W 2021-12-18 15:25:45 tacotron2:366] Reached max decoder steps 1000.

    Epoch 0:  95%|▉| 585/617 [09:20<00:30,  1.04it/s, loss=2, v_num=6-09, training_l[NeMo W 2021-12-18 15:25:47 tacotron2:366] Reached max decoder steps 1000.

    Validating:  65%|████████████████████▏          | 60/92 [01:34<00:46,  1.45s/it][NeMo W 2021-12-18 15:25:48 tacotron2:366] Reached max decoder steps 1000.

    Epoch 0:  95%|▉| 587/617 [09:23<00:28,  1.04it/s, loss=2, v_num=6-09, training_l[NeMo W 2021-12-18 15:25:49 tacotron2:366] Reached max decoder steps 1000.

    Validating:  67%|████████████████████▉          | 62/92 [01:37<00:43,  1.45s/it][NeMo W 2021-12-18 15:25:51 tacotron2:366] Reached max decoder steps 1000.

    Epoch 0:  95%|▉| 589/617 [09:26<00:26,  1.04it/s, loss=2, v_num=6-09, training_l[NeMo W 2021-12-18 15:25:52 tacotron2:366] Reached max decoder steps 1000.

    Validating:  70%|█████████████████████▌         | 64/92 [01:40<00:40,  1.46s/it][NeMo W 2021-12-18 15:25:54 tacotron2:366] Reached max decoder steps 1000.

    Epoch 0:  96%|▉| 591/617 [09:29<00:25,  1.04it/s, loss=2, v_num=6-09, training_l[NeMo W 2021-12-18 15:25:55 tacotron2:366] Reached max decoder steps 1000.

    Validating:  72%|██████████████████████▏        | 66/92 [01:43<00:37,  1.45s/it][NeMo W 2021-12-18 15:25:57 tacotron2:366] Reached max decoder steps 1000.

    Epoch 0:  96%|▉| 593/617 [09:32<00:23,  1.04it/s, loss=2, v_num=6-09, training_l[NeMo W 2021-12-18 15:25:58 tacotron2:366] Reached max decoder steps 1000.

    Validating:  74%|██████████████████████▉        | 68/92 [01:46<00:34,  1.45s/it][NeMo W 2021-12-18 15:26:00 tacotron2:366] Reached max decoder steps 1000.

    Epoch 0:  96%|▉| 595/617 [09:34<00:21,  1.03it/s, loss=2, v_num=6-09, training_l[NeMo W 2021-12-18 15:26:01 tacotron2:366] Reached max decoder steps 1000.

    Validating:  76%|███████████████████████▌       | 70/92 [01:49<00:31,  1.45s/it][NeMo W 2021-12-18 15:26:02 tacotron2:366] Reached max decoder steps 1000.

    Epoch 0:  97%|▉| 597/617 [09:37<00:19,  1.03it/s, loss=2, v_num=6-09, training_l[NeMo W 2021-12-18 15:26:04 tacotron2:366] Reached max decoder steps 1000.

    Validating:  78%|████████████████████████▎      | 72/92 [01:51<00:28,  1.45s/it][NeMo W 2021-12-18 15:26:05 tacotron2:366] Reached max decoder steps 1000.

    Epoch 0:  97%|▉| 599/617 [09:40<00:17,  1.03it/s, loss=2, v_num=6-09, training_l[NeMo W 2021-12-18 15:26:07 tacotron2:366] Reached max decoder steps 1000.

    Validating:  80%|████████████████████████▉      | 74/92 [01:54<00:26,  1.44s/it][NeMo W 2021-12-18 15:26:08 tacotron2:366] Reached max decoder steps 1000.

    Epoch 0:  97%|▉| 601/617 [09:43<00:15,  1.03it/s, loss=2, v_num=6-09, training_l[NeMo W 2021-12-18 15:26:10 tacotron2:366] Reached max decoder steps 1000.

    Validating:  83%|█████████████████████████▌     | 76/92 [01:57<00:23,  1.44s/it][NeMo W 2021-12-18 15:26:11 tacotron2:366] Reached max decoder steps 1000.

    Epoch 0:  98%|▉| 603/617 [09:46<00:13,  1.03it/s, loss=2, v_num=6-09, training_l[NeMo W 2021-12-18 15:26:13 tacotron2:366] Reached max decoder steps 1000.

    Validating:  85%|██████████████████████████▎    | 78/92 [02:00<00:20,  1.44s/it][NeMo W 2021-12-18 15:26:14 tacotron2:366] Reached max decoder steps 1000.

    Epoch 0:  98%|▉| 605/617 [09:49<00:11,  1.03it/s, loss=2, v_num=6-09, training_l[NeMo W 2021-12-18 15:26:15 tacotron2:366] Reached max decoder steps 1000.

    Validating:  87%|██████████████████████████▉    | 80/92 [02:03<00:17,  1.43s/it][NeMo W 2021-12-18 15:26:17 tacotron2:366] Reached max decoder steps 1000.

    Epoch 0:  98%|▉| 607/617 [09:52<00:09,  1.02it/s, loss=2, v_num=6-09, training_l[NeMo W 2021-12-18 15:26:18 tacotron2:366] Reached max decoder steps 1000.

    Validating:  89%|███████████████████████████▋   | 82/92 [02:06<00:14,  1.43s/it][NeMo W 2021-12-18 15:26:20 tacotron2:366] Reached max decoder steps 1000.

    Epoch 0:  99%|▉| 609/617 [09:55<00:07,  1.02it/s, loss=2, v_num=6-09, training_l[NeMo W 2021-12-18 15:26:21 tacotron2:366] Reached max decoder steps 1000.

    Validating:  91%|████████████████████████████▎  | 84/92 [02:09<00:11,  1.43s/it][NeMo W 2021-12-18 15:26:23 tacotron2:366] Reached max decoder steps 1000.

    Epoch 0:  99%|▉| 611/617 [09:57<00:05,  1.02it/s, loss=2, v_num=6-09, training_l[NeMo W 2021-12-18 15:26:24 tacotron2:366] Reached max decoder steps 1000.

    Validating:  93%|████████████████████████████▉  | 86/92 [02:12<00:08,  1.42s/it][NeMo W 2021-12-18 15:26:25 tacotron2:366] Reached max decoder steps 1000.

    Epoch 0:  99%|▉| 613/617 [10:00<00:03,  1.02it/s, loss=2, v_num=6-09, training_l[NeMo W 2021-12-18 15:26:27 tacotron2:366] Reached max decoder steps 1000.

    Validating:  96%|█████████████████████████████▋ | 88/92 [02:14<00:05,  1.42s/it][NeMo W 2021-12-18 15:26:28 tacotron2:366] Reached max decoder steps 1000.

    Epoch 0: 100%|▉| 615/617 [10:03<00:01,  1.02it/s, loss=2, v_num=6-09, training_l[NeMo W 2021-12-18 15:26:30 tacotron2:366] Reached max decoder steps 1000.

    Validating:  98%|██████████████████████████████▎| 90/92 [02:17<00:02,  1.42s/it][NeMo W 2021-12-18 15:26:31 tacotron2:366] Reached max decoder steps 1000.

    Epoch 0: 100%|█| 617/617 [10:06<00:00,  1.02it/s, loss=2, v_num=6-09, training_l[NeMo W 2021-12-18 15:26:32 tacotron2:366] Reached max decoder steps 1000.

    Epoch 0: 100%|█| 617/617 [10:08<00:00,  1.01it/s, loss=2, v_num=6-09, training_l
                                                                                Epoch 0, global step 524: val_loss reached 19.76913 (best 19.76913), saving model to "/home/ubuntu/CoRePooL/benchmarks/badaga/notebooks/tactron2-rbg-badaga-tts-f/Tacotron2/2021-12-18_15-16-09/checkpoints/Tacotron2---val_loss=19.77-epoch=0.ckpt" as top 3
    Epoch 0, global step 524: val_loss reached 19.76913 (best 19.76913), saving model to "/home/ubuntu/CoRePooL/benchmarks/badaga/notebooks/tactron2-rbg-badaga-tts-f/Tacotron2/2021-12-18_15-16-09/checkpoints/Tacotron2---val_loss=19.77-epoch=0.ckpt" as top 3
    Epoch 1:  77%|▊| 478/617 [07:10<02:05,  1.11it/s, loss=1.6, v_num=6-09, training