```python
!cd content/CMU/*
```

    /bin/bash: line 0: cd: content/CMU/*: No such file or directory

```python
!git clone https://github.com/resemble-ai/Resemblyzer.git
```

    Cloning into 'Resemblyzer'...
    remote: Enumerating objects: 602, done.
    remote: Total 602 (delta 0), reused 0 (delta 0), pack-reused 602
    Receiving objects: 100% (602/602), 101.46 MiB | 36.15 MiB/s, done.
    Resolving deltas: 100% (107/107), done.

```python
%cd /content/Resemblyzer/
```

    /content/Resemblyzer

```python
!pip install -r  /content/Resemblyzer/requirements_demos.txt
```

    Requirement already satisfied: librosa>=0.6.1 in /usr/local/lib/python3.7/dist-packages (from -r /content/Resemblyzer/requirements_demos.txt (line 1)) (0.8.1)
    Requirement already satisfied: numpy>=1.10.1 in /usr/local/lib/python3.7/dist-packages (from -r /content/Resemblyzer/requirements_demos.txt (line 2)) (1.19.5)
    Collecting webrtcvad>=2.0.10
      Downloading webrtcvad-2.0.10.tar.gz (66 kB)
         |████████████████████████████████| 66 kB 2.4 MB/s
    Requirement already satisfied: torch>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from -r /content/Resemblyzer/requirements_demos.txt (line 4)) (1.10.0+cu111)
    Requirement already satisfied: scipy>=1.2.1 in /usr/local/lib/python3.7/dist-packages (from -r /content/Resemblyzer/requirements_demos.txt (line 5)) (1.4.1)
    Collecting typing
      Downloading typing-3.7.4.3.tar.gz (78 kB)
         |████████████████████████████████| 78 kB 5.4 MB/s
    Requirement already satisfied: matplotlib>=3.0.0 in /usr/local/lib/python3.7/dist-packages (from -r /content/Resemblyzer/requirements_demos.txt (line 7)) (3.2.2)
    Collecting sounddevice
      Downloading sounddevice-0.4.4-py3-none-any.whl (31 kB)
    Requirement already satisfied: tqdm in /usr/local/lib/python3.7/dist-packages (from -r /content/Resemblyzer/requirements_demos.txt (line 9)) (4.62.3)
    Collecting umap-learn
      Downloading umap-learn-0.5.2.tar.gz (86 kB)
         |████████████████████████████████| 86 kB 4.5 MB/s
    Requirement already satisfied: joblib>=0.14 in /usr/local/lib/python3.7/dist-packages (from librosa>=0.6.1->-r /content/Resemblyzer/requirements_demos.txt (line 1)) (1.1.0)
    Requirement already satisfied: scikit-learn!=0.19.0,>=0.14.0 in /usr/local/lib/python3.7/dist-packages (from librosa>=0.6.1->-r /content/Resemblyzer/requirements_demos.txt (line 1)) (1.0.1)
    Requirement already satisfied: decorator>=3.0.0 in /usr/local/lib/python3.7/dist-packages (from librosa>=0.6.1->-r /content/Resemblyzer/requirements_demos.txt (line 1)) (4.4.2)
    Requirement already satisfied: audioread>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from librosa>=0.6.1->-r /content/Resemblyzer/requirements_demos.txt (line 1)) (2.1.9)
    Requirement already satisfied: resampy>=0.2.2 in /usr/local/lib/python3.7/dist-packages (from librosa>=0.6.1->-r /content/Resemblyzer/requirements_demos.txt (line 1)) (0.2.2)
    Requirement already satisfied: numba>=0.43.0 in /usr/local/lib/python3.7/dist-packages (from librosa>=0.6.1->-r /content/Resemblyzer/requirements_demos.txt (line 1)) (0.51.2)
    Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.7/dist-packages (from librosa>=0.6.1->-r /content/Resemblyzer/requirements_demos.txt (line 1)) (21.3)
    Requirement already satisfied: pooch>=1.0 in /usr/local/lib/python3.7/dist-packages (from librosa>=0.6.1->-r /content/Resemblyzer/requirements_demos.txt (line 1)) (1.5.2)
    Requirement already satisfied: soundfile>=0.10.2 in /usr/local/lib/python3.7/dist-packages (from librosa>=0.6.1->-r /content/Resemblyzer/requirements_demos.txt (line 1)) (0.10.3.post1)
    Requirement already satisfied: typing-extensions in /usr/local/lib/python3.7/dist-packages (from torch>=1.0.1->-r /content/Resemblyzer/requirements_demos.txt (line 4)) (3.10.0.2)
    Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib>=3.0.0->-r /content/Resemblyzer/requirements_demos.txt (line 7)) (3.0.6)
    Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.7/dist-packages (from matplotlib>=3.0.0->-r /content/Resemblyzer/requirements_demos.txt (line 7)) (0.11.0)
    Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib>=3.0.0->-r /content/Resemblyzer/requirements_demos.txt (line 7)) (2.8.2)
    Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib>=3.0.0->-r /content/Resemblyzer/requirements_demos.txt (line 7)) (1.3.2)
    Requirement already satisfied: llvmlite<0.35,>=0.34.0.dev0 in /usr/local/lib/python3.7/dist-packages (from numba>=0.43.0->librosa>=0.6.1->-r /content/Resemblyzer/requirements_demos.txt (line 1)) (0.34.0)
    Requirement already satisfied: setuptools in /usr/local/lib/python3.7/dist-packages (from numba>=0.43.0->librosa>=0.6.1->-r /content/Resemblyzer/requirements_demos.txt (line 1)) (57.4.0)
    Requirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from pooch>=1.0->librosa>=0.6.1->-r /content/Resemblyzer/requirements_demos.txt (line 1)) (2.23.0)
    Requirement already satisfied: appdirs in /usr/local/lib/python3.7/dist-packages (from pooch>=1.0->librosa>=0.6.1->-r /content/Resemblyzer/requirements_demos.txt (line 1)) (1.4.4)
    Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/dist-packages (from python-dateutil>=2.1->matplotlib>=3.0.0->-r /content/Resemblyzer/requirements_demos.txt (line 7)) (1.15.0)
    Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from scikit-learn!=0.19.0,>=0.14.0->librosa>=0.6.1->-r /content/Resemblyzer/requirements_demos.txt (line 1)) (3.0.0)
    Requirement already satisfied: cffi>=1.0 in /usr/local/lib/python3.7/dist-packages (from soundfile>=0.10.2->librosa>=0.6.1->-r /content/Resemblyzer/requirements_demos.txt (line 1)) (1.15.0)
    Requirement already satisfied: pycparser in /usr/local/lib/python3.7/dist-packages (from cffi>=1.0->soundfile>=0.10.2->librosa>=0.6.1->-r /content/Resemblyzer/requirements_demos.txt (line 1)) (2.21)
    Collecting pynndescent>=0.5
      Downloading pynndescent-0.5.5.tar.gz (1.1 MB)
         |████████████████████████████████| 1.1 MB 23.0 MB/s
    Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests->pooch>=1.0->librosa>=0.6.1->-r /content/Resemblyzer/requirements_demos.txt (line 1)) (3.0.4)
    Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests->pooch>=1.0->librosa>=0.6.1->-r /content/Resemblyzer/requirements_demos.txt (line 1)) (1.24.3)
    Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests->pooch>=1.0->librosa>=0.6.1->-r /content/Resemblyzer/requirements_demos.txt (line 1)) (2021.10.8)
    Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->pooch>=1.0->librosa>=0.6.1->-r /content/Resemblyzer/requirements_demos.txt (line 1)) (2.10)
    Building wheels for collected packages: webrtcvad, typing, umap-learn, pynndescent
      Building wheel for webrtcvad (setup.py) ... done
      Created wheel for webrtcvad: filename=webrtcvad-2.0.10-cp37-cp37m-linux_x86_64.whl size=72371 sha256=7621f4ae82593fcd07bed8d20dbae14244801d282e036e3ccae3bc29a5c95bee
      Stored in directory: /root/.cache/pip/wheels/11/f9/67/a3158d131f57e1c0a7d8d966a707d4a2fb27567a4fe47723ad
      Building wheel for typing (setup.py) ... done
      Created wheel for typing: filename=typing-3.7.4.3-py3-none-any.whl size=26324 sha256=fbd02a1c9f53568e1fc1325fe365404b2e64401b5d3e3300c61e00db30c1c037
      Stored in directory: /root/.cache/pip/wheels/35/f3/15/01aa6571f0a72ee6ae7b827c1491c37a1f72d686fd22b43b0e
      Building wheel for umap-learn (setup.py) ... done
      Created wheel for umap-learn: filename=umap_learn-0.5.2-py3-none-any.whl size=82709 sha256=ba9cc8bd29661af264bf40ab0438f52001799bd4db06a4047b8d6e9d3370d702
      Stored in directory: /root/.cache/pip/wheels/84/1b/c6/aaf68a748122632967cef4dffef68224eb16798b6793257d82
      Building wheel for pynndescent (setup.py) ... done
      Created wheel for pynndescent: filename=pynndescent-0.5.5-py3-none-any.whl size=52603 sha256=baff4af0093c996b086ed9c9f33e3ee2e24de44b6cf5cfc68aec0a949a5146dd
      Stored in directory: /root/.cache/pip/wheels/af/e9/33/04db1436df0757c42fda8ea6796d7a8586e23c85fac355f476
    Successfully built webrtcvad typing umap-learn pynndescent
    Installing collected packages: pynndescent, webrtcvad, umap-learn, typing, sounddevice
    Successfully installed pynndescent-0.5.5 sounddevice-0.4.4 typing-3.7.4.3 umap-learn-0.5.2 webrtcvad-2.0.10

```python
!mkdir /content/new
```

```python
from resemblyzer import preprocess_wav, VoiceEncoder
from pathlib import Path

#give the file path to your audio file (Noagenda-0690-2015-01-25-Noagenda-0690-2015-01-25 from 13:37 to 15:12)
audio_file_path = '/content/new/noagenda-extract.wav'
wav_fpath = Path(audio_file_path)

wav = preprocess_wav(wav_fpath)
encoder = VoiceEncoder("cpu")
_, cont_embeds, wav_splits = encoder.embed_utterance(wav, return_partials=True, rate=16)
print(cont_embeds.shape)
```

    Loaded the voice encoder model on cpu in 0.04 seconds.
    (1548, 256)

```python
!pip install spectralcluster
```

    Collecting spectralcluster
      Downloading spectralcluster-0.2.4-py3-none-any.whl (22 kB)
    Installing collected packages: spectralcluster
    Successfully installed spectralcluster-0.2.4

```python
from spectralcluster import configs

labels = configs.icassp2018_clusterer.predict(cont_embeds)

#labels = clusterer.predict(cont_embeds)
```

```python
from spectralcluster import RefinementOptions
from spectralcluster import ThresholdType
from spectralcluster import ICASSP2018_REFINEMENT_SEQUENCE

refinement_options = RefinementOptions(
    gaussian_blur_sigma=1,
    p_percentile=0.95,
    thresholding_soft_multiplier=0.01,
    thresholding_type=ThresholdType.RowMax,
    refinement_sequence=ICASSP2018_REFINEMENT_SEQUENCE)
```

```python
from spectralcluster import SpectralClusterer

clusterer = SpectralClusterer(
    min_clusters=2,
    max_clusters=7,
    autotune=None,
    laplacian_type=None,
    refinement_options=refinement_options,
    custom_dist="cosine")
print("Done")
labels = clusterer.predict(cont_embeds)
```

    Done

```python
labels
```

    array([0, 0, 0, ..., 0, 0, 0])

```python
from spectralcluster import AutoTune

autotune = AutoTune(
    p_percentile_min=0.60,
    p_percentile_max=0.95,
    init_search_step=0.01,
    search_level=3)
```

```python
def create_labelling(labels,wav_splits):
    from resemblyzer import sampling_rate
    times = [((s.start + s.stop) / 2) / sampling_rate for s in wav_splits]
    labelling = []
    start_time = 0

    for i,time in enumerate(times):
        if i>0 and labels[i]!=labels[i-1]:
            temp = [str(labels[i-1]),start_time,time]
            labelling.append(tuple(temp))
            start_time = time
        if i==len(times)-1:
            temp = [str(labels[i]),start_time,time]
            labelling.append(tuple(temp))

    return labelling
```

```python
labelling = create_labelling(labels,wav_splits)
```

```python
labelling
```

    [('0', 0, 16.34),
     ('1', 16.34, 17.78),
     ('0', 17.78, 22.4),
     ('1', 22.4, 24.14),
     ('0', 24.14, 27.86),
     ('1', 27.86, 30.38),
     ('0', 30.38, 57.74),
     ('1', 57.74, 62.06),
     ('0', 62.06, 69.38),
     ('1', 69.38, 69.5),
     ('0', 69.5, 70.94),
     ('1', 70.94, 74.9),
     ('0', 74.9, 79.76),
     ('1', 79.76, 80.18),
     ('0', 80.18, 93.62)]

```python
pip install pydub
```

    Collecting pydub
      Downloading pydub-0.25.1-py2.py3-none-any.whl (32 kB)
    Installing collected packages: pydub
    Successfully installed pydub-0.25.1

```python
from pydub import AudioSegment

def mp3_to_wav(audio_file_path):
    sound = AudioSegment.from_mp3(audio_file_path)
    audio_file_path = audio_file_path.split('.')[0] + '.wav'
    sound.export(audio_file_path, format="wav")
    return audio_file_path

#audio_file_path = mp3_to_wav(audio_file_path)
#print(audio_file_path)
```

```python
pip install spectralcluster
```

    Requirement already satisfied: spectralcluster in /usr/local/lib/python3.7/dist-packages (0.2.4)

```python
!pip install ffmpeg-python
```

    Collecting ffmpeg-python
      Downloading ffmpeg_python-0.2.0-py3-none-any.whl (25 kB)
    Requirement already satisfied: future in /usr/local/lib/python3.7/dist-packages (from ffmpeg-python) (0.16.0)
    Installing collected packages: ffmpeg-python
    Successfully installed ffmpeg-python-0.2.0

```python
import ffmpeg
def create_output_files(path):
    # For each tuple in the labelling file, extract ele[0] as the speaker
    # label, ele[1] as the start time, and ele[2] as the end time
    num = 1
    input = ffmpeg.input(path)
    for ele in labelling:
        audio_trim = input.audio.filter('atrim', start=ele[1], end=ele[2])
        numstr = str(num)
        while len(numstr) < 4:
            numstr = "0" + numstr
        output = ffmpeg.output(audio_trim, "/content/extract{}.wav".format(numstr))
        output.run(capture_stdout=True, capture_stderr=True, overwrite_output=True)
        num += 1

```

```python
try:
    create_output_files(audio_file_path)
except ffmpeg.Error as e:
    print(e.stderr)
```

```python
!pip install SpeechRecognition
```

    Collecting SpeechRecognition
      Downloading SpeechRecognition-3.8.1-py2.py3-none-any.whl (32.8 MB)
         |████████████████████████████████| 32.8 MB 1.6 MB/s
    Installing collected packages: SpeechRecognition
    Successfully installed SpeechRecognition-3.8.1

```python
import speech_recognition as sr
def transcribe_audio(path):
    r = sr.Recognizer()
    audio_file = sr.AudioFile(path)
    with audio_file as source:
        r.adjust_for_ambient_noise(source)
        audio = r.record(source)
    try:
        res = r.recognize_google(audio, language="en-US")
    except sr.UnknownValueError:
        res = "(The speech is unintelligible)"
    except sr.RequestError:
        res = "(The speech recognition operation failed)"
    return res
```

```python
import os
import os.path
list_dir = os.listdir("/content")
i = 0
for e in list_dir:
    if e.startswith('extract'):
        path = os.path.join("/content", e)
        transcription = transcribe_audio(path)
        print("Speaker {}: {}".format(labelling[i][0], transcription))
        i += 1
```

    Speaker 0: (The speech is unintelligible)
    Speaker 1: (The speech is unintelligible)
    Speaker 0: (The speech is unintelligible)
    Speaker 1: the commitments
    Speaker 0: negotiate
    Speaker 1: I have instability in financial markets believe that a couple of years ago
    Speaker 0: (The speech is unintelligible)
    Speaker 1: it'll be between
    Speaker 0: (The speech is unintelligible)
    Speaker 1: Eurasian or renegotiate
    Speaker 0: (The speech is unintelligible)
    Speaker 1: it's true I heard it too did you get this from TV or from the web
    Speaker 0: commitment
    Speaker 1: (The speech is unintelligible)
    Speaker 0: (The speech is unintelligible)

```python
!pip install pyannote.metrics
```

To calculate the DER and the JER, several tools can be used such as *dscore* or *pyannote*

# Dscore
To use *dscore*, run the following commands.

```python
!git clone https://github.com/nryant/dscore.git
```

    fatal: destination path 'dscore' already exists and is not an empty directory.

```python
%cd ../dscore
```

    /content/dscore

```python
!pip install -r requirements.txt
```

    Collecting intervaltree>=3.0.0
      Downloading intervaltree-3.1.0.tar.gz (32 kB)
    Requirement already satisfied: numpy>=1.16.2 in /usr/local/lib/python3.7/dist-packages (from -r requirements.txt (line 2)) (1.19.5)
    Requirement already satisfied: scipy>=0.17.0 in /usr/local/lib/python3.7/dist-packages (from -r requirements.txt (line 3)) (1.4.1)
    Requirement already satisfied: tabulate>=0.5.0 in /usr/local/lib/python3.7/dist-packages (from -r requirements.txt (line 4)) (0.8.9)
    Requirement already satisfied: sortedcontainers<3.0,>=2.0 in /usr/local/lib/python3.7/dist-packages (from intervaltree>=3.0.0->-r requirements.txt (line 1)) (2.4.0)
    Building wheels for collected packages: intervaltree
      Building wheel for intervaltree (setup.py) ... done
      Created wheel for intervaltree: filename=intervaltree-3.1.0-py2.py3-none-any.whl size=26119 sha256=896a15e34262e5ed51f8664527cb236e81e4bba51a03d416f7bbcc42c7288cb6
      Stored in directory: /root/.cache/pip/wheels/16/85/bd/1001cbb46dcfb71c2001cd7401c6fb250392f22a81ce3722f7
    Successfully built intervaltree
    Installing collected packages: intervaltree
      Attempting uninstall: intervaltree
        Found existing installation: intervaltree 2.1.0
        Uninstalling intervaltree-2.1.0:
          Successfully uninstalled intervaltree-2.1.0
    Successfully installed intervaltree-3.1.0

The script *score.py* takes one or several RTTM files for the reference and the system.

Rich Transcription Time Marked (RTTM) files are space-delimited text files containing one turn per line, each line containing ten fields:

Type -- segment type; should always by SPEAKER

File ID -- file name; basename of the recording minus extension (e.g., rec1_a)

Channel ID -- channel (1-indexed) that turn is on; should always be 1

Turn Onset -- onset of turn in seconds from beginning of recording

Turn Duration -- duration of turn in seconds

Orthography Field -- should always be ```<NA>```

Speaker Type -- should always be ```<NA>```

Speaker Name -- name of speaker of turn; should be unique within scope of each file

Confidence Score -- system confidence (probability) that information is correct; should always be ```<NA>```

Signal Lookahead Time -- should always be ```<NA>```

## Example
```
SPEAKER CMU_20020319-1400_d01_NONE 1 130.430000 2.350 <NA> <NA> juliet <NA> <NA>
SPEAKER CMU_20020319-1400_d01_NONE 1 157.610000 3.060 <NA> <NA> tbc <NA> <NA>
SPEAKER CMU_20020319-1400_d01_NONE 1 130.490000 0.450 <NA> <NA> chek <NA> <NA>
```

You can then execute *score.py* with the reference files and the system files

```python
%cd /content/dscore/
```

```python
!python score.py -r ../CMU_20030109-1600.sdm.overlap.rttm -s ../out.rttm
```

# PyAnnote

First, install PyAnnote with the following command:

# New Section

**Start to execute here: don't take into account the cells above (except for Resemblyzer)**

```python
!pip install pyannote.metrics
```

    Collecting pyannote.metrics
      Downloading pyannote.metrics-3.1-py3-none-any.whl (49 kB)
         |████████████████████████████████| 49 kB 3.5 MB/s
    Requirement already satisfied: sympy>=1.1 in /usr/local/lib/python3.7/dist-packages (from pyannote.metrics) (1.7.1)
    Collecting pyannote.core>=4.1
      Downloading pyannote.core-4.3-py3-none-any.whl (59 kB)
         |████████████████████████████████| 59 kB 8.9 MB/s
    Collecting pyannote.database>=4.0.1
      Downloading pyannote.database-4.1.1-py3-none-any.whl (41 kB)
         |████████████████████████████████| 41 kB 331 kB/s
    Requirement already satisfied: scikit-learn>=0.17.1 in /usr/local/lib/python3.7/dist-packages (from pyannote.metrics) (1.0.1)
    Requirement already satisfied: docopt>=0.6.2 in /usr/local/lib/python3.7/dist-packages (from pyannote.metrics) (0.6.2)
    Requirement already satisfied: scipy>=1.1.0 in /usr/local/lib/python3.7/dist-packages (from pyannote.metrics) (1.4.1)
    Requirement already satisfied: matplotlib>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from pyannote.metrics) (3.2.2)
    Requirement already satisfied: pandas>=0.19 in /usr/local/lib/python3.7/dist-packages (from pyannote.metrics) (1.1.5)
    Requirement already satisfied: tabulate>=0.7.7 in /usr/local/lib/python3.7/dist-packages (from pyannote.metrics) (0.8.9)
    Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.7/dist-packages (from matplotlib>=2.0.0->pyannote.metrics) (0.11.0)
    Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib>=2.0.0->pyannote.metrics) (2.8.2)
    Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib>=2.0.0->pyannote.metrics) (1.3.2)
    Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib>=2.0.0->pyannote.metrics) (3.0.6)
    Requirement already satisfied: numpy>=1.11 in /usr/local/lib/python3.7/dist-packages (from matplotlib>=2.0.0->pyannote.metrics) (1.19.5)
    Requirement already satisfied: pytz>=2017.2 in /usr/local/lib/python3.7/dist-packages (from pandas>=0.19->pyannote.metrics) (2018.9)
    Requirement already satisfied: typing-extensions>=3.7.4.1 in /usr/local/lib/python3.7/dist-packages (from pyannote.core>=4.1->pyannote.metrics) (3.10.0.2)
    Collecting simplejson>=3.8.1
      Downloading simplejson-3.17.6-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (130 kB)
         |████████████████████████████████| 130 kB 42.0 MB/s
    Requirement already satisfied: sortedcontainers>=2.0.4 in /usr/local/lib/python3.7/dist-packages (from pyannote.core>=4.1->pyannote.metrics) (2.4.0)
    Requirement already satisfied: pyYAML>=3.12 in /usr/local/lib/python3.7/dist-packages (from pyannote.database>=4.0.1->pyannote.metrics) (3.13)
    Collecting typer[all]>=0.2.1
      Downloading typer-0.4.0-py3-none-any.whl (27 kB)
    Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/dist-packages (from python-dateutil>=2.1->matplotlib>=2.0.0->pyannote.metrics) (1.15.0)
    Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from scikit-learn>=0.17.1->pyannote.metrics) (3.0.0)
    Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.7/dist-packages (from scikit-learn>=0.17.1->pyannote.metrics) (1.1.0)
    Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.7/dist-packages (from sympy>=1.1->pyannote.metrics) (1.2.1)
    Requirement already satisfied: click<9.0.0,>=7.1.1 in /usr/local/lib/python3.7/dist-packages (from typer[all]>=0.2.1->pyannote.database>=4.0.1->pyannote.metrics) (7.1.2)
    Collecting colorama<0.5.0,>=0.4.3
      Downloading colorama-0.4.4-py2.py3-none-any.whl (16 kB)
    Collecting shellingham<2.0.0,>=1.3.0
      Downloading shellingham-1.4.0-py2.py3-none-any.whl (9.4 kB)
    Installing collected packages: typer, simplejson, shellingham, colorama, pyannote.core, pyannote.database, pyannote.metrics
    Successfully installed colorama-0.4.4 pyannote.core-4.3 pyannote.database-4.1.1 pyannote.metrics-3.1 shellingham-1.4.0 simplejson-3.17.6 typer-0.4.0

It seems that you have to add the segments directly in the code. The letters ```A```, ```B```, ```C``` and ```D``` identify the different speakers.

```python
metric = DiarizationErrorRate()
metric(reference, hypothesis, detailed=True)
```

    /usr/local/lib/python3.7/dist-packages/pyannote/metrics/utils.py:184: UserWarning: 'uem' was approximated by the union of 'reference' and 'hypothesis' extents.
      "'uem' was approximated by the union of 'reference' "

    {'confusion': 7.0,
     'correct': 22.0,
     'diarization error rate': 0.5161290322580645,
     'false alarm': 7.0,
     'missed detection': 2.0,
     'total': 31.0}

```python
import librosa
import librosa.display # You need this in librosa to be able to plot
import os, sys, glob
import matplotlib.pyplot as plt
from pathlib import Path
import numpy as np

#give the file path to your audio file
audio_file_path = '/content/CMU/*'
clip_dir=glob.glob(audio_file_path)
```

```python
!pip install spectralcluster
```

    Collecting spectralcluster
      Downloading spectralcluster-0.2.4-py3-none-any.whl (22 kB)
    Installing collected packages: spectralcluster
    Successfully installed spectralcluster-0.2.4

```python
from spectralcluster import SpectralClusterer

clusterer = SpectralClusterer(
    min_clusters=2,
    max_clusters=7,
    autotune=None,
    laplacian_type=None,
    refinement_options=None,
    custom_dist="cosine")

```

```python
def create_labelling(labels,wav_splits, prefix="", output_file_name="/content/labelling.txt"):
    from resemblyzer import sampling_rate
    times = [((s.start + s.stop) / 2) / sampling_rate for s in wav_splits]
    start_time = 0
    with open(output_file_name, 'a+') as f_out:
        for i,time in enumerate(times):
            if i>0 and labels[i]!=labels[i-1]:
                temp = [prefix + str(labels[i-1]),start_time,time]
                f_out.write("{} {} {}\n".format(temp[0], temp[1], temp[2]))
                start_time = time
            if i==len(times)-1:
                temp = [prefix + str(labels[i]),start_time,time]
                f_out.write("{} {} {}\n".format(temp[0], temp[1], temp[2]))

```

```python
%cd Resemblyzer/
```

    /content/Resemblyzer

```python
def labelling_to_rttm(input_file, output_file):
    with open(input_file, 'r') as f_in:
        with open(output_file, 'a+') as f_out:
            lines = f_in.readlines()
            for l in lines:
                fields = l.split(' ')
                if len(fields) == 3:
                    speaker = fields[0]
                    turn_onset = float(fields[1])
                    turn_duration = float(fields[2][:-1]) - turn_onset
                    output_line = "SPEAKER {} 1 {:.3f} {:.3f} <NA> <NA> {} <NA> <NA>".format(speaker, turn_onset, turn_duration, speaker)
                    f_out.write(output_line + "\n")
```

```python

```

```python
def time_labelling_to_rttm(input_file, output_file):
    with open(input_file, 'r') as f_in:
        with open(output_file, 'a+') as f_out:
            lines = f_in.readlines()
            for l in lines:
                fields = l.split()
                print(fields)
                if len(fields) == 3:
                    print(fields)
                    speaker = fields[0]
                    turn_onset= (datetime.datetime.strptime((fields[1]).strip().replace('\n',''),"%H:%M:%S,%f") - datetime.datetime(1900, 1, 1)).total_seconds()
                    #turn_onset = float(fields[1])
                    print(turn_onset)
                    turn_duration = (datetime.datetime.strptime((fields[2]).strip().replace('\n',''),"%H:%M:%S,%f") - datetime.datetime(1900, 1, 1)).total_seconds() -turn_onset
                    #turn_onset =- turn_onset
                    output_line = "SPEAKER {} 1 {:.3f} {:.3f} <NA> <NA> {} <NA> <NA>".format(speaker, turn_onset, turn_duration, speaker)
                    f_out.write(output_line + "\n")
```

```python
time_labelling_to_rttm("annotation", "/content/out.rttm")
```

    ['Speaker1', '00:00:00,630', '00:00:07,020']
    ['Speaker1', '00:00:00,630', '00:00:07,020']
    0.63
    ['Speaker1', '00:00:09,719', '00:00:15,449']
    ['Speaker1', '00:00:09,719', '00:00:15,449']
    9.719
    ['Speaker1', '00:00:16,560', '00:00:36,630']
    ['Speaker1', '00:00:16,560', '00:00:36,630']
    16.56
    ['Speaker2', '00:00:42,810', '00:00:46,290']
    ['Speaker2', '00:00:42,810', '00:00:46,290']
    42.81
    ['Speaker2', '00:00:51,300', '00:01:01,020']
    ['Speaker2', '00:00:51,300', '00:01:01,020']
    51.3
    ['Speaker2', '00:00:55,830', '00:01:01,020']
    ['Speaker2', '00:00:55,830', '00:01:01,020']
    55.83
    ['Speaker2', '00:01:03,180', '00:01:07,050']
    ['Speaker2', '00:01:03,180', '00:01:07,050']
    63.18
    ['Speaker2', '00:01:09,780', '0:01:24,180']
    ['Speaker2', '00:01:09,780', '0:01:24,180']
    69.78
    ['Speaker2', '00:01:24,180', '00:01:58,290']
    ['Speaker2', '00:01:24,180', '00:01:58,290']
    84.18
    ['Speaker3', '00:01:58,290', '00:02:01,740']
    ['Speaker3', '00:01:58,290', '00:02:01,740']
    118.29
    ['Speaker3', '00:02:01,740', '00:02:02,760']
    ['Speaker3', '00:02:01,740', '00:02:02,760']
    121.74
    ['Speaker2', '00:02:04,170', '00:02:07,800']
    ['Speaker2', '00:02:04,170', '00:02:07,800']
    124.17
    ['Speaker3', '00:02:07,800', '00:02:11,220']
    ['Speaker3', '00:02:07,800', '00:02:11,220']
    127.8
    ['Speaker3', '00:02:11,220', '00:02:14,430']
    ['Speaker3', '00:02:11,220', '00:02:14,430']
    131.22
    ['Speaker3', '00:02:18,600', '00:02:22,140']
    ['Speaker3', '00:02:18,600', '00:02:22,140']
    138.6
    ['Speaker1', '00:02:22,170', '00:02:30,630']
    ['Speaker1', '00:02:22,170', '00:02:30,630']
    142.17
    ['Speaker2', '00:02:30,630', '00:02:34,290']
    ['Speaker2', '00:02:30,630', '00:02:34,290']
    150.63
    ['Speaker2', '00:02:35,670', '00:02:38,220']
    ['Speaker2', '00:02:35,670', '00:02:38,220']
    155.67
    ['Speaker3', '00:02:38,220', '00:02:56,730']
    ['Speaker3', '00:02:38,220', '00:02:56,730']
    158.22
    ['Speaker2', '00:02:56,850', '00:03:13,020']
    ['Speaker2', '00:02:56,850', '00:03:13,020']
    176.85
    ['Speaker1', '00:03:14,190', '00:04:03,810']
    ['Speaker1', '00:03:14,190', '00:04:03,810']
    194.19
    ['Speaker2', '00:04:03,810', '00:04:26,340']
    ['Speaker2', '00:04:03,810', '00:04:26,340']
    243.81
    ['Speaker1', '00:04:26,340', '00:04:29,460']
    ['Speaker1', '00:04:26,340', '00:04:29,460']
    266.34
    ['Speaker1', '00:04:29,460', '00:04:35,850']
    ['Speaker1', '00:04:29,460', '00:04:35,850']
    269.46
    ['Speaker2', '00:04:35,850', '00:05:15,450']
    ['Speaker2', '00:04:35,850', '00:05:15,450']
    275.85
    ['Speaker3', '00:05:16,470', '00:05:32,730']
    ['Speaker3', '00:05:16,470', '00:05:32,730']
    316.47
    ['Speaker2', '00:05:32,820', '00:05:46,170']
    ['Speaker2', '00:05:32,820', '00:05:46,170']
    332.82
    ['Speaker3', '00:05:46,170', '00:05:49,560']
    ['Speaker3', '00:05:46,170', '00:05:49,560']
    346.17
    ['Speaker1', '0:05:49,560', '00:06:02,670']
    ['Speaker1', '0:05:49,560', '00:06:02,670']
    349.56
    ['Speaker1', '00:06:02,670', '00:06:45,330']
    ['Speaker1', '00:06:02,670', '00:06:45,330']
    362.67
    ['Speaker3', '00:06:45,330', '00:06:52,560']
    ['Speaker3', '00:06:45,330', '00:06:52,560']
    405.33

```python

```

```python
'ffggg\n'.strip()
```

    'ffggg'

```python
%cd Resemblyzer/
```

    [Errno 2] No such file or directory: 'Resemblyzer/'
    /content/Resemblyzer

```python
help("spectralcluster.SpectralClusterer.predict")
```

    Help on function predict in spectralcluster.SpectralClusterer:

    spectralcluster.SpectralClusterer.predict = predict(self, embeddings, constraint_matrix=None)
        Perform spectral clustering on data embeddings.

        The spectral clustering is performed on an affinity matrix.

        Args:
          embeddings: numpy array of shape (n_samples, n_features)
          constraint_matrix: numpy array of shape (n_samples, n_samples). The
            constraint matrix with prior information

        Returns:
          labels: numpy array of shape (n_samples,)

        Raises:
          TypeError: if embeddings has wrong type
          ValueError: if embeddings has wrong shape

```python
%cd ./Resemblyzer/
```

    [Errno 2] No such file or directory: './Resemblyzer/'
    /content/Resemblyzer

```python
from spectralcluster import SpectralClusterer
from resemblyzer import preprocess_wav, VoiceEncoder
from pathlib import Path
import os
#print("fff")
#print(wav_fpath)

clip_dir = '/content/new/'
# list of files in clip_dir
list_files = [f for f in os.listdir(clip_dir) if not f.startswith('.')]
utterance_spec=[]
total_labelling = []
for idx,showName in enumerate(list_files):
    base, ext = os.path.splitext(showName)
    path = os.path.join(clip_dir, showName)
    wav = preprocess_wav(path)
    utterance_spec.append(wav)
    encoder = VoiceEncoder("cpu")
    _, cont_embeds, wav_splits = encoder.embed_utterance(wav, return_partials=True, rate=16)
    print(cont_embeds.shape)

    clusterer = SpectralClusterer(
        min_clusters=3,
        max_clusters=7,
        autotune=None,
        laplacian_type=None,
        refinement_options=None,
        custom_dist="cosine"
    )
    labels = clusterer.predict(cont_embeds)
    create_labelling(labels, wav_splits, prefix=base)
    print('Lablling created for {}'.format(showName))
    del wav, encoder, cont_embeds, wav_splits, clusterer, labels

#utterance_spec = np.array(utterance_spec)

```

    Loaded the voice encoder model on cpu in 0.01 seconds.
    (3417, 256)
    Lablling created for extract.wav

```python
labelling_to_rttm("/content/labelling.txt", "/content/out.rttm")
```

```python
from pyannote.core import Segment, Timeline, Annotation
from pyannote.metrics.diarization import DiarizationErrorRate

#speaker=['A','B','C','D','E']
def annotate(path, uri='file1'):
    res = Annotation(uri=uri)

    with open(path, "r") as f:
        # current speaker in the reference
        current_speaker_ref = ''
        # current begin time
        current_begin = 0
        # current end time
        current_end = 0
        rttmspeaker=dict()
        lines = f.readlines()
        for index, l in enumerate(lines):
            # for each line, we check whether the line starts with SPEAKER
            if l.startswith('SPEAKER'):
                num=0
                currentspeaker=0
                fields = l.split(' ')
                if len(l) > 7:
                    speaker = fields[7]
                    if speaker not in rttmspeaker.keys():
                      rttmspeaker[speaker]=num
                      num=num+1
                    else:
                     currentspeaker= rttmspeaker[speaker]

                    start = float(fields[3])
                    end = start + float(fields[4]) # end = start + duration
                    #if speaker == current_speaker_ref and index < len(lines)-1:
                    #    current_end = end
                    #elif current_speaker_ref == '':
                    #    current_speaker_ref = speaker
                    #   current_begin = start
                    #    current_end = end
                    #else:
                    print(str(start)+ ' ' + str(end)+ ' '+str(speaker))
                    res[Segment(start, end)] = speaker
    return res

reference = annotate("/content/out.rttm")
#reference = annotate("/content/CMU_20030109-1530.mdm.overlap.rttm")
#hypothesis = annotate("/content/out.rttm")
```

    0.63 7.02 Speaker1
    9.719 15.449 Speaker1
    16.56 36.629999999999995 Speaker1
    42.81 46.29 Speaker2
    51.3 61.019999999999996 Speaker2
    55.83 61.019999999999996 Speaker2
    63.18 67.05 Speaker2
    69.78 84.18 Speaker2
    84.18 118.29 Speaker2
    118.29 121.74000000000001 Speaker3
    121.74 122.75999999999999 Speaker3
    124.17 127.8 Speaker2
    127.8 131.22 Speaker3
    131.22 134.43 Speaker3
    138.6 142.14 Speaker3
    142.17 150.63 Speaker1
    150.63 154.29 Speaker2
    155.67 158.22 Speaker2
    158.22 176.73 Speaker3
    176.85 193.01999999999998 Speaker2
    194.19 243.81 Speaker1
    243.81 266.34000000000003 Speaker2
    266.34 269.46 Speaker1
    269.46 275.84999999999997 Speaker1
    275.85 315.45000000000005 Speaker2
    316.47 332.73 Speaker3
    332.82 346.17 Speaker2
    346.17 349.56 Speaker3
    349.56 362.67 Speaker1
    362.67 405.33000000000004 Speaker1
    405.33 412.56 Speaker3

```python
diarization
```

```python
metric = DiarizationErrorRate()
metric(reference, diarization, detailed=True)
```

    /usr/local/lib/python3.7/dist-packages/pyannote/metrics/utils.py:184: UserWarning:

    'uem' was approximated by the union of 'reference' and 'hypothesis' extents.

    {'confusion': 153.4279999999999,
     'correct': 195.8082499999997,
     'diarization error rate': 0.5600233021349015,
     'false alarm': 25.16768749999996,
     'missed detection': 38.603750000000225,
     'total': 387.83999999999986}

```python
report = metric.report(display=True)
```

          diarization error rate  total correct correct false alarm false alarm missed detection missed detection confusion confusion
                               %                      %                       %                                 %                   %
    item
    file1                  54.67 905.88  556.70   61.45      146.07       16.13           287.68            31.76     61.50      6.79
    TOTAL                  54.67 905.88  556.70   61.45      146.07       16.13           287.68            31.76     61.50      6.79

```python
report = metric.report(display=True)
```

          diarization error rate  total correct correct false alarm false alarm missed detection missed detection confusion confusion
                               %                      %                       %                                 %                   %
    item
    file1                  56.00 387.84  195.81   50.49       25.17        6.49            38.60             9.95    153.43     39.56
    TOTAL                  56.00 387.84  195.81   50.49       25.17        6.49            38.60             9.95    153.43     39.56

```python
!pip install ffmpeg-python
```

    Collecting ffmpeg-python
      Downloading ffmpeg_python-0.2.0-py3-none-any.whl (25 kB)
    Requirement already satisfied: future in /usr/local/lib/python3.7/dist-packages (from ffmpeg-python) (0.16.0)
    Installing collected packages: ffmpeg-python
    Successfully installed ffmpeg-python-0.2.0

```python
time_labelling_to_rttm("annotation", "/content/out.rttm")
```

```python
import datetime
```

```python
date_time=datetime.datetime.strptime('00:00:55,830',"%H:%M:%S,%f")
a_timedelta = date_time - datetime.datetime(1900, 1, 1)
seconds = a_timedelta.total_seconds()
```

```python

```

```python
seconds
```

    55.83

https://github.com/kaldi-asr/kaldi/blob/master/egs/wsj/s5/steps/segmentation/convert_utt2spk_and_segments_to_rttm.py

```python
import ffmpeg
input = ffmpeg.input('/content/nashnote.mp3')
audio_trim = input.audio.filter('atrim', start=0, end=410)
output = ffmpeg.output(audio_trim, "/content/extract.wav")
output.run(capture_stdout=True, capture_stderr=True, overwrite_output=True)

```

    (b'',
     b"ffmpeg version 3.4.8-0ubuntu0.2 Copyright (c) 2000-2020 the FFmpeg developers\n  built with gcc 7 (Ubuntu 7.5.0-3ubuntu1~18.04)\n  configuration: --prefix=/usr --extra-version=0ubuntu0.2 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --enable-gpl --disable-stripping --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-librsvg --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libopencv --enable-libx264 --enable-shared\n  libavutil      55. 78.100 / 55. 78.100\n  libavcodec     57.107.100 / 57.107.100\n  libavformat    57. 83.100 / 57. 83.100\n  libavdevice    57. 10.100 / 57. 10.100\n  libavfilter     6.107.100 /  6.107.100\n  libavresample   3.  7.  0 /  3.  7.  0\n  libswscale      4.  8.100 /  4.  8.100\n  libswresample   2.  9.100 /  2.  9.100\n  libpostproc    54.  7.100 / 54.  7.100\n[mp3 @ 0x563d832c6000] Estimating duration from bitrate, this may be inaccurate\nInput #0, mp3, from '/content/nashnote.mp3':\n  Metadata:\n    title           : PC20-49-2021-08-06\n    TIT3            : BOTG\n    album           : Podcasting 2.0\n    artist          : Adam Curry & Dave Jones\n    comment         : Podcasting 2.0 for August 6th 2021 Episode 49: BOTG \n                    : Adam & Dave discuss the week's developments on podcastindex.org live from Podcast Movement in Nashville Tennessee and we're joined by Renowned Industry Expert Daniel J Lewis \n                    : I'm Adam Curry in the He...\n    lyrics-eng      : Podcasting 2.0 for August 6th 2021 Episode 49: BOTG \n                    : Adam & Dave discuss the week's developments on podcastindex.org live from Podcast Movement in Nashville Tennessee and we're joined by Renowned Industry Expert Daniel J Lewis \n                    : I'm Adam Curry in the Heart of Hill Country Texas \n                    : And In Nashville Tennessee, The True Live Music Capitol of the World - My Friend on the Other End -- Dave Jones! \n                    : Crank those value sliders up on any of these value4value streaming payment apps: \n                    : Breez app \n                    : CurioCaster \n                    : Fountain.FM \n                    : Podfriend \n                    : Podstation Browser extension \n                    : Podverse \n                    : Sphinx Chat App \n                    : Value4Value.io Explains how this works! \n                    : Download the mp3 \n                    : Podcast Feed \n                    : PodcastIndex.org \n                    : Preservepodcasting.com \n                    : Check out the podcasting 2.0 apps and services newpodcastapps.com \n                    : Support us with your Time Talent and Treasure \n                    : ShowNotes \n                    : Daniel J Lewis \n                    : Podcasting advocate and educator, award-winning host of The Audacity to Podcast, creator of My Podcast Reviews, curator of Podcast Industry Insights. \n                    : Boostagram \n                    : Fast Follow \n                    : \n    composer        : Podcast Index LLC\n    track           : 49\n    encoded_by      : Hindenburg Journalist Pro 1.86.2380\n    genre           : Podcast\n    copyright       : 2020-2021 Podcast Index LLC\n    date            : 2021\n  Duration: 01:10:19.83, start: 0.000000, bitrate: 96 kb/s\n    Stream #0:0: Audio: mp3, 44100 Hz, stereo, s16p, 96 kb/s\n    Stream #0:1: Video: mjpeg, yuvj420p(pc, bt470bg/unknown/unknown), 300x300 [SAR 1:1 DAR 1:1], 90k tbr, 90k tbn, 90k tbc\n    Metadata:\n      title           : Image\n      comment         : Other\nStream mapping:\n  Stream #0:0 (mp3) -> atrim\n  atrim -> Stream #0:0 (pcm_s16le)\nPress [q] to stop, [?] for help\nOutput #0, wav, to '/content/extract.wav':\n  Metadata:\n    INAM            : PC20-49-2021-08-06\n    TIT3            : BOTG\n    IPRD            : Podcasting 2.0\n    IART            : Adam Curry & Dave Jones\n    ICMT            : Podcasting 2.0 for August 6th 2021 Episode 49: BOTG \n                    : Adam & Dave discuss the week's developments on podcastindex.org live from Podcast Movement in Nashville Tennessee and we're joined by Renowned Industry Expert Daniel J Lewis \n                    : I'm Adam Curry in the He...\n    lyrics-eng      : Podcasting 2.0 for August 6th 2021 Episode 49: BOTG \n                    : Adam & Dave discuss the week's developments on podcastindex.org live from Podcast Movement in Nashville Tennessee and we're joined by Renowned Industry Expert Daniel J Lewis \n                    : I'm Adam Curry in the Heart of Hill Country Texas \n                    : And In Nashville Tennessee, The True Live Music Capitol of the World - My Friend on the Other End -- Dave Jones! \n                    : Crank those value sliders up on any of these value4value streaming payment apps: \n                    : Breez app \n                    : CurioCaster \n                    : Fountain.FM \n                    : Podfriend \n                    : Podstation Browser extension \n                    : Podverse \n                    : Sphinx Chat App \n                    : Value4Value.io Explains how this works! \n                    : Download the mp3 \n                    : Podcast Feed \n                    : PodcastIndex.org \n                    : Preservepodcasting.com \n                    : Check out the podcasting 2.0 apps and services newpodcastapps.com \n                    : Support us with your Time Talent and Treasure \n                    : ShowNotes \n                    : Daniel J Lewis \n                    : Podcasting advocate and educator, award-winning host of The Audacity to Podcast, creator of My Podcast Reviews, curator of Podcast Industry Insights. \n                    : Boostagram \n                    : Fast Follow \n                    : \n    composer        : Podcast Index LLC\n    IPRT            : 49\n    ITCH            : Hindenburg Journalist Pro 1.86.2380\n    IGNR            : Podcast\n    ICOP            : 2020-2021 Podcast Index LLC\n    ICRD            : 2021\n    ISFT            : Lavf57.83.100\n    Stream #0:0: Audio: pcm_s16le ([1][0][0][0] / 0x0001), 44100 Hz, stereo, s16, 1411 kb/s\n    Metadata:\n      encoder         : Lavc57.107.100 pcm_s16le\nsize=   35329kB time=00:03:25.29 bitrate=1409.7kbits/s speed= 411x    \rsize=   70629kB time=00:06:50.00 bitrate=1411.2kbits/s speed= 413x    \nvideo:0kB audio:70629kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000758%\n")

```python
pip install pyannote.audio==1.1.1
```

```python
pip install pyannote.core
```

    Requirement already satisfied: pyannote.core in /usr/local/lib/python3.7/dist-packages (4.3)
    Requirement already satisfied: numpy>=1.10.4 in /usr/local/lib/python3.7/dist-packages (from pyannote.core) (1.19.5)
    Requirement already satisfied: simplejson>=3.8.1 in /usr/local/lib/python3.7/dist-packages (from pyannote.core) (3.17.6)
    Requirement already satisfied: scipy>=1.1 in /usr/local/lib/python3.7/dist-packages (from pyannote.core) (1.4.1)
    Requirement already satisfied: sortedcontainers>=2.0.4 in /usr/local/lib/python3.7/dist-packages (from pyannote.core) (2.4.0)
    Requirement already satisfied: typing-extensions>=3.7.4.1 in /usr/local/lib/python3.7/dist-packages (from pyannote.core) (3.10.0.2)

```python
import torch
pipeline = torch.hub.load('pyannote/pyannote-audio', 'dia_dihard')
```

    Using cache found in /root/.cache/torch/hub/pyannote_pyannote-audio_master

    Downloading list of pretrained models and pipelines to "/root/.pyannote/hub/pretrained.yml".

      0%|          | 0.00/901 [00:00<?, ?B/s]

    Downloading pretrained pipeline "dia_dihard" to "/root/.pyannote/hub/pipelines/dia_dihard.zip".

      0%|          | 0.00/1.49k [00:00<?, ?B/s]

    Downloading pretrained model "sad_dihard" to "/root/.pyannote/hub/models/sad_dihard.zip".

    Using cache found in /root/.cache/torch/hub/pyannote_pyannote-audio_master

      0%|          | 0.00/2.41M [00:00<?, ?B/s]

    Using cache found in /root/.cache/torch/hub/pyannote_pyannote-audio_master

    Downloading pretrained model "scd_dihard" to "/root/.pyannote/hub/models/scd_dihard.zip".

      0%|          | 0.00/2.40M [00:00<?, ?B/s]

    Downloading pretrained model "emb_voxceleb" to "/root/.pyannote/hub/models/emb_voxceleb.zip".

    Using cache found in /root/.cache/torch/hub/pyannote_pyannote-audio_master

      0%|          | 0.00/16.3M [00:00<?, ?B/s]

    /usr/local/lib/python3.7/dist-packages/pyannote/audio/embedding/approaches/arcface_loss.py:170: FutureWarning:

    The 's' parameter is deprecated in favor of 'scale', and will be removed in a future release

    /usr/local/lib/python3.7/dist-packages/pyannote/audio/features/pretrained.py:156: UserWarning:

    Model was trained with 4s chunks and is applied on 2s chunks. This might lead to sub-optimal results.

    Using cache found in /root/.cache/torch/hub/pyannote_pyannote-audio_master

```python
from pyannote.database import get_protocol
from pyannote.database import FileFinder
preprocessors = {'audio': FileFinder()}
protocol = get_protocol('AMI.SpeakerDiarization.MixHeadset',
                        preprocessors=preprocessors)
test_file = next(protocol.test())
```

```python
test_file = {'uri': 'filename', 'audio': '/content/extract.wav'}
```

```python
diarization = pipeline(test_file)
```

```python
diarization
```

```python
reference
```

```python

```

```python

```

```python

```

```python

```

```python

```

```python

```


# Python support library: `epic_kitchens`

We provide a support library `epic_kitchens` that is installed in your conda environment, this contains a variety of helper methods and scripts for...

* Gulping frames: `python -m epic_kitchens.gulp`
* Converting between RGB frame indices and Flow frame indices: `epic_kitchens.time.*`
* Dataset classes for use in training loops: `epic_kitchens.dataset.epic_dataset`

Find out more:

* [Github repo](https://github.com/epic-kitchens/epic-lib)
* [read the docs](http://epic-kitchens.readthedocs.io/en/latest/index.html)
* [PyPI package](https://pypi.org/project/epic-kitchens/)

---

# Loading frames

We'll take a look at how you can load the frames off disk and query metadata of the frame segments using a package called [GulpIO](https://github.com/TwentyBN/GulpIO).

```python
import numpy as np
from gulpio import GulpDirectory
from epic_kitchens.dataset.epic_dataset import EpicVideoDataset, EpicVideoFlowDataset, GulpVideoSegment
from pathlib import Path
```

## What is Gulp?

First we should take a little look at what [GulpIO](https://github.com/TwentyBN/GulpIO) is and what data is contained.

GulpIO consists of three main concepts: *ingestion*, *directories* and *chunks*. *Ingestion* is the process of 'Gulping' a dataset, i.e. transforming it from a set of individual image files on disk into a set of gulp *chunks* each containing a number of images and gulp *directories* containing both gulp *chunks* and metadata files.

At its core gulp concatenates image files into larger files (gulp chunks) and produces a metadata file containing the offsets and lengths of each image file inside the chunks.

We recommend storing frames with gulp and our library `epic_kitchens` assumes this if you wish to use the dataset adapters provided.

When producing gulp files we can store arbitrary metadata associated with each segment, we store all information provided in the labels file (verb_class, noun_class, participant_id, ....) as well as details on the image itself: frame_size (width, height, channels) and the number of frames in the segment.

We provide dataset objects that make loading frames a little easier than with raw gulp from the `epic_kitchens` python package.

```python
gulp_root = Path('../data/processed/gulp')
```

```python
# The EpicVideoDataset class encapsulates the logic for reading frames
# from a video and accessing the stored metadata about a segment
# The class_type determines the labels yielded by the dataset object.
# class_type value choices: ['verb', 'noun', 'verb+noun', None], None is
# used for the test sets, where there are no labels.
class_type = 'verb+noun'
rgb_train = EpicVideoDataset(gulp_root / 'rgb_train', class_type)
```

The dataset objects give you access to the [GulpDirectory](https://github.com/TwentyBN/GulpIO#read-a-gulped-dataset). Which gives us access to all gulp metadata contained in the directory, we can see which segments are in the gulp file by inspecting the UIDs.

```python
segment_uids = list(rgb_train.gulp_dir.merged_meta_dict.keys())
print(segment_uids[:10], "...")
print("Number of segments", len(segment_uids))
```

    ['0', '1', '2', '3', '4', '5', '6', '7', '8', '9'] ...
    Number of segments 28472

Each entry in the `merged_meta_dict` has two keys, a `frame_info` key storing internal details about segment's frame locations and `meta_data`, arbitrary metadata we store when ingesting the frames, it is in this that we store details about the segment.

```python
example_meta_entry = rgb_train.gulp_dir.merged_meta_dict[segment_uids[10]]
example_meta_entry.keys()
```

    odict_keys(['frame_info', 'meta_data'])

```python
# Frame info contains the byte offsets of each jpeg, the padding offset
# and the length of the jpeg in the gulp chunk

# Format: (offset, padding, length) (all in bytes)

example_meta_entry['frame_info'][:10]
```

    [[25991980, 0, 27348],
     [26019328, 1, 27412],
     [26046740, 3, 29040],
     [26075780, 1, 29372],
     [26105152, 1, 30944],
     [26136096, 1, 30944],
     [26167040, 3, 30988],
     [26198028, 3, 30656],
     [26228684, 3, 30832],
     [26259516, 2, 30900]]

Gulp stores the metadata dictionaries in a list, however there is only ever one dictionary for each segment

```python
example_meta_entry['meta_data'][0]
```

    OrderedDict([('participant_id', 'P01'),
                 ('video_id', 'P01_01'),
                 ('narration', 'open drawer'),
                 ('start_timestamp', '00:00:24.97'),
                 ('stop_timestamp', '00:00:26.20'),
                 ('start_frame', 1498),
                 ('stop_frame', 1572),
                 ('verb', 'open'),
                 ('verb_class', 2),
                 ('noun', 'drawer'),
                 ('noun_class', 9),
                 ('all_nouns', ['drawer']),
                 ('all_noun_classes', [9]),
                 ('uid', 10),
                 ('frame_size', [256, 456, 3]),
                 ('num_frames', 74)])

# Reading frames

You don't have to deal with fiddling around with any of these details as we wrap them up in `GulpVideoSegment` objects that encapsulate the access to segment metadata including the number of frames and labels. You should work with these.

```python
example_segment = rgb_train.video_segments[10]
example_segment
```

    GulpVideoSegment(OrderedDict([('participant_id', 'P01'), ('video_id', 'P01_01'), ('narration', 'open drawer'), ('start_timestamp', '00:00:24.97'), ('stop_timestamp', '00:00:26.20'), ('start_frame', 1498), ('stop_frame', 1572), ('verb', 'open'), ('verb_class', 2), ('noun', 'drawer'), ('noun_class', 9), ('all_nouns', ['drawer']), ('all_noun_classes', [9]), ('uid', 10), ('frame_size', [256, 456, 3]), ('num_frames', 74)]), <function <lambda> at 0x7fb2dd6366a8>)

We can load all the frames for a segment:

```python
example_frames = rgb_train.load_frames(example_segment)
```

```python
example_frames[0]
```

Frames are loaded as [PIL Images](https://pillow.readthedocs.io/en/5.1.x/reference/Image.html#the-image-class)

```python
type(example_frames[0])
```

    PIL.Image.Image

Which can be easily converted to numpy arrays

```python
np.array(example_frames[0]).shape
```

    (256, 456, 3)

And what is the label of this segment?

```python
example_segment.label
```

    {'verb': 2, 'noun': 9}

If we wished to access other metadata using dot notation

```python
example_segment.verb
```

    'open'

```python
example_segment.noun
```

    'drawer'

or indexing notation

```python
example_segment['verb']
```

    'open'

```python
example_segment['noun']
```

    'drawer'

Or access the metadata dictionary directly

```python
example_segment.metadata
```

    OrderedDict([('participant_id', 'P01'),
                 ('video_id', 'P01_01'),
                 ('narration', 'open drawer'),
                 ('start_timestamp', '00:00:24.97'),
                 ('stop_timestamp', '00:00:26.20'),
                 ('start_frame', 1498),
                 ('stop_frame', 1572),
                 ('verb', 'open'),
                 ('verb_class', 2),
                 ('noun', 'drawer'),
                 ('noun_class', 9),
                 ('all_nouns', ['drawer']),
                 ('all_noun_classes', [9]),
                 ('uid', 10),
                 ('frame_size', [256, 456, 3]),
                 ('num_frames', 74)])

Once the frames are loaded, [moviepy](https://zulko.github.io/moviepy/) can be used to inspect the segments

```python
!pip install moviepy
```

    Requirement already satisfied: moviepy in /jmain01/home/JAD026/dxd01/shared/miniconda3/envs/ar/lib/python3.6/site-packages (0.2.3.5)
    Requirement already satisfied: tqdm<5.0,>=4.11.2 in /jmain01/home/JAD026/dxd01/shared/miniconda3/envs/ar/lib/python3.6/site-packages (from moviepy) (4.25.0)
    Requirement already satisfied: decorator<5.0,>=4.0.2 in /jmain01/home/JAD026/dxd01/shared/miniconda3/envs/ar/lib/python3.6/site-packages (from moviepy) (4.3.0)
    Requirement already satisfied: numpy in /jmain01/home/JAD026/dxd01/shared/miniconda3/envs/ar/lib/python3.6/site-packages (from moviepy) (1.13.3)
    Requirement already satisfied: imageio<3.0,>=2.1.2 in /jmain01/home/JAD026/dxd01/shared/miniconda3/envs/ar/lib/python3.6/site-packages (from moviepy) (2.3.0)
    Requirement already satisfied: pillow in /jmain01/home/JAD026/dxd01/shared/miniconda3/envs/ar/lib/python3.6/site-packages (from imageio<3.0,>=2.1.2->moviepy) (5.2.0)
    You are using pip version 10.0.1, however version 18.0 is available.
    You should consider upgrading via the 'pip install --upgrade pip' command.

```python
from moviepy.editor import ImageSequenceClip
segment_clip = ImageSequenceClip([np.array(frame) for frame in example_frames], fps=60)
segment_clip.ipython_display()
```

    100%|██████████| 75/75 [00:00<00:00, 644.71it/s]

---

Flow can be dealt with similarly as we provide an `EpicVideoFlowDataset` class sharing the same API as `EpicVideoDataset`

```python
flow_train = EpicVideoFlowDataset(gulp_root / 'flow_train', class_type)
```

```python
example_flow_frames = flow_train.load_frames(example_segment, range(0, example_segment.num_frames))
```

Flow frames are stored in alternate $(u, v)$ pairs, all $u$ frames have an even index, whereas $v$ frames have an odd index, so if $g$ is the set of frames in the gulp segment:

$$
\begin{align}
u_t &= g_{2t} \\
v_t &= g_{2t + 1}
\end{align}
$$

```python
def grey_to_rgb(frames):
    """
    Convert frame(s) from gray (2D array) to RGB (3D array)
    """
    n_channels = 3
    return np.repeat(np.array(frames)[..., np.newaxis], n_channels, axis=-1)

def stitch_flow_uv_frames(uv_frames, width_axis=2):
    """
    Destack (u, v) frames and concatenate them side by side for display purposes
    """
    u_frames = np.array(uv_frames[::2])
    v_frames = np.array(uv_frames[1::2])
    stacked_frames = np.concatenate((u_frames, v_frames), axis=width_axis)
    return stacked_frames

def clipify_flow(frames, fps=30):
    """
    Destack flow frames, join them side by side and then create an ImageSequenceClip
    for display
    """
    frames = grey_to_rgb(stitch_flow_uv_frames(list(map(np.array, frames))))
    segment_clip = ImageSequenceClip(list(frames), fps=fps)
    return segment_clip
```

```python
example_segment = flow_train.video_segments[10]
# Note you can also pass an iterable of frame indices to extract, this is useful for
# methods like TSN which only need a few frames
example_flow_frames =  flow_train.load_frames(example_segment, range(0, example_segment.num_frames))
clipify_flow(example_flow_frames).ipython_display()
```

    100%|██████████| 38/38 [00:00<00:00, 330.65it/s]


# Jupyter notebook tips

## Magics

You can run different languages within a notebook.

Use % to start a single-line expression to run with the magics command.
Use %% to run a multi-line expression.

```python
# List all available magics
%lsmagic
```

    Available line magics:
    %alias  %alias_magic  %autocall  %automagic  %autosave  %bookmark  %cat  %cd  %clear  %colors  %config  %connect_info  %cp  %debug  %dhist  %dirs  %doctest_mode  %ed  %edit  %env  %gui  %hist  %history  %killbgscripts  %ldir  %less  %lf  %lk  %ll  %load  %load_ext  %loadpy  %logoff  %logon  %logstart  %logstate  %logstop  %ls  %lsmagic  %lx  %macro  %magic  %man  %matplotlib  %mkdir  %more  %mv  %notebook  %page  %pastebin  %pdb  %pdef  %pdoc  %pfile  %pinfo  %pinfo2  %popd  %pprint  %precision  %profile  %prun  %psearch  %psource  %pushd  %pwd  %pycat  %pylab  %qtconsole  %quickref  %recall  %rehashx  %reload_ext  %rep  %rerun  %reset  %reset_selective  %rm  %rmdir  %run  %save  %sc  %set_env  %store  %sx  %system  %tb  %time  %timeit  %unalias  %unload_ext  %who  %who_ls  %whos  %xdel  %xmode

    Available cell magics:
    %%!  %%HTML  %%SVG  %%bash  %%capture  %%debug  %%file  %%html  %%javascript  %%js  %%latex  %%markdown  %%perl  %%prun  %%pypy  %%python  %%python2  %%python3  %%ruby  %%script  %%sh  %%svg  %%sx  %%system  %%time  %%timeit  %%writefile

    Automagic is ON, % prefix IS NOT needed for line magics.

```python
# List your environment variables
%env
```

    {'PYENV_VIRTUALENV_DISABLE_PROMPT': '1',
     'LANG': 'en_CA.UTF-8',
     'LESS': 'FRXS',
     'DISPLAY': ':0',
     'PYENV_ROOT': '/home/gilles/.pyenv',
     'OLDPWD': '/home/gilles',
     'INVOCATION_ID': 'df1cf28990bc482789c905c8050d9fd1',
     'EDITOR': 'nvim',
     'NODE_PATH': '/home/gilles/.npm-packages/lib/node_modules:',
     'WM': 'i3',
     'PERL_MB_OPT': '--install_base "/home/alp/.local"',
     'MOZ_PLUGIN_PATH': '/usr/lib/mozilla/plugins',
     'PYENV_HOOK_PATH': '/home/gilles/.pyenv/pyenv.d:/usr/local/etc/pyenv.d:/etc/pyenv.d:/usr/lib/pyenv/hooks:/home/gilles/.pyenv/plugins/pyenv-virtualenv/etc/pyenv.d',
     'XDG_VTNR': '1',
     'LESS_TERMCAP_so': '\x1b[1m\x1b[45m\x1b[37m',
     'LESS_TERMCAP_se': '\x1b[0m',
     'XDG_SESSION_ID': 'c1',
     'USER': 'gilles',
     'ENV': '/home/gilles/.rc',
     'PYENV_DIR': '/home/gilles/documents/websites/support.gillespilon.ca/fil',
     'PAGER': 'cat',
     'RBENV_SHELL': 'zsh',
     'PYTHONDOCS': '/usr/share/doc/python/html/library',
     'PWD': '/home/gilles/documents/websites/support.gillespilon.ca/fil',
     'HOME': '/home/gilles',
     'JOURNAL_STREAM': '9:24156',
     'BROWSER': 'chromium',
     'PYENV_VERSION': '3.6.6',
     'PERL_MM_OPT': 'INSTALL_BASE=/home/alp/.local',
     'LESS_TERMCAP_mb': '\x1b[1m\x1b[31m',
     'LESS_TERMCAP_md': '\x1b[1m\x1b[31m',
     'LESS_TERMCAP_me': '\x1b[0m',
     'MAIL_ACCOUNT': 'gillespilon.ca',
     'PERL_LOCAL_LIB_ROOT': '/home/alp/.local',
     'GTK_MODULES': 'canberra-gtk-module',
     'MAIL': '/var/spool/mail/gilles',
     'VISUAL': 'nvim',
     'WINDOWPATH': '1',
     'TERM': 'xterm-color',
     'SHELL': '/usr/bin/zsh',
     'PERL5LIB': '/home/gilles/.local/lib/perl5:/home/gilles/.local/lib/perl5:',
     'PYENV_SHELL': 'zsh',
     'XDG_SEAT': 'seat0',
     'SHLVL': '3',
     'MANPATH': '/home/gilles/.local/share/man:/home/gilles/.cabal/share/man:',
     'WINDOWID': '56623105',
     'LOGNAME': 'gilles',
     'DBUS_SESSION_BUS_ADDRESS': 'unix:path=/run/user/1000/bus',
     'XDG_RUNTIME_DIR': '/run/user/1000',
     'XAUTHORITY': '/home/gilles/.Xauthority',
     'LESS_TERMCAP_ue': '\x1b[0m',
     'LESS_TERMCAP_us': '\x1b[1m\x1b[32m',
     'PATH': '/home/gilles/.pyenv/versions/3.6.6/bin:/home/gilles/.pyenv/versions/3.6.6/bin:/home/gilles/.pyenv/libexec:/home/gilles/.pyenv/plugins/python-build/bin:/home/gilles/.pyenv/plugins/pyenv-virtualenv/bin:/home/gilles/.pyenv/shims:/home/gilles/.rbenv/shims:/home/gilles/.cabal/bin:/home/gilles/.local/bin:/home/gilles/.pyenv/bin:/home/gilles/.rbenv/bin:/home/gilles/.npm-packages/bin:/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/lib/jvm/default/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl:/opt/x-tools/bin:/opt/heroku/bin',
     'SDL_VIDEO_FULLSCREEN_HEAD': '0',
     'JPY_PARENT_PID': '17999',
     'CLICOLOR': '1',
     'GIT_PAGER': 'cat',
     'MPLBACKEND': 'module://ipykernel.pylab.backend_inline'}

```python
# ! to run a shell command
# For example, to see what version of pandas is installed
!pip freeze | grep pandas
```

    pandas==0.23.4

```python
# Show matplotlib plots inline the notebook
%matplotlib inline
```

```python
# Upload code to pastebin and get the url returned
#% pastebin 'file.py'
```

```python
# Run the cell with bash in a subprocess
```

```bash
%%bash
for i in {1..5}
do
   echo "i is $i"
done
```

    i is 1
    i is 2
    i is 3
    i is 4
    i is 5

```python
# Time an evaluation
%time
```

    CPU times: user 2 µs, sys: 0 ns, total: 2 µs
    Wall time: 5.48 µs

```python
# Render cell contents as LaTex
```

```latex
%%latex
\begin{align}
a = \frac{i}{2} && b = \frac{1}{3} && c = \frac{1}{4} \\
a && b && c \\
1 && 2 && 3
\end{align}
```

\begin{align}
a = \frac{i}{2} && b = \frac{1}{3} && c = \frac{1}{4} \\
a && b && c \\
1 && 2 && 3
\end{align}

```python
%timeit x = range(10000)
```

    261 ns ± 4.94 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

```python
# Render the cell as HTML
```

```python
%%HTML
Jupyter Notebooks can render HTML</h1>
```

Jupyter Notebooks can render HTML</h1>

```python
import rpy2.ipython
%load_ext rpy2.ipython
```

    /home/gilles/.pyenv/versions/3.6.6/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88
      return f(*args, **kwds)

```r
%%R
x <- runif(10)
y <- runif(10)
```

```python
%R plot(x, y)
```

You can use %Rpush and %Rpull to move values back and forth between R and Python.

```python
import numpy as np
X = np.array([4.5,6.3,7.9])
X.mean()
```

    6.233333333333334

```python
%Rpush X
%R mean(X)
```

    array([6.23333333])

## Batch, scheduling and reports

```python
import pandas as pd
import folium
```

# References

This jupyter notebook is an exercise for me and is found in these URLs.

[Advanced jupyter notebook tricks part](https://blog.dominodatalab.com/lesser-known-ways-of-using-notebooks/)

[Building interactive dashboards with jupyter](https://blog.dominodatalab.com/interactive-dashboards-in-jupyter/)