# Detector simulation
In this notebook we load a track dataset generated by `edep-sim` and we calculate the ADC counts corresponding to each pixel. The result is exported to a HDF5 file.

```python
# This is need so you can import larndsim without doing python setup.py install
import os,sys,inspect
currentdir = os.path.dirname(os.path.abspath(inspect.getfile(inspect.currentframe())))
parentdir = os.path.dirname(currentdir)
sys.path.insert(0,parentdir)
```

```python
from math import ceil
from time import time

import matplotlib.pyplot as plt
from matplotlib import cm, colors
import mpl_toolkits.mplot3d.art3d as art3d

import numpy as np
import cupy as cp
import fire
import h5py

from numba import cuda
from numba.cuda.random import create_xoroshiro128p_states

import matplotlib as mpl
mpl.rcParams['figure.dpi'] = 100

from tqdm.notebook import tqdm
```

```python
from larndsim import consts
import importlib
importlib.reload(consts)
consts.load_detector_properties("/global/homes/k/kwood/dunend/software/test_ndlarmod/larnd-sim/larndsim/detector_properties/ndlar-module.yaml","/global/homes/k/kwood/dunend/software/test_ndlarmod/larnd-sim/larndsim/pixel_layouts/multi_tile_layout-3.0.40.yaml")

from larndsim import quenching, drifting, detsim, pixels_from_track, fee
```

    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.pixel_trim_dac using bits (0, 512)
    	<function _list_property at 0x2aab38f04170>((<class 'int'>, 0, 31, 64, 8))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.threshold_global using bits (512, 520)
    	<function _basic_property at 0x2aab38f040e0>((<class 'int'>, 0, 255))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.csa_gain using bits (520, 521)
    	<function _compound_property at 0x2aab38f04200>((['csa_gain', 'csa_bypass_enable', 'bypass_caps_en'], (<class 'int'>, <class 'bool'>), 0, 1))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.csa_bypass_enable using bits (521, 522)
    	<function _compound_property at 0x2aab38f04200>((['csa_gain', 'csa_bypass_enable', 'bypass_caps_en'], (<class 'int'>, <class 'bool'>), 0, 1))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.bypass_caps_en using bits (522, 523)
    	<function _compound_property at 0x2aab38f04200>((['csa_gain', 'csa_bypass_enable', 'bypass_caps_en'], (<class 'int'>, <class 'bool'>), 0, 1))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.csa_enable using bits (528, 592)
    	<function _list_property at 0x2aab38f04170>(((<class 'int'>, <class 'bool'>), 0, 1, 64, 1))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.ibias_tdac using bits (592, 596)
    	<function _basic_property at 0x2aab38f040e0>((<class 'int'>, 0, 15))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.ibias_comp using bits (600, 604)
    	<function _basic_property at 0x2aab38f040e0>((<class 'int'>, 0, 15))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.ibias_buffer using bits (608, 612)
    	<function _basic_property at 0x2aab38f040e0>((<class 'int'>, 0, 15))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.ibias_csa using bits (616, 620)
    	<function _basic_property at 0x2aab38f040e0>((<class 'int'>, 0, 15))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.ibias_vref_buffer using bits (624, 628)
    	<function _basic_property at 0x2aab38f040e0>((<class 'int'>, 0, 15))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.ibias_vcm_buffer using bits (632, 636)
    	<function _basic_property at 0x2aab38f040e0>((<class 'int'>, 0, 15))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.ibias_tpulse using bits (640, 644)
    	<function _basic_property at 0x2aab38f040e0>((<class 'int'>, 0, 15))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.ref_current_trim using bits (648, 653)
    	<function _compound_property at 0x2aab38f04200>((['ref_current_trim', 'override_ref', 'ref_kickstart'], <class 'int'>, 0, 31))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.override_ref using bits (653, 654)
    	<function _compound_property at 0x2aab38f04200>((['ref_current_trim', 'override_ref', 'ref_kickstart'], (<class 'int'>, <class 'bool'>), 0, 1))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.ref_kickstart using bits (654, 655)
    	<function _compound_property at 0x2aab38f04200>((['ref_current_trim', 'override_ref', 'ref_kickstart'], (<class 'int'>, <class 'bool'>), 0, 1))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.vref_dac using bits (656, 664)
    	<function _basic_property at 0x2aab38f040e0>((<class 'int'>, 0, 255))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.vcm_dac using bits (664, 672)
    	<function _basic_property at 0x2aab38f040e0>((<class 'int'>, 0, 255))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.csa_bypass_select using bits (672, 736)
    	<function _list_property at 0x2aab38f04170>(((<class 'int'>, <class 'bool'>), 0, 1, 64, 1))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.csa_monitor_select using bits (736, 800)
    	<function _list_property at 0x2aab38f04170>(((<class 'int'>, <class 'bool'>), 0, 1, 64, 1))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.csa_testpulse_enable using bits (800, 864)
    	<function _list_property at 0x2aab38f04170>(((<class 'int'>, <class 'bool'>), 0, 1, 64, 1))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.csa_testpulse_dac using bits (864, 872)
    	<function _basic_property at 0x2aab38f040e0>((<class 'int'>, 0, 255))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.current_monitor_bank0 using bits (872, 876)
    	<function _list_property at 0x2aab38f04170>(((<class 'int'>, <class 'bool'>), 0, 1, 4, 1))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.current_monitor_bank1 using bits (880, 884)
    	<function _list_property at 0x2aab38f04170>(((<class 'int'>, <class 'bool'>), 0, 1, 4, 1))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.current_monitor_bank2 using bits (888, 892)
    	<function _list_property at 0x2aab38f04170>(((<class 'int'>, <class 'bool'>), 0, 1, 4, 1))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.current_monitor_bank3 using bits (896, 900)
    	<function _list_property at 0x2aab38f04170>(((<class 'int'>, <class 'bool'>), 0, 1, 4, 1))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.voltage_monitor_bank0 using bits (904, 907)
    	<function _list_property at 0x2aab38f04170>(((<class 'int'>, <class 'bool'>), 0, 1, 3, 1))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.voltage_monitor_bank1 using bits (912, 915)
    	<function _list_property at 0x2aab38f04170>(((<class 'int'>, <class 'bool'>), 0, 1, 3, 1))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.voltage_monitor_bank2 using bits (920, 923)
    	<function _list_property at 0x2aab38f04170>(((<class 'int'>, <class 'bool'>), 0, 1, 3, 1))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.voltage_monitor_bank3 using bits (928, 931)
    	<function _list_property at 0x2aab38f04170>(((<class 'int'>, <class 'bool'>), 0, 1, 3, 1))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.voltage_monitor_refgen using bits (936, 944)
    	<function _list_property at 0x2aab38f04170>(((<class 'int'>, <class 'bool'>), 0, 1, 8, 1))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.digital_monitor_enable using bits (944, 945)
    	<function _compound_property at 0x2aab38f04200>((['digital_monitor_enable', 'digital_monitor_select'], (<class 'int'>, <class 'bool'>), 0, 1))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.digital_monitor_select using bits (945, 949)
    	<function _compound_property at 0x2aab38f04200>((['digital_monitor_enable', 'digital_monitor_select'], (<class 'int'>, <class 'bool'>), 0, 10))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.digital_monitor_chan using bits (952, 958)
    	<function _basic_property at 0x2aab38f040e0>((<class 'int'>, 0, 63))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.adc_hold_delay using bits (960, 976)
    	<function _basic_property at 0x2aab38f040e0>((<class 'int'>, 0, 65535))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.chip_id using bits (976, 984)
    	<function _basic_property at 0x2aab38f040e0>((<class 'int'>, 0, 255))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.enable_tx_dynamic_powerdown using bits (984, 985)
    	<function _compound_property at 0x2aab38f04200>((['enable_tx_dynamic_powerdown', 'load_config_defaults', 'enable_fifo_diagnostics', 'clk_ctrl', 'tx_dynamic_powerdown_cycles'], (<class 'int'>, <class 'bool'>), 0, 1))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.load_config_defaults using bits (985, 986)
    	<function _compound_property at 0x2aab38f04200>((['enable_tx_dynamic_powerdown', 'load_config_defaults', 'enable_fifo_diagnostics', 'clk_ctrl', 'tx_dynamic_powerdown_cycles'], (<class 'int'>, <class 'bool'>), 0, 1))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.enable_fifo_diagnostics using bits (986, 987)
    	<function _compound_property at 0x2aab38f04200>((['enable_tx_dynamic_powerdown', 'load_config_defaults', 'enable_fifo_diagnostics', 'clk_ctrl', 'tx_dynamic_powerdown_cycles'], (<class 'int'>, <class 'bool'>), 0, 1))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.clk_ctrl using bits (987, 989)
    	<function _compound_property at 0x2aab38f04200>((['enable_tx_dynamic_powerdown', 'load_config_defaults', 'enable_fifo_diagnostics', 'clk_ctrl', 'tx_dynamic_powerdown_cycles'], (<class 'int'>, <class 'bool'>), 0, 3))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.tx_dynamic_powerdown_cycles using bits (989, 992)
    	<function _compound_property at 0x2aab38f04200>((['enable_tx_dynamic_powerdown', 'load_config_defaults', 'enable_fifo_diagnostics', 'clk_ctrl', 'tx_dynamic_powerdown_cycles'], (<class 'int'>, <class 'bool'>), 0, 7))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.enable_piso_upstream using bits (992, 996)
    	<function _list_property at 0x2aab38f04170>(((<class 'int'>, <class 'bool'>), 0, 1, 4, 1))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.enable_piso_downstream using bits (1000, 1004)
    	<function _list_property at 0x2aab38f04170>(((<class 'int'>, <class 'bool'>), 0, 1, 4, 1))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.enable_posi using bits (1008, 1012)
    	<function _list_property at 0x2aab38f04170>(((<class 'int'>, <class 'bool'>), 0, 1, 4, 1))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.test_mode_uart0 using bits (1016, 1018)
    	<function _compound_property at 0x2aab38f04200>((['test_mode_uart0', 'test_mode_uart1', 'test_mode_uart2', 'test_mode_uart3'], <class 'int'>, 0, 3))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.test_mode_uart1 using bits (1018, 1020)
    	<function _compound_property at 0x2aab38f04200>((['test_mode_uart0', 'test_mode_uart1', 'test_mode_uart2', 'test_mode_uart3'], <class 'int'>, 0, 3))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.test_mode_uart2 using bits (1020, 1022)
    	<function _compound_property at 0x2aab38f04200>((['test_mode_uart0', 'test_mode_uart1', 'test_mode_uart2', 'test_mode_uart3'], <class 'int'>, 0, 3))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.test_mode_uart3 using bits (1022, 1024)
    	<function _compound_property at 0x2aab38f04200>((['test_mode_uart0', 'test_mode_uart1', 'test_mode_uart2', 'test_mode_uart3'], <class 'int'>, 0, 3))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.enable_cross_trigger using bits (1024, 1025)
    	<function _compound_property at 0x2aab38f04200>((['enable_cross_trigger', 'enable_periodic_reset', 'enable_rolling_periodic_reset', 'enable_periodic_trigger', 'enable_rolling_periodic_trigger', 'enable_periodic_trigger_veto', 'enable_hit_veto'], <class 'int'>, 0, 1))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.enable_periodic_reset using bits (1025, 1026)
    	<function _compound_property at 0x2aab38f04200>((['enable_cross_trigger', 'enable_periodic_reset', 'enable_rolling_periodic_reset', 'enable_periodic_trigger', 'enable_rolling_periodic_trigger', 'enable_periodic_trigger_veto', 'enable_hit_veto'], <class 'int'>, 0, 1))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.enable_rolling_periodic_reset using bits (1026, 1027)
    	<function _compound_property at 0x2aab38f04200>((['enable_cross_trigger', 'enable_periodic_reset', 'enable_rolling_periodic_reset', 'enable_periodic_trigger', 'enable_rolling_periodic_trigger', 'enable_periodic_trigger_veto', 'enable_hit_veto'], <class 'int'>, 0, 1))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.enable_periodic_trigger using bits (1027, 1028)
    	<function _compound_property at 0x2aab38f04200>((['enable_cross_trigger', 'enable_periodic_reset', 'enable_rolling_periodic_reset', 'enable_periodic_trigger', 'enable_rolling_periodic_trigger', 'enable_periodic_trigger_veto', 'enable_hit_veto'], <class 'int'>, 0, 1))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.enable_rolling_periodic_trigger using bits (1028, 1029)
    	<function _compound_property at 0x2aab38f04200>((['enable_cross_trigger', 'enable_periodic_reset', 'enable_rolling_periodic_reset', 'enable_periodic_trigger', 'enable_rolling_periodic_trigger', 'enable_periodic_trigger_veto', 'enable_hit_veto'], <class 'int'>, 0, 1))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.enable_periodic_trigger_veto using bits (1029, 1030)
    	<function _compound_property at 0x2aab38f04200>((['enable_cross_trigger', 'enable_periodic_reset', 'enable_rolling_periodic_reset', 'enable_periodic_trigger', 'enable_rolling_periodic_trigger', 'enable_periodic_trigger_veto', 'enable_hit_veto'], <class 'int'>, 0, 1))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.enable_hit_veto using bits (1030, 1031)
    	<function _compound_property at 0x2aab38f04200>((['enable_cross_trigger', 'enable_periodic_reset', 'enable_rolling_periodic_reset', 'enable_periodic_trigger', 'enable_rolling_periodic_trigger', 'enable_periodic_trigger_veto', 'enable_hit_veto'], <class 'int'>, 0, 1))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.shadow_reset_length using bits (1032, 1040)
    	<function _basic_property at 0x2aab38f040e0>((<class 'int'>, 0, 255))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.adc_burst_length using bits (1040, 1048)
    	<function _basic_property at 0x2aab38f040e0>((<class 'int'>, 0, 255))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.channel_mask using bits (1048, 1112)
    	<function _list_property at 0x2aab38f04170>((<class 'int'>, 0, 1, 64, 1))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.external_trigger_mask using bits (1112, 1176)
    	<function _list_property at 0x2aab38f04170>((<class 'int'>, 0, 1, 64, 1))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.cross_trigger_mask using bits (1176, 1240)
    	<function _list_property at 0x2aab38f04170>((<class 'int'>, 0, 1, 64, 1))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.periodic_trigger_mask using bits (1240, 1304)
    	<function _list_property at 0x2aab38f04170>((<class 'int'>, 0, 1, 64, 1))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.periodic_reset_cycles using bits (1304, 1328)
    	<function _basic_property at 0x2aab38f040e0>((<class 'int'>, 0, 16777215))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.periodic_trigger_cycles using bits (1328, 1360)
    	<function _basic_property at 0x2aab38f040e0>((<class 'int'>, 0, 4294967295))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.enable_dynamic_reset using bits (1360, 1361)
    	<function _compound_property at 0x2aab38f04200>((['enable_dynamic_reset', 'enable_min_delta_adc', 'threshold_polarity', 'reset_length', 'mark_first_packet'], (<class 'int'>, <class 'bool'>), 0, 1))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.enable_min_delta_adc using bits (1361, 1362)
    	<function _compound_property at 0x2aab38f04200>((['enable_dynamic_reset', 'enable_min_delta_adc', 'threshold_polarity', 'reset_length', 'mark_first_packet'], (<class 'int'>, <class 'bool'>), 0, 1))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.threshold_polarity using bits (1362, 1363)
    	<function _compound_property at 0x2aab38f04200>((['enable_dynamic_reset', 'enable_min_delta_adc', 'threshold_polarity', 'reset_length', 'mark_first_packet'], (<class 'int'>, <class 'bool'>), 0, 1))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.reset_length using bits (1363, 1366)
    	<function _compound_property at 0x2aab38f04200>((['enable_dynamic_reset', 'enable_min_delta_adc', 'threshold_polarity', 'reset_length', 'mark_first_packet'], <class 'int'>, 0, 7))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.mark_first_packet using bits (1366, 1367)
    	<function _compound_property at 0x2aab38f04200>((['enable_dynamic_reset', 'enable_min_delta_adc', 'threshold_polarity', 'reset_length', 'mark_first_packet'], (<class 'int'>, <class 'bool'>), 0, 1))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.reset_threshold using bits (1368, 1376)
    	<function _basic_property at 0x2aab38f040e0>((<class 'int'>, 0, 255))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.min_delta_adc using bits (1376, 1384)
    	<function _basic_property at 0x2aab38f040e0>((<class 'int'>, 0, 255))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.digital_threshold using bits (1384, 1896)
    	<function _list_property at 0x2aab38f04170>((<class 'int'>, 0, 255, 64, 8))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.RESERVED using bits (1896, 1912)
    	<function _basic_property at 0x2aab38f040e0>((<class 'int'>, 0, 0))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.tx_slices0 using bits (1912, 1916)
    	<function _compound_property at 0x2aab38f04200>((['tx_slices0', 'tx_slices1'], <class 'int'>, 0, 15))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.tx_slices1 using bits (1916, 1920)
    	<function _compound_property at 0x2aab38f04200>((['tx_slices0', 'tx_slices1'], <class 'int'>, 0, 15))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.tx_slices2 using bits (1920, 1924)
    	<function _compound_property at 0x2aab38f04200>((['tx_slices2', 'tx_slices3'], <class 'int'>, 0, 15))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.tx_slices3 using bits (1924, 1928)
    	<function _compound_property at 0x2aab38f04200>((['tx_slices2', 'tx_slices3'], <class 'int'>, 0, 15))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.i_tx_diff0 using bits (1928, 1932)
    	<function _compound_property at 0x2aab38f04200>((['i_tx_diff0', 'i_tx_diff1'], <class 'int'>, 0, 15))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.i_tx_diff1 using bits (1932, 1936)
    	<function _compound_property at 0x2aab38f04200>((['i_tx_diff0', 'i_tx_diff1'], <class 'int'>, 0, 15))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.i_tx_diff2 using bits (1936, 1940)
    	<function _compound_property at 0x2aab38f04200>((['i_tx_diff2', 'i_tx_diff3'], <class 'int'>, 0, 15))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.i_tx_diff3 using bits (1940, 1944)
    	<function _compound_property at 0x2aab38f04200>((['i_tx_diff2', 'i_tx_diff3'], <class 'int'>, 0, 15))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.i_rx0 using bits (1944, 1948)
    	<function _compound_property at 0x2aab38f04200>((['i_rx0', 'i_rx1'], <class 'int'>, 0, 15))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.i_rx1 using bits (1948, 1952)
    	<function _compound_property at 0x2aab38f04200>((['i_rx0', 'i_rx1'], <class 'int'>, 0, 15))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.i_rx2 using bits (1952, 1956)
    	<function _compound_property at 0x2aab38f04200>((['i_rx2', 'i_rx3'], <class 'int'>, 0, 15))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.i_rx3 using bits (1956, 1960)
    	<function _compound_property at 0x2aab38f04200>((['i_rx2', 'i_rx3'], <class 'int'>, 0, 15))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.i_rx_clk using bits (1960, 1964)
    	<function _compound_property at 0x2aab38f04200>((['i_rx_clk', 'i_rx_rst'], <class 'int'>, 0, 15))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.i_rx_rst using bits (1964, 1968)
    	<function _compound_property at 0x2aab38f04200>((['i_rx_clk', 'i_rx_rst'], <class 'int'>, 0, 15))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.i_rx_ext_trig using bits (1968, 1972)
    	<function _basic_property at 0x2aab38f040e0>((<class 'int'>, 0, 15))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.r_term0 using bits (1976, 1981)
    	<function _basic_property at 0x2aab38f040e0>((<class 'int'>, 0, 31))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.r_term1 using bits (1984, 1989)
    	<function _basic_property at 0x2aab38f040e0>((<class 'int'>, 0, 31))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.r_term2 using bits (1992, 1997)
    	<function _basic_property at 0x2aab38f040e0>((<class 'int'>, 0, 31))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.r_term3 using bits (2000, 2005)
    	<function _basic_property at 0x2aab38f040e0>((<class 'int'>, 0, 31))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.r_term_clk using bits (2008, 2013)
    	<function _basic_property at 0x2aab38f040e0>((<class 'int'>, 0, 31))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.r_term_reset using bits (2016, 2021)
    	<function _basic_property at 0x2aab38f040e0>((<class 'int'>, 0, 31))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.r_term_ext_trig using bits (2024, 2029)
    	<function _basic_property at 0x2aab38f040e0>((<class 'int'>, 0, 31))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.v_cm_lvds_tx0 using bits (2032, 2035)
    	<function _compound_property at 0x2aab38f04200>((['v_cm_lvds_tx0', 'v_cm_lvds_tx0'], <class 'int'>, 0, 7))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.v_cm_lvds_tx1 using bits (2036, 2039)
    	<function _compound_property at 0x2aab38f04200>((['v_cm_lvds_tx0', 'v_cm_lvds_tx0'], <class 'int'>, 0, 7))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.v_cm_lvds_tx2 using bits (2040, 2043)
    	<function _compound_property at 0x2aab38f04200>((['v_cm_lvds_tx2', 'v_cm_lvds_tx3'], <class 'int'>, 0, 7))
    Generate <class 'larpix.configuration.configuration_v2b.Configuration_v2b'>.v_cm_lvds_tx3 using bits (2044, 2047)
    	<function _compound_property at 0x2aab38f04200>((['v_cm_lvds_tx2', 'v_cm_lvds_tx3'], <class 'int'>, 0, 7))

```python
response = cp.load('larndsim/response_44.npy')
print(response)
```

    [[[ 0.          0.          0.         ...  0.          0.
        0.        ]
      [ 0.          0.          0.         ...  0.          0.
        0.        ]
      [ 0.          0.          0.         ...  0.          0.
        0.        ]
      ...
      [ 0.00159894  0.00159894  0.00159894 ...  0.          0.
        0.        ]
      [ 0.00154477  0.00154477  0.00154477 ...  0.          0.
        0.        ]
      [ 0.00144699  0.00144699  0.00144699 ...  0.          0.
        0.        ]]

     [[ 0.          0.          0.         ...  0.          0.
        0.        ]
      [ 0.          0.          0.         ...  0.          0.
        0.        ]
      [ 0.          0.          0.         ...  0.          0.
        0.        ]
      ...
      [ 0.00162965  0.00162965  0.00162965 ...  0.          0.
        0.        ]
      [ 0.00155165  0.00155165  0.00155165 ...  0.          0.
        0.        ]
      [ 0.00142135  0.00142135  0.00142135 ...  0.          0.
        0.        ]]

     [[ 0.          0.          0.         ...  0.          0.
        0.        ]
      [ 0.          0.          0.         ...  0.          0.
        0.        ]
      [ 0.          0.          0.         ...  0.          0.
        0.        ]
      ...
      [ 0.00161736  0.00161736  0.00161736 ...  0.          0.
        0.        ]
      [ 0.00154444  0.00154444  0.00154444 ...  0.          0.
        0.        ]
      [ 0.00144821  0.00144821  0.00144821 ...  0.          0.
        0.        ]]

     ...

     [[ 0.00159894  0.00159894  0.00159894 ...  0.          0.
        0.        ]
      [ 0.00162965  0.00162965  0.00162965 ...  0.          0.
        0.        ]
      [ 0.00161736  0.00161736  0.00161736 ...  0.          0.
        0.        ]
      ...
      [ 0.00126394  0.00126394  0.00126394 ...  0.          0.
        0.        ]
      [ 0.00122165  0.00122165  0.00122165 ...  0.          0.
        0.        ]
      [ 0.00116178  0.00116178  0.00116178 ...  0.          0.
        0.        ]]

     [[ 0.00154477  0.00154477  0.00154477 ...  0.          0.
        0.        ]
      [ 0.00155165  0.00155165  0.00155165 ...  0.          0.
        0.        ]
      [ 0.00154444  0.00154444  0.00154444 ...  0.          0.
        0.        ]
      ...
      [ 0.00122165  0.00122165  0.00122165 ...  0.          0.
        0.        ]
      [ 0.00115429  0.00115429  0.00115429 ...  0.          0.
        0.        ]
      [ 0.00110303  0.00110303  0.00110303 ...  0.          0.
        0.        ]]

     [[ 0.00144699  0.00144699  0.00144699 ...  0.          0.
        0.        ]
      [ 0.00142135  0.00142135  0.00142135 ...  0.          0.
        0.        ]
      [ 0.00144821  0.00144821  0.00144821 ...  0.          0.
        0.        ]
      ...
      [ 0.00116178  0.00116178  0.00116178 ...  0.          0.
        0.        ]
      [ 0.00110303  0.00110303  0.00110303 ...  0.          0.
        0.        ]
      [ 0.00103613  0.00103613  0.00103613 ... -0.00283607 -0.00648338
        0.        ]]]

resp### Dataset import
First of all we load the `edep-sim` output. For this sample we need to invert $z$ and $y$ axes.

```python
with h5py.File('/global/homes/k/kwood/dunend/software/test_ndlarmod/larnd-sim/inputs/LAr.neutrino.100.edep.h5', 'r') as f:
    tracks = np.array(f['segments'])

x_start = np.copy(tracks['x_start'] )
x_end = np.copy(tracks['x_end'])
x = np.copy(tracks['x'])

tracks['x_start'] = np.copy(tracks['z_start'])
tracks['x_end'] = np.copy(tracks['z_end'])
tracks['x'] = np.copy(tracks['z'])

tracks['z_start'] = x_start
tracks['z_end'] = x_end
tracks['z'] = x

selected_tracks = tracks[:20]
```

### Quenching and drifting
We calculate the number of electrons after recombination (`quenching` module) and the position and number of electrons after drifting (`drifting` module).

```python
TPB = 256
BPG = ceil(selected_tracks.shape[0] / TPB)
quenching.quench[BPG,TPB](selected_tracks, consts.birks)
drifting.drift[BPG,TPB](selected_tracks)
```

    /global/homes/k/kwood/.conda/envs/kenv/lib/python3.7/site-packages/numba/cuda/compiler.py:865: NumbaPerformanceWarning: Grid size (1) < 2 * SM count (160) will likely result in GPU under utilization due to low occupancy.
      warn(NumbaPerformanceWarning(msg))
    /global/homes/k/kwood/.conda/envs/kenv/lib/python3.7/site-packages/numba/cuda/cudadrv/devicearray.py:790: NumbaPerformanceWarning: Host array used in CUDA kernel will incur copy overhead to/from device.
      warn(NumbaPerformanceWarning(msg))
    /global/homes/k/kwood/.conda/envs/kenv/lib/python3.7/site-packages/numba/cuda/compiler.py:865: NumbaPerformanceWarning: Grid size (1) < 2 * SM count (160) will likely result in GPU under utilization due to low occupancy.
      warn(NumbaPerformanceWarning(msg))
    /global/homes/k/kwood/.conda/envs/kenv/lib/python3.7/site-packages/numba/cuda/cudadrv/devicearray.py:790: NumbaPerformanceWarning: Host array used in CUDA kernel will incur copy overhead to/from device.
      warn(NumbaPerformanceWarning(msg))

We find the pixels intersected by the projection of the tracks on the anode plane using the Bresenham's algorithm. We also take into account the neighboring pixels, due to the transverse diffusion of the charges.

```python
# Here we build a map between tracks and event IDs
unique_eventIDs = cp.unique(selected_tracks['eventID'])
event_id_map = cp.searchsorted(unique_eventIDs,cp.asarray(selected_tracks['eventID']))

longest_pix = ceil(max(selected_tracks["dx"])/consts.pixel_pitch)
max_radius = ceil(max(selected_tracks["tran_diff"])*5/consts.pixel_pitch)

MAX_PIXELS = int((longest_pix*4+6)*max_radius*1.5)
MAX_ACTIVE_PIXELS = int(longest_pix*1.5)

active_pixels = cp.full((selected_tracks.shape[0], MAX_ACTIVE_PIXELS, 2), -1, dtype=np.int32)
neighboring_pixels = cp.full((selected_tracks.shape[0], MAX_PIXELS, 2), -1, dtype=np.int32)
n_pixels_list = cp.zeros(shape=(selected_tracks.shape[0]))
TPB = 128
BPG = ceil(selected_tracks.shape[0] / TPB)
pixels_from_track.get_pixels[BPG,TPB](selected_tracks,
                                      active_pixels,
                                      neighboring_pixels,
                                      n_pixels_list,
                                      max_radius+1)
```

    /global/homes/k/kwood/.conda/envs/kenv/lib/python3.7/site-packages/numba/cuda/compiler.py:865: NumbaPerformanceWarning: Grid size (1) < 2 * SM count (160) will likely result in GPU under utilization due to low occupancy.
      warn(NumbaPerformanceWarning(msg))
    /global/homes/k/kwood/.conda/envs/kenv/lib/python3.7/site-packages/numba/cuda/cudadrv/devicearray.py:790: NumbaPerformanceWarning: Host array used in CUDA kernel will incur copy overhead to/from device.
      warn(NumbaPerformanceWarning(msg))

```python
def cupy_unique_axis0(array):
    # axis is still not supported for cupy.unique, this
    # is a workaround
    if len(array.shape) != 2:
        raise ValueError("Input array must be 2D.")
    sortarr     = array[cp.lexsort(array.T[::-1])]
    mask        = cp.empty(array.shape[0], dtype=cp.bool_)
    mask[0]     = True
    mask[1:]    = cp.any(sortarr[1:] != sortarr[:-1], axis=1)
    return sortarr[mask]

shapes = neighboring_pixels.shape
joined = neighboring_pixels.reshape(shapes[0]*shapes[1],2)
unique_pix = cupy_unique_axis0(joined)
unique_pix = unique_pix[(unique_pix[:,0] != -1) & (unique_pix[:,1] != -1),:]
print(len(unique_pix))
#print(cp.remainder(list(unique_pix),[70,70]))
print(unique_pix)
#for pix in unique_pix:
#    pix[0] = pix[0]%128
#    pix[1] = pix[1]%80
#print(unique_pix)
print(type(unique_pix))
```

    311
    [[  4  54]
     [  5  50]
     [  5  51]
     [  5  52]
     [  5  53]
     [  5  54]
     [  6  50]
     [  6  51]
     [  6  52]
     [  6  53]
     [  6  54]
     [  7  50]
     [  7  51]
     [  7  52]
     [  7  53]
     [  7  54]
     [  8  50]
     [  8  51]
     [  8  52]
     [  8  53]
     [  8  54]
     [ 34  47]
     [ 35  43]
     [ 35  44]
     [ 35  45]
     [ 35  46]
     [ 35  47]
     [ 36  43]
     [ 36  44]
     [ 36  45]
     [ 36  46]
     [ 36  47]
     [ 37  43]
     [ 37  44]
     [ 37  45]
     [ 37  46]
     [ 37  47]
     [ 38  43]
     [ 38  44]
     [ 38  45]
     [ 38  46]
     [ 38  47]
     [ 56  55]
     [ 57  51]
     [ 57  52]
     [ 57  53]
     [ 57  54]
     [ 57  55]
     [ 58  51]
     [ 58  52]
     [ 58  53]
     [ 58  54]
     [ 58  55]
     [ 59  51]
     [ 59  52]
     [ 59  53]
     [ 59  54]
     [ 59  55]
     [ 60  51]
     [ 60  52]
     [ 60  53]
     [ 60  54]
     [ 60  55]
     [ 60  56]
     [ 61  52]
     [ 61  53]
     [ 61  54]
     [ 61  55]
     [ 61  56]
     [ 61 143]
     [ 62  52]
     [ 62  53]
     [ 62  54]
     [ 62  55]
     [ 62  56]
     [ 62 134]
     [ 62 135]
     [ 62 136]
     [ 62 137]
     [ 62 138]
     [ 62 139]
     [ 62 140]
     [ 62 141]
     [ 62 142]
     [ 62 143]
     [ 63  52]
     [ 63  53]
     [ 63  54]
     [ 63  55]
     [ 63  56]
     [ 63 134]
     [ 63 135]
     [ 63 136]
     [ 63 137]
     [ 63 138]
     [ 63 139]
     [ 63 140]
     [ 63 141]
     [ 63 142]
     [ 63 143]
     [ 64  52]
     [ 64  53]
     [ 64  54]
     [ 64  55]
     [ 64  56]
     [ 64 134]
     [ 64 135]
     [ 64 136]
     [ 64 137]
     [ 64 138]
     [ 64 139]
     [ 64 140]
     [ 64 141]
     [ 64 142]
     [ 64 143]
     [ 65 134]
     [ 65 135]
     [ 65 136]
     [ 65 137]
     [ 65 138]
     [ 65 139]
     [ 65 140]
     [ 65 141]
     [ 65 142]
     [ 65 143]
     [ 66 134]
     [ 66 135]
     [ 66 136]
     [ 66 137]
     [ 66 138]
     [128  30]
     [128  31]
     [128  32]
     [128  33]
     [128  34]
     [129  30]
     [129  31]
     [129  32]
     [129  33]
     [129  34]
     [130  30]
     [130  31]
     [130  32]
     [130  33]
     [130  34]
     [131  30]
     [131  31]
     [131  32]
     [131  33]
     [131  34]
     [132  30]
     [132  31]
     [132  32]
     [132  33]
     [132  34]
     [133  30]
     [133  31]
     [133  32]
     [133  33]
     [133  34]
     [197  85]
     [198  81]
     [198  82]
     [198  83]
     [198  84]
     [198  85]
     [199  81]
     [199  82]
     [199  83]
     [199  84]
     [199  85]
     [200  81]
     [200  82]
     [200  83]
     [200  84]
     [200  85]
     [201  81]
     [201  82]
     [201  83]
     [201  84]
     [201  85]
     [205  41]
     [205  42]
     [205  43]
     [205  44]
     [205  45]
     [206  41]
     [206  42]
     [206  43]
     [206  44]
     [206  45]
     [207  41]
     [207  42]
     [207  43]
     [207  44]
     [207  45]
     [208  41]
     [208  42]
     [208  43]
     [208  44]
     [208  45]
     [209  41]
     [209  42]
     [209  43]
     [209  44]
     [209  45]
     [211  52]
     [212  48]
     [212  49]
     [212  50]
     [212  51]
     [212  52]
     [213  48]
     [213  49]
     [213  50]
     [213  51]
     [213  52]
     [214  48]
     [214  49]
     [214  50]
     [214  51]
     [214  52]
     [215  48]
     [215  49]
     [215  50]
     [215  51]
     [215  52]
     [245  16]
     [246  12]
     [246  13]
     [246  14]
     [246  15]
     [246  16]
     [247  12]
     [247  13]
     [247  14]
     [247  15]
     [247  16]
     [248  12]
     [248  13]
     [248  14]
     [248  15]
     [248  16]
     [249  12]
     [249  13]
     [249  14]
     [249  15]
     [249  16]
     [451 121]
     [452 117]
     [452 118]
     [452 119]
     [452 120]
     [452 121]
     [453 117]
     [453 118]
     [453 119]
     [453 120]
     [453 121]
     [454 117]
     [454 118]
     [454 119]
     [454 120]
     [454 121]
     [455 117]
     [455 118]
     [455 119]
     [455 120]
     [455 121]
     [464 125]
     [465 121]
     [465 122]
     [465 123]
     [465 124]
     [465 125]
     [466 121]
     [466 122]
     [466 123]
     [466 124]
     [466 125]
     [467 121]
     [467 122]
     [467 123]
     [467 124]
     [467 125]
     [468 121]
     [468 122]
     [468 123]
     [468 124]
     [468 125]
     [484  20]
     [485  16]
     [485  17]
     [485  18]
     [485  19]
     [485  20]
     [486  16]
     [486  17]
     [486  18]
     [486  19]
     [486  20]
     [487  16]
     [487  17]
     [487  18]
     [487  19]
     [487  20]
     [488  16]
     [488  17]
     [488  18]
     [488  19]
     [488  20]]
    <class 'cupy.core.core.ndarray'>

### Charge distribution calculation
Here we calculate the current induced by each track on the pixels, taking into account longitudinal and transverse diffusion. The track segment is parametrized as:
\begin{align}
x'(r') &=x_s + \frac{\Delta x}{\Delta r}r'\\
y'(r') &=y_s + \frac{\Delta y}{\Delta r}r'\\
z'(r') &=z_s + \frac{\Delta z}{\Delta r}r',
\end{align}
where $\Delta r$ is the segment length. Here we assume $z$ as the drift direction.
The diffused charge distribution is calculated with the following integral:
\begin{equation}
\rho(x,y,z) = \frac{Q}{\sqrt{(2\pi)^3}\sigma_x\sigma_y\sigma_z\Delta r}\exp\left[-\frac{(x-x_s)^2}{2\sigma_x^2}-\frac{(y-y_s)^2}{2\sigma_y^2}-\frac{(z-z_s)^2}{2\sigma_z^2}\right]\int^{r'=\Delta r}_{r'=0}dr'\exp[-(ar'^2+br')],
\end{equation}
where
\begin{align}
a &= \left[\left(\frac{\Delta x}{\Delta r}\right)^2\frac{1}{2\sigma_x^2} + \left(\frac{\Delta y}{\Delta r}\right)^2\frac{1}{2\sigma_y^2} + \left(\frac{\Delta z}{\Delta r}\right)^2\frac{1}{2\sigma_z^2} \right]\\
b &= -\left[\frac{(x-x_s)}{\sigma_x^2}\frac{\Delta x}{\Delta r}+
\frac{(y-y_s)}{\sigma_y^2}\frac{\Delta y}{\Delta r}+
\frac{(z-z_s)}{\sigma_z^2}\frac{\Delta z}{\Delta r}\right].
\end{align}

The simmetry of the transverse diffusion along the track allows to take a slice on the $xy$ plane and solve the integral once at a fixed $z$ coordinate (e.g. at $z_{m} = (z_s+z_e)/2$) and re-use it at other $z$ coordinates away from the endpoints (where $\rho(x,y,z)$ varies along $z$ so must be calculated at each $z$).

```python
max_length = cp.array([0])
track_starts = cp.empty(selected_tracks.shape[0])
threadsperblock = 128
blockspergrid = ceil(selected_tracks.shape[0] / threadsperblock)
detsim.time_intervals[blockspergrid,threadsperblock](track_starts, max_length,  event_id_map, selected_tracks)

signals = cp.zeros((selected_tracks.shape[0],
                    neighboring_pixels.shape[1],
                    cp.asnumpy(max_length)[0]), dtype=np.float32)
threadsperblock = (1,1,64)
importlib.reload(detsim)
blockspergrid_x = ceil(signals.shape[0] / threadsperblock[0])
blockspergrid_y = ceil(signals.shape[1] / threadsperblock[1])
blockspergrid_z = ceil(signals.shape[2] / threadsperblock[2])
blockspergrid = (blockspergrid_x, blockspergrid_y, blockspergrid_z)
detsim.tracks_current[blockspergrid,threadsperblock](signals,
                                                     neighboring_pixels,
                                                     selected_tracks,
                                                     response)
```

    /global/homes/k/kwood/.conda/envs/kenv/lib/python3.7/site-packages/numba/cuda/compiler.py:865: NumbaPerformanceWarning: Grid size (1) < 2 * SM count (160) will likely result in GPU under utilization due to low occupancy.
      warn(NumbaPerformanceWarning(msg))
    /global/homes/k/kwood/.conda/envs/kenv/lib/python3.7/site-packages/numba/cuda/cudadrv/devicearray.py:790: NumbaPerformanceWarning: Host array used in CUDA kernel will incur copy overhead to/from device.
      warn(NumbaPerformanceWarning(msg))
    /global/homes/k/kwood/.conda/envs/kenv/lib/python3.7/site-packages/numba/cuda/cudadrv/devicearray.py:790: NumbaPerformanceWarning: Host array used in CUDA kernel will incur copy overhead to/from device.
      warn(NumbaPerformanceWarning(msg))

Here we create a map that associates each track to the pixels where it induces a current

```python
pixel_index_map = cp.full((selected_tracks.shape[0], neighboring_pixels.shape[1]), -1)

try:
    compare = neighboring_pixels[..., np.newaxis, :] == unique_pix
    indices = cp.where(cp.logical_and(compare[..., 0], compare[..., 1]))
except cp.cuda.memory.OutOfMemoryError:
    print("out of memory")

pixel_index_map[indices[0], indices[1]] = indices[2]
```

The three-dimensional array can contain more than one signal for each pixel at different times. If we want to plot the full induced signal on the pixel, we need to join the signals corresponding to the same pixel. First, we find the start time of each signal with `time_intervals`:

```python
# Mapping between unique pixel array and track array index
track_pixel_map = cp.full((unique_pix.shape[0], detsim.MAX_TRACKS_PER_PIXEL), -1)
TPB = 32
BPG = ceil(unique_pix.shape[0] / TPB)
detsim.get_track_pixel_map[BPG, TPB](track_pixel_map, unique_pix, neighboring_pixels)

threadsperblock = (1,1,64)
blockspergrid_x = ceil(signals.shape[0] / threadsperblock[0])
blockspergrid_y = ceil(signals.shape[1] / threadsperblock[1])
blockspergrid_z = ceil(signals.shape[2] / threadsperblock[2])
blockspergrid = (blockspergrid_x, blockspergrid_y, blockspergrid_z)
pixels_signals = cp.zeros((len(unique_pix), len(consts.time_ticks)*3))
pixels_tracks_signals = cp.zeros((len(unique_pix),len(consts.time_ticks)*3,track_pixel_map.shape[1]))

detsim.sum_pixel_signals[blockspergrid,threadsperblock](pixels_signals,
                                                        signals,
                                                        track_starts,
                                                        pixel_index_map,
                                                        track_pixel_map,
                                                        pixels_tracks_signals)
currents = np.sum(pixels_signals,axis=1)*consts.t_sampling/consts.e_charge
```

    /global/homes/k/kwood/.conda/envs/kenv/lib/python3.7/site-packages/numba/cuda/compiler.py:865: NumbaPerformanceWarning: Grid size (10) < 2 * SM count (160) will likely result in GPU under utilization due to low occupancy.
      warn(NumbaPerformanceWarning(msg))

Now, let's verify that the integral of our induced currents

```python
print("Ratio: ", sum(currents)/sum(selected_tracks['n_electrons']))
```

    Ratio:  1.2072718575214823

### 3D event display

```python
cmap = cm.Spectral_r
from matplotlib import colors
norm = colors.Normalize(vmin=0, vmax=256)
m = cm.ScalarMappable(norm=norm, cmap=cmap)

cmap = cm.viridis
norm_curr = colors.LogNorm(vmin=min(currents[currents>0]), vmax=max(currents))
m_curr = cm.ScalarMappable(norm=norm_curr, cmap=cmap)
fig = plt.figure(figsize=(7,7))
ax = fig.add_subplot(111, projection='3d')

for it,t in enumerate(selected_tracks):
    if it == 0:
        ax.plot((t["x_start"], t["x_end"]),
            (t["z_start"], t["z_end"]),
            (t["y_start"], t["y_end"]),
            c='r',
            lw=1,
            alpha=1,
            zorder=10,
            label='Geant4 detector segment')
    else:
        ax.plot((t["x_start"], t["x_end"]),
                (t["z_start"], t["z_end"]),
                (t["y_start"], t["y_end"]),
                c='r',
                lw=1,
                alpha=1,
                zorder=9999)
    #print(t['pixel_plane'])
    if 0<=t['pixel_plane']<=999:
        ax.plot((t["x_start"], t["x_end"]),
            (consts.tpc_borders[t['pixel_plane']][2][0], consts.tpc_borders[t['pixel_plane']][2][0]),
            (t["y_start"], t["y_end"]),
            c='r',
            lw=1,
            ls=':',
            alpha=1,
            zorder=9999)

for ip, p in enumerate(unique_pix):
    x_rect, y_rect = detsim.get_pixel_coordinates(p.get())
    pixel_plane = int(p[0] // consts.n_pixels[0])
    #print(pixel_plane)
    row = int(pixel_plane // 4)
    column = int(pixel_plane % 4)
    if currents[ip] > 0:
        rect = plt.Rectangle((x_rect, y_rect),
                             consts.pixel_pitch, consts.pixel_pitch,
                             linewidth=0.1, fc=m_curr.to_rgba(currents[ip].get()),
                             edgecolor='gray')
        ax.add_patch(rect)
        art3d.pathpatch_2d_to_3d(rect, z=consts.tpc_borders[pixel_plane][2][0], zdir="y")

anode1 = plt.Rectangle((consts.tpc_borders[0][0][0], consts.tpc_borders[0][1][0]),
                        consts.tpc_borders[0][0][1]-consts.tpc_borders[0][0][0],
                        consts.tpc_borders[0][1][1]-consts.tpc_borders[0][1][0],
                       linewidth=1, fc='none',
                       edgecolor='gray', label=('Pixel' if ip == 5 else ''))
ax.add_patch(anode1)
art3d.pathpatch_2d_to_3d(anode1, z=consts.tpc_borders[0][2][0], zdir="y")

anode2 = plt.Rectangle((consts.tpc_borders[0][0][0], consts.tpc_borders[0][1][0]),
                        consts.tpc_borders[0][0][1]-consts.tpc_borders[0][0][0],
                        consts.tpc_borders[0][1][1]-consts.tpc_borders[0][1][0],
                       linewidth=1, fc='none',
                       edgecolor='gray', label=('Pixel' if ip == 5 else ''))
ax.add_patch(anode2)
art3d.pathpatch_2d_to_3d(anode2, z=consts.tpc_borders[1][2][0], zdir="y")

cathode = plt.Rectangle((consts.tpc_borders[0][0][0], consts.tpc_borders[0][1][0]),
                        consts.tpc_borders[0][0][1]-consts.tpc_borders[0][0][0],
                        consts.tpc_borders[0][1][1]-consts.tpc_borders[0][1][0],
                        linewidth=1, fc='gray', alpha=0.5,
                        edgecolor='gray', label=('Pixel' if ip == 5 else ''))
ax.add_patch(cathode)
art3d.pathpatch_2d_to_3d(cathode, z=0, zdir="y")

ax.plot((consts.tpc_borders[0][0][0],consts.tpc_borders[0][0][0]),(consts.tpc_borders[0][2][0],consts.tpc_borders[1][2][0]),
        (consts.tpc_borders[0][1][0],consts.tpc_borders[0][1][0]), lw=1,color='gray')

ax.plot((consts.tpc_borders[0][0][0],consts.tpc_borders[0][0][0]),(consts.tpc_borders[0][2][0],consts.tpc_borders[1][2][0]),
        (consts.tpc_borders[0][1][1],consts.tpc_borders[0][1][1]), lw=1,color='gray')

ax.plot((consts.tpc_borders[0][0][1],consts.tpc_borders[0][0][1]),(consts.tpc_borders[0][2][0],consts.tpc_borders[1][2][0]),
        (consts.tpc_borders[0][1][0],consts.tpc_borders[0][1][0]), lw=1,color='gray')

ax.plot((consts.tpc_borders[0][0][1],consts.tpc_borders[0][0][1]),(consts.tpc_borders[0][2][0],consts.tpc_borders[1][2][0]),
        (consts.tpc_borders[0][1][1],consts.tpc_borders[0][1][1]), lw=1,color='gray')

# ax.set_ylim(consts.module_borders[pixel_plane][2][0],50)
ax.set_xlim(consts.tpc_borders[0][0][0],consts.tpc_borders[1][0][1])
ax.set_ylim(consts.tpc_borders[0][2][0],consts.tpc_borders[1][2][0])
ax.set_zlim(consts.tpc_borders[0][1][0],consts.tpc_borders[1][1][1])

ax.set_box_aspect((2,2,4))
ax.grid(False)
ax.xaxis.set_major_locator(plt.MaxNLocator(3))
ax.yaxis.set_major_locator(plt.MaxNLocator(3))
# ax.set_axis_off()
ax.w_xaxis.set_pane_color((1.0, 1.0, 1.0, 1.0))
ax.w_yaxis.set_pane_color((1.0, 1.0, 1.0, 1.0))
ax.w_zaxis.set_pane_color((1.0, 1.0, 1.0, 1.0))

ax.view_init(azim=20)
def rotate(angle):
    ax.view_init(azim=angle)

from matplotlib import animation

rot_animation = animation.FuncAnimation(fig, rotate, frames=np.arange(0,362,2),interval=50)

ax.set_ylabel("z [cm]")
ax.set_xlabel("x [cm]")
ax.set_zlabel("y [cm]")
_ = plt.colorbar(m_curr,fraction=0.035, pad=0.05,label='Induced current integral [# electrons]')

```

```python
#rot_animation.save('animation.gif', writer='imagemagick', fps=10, bitrate=10, dpi=100)
```

### Electronics response and digitization
Here we simulate the electronics response (the self-triggering cycle) and the signal digitization.

```python
time_ticks = cp.linspace(0, len(unique_eventIDs)*consts.time_interval[1]*3, pixels_signals.shape[1]+1)
integral_list = cp.zeros((pixels_signals.shape[0], fee.MAX_ADC_VALUES))
adc_ticks_list = cp.zeros((pixels_signals.shape[0], fee.MAX_ADC_VALUES))
TPB = 128
BPG = ceil(pixels_signals.shape[0] / TPB)
rng_states = create_xoroshiro128p_states(TPB * BPG, seed=2)
integrate = cp.zeros((pixels_signals.shape[0], time_ticks.shape[0]))
backtracked_ids = cp.zeros((pixels_signals.shape[0], fee.MAX_ADC_VALUES, track_pixel_map.shape[1]))
importlib.reload(fee)
fee.get_adc_values[BPG,TPB](pixels_signals,
                            pixels_tracks_signals,
                            time_ticks,
                            integral_list,
                            adc_ticks_list,
                            0,
                            rng_states,
                            backtracked_ids)
adc_list = fee.digitize(integral_list)
```

    /global/homes/k/kwood/.conda/envs/kenv/lib/python3.7/site-packages/numba/cuda/compiler.py:865: NumbaPerformanceWarning: Grid size (3) < 2 * SM count (160) will likely result in GPU under utilization due to low occupancy.
      warn(NumbaPerformanceWarning(msg))

As an example, we plot the pixel with the largest amount of deposited charge

```python
largest_pix = np.argmax(pixels_signals.sum(axis=1)).get()
```

```python
fig,ax=plt.subplots(1,1)
current = pixels_signals[largest_pix][:2001].get()
ax.plot(consts.time_ticks,pixels_tracks_signals[largest_pix][:2001].get())
ax.plot(consts.time_ticks,current,label='Total track contribution',c='k',lw=2)
ax.plot(consts.time_ticks,np.sum(pixels_tracks_signals[largest_pix][:2001].get(),axis=1),c='r',ls='--',label='Sum of each track contribution')
start_time = np.nonzero(current)[0][0]
ax.set_xlim(consts.time_ticks[start_time]-10,consts.time_ticks[start_time]+30)
ax.set_xlabel(r"Time [$\mathrm{\mu}$s]")
ax.set_ylabel(r"Current [A]")
ax.set_ylim(-0.1e-14, max(current)*1.3)
_=ax.legend()
```

### 2D event display with induced current and ADC counts

```python
from matplotlib import colors
norm = colors.Normalize(vmin=0, vmax=256)
m = cm.ScalarMappable(norm=norm, cmap=cmap)

cmap = cm.viridis
norm_curr = colors.LogNorm(vmin=min(currents[currents>0]), vmax=max(currents))
m_curr = cm.ScalarMappable(norm=norm_curr, cmap=cmap)

fig,ax = plt.subplots(1,2,figsize=(10,4.5),sharex=True,sharey=True)
importlib.reload(detsim)
for ip, p in enumerate(unique_pix):
    x_rect, y_rect = detsim.get_pixel_coordinates(p.get())
    pixel_plane = int(p[0] // consts.n_pixels[0])
    c = currents[ip].get()
    if c >= 1:
        rect = plt.Rectangle((x_rect, y_rect),
                             consts.pixel_pitch, consts.pixel_pitch,
                             linewidth=0.2, fc=m_curr.to_rgba(c),
                             edgecolor='grey')
        ax[0].add_patch(rect)

    a = adc_list[ip][adc_list[ip]>fee.digitize(0)]
    if len(a):
        rect = plt.Rectangle((x_rect, y_rect),
                             consts.pixel_pitch, consts.pixel_pitch,
                             linewidth=0.2, fc=m.to_rgba(np.sum(a.get())),
                             edgecolor='grey')
        ax[1].add_patch(rect)

for it,t in enumerate(selected_tracks):
    ax[0].plot((t["x_start"], t["x_end"]),
#            (t["y_start"]+21.8236, t["y_end"]+21.8236),
            (t["y_start"], t["y_end"]),
            c='r',
            lw=1.25,
            ls=':',
            alpha=1,
            zorder=10)
    ax[1].plot((t["x_start"], t["x_end"]),
            (t["y_start"], t["y_end"]),
#            (t["y_start"]+21.8236, t["y_end"]+21.8236),
            c='r',
            lw=1.25,
            ls=':',
            alpha=1,
            zorder=100)
    ax[0].scatter((t["x_start"], t["x_end"]),
                  (t["y_start"], t["y_end"]),
#                  (t["y_start"]+21.8236, t["y_end"]+21.8236),
                  c='r', s=1, zorder=99999)
#     ax[1].scatter((t["x_start"], t["x_end"]),
#                   (t["y_start"], t["y_end"]),
#                   c='r', s=1, zorder=99999)

ax[0].set_aspect("equal")
ax[1].set_aspect("equal")
ax[0].set_xlabel("x [cm]")
ax[1].set_xlabel("x [cm]")
ax[0].set_ylabel("y [cm]")
from mpl_toolkits.axes_grid1.axes_divider import make_axes_locatable

divider0 = make_axes_locatable(ax[1])
cax0 = divider0.append_axes("right", size="7%", pad=0.07)
fig.colorbar(m, ax=ax[1], cax=cax0, label='ADC counts sum')

divider1 = make_axes_locatable(ax[0])
cax1 = divider1.append_axes("right", size="7%", pad=0.07)
fig.colorbar(m_curr, ax=ax[0], cax=cax1, label='Induced current integral [# electrons]')

plt.subplots_adjust(hspace=0.5)
fig.savefig("currentadc.pdf")
```

### Export result
As a last step we backtrack the ADC counts to the Geant4 tracks and we export the result in a HDF5 file.

```python
!rm "LAr.neutrino.100.edep.larnd-fromnotebook.h5"
track_pixel_map=cp.repeat(track_pixel_map[:, cp.newaxis], 10, axis=1)
pc = fee.export_to_hdf5(adc_list.get(),
                        adc_ticks_list.get(),
                        unique_pix.get(),
                        backtracked_ids.get(),
                        track_pixel_map.get(),
                        "LAr.neutrino.100.edep.larnd-fromnotebook.h5")#,
#                        bad_channels=None)
```

    Writing to HDF5...: 100%|██████████| 311/311 [00:00<00:00, 52374.07it/s]

```python
with h5py.File('LAr.neutrino.100.edep.larnd-fromnotebook.h5', 'a') as f:
    f.create_dataset("tracks", data=selected_tracks)
```

## Analyze the result
Here we read back the file we just produced and compare it with the _true_ tracks. So first we create a dictionary that associates the chip key to a $(x,y)$ position:

```python
from collections import defaultdict
import yaml

geometry_yaml = yaml.load(open("/global/homes/k/kwood/dunend/software/test_ndlarmod/larnd-sim/larndsim/pixel_layouts/multi_tile_layout-3.0.40.yaml"), Loader=yaml.FullLoader)
det_yaml = yaml.load(open("/global/homes/k/kwood/dunend/software/test_ndlarmod/larnd-sim/larndsim/detector_properties/ndlar-module.yaml"),Loader=yaml.FullLoader)

pixel_pitch = geometry_yaml['pixel_pitch']
is_multi_tile = True
chip_channel_to_position = geometry_yaml['chip_channel_to_position']
tile_orientations = geometry_yaml['tile_orientations']
tile_positions = geometry_yaml['tile_positions']
tpc_centers = det_yaml['tpc_offsets']
tile_indeces = geometry_yaml['tile_indeces']
xs = np.array(list(chip_channel_to_position.values()))[:, 0] * pixel_pitch
ys = np.array(list(chip_channel_to_position.values()))[:, 1] * pixel_pitch
x_size = max(xs)-min(xs)+pixel_pitch
y_size = max(ys)-min(ys)+pixel_pitch

def _rotate_pixel(pixel_pos, tile_orientation):
    return pixel_pos[0]*tile_orientation[2], pixel_pos[1]*tile_orientation[1]

tile_geometry = defaultdict(int)
geometry = defaultdict(lambda: (0,0))
io_group_io_channel_to_tile = {}

for tile in geometry_yaml['tile_chip_to_io']:
    tile_orientation = tile_orientations[tile]
    tile_geometry[tile] = tile_positions[tile], tile_orientations[tile]

    for chip in geometry_yaml['tile_chip_to_io'][tile]:
        io_group_io_channel = geometry_yaml['tile_chip_to_io'][tile][chip]
        io_group = io_group_io_channel//1000
        io_channel = io_group_io_channel % 1000
        io_group_io_channel_to_tile[(
            io_group, io_channel)] = tile
    for chip_channel in geometry_yaml['chip_channel_to_position']:
        chip = chip_channel // 1000
        channel = chip_channel % 1000
        try:
            io_group_io_channel = geometry_yaml['tile_chip_to_io'][tile][chip]
        except KeyError:
            #print("Chip %i on tile %i not present in network" % (chip,tile))
            continue

        io_group = io_group_io_channel // 1000
        io_channel = io_group_io_channel % 1000
        x = chip_channel_to_position[chip_channel][0] * \
            pixel_pitch - x_size / 2 + pixel_pitch / 2
        y = chip_channel_to_position[chip_channel][1] * \
            pixel_pitch - y_size / 2 + pixel_pitch / 2

        x, y = _rotate_pixel((x, y), tile_orientation)
        x += tile_positions[tile][2] + \
            10*tpc_centers[tile_indeces[tile][1]-1][2]
        y += tile_positions[tile][1] + \
            10*tpc_centers[tile_indeces[tile][1]-1][1]

        geometry[(io_group, io_channel,
                       chip, channel)] = x, y
```

Then, we can compare the digitized charge associated to packets with the number of true electrons that reach the anode

```python
f = h5py.File("LAr.neutrino.100.edep.larnd-fromnotebook.h5", "r")
#f = h5py.File("LAr.neutrino.100.edep.larnd.h5", "r")
datawords = f['packets']['dataword'][f['packets']['packet_type']==0]
print("Digitized charge/true charge:", sum(datawords/fee.ADC_COUNTS*(fee.V_REF-fee.V_CM)+fee.V_CM-fee.V_PEDESTAL)/fee.GAIN/sum(f['tracks']['n_electrons']))
```

    Digitized charge/true charge: 0.8754515919498779

Here we verify the association between the _true_ track and the hit. The color of the pixel corresponds to the color of the first associated track (a hit can be associated to more tracks, not shown here).

```python
fig,ax = plt.subplots(1,1,constrained_layout=True)
plotted_tracks = []
colors = ['tab:blue','tab:orange','tab:green','tab:red','tab:purple','tab:brown']

for ip,packet in enumerate(f['packets']):
    if packet['packet_type'] == 0:
        io_group, io_channel, chip, channel = packet['io_group'], packet['io_channel'], packet['chip_id'], packet['channel_id']
        x,y = geometry[(io_group, io_channel, chip, channel)]
        #print(x,y)
        track_ids = f['mc_packets_assn']['track_ids'][ip]
        fractions = f['mc_packets_assn']['fraction'][ip]

        rect = plt.Rectangle((x-consts.pixel_pitch*5, y-consts.pixel_pitch*5),
                     consts.pixel_pitch*10, consts.pixel_pitch*10,
                     linewidth=0.5, fc='b',
                     edgecolor='k',alpha=0.25)

        ax.add_patch(rect)

        for trackid in track_ids:
            if trackid >= 0 and trackid not in plotted_tracks:
                plotted_tracks.append(trackid)
                ax.plot((f['tracks'][trackid]['x_start']*10,f['tracks'][trackid]['x_end']*10),
                        (f['tracks'][trackid]['y_start']*10,f['tracks'][trackid]['y_end']*10),c='r',alpha=0.25)
#                        (f['tracks'][trackid]['y_start']*10+218.236,f['tracks'][trackid]['y_end']*10+218.236),c='r',alpha=0.25)

tpc_rect = plt.Rectangle((4119.6,-1446.13), 1000, 3000, linewidth=1, edgecolor='b', facecolor='none')

#ax.set_ylim([-1446.13,1553.87])
ax.set_xlim([4119.6,5127.6])
ax.set_ylim([-1500,-1300.])
#ax.set_xlim([4119.6,4407.6])
ax.set_aspect("equal")
ax.set_xlabel("x [mm]")
ax.set_ylabel("y [mm]")
ax.add_patch(tpc_rect)

fig.savefig("mc.pdf")
```

```python

```


In this blog-post we discuss the concept of transfer-learning and show an implementation in Python, using tensorflow. Namely, we are using the pre-trained model *MobileNetV2* and apply it on the *Oxford Flower 102* dataset, in order to build a flower classification model. Lastly, we deploy the trained model on an ios device to make live predictions, using the phone's camera. The Github Repository for this project can be found [here](https://github.com/paulmora-statworx/flower_detection).

# Overview

The concept of transfer-learning is best understood when applied to real-life examples. If, for example, a person has played the piano for many years, this person will have an easier time picking up guitar, compared to a person who never played any musical instrument before.

That does not imply, of course, that the person who has prior piano experience is going to be perfect at playing the guitar right away, but the yearslong finger practice of playing piano and the development of having a good ear in music makes it easier for that person to learn.

It is important to note that these synergy effect were only possible because playing piano and playing the guitar were somewhat related. If the person who played piano for many years is now trying to learn to play American Football, it is rather less likely that this person will have an advantage. Hence, whether the previous training of concept A is beneficial for concept B depends on whether concept A and B bear any similarities to oneanother.

The very same idea applies to transfer-learning. Let us assume that we have a model which is very strong in detecting whether an apple is portrayed in a given image. Chances are good that we could use this model to detect pears as well, since pears and apples are similar (both round fruits). It will definitely not be as strong as a model thst was trained entirely on pears, but in situation where we do not have that many trainings images of pears, it is better than nothing.

That the apple detection would work also on detecting pears is only because apples and pears bear some common characteristics, like for example color and shape. If we would, on the other hand, take the apple detection model and try to detect cars with it, we will likely have terrible performance.

The question might arise why we are not simply building one model for each job: one model for detecting apples, one model for detecting pears, and so on.  The reason why this is sometimes not feasible are multi-fold. It could be, for example, that we lack the computational power, the time, or even the amount of images needed to train a  classification model that performs well. Especially the latter reason is a common problem within image detection.

The reason why a small amount of images leads to a poor performing model is easily understood when considering the workings of a neural network. When initializing a neural network, all weight parameters are initialized randomly. Through the training process these weight parameters are constantly adjusted, using back-propagation based on gradient descent. If we do not have enough images of the object we would like to classify, the network is not going to have a sufficient amount of data in order to adjust the weights appropriately (i.e., learn).

Transfer-learning can help in those situations. That is because of the workings of convolutional neural networks (CNNs), which are the gold standard when working with image data. While many details of CNNs are not fully explored, it is established that especially the lower levels of those models learn general shapes and patterns of the images. In the final layers of the networks all these shapes and patterns are put together to make up for the final object.

When using a pre-trained model on a different domain than it was originally trained for, we cannot make use of the top-layers of the pre-trained model since they are too specific to the initial use-case. The lower-levels of the network, on the other hand, come in very handy since detecting shapes and patterns is an essential part of image recognition, as is going to be needed for the new domain as well.

Therefore, we simply separate the pre-trained model into two pieces. The first piece, the layers below the very top are referred to as the **base-model**. The job of these layers is to detect shapes and patterns and putting them together into a combination of these intial shapes and patterns. The second piece is the top-layers. These layers are specific to the objects the model is classifying. This second piece is unwanted when changing the domain the model is supposed to classify, and therefore simply removed. Afterwards we stack on top of the base-model, one or several untrained layers, which comply with the number of categories we would like to classify within the new domain. That means, if the pre-trained model was originally trained to classify 50 different dog breeds, we can remove the last dense layer which outputs a vector with length 50, and add a layer with 30 categories, in order to classify 30 different cat breeds.

When stacking one or more top-layers on top of the lower levels, we have to re-train the entire model. It is important to note that we are solely intending to train the top-layers, not the base-model. We achieve that by freezing the weights of the base-model and therefore only training the newly added top layer with image data from the new domain. That is because jointly training the pre-trained model with the randomly initialized top-layers would result in gradient updates that are too large, and the pre-trained model would forget what it originally learned.

Through that approach the model is much faster in correctly classifying the objects from the new domain, since it does not spend time with learning how to detect shapes and patterns, but solely on how to identify what the combination of such shapes and patterns represent.

After training the top-layers we then can go one step further and train the entire model a bit more, by unfreezing some top-layers of the base-model and re-training those as well. It is crucial at this point to use a smaller learning rate, in order to not cause large gradient updates which would shake the foundation of the model.

# MobileNet V2

After covering the general idea of transfer-learning, we are now moving in the direction of implementing our own transfer-learning model. For that we first have to find a solid pre-trained model. As explained above, it is important for the cross-learning effects to happen that the pre-trained model is generalizable. Therefore, most of these pre-trained models are initially trained on very large image datasets with hundreds of categories. For that reason, many of the pre-trained models are trained using the [ImageNet database](https://www.image-net.org). This database offers 1000 categories and over one million training images.

When choosing which pre-trained model to go for, we were influenced in our decision by the fact that we are interested in deploying our final model on a mobile device. This causes obvious limitations in storage space, processing speed and energy usage, due to the decreased capactities of mobile phones. Lucky for us, there is a pre-trained model for exactly that purpose, namely the [**MobileNet V2**]((https://arxiv.org/pdf/1704.04861.pdf)), which was developed by Google.

## Model architecture

Before implementing that model right away, we take a look at its workings and what makes the model the better choice for being deployed on a mobile phone in contrast to any other model. For that we take a look at the [original paper](https://arxiv.org/pdf/1801.04381v4.pdf). More specifically, the model architecture of the network. The graphic below shows the different layers which were used when initially training this model. Each line represents one layer, which is repeated *n* times.

[Source](https://arxiv.org/pdf/1801.04381v4.pdf)

The MobileNetV2 starts with a traditional convolution before starting to apply the so-called bottleneck operator. This operator is elaborated on in more detail in the following sections. The main idea of these bottleneck layers is that they try to maintain the same level of accuracy as original convolutional networks, while being computationally cheaper. The size of the feature maps being passed from one layer to the next, are denoted by the variable *c*, while the stride parameter is denoted as *s*. The parameter *t* describes the *expansion factor*, which is the amount by which the number of channels are expanded within the bottleneck methodology.

As with most image classification models we notice how the number of channels gradually increases, while the image size decreases before collapsing the image into a 1x1 with k channels using average pooling and again a traditional convolution.

### Bottleneck Sequence

The main difference between the workings of the MobileNetV2 and other image classification model architectures are the usage of these so-called bottlenecks. The original paper describes bottlenecks as a sequence of three things - expansion layers, a depth wise convolution, and a projection layer. The following table, which was extracted from the original paper, nicely shows how the input and output sizes change when applying the bottleneck sequence.

One noteworthy aspect we gain from the table is that this approach is not using any pooling mechanism, and is altering the image's height and width solely using the stride parameter.

[Source](https://arxiv.org/pdf/1801.04381v4.pdf)

#### Expansion Layer

As the name suggests, what the expansion layer does is that it increases the number of feature maps of the output. This is done by applying multiple 1x1 kernels on the image and through that, increasing the size of the channels without altering the height or width of the image. This can also be seen from the table above. The original paper describes this process as a form of *unzipping* the image into a larger workbench. By how much we are increasing the number feature maps is a user defined input. The original paper set the default value equal to six.

#### Depth wise convolution

After increasing the number of channels through the expansion layer, we then apply a so-called depth-wise convolution. Depth wise convolution is very similar to traditional convolution, with the only difference being that the result is not one pixel within a feature map, but multiple.

To better understand that, we quickly explain the workings of traditional convolution. Traditional convolution applies one (usually) squared kernel to an image, which then calculates the dot product with every feature map respectively and then calculates a weighted sum of all dot product results in return one single number. The resulting number of feature maps when applying a kernel to an image is equal to one.

When applying depth wise convolution, we still apply the kernel to the image and calculate the dot product for every feature map. The difference is that we are then not summing the results of *all* feature maps together, but rather only sum the dot products for each feature map individually. This approach results in us having the same amount of feature maps before and after applying the convolution. This is also visible by looking at the second row of the table above, in which it says that both the input and output are equal to $tk$.

[Source](https://machinethink.net/blog/googles-mobile-net-architecture-on-iphone/)

#### Projection layer

Lastly, we apply a so-called projection layer. What this layer is doing is that it shrinks the number of feature maps. This is done by simply using again a 1x1 kernel, but this time not in order to increase the number of feature maps, but rather in order to decrease them. The amount by which the projection layer shrinks the number of feature maps is a user-defined input, denoted as *c*.

[Source](https://machinethink.net/blog/googles-mobile-net-architecture-on-iphone/)

#### Graphical Example

To better understanding the combined workings of the bottleneck sequence, we will take a look at an example. Let us assume that we have an image with the sizes 128x128x16. When applying the expansion layer we increase the number of feature maps of the image. Using the default expansion factor of six, the number of channels increases to $16*6=96$, while not changing the width or height of the image.

The second step would then be to apply the depth wise convolution. Given that the depth wise convolution does not change the number of channels, we do not have any change in the number of feature maps. Assuming a stride equal to 1, we also do not change the number of height or width.

Lastly, we decrease the number of feature channels again, using the projection layer. Herein we set the number of desired output channels equal to 24, which is therefore going to be the resulting number of output channels.

#### Motivation

The question might be asked why we are doing all of this instead of simply using traditional convolution. The reason for this is mainly already answered by the invention of [MobileNetV1](https://arxiv.org/pdf/1704.04861.pdf). In contrast to MobileNetV2, V1 consists only of the depth wise convolution plus the projection layer. This combination is by the original paper referred to as **Depth wise separable convolution**.

In order to gain a more mathematical understanding of why the depth wise separable convolution is computationally beneficial is found when considering the number of computations both methods have to go through. Traditional convolutions have a computational cost of:

$$D_K \cdot D_K \cdot M \cdot N \cdot D_F \cdot D_F$$

$$\begin{align}
D_K &= \textrm{Kernel size} \\
M &= \textrm{Number of input channels} \\
N &= \textrm{Number of output channels} \\
D_F &= \textrm{Feature map size} \\
\end{align}$$

In contrast to traditional convolution, depth wise convolution does not take the number of output channels into consideration, since it creates as many output channels as it has input channels by definition. Adjusting the number of output channels is then conducted by the projection layer. Both costs are then in the end summed together, resulting in the computational cost of the depth wise separable convolution:

$$D_K \cdot D_K \cdot M \cdot D_F \cdot D_F + M \cdot N \cdot D_F \cdot D_F$$

$$\begin{align}
D_K &= \textrm{Kernel size} \\
M &= \textrm{Number of input channels} \\
N &= \textrm{Number of output channels} \\
D_F &= \textrm{Feature map size} \\
\end{align}$$

When then calculating how many times the depth wise separable convolution is superior to the traditional convolution, we find the following:

$$\frac{D_K \cdot D_K \cdot M \cdot D_F \cdot D_F + M \cdot N \cdot D_F \cdot D_F}{D_K \cdot D_K \cdot M \cdot N \cdot D_F \cdot D_F} = \frac{1}{N} + \frac{1}{D^2_K}$$

$$\begin{align}
D_K &= \textrm{Kernel size} \\
M &= \textrm{Number of input channels} \\
N &= \textrm{Number of output channels} \\
D_F &= \textrm{Feature map size} \\
\end{align}$$

Assuming a Kernel size of 3, we then find that the convolution method of MobileNet is 8 to 9 times more efficient compared to traditional convolution.

The computation gains are the main motivation of using MobileNet overall. The difference between V2 and V1 are mostly the idea of zipping and unzipping the image within each bottleneck sequence, which further reduces the computational costs.

#### Side note: ReLU6

It is also interesting to note that the MobileNetV2 does not use a traditional ReLU function, but rather a so-called ReLU6 activation function. As the name probably already suggests, this caps all positive values at positive six, preventing the activations from becoming too large.

# Oxford Flower 102

In order to show the power of transfer-learning we chose the Oxford Flower 102 dataset, which can be found [here](https://www.robots.ox.ac.uk/~vgg/data/flowers/102/). This dataset contains, as the name suggests, 102 different categories of flowers in the United Kingdom. Each flower category contains between 40 and 258 images, respectively. Obviously, this amount of data is far too little to train a sophisticated neural network from, which makes it a good testing example of transfer-learning.

[Source](https://www.researchgate.net/figure/Examples-of-images-in-the-Oxford-Flower-102-Dataset-Corresponding-categories-are-given_fig7_318204948)

# Code implementation

The implementation of this transfer learning example was done in tensorflow. Choosing tensorflow over PyTorch did not have any particular reason, as both frameworks have a significant amount of content about transfer-learning.
The repository for this project is found [here](https://github.com/paulmora-statworx/flower_detection). The final implementation of the trained model into an ios application is conducted following the tutorial from [tensorflow's repository](https://github.com/tensorflow/examples/tree/master/lite/examples/image_classification/ios).

## Data Structure

In contrast to many other blog-posts which try to show intuitive output after every code cell, this post works a bit differently given the packaged code. Instead, this post shows all classes used in building the model individually and elaborates on its workings.

In order to have a better understanding of how the different classes interact with each other, we start by showing the  src  folder for this project.

```python
import seedir as sd
sd.seedir("../src", style="lines", exclude_folders="__pycache__")
```

    src/
    ├─config.json
    ├─data_loader.py
    ├─utils/
    │ ├─config.py
    │ └─args.py
    ├─coreml_converter.py
    ├─model.py
    ├─trainer.py
    └─main.py

As usually done, all major classes are constructed in their own file and then called and executed within  main.py . All kinds of hyper-parameters for every class are stated within the  config.json  file and called by using an argparser function defined in the folder utils.

## Data Loader

The first step we have to do after downloading the images is to load them into Python. This step was less straight-forward than originally thought. This is because the Oxford Flower dataset has the interesting property of having substantially more test data than training data. This might be an interesting challenge for many, but for our use-case, in which we would like to end up building a strong classification model, we would rather have more trainings data.

In order to self-set the train to test data ratio, we have to unpack all images and shuffle them ourselves. These steps are outlined in the methods  _loading_images_array  and  _load_labels . Since labels and images are stored within two separate files, we have to make sure to correctly align and match image and label. This is done by sorting the image names in an ascending order before attaching the labels. The labels have to be one-hot encoded, in order to be properly used within the prediction algorithm.

We decided to use the preprocessing class  ImageDataGenerator  from the preprocessing package from tensorflow. Using this preprocessing method allows us to easily apply the appropriate data augmentation settings for the images. One has to be aware that when using the pre-trained model MobileNetV2, one has to apply the related pre-processing function for that very model. This is necessary since the input images have to resemble the same kind of images which were used when training the model in the first place.

Furthermore, we applied several data augmentation techniques to the training data. This is commonly done in situation in which we have only a limited amount of training data. In contrast to the training data, the validation and test data is not augmented in any way other than the necessary MobileNetV2 pre-processing. Classifying the flower category becomes a much harder challenge for the model when the image is heavily augmented. Since the validation data is not altered in any way, those examples are much easier for the model, which results in a better performance of on the validation data in contrast to the training data, a phenomena which rarely occurs.

```python
# %% Packages

import os
import numpy as np
import tensorflow as tf
from scipy.io import loadmat
from sklearn.preprocessing import OneHotEncoder
from sklearn.model_selection import train_test_split
from tensorflow.keras.preprocessing.image import ImageDataGenerator

# %% Classes

class OxfordFlower102DataLoader:
    """
    This class loads the images and labels and embeds them into ImageDataGenerators.
    """

    def __init__(self, config):
        self.config = config
        (
            self.train_generator,
            self.val_generator,
            self.test_generator,
        ) = self.create_generators()

    def create_generators(self):
        """
        This method loads the labels and images, which are already split into train, test and validation.
        Furthermore, we add an additional step to the preprocessing function, which is required for the pre-trained
        model. Afterwards we create ImageGenerators from tensorflow for train, test and validation.
        :return: ImageDataGenerator for training, validation and testing
        """
        X_train, X_val, X_test, y_train, y_val, y_test = self._image_and_labels()
        train_augment_settings, test_augment_settings = self._add_preprocess_function()

        # Data Augmentation setup initialization
        train_data_gen = ImageDataGenerator(**train_augment_settings)
        valid_data_gen = ImageDataGenerator(**test_augment_settings)
        test_data_gen = ImageDataGenerator(**test_augment_settings)

        # Setting up the generators
        training_generator = train_data_gen.flow(
            x=X_train, y=y_train, batch_size=self.config.data_loader.batch_size
        )
        validation_generator = valid_data_gen.flow(
            x=X_val, y=y_val, batch_size=self.config.data_loader.batch_size
        )
        test_generator = test_data_gen.flow(
            x=X_test, y=y_test, batch_size=self.config.data_loader.batch_size
        )
        return training_generator, validation_generator, test_generator

    def _add_preprocess_function(self):
        """
        This function adds the pre-processing function for the MobileNet_v2 to the settings dictionary.
        The pre-processing function is needed since the base-model was trained using it.
        :return: Dictionaries with multiple items of image augmentation
        """
        train_augment_settings = self.config.data_loader.train_augmentation_settings
        test_augment_settings = self.config.data_loader.test_augmentation_settings
        train_augment_settings.update(
            {
                "preprocessing_function": tf.keras.applications.mobilenet_v2.preprocess_input
            }
        )
        test_augment_settings.update(
            {
                "preprocessing_function": tf.keras.applications.mobilenet_v2.preprocess_input
            }
        )
        return train_augment_settings, test_augment_settings

    def _image_and_labels(self):
        """
        This method loads labels and images and afterwards split them into training, validation and testing set
        :return: Trainings, Validation and Testing Images and Labels
        """
        y = self._load_labels()
        X = self._loading_images_array()
        X_train, X_test, y_train, y_test = train_test_split(
            X,
            y,
            train_size=self.config.data_loader.train_size,
            random_state=self.config.data_loader.random_state,
            shuffle=True,
            stratify=y,
        )
        X_train, X_val, y_train, y_val = train_test_split(
            X_train,
            y_train,
            train_size=self.config.data_loader.train_size,
            random_state=self.config.data_loader.random_state,
            shuffle=True,
            stratify=y_train,
        )
        return X_train, X_val, X_test, y_train, y_val, y_test

    def _load_labels(self):
        """
        Loading the matlab file and one-hot encodes them.
        :return: Numpy array of one-hot encoding labels
        """
        imagelabels_file_path = "./data/imagelabels.mat"
        image_labels = loadmat(imagelabels_file_path)["labels"][0]
        image_labels_2d = image_labels.reshape(-1, 1)

        encoder = OneHotEncoder(sparse=False)
        one_hot_labels = encoder.fit_transform(image_labels_2d)
        return one_hot_labels

    def _loading_images_array(self):
        """
        Loading the flower images and resizes them into the appropriate size. Lastly we turn the images into a numpy array
        :return: Numpy array of the images
        """
        image_path = "./data/jpg"
        image_file_names = os.listdir(image_path)
        image_file_names.sort()
        image_array_list = []
        for image_file_name in image_file_names:
            tf_image = tf.keras.preprocessing.image.load_img(
                path=f"{image_path}/{image_file_name}",
                grayscale=False,
                target_size=(
                    self.config.data_loader.target_size,
                    self.config.data_loader.target_size,
                ),
            )
            img_array = tf.keras.preprocessing.image.img_to_array(tf_image)
            image_array_list.append(img_array)
        return np.array(image_array_list)
```

Given that we have quite a large number of flower categories to predict (102), and the fact that these categories are not balanced, we have to make sure that we have the same proportion of each class within the training, validation and test data in order to have a stronger model and a more meaningful model evaluation. This balance is ensured by using the  stratify  argument within the train-test split from  sklearn . The following image shows the result of using that parameter: We can see that we have same proportions within the train, test and validation data.

In order to also have a better understanding what the pre-processing of the images actually looks like, we show in the following nine example images from the trainings data. We see that all images are much darker than the original ones we saw before. That change of lighting comes from the MobileNetV2 pre-process function we applied. The image in the very middle of the lower matrix nicely shows the level of distortion we apply to the images. These augmentations of images are especially useful in cases like this one where we have such training little data, since it artificially increases the pool of images we can train our model with. It is to be said, though, that we are not applying these distortions on the test and validation data, since these heavy distortions don't occur in the model's final application and should therefore not be considered in the model's performance on real flower-images.

## Model

Now it is time to build our model. This is done in two steps. The first step loads the pre-trained model and freezes all parameters within it. We then stack a dense layer on top and solely train these weights. Furthermore, we add a dropout layer in order to prevent overfitting of the model.

The second step, as already outlined in the explanation of transfer-learning, then describes the fine-tuning process of transfer-learning. Herein we unfreeze several of the top-layers of the pre-trained model and train them using a small learning rate in order to marginally adjust the pre-trained model in a beneficial direction.

We are using RMSprop for compiling the model, as well as a learning rate of 1e-3 for the training within the first step, and a learning rate of 1e-4 for the fine-tuning.

```python
# %% Packages

import tensorflow as tf
from tensorflow.keras import Sequential
from tensorflow.keras.layers import Dense, Dropout
from tensorflow.keras.applications import MobileNetV2

# %% Classes

class OxfordFlower102Model:
    """
    This class is initializing the model
    """

    def __init__(self, config):
        self.config = config
        self.base_model = self.build_model()
        tf.random.set_seed(self.config.model.random_seed)

    def build_model(self):
        """
        This method build the basic model. The basic model describes the pre-trained model plus a dense layer
        on top which is individualized to the number of categories needed. The model is also compiled
        :return: A compiled tensorflow model
        """
        pre_trained_model = self.initialize_pre_trained_model()
        top_model = self.create_top_layers()

        model = Sequential()
        model.add(pre_trained_model)
        model.add(top_model)

        model.compile(
            loss=self.config.model.loss,
            metrics=[self.config.model.metrics],
            optimizer=tf.keras.optimizers.RMSprop(
                learning_rate=self.config.model.learning_rate
            ),
        )
        model.summary()
        return model

    def unfreeze_top_n_layers(self, model, ratio):
        """
        This method unfreezes a certain number of layers of the pre-trained model and combines it subsequently with the
        pre-trained top layer which was added within the 'create_top_layers' method and trained within the 'build_model'
        class
        :param model: Tensorflow model which was already fitted
        :param ratio: Float of how many layers should not be trained of the entire model
        :return: Compiled tensorflow model
        """
        base_model = model.layers[0]
        trained_top_model = model.layers[1]

        base_model.trainable = True
        number_of_all_layers = len(base_model.layers)
        non_trained_layers = int(number_of_all_layers * ratio)
        for layer in base_model.layers[:non_trained_layers]:
            layer.trainable = False

        fine_tune_model = Sequential()
        fine_tune_model.add(base_model)
        fine_tune_model.add(trained_top_model)

        adjusted_learning_rate = (
            self.config.model.learning_rate / self.config.model.learning_rate_shrinker
        )
        fine_tune_model.compile(
            loss=self.config.model.loss,
            metrics=[self.config.model.metrics],
            optimizer=tf.keras.optimizers.RMSprop(learning_rate=adjusted_learning_rate),
        )
        fine_tune_model.summary()
        return fine_tune_model

    def initialize_pre_trained_model(self):
        """
        This method calls the pre-trained model. In this case we are loading the MobileNetV2
        :return: Tensorflow model
        """
        image_shape = (
            self.config.data_loader.target_size,
            self.config.data_loader.target_size,
            3,
        )
        base_model = MobileNetV2(
            input_shape=image_shape, include_top=False, pooling="avg"
        )
        base_model.trainable = False
        return base_model

    def create_top_layers(self):
        """
        Creating the tensorflow top-layer of a model
        :return: Tensorflow Sequential model
        """
        top_model = Sequential()
        top_model.add(
            Dense(self.config.model.number_of_categories, activation="softmax")
        )
        top_model.add(Dropout(rate=self.config.model.dropout_rate))
        return top_model

```

## Trainer

As the last class, we define the training process. This class first triggers the training of the base-model, which comprises of the pre-trained model with the shallow top-layers on top for ten epochs. Afterwards, we call the unfreezing method of the model, which is defined within the model class, and continue training for ten more epochs. In order to stop any potential overfitting we use Early-stopping. This method stops the model training after the validation accuracy leveled for a user-defined number of epochs (we chose three).

```python
# %% Packages

import matplotlib.pyplot as plt
from tensorflow.keras.callbacks import EarlyStopping

# %% Classes

class OxfordFlower102Trainer:
    """
    This class is training the base-model and fine-tunes the model
    """

    def __init__(self, model, data_generator, config):
        self.config = config
        self.model = model
        self.train_data_generator = data_generator.train_generator
        self.val_data_generator = data_generator.val_generator
        self.loss = []
        self.acc = []
        self.val_loss = []
        self.val_acc = []

        self._init_callbacks()
        print("Train the base Model!")
        self.train_model()
        print("Fine tune the Model!")
        self.train_fine_tune()
        self.save_model()

    def _init_callbacks(self):
        self.custom_callbacks = [
            EarlyStopping(
                monitor="val_accuracy",
                mode="max",
                patience=self.config.trainer.early_stopping_patience,
            )
        ]

    def train_model(self):
        """
        This method is training the base_model
        :return: /
        """
        history = self.model.base_model.fit(
            self.train_data_generator,
            verbose=self.config.trainer.verbose_training,
            epochs=self.config.trainer.number_of_base_epochs,
            validation_data=self.val_data_generator,
            callbacks=self.custom_callbacks,
        )
        self.append_model_data(history)

    def train_fine_tune(self):
        """
        This method is unfreezing some layers of the already trained model and re-trains the model
        :return: /
        """
        total_epochs = (
            self.config.trainer.number_of_base_epochs
            + self.config.trainer.number_of_fine_tune_epochs
        )
        self.fine_tune_model = self.model.unfreeze_top_n_layers(
            self.model.base_model, self.config.trainer.percentage_of_frozen_layers
        )

        fine_tune_history = self.fine_tune_model.fit(
            self.train_data_generator,
            verbose=self.config.trainer.verbose_training,
            initial_epoch=self.config.trainer.number_of_base_epochs,
            epochs=total_epochs,
            validation_data=self.val_data_generator,
            callbacks=self.custom_callbacks,
        )
        self.append_model_data(fine_tune_history)
        self.plot_history("fine_tune_model")

    def append_model_data(self, history):
        """
        This method is
        :param history: Tensorflow model history
        :return: /
        """
        self.loss.extend(history.history["loss"])
        self.val_loss.extend(history.history["val_loss"])

        self.acc.extend(history.history["accuracy"])
        self.val_acc.extend(history.history["val_accuracy"])

    def plot_history(self, title):
        """
        This method is plotting the accuracy and loss of the plots
        :param title: str - Used to save the png
        :return: /
        """
        fig, axs = plt.subplots(figsize=(10, 5), ncols=2)
        axs = axs.ravel()
        axs[0].plot(self.loss, label="Training")
        axs[0].plot(self.val_loss, label="Validation")
        axs[0].set_title("Loss")
        axs[0].axvline(
            x=(self.config.trainer.number_of_base_epochs - 1),
            ymin=0,
            ymax=1,
            label="BaseEpochs",
            color="green",
            linestyle="--",
        )
        axs[0].legend()

        axs[1].plot(self.acc, label="Training")
        axs[1].plot(self.val_acc, label="Validation")
        axs[1].set_title("Accuracy")
        axs[1].axvline(
            x=(self.config.trainer.number_of_base_epochs - 1),
            ymin=0,
            ymax=1,
            label="BaseEpochs",
            color="green",
            linestyle="--",
        )
        axs[1].legend()

        fig.savefig(f"./reports/figures/history_{title}.png")

    def save_model(self):
        """
        Saving the fine-tuned model
        :return: /
        """
        path = "./models/oxford_flower102_fine_tuning.h5"
        self.fine_tune_model.save(filepath=path)

```

## Run the model

Lastly we call all of the aforementioned classes within the  main.py  file and trigger them one after another.

```python
# %% Packages

from utils.args import get_args
from utils.config import process_config
from model import OxfordFlower102Model
from data_loader import OxfordFlower102DataLoader
from trainer import OxfordFlower102Trainer

# %% Main Script

def main():

    args = get_args()
    config = process_config(args.config)

    print("Creating the Data Generator!")
    data_loader = OxfordFlower102DataLoader(config)

    print("Creating the Model!")
    model = OxfordFlower102Model(config)

    print("Creating the Trainer!")
    trainer = OxfordFlower102Trainer(model, data_loader, config)

if __name__ == "__main__":
    main()

```

## Model evaluation

The plot below gives us some interesting insights into how the model training went. We see that the model reaches a relatively strong performance after only a small amount of training epochs, but then seems to starting leveling off. After the fine-tuning kicks in, we then witness a significant drop in accuracy, which suggests that the learning rate was probably too high, triggering too large weight changes within the back-propagation of the network. However, after a couple of epochs, the model is back on track, reaching performance levels which were not attainable earlier.

Looking at the examples from the tensorflow [website](https://www.tensorflow.org/tutorials/images/transfer_learning), we did not spot any drop in accuracy to the extent that we encountered. Unsure whether that problem was only due to a potentially higher learning rate, we tried a range of learning rates, always encountering the same problem. We therefore suspect that the drop of the learning rate is likely going to be a result of having so little training data, compared to the example shown on the tensorflow website, which uses the well-known [cats vs. dogs dataset](https://www.kaggle.com/c/dogs-vs-cats).

Overall we are happy with the model performance, which reaches an accuracy of **93.17%** on the unseen test data.

# Phone Application

Finally, we thought it would be a nice to deploy the model on a mobile application, as that was also the motivation of choosing the MobileNetV2 network. In order to do so, one has to convert the h5 format the model is currently saved as, into a tflite file. Doing that compresses the model and brings it into the right format for the job. Furthermore, we have to sort and store all the labels into a text file and put them into the  Model  folder of the application parent folder.

This app folder is pulled from the official tensorflow repository, found [here](https://github.com/tensorflow/examples/tree/master/lite/examples/image_classification/ios).

```python
sd.seedir("../app/model", style="lines", exclude_folders="__pycache__")
```

    model/
    ├─.DS_Store
    ├─oxford_flower_102.tflite
    ├─labels_flowers.txt
    ├─mobilenet_quant_v1_224.tflite
    ├─labels.txt
    └─.gitignore

```python
# %% Packages

import json
import tensorflow as tf

# %% Loading models and data

# Model
keras_path = "./models/oxford_flower102_fine_tuning.h5"
keras_model = tf.keras.models.load_model(keras_path)
converter = tf.lite.TFLiteConverter.from_keras_model(keras_model)
tflite_model = converter.convert()
with open("./models/oxford_flower_102.tflite", "wb") as f:
    f.write(tflite_model)

# Labels
labels_path = "./data/cat_to_name.json"
with open(labels_path) as json_file:
    labels_dict = json.load(json_file)
sorted_labels_dict = sorted(labels_dict.items(), key=lambda x: int(x[0]))
label_values = [x[1] for x in sorted_labels_dict]
textfile = open("./models/labels_flowers.txt", "w")
for element in label_values:
    textfile.write(element + "\n")
textfile.close()

```

After some adjustment in xcode, we can then deploy the app on any iOs device and use the camera for live prediction. The result of which can be seen on the gif below.