"""
--- Day 10: The Stars Align ---

It's no use; your navigation system simply isn't capable of providing walking
directions in the arctic circle, and certainly not in 1018.

The Elves suggest an alternative. In times like these, North Pole rescue
operations will arrange points of light in the sky to guide missing Elves back
to base. Unfortunately, the message is easy to miss: the points move slowly
enough that it takes hours to align them, but have so much momentum that they
only stay aligned for a second. If you blink at the wrong time, it might be
hours before another message appears.

You can see these points of light floating in the distance, and record their
position in the sky and their velocity, the relative change in position per
second (your puzzle input). The coordinates are all given from your
perspective; given enough time, those positions and velocities will move the
points into a cohesive message!

Rather than wait, you decide to fast-forward the process and calculate what the
points will eventually spell.

For example, suppose you note the following points:

position=< 9,  1> velocity=< 0,  2>
position=< 7,  0> velocity=<-1,  0>
position=< 3, -2> velocity=<-1,  1>
position=< 6, 10> velocity=<-2, -1>
position=< 2, -4> velocity=< 2,  2>
position=<-6, 10> velocity=< 2, -2>
position=< 1,  8> velocity=< 1, -1>
position=< 1,  7> velocity=< 1,  0>
position=<-3, 11> velocity=< 1, -2>
position=< 7,  6> velocity=<-1, -1>
position=<-2,  3> velocity=< 1,  0>
position=<-4,  3> velocity=< 2,  0>
position=<10, -3> velocity=<-1,  1>
position=< 5, 11> velocity=< 1, -2>
position=< 4,  7> velocity=< 0, -1>
position=< 8, -2> velocity=< 0,  1>
position=<15,  0> velocity=<-2,  0>
position=< 1,  6> velocity=< 1,  0>
position=< 8,  9> velocity=< 0, -1>
position=< 3,  3> velocity=<-1,  1>
position=< 0,  5> velocity=< 0, -1>
position=<-2,  2> velocity=< 2,  0>
position=< 5, -2> velocity=< 1,  2>
position=< 1,  4> velocity=< 2,  1>
position=<-2,  7> velocity=< 2, -2>
position=< 3,  6> velocity=<-1, -1>
position=< 5,  0> velocity=< 1,  0>
position=<-6,  0> velocity=< 2,  0>
position=< 5,  9> velocity=< 1, -2>
position=<14,  7> velocity=<-2,  0>
position=<-3,  6> velocity=< 2, -1>

Each line represents one point. Positions are given as <X, Y> pairs: X
represents how far left (negative) or right (positive) the point appears,
while Y represents how far up (negative) or down (positive) the point appears.

At 0 seconds, each point has the position given. Each second, each point's
velocity is added to its position. So, a point with velocity <1, -2> is moving
to the right, but is moving upward twice as quickly. If this point's initial
position were <3, 9>, after 3 seconds, its position would become <6, 3>.

Over time, the points listed above would move like this:

Initially:
........#.............
................#.....
.........#.#..#.......
......................
#..........#.#.......#
...............#......
....#.................
..#.#....#............
.......#..............
......#...............
...#...#.#...#........
....#..#..#.........#.
.......#..............
...........#..#.......
#...........#.........
...#.......#..........

After 1 second:
......................
......................
..........#....#......
........#.....#.......
..#.........#......#..
......................
......#...............
....##.........#......
......#.#.............
.....##.##..#.........
........#.#...........
........#...#.....#...
..#...........#.......
....#.....#.#.........
......................
......................

After 2 seconds:
......................
......................
......................
..............#.......
....#..#...####..#....
......................
........#....#........
......#.#.............
.......#...#..........
.......#..#..#.#......
....#....#.#..........
.....#...#...##.#.....
........#.............
......................
......................
......................

After 3 seconds:
......................
......................
......................
......................
......#...#..###......
......#...#...#.......
......#...#...#.......
......#####...#.......
......#...#...#.......
......#...#...#.......
......#...#...#.......
......#...#..###......
......................
......................
......................
......................

After 4 seconds:
......................
......................
......................
............#.........
........##...#.#......
......#.....#..#......
.....#..##.##.#.......
.......##.#....#......
...........#....#.....
..............#.......
....#......#...#......
.....#.....##.........
...............#......
...............#......
......................
......................

After 3 seconds, the message appeared briefly: HI. Of course, your message will
be much longer and will take many more seconds to appear.

What message will eventually appear in the sky?
"""
import itertools
from typing import List, Tuple

def read_input(
    filename: str
    ) -> Tuple[List[Tuple[int, int]], List[Tuple[int, int]]]:

    position = []
    velocity = []

    with open(filename) as f:
        for line in f:
            splitted = line.strip().lstrip('position=<').rstrip('>').split(',')
            mid = splitted[1].split('> velocity=<')
            px = int(splitted[0])
            py = int(mid[0])
            vx = int(mid[-1])
            vy = int(splitted[-1])
            position.append((px, py))
            velocity.append((vx, vy))

    return position, velocity

def draw_message(
    t: int,
    position: List[Tuple[int, int]],
    velocity: List[Tuple[int, int]]
    ) -> None:

    result = []
    for (y0, x0), (dy, dx) in zip(position, velocity):
        x, y = x0 + dx * t, y0 + dy * t
        result.append((x, y))

    result.sort(key=lambda x: x[0])

    min_height = min(x for x, y in result)
    max_height = max(x for x, y in result)

    min_width = min(y for x, y in result)
    max_width = max(y for x, y in result)

    grid = [[' '] * (max_width - min_width + 1) for _ in range(max_height - min_height + 1)]

    for x, y in result:
        grid[x - min_height][y - min_width] = '#'

    for row in grid:
        print(''.join(row))

def find_min(
    position: List[Tuple[int, int]],
    velocity: List[Tuple[int, int]]
    ) -> int:

    result = []

    for t in range(20000):

        min_height = min(x + t * vx for (y, x), (vy, vx) in zip(position, velocity))
        max_height = max(x + t * vx for (y, x), (vy, vx) in zip(position, velocity))

        min_width = min(y + t * vy for (y, x), (vy, vx) in zip(position, velocity))
        max_width = max(y + t * vy for (y, x), (vy, vx) in zip(position, velocity))

        result.append(max_width - min_width + max_height - min_height)

    return result.index(min(result))

if __name__ == "__main__":

    pos, vel = read_input('test.txt')
    t = find_min(pos, vel)
    pos, vel = read_input('test.txt')
    print(f"After {t} seconds:")
    draw_message(t, pos, vel)

    pos, vel = read_input('input.txt')
    t = find_min(pos, vel)
    pos, vel = read_input('input.txt')
    print(f"After {t} seconds:")
    draw_message(t, pos, vel)


# -*- coding: utf-8 -*-
"""
The AID configuration selection is getting a mjor update right now
"""
from __future__ import absolute_import, division, print_function
import utool as ut
import six  # NOQA
from ibeis.init import old_main_helpers
(print, print_, printDBG, rrr, profile) = ut.inject(__name__, '[main_helpers]')

# DEPRICATE
get_test_daids = old_main_helpers.get_test_daids
get_test_qaids = old_main_helpers.get_test_qaids

VERB_TESTDATA, VERYVERB_TESTDATA = ut.get_verbflag('testdata', 'td')
VERYVERB_MAIN_HELPERS = VERYVERB_TESTDATA
VERB_MAIN_HELPERS = VERB_TESTDATA

#VERB_TESTDATA = ut.get_argflag(('--verbose-testdata', '--verbtd')) or VERYVERB_TESTDATA
#VERB_MAIN_HELPERS = ut.get_argflag(('--verbose-main-helpers', '--verbmhelp')) or ut.VERBOSE or VERB_TESTDATA

def testdata_pipecfg():
    r"""
    Returns:
        dict: pcfgdict

    CommandLine:
        python -m ibeis.init.main_helpers --exec-testdata_pipecfg

    Example:
        >>> # ENABLE_DOCTEST
        >>> from ibeis.init.main_helpers import *  # NOQA
        >>> pcfgdict = testdata_pipecfg()
        >>> result = ('pcfgdict = %s' % (str(pcfgdict),))
        >>> print(result)
    """
    from ibeis.experiments import experiment_helpers
    test_cfg_name_list = ut.get_argval('-t', type_=list, default=['default'])
    pcfgdict_list = experiment_helpers.get_pipecfg_list(test_cfg_name_list)[0]
    assert len(pcfgdict_list) == 1, 'can only specify one pipeline config here'
    pcfgdict = pcfgdict_list[0]
    return pcfgdict

def testdata_filtcfg(default=None):
    from ibeis.experiments import cfghelpers
    if default is None:
        default = ['']
    filt_cfg = cfghelpers.parse_argv_cfg(('--filt', '-f'), default=default)[0]
    return filt_cfg

def testdata_qres(defaultdb='testdb1'):
    r"""
    Args:
        defaultdb (str): (default = 'testdb1')

    Returns:
        tuple: (ibs, test_result)

    CommandLine:
        python -m ibeis.init.main_helpers --exec-testdata_qres
        python -m ibeis.init.main_helpers --exec-testdata_qres --qaid 1

    Example:
        >>> # ENABLE_DOCTEST
        >>> from ibeis.init.main_helpers import *  # NOQA
        >>> defaultdb = 'testdb1'
        >>> (ibs, qreq_, qres) = testdata_qres(defaultdb)
        >>> result = ('(ibs, qreq_, qres) = %s' % (str((ibs, qreq_, qres)),))
        >>> print(result)
    """
    ibs, qaids, daids = testdata_ibeis(defaultdb=defaultdb)
    pcfgdict = testdata_pipecfg()
    qreq_ = ibs.new_query_request(qaids, daids, cfgdict=pcfgdict)
    print('qaids = %r' % (qaids,))
    assert len(qaids) == 1, 'only one qaid for this tests'
    qres = qreq_.load_cached_qres(qaids[0])
    print('qreq_ = %r' % (qreq_,))
    return ibs, qreq_, qres

def testdata_expts(defaultdb='testdb1',
                   default_acfgstr_name_list=['default'],
                   #default_acfgstr_name_list=['controlled:qsize=20,dper_name=1,dsize=10',
                   #                           'controlled:qsize=20,dper_name=10,dsize=100'],
                   #default_test_cfg_name_list=['default', 'default:fg_on=False']
                   default_test_cfg_name_list=['default'],
                   a=None,
                   t=None,
                   qaid_override=None,
                   ):
    """
    Command line interface to quickly get testdata for test_results
    """
    import ibeis
    from ibeis.experiments import experiment_harness
    from ibeis.experiments import experiment_storage
    if a is not None:
        default_acfgstr_name_list = a
    if t is not None:
        default_test_cfg_name_list = t

    #from ibeis.experiments import experiment_helpers
    ibs = ibeis.opendb(defaultdb=defaultdb)
    acfg_name_list = ut.get_argval(('--aidcfg', '--acfg', '-a'), type_=list, default=default_acfgstr_name_list)
    test_cfg_name_list = ut.get_argval('-t', type_=list, default=default_test_cfg_name_list)
    test_result_list = experiment_harness.run_test_configurations2(
        ibs, acfg_name_list, test_cfg_name_list, qaid_override=qaid_override)
    test_result = experiment_storage.combine_test_results(ibs, test_result_list)
    return ibs, test_result
    #return ibs, test_result_list

def testdata_ibeis(default_qaids=[1], default_daids='all', defaultdb='testdb1', ibs=None, verbose=False, return_annot_info=False):
    r"""
    Args:
        default_qaids (list): (default = [1])
        default_daids (str): (default = 'all')
        defaultdb (str): (default = 'testdb1')
        ibs (IBEISController):  ibeis controller object(default = None)
        verbose (bool):  verbosity flag(default = False)
        return_annot_info (bool): (default = False)

    Returns:
        ibs, qaid_list, daid_list, annot_info:

    CommandLine:
        python -m ibeis.init.main_helpers --exec-testdata_ibeis --db NNP_Master3
        python -m ibeis.init.main_helpers --exec-testdata_ibeis --db PZ_MTEST --acfg default:aids=gt,shuffle,index=0:25 --verbose-testdata
        python -m ibeis.init.main_helpers --exec-testdata_ibeis --db PZ_MTEST --acfg default:aids=gt,index=0:25 --verbose-testdata
        python -m ibeis.init.main_helpers --exec-testdata_ibeis --db NNP_Master3 --verbose-testdata -a controlled
        python -m ibeis.init.main_helpers --exec-testdata_ibeis --db NNP_Master3 --verbose-testdata --aidcfg controlled
        python -m ibeis.init.main_helpers --exec-testdata_ibeis --db NNP_Master3 --verbose-testdata --aidcfg default:species=None

        python -m ibeis.init.main_helpers --exec-testdata_ibeis --db NNP_Master3 --acfg controlled --verbose-testdata
        python -m ibeis.init.main_helpers --exec-testdata_ibeis --db PZ_Master0 --acfg controlled --verbose-testdata
        python -m ibeis.init.main_helpers --exec-testdata_ibeis --db GZ_ALL --acfg controlled --verbose-testdata

    Example:
        >>> # ENABLE_DOCTEST
        >>> from ibeis.init.main_helpers import *  # NOQA
        >>> import ibeis
        >>> from ibeis.experiments import annotation_configs
        >>> default_qaids = [1]
        >>> default_daids = 'all'
        >>> defaultdb = 'testdb1'
        >>> ibs = None
        >>> verbose = False
        >>> return_annot_info = True
        >>> ibs, qaid_list, daid_list, aidcfg = testdata_ibeis(default_qaids, default_daids, defaultdb, ibs, verbose, return_annot_info)
        >>> print('Printing annot config')
        >>> annotation_configs.print_acfg(aidcfg)
        >>> print('Printing annotconfig stats')
        >>> #print('qaid_list = %r' % (np.array(qaid_list),))
        >>> ibs.get_annotconfig_stats(qaid_list, daid_list)
        >>> print('Combined annotconfig stats')
        >>> ibs.print_annot_stats(qaid_list + daid_list, yawtext_isect=True)
    """
    print('[testdata_ibeis] Getting test annot configs')
    import ibeis
    if ibs is None:
        ibs = ibeis.opendb(defaultdb=defaultdb)
    # TODO: rectify command line with function arguments
    from ibeis.experiments import experiment_helpers
    aidcfg_name_list = ut.get_argval(('--aidcfg', '--acfg', '-a'), type_=list, default=['default'])
    acfg_list, expanded_aids_list = experiment_helpers.get_annotcfg_list(ibs, aidcfg_name_list)

    #aidcfg = old_main_helpers.get_commandline_aidcfg()
    assert len(acfg_list) == 1, 'multiple acfgs specified, but this function is built to return only 1. len(acfg_list)=%r' % (len(acfg_list),)
    aidcfg = acfg_list[0]

    qaid_list, daid_list = expanded_aids_list[0]

    #ibs.get_annotconfig_stats(qaid_list, daid_list)

    if ut.VERYVERBOSE:
        ibeis.other.dbinfo.print_qd_info(ibs, qaid_list, daid_list, verbose=True)
    if return_annot_info:
        return ibs, qaid_list, daid_list, aidcfg
    else:
        return ibs, qaid_list, daid_list

#def register_utool_aliases():
#    """
#    registers commmon class names with utool so they are printed nicely
#    """
#    #print('REGISTER UTOOL ALIASES')
#    import utool as ut
#    import matplotlib as mpl
#    from ibeis.control import IBEISControl, SQLDatabaseControl
#    from ibeis.gui import guiback
#    #from ibeis.gui import guifront
#    ut.extend_global_aliases([
#        (SQLDatabaseControl.SQLDatabaseController, 'sqldb'),
#        (IBEISControl.IBEISController, 'ibs'),
#        (guiback.MainWindowBackend, 'back'),
#        #(guifront.MainWindowFrontend, 'front'),
#        (mpl.figure.Figure, 'fig')
#    ])

if __name__ == '__main__':
    """
    CommandLine:
        python -m ibeis.init.main_helpers
        python -m ibeis.init.main_helpers --allexamples
        python -m ibeis.init.main_helpers --allexamples --noface --nosrc
    """
    import multiprocessing
    multiprocessing.freeze_support()  # for win32
    import utool as ut  # NOQA
    ut.doctest_funcs()


# -*- coding: utf-8 -*-

import numpy as np
from scipy.integrate import solve_bvp, odeint, solve_ivp, simps, romb, romberg, quad
from scipy.interpolate import interp1d, CubicSpline
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
from tqdm import trange

k = 100 #Reaction rate constant.
DG = 1.16 #Diffusion coefficient of Gallium.
DA = 0 #Diffusion coefficient of Arsenic.
F0 = 0.04 #Deposition flux of Arsenic.
F = 2 * F0 #Resdience time of Arsenic.
tau = 1/F0 #Resdience time of Arsenic.
GaAs_normalisation = 500
y1a = 1.0 #Surface adatom concentration of Gallium at r = rD.
y3a = 0.0 #Surface adatom concentration of Arsenic at r = rD.
CAs_c = F * tau #Surface adatom concentration of Arsenic for large r.
kappa_0 = k * y1a * CAs_c / 500 #Normalisation constant

rDi = 1.0#Initial droplet radius (first r-mesh point at t = 0).
rb = 10.0 #Last r-mesh point.
rdivs = 513 #484, 0.003, 20000, 165 #Number of r-mesh points.#300 when tol=0.01
tolerance = 1e-7 #Accuracy to which CGa and CAs are found
max_mesh_nodes = 4e7 #Number of mesh nodes algorithm for bvp solver uses to find concentration profiles

tf = 0.1473 #Final time
rD_cutoff = 0.003 #point at which radius stops decreasing to avoid small number errors
tdivs = 513 #Number of t-mesh points.
V_Ga = 1.0 #Ga primitive cell volume
V_GaAs = 1.0 #GaAs primitive cell volume
beta_theta = 1.0 #Contact angle constant
A = 2.57142857 #Constant coefficient of first term in drD/dt expression.
B = 1.5 #Constant coefficient of second term in drD/dt expression.
b = 1.0 * rDi #Constant coefficient of Gaussian expression.
w = 0.15 * rDi #Width of Gaussian.
hi = 0.0 #Initial height of inner/outer ring (at all points).

plt.rcParams['font.size'] = 30

def dydr(r, y):
    return np.vstack((y[1],
                      -y[1]/r + k * y[0] * F/(DG * (k * y[0] + 1/tau))))

def y_bc(ya, yb):
    return np.array([ya[0] - y1a, yb[0]])

rmeshi = np.linspace(rDi, rb, rdivs)
rmeshf = np.linspace(0, rb - rDi, rdivs)
y_at_ti_guess = np.zeros((2, rmeshi.size))
calc_y_at_ti_vs_r = solve_bvp(dydr, y_bc, rmeshi, y_at_ti_guess, tol = tolerance, max_nodes = max_mesh_nodes)
calc_y_at_ti_vs_r.status

def plot_y_at_ti_vs_r():
    plt.figure(1, figsize = (11,11))
    plt.plot(rmeshi, calc_y_at_ti_vs_r.sol(rmeshi)[0], 'r--', lw = '3', label = r'$C_{Ga}/C^0_{Ga}$')
    y3 = F / (k * calc_y_at_ti_vs_r.sol(rmeshi)[0] + 1/tau)
    plt.plot(rmeshi, y3, 'g', linestyle = (0, (1, 1)), lw = '3', label = r'$C_{As}/C^c_{As}$')
    plt.plot(rmeshi, k * calc_y_at_ti_vs_r.sol(rmeshi)[0] * y3 / kappa_0, color = 'k', linewidth = 5, label = r'$k_rC_{Ga}C_{As} / \kappa_0$')
    plt.plot([rmeshi[0], rmeshi[0]], [0, y1a], color = 'k', lw = '2', linestyle = (0, (1, 10)))
    plt.xlabel(r'$r/r_D(0)$')
    plt.legend()

plot_y_at_ti_vs_r() #This produces a graph of the concentration profiles.

"""
Droplet radius
"""

def y2_value_at_rD(rD):
    def dydr(r, y):
        return np.vstack((y[1],
                          -y[1]/r + k * y[0] * F/(DG * (k * y[0] + 1/tau))))

    def y_bc(ya, yb):
        return np.array([ya[0] - y1a, yb[0]])

    rmesh = np.linspace(rD, rb - (rDi - rD), rdivs)
    y_guess = np.zeros((2, rmesh.size))
    return solve_bvp(dydr, y_bc, rmesh, y_guess, tol = tolerance, max_nodes = max_mesh_nodes).sol(rD)[1]

def drDdt(t, rD):
    if rD > rD_cutoff:
        return (1 / rD) * (V_Ga / beta_theta) * (DG * A * y2_value_at_rD(rD[0]) - B * b * w / V_GaAs) #(Sould include DG dependence later because this is one of the parameters varied in the paper.)
    else:
        return 0.0

tmesh = np.linspace(0, tf, tdivs)
calc_rD_vs_t = solve_ivp(drDdt, [0.0, tf], [rDi], dense_output = True, atol = 1e-6)
rD_vs_t = calc_rD_vs_t.sol(tmesh)

y2a_vs_t = [y2_value_at_rD(i) for i in rD_vs_t[0]]

plt.figure(652, figsize = (11,11))
plt.xlabel(r'$t / t_f$')
plt.ylabel(r'$-\frac{dC_{Ga}(r_D, t)}{dr}$')
plt.yscale('log')
plt.plot(tmesh / tf, [-i for i in y2a_vs_t], color = 'r', lw = 4)

def plot_rD_vs_t():
        plt.figure(4, figsize = (11,11))
        plt.xlabel(r'$t / t_f$')
        plt.ylabel(r'$r_D(t) / r_D(0)$')
        x = np.linspace(0, tf, tdivs*500)
        x = tmesh
        plt.plot(x/tf, calc_rD_vs_t.sol(x)[0], color = 'r', lw = 4)

plot_rD_vs_t() #This produces a graph of droplet radius as a function of time.

def rD_value_at_t(t):
    if t <= tf:
        return calc_rD_vs_t.sol(t)[0]
    elif t > tf:
        return 0

"""
Inner ring
"""

def dhIdt(t, r):
    if rD_value_at_t(t) > rD_cutoff:
        return b*np.exp(-((r - rD_value_at_t(t))**2)/(2*w**2))
    else:
       return 0.0

hIf_vs_r = []
for r in rmeshf:
    hOf_at_r = romberg(dhIdt, 0, tf, args = (r,))
    hIf_vs_r.append(hOf_at_r)

hI1_vs_r = []
for r in rmeshf:
    hOf_at_r = romberg(dhIdt, 0, tf*(1/4), args = (r,))
    hI1_vs_r.append(hOf_at_r)

hI2_vs_r = []
for r in rmeshf:
    hOf_at_r = romberg(dhIdt, 0, tf*(2/4), args = (r,))
    hI2_vs_r.append(hOf_at_r)

hI3_vs_r = []
for r in rmeshf:
    hOf_at_r = romberg(dhIdt, 0, tf*(3/4), args = (r,))
    hI3_vs_r.append(hOf_at_r)

plt.figure(908, figsize = (11,11))
plt.plot(rmeshf, hI1_vs_r, color = 'r', label = r'$t = \frac{1}{4}t_f$')
plt.plot(rmeshf, hI2_vs_r, color = 'g', label = r'$t = \frac{1}{2}t_f$')
plt.plot(rmeshf, hI3_vs_r, color = 'b', label = r'$t = \frac{3}{4}t_f$')
plt.plot(rmeshf, hIf_vs_r, color = 'k', label = r'$t = t_f$')
plt.xlabel(r'$r/r_D(0)$')
plt.yticks([])
plt.legend()

"""
Outer ring
"""

y_vs_r_at_all_t = []

def Append_solution(rD):
    def dydr(r, y):
        return np.vstack((y[1],
                          -y[1]/r + k * y[0] * F/(DG * (k * y[0] + 1/tau))))

    def y_bc(ya, yb):
        return np.array([ya[0] - y1a, yb[0]])

    rmesh = np.linspace(rD, rb - (rDi - rD), rdivs)
    y_guess = np.zeros((2, rmesh.size))
    calc_y_vs_r_at_t = solve_bvp(dydr, y_bc, rmesh, y_guess, tol = tolerance, max_nodes = max_mesh_nodes).sol
    y_vs_r_at_all_t.append(calc_y_vs_r_at_t)
    return None

for i in trange(len(rD_vs_t[0])):
    Append_solution(rD_vs_t[0][i])

y1y3_values_mesh_points = []
for t in range(len(tmesh)):
    y1y3_over_rmesh_fixed_t = []
    for r in rmeshf:
        y1 = y_vs_r_at_all_t[t](r)[0]
        y3 = F / (k * y_vs_r_at_all_t[t](r)[0] + 1/tau)
        y1y3 = k * V_GaAs * y1 * y3
        if y1y3 > 1e-17 and rD_value_at_t(tmesh[t]) > rD_cutoff:
            y1y3_over_rmesh_fixed_t.append(y1y3)
        else:
            y1y3_over_rmesh_fixed_t.append(0.0)
    y1y3_values_mesh_points.append(y1y3_over_rmesh_fixed_t)

plt.figure(903)
for i in range(len(tmesh)):
    plt.plot(rmeshf, y1y3_values_mesh_points[i])

y1y3_values_mesh_points_r_vs_t = []
for r in rmeshf:
    y1y3_over_rmesh_fixed_r = []
    for t in range(len(tmesh)):
        y1 = y_vs_r_at_all_t[t](r)[0]
        y3 = F / (k * y_vs_r_at_all_t[t](r)[0] + 1/tau)
        y1y3 = k * V_GaAs * y1 * y3
        if y1y3 > 0 and rD_value_at_t(tmesh[t]) > rD_cutoff: #1e-17:
            y1y3_over_rmesh_fixed_r.append(y1y3)
        else:
            y1y3_over_rmesh_fixed_r.append(0.0)
    y1y3_values_mesh_points_r_vs_t.append(y1y3_over_rmesh_fixed_r)

plt.figure(904)
for i in range(len(rmeshf)):
    plt.plot(tmesh, y1y3_values_mesh_points_r_vs_t[i])

delta_t = tf/(tdivs - 1)

hOf_vs_r = []
for i in range(len(rmeshf)):
    hOf_at_r = simps(y1y3_values_mesh_points_r_vs_t[i], tmesh)
    hOf_vs_r.append(hOf_at_r)

hO1_vs_r = []
for i in range(len(rmeshf)):
    hOf_at_r = simps(y1y3_values_mesh_points_r_vs_t[i][:int((1/4) * tdivs)], tmesh[:int((1/4) * tdivs)])
    hO1_vs_r.append(hOf_at_r)

hO2_vs_r = []
for i in range(len(rmeshf)):
    hOf_at_r = simps(y1y3_values_mesh_points_r_vs_t[i][:int((2/4) * tdivs)], tmesh[:int((2/4) * tdivs)])
    hO2_vs_r.append(hOf_at_r)

hO3_vs_r = []
for i in range(len(rmeshf)):
    hOf_at_r = simps(y1y3_values_mesh_points_r_vs_t[i][:int((3/4) * tdivs)], tmesh[:int((3/4) * tdivs)])
    hO3_vs_r.append(hOf_at_r)

plt.figure(905)
plt.plot(rmeshf, hO1_vs_r, color = 'r', label = r'$t = \frac{1}{4}t_f$')
plt.plot(rmeshf, hO2_vs_r, color = 'g', label = r'$t = \frac{1}{2}t_f$')
plt.plot(rmeshf, hO3_vs_r, color = 'b', label = r'$t = \frac{3}{4}t_f$')
plt.plot(rmeshf, hOf_vs_r, color = 'k', label = r'$t = t_f$')
plt.xlabel(r'$r/r_D(0)$')
plt.yticks([])
plt.legend()

normalise = 5

h1_vs_r = np.array(hI1_vs_r) + normalise * np.array(hO1_vs_r)
h2_vs_r = np.array(hI2_vs_r) + normalise * np.array(hO2_vs_r)
h3_vs_r = np.array(hI3_vs_r) + normalise * np.array(hO3_vs_r)
hf_vs_r = np.array(hIf_vs_r) + normalise * np.array(hOf_vs_r)

h1_vs_r_T = np.insert(h1_vs_r, 0, h1_vs_r[::-1][:-1])
h2_vs_r_T = np.insert(h2_vs_r, 0, h2_vs_r[::-1][:-1])
h3_vs_r_T = np.insert(h3_vs_r, 0, h3_vs_r[::-1][:-1])
hf_vs_r_T = np.insert(hf_vs_r, 0, hf_vs_r[::-1][:-1])

rmeshf_T = np.linspace(-rmeshf[-1], rmeshf[-1], rdivs * 2 - 1)

plt.figure(906)
plt.plot(rmeshf_T, h1_vs_r_T, color = 'r', label = r'$t = \frac{1}{4}t_f$')
plt.plot(rmeshf_T, h2_vs_r_T, color = 'g', label = r'$t = \frac{1}{2}t_f$')
plt.plot(rmeshf_T, h3_vs_r_T, color = 'b', label = r'$t = \frac{3}{4}t_f$')
plt.plot(rmeshf_T, hf_vs_r_T, color = 'k', label = r'$t = t_f$')
plt.xlabel(r'$r/r_D(0)$')
plt.yticks([])
plt.legend()

hI1_vs_r_T = np.insert(np.array(hI1_vs_r), 0, np.array(hI1_vs_r)[::-1][:-1])
hI2_vs_r_T = np.insert(np.array(hI2_vs_r), 0, np.array(hI2_vs_r)[::-1][:-1])
hI3_vs_r_T = np.insert(np.array(hI3_vs_r), 0, np.array(hI3_vs_r)[::-1][:-1])
hIf_vs_r_T = np.insert(np.array(hIf_vs_r), 0, np.array(hIf_vs_r)[::-1][:-1])

hO1_vs_r_T = normalise * np.insert(np.array(hO1_vs_r), 0, np.array(hO1_vs_r)[::-1][:-1])
hO2_vs_r_T = normalise * np.insert(np.array(hO2_vs_r), 0, np.array(hO2_vs_r)[::-1][:-1])
hO3_vs_r_T = normalise * np.insert(np.array(hO3_vs_r), 0, np.array(hO3_vs_r)[::-1][:-1])
hOf_vs_r_T = normalise * np.insert(np.array(hOf_vs_r), 0, np.array(hOf_vs_r)[::-1][:-1])

plt.figure(888, figsize = (11,11))
plt.plot(rmeshf_T, hf_vs_r_T, color = 'k', lw = 8, label = r'$h(r/r_D(0), t = t_f)$')
plt.plot(rmeshf_T, hIf_vs_r_T, color = 'g', lw = 5, label = r'$h_I(r/r_D(0), t = t_f)$')
plt.plot(rmeshf_T, hOf_vs_r_T, color = 'r', lw = 5, label = r'$h_O(r/r_D(0), t = t_f)$')
plt.xlabel(r'$r/r_D(0)$')
plt.yticks([])
#plt.legend()

plt.figure(889, figsize = (11,11))
plt.plot(rmeshf_T, hf_vs_r_T, color = 'k', lw = 8, label = r'$h(r/r_D(0), t = t_f)$')
plt.ylim(0, 1.7 * max(hf_vs_r_T))
plt.xlabel(r'$r/r_D(0)$')
plt.ylabel(r'$h(r/r_D(0))$')
plt.yticks([])

#Following piece of code for 3D plot adapted from code written by ImportanceOfBeingErnest on https://stackoverflow.com/questions/47333811/how-do-i-create-a-surface-plot-with-matplotlib-of-a-closed-loop-revolve-about-an

x_values = rmeshf
# input xy coordinates
x_y_coordinates = [[x_values[i], hf_vs_r[i]] for i in range(len(x_values))]
xy = np.array(x_y_coordinates)
# radial component is x values of input
r = xy[:,0]
# angular component is one revolution of 60 steps
phi = np.linspace(0, 2*np.pi, 60)
# create grid
R,Phi = np.meshgrid(r,phi)
# transform to cartesian coordinates
X = R*np.cos(Phi)
Y = R*np.sin(Phi)
# Z values are y values, repeated 60 times
Z = np.tile(xy[:,1],len(Y)).reshape(Y.shape)

fig = plt.figure()
ax = fig.add_subplot(1, 1, 1, projection='3d')
ax2 = fig.add_axes([0.05,0.7,0.15,.2])
ax2.plot(xy[:,0],xy[:,1], color="k")

ax.plot_surface(X, Y, Z, alpha=0.5, color='gold', rstride=1, cstride=1)

plt.show()


# -*- coding: utf-8 -*-
# @Time    : 2020/12/6
# @Author  : Lart Pang
# @GitHub  : https://github.com/lartpang

from torch import nn

from backbone.origin.from_origin import Backbone_V16_Custumed
from module.BaseBlocks import BasicConv2d
from utils.tensor_ops import cus_sample, upsample_add

class SIM(nn.Module):
    def __init__(self, h_C, l_C):
        super(SIM, self).__init__()
        self.h2l_pool = nn.AvgPool2d((2, 2), stride=2)
        self.l2h_up = cus_sample

        self.h2l_0 = nn.Conv2d(h_C, l_C, 3, 1, 1)
        self.h2h_0 = nn.Conv2d(h_C, h_C, 3, 1, 1)
        self.bnl_0 = nn.BatchNorm2d(l_C)
        self.bnh_0 = nn.BatchNorm2d(h_C)

        self.h2h_1 = nn.Conv2d(h_C, h_C, 3, 1, 1)
        self.h2l_1 = nn.Conv2d(h_C, l_C, 3, 1, 1)
        self.l2h_1 = nn.Conv2d(l_C, h_C, 3, 1, 1)
        self.l2l_1 = nn.Conv2d(l_C, l_C, 3, 1, 1)
        self.bnl_1 = nn.BatchNorm2d(l_C)
        self.bnh_1 = nn.BatchNorm2d(h_C)

        self.h2h_2 = nn.Conv2d(h_C, h_C, 3, 1, 1)
        self.l2h_2 = nn.Conv2d(l_C, h_C, 3, 1, 1)
        self.bnh_2 = nn.BatchNorm2d(h_C)

        self.relu = nn.ReLU(True)

    def forward(self, x):
        h, w = x.shape[2:]

        # first conv
        x_h = self.relu(self.bnh_0(self.h2h_0(x)))
        x_l = self.relu(self.bnl_0(self.h2l_0(self.h2l_pool(x))))

        # mid conv
        x_h2h = self.h2h_1(x_h)
        x_h2l = self.h2l_1(self.h2l_pool(x_h))
        x_l2l = self.l2l_1(x_l)
        x_l2h = self.l2h_1(self.l2h_up(x_l, size=(h, w)))
        x_h = self.relu(self.bnh_1(x_h2h + x_l2h))
        x_l = self.relu(self.bnl_1(x_l2l + x_h2l))

        # last conv
        x_h2h = self.h2h_2(x_h)
        x_l2h = self.l2h_2(self.l2h_up(x_l, size=(h, w)))
        x_h = self.relu(self.bnh_2(x_h2h + x_l2h))

        return x_h

class conv_2nV1(nn.Module):
    def __init__(self, in_hc=64, in_lc=256, out_c=64, main=0):
        super(conv_2nV1, self).__init__()
        self.main = main
        mid_c = min(in_hc, in_lc)
        self.relu = nn.ReLU(True)
        self.h2l_pool = nn.AvgPool2d((2, 2), stride=2)
        self.l2h_up = nn.Upsample(scale_factor=2, mode="nearest")

        # stage 0
        self.h2h_0 = nn.Conv2d(in_hc, mid_c, 3, 1, 1)
        self.l2l_0 = nn.Conv2d(in_lc, mid_c, 3, 1, 1)
        self.bnh_0 = nn.BatchNorm2d(mid_c)
        self.bnl_0 = nn.BatchNorm2d(mid_c)

        # stage 1
        self.h2h_1 = nn.Conv2d(mid_c, mid_c, 3, 1, 1)
        self.h2l_1 = nn.Conv2d(mid_c, mid_c, 3, 1, 1)
        self.l2h_1 = nn.Conv2d(mid_c, mid_c, 3, 1, 1)
        self.l2l_1 = nn.Conv2d(mid_c, mid_c, 3, 1, 1)
        self.bnl_1 = nn.BatchNorm2d(mid_c)
        self.bnh_1 = nn.BatchNorm2d(mid_c)

        if self.main == 0:
            # stage 2
            self.h2h_2 = nn.Conv2d(mid_c, mid_c, 3, 1, 1)
            self.l2h_2 = nn.Conv2d(mid_c, mid_c, 3, 1, 1)
            self.bnh_2 = nn.BatchNorm2d(mid_c)

            # stage 3
            self.h2h_3 = nn.Conv2d(mid_c, out_c, 3, 1, 1)
            self.bnh_3 = nn.BatchNorm2d(out_c)

            self.identity = nn.Conv2d(in_hc, out_c, 1)

        elif self.main == 1:
            # stage 2
            self.h2l_2 = nn.Conv2d(mid_c, mid_c, 3, 1, 1)
            self.l2l_2 = nn.Conv2d(mid_c, mid_c, 3, 1, 1)
            self.bnl_2 = nn.BatchNorm2d(mid_c)

            # stage 3
            self.l2l_3 = nn.Conv2d(mid_c, out_c, 3, 1, 1)
            self.bnl_3 = nn.BatchNorm2d(out_c)

            self.identity = nn.Conv2d(in_lc, out_c, 1)

        else:
            raise NotImplementedError

    def forward(self, in_h, in_l):
        # stage 0
        h = self.relu(self.bnh_0(self.h2h_0(in_h)))
        l = self.relu(self.bnl_0(self.l2l_0(in_l)))

        # stage 1
        h2h = self.h2h_1(h)
        h2l = self.h2l_1(self.h2l_pool(h))
        l2l = self.l2l_1(l)
        l2h = self.l2h_1(self.l2h_up(l))
        h = self.relu(self.bnh_1(h2h + l2h))
        l = self.relu(self.bnl_1(l2l + h2l))

        if self.main == 0:
            # stage 2
            h2h = self.h2h_2(h)
            l2h = self.l2h_2(self.l2h_up(l))
            h_fuse = self.relu(self.bnh_2(h2h + l2h))

            # stage 3
            out = self.relu(self.bnh_3(self.h2h_3(h_fuse)) + self.identity(in_h))
            # 这里使用的不是in_h，而是h
        elif self.main == 1:
            # stage 2
            h2l = self.h2l_2(self.h2l_pool(h))
            l2l = self.l2l_2(l)
            l_fuse = self.relu(self.bnl_2(h2l + l2l))

            # stage 3
            out = self.relu(self.bnl_3(self.l2l_3(l_fuse)) + self.identity(in_l))
        else:
            raise NotImplementedError

        return out

class conv_3nV1(nn.Module):
    def __init__(self, in_hc=64, in_mc=256, in_lc=512, out_c=64):
        super(conv_3nV1, self).__init__()
        self.upsample = nn.Upsample(scale_factor=2, mode="nearest")
        self.downsample = nn.AvgPool2d((2, 2), stride=2)

        mid_c = min(in_hc, in_mc, in_lc)
        self.relu = nn.ReLU(True)

        # stage 0
        self.h2h_0 = nn.Conv2d(in_hc, mid_c, 3, 1, 1)
        self.m2m_0 = nn.Conv2d(in_mc, mid_c, 3, 1, 1)
        self.l2l_0 = nn.Conv2d(in_lc, mid_c, 3, 1, 1)
        self.bnh_0 = nn.BatchNorm2d(mid_c)
        self.bnm_0 = nn.BatchNorm2d(mid_c)
        self.bnl_0 = nn.BatchNorm2d(mid_c)

        # stage 1
        self.h2h_1 = nn.Conv2d(mid_c, mid_c, 3, 1, 1)
        self.h2m_1 = nn.Conv2d(mid_c, mid_c, 3, 1, 1)
        self.m2h_1 = nn.Conv2d(mid_c, mid_c, 3, 1, 1)
        self.m2m_1 = nn.Conv2d(mid_c, mid_c, 3, 1, 1)
        self.m2l_1 = nn.Conv2d(mid_c, mid_c, 3, 1, 1)
        self.l2m_1 = nn.Conv2d(mid_c, mid_c, 3, 1, 1)
        self.l2l_1 = nn.Conv2d(mid_c, mid_c, 3, 1, 1)
        self.bnh_1 = nn.BatchNorm2d(mid_c)
        self.bnm_1 = nn.BatchNorm2d(mid_c)
        self.bnl_1 = nn.BatchNorm2d(mid_c)

        # stage 2
        self.h2m_2 = nn.Conv2d(mid_c, mid_c, 3, 1, 1)
        self.l2m_2 = nn.Conv2d(mid_c, mid_c, 3, 1, 1)
        self.m2m_2 = nn.Conv2d(mid_c, mid_c, 3, 1, 1)
        self.bnm_2 = nn.BatchNorm2d(mid_c)

        # stage 3
        self.m2m_3 = nn.Conv2d(mid_c, out_c, 3, 1, 1)
        self.bnm_3 = nn.BatchNorm2d(out_c)

        self.identity = nn.Conv2d(in_mc, out_c, 1)

    def forward(self, in_h, in_m, in_l):
        # stage 0
        h = self.relu(self.bnh_0(self.h2h_0(in_h)))
        m = self.relu(self.bnm_0(self.m2m_0(in_m)))
        l = self.relu(self.bnl_0(self.l2l_0(in_l)))

        # stage 1
        h2h = self.h2h_1(h)
        m2h = self.m2h_1(self.upsample(m))

        h2m = self.h2m_1(self.downsample(h))
        m2m = self.m2m_1(m)
        l2m = self.l2m_1(self.upsample(l))

        m2l = self.m2l_1(self.downsample(m))
        l2l = self.l2l_1(l)

        h = self.relu(self.bnh_1(h2h + m2h))
        m = self.relu(self.bnm_1(h2m + m2m + l2m))
        l = self.relu(self.bnl_1(m2l + l2l))

        # stage 2
        h2m = self.h2m_2(self.downsample(h))
        m2m = self.m2m_2(m)
        l2m = self.l2m_2(self.upsample(l))
        m = self.relu(self.bnm_2(h2m + m2m + l2m))

        # stage 3
        out = self.relu(self.bnm_3(self.m2m_3(m)) + self.identity(in_m))
        return out

class AIM(nn.Module):
    def __init__(self, iC_list, oC_list):
        super(AIM, self).__init__()
        ic0, ic1, ic2, ic3, ic4 = iC_list
        oc0, oc1, oc2, oc3, oc4 = oC_list
        self.conv0 = conv_2nV1(in_hc=ic0, in_lc=ic1, out_c=oc0, main=0)
        self.conv1 = conv_3nV1(in_hc=ic0, in_mc=ic1, in_lc=ic2, out_c=oc1)
        self.conv2 = conv_3nV1(in_hc=ic1, in_mc=ic2, in_lc=ic3, out_c=oc2)
        self.conv3 = conv_3nV1(in_hc=ic2, in_mc=ic3, in_lc=ic4, out_c=oc3)
        self.conv4 = conv_2nV1(in_hc=ic3, in_lc=ic4, out_c=oc4, main=1)

    def forward(self, *xs):
        # in_data_2, in_data_4, in_data_8, in_data_16, in_data_32
        out_xs = []
        out_xs.append(self.conv0(xs[0], xs[1]))
        out_xs.append(self.conv1(xs[0], xs[1], xs[2]))
        out_xs.append(self.conv2(xs[1], xs[2], xs[3]))
        out_xs.append(self.conv3(xs[2], xs[3], xs[4]))
        out_xs.append(self.conv4(xs[3], xs[4]))

        return out_xs

class MINet_VGG16(nn.Module):
    def __init__(self):
        super(MINet_VGG16, self).__init__()
        self.upsample_add = upsample_add
        self.upsample = cus_sample

        self.encoder1, self.encoder2, self.encoder4, self.encoder8, self.encoder16 = Backbone_V16_Custumed(3)
        self.trans = AIM((64, 128, 256, 512, 512), (32, 64, 64, 64, 64))

        self.sim16 = SIM(64, 32)
        self.sim8 = SIM(64, 32)
        self.sim4 = SIM(64, 32)
        self.sim2 = SIM(64, 32)
        self.sim1 = SIM(32, 16)

        self.upconv16 = BasicConv2d(64, 64, kernel_size=3, stride=1, padding=1)
        self.upconv8 = BasicConv2d(64, 64, kernel_size=3, stride=1, padding=1)
        self.upconv4 = BasicConv2d(64, 64, kernel_size=3, stride=1, padding=1)
        self.upconv2 = BasicConv2d(64, 32, kernel_size=3, stride=1, padding=1)
        self.upconv1 = BasicConv2d(32, 32, kernel_size=3, stride=1, padding=1)

        self.classifier = nn.Conv2d(32, 1, 1)

    def forward(self, data):
        in_data = data["image"]
        in_data_1 = self.encoder1(in_data)
        in_data_2 = self.encoder2(in_data_1)
        in_data_4 = self.encoder4(in_data_2)
        in_data_8 = self.encoder8(in_data_4)
        in_data_16 = self.encoder16(in_data_8)

        in_data_1, in_data_2, in_data_4, in_data_8, in_data_16 = self.trans(
            in_data_1, in_data_2, in_data_4, in_data_8, in_data_16
        )

        out_data_16 = self.upconv16(self.sim16(in_data_16) + in_data_16)  # 1024
        out_data_8 = self.upsample_add(out_data_16, in_data_8)
        out_data_8 = self.upconv8(self.sim8(out_data_8) + out_data_8)  # 512
        out_data_4 = self.upsample_add(out_data_8, in_data_4)
        out_data_4 = self.upconv4(self.sim4(out_data_4) + out_data_4)  # 256
        out_data_2 = self.upsample_add(out_data_4, in_data_2)
        out_data_2 = self.upconv2(self.sim2(out_data_2) + out_data_2)  # 64
        out_data_1 = self.upsample_add(out_data_2, in_data_1)
        out_data_1 = self.upconv1(self.sim1(out_data_1) + out_data_1)  # 32
        seg_logits = self.classifier(out_data_1)

        return dict(seg=seg_logits)


import re

#res=re.match((25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[0]*[0-9][0-9]*)\.(25[0-5]|2[0-4][0-9]|[01][0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?),a)

import numpy as np
c = np.array([[5,6,7],[7,6,5],[0,8,7]])
print(c.shape)
print(c[:,1])

# Read the input list
import numpy as np

# Convert the input list to a NumPy array
array_2d =np.array([[11, 12, 13, 14], [21, 22, 23, 24], [31, 32, 33, 34]])

# Extract the number of rows and columns of the array
rows = len(array_2d[:, 0])
cols = len(array_2d[0, :])

# Extract the first column, first row, last column and last row respectively using
# appropriate indexing
col_1 = array_2d[:, 0]
row_1 = array_2d[0, :]
col_last = array_2d[:, cols-1]
row_last = array_2d[rows-1, :]

print(col_1)
print(row_1)
print(col_last)
print(row_last)

import numpy as np
seed=0
n=10
p=0.5
#write your code here
np.random.seed(0)
s = np.random.binomial(n, p, n)
print(s)

# import scipy.stats
# scipy.stats.norm(100, 12).pdf(98)

# Python 3.x code to demonstrate star pattern

# Function to demonstrate printing pattern of numbers
def numpat(n):
    # initialising starting number
    num = 1
    # outer loop to handle number of rows
    for i in range(0, n):
        # re assigning num
        num = 1
        # inner loop to handle number of columns
        # values changing acc. to outer loop
        for j in range(0, i + 1):
            # printing number
            a  = "{}".format(num)
            print(int(a), end="")
            # incrementing number at each column
            num = num + 1
        for k in range(i ,0,-1):
            # printing number
            b = "{}".format(k)
            print(int(b), end="")
            # incrementing number at each column

        # ending line after each row

        if  i != n-1 :
            print("\r")

# Driver code
n = 4
# numpat(n)

# for i in range(1,5):
#     print (((10 ** i - 1) // 9) ** 2)
import math
print (0.5 * (1 + math.erf(100 - 90)/math.sqrt(2 * 10**2)))

# from statistics import NormalDist
#
# NormalDist(mu=100, sigma=12).pdf(98)
#
# n = 15
# a =  n - 3
# b = n - 1
# c  = a/b
# d = "{:.4f}".format(c)
# e = float(d)
# print(e)
j  = 100
if j > 10 and j < 200 :
    print("ss")
# import pandas as pd
# df  = pd.read_csv("sample.csv")
# df.sort_values("ff",ascending=True)

lisst = ["USA", "IND"]
import requests
from bs4 import BeautifulSoup

html_page  = requests.get("https://en.wikipedia.org/wiki/List_of_territorial_entities_where_English_is_an_official_language")
print(dir(html_page))
print(html_page.content)
soup = BeautifulSoup(html_page.content, 'html5lib')
print(soup)
print(soup.prettify())

# data_frame1 = ''
# data_frame1 =
#
# data_frame1.loc[dataframe_1["funding_round_type"] >= 5000000 and  [dataframe_1["funding_round_type"] < 150000000,["permalink", "funding_round_type", "raised_amount_usd", "category_list",  "country_code" ]]

import time

start_time = time.time()
def sample():
    for i in range(100000):
        print("aa")
sample()
end_time = time.time()
print(end_time-start_time)


#
# Plot for generating Figures 10.8, 10.9 and 10.10 of the Pathway Modeling book
#
import pylab as plt; import numpy as np
import lmfit; import tellurium as te
import time; import copy; import random; import sys
import emcee
import time

# Fix the data, not change in the errors in the data between runs
np.random.seed (126)
#np.random.seed (2)

nsims = 0

r = te.loada("""
   S1 -> S2; k1*S1;
   S2 -> S3; k2*S2;

   S1 = 1; S2 = 0; S3 = 0;
   k1 = 0.45; k2 = 0.15;
""")

toFit = ['k1', 'k2']; nParameters = len (toFit)

nDataPoints = 32
timeToSimulate = 20
# Create the experimental data
# First column is time, other columns are species
m = r.simulate (0, timeToSimulate, nDataPoints)

# Plot ground truth
plt.figure(figsize=(7,5))
plt.plot (m['time'], m['[S1]'], linewidth=3, label='S1')
plt.plot (m['time'], m['[S2]'], linewidth=3, label='S2')
plt.plot (m['time'], m['[S3]'], linewidth=3, label='S3')
plt.xlabel('Time', fontsize=18)
plt.ylabel('Concentation', fontsize=18)
plt.legend (fontsize=18)
plt.savefig  ('c:\\tmp\\groundtruth_simpleModel.pdf')
plt.show()

# Change this index to use a different variables
# These are the variables that will be used to fit the model
SIndexList = [3] # 1 = S1, 2 = S2, 3 = S3
x_data = m['time']; # Extract the time column

# Extract the SIndexList columns into y_data
y_data = []
for i in range (len(SIndexList)):
    y_data.append (m[:,SIndexList[i]])

# Create the 'experimental' data by adding noise
y_noise = np.empty([nDataPoints])
for k in range (len (SIndexList)):
   for i in range (0, len (y_data[k])):
       y_noise[i] = 0.1 # standard deviation of noise
       # Might be better to use lognormal here?
       ln = np.random.normal (0, y_noise[i]);
       while y_data[k][i] + ln < 0:
             ln = np.random.normal (0, y_noise[i]);
       y_data[k][i] = y_data[k][i] + ln # Add noise

# Plot ground truth
#plt.figure(figsize=(7,5))
index = 0
for i in SIndexList:
    plt.plot (m['time'], y_data[index], 'or', linewidth=3, label='S' + str (SIndexList));
    plt.xlabel('Time', fontsize=18)
    plt.ylabel('Concentation', fontsize=18)
    plt.legend (fontsize=18)
    plt.savefig  ('c:\\tmp\\groundtruthS' + str (i) + '.pdf')
    plt.show()
    index = index + 1

# Compute the simulation at the current parameter values
# Return the time series variable indicated by SIndex
# This is very inefficient since it has to be called for
# each variable we use to fit the model. Eg if SIndexList = [1,2,3]
# then we'll have to call this three times, ideally we should call
# it once and collect all SIndex variables at once.
def computeSimulationData(p, SIndex):
    global nsims
    r.reset()
    pp = p.valuesdict()
    for i in range(0, nParameters):
       r.model[toFit[i]] = pp[toFit[i]]
    m = r.simulate (0, timeToSimulate, nDataPoints)
    nsims = nsims + 1
    return m[:,SIndex]

# Compute the residuals between objective and experimental data
#def weightedResiduals(p):
#    return (y_data - my_ls_func (p))/y_weight

# Compute the residuals between objective and experimental data
def residuals(p):
    global y_data, SIndexList
    y1 = (y_data[0] - computeSimulationData (p, SIndexList[0]));
    y1 = np.concatenate ((y1, ))
    for k in range (1, len (SIndexList)):
        y1 = np.concatenate ((y1, (y_data[k] - computeSimulationData (p, SIndexList[k]))))
    return y1

def unWeightedResiduals(p):
    y1 = (y_data[0] - computeSimulationData (p, SIndexList[0]))
    return y1

# Randomize the work of the optimizer
rd = int (time.time())
print ('Random number = ', rd)
np.random.seed (rd)

# Set up the parameters that we will fit
params = lmfit.Parameters()
params.add('k1', value=1, min=0, max=10)
params.add('k2', value=1, min=0, max=10)

# Compute the fitted parameters
minimizer = lmfit.Minimizer(residuals, params)
result = minimizer.minimize(method='differential_evolution')#'leastsqr')
lmfit.report_fit(result.params, min_correl=0.5)
result = minimizer.minimize(method='leastsqr')
lmfit.report_fit(result.params, min_correl=0.5)

# Assign fitted parameters to the model
r.reset()
for i in range(0, nParameters):
   r.model[toFit[i]] = result.params[toFit[i]].value
m = r.simulate (0, timeToSimulate, 100)

# Set up some convenient font sizes
plt.rcParams.update({'axes.titlesize': 16})
plt.rcParams.update({'axes.labelsize': 14})
plt.rcParams.update({'xtick.labelsize': 13})
plt.rcParams.update({'ytick.labelsize': 13})

# Plot experimental data
plt.figure (figsize=(7,5))
ldata, = plt.plot (x_data, y_data[0], 'dm', markersize=8)
for k in range (1, len (SIndexList)):
    ldata2, = plt.plot (x_data, y_data[k], 'dm', markersize=8)

# Plot the fitted lines for S1, S2 and S3
# Retrieve lfit to use in the legend
lfit, = plt.plot (m[:,0], m[:,1], '-g', linewidth=2)
plt.plot (m[:,0], m[:,2], '-g', linewidth=2)
plt.plot (m[:,0], m[:,3], '-g', linewidth=2)

# Plot the residuals
resids = unWeightedResiduals(result.params)
lresids, = plt.plot (x_data, resids, 'bo', markersize=6)
plt.vlines(x_data, [0], resids, color='r', linewidth=2)

theResiduals = copy.deepcopy (resids)
#finalFittedData = copy.deepcopy (y_data)
originalYData = copy.deepcopy (y_data)

plt.tick_params(axis='both', which='major', labelsize=16)
plt.xlabel('Time')
plt.ylabel("Concentration", fontsize=16)
plt.legend([ldata, lfit, lresids],['Data', 'Best fit', 'Residuals'], loc=0, fontsize=10)
plt.axhline (y=0, color='k')
plt.savefig('fittedCurves.pdf')
plt.show()

to = time.time()
# Boostrapping analysis
if True:
    k1Sample = []; k2Sample = []; NSamples = 6000

    print ("\nStart Monte Carlo Estimation")

    chis = []
    to = time.time()
    nsims = 0
    # Start the Monte Carlo parameter confidence estimation
    for n in range (NSamples):
        if n % 100 == 0:
           print (n)

        for j in range (len (y_data[0])):
            for i in range (len (SIndexList)):
                y_data[i][j] = originalYData[i][j] + random.choice (theResiduals)

        #result = minimizer.minimize(method='differential_evolution')
        result = minimizer.minimize(method='leastsqr')
        result = minimizer.minimize(method='leastsqr')
        if result.success == False:
            print ('Failed')
        # Not all fits will work so we test against a threshold
        if result.redchi < 500:
           chis.append (result.redchi)
           pp = result.params.valuesdict()
           # Collect the fitted parameters
           k1Sample.append (pp['k1'])
           k2Sample.append (pp['k2'])
        else:
           print (result.redchi)

    print ("Finished Monte Carlo Estimation.....")
    if len (k1Sample) == 0:
        print ("Failed to compute Monto Carlo estimate, unable to generate fits, bad model or insufficient data")
        sys.exit("Unable to continue")

    print ('Time for Bootstrap = ', time.time() - to)
    print ('Number of simulations = ', nsims)

    # Compute the mean values of the k1 and k2 samples
    meank1 = np.mean (k1Sample); meank2 = np.mean (k2Sample)

    # Compute 95% percentiles
    plusk1 = np.percentile (k1Sample, 97.5) - meank1
    minusk1 = meank1 - np.percentile (k1Sample, 2.5)

    plusk2 = np.percentile (k2Sample, 97.5) - meank2
    minusk2 = meank2 - np.percentile (k2Sample, 2.5)

    # Note that the limits are not symmetric
    print ('Computed 95 percent percentiles from the Monte Carlo run:')
    print ('k1: ', meank1, "+/- ", plusk1, minusk1)
    print ('k2: ', meank2, "+/- ", plusk2, minusk2)

    plt.hist (k1Sample, 10, range=[0,1], color='peachpuff')
    plt.ylabel('Frequency');
    plt.xlabel ('k1'); plt.title ('k1 variation')
    plt.savefig('k1Distribution.pdf')
    plt.show()
    plt.hist (k2Sample, 10, range=[0,1], color='peachpuff')
    plt.ylabel('Frequency');
    plt.xlabel ('k2'); plt.title ('k2 variation');
    plt.savefig('k2Distribution.pdf')
    plt.show()

    # Scatter plot of k1 versus k2
    plt.xlim((0.2, 0.65)); plt.ylim ((0.0, 0.3))
    plt.plot (k1Sample, k2Sample, '.', color='cornflowerblue')
    plt.xlabel ('k1'); plt.ylabel ('k2')
    plt.title ('Scatter plot of k1 against k2')
    plt.savefig('k1_k2_scatter.pdf')
    plt.show()
print ('Time for monte carlo = ', time.time() - to)

# Chi Square Analysis
if False:
    to = time.time()
    ci = lmfit.conf_interval(minimizer, result)
    print ('Time for conf Interval = ', time.time() - to)
    lmfit.printfuncs.report_ci(ci)
    #
    cx, cy, grid = lmfit.conf_interval2d(minimizer, result, 'k1', 'k2', 100, 100)
    plt.contourf(cx, cy, grid, np.linspace(0, 1, 11))
    plt.xlabel('k1')
    plt.colorbar()
    plt.ylabel('k2')

to = time.time()
# MCMC Analysis
if False:
    nsims = 0
    np.set_printoptions(precision=4, linewidth=150)

    print (result.covar)

    from tqdm import tqdm
    to = time.time()
    res = minimizer.emcee(params=params,burn=5000, steps=15000, thin=20,is_weighted=False,progress=True)
    #res = lmfit.minimize(residuals, method='emcee', nan_policy='omit', burn=500, steps=5000, thin=20, progress=True,
    #                         params=params, is_weighted=False)
    import corner
    print('Plot results of emcee...')
    corner.corner(res.flatchain, labels=res.var_names, truths=list(res.params.valuesdict().values()), range=[(0,0.3),(0,1.2), 1])
    plt.savefig ('c:\\tmp\emcee.pdf')
    print ('Time for emcee = ', time.time() - to)
    print ('Number of simulations = ', nsims)


#!/usr/bin/env python
# -*- coding: utf-8 -*-

import unirest
import oauth2
import json

from gdata.analytics.client import AnalyticsClient, DataFeedQuery

APP_NAME = 'GA Dashboard'

class AnalyticsMetrics(object):

    def __init__(self, login, pwd):
        self.client = AnalyticsClient(source=APP_NAME)
        self.client.client_login(login, pwd, source=APP_NAME)

    def get_metrics(self, ids, start_date, end_date):

        query_metrics = ['ga:users',
                         'ga:sessions',
                         'ga:pageviews',
                         'ga:uniquePageviews',
                         'ga:avgSessionDuration',
                         'ga:avgTimeOnPage',
                         'ga:percentNewSessions']

        query = DataFeedQuery({
            'ids': ids,
            'start-date': start_date,
            'end-date': end_date,
            'metrics': ','.join(query_metrics)})

        feed = self.client.GetDataFeed(query)

        metrics = {}

        for metric in query_metrics:
            met = feed.aggregates.get_metric(metric)

            if met:
                metrics[met.name] = met.value

        return metrics

    def get_pageviews(self, ids, start_date, end_date):

        query = DataFeedQuery({
            'ids': ids,
            'start-date': start_date,
            'end-date': end_date,
            'metrics': 'ga:pageviews'})

        feed = self.client.GetDataFeed(query)
        met = feed.aggregates.get_metric('ga:pageviews')

        if met:
            return int(met.value)

        return 0

    def get_top_pages_count(self, ids, start_date, end_date, top_count=5):
        query = DataFeedQuery({
            'ids': ids,
            'start-date': start_date,
            'end-date': end_date,
            'dimensions': 'ga:pagePath,ga:pageTitle',
            'metrics': 'ga:pageviews',
            'sort': '-ga:pageviews',
            'max-results': top_count })

        feed = self.client.GetDataFeed(query)

        metrics = []

        for entry in feed.entry:
            metrics.append({
                'ga:pagePath': entry.get_dimension('ga:pagePath').value,
                'ga:pageTitle': entry.get_dimension('ga:pageTitle').value,
                'ga:pageviews': entry.get_metric('ga:pageviews').value })

        return metrics

class SocialMetrics(object):

    def __init__(self, gplus_key=None, twitter_auth=None):
        self.gplus_key = gplus_key
        self.twitter_auth = twitter_auth or {}

    def get_facebook_likes(self, id):
        res = unirest.get('https://graph.facebook.com/%s' % id)

        if res.code != 200:
            return 0

        return res.body.get('likes', 0)

    def get_google_plus_followers(self, id=None):
        res = unirest.get('https://www.googleapis.com/plus/v1/people/%s?key=%s' % (id, self.gplus_key))

        if res.code != 200:
            return 0

        return res.body.get('plusOneCount', 0) or res.body.get('circledByCount', 0)

    def get_twitter_followers(self, id=None):
        consumer = oauth2.Consumer(key=self.twitter_auth.get('api_key', ''), secret=self.twitter_auth.get('api_secret', ''))
        token = oauth2.Token(key=self.twitter_auth.get('token_key', ''), secret=self.twitter_auth.get('token_secret', ''))
        client = oauth2.Client(consumer, token)
        res, body = client.request('https://api.twitter.com/1.1/users/lookup.json?screen_name=%s' % id, method='GET')

        if res.get('status') == '200':
            r = json.loads(body)[0]
            return r.get('followers_count', 0)
        else:
            return 0

    def get_youtube_subscribers(self, id=None):
        res = unirest.get('http://gdata.youtube.com/feeds/api/users/%s?alt=json' % id)

        if res.code != 200:
            return 0

        return res.body.get('entry', {}).get('yt$statistics', {}).get('subscriberCount', 0)

if __name__ == '__main__':
    from datetime import datetime
    from keys import GA_ID, GA_USER, GA_PWD

    M = AnalyticsMetrics(GA_USER, GA_PWD)
    date = datetime.today().strftime('%Y-%m-%d')
    metrics = M.get_metrics(GA_ID, date, date)
    count = M.get_top_pages_count(GA_ID, date, date, 5)

    print(metrics)
    print(count)


import pandas as pd
import numpy as np

def normalize_data(dataframe, mode):
    if mode == 'abs':
        from sklearn.preprocessing import MaxAbsScaler
        max_abs = MaxAbsScaler(copy=True)  #save for retransform later
        max_abs.fit(dataframe)
        data_norm = max_abs.transform(dataframe)

        return data_norm, max_abs

    if mode == 'robust':
        from sklearn.preprocessing import RobustScaler
        robust = RobustScaler(copy=True)  #save for retransform later
        robust.fit(dataframe)
        data_norm = robust.transform(dataframe)

        return data_norm, robust

    if mode == 'min_max':
        from sklearn.preprocessing import MinMaxScaler
        minmax = MinMaxScaler(feature_range=(0, 1), copy=True)  #save for retransform later
        minmax.fit(dataframe)
        data_norm = minmax.transform(dataframe)

        return data_norm, minmax
    if mode == 'std':
        from sklearn.preprocessing import StandardScaler
        stdscaler = StandardScaler(copy=True, with_mean=True, with_std=True)
        stdscaler.fit(dataframe)
        data_norm = stdscaler.transform(dataframe)

        return data_norm, stdscaler

def extract_data(dataframe, window_size=5, target_timstep=1, cols_x=[], cols_y=[], cols_gt=[],mode='std'):
    dataframe, scaler = normalize_data(dataframe, mode)

    xs = [] # return input data
    ys = [] # return output data
    ygt = [] # return groundtruth data

    if target_timstep != 1:
        for i in range(dataframe.shape[0] - window_size - target_timstep):
            xs.append(dataframe[i:i + window_size, cols_x])
            ys.append(dataframe[i + window_size:i + window_size + target_timstep,
                                cols_y].reshape(target_timstep, len(cols_y)))
            ygt.append(dataframe[i + window_size:i + window_size + target_timstep,
                       cols_gt].reshape(target_timstep, len(cols_gt)))
    else:
        for i in range(dataframe.shape[0] - window_size - target_timstep):
            xs.append(dataframe[i:i + window_size, cols_x])
            ys.append(dataframe[i + window_size:i + window_size + target_timstep, cols_y].reshape(len(cols_y)))
            ygt.append(dataframe[i + window_size:i + window_size + target_timstep, cols_gt].reshape(len(cols_gt)))
    return np.array(xs), np.array(ys), scaler, np.array(ygt)

def ed_extract_data(dataframe, window_size=5, target_timstep=1, cols_x=[], cols_y=[], mode='std'):
    dataframe, scaler = normalize_data(dataframe, mode)

    en_x = []
    de_x = []
    de_y = []

    for i in range(dataframe.shape[0] - window_size - target_timstep):
        en_x.append(dataframe[i:i + window_size, cols_x])

        #decoder input is q and h of 'window-size' days before
        de_x.append(dataframe[i + window_size - 1:i + window_size + target_timstep - 1,
                              cols_y].reshape(target_timstep, len(cols_y)))
        de_y.append(dataframe[i + window_size:i + window_size + target_timstep,
                              cols_y].reshape(target_timstep, len(cols_y)))

    en_x = np.array(en_x)
    de_x = np.array(de_x)
    de_y = np.array(de_y)
    de_x[:, 0, :] = 0

    return en_x, de_x, de_y, scaler

def atted_extract_data(dataframe, window_size=5, cols=[], mode='l2'):  #NOTE: unuse!!!
    dataframe, scaler = normalize_data(dataframe, mode)

    en_x = []
    de_x = []
    de_y = []

    for i in range(dataframe.shape[0] - 2 * window_size - 1):
        en_x.append(dataframe[i:i + window_size, cols])

        #decoder input is q and h of 'window-size' days before
        de_x.append(dataframe[(i + window_size - 1):(i + 2 * window_size - 1), [7, 5]])
        de_y.append(dataframe[(i + window_size):(i + 2 * window_size), [7, 5]])

    en_x = np.array(en_x)
    de_x = np.array(de_x)
    de_y = np.array(de_y)

    de_x[:, 0, :] = 0

    return en_x, de_x, de_y, scaler

def roll_data(dataframe, cols_x, cols_y, mode='min_max'):
    dataframe, scaler = normalize_data(dataframe, mode)
    #dataframe = dataframe.drop('time', axis=1)

    X = dataframe[:, cols_x]
    y = dataframe[:, cols_y]

    return X, y, scaler

def process_evaporation(full_dat, vapor_dat):
    full_dat['vapor'] = 0
    length = len(full_dat)

    full_dat.index = pd.to_datetime(full_dat['time'])
    full_dat = full_dat.drop('time', axis=1)

    vapor_dat.index = pd.to_datetime(vapor_dat['Time'])
    vapor_dat = vapor_dat.drop('Time', axis=1)

    vapor_dat['KonTum'] = vapor_dat['KonTum'] / 12

    print(full_dat.head())
    print(vapor_dat.head())
    print(length)
    #vapor_dat.to_csv('./RawData/KonTum.csv')
    import datetime
    for i in range(length):
        month = full_dat.index[i].month
        year = full_dat.index[i].year
        full_dat.iloc[i, 8] = vapor_dat.loc[datetime.datetime(year, month, 1), 'KonTum']
        if i < 10:
            print(vapor_dat.loc[datetime.datetime(year, month, 1), 'KonTum'])

    print(full_dat.head())
    full_dat.to_csv('./Kontum-daily.csv')

def plot_compare_model():
    dat = pd.read_csv('./Log/DataAnalysis/predict_val.csv', header=0)
    # dat_arima = pd.read_csv('./Log/Arima/arima.csv', header=0, index_col=0)

    # dat['arima_q'] = dat_arima['arima_q_hanoi']
    # dat['arima_h'] = dat_arima['arima_h_hanoi']

    # dat.to_csv('./Log/DataAnalysis/predict_val.csv', index=None)
    import matplotlib.pyplot as plt

    plt.plot(dat['real_q'], label='ground_truth')
    plt.plot(dat['ensemble_q'], label='ensemble')
    plt.plot(dat['rnn_cnn_q'], label='rnn_cnn')
    plt.plot(dat['en_de_q'], label='encoder_decoder')
    plt.legend(loc='best')
    plt.xlabel('Time')
    plt.ylabel('Q')
    plt.title('Các mô hình đề xuất')
    plt.savefig('./Log/DataAnalysis/our_model_q.png')
    plt.clf()

    plt.plot(dat['real_h'], label='ground_truth')
    plt.plot(dat['ensemble_h'], label='ensemble')
    plt.plot(dat['rnn_cnn_h'], label='rnn_cnn')
    plt.plot(dat['en_de_h'], label='encoder_decoder')
    plt.legend(loc='best')
    plt.xlabel('Time')
    plt.ylabel('H')
    plt.title('Các mô hình đề xuất')

    plt.savefig('./Log/DataAnalysis/our_model_h.png')
    plt.clf()

    plt.plot(dat['lstm_q'], label='lstm')
    plt.plot(dat['ann_q'], label='ann')
    plt.plot(dat['arima_q'], label='arima')
    plt.plot(dat['real_q'], label='ground_truth')

    plt.legend(loc='best')
    plt.xlabel('Time')
    plt.ylabel('Q')
    plt.title('Các mô hình hiện tại')
    plt.savefig('./Log/DataAnalysis/paper_model_q.png')
    plt.clf()

    plt.plot(dat['lstm_h'], label='lstm')
    plt.plot(dat['ann_h'], label='ann')
    plt.plot(dat['arima_h'], label='arima')
    plt.plot(dat['real_h'], label='ground_truth')

    plt.legend(loc='best')
    plt.xlabel('Time')
    plt.ylabel('H')
    plt.title('Các mô hình hiện tại')
    plt.savefig('./Log/DataAnalysis/paper_model_h.png')
    plt.clf()

    plt.plot(dat['real_q'], label='ground_truth')
    plt.plot(dat['ensemble_q'], label='ensemble')
    plt.plot(dat['rnn_cnn_q'], label='rnn_cnn')
    plt.plot(dat['en_de_q'], label='encoder_decoder')
    plt.plot(dat['lstm_q'], label='lstm')
    plt.plot(dat['ann_q'], label='ann')
    plt.plot(dat['arima_q'], label='arima')
    plt.legend(loc='best')
    plt.xlabel('Time')
    plt.ylabel('Q')
    plt.title('Mọi mô hình')
    plt.savefig('./Log/DataAnalysis/compare_q.png')
    plt.clf()

    plt.plot(dat['real_h'], label='ground_truth')
    plt.plot(dat['ensemble_h'], label='ensemble')
    plt.plot(dat['rnn_cnn_h'], label='rnn_cnn')
    plt.plot(dat['en_de_h'], label='encoder_decoder')
    plt.plot(dat['lstm_h'], label='lstm')
    plt.plot(dat['ann_h'], label='ann')
    plt.plot(dat['arima_h'], label='arima')
    plt.legend(loc='best')
    plt.xlabel('Time')
    plt.ylabel('H')
    plt.title('Mọi mô hình')
    plt.savefig('./Log/DataAnalysis/compare_H.png')
    plt.clf()

def plot_PM():
    import matplotlib.pyplot as plt

    gt = pd.read_csv('./RawData/PM/groundtruth.csv', header=None)
    pre = pd.read_csv('./RawData/PM/preds.csv', header=None)

    plt.plot(gt[1], label='ground_truth')
    plt.plot(pre[1], label='predict')

    plt.legend(loc='best')
    plt.xlabel('Thời gian')
    plt.ylabel('Lượng mưa')
    plt.title('Kết quả mô hình')
    plt.savefig('./Log/DataAnalysis/compare_pm.png')

if __name__ == '__main__':
    # full_dat = pd.read_csv('./RawData/Kontum-daily.csv', header=0, index_col=0)
    # vapor_dat = pd.read_csv('./RawData/KonTum.csv', header=0, index_col=None)

    # process_evaporation(full_dat, vapor_dat)

    # data = pd.read_csv('./RawData/Hanoi/Merge_HN.csv', header=0, index_col=0)
    # data = data.set_index('date')
    # data = data.drop(['date.1', 'date.2'], axis=1)
    # print(data.head())
    # data.to_csv('./RawData/Hanoi/Merge_HN.csv')
    # data = data.to_numpy()
    # print(data.shape)
    plot_PM()


'''
Created on June 3, 2016

    This program will report the status of the current languages supported for QuantaStor. They are statically
    defined in the program. Because of that if additional languages are supported the program will have to be
    altered. In addition to the status report a translator's file will be generated which can be sent to the
    translators. The translation files will have  names of the form, NotYetTranslated_FR.txt in the case of French.
    It is in UTF-8 and contains a list of token - text pairs.

    Finally, a file called Msgs_FR.tmp also in UTF-8 format will be created to take the place of the old Msgs_FR.properties file.
    Any untranslated tokens in this file will be listed left as English.

    The program was written in Python 3.5.

@author: Charles
Property of OsNexus

'''
import os
from os.path import isfile
import datetime

from builtins import input

def main():
    foreignFileNames=[]
    foreignFiles=[]
    msgsPath=[]
    msgTokenText=[]

    '''        As more languages are supported enter their language tags below in foriegnTags[].

        Language_option = [["Arabic","Msgs_ar.properties"], ["Bulgarian","Msgs_bg.properties"], ["Chinese","Msgs_zh-CN.properties"], ["Croatian","Msgs_hr.properties"], ["Czech","Msgs_cs.properties"],
     ["Danish","Msgs_da.properties"], ["Dutch","Msgs_nl.properties"], ["Finnish","Msgs_fi.properties"], ["French","Msgs_fr.properties"], ["German","Msgs_de.properties"],
     ["Greek","Msgs_el.properties"], ["Hindi","Msgs_hi.properties"], ["Italian","Msgs_it.properties"], ["Japanese","Msgs_ja.properties"], ["Korean","Msgs_ko.properties"], ["Norwegian","Msgs_no.properties"],
     ["Polish","Msgs_pl.properties"], ["Portugese","Msgs_pt.properties"],["Romanian","Msgs_ro.properties"], ["Russian","Msgs_ru.properties"], ["Spanish","Msgs_es.properties"], ["Swedish","Msgs_sv.properties"]]
    '''
    foriegnTags = ["es", "fr", "ja", "de", "it"]
    defaultStatusPath = "C:\\Sandbox"
    defaultMsgsPath = "C:\\Sandbox\\quantastor\\web\\workspace\\QuantaStor\\src\\com\\osnexus\\quantastor\\client\\resources"

    #    build the foreign files list
    for tag in foriegnTags:
        foreignFileNames.append("Msgs_"  + tag)

    print("\nThis program determines the current translation status.\n\n\
    You will be prompted for the paths for the foreign message files and the location\n\
     for the status report and the tokens to translate for each language supported.")

    #    Establish the location of the Msgs_xx.properties files
    loop = True
    while loop:
        print("\n\nThe default path for Msgs_xx.properties is " + defaultMsgsPath)
        msgsPath = input("Please enter the path for the foreign message files:")

        if msgsPath:
            if msgsPath.endswith('\\'):
                msgsPath = msgsPath[:len(msgsPath) - 1]
        else:
            msgsPath = defaultMsgsPath

        if os.path.isdir(msgsPath):
            for fileName in foreignFileNames:
                if not isfile(msgsPath + "\\" + fileName  + ".properties"):
                    print("\n\tNot all the foreign message files could be found at " + msgsPath + \
                         ".\n\tSpecifically the file " + fileName + " is not present.")
                    break
        else:
            print("The path you offered does not exist.\nPlease try again.")
            break

        loop = False

    #    create the list of foreign path and file names with paths and extension
    for aName in foreignFileNames:
        foreignFiles.append(msgsPath + "\\" + aName  + ".properties")

    #    Establish intended location for the Foreign Msgs Status.txt file
    loop = True
    while loop:
        print("\nThe default path for the file, Status Information.txt, is " + defaultStatusPath)
        statusPath = input("Please enter the path for the status report file:")

        if statusPath:
            if statusPath.endswith('\\'):
                statusPath = statusPath[:len(statusPath) - 1]
        else:
            statusPath = defaultStatusPath

        if os.path.isdir(statusPath):
            loop = False
        else:
            print("\n\tThe path offered does not exist, please try again.")

    #    Gather the token string pairs for comparison with foreign text
    msgsFp = open (msgsPath+"\\Msgs.properties", "r", encoding="utf-8")
    firstLine = True
    for line in msgsFp:
        #  had to do this because UTF-8 has a special character at beginning of the file
        if firstLine:
            if line.find("=") > -1 and line.find("#") == -1:
                msgTokenText.append(line)

            firstLine = False
        else:
            if not line.startswith("#"):
                if line.find("=") > 0:
                    msgTokenText.append(line)

    msgsFp.close

    #    Begin logging to the status file
    statusFile = open(statusPath + "\\Status Information.txt", "w+")
    statusFile.write("The foreign files status for "  + '{0:%Y-%m-%d %H:%M:%S}'.format(datetime.datetime.now()) + " is...\n\n")

    #    Begin reviewing each foreign file and compare token and strings with the English
    #    for file in foreginFiledata.values():
    index = -1
    for aFileName in foreignFiles:
        translationCount = 0
        firstLine = True
        index += 1

        print("\n\tNow working on " + aFileName)
        #We will remove list items till all that remains are new token string pairs that are not yet in foreign Msgs
        newTokensFromMsgs = list(msgTokenText)

        foreignFp = open(foreignFiles[index], 'r', encoding="utf-8")
        translatorsFp = open(statusPath + "\\" + foreignFileNames[index] + " to Translator.txt", 'w+', encoding="utf-8")

        for msgsLine in foreignFp:
            #  had to do this because UTF-8 has a special character at beginning of the file
            if firstLine:
                firstLine = False
            else:
                equalIndex = msgsLine.find("=")
                commentIndex = msgsLine.find("#")
                if msgsLine.startswith("#") and equalIndex > -1:
                    item = ""

                    token = msgsLine[commentIndex + 1:equalIndex + 1]
                    tokenText = msgsLine[equalIndex + 1:]

                    tokensIndex = 0
                    #    Search the list of tokens from Msgs.properties
                    for item in newTokensFromMsgs:
                        if item.find(token) > -1:
                            break
                        else:
                            item = ""
                            tokensIndex += 1

                    #    this is an orphan token and should removed from translation file
                    if item == "":
                        translatorsFp.write("***This is an orphan token and should be removed from the file " + foreignFileNames[index] + "\n\t" + token + tokenText + "\n\n")
                    else:
                        englishText = item[item.find("=") + 1:]
                        if not englishText == tokenText:
                            translationCount += 1
                            translatorsFp.write(msgsLine + token + "\n\n")

                        del newTokensFromMsgs[tokensIndex]

        #    These are new token string pairs for translator
        for item in newTokensFromMsgs:
            translationCount += 1
            translatorsFp.write("#" + item + item[:item.find("=")+1] +"\n\n")

        translatorsFp.close()
        foreignFp.close()

        if translationCount > 0:
            statusFile.write( aFileName + " has " + str(translationCount) + " tokens to be translated.\n")

    statusFile.write("\n\nThe total number of tokens in Msgs.properties is  "  + str(len(msgTokenText)))
    statusFile.close()

    print("\nThats all folks...\n")
    return

if __name__ == '__main__':
    main()
    pass


"""
    1. 研究如何写才能真正实现协程并发。（如何正确使用asyncio）
    2. Based on python 3.7.4.
"""

import asyncio
import time

def show_perf(func):
    print('*' * 20)
    start = time.perf_counter()
    asyncio.run(func())
    end = time.perf_counter()
    print(f'{func.__name__} Cost: {end - start}')

async def a():
    print('Suspending a')
    await asyncio.sleep(3)
    print('Resuming a')

async def b():
    print('Suspending b')
    await asyncio.sleep(1)
    print('Resuming b')

async def main_1():
    # 这样写不能并发，变成串行了。4秒
    await a()
    await b()

async def main_2():
    # 这样写可以。3秒。
    task_1 = asyncio.create_task(a())
    task_2 = asyncio.create_task(b())
    await task_1
    await task_2

async def main_3():
    # 这样写可以。3秒
    await asyncio.gather(a(), b())

async def main_4():
    # 这样写可以。3秒
    task_1 = asyncio.create_task(a())
    await b()
    await task_1

async def main_5():
    # 这样写不行。4秒
    task_1 = asyncio.create_task(a())
    await task_1
    await b()

async def main_6():
    # 这样写不行。4秒
    await asyncio.create_task(a())
    await asyncio.create_task(b())

# show_perf(main_1)
# show_perf(main_2)
# show_perf(main_3)
# show_perf(main_4)
# show_perf(main_5)
show_perf(main_6)


import numpy as np
import yaml
from utils.image_utils import cropImageToAnnoRegion

def pyroidbTransform_cropImageToBox(inputs,**kwargs):
    im_orig = inputs[0]
    box = inputs[1]
    clean_box(box,kwargs['sample']['width'],kwargs['sample']['height'])
    return cropImageToAnnoRegion(im_orig,box)

def pyroidbTransform_normalizeBox(inputs,**kwargs):
    sample = kwargs['sample']
    annoIndex = kwargs['annoIndex']
    if checkNormalizeSample(sample,annoIndex):
        initNormalizeSample(sample)
        inputs = inputs.astype(np.float64)
        inputs = clean_box(inputs,sample['width'],sample['height'])
        inputs[::2] /= sample['width']
        inputs[1::2] /= sample['height']
        if np.any(inputs > 1):
            print("inputs greater than 1")
        assert np.all(inputs <= 1)
        assert np.all(inputs >= 0)
        updateNormalizeSample(sample,annoIndex)
    return inputs

def printPyroidbSetCounts(pyroidb,rootDir):
    import yaml
    fn = "lib/datasets/ymlConfigs/default_dataset_index.yml"
    fn = osp.join(cfg.ROOT_DIR,fn)
    with open(fn,"r") as f:
        setIds = yaml.load(f)

    idsToSet = ["" for _ in range(8)]
    for val,idx in setIds.items():
        idsToSet[idx] = val
    sizes = dict.fromkeys(idsToSet, 0)
    for elem,target in pyroidb:
        sizes[idsToSet[target]] += 1
    pp.pprint(sizes)

def checkNormalizeSample(sample,annoIndex):
    return ("bbox_noramlized?" in sample.keys() and sample["bbox_noramlized?"][annoIndex] is False) or ("bbox_noramlized?" not in sample.keys())

def initNormalizeSample(sample):
    if "bbox_noramlized?" not in sample.keys():
        sample["bbox_noramlized?"] = [False for _ in range(len(sample['boxes']))]

def updateNormalizeSample(sample,annoIndex):
    sample["bbox_noramlized?"][annoIndex] = True


import numpy as np
import time
import matplotlib.pyplot as plt

pmt=18
path='/home/gerak/Desktop/DireXeno/301120B/PulserA/PMT{}/'.format(pmt)
BL=np.load(path+'BL.npz')['BL']
x=np.arange(1000)

l=40
ly=-0.01
r=605
ry=1.14

bl=np.array(BL)
bl[l]=ly
bl[r]=ry

a=(ly-ry)/(l-r)
b=ry-a*r
bl[l:r]=a*x[l:r]+b

plt.plot(x, BL, 'k.')
plt.plot(x, bl, 'r.')
np.savez(path+'BL'.format(pmt), BL=bl)

plt.show()


# https://school.programmers.co.kr/learn/courses/30/lessons/131705

#1 모든 3개로 구성된 쌍을 만든다.
#2 합이 0이면 answer += 1

# 풀이1
def solution1(number):
    answer = 0
    for n1 in range(0, len(number) - 2):
        for n2 in range(n1 + 1, len(number) - 1):
            for n3 in range(n2 + 1, len(number)):
                if number[n1] + number[n2] + number[n3] == 0:
                    answer += 1
    return answer

# 풀이2
from itertools import combinations

def solution(number):
    set_list = combinations(number, 3)
    return list(map(lambda x: sum(x), set_list)).count(0)

print(solution([-2, 3, 0, 2, -5]))
print(solution([-3, -2, -1, 0, 1, 2, 3]))


import time
import copy
import logging
import datetime
import argparse

from utils import *
from data_loader import *
from paths import *
from sklearn.decomposition import TruncatedSVD

if __name__ == '__main__':

    parser = argparse.ArgumentParser()
    parser.add_argument('--num_pc', '-k', type=int, default=2)
    parser.add_argument('--num_participants', '-p', type=int, default=10)
    parser.add_argument('--num_samples', '-s', type=int, default=1000)
    parser.add_argument('--dataset', '-d', type=str, default='load_synthetic')
    parser.add_argument('--block', '-b', type=int, default=10)
    parser.add_argument('--num_feature', '-f', type=int, default=1000)
    parser.add_argument('--only_time', '-t', type=str, default='False')
    parser.add_argument('--output_pkl', '-o', type=str, default='False')
    parser.add_argument('--log_dir', '-l', type=str, default='')
    args = parser.parse_args()

    # Parameters
    num_participants = args.num_participants
    num_samples = args.num_samples  # per participant

    only_evaluate_time = True if args.only_time == 'True' else False
    save_pickle = True if args.output_pkl == 'True' else False

    # Generate data using Gaussian distribution
    X = np.random.randn(args.num_feature, args.num_participants*args.num_samples)
    label = None

    print('PCA mode, subtracting the mean')
    X = X.T
    X -= np.mean(X, axis=0)
    X = X.T

    # Split the data for participants
    Xs = [X[:, e * num_samples: e * num_samples + num_samples] for e in range(num_participants)]
    if label is not None:
        Ys = [label[e * num_samples: e * num_samples + num_samples] for e in range(num_participants)]
    else:
        Ys = None

    # Init the efficiency file
    log_file_name = os.path.join(log_dir, args.log_dir,
                                 'FedSVD_' + datetime.datetime.now().strftime('%Y%m%d%H%M%S') + '.log')
    logging.basicConfig(level=logging.INFO, format='%(asctime)s-%(levelname)s: %(message)s',
                        filename=log_file_name)
    logging.info(str(args))
    print(str(args))

    ground_truth = np.concatenate(Xs, axis=1)
    m, n = ground_truth.shape

    mape_denominator = copy.deepcopy(ground_truth)
    mape_denominator[np.where(mape_denominator == 0)] = 1e-10

    # Standalone SVD
    start = time.time()
    truncated_svd = TruncatedSVD(n_components=args.num_pc, algorithm='arpack')
    truncated_svd.fit(ground_truth.T)
    normal_components = truncated_svd.components_
    normal_explained_var = truncated_svd.explained_variance_
    normal_explained_var_ratio = truncated_svd.explained_variance_ratio_
    time_standalone_svd = time.time() - start
    logging.info('StandalonePCA time %s' % time_standalone_svd)
    print('StandalonePCA time %s' % time_standalone_svd)
    logging.info('StandalonePCA explained var ratio %s ' % normal_explained_var_ratio)
    print('StandalonePCA explained var ratio %s ' % normal_explained_var_ratio)

    # FedSVD, Start the simulation
    logging.info('Simulation Start!')

    comm_each_data_holder = 0

    # Masking Server: Generate random orthogonal matrix P and Q
    start = time.time()
    P = generate_orthogonal_matrix(n=X.shape[0], reuse=False, block_reduce=args.block)
    t1 = time.time()
    Q = generate_orthogonal_matrix(n=np.sum([e.shape[1] for e in Xs]), reuse=False, block_reduce=args.block)
    t2 = time.time()
    Qs = [Q[e * num_samples: e * num_samples + num_samples] for e in range(num_participants)]
    time_generate_orthogonal = t2 - start
    if not only_evaluate_time:
        comm_each_data_holder += get_object_size(P)
        comm_each_data_holder += (get_object_size(Qs) / num_participants)
    logging.info('Generate orthogonal matrix %s done. Using %s seconds.' % (P.shape[0], t1-start))
    logging.info('Generate orthogonal matrix %s done. Using %s seconds.' % (Q.shape[0], t2-start))
    print('Generate orthogonal matrix done. Using %s seconds.' % time_generate_orthogonal)

    # Data Holders & Factorization Server: SecureAggregation to get X'
    start = time.time()
    X_mask_partitions = []
    for i in range(num_participants):
        X_mask_partitions.append(P @ Xs[i] @ Qs[i])
    X_mask, comm_size = secure_aggregation(X_mask_partitions, only_evaluate_time)
    # The time consumption of applying random mask runs in parallel by all the participants
    time_apply_orthogonal = (time.time() - start) / num_participants
    comm_each_data_holder += comm_size
    logging.info('Apply distortion done. Using %s seconds.' % time_apply_orthogonal)
    print('Apply distortion done. Using %s seconds.' % time_apply_orthogonal)

    # Decrypt the distorted_X, and perform the SVD decomposition
    start = time.time()
    truncated_fed_svd = TruncatedSVD(n_components=args.num_pc, algorithm='arpack')
    truncated_fed_svd.fit(X_mask.T)
    fed_components_mask = truncated_fed_svd.components_
    fed_explained_var = truncated_fed_svd.explained_variance_
    fed_explained_var_ratio = truncated_fed_svd.explained_variance_ratio_
    time_svd = time.time() - start
    logging.info('SVD done. Using %s seconds.' % time_svd)
    print('SVD done. Using %s seconds.' % time_svd)
    logging.info('Truncated FedSVD explained var ratio %s ' % fed_explained_var_ratio)
    print('Truncated FedSVD explained var ratio %s ' % fed_explained_var_ratio)

    # Recover the real singular values and vectors
    start = time.time()
    fed_components = (P.T @ fed_components_mask.T).T

    # Evaluation (FedSVD): reconstruct to measure the precision
    mape_to_normal = np.mean(np.abs((np.abs(normal_components) - np.abs(fed_components)) / np.abs(normal_components)))
    logging.info('MAPE to normal PCA %s ' % mape_to_normal)
    print('MAPE to normal PCA %s ' % mape_to_normal)

    # End Simulation
    logging.info('Finished!')
    # Collect the time consumption
    time_consumption = [time_generate_orthogonal, time_apply_orthogonal, time_svd]
    logging.info('Truncated FedSVD totally uses %s seconds' % np.sum(time_consumption))
    print('Truncated FedSVD totally uses %s seconds' % np.sum(time_consumption))

    if save_pickle:
        # Save the results to pickle file
        result = {
            'dataset': args.dataset,
            'num_participants': args.num_participants,
            'num_samples': args.num_samples,
            'Xs': Xs,
            'Ys': Ys,
            'block_reduce': args.block,
            'fed_svd': {
                'var': fed_explained_var,
                'var_ratio': fed_explained_var_ratio,
                'pcs': fed_components
            },
            'standalone_svd': {
                'var': normal_explained_var,
                'var_ratio': normal_explained_var_ratio,
                'pcs': normal_components
            },
            'time_consumption': time_consumption,
            'comm_size': comm_each_data_holder,
            'mape_to_normal': mape_to_normal,
            'log': log_file_name
        }

        with open(os.path.join(results_dir, save_file_name + '.pkl'), 'wb') as f:
            pickle.dump(result, f)


from django.contrib import admin
from django.urls import path,include,re_path
from . import views,models
from django.conf.urls import url

#from . import views, settings
#from django.contrib.staticfiles.urls import static
#from django.contrib.staticfiles.urls import staticfiles_urlpatterns

from django.conf import settings
from django.conf.urls.static import static
#from users import views as user_views
#from ms.views import template_list as a
app_name='main'

urlpatterns=[
    path('admin/', admin.site.urls),
    path('',views.login),
    #url(r'repeat/(?P<id>\d+)/',views.repeat,name='repeat')
    path('pscores/', views.pscores, name='pscores'),
    path('logical/', views.object_list, name='logical'),
    path('quantitative/', views.quantitative, name='quantitative'),
    path('spatial/', views.spatial, name='spatial'),
    path('login/', views.login, name='login'),
    path('register/', views.register, name='register'),
    url('logout/', views.logout, name='logout'),
    path('verbal/',views.verbal,name='verbal'),
    path('tts/', views.tts, name='tts'),
    path('tts_s/', views.tts_s, name='tts_s'),
    path('tts1/', views.tts1, name='tts1'),
    path('cam/', views.cam, name='cam'),
    path('tts_repeat/', views.tts_repeat, name='tts_repeat'),
    path('home/', views.home, name='home'),
    path('instructions/', views.instructions, name='instructions'),
    path('register/', views.register, name='register'),
    path('sec1ins/', views.sec1ins, name='sec1ins'),
    path('sec3ins/', views.sec3ins, name='sec3ins'),
    path('sec1sub/', views.sec1sub, name='sec1sub'),
    path('sec3sub/', views.sec3sub, name='sec3sub'),
    path('sec2ins/', views.sec2ins, name='sec2ins'),
    path('sec4ins/', views.sec4ins, name='sec4ins'),
    path('sec2sub/', views.sec2sub, name='sec2sub'),
    path('sec4sub/', views.sec4sub, name='sec4sub'),
    path('result/', views.result, name='result'),

    ]
urlpatterns+= static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT)


from django.contrib import admin
from django.urls import path, include
from .import views

urlpatterns = [
    path('', views.home, name='home'),
    path('practitioners', views.practitioners, name='practitioners'),
    path('visitors', views.visitors, name='visitors'),
    path('signup', views.handleSignup, name='handleSignup'),
    path('login', views.handlelogin, name='handlelogin'),
    path('logout', views.handlelogout, name='handlelogout')
]


#!/usr/bin/env python

"""
    models.py
"""

from __future__ import division
from __future__ import print_function

from functools import partial

import torch
from torch import nn
from torch.nn import functional as F

from lr import LRSchedule

from nn_modules import GraphConvolution,IdEdgeAggregator

# --
# Model

class HINGCN_GS(nn.Module):
    def __init__(self,
                 n_mp,
                 problem,
                 prep_len,
                 n_head,
                 layer_specs,
                 aggregator_class,
                 mpaggr_class,
                 edgeupt_class,
                 prep_class,
                 sampler_class,
                 dropout,
                 batchnorm,
                 attn_dropout=0,
                 ):

        super(HINGCN_GS, self).__init__()

        # --
        # Input Data
        self.edge_dim = problem.edge_dim
        self.input_dim = problem.feats_dim
        self.n_nodes = problem.n_nodes
        self.n_classes = problem.n_classes
        self.n_head = n_head

        # self.feats
        self.register_buffer('feats', problem.feats)

        # self.edge_emb_mp
        for i, key in enumerate(problem.edge_emb):
            self.register_buffer('edge_emb_{}'.format(i),
                                 problem.edge_emb[key])
        # self.adjs_mp
        for i, key in enumerate(problem.adj):
            self.register_buffer('adjs_{}'.format(i), problem.adj[key])

        # Define network
        self.n_mp = n_mp
        self.depth = len(layer_specs)
        self.dropout = dropout
        self.attn_dropout = attn_dropout
        self.batchnorm = batchnorm

        # Sampler
        self.train_sampler = sampler_class()
        self.val_sampler = sampler_class()
        self.train_sample_fns = [partial(
            self.train_sampler, n_samples=s['n_train_samples']) for s in layer_specs]
        self.val_sample_fns = [
            partial(self.val_sampler, n_samples=s['n_val_samples']) for s in layer_specs]

        # Prep
        self.prep = prep_class(input_dim=problem.feats_dim, n_nodes=problem.n_nodes,
                               embedding_dim=prep_len
                               # output_dim=prep_len
                               )
        self.input_dim = self.prep.output_dim

        # Network
        for mp in range(self.n_mp):
            # agg_layers = []
            # edge_layers = []
            input_dim = self.input_dim
            out_dim = 0
            for i, spec in enumerate(layer_specs):
                if i == 0:
                    edge = IdEdgeAggregator(
                        input_dim=input_dim,
                        edge_dim=self.edge_dim,
                        activation=spec['activation'],
                        dropout=self.dropout,
                        batchnorm=self.batchnorm,
                    )
                else:
                    edge = edgeupt_class(
                        input_dim=input_dim,
                        edge_dim=self.edge_dim,
                        activation=spec['activation'],
                        dropout=self.dropout,
                        batchnorm=self.batchnorm,
                    )
                agg = nn.ModuleList([aggregator_class(
                    input_dim=input_dim,
                    edge_dim=problem.edge_dim,
                    output_dim=spec['output_dim'],
                    activation=spec['activation'],
                    concat_node=spec['concat_node'],
                    concat_edge=spec['concat_edge'],
                    dropout=self.dropout,
                    attn_dropout=self.attn_dropout,
                    batchnorm=self.batchnorm,
                ) for _ in range(n_head)])
                # agg_layers.append(agg)
                # May not be the same as spec['output_dim']
                input_dim = agg[0].output_dim * n_head
                out_dim += input_dim

                # edge_layers.append(edge)
                self.add_module('agg_{}_{}'.format(mp, i), agg)
                self.add_module('edge_{}_{}'.format(mp, i), edge)
        input_dim = out_dim

        self.mp_agg = mpaggr_class(
            input_dim, n_head=self.n_mp+int(self.bias), dropout=self.dropout, batchnorm=self.batchnorm,)

        self.fc = nn.Sequential(*[
            nn.Linear(self.mp_agg.output_dim, 32, bias=True),
            nn.ReLU(), nn.Dropout(self.dropout),
            nn.Linear(32, problem.n_classes, bias=True),
        ])

    # Forward IDs only to facilitate nn.DataParallelism
    def forward(self, ids, train=True):

        # print("\tIn Model: input size ", ids.shape)
        # ids.to(self.feats.device)

        # Sample neighbors
        sample_fns = self.train_sample_fns if train else self.val_sample_fns

        has_feats = self.feats is not None

        output = []
        tmp_ids = ids
        for mp in range(self.n_mp):
            ids = tmp_ids
            tmp_feats = self.feats[ids] if has_feats else None
            #tmp_feats = torch.nn.functional.dropout( self.prep(ids, tmp_feats, layer_idx=0),0.2,training=train)
            tmp_feats = self.prep(ids, tmp_feats, layer_idx=0)
            all_feats = [tmp_feats]
            all_edges = []
            all_masks = []
            for layer_idx, sampler_fn in enumerate(sample_fns):
                neigh, edges, mask = sampler_fn(
                    adj=getattr(self, 'adjs_{}'.format(mp)), ids=ids)

                all_edges.append(getattr(self, 'edge_emb_{}'.format(mp))[
                                 edges.contiguous().view(-1)])
                all_masks.append(mask)

                ids = neigh.contiguous().view(-1)
                tmp_feats = self.feats[ids] if has_feats else None
                all_feats.append(
                    self.prep(ids, tmp_feats, layer_idx=layer_idx + 1))

            # Sequentially apply layers, per original (little weird, IMO)
            # Each iteration reduces length of array by one
            tmp_out = []
            for i in range(self.depth):
                all_edges = [getattr(self, 'edge_{}_{}'.format(mp, i))(all_feats[k], all_feats[k + 1],
                                                                       all_edges[k], mask=all_masks[k]
                                                                       ) for k in range(len(all_edges))]
                all_edges = [
                    F.dropout(i, self.dropout, training=self.training) for i in all_edges]

                all_feats = [torch.cat([getattr(self, 'agg_{}_{}'.format(mp, i))[h](all_feats[k], all_feats[k + 1],
                                                                                all_edges[k], mask=all_masks[k]) for h in range(self.n_head)], dim=1)
                             for k in range(len(all_feats) - 1)]
                all_feats = [
                    F.dropout(i, self.dropout, training=self.training) for i in all_feats]

                tmp_out.append(all_feats[0])
            assert len(all_feats) == 1, "len(all_feats) != 1"
            output.append(torch.cat(tmp_out, dim=-1).unsqueeze(0))

        output = torch.cat(output)

        # output = F.normalize(output, dim=2) #normalize before attention

        output, weights = self.mp_agg(output)
        # print(weights)
        # output = F.normalize(output, dim=1)  # ?? Do we actually want this? ... Sometimes ...
        output = F.dropout(output, self.dropout, training=self.training)
        output = (self.fc(output))
        return output, weights

class MyDataParallel(nn.DataParallel):
    def __getattr__(self, name):
        try:
            return super().__getattr__(name)
        except AttributeError:
            return getattr(self.module, name)


import argparse
import urllib3
import json
import certifi
import re
import lxml.html
from lxml import etree
import xml.etree.ElementTree as etree
from xml.dom.minidom import parse
import xml.dom.minidom

Rxcui = '88014'
NUI = 'N0000152900'
ndc_prop = '0069-3150-83'
drug_name = 'cymbalta'
##NDFRT
id_type = 'RXCUI'
id_string = '161'
concept_name = 'morphine'
kind_name = 'DRUG_KIND'
concept_kind = 'pharmacokinetics_kind'
nui_all_info = 'N0000152900'
pharmacokinetics_kind = 'DRUG_KIND'
rxcui = '1234567890'

RxNav_base_url = 'https://rxnav.nlm.nih.gov/REST/'
FDA_base_url = 'https://api.fda.gov/drug/'
PUBMED_base_url = 'https://eutils.ncbi.nlm.nih.gov/entrez/eutils/'
term = 'asthma[mesh]+AND+leukotrienes[mesh]+AND+20089[pdat]'
PUBMED_query = PUBMED_base_url + 'esearch.fcgi?db=pubmed&term=' + term + 'usehistory=y'

FDA_Adverse_Event =FDA_base_url +  'event.json?'
FDA_enforcement = FDA_base_url +'enforcement.json?'

adverse_search = FDA_Adverse_Event + 'search=patient.drug.openfda.pharm_class_epc:\"nonsteroidal+anti-inflammatory+drug\"&'
adverse_result_count = 'count=patient.reaction.reactionmeddrapt.exact'

RX_NORM_INFO = RxNav_base_url + '/RxTerms/'
RX_NORM_Drug_Interactions = RxNav_base_url + '/interaction/interaction.json?rxcui=' + Rxcui + '&sources=ONCHigh'
RX_NORM_AllProperties = RxNav_base_url + '/rxcui/' + rxcui + '/allProperties.json?prop=all'
## search=field:term

## DISEASE_KIND, DOSE_FORM_KIND, DRUG_KIND, INGREDIENT_KIND, MECHANISM_OF_ACTION_KIND
## PHARMACOKINETICS_KIND, PHYSIOLOGIC_EFFECT_KIND, THERAPEUTIC_CATEGORY_KIND

NDFRT_concept_by_id = RxNav_base_url + '/Ndfrt/id?idType='+ id_type + '&idString=' + id_string
NDFRT_search = RxNav_base_url + '/Ndfrt/search?conceptName=' + concept_name + '&kindName=' + kind_name
NDFRT_get_all_concepts = RxNav_base_url + '/Ndfrt/allconcepts?kind=' + pharmacokinetics_kind
NDFRT_all_info = RxNav_base_url + '/Ndfrt/allInfo.json?nui=' + nui_all_info
NDFRT_get_child_concepts = RxNav_base_url + '/Ndfrt/childConcepts?nui=' + nui_all_info + '&transitive=true'

DAILY_MED_DOWNLOADS = 'https://dailymed.nlm.nih.gov/dailymed/spl-resources-all-drug-labels.cfm'

class DrugAPI:
	def  __init__(self,type):
		parser = argparse.ArgumentParser(description='HTTP Client Example')
		if type == 'rxnorm':
			parser.add_argument('--host', action="store",dest="host",  default=RxNav_base_url)
		elif type == 'spl':
			parser.add_argument('--host', action="store",dest="host",  default=FDA_base_url)
		elif type == 'pubmed':
			parser.add_argument('--host', action="store",dest="host",  default=PUBMED_base_url)

		given_args = parser.parse_args()
		self.host = given_args.host
		self.http = urllib3.PoolManager(cert_reqs = 'CERT_REQUIRED', ca_certs=certifi.where())

	def run_test(self):
		drugInteractions = getDrugInteractions(Rxcui)
		ndfrt_allinfo = get_NDFRT_allinfo()
		getAll_NDC_Properties = getAll_NDC_Properties(ndc_prop)
		get_drugs = getDrugInfo(drug_name)

	def do_query(http,query_type,params):

		http_string = ''
		if query_type == 'getAllRxNormInfo':
			http_string = getAllRxNormInfo(params)
		response = self.http.request('GET',http_string)
		drug_data = response.data.decode('utf-8')

	def getAllRxNormInfo(rxcuid):
		## returns
		##<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
		##<rxtermsdata>
		##<rxtermsProperties>
		##       <brandName></brandName>
		##       <displayName>Ondansetron (Oral Disintegrating)</displayName>
		##        <synonym></synonym>
		##        <fullName>Ondansetron 4 MG Disintegrating Oral Tablet</fullName>
		##        <fullGenericName>Ondansetron 4 MG Disintegrating Oral Tablet</fullGenericName>
		##         <strength>4 mg</strength>
		##         <rxtermsDoseForm>Tab</rxtermsDoseForm>
		##         <route>Oral Disintegrating</route>
		##         <termType>SCD</termType>
		##          <rxcui>104894</rxcui>
		##          <genericRxcui>0</genericRxcui>
		##           <rxnormDoseForm>Disintegrating Oral Tablet</rxnormDoseForm>
		##          <suppress></suppress></rxtermsProperties></rxtermsdata>

		get_rx_norm_info = RX_NORM_INFO + rxcuid + '/allinfo'
		return (get_rx_norm_info)

	def getRXNormDrugInfo(drug_name):
		get_drugs = 'drugs?name=' + drug_name
		http_string = RxNav_base_url+get_drugs

	def get_NDFRT_allinfo(NUI):
		ndfrt_allinfo = 'allInfo.json?nui='+NUI
		http_string = RxNav_base_url+ ndfrt_allinfo

	def getDrugInteractionsSources():
		getDrugInteractionSource = '/interaction/sources.json'
		http_string = RxNav_base_url+getDrugInteractionSource

	def getDrugInteractions(rxcui):
		drugInteractions = 'interaction/interaction.json?rxcui=' + rxcui +'&sources=ONCHigh'
		drugInteractionsList = 'interaction/list.json?rxcuis=' ##iterate and append array elem with rxcui
		http_string = RxNav_base_url+ drugInteractions

	def getAll_NDC_Properties(ndc_prop):
		## NDC 2 segment   67544-355
		## NDC 3 segment   0781-1506-10
		## NDC11                 00904629161
		## RxCUI                   213270
		## SPL_SET_ID          1C5BC1DD-E9EC-44C1-9281-67AD482315D9
		getAll_NDC_Properties = 'ndcproperties?id=' + ndc_prop
		http_string = RxNav_base_url+ getAll_NDC_Properties