```python
%matplotlib notebook
```

```python
import numpy as np
import matplotlib.pyplot as plt
import random
import string
from collections import OrderedDict
from nltk.stem import PorterStemmer
from nltk.tokenize import RegexpTokenizer
from nltk.corpus import stopwords
from nltk.tokenize import word_tokenize
```

```python
# Normalize the files to use i.e. same number of lines
f = open("126015.txt", "r")
book1 = f.read()
book1_lines = book1.split('\n')
random.shuffle(book1_lines)
book1_part1 = book1_lines[:len(book1_lines)//2]
book1_part2 = book1_lines[len(book1_lines)//2:]
f.close()

wf = open("126015_1.txt", "w+")
wf.write('\n'.join(book1_part1))
wf.close()

wf = open("126015_2.txt", "w+")
wf.write('\n'.join(book1_part2))
wf.close()
```

```python
# Normalize second file.
f = open("524590.txt", "r")
book2 = f.read()
book2_lines = book2.split('\n')
random.shuffle(book2_lines)
book2_normalized = book2_lines[:len(book1_lines)//2]
f.close()

wf = open("524590_1.txt", "w+")
wf.write('\n'.join(book2_normalized))
wf.close()
```

```python
def get_word_len_freq(path):
    f = open(path, "r")
    book1 = f.read()

    # Tokenize based on NLTK and regexp split criteria.
    book1_words = word_tokenize(book1)
    tokenizer = RegexpTokenizer(r'\w+')
    book1_words = tokenizer.tokenize(" ".join(book1_words))

    stop_words = set(stopwords.words('english'))

    # Remove punctuation.
    book1_words = [x.lower() for x in book1_words if not x in string.punctuation]
    # Remove stop words.
    book1_words = [x for x in book1_words if not x in stop_words]
    # Stem words.
    ps = PorterStemmer()
    book1_words = [ps.stem(x) for x in book1_words]

    book1_word_freq = {}
    for word in book1_words:
        if book1_word_freq.get(len(word)):
            book1_word_freq[len(word)] += 1
        else:
            book1_word_freq[len(word)] = 1

    return book1_word_freq
```

```python
def get_word_freq(path):
    f = open(path, "r")
    book1 = f.read()

    # Tokenize based on NLTK and regexp split criteria.
    book1_words = word_tokenize(book1)
    tokenizer = RegexpTokenizer(r'\w+')
    book1_words = tokenizer.tokenize(" ".join(book1_words))

    stop_words = set(stopwords.words('english'))

    # Remove punctuation.
    book1_words = [x.lower() for x in book1_words if not x in string.punctuation]
    # Remove stop words.
    book1_words = [x for x in book1_words if not x in stop_words]
    # Stem words.
    ps = PorterStemmer()
    book1_words = [ps.stem(x) for x in book1_words]

    book1_word_freq = {}
    for word in book1_words:
        if book1_word_freq.get(word):
            book1_word_freq[word] += 1
        else:
            book1_word_freq[word] = 1

    return book1_word_freq
```

```python
# 126015 - Perfect Obedience A Bride’s Vow
# Genre: Historical Romance Fiction
book1_len_freq = get_word_len_freq("126015_1.txt")
# 126015 - Perfect Obedience A Bride’s Vow
# Genre: Historical Romance Fiction
book2_len_freq = get_word_len_freq("126015_2.txt")
# 524590 - 48 Hours To Die
# Genre: Thriller Fiction
book3_len_freq = get_word_len_freq("524590_1.txt")
```

```python
book1_len_freq
```

    {4: 7207,
     6: 2750,
     5: 4135,
     7: 1092,
     3: 2160,
     2: 222,
     8: 398,
     9: 131,
     1: 7,
     12: 9,
     10: 43,
     11: 3}

```python
book2_len_freq
```

    {4: 7223,
     6: 2657,
     5: 4200,
     3: 2101,
     8: 418,
     7: 1145,
     2: 174,
     1: 8,
     10: 41,
     9: 133,
     13: 2,
     11: 2,
     12: 3}

```python
min_length = 1
max_length = max(max(book1_len_freq.keys()), max(book2_len_freq.keys()), max(book3_len_freq.keys()))
```

```python
xvalues = [i for i in range(min_length, max_length+1)]

yvalues1_norm = sum(book1_len_freq.values())
yvalues1 = []
for i in range(min_length, max_length+1):
    if book1_len_freq.get(i):
        yvalues1.append(book1_len_freq.get(i) / yvalues1_norm)
    else:
        yvalues1.append(0)

yvalues2_norm = sum(book2_len_freq.values())
yvalues2 = []
for i in range(min_length, max_length+1):
    if book2_len_freq.get(i):
        yvalues2.append(book2_len_freq.get(i) / yvalues2_norm)
    else:
        yvalues2.append(0)

yvalues3_norm = sum(book3_len_freq.values())
yvalues3 = []
for i in range(min_length, max_length+1):
    if book3_len_freq.get(i):
        yvalues3.append(book3_len_freq.get(i) / yvalues3_norm)
    else:
        yvalues3.append(0)
```

```python
sum_yvalues1 = [sum(yvalues1[:i+1]) for i in range(len(yvalues1))]
sum_yvalues2 = [sum(yvalues2[:i+1]) for i in range(len(yvalues2))]
sum_yvalues3 = [sum(yvalues3[:i+1]) for i in range(len(yvalues3))]
```

```python
plt.figure(figsize=(9,4))
plt.subplot(1, 2, 1)
plt.title("Pages PDF")
plt.xlabel("Word Length")
plt.ylabel("Probability")
plt.plot(xvalues, yvalues1, label="Fragment 1", alpha=0.6)
plt.plot(xvalues, yvalues2, label="Fragment 2", alpha=0.6)
plt.plot(xvalues, yvalues3, label="Fragment 3", alpha=0.6)
plt.legend()
plt.subplot(1, 2, 2)
plt.title("Pages EDF")
plt.xlabel("Word Length")
plt.ylabel("Cumulative Probability")
plt.plot(xvalues, sum_yvalues1, label="Fragment 1", alpha=0.6)
plt.plot(xvalues, sum_yvalues2, label="Fragment 2", alpha=0.6)
plt.plot(xvalues, sum_yvalues3, label="Fragment 3", alpha=0.6)
plt.legend()
plt.show()
```

    <IPython.core.display.Javascript object>

$\begin{equation}D_{F, G} = \max_{x_i\in x} \{ |F(x_i)-G(x_i)| \} \end{equation}$

```python

```


# Introduction

My master thesis looked into service discovery recommendation system research based on data available at the programmable web [website](https://www.programmableweb.com/).

In this notebook, I will share some of what I learned and use Latent Dirichlet Allocation and the Author Topic Model are implemented and the results were compared in terms of Mean Average Precision@N and Recall@N.

## Importing Libraries and Loading Data

```python
import numpy as np
import pandas as pd
import re
from collections import  Counter
import ml_metrics as metrics
from nltk.corpus import stopwords
from nltk.stem.porter import PorterStemmer

#Mashup Data
mdata = pd.read_csv("data/mashups_2k18.csv",sep='\t',encoding='utf-8')
#list of unique APIs
u_apis= pd.read_csv("data/unique_apis.csv",sep='\t',encoding='utf-8')
mdata.sort_values('title',inplace=True)
u_apis.sort_values('api_name',inplace=True)
print('Total Number of Mashups: ', mdata.shape[0])
print('Total Number of Mashups: ', u_apis.shape[0])

mdata.head()

```

    Total Number of Mashups:  6962
    Total Number of Mashups:  1520

      Unnamed: 0
      description
      title
      date
      Related_APIs
      Tags

      906
      906
      #API Christmas is the Nativity Story, remixed ...
      #API Christmas
      2012-12-20
      ['instagram-graph']
      ['aggregation', 'holidays', 'humor', 'social',...

      2486
      2486
      #Ask4Stuff is a new, Twitter-based service tha...
      #Ask4Stuff: WorldCat Twitter Search
      2010-06-27
      ['twitter', 'worldcat-search']
      ['library', 'search', 'books']

      610
      610
      To discover the most popular beer in over 1500...
      #BeerMap - The Top 2,500 Beers on Twitter
      2013-09-19
      ['google-maps', 'twitter', 'peekyou-social-ana...
      ['analytics', 'beer', 'humor']

      2922
      2922
      Get some shiny stats about countries, cities a...
      #LinS Stats on Your LinkedIn Social Graph
      2009-11-27
      ['google-chart', 'linkedin']
      ['social', 'statistics', 'social', 'enterprise']

      577
      577
      The #S42AT project lets you bookmark links you...
      #S42AT
      2013-11-11
      ['facebook', 'twitter', 'instagram-graph', 'go...
      ['bookmarks', 'tools', 'tweets']

## Data Pre-processing

As machine learning models can not deal with raw text data, a set of pre- processing steps is needed to give a meaningful representation of the text. The text is transformed to a Bag-of-Words representation which assigns a unique number to every word in the vocabulary, this way any document can be represented as a vector of length $\vert V\vert$ the number of unique words in the vocabulary. Each cell in the vector contains the count of the word in the document.
This can be done by assigning each word a unique number. Then any document we see can be encoded as a fixed-length vector with the length of the vocabulary of known words. The value in each position in the vector could be filled with a count or frequency of each word in the encoded document.

A high number of words generally translates into more computation and sparsity in the bag of word model, meaning that words should be carefully selected from the vocabulary and words that do not add meaning to the machine learning model should be dropped. The pre-processing steps follow the general guidelines of working with text in the Nature Language Processing field and the reviewed work on service discovery RS:

####  Tokenization: dividing a stream of text into a sequence of words according to a delimiter (space in this case)
#### Pruning: Removing words that do not add meaning to the sentence, also known as stop-words removal, a list of stopwords was taken from the Natural Language Toolkit (nltk) python library to this end
##### item Stemming: Stripping words from suffixes and/or prefixes to get to the word stem/root. This way word like 'map','mapping' and 'maps' would be counted as 'map' adding relevance for the model. The Porter Stemmer of the nltk library was used.
#### Text Augmentation: Since Mashup and API titles and categories add meaning to the representation of a mashup, they are added to the description of the mashups.

```python
#storing the mashup corpus as a list of words
corpus_list=[]
#storing the corpus as strings
corpus_string = []
#APIs related to each row of the mashup data
rapis= []
#storing the API corpus as strings
corpus_api_string = []
#storing the API corpus as a list of word_ids
corpus_api_list = []
#dictionary for mashup and its corespending APIs
dict_mashup_apis = dict()
#Number of times each API was used in a mashup
api_count = []
#calculating tag counts for EDA
tag_count  = []
#mashup_api matrice for EDA
ma_matrice = np.zeros((mdata.shape[0],u_apis.shape[0]),dtype=int)

for i in range(0, mdata.shape[0]):

    # get each related api seperatly
    apis = list(filter(None,re.sub('[^\w+\d+-]',' ', mdata['Related_APIs'][i]).strip().split(' ')))
    rapis.append(apis)
    dict_mashup_apis[i] = []

    for a in apis:
        dict_mashup_apis[i].append(a)
        api_index = u_apis.loc[u_apis['api_name']==a].index
        ma_matrice[i][api_index]= ma_matrice[i][api_index] + 1

    api_count.extend(apis)

    #Extract Description and tags while removing special characters
    review = re.sub('[^a-zA-Z-\d+]', ' ', mdata['description'][i])
    tags =  re.sub('[^a-zA-Z-\d+]',' ',mdata['Tags'][i])
    tag_count.extend(tags.split())
    review = review.lower()
    review = review.split()
    #Changing each word to stem form
    ps = PorterStemmer()
    stopwords_ = list(set(stopwords.words('english')))
    stopwords_.append('mashup')
    review = [ps.stem(word) for word in review if not word in set(stopwords_)]
    #adding the tags 3 times and the title of the mashup to the mashup description
    for j in range(3):
        review.append(tags)

    review.append(mdata['title'][i])
    corpus_list.append(review)
    review = ' '.join(review)
    corpus_string.append(review)

print(mdata['description'][10],'\n After Pre-processing')
corpus_string[10]

```

    Goodneighbor is a platform that allows you to define, run and manage Twitter avatars, totally autonomous Twitter identities that manage their own Twitter accounts, concerned with any subject matter and interested in any topic.
     After Pre-processing

    'goodneighbor platform allow defin run manag twitter avatar total autonom twitter ident manag twitter account concern subject matter interest topic   artificial-intelligence    social     artificial-intelligence    social     artificial-intelligence    social   Goodneighbor'

So far we have tokenized and stemed the mashup and APIs description, and mapped each API (Parent) to its mashup (child).

Now let's change api names and words to numerical IDs, and create our bag of word model

```python

# Changing API name to API ids
unique_apis = list(u_apis['api_name'])
api_id_dict = dict(zip(unique_apis, range(len(unique_apis))))
api_id_dict_reverse = dict(zip( range(len(unique_apis)),unique_apis))

#Create dictionary of Mashups and Related APIs IDs
for mash, rapi in dict_mashup_apis.items():
    for i, api_id in enumerate(rapi):

        dict_mashup_apis[mash][i] = api_id_dict[api_id]

# Let's create the bag of word model
from sklearn.feature_extraction.text import CountVectorizer
cv = CountVectorizer(min_df=5,max_df=0.8,stop_words='english')
L = cv.fit_transform(corpus_string + corpus_api_string).toarray()
features = cv.get_feature_names()

```

```python
#Now let's encode the corpus with only the features extracted from the count vectorizer and create appropriate word dict
corpus_encoded = []
word_id_dict = dict(zip(features, range(len(features))))
word_id_dict_reverse = dict(zip( range(len(features)),features))
corpus_encoded = []
for each in corpus_list:
    words_encoded = []
    for e in each:
        if(e in word_id_dict):
            words_encoded.append(word_id_dict[e])
    corpus_encoded.append(words_encoded)

```

# EDA

## What APIs are used the most for creating Mashups?

The first graph plots the API use count, and we can clearly see that very few APIs are responsible for most mashups. Leading the pack is 'google-maps' with more than 2300 mashup to its count, followed by Twitter that has more than 750 mashup.
On the other hand, the top 40 - 50 APIs are only used to create around 40 mashups

```python
from collections import Counter
import seaborn as sns
import matplotlib.pyplot as plt
from matplotlib import pyplot
%matplotlib inline

counter2 = Counter(api_count)
eda_apis =  pd.DataFrame.from_dict(counter2, orient='Index').reset_index()
eda_apis.columns = ['api','count']
f_apid = lambda x : api_id_dict[x]
eda_apis['id'] = eda_apis['api'].apply(f_apid)
eda_apis.sort_values('count',ascending=False,inplace=True)
top_50_apis = eda_apis[["count",'api']][:50].copy()

plt.subplots(figsize=(22, 8))
plt.xticks(rotation=90)
sns.barplot(x='api', y="count", data=top_50_apis, palette="vlag").set_title("50 Most Used APIs")
plt.show()

```

```python
top_50_apis.tail(10)
```

      count
      api

      241
      50
      freebase

      359
      48
      panoramio

      83
      47
      eventful

      341
      44
      facebook-graph

      192
      44
      google-calendar

      447
      43
      google-earth

      279
      43
      salesforcecom

      302
      42
      google-translate

      53
      41
      indeed

      555
      40
      google-maps-flash

# What are the most popular Tags ?

We can see from the graph that the tag distribution is similar to that of APIs (few tags present with most mashups)
It is normal to see such distribution since the APIs used with a mashup influence the possible functionalities of the mashup (hence tags as well)

```python
counter = Counter(tag_count)
tag_count_result= list(counter.most_common())
eda_tags = pd.DataFrame.from_dict(counter, orient='Index').reset_index()
eda_tags.columns = ['tag','count']
eda_tags.sort_values('count',ascending=False,inplace=True)
top_50_tags = eda_tags[["count",'tag']][:50].copy()

plt.subplots(figsize=(22, 8))
plt.xticks(rotation=90)
sns.barplot(x='tag', y="count", data=top_50_tags, palette="vlag").set_title("50 Most Used Tags")
plt.show()
```

#  The API - Mashup Matrix

Now let's have a look at the mashups and the APIs they use

Horizantal lines indicate that an API has been used to create a lot of mashups

```python
df = pd.DataFrame(ma_matrice.T,columns=mdata['title'].values)
df.index = u_apis['api_name'].values

plt.figure(figsize=(20,8))
ax = sns.heatmap(df,vmin=0, vmax=1,cmap='rocket' )
ax.set_xlabel(xlabel='Mashups', fontsize=16)
ax.set_ylabel(ylabel='APIs', fontsize=16)
ax.set_title(label='API invocation Matrix', fontsize=20)
plt.show()

# (10, 27)
```

# How many APIs occur together in mashups ?

Let's have a look at the co-occurance of APIs, to do that we take the dot product of the mashup API matrix with its inverse (transpose)

We can clearly see that the matrix is very sparse (a lot of 0s, thus the dark color)
the horizontal/vertical colored line indicate that it is a popular API and that it has been used a lot for creating mashups

The white (light) dotes in the graph indicate that a pair of APIs has been used a lot together (at least 5 times)

```python
import numpy as np
#calculating API co occurences
#A = np.dot(ma_matrice.transpose(),ma_matrice)
#df = pd.DataFrame(A,columns = u_apis['api_name'].values)
#df.set_index(u_apis['api_name'].values,inplace=True)
#df.to_csv('data/api_coocu.csv', sep='\t', encoding='utf-8')
#df.head()

#API Co occurence matrice - To avoid a long processing time,
#I will load the API co occurence matrice from a file
A = pd.read_csv("data/api_coocu.csv",sep='\t',encoding='utf-8').drop('Unnamed: 0',axis=1)
A.set_index(u_apis['api_name'].values,inplace=True)
A.head()

```

      123-shop-pro
      123contactform
      12secondstv
      140-proof
      18amail
      1map
      23
      30-boxes
      3dcart
      411sync
      ...
      zixxo
      zoho-crm
      zomato-foodie-index
      zoomin
      zoominfo
      zooomr
      zoopla
      zootool
      zotero
      zvents

      123-shop-pro
      18
      0
      0
      0
      0
      0
      0
      0
      0
      0
      ...
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0

      123contactform
      0
      1
      0
      0
      0
      0
      0
      0
      0
      0
      ...
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0

      12secondstv
      0
      0
      1
      0
      0
      0
      0
      0
      0
      0
      ...
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0

      140-proof
      0
      0
      0
      2
      0
      0
      0
      0
      0
      0
      ...
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0

      18amail
      0
      0
      0
      0
      1
      0
      0
      0
      0
      0
      ...
      0
      0
      0
      0
      0
      0
      0
      0
      0
      0

5 rows × 1520 columns

```python
plt.figure(figsize=(20,15))
ax = sns.heatmap(A,vmin=0, vmax=5,cmap='rocket' )
ax.set_xlabel(xlabel='APIs', fontsize=16)
ax.set_ylabel(ylabel='APIs', fontsize=16)
ax.set_title(label='API co-occurence Matrix', fontsize=20)
plt.show()

```

# Recomendation Approach (Author Topic Model)

The Programmable Web API recommendation problem can be defined as follow:
Given a list of Mashups and and their component APIs descriptions, build a recommender that returns a ranked list of APIs that would satisfy the user query. For the sake of off-line evaluation, the user query is represented by unseen mashup descriptions and categories

## Graphical explanation of the recomendation process

If you compare the description of the mashup Holidayen and its component service Flickr API in the picture below, you quickly notice that there is not much in common in their respective descriptions and that’s because each one serves a different function.

Holidayen helps users with planning their holidays and Flickr API provides access to photos from the Flickr platform using different feeds. To a mashup creator, it is clear that Flickr was used to provide adequate pictures for different locations and make the user experience pleasant, though this relation can not be captured by the service description because the API provider does not have to provide information about all the potential use cases of its services. The mashup descriptions on the other hand are more likely to give insight of the potential use cases of an API because they link the service Flickr to the ‘Travel’ use case in this example.

Starting from this observation, the authors of [the reconstructed service profile paper](https://ieeexplore.ieee.org/document/7763869/) devised a way to build new API service description based on the description of the mashups that invoked said APIs. To put it more in context, let’s use the Holidayen example, the words used to describe the mashup can be attributed to each related API as shown in the figure below. Doing the stated operation for every mashup gives a new set of words to describe the APIs that better captures the relationship between the APIs and their use cases. The attribution of words from mashups to APIs is done using a generative probabilistic approach called The Author Topic model.

The [author topic model model](https://arxiv.org/abs/1207.4169) was first introduced to explore the relationships between authors, documents, topics, and words in a collection of academics papers and paper abstracts. The author topic model views each words in a document as the manifestation of one of the authors’ interest (topics). According to the Author Topic Model,  to create each word w in a document d, an author is chosen at random from the uniform distribution of authors over document, and a word is picked from a topic distribution specific to that author. Accordingly, each author is associated with a multinomial distribution over topics, each topic is associated with a multinomial distribution over words,  and each word is associated with an author and a topic assignment. The topic distribution of a document is then a mixture of topic assignments of the author of the documents.

If each document was written by a unique author, then the author assignment becomes trivial, meaning that topics are determined by documents which is the same as using LDA model.
Latent Dirichlet Allocation (LDA) is a generative statistical model that treats each document as a mixture of distinct topics (clusters of words) that are responsible for the appearance of each word in the document with a certain probability. The generative process to create mashups using LDA and the graph notation of LDA is presented below:

In the mashup service discovery context, we can think of APIs as being authors of mashup descriptions.  Each word in the description can then be attributed to one of the API’s topic assignment. the figure below depicts the plate notation of the generative process to mashup words creation with the author-topic model.
The generative process to create mashups works as follows:

However; the recommendation approach I am using here is only needs the author assignment of the Author Topic Model, we use to calculate the following conditional probability. The probability can be seen as wether an API (s for Service) will satisfy a user if he inputs a word (w) in his query

\begin{equation}
\label{eq:psw} p(s|w)=\frac {\text {RP}(s,w)}{\sum _{k}\text {RP}(k,w)}.
\end{equation}

Once the relevance between services and words is calculated, the query relevance score can be defined as follow:

\begin{equation} r(s,Q)=\sum \limits _{w\in Q}p(s|w). \end{equation}

To account for the popularity of APIs, a list of dominant words is generated by looking at the co-occurrences of service - word pairs . Basically, if a word in the reconstructed profiles is overwhelmingly associated with a service $p(s\vert w) ~=1$, the services and the words are associated and added to a dominant words list. If a dominant word is present in the user query, then its corresponding service score is boosted by adding the length of the query to the final relevance score. The authors [the reconstructed service profile paper](https://ieeexplore.ieee.org/document/7763869/) propose two simple rules to identify dominant words.

     p(s|w) > g where g is a constant between 0 and 1
     RP(s,w) > h where h is constant positive integer

### Functions for calculating relevance score

```python
dominant_words = dict()
def rspsim(word,api,z,l):

    dist = (rsp[api][word])/z
    if(rsp[api][word] / z > 0.3 and rsp[api][word]>0.5):
        dist = (rsp[api][word]/z)+ l
        if(word in dominant_words):
            if(not (api in dominant_words[word])):
                dominant_words[word].append(api)
        else:
            dominant_words[word]=[]
            dominant_words[word].append(api)

    return  dist

def get_rspsims(words,top_n=20):
    ranked_list = np.zeros(len(unique_apis),dtype='float')
    l = len(words)
    for word in words:

        z = np.sum(rsp[:,word])

        if(z > 0):
            sims = np.array([rspsim(word,api_id_dict[api],z,l) for api in unique_apis ])
        else:
            sims = np.zeros(len(unique_apis))
        ranked_list = ranked_list + sims

    df = pd.DataFrame(list(zip(unique_apis, list(ranked_list))), columns=['Author', 'Score'])
    return df

def recall(actual, predicted):
    la = len(actual)
    r = []

    for i in range(len(predicted)):
        tp = [doc for doc in predicted[:i+1] if doc in actual]
        r.append(len(tp) * 1.0 / la)

    return r

```

### Building Train and Test Set

```python
rsp = np.zeros([len(unique_apis),len(features)])
rsp_dict = dict()
for each in unique_apis:
    rsp_dict[each] = []

most_common_5 = ['google-maps', 'twitter', 'youtube', 'flickr','facebook']
                  #,'amazon-product-advertising', 'twilio','lastfm', 'ebay', 'twilio-sms', 'google-search', 'microsoft-bing-maps',
                  #'delicious-0', 'google-ajax-search', 'foursquare', 'google-app-engine',
                  #'google-homepage', 'yahoo-search', 'docusign-enterprise', 'yahoo-maps']

from sklearn.model_selection import train_test_split
X_Train, X_Test, Y_Train, Y_Test,dict_train,dict_test = train_test_split(corpus_encoded, rapis,dict_mashup_apis, test_size = 0.2, random_state = 0)

print('Size of train records:', len(X_Train))
print('Size of test records:', len(X_Test))

```

    Size of train records: 5569
    Size of test records: 1393

### Running the Author Topic Model
 may take some time

```python
n_doc = len(X_Train)
n_topic = 50
n_author = len(unique_apis)
n_voca = len(features)
max_iter = 50
from original import AuthorTopicModel
model = AuthorTopicModel(n_doc, n_voca, n_topic, n_author,alpha = 1, beta=0.01)
model.fit(X_Train, dict_train, max_iter=max_iter)
```

    2019-01-17 12:30:45 INFO:AuthorTopicModel:[INIT] 0	elapsed_time:3.71	log_likelihood:-821804.09
    2019-01-17 12:30:49 INFO:AuthorTopicModel:[INIT] 1	elapsed_time:3.71	log_likelihood:-772327.43
    2019-01-17 12:30:53 INFO:AuthorTopicModel:[INIT] 2	elapsed_time:3.79	log_likelihood:-736197.19
    2019-01-17 12:30:57 INFO:AuthorTopicModel:[INIT] 3	elapsed_time:3.79	log_likelihood:-702732.27
    2019-01-17 12:31:00 INFO:AuthorTopicModel:[INIT] 4	elapsed_time:3.84	log_likelihood:-674102.71
    2019-01-17 12:31:04 INFO:AuthorTopicModel:[INIT] 5	elapsed_time:3.66	log_likelihood:-653100.10
    2019-01-17 12:31:08 INFO:AuthorTopicModel:[INIT] 6	elapsed_time:4.22	log_likelihood:-638377.57
    2019-01-17 12:31:12 INFO:AuthorTopicModel:[INIT] 7	elapsed_time:3.98	log_likelihood:-627653.05
    2019-01-17 12:31:16 INFO:AuthorTopicModel:[INIT] 8	elapsed_time:3.85	log_likelihood:-620515.39
    2019-01-17 12:31:20 INFO:AuthorTopicModel:[INIT] 9	elapsed_time:4.11	log_likelihood:-613980.95
    2019-01-17 12:31:24 INFO:AuthorTopicModel:[INIT] 10	elapsed_time:4.09	log_likelihood:-609679.92
    2019-01-17 12:31:29 INFO:AuthorTopicModel:[INIT] 11	elapsed_time:4.15	log_likelihood:-605891.41
    2019-01-17 12:31:33 INFO:AuthorTopicModel:[INIT] 12	elapsed_time:4.09	log_likelihood:-603119.40
    2019-01-17 12:31:36 INFO:AuthorTopicModel:[INIT] 13	elapsed_time:3.72	log_likelihood:-600996.65
    2019-01-17 12:31:40 INFO:AuthorTopicModel:[INIT] 14	elapsed_time:3.76	log_likelihood:-597663.51
    2019-01-17 12:31:44 INFO:AuthorTopicModel:[INIT] 15	elapsed_time:4.13	log_likelihood:-595624.43
    2019-01-17 12:31:48 INFO:AuthorTopicModel:[INIT] 16	elapsed_time:3.97	log_likelihood:-593554.38
    2019-01-17 12:31:52 INFO:AuthorTopicModel:[INIT] 17	elapsed_time:3.93	log_likelihood:-591764.44
    2019-01-17 12:31:56 INFO:AuthorTopicModel:[INIT] 18	elapsed_time:3.96	log_likelihood:-592220.90
    2019-01-17 12:32:00 INFO:AuthorTopicModel:[INIT] 19	elapsed_time:4.07	log_likelihood:-590981.77
    2019-01-17 12:32:04 INFO:AuthorTopicModel:[INIT] 20	elapsed_time:3.93	log_likelihood:-590025.87
    2019-01-17 12:32:08 INFO:AuthorTopicModel:[INIT] 21	elapsed_time:4.25	log_likelihood:-588941.28
    2019-01-17 12:32:12 INFO:AuthorTopicModel:[INIT] 22	elapsed_time:3.84	log_likelihood:-588545.22
    2019-01-17 12:32:16 INFO:AuthorTopicModel:[INIT] 23	elapsed_time:3.76	log_likelihood:-587117.20
    2019-01-17 12:32:20 INFO:AuthorTopicModel:[INIT] 24	elapsed_time:4.26	log_likelihood:-586218.23
    2019-01-17 12:32:25 INFO:AuthorTopicModel:[INIT] 25	elapsed_time:4.69	log_likelihood:-585542.73
    2019-01-17 12:32:30 INFO:AuthorTopicModel:[INIT] 26	elapsed_time:5.08	log_likelihood:-584895.95
    2019-01-17 12:32:35 INFO:AuthorTopicModel:[INIT] 27	elapsed_time:4.43	log_likelihood:-583176.24
    2019-01-17 12:32:38 INFO:AuthorTopicModel:[INIT] 28	elapsed_time:3.74	log_likelihood:-582331.76
    2019-01-17 12:32:42 INFO:AuthorTopicModel:[INIT] 29	elapsed_time:4.21	log_likelihood:-581552.69
    2019-01-17 12:32:47 INFO:AuthorTopicModel:[INIT] 30	elapsed_time:4.10	log_likelihood:-581681.61
    2019-01-17 12:32:51 INFO:AuthorTopicModel:[INIT] 31	elapsed_time:3.95	log_likelihood:-580951.03
    2019-01-17 12:32:54 INFO:AuthorTopicModel:[INIT] 32	elapsed_time:3.93	log_likelihood:-580535.88
    2019-01-17 12:33:00 INFO:AuthorTopicModel:[INIT] 33	elapsed_time:5.05	log_likelihood:-579990.14
    2019-01-17 12:33:04 INFO:AuthorTopicModel:[INIT] 34	elapsed_time:4.58	log_likelihood:-578482.94
    2019-01-17 12:33:09 INFO:AuthorTopicModel:[INIT] 35	elapsed_time:4.49	log_likelihood:-578701.60
    2019-01-17 12:33:13 INFO:AuthorTopicModel:[INIT] 36	elapsed_time:4.33	log_likelihood:-578560.24
    2019-01-17 12:33:17 INFO:AuthorTopicModel:[INIT] 37	elapsed_time:4.04	log_likelihood:-578779.55
    2019-01-17 12:33:21 INFO:AuthorTopicModel:[INIT] 38	elapsed_time:3.93	log_likelihood:-577578.12
    2019-01-17 12:33:25 INFO:AuthorTopicModel:[INIT] 39	elapsed_time:4.27	log_likelihood:-577411.26
    2019-01-17 12:33:29 INFO:AuthorTopicModel:[INIT] 40	elapsed_time:3.99	log_likelihood:-577824.52
    2019-01-17 12:33:33 INFO:AuthorTopicModel:[INIT] 41	elapsed_time:4.09	log_likelihood:-576495.75
    2019-01-17 12:33:37 INFO:AuthorTopicModel:[INIT] 42	elapsed_time:3.72	log_likelihood:-577012.47
    2019-01-17 12:33:41 INFO:AuthorTopicModel:[INIT] 43	elapsed_time:3.99	log_likelihood:-576679.63
    2019-01-17 12:33:45 INFO:AuthorTopicModel:[INIT] 44	elapsed_time:4.06	log_likelihood:-576703.22
    2019-01-17 12:33:49 INFO:AuthorTopicModel:[INIT] 45	elapsed_time:3.98	log_likelihood:-575346.33
    2019-01-17 12:33:53 INFO:AuthorTopicModel:[INIT] 46	elapsed_time:3.90	log_likelihood:-575704.89
    2019-01-17 12:33:57 INFO:AuthorTopicModel:[INIT] 47	elapsed_time:3.83	log_likelihood:-575248.63
    2019-01-17 12:34:01 INFO:AuthorTopicModel:[INIT] 48	elapsed_time:3.87	log_likelihood:-574341.08
    2019-01-17 12:34:04 INFO:AuthorTopicModel:[INIT] 49	elapsed_time:3.67	log_likelihood:-574057.04

### Building Reconstructed  API description

```python

authors_assignment = model.author_assigned

#Building the Reconstructed Service Profile from the author assignment
for i in range(len(X_Train)):
    for j in range(len(X_Train[i])):
        rsp[authors_assignment[i][j]][X_Train[i][j]] = rsp[authors_assignment[i][j]][X_Train[i][j]] + 1
        if(not word_id_dict_reverse[X_Train[i][j]] in rsp_dict[api_id_dict_reverse[authors_assignment[i][j]]]):
            rsp_dict[api_id_dict_reverse[authors_assignment[i][j]]].append(word_id_dict_reverse[X_Train[i][j]])

```

### Getting the predictions

```python
pred_most_common =[]
pred_atm_dw = []
lists = []
for test in X_Test:
    pred_most_common.append(most_common_5)
    df1= get_rspsims(test,5)
    df = df1.sort_values('Score', ascending=False)[:5]
    pred_atm_dw.append(list(df['Author'].values))

```

### Result Evaluation MAP@5 and Recall@5

The most common apis is a good comparaison baseline as the most common APIs account for most of the API invocations.

We see that the reconstructed profile approach is outperforming the baseline ^^

```python
results_map_most_common = (metrics.mapk(Y_Test,pred_most_common,5))
results_map_atm_dw = (metrics.mapk(Y_Test,pred_atm_dw,5))

results_recall_atm_dw = np.mean([recall(a,p) for (a,p) in zip(Y_Test,pred_atm_dw)] )
results_recall_most_common = np.mean([recall(a,p) for (a,p) in zip(Y_Test,pred_most_common)] )

results_df= pd.DataFrame([[results_map_atm_dw,results_map_most_common],
                          [results_recall_atm_dw,results_recall_most_common]]
                         ,columns=['RSP','Most_Common'])
results_df.index = ['MAP@5','R@5']

results_df.plot(kind='bar',rot=0)

results_df.head()
```

### Let's have a look at the new descrption of APIs

```python
keys  = []
values = []
for key,value in rsp_dict.items():
    if(value!=[]):
        keys.append(key)
        values.append(' '.join(value))

rsp_df= pd.DataFrame(keys,columns=['API'])
rsp_df['rsp_description'] = values
rsp_df.head(100)
```

      API
      rsp_description

      0
      123-shop-pro
      order

      1
      123contactform
      paint famili member

      2
      18amail
      kind

      3
      1map
      parent

      4
      23
      view applic photo rotat anywher execut

      5
      30-boxes
      ad list 30 box calendar

      6
      3dcart
      provid txt support store entiti file import re...

      7
      411sync
      latest cell phone real time use wap capabl sim...

      8
      43-places
      offer solut

      9
      43things
      book button

      10
      4shared
      search allow

      11
      7digital
      guess artist song clip pictur question randoml...

      12
      8coupons
      includ deal daili map portion page sourc displ...

      13
      99designs-tasks
      design assist applic

      14
      9flats
      accommod 200 000 return short term apart

      15
      abebooks
      addit

      16
      accuweather
      weather zombi provid local forecast

      17
      active
      sport social network googl api

      18
      activecom
      event allow user base locat leverag activ com ...

      19
      actual-reports
      busi label order minut code imag tabl

      20
      addthis-menu
      plan brows thank com allow share content faceb...

      21
      aden-forshaws-cat
      android app use com everi second anoth cat dis...

      22
      adility
      food pet deal

      23
      adobe-air-bus-tour
      bu applic map current locat adob air promot tour

      24
      aerisweather
      weather center app data user need track gener ...

      25
      after-deadline
      checker spell use googl visitor day

      26
      agendize-action
      use

      27
      agentrank
      wordpress plugin conveni display agent profil ...

      28
      aiderss
      use googl reader extens bring filter view rank...

      29
      airbnb
      busi rang world

      ...
      ...
      ...

      70
      artbeat
      bubbl featur popular art design event tokyo bu...

      71
      artistdata
      blog profession

      72
      att-app-messaging
      messag protect person mobil data individu fami...

      73
      att-call-management
      danger music turn new parent incom custom repl...

      74
      australian-business-number-lookup
      onlin commun australian busi share inform bad ...

      75
      authentic-jobs
      custom rss feed authent job filter type locat ...

      76
      authorizenet
      retail product use fedex net track direct impo...

      77
      autodesk-freewheel
      preview free

      78
      automattic-wordpress-widgets
      creat valu

      79
      aviary-effects
      night effect allow user appli varieti filter u...

      80
      aviary-suite
      use

      81
      avvo
      simpl android nativ applic lawyer unit state s...

      82
      awesm
      applic associ document enter event viral

      83
      baby-names
      look babi perfect list discov origin mean mash...

      84
      backpack
      parti

      85
      backtweets
      api monitor tweet competitor domain add url creat

      86
      backtype
      twitter link domain monitor peopl post tweet url

      87
      baidu
      manag space follow secur video

      88
      bandcamp
      discoveri popul playlist artist discographi brows

      89
      bandsintown
      mash

      90
      bank-russia-daily-info
      mobil rate differ applic bank invest metal api...

      91
      bart
      bu train area washington

      92
      basecamp
      onlin invoic servic api use basecamp integr da...

      93
      batchbook
      kind number record form 100 use data drawloop saa

      94
      bbc
      delay creation bbc updat play aggreg filter ya...

      95
      bbc-music
      convert day track event realtim sourc

      96
      bbc-nitro
      map channel

      97
      beatport
      new track help brows preview latest releas hou...

      98
      beebotte
      simpl web app demo demonstr real time chat app...

      99
      behance
      pop mobil applic allow user transform app idea...

100 rows × 2 columns

# Thank you for following

I hope this was helpful, feel free to get in contact for more information and please share any suggestions/comments you have ^^

#### Sources

The author topic model implementation was found in this [git](https://github.com/dongwookim-ml/python-topic-model)

Text Data -  API Prediction/API_Prediction.ipynb


# Introduction

Systems biology has been brought to the forefront of life science-based research and development. The need for systems analysis is made apparent by the inability of focused studies to explain whole network, cell, or organism behavior, and the availability of component data is what is fueling and enabling the effort. This massive amount of experimental information is a reflection of the complex molecular networks that underlie cellular functions. Reconstructed networks represent a common denominator in systems biology. They are used for data interpretation, comparing organism capabilities, and as the basis for computing their functional states. The companion book _Systems Biology: Properties of Reconstructed Networks_ details the topological features and assessment of functional states of biochemical reaction networks and how these features are represented by the stoichiometric matrix. In this book, we turn our attention to the kinetic properties of the reactions that make up a network. We will focus on the formulation of dynamic simulators and how they are used to generate and study the dynamic states of biological networks.

## Biological Networks
Cells are made up of many chemical constituents that interact to form networks. Networks are fundamentally comprised of _nodes_ (the compounds) and the _links_ (chemical transformations) between them. The networks take on functional states that we wish to compute, and it is these physiological states that we observe. This text is focused on dynamic states of networks.

There are many different kinds of biological networks of interest, and they can be defined in different ways. One common way to define networks is based on a preconceived notion of what they do. Examples include metabolic, signaling, and regulatory networks, see Figure 1.1. This approach is driven by a large body of literature that has grown around a particular cellular function.

**Figure 1.1:** Three examples of networks that are defined by major function.  (a) Metabolism.  (b) Signaling. From Arisi, et al. BMC Neuroscience 2006  7(Suppl 1):S6 DOI:10.1186/1471-2202-7-S1-S6.  (c) Transcriptional regulatory networks. Image courtesy of Christopher Workman, Center for Biological Sequence Analysis, Technical University of Denmark.

### Metabolic networks
Metabolism is ubiquitous in living cells and is involved in essentially all cellular functions. It has a long history - glycolysis was the first pathway elucidated in the 1930s - and is thus well-known in biochemical terms. Many of the enzymes and the corresponding genes have been discovered and characterized. Consequently, the development of dynamic models for metabolism is the most advanced at the present time.

A few large-scale kinetic models of metabolic pathways and networks now exist. Genome-scale reconstructions of metabolic networks in many organisms are now available. With the current developments in metabolomics and fluxomics, there is a growing number of large-scale data sets becoming available. However, there are no genome-scale dynamic models yet available for metabolism.

### Signaling networks
Living cells have a large number of sensing mechanisms to measure and evaluate their environment. Bacteria have a surprising number of 2-component sensing systems that inform the organism about its nutritional, physical, and biological environment. Human cells in tissues have a large number of receptor systems in their membranes to which specific ligands bind, such as growth factors or chemokines. Such signaling influences the cellular fate processes: differentiation, replication, apoptosis, and migration.

The function of many of the signaling pathways that is initiated by a sensing event are presently known, and this knowledge is becoming more detailed. Only a handful of signaling networks are well-known, such as the JAK-STAT signaling network in lymphocytes and the Toll-like receptor system in macrophages. A growing number of dynamic models for individual signaling pathways are becoming available.

### Regulatory networks
There is a complex network of interactions that determine the DNA binding state of most proteins, which in turn determine if genes are being expressed. The RNA polymerase must bind to DNA, as do transcription factors and various other proteins. The details of these chemical interactions are being worked out, but in the absence of such information, most of the network models that have been built are discrete, stochastic, and logistical in nature.

With the rapid development of experimental methods that measure expression states, the binding sites, and their occupancy, we may soon see large-scale reconstructions of transcriptional regulatory networks. Once these are available, we can begin to plan the process to build models that will describe their dynamic states.

### Unbiased network definitions
An alternative way to define networks is based on chemical assays. Measuring all protein-protein interactions regardless of function provides one such example, see Figure 1.2. Another example is a genome-wide measurement of the binding sites of a DNA-binding protein. This approach is driven by data-generating capabilities. It does not have an _a priori_ bias about the function of molecules being examined.

**Figure 1.2:** Two examples of networks that are defined by high-throughput chemical assays. Images courtesy of Markus Herrgard.

### Network reconstruction
Metabolic networks are currently the best-characterized biological networks for which the most detailed reconstructions are available. The conceptual basis for their reconstruction has been reviewed (Reed, 2006), the workflows used detailed (Feist, 2009) and a detailed standard operating procedure (SOP) is available (Thiele, 2010). Some of the fundamental issues associated with the generation of dynamic models describing their functions have been articulated (Jashmidi, 2008).

There is much interest in reconstructing signaling and regulatory networks in a similar way. The prospects for reconstruction of large-scale signaling networks have been discussed (Hyduke, 2010). Given the development of new omics data types and other information, it seems likely that we will be able to obtain reliable reconstructions of these networks in the not too distant future.

### Public information about pathways and networks
There is a growing number of networks that underlie cellular functions that are being unraveled and reconstructed. Many publicly available sources contain this information, Table 1.1. We wish to study the dynamic states of such networks. To do so, we need to describe them in chemical detail and incorporate thermodynamic information and formulate a mathematical model.

**Table 1.1:** Web resources that contain information about biological networks. Prepared by Jan Schellenberger.

## Why Build and Study Models?
Mathematical modeling is practiced in various branches of science and engineering. The construction of models is a laborious and detailed task. It also involves the use of numerical and mathematical analysis, both of which are intellectually intensive and unforgiving undertakings. So why bother?

### Bailey's five reasons
The purpose and utility of model building has been succinctly summarized and discussed (Bailey, 1998):

**1.** _"To organize disparate information into a coherent whole."_ The information that goes into building models is often found in many different sources and the model-builder has to look for these, evaluate them, and put them in context. In our case, this comes down to building data matrices (see Table 1.3) and determining conditions of interest.

**2.** _"To think (and calculate) logically about what components and interactions are important in a complex system."_ Once the information has been gathered it can be mathematically represented in a self-consistent format. Once equations have been formulated using the information gathered and according to the laws of nature, the information can be mathematically interrogated. The interactions among the different components are evaluated and the behavior of the model is compared to experimental data.

**3.** _"To discover new strategies."_ Once a model has been assembled and studied, it often reveals relationships among its different components that were not previously known. Such observations often lead to new experiments, or form the basis for new designs. Further, when a model fails to reproduce the functions of the process being described, it means there is either something critical missing in the model or the data that led to its formulation is inconsistent. Such an occurrence then leads to a re-examination of the information that led to the model formulation. If no logical flaw is found, the analysis of the discrepancy may lead to new experiments to try to discover the missing information.

**4.** _"To make important corrections to the conventional wisdom."_ The properties of a model may differ from the governing thinking about process phenomena that is inferred based on qualitative reasoning. Good models may thus lead to important new conceptual developments.

**5.** _"To understand the essential qualitative features."_ Since a model accounts for all described interactions among its parts, it often leads to a better understanding of the whole. In the present case, such qualitative features relate to multi-scale analysis in time and an understanding of how multiple chemical events culminate in coherent physiological features.

## Characterizing Dynamic States
The dynamic analysis of complex reaction networks involves the tracing of time-dependent changes of concentrations and reaction fluxes over time. The concentrations typically considered are those of metabolites, proteins, or other cellular constituents. There are three key characteristics of dynamics states that we mention here right at the outset, and they are described in more detail in Section 2.1.

### Time constants
Dynamic states are characterized by change in time, thus the rate of change becomes the key consideration. The rate of change of a variable is characterized by a _time constant_. Typically, there is a broad spectrum of time constants found in biochemical reaction networks. This leads to time scale separation where events may be happening on the order of milliseconds all the way to hours, if not days. The determination of the spectrum of time constants is thus central to the analysis of network dynamics.

### Aggregate variables
An associated issue is the identification of the biochemical, and ultimately physiological, events that are unfolding on every time scale. Once identified, one begins to form _aggregate concentration variables,_ or _pooled variables_. These variables will be combinations of the original concentration variables. For example, two concentration variables may inter-convert rapidly, on the order of milliseconds, and thus on every time scale longer than milliseconds, these two concentrations will be "connected." They can therefore be "pooled" together to form an aggregate variable. An example is given in Figure 1.3.

The determination of such aggregate variables becomes an intricate mathematical problem. Once solved, it allows us to determine the _dynamic structure of a network_. In other words, we move hierarchically away from the original concentration variables to increasingly interlinked aggregate variables that ultimately culminate in the overall dynamic features of a network on slower time scales. Temporal decomposition therefore involves finding the time scale spectrum of a network and determining what moves on each one of these time scales. A network can then be studied on any one of these time scales.

**Figure 1.3:** Time-scale hierarchy and the formation of aggregate variables in glycolysis.The 'pooling' process culminates in the formation of one pool (shown in a box at the bottom) that is filled by hexokinase (HK) and drained by ATPase. This pool represents the inventory of high energy phosphate bonds. From (Jashmidi, 2008).

### Transitions
Complex networks can transition from one steady state (i.e., homeostatic state) to another. There are distinct types of transitions that characterize the dynamic states of a network. Transitions are analyzed by _bifurcation theory_. The most common bifurcations involve the emergence of _multiple steady states,_ _sustained oscillations,_ and _chaotic behavior_. Such dynamic features call for a yet more sophisticated mathematical treatment. Such changes in dynamic states have been called _creative functions,_ which in turn represent willful physiological changes in organism behavior. In this book, we will only encounter relatively simple types of such transitions.

## 1.4 Formulating Dynamic Network Models
### Approach
Mechanistic kinetic models based on differential equations represent a _bottom-up approach_. This means that we identify all the detailed events in a network and systematically build it up in complexity by adding more and more new information about the components of a network and how they interact. A complementary approach to the analysis of a biochemical reaction network is a _top-down approach_ where one collects data and information about the state of the whole network at one time. This approach is not covered in this text but typically requires a Bayesian or Boolean analysis that represents causal or statistically-determined relationships between network components. The bottom-up approach requires a mechanistic understanding of component interactions. Both the top-down and bottom-up approaches are useful and complimentary in studying the dynamic states of networks.

### Simplifying assumptions
Kinetic models are typically formulated as a set of deterministic ordinary differential equations (ODEs). There are a number of important assumptions made in such formulations that often are not fully described and delineated. Five assumptions will be discussed here (Table 1.2).

**Table 1.2:** Assumptions used in the formulation of biological network models.

**1.** Using deterministic equations to model biochemistry essentially implies a "clockwork" of functionality. However, this modeling assumption needs justification. There are three principal sources of variability in biological dynamics: internal thermal noise, changes in the environment, and cell-to-cell variation. Inside cells, all components experience thermal effects that result in _random molecular motion_. This process is, of course, one of molecular diffusion, called _Brownian motion_ with larger observable objects. The ordinary differential equation assumption involves taking an ensemble of molecules and averaging out the stochastic effects. In cases where there are very few molecules of a particular species inside a cell or a cellular compartment, this assumption may turn out to be erroneous.

**2.** The finer architecture of cells is also typically not considered in kinetic models. Cells are highly structured down to the 100 nm length scale and are thus not homogeneous (see Figure 1.4). Rapidly diffusing compounds, such as metabolites and ions, will distribute rapidly throughout the compartment and one can justifiably consider the concentration to be relatively uniform. However, with larger molecules whose diffusion is hindered and confined, one may have to consider their spatial location. Studying and describing cellular functions of the 100 nm length scale is likely to represent an interesting topic in systems biology as it unfolds.

**3.** Another major assumption in most kinetic models is that of _constant volume_. Cells and cellular compartments typically have fluctuations in their volume. Treating variable volume turns out to be mathematically difficult. It is, therefore, often ignored. However, minor fluctuations in the volume of a cellular compartment may change all the concentrations in that compartment and therefore all kinetic and regulatory effects.

**4.** Temperature is typically considered to be a constant. Larger organisms have the capability to control their temperature. Small organisms have a high surface to volume ratio making it hard to control heat flux at their periphery. Further, small cellular dimensions lead to rapid thermal diffusivity and a strong dependency on the thermal characteristics of the environment. Rate constants are normally a strong function of temperature, often described by Arrhenius' law. Thus, treating cells as isothermal systems is a simplification under which the kinetic properties are described by kinetic constants.

**5.** All cells and cellular compartments must maintain electro-neutrality and therefore the exchange of any species in and out of a compartment or a cell must also obey electro-neutrality. Considering the charge of molecules and their pH dependence is yet another complicated mathematical subject and thus often ignored. Similarly, significant internal osmotic pressure must be balanced with that of the environment. Cells in tissues are in an isotonic environment, while single cellular organisms and cells in plants build rigid walls to maintain their integrity.

**Figure 1.4:** The crowded state of the intracellular environment.Some of the physical characteristics are viscosity $(>100\ \mu_{H2O})$, osmotic pressure $(<150 atm)$, electrical gradients $(\approx 300,000\ V/cm)$, and a near crystalline state. © David S.Goodsell 1999.

### The dynamic mass balance equations
Applying these simplifying assumptions, we arrive at the dynamic mass balance equations as the starting point for modeling the dynamic states of biochemical reaction networks. The basic notion of a dynamic mass balance on a single compound, $x_i$, is shown in Figure 1.5.

**Figure 1.5:** The dynamic mass balance on a single compound. (a) All the rates of formation and degradation of a compound $x_i$ (a graphical representation called a node map). (b) The corresponding dynamic mass balance equation that simply states that the rate of change of the concentrations $x_i$ is equal to the sum of the rates of formation minus the sum of the rates of degradation. This summation can be represented as an inner product between a row vector and the flux vector. This row vector becomes a row in the stoichiometric matrix in Eq. 1.1.

The combination of all the dynamic mass balances for all concentrations ($\textbf{x}$)in a biochemical reaction network are given by a matrix equation:

$$\begin{equation} \frac{d\textbf{x}}{dt} = \textbf{Sv}(\textbf{x}) \tag{1.1} \end{equation}$$

where $\textbf{S}$ is the stoichiometric matrix, $\textbf{v}$ is the vector of reaction fluxes $(v_j)$, and $\textbf{x}$ is the vector of concentrations $(x_i)$. Equation 1.1 will be the 'master' equation that will be used to describe network dynamics states in this book.

### Alternative views

There are a number of considerations that come with the differential equation formalism described in this book and in the vast majority of the research literature on this subject matter. Perhaps the most important issue is the treatment of cells as behaving deterministically and displaying continuous changes over time. It is possible, though, that cells ultimately will be viewed as essentially a _liquid crystalline state_ where transitions will be discreet from one state to the next, and not continuous.

## The Basic Information is in a Matrix Format
The natural mathematical language for describing network states using dynamic mass balances is that of matrix algebra. In studying the dynamic states of networks, there are three fundamental matrices of interest: the stoichiometric, the gradient, and the Jacobian matrices.

**Table 1.3:** Comparison of some of the attributes and properties of the stoichiometric and the gradient matrices. Adapted from (Jashmidi, 2008).

### The stoichiometric matrix
The stoichiometric matrix, the properties of which are detailed in (SB1), represents the reaction topology of a network. Every row in this matrix represents a compound (alternative states require multiple rows) and every column represents a link between compounds, or a chemical reaction. All the entries in this matrix are _stoichiometric coefficients_.  This matrix is "knowable," since it is comprised of integers that have no error associated with them. The stoichiometric matrix is _genomically-derived_ and thus all members of a species or a biopopulation will have the same stoichiometric matrix.

Mathematically, the stoichiometric matrix has important features. It is a _sparse matrix,_ which means that few of its elements are non-zero. Typically, less than 1% of the elements of a genome-scale stoichiometric matrix are non-zero, and those non-zero elements are almost always 1 or -1. Occasionally there will be an entry of numerical value 2, which may represent the formation of a homodimer. The fact that all the elements of $\textbf{S}$ are of the same order of magnitude makes it a convenient matrix to deal with from a numerical standpoint. The properties of the stoichiometric matrix have been extensively studied (SB1).

### The gradient matrix
Each link in a reaction map has kinetic properties with which it is associated. The reaction rates that describe the kinetic properties are found in the rate laws, $\textbf{v}(\textbf{x}; \textbf{k})$, where the vector $\textbf{k}$ contains all the kinetic constants that appear in the rate laws. Ultimately, these properties represent time constants that tell us how quickly a link in a network will respond to the concentrations that are involved in that link. The reciprocal of these time constants are found in the gradient matrix, $\textbf{G}$, whose elements are:

$$\begin{equation} g_{ij} = \frac{\partial v_i}{\partial x_j}[\text{time}^{-1}] \tag{1.2} \end{equation}$$

These constants may change from one member to the next in a biopopulation given the natural sequence diversity that exists. Therefore, the gradient matrix, is a _genetically-determined_ matrix. Two members of the population may have a different $\textbf{G}$ matrix. This difference is especially important in cases where mutations exist that significantly change the kinetic properties of critical steps in the network and change its dynamic structure. Such changes in the properties of a single link in a network may change the properties of the entire network.

Mathematically speaking, $\textbf{G}$ has several challenging features. Unlike the stoichiometric matrix, its numerical values vary over many orders of magnitude. Some links have very fast response times, while others have long response times. The entries of $\textbf{G}$ are real numbers and therefore are not "knowable." The values of $\textbf{G}$ will always come with an error bar associated with the experimental method used to determine them. Sometimes only order-of-magnitude information about the numerical values of the entries in $\textbf{G}$ is sufficient to allow us to determine the overall dynamic properties of a network. The matrix $\textbf{G}$ has the same sparsity properties as the matrix $\textbf{S}$.

### The Jacobian matrix
The Jacobian matrix, $\textbf{J}$, is the matrix that characterizes network dynamics. It is a product of the stoichiometric matrix and the gradient matrix. The stoichiometric matrix gives us network structure, and the gradient matrix gives us kinetic parameters of the links in the network. The product of these two matrices gives us the network dynamics.

The three matrices described above are thus not independent. The Jacobian is given by

$$\begin{equation} \textbf{J} = \textbf{SG} \tag{1.3} \end{equation}$$

The properties of $\textbf{S}$ and $\textbf{G}$ are compared in Table 1.3.

## Summary
* Large-scale biological networks that underlie various cellular functions can now be reconstructed from detailed data sets and published literature.

* Mathematical models can be built to study the dynamic states of networks. These dynamic states are most often described by ordinary differential equations.

* There are significant assumptions leading to an ordinary differential equations formalism to describe dynamic states. Most notably is the elimination of molecular noise, assuming volumes and temperature to be constant, and considering spatial structure to be insignificant.

* Networks have dynamic states that are characterized by time constants, pooling of variables, and characteristic transitions. The basic dynamic properties of a network come down to analyzing the spectrum of time-scales associated with a network and how the concentrations move on these time-scales.

* Sometimes the concentrations move in tandem at certain time-scales, leading to the formation of aggregate variables. This property is key to the hierarchical decomposition of networks and the understanding of how physiological functions are formed.

* The data used to formulate models to describe dynamic states of networks is found in two matrices: the stoichiometric matrix, that is typically well-known, the gradient matrix, whose elements are harder to determine.

* Dynamic states can be studied by simulation or mathematical analysis. This text is focused on the process of dynamic simulation and its uses.

$\tiny{\text{© B. Ø. Palsson 2011;}\ \text{This publication is in copyright.}\\ \text{Subject to statutory exception and to the provisions of relevant collective licensing agreements,}\\ \text{no reproduction of any part may take place without the written permission of Cambridge University Press.}}$