# Algorithmic Fairness: Considering Different Definitions

Approximate notebook time: 2 hours

## Introduction

Decision making within the United States criminal justice system relies heavily on risk assessment, which determines the potential risk that a released defendant will fail to appear in court or cause harm to the public. Judges use these assessments to decide if bail can be set or if a defendant should be detained before trial. While this is not new in the legal system, the use of risk scores determined by an algorithm are gaining prevalence and support. Proponents promote the use of risk scores to guide judges in their decision making, arguing that machine learning could lead to greater efficiency, accountability, and less biased decisions compared with human judgment ([Henry](https://theappeal.org/risk-assessment-explained/)). On the other hand, critical voices raise the concern that biases can creep into these algorithms at any point in the process, and that algorithms are often applied to the wrong situations ([Henry](https://theappeal.org/risk-assessment-explained/)). Further, they exacerbate the racism embedded deep within the criminal justice system by perpetuating inequalities found in historical data ([Henry](https://theappeal.org/risk-assessment-explained/)).

In the debate about the use of risk assessment algorithms, people have used data analysis to determine the extent to which these algorithms are fair to different groups of people. In this homework, **you will explore some of the many definitions and metrics (different ways of operationalizing data to quantify those definitions) of fairness that can be applied to the risk assessment tool COMPAS**. In doing so, you will understand and provide evidence for or against the presence of bias within the algorithm. You will examine the arguments and analyses made by the company that created COMPAS and the critics of this risk assessment tool to gain a deeper understanding of the technical and societal interpretations and implications of fairness.

**NOTE**: When we discuss bias in this module, we define it most generally as prejudice or an inclination in favor of one person, thing, or group compared to another. In the context of machine learning, bias is a “phenomenon that occurs when an algorithm produces results that are systemically prejudiced due to erroneous assumptions in the machine learning process” ([Rouse](https://searchenterpriseai.techtarget.com/definition/machine-learning-bias-algorithm-bias-or-AI-bias#:~:text=Machine%20learning%20bias%2C%20also%20sometimes,in%20the%20machine%20learning%20process)).

## Table of Contents:
* [Part 0. COMPAS](#part-zero)
* [Part 1. ProPublica's Perspective](#part-one)
* [Part 2. Northpointe's Perspective](#part-two)
* [Part 3. Yet Another Definition of Fairness](#part-three)
* [Part 4. Conclusion](#part-four)

## Setup

Let's begin by importing the packages we need.

```python
import numpy as np
import pandas as pd
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import roc_curve, roc_auc_score
import matplotlib.pyplot as plt
import seaborn as sns
!pip install aif360
!pip install BlackBoxAuditing
from aif360.algorithms.preprocessing import DisparateImpactRemover
from aif360.datasets import BinaryLabelDataset
```

    Requirement already satisfied: aif360 in /Users/eva/opt/anaconda3/lib/python3.8/site-packages (0.3.0)
    Requirement already satisfied: numpy>=1.16 in /Users/eva/opt/anaconda3/lib/python3.8/site-packages (from aif360) (1.18.5)
    Requirement already satisfied: scipy>=1.2.0 in /Users/eva/opt/anaconda3/lib/python3.8/site-packages (from aif360) (1.5.0)
    Requirement already satisfied: scikit-learn>=0.21 in /Users/eva/opt/anaconda3/lib/python3.8/site-packages (from aif360) (0.23.1)
    Requirement already satisfied: matplotlib in /Users/eva/opt/anaconda3/lib/python3.8/site-packages (from aif360) (3.2.2)
    Requirement already satisfied: pandas>=0.24.0 in /Users/eva/opt/anaconda3/lib/python3.8/site-packages (from aif360) (1.0.5)
    Requirement already satisfied: joblib>=0.11 in /Users/eva/opt/anaconda3/lib/python3.8/site-packages (from scikit-learn>=0.21->aif360) (0.16.0)
    Requirement already satisfied: threadpoolctl>=2.0.0 in /Users/eva/opt/anaconda3/lib/python3.8/site-packages (from scikit-learn>=0.21->aif360) (2.1.0)
    Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /Users/eva/opt/anaconda3/lib/python3.8/site-packages (from matplotlib->aif360) (2.4.7)
    Requirement already satisfied: cycler>=0.10 in /Users/eva/opt/anaconda3/lib/python3.8/site-packages (from matplotlib->aif360) (0.10.0)
    Requirement already satisfied: kiwisolver>=1.0.1 in /Users/eva/opt/anaconda3/lib/python3.8/site-packages (from matplotlib->aif360) (1.2.0)
    Requirement already satisfied: python-dateutil>=2.1 in /Users/eva/opt/anaconda3/lib/python3.8/site-packages (from matplotlib->aif360) (2.8.1)
    Requirement already satisfied: pytz>=2017.2 in /Users/eva/opt/anaconda3/lib/python3.8/site-packages (from pandas>=0.24.0->aif360) (2020.1)
    Requirement already satisfied: six in /Users/eva/opt/anaconda3/lib/python3.8/site-packages (from cycler>=0.10->matplotlib->aif360) (1.15.0)
    Requirement already satisfied: BlackBoxAuditing in /Users/eva/opt/anaconda3/lib/python3.8/site-packages (0.1.54)
    Requirement already satisfied: pandas in /Users/eva/opt/anaconda3/lib/python3.8/site-packages (from BlackBoxAuditing) (1.0.5)
    Requirement already satisfied: matplotlib in /Users/eva/opt/anaconda3/lib/python3.8/site-packages (from BlackBoxAuditing) (3.2.2)
    Requirement already satisfied: networkx in /Users/eva/opt/anaconda3/lib/python3.8/site-packages (from BlackBoxAuditing) (2.4)
    Requirement already satisfied: numpy in /Users/eva/opt/anaconda3/lib/python3.8/site-packages (from BlackBoxAuditing) (1.18.5)
    Requirement already satisfied: pytz>=2017.2 in /Users/eva/opt/anaconda3/lib/python3.8/site-packages (from pandas->BlackBoxAuditing) (2020.1)
    Requirement already satisfied: python-dateutil>=2.6.1 in /Users/eva/opt/anaconda3/lib/python3.8/site-packages (from pandas->BlackBoxAuditing) (2.8.1)
    Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /Users/eva/opt/anaconda3/lib/python3.8/site-packages (from matplotlib->BlackBoxAuditing) (2.4.7)
    Requirement already satisfied: cycler>=0.10 in /Users/eva/opt/anaconda3/lib/python3.8/site-packages (from matplotlib->BlackBoxAuditing) (0.10.0)
    Requirement already satisfied: kiwisolver>=1.0.1 in /Users/eva/opt/anaconda3/lib/python3.8/site-packages (from matplotlib->BlackBoxAuditing) (1.2.0)
    Requirement already satisfied: decorator>=4.3.0 in /Users/eva/opt/anaconda3/lib/python3.8/site-packages (from networkx->BlackBoxAuditing) (4.4.2)
    Requirement already satisfied: six>=1.5 in /Users/eva/opt/anaconda3/lib/python3.8/site-packages (from python-dateutil>=2.6.1->pandas->BlackBoxAuditing) (1.15.0)

# Part 0. COMPAS: Why it was created and how it exists in the court system

COMPAS (Correctional Offender Management Profiling for Alternative Sanctions) is a commercial tool produced by the for-profit company Northpointe (acquired by equivant) known as a recidivism risk assessment system. **Tools like COMPAS are used to predict the risk of future crimes for an individual who has entered the US criminal justice system by outputting a risk score from 1-10**. While COMPAS was initially intended to aid decisions made by probation officers on treatment and supervision of those who are incarcerated, Northpointe has since emphasized the scalability of the tool to “fit the needs of many different decision points” including pre-screening assessments, pretrial release decisions (whether or not to hold an arrested individual in jail until their trial), and post-trial next steps for the defendant ([Northpointe](http://www.northpointeinc.com/files/downloads/FAQ_Document.pdf)). These algorithms are believed by many to provide the ability to make the court system more just by removing or correcting for bias of criminal justice officials.

### Question 0a
Explain 3 parties that are impacted by the COMPAS tool. In what ways are they impacted? (Can you think of impacts beyond those in the courtroom for at least one of your examples?)

*Student Written Answer Here*

### Question 0b
Based on your initial reading, what is one problem of the criminal justice system that the COMPAS tool could potentially alleviate? What is one potential problem that using the COMPAS algorithm could introduce?

*Student Written Answer Here*

## Dataset Setup

We will be using the data that was obtained and used by ProPublica in their own analysis of the COMPAS tool from Broward County public records of people who were scored between 2013 and 2014 ([ProPublica](https://www.propublica.org/article/how-we-analyzed-the-compas-recidivism-algorithm)). In order to replicate ProPublica's analysis, we remove any cases where the charge was not within 30 days of the score (ProPublica did this in order to match the COMPAS score with the correct criminal case). We are left with 6172 rows in the dataset.

```python
data = pd.read_csv('compas-scores-two-years.csv')
data = data.query('days_b_screening_arrest <= 30 & days_b_screening_arrest >= -30')
data
```

      id
      name
      first
      last
      compas_screening_date
      sex
      dob
      age
      age_cat
      race
      ...
      v_decile_score
      v_score_text
      v_screening_date
      in_custody
      out_custody
      priors_count.1
      start
      end
      event
      two_year_recid

      0
      1
      miguel hernandez
      miguel
      hernandez
      2013-08-14
      Male
      1947-04-18
      69
      Greater than 45
      Other
      ...
      1
      Low
      2013-08-14
      2014-07-07
      2014-07-14
      0
      0
      327
      0
      0

      1
      3
      kevon dixon
      kevon
      dixon
      2013-01-27
      Male
      1982-01-22
      34
      25 - 45
      African-American
      ...
      1
      Low
      2013-01-27
      2013-01-26
      2013-02-05
      0
      9
      159
      1
      1

      2
      4
      ed philo
      ed
      philo
      2013-04-14
      Male
      1991-05-14
      24
      Less than 25
      African-American
      ...
      3
      Low
      2013-04-14
      2013-06-16
      2013-06-16
      4
      0
      63
      0
      1

      5
      7
      marsha miles
      marsha
      miles
      2013-11-30
      Male
      1971-08-22
      44
      25 - 45
      Other
      ...
      1
      Low
      2013-11-30
      2013-11-30
      2013-12-01
      0
      1
      853
      0
      0

      6
      8
      edward riddle
      edward
      riddle
      2014-02-19
      Male
      1974-07-23
      41
      25 - 45
      Caucasian
      ...
      2
      Low
      2014-02-19
      2014-03-31
      2014-04-18
      14
      5
      40
      1
      1

      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...
      ...

      7209
      10996
      steven butler
      steven
      butler
      2013-11-23
      Male
      1992-07-17
      23
      Less than 25
      African-American
      ...
      5
      Medium
      2013-11-23
      2013-11-22
      2013-11-24
      0
      1
      860
      0
      0

      7210
      10997
      malcolm simmons
      malcolm
      simmons
      2014-02-01
      Male
      1993-03-25
      23
      Less than 25
      African-American
      ...
      5
      Medium
      2014-02-01
      2014-01-31
      2014-02-02
      0
      1
      790
      0
      0

      7211
      10999
      winston gregory
      winston
      gregory
      2014-01-14
      Male
      1958-10-01
      57
      Greater than 45
      Other
      ...
      1
      Low
      2014-01-14
      2014-01-13
      2014-01-14
      0
      0
      808
      0
      0

      7212
      11000
      farrah jean
      farrah
      jean
      2014-03-09
      Female
      1982-11-17
      33
      25 - 45
      African-American
      ...
      2
      Low
      2014-03-09
      2014-03-08
      2014-03-09
      3
      0
      754
      0
      0

      7213
      11001
      florencia sanmartin
      florencia
      sanmartin
      2014-06-30
      Female
      1992-12-18
      23
      Less than 25
      Hispanic
      ...
      4
      Low
      2014-06-30
      2015-03-15
      2015-03-15
      2
      0
      258
      0
      1

6172 rows × 53 columns

We are also able to filter out any information that was not used by ProPublica and select fields for severity of charge, number of priors, demographics, age, sex, compas scores, and whether each person was accused of a crime within two years.

```python
select_data = data[["age", "c_charge_degree", "race", "age_cat", "score_text", "sex", "priors_count",
                    "days_b_screening_arrest", "decile_score", "is_recid", "two_year_recid", "c_jail_in", "c_jail_out"]]
```

### Question 0c
Explore the dataset. What is the granularity of this dataset?

*Student Written Answer Here*

***Sensitive features*** are attributes within a dataset that are given special consideration and treatment for potential legal, social, or ethical reasons. Often, these features are recognized and protected by antidiscrimination or privacy laws. One example of a sensitive feature is age.

### Question 0d
Identify 2 sensitive features in the dataset that we have not already mentioned.

*Student Written Answer Here*

### Question 0e
Pick one of the sensitive features you have identified. Identify at least 2 features in the dataset that are proxies for that sensitive feature.

*Student Written Answer Here*

### Question 0f
As a data scientist, why is it important to give special consideration to these kinds of features?

*Student Written Answer Here*

# Part 1. ProPublica’s Perspective

### Who is ProPublica?

ProPublica is a nonprofit organization that “produces investigative journalism with moral force” ([ProPublica](https://www.propublica.org/about/)). ProPublica was founded as a nonpartisan newsroom aiming to expose and question abuses of power, justice, and public trust, often by systems and institutions deeply ingrained in the US.

In 2016, ProPublica investigated the COMPAS algorithm to assess the accuracy of and potential racial bias within the tool, as it became more popular within the United States court system nationwide. In their analysis, ProPublica used data from defendants with risk scores from Broward County, FL from 2013 to 2014 to test for statistical differences in outcomes for Black and white defendants, which ultimately highlighted racial disparities that exist within the algorithm. ProPublica came to the conclusion that COMPAS utilizes data from a criminal justice system with a history of racial injustices, thus continuing to disproportionately target and arrest Black people in comparison to their white counterparts. While the COMPAS algorithm treats unequal groups alike, which may appear neutral, ProPublica’s data analysis and reporting emphasized the bias against Black defendants and their communities that COMPAS produced from this line of thinking, a claim that Northpointe has disputed (as we will see later).

Let's retrace ProPublica's statistical analysis in order to better understand ProPublica's argument and engage with the metric of fairness that it uses.

## Question 1. Logistic Regression: What are the odds of getting a high risk score?

ProPublica’s first attempt at understanding the disparity in risk scores from the COMPAS tool was through logistic regression to model the chance of getting a “higher” (i.e. more "risky") score. COMPAS labels scores 1-4 as low, 5-7 as medium, and 8-10 as high scores. For the purposes of their analysis, ProPublica labeled any score above a low score as high.

### Question 1a (i)
Create a logistic regression model to predict the score of defendants based on their sex, age, race, previous arrests, seriousness of the crime, and future criminal behavior.

```python
# Create logistic regression model
# Student Code Here
```

### Question 1a (ii)
Print out the coefficients paired with the corresponding feature names.

```python
# Pair the coefficients with feature names
# Student Code Here
```

### Question 1b
What features are most predictive?

*Student Written Answer Here*

### Question 1c
Are Black defendants more likely to get a high risk score opposed to white defendants? If so, by how much? Show your calculations.

```python
# Student Calculations Here
```

*Student Written Answer Here*

## Question 2. FPR and FNR: Does COMPAS overpredict or underpredict across groups?

In order to answer this question and understand the ways in which bias is present in the risk scores, ProPublica used the False Positive Rate (FPR) and False Negative Rate (FNR) as their metrics to understand and quantify fairness.

### Question 2a
Complete the following functions to calculate the FPR and FNR. Afterwards, apply these functions to each racial subgroup: Black defendants and white defendants. Keep in mind that ProPublica defines a high score as anything above 4, and therefore a false positive would be a defendant with a high score who did not recidivate.

```python
def fpr(race_feature, data):
    # Return the False Positive Rate of scores for the specified race_feature

    return 0

def fnr(race_feature, data):
    # Return the False Negative Rate of scores for the specified race_feature

    return 0

```

```python
# Apply the metrics to the dataset - Replace zeros with correct metrics
print("FPR for Black defendants:", 0)
print("FPR for white defendants:", 0)
print("FNR for Black defendants:", 0)
print("FNR for white defendants:", 0)
```

    FPR for Black defendants: 0
    FPR for white defendants: 0
    FNR for Black defendants: 0
    FNR for white defendants: 0

### Question 2b
What can you conclude from these metrics about the overprediction of risk scores for Black and white defendants? By how much is the tool overpredicting?  (Hint: Look at your calculations for the FPR.)

```python
# Calculation of ratio for FPR
```

*Student Written Answer Here*

### Question 2c
What can you conclude from these metrics about the underprediction of risk scores for Black and white defendants? By how much is the tool underpredicting? (Hint: Look at your calculations for the FNR.)

```python
# Calculation of ratio for FNR
```

*Student Written Answer Here*

### Question 2d
What is the importance of overprediction and underprediction in regard to ProPublica’s analysis? How might these observations have real impacts on the defendants who receive scores?

*Student Written Answer Here*

##  Question 3.

### Question 3a (i)
Utilizing your answers from 1b and 2b, what problems does ProPublica highlight in the COMPAS algorithm?

*Student Written Answer Here*

### Question 3a (ii)
How would you describe ProPublica’s definition of fairness, after learning and utilizing the metrics they used?

*Student Written Answer Here*

### Question 3b
Why did ProPublica choose to investigate bias between races rather than a different sensitive feature? (Hint: think about how ProPublica’s conclusions reflect the racial disparities in our current criminal justice system.)

*Student Written Answer Here*

### Question 3c
What is ProPublica’s agenda as an investigative journalism organization? How do we see this in their analysis and conclusions?

*Student Written Answer Here*

We mentioned earlier that Northpointe disagreed with ProPublica's argument that the COMPAS algorithm is racially biased. Now that we’ve analyzed ProPublica’s perspective and seen the way in which they define and operationalize the concept of fairness, let’s move on to Northpointe’s.

# Part 2. Northpointe's Perspective

### Who is Northpointe?

Northpointe (merged with two other companies to create *equivant* in 2017) is a for-profit computer software company that aims to advance justice by informing and instilling confidence in decision makers at every stage of the criminal justice system ([equivant](https://www.equivant.com/)). In addition to operating and continuing to develop COMPAS, *equivant* has developed a variety of technologies for use in court case management, attorney case management, inmate classification, and risk/needs assessment strategies.

In the wake of criticism from ProPublica and other researchers alike, Northpointe produced a [detailed response](http://go.volarisgroup.com/rs/430-MBX-989/images/ProPublica_Commentary_Final_070616.pdf) to ProPublica’s allegations, claiming that these critiques of their tool utilized the wrong type of classification statistics in their analysis and portrayed the tool incorrectly. The company provided their own analysis of the COMPAS algorithm by using different statistical methods and responding individually to each of ProPublica’s claims of racial bias against Black defendants.

Upon examining their tool’s fairness through accuracy equity and predictive parity (which are metrics that were left out of ProPublica’s analysis), as well as the fact that the model was not trained with a race feature, Northpointe concluded that their algorithm treats all citizens and specified groups equally, and therefore does not exhibit signs of bias or inequality for specified groups. Now, let’s take a look at how Northpointe supported this argument.

## Question 4. Accuracy Equity: Is each group being discriminated against equally?

Instead of analyzing and comparing the model errors FNR and FPR, Northpointe utilized the complement of FNR, known as the TPR (or what is often referred to as *Sensitivity*), paired with the FPR to prove what they refer to as ***Accuracy Equity*** through the use of a *ROC Curve*. Accuracy equity, according to [Northpointe](http://go.volarisgroup.com/rs/430-MBX-989/images/ProPublica_Commentary_Final_070616.pdf), is exhibited in the model “if it can discriminate recidivists and nonrecidivists equally well for two different groups such as blacks and whites.” Recall that we use ROC curves and the *Area Under the Curve* to understand how much a model is capable of distinguishing between classes.

### Question 4a
Utilize the sklearn metrics package to calculate TPR and FPR, visualize the ROC curve for both white and Black defendants, and calculate the AUC for each curve.

```python
# Calculate FPR and FNR from metrics package - Black defendants

# Calculate FPR and FNR from metrics package - White defendants

# Plot the ROC
plt.subplots(1, figsize=(10,10))
plt.title('ROC - Black Defendents')
plt.ylabel('Sensitivity')
plt.xlabel('1 - Specificity')
plt.plot([0, 1], ls="--")
plt.show()

```

```python
# Calculate the AUC - Replace zeros with correct AUC's
print("AUC for Black defendants:", 0)
print("AUC for white defendants:", 0)
```

    AUC for Black defendants: 0
    AUC for white defendants: 0

### Question 4b (i)
What do you notice from the ROC curve and the AUC calculation? List at least two general observations.

*Student Written Answer Here*

### Question 4b (ii)
What could Northpointe take away from this visualization to prove their point? Is accuracy equity being represented here? (Hint: Is each racial group being discriminated against equally?)

*Student Written Answer Here*

## Question 5. Predictive Parity: Is the likelihood of recidivism equal across groups?

In addition to the metric outlined above, Northpointe also utilized positive predictive values to explore the likelihood of defendants to reoffend, and to therefore prove that ***Predictive Parity*** is achieved. Predictive parity, according to [Northpointe](http://go.volarisgroup.com/rs/430-MBX-989/images/ProPublica_Commentary_Final_070616.pdf), is exhibited in a model “if the classifier obtains similar predictive values for two different groups such as blacks and whites, for example, the probability of recidivating, given a high risk score, is similar for blacks and whites.” Let’s explore how they analyzed this.

### Question 5a

Complete the following functions to calculate the positive predictive values and negative predictive values. Afterwards, apply these functions to the data of white defendants and the data of Black defendants.

```python
def ppv(race_feature, data):
    # Return the Positive Predictive Value of scores for the specified race_feature

    return 0

def npv(race_feature, data):
    # Return the Negative Predictive Value of scores for the specified race_feature

    return 0

```

```python
# Apply metrics to the dataset - Replace zeros with correct metrics
print("PPV for Black defendants:", 0)
print("NPV for Black defendants:", 0)
print("PPV for white defendants:", 0)
print("NPV for white defendants:", 0)
```

    PPV for Black defendants: 0
    NPV for Black defendants: 0
    PPV for white defendants: 0
    NPV for white defendants: 0

### Question 5b
Use the metrics you calculated above to fill in the table below.

| |  White  | Black    |
|---:|:-------------|:-----------|
| Labeled higher risk, but didn't reoffend | % | % |
| Labeled lower risk, but did reoffend | % | % |

```python
# Student calculations for above table
```

### Question 5c (i)
What do you notice about the positive predictive values for each group? List at least one general observation.

*Student Written Answer Here*

### Question 5c (ii)
What could Northpointe conclude from these findings? Is predictive parity represented here? (Hint: Is the likelihood of recidivism relatively equal for each racial group?)

*Student Written Answer Here*

## Question 6.

### Question 6a
How would you describe Northpointe’s definition of fairness, after learning and utilizing the metrics they used? How is this different from your description of ProPublica’s definition from Q3aii?

*Student Written Answer Here*

### Question 6b

If anything, what are ProPublica and Northpointe each not considering in their definitions? (Hint: Think about other goodness metrics in ML, as well as your knowledge of the historical context of policing data)

*Student Written Answer Here*

So far, we’ve investigated ProPublica’s and Northpointe’s definitions of fairness. In the world of machine learning there are [many more](https://www.google.com/url?q=https://fairmlbook.org/tutorial2.html&sa=D&ust=1606727018134000&usg=AOvVaw06zU_fm8h7xp71d8igA8KI), so in the next section we will take a look at a third definition.

# Part 3. Yet Another Definition of Fairness

In this section, you will go through yet another metric and definition used to evaluate fairness in machine learning: **disparate impact**. Disparate impact is a legal doctrine that determines if there is unintended discrimination towards a protected class ([Society for Human Resource Management](https://www.shrm.org/resourcesandtools/tools-and-samples/hr-qa/pages/disparateimpactdisparatetreatment.aspx)). In machine learning, disparate impact is a metric to evaluate fairness in a model. It is a form of bias within an algorithm that reflects systemic discrimination when a model’s outputs are dependent on a ***sensitive feautre*** (the protected class). This is often considered unintentional (like the legal doctrine) due to the fact that the sensitive feature is omitted from the model, though it is still correlated with the output through proxy variables ([Wang et al.](https://arxiv.org/pdf/1801.05398.pdf#:~:text=Abstract%E2%80%94In%20the%20context%20of,e.g.%2C%20race%20or%20gender)).

Not only will you evaluate the fairness of the tool (as Northpointe and ProPublica did) by measuring the bias reflected in the outputs of the model, but you will remove it to actually change those outputs and therefore eliminate the dependencies between the risk scores and the race feature. In order to computationally remove the disparate impact that we quantify, we can use tools like aif360’s [Disparate Impact Remover](https://aif360.readthedocs.io/en/latest/modules/generated/aif360.algorithms.preprocessing.DisparateImpactRemover.html). aif360 is a package created by IBM’s AI Research team, which contains a variety of tools to “help you examine, report, and mitigate discrimination and bias in machine learning models throughout the AI application lifecycle” ([AI Fairness 360](https://aif360.mybluemix.net/)).

## Question 7. Disparate Impact: Quantification and Removal

First, let’s visualize the disparity that we would like to remove from the dataset. In order to do that we need to distinguish between a privileged group and an unprivileged group. In technical terms, the privileged group receives higher scores from a trained model, so therefore the Black defendants will be considered "privileged" and the white defendants will be considered "unprivileged" in this case.

### Question 7a

Use a histogram to plot the scores for Black defendants and the scores for white defendants. Visualize both histograms on the same plot.

```python
# Student Code for Histogram
```

### Question 7b

What do you notice from the plot? (Hint: how do the distributions differ across racial groups with respect to mean, shape of distribution, etc.)

*Student Written Answer Here*

Now, we need to quantify the disparate impact we are seeing in the plot. In machine learning, we can understand disparate impact as the proportion of individuals that get positive outcomes (did they get a high score) for the two groups described above:

\begin{align}
\Pr(Y=1|D=Unprivileged) \ / \ \Pr(Y=1|D=Privileged)
\end{align}

In this equation Y is 1 if the defendant received a high score and 0 if they received a low score.

### Question 7c (i)

Create a function to calculate the proportion of individuals from a specified racial group that get positive outcomes.

```python
def proportion(data, group):
    # Returns the proportion of individuals in data from the group who recidivate

    return 0
```

### Question 7c (ii)

Use this function to calculate the disparate impact, using the equation from above.

```python
# Student Code Here
```

If the proportion of unprivileged individuals receiving positive outcomes to privileged individuals receiving positive outcomes is less than 80%, there is a disparate impact violation. In order to stop a trained model from replicating these biases in its output, we can now use aif360’s Disparate Impact Remover to remove the bias we just calculated.

### Question 7d

Create a Disparate Impact Remover type object and use the function fit_transform on our data. In order to do this you will need to first create a BinaryLabelDataset from our dataset to use for fit_transform. Check out the documentation [here](https://aif360.readthedocs.io/en/latest/modules/generated/aif360.datasets.BinaryLabelDataset.html#aif360.datasets.BinaryLabelDataset) to see how to implement this.

```python
# Create new DataFrame with just the necessary columns - Only numeric values for race, decile score, and two year recid

# The BinaryLabelDataset requires decile_scores to be continuous -->
# Use this line of code noise = np.random.normal(0, 0.1, race_data.shape[0]) to add noise to your decile_score column
# Student Code Here
```

### Question 7e

Similar to part a, use a histogram to plot the scores on the modified dataset. Afterwards, use the proportion function created above to calculate the disparate impact of the transformed dataset.

```python
# Transform output from DIRemover into usable DataFrame

# Plot the new overlayed histogram
```

```python
# Student Calculation Here
```

### Question 7f

What has changed from our original histogram? Please explain why this change has happened.

*Student Written Answer Here*

### Question 7g

How would you describe this third definition of fairness, after learning and utilizing these new metrics?

*Student Written Answer Here*

### Question 7h
How does this definition of fairness differ from ProPublica’s and Northpointe’s?

*Student Written Answer Here*

## Question 8. Considering Expertise Outside of Data Science

Just now, you used your technical data science skills to computationally remove bias from the data set. By removing bias, we’ve made the outputs of the algorithm statistically fair in regards to one definition of fairness. However, it is important to consider many types of knowledge and experiences beyond data science expertise when analyzing and creating an algorithm like COMPAS. As such, you will think through issues of expertise and fairness in the next set of questions.

### Question 8a

Look back to your answer from Q0a. Now that you’ve gone through several definitions of fairness, how would you add to or revise your answer: Explain 3 parties that are impacted by the COMPAS tool. In what ways are they impacted?

*Student Written Answer Here*

### Question 8b

What expertise and lived experiences are necessary to understand and critically think about the issues produced by COMPAS?

*Student Written Answer Here*

### Question 8c

Why is this third definition of fairness still inadequate as a measurement of justice in the court system? (Hint: look at the previous two questions and answers).

*Student Written Answer Here*

 # Part 4. Conclusion

## Question 9. Which Definition Is Fair? And Who Decides?

We’ve now gone through three definitions of fairness, each one with a different idea of how to operationalize fairness and to judge whether or not an algorithm is fair. As a data scientist, you may encounter situations where you will need to make decisions that affect real-world outcomes and people! Let’s try to do this for COMPAS.

### Question 9a
If you had to decide between the three definitions of fairness above, which definition do you think would make “fair” decisions for everyone who goes through the court system? What values did you consider as you made this decision? If you cannot come to a decision, what challenges did you come across when considering this?

*Student Written Answer Here*

### Question 9b
Take a step back and think about how different actors who created, utilize, and are affected by COMPAS would consider which definition is most fair. Name two relevant actors, and discuss what they would value in *their own* definitions of fairness. Of the three definitions you have explored, which would they decide is most fair from the perspective of that actor? If you don’t think they’d choose any of them, explain why. (Examples of actors, which you’re welcome to use: judges, defendants, police, policy makers, community members)

*Student Written Answer Here*

Choosing one definition of fairness can be incredibly difficult when you need to consider all the actors at play. Throughout this module we have examined where and how the COMPAS algorithm is appropriate to use. It is also important to recognize the problems that are not solvable by an algorithm and think through how we can make the ecosystem that COMPAS is in  (which includes but is not limited to the legal system, affected communities, the tech industry, etc.) more equitable.

### Question 9c
What issues that are relevant to the COMPAS ecosystem but outside of the algorithm itself need to be addressed to be able to create a more equitable system, with or without the algorithm?

*Student Written Answer Here*

You’ve now begun to think through the very complex systems in which the COMPAS algorithm functions. **Congratulations!** Through considering a few of the differing definitions of fairness connected to COMPAS, hopefully you can begin to understand some of the human contexts of  creating algorithms that intentionally affect people and their decision-making.

```python

```

COMPAS/.ipynb_checkpoints/COMPAS Project - No Solutions-checkpoint.ipynb


# A minimal test that DocOnce works

**Hans Petter Langtangen**

Date: **Jan 1, 2031**

**Summary.** This is a small document for testing the installation of DocOnce.

# Test of plain text

## Test standard text with lists and inline tagging

Here are some DocOnce features:

  * DocOnce addresses small and large documents containing
    *text with much computer source code and
    LaTeX mathematics*, where the output is desired in different formats
    such as LaTeX, pdfLaTeX, Sphinx, HTML,
    MediaWiki, blogger.com, and wordpress.com.
    A piece of DocOnce text can enter (e.g.) a classical
    science book, an ebook, a web document, and a blog post.

  * DocOnce offers a range of HTML designs, including many
    Bootstrap and Sphinx styles and [solarized color schemes](http://ethanschoonover.com/solarized).
    A special feature is the many styles for **admonitions** (boxes for
    warning, notice, question, etc.) in HTML and LaTeX.

  * DocOnce targets in particular large book projects where many different
    pieces of text and software can be assembled in different ways
    and published in different formats for different devices
    (see [example](http://hplgit.github.io/setup4book-doconce/doc/web/index.html)).

  * DocOnce enables authors who write for many times of media
    (blog posts, wikis, LaTeX manuscripts, Sphinx, HTML) to use a common
    source language such that lots of different pieces can easily be
    brought together later to form a coherent (big) document.

  * DocOnce has good support for copying computer code
    directly from the source code files via regular expressions
    for the start and end lines.

  * DocOnce first runs two preprocessors (Preprocess and Mako), which
    allow programming constructs (includes, if-tests, function calls)
    as part of the text. This feature makes it easy to write *one text*
    with different flavors: long vs short text, Python vs Matlab code
    examples, experimental vs mature content.

  * DocOnce can be converted to plain *untagged* text,
    often desirable for email and computer code documentation.

  * DocOnce markup does include tags, so the format is more tagged than
    Markdown, but less than reST, and very much less than
    LaTeX and HTML.

  * Compared to the related tools Sphinx and Markdown, DocOnce
    allows more types of equations (especially systems of
    equations with references), has more flexible
    inclusion of source code, integrates preprocessors, has
    special support for exercises, and produces
    cleaner LaTeX and HTML output.

## Test of figures and movies

Inline figure:

[Figure](#fig1) has a caption.

Figure with caption.

Here is a YouTube movie:

```
from IPython.display import HTML
_s = """

"""
HTML(_s)
```

```
_s = """
<div>

</div>
<p><em>Movie on the hard disk: 1D wave motion.</em></p>

<!-- Issue warning if in a Safari browser -->

"""
HTML(_s)
```

# Test of math, code, admons, quiz

## Math

We have

$$
\begin{equation}
F = \int_a^b f(x)dx.
\label{_auto1} \tag{1}
\end{equation}
$$

## Code

We can do the integral by

```
from sympy import *
x = 'symbols('x')
f = x*sin(x)*exp(-x)
integrate(f, x(0, 4))
```

Or we can do it numerically via the Trapezoidal rule:

```
import numpy

def trapezoidal(f, a, b, n=100):
    """Integrate f from a to b with 100 intervals."""
    x = numpy.linspace(a, b, n+1)
    F = (b-a)/float(n)*(numpy.sum(f(x)) - 0.5*(f(a) + f(b)))
    return F

def f(x):
    return x*numpy.sin(x)*numpy.exp(-x)

print trapezoidal(f, 0, 4)
```

## Admonitions

**Question.**

How do adminitions look like? That depends on the output format and
what type of admon design in the format one has chosen.

**White space is important!**

Many DocOnce errors arise from wrong use of white space. The white space
is not as critical as in reStructuredText, but is not ignored either,
as in LaTeX and HTML.


[Home Page](../../Start_Here.ipynb)

[1]
[2](../diffusion_1d/Diffusion_Problem_Notebook.ipynb)
[3](../spring_mass/Spring_Mass_Problem_Notebook.ipynb)
[4](../chip_2d/Challenge_CFD_Problem_Notebook.ipynb)

[Next Notebook](../diffusion_1d/Diffusion_Problem_Notebook.ipynb)

# Introduction to Physics Informed Neural Networks (PINNs)

In this notebook we will see the advantages of Physics Informed modeling over data-driven modeling and will also outline the brief theory about Physics Informed Neural Networks (PINNs).

## PINNs vs Data-Driven Methods

The data-driven methods are very useful in applications where the data is already available to learn from, or in situations where the underlying physics is not known. For such methods, the generalization errors are dependent on the amount of training data. Usually, obtaining the training data from simulations is a resource-hungry process and can limit the amounts of data the model is trained on. This places restrictions on the quality of results obtained from the data-driven methods and it usually takes a large amount of training data and fine-tuning of the model to capture the physics better.

Since for most of the physical systems, we know the underlying governing equations (e.g. Navier Stokes equations for the fluid mechanics problems), we can make the neural networks more cognizant of these physical laws by incorporating these governing equations in training framework. In such cases we can train the models on lesser data, or even train completely without any data from a solver. Also, in such Physics-Informed Neural Networks, treatment of inverse problems and parameterized problems (e.g. problems with parameterized geometry/material properties/boundary conditions) is easy. Due to these advantages, we will now see how to apply PINNs for physics-based problems using the SimNet library.

Let's start with discussing the theory behind the PINNs briefly.

## Neural Network Solver Methodology

In this section we provide a brief introduction to solving differential equations with neural networks. The idea is to use a neural network to approximate the solution to the given differential equation and boundary conditions. We train this neural network by constructing a loss function for how well the neural network is satisfying the differential equation and boundary conditions. If the network is able to minimize this loss function then it will in effect, solve the given differential equation.

To illustrate this idea we will give an example of solving the following problem,

\begin{equation} \label{1d_equation}
    \mathbf{P} : \left\{\begin{matrix}
\frac{\delta^2 u}{\delta x^2}(x) = f(x), \\
\\
u(0) = u(1) = 0,
\end{matrix}\right.
\end{equation}

We start by constructing a neural network $u_{net}(x)$. The input to this network is a single value $x \in \mathbb{R}$ and its output is also a single value $u_{net}(x) \in \mathbb{R}$. We suppose that this neural network is infinitely differentiable, $u_{net} \in C^{\infty}$. The typical neural network used is a deep fully connected network where the activation functions are infinitely differentiable.

Next we need to construct a loss function to train this neural network. We easily encode the boundary conditions as a loss in the following way:

\begin{equation}
  L_{BC} = u_{net}(0)^2 + u_{net}(1)^2
\end{equation}

For encoding the equations, we need to compute the derivatives of $u_{net}$. Using automatic differentiation we can do so and compute $\frac{\delta^2 u_{net}}{\delta x^2}(x)$. This allows us to write a loss function of the form:

\begin{equation} \label{sumation_loss}
  L_{residual} = \frac{1}{N}\sum^{N}_{i=0} \left( \frac{\delta^2 u_{net}}{\delta x^2}(x_i) - f(x_i) \right)^2
\end{equation}

Where the $x_i$'s are a batch of points sampled in the interior, $x_i \in (0, 1)$. Our total loss is then $L = L_{BC} + L_{residual}$. Optimizers such as Adam are used to train this neural network. Given $f(x)=1$, the true solution is $\frac{1}{2}(x-1)x$. Upon solving the problem, you can obtain good agreement between the exact solution and the neural network solution as shown in Figure below.

## Parmeterized Problems

One important advantage of a neural network solver over traditional numerical methods is its ability to solve parameterized geometries. To illustrate this concept we solve a parameterized version of the above problem. Suppose we want to know how the solution to this equation changes as we move the position on the boundary condition $u(l)=0$. We can parameterize this position with a variable $l \in [1,2]$ and our equation now has the form,

\begin{equation} \label{1d_equation2}
    \mathbf{P} : \left\{\begin{matrix}
\frac{\delta^2 u}{\delta x^2}(x) = f(x), \\
\\
u(0) = u(l) = 0,
\end{matrix}\right.
\end{equation}

To solve this parameterized problem we can have the neural network take $l$ as input, $u_{net}(x,l)$. The losses then take the form,

\begin{equation}
  L_{residual} = \int_1^2 \int_0^l \left( \frac{\delta^2 u_{net}}{\delta x^2}(x,l) - f(x) \right)^2 dx dl \approx \left(\int_1^2 \int^l_0 dxdl\right) \frac{1}{N} \sum^{N}_{i=0} \left(\frac{\delta^2 u_{net}}{\delta x^2}(x_i, l_i) - f(x_i)\right)^2
\end{equation}

\begin{equation}
  L_{BC} = \int_1^2 (u_{net}(0,l))^2 + (u_{net}(l,l) dl \approx \left(\int_1^2 dl\right) \frac{1}{N} \sum^{N}_{i=0} (u_{net}(0, l_i))^2 + (u_{net}(l_i, l_i))^2
\end{equation}

In figure below we see the solution to the differential equation for various $l$ values after optimizing the network on this loss. While this example problem is overly simplistic, the ability to solve parameterized geometries presents significant industrial value. Instead of performing a single simulation we can solve multiple designs at the same time and for reduced computational cost. More examples of this can be found in *SimNet User Guide Chapter 13*.

## Inverse Problems

Another useful application of a neural network solver is solving inverse problems. In an inverse problem, we start with a set of observations and then use those observations to calculate the causal factors that produced them. To illustrate how to solve inverse problems with a neural network solver, we give the example of inverting out the source term $f(x)$ from same equation from above problem. Suppose we are given the solution $u_{true}(x)$ at 100 random points between 0 and 1 and we want to determine the $f(x)$ that is causing it. We can do this by making two neural networks $u_{net}(x)$ and $f_{net}(x)$ to approximate both $u(x)$ and $f(x)$. These networks are then optimized to minimize the following losses;

\begin{equation}
  L_{residual} \approx \left(\int^1_0 dx\right) \frac{1}{N} \sum^{N}_{i=0} \left(\frac{\delta^2 u_{net}}{\delta x^2}(x_i, l_i) - f_{net}(x_i)\right)^2
\end{equation}

\begin{equation}
  L_{data} = \frac{1}{100} \sum^{100}_{i=0} (u_{net}(x_i) - u_{true}(x_i))^2
\end{equation}

Using the function $u_{true}(x)=\frac{1}{48} (8 x (-1 + x^2) - (3 sin(4 \pi x))/\pi^2)$ the solution for $f(x)$ is $x + sin(4 \pi x)$. We solve this problem and compare the results in Figures below.

Comparison of true solution for $f(x)$ and the function approximated by the NN:

Comparison of $u_{net}(x)$ and the train points from $u_{true}$:

More examples of solving an inverse problem can be found in the *SimNet User Guide Chapter 12*.

```python
from IPython.display import IFrame, display
#filepath = "http://wikipedia.org" # works with websites too!
filepath = "SimNet_v21.06_User_Guide.pdf"
IFrame(filepath, width=700, height=400)
```

# Licensing
This material is released by NVIDIA Corporation under the Creative Commons Attribution 4.0 International (CC BY 4.0)

[Home Page](../../Start_Here.ipynb)

[1]
[2](../diffusion_1d/Diffusion_Problem_Notebook.ipynb)
[3](../spring_mass/Spring_Mass_Problem_Notebook.ipynb)
[4](../chip_2d/Challenge_CFD_Problem_Notebook.ipynb)

[Next Notebook](../diffusion_1d/Diffusion_Problem_Notebook.ipynb)


```python
import numpy as np
np.set_printoptions(suppress=True)
import math
import matplotlib.pyplot as plt
import sympy
from matplotlib.patches import Ellipse, Arc
from matplotlib import cm, colors, patches
```

```python
def Draw_truss(Nodes,Element,TotalNodes,TotalElements,x_min,x_max,y_min,y_max,R,mr):
    fig = plt.figure(figsize=(12,9))
    ax = fig.add_subplot(1,1,1)
    L1,L2 = (x_max-x_min)/6,(y_max-y_min)/6
    L = min(L1,L2)

    for i in range(TotalElements):
        n1,n2 = Element[i]['From_To'][0], Element[i]['From_To'][1]
        X = [Nodes[n1]['coordinates'][0],Nodes[n2]['coordinates'][0]]
        Y = [Nodes[n1]['coordinates'][1],Nodes[n2]['coordinates'][1]]
        ax.plot(X,Y,linewidth=1,color='b')

        ### ELEMENT ###
        x0,y0 = (X[0]+X[1])/2,(Y[0]+Y[1])/2
        ax.text(x0-0.1*L1, y0+0.1*L2, "E"+str(i+1))

    if mr==1:
        print("\nExternal Forces\n")
    if mr==2:
        print("\nReaction Forces\n")

    for i in range(TotalNodes):
        x,y = Nodes[i]['coordinates'][0],Nodes[i]['coordinates'][1]

        if Nodes[i]['joint'] == 'h':
            ax.plot(x,y,marker='^',color='orange',markersize=18)
        elif Nodes[i]['joint'] == 'r':
            ax.plot(x,y,marker='o',color='orange',markersize=15)
        elif Nodes[i]['joint'] == 'f':
            ax.plot(x,y,marker='s',color='orange',markersize=15)
        else:
            ax.plot(x,y,marker='s',color='k',markersize=6)
        if mr==1:
            Fx,Fy,Mz = round(Nodes[i]['external_force'][0],3),round(Nodes[i]['external_force'][1],3),round(Nodes[i]['external_force'][2],3)
        if mr==2:
            Fx,Fy,Mz = round(R[3*i,0],3),round(R[3*i+1,0],3),round(R[3*i+2,0],3)
        dx,dy = 0.5*L1,0.5*L2
        head_width = 0.2*L
        x1,y1 = head_width+1.1*dx,head_width+1.1*dy
        if Fx<0:
            ax.arrow(x+x1, y, -dx, 0, head_width = head_width, width = 0.03*L2, ec ='green')
            ax.text(x+x1, y+0.1*L2, "Fx = "+str(Fx))
        if Fx>0:
            ax.arrow(x-x1, y, dx, 0, head_width = head_width, width = 0.03*L2, ec ='green')
            ax.text(x-2*x1, y+0.1*L2, "Fx = "+str(Fx))
        if Fy<0:
            ax.arrow(x, y+y1, 0, -dy, head_width = head_width, width = 0.03*L1, ec ='green')
            ax.text(x, y+y1*1.1, "Fy = "+str(Fy))
        if Fy>0:
            ax.arrow(x, y-y1, 0, dy, head_width = head_width, width = 0.03*L1, ec ='green')
            ax.text(x, y-y1*1.1, "Fy = "+str(Fy))
        r1,r2,r3 = 0.9*L1*0.8,0.9*L2*0.8,0.45*L2*0.8
        if Mz>0 and Fy<=0:
            ellipse = Arc([x,y],r1,r2,0,0,270,color='green', linewidth=1)
            ax.add_patch(ellipse)
            ax.plot(x,y-r3,marker='>',color='green',markersize=8)
            ax.text(x, y-(r3*1.3), "Mz = "+str(Mz))
        if Mz>0 and Fy>0:
            ellipse = Arc([x,y],r1,r2,0,180,450,color='green', linewidth=1)
            ax.add_patch(ellipse)
            ax.plot(x,y+r3,marker='<',color='green',markersize=8)
            ax.text(x, y+(r3*1.3), "Mz = "+str(Mz))
        if Mz<0 and Fy<=0:
            ellipse = Arc([x,y],r1,r2,0,-90,180,color='green', linewidth=1)
            ax.add_patch(ellipse)
            ax.plot(x,y-r3,marker='<',color='green',markersize=8)
            ax.text(x, y-(r3*1.3), "Mz = "+str(Mz))
        if Mz<0 and Fy>0:
            ellipse = Arc([x,y],r1,r2,0,90,360,color='green', linewidth=1)
            ax.add_patch(ellipse)
            ax.plot(x,y+r3,marker='>',color='green',markersize=8)
            ax.text(x, y+(r3*1.3), "Mz = "+str(Mz))

        ### NODE ###
        ax.text(x+0.2*L1, y+0.2*L2, "N"+str(i+1))

        ### FORCE ###

    ax.set_ylim(y_min-2*L2, y_max+2*L2)
    ax.set_xlim(x_min-2*L1,x_max+2*L1)
    plt.show()
```

```python
def main():
    ##### 2D FEM FOR TRUSS #####

    TotalNodes = int(input("Enter total number of nodes: "))
    dof = 3*TotalNodes

    ### DEFINING NODE QUANTITIES ###

    Nodes = {}
    freeNode = []
    F = []
    x_min,x_max,y_min,y_max = 1e10,-1e10,1e10,-1e10

    for i in range(TotalNodes):
        print()
        X = input("Enter (x,y) coordinates of node-"+str(i+1)+" (Example: 2,-3.7): ").split(',')
        support = input("Enter support type at node-"+str(i+1)+" ---> write 'P/p' for Planar and 'I/i' for Inclined: ")
        if support.upper() == 'P':
            angle = 0
        elif support.upper() == 'I':
            angle = float(input("Enter inclination angle in degrees (Example: 45.3): "))
        joint = input("Enter constraint type at node-"+str(i+1)+" ---> write 'f' for Fixed, 'fr' for Free, 'h' for hinged and 'r' for Roller support: ")
        if joint == 'fr':
            freeNode.append(3*i)
            freeNode.append(3*i+1)
            freeNode.append(3*i+2)
            Ext_F = input("Enter external force (Fx,Fy,Mz) at node-"+str(i+1)+" (Example: 1.2,-2,4.76 or 0,2.75,0): ").split(',')
            F.append(float(Ext_F[0]))
            F.append(float(Ext_F[1]))
            F.append(float(Ext_F[2]))
        elif joint == 'r':
            freeNode.append(3*i)
            freeNode.append(3*i+2)
            Ext_Fx = input("Enter external force (Fx,Mz) at node-"+str(i+1)+" (Example: 1.2,-3.7): ").split(',')
            Ext_F = [Ext_Fx[0],'0',Ext_Fx[1]]
            F.append(float(Ext_F[0]))
            F.append(float(Ext_F[2]))
        elif joint == 'h':
            freeNode.append(3*i+2)
            Ext_Fx = input("Enter external force Mz at node-"+str(i+1)+" (Example: 0.75): ")
            Ext_F = ['0','0',Ext_Fx]
            F.append(float(Ext_F[2]))
        else:
            Ext_F = ['0','0','0']

        Nodes[i] = {'coordinates':[float(X[0]),float(X[1])],
                    'angle': angle,
                    'joint': joint,
                    'external_force': [float(Ext_F[0]),float(Ext_F[1]),float(Ext_F[2])]}

        if x_min>Nodes[i]['coordinates'][0]:
            x_min = Nodes[i]['coordinates'][0]
        if y_min>Nodes[i]['coordinates'][1]:
            y_min = Nodes[i]['coordinates'][1]
        if x_max<Nodes[i]['coordinates'][0]:
            x_max = Nodes[i]['coordinates'][0]
        if y_max<Nodes[i]['coordinates'][1]:
            y_max = Nodes[i]['coordinates'][1]

    print()

    ### NODE ELEMENT RELATIONSHIP TABLE ###

    TotalElements = int(input("Enter total number of elements: "))

    Element = {}

    for i in range(TotalElements):
        print()
        FromTo = input("Enter (From_node, To_node) for element-"+str(i+1)+" (Example: 2,3): ").split(',')
        E = sympy.sympify(input("Enter Young's modulus of element-"+str(i+1)+" (Example: 2.27e10 if E = 2.27*10^10 or 2.7e10*(2**(0.5)) if E = 2.7*sqrt(2)*10^10): "))
        A = sympy.sympify(input("Enter area of element-"+str(i+1)+" (Example: 2.27e-3 if A = 2.27*10^-3 or 2.7*(2**(0.5)) if A = 2.7*sqrt(2)): "))
        I = sympy.sympify(input("Enter Izz of element-"+str(i+1)+" (Example: 2.27e-3 if I = 2.27*10^-3 or 2.7*(2**(0.5)) if I = 2.7*sqrt(2)): "))
        X,Y = np.array(Nodes[int(FromTo[0])-1]['coordinates']), np.array(Nodes[int(FromTo[1])-1]['coordinates'])
        L,Z = np.linalg.norm(X-Y),Y-X
        theta = np.arctan2(Z[1], Z[0])
        s,c = math.sin(theta),math.cos(theta)

        ### LOCAL STIFNESS MATRIX ###

        c1 = A*c*c + (12*I*s*s/(L*L))
        c2 = (A - (12*I/(L*L)))*c*s
        c3 = 6*I*s/L
        c4 = A*s*s + (12*I*c*c/(L*L))
        c5 = 6*I*c/L

        k = np.array([[c1, c2, -c3, -c1, -c2, -c3],
                      [c2, c4, c5, -c2, -c4, c5],
                      [-c3, c5, 4*I, c3, -c5, 2*I],
                      [-c1, -c2, c3, c1, c2, c3],
                      [-c2, -c4, -c5, c2, c4, -c5],
                      [-c3, c5, 2*I, c3, -c5, 4*I]])
        k = k*(E/L)
        Element[i] = {'s':s,
                      'c':c,
                      'L':L,
                      'A':A,
                      'E':E,
                      'I':I,
                      'k':k,
                      'From_To':[int(FromTo[0])-1,int(FromTo[1])-1]}

    ### ASSEMBLE GLOBAL STIFFNESS MATRIX ###
    print()

    K = np.zeros((dof,dof))
    for i in range(TotalElements):
        n1,n2 = Element[i]['From_To'][0], Element[i]['From_To'][1]
        k = Element[i]['k']
        print('LOCAL STIFFNESS MATRIX k'+str(i+1)+':\n')
        print(np.around(k.astype(np.double),4))
        print()
        map_local_to_global = {0:3*n1,
                               1:3*n1+1,
                               2:3*n1+2,
                               3:3*n2,
                               4:3*n2+1,
                               5:3*n2+2}
        for i in range(6):
            for j in range(6):
                K[map_local_to_global[i],map_local_to_global[j]] += k[i,j]

    print('GLOBAL STIFFNESS MATRIX K:\n')
    print(np.around(K,4))
    print()

    ### TRANSFORM GLOBAL STIFFNESS MATRIX ###

    T = np.zeros((dof,dof))
    for i in range(TotalNodes):
        alpha = (Nodes[i]['angle'])*np.pi/180
        s,c = math.sin(alpha),math.cos(alpha)
        t = np.array([[c,s,0],
                      [-s,c,0],
                      [0,0,1]])
        T[3*i:3*i+3,3*i:3*i+3] = t

    K = (T@K)@(T.T)
    print('TRANSFORMED GLOBAL STIFFNESS MATRIX K:\n')
    print(np.around(K,4))
    print()

    ### SOLVING KQ = F ###

    K_small = np.zeros((len(freeNode),len(freeNode)))

    map_to_small = {}
    for i in range(len(freeNode)):
        map_to_small[freeNode[i]] = i

    for i in range(dof):
        if i not in freeNode:
            continue
        for j in range(dof):
            if j not in freeNode:
                continue
            K_small[map_to_small[i],map_to_small[j]] = K[i,j]

    F = np.array(F).reshape(len(freeNode),1)
    q = (np.linalg.inv(K_small))@F

    Q = np.zeros((dof,1))
    for i in range(dof):
        if i not in freeNode:
            continue
        Q[i,0] = q[map_to_small[i],0]

    print("DISPLACEMENT MATRIX Q:\n")
    print(np.around(Q,4))
    print()

    ### REACTION FORCES AND STRESSES ###

    Z = K@Q
    R = Z[:,:]
    for i in range(TotalNodes):
        if Nodes[i]['joint']=='fr':
            R[3*i,0],R[3*i+1,0],R[3*i+2,0] = 0,0,0
        elif Nodes[i]['joint']=='r':
            R[3*i,0],R[3*i+2,0] = 0,0
        elif Nodes[i]['joint']=='h':
            R[3*i+2,0] = 0
        else:
            pass

    print("REACTION FORCE R:\n")
    print(np.around(R,4))
    print()

    '''
    for i in range(TotalElements):
        n1,n2 = Element[i]['From_To'][0], Element[i]['From_To'][1]
        q = np.array([Q[2*n1,0],Q[2*n1+1,0],Q[2*n2,0],Q[2*n2+1,0]])
        c,s = Element[i]['c'],Element[i]['s']
        E,L = Element[i]['E'],Element[i]['L']
        C = np.array([-c,-s,c,s])
        S = np.dot(C,q)
        S = S*(E/L)
        if S<0:
            print("ELEMENT-"+str(i+1)+" IS UNDER COMPRESSIVE STRESS OF",S,"UNITS")
            print()
        elif S>0:
            print("ELEMENT-"+str(i+1)+" IS UNDER TENSILE STRESS OF",S,"UNITS")
            print()
        else:
            print("ELEMENT-"+str(i+1)+" IS STRESS FREE")
            print()
    '''
    Draw_truss(Nodes,Element,TotalNodes,TotalElements,x_min,x_max,y_min,y_max,R,1)
    Draw_truss(Nodes,Element,TotalNodes,TotalElements,x_min,x_max,y_min,y_max,R,2)
    print("\nAnalysis ends here!")

main()
```


Week 8 Notebook: Extending the Model
===============================================================

This week, we will look at graph neural networks using the PyTorch Geometric library: <https://pytorch-geometric.readthedocs.io/>. See {cite:p}`PyTorchGeometric` for more details.

```python
import torch
import torch_geometric
device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
from tqdm.notebook import tqdm
import numpy as np
local = False
```

```python
import yaml

with open('definitions.yml') as file:
    # The FullLoader parameter handles the conversion from YAML
    # scalar values to Python the dictionary format
    definitions = yaml.load(file, Loader=yaml.FullLoader)

features = definitions['features']
spectators = definitions['spectators']
labels = definitions['labels']

nfeatures = definitions['nfeatures']
nspectators = definitions['nspectators']
nlabels = definitions['nlabels']
ntracks = definitions['ntracks']
```

## Graph datasets
Here we have to define the graph dataset. We do this in a separate class following this example: https://pytorch-geometric.readthedocs.io/en/latest/notes/create_dataset.html#creating-larger-datasets

Formally, a graph is represented by a triplet $\mathcal G = (\mathbf{u}, V, E)$, consisting of a graph-level, or *global*, feature vector $\mathbf{u}$, a set of $N^v$ nodes $V$, and a set of $N^e$ edges $E$.
The nodes are given by $V = \{\mathbf{v}_i\}_{i=1:N^v}$, where $\mathbf{v}_i$ represents the $i$th node's attributes.
The edges connect pairs of nodes and are given by $E = \{\left(\mathbf{e}_k, r_k, s_k\right)\}_{k=1:N^e}$, where $\mathbf{e}_k$ represents the $k$th edge's attributes, and $r_k$ and $s_k$ are the indices of the "receiver" and
"sender" nodes, respectively, connected by the $k$th edge (from the sender node to the receiver node).
The receiver and sender index vectors are an alternative way of encoding the directed adjacency matrix.

```python
from GraphDataset import GraphDataset
if local:
    file_names = ['/teams/DSC180A_FA20_A00/b06particlephysics/train/ntuple_merged_10.root']
    file_names_test = ['/teams/DSC180A_FA20_A00/b06particlephysics/test/ntuple_merged_0.root']
else:
    file_names = ['root://eospublic.cern.ch//eos/opendata/cms/datascience/HiggsToBBNtupleProducerTool/HiggsToBBNTuple_HiggsToBB_QCD_RunII_13TeV_MC/train/ntuple_merged_10.root']
    file_names_test = ['root://eospublic.cern.ch//eos/opendata/cms/datascience/HiggsToBBNtupleProducerTool/HiggsToBBNTuple_HiggsToBB_QCD_RunII_13TeV_MC/test/ntuple_merged_0.root']

graph_dataset = GraphDataset('gdata_train', features, labels, spectators, n_events=1000, n_events_merge=1,
                             file_names=file_names)

test_dataset = GraphDataset('gdata_test', features, labels, spectators, n_events=2000, n_events_merge=1,
                             file_names=file_names_test)
```

## Graph neural network

Here, we recapitulate the "graph network" (GN) formalism {cite:p}`battaglia2018relational`, which generalizes various GNNs and other similar methods.
GNs are graph-to-graph mappings, whose output graphs have the same node and edge structure as the input.
Formally, a GN block contains three "update" functions, $\phi$, and three "aggregation" functions, $\rho$.
The stages of processing in a single GN block are:

$
\begin{align}
    \mathbf{e}'_k &= \phi^e\left(\mathbf{e}_k, \mathbf{v}_{r_k}, \mathbf{v}_{s_k}, \mathbf{u} \right) & \mathbf{\bar{e}}'_i &= \rho^{e \rightarrow v}\left(E'_i\right) & \text{(Edge block),}\\
    \mathbf{v}'_i &= \phi^v\left(\mathbf{\bar{e}}'_i, \mathbf{v}_i, \mathbf{u}\right) &
    \mathbf{\bar{e}}' &= \rho^{e \rightarrow u}\left(E'\right) &  \text{(Node block),}\\
    \mathbf{u}' &= \phi^u\left(\mathbf{\bar{e}}', \mathbf{\bar{v}}', \mathbf{u}\right) &
    \mathbf{\bar{v}}' &= \rho^{v \rightarrow u}\left(V'\right) &\text{(Global block).}
  \label{eq:gn-functions}
\end{align}
$

where $E'_i = \left\{\left(\mathbf{e}'_k, r_k, s_k \right)\right\}_{r_k=i,\; k=1:N^e}$ contains the updated edge features for edges whose receiver node is the $i$th node, $E' = \bigcup_i E_i' = \left\{\left(\mathbf{e}'_k, r_k, s_k \right)\right\}_{k=1:N^e}$ is the set of updated edges, and $V'=\left\{\mathbf{v}'_i\right\}_{i=1:N^v}$ is the set of updated nodes.

We will define an interaction network model similar to Ref. {cite:p}`Moreno:2019neq`, but just modeling the particle-particle interactions. It will take as input all of the tracks (with 48 features) without truncating or zero-padding. Another modification is the use of batch normalization {cite:p}`bn` layers to improve the stability of the training.

```python
import torch.nn as nn
import torch.nn.functional as F
import torch_geometric.transforms as T
from torch_geometric.nn import EdgeConv, global_mean_pool
from torch.nn import Sequential as Seq, Linear as Lin, ReLU, BatchNorm1d
from torch_scatter import scatter_mean
from torch_geometric.nn import MetaLayer

inputs = 48
hidden = 128
outputs = 2

class EdgeBlock(torch.nn.Module):
    def __init__(self):
        super(EdgeBlock, self).__init__()
        self.edge_mlp = Seq(Lin(inputs*2, hidden),
                            BatchNorm1d(hidden),
                            ReLU(),
                            Lin(hidden, hidden))

    def forward(self, src, dest, edge_attr, u, batch):
        out = torch.cat([src, dest], 1)
        return self.edge_mlp(out)

class NodeBlock(torch.nn.Module):
    def __init__(self):
        super(NodeBlock, self).__init__()
        self.node_mlp_1 = Seq(Lin(inputs+hidden, hidden),
                              BatchNorm1d(hidden),
                              ReLU(),
                              Lin(hidden, hidden))
        self.node_mlp_2 = Seq(Lin(inputs+hidden, hidden),
                              BatchNorm1d(hidden),
                              ReLU(),
                              Lin(hidden, hidden))

    def forward(self, x, edge_index, edge_attr, u, batch):
        row, col = edge_index
        out = torch.cat([x[row], edge_attr], dim=1)
        out = self.node_mlp_1(out)
        out = scatter_mean(out, col, dim=0, dim_size=x.size(0))
        out = torch.cat([x, out], dim=1)
        return self.node_mlp_2(out)

class GlobalBlock(torch.nn.Module):
    def __init__(self):
        super(GlobalBlock, self).__init__()
        self.global_mlp = Seq(Lin(hidden, hidden),
                              BatchNorm1d(hidden),
                              ReLU(),
                              Lin(hidden, outputs))

    def forward(self, x, edge_index, edge_attr, u, batch):
        out = scatter_mean(x, batch, dim=0)
        return self.global_mlp(out)

class InteractionNetwork(torch.nn.Module):
    def __init__(self):
        super(InteractionNetwork, self).__init__()
        self.interactionnetwork = MetaLayer(EdgeBlock(), NodeBlock(), GlobalBlock())
        self.bn = BatchNorm1d(inputs)

    def forward(self, x, edge_index, batch):

        x = self.bn(x)
        x, edge_attr, u = self.interactionnetwork(x, edge_index, None, None, batch)
        return u

model = InteractionNetwork().to(device)
optimizer = torch.optim.Adam(model.parameters(), lr = 1e-2)
```

## Define training loop

```python
@torch.no_grad()
def test(model, loader, total, batch_size, leave=False):
    model.eval()

    xentropy = nn.CrossEntropyLoss(reduction='mean')

    sum_loss = 0.
    t = tqdm(enumerate(loader), total=total/batch_size, leave=leave)
    for i, data in t:
        data = data.to(device)
        y = torch.argmax(data.y, dim=1)
        batch_output = model(data.x, data.edge_index, data.batch)
        batch_loss_item = xentropy(batch_output, y).item()
        sum_loss += batch_loss_item
        t.set_description("loss = %.5f" % (batch_loss_item))
        t.refresh() # to show immediately the update

    return sum_loss/(i+1)

def train(model, optimizer, loader, total, batch_size, leave=False):
    model.train()

    xentropy = nn.CrossEntropyLoss(reduction='mean')

    sum_loss = 0.
    t = tqdm(enumerate(loader), total=total/batch_size, leave=leave)
    for i, data in t:
        data = data.to(device)
        y = torch.argmax(data.y, dim=1)
        optimizer.zero_grad()
        batch_output = model(data.x, data.edge_index, data.batch)
        batch_loss = xentropy(batch_output, y)
        batch_loss.backward()
        batch_loss_item = batch_loss.item()
        t.set_description("loss = %.5f" % batch_loss_item)
        t.refresh() # to show immediately the update
        sum_loss += batch_loss_item
        optimizer.step()

    return sum_loss/(i+1)
```

## Define training, validation, testing data generators

```python
from torch_geometric.data import Data, DataListLoader, Batch
from torch.utils.data import random_split

def collate(items):
    l = sum(items, [])
    return Batch.from_data_list(l)

torch.manual_seed(0)
valid_frac = 0.20
full_length = len(graph_dataset)
valid_num = int(valid_frac*full_length)
batch_size = 32

train_dataset, valid_dataset = random_split(graph_dataset, [full_length-valid_num,valid_num])

train_loader = DataListLoader(train_dataset, batch_size=batch_size, pin_memory=True, shuffle=True)
train_loader.collate_fn = collate
valid_loader = DataListLoader(valid_dataset, batch_size=batch_size, pin_memory=True, shuffle=False)
valid_loader.collate_fn = collate
test_loader = DataListLoader(test_dataset, batch_size=batch_size, pin_memory=True, shuffle=False)
test_loader.collate_fn = collate

train_samples = len(train_dataset)
valid_samples = len(valid_dataset)
test_samples = len(test_dataset)
print(full_length)
print(train_samples)
print(valid_samples)
print(test_samples)
```

## Train

```python
import os.path as osp

n_epochs = 10
stale_epochs = 0
best_valid_loss = 99999
patience = 5
t = tqdm(range(0, n_epochs))

for epoch in t:
    loss = train(model, optimizer, train_loader, train_samples, batch_size, leave=bool(epoch==n_epochs-1))
    valid_loss = test(model, valid_loader, valid_samples, batch_size, leave=bool(epoch==n_epochs-1))
    print('Epoch: {:02d}, Training Loss:   {:.4f}'.format(epoch, loss))
    print('           Validation Loss: {:.4f}'.format(valid_loss))

    if valid_loss < best_valid_loss:
        best_valid_loss = valid_loss
        modpath = osp.join('interactionnetwork_best.pth')
        print('New best model saved to:',modpath)
        torch.save(model.state_dict(),modpath)
        stale_epochs = 0
    else:
        print('Stale epoch')
        stale_epochs += 1
    if stale_epochs >= patience:
        print('Early stopping after %i stale epochs'%patience)
        break
```

## Evaluate on testing data

```python
model.eval()
t = tqdm(enumerate(test_loader),total=test_samples/batch_size)
y_test = []
y_predict = []
for i,data in t:
    data = data.to(device)
    batch_output = model(data.x, data.edge_index, data.batch)
    y_predict.append(batch_output.detach().cpu().numpy())
    y_test.append(data.y.cpu().numpy())
y_test = np.concatenate(y_test)
y_predict = np.concatenate(y_predict)
```

```python
from sklearn.metrics import roc_curve, auc
import matplotlib.pyplot as plt
import mplhep as hep
plt.style.use(hep.style.ROOT)
# create ROC curves
fpr_gnn, tpr_gnn, threshold_gnn = roc_curve(y_test[:,1], y_predict[:,1])

# plot ROC curves
plt.figure()
plt.plot(tpr_gnn, fpr_gnn, lw=2.5, label="GNN, AUC = {:.1f}%".format(auc(fpr_gnn,tpr_gnn)*100))
plt.xlabel(r'True positive rate')
plt.ylabel(r'False positive rate')
plt.semilogy()
plt.ylim(0.001, 1)
plt.xlim(0, 1)
plt.grid(True)
plt.legend(loc='upper left')
plt.show()
```

```python

```


```python
# Always run this first
# NOTE: Do not define new basic variables in this notebook;
#       define them in Variables_Q.ipynb.  Use this notebook
#       to define new expressions built from those variables.

from __future__ import division # This needs to be here, even though it's in Variables.py
import sys
sys.path.insert(0, '..') # Look for modules in directory above this one
execfile('../Utilities/ExecNotebook.ipy')
#try: execnotebook(VariablesNotebook)
#except: execnotebook('../PNTerms/Variables_Q.ipynb')
```

This notebook provides a couple examples for how to convert long $\LaTeX$ expression into sympy format, via Mathematica.

# EMRI terms

The first step is to select the equation you want from the original source (possibly obtained from the "Other Formats" link on the paper's arXiv page), and put it in its own file.  Here, we have an example named `EMRIGWFlux_7PN.tex` taken from [Fujita (2012)](http://arxiv.org/abs/1211.5535v1).  It is best to copy this exactly, without making any changes

The next step is to run this through perl, and let perl make any necessary replacements.  Mathematica won't want any `\begin{equation}` statements, so we remove them first.  Next, we probably want to remove the left-hand side of the equation, which is just the variable name that this expression represents.  We correct any mistakes in the original (the use of `{\rm ln}` instead of `\ln`, for example), and remove anything else Mathematica won't understand.

```bash
%%bash
perl -nlw \
    -e 's/\\begin\{eqnarray\*\}//g; s/\\end\{eqnarray\*\}//g; ' `# remove environment for Mathematica` \
    -e 's/\{dE\\over dt\}=&&\\left\(\{dE\\over dt\}\\right\)_N//;' `# remove definition statement` \
    -e 's/\{\\rm ln\}/\\ln/g;' `# Correct bad notation for logarithm` \
    -e 's/\\cr//g; s/\\displaystyle//g;' `# These shouldn't have been in there to begin with` \
    -e 's/(\\ln\(.\))(\^\{.\})/($1)$2/g;' `# group logarithm powers correctly` \
    -e 's/\\\{/(/g; s/\\\}/)/g; s/\[/(/g; s/\]/)/g;' `# convert braces and brackets to parentheses` \
    -e 's/\),/\)/;' `# remove final punctuation` \
    -e 'print if /\S/;' `# only print line if nonempty` \
    EMRIGWFlux_7PN.tex > EMRIGWFlux_7PN_Simplified.tex
#cat EMRIGWFlux_7PN_Simplified.tex
```

Next, we just need to run this through Mathematica, which has a good TeX conversion method.  Of course, Mathematica's output is weird — all the function names are capitalized; function arguments come in square brackets; etc.  So we just replace them in the output.  Then, we go through and print the terms at each order, ready to be copied to another notebook.  (Or we could just use the sympy object `Flux` below.)

*NOTE:* You will need to adjust the `MathKernel` path below.  On Linux, you will need the executable named `math` instead.

```python
MathKernel='/Applications/Local/Mathematica.app/Contents/MacOS/MathKernel'
FluxCommand = r"""
\[Gamma] = EulerGamma;
\[Zeta] = Zeta;
HornerForm[ToExpression[Import[
    "EMRIGWFlux_7PN_Simplified.tex",
    "Text"], TeXForm]] >> /tmp/Flux.cpp
Exit[];
"""
! {MathKernel} -run '{FluxCommand}' >& /dev/null
Flux = !cat /tmp/flux.cpp
Flux = ''.join(Flux).replace(' ','').replace('Pi','pi').replace('Log','log').replace('Zeta','zeta').replace('Power','Pow')
Flux = Flux.replace('[','(').replace(']',')').replace('^','**')
Flux = sympify(Flux)

logv = symbols('logv')
FluxDictionary = Poly(Flux.subs('log(v)', logv), Flux.atoms(Symbol).pop()).as_dict()

for key in sorted(FluxDictionary) :
    if(key[0]>7) :
        print("FluxTerms['IncompleteNonspinning'][{0}] = {1}".format(key[0], FluxDictionary[key].subs(logv, log(v))))
```

    FluxTerms['IncompleteNonspinning'][8] = 232597*log(v)/4410 - 1369*pi**2/126 - 323105549467/3178375200 - 47385*log(3)/1568 + 232597*EulerGamma/4410 + 39931*log(2)/294
    FluxTerms['IncompleteNonspinning'][9] = -6848*pi*log(v)/105 - 13696*pi*log(2)/105 - 6848*EulerGamma*pi/105 + 265978667519*pi/745113600
    FluxTerms['IncompleteNonspinning'][10] = 916628467*log(v)/7858620 - 2500861660823683/2831932303200 - 424223*pi**2/6804 - 83217611*log(2)/1122660 + 916628467*EulerGamma/7858620 + 47385*log(3)/196
    FluxTerms['IncompleteNonspinning'][11] = 177293*pi*log(v)/1176 - 142155*pi*log(3)/784 + 8399309750401*pi/101708006400 + 177293*EulerGamma*pi/1176 + 8521283*pi*log(2)/17640
    FluxTerms['IncompleteNonspinning'][12] = 1465472*log(v)**2/11025 - 246137536815857*log(v)/157329572400 - 27392*pi**2*log(v)/315 + 2930944*EulerGamma*log(v)/11025 + 5861888*log(2)*log(v)/11025 - 271272899815409*log(2)/157329572400 - 54784*pi**2*log(2)/315 - 246137536815857*EulerGamma/157329572400 - 437114506833*log(3)/789268480 - 256*pi**4/45 - 27392*EulerGamma*pi**2/315 - 27392*zeta(3)/105 - 37744140625*log(5)/260941824 + 1465472*EulerGamma**2/11025 + 5861888*EulerGamma*log(2)/11025 + 5861888*log(2)**2/11025 + 2067586193789233570693/602387400044430000 + 3803225263*pi**2/10478160
    FluxTerms['IncompleteNonspinning'][13] = 300277177*pi*log(v)/436590 - 81605095538444363*pi/20138185267200 - 42817273*pi*log(2)/71442 + 300277177*EulerGamma*pi/436590 + 142155*pi*log(3)/98
    FluxTerms['IncompleteNonspinning'][14] = -52525903*log(v)**2/154350 - 471188717*log(2)*log(v)/231525 - 52525903*EulerGamma*log(v)/77175 + 9640384387033067*log(v)/17896238860500 + 1848015*log(3)*log(v)/2744 + 531077*pi**2*log(v)/6615 - 6136997968378863*log(3)/1256910054400 - 142155*pi**2*log(3)/392 - 5811697*log(2)**2/2450 - 9523*pi**4/945 - 471188717*EulerGamma*log(2)/231525 - 52525903*EulerGamma**2/154350 + 531077*zeta(3)/2205 + 9640384387033067*EulerGamma/17896238860500 + 1848015*log(3)**2/5488 + 1848015*EulerGamma*log(3)/2744 + 531077*EulerGamma*pi**2/6615 + 1848015*log(2)*log(3)/2744 + 19402232550751339*log(2)/17896238860500 + 9926708984375*log(5)/5088365568 + 128223*pi**2*log(2)/245 + 58327313257446476199371189/8332222517414555760000 + 2621359845833*pi**2/2383781400

Just for fun, let's look at the actual expression:

```python
Flux
```

# PN angular momentum

```python
from sympy import *
ellHat= Symbol('ellHat')
nHat= Symbol('nHat')
lambdaHat= Symbol('lambdaHat')
var('v, m, nu, G, c, x');
```

```python
def MathematicaToSympy(L):
    import re
    L = ''.join(L).replace(' ','')
    L = L.replace(r'\[ScriptL]','ell')
    MathematicaCapitalGreek = re.compile(r'\\ \[ Capital(.*?) \]', re.VERBOSE)
    L = MathematicaCapitalGreek.sub(r'\1',L)
    MathematicaGreek = re.compile(r'\\ \[ (.*?) \]', re.VERBOSE)
    L = MathematicaGreek.sub(lambda m: m.group(1).lower(),L)
    OverHat = re.compile(r'OverHat\[ (.*?) \]', re.VERBOSE)
    L = OverHat.sub(r'\1Hat',L)
    Subscript = re.compile(r'Subscript\[ (.*?), (.*?) \]', re.VERBOSE)
    L = Subscript.sub(r'\1_\2',L)
    Sqrt = re.compile(r'Sqrt\[ (.*?) \]', re.VERBOSE)
    L = Sqrt.sub(r'sqrt(\1)',L)
    L = L.replace('Pi','pi').replace('Log','log').replace('Zeta','zeta').replace('Power','Pow')
    L = L.replace('^','**')
    return L
```

```python
MathKernel='/Applications/Local/Mathematica.app/Contents/MacOS/MathKernel'
MCommand = r"""
\[Gamma] = EulerGamma;
\[Zeta] = Zeta;
HornerForm[ToExpression[Import[
    "AngularMomentum.tex",
    "Text"], TeXForm],x] >> /tmp/AngularMomentum.cpp
Exit[];
"""
! {MathKernel} -run '{MCommand}' >& /dev/null
L = !cat /tmp/AngularMomentum.cpp
L = MathematicaToSympy(L)
L = sympify(L).subs('sqrt(x)',v).subs('x',v**2).subs('G',1).subs('c',1).simplify()
```

```python
L_ellHat = horner( (v*L).simplify().expand().coeff(ellHat) )/v
L_nHat = horner( (v*L).simplify().expand().coeff(nHat) )/v
L_lambdaHat = horner( (v*L).simplify().expand().coeff(lambdaHat) )/v
```

```python
L_set = [L_ellHat, L_nHat, L_lambdaHat]

for n in range(3,9,2):
    print("""AngularMomentum_Spin.AddDerivedVariable('L_SO_{0}',
    ({1})*ellHat
    + ({2})*nHat
    + ({3})*lambdaHat,
    datatype=ellHat.datatype)""".format(n,
                                        (v*L_ellHat/(m**2*nu)).expand().coeff(v**n).simplify(),
                                        (v*L_nHat/(m**2*nu)).expand().coeff(v**n).simplify(),
                                        (v*L_lambdaHat/(m**2*nu)).expand().coeff(v**n).simplify()))
```

    BindingEnergy_Spin.AddDerivedVariable('L_SO_3',
        (-(35*S_ell + 15*Sigma_ell*delta)/(6*m**2))*ellHat
        + ((S_n + Sigma_n*delta)/(2*m**2))*nHat
        + (-(3*S_lambda + Sigma_lambda*delta)/m**2)*lambdaHat,
        datatype=ellHat.datatype)
    BindingEnergy_Spin.AddDerivedVariable('L_SO_5',
        (7*(61*S_ell*nu - 99*S_ell + 30*Sigma_ell*delta*nu - 27*Sigma_ell*delta)/(72*m**2))*ellHat
        + ((-19*S_n*nu + 33*S_n - 10*Sigma_n*delta*nu + 33*Sigma_n*delta)/(24*m**2))*nHat
        + ((18*S_lambda*nu - 21*S_lambda + 8*Sigma_lambda*delta*nu - 3*Sigma_lambda*delta)/(6*m**2))*lambdaHat,
        datatype=ellHat.datatype)
    BindingEnergy_Spin.AddDerivedVariable('L_SO_7',
        ((-29*S_ell*nu**2 + 1101*S_ell*nu - 405*S_ell - 15*Sigma_ell*delta*nu**2 + 468*Sigma_ell*delta*nu - 81*Sigma_ell*delta)/(16*m**2))*ellHat
        + ((11*S_n*nu**2 - 1331*S_n*nu + 183*S_n + 5*Sigma_n*delta*nu**2 - 734*Sigma_n*delta*nu + 183*Sigma_n*delta)/(48*m**2))*nHat
        + ((-32*S_lambda*nu**2 + 2*S_lambda*nu - 174*S_lambda - 16*Sigma_lambda*delta*nu**2 - 79*Sigma_lambda*delta*nu - 12*Sigma_lambda*delta)/(24*m**2))*lambdaHat,
        datatype=ellHat.datatype)

```python
for var in [L_ellHat, L_nHat, L_lambdaHat]:
    print ccode(N(var,16))
```

    (nu + pow(v, 2)*(nu*(0.1666666666666667*nu + 1.5) + v*(-5.833333333333333*S_ell*nu - 2.5*Sigma_ell*delta*nu + v*(nu*(nu*(0.04166666666666667*nu - 2.375) + 3.375) + v*(S_ell*nu*(5.930555555555556*nu - 9.625) + Sigma_ell*delta*nu*(2.916666666666667*nu - 2.625) + v*(nu*(nu*(nu*(0.005401234567901235*nu + 1.291666666666667) - 30.97970359258346) + 8.4375) + v*(S_ell*nu*(nu*(-1.8125*nu + 68.8125) - 25.3125) + Sigma_ell*delta*nu*(nu*(-0.9375*nu + 29.25) - 5.0625))))))))/v
    pow(v, 2)*(0.5*S_n*nu + 0.5*Sigma_n*delta*nu + pow(v, 2)*(S_n*nu*(-0.7916666666666667*nu + 1.375) + Sigma_n*delta*nu*(-0.4166666666666667*nu + 1.375) + pow(v, 2)*(S_n*nu*(nu*(0.2291666666666667*nu - 27.72916666666667) + 3.8125) + Sigma_n*delta*nu*(nu*(0.1041666666666667*nu - 15.29166666666667) + 3.8125))))
    pow(v, 2)*(-3.0*S_lambda*nu - Sigma_lambda*delta*nu + pow(v, 2)*(S_lambda*nu*(3.0*nu - 3.5) + Sigma_lambda*delta*nu*(1.333333333333333*nu - 0.5) + pow(v, 2)*(S_lambda*nu*(nu*(-1.333333333333333*nu + 0.08333333333333333) - 7.25) + Sigma_lambda*delta*nu*(nu*(-0.6666666666666667*nu - 3.291666666666667) - 0.5))))

# Binding energy

```bash
%%bash --out tmp
perl -nlw \
    -e 's/\\begin\{eqnarray\}//g; s/\\end\{eqnarray\}//g; ' `# remove environment for Mathematica` \
    -e 's/\\label\{eB6PN\}//;' `# remove equation label` \
    -e 's/\\nonumber//g; s/\\biggl//g; s/\\Biggl//g; s/\\left//g; s/\\right//g;' `# remove irrelevant formatting` \
    -e 's/\&//g; s/\\\\//g;' `# remove alignments and newlines` \
    -e 's/\\\{/(/g; s/\\\}/)/g; s/\[/(/g; s/\]/)/g;' `# convert braces and brackets to parentheses` \
    -e 's/\),/\)/;' `# remove final punctuation` \
    -e 'print if /\S/;' `# only print line if nonempty` \
    BindingEnergy.tex > BindingEnergy_Simplified.tex
cat BindingEnergy_Simplified.tex
```

```python
tmp
```

    'e_{\\rm 1PN} (\\nu)=  -\\frac{1}{12}\\nu-\\frac34 \ne_{\\rm 2PN} (\\nu)=  -\\frac{1}{24}\\nu^2+\\frac{19}8 \\nu-\\frac{27}{8}\ne_{\\rm 3PN} (\\nu)= -\\frac{35}{5184}\\nu^3 -\\frac{155}{96}\\nu^2+\\frac{34445}{576}\\nu-\\frac{205}{96}\\pi^2\\nu-\\frac{675}{64}  \ne_{\\rm 4PN}^c (\\nu)= -\\frac{3969}{128}+\\frac{77}{31104}\\nu^4+\\frac{301}{1728}\\nu^3\n+(-\\frac{498449}{3456} +\\frac{3157}{576}\\pi^2)\\nu^2\n+(-\\frac{123671}{5760}+\\frac{1792}{15} \\ln(2)+\\frac{9037}{1536} \\pi^2\n+\\frac{896}{15} \\gamma)\\nu \ne_{\\rm 4PN}^{\\ln}(\\nu)= \\frac{448}{15}\\nu  \ne_{\\rm 5PN}^c (\\nu)= -\\frac{45927}{512}+\\frac{1}{512}\\nu^5+\\frac{55}{512}\\nu^4\n+(-\\frac{1353}{256}\\pi^2+\\frac{69423}{512} )\\nu^3\n+(-\\frac{21337}{1024}\\pi^2+3 a_6^{c1}\n-\\frac{896}{5}\\ln(2)-\\frac{448}{5}\\gamma+\\frac{189745}{576}+\\frac{2}{3}a_{6}^{\\ln 1})\\nu^2\n+(-\\frac{228916843}{115200}-\\frac{9976}{35}\\gamma+\\frac{729}{7}\\ln(3) \n -\\frac{23672}{35}\\ln(2)+\\frac{126779}{512}\\pi^2 )\\nu \ne_{\\rm 5PN}^{\\ln}(\\nu)= -\\frac{4988}{35}\\nu-\\frac{224}{5}\\nu^2+3\\nu^2 a_6^{\\ln 1}\ne_{\\rm 5.5PN}(\\nu)=\\frac{10}{3}\\nu (\\frac{13696}{525}\\pi+\\nu  a_{6.5}^{c1})\ne_{\\rm 6PN}^c (\\nu)= -\\frac{264627}{1024}+\\frac{2717}{6718464}\\nu^6+\\frac{5159}{248832}\\nu^5\n+(\\frac{272855}{124416}\\pi^2-\\frac{20543435}{373248} )\\nu^4\n+(\\frac{1232}{27}\\gamma+\\frac{6634243}{110592}\\pi^2-\\frac{11}{2} a_6^{c1}\n-\\frac{75018547}{51840}+\\frac{2464}{27}\\ln(2) -\\frac{20}{9}a_6^{\\ln 1} )\\nu^3\n+ (\\frac{113594718743}{14515200}+\\frac{18491}{2304}\\pi^4\n+\\frac{246004}{105}\\ln(2)+\\frac{112772}{105}\\gamma+\\frac{11}{2} a_6^{c1}+a_6^{\\ln 1}+\\frac{2}{3}a_7^{\\ln 1}\n+ \\frac{11}{3} a_7^{c1}-\\frac{86017789}{110592}\\pi^2-\\frac{2673}{14}\\ln(3))\\nu^2\n+(- \\frac{389727504721}{43545600}\n +\\frac{74888}{243}\\ln(2)\n- \\frac{7128}{7}\\ln(3)-\\frac{30809603}{786432}\\pi^4\n-\\frac{3934568}{8505}\\gamma +\\frac{9118627045}{5308416}\\pi^2)\\nu\ne_{\\rm 6PN}^{\\ln}(\\nu)= -\\frac{1967284}{8505}\\nu+\\frac{616}{27}\\nu^3+\\frac{56386}{105}\\nu^2 -\\frac{11}{2}\\nu^3 a_6^{\\ln 1}+\\frac{11}{3}a_7^{\\ln 1}+\\frac{11}{2}\\nu^2 a_6^{\\ln 1}\n'

```python
from __future__ import print_function
Terms = ['E_2', 'E_4', 'E_6', 'E_8', 'E_lnv_8', 'E_10', 'E_lnv_10', 'E_11', 'E_12', 'E_lnv_12']
Expressions = [s[(s.index('=')+1):].replace('\n','') for s in tmp.split('\ne')]
for t,e in zip(Terms,Expressions):
    print(e, file=open('/tmp/'+t+'.tex','w+'))
```

```python
import re
MathKernel='/Applications/Local/Mathematica.app/Contents/MacOS/MathKernel'
BindingEnergyCommand = r"""
\[Gamma] = EulerGamma;
HornerForm[ToExpression[Import[
    "/tmp/{0}.tex",
    "Text"], TeXForm]] >> /tmp/{0}.cpp
Exit[];
"""
for t in Terms:
    ! {MathKernel} -run '{BindingEnergyCommand.format(t)}' >& /dev/null
    BindingEnergy = !cat /tmp/{t}.cpp
    BindingEnergy = ''.join(BindingEnergy).replace(' ','').replace('Pi','pi').replace('Log','log').replace('Nu','nu')
    BindingEnergy = BindingEnergy.replace('[','(').replace(']',')').replace('Power','Pow').replace('^','**').replace('\\','')
    BindingEnergy = re.sub(r'Subscript\((.*?),([0-9])[.5]*\)\*\*(c|ln)', r'\1_\2__\g<3>1', BindingEnergy)
    BindingEnergy = sympify(BindingEnergy)

    logv = symbols('logv')
    BindingEnergy = BindingEnergy.subs('log(v)', logv)

    print("BindingEnergy_NoSpin_B.AddDerivedConstant('{0}', {1})".format(t, BindingEnergy))
    print()
```

    BindingEnergy_NoSpin_B.AddDerivedConstant('E_2', -nu/12 - 3/4)

    BindingEnergy_NoSpin_B.AddDerivedConstant('E_4', nu*(-nu/24 + 19/8) - 27/8)

    BindingEnergy_NoSpin_B.AddDerivedConstant('E_6', nu*(nu*(-35*nu/5184 - 155/96) - 205*pi**2/96 + 34445/576) - 675/64)

    BindingEnergy_NoSpin_B.AddDerivedConstant('E_8', nu*(nu*(nu*(77*nu/31104 + 301/1728) - 498449/3456 + 3157*pi**2/576) - 123671/5760 + 896*EulerGamma/15 + 9037*pi**2/1536 + 1792*log(2)/15) - 3969/128)

    BindingEnergy_NoSpin_B.AddDerivedConstant('E_lnv_8', 448*nu/15)

    BindingEnergy_NoSpin_B.AddDerivedConstant('E_10', nu*(nu*(3*a_6__c1 + 2*a_6__ln1/3 + nu*(nu*(nu/512 + 55/512) - 1353*pi**2/256 + 69423/512) - 21337*pi**2/1024 - 896*log(2)/5 - 448*EulerGamma/5 + 189745/576) - 228916843/115200 - 23672*log(2)/35 - 9976*EulerGamma/35 + 729*log(3)/7 + 126779*pi**2/512) - 45927/512)

    BindingEnergy_NoSpin_B.AddDerivedConstant('E_lnv_10', nu*(nu*(3*a_6__ln1 - 224/5) - 4988/35))

    BindingEnergy_NoSpin_B.AddDerivedConstant('E_11', 10*nu(a_6__c1*nu + 13696*pi/525)/3)

    BindingEnergy_NoSpin_B.AddDerivedConstant('E_12', nu*(nu*(11*a_6__c1/2 + a_6__ln1 + 11*a_7__c1/3 + 2*a_7__ln1/3 + nu*(-11*a_6__c1/2 - 20*a_6__ln1/9 + nu*(nu*(2717*nu/6718464 + 5159/248832) - 20543435/373248 + 272855*pi**2/124416) - 75018547/51840 + 1232*EulerGamma/27 + 2464*log(2)/27 + 6634243*pi**2/110592) - 86017789*pi**2/110592 - 2673*log(3)/14 + 112772*EulerGamma/105 + 18491*pi**4/2304 + 246004*log(2)/105 + 113594718743/14515200) - 389727504721/43545600 - 30809603*pi**4/786432 - 7128*log(3)/7 - 3934568*EulerGamma/8505 + 74888*log(2)/243 + 9118627045*pi**2/5308416) - 264627/1024)

    BindingEnergy_NoSpin_B.AddDerivedConstant('E_lnv_12', 11*a_7__ln1/3 + nu*(nu*(11*a_6__ln1/2 + nu*(-11*a_6__ln1/2 + 616/27) + 56386/105) - 1967284/8505))

```python
import re
from sympy import sympify, symbols
for i in [1]:
    BindingEnergy = !cat /tmp/E_lnv_12.cpp
    print(BindingEnergy)
    BindingEnergy = ''.join(BindingEnergy).replace(' ','').replace('Pi','pi').replace('Log','log').replace('Nu','nu')
    print(BindingEnergy)
    BindingEnergy = BindingEnergy.replace('[','(').replace(']',')').replace('Power','Pow').replace('^','**').replace('\\','')
    print(BindingEnergy)
    BindingEnergy = re.sub(r'Subscript\((.*?),([0-9])[.5]*\)\*\*(c|ln)', r'\1_\2__\g<3>1', BindingEnergy)
    print(BindingEnergy)
    BindingEnergy = sympify(BindingEnergy)
    print(BindingEnergy)

    logv = symbols('logv')
    BindingEnergy = BindingEnergy.subs('log(v)', logv)

    print("BindingEnergy_NoSpin_B.AddDerivedConstant('E_lnv_12', {0})".format(BindingEnergy))
    print()
```

    ['\\[Nu]*(-1967284/8505 + \\[Nu]*(56386/105 + (11*Subscript[a, 6]^ln)/2 + ', '     \\[Nu]*(616/27 - (11*Subscript[a, 6]^ln)/2))) + (11*Subscript[a, 7]^ln)/3']
    \[nu]*(-1967284/8505+\[nu]*(56386/105+(11*Subscript[a,6]^ln)/2+\[nu]*(616/27-(11*Subscript[a,6]^ln)/2)))+(11*Subscript[a,7]^ln)/3
    (nu)*(-1967284/8505+(nu)*(56386/105+(11*Subscript(a,6)**ln)/2+(nu)*(616/27-(11*Subscript(a,6)**ln)/2)))+(11*Subscript(a,7)**ln)/3
    (nu)*(-1967284/8505+(nu)*(56386/105+(11*a_6__ln1)/2+(nu)*(616/27-(11*a_6__ln1)/2)))+(11*a_7__ln1)/3
    11*a_7__ln1/3 + nu*(nu*(11*a_6__ln1/2 + nu*(-11*a_6__ln1/2 + 616/27) + 56386/105) - 1967284/8505)
    BindingEnergy_NoSpin_B.AddDerivedConstant('E_lnv_12', 11*a_7__ln1/3 + nu*(nu*(11*a_6__ln1/2 + nu*(-11*a_6__ln1/2 + 616/27) + 56386/105) - 1967284/8505))
    ()

```python
nu*(nu*(11*a_6__ln1/2 + nu*(-11*a_6__ln1/2 + 616/27) + 56386/105) - 1967284/8505)
```

```python
11*self.a_7__ln1/3 + nu*(nu*(11*self.a_6__ln1/2 + nu*(-11*self.a_6__ln1/2 + 616/27 + 56386/105) - 1967284/8505))
```