---
Replication Project for Microeconometrics course | Summer 2021, M.Sc. Economics, Bonn University | [Aysu Avcı](https://github.com/aysuavci)

# Replication of Zimmermann (2020)
---

This notebook contains the replication of the following study:

> [Zimmermann, F. (2020). The Dynamics of Motivated Beliefs. American Economic Review, 110(2), 337–363](https://www.aeaweb.org/articles?id=10.1257/aer.20180728).

##### Notes:

* I try to remain true to the original naming of the variables, however, I renamed all of the figures and tables according to sections in this notebook to avoid any confusions that might arise due to having two replications.
* For convenience, I decided to keep some of the coding in the notebook which I have seen essential for understanding the steps taken in replication.

* The responsitory can be downloaded from [here](https://github.com/OpenSourceEconomics/ose-data-science-course-project-aysuavci), for the best viewing.

## **Table of Contents**
* [Introduction](#sec1)
* [Background for the Experiment and Hypothesis](#sec2)
* [Experiment Design](#sec3)
* [Identification](#sec4)
* [Empirical Strategy](#sec5)
* [Replication of Zimmermann (2020)](#sec6)
    * [Main Results](#sec6_1)
        * [Data & Descriptive Statistics](#sec6_1_1)
        * [Belief Adjustment Distribution¶](#sec6_1_2)
        * [Belief Adjustment Distribution & Test performances](#sec6_1_3)
        * [Regression Outputs](#sec6_1_4)
    * [Robustness Checks](#sec6_2)
        * [Appendix A.4 - Alternative Definition of Positive/Negative Feedback](#sec6_2_1)
        * [Appendix A.6 - Updating in the Short-Run](#sec6_2_2)
        * [Appendix A.7 - No Feedback Condition](#sec6_2_3)
        * [Appendix A.8 - Figures Bayesian Posteriors](#sec6_2_4)
* [Extension](#sec7)
    * [Replication of Eil and Rao (2011)](#sec7_1)
        * [Introduction](#sec7_1_1)
        * [Experiment Design](#sec7_1_2)
        * [Emprical Strategy](#sec7_1_3)
        * [Replication](#sec7_1_4)
        * [Comparison with Zimmermann (2020)](#sec7_1_5)
    * [Further Extension](#sec7_2)
        * [Alternative Data Visualization](#sec7_2_1)
        * [Further Analysis with No Feedback Group](#sec7_2_2)
            * [Emprical Strategy](#sec7_2_2_1)
            * [Estimation](#sec7_2_2_2)
* [Conclusion](#sec8)
* [References](#sec9)

```python
%matplotlib inline
!pip install stargazer
!pip install matplotlib
!pip install -U matplotlib
import numpy as np
import pandas as pd
import pandas.io.formats.style
import seaborn as sns
import statsmodels as sm
import statsmodels.formula.api as smf
import statsmodels.api as sm_api
import matplotlib as plt
from IPython.display import HTML
from stargazer.stargazer import Stargazer, LineLocation
from statsmodels.iolib.summary2 import summary_col
```

```python
from auxiliary.auxiliary_tools import *
from auxiliary.auxiliary_plots import *
from auxiliary.auxiliary_tables import *
```

---
# 1.Introduction
---

The study of Zimmermann(2020) is a lab experiment aiming to examine how motivated beliefs held by individuals continues to be sustained after receiving positive or negative feedback.
The study can be divided into 3 parts:

1. The first part of the study-named “**Motivated Belief Dynamics**” in the paper- is the main study for investigating the causality between different types of feedback (positive or negative) received and reconstruction of belief patterns depending on the elicitation time (directly or 1 month) of the belief after the experiment.
2. The second part of the study-named “**The Role of Memory**” in the paper- investigates the asymmetry in the accuracy of feedback recall and also recall of IQ test in general that they solve as a part of the experiment.
3. The final part of the study-named “**The Trade-Off between Motivated and Accurate Belief**” in the paper- that questions whether incentivizing for recall accuracy could mitigate the motivated reasoning that participants employ.

In this project notebook, the main study-first part- will be replicated; the part where Zimmermann (2020) used also **difference-in-difference (DID) models** for their estimations. In the next section (Section 2), I will give a quick literature background for the experiment and also specify the researcher’s hypothesis. In Section 3, I will explain the relevant parts of the experimental design and introduce treatment groups and variables for convenience. The identification part (Section 4) will explain the causality studied by Zimmermann (2020) and the empirical strategy part (Section 5) will present the established models for estimations. Section 6 is the replication part where the figures and tables from the main study and relevant Appendix parts are presented. The extension of this project is located in Section 7 where I replicate the study of Eil and Rao (2022)  as well as including some data visualization and further analysis. Finally, Section 8 includes my concluding remarks.

---
# 2. Background for the Experiment and Hypothesis
---
Personally, I have always been interested in the question of how individual beliefs are formed and sustained or changed over time. In the neoclassical economics framework, any changes in beliefs follow Bayes Rule’s since it is accepted that all agents are rational and unbiased. However, many studies in the literature of motivated beliefs showed that individuals deviate from these Bayesian predictions and manipulate their beliefs in a self-serving way; and some studies like Eil and Rao 2011 or Zimmermann 2020 tried to address the asymmetry in belief updating as well. I see the study of Zimmermann (2020) as an influential study to study motivated belief dynamics because of the experiment design that is been employed. The design of Zimmermann (2020) allows comparing the belief updating across signals received (asymmetry) and also across time (short-run and long-run updating) by employing control and various treatment groups.

**Main Hypothesis:** *Individuals’ engagement with belief updating follows an asymmetric pattern, putting more weight on positive signals and less on the negative signals, which is more pronounced for long-term since negative signal’s effects fade over time.*

Using DID models for analysis is suitable for this study because, first of all, it is impossible to make one experiment subject expose to two different treatments while keeping everything constant in this experiment to examine individual-level effects. Secondly, since this is a randomly assigned experiment we can assume subjects do not differ between the control group and treatment groups. Also, between the control group and treatment group Zimmermann (2020) made sure to keep everything constant and only made the feedback condition vary. Also, DID was easily implemented as an interaction term between time treatment and feedback dummy allowing for causal inferences on belief dynamics in line with the researcher’s goal and with the group of interest which is the individuals who received a negative signal and in the treatment group 1 month and their differences compared to other groups in belief updating. In addition to these, employing DID models allowed Zimmermann (2020) to add the fixed rank effects interacted with treatment which serve as a control for the possible characteristic differences due to rank groups between treatments. Also, having a control condition for 1-month treatment which serve as a robustness check for potential beliefs changes due to time trends that may exist in the absence of feedback.

---
# 3. Experimental Design
---
1. Subjects completed an IQ test where they need to solve 10 Raven matrices.
2. Subjects were randomly assigned to groups of 10.
3. Subjects were asked to estimate their belief about the likelihood of being ranked in the upper half of their group in terms of a percentage point. Full probability distributions for every possible rank is also elicited.
4. Subjects were given noisy but true information about their rank by randomly selecting 3 participants from their group and telling them whether they were rank higher or lower than these 3 participants. If a subject is told that they ranked higher than 2 or 3 participants in their group, the subject is considered to be in the positive feedback group. They considered being in the negative feedback group if otherwise.
5. After receiving the feedbacks, subjects are divided into two treatment groups randomly: `condifence_direct` and `confidence_1monthlater`.
6. In the `confidence_1monthlater` treatment, subjects are asked to elicit their beliefs 1 month later; and in the `condifence_direct` treatment, group subjects are asked to elicit their beliefs after the feedback. `condifence_direct`has also two subgroups: `condifence_direct_immediate` and `condifence_direct_15minutelater`.

**Main Variables**

|Treatment groups|Feedback types|Outcomes|Other variables|
|---|---|---|---|
|`confidence_1monthlater`|Positive|Belief adjustment|Bayesian belief adjustment||
|`condifence_direct_immediate`|Negative||rank|
|`condifence_direct_15minutelater`|||
|`no_feedback`|||

---
# 4. Identification
---

The first part of the experiment is dedicated to investigating the causality between the effect of feedback and belief adjustments of participants that are in two different treatment groups that vary depending on time: _CondifenceDirect_ and _Condifence1month_. The causality is established by two key components in the experiment. First of all, Zimmermann(2020) elicit peoples prior beliefs about their probability of ranking in the upper half of the group and then they elicited again after participants received the feedback. So, for each participant, they had a clear measure of belief updating which is possibly caused by the feedback and in line with the type of feedback. The second and the most important component of the experiment, that ensured causal identification, is the noisy feedback component. In this experiment rather than randomly assigning a subject into a negative or positive feedback group directly, Zimmermann(2020) randomly choose 3 other participants within the subject’s group to compare their rank and provide them with 3 comparisons. In this way, subjects indeed received true feedback but a noisy one. So, it is possible for two subjects with the same rank to receive different types of feedback. By adopting such an experimental design, Zimmermann(2020) ensured that potential asymmetries in the belief dynamics cannot be explained by the individual difference between participants.

In addition to investigating the causal relationship between belief dynamics and feedback, another goal of the experiment is to observe the effect of time on this relationship. For that, Zimmermann(2020) assigned participants randomly to two groups: Confidence1month and CondifenceDirect. ConfidenceDirect is also divided into two subgroups(immediate and 15 minutes later belief elicitation) to investigate the possible short-term dynamics in belief adjustment, however, that part of the study will not be covered in this replication.

---
# 5. Empirical Strategy
---

In the study, the belief adjustment is defined as the difference between the elicited belief after the feedback and the belief before the feedback, formally

\begin{equation}
Pr(upperhalf)^{post}_{i} - Pr(upperhalf)^{prior}_{i}
\end{equation}

where $Pr(upperhalf)_{i}$ represents the subject ${i}$'s given probability of being in the upperhalf of the group.

To compare belief adjustments between positive and negative feedbacks, a normalized version of the belief adjustment norm is also formed as below:

$$
beliefadjustmentnorm_i = \left\{
    \begin{array}\\
        Pr(upperhalf)^{post}_{i} - Pr(upperhalf)^{prior}_{i} & \mbox{if } \ feedback \ positive \\
        (-1) x (Pr(upperhalf)^{post}_{i} - Pr(upperhalf)^{prior}_{i}) & \mbox{if } \ feedback \ negative \\
    \end{array}
\right.
$$

The difference-in-difference models, formally in the form of

\begin{equation}
beliefadjustmentnorm_i = \alpha + \beta feedback_i + \gamma T_i + \delta I_i + X_i \gamma + \epsilon_i
\end{equation}

are estimated for the purpose of establishing dynamic belief patterns. $feedback_i$ is the dummy variable for feedback and $T_i$ is the type of treatment. $I_i$ is for the interation term that is a generated dummy for the interest group, interest group being subjects who are in the `confidence_1monthlater`and received a negative feedback. In other words, if subjects were in `confidence_1monthlater` treatment and received a negative information; treatment dummy, feedback dummy and interaction term are all equal to 1. In this case, parameter $\delta$ is therefore captures the belief dynamics. In other words, if the subject is in `confidence_1monthlater` treatment and received a negative information; treatment dummy, feedback dummy and interaction term are equal to 1.

In order to form a causal inference, Zimmermann (2020) control for rank and IQ test scores; these variables are captured by $X_i$, set of control variables. As another control variable, Bayesian belief adjustments are also used for some specifications. Zimmermann (2020) calculated Bayesian predictions, using individuals full prior probability distribution and the feedback type that they received. The Bayesian belief adjustment is then, produced in the same way as the belief adjustment

\begin{equation}
Pr(upperhalf)^{postBayes}_{i} - Pr(upperhalf)^{prior}_{i}
\end{equation}

where $Pr(upperhalf)^{postBayes}_{i}$ is the calculated Bayesian prediction for the subject $i$.

Another specification within the study is adding the rank fixed effects interacted with treatment. In this way, Zimmermann (2020) was able to control the differentiation of possible different characteristics due to ranking between treatment groups.

Zimmermann (2020) choose to perform ordinary least square (OLS) estimates with type HC1 heteroscedasticity robust covariance. Most of the main results are produced with this linear regression and I will use this type of regression in the table replications.

---
# 6. Replication of Zimmermann (2020)
---
## 6.1. Main Results
### 6.1.1 Data & Descriptive Statistics

```python
df = pd.read_stata('data/data.dta')
```

```python
#Adjustments for better viewing
pd.set_option("display.max.columns", None)
pd.set_option("display.precision", 7)
pd.set_option("display.max.rows", 30)
```

```python
#Viewing data
df.head()
```

      treatment
      rscore
      performance1
      performance2
      performance3
      performance4
      performance5
      performance6
      performance7
      performance8
      performance9
      wspt
      donation
      prior_median
      prior_1
      prior_2
      prior_3
      prior_4
      prior_5
      prior_6
      prior_7
      prior_8
      prior_9
      prior_10
      rank
      pos_comparisons
      neg_comparisons
      repeat_neg_comparisons
      repeat_pos_comparisons
      risk_investment
      recall_neg_comparisons
      recall_pos_comparisons
      big5_1
      big5_2
      big5_3
      big5_4
      big5_5
      big5_6
      big5_7
      big5_8
      big5_9
      big5_10
      big5_11
      big5_12
      big5_13
      big5_14
      big5_15
      gender
      field_of_study
      student
      posterior_median
      recallfeedback
      qualtrics
      recall_part_IQtest
      recall_part_feedback
      recall_parts_nonIQ
      pricelist1
      pricelist2
      pricelist3
      pricelist4
      pricelist5
      pricelist6
      pricelist7
      pricelist8
      pricelist9
      pricelist10
      pricelist11
      pricelist12
      pricelist13
      pricelist14
      pricelist15

      0
      belief_announcement
      4
      1
      3
      4
      4
      5
      5
      5
      6
      8
      6
      1
      53
      2
      4
      9
      14
      24
      16
      13
      9
      7
      2
      7
      2.0
      1.0
      1.0
      2.0
      80
      1.0
      2.0
      1
      2
      4
      3
      2
      4
      3
      4
      2
      5
      3
      4
      3
      4
      4
      0
      Geographie
      1
      68.0
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN

      1
      belief_announcement
      6
      3
      4
      5
      5
      6
      6
      7
      8
      9
      7
      5
      70
      10
      10
      10
      20
      20
      10
      5
      5
      5
      5
      4
      2.0
      1.0
      1.0
      2.0
      150
      1.0
      2.0
      2
      2
      6
      2
      5
      2
      7
      2
      2
      7
      3
      6
      2
      2
      3
      1
      psychologie
      1
      60.0
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN

      2
      belief_announcement
      8
      4
      6
      7
      7
      7
      8
      8
      9
      10
      5
      3
      85
      1
      4
      30
      30
      20
      9
      2
      2
      1
      1
      3
      3.0
      0.0
      0.0
      2.0
      80
      0.0
      2.0
      3
      5
      6
      3
      2
      3
      4
      4
      2
      2
      3
      2
      2
      3
      5
      1
      Medienwissenschaft
      1
      85.0
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN

      3
      belief_announcement
      5
      4
      6
      7
      7
      7
      8
      8
      9
      10
      5
      0
      80
      3
      6
      6
      25
      40
      5
      5
      4
      3
      3
      9
      0.0
      3.0
      3.0
      0.0
      150
      3.0
      0.0
      1
      2
      7
      3
      2
      2
      6
      4
      2
      4
      1
      4
      1
      2
      3
      0
      Lebensmittelchemie
      1
      45.0
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN

      4
      belief_announcement
      5
      1
      3
      4
      4
      5
      5
      5
      6
      8
      3
      0
      25
      0
      0
      0
      0
      25
      5
      5
      20
      20
      25
      4
      3.0
      0.0
      0.0
      3.0
      100
      0.0
      3.0
      4
      4
      7
      3
      2
      1
      7
      2
      6
      3
      2
      4
      1
      1
      4
      1
      Geographie
      1
      70.0
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN
      NaN

```python
#Renaming treatgroup for easier coding
tg_conditions = [
    (df['treatment'] == 'belief_announcement'),
    (df['treatment'] == 'confidence_1monthlater'),
    (df['treatment'] == 'confidence_direct_15minuteslater'),
    (df['treatment'] == 'confidence_direct_immediate'),
    (df['treatment'] == 'memory'),
    (df['treatment'] == 'memory_high'),
    (df['treatment'] == 'nofeedback'),
    (df['treatment'] == 'tournament_announcement'),
]
tg_values = [1, 2, 3, 4, 5, 6, 7, 8]
df['treatgroup'] = np.select(tg_conditions, tg_values)
```

Throughout this study feedback groups will be addressed using the dummy variable `dummynews_goodbad`. As it is mentioned above, if a subject received more than 2 positive comparisons they are considered to be in the positive feedback group; if else they are considered to be in the negative feedback group.

```python
#Creating dummy variable for good/bad news (if good = 0, if bad = 1)
df["dummynews_goodbad"] = np.nan
df.loc[(df['pos_comparisons'] == 2) | (df['pos_comparisons'] == 3), 'dummynews_goodbad'] = 0
df.loc[(df['pos_comparisons'] == 0) | (df['pos_comparisons'] == 1), 'dummynews_goodbad'] = 1
```

```python
#Belief adjustment
df["beliefadjustment"] = df["posterior_median"] - df["prior_median"]
#Normalized belief adjustment
df["beliefadjustment_normalized"] = np.nan
df.loc[df['dummynews_goodbad'] == 0, 'beliefadjustment_normalized'] = df['beliefadjustment']
df.loc[df['dummynews_goodbad'] == 1, 'beliefadjustment_normalized'] = df['beliefadjustment']*-1
```

### 6.1.2 Belief Adjustment Distribution

```python
Main_Figure1(df)
```

Figure 1 is an easy way to view the belief adjustments of the subjects, even though, they aren't enough for causal inference. As it is observable from Panel A. subjects adjusted their beliefs in line with the feedback they have received. For instance, in the positive feedback group, we see the accumulation of observations between 0 and +50 while an opposite pattern is observable for the negative feedback group.

In Panel B. we see that for the individuals who have received positive feedback, the pattern observe is not so much different from the pattern in Panel A. However, the graph significantly changes for the negative feedback when subjects elicit their beliefs 1 month after the feedback.

### 6.1.3 Belief Adjustment Distribution  & Test performances

**Calculating Bayesian Predictions**

```python
#Bayesian predictions
get_Bayesian_predictions(df)
```

```python
#Calculating Bayesian Posterior Median Beliefs
df["posterior_median_bayes"] = (df["post_1"] + df["post_2"] + df["post_3"] + df["post_4"] + df["post_5"])*100
```

```python
##Bayesian predictions for belief adjustment
df["beliefadjustment_bayes"] = df["posterior_median_bayes"] - df["prior_median"]

#Normalization of Bayesian predictions
df["beliefadjustment_bayes_norm"] = np.nan
df.loc[df['dummynews_goodbad'] == 0, 'beliefadjustment_bayes_norm'] = df['beliefadjustment_bayes']
df.loc[df['dummynews_goodbad'] == 1, 'beliefadjustment_bayes_norm'] = df['beliefadjustment_bayes']*-1
```

There are two sub treatment groups in the immediate treatment: confidence_direct_15minuteslater' and 'confidence_direct_immediate'. Zimmermann(2020) made this differentiation for a robustness check to see whether there would be a difference between eliciting beliefs immediately after feedback or 15 minutes later. However, the main results derived from the comparison between 1-month treatment and pooled treatment which combines 15 minutes later and immediate treatment. So, a dummy is created for further analysis; with 1 indicating treatment 1 month and 0 indicating pooled direct treatment.

```python
#Generate dummy for direct treatment and 1 month treatment
df["dummytreat_direct1month"] = np.nan
df.loc[(df['treatgroup'] == 3) | (df['treatgroup'] == 4), 'dummytreat_direct1month'] = 0
df.loc[df['treatgroup'] == 2, 'dummytreat_direct1month'] = 1
```

```python
#Grouping IQ test scores
df["test_1"] = np.nan
df.loc[(df['rscore'] == 0) | (df['rscore'] == 1) | (df['rscore'] == 2) | (df['rscore'] == 3) | (df['rscore'] == 4), 'test_1'] = 1
df.loc[(df['rscore'] == 5), 'test_1'] = 2
df.loc[(df['rscore'] == 6), 'test_1'] = 3
df.loc[(df['rscore'] == 7) | (df['rscore'] == 8) | (df['rscore'] == 9) | (df['rscore'] == 10), 'test_1'] = 4
```

```python
#Average prior belief
df['prior_av'] = df.groupby(['test_1', 'dummytreat_direct1month'])["prior_median"].transform('mean')
```

```python
#Average posterior belief if feedback is positive
df['post_av_pos'] = df[df["dummynews_goodbad"] == 0].groupby(['test_1', 'dummytreat_direct1month'])["posterior_median"].transform('mean')
```

```python
#Average posterior belief if feedback is negative
df['post_av_neg'] = df[df["dummynews_goodbad"] == 1].groupby(['test_1', 'dummytreat_direct1month'])["posterior_median"].transform('mean')
```

```python
Main_Figure2(df)
```

Figure 2 plots the individuals' prior and posterior belief of being in the upper half of their group for differents levels in test performances. Test performances are grouped as 1, 2, 3 and 4; with 1 being the lowest score group. Also, posteriors are plotted with two different lines.

It is easy to see that in Panel A. ConfidenceDirect, subject's posterior beliefs visibly differ from their prior beliefs in line with the feedback they received. However, in Panel B, the posterior beliefs of subjects that received negative feedback are visibly closer to the prior beliefs.

Compared to the first graph, here causal inference is made possible by grouping individuals according to their test scores.

These figures are also a great way to observe ex-ante overconfident-average priors lie consistently above 50 percent- which can be theoretically explained by Bayesian updating (see. Benoît and Dubra, 2011) and also visible in the figures from the study of Zimmermann (2020).

### 6.1.4. Regression Outputs

In the study, Zimmermann (2020) perform 8 OLS estimations(see in Table 1) with type HC1 heteroscedasticity robust covariance. :

(1) Regressing $y_i$ (normalized belief adjustment) on $T_i$ (treatment type) in case of positive feedback

(2) Same as (1) but controling for $X_i$ (rank and normalized bayesian belief adjustment)

(3) Regressing $y_i$ (normalized belief adjustment) on $T_i$ (treatment type) in case of negative feedback

(4) Same as (3) but controling for $X_i$ (rank and normalized bayesian belief adjustment)

(5) Regressing $y_i$ (normalized belief adjustment) on $T_i$ (treatment type), $feedback_i$ (feedback type) and $I_i$ (interaction term indicating whether individual is in the interest group or not)

(6) Same as (5) but controling for $X_i$ (rank and normalized bayesian belief adjustment)

(7) Same as (5) but including rank fixed effects with their interaction with treatment

(8) Same as (7) but controling for $X_i$ (normalized bayesian belief adjustment)

```python
df_good = pd.DataFrame({"beliefadjustment_normalized": df[df["dummynews_goodbad"] == 0]['beliefadjustment_normalized'], "dummytreat_direct1month": df[df["dummynews_goodbad"] == 0]['dummytreat_direct1month'], "rank": df[df["dummynews_goodbad"] == 0]['rank'], "beliefadjustment_bayes_norm": df[df["dummynews_goodbad"] == 0]['beliefadjustment_bayes_norm']})
#(1)
model_ols = smf.ols(formula="beliefadjustment_normalized ~ dummytreat_direct1month", data=df_good)
reg_1 = model_ols.fit(cov_type='HC1')
#(2)
model_ols = smf.ols(formula="beliefadjustment_normalized ~ dummytreat_direct1month + rank + beliefadjustment_bayes_norm", data=df_good)
reg_2 = model_ols.fit(cov_type='HC1')
```

```python
df_bad = pd.DataFrame({"beliefadjustment_normalized": df[df["dummynews_goodbad"] == 1]['beliefadjustment_normalized'], "dummytreat_direct1month": df[df["dummynews_goodbad"] == 1]['dummytreat_direct1month'], "rank": df[df["dummynews_goodbad"] == 1]['rank'], "beliefadjustment_bayes_norm": df[df["dummynews_goodbad"] == 1]['beliefadjustment_bayes_norm']})
#(3)
model_ols = smf.ols(formula="beliefadjustment_normalized ~ dummytreat_direct1month", data=df_bad)
reg_3 = model_ols.fit(cov_type='HC1')
#(4)
model_ols = smf.ols(formula="beliefadjustment_normalized ~ dummytreat_direct1month + rank + beliefadjustment_bayes_norm", data=df_bad)
reg_4 = model_ols.fit(cov_type='HC1')
```

Next regressions are DID models, therefore interaction term for our interest group is added.

```python
#Generating interaction term
df["interact_direct1month"] = df["dummytreat_direct1month"]*df["dummynews_goodbad"]
```

```python
#(5)
model_ols = smf.ols(formula= "beliefadjustment_normalized ~ dummytreat_direct1month + dummynews_goodbad + interact_direct1month", data=df)
reg_5 = model_ols.fit(cov_type='HC1')
#(6)
model_ols = smf.ols(formula= "beliefadjustment_normalized ~ dummytreat_direct1month + dummynews_goodbad + rank + interact_direct1month + beliefadjustment_bayes_norm", data=df)
reg_6 = model_ols.fit(cov_type='HC1')
```

Next spesifications are with rank fixed effects to include their interaction with treatment. So rank dummies are generated and added to the regression as well as their interaction term: `rankdummy`$_i$ *`dummytreat_direct1month`.

```python
#Rank dummies
get_rank_dummies(df)
```

```python
#Intereaction term: Rank dummy*Treatment dummy
get_rank_interation_term(df)
```

```python
#(7)
model_ols = smf.ols(formula= "beliefadjustment_normalized ~ dummytreat_direct1month + dummynews_goodbad + interact_direct1month + rankdummy1 + rankdummy2 + rankdummy3 + rankdummy4 + rankdummy5 + rankdummy6 + rankdummy7 + rankdummy8 + rankdummy9 + rankdummy1_interact + rankdummy2_interact + rankdummy3_interact + rankdummy4_interact + rankdummy5_interact + rankdummy6_interact + rankdummy7_interact + rankdummy8_interact + rankdummy9_interact", data=df)
reg_7 = model_ols.fit(cov_type='HC1')
#(8)
model_ols = smf.ols(formula= "beliefadjustment_normalized ~ dummytreat_direct1month + dummynews_goodbad + interact_direct1month + beliefadjustment_bayes_norm + rankdummy1 + rankdummy2 + rankdummy3 + rankdummy4 + rankdummy5 + rankdummy6 + rankdummy7 + rankdummy8 + rankdummy9 + rankdummy1_interact + rankdummy2_interact + rankdummy3_interact + rankdummy4_interact + rankdummy5_interact + rankdummy6_interact + rankdummy7_interact + rankdummy8_interact + rankdummy9_interact", data=df)
reg_8 = model_ols.fit(cov_type='HC1')
```

```python
Main_Table_1(df)
```

Table 1 - Belief Adjustment: Direct versus One Month Later

	Normalized Belief Adjustment - beliefadjustment_normalized

	Positive Information	Negative Information	Difference-in-difference	Difference-in-difference with rank fixed effects

	(1)	(2)	(3)	(4)	(5)	(6)	(7)	(8)

Intercept	10.812***	-6.705**	14.247***	15.219***	10.812***	4.588**	9.237**	-2.317

	(1.604)	(2.762)	(1.687)	(5.600)	(1.604)	(2.217)	(4.421)	(4.209)

beliefadjustment_bayes_norm		0.674***		0.252***		0.391***		0.431***

		(0.071)		(0.081)		(0.055)		(0.061)

dummynews_goodbad					3.436	2.403	5.319	3.558

					(2.328)	(2.847)	(3.298)	(2.932)

dummytreat_direct1month	0.301	1.504	-10.411***	-11.006***	0.301	0.392	-0.245	-1.835

	(3.564)	(3.062)	(2.540)	(2.540)	(3.564)	(3.238)	(8.909)	(8.193)

interact_direct1month					-10.712**	-11.379***	-18.060**	-21.036***

					(4.377)	(4.144)	(7.698)	(7.628)

rank		1.416**		-0.910		-0.256

		(0.645)		(0.745)		(0.484)

rankdummy1							8.888	7.144

							(7.624)	(5.666)

rankdummy1_interact							-8.505	-0.388

							(11.556)	(9.580)

rankdummy2							3.526	6.168

							(5.247)	(4.381)

rankdummy2_interact							-2.651	-4.270

							(11.072)	(9.672)

rankdummy3							2.338	6.993

							(7.110)	(5.746)

rankdummy3_interact							-2.517	-1.159

							(12.656)	(11.293)

rankdummy4							-0.957	3.897

							(5.276)	(4.750)

rankdummy4_interact							7.368	13.099

							(11.043)	(11.020)

rankdummy5							-1.133	3.533

							(5.154)	(4.688)

rankdummy5_interact							7.657	12.868*

							(8.664)	(7.543)

rankdummy6							-0.485	3.931

							(4.809)	(4.433)

rankdummy6_interact							8.646	12.852

							(8.203)	(8.086)

rankdummy7							-5.460	3.522

							(4.343)	(4.370)

rankdummy7_interact							13.358	12.417

							(9.126)	(7.615)

rankdummy8							3.480	6.731

							(4.976)	(4.711)

rankdummy8_interact							2.621	6.698

							(6.694)	(6.624)

rankdummy9							-0.243	0.373

							(4.217)	(3.598)

rankdummy9_interact							12.301**	17.087***

							(6.068)	(5.082)

Observations	138	137	148	148	286	285	286	285

R2	0.000	0.308	0.097	0.175	0.044	0.195	0.076	0.234

Adjusted R2	-0.007	0.293	0.090	0.158	0.034	0.181	0.003	0.169

Residual Std. Error	18.466 (df=136)	15.512 (df=133)	15.495 (df=146)	14.911 (df=144)	16.993 (df=282)	15.668 (df=279)	17.267 (df=264)	15.775 (df=262)

F Statistic	0.007 (df=1; 136)	35.171*** (df=3; 133)	16.794*** (df=1; 146)	7.343*** (df=3; 144)	5.733*** (df=3; 282)	12.332*** (df=5; 279)	2.002*** (df=21; 264)	5.052*** (df=22; 262)

Note:

  *p<0.1;
  **p<0.05;
  ***p<0.01

From the first two regressions, it is clear that belief adjustments of individuals in the direct treatment group and the 1-month treatment group did not differ significantly. In other words, time has no significant effect on individuals belief adjustment if they have received positive feedback. These findings overlap with the finding in Zimmermann(2020) that state positive feedback has a persistent effect on belief dynamics.

The (3) and (4) regression also correlates with Zimmermann(2020) findings since it is clear that the belief adjustments are significantly different between treatment groups at the .01 level, referring to the short-term effect of negative feedback.

As it is observable from the column 5 and 6 the interaction term for negative feedback and 1-month treatment group is significant again at the .01 level, in line with the findings from the previous columns. The regression output columns with control variables (rank and Bayesian belief adjustment) also gave the same results in terms of significance levels which strengthens the results. Again these are also in line with the findings of Zimmermann (2020).

The last columns are the DID models with rank effects interacted with treatment group and also includes rank dummy controls, as it is observable the interaction term for negative feedback and 1-month treatment group is significant again at the .05 level in (7) and at the .01 level in (8).

Overall, all of the results and all of the values are exactly the same as the ones in the study of Zimmermann (2020); confirming the researcher's results.

---
## 6.2. Robustness Checks
---
Some parts of the Appendix that would be relavant for the extension part of this project is added to the project notebook.

### 6.2.1 Appendix A.4 - Alternative Definition of Positive/Negative Feedback

##### **Heterogenous Feedbacks**

Basically, to be considered to receive a positive or negative feedback, a subject should receive all the comparisons as the same kind; positive feedback if he/she received 3 positive comparisons and negative feedback if he/she received 3 negative comparisons.

```python
#Creating the new feedback dummy accordingly
df["dummynews_goodbad_h"] = np.nan
df.loc[df['pos_comparisons'] == 3, 'dummynews_goodbad_h'] = 0
df.loc[df['pos_comparisons'] == 0, 'dummynews_goodbad_h'] = 1
```

```python
Appendix_Table_1(df)
```

Appendix Table 1 - Belief Adjustment: Direct versus One Month Later

	Normalized Belief Adjustment - beliefadjustment_normalized

	Positive Information	Negative Information	Difference-in-difference

	(1)	(2)	(3)	(4)	(5)	(6)

Intercept	15.511***	-0.580	20.795***	31.651***	15.511***	8.473***

	(2.483)	(2.708)	(2.324)	(11.841)	(2.481)	(3.203)

beliefadjustment_bayes_norm		0.640***		0.121		0.384***

		(0.069)		(0.097)		(0.070)

dummynews_goodbad_h					5.284	7.631

					(3.401)	(6.612)

dummytreat_direct1month	0.447	0.855	-15.895***	-16.923***	0.447	0.137

	(3.919)	(2.930)	(2.840)	(3.113)	(3.917)	(3.067)

interact_direct1month					-16.343***	-16.571***

					(4.839)	(4.254)

rank		-0.288		-1.825		-1.161

		(0.872)		(1.294)		(0.880)

Observations	69	69	74	74	143	143

R2	0.000	0.459	0.265	0.326	0.133	0.319

Adjusted R2	-0.015	0.434	0.255	0.297	0.114	0.294

Residual Std. Error	16.051 (df=67)	11.986 (df=65)	13.181 (df=72)	12.799 (df=70)	14.635 (df=139)	13.068 (df=137)

F Statistic	0.013 (df=1; 67)	30.393*** (df=3; 65)	31.317*** (df=1; 72)	11.025*** (df=3; 70)	12.301*** (df=3; 139)	11.697*** (df=5; 137)

Note:

  *p<0.1;
  **p<0.05;
  ***p<0.01

All of the values mimic the values in Table 1 in terms of significance, which means the definition of feedback in the main study was robust. However, for the negative feedback, the coefficients are larger than the previous results which might suggest more negative feedback may affect belief adjustment more negatively.

---
### 6.2.2 Appendix A.6 - Updating in the Short-Run
---
Looking specifically at short-run updating is not the main aim of Zimmermann(2020), however, the researcher finds it relevant to include the results from data. As it is also mentioned in the study, previous findings on short-term belief updating suggest two phenomena: conservatism (Möbius et al., 2013) and asymmetry (Eil and Rao, 2011).

**Conservatism**

According to Bayes' rule, people should adjust their belief by about 20 percentage points, on average after a signal. However, Zimmermann (2020) found a way lower normalized belief adjustment value of 11.80 while the correlation between Bayesian predictions and belief adjustment is being 0.459.

```python
df_short = df[df['treatgroup'] == 4]
df_short['beliefadjustment_normalized'].mean()
```

    11.802197802197803

```python
correlation = df_short[['beliefadjustment_normalized','beliefadjustment_bayes_norm']].corr(method='pearson')
correlation
```

      beliefadjustment_normalized
      beliefadjustment_bayes_norm

      beliefadjustment_normalized
      1.0000000
      0.4585338

      beliefadjustment_bayes_norm
      0.4585338
      1.0000000

**Asymmetry**

In order to investigate asymmetry, Zimmermann (2020) analyze the effect of Bayesian predictions on belief adjustments, separately for positive and negative feedback.

```python
Appendix_Table_3(df)
```

Table 8: Belief Adjustment in the Short-Run

	Belief Adjustmentbeliefadjustment_normalized

	Positive Information	Negative Information	Difference-in-difference

	(1)	(2)	(3)

Intercept	1.636	5.930*	1.636

	(1.673)	(3.385)	(1.672)

beliefadjustment_bayes_norm	0.520***	0.306***	0.520***

	(0.168)	(0.111)	(0.168)

dummynews_goodbad			4.294

			(3.776)

interact_negative_bayes			-0.213

			(0.201)

Observations	45	46	91

R2	0.274	0.153	0.224

Adjusted R2	0.257	0.133	0.198

Residual Std. Error	13.566 (df=43)	13.907 (df=44)	13.739 (df=87)

F Statistic	9.595*** (df=1; 43)	7.558*** (df=1; 44)	9.610*** (df=3; 87)

Note:

  *p<0.1;
  **p<0.05;
  ***p<0.01

As it is suggested by the findings in Column 3, DID reveals no significant relationship. In other words, the Bayesian predictions don’t significantly differ between feedback groups. However, clearly, the coefficient for Bayesian prediction is slightly larger in positive information than in negative information which suggests some asymmetry.

---
### 6.2.3. Appendix A.7 - No Feedback Condition
---

```python
Appendix_Figure_1(df)
```

The figure clearly demonstrates that in absence of feedback, many subjects do not change their beliefs and a clear symmetry is observable for those who changed their beliefs.

```python
#Stats for control group; the only numbers I couldn't get exactly from the study of Zimmermann (2020) is the numbers below.
#The reported mean belief adjustment being 0.22, with a standard deviation of 17.83.
df[df['treatgroup'] == 7]['beliefadjustment'].describe()
```

    count    51.0000000
    mean      0.8039216
    std      15.4687034
    min     -50.0000000
    25%      -5.0000000
    50%       0.0000000
    75%       5.0000000
    max      40.0000000
    Name: beliefadjustment, dtype: float64

---
### 6.2.4. Appendix A.8 - Figures Bayesian Posteriors
---

```python
#Calculating Bayesian Posterior Averages
df['prior_av_b'] = df.groupby(['test_1', 'dummytreat_direct1month'])["prior_median"].transform('mean')
df['post_av_pos_b'] = df[df["dummynews_goodbad"] == 0].groupby(['test_1', 'dummytreat_direct1month'])["posterior_median_bayes"].transform('mean')
df['post_av_neg_b'] = df[df["dummynews_goodbad"] == 1].groupby(['test_1', 'dummytreat_direct1month'])["posterior_median_bayes"].transform('mean')
```

```python
Appendix_Figure_2(df)
```

This figure is the same as Figure 2 only here Bayes prediction of posterior beliefs are used instead of actual posteriors.

---
# 7. Extension
---

The extension consist of two parts.

In the first part, I will replicate the first part of the study of Eil and Rao (2011) which is a quite similar experiment in terms of experimental design and empirical strategy that has been used. I think replicating an experiment that Zimmermann(2020) had taken reference would be beneficial to understand the motives and aims of Zimmermann(2020) while forming his study in the way it has been done. Also, I made use of some parts of the Eil and Rao (2011) study in the second part of the extension.

In the second part, I will present the extensions I have added such as data visualizations and further analysis on the effect of feedback.

---
## 7.1. Replication of Eil and Rao (2011)

As an extension, I decided to replicate the study of Eil and Rao (2011) which is a relevant experiment that is also quite oftenly mentioned by Zimmermann (2020) in their study.

### 7.1.1. Introduction
The experiment consists of 3 main parts. I decided to replicate the first part of the experiment that analyzes the posterior beliefs of the subjects receiving the most “extreme” feedback(all positive or all negative).

Eil and Rao (2011) main aim in this experiment is to observe the asymmetry in the belief updating when subjects receive different types (positive and negative) of signals. Unlike Zimmermann (2020), Eil and Rao (2011) didn’t vary the time between treatments but varied the scoring method(referred to as image task) of subjects. The subjects who are assigned to an image task, either scored according to their IQ test or physical attractiveness(beauty) and also a control group task.

### 7.1.2. Experiment Design

The experiment is also quite similar to the study of Zimmermann(2020) by nature.

1. Subjects were first either taken an IQ test and assigned to groups of 10 or went through a series of speed dates with 10 other people and asked to give a rating to each in terms of their physical attractiveness from 1 to 10.
2. After the tasks, subjects were asked to elicit their belief about their ranking within their group of 10.
3. Then subjects received 3 cards that revealed whether they were ranked higher or lower than their randomly picked groupmates. For the control task, they were told the number on the cards was randomly picked. Belief elicitation was done in three rounds since subjects were asked to elicit their beliefs about their ranks within the group after each card.

**Variable Definitions**

|Variable name|Definition|Beauty|IQ|Control|
|---|---|---|---|---|
|`meanbelief_priorbayesimage`|Mean Bayesian belief prediction calculated from prior belief|√|√||
|`meanbelief_priorbayescard`|Mean Bayesian belief prediction calculated from prior belief|||√|
|`frac_upimage`|0 if good news, 1 if bad news|√|√||
|`frac_upcard`||||√|
|`mb_fracup`|interaction term: $meanbelief_priorbayesimage_i$*$frac_upimage_i$|√|√||
|`mb_fraccard`|interaction term: $meanbelief_priorbayescard_i$*$frac_upcard_i$|||√|

### 7.1.3. Emprical Strategy

Even though, a model is not specified in the study of Eil and Rao (2011). One can come up with the following DID model for the estimations presented in the Table 1:

\begin{equation}
meanbelief_i = \alpha + \beta meanBayesian_i + \gamma feedback_i + \delta I_i + \epsilon_i
\end{equation}

where $feedback_i$ is the dummy variable for feedback and $meanBayesian_i$ is the mean Bayesian belief prediction. $I_i$ is for the interaction term that is generated by $meanBayesian_i$*$feedback_i$. Bayesian predictions are calculated using reported priors and the received signal type; then the term mean Bayesian, `meanbelief_priorbayesimage`, is produced by taking the expected rank according to posteriors reported through the rounds.

### 7.1.4. Replication

```python
#Importing the adjusted and merged data set provided by the Eil and Rao (2011)
df_ex = pd.read_stata('data/extension.dta')
```

The first figure presented by Eil and Rao (2011) is formed by regressing the subject's observed mean belief on the predicted mean belief using Bayes' Rule and grouping subjects by the feedback type that they have received.

```python
Extension_Figure_1(df_ex)
```

At first glance, a visible asymmetry can be observed in the Beauty group between good and bad news; while such asymmetry is not that pronounced in the IQ test group. We interpret this asymmetry by the difference in slopes of the lines. As it can be seen in Panel A. bad news condition has a flatter slope, meaning that actual posteriors' have a shorter value length which means even though bayesian posteriors might suggest extreme updating the actual updating is relatively more conservative. In other words, the slope of the bad news line suggests an insensitivity to signal strength. Compared to bad news, we see a steeper slope for good news which suggests an exaggerated response to signal strength. Therefore, an asymmetry in responses for Beauty treatment is pronounced. If we look at Panel C. Control group we see a constant slope which implies that such asymmetries are the results of image treatments. The slope of the good news line in IQ treatments is also steeper compared to the control group however, not as much as it is in Beauty treatment.

Another feature to analyze in this figure is the observations scattered around the fitted line. If we compare the two treatment groups, it is clear that beliefs are noisier for bad news as it is for good news.

Also, it is worth noting that having slopes equal to zero and negative intercept would imply optimism in beliefs in general.

```python
#Used regressions in Extension Figure 2 with the standard errors clustered at subject level
reg_B_b_ = cluster_fit('meanbeliefimage ~ meanbelief_priorbayesimage + mb_fracup + frac_upimage',
                           data=df_ex[(df_ex['frac_upimage'] == 0) & (df_ex['IQtask'] ==0)], group_var='ID')
reg_B_g_ = cluster_fit('meanbeliefimage ~ meanbelief_priorbayesimage + mb_fracup + frac_upimage',
                           data=df_ex[(df_ex['frac_upimage'] == 1) & (df_ex['IQtask'] ==0)], group_var='ID')

reg_IQ_b_ = cluster_fit('meanbeliefimage ~ meanbelief_priorbayesimage + mb_fracup + frac_upimage',
                            data=df_ex[(df_ex['frac_upimage'] == 0) & (df_ex['IQtask'] ==1)], group_var='ID')
reg_IQ_g_ = cluster_fit('meanbeliefimage ~ meanbelief_priorbayesimage + mb_fracup + frac_upimage',
                            data=df_ex[(df_ex['frac_upimage'] == 1) & (df_ex['IQtask'] ==1)], group_var='ID')

reg_C_b_ = cluster_fit('meanbeliefcard ~ meanbelief_priorbayescard + mb_fracupcard + frac_upcard',
                           data=df_ex[(df_ex['frac_upcard'] == 0)], group_var='ID')
reg_C_g_ = cluster_fit('meanbeliefcard ~ meanbelief_priorbayescard + mb_fracupcard + frac_upcard',
                           data=df_ex[(df_ex['frac_upcard'] == 0)], group_var='ID')
```

```python
Extension_Figure_2(df_ex)
```

Panel. D represents the plots of the R^2s from the regressions used in Extension Table 1 but divided according to the type of feedback. Unfortunately, even though I used regressions with standard errors clustered at the subject level as is described by Eil and Rao (2011); I couldn’t get the R^2 value of the IQ test/good newsgroup right, which is confusing since I get exactly the same result from the pooled regression in the Extension Table 1. According to Eil and Rao (2011), the $R^2$ for IQ/good newsgroup should be below 0.7.

Taking the result of Eil and Rao (2011) as a reference point, R^2 differ for the Beauty and IQ group while that is obviously not the case for the Control group.

**Table 1**

```python
#Used regressions in Extension Table 1 with the standard errors clustered at subject level
reg_B = cluster_fit('meanbeliefimage ~ meanbelief_priorbayesimage + mb_fracup + frac_upimage', data=df_ex[(((df_ex['frac_upimage'] == 0) | (df_ex['frac_upimage'] == 1)) & (df_ex['IQtask'] ==0))], group_var='ID')
reg_IQ = cluster_fit('meanbeliefimage ~ meanbelief_priorbayesimage + mb_fracup + frac_upimage', data=df_ex[(((df_ex['frac_upimage'] == 0) | (df_ex['frac_upimage'] == 1)) & (df_ex['IQtask'] ==1))], group_var='ID')
reg_C = cluster_fit('meanbeliefcard ~ meanbelief_priorbayescard + mb_fracupcard + frac_upcard', data=df_ex[(df_ex['frac_upcard'] == 0) | (df_ex['frac_upcard'] == 1)], group_var='ID')
```

```python
Extension_Table_1(df_ex)
```

    ===================================================
                                Beauty     IQ   Control
    ---------------------------------------------------
    Intercept                  2.05***  1.05    2.29***
                               (0.52)   (0.76)  (0.76)
    R-squared                  0.87     0.87    0.72
    R-squared Adj.             0.86     0.87    0.71
    frac_upcard                                 -0.53
                                                (0.95)
    frac_upimage               -2.31*** -0.58
                               (0.54)   (0.81)
    mb_fracup                  0.41***  0.07
                               (0.10)   (0.13)
    mb_fracupcard                               -0.00
                                                (0.19)
    meanbelief_priorbayescard                   0.59***
                                                (0.10)
    meanbelief_priorbayesimage 0.64***  0.85***
                               (0.08)   (0.10)
    ===================================================
    Standard errors in parentheses.
    * p<.1, ** p<.05, ***p<.01

---
*See below for a reader friendly version of the Extension Table 1 that I created with the same values.*

||Beauty|IQ|Control|
|---|---|---|---|
|Intercept|**2.05** |1.05 |**2.29** |
||(0.52)|(0.76)|(0.76)|
|Bayesian μ|**0.64** |**0.85** |**0.59** |
||(0.08)|(0.85)|(0.10)|
|Bayesian μ *1{All positive feedback}|**0.41** |0.07 |-0.00 |
||(0.10)|(0.13)|(0.19)|
|1{All positive feedback}|**-2.31** |-0.58 |-0.53 |
||(0.54)|(0.81)|(0.95)|
|Intercept|**2.05** |**1.05** |**2.29**|
||(0.52)|(0.76)|(0.76)|
|$R^2$|0.87|0.87|0.72 |
|$R^2$ Adj.|0.86|0.87 |0.71|
||Standard errors in parentheses. * p<.1, *p<.05*, ***p<.01|

The findings found in Extention Table1 are in furtherance of the interpretations made for the Extention Figure2. Like it is suggested by the figure, the asymmetry in belief dynamics-the difference between positive and negative feedback conditions- is only significant in Beauty at the 0.01 level of significance.

### 7.1.5. Comparison with Zimmermann (2020)

It is easy to see the two experiments share a lot of experimental and empirical strategies. First of all, both studies have a longitudinal feature since they elicit subjects’ beliefs two times. Secondly, the two studies have the same focus which is the asymmetry in belief dynamics that arise as a result of feedback type. In addition, both studies make use of DID models in their predictions. Nevertheless, while Eil and Rao (2011) are used the heterogeneous definition of feedbacks, Zimmermann(2020) choose to allow homogenous comparison types within feedback groups- even though a robustness check is made (see Appendix A.4)-.  A fundamental difference is the employment of treatment groups. While Eil and Rao (2011) were interested in belief updating in different tasks (Beauty and IQ), Zimmermann (2020) was interested in belief updating when elicitation time differs (Direct and 1 month). Therefore, all of the results Eil and Rao (2011) found was derived from short-run updating. At this point, it is intriguing that whether the difference between the tasks would be as pronounce as it was; if Zimmermann (2020) also had a Beauty image task that is also measured after 1 month.

---
## 7.2. Further Extension
---
### 7.2.1. Alternative Data Visualizations

```python
Extension_Figure_3(df)
```

As a small extension, I decided to create a figure to combine the figure from Appendix A.8 with Figure 2 since it would be a good way to view posterior beliefs in comparison with Bayesian predictions.

If we look at the first row we see that observed beliefs follow a relatively similar pattern, in terms of subjects' reported percentages of being in the upper half of the group, to Bayesian predictions compared to the second row where graphs differ visibly.

Also, in the short run, it is easy to see that people are conservative in belief updating. For instance, in the group of high test scores and positive feedback, according to Bayes' Rule, one would expect a posterior slightly above %90 while it is only slightly above %80; suggesting a difference near %10 percent.

The opposite can be observable in long-run belief updating in the negative feedback group since in every test score group people reported posterior beliefs that are much higher than Bayesian predictions-especially for the individuals with lower test scores. What I found interesting in the results from long-run, is the difference in belief updating when feedback is positive. In the last figure, we see that no matter the test scores individuals tend to report a belief near %80 percent when they receive positive feedback. I think this would be a good starting point for further research in dynamic beliefs.

---
Making use of Extention Figure 2 from the study of Eil and Rao (2011), I created two similar data visualization using the data of Zimmermann (2020).

```python
Extension_Figure_4(df)
```

In both figures, negative feedback lines are flatter and specifically very flat in Panel B. 1 month treatment. Meaning that in both treatment groups subjects'  weighed negative information less heavily than positive information and even much less heavily in 1-month treatment.

```python
Extension_Figure_5(df)
```

In this visiualization, it is easier to see the asymmetry between positive and negative feedback. If we look at Panel A. Positive Feedback, we see that slopes of fitted lines of direct and 1month treatment are almost the same; while in Panel B. they clearly differ.

## 7.2.2 Further Analysis with No Feedback Group

As a control group, Zimmermann (2020) also added a group where no feedback is given to the subjects. The main aim of adding this group was the investigate the potential time trends in beliefs that may exist independently of receiving feedback. However, I couldn't find any further analyzes with this group neither in the study or appendix-except the visiualization in Appendix A.7.

So as a further analyze I decided to investigate the difference between feedback(positive or negative) and no feedback groups within the 1-month condition to see whether receiving feedback itself cause a difference in belief adjustment.

### 7.2.2.1. Estimation Strategy

For my analysis, I made use of a simple difference-in-difference model of the following kind:

\begin{equation}
belief_i = \alpha + \beta dummyfeedback_i + \gamma secondwave_i + \delta I_i + \epsilon_i
\end{equation}

where $dummyfeedback_i$ is the dummy variable for receiving feedback and $secondwave_i$ is the dummy for posterior belief. $I_i$ is for the interaction term that is generated by $dummyfeedback_i$*$secondwave_i$. If the subjects obtained feedback and the belief is a posterior belief, (i)feedback dummy (ii)secondwave dummy and (iii) interaction term are equal to 1; making the $ \delta$ the coefficient of interest.

For the estimation, I reshaped data to have a single belief variable combining posterior and prior beliefs and to have a time variable that qualifies in which time the belief is elicited.

### 7.2.2.2. Estimation

```python
#Making a copy of the original data to start the reshape
df_control = df
```

```python
#Generating an ID number for each participant
df_control['ID'] = df_control.index + 1
```

```python
#Filtering data to have only 1 month and no feedback treatment
df_control = df[(df['treatgroup'] == 2) | (df['treatgroup'] == 7)]
```

```python
#Filtering and adjusting data
df_control = df[(df['treatgroup'] == 2) | (df['treatgroup'] == 7)]
df_control = df_control[['ID', 'treatment', 'posterior_median', 'prior_median']]
df_control.rename(columns = {"prior_median": "belief1", "posterior_median": "belief2" }, inplace=True)
df_control["ID"] = df_control.index
```

```python
#Transforming data from a wide format to long format
df_control = pd.wide_to_long(df_control, stubnames='belief', i="ID", j="time")
```

```python
#Creating a column with time index
df_control = df_control.reset_index(level=1)
```

```python
#Final data
df_control
```

      time
      treatment
      belief

      ID

      115
      2
      confidence_1monthlater
      90.0

      116
      2
      confidence_1monthlater
      50.0

      117
      2
      confidence_1monthlater
      30.0

      118
      2
      confidence_1monthlater
      80.0

      119
      2
      confidence_1monthlater
      0.0

      ...
      ...
      ...
      ...

      679
      1
      nofeedback
      100.0

      680
      1
      nofeedback
      55.0

      681
      1
      nofeedback
      50.0

      682
      1
      nofeedback
      25.0

      683
      1
      nofeedback
      70.0

318 rows × 3 columns

```python
get_variables_for_extension(df_control)
```

```python
Extension_Table_2(df_control)
```

OLS Regression Results

  Dep. Variable:	         belief	        R-squared:         	    0.005

  Model:	                   OLS	         Adj. R-squared:    	   -0.005

  Method:	             Least Squares	    F-statistic:       	   0.5177

  Date:	             Thu, 22 Jul 2021	   Prob (F-statistic):	   0.670

  Time:	                 23:31:30	       Log-Likelihood:    	  -1458.1

  No. Observations:	         318	        AIC:               	    2924.

  Df Residuals:	             314	        BIC:               	    2939.

  Df Model:	                   3

  Covariance Type:	         HC1

           	              coef	     std err	      z	      P>|z|	  [0.025	    0.975]

  Intercept	              65.9608	     3.797	    17.371	  0.000	    58.518	    73.403

  dummy_feedback	         -3.1182	     4.358	    -0.715	  0.474	   -11.660	     5.424

  secondwave	              0.2157	     5.185	     0.042	  0.967	    -9.947	    10.378

  feedback_secondwave	     3.2843	     6.035	     0.544	  0.586	    -8.544	    15.113

  Omnibus:	        9.123	   Durbin-Watson:     	    1.766

  Prob(Omnibus):	  0.010	   Jarque-Bera (JB):  	    8.163

  Skew:	          -0.327	   Prob(JB):          	   0.0169

  Kurtosis:	       2.566	   Cond. No.          	     8.62

Notes:
[1] Standard Errors are heteroscedasticity robust (HC1)

The p-value of the interaction term implies that the difference is insignificant. Therefore, one can conclude that receiving feedback does not have an effect on beliefs solely. In other words, the change in beliefs is not significant between feedback and no feedback group.

---
# 8. Conclusion
---

In sum, all of the results I get from using the data and STATA codes provided by Zimmmermann (2020) overlaps with the ones in the study. Except for the stats in Appendix A.7., I manage to get the same numbers and figures presented in the study. Similarly, for the extension replication, I used the merged and reshaped data provided by Eil and Rao. Using the Stata codes provided by the researchers, I manage to get the same results except the $R^2$ values represented in the Extension Figure 2. In light of these, I can conclude that even though individuals adjust their beliefs accordingly- and in line with Bayesian predictions- in short term; the effects of negative feedback tend to fade away over time while positive feedback has a long-lasting effect on belief adjustment. According to my analysis done with the control group, the feedback itself solely doesn't have a significant effect on beliefs. Finally, the Difference-in-Difference models used in both of the studies and my analysis, are good tools when researchers want to analyze observations draw from a between-subjects experimental design.

---
# 9. References
---

Benoît, Jean-Pierre, and Juan Dubra. 2011. “Apparent Overconfidence?” [*Econometrica* 79 (5):
1591–1625.](https://doi.org/10.3982/ECTA8583)

Eil, David, and Justin M. Rao. 2011. “The Good News-Bad News Effect: Asymmetric Processing of Objective Information about Yourself.” [*American Economic Journal: Microeconomics* 3 (2): 114–38.](https://doi.org/10.1257/mic.3.2.114)

Möbius, Markus M., Muriel Niederle, Paul Niehaus, and Tanya S. Rosenblat. 2013. “Managing
Self-Confidence: Theory and Experimental Evidence.” Unpublished.

Zimmermann, F. 2020. “The Dynamics of Motivated Beliefs.” [*American Economic Journal: Microeconomics* 110 (2): 337–363.](https://doi.org/10.1257/aer.20180728)

-------
Notebook by Aysu Avcı | Find me on GitHub at https://github.com/aysuavci.

---


```
!pip install cirq
```

    Requirement already satisfied: cirq in /usr/local/lib/python3.6/dist-packages (0.8.1)
    Requirement already satisfied: google-api-core[grpc]<2.0.0dev,>=1.14.0 in /usr/local/lib/python3.6/dist-packages (from cirq) (1.16.0)
    Requirement already satisfied: typing-extensions in /usr/local/lib/python3.6/dist-packages (from cirq) (3.6.6)
    Requirement already satisfied: networkx~=2.4 in /usr/local/lib/python3.6/dist-packages (from cirq) (2.4)
    Requirement already satisfied: requests~=2.18 in /usr/local/lib/python3.6/dist-packages (from cirq) (2.23.0)
    Requirement already satisfied: scipy in /usr/local/lib/python3.6/dist-packages (from cirq) (1.4.1)
    Requirement already satisfied: sortedcontainers~=2.0 in /usr/local/lib/python3.6/dist-packages (from cirq) (2.2.2)
    Requirement already satisfied: sympy in /usr/local/lib/python3.6/dist-packages (from cirq) (1.1.1)
    Requirement already satisfied: dataclasses; python_version < "3.7" in /usr/local/lib/python3.6/dist-packages (from cirq) (0.7)
    Requirement already satisfied: matplotlib~=3.0 in /usr/local/lib/python3.6/dist-packages (from cirq) (3.2.2)
    Requirement already satisfied: protobuf==3.8.0 in /usr/local/lib/python3.6/dist-packages (from cirq) (3.8.0)
    Requirement already satisfied: pandas in /usr/local/lib/python3.6/dist-packages (from cirq) (1.0.5)
    Requirement already satisfied: freezegun~=0.3.15 in /usr/local/lib/python3.6/dist-packages (from cirq) (0.3.15)
    Requirement already satisfied: numpy~=1.16 in /usr/local/lib/python3.6/dist-packages (from cirq) (1.18.5)
    Requirement already satisfied: setuptools>=34.0.0 in /usr/local/lib/python3.6/dist-packages (from google-api-core[grpc]<2.0.0dev,>=1.14.0->cirq) (47.3.1)
    Requirement already satisfied: pytz in /usr/local/lib/python3.6/dist-packages (from google-api-core[grpc]<2.0.0dev,>=1.14.0->cirq) (2018.9)
    Requirement already satisfied: google-auth<2.0dev,>=0.4.0 in /usr/local/lib/python3.6/dist-packages (from google-api-core[grpc]<2.0.0dev,>=1.14.0->cirq) (1.17.2)
    Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in /usr/local/lib/python3.6/dist-packages (from google-api-core[grpc]<2.0.0dev,>=1.14.0->cirq) (1.52.0)
    Requirement already satisfied: six>=1.10.0 in /usr/local/lib/python3.6/dist-packages (from google-api-core[grpc]<2.0.0dev,>=1.14.0->cirq) (1.12.0)
    Requirement already satisfied: grpcio<2.0dev,>=1.8.2; extra == "grpc" in /usr/local/lib/python3.6/dist-packages (from google-api-core[grpc]<2.0.0dev,>=1.14.0->cirq) (1.30.0)
    Requirement already satisfied: decorator>=4.3.0 in /usr/local/lib/python3.6/dist-packages (from networkx~=2.4->cirq) (4.4.2)
    Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.6/dist-packages (from requests~=2.18->cirq) (2.9)
    Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.6/dist-packages (from requests~=2.18->cirq) (1.24.3)
    Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.6/dist-packages (from requests~=2.18->cirq) (3.0.4)
    Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.6/dist-packages (from requests~=2.18->cirq) (2020.6.20)
    Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.6/dist-packages (from sympy->cirq) (1.1.0)
    Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib~=3.0->cirq) (2.4.7)
    Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib~=3.0->cirq) (2.8.1)
    Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.6/dist-packages (from matplotlib~=3.0->cirq) (0.10.0)
    Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib~=3.0->cirq) (1.2.0)
    Requirement already satisfied: cachetools<5.0,>=2.0.0 in /usr/local/lib/python3.6/dist-packages (from google-auth<2.0dev,>=0.4.0->google-api-core[grpc]<2.0.0dev,>=1.14.0->cirq) (4.1.0)
    Requirement already satisfied: rsa<5,>=3.1.4; python_version >= "3" in /usr/local/lib/python3.6/dist-packages (from google-auth<2.0dev,>=0.4.0->google-api-core[grpc]<2.0.0dev,>=1.14.0->cirq) (4.6)
    Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.6/dist-packages (from google-auth<2.0dev,>=0.4.0->google-api-core[grpc]<2.0.0dev,>=1.14.0->cirq) (0.2.8)
    Requirement already satisfied: pyasn1>=0.1.3 in /usr/local/lib/python3.6/dist-packages (from rsa<5,>=3.1.4; python_version >= "3"->google-auth<2.0dev,>=0.4.0->google-api-core[grpc]<2.0.0dev,>=1.14.0->cirq) (0.4.8)

```
import cirq
```

```
#Applying NOT gate to a qubit and measuring it.
qubit = cirq.GridQubit(0, 0) #picking the qubit

circuit = cirq.Circuit([
                                 cirq.X(qubit),
                                 cirq.measure(qubit, key = 'm')
                                 ]
)
print("The circuit:")
print(circuit)
```

    The circuit:
    (0, 0): ───X───M('m')───

```
simulator = cirq.Simulator()
result = simulator.run(circuit, repetitions = 10)
print("Results:")
print(result)
```

    Results:
    m=1111111111

```
#Preparing a Bell States |phi^+>

import cirq

qreg = [cirq.LineQubit(x) for x in range(2)]
circ = cirq.Circuit()

circ.append([cirq.H(qreg[0]), cirq.CNOT(qreg[0], qreg[1])])

print("Circuit")
print(circ)

circ.append(cirq.measure(*qreg, key="z"))

sim = cirq.Simulator()
res = sim.run(circ, repetitions = 100)

print("\nMeasurements:")
print(res.histogram(key="z"))

```

    Circuit
    0: ───H───@───
              │
    1: ───────X───

    Measurements:
    Counter({3: 56, 0: 44})

```

```


# Damage Mapper Examples

This Jupyter notebook mainly demonstrates the various function in the Damage Mapper section of the project

```python
!pip install -e .
!pip install -r requirements.txt
import armageddon
import numpy as np
import matplotlib.pyplot as plt
from collections import OrderedDict
```

    Obtaining file:///home/yhc97/acse/ACSE-4/1-armageddon
    Installing collected packages: Armageddon
      Attempting uninstall: Armageddon
        Found existing installation: Armageddon 1.0
        Uninstalling Armageddon-1.0:
          Successfully uninstalled Armageddon-1.0
      Running setup.py develop for Armageddon
    Successfully installed Armageddon
    Requirement already satisfied: numpy>=1.13.0 in /home/yhc97/anaconda3/lib/python3.8/site-packages (from -r requirements.txt (line 1)) (1.18.5)
    Requirement already satisfied: scipy in /home/yhc97/anaconda3/lib/python3.8/site-packages (from -r requirements.txt (line 2)) (1.5.0)
    Requirement already satisfied: sympy in /home/yhc97/anaconda3/lib/python3.8/site-packages (from -r requirements.txt (line 3)) (1.6.1)
    Requirement already satisfied: pandas in /home/yhc97/anaconda3/lib/python3.8/site-packages (from -r requirements.txt (line 4)) (1.0.5)
    Requirement already satisfied: matplotlib in /home/yhc97/anaconda3/lib/python3.8/site-packages (from -r requirements.txt (line 5)) (3.2.2)
    Requirement already satisfied: mpltools in /home/yhc97/anaconda3/lib/python3.8/site-packages (from -r requirements.txt (line 6)) (0.2.0)
    Requirement already satisfied: pytest in /home/yhc97/anaconda3/lib/python3.8/site-packages (from -r requirements.txt (line 7)) (5.4.3)
    Requirement already satisfied: pytest-timeout in /home/yhc97/anaconda3/lib/python3.8/site-packages (from -r requirements.txt (line 8)) (1.4.2)
    Requirement already satisfied: sphinx in /home/yhc97/anaconda3/lib/python3.8/site-packages (from -r requirements.txt (line 9)) (3.1.2)
    Requirement already satisfied: dask in /home/yhc97/anaconda3/lib/python3.8/site-packages (from -r requirements.txt (line 10)) (2.20.0)
    Requirement already satisfied: distributed in /home/yhc97/anaconda3/lib/python3.8/site-packages (from -r requirements.txt (line 11)) (2.20.0)
    Requirement already satisfied: googledrivedownloader in /home/yhc97/anaconda3/lib/python3.8/site-packages (from -r requirements.txt (line 12)) (0.4)
    Requirement already satisfied: folium in /home/yhc97/anaconda3/lib/python3.8/site-packages (from -r requirements.txt (line 13)) (0.12.0)
    Requirement already satisfied: mpmath>=0.19 in /home/yhc97/anaconda3/lib/python3.8/site-packages (from sympy->-r requirements.txt (line 3)) (1.1.0)
    Requirement already satisfied: python-dateutil>=2.6.1 in /home/yhc97/anaconda3/lib/python3.8/site-packages (from pandas->-r requirements.txt (line 4)) (2.8.1)
    Requirement already satisfied: pytz>=2017.2 in /home/yhc97/anaconda3/lib/python3.8/site-packages (from pandas->-r requirements.txt (line 4)) (2020.1)
    Requirement already satisfied: kiwisolver>=1.0.1 in /home/yhc97/anaconda3/lib/python3.8/site-packages (from matplotlib->-r requirements.txt (line 5)) (1.2.0)
    Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /home/yhc97/anaconda3/lib/python3.8/site-packages (from matplotlib->-r requirements.txt (line 5)) (2.4.7)
    Requirement already satisfied: cycler>=0.10 in /home/yhc97/anaconda3/lib/python3.8/site-packages (from matplotlib->-r requirements.txt (line 5)) (0.10.0)
    Requirement already satisfied: configobj>=4.4.0 in /home/yhc97/anaconda3/lib/python3.8/site-packages (from mpltools->-r requirements.txt (line 6)) (5.0.6)
    Requirement already satisfied: future>=0.12.4 in /home/yhc97/anaconda3/lib/python3.8/site-packages (from mpltools->-r requirements.txt (line 6)) (0.18.2)
    Requirement already satisfied: py>=1.5.0 in /home/yhc97/anaconda3/lib/python3.8/site-packages (from pytest->-r requirements.txt (line 7)) (1.9.0)
    Requirement already satisfied: packaging in /home/yhc97/anaconda3/lib/python3.8/site-packages (from pytest->-r requirements.txt (line 7)) (20.4)
    Requirement already satisfied: attrs>=17.4.0 in /home/yhc97/anaconda3/lib/python3.8/site-packages (from pytest->-r requirements.txt (line 7)) (18.2.0)
    Requirement already satisfied: more-itertools>=4.0.0 in /home/yhc97/anaconda3/lib/python3.8/site-packages (from pytest->-r requirements.txt (line 7)) (8.4.0)
    Requirement already satisfied: pluggy<1.0,>=0.12 in /home/yhc97/anaconda3/lib/python3.8/site-packages (from pytest->-r requirements.txt (line 7)) (0.13.1)
    Requirement already satisfied: wcwidth in /home/yhc97/anaconda3/lib/python3.8/site-packages (from pytest->-r requirements.txt (line 7)) (0.2.5)
    Requirement already satisfied: snowballstemmer>=1.1 in /home/yhc97/anaconda3/lib/python3.8/site-packages (from sphinx->-r requirements.txt (line 9)) (2.0.0)
    Requirement already satisfied: sphinxcontrib-jsmath in /home/yhc97/anaconda3/lib/python3.8/site-packages (from sphinx->-r requirements.txt (line 9)) (1.0.1)
    Requirement already satisfied: Pygments>=2.0 in /home/yhc97/anaconda3/lib/python3.8/site-packages (from sphinx->-r requirements.txt (line 9)) (2.6.1)
    Requirement already satisfied: sphinxcontrib-devhelp in /home/yhc97/anaconda3/lib/python3.8/site-packages (from sphinx->-r requirements.txt (line 9)) (1.0.2)
    Requirement already satisfied: setuptools in /home/yhc97/anaconda3/lib/python3.8/site-packages (from sphinx->-r requirements.txt (line 9)) (49.2.0.post20200714)
    Requirement already satisfied: sphinxcontrib-serializinghtml in /home/yhc97/anaconda3/lib/python3.8/site-packages (from sphinx->-r requirements.txt (line 9)) (1.1.4)
    Requirement already satisfied: docutils>=0.12 in /home/yhc97/anaconda3/lib/python3.8/site-packages (from sphinx->-r requirements.txt (line 9)) (0.16)
    Requirement already satisfied: sphinxcontrib-htmlhelp in /home/yhc97/anaconda3/lib/python3.8/site-packages (from sphinx->-r requirements.txt (line 9)) (1.0.3)
    Requirement already satisfied: babel>=1.3 in /home/yhc97/anaconda3/lib/python3.8/site-packages (from sphinx->-r requirements.txt (line 9)) (2.8.0)
    Requirement already satisfied: alabaster<0.8,>=0.7 in /home/yhc97/anaconda3/lib/python3.8/site-packages (from sphinx->-r requirements.txt (line 9)) (0.7.12)
    Requirement already satisfied: sphinxcontrib-applehelp in /home/yhc97/anaconda3/lib/python3.8/site-packages (from sphinx->-r requirements.txt (line 9)) (1.0.2)
    Requirement already satisfied: imagesize in /home/yhc97/anaconda3/lib/python3.8/site-packages (from sphinx->-r requirements.txt (line 9)) (1.2.0)
    Requirement already satisfied: Jinja2>=2.3 in /home/yhc97/anaconda3/lib/python3.8/site-packages (from sphinx->-r requirements.txt (line 9)) (2.11.2)
    Requirement already satisfied: sphinxcontrib-qthelp in /home/yhc97/anaconda3/lib/python3.8/site-packages (from sphinx->-r requirements.txt (line 9)) (1.0.3)
    Requirement already satisfied: requests>=2.5.0 in /home/yhc97/anaconda3/lib/python3.8/site-packages (from sphinx->-r requirements.txt (line 9)) (2.22.0)
    Requirement already satisfied: pyyaml in /home/yhc97/anaconda3/lib/python3.8/site-packages (from dask->-r requirements.txt (line 10)) (5.3.1)
    Requirement already satisfied: tornado>=6.0.3; python_version >= "3.8" in /home/yhc97/anaconda3/lib/python3.8/site-packages (from distributed->-r requirements.txt (line 11)) (6.0.4)
    Requirement already satisfied: zict>=0.1.3 in /home/yhc97/anaconda3/lib/python3.8/site-packages (from distributed->-r requirements.txt (line 11)) (2.0.0)
    Requirement already satisfied: toolz>=0.8.2 in /home/yhc97/anaconda3/lib/python3.8/site-packages (from distributed->-r requirements.txt (line 11)) (0.10.0)
    Requirement already satisfied: msgpack>=0.6.0 in /home/yhc97/anaconda3/lib/python3.8/site-packages (from distributed->-r requirements.txt (line 11)) (1.0.0)
    Requirement already satisfied: psutil>=5.0 in /home/yhc97/anaconda3/lib/python3.8/site-packages (from distributed->-r requirements.txt (line 11)) (5.7.0)
    Requirement already satisfied: tblib>=1.6.0 in /home/yhc97/anaconda3/lib/python3.8/site-packages (from distributed->-r requirements.txt (line 11)) (1.6.0)
    Requirement already satisfied: cloudpickle>=1.3.0 in /home/yhc97/anaconda3/lib/python3.8/site-packages (from distributed->-r requirements.txt (line 11)) (1.5.0)
    Requirement already satisfied: sortedcontainers!=2.0.0,!=2.0.1 in /home/yhc97/anaconda3/lib/python3.8/site-packages (from distributed->-r requirements.txt (line 11)) (2.2.2)
    Requirement already satisfied: click>=6.6 in /home/yhc97/anaconda3/lib/python3.8/site-packages (from distributed->-r requirements.txt (line 11)) (7.1.2)
    Requirement already satisfied: branca>=0.3.0 in /home/yhc97/anaconda3/lib/python3.8/site-packages (from folium->-r requirements.txt (line 13)) (0.4.2)
    Requirement already satisfied: six>=1.5 in /home/yhc97/.local/lib/python3.8/site-packages (from python-dateutil>=2.6.1->pandas->-r requirements.txt (line 4)) (1.15.0)
    Requirement already satisfied: MarkupSafe>=0.23 in /home/yhc97/anaconda3/lib/python3.8/site-packages (from Jinja2>=2.3->sphinx->-r requirements.txt (line 9)) (1.1.1)
    Requirement already satisfied: certifi>=2017.4.17 in /home/yhc97/anaconda3/lib/python3.8/site-packages (from requests>=2.5.0->sphinx->-r requirements.txt (line 9)) (2019.11.28)
    Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /home/yhc97/anaconda3/lib/python3.8/site-packages (from requests>=2.5.0->sphinx->-r requirements.txt (line 9)) (1.25.7)
    Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/yhc97/anaconda3/lib/python3.8/site-packages (from requests>=2.5.0->sphinx->-r requirements.txt (line 9)) (3.0.4)
    Requirement already satisfied: idna<2.9,>=2.5 in /home/yhc97/anaconda3/lib/python3.8/site-packages (from requests>=2.5.0->sphinx->-r requirements.txt (line 9)) (2.8)
    Requirement already satisfied: heapdict in /home/yhc97/anaconda3/lib/python3.8/site-packages (from zict>=0.1.3->distributed->-r requirements.txt (line 11)) (1.0.1)

```python
# call by keyword
planet = armageddon.Planet()
locator = armageddon.PostcodeLocator()
```

```python
params = {'radius': 60, 'angle': 20, 'strength': 1e6,
          'density': 3000, 'velocity': 19e3,
          'lat': 55., 'lon': -2, 'bearing': 180.}

result = planet.solve_atmospheric_entry(params['radius'], params['velocity'],
                                        params['density'], params['strength'], params['angle'])
outcome = planet.analyse_outcome(result=planet.calculate_energy(result))
outcome
```

    {'outcome': 'Airburst',
     'burst_peak_dedz': 12925.271090322896,
     'burst_altitude': 16460.83220034911,
     'burst_distance': 238856.6954917025,
     'burst_energy': 79421.66078548192}

```python
# use a sample strike event
zb = outcome['burst_altitude']
Ek = outcome['burst_energy']
# using the pressure function defined in the DamageMapper.ipynb
P = lambda r: 3.14 * 10.**11 * ((r**2 + zb**2)/ Ek**(2./3))**(-1.3) + \
                1.8 * 10.**7 * ((r**2 + zb**2)/ Ek**(2./3))**(-0.565)

entry_point = [params['lat'], params['lon']]
pressures = [1e3, 3.5e3, 27e3, 43e3]
blat, blon, damrad = armageddon.damage_zones(outcome, entry_point[0], entry_point[1], params['bearing'], pressures)
burst_point = [blat, blon]

x = np.linspace(0, 4e5, 1000)

# verification plots, plotting the damrad on the function P to check the results visually
plt.plot(x, P(x), label="Pressure Function")
i = 1
for (loc, pres) in zip(damrad, pressures):
    plt.plot(loc, pres, 'o', label='damage level: ' + str(i) + ' boundary')
    plt.hlines(pres, 0, loc, linestyles='dashed')
    i += 1
plt.title("Damage radius of the sample burst")
plt.legend()
```

## Extension one

Following cells show how the different damage zone and population data can be plotted on the map for effective risk assessment and evacuation planning

```python
# The mapper function then plot the damage zones onto google maps
armageddon.plot_damage_zones(entry_point, burst_point, damrad)
```

Make this Notebook Trusted to load map: File -> Trust Notebook

```python
# to get the impacted postcodes in each damage zone:
sector = True
z = locator.get_postcodes_by_radius(burst_point, damrad, sector=sector)
```

    Number of postcode sectors in damage zone 1: ~10% glass windows shatter: 9269
    Number of postcode sectors in damage zone 2: ~90% glass windows shatter: 2796
    Number of postcode sectors in damage zone 3: Wood frame buildings collapse: 113
    Number of postcode sectors in damage zone 4: Multistory brick buildings collapse: 37

```python
damage_zones = [
    '1: ~10% glass windows shatter',
    '2: ~90% glass windows shatter',
    '3: Wood frame buildings collapse',
    '4: Multistory brick buildings collapse'
]

for i in range(len(z)):
    if sector:
        print('Number of postcode sectors in damage zone ' + damage_zones[i] + ': ' + str(len(z[i])))
    else:
        print('Number of postcodes in damage zone ' + damage_zones[i] + ': ' + str(len(z[i])))
```

    Number of postcode sectors in damage zone 1: ~10% glass windows shatter: 9269
    Number of postcode sectors in damage zone 2: ~90% glass windows shatter: 2796
    Number of postcode sectors in damage zone 3: Wood frame buildings collapse: 113
    Number of postcode sectors in damage zone 4: Multistory brick buildings collapse: 37

```python
# to calculate the population affected in each damage zones
[p1, p2, p3, p4] = locator.get_population_of_postcode(z, sector=sector)
```

    estimaed population in damage zone 1 by postcode sector data: 53740669
    estimaed population in damage zone 2 by postcode sector data: 16242468
    estimaed population in damage zone 3 by postcode sector data: 765397
    estimaed population in damage zone 4 by postcode sector data: 293389

```python
p = [int(sum(p1)), int(sum(p2)), int(sum(p3)), int(sum(p4))]
for i in range(len(p)):
    if sector:
        print('estimaed population in damage zone ' + str(i+1) + ' by postcode sector data: ' + str(p[i]))
    else:
        print('estimaed population in damage zone ' + str(i+1) + ' by postcode data: ' + str(p[i]))
```

    estimaed population in damage zone 1 by postcode sector data: 53740669
    estimaed population in damage zone 2 by postcode sector data: 16242468
    estimaed population in damage zone 3 by postcode sector data: 765397
    estimaed population in damage zone 4 by postcode sector data: 293389

## Extension 2: Uncertainty Analysis

Utilising the input parametres used above for Extension 1, plus the standard deviations of the variables provided in the DamageMapper.ipynb notebook, an example of the impact risk analysis is conducted below.

```python
fiducial_stdevs = {'radius': 1, 'angle': 1, 'strength': 5e5,
               'density': 500, 'velocity': 1e3,
               'lat': 0.025, 'lon': 0.025, 'bearing': 0.5}

# call by keyword
result = armageddon.impact_risk(planet, means=params, stdevs=fiducial_stdevs,
            pressure=27.e3, nsamples=100, sector=True)
# showing the top 5 risky postcode sectors in this example
result.head()
```

      risk

      sector

      ST163
      14172.30

      ST179
      12609.36

      WS124
      12151.62

      WS152
      11820.80

      ST158
      11452.16

# Breaking News Example

Asteroid fact file:

Size: approximately 20-m radius
Composition: iron (typical density, 8000 kg/m^3; typical strength, 10 MPa)
Speed: 17 km/s

The asteroid is expected to enter Earth’s atmosphere (100-km altitude) at coordinates (53.79, -3.55), over the Irish Sea, on a bearing of 112 degrees and at an angle of 45 degrees to the horizontal.

```python
params = {'radius': 20, 'angle': 45., 'strength': 10e6,
          'density': 8000, 'velocity': 17e3,
          'lat': 53.79, 'lon': -3.55, 'bearing': 112.}
stdevs = {'radius': 1, 'angle': 1, 'strength': 5e5,
          'density': 500, 'velocity': 1e3,
          'lat': 0.025, 'lon': 0.025, 'bearing': 0.5}
```

## Study of the mean case

```python
result = planet.solve_atmospheric_entry(params['radius'], params['velocity'],
                                        params['density'], params['strength'], params['angle'])
outcome = planet.analyse_outcome(result=planet.calculate_energy(result))
```

```python
zb = outcome['burst_altitude']
Ek = outcome['burst_energy']
# using the pressure function defined in the DamageMapper.ipynb
P = lambda r: 3.14 * 10.**11 * ((r**2 + zb**2)/ Ek**(2./3))**(-1.3) + \
                1.8 * 10.**7 * ((r**2 + zb**2)/ Ek**(2./3))**(-0.565)

entry_point = [params['lat'], params['lon']]
pressures = [1e3, 3.5e3, 27e3, 43e3]
blat, blon, damrad = armageddon.damage_zones(outcome, entry_point[0], entry_point[1], params['bearing'], pressures)
burst_point = [blat, blon]

x = np.linspace(0, 4e5, 1000)

# verification plots, plotting the damrad on the function P to check the results visually
plt.plot(x, P(x), label="Pressure Function")
i = 1
for (loc, pres) in zip(damrad, pressures):
    plt.plot(loc, pres, 'o', label='damage level: ' + str(i) + ' boundary')
    plt.hlines(pres, 0, loc, linestyles='dashed')
    i += 1
plt.title("Damage radius of the sample burst")
plt.legend()
```

```python
# to get the impacted postcodes in each damage zone:
sector_op = True
# %timeit locator.get_postcodes_by_radius(burst_point, damrad, sector=sector_op)
z = locator.get_postcodes_by_radius(burst_point, damrad, sector=sector_op)
```

```python
damage_zones = [
    '1: ~10% glass windows shatter',
    '2: ~90% glass windows shatter',
    '3: Wood frame buildings collapse',
    '4: Multistory brick buildings collapse'
]

for i in range(len(z)):
    if sector_op:
        print('Number of postcode sectors in damage zone ' + damage_zones[i] + ': ' + str(len(z[i])))
    else:
        print('Number of postcodes in damage zone ' + damage_zones[i] + ': ' + str(len(z[i])))
```

    Number of postcode sectors in damage zone 1: ~10% glass windows shatter: 3022
    Number of postcode sectors in damage zone 2: ~90% glass windows shatter: 851
    Number of postcode sectors in damage zone 3: Wood frame buildings collapse: 145

```python
# to calculate the population affected in each damage zones
[p1, p2, p3] = locator.get_population_of_postcode(z, sector=sector_op)
```

```python
p = [int(sum(p1)), int(sum(p2)), int(sum(p3))]
for i in range(len(p)):
    if sector_op:
        print('estimaed population in damage zone ' + str(i+1) + ' by postcode sector data: ' + str(p[i]))
    else:
        print('estimaed population in damage zone ' + str(i+1) + ' by postcode data: ' + str(p[i]))
```

    estimaed population in damage zone 1 by postcode sector data: 16482024
    estimaed population in damage zone 2 by postcode sector data: 4916634
    estimaed population in damage zone 3 by postcode sector data: 713643

```python
armageddon.plot_damage_zones(entry_point, burst_point, damrad, p)
```

Make this Notebook Trusted to load map: File -> Trust Notebook

## Full Impact Risk Analysis

```python
# call by keyword
result = armageddon.impact_risk(planet, means=params, stdevs=fiducial_stdevs,
            pressure=27.e3, nsamples=100, sector=True)
# showing the top 5 risky postcode sectors in this example
result.head()
```