@beaconlabs-io/evidence 1.1.0

package/evidence/02.mdx

@@ -0,0 +1,72 @@
+---
+evidence_id: "02"
+results:
+  - intervention: "Whether grants were received or token incentives were provided"
+    outcome_variable: "Developer retention, user activity, and Total Value Locked (TVL) of the network"
+    outcome: "+-"
+strength: "3"
+methodologies:
+  - "Synthetic Control Method"
+version: "2.0.0"
+datasets:
+  - "https://github.com/opensource-observer/insights/tree/main/analysis/optimism/syncon"
+title: "Effect of a Grant Program on Developer Activity "
+date: "2024-11-18"
+tags:
+  - "oss"
+  - "public goods funding"
+citation:
+  - name: "Early experiments with synthetic controls and causal inference"
+    src: "https://docs.oso.xyz/blog/synthetic-controls/"
+    type: "link"
+author: "BeaconLabs"
+---
+
+## Key Points
+
+On average, there was an observed increase of approximately 150 to 200 monthly active developers.
+
+## Background
+
+- Open Source Observer (OSO) is exploring advanced metrics to better measure the impact of certain types of interventions on public goods ecosystems.
+- For example, it aims to compare the performance of projects or users who received token incentives against those who did not.
+- However, in real-world economies, it is impossible to randomly assign treatment and control groups like in controlled A/B tests.
+- Therefore, advanced statistical techniques must be employed to estimate the causal effect of a treatment on a target cohort, while controlling for other factors such as market conditions, competing incentives, and geopolitical events.
+
+## Analysis Method
+
+### Dataset
+
+- The synthetic control work is part of a broader initiative to build a flexible analytics engine capable of analyzing virtually all metrics over time.
+- OSO is currently rolling out a suite of timeseries metrics. These models allow metrics to be computed for any cohort over any time period, enabling "time travel" to evaluate past performance.
+- Most timeseries metrics are computed using a rolling window with daily buckets. For example, rather than measuring monthly active developers as a static monthly count, OSO uses 30-day and 90-day rolling windows to provide a more detailed view of cohort performance.
+- Sample SQL queries show the use of tables such as `timeseries_metrics_by_collection_v0`, `metrics_v0`, and `collections_v1`.
+
+### Intervation / Explanatory Variable
+
+- OSO is interested in measuring the impact of specific types of interventions on public goods ecosystems.
+- Specifically, it seeks to assess how grants and incentives affect outcomes such as developer retention, user activity, and network TVL (Total Value Locked).
+- The initial experiment evaluates an intervention targeting a cohort of projects that received Optimism Retro Funding in January 2024.
+- The explanatory variable is `new_contributors_over_90_day` (new contributors over 90 days), `commits_over_90_day` (commits over 90 days), and `issues_opened_over_90_day` (issues opened over 90 days).
+
+### Dependent Variable
+
+- The dependent variable is `active_developers_over_90_day` (90-day active developers)
+
+### Identification Strategy
+
+- As an early experiment in crypto network economics, OSO is exploring methods such as synthetic controls and causal inference.
+- Synthetic control methods are widely used to assess the impact of interventions in complex systems.
+- In this approach, a synthetic control is a weighted average of several units, constructed to replicate the trajectory that the treated unit would have followed in the absence of the intervention.
+- Weights are selected in a data-driven way so that the resulting synthetic control closely resembles the treated unit with respect to key predictors of the outcome variable.
+- Unlike difference-in-differences approaches, this method allows for adjustments for time-varying confounders by weighting the control group to better match the treatment group in the pre-intervention period.
+- Economists frequently use synthetic controls to evaluate policy impacts in non-experimental settings (e.g., Abadie and Gardeazabal’s study on the economic impact of the Basque separatist conflict).
+- One key advantage of synthetic controls is the ability to systematically select comparison groups. In OSO’s case, this means comparing grant recipients to similar non-recipient projects.
+- Inspired by work from Counterfactual Labs, OSO uses the pysyncon package to estimate treatment effects across the range of timeseries metrics available within OSO.
+- Each analysis request includes a pre-period start and end date, an optimization period start and end date, a dependent variable, treatment identifier, control identifiers, and predictor variables.
+
+## Results
+
+- In early findings, OSO analyzed monthly active developers over a 90-day rolling window for a cohort of projects that received Optimism Retro Funding in January 2024.
+- The results indicate that the gap between the treated group and the synthetic control group reflects the treatment effect, with an average increase of approximately 150 to 200 monthly active developers.
+- OSO is in the early stages of applying advanced metrics like synthetic control to measure the impact of incentives in crypto networks and plans to share further insights in the future.

package/evidence/03.mdx

@@ -0,0 +1,74 @@
+---
+evidence_id: "03"
+results:
+  - intervention: "SuperStacks: A New Approach to Rewards on the Superchain"
+    outcome_variable: "Establishing a sustainable DeFi growth loop"
+    outcome: "+"
+strength: "3"
+methodologies:
+  - "Pro-rata model, One-sided t-tests"
+version: "1.0.0"
+title: "SuperStacks for Sustainable DeFi Liquidity on the Superchain"
+date: "2025-08-15"
+citation:
+  - name: "SuperStacks Impact Analysis"
+    src: "https://gov.optimism.io/t/superstacks-impact-analysis/10225"
+    type: "link"
+author: "BeaconLabs"
+---
+
+## Key Points
+- The SuperStacks program achieved **$58.0M in net TVL inflows** during its implementation period, with **$53.7M retained 30 days after the program concluded**. This translates to **$23.2/OP and $21.5/OP** after costs, respectively.
+- Findings supported the theory that strong demand-side activity can lift post-incentive equilibrium levels, though a more rigorous analysis is still needed.
+- **Incentivized lending pools generally demonstrated higher liquidity retention**.
+- However, TVL retention in DEXs appeared **inflated by Uniswap's concurrent Gauntlet campaign**. Specifically, when the two co-incentivized DEX pools were excluded, retained TVL inflows dropped to $48.2M, revealing a significant divergence between the lending and DEX verticals.
+- Metrics for measuring demand-side traction included trading volume per TVL for DEX pools and utilization rate for lending pools. For DEX pools, a moderate correlation was observed between peak net TVL inflows and trading volume per TVL, hinting at possible synergistic effects.
+- A key limitation was that isolating the program's causal effect alone was extremely difficult due to numerous confounding variables, such as external co-incentives and broader market volatility.
+
+## Background
+
+SuperStacks was the **Optimism Foundation's first proactive DeFi incentive pilot program**. It was designed with the goal of **increasing liquidity for interoperable assets (e.g., USD₮0) across the Superchain**. At its core, the program aimed to help these assets overcome the cold start problem and **test new mechanisms for establishing sustainable DeFi growth loops**.
+
+The program's design was based on a theoretical framework suggesting that a two-pronged approach, focusing on both supply-side and demand-side activity, could initiate a sustainable flywheel effect for interoperable assets on the Superchain, ultimately catalyzing lasting liquidity growth.
+
+This theory is broken down into three phases:
+
+1.  **Supply-Side Growth**: Available incentives attract deposits into DEX pools, which creates deeper liquidity and improves the execution quality for trades. On the lending side, an increase in lending supply reduces the borrow rate.
+2.  **Demand-Side Growth**: Improved trading conditions draw in more trades that are routed through the incentivized DEX pools, which increases trading volumes and generates more fees for LPs. Similarly, more competitive borrow rates attract more borrowers, which raises utilization rates and generates a higher yield for lenders.
+3.  **Sustainable Traction**: Once incentives are switched off, a portion of the TVL (Total Value Locked) and net liquidity leaves the incentivized pools, but due to higher fee and yield generation for LPs and lenders, liquidity settles at higher baseline levels compared to the pre-incentives period.
+
+## Analysis Method
+
+### Dataset
+
+The datasets and methods used for the SuperStacks analysis were:
+
+- Period: 76-day incentive period (April 16 – June 30) and a 30-day retention evaluation period after program end (through July 30).
+- Data sources: 25 pools and vaults targeted by SuperStacks incentives.
+- Metrics:
+  - TVL (Total Value Locked): Net TVL inflows during the program and retained inflows after program end.
+  - Trading volume: Indicator of demand-side activity in DEX pools.
+  - Utilization: Indicator of demand-side activity in lending pools.
+  - Cost efficiency: Net TVL inflows per OP token ($/OP).
+
+### Intervation / Explanatory Variable
+
+SuperStacks Program:
+
+SuperStacks was the Optimism Foundation’s first attempt at a “proactive DeFi incentive,” designed as a pilot program to increase liquidity of interoperable assets across the Superchain. The program was built on a two-pronged approach targeting both supply-side and demand-side activities. Specifically, incentives were provided to encourage supply-side actions (deposits into DEX pools, increased lending supply), which in turn aimed to stimulate demand-side activities (higher trading volume, increased utilization).
+
+### Dependent Variable
+
+Establishing a sustainable DeFi growth loop: Testing a new mechanism for interoperable assets to overcome the cold-start problem and achieve sustained growth.
+
+### Identification Strategy
+
+- A pro-rata model was applied to disentangle the complexity of overlapping incentive programs, attributing impact based on each program’s share of total USD incentives.
+- To evaluate pool-level performance, incentivized pools (treatment group) were paired with comparable non-incentivized pools (control group) on the same chain and protocol. One-sided t-tests were conducted on changes in TVL and trading volume.
+- The analysis focused on DEX pools (9 pairs) that passed statistical filtering.
+
+## Results
+
+- Theoretical Support: The analysis confirmed the theory that strong demand-side activity raises the equilibrium level after incentives end. The program was designed to test new mechanisms by which interoperable assets on the Superchain could overcome the cold-start problem and establish a sustainable DeFi growth loop.
+- Mechanism Validation: The two-pronged approach targeting both supply-side and demand-side activities supported the theoretical framework that sustained liquidity growth of interoperable assets on the Superchain can be promoted. The hypothesis was that incentives would increase TVL (supply side), leading to higher trading volume and utilization (demand side), and ultimately these metrics would settle at higher equilibrium levels even after incentives stopped.
+- Challenges: However, more rigorous analysis of this theory is needed. While trading volume per TVL emerged as a potential predictor of net TVL inflows, its explanatory power was inconsistent, requiring further careful analysis.

package/evidence/04.mdx

@@ -0,0 +1,81 @@
+---
+evidence_id: "04"
+results:
+  - intervention: "SuperStacks: A New Approach to Rewards on the Superchain"
+    outcome_variable: "Growth and retention of TVL"
+    outcome: "+"
+strength: "3"
+methodologies:
+  - "Pro-rata model, One-sided t-tests"
+version: "1.0.0"
+title: "SuperStacks increase TVL"
+date: "2025-08-15"
+citation:
+  - name: "SuperStacks Impact Analysis"
+    src: "https://gov.optimism.io/t/superstacks-impact-analysis/10225"
+    type: "link"
+author: "BeaconLabs"
+---
+
+## Key Points
+
+- The SuperStacks program achieved **$58.0M in net TVL inflows** during its implementation period, with **$53.7M retained 30 days after the program concluded**. This translates to **$23.2/OP and $21.5/OP** after costs, respectively.
+- Findings supported the theory that strong demand-side activity can lift post-incentive equilibrium levels, though a more rigorous analysis is still needed.
+- **Incentivized lending pools generally demonstrated higher liquidity retention**.
+- However, TVL retention in DEXs appeared **inflated by Uniswap's concurrent Gauntlet campaign**. Specifically, when the two co-incentivized DEX pools were excluded, retained TVL inflows dropped to $48.2M, revealing a significant divergence between the lending and DEX verticals.
+- Metrics for measuring demand-side traction included trading volume per TVL for DEX pools and utilization rate for lending pools. For DEX pools, a moderate correlation was observed between peak net TVL inflows and trading volume per TVL, hinting at possible synergistic effects.
+- A key limitation was that isolating the program's causal effect alone was extremely difficult due to numerous confounding variables, such as external co-incentives and broader market volatility.
+
+## Background
+
+SuperStacks was the **Optimism Foundation's first proactive DeFi incentive pilot program**. It was designed with the goal of **increasing liquidity for interoperable assets (e.g., USD₮0) across the Superchain**. At its core, the program aimed to help these assets overcome the cold start problem and **test new mechanisms for establishing sustainable DeFi growth loops**.
+
+The program's design was based on a theoretical framework suggesting that a two-pronged approach, focusing on both supply-side and demand-side activity, could initiate a sustainable flywheel effect for interoperable assets on the Superchain, ultimately catalyzing lasting liquidity growth.
+
+This theory is broken down into three phases:
+
+1.  **Supply-Side Growth**: Available incentives attract deposits into DEX pools, which creates deeper liquidity and improves the execution quality for trades. On the lending side, an increase in lending supply reduces the borrow rate.
+2.  **Demand-Side Growth**: Improved trading conditions draw in more trades that are routed through the incentivized DEX pools, which increases trading volumes and generates more fees for LPs. Similarly, more competitive borrow rates attract more borrowers, which raises utilization rates and generates a higher yield for lenders.
+3.  **Sustainable Traction**: Once incentives are switched off, a portion of the TVL (Total Value Locked) and net liquidity leaves the incentivized pools, but due to higher fee and yield generation for LPs and lenders, liquidity settles at higher baseline levels compared to the pre-incentives period.
+
+## Analysis Method
+
+### Dataset
+
+The datasets and methods used for the SuperStacks analysis were:
+
+- Period: 76-day incentive period (April 16 – June 30) and a 30-day retention evaluation period after program end (through July 30).
+- Data sources: 25 pools and vaults targeted by SuperStacks incentives.
+- Metrics:
+  - TVL (Total Value Locked): Net TVL inflows during the program and retained inflows after program end.
+  - Trading volume: Indicator of demand-side activity in DEX pools.
+  - Utilization: Indicator of demand-side activity in lending pools.
+  - Cost efficiency: Net TVL inflows per OP token ($/OP).
+
+### Intervation / Explanatory Variable
+
+SuperStacks Program:
+
+SuperStacks was the Optimism Foundation’s first attempt at a “proactive DeFi incentive,” designed as a pilot program to increase liquidity of interoperable assets across the Superchain. The program was built on a two-pronged approach targeting both supply-side and demand-side activities. Specifically, incentives were provided to encourage supply-side actions (deposits into DEX pools, increased lending supply), which in turn aimed to stimulate demand-side activities (higher trading volume, increased utilization).
+
+### Dependent Variable
+
+Growth and retention of TVL (Total Value Locked): Measuring both the increase in TVL through incentives and the extent to which that TVL was maintained after incentives ended. The analysis centered on “retained TVL inflows.”
+
+### Identification Strategy
+
+- A pro-rata model was applied to disentangle the complexity of overlapping incentive programs, attributing impact based on each program’s share of total USD incentives.
+- To evaluate pool-level performance, incentivized pools (treatment group) were paired with comparable non-incentivized pools (control group) on the same chain and protocol. One-sided t-tests were conducted on changes in TVL and trading volume.
+- The analysis focused on DEX pools (9 pairs) that passed statistical filtering.
+
+## Results
+
+- Overall TVL Inflow and Retention:
+  - During the program, the SuperStacks initiative achieved $58M in net TVL inflows, with $53.7M retained 30 days after the program ended.
+  - Considering costs, this corresponds to $23.2 net TVL per 1 OP during the program, and $21.5 per 1 OP after the program.
+  - Across all incentivized pools and vaults, $87.7M of TVL was retained 30 days after incentives ended.
+  - Net TVL inflows peaked before the program ended, at $15.7M in DEXs and $70M in lending markets.
+- DEX vs. Lending Markets:
+  - Of the net TVL inflows during the program, DEXs accounted for 31.6% and lending markets 68.4%.
+  - Lending markets showed stronger liquidity retention compared to DEXs. Lending pools, especially deposit-only vaults such as Morpho’s, often showed growth.
+  - DEX TVL retention appeared inflated by concurrent incentive programs run by the Uniswap Foundation and Gauntlet. Excluding the two DEX pools covered by these joint incentives, retained TVL inflows fell to $48.2M, revealing a notable divergence between lending and DEX outcomes.

package/evidence/05.mdx

@@ -0,0 +1,83 @@
+---
+evidence_id: "05"
+results:
+  - intervention: "SuperStacks: A New Approach to Rewards on the Superchain"
+    outcome_variable: "Trading volume and utilization"
+    outcome: "+"
+strength: "4"
+methodologies:
+  - "Pro-rata model, One-sided t-tests"
+version: "1.0.0"
+datasets:
+  - ""
+title: "Trading Volume and Utilization Outcomes of SuperStacks"
+date: "2025-08-15"
+
+citation:
+  - name: "SuperStacks Impact Analysis"
+    src: "https://gov.optimism.io/t/superstacks-impact-analysis/10225"
+    type: "link"
+author: "BeaconLabs"
+---
+
+## Key Points
+
+- The SuperStacks program achieved **$58.0M in net TVL inflows** during its implementation period, with **$53.7M retained 30 days after the program concluded**. This translates to **$23.2/OP and $21.5/OP** after costs, respectively.
+- Findings supported the theory that strong demand-side activity can lift post-incentive equilibrium levels, though a more rigorous analysis is still needed.
+- **Incentivized lending pools generally demonstrated higher liquidity retention**.
+- However, TVL retention in DEXs appeared **inflated by Uniswap's concurrent Gauntlet campaign**. Specifically, when the two co-incentivized DEX pools were excluded, retained TVL inflows dropped to $48.2M, revealing a significant divergence between the lending and DEX verticals.
+- Metrics for measuring demand-side traction included trading volume per TVL for DEX pools and utilization rate for lending pools. For DEX pools, a moderate correlation was observed between peak net TVL inflows and trading volume per TVL, hinting at possible synergistic effects.
+- A key limitation was that isolating the program's causal effect alone was extremely difficult due to numerous confounding variables, such as external co-incentives and broader market volatility.
+
+## Background
+
+SuperStacks was the **Optimism Foundation's first proactive DeFi incentive pilot program**. It was designed with the goal of **increasing liquidity for interoperable assets (e.g., USD₮0) across the Superchain**. At its core, the program aimed to help these assets overcome the cold start problem and **test new mechanisms for establishing sustainable DeFi growth loops**.
+
+The program's design was based on a theoretical framework suggesting that a two-pronged approach, focusing on both supply-side and demand-side activity, could initiate a sustainable flywheel effect for interoperable assets on the Superchain, ultimately catalyzing lasting liquidity growth.
+
+This theory is broken down into three phases:
+
+1.  **Supply-Side Growth**: Available incentives attract deposits into DEX pools, which creates deeper liquidity and improves the execution quality for trades. On the lending side, an increase in lending supply reduces the borrow rate.
+2.  **Demand-Side Growth**: Improved trading conditions draw in more trades that are routed through the incentivized DEX pools, which increases trading volumes and generates more fees for LPs. Similarly, more competitive borrow rates attract more borrowers, which raises utilization rates and generates a higher yield for lenders.
+3.  **Sustainable Traction**: Once incentives are switched off, a portion of the TVL (Total Value Locked) and net liquidity leaves the incentivized pools, but due to higher fee and yield generation for LPs and lenders, liquidity settles at higher baseline levels compared to the pre-incentives period.
+
+## Analysis Method
+
+### Dataset
+
+The datasets and methods used for the SuperStacks analysis were:
+
+- Period: 76-day incentive period (April 16 – June 30) and a 30-day retention evaluation period after program end (through July 30).
+- Data sources: 25 pools and vaults targeted by SuperStacks incentives.
+- Metrics:
+  - TVL (Total Value Locked): Net TVL inflows during the program and retained inflows after program end.
+  - Trading volume: Indicator of demand-side activity in DEX pools.
+  - Utilization: Indicator of demand-side activity in lending pools.
+  - Cost efficiency: Net TVL inflows per OP token ($/OP).
+
+### Intervation / Explanatory Variable
+
+SuperStacks Program:
+
+SuperStacks was the Optimism Foundation’s first attempt at a “proactive DeFi incentive,” designed as a pilot program to increase liquidity of interoperable assets across the Superchain. The program was built on a two-pronged approach targeting both supply-side and demand-side activities. Specifically, incentives were provided to encourage supply-side actions (deposits into DEX pools, increased lending supply), which in turn aimed to stimulate demand-side activities (higher trading volume, increased utilization).
+
+### Dependent Variable
+
+Trading volume and utilization: Determining whether TVL growth led to higher demand-side trading volume and lending utilization, and whether these metrics remained at elevated equilibrium levels after incentives stopped.
+
+### Identification Strategy
+
+- A pro-rata model was applied to disentangle the complexity of overlapping incentive programs, attributing impact based on each program’s share of total USD incentives.
+- To evaluate pool-level performance, incentivized pools (treatment group) were paired with comparable non-incentivized pools (control group) on the same chain and protocol. One-sided t-tests were conducted on changes in TVL and trading volume.
+- The analysis focused on DEX pools (9 pairs) that passed statistical filtering.
+
+## Results
+
+- DEX Trading Volume:
+  - In DEX pools, trading volume per TVL was used as a measure of liquidity “productivity.”
+  - CL100-USD₮0/kBTC on OP Mainnet: Alongside rapid TVL growth, trading volume increased by 126.7% during the program, and remained 42.7% above baseline after incentives ended (p < 0.01).
+  - CL1-USD₮0/USDC on OP Mainnet: TVL rose, while trading volume saw a modest 2.5% increase during the program. However, after incentives ended, volume accelerated and stabilized 25.1% above baseline (p < 0.01).
+  - BV-WETH/weETH on OP Mainnet: Beyond TVL growth, trading volume rose even more sharply, increasing 123.8% during the program and remaining 49.4% above baseline afterward (p < 0.01).
+- Lending Market Utilization:
+  - In lending pools, utilization rate was used as the demand-side traction metric.
+  - The lending pool with the highest performance at peak net TVL inflow showed a healthy 60.2% utilization rate, but the relationship between supply and demand momentum was less clear than in DEXs. Theoretically, competitive borrowing rates were expected to attract more borrowers, boost utilization, and deliver higher yields to lenders.

package/evidence/06.mdx

@@ -0,0 +1,61 @@
+---
+evidence_id: "06"
+results:
+  - intervention: "The platform owner enters the market by introducing its proprietary product, which is based on the complementary firm’s OSS."
+    outcome_variable: "OSS contributions by existing contributors(the number of commits, lines of code changed/added, and files of code changed/added)"
+    outcome: "-"
+strength: "3"
+methodologies:
+  - "DID, PSE"
+version: "1.0.0"
+datasets:
+  - ""
+title: "How Platform Owner Entry Affects OSS Contributions by Existing Contributors: An Experiment with AWS Elasticsearch"
+date: "2024-06-20"
+
+citation:
+  - name: "How Platform Owner Entry Affects Open Source Contribution? Evidence from GitHub Developers"
+    src: "https://questromworld.bu.edu/platformstrategy/wp-content/uploads/sites/49/2024/06/PlatStrat2024_paper_100.pdf"
+    type: "link"
+author: "BeaconLabs"
+---
+
+## Key Points
+
+This study explores how a platform owner’s market entry—by leveraging a complementary firm’s open-source technology—affects the external knowledge sourcing of the complementary firm, focusing on the willingness of GitHub developers to contribute. Using the staggered rollout of Amazon AWS’s Elasticsearch as a natural experiment, the analysis shows that the platform owner’s entry reduces contributions from existing contributors but substantially increases contributions from new contributors, leading to an overall increase in contributions to the open-source technology. This finding provides a new perspective: contrary to common concerns that platform owners’ entry harms open-source startups, from a technology development standpoint, such entry may not necessarily be detrimental.
+
+## Background
+
+Existing contributors are individuals who had begun contributing to the complementary firm’s open-source project before the platform owner’s entry. Based on prospect theory, they are likely to perceive strong potential losses to their existing “endowments,” such as their commitment to open-source philosophy, community interactions, and established reputation, as a result of the platform owner’s entry. Thus, it was hypothesized that they might reduce their involvement due to concerns about their contributions being exploited by the platform owner and about threats to community sustainability.
+
+## Analysis Method
+
+### Dataset
+
+Data was collected from AWS and GitHub. Specifically, AWS official announcements provided information about the date and location of the Amazon Elasticsearch Service launch, while GitHub data provided developers’ contributions to Elasticsearch (contribution volume, personal GitHub experience, popularity, etc.). Commits by Elastic employees were excluded to focus on external developers’ contributions.
+
+### Intervation / Explanatory Variable
+
+- AWS’s market entry with Amazon Elasticsearch Service.
+  - Entry is a dummy variable equal to 1 if AWS entered the country where the contributor resides, and 0 otherwise.
+  - After is a dummy variable equal to 1 if AWS had already entered the contributor’s country during or before a given period, and 0 otherwise.
+  - The interaction term Entry × After was used to measure the treatment effect.
+
+### Dependent Variable
+
+- OSS contributions by existing contributors(the number of commits, lines of code changed/added, and files of code changed/added)
+  - Contributions by existing contributors were measured as the natural logarithm of the number of commits to Elasticsearch.
+  - As robustness checks, the natural logarithm of lines of code changed/added and files changed/added were also used.
+
+### Identification Strategy
+
+- AWS’s staggered introduction of managed Elasticsearch services across countries/regions was treated as a natural experiment, and stacked difference-in-differences analysis was applied.
+- This method addresses the “forbidden comparison problem” in staggered interventions and strengthens analytical robustness.
+- The treatment group consisted of contributors in countries where AWS launched Elasticsearch services; the control group consisted of contributors in countries where AWS had not yet launched or never launched the service.
+- Propensity score matching (PSM) was used to align contributors across groups. The analysis was conducted at the contributor-week level, with 110,249 observations across 459 contributors.
+
+## Results
+
+- The analysis provides empirical evidence that the platform owner’s entry reduces contributions by existing contributors.
+- On average, the number of commits decreased by about 4.33%. Notably, contributors with stronger intrinsic motivations reduced their contributions more sharply after the platform owner’s entry, while contributors with stronger extrinsic motivations showed a significantly smaller decline.
+- These results support the prospect theory–based hypothesis that existing contributors decrease participation due to concerns over losses to their "endowments," such as community attachment and identity.

package/evidence/07.mdx

@@ -0,0 +1,61 @@
+---
+evidence_id: "07"
+results:
+  - intervention: "The platform owner enters the market by introducing its proprietary product, which is based on the complementary firm’s OSS."
+    outcome_variable: "OSS contributions by new contributors (both the number of new entrants and the average contribution per new entrant).(the number of commits, lines of code changed/added, and files of code changed/added)"
+    outcome: "+"
+strength: "3"
+methodologies:
+  - "DID, PSE"
+version: "1.0.0"
+datasets:
+  - ""
+title: "How Platform Owner Entry Affects OSS Contributions by New Contributors: An Experiment with AWS Elasticsearch"
+date: "2024-06-20"
+
+citation:
+  - name: "How Platform Owner Entry Affects Open Source Contribution? Evidence from GitHub Developers"
+    src: "https://questromworld.bu.edu/platformstrategy/wp-content/uploads/sites/49/2024/06/PlatStrat2024_paper_100.pdf"
+    type: "link"
+author: "BeaconLabs"
+---
+
+## Key Points
+
+This study explores how a platform owner’s market entry—by leveraging a complementary firm’s open-source technology—affects the external knowledge sourcing of the complementary firm, focusing on the willingness of GitHub developers to contribute. Using the staggered rollout of Amazon AWS’s Elasticsearch as a natural experiment, the analysis shows that the platform owner’s entry reduces contributions from existing contributors but substantially increases contributions from new contributors, leading to an overall increase in contributions to the open-source technology. This finding provides a new perspective: contrary to common concerns that platform owners’ entry harms open-source startups, from a technology development standpoint, such entry may not necessarily be detrimental.
+
+## Background
+
+New contributors are individuals who made their first contribution to the open-source project in a given period. Unlike existing contributors, they have no prior investments or ties to the project, and thus hold different reference points. From a prospect theory perspective, it was hypothesized that new contributors may perceive the platform owner’s entry as an opportunity for potential gains—such as enhanced professional reputation or career opportunities through increased visibility of the technology.
+
+## Analysis Method
+
+### Dataset
+
+Data was collected from AWS and GitHub. Specifically, AWS official announcements provided information about the date and location of the Amazon Elasticsearch Service launch, while GitHub data provided developers’ contributions to Elasticsearch (contribution volume, personal GitHub experience, popularity, etc.). Commits by Elastic employees were excluded to focus on external developers’ contributions.
+
+### Intervation / Explanatory Variable
+
+- AWS’s market entry with Amazon Elasticsearch Service.
+  - Entry is a dummy variable equal to 1 if AWS entered the country where the contributor resides, and 0 otherwise.
+  - After is a dummy variable equal to 1 if AWS had already entered the contributor’s country during or before a given period, and 0 otherwise.
+  - The interaction term Entry × After was used to measure the treatment effect.
+
+### Dependent Variable
+
+- OSS contributions by new contributors(the number of commits, lines of code changed/added, and files of code changed/added)
+  - Contributions by new contributors were measured as the natural logarithm of the total number of commits by all new contributors in each period.
+  - Additionally, the natural logarithm of the number of new contributors (extensive margin) and the average contribution per new contributor (intensive margin) were measured.
+  - As robustness checks, the natural logarithm of lines of code and files changed/added was also included.
+
+### Identification Strategy
+
+- AWS’s staggered market entry was treated as a natural experiment and stacked difference-in-differences analysis was applied.
+- The analysis was conducted at the country-week level with 5,635 observations across 16 treatment countries and 13 control countries.
+- Control countries were matched to treatment countries using PSM.
+
+## Results
+
+- The analysis strongly supports the hypothesis that the platform owner’s entry increases contributions by new contributors.
+- On average, the number of commits increased by about 7.27%.
+- This increase was driven by both a rise in the number of new contributors (awareness effect) and an increase in the average contribution per new contributor (reward effect). Moreover, after the platform owner's entry, new contributors tended to show stronger extrinsic motivations, suggesting a selection effect in the types of new contributors.

package/evidence/08.mdx

@@ -0,0 +1,57 @@
+---
+evidence_id: "08"
+results:
+  - intervention: "The platform owner enters the market by introducing its proprietary product, which is based on the complementary firm’s OSS."
+    outcome_variable: "Overall OSS contributions by combining both existing and new contributors(the number of commits, lines of code changed/added, and files of code changed/added)"
+    outcome: "+"
+strength: "3"
+methodologies:
+  - "DID"
+version: "1.0.0"
+datasets:
+  - ""
+title: "How Platform Owner Entry Affects OSS Contributions by Existing and New Contributors: An Experiment with AWS Elasticsearch"
+date: "2024-06-20"
+
+citation:
+  - name: "How Platform Owner Entry Affects Open Source Contribution? Evidence from GitHub Developers"
+    src: "https://questromworld.bu.edu/platformstrategy/wp-content/uploads/sites/49/2024/06/PlatStrat2024_paper_100.pdf"
+    type: "link"
+author: "BeaconLabs"
+---
+
+## Key Points
+
+This study explores how a platform owner’s market entry—by leveraging a complementary firm’s open-source technology—affects the external knowledge sourcing of the complementary firm, focusing on the willingness of GitHub developers to contribute. Using the staggered rollout of Amazon AWS’s Elasticsearch as a natural experiment, the analysis shows that the platform owner’s entry reduces contributions from existing contributors but substantially increases contributions from new contributors, leading to an overall increase in contributions to the open-source technology. This finding provides a new perspective: contrary to common concerns that platform owners’ entry harms open-source startups, from a technology development standpoint, such entry may not necessarily be detrimental.
+
+## Background
+
+Given the opposing effects of reduced contributions by existing contributors and increased contributions by new contributors, it was an important question to assess how the platform owner’s entry affects total contributions to the open-source project.
+
+## Analysis Method
+
+### Dataset
+
+Weekly contributions from all cloud developers were aggregated at the country level and analyzed at the country-week level.
+
+### Intervation / Explanatory Variable
+
+- AWS’s market entry with Amazon Elasticsearch Service.
+  - Entry is a dummy variable equal to 1 if AWS entered the country where the contributor resides, and 0 otherwise.
+  - After is a dummy variable equal to 1 if AWS had already entered the contributor’s country during or before a given period, and 0 otherwise.
+  - The interaction term Entry × After was used to measure the treatment effect.
+
+### Dependent Variable
+
+- Overall OSS contributions by combining both existing and new contributors(the number of commits, lines of code changed/added, and files of code changed/added)
+  - Overall contributions were measured as the natural logarithm of the total number of commits by all contributors.
+  - Robustness checks also included the natural logarithm of lines of code and files changed/added.
+
+### Identification Strategy
+
+- After aggregating data from both existing and new contributors, stacked difference-in-differences analysis was applied at the country-week level.
+
+## Results
+
+- The analysis shows that the platform owner’s entry increases overall contributions to the open-source software project by cloud developers.
+- This result suggests that, contrary to concerns that platform owners' entry harms open-source startups or communities, such entry may actually enhance the complementary firm's ability to attract external knowledge contributions.

package/evidence/09.mdx

@@ -0,0 +1,63 @@
+---
+evidence_id: "09"
+results:
+  - intervention: "Participation in Wikipedia offline meetings"
+    outcome_variable: "Increase in contributions to Wikipedia(overall change in editing activity)"
+    outcome: "+"
+strength: "3"
+methodologies:
+  - "DID, Covariate matching"
+version: "1.0.0"
+datasets:
+  - ""
+title: "How Wikipedia Offline Meetings Shape Participants’ Editing Activity: An Empirical Analysis of the German-Language Community"
+date: "2024-11-05"
+
+citation:
+  - name: "How offline meetings affect online activities: the case of Wikipedia"
+    src: "https://epjdatascience.springeropen.com/articles/10.1140/epjds/s13688-024-00506-w"
+    type: "link"
+author: "BeaconLabs"
+---
+
+## Key Points
+
+- Participation in offline meetings exerts a positive, statistically significant effect on users’ contribution behavior over the short term (1 week), medium term (1 month), and long term (1 year).
+- The magnitude of decline in editing activity among participants is significantly smaller than the decline observed among comparable non-participants.
+- Notably, users attending their first meeting were observed to increase their editing thereafter.
+
+## Background
+
+Open-source communities and peer-production projects face challenges in long-term sustainability, and offline gatherings are increasingly recognized for promoting community resilience. Although Wikipedia struggles with declining activity and retention of new users, the German-language Wikipedia hosts regular offline meetings. These meetings provide opportunities to form personal ties, and face-to-face interaction is thought to strengthen commitment to the project and reinforce identity.
+
+## Analysis Method
+
+### Dataset
+
+- We combine a comprehensive dataset on informal offline meetings in the German-language Wikipedia community from 2001 to 2020 with large-scale online activity data.
+- The dataset includes information on 4,408 small-scale meetings and 4,013 participating users.
+- All online actions on Wikipedia are recorded, and users’ editing activities are measured from metadata dumps.
+
+### Intervation / Explanatory Variable
+
+- The intervention for this outcome is participation in offline meetings.
+- The analysis examines whether a user attended a meeting, and in particular whether it was their first meeting.
+
+### Dependent Variable
+
+- The outcome variable is the volume of a user’s editing activity on Wikipedia (number of edits).
+- This is measured separately as the total number of edits across all namespaces and edits in the article main namespace.
+- Activity is analyzed over windows of 1 week (7 days), 1 month (28 days), and 1 year (364 days) before and after the meeting.
+
+### Identification Strategy
+
+- Quasi-experimental approach: We employ a difference-in-differences (DiD) design comparing meeting participants (treatment group) with comparable non-participants selected via matching (control group).
+- Covariate matching: From a pool of non-participants, we construct a control group most similar to participants based on five features (days since registration; cumulative activity in mainspace and outside mainspace from registration to the meeting; and recent activity in mainspace and outside mainspace over the 7-day, 1-month, 2-month, and 1-year periods prior to the meeting). This aims to minimize pre-existing differences between groups.
+- Statistical models: For the binary outcome of resuming activity, we use a multilevel linear probability model (LPM); for changes in activity volume, we use multilevel negative binomial models. Control variables (prior activity level, tenure, administrator status, and meeting year) are included.
+
+## Results
+
+- Compared to the control group, participants’ contributions increased significantly over the short, medium, and long terms.
+- Among users inactive before the meeting, the probability of resuming editing after the meeting increased substantially relative to the control group (e.g., the probability of resuming edits in mainspace rose from 16.7% in the control group to 33.4% among participants).
+- While the control group tended to reduce their editing, the decline among participants was significantly smaller, suggesting that offline interaction helps mitigate the broader decline of online communities.
+- Attending a first meeting showed a particularly strong positive effect, increasing editing activity more than attending other meetings.

package/evidence/10.mdx

@@ -0,0 +1,60 @@
+---
+evidence_id: "10"
+results:
+  - intervention: "Type of offline meeting (work-oriented vs. social)"
+    outcome_variable: "Increase the volume of users’ editing activity (number of edits) and the posibility of presence of editing"
+    outcome: "+-"
+strength: "3"
+methodologies:
+  - "DID, Covariate matching"
+version: "1.0.0"
+title: "Effects of Work-Oriented vs. Social Offline Meetings on User Editing Activity"
+date: "2024-11-05"
+
+citation:
+  - name: "How offline meetings affect online activities: the case of Wikipedia"
+    src: "https://epjdatascience.springeropen.com/articles/10.1140/epjds/s13688-024-00506-w"
+    type: "link"
+author: "BeaconLabs"
+---
+
+## Key Points
+
+- In the short term (1 week), work-related meetings showed a stronger positive effect than social meetings in terms of raising the likelihood of resuming editing.
+- However, this effect did not extend to the level (intensity) of editing, and in the medium term (1 month) and long term (1 year), no significant differences were observed between work-oriented and social meetings.
+- Overall, the value of social meetings for community building appears to match the effect of work meetings on long-term editor productivity.
+
+## Background
+
+Offline meetings include those primarily for social interaction and those that are work-oriented, directly focusing on improving Wikipedia. Theoretically, work-oriented meetings could yield larger increases in contributions by enhancing users’ capacity to contribute or heightening their awareness of needs.
+
+## Analysis Method
+
+### Dataset
+
+- We combine a comprehensive dataset on informal offline meetings in the German-language Wikipedia community from 2001 to 2020 with large-scale online activity data.
+- The dataset includes information on 4,408 small-scale meetings and 4,013 participating users.
+- All online actions on Wikipedia are recorded, and users’ editing activities are measured from metadata dumps.
+- Meetings are classified mainly into social (77%) and work-oriented (23%).
+
+### Intervation / Explanatory Variable
+
+- The intervention for this outcome is the type of offline meeting (work-oriented vs. social).
+- The models include indicators for meeting type and their interaction terms.
+
+### Dependent Variable
+
+- Outcome variables are the volume of users’ editing activity (number of edits) and the presence/absence of editing.
+- Analyses are conducted over short (1 week), medium (1 month), and long (1 year) horizons.
+
+### Identification Strategy
+
+- Quasi-experimental approach: We employ a difference-in-differences (DiD) design comparing meeting participants (treatment group) with comparable non-participants selected via matching (control group).
+- Covariate matching: From a pool of non-participants, we construct a control group most similar to participants based on five features (days since registration; cumulative activity in mainspace and outside mainspace from registration to the meeting; and recent activity in mainspace and outside mainspace over the 7-day, 1-month, 2-month, and 1-year periods prior to the meeting). This aims to minimize pre-existing differences between groups.
+- Statistical models: For the binary outcome of resuming activity, we use a multilevel linear probability model (LPM); for changes in activity volume, we use multilevel negative binomial models. Control variables (prior activity level, tenure, administrator status, and meeting year) are included.
+
+## Results
+
+- Short term (1 week): For the likelihood that previously inactive users resume editing, work-related meetings showed a stronger positive effect than social meetings. However, they did not affect the intensity of editing itself.
+- Medium term (1 month) and long term (1 year): No significant differences were observed between work-oriented and social meetings in either resuming editing or editing intensity.
+- These results provide only limited support for Hypothesis that work-related meetings increase contributions more than social meetings. While there is a short-term difference in re-engagement, there is no clear difference in long-term contribution growth.