Arbitrum DAO STIP Results

idm
2023-10-29

Introduction

This report is a retrospective look at the Arbitrum DAO Short Term Incentive Program (STIP). To quickly recap the STIP timeline, a DAO vote was held to authorize the program and determine its budget. As a result of this vote, STIP was launched with a 50m ARB budget. Next, the DAO voted on individual STIP applications to determine whether they met the criteria for funding. Upon the conclusion of voting, it was discovered that the DAO had approved more applications than it had budgeted for.

With this budgetary discrepancy in mind, we now turn to the voting data. We found that most DAO voters’ first choice was to authorize the smallest budget: 25m ARB. However, among the most influential voters and delegates, the first choice was more bullish: 50m ARB. We found that the most-influential voters - especially those who prevailed in the initial approval vote - tended to approve more ARB than they originally budgeted for.

Two methods were used to evaluate the budget: an initial ranked-choice vote, which occurred before proposals were submitted, and a fine-grained method that considered each proposal. Ultimately, the second method approved applications totaling an allocation slightly over 75m ARB - which, coincidentally, was one of the original ranked-choice options. This discrepancy between the two budget evaluation methods (predicted versus actual) suggests the initial vote under-estimated the DAO’s actual intentions and merit assessment, once applications were known.

STIP authorization vote

From September 10-17, the Arbitrum DAO voted on a proposal called Arbitrum Short-term incentive Program. This proposal was described as:

This proposal outlines a one-time, community-created program to distribute up to 75,000,000 ARB of DAO-funded incentives targeting active Arbitrum protocols. Aimed at fulfilling short-term community needs, the program plans to allocate DAO-owned ARB towards incentives while ensuring transparent consensus and distribution methods. It spans two voting rounds and provides incentives to eligible programs designed to fund incentives through the January 31, 2024. It asks all participating grantees to self report data, and dashboards, summarizing grant performance.

Procedurally, the DAO hosted a ranked choice vote on snapshot.org with the following 4 choices:

  1. launch STIP with 25m ARB budget
  2. launch STIP with 50m ARB budget
  3. launch STIP with 75m ARB budget
  4. reject proposal; do not launch STIP

A vote was cast by arranging the 4 choices into a ranking of preferences. When voting concluded, the result was calculated using the ranked choice voting algorithm, as described on snapshot.

For this vote, the result was determined by the following:

  1. first-choices were examined and no single option achieved greater than 50% support
  2. the choice with fewest first-choice votes (probably reject) was eliminated; voters who chose reject had their second-choice votes counted
  3. the remaining choice with fewest first-choice votes (probably 75m) was eliminated; voters who chose 75m had their second-choice votes counted

After step 3, both Reject and 75m had been eliminated. 50m had achieved a simple majority.

Individual choice ranking results

We can examine the un-weighted numbers of voters who endorsed a given budget, as well as their rankings. These un-weighted counts disregard voting power, which considers the quantity of ARB held by a voter; we will examine that a little later. From the distribution of unweighted ranked choices, several patterns emerge.

According to rankings, most voters preferred a 25m budget, then a 50m budget, then a 75m budget. The least-preferred choice was to be against the STIP. It seems the DAO wished to fund the program - and the general preference was towards the smallest budget. Even so, the DAO still would have preferred a 75m budget over rejecting the STIP entirely.

Weighted outcomes

Of course, the vote was not determined as the simple count of votes towards a choice. For the Arbitrum DAO, as for many DAOs, the quantity of governance token - ARB, in this case - is proportional to voting power. Voters may also allocate their ARB to a voting delegate who may vote on their behalf.

In fact, when we examine the same ranked choice but include voting power (vote weight), the outcome looks different. More influential voters/delegates favored 50m as their first choice, which is a more aggressive allocation than the general public indicated. Consistent with the unweighted count, the reject option is dropped first during ranked scoring. 25m was the preferred second- and third-choice, which is also consistent with the preponderance of unweighted votes gravitating towards the slimmest budget. Examining the least-preferred choices, we see that reject is just unpopular, no matter whether counting weighted or unweighted votes.

Interpreting unweighted vs weighted votes

For the most part, the outcome is largely unchanged whether we consider ARB or not: 50m is well supported under both frameworks. The discrepancy among first-choice options could be taken as an indication that DAO delegates were more bullish on the STIP program than the general public. The weighted preference for 50m could also reflect a better prediction of the budget requirements for funding successful STIP applications. The “wisdom of the crowd” produced a worse estimate of the final STIP approvals than the weighted votes did.

Funding: whether a successful STIP was actually granted

Having determined the budget, the DAO opened the STIP to applications. In order for an application to be considered fundable, it would have to achieve quorum (in excess of 70m ARB) with greater than 50% voting in favor. In case the full 50m budget was not allocated - presumably, due to a lack of proposals with merit - then the remainder would be reserved for a future round of proposals.

To implement voting, each STIP application was allocated its own snapshot.org proposal. The proposals all had the same structure, providing simple yes/no/abstain voting choices; this vote was not ranked choice. Voting remained open for one week.

Upon the conclusion of individual STIP voting, the number of proposals that were deemed fundable had exceeded the 50m ARB budget. To determine which proposals would be funded, they were ranked according to the absolute votes in favor. Due to budget limitations, a number of fundable proposals were not accepted.

From the plot, a red vertical line separates the proposals that were funded; any proposal with a lower count of yes-votes could not be funded due to budgetary constraints.

The number of unfunded proposals accounts for about half of all approved proposals.

Predicted Budget vs Actual Budget

Now we will dig into the relationship between the budgetary vote and the eventual individual budgeting choices by those same delegates.

Votes on the STIP budget underestimated their own STIP approvals

We examined all voters with a “voting power” above 100k (the most influential voters who participated). For this plot, we considered their first-choice budget in the original STIP authorization. We already know 50m received the greatest support by delegates - but among those delegates, we can see that the median allocation was greater than 60m. Delegates who supported either 25m or 75m approved a lower median allocation across individual STIP applications. Even so, the delegates who prevailed in the initial approval vote tended to be the same who approved far greater than their own budgetary allocation.

Funded Grants tended to be Bigger Asks

We examined all grants to achieve quorum. Those grants that were within the funded cohort tended to be larger than those that were not funded. This is a curious finding; at first glance, the smaller the ask, the less impact upon the overall STIP budget. On the other hand, larger protocols tend to have larger communities and a greater ability to demonstrate both the need and the ability to manage larger amounts. Even so, this suggests there was a tendency towards funding fewer popular protocols with larger amounts - to the exclusion of smaller protocols with smaller asks. This may be at odds with the original objective of granting ARB allocations to a wide diversity of protocols.

Larger asks tended to be met by greater approval

The previous relationship - that funded applications asked for more - also works in the opposite direction. We find that those applications requesting larger allocations tended to gather more votes in favor. As we previously noted, there seems to be a congruence between capacity and quantity: bigger protocols can handle more. However, this plot highlights another detail: that a larger grant conveys greater value to the protocol that receives it, thereby increasing the overall expected value of the grant. On the basis that some grants were more “valuable” and “rewarding” than others, this plot could support the rationale that there may have been greater incentive to ensure that those grants were funded.

Industry Categories

We examined how funds were distributed across protocol categories.

In general, the number of funded protocols in each category is roughly half of the total approved.

When examining the total ARB allocated to any category, it is not evenly distributed.

Beacon Grants

Beacon Grants had lowest rate of funding

Of successful grants (i.e. those that were voted on to receive funding), acceptance rates were lowest for the smallest funding tier (“Beacon Grants”). Just 30.8% of successful Beacon grants were funded, compared to 60% of Siren grants, 57.1% of Lighthouse grants, and 71.4% of Pinnacle grants.

Number of successful STIPs funded at each grant tier

This plot breaks down the successful grants according to their grant tier (Beacon, Siren, Lighthouse, and Pinnacle) as well as their funding status. Beacon grants, for amounts up to 200,000 ARB, were the smallest tier. When we focus on the Beacon grants, we see that 4 were funded while 9 were below the budget cutoff. For all other grant tiers, the number of grants that received funding was greater than the number that were below the budget cutoff.

Amounts funded at each grant tier

For most grant tiers, a greater proportion of successful STIP applications was funded than not. However, specifically for Beacon grants, this trend was reversed.

Voting over Time

What were the time dynamics of STIP voting activity? We examined the votes cast over time to see whether activity was constant throughout STIP application review or if there were bursts. For this analysis, we examined both the unweighted count of votes as well as the ARB-weighted votes.

Voting per day

Examining the raw count of votes per day, we can see that the greatest activity occurred on the final day of the program.

When we examine the ARB-weighted votes per day, again we see the greatest activity occurred on the final day. As this plot considers voting influence, the concentration of activity is even greater.

Most votes were cast at the end

When we break down the time dynamics per hour, we can see even more clearly that most votes were cast at the very the end of the process.

Backfunding Strategies for the Long Tail

We examined a budget for backfunding.

The simplest approach that implies no additional criteria is to backfund all successful proposals according to their original ask amount.

Reviewing the STIP’s original motivation

We conclude by revisiting the language of the original STIP authorization, which put forth a statement of motivation:

This provisional framework aims to distribute incentives fairly and effectively using transparent criteria agreed upon by the community, while a thorough system is developed (1 19, 2 7, 3 4, 4 5). The goals of the program are to:

With the current findings in mind, we have identified several ways the voting choices are at odds with themselves; the prediction does not match the final votes. Because it was originally predicted the first round of voting would not allocate the entire 50m ARB budget, we can surmise the actual voting outcome was surprising. Since the outcome was unexpected, it is worth revisiting whether the original STIP motivation was satisfied.

Indeed, this was a valid investigation into new models for grants and developer support. However, the large proportion of fundable-but-unfunded proposals reduces grant distribution and, because fewer protocols were funded, may not maximize activity on the Arbitrum network. There may be follow-on effects due to the lack of funds already allocated for a subsequent STIP round. Doubtless, the results of this grant program will be examined by DAOs and protocols, alike. DAOs will therefore be tasked with the responsibility of predicting their grant budgets so as to not waste resources approving grants in vain.