Exploring the GCA Protocol Binary Bug

Simon Boccara / April 10, 2024

Overview

On December 18, 2023 Glow went into guarded launch mode on Ethereum Mainnet. During the launch, there was an off-chain bug was caught in the GCA laptop that resulted in only an Ethereum public key being saved on the secure laptop, rather than a public + private key pair. This meant that the first GCA did not have access to their private key to submit reports on-chain. This bug was quickly fixed, but the new GCA public key needed to be elected through Governance on the Glow Protocol. This process can take anywhere from 4-5 weeks, and as a result, the GCA was unable to submit reports for the data contained in weeks 0-8. This resulted in having to aggregate the data from weeks [0-8] into a single report.

The sections below contain the logistics, impact on carbon credit production and the rewards for farms, and the process that went into creating the first on-chain proposal.

Logistics

GCA That Submit The Report: 0xB2d687b199ee40e6113CD490455cC81eC325C496

Transaction Link: https://etherscan.io/tx/0x26e6164995bf746d522e911d4cb88c1672c9e31757355e67dbf9496d8b1b06d4

Function Inputs

bucketId	uint256	9	(Human Readable)
totalNewGCC	uint256	689843857893639930	.6898 Carbon Credits
totalGlwRewardsWeight	uint256	3067431198	$3067
totalGRCRewardsWeight	uint256	689844	.6898
root	bytes32	0x03367cff4ec5a2ba7da1a41d477056299e7e8ca9a37f825bad1626ada08b8513

Summary

Total New GCC: 689843857893639930 represents about .69 carbon credits in 18 decimal form

Total Glow Rewards Weight: 3067431198 represents about $3067 of protocol fees.

Total UDSG Rewards Weight: (totalGRCRewardsWeight): 689844 represents about .69 carbon credits in 6 decimal form.

Root: The merkle root of all reports

Impact

The impact of the bug can best be understood by breaking it down into two parts: Carbon Credit Production and Rewards for Farms.

Impact On Carbon Credits

The amount of carbon credits on-chain are not affected from this bug since the on-chain report aggregated the values for missed weeks into one report. The total credits submitted for the first report is equal to the sum of the carbon credits produced in weeks [0-8].

However, it should be noted, that the expectation for GCAs is that under normal circumstances, the weekly reports should be scoped to the week in question. This means that the carbon credits produced and protocol in week 0 should be contained in the week 0 report , and the carbon credits produced in week 1 should be contained in week 1 report. The same is true for the reporting of protocol fees paid by farms.

It's essential that the contributions of each farm are accurately described in the weekly reports, because the Glow Protocol rewards farms USDG and Glow on a weekly schedule based on the carbon credits they produce in relation to other farms and based on the protocol fees farms have paid in relation to other farms respectively (this process is covered in more detail in the next section.). By aggregating the data from weeks [0-8] into a single report, the GCA had to change the reward weights to correctly match the reported credits and protocol fees, which influenced the rewards for farms.

It was early enough in the protocol that the impact of this bug is minimal, especially as there were no USDG rewards at the time, but it is important to note that this is not the intended behavior for GCAs. The intended behavior is that the GCA should report the carbon credits produced and protocol fees paid in the week they were produced and paid, respectively. While the impact on the carbon credit reporting was minimal, the impact on the rewards for farms was more significant. The below section dives deeper into how this affected the rewards for farms.

Impact And Reconciliation On Farm Rewards

Before diving into how this affected the rewards for farms, let’s dive in a bit into the reward mechanisms that make up Glow.

Farms can earn two types of rewards: Glow and USDG. Farms earn Glow based on their protocol fee payment in relation to the total protocol fee payments. Put simply, if a farm has paid $1,000 in protocol fees and there are a total of $10,000 of protocol fees, that farm will earn 10% of the Glow rewards available in the bucket. Farms also earn USDG, but the distribution is based on their carbon credit production output rather than protocol fee payments. Similar to Glow distribution, farms earn the USDG as a function of their carbon credit production in relation to others. As a numerical example, if a farm produced 1 of the 100 carbon credits in a given week, then they can earn 1% of the USDG rewards. Another distinction between both reward types is that Glow rewards are fixed at 175,000 tokens per week, while the USDG available is variable and depends on multiple factors.

The impact of the bug on the rewards is that the GCA treated the aggregate of the weeks protocol fee payments and credit as a single week. Let’s take a simple example of two farms with the following timeline.

Week 0: Farm A pays $1,000 of protocol fees.

Week 0-4: Farm A produces 1 credit per week

Week 5. Farm B joins and pays $1,000 of protocol fees

Weeks 5-8: Farm A produces 1 credit per week and Farm B produces 1 credit per week

The end state is that Farm A has been $1,000 of protocol fees and produced 9 carbon credits,

and Farm B has paid $1,000 of protocol fees and produced 4 carbon credits.

A note on the Glow Rewards and Protocol fee vesting

Protocol fee accounting is a bit more involved than stated in the previous section. Feel free to skip this section if you’re not interested in the details. Farms DO earn the Glow rewards based on their protocol fee payments in relation to other farms, but it’s on a weekly basis, not on a gross basis. From the week a farm pays a protocol fee, their protocol fee gets distributed over 192 weeks. This means that farms will earn Glow from their protocol fee payment only for the following 192 weeks after they’ve paid.

In the example above, the correct weights for Farm A is actually

$1000/192 _ 9 , or , protocol fee payment / number_of_vesting_weeks _ weeks_to_distribute_the_fee_over. This results in a total of $46.88 vested over 9 weeks.

The correct weight for Farm B is:

$1000 / 192 * 4 = $20.833. The weeks_to_distribute_the_fee_over is only 4 since the farm paid their protocol fee on week 5, and the report aggregated data from weeks [0-8].

This means that Farm A got 46.88/(46.88+20.833) of the glow rewards available and Farm B got the remainder of (20.833)/(46.88+20.833).

For weeks [0-8], there were 0 USDG rewards available, but we still do calculate the percent of the rewards reserved for the farms. Farm A produced 9 credits over 9 weeks, and Farm B produced 4 credits over 4 weeks.

Farm A was entitled to 9/13 of the USDG rewards available, and Farm B the remaining 4/13 USDG rewards. There were no USDG rewards to earn, but should there have been, the glow protocol would have distributed them accordingly.

Impact The overall impact of the bug is that the GCA had to aggregate the data from weeks [0-8] into a single report. This meant that the GCA had to adjust the rewards for farms to correctly match the reported credits and protocol fees. The GCA did this by calculating the correct weights for each farm based on the aggregated data. The GCA then submitted the correct weights on-chain, and the Glow Protocol distributed the rewards accordingly. This lead to some farms earning less than they should have, and some farms earning more than they should have as a result of the bug.

Reconciliation Upon the Glow relaunch, farms will be made whole for the rewards they missed out on due to the bug. This means that farms will receive the rewards they would have received had the bug not occurred.

Other Reward Considerations

Since Glow has a fixed reward rate of 175,000 tokens per week, there are technically 4 weeks of rewards (Weeks 4-7) 700,000 Glow tokens that have not been distributed as rewards. Weeks (0-3) are not included as no farms were online during that time-frame, and would not have earned rewards. The rewards for these weeks will be distributed based on what they would have been had Glow had no launch bugs. This will be made correct in the Glow relaunch.

Aggregation Process

The rest of this document covers the processes that went in gathering the data and lives to give transparency into the process and numbers that went into forming the aggregated report.

This doc covers the process that was underwent to create the first on-chain proposal. The first on-chain proposal consolidated the stats from weeks [0-8]. Submitting bulk reports in one week means that farms earn less glow tokens than they are supposed to. To rectify this, glow will make full the farms that received unfair rewards when the unguarded production contracts are deployed.

How did we get to these numbers?

We aggregated the data from weeks [0-8] for the GCA Endpoint:

http://95.217.194.59:35015

Carbon Credit Production Routes Used

Week 0: http://95.217.194.59:35015/api/v1/all-device-stats?timeslot_offset=0

Week 1: http://95.217.194.59:35015/api/v1/all-device-stats?timeslot_offset=2016

Week 2: http://95.217.194.59:35015/api/v1/all-device-stats?timeslot_offset=4032

Week 3: http://95.217.194.59:35015/api/v1/all-device-stats?timeslot_offset=6048

Week 4: http://95.217.194.59:35015/api/v1/all-device-stats?timeslot_offset=8064

Week 5: http://95.217.194.59:35015/api/v1/all-device-stats?timeslot_offset=10080

Week 6: http://95.217.194.59:35015/api/v1/all-device-stats?timeslot_offset=12096

Week 7: http://95.217.194.59:35015/api/v1/all-device-stats?timeslot_offset=14112

Week 8: http://95.217.194.59:35015/api/v1/all-device-stats?timeslot_offset=16128

Equipment Data / Protocol Fee Route Used

http://95.217.194.59:35015/api/v1/equipment

Determining How Many Credits Were Made

Carbon credits for a single slot for a farm are determined by multiplying the power output at every 5 minute slot with it’s respective impact power. By summing up all the 2016 slot in which a farm reports and dividing that number by 1e15, we can determine how many carbon offsets the farm has made.
The actual reported carbon credits are multiplied by 0.65 (made smaller) to underestimate the amount of carbon credits actually produced.

https://docs.google.com/spreadsheets/d/1SrxffsCmrifOM91hTzC408wU48g59arREZxxujFfFfM/edit?usp=sharing

Determining The Protocol Fees That Were Paid

The GCA server also contains an endpoint to query the protocol fee for every farm as well as the fee’s payment date. From this, we can backtrack weight of the payment in the given week for the farm.

Matching Each Farm To A Reward Address

The solar farms thus far were all onboarded through Glow Solutions. Glow solutions provided the GCA with a google sheet containing all protocol fee payments and receiving addresses for the rewards. This was all double checked by the GCA to ensure the data matched the on-chain records.
For visibility, a forked version of the google sheet can be found here: https://docs.google.com/spreadsheets/d/1SrxffsCmrifOM91hTzC408wU48g59arREZxxujFfFfM/edit?usp=sharing
The GCA split up the aggregate protocol fees and carbon credit production between the farm and Glow Solutions to account for the glow solutions service fee they charge to every farm.

The final result was a merkle tree with the following leaves:

The tree can also be inspected here

It should be noted that usdcWeight and glowWeight are with 6 decimals.

[
  {
    "address": "0x2e2771032d119fe590FD65061Ad3B366C8e9B7b9",
    "glowWeight": "182654010",
    "usdcWeight": "50770"
  },
  {
    "address": "0x1f00e91A9e467FfE8038e520c498D371F63dFE56",
    "glowWeight": "300241125",
    "usdcWeight": "228346"
  },
  {
    "address": "0xCB0695C5e231D04a36feb07841e26D44e6D08c9d",
    "glowWeight": "1407219198",
    "usdcWeight": "4319"
  },
  {
    "address": "0x09EfEe2b1fC9105Ff080Ec2d379f21aff697455C",
    "glowWeight": "511513151",
    "usdcWeight": "156890"
  },
  {
    "address": "0x0e9c3c8c10900c899c5681f87114fe0b6fb2a198",
    "glowWeight": "511513151",
    "usdcWeight": "184856"
  },
  {
    "address": "0x5bC1A82995C73eED31183DAE1b7ce70E70eBF3Cf",
    "glowWeight": "154290563",
    "usdcWeight": "64663"
  }
]

Appendix / More Resources

The data derived when summing up the first 8 weeks
1. The device is the hexlified version of the PublicKey output on the GCA Servers

export const data = [
  [],
  [],
  [],
  [
    {
      device:
        "0xcc437c465421e81beda4413166569aef4613494f6f22f5d04fc221fc108c1ae3",
      powerOutput: 0,
      impactRate: 1306877635.857641,
      credits: 0,
      glowWeight: 55.600208333333335,
      shortId: "1",
    },
    {
      device:
        "0xcfe170dbe83d48d3c3ef03b34f0848cf2a7e5c587ab6a4041bfdb6cc84e0f622",
      powerOutput: 0,
      impactRate: 1350593507.701131,
      credits: 0,
      glowWeight: 95.62078125,
      shortId: "2",
    },
    {
      device:
        "0x73186f7582968d9c6a3123884f783ede6c82d7f74af07cfd307b23df1e6eecf6",
      powerOutput: 0,
      impactRate: 1350593507.701131,
      credits: 0,
      glowWeight: 95.62078125,
      shortId: "3",
    },
  ],
  [
    {
      device:
        "0xcc437c465421e81beda4413166569aef4613494f6f22f5d04fc221fc108c1ae3",
      powerOutput: 60109672,
      impactRate: 1330976544.8833704,
      credits: 0.03905347662394888,
      glowWeight: 55.600208333333335,
      shortId: "1",
    },
    {
      device:
        "0xcfe170dbe83d48d3c3ef03b34f0848cf2a7e5c587ab6a4041bfdb6cc84e0f622",
      powerOutput: 0,
      impactRate: 1348884825.2433434,
      credits: 0,
      glowWeight: 95.62078125,
      shortId: "2",
    },
    {
      device:
        "0x73186f7582968d9c6a3123884f783ede6c82d7f74af07cfd307b23df1e6eecf6",
      powerOutput: 0,
      impactRate: 1348884825.2433434,
      credits: 0,
      glowWeight: 95.62078125,
      shortId: "3",
    },
    {
      device:
        "0x5335235097f9646f90f01431fd4746bd8d11485f3efe88d417e7a84dbecf0fd9",
      powerOutput: 0,
      impactRate: 1322909858.17529,
      credits: 0,
      glowWeight: 51.492604166666666,
      shortId: "4",
    },
    {
      device:
        "0xe77caf2d3ec3e6447d9632f41d7155a9abbe0e188ac8e013d8e4b1f632691674",
      powerOutput: 0,
      impactRate: 1322909858.17529,
      credits: 0,
      glowWeight: 107.68697916666667,
      shortId: "5",
    },
    {
      device:
        "0xec40f7b230b24e8092c2e978c8d8b3578f4ecb052d5a197b1a92cbfe581364ad",
      powerOutput: 0,
      impactRate: 1322909858.17529,
      credits: 0,
      glowWeight: 107.68697916666667,
      shortId: "6",
    },
  ],
  [
    {
      device:
        "0x5335235097f9646f90f01431fd4746bd8d11485f3efe88d417e7a84dbecf0fd9",
      powerOutput: 0,
      impactRate: 1330581012.3367293,
      credits: 0,
      glowWeight: 51.492604166666666,
      shortId: "4",
    },
    {
      device:
        "0xe77caf2d3ec3e6447d9632f41d7155a9abbe0e188ac8e013d8e4b1f632691674",
      powerOutput: 13920696,
      impactRate: 1329921942.6231194,
      credits: 0.009812839657273531,
      glowWeight: 107.68697916666667,
      shortId: "5",
    },
    {
      device:
        "0xec40f7b230b24e8092c2e978c8d8b3578f4ecb052d5a197b1a92cbfe581364ad",
      powerOutput: 4380219,
      impactRate: 1329939632.7255492,
      credits: 0.002933023501127597,
      glowWeight: 107.68697916666667,
      shortId: "6",
    },
    {
      device:
        "0xcc437c465421e81beda4413166569aef4613494f6f22f5d04fc221fc108c1ae3",
      powerOutput: 200503358,
      impactRate: 1330581012.3367293,
      credits: 0.13264762183550693,
      glowWeight: 55.600208333333335,
      shortId: "1",
    },
    {
      device:
        "0xcfe170dbe83d48d3c3ef03b34f0848cf2a7e5c587ab6a4041bfdb6cc84e0f622",
      powerOutput: 10040502,
      impactRate: 1332708360.5520408,
      credits: 0.006962900288634049,
      glowWeight: 95.62078125,
      shortId: "2",
    },
    {
      device:
        "0x73186f7582968d9c6a3123884f783ede6c82d7f74af07cfd307b23df1e6eecf6",
      powerOutput: 0,
      impactRate: 1332708360.5520408,
      credits: 0,
      glowWeight: 95.62078125,
      shortId: "3",
    },
  ],
  [
    {
      device:
        "0xcfe170dbe83d48d3c3ef03b34f0848cf2a7e5c587ab6a4041bfdb6cc84e0f622",
      powerOutput: 0,
      impactRate: 1398930125.0178072,
      credits: 0,
      glowWeight: 95.62078125,
      shortId: "2",
    },
    {
      device:
        "0x73186f7582968d9c6a3123884f783ede6c82d7f74af07cfd307b23df1e6eecf6",
      powerOutput: 0,
      impactRate: 1398930125.0178072,
      credits: 0,
      glowWeight: 95.62078125,
      shortId: "3",
    },
    {
      device:
        "0x5335235097f9646f90f01431fd4746bd8d11485f3efe88d417e7a84dbecf0fd9",
      powerOutput: 0,
      impactRate: 1386099357.647629,
      credits: 0,
      glowWeight: 51.492604166666666,
      shortId: "4",
    },
    {
      device:
        "0xe77caf2d3ec3e6447d9632f41d7155a9abbe0e188ac8e013d8e4b1f632691674",
      powerOutput: 149960831,
      impactRate: 1386099357.647629,
      credits: 0.10375115167927806,
      glowWeight: 107.68697916666667,
      shortId: "5",
    },
    {
      device:
        "0xec40f7b230b24e8092c2e978c8d8b3578f4ecb052d5a197b1a92cbfe581364ad",
      powerOutput: 152467623,
      impactRate: 1386099357.647629,
      credits: 0.10607421111958493,
      glowWeight: 107.68697916666667,
      shortId: "6",
    },
    {
      device:
        "0xcc437c465421e81beda4413166569aef4613494f6f22f5d04fc221fc108c1ae3",
      powerOutput: 133433338,
      impactRate: 1386099357.647629,
      credits: 0.09169600513283077,
      glowWeight: 55.600208333333335,
      shortId: "1",
    },
  ],
  [
    {
      device:
        "0x73186f7582968d9c6a3123884f783ede6c82d7f74af07cfd307b23df1e6eecf6",
      powerOutput: 0,
      impactRate: 1385961011.9747725,
      credits: 0,
      glowWeight: 95.62078125,
      shortId: "3",
    },
    {
      device:
        "0x5335235097f9646f90f01431fd4746bd8d11485f3efe88d417e7a84dbecf0fd9",
      powerOutput: 0,
      impactRate: 1388966061.426022,
      credits: 0,
      glowWeight: 51.492604166666666,
      shortId: "4",
    },
    {
      device:
        "0xe77caf2d3ec3e6447d9632f41d7155a9abbe0e188ac8e013d8e4b1f632691674",
      powerOutput: 18227578,
      impactRate: 1388966061.426022,
      credits: 0.01231588145286776,
      glowWeight: 107.68697916666667,
      shortId: "5",
    },
    {
      device:
        "0xec40f7b230b24e8092c2e978c8d8b3578f4ecb052d5a197b1a92cbfe581364ad",
      powerOutput: 26461323,
      impactRate: 1388966061.426022,
      credits: 0.017941642515139362,
      glowWeight: 107.68697916666667,
      shortId: "6",
    },
    {
      device:
        "0x9a8126f231cd3a73441d3161ced733331e1b942dc553d09afd12948bca529f65",
      powerOutput: 0,
      impactRate: 1380223068.4942706,
      credits: 0,
      glowWeight: 85.71697916666666,
      shortId: "7",
    },
    {
      device:
        "0xcc437c465421e81beda4413166569aef4613494f6f22f5d04fc221fc108c1ae3",
      powerOutput: 31701585,
      impactRate: 1388966061.426022,
      credits: 0.021368761672913437,
      glowWeight: 55.600208333333335,
      shortId: "1",
    },
    {
      device:
        "0xcfe170dbe83d48d3c3ef03b34f0848cf2a7e5c587ab6a4041bfdb6cc84e0f622",
      powerOutput: 0,
      impactRate: 1385961011.9747725,
      credits: 0,
      glowWeight: 95.62078125,
      shortId: "2",
    },
  ],
  [
    {
      device:
        "0xa2935e1fb94d362893e01e887016a9791c67094c556662b8608f5e393273bf13",
      powerOutput: 0,
      impactRate: 917559304.6866393,
      credits: 0,
      glowWeight: 80.61380208333334,
      shortId: "8",
    },
    {
      device:
        "0xcfe170dbe83d48d3c3ef03b34f0848cf2a7e5c587ab6a4041bfdb6cc84e0f622",
      powerOutput: 0,
      impactRate: 1312508985.1311812,
      credits: 0,
      glowWeight: 95.62078125,
      shortId: "2",
    },
    {
      device:
        "0x5335235097f9646f90f01431fd4746bd8d11485f3efe88d417e7a84dbecf0fd9",
      powerOutput: 45628,
      impactRate: 1383137399.471523,
      credits: 0.00003132934603659245,
      glowWeight: 51.492604166666666,
      shortId: "4",
    },
    {
      device:
        "0xec40f7b230b24e8092c2e978c8d8b3578f4ecb052d5a197b1a92cbfe581364ad",
      powerOutput: 245865626,
      impactRate: 1383137399.471523,
      credits: 0.17241251503817387,
      glowWeight: 107.68697916666667,
      shortId: "6",
    },
    {
      device:
        "0x9a8126f231cd3a73441d3161ced733331e1b942dc553d09afd12948bca529f65",
      powerOutput: 158014567,
      impactRate: 1383137399.471523,
      credits: 0.11053447786616954,
      glowWeight: 85.71697916666666,
      shortId: "7",
    },
    {
      device:
        "0xcc437c465421e81beda4413166569aef4613494f6f22f5d04fc221fc108c1ae3",
      powerOutput: 150440855,
      impactRate: 1383137399.471523,
      credits: 0.10556928011145325,
      glowWeight: 55.600208333333335,
      shortId: "1",
    },
    {
      device:
        "0x73186f7582968d9c6a3123884f783ede6c82d7f74af07cfd307b23df1e6eecf6",
      powerOutput: 0,
      impactRate: 1312508985.1311812,
      credits: 0,
      glowWeight: 95.62078125,
      shortId: "3",
    },
    {
      device:
        "0xe77caf2d3ec3e6447d9632f41d7155a9abbe0e188ac8e013d8e4b1f632691674",
      powerOutput: 182709135,
      impactRate: 1383137399.471523,
      credits: 0.12819312507235353,
      glowWeight: 107.68697916666667,
      shortId: "5",
    },
  ],
];