FilCDN: SP Payout Architecture

Summary

This design document outlines the architecture for an egress-based payout system for a filCDN. The system tracks egress data provided by SPs and rewards them based on cache misses and egress usage when users request files.

The PandoraService contract is already live and is responsible for managing user opt-ins and user payments for CDN services. filCDNPayout, a separate payout smart contract, is responsible for handling the payments to SPs based on the egress data (and cache misses) associated with each file request.

Users make a lump sum payment via PandoraService for CDN access (withCDN flag enabled), which covers their egress, and SPs are compensated through filCDNPayout based on the egress provided during cache misses. The system ensures that SP payments are processed accurately with flexible payout mechanisms (pull-based [Optional] or push-based). Payments are made using USDFC, a stablecoin, and the payment rates can be adjusted dynamically.

This document describes the system's components, the flow of interactions, and the integration of both PandoraService for user payments and filCDNPayout for SP compensation.

Goals

Track egress and cache hits/misses for both users and Service Providers (SPs), with data already recorded in the D1 database.

Pay Service Providers (SPs) only based on the egress delivered during cache misses.

Implement the filCDNPayout smart contract to manage payments to SPs using USDFC.

Optimize payments to SPs based on actual egress data, factoring in both perc_max and perc_actual.

Allow for flexible payment models (pull-based or push-based) for payouts to SPs. (Pull-based model is optional).

Handle payments to SPs through the filCDNPayout contract, which keeps a state of funds eligible for withdrawal for SPs.

Non-Goals

Real-time streaming of data: The system is designed for traditional CDN services and does not handle real-time or live-streaming data.

Per-request payment model: The user pays a lump sum upfront, not on a per-request basis.

Split the revenue on the smart contract level: The system does not split revenue between different entities at the smart contract level, meaning that SPs just receive an aggregated sum rather than a per-user payment.

Slash SPs for bad retrieval service: There is no functionality for slashing SPs in the case of bad retrieval or service degradation.

Reward based on egress quality (speed, TTFB): The system does not reward SPs based on service quality metrics like speed or Time To First Byte (TTFB).

Components Overview

The system comprises several key components, each with its distinct responsibilities. Below is a breakdown of each component:

1. PandoraService Smart Contract (User Opt-in and Payments)

Description: PandoraService is an already live smart contract responsible for handling user opt-in and user payment for CDN services. The user makes a lump sum payment to access the CDN, which will cover their egress for a specified period.

Key Tasks:
- PandoraService contract manages user opt-in for the CDN service.
- Users pay a lump sum through PandoraService for CDN access.
- The payment is stored, and the D1 database tracks the egress consumed by the user.
- Record payment data and link the user’s payment with their egress usage.

2. Egress Tracking in D1 Database

Status: Live

Description: The D1 database tracks egress data (cache hits and misses) associated with each user and SP. This data is already being tracked and is crucial for calculating SP payments.c

Key Tasks:
- D1 database records and tracks cache hits and misses.
- Egress data is linked to individual users and their usage.
- Calculate egress delivered by SPs during cache misses.
- Data on cache hits and misses is captured in real-time.

3. Cloudflare Worker for Egress Summation

Description: A Cloudflare Worker runs daily to aggregate egress data for cache misses and calculate payouts for SPs.

Key Tasks:
- Summing up daily egress data for cache misses.
- Calculating the perc_actual for SPs’ payout amount.
- Trigger payout calculations for SPs through filCDNPayout based on egress data.

4. filCDNPayout Smart Contract (SP Payments)

Description: filCDNPayout is the smart contract responsible for managing payments to Service Providers (SPs). The contract tracks egress data per SP and forwards payments to SPs based on the amount of egress delivered during cache misses.

Key Tasks:
- Handle payout distributions using USDFC (stablecoin).
- Support both pull-based and push-based payout models (Pull-based is optional).
  - Push-based: The worker triggers the payout each time interval (Each day in this case), and the SP does not have to take any action.
  - Pull-based: The SP triggers the payout directly through the filCDNPayout smart contract, and the worker only updates the state that tracks the funds eligible for payout.

5. User Payment Management

Description: The user makes a lump sum payment to access the CDN. This lump sum is used to cover their egress for a specific amount of data. If the user exceeds this limit, they will be throttled (This functionality is part of a different issue.).

Key Tasks:
- Manage lump sum payments and user egress tracking.
- Implement throttling logic for users exceeding their limit.

Diagrams

Below are three types of diagrams to visualise the system architecture and processes:

1. Flowchart

The flowchart provides an overview of the entire process, from user opt-in to the final payout to the SP.

graph TD
    A[User opts into CDN] --> B[User pays lump sum through PandoraService]
    B --> D{Request from User}
    D -->|Cache Hit| E[CDN serves data from cache]
    D -->|Cache Miss| F[CDN fetches data from SP]
    F --> G[Track egress from SP in D1 database]
    G --> H{End of day}
    H --> I[Cloudflare worker sums egress data]
    I --> J[Calculate perc_actual and perc_max]
    J --> K[Trigger payout calculation for SPs via filCDNPayout]
    K --> L[Smart contract processes payment to SP]
    L --> M[SP receives payout in USDFC]

2. Sequence Diagram

The sequence diagram shows the interactions between the User, CDN System, Service Providers (SPs), Cloudflare Worker, PandoraService smart contract, and the filCDNPayout smart contract.

sequenceDiagram
    participant U as User
    participant CDN as CDN System
    participant SP as Service Provider
    participant DB as D1 Database
    participant Worker as Cloudflare Worker
    participant PS as PandoraService (Filecoin Smart Contract)
    participant FCP as filCDNPayout (Filecoin Smart Contract)

    U->>PS: Opt-in and pay lump sum via PandoraService
    CDN->>DB: Record egress usage (cache hit or miss)
    U->>CDN: Request file
    CDN->>SP: Fetch data (cache miss)
    CDN->>DB: Log egress data for SP
    CDN->>DB: Track cache miss
    Worker->>DB: Sum daily egress data (cache misses)
    Worker->>FCP: Calculate payment for SPs and update state of filCDNPayout
    FCP->>SP: Transfer payout in USDFC
    SP->>SP: Receive payment in USDFC

Explanation of `perc_max` and `perc_actual`

The system utilises perc_max and perc_actual to calculate the payment to Service Providers (SPs) based on the egress data delivered and the number of cache misses.

perc_max

Definition: perc_max represents the maximum percentage of the lump sum paid by the user that can be allocated to SPs, assuming every request results in a cache miss. This value is set to X% of the lump sum.

Purpose: It defines the upper limit of how much of the lump sum can be paid out to SPs in scenarios where all user requests miss the cache.

perc_actual

Definition: perc_actual represents the actual percentage of the lump sum allocated to the SPs, based on the actual egress delivered. This percentage is dynamically calculated based on the real cache miss rate and egress usage.

Purpose: It adjusts the payment to the SPs according to how much egress is actually delivered. If fewer cache misses occur, the payment to SPs will be reduced proportionally. The actual payment is calculated by the Cloudflare Worker based on the daily egress data and cache miss statistics.

The relationship between perc_max and perc_actual ensures that payments to SPs are fairly distributed based on the actual usage of the CDN and the actual cache miss rate.

The formula to calculate perc_actual is:

$perc_{actual} = \frac{\sum egress_{cacheMiss}}{egress_{max}} \times perc_{max}$

Example:

The payout period is 24h.

The amount of egress paid for by the user through the PandoraService is $1TB = egress_{max}$
$perc_{max}$ is 50%, meaning that at most 50% of the funds that a user paid in the PandoraService for CDN enablement are paid out to the SP (This is the case if cache miss is 100% and egress usage is 100%).

In 24h the amount of egress that was used and was a cache miss is $\sum{egress_{cacheMiss}} = 1/30 TB$ .

This results in: $perc_{actual} = \frac{\frac{1}{30}}{1} \times 0.5$ meaning that 1.67% of the fees paid by the user for enabling CDN are paid out to the SP on day 1.