FilCDN Overview

Filecoin is lacking reliable retrievals. Checking retrievability using synthetic traffic give us probabilistic guarantees/estimates, but that does not guarantee the performance of retrieval requests made by the real users. We don’t even have visibility into that data.

Customers buying storage products want retrievals that work. They are not interested in measuring retrievability per se. They expect the data to be (almost) never lost and (almost) always retrievable, with SLA as close to 100% as possible.

Decentralised CDNs don’t bring enough value and are extremely difficult to build. If we leverage existing CDN like Cloudflare, we can ship fast retrievals much sooner. By using a trusted CDN setup, we can measure retrieval SLI as part of the retrieval flow. This way, we replace a proxy metric (retrieval success rate for synthetic traffic) with the measurements of real-user retrieval requests and avoid attack vectors like SPs offering better QoS for checker requests.

Using a CDN unlocks the following benefits:

• We keep the requirements on SPs much lower. They are not expected to handle internet-level load, just be a good cache-miss layer.

• Potential feature: Retrieval requests from our CDN to SPs can be authorised - SPs can serve retrievals to our CDN only, they don’t have to serve retrievals to any random client.

• We can record the telemetry about the performance of real retrievals requests to produce the on-chain SLI data consumed by the SLA contract(s) gating payouts.

• We get fast retrievals all around the world by caching the response from SP on the edge, assuming SPs serve cache-miss requests well.

• The trusted CDN layer can provide trusted retrieval metrics that can be used to reward SPs for providing good retrieval service for cache-miss requests.

Retrievals from Filecoin are not a solution people would buy on its own. A significant group of users requires a combo of easy file upload plus fast reliable retrievals.

By combining the recently introduced PDP mechanism with Cloudflare edge cache, we can quickly build a minimal file storage service as a tiny layer on top of Filecoin & Filecoin Services. (Initially, this service will be for publicly-accessible data only, e.g. pictures shown on a personal website or product images shown on e-commerce websites. Access controls are out of the scope of the initial iterations.)

Our goal is to validate whether there is demand for such a paid service.

Happy path demo

Web GUI flow (based on hotvault-demo):

1. Open the Filecoin Services (FS) website

2. Follow the hotvault-demo flow onboarding flow to create a paid deal (a new ProofSet = new bucket). Check the check-box to include CDN retrievals. The UI should link to filcdn.com and mention how much egress is included in the package.

3. Follow the hotvault-demo flow to upload your file to PDP.

4. After you upload the file, you will get back CommP CID.

5. The website gives you an URL for downloading the file. This URL points to FilCDN subdomain specific to your ProofSet and includes CommP CID in the path.

Synapse SDK flow:

1. Setup the Synapse SDK client, connect your wallet

2. Follow the Synapse flow to create a paid deal (a new ProofSet = new bucket)

3. Follow the Synapse flow to upload your file to PDP.

4. As part of this process, Synapse gives you CommP CID of your file.

5. Synapse SDK also gives you an URL for downloading the file. This URL points to FilCDN subdomain specific to your ProofSet and includes CommP CID in the path.

Code example based on https://www.notion.so/filecoindev/Specify-the-feature-sets-to-build-for-the-hackathon-M1-1fbdc41950c180ca877fe1e772abe562?pvs=4#1fcdc41950c180cab0a1d84e26feb37d

const synapse = new Synapse({ privateKey: '...', withCDN: true })

// (...)

const uploadTask = synapse.upload(myByteBlob) // assume single proof set for now?

await uploadTask.done()

/* RETRIEVE */
const commp = await uploadTask.commp()

// Get CDN URL you can use e.g. in `<img>` tags
const url = await synapse.getRetrievalUrl(commp)
// (...)

// Alternatively, you can download it via SDK
// Benefit: you can verify the response matches CommP
const myByteBlob = await synapse.download(commp)

Pricing model

We expect the pricing to evolve as we get more customer feedback.

M1 : June hackathon

According to https://www.notion.so/filecoindev/Specify-the-feature-sets-to-build-for-the-hackathon-M1-1fbdc41950c180ca877fe1e772abe562, there will be a fixed fee 1 USDFC for CDN component.

Based on the calculation below, that would mean ~3-5GiB of egress.

• Does this enable all hackathon use cases? If they stream video, it might not be enough. The CDN shouldn’t be a reason why a project isn’t possible. Post-hackathon it’s different, because we’ll allow people to pay for more. For now, what about something like 100GiB instead?

• Question for FilOz: What kind of files do you expect hackathon participants to store? (What is a typical and max allowed root size?) What’s a reasonable egress allowance per ProofSet we should provide for the hackathon participants?

M2: November 2025

For the public launch in November 2025, we are thinking about a pay-as-you go pricing model, charging $X USDFC per 1GiB of egress traffic.

Note:

• There are two costs involved: paying for the traffic between the retrieval clients and FilCDN and paying for the traffic between FilCDN edge and SPs on cache-misses. How can we fairly compensate SPs for their egress?

  Idea: make this transparent and charge customers two prices:

  1. $X per 1GiB of edge egress, the proceeds are paid to FilCDN.
  2. $Y for 1GiB of cache-miss traffic, the proceeds are paid to the SP.

  Strawman proposal to be clarified after June 6:

  • $0.2 USDFC per 1 GiB of edge egress
  • $0.1 USDFC per 1GiB of cache-misses to SP

  Discussion:

• We may be able to “swallow” the costs of SP cache-miss traffic by adding https://developers.cloudflare.com/cache/advanced-configuration/cache-reserve/ to our mix.

Technical Architecture

The FilCDN is a service which forwards file retrieval requests to the right SPs serving PDP file retrieval. It leverages the caching mechanism of Cloudflare and offers a single point of entry for file retrievals.

Entities & Components

• Developer is the person using Synapse SDK to build a software uploading & downloading data.

• User is the person downloading data stored on Filecoin. The data can be provided by the user (e.g. when the developer created project allowing users to upload their content) or by the developer (e.g. when a personal website with a gallery is hosted on Filecoin).

• CloudFlare is a web2 provider of CDN & Serverless services.

• FilCDN is a service built on top of CloudFlare that serves retrieval requests from users by using a Cloudflare-based cache with cache-misses forwarded to the SP.

• SP is the entity storing content provided by the application built by the Developer.

Retrieval flow

sequenceDiagram
participant u as User
participant fcdn as filcdn.io<br/>(running on CloudFlare)
participant sp as SP
participant cv as FilCDN Verifier Contract

u->>fcdn: Make file retrieval request
alt Cache Hit
        fcdn->>u: Return File
else Cache Miss
        fcdn->>sp: Request CommP content
        sp->>fcdn: Return the content
        fcdn->>fcdn: Cache File
        fcdn->>u: Return File

end

loop On schedule
	fcdn->>cv: Submit aggregated telemetry to the Verifier contract
end

See FilCDN Architecture for a detailed version.

What’s missing:

How is FilCDN going to interact with the on-chain data indicating which ProofSets opted into CDN retrievals.

Payment Flow

To be determined.

Assuming the product design outlined here:

• Product 1: 2 USDFC for 1 TiB of storage. Data will be retrieved directly from the SPs.

• Product 2: 3 USDFC for 1 TiB of storage, with CDN add-on and data will be retrieved from the CDN.

When the deal includes a CDN add-on, the extra 1 USDFC will be paid to FilCDN for providing the retrieval service.

Roadmap (WIP)

See also what FilOz wants to build by June 6 (M1):

M0.1: Proof of Concept (shipped)

• A basic caching CDN service running on Cloudflare.

• Request format: GET https://calibration.filcdn.io/{ProofSetId}/{CommP}

• Only one SP supported - all cache-miss requests go to yablu.net.

• No validation of whether the request is valid or not (CommP is live, belongs to ProofSetId, ProofSet is live, and CDN flag is enabled in ProofSet metadata).

• Minimal cache eviction (i.e. Cloudflare defaults).

• No subdomain isolation.

• No visibility into CDN performance for FilCDN customers (hackathon developers), i.e. no dashboard, no retrieval stats.

M1a: Working CDN without payments

• Builts on top of M0.1

• Adds validation of whether the request is valid or not (CommP is live, belongs to ProofSetId, ProofSet is live, and CDN flag is enabled in ProofSet metadata).

Stretch goals:

Out of scope:

• More sensible cache eviction

• Subdomain isolation

M1b: CDN with Payments (stretch goal)

• Builts on top of M1a

• FilCDN receives a cut from the deal fee as a compensation for the CDN service.

• Details TBD.

Post-M1 (June-August)

• Sensible cache eviction

• Dynamic billing model, e.g. pay-as-you-go based on egress consumed.
  • Overview of existing Web2 and Web3 egress prices:

• How to compensate the SP for serving cache-miss requests.

• Isolation via subdomains - when a ProofSet contains bad content that becomes blocked at DNS level, other ProofSets must remain accessible. (Do the same thing that IPFS Gateways and GitHub does.)

Ideas to consider:

• Custom domains. Allow customers to bring their own domain replacing bucket.filcdn.io
  • Cloudflare for SaaS  allow us to let customers use their own domain name for our CDN service.

Questions to answer:

• Who is our target audience?
  • Are we targeting web3/crypto-native users?

• How to deal with users uploading bad content we don’t want to serve publicly under our brand (e.g. explicit content).

• How much is this going to cost us on Cloudflare bills? What would a sustainable financial model look like? Will it be competitive with what AWS/GCP/Azure offers?

== RAW NOTES & IDEAS, POSSIBLY OUTDATED ==

Suggestions from Adrian Lanzafame (at FDS-6):

• Keep it simple. Maybe we don’t need FWS for the initial MVP. Maybe not even any payments.

• If we build a demo running on testnet, with free service, storing data for 30 days only, Filecoin-branded, then FF may get us on the filecoin.io page.

• Adrian is operating an SP in Amsterdam on behalf of FF, he is happy for us to use it as the SP storing files uploaded from our service.

• Re: verifiable retrievals - start with what’s simpler to use, i.e. raw data download.
  • We can add verifiable retrievals later if there is demand.

Architecture — OUTDATED

Upload workflow:

• TBD, https://github.com/FilOzone/hotvault-demo

• At the end, user’s file is stored as a new PDP root

Download workflow:

• A Cloudflare worker fetches the PDP root (CommP) from the SP, verifies content and returns it back to the caller. The response headers will instruct Cloudflare to cache the response. We will need to tweak the cache eviction rules later.

• The Cloudflare worker will report whether the retrieval was successful or not to the FWS SLI contract.

SLI/SLA:

• Based on the data reported by the Cloudflare worker.

• Can we leave it out of the initial demo?

Payments:

• We can use FWS, or build something simpler ourselves, or leave payments out of the initial demo.

Future improvements

• Store each file with multiple SPs to get enough redundancy and handle PDP failures.
  • We can e.g. use 2+1 erasure encoding scheme and store each file with 3 SPs.
  • Ideally each SP should be in a different geographical region.

• Paid retrievals. Since we can measure the number of cache-miss requests sent to SP and the amount of bandwidth consumed, we can start paying the SP for serving retrievals. (The FilStore user will pay for the retrievals, not the retrieval client.)
  • This can potentially be another SLI/SLA in FWS.