Public Alpha was announced, what’s next?

Product-market fit

• Get the first module to run on Zinnia.
  • Find the first team to commit to build and ship their module for Zinnia.
  • Along the way, find out what APIs and other functionality they are missing, and build it.

• Get the first module to pay rewards to Station operators.
  • Without rewards, the entire proposition of Station is a kind of busted, isn’t it?
  • Can we (the Station as a deployment platform) take a cut from the rewards, to cover the costs of building & running this project?

Marketing

• Present Zinnia at the Mother of all demos in March (2022-03-17).

• Present Zinnia at EngRes AllHands spotlight (2022-03-09)
  • Caveat: I will be working from a different place, the internet connection can be unreliable sometimes. Maybe I can pre-record the video?
  • Miroslav to reach out to Misty

• Give a talk about Zinnia at IPFS Thing 2023 in April.
  • Maybe we can run a workshop too?

• Public hackathon in June - does this still make sense?

Developer Experience

• ✅Build binaries for each release #23 (Super important to do ASAP, IMO.)

• ✅ES Modules #43

Functionality & APIs

• p2p: stream response payload #56
  • Lift the current somewhat-arbitrarily-chosen response size limit of 10MB.
  • I will also learn more about rust-libp2p and Deno internals.

• ✅ Web Cryptography API #33
  • A low-hanging fruit - just expose the Deno implementation. 🤞🏻

Maintenance →

• ✅ Upgrade Deno core & runtime to the new version released on Feb 25th.
  • They have reworked internals to use ESM, which is great for us too, but requires extra work on our side.

• ✅ Upgrade rust-libp2p to v0.51.0 released on Feb 24th.

• Clean up left-over TODOs from the sprint to Public Alpha.

On hold and/or waiting for demand

• Sandboxing and smart resource usage → Let’s defer to Q2’23 or later

• Filecoin Station Integration → We can wait for the first module as we have an easy short-term solution.

• Modules built using Rust/WASM → Work on getting more feedback - how much demand there is?

Ideas & Brainstorming

Feb’23 → v0.1 Public Alpha 🚀 [shipped]

GitHub Milestone: https://github.com/filecoin-station/zinnia/milestone/1

next → incremental DX Improvements

Tracking GitHub issue: https://github.com/filecoin-station/zinnia/issues/30

Make it easier to build modules. Themes to choose from:

• Address feedback from module builders.

• Zinnia CLI installation. We should have an automated CD workflow to build & publish binary artefacts for all supported platforms to GitHub Releases.
  • Should have: Apple Silicon support, code signing and notarization.
  • Nice to have: package managers like Homebrew (macOS), apt-get & rpm (Linux), winget or Chocolatey (Windows).

• ESM imports. Allow modules to be composed of multiple source code files. Nice to have: support for third-party dependencies, e.g. using the Deno approach (import from HTTPS).

• Type definitions. When building modules in VS Code, developers should have type-aware auto-completion for Zinnia APIs. There should be an automated way of running a TypeScript-powered check of all module source code.

• Testing. How should module authors write automated tests for their modules?

• More platform APIs - Web APIs, libp2p features, IPFS client (Iroh)?

• More comprehensive documentation for module builders.

Restrict which resources a module can access

• E.g. hosts in the local network?

Start limiting resource consumption

• E.g. number of concurrent network connections, number of requests send per second, up/down bandwidth

Persistence - a key/value storage

Jun’23 → Public Hackathon

• We need to determine what work is required to make this happen. Are there any features we consider required for successful hacking?

• Maybe this can be a public beta release?

On-demand: Filecoin Station Integration

GitHub Milestone: https://github.com/filecoin-station/zinnia/issues/75

Once the first module is built on top of Zinnia, we should quickly integrate Zinnia into the Station.

For example, as a short-term solution, we can use the current approach where each module is a standalone executable binary, with the slight modification that new modules will contain two pieces - Zinnia CLI and the module JS bundle.

On the Zinnia side, we may need to implement platform APIs for the following features:

• Get the FIL wallet address for receiving rewards.

• Publish Activity Log items.

• Report completion of jobs.

On-demand: Modules built using Rust/WASM

GitHub Issue: https://github.com/filecoin-station/zinnia/issues/74

Some of the work required to make this happen:

• Tooling for building a Rust project for Station. This will typically involve Deno’s wasmbuild and possibly other build steps. A minimal solution is to write documentation explaining the manual steps.

• SDK exposing Zinnia Platform APIs. This can be a light wrapper around js_sys and web_sys crates, plus functions using wasm-bindgen to invoke our custom APIs, e.g. libp2p, Activity Log reporting, and so on.

• Zinnia CLI need to know how to load and run these modules.

• Documentation for module builders

Future features

• Website - a different landing page for Module Authors, as part of the main website.

• Documentation for module developers

• WASM API for implementing handlers for custom libp2p protocols

• Zinnia needs to tell the module what is the FIL address for receiving rewards. This can be pushed by Zinnia (via something like WASI ENV vars) or pulled by the module (via a getter function provided by our WASM API).

• Network sandboxing - reject connections to hosts in the local network. (Is there a sufficiently reliable way how to detect whether a hostname or an IP address is in the local network vs. the public internet?)

• Static limiting of network bandwidth - throttle read & writes to keep the overall bandwidth usage under 10MB/s. (Or a different threshold we agree on.)

• Static limiting of network traffic - number of open connections at a given time, number of new open connections per minute, number of different hosts connected to per minute.

• Static limiting of CPU usage. To keep things simple, let’s start by ensuring that all Zinnia code executes on a single CPU or maybe in a single thread?

• Key/value storage (block storage)
  • WASM API
  • static limit on storage used (e.g. 20GB)
  • dynamic limit on storage used (e.g. 60% of available free space)

• Dynamic limiting of network bandwidth

• Dynamic limiting of CPU usage. Detect when the computer is (relatively) idle and allow modules to consumer more CPU power.

• Headless mode for running Zinnia on the server - see Station Core Roadmap

• Auto-update modules
  • Periodically check GitHub Releases of each module.
  • When a new release is available, download the WASM bundle and restart the module.
  • Do we need some sort of migration mechanism for modules, to allow them to update their data stored in the key/value storage?
  • At minimum, we need some way to tell the module to cleanly shut down. (Unregister from the network, etc.)

• Virtual FS on top of our key/value store

• IPFS content retrieval client
  • MVP:fetchIpfs(cid, protocol, remote_address)
    • This enables a module to check retrievability of content from Filecoin SP
  • Better: fetch("ipfs://{cid}")
    • This is more practical for general use.

• IPFS content provider
  • Announce content to the network, allow other peers to retrieve it.

OLD CONTENT - NO LONGER RELEVANT

June 2023: v0.3 - Content Retrieval Checker

GitHub Milestone: https://github.com/filecoin-station/zinnia/milestone/3

Retrieval checker module

• At a hard-coded interval, the module dials our server and invokes a custom libp2p protocol to receive instruction for what to check - CID, protocol, remote_address.

• Next, the module attempts to perform the check by retrieving the given CID and captures telemetry like time to first byte, etc.

• Finally, the module invokes another custom libp2p protocol to report the check results.

• The module should report status to Station’s Activity Log (especially when transitioning between “can run checks” and “cannot reach the orchestrator” modes).

• The service to pick a random checking task and collect check results will be centralised for now, operated by the Station team with no guarantees on availability and reliability.

Zinnia features

• WASM API for dialing custom protocols
  • this requires WASM API for async streams

• Lightweight integration with Iroh - WASM API fetch(cid, protocol, remote_address)

• WASM API for creating Activity Log items (info, error)

Feb’23 → v0.1 Public Alpha 🚀 [shipped]

GitHub Milestone: https://github.com/filecoin-station/zinnia/milestone/1

Allow the first adventurous module builders to experiment with the new platform and let us know what features they are missing in Zinnia.

“If you aren’t embarrassed by the first version of your product, you shipped too late.”
― Reid Hoffman [source]

Features

• Modules built using JavaScript ✅

• A CLI to run the module (zinnia run module.js) ✅

• A subset of Timers API, e.g. setTimeout, setInterval

• Console logging APIs, e.g. console.log (see Web APIs :: Console) ✅

• A subset of networking APIs, like Web APIs :: Fetch API and libp2p APIs ✅

• Minimal documentation for module builders 🟢

Out of scope

• Modules built using Rust/WASM

• ESM imports (each module is a single JS file)

Product work

• Which modules require which APIs from Zinnia? 🟠

• Clarify priority of these requirements to inform planning of our work in March-June work. 🟠

• Setup module-builders-wg ✅

• GTM for Zinnia Public Alpha ✅