| Concern |
Design |
Status |
| Control plane address |
Solo binds to 127.0.0.1 by default. A Lab binds to the LAN or a WireGuard interface, chosen at setup. |
Planned |
| Agents joining |
Each agent dials out to the control plane: one long-polled call, POST /agent/v1/sync over HTTPS with mutual TLS (ADR-0016), so it works behind home routers, NAT, and CGNAT. No inbound ports are opened on machines. In Solo, the agent runs inside the control plane’s process, with no network at all. |
In-process agent built; network transport Phase 2 |
| Enrolment |
A one-time token from the dashboard, after which the agent gets its own certificate, revocable per machine |
Planned |
| Reaching your workloads |
On the same machine or LAN: direct. From elsewhere: a relay through the control plane’s agent connection (notebook UI, endpoint API, VM SSH), authenticated per user. |
Planned (Phase 2) |
| Guest egress (non-owners) |
No access to the host, the LAN, or private and link-local ranges. Community workloads have no internet by default (ADR-0014). |
Planned |
| Owner workloads |
Normal network access, like any app on the machine |
— |
| Labs across sites |
Optional WireGuard between sites. Agents still dial out, so sites don’t need public IPs. |
Planned |
| Range |
Use |
Set in |
| 10.0.10.0/24 |
Node network |
inventory node_ip |
| 10.0.10.200–220 |
MetalLB pool (Gateway and LoadBalancer IPs) |
deploy/environments/*.yaml |
| 10.42.0.0/16 |
Pod CIDR |
infra/ansible/group_vars/all/main.yml |
| 10.43.0.0/16 |
Service CIDR |
same |
| 10.99.0.0/24 |
WireGuard admin VPN |
inventory wireguard_address |
| Public IPs |
Gateway, and VM SSH later |
ISP or colocation |
Give each region its own non-overlapping ranges, so regions can be linked later without
renumbering (ADR-0009).
- Cilium provides the CNI, kube-proxy replacement, NetworkPolicy enforcement, Hubble, and Gateway API (ADR-0007).
- MetalLB in L2 mode hands out LoadBalancer IPs on bare metal.
- Gateway API: one
Gateway named einsora with HTTPS listeners for api., app., and auth.. cert-manager issues certificates through ACME HTTP-01.
- WireGuard is the intended admin path. The firewall that enforces it is a Phase 3 task.
- Control plane versus regional entry points:
api., app., and auth. are served by the control plane. Workload traffic (*.<region>.ep. and *.<region>.nb. hosts, plus VM SSH) enters its region’s own Gateway directly.
- Inference gateway (planned, ADR-0010): API keys, token counts, rate limits, and failover. It routes by API key → tenant → endpoint, never by hostname alone.
These are generated by manifests.Tenant for every tenant namespace.
| Direction |
Allowed |
| Ingress |
From the same namespace, and from the Gateway proxy (the Cilium ingress entity) |
| Egress |
To the same namespace, DNS, and the model store (einsora.dev/role: model-store). Also the public internet (0.0.0.0/0 minus private and link-local ranges). |
Gaps to close:
- IPv6: the policies are IPv4-only, so keep clusters single-stack IPv4 until that changes.
- Node public IPs: add a Cilium cluster-wide policy that stops tenant pods reaching nodes on their public addresses.
- Abuse: block port 25, rate-limit new connections, and alert on scan-like flows.
| Workload |
Proposal |
| Endpoint |
<id>.<region>.ep.<domain>, with a wildcard certificate per region (DNS-01), through the inference gateway |
| Notebook |
<id>.<region>.nb.<domain>, behind an authenticating proxy (oauth2-proxy with the IdP) |
| VM SSH |
A shared IP with a port per VM, or a MetalLB IP per VM, or a bastion |
- At least 2× 25 GbE per node, plus a 1 GbE BMC port on a separate management switch.
- Ceph wants its own VLAN or NICs at 25 GbE or more.
- Dual ISP uplinks and static public IPs per site, and check the provider’s DDoS protection.
- Multi-node training (InfiniBand or RoCE) is out of scope until there’s demand.
