Appearance

Planet-scale distribution for the BEAM.

Partisan is a scalable and flexible membership system and distribution layer for the BEAM. With Partisan, you can push the BEAM distributed actor model to achieve planet-scale geo-distribution.

DocsSource
Latest release on Hex
01 — Features

Key Features

Partisan provides three key features and optimizations for building large distributed actor systems on the BEAM.

App-specific Overlay Topology Selection

Select the appropriate overlay topology for your app at runtime while keeping the BEAM's programming model. You can choose between the provided topologies or build your own.

  • Full-mesh (~200 nodes): nodes are fully connected and use direct messaging.
  • Client-server (~1,000 nodes): a hub-spoke model where clients do not see one another and messages are routed through servers.
  • Peer-to-peer (~2,000 nodes): nodes partially connected using HyParView, message routing transitively through peers.

Named Channels and Affinitized Parallelism

The BEAM distribution system allows only a single connection between a pair of nodes. Partisan overcomes this limitation offering multiple connections.

  • A named channel is a set of one or more network connections
  • Each named channel can be configured individually e.g. parallelism (number of connections) and compression level.
  • Partition messages across named channels based on type e.g. separate control plane messages from user requests.
  • Maintain channel affinity: Partition actor communication amongst channel connections using an identifier.

Messaging Patterns and Delivery Guarantees

Partisan offers support for message delivery patterns and guarantees.

  • Broadcasting: via its implementation of Epidemic Broadcast Trees.
  • Acknowledged delivery and retransmits using an in-memory message store.
  • Causal delivery (Experimental) using an in-memory causal store.
02 — Topologies

Overlay Topologies

Select the appropriate overlay topology for your app at runtime while keeping the BEAM's programming model.

Full-mesh

Every node holds a connection to every other node, and messages go direct.

Client-server

Servers form a full mesh. Each client connects to a server, never to another client, and servers relay messages between clients.

failed

Peer-to-peer (HyParView)

Each node keeps a small active view of connected peers and a larger passive view of backups. When a peer fails, its neighbours promote replacements from their passive views. With transitive forwarding enabled, messages reach other nodes through peers.

active viewpassive view
e.g. ring e.g. tree e.g. regions

Bring your own

Implement the partisan_peer_service_manager behaviour and select your module with the peer_service_manager option. It defines the overlay topology and how the membership view is maintained.

03 — Reference

Academic Papers

  1. Partisan: Scaling the Distributed Actor RuntimeChristopher S. Meiklejohn, Heather Miller, Peter Alvaro
  2. Partisan: Enabling Real-World Protocol EvaluationChristopher S. Meiklejohn
  3. Partisan: Enabling Cloud-Scale Erlang ApplicationsChristopher Meiklejohn, Heather Miller
  4. Lasp: A Language for Distributed, Coordination-Free ProgrammingChristopher Meiklejohn, Peter Van Roy
  5. Lasp: A Model for Distributed, Convergent, Edge ComputationChristopher Meiklejohn
  6. Epidemic Broadcast TreesJoão Leitão, José Pereira, Luís Rodrigues
  7. HyParView: a membership protocol for reliable gossip-based broadcastJoão Leitão, José Pereira, Luís Rodrigues
  8. X-BOT: A Protocol for Resilient Optimization of Unstructured Overlay NetworksJoão Leitão, João Pedro Marques, José Pereira, Luís Rodrigues
04 — Reference

Presentations

Introducing Bondy: The IoT, Web and Mobile Platform

Code BEAM Europe 2023, Alejandro Ramallo

Partisan: Scaling the Distributed Actor Runtime

2019 USENIX Annual Technical Conference, Christopher S. Meiklejohn

Partisan: testable, high performance, large scale distributed Erlang

Code BEAM SF 2019, Christopher S. Meiklejohn

Distributed Erlang: From Datacenter Applications to Planetary Scale Applications

Christopher Meiklejohn

Rethinking the Language Runtime for Scale

Erlang User Conference 2016, Christopher S. Meiklejohn

05 — Community

Join the Community

Partisan's community meets in the #partisan channel of the Bondy Zulip. Bondy runs on Partisan and the same team maintains both, so the two projects share one workspace.

  1. Join the workspace

    Create your account on the Bondy Zulip with the invitation link. It is free, and one account covers every Bondy and Partisan channel.

  2. Subscribe to #partisan

    Ask about using or contributing to Partisan: membership, overlays, channels and configuration, from Erlang, Elixir or Bondy.

  3. Post under a topic

    Every Zulip message lives under a channel and a topic. Reply in an existing topic, or start one with a short, descriptive name. Link code and paste errors verbatim.

Join us on ZulipOpen #partisanReport an issue

Everything on Zulip and GitHub follows the Bondy Code of Conduct.