DataFusion Distributed

DataFusion Distributed is a library that enhances Apache DataFusion with distributed capabilities.

These docs will guide you towards using the library for building your own Distributed DataFusion cluster, and how to contribute changes to the library yourself.

User Guide

• Index
• Getting Started
  • Next steps
• Building a TaskEstimator
• Building a ChannelResolver
• Building an Arrow Flight endpoint
• Concepts
  • DistributedPhysicalOptimizerRule
  • TaskEstimator
  • DistributedTaskContext
  • ChannelResolver
  • NetworkBoundary
• How a Distributed Plan is Built

Contributor Guide

• Index
• Setup
  • Prerequisites
  • Clone and Setup
  • Running Examples
  • Resources
• Tests
  • Running Unit Tests
  • Running Integration Tests
  • Resources
• Benchmarks
  • Local Benchmarks
  • Remote Benchmarks