Research.

Systalyze grew out of years of work at MIT CSAIL on making machine learning systems efficient. We keep publishing, and we turn what we learn into the platform.

All blog posts

Papers.

  • NSDI 2026
    Checkmate: Zero-overhead model checkpointing via network gradient replication

    Checkpoints every training iteration with no slowdown, cutting repeated work after a failure by 80–97%.

    Read the abstract

    Checkmate enables per-iteration checkpointing in DNN training without slowdown. Its key insight is that in data-parallel training, all information necessary to create a checkpoint already exists in the network as gradients. A multicast abstraction simultaneously delivers gradients to a separate CPU-based shadow cluster, which maintains a checkpoint by applying them to a copy of the model. Checkmate performs per-iteration checkpointing with throughput comparable to an ideal no-checkpoint baseline. It achieves 5 to 34.5× more frequent checkpointing than state-of-the-art systems, reducing repeated work per failure by 80% to 97.1%, and delivers 1.3× to 6.5× throughput at the same checkpointing frequency.

  • NSDI 2024
    Cassini: Network-aware job scheduling in machine learning clusters

    Interleaves the communication of jobs sharing a link, improving tail completion time by up to 2.5×.

    Read the abstract

    CASSINI is a network-aware job scheduler for machine learning clusters. Its geometric abstraction and affinity graph find time-shift values that interleave the communication patterns of jobs sharing a network link. Experiments with 13 common ML models on a 24-server testbed show that CASSINI improves average and tail completion time by up to 1.6× and 2.5×, respectively, compared to state-of-the-art ML schedulers. It also reduces ECN-marked packets by up to 33×.

  • HOTI 2024
    Rail-only: A low-cost network for training trillion-parameter LLMs

    Matches conventional training performance while cutting network cost by 38–77%.

    Read the abstract

    This paper presents a low-cost network architecture for training large language models at hyperscale. Because LLM training generates sparse communication patterns, it does not require an any-to-any full-bisection network. The rail-only design eliminates the spine layer in traditional GPU clusters and maintains training performance while reducing network cost by 38% to 77% and power consumption by 37% to 75%. It also supports Mixture-of-Expert models through forwarding, with only 8.2% to 11.2% completion-time overhead. The paper studies failure robustness and the impact of network and training parameters.

  • arXiv 2024
    MLTCP: Congestion control for DNN training

    A few dozen lines added to standard congestion control speed up training iterations by up to 2×.

    Read the abstract

    MLTCP augments congestion-control algorithms to accelerate DNN training jobs in shared GPU clusters by interleaving the communication phases of jobs competing for bandwidth. Its central principle is that DNN training flows should scale their congestion window based on bytes sent in each training iteration. Adding 30–60 lines of code to Reno, CUBIC, or DCQCN stabilizes flows into an interleaved state within a few iterations, regardless of flow counts or start times. MLTCP accelerates average and 99th-percentile training iteration time by up to 2× and 4×, respectively.

  • IEEE CloudNet 2024
    Nona: A stochastic congestion-aware scheduler for real-time inference

    Queueing-theoretic scheduling that balances compute and network to cut inference completion time.

    Read the abstract

    Nona combines a stochastic scheduler with an offline optimization formulation rooted in queueing theory to minimize the average completion time of heterogeneous inference queries. It incorporates compute and network tradeoffs, uses the Pollaczek–Khinchine formula to estimate queueing latency, and builds on Jackson networks to capture dependencies between computation and communication operations. A graph-contraction procedure lets cloud providers solve the optimization in practical settings. Evaluation with models including AlexNet, ResNet, DenseNet, VGG, and GPT-2 shows that Nona outperforms state-of-the-art schedulers by up to 350×.

  • HotNets 2022
    Congestion control in machine learning clusters

    Shows that deliberately unfair sharing can make every competing training job faster.

    Read the abstract

    This paper argues that fair sharing is not necessarily desirable in machine learning training clusters. For compatible combinations of jobs, deliberately introducing unfairness improves training time for every competing job. A geometric abstraction rolls time around a circle and rotates communication phases to identify fully compatible jobs, demonstrating up to 1.3× improvement in average training iteration time. The paper proposes three directions for resource management: adaptively unfair congestion control, priority queues on switches, and precise flow scheduling.

See the research at work on your own workloads.

Sign up for Sovereign AI