AI Tool

Soup CLI Review

Soup CLI is an open-source command-line interface tool that simplifies the process of training and fine-tuning machine learning models by automating configurations and data management.

shipped Aug 9, 2026free

Domain rating34

Why it matters

1Enables fine-tuning of 8B LLMs on GPUs with as little as 4GB VRAM using layer streaming.

2Automates configuration for LLM training, including GPU detection and batch size adjustment.

3Offers comprehensive post-training tools for data validation, evaluation, and model serving.

4Version 0.72.4 expanded layer streaming to support DPO, ORPO, SimPO, and KTO preference losses.

About Soup CLI

Business Model

Open Source

Platforms

CLI, Web

Target Audience

Machine Learning practitioners and researchers

Pricing Plans

Free

Leadership

Alpamys Makazhan

GitHubOpen Source

Specs

API Docs

View Documentation →

GitHub

View Repository →

API Available

Yes, public API

overview

What is Soup CLI?

Soup CLI is a machine learning operations (MLOps) tool developed by Alpamys Makazhan that enables developers, ML engineers, researchers, and data scientists to fine-tune and manage large language models (LLMs). It simplifies the process of training and fine-tuning machine learning models by automating configurations and data management, designed to handle large models efficiently with features like layer streaming and auto-configuration. A core capability is its "layer streaming" technique, which allows fine-tuning of models like Meta's Llama 3.1 8B Instruct on GPUs with as little as 4GB of VRAM, by keeping frozen base model decoder layers in main memory (RAM or NVMe) and streaming them one layer at a time to VRAM, while only LoRA adapters remain resident. Beyond training, Soup CLI offers tools for data pre-flight analysis (soup advise, soup data doctor), model evaluation (soup ship), and conversion to suitable execution formats such as GGUF. It also supports knowledge editing and unlearning specific data using methods like NPO, SimNPO, RMU, ROME, MEMIT, and AlphaEdit.

features

Key Features of Soup CLI

Soup CLI provides a comprehensive set of features designed to streamline the LLM fine-tuning and deployment workflow, emphasizing efficiency and ease of use, particularly for resource-constrained environments. Its capabilities include automated configuration, advanced data management, and robust evaluation tools.

Automated configuration of fine-tuning parameters, including GPU detection, batch size adjustment, and quantization.
Layer streaming for efficient resource usage, enabling 8B models on 4 GB laptop GPUs by streaming decoder layers from RAM or NVMe.
Support for multiple ML methods and frameworks, including Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), ORPO, SimPO, GRPO, and KTO.
Pre-flight data validation and linting through commands like soup advise and soup data doctor.
Comprehensive model evaluation, diagnosis, and interpretation of LLM performance and behavior via the soup ship command, which includes seven bundled offline evaluation suites.
Exporting and serving fine-tuned LLMs in various formats (e.g., GGUF) and via an OpenAI-compatible API server.
Knowledge editing and unlearning capabilities using methods such as NPO, SimNPO, RMU, ROME, MEMIT, and AlphaEdit.
Open-source under the Apache-2.0 License, ensuring no vendor lock-in and community-driven development.

use cases

Who Should Use Soup CLI?

Soup CLI is designed for machine learning practitioners and researchers who require efficient and simplified tools for fine-tuning and deploying large language models, especially when working with limited hardware resources.

Developers and ML Engineers: For fine-tuning Large Language Models (LLMs) with various methods (SFT, DPO, GRPO, KTO) and deploying them via an OpenAI-compatible API server.
Researchers: For optimizing LLM training on resource-constrained hardware (e.g., 4 GB GPUs) using layer streaming and for evaluating, diagnosing, and interpreting LLM performance and behavior.
Data Scientists: For pre-flight data validation and linting for LLM training, ensuring data quality before model training begins.
Teams transitioning from other ML tools: Soup CLI provides direct migration paths from tools like LLaMA-Factory, Axolotl, and Unsloth, facilitating adoption.

how to use

How to Use Soup CLI

Soup CLI simplifies the LLM fine-tuning process, allowing users to initiate training with a single command and a YAML configuration file. The tool automates many complex setup tasks, making it accessible for various users.

1Installation: Install Soup CLI via pip or by cloning the GitHub repository.
2Data Preparation: Prepare training data, typically in a JSONL format, and specify its path in a YAML configuration file.
3Configuration: Define fine-tuning parameters, model architecture, and training methods (e.g., SFT, DPO) within the YAML configuration.
4Fine-tuning: Execute the soup train command, referencing the configuration file, to initiate the fine-tuning process.
5Evaluation: Utilize the soup ship command to evaluate the fine-tuned model's performance and receive a deployment verdict.
6Deployment: Export the fine-tuned LLM into formats like GGUF or serve it using the integrated OpenAI-compatible API server.

pricing

Soup CLI Pricing & Plans

Soup CLI is an open-source tool distributed under the Apache-2.0 License, making it available for free. There are no paid tiers or subscription plans for the core CLI functionality. API rate limits apply to its integrated services.

Free: All core features, including layer streaming, automated configuration, and post-training tools, are available at no cost.
API Rate Limits: GET requests are limited to 1,000 requests per minute per access token. POST, PUT, and DELETE requests are limited to 100 requests per minute per access token. Limits for the CMS Delivery API and OpenAI-compatible API server are currently unknown.

Pros

+Enables fine-tuning of 8B LLMs on GPUs with as little as 4GB VRAM through its layer streaming technique.
+Automates complex configuration tasks, reducing setup time for LLM training environments.
+Provides a comprehensive post-training stack, including data validation (soup data doctor) and model evaluation (soup ship).
+Supports various fine-tuning methods, including SFT, DPO, ORPO, SimPO, and KTO, for diverse use cases.
+Offers an OpenAI-compatible API server for serving fine-tuned models, simplifying deployment.
+Open-source under Apache-2.0 License, promoting transparency and community contributions.

Cons

−The layer streaming feature is currently in beta, which may imply potential instability or ongoing development.
−Specific API rate limits for the CMS Delivery API and OpenAI-compatible API server are currently unknown.
−While simplifying the workflow, it might offer less granular control over individual components compared to highly modular frameworks like Kedro or Hydra.
−Community support is primarily via Discord and GitHub, which may not suit all users compared to dedicated enterprise support channels.

Similar Tools

Soup CLI vs Competitors

Soup CLI differentiates itself in the competitive landscape of machine learning tools by focusing on resource efficiency, simplified workflows, and comprehensive post-training capabilities, particularly for LLM fine-tuning.

MLflowOn Stork

It provides a platform for managing the end-to-end machine learning lifecycle, including experiment tracking, reproducible runs, and model deployment.

While Soup CLI focuses on simplifying the training and fine-tuning process with automation, MLflow offers a broader suite of tools for experiment tracking, model management, and reproducibility across the entire ML lifecycle. You might need to integrate MLflow more explicitly into your training scripts compared to Soup CLI's more opinionated automation.

DVCOn Stork

It brings Git-like version control to data and machine learning models, enabling reproducible experiments and pipelines.

Soup CLI includes data management as part of its broader training automation. DVC specializes purely in data and model versioning and pipeline definition, requiring you to integrate it into your existing training scripts rather than automating the training loop itself.

Hydra↗

It provides a framework for elegantly configuring complex applications, especially useful for machine learning experiments with many parameters.

Soup CLI offers auto-configuration as part of its end-to-end training simplification. Hydra focuses exclusively on robust and dynamic configuration management, giving you fine-grained control over experiment parameters but not handling data management or the training execution itself.

KedroOn Stork

It's a Python framework that helps build robust, scalable, and reproducible data science and machine learning pipelines with a strong emphasis on code modularity.

Soup CLI aims to simplify the direct training and fine-tuning process. Kedro provides a structured framework for building the entire ML project, including data processing and model training pipelines, offering more control over project architecture at the cost of less out-of-the-box training automation.

See every Soup CLI alternative, compared→

Related AI Tools

Other tools in this category, matched by shared tags

Bid to Up NanoImage SaySorry.in Nerdy — Workout Tracker (No LLM)Marketplace Operations AfterMeeting

Visit Soup CLI↗

Connect

⌘

GitHubgithub.com/MakazhanAlpamys

💬

Discorddiscord.gg/8RgVbFA6Zq