v.1.4.1

Changelog

• Support comparator to be a centralized coordinator (PR #69).
• Allow to deploy LLM agents, reward model and comparator in different devices (commit 7cbb792).
• Allow copying current models as comparators on new devices (commit d8d493a)
• Optimize GPU memory usage by timely releasing the unused models (commit d0c6376).

v.1.4.0

Version 1.4.0

Changelog

• Add online preference-learning methods. (#68).
• Add dataset drift logger.

v.1.3.9

Changelog

• Add offline preference-learning methods, MADPO, MARLHF, etc. (#67)

v.1.3.8

Changelog

• Allow using the reference model for all algorithms (PR #65).
• Unify the logger between different trainers (PR #66).

Check both of these pull requests to see performance results and resource estimates.

v.1.3.7

Changelog

• Remove the redundant sampling hyperparameters in algorithms.
• Allow multi-gpu training with MP (see test results at PR #62).

Notes: Currently, we limit the scope of parallel training via model parallel for research use (4x7/15B models should be fine). Ray-based distributed training for a larger scope is unnecessary and has not been developed yet.

v1.3.6

This release (v1.3.6) provides the code used in Learning Decentralized LLM Collaboration with Multi-Agent Actor Critic.
See CoMLRL GitHub and Docs for more details about the latest version.

Core Features

1. MAGRPO, IAC, MAAC Trainers for Decentralized LLM Collaboration
2. Environments including Writing, Coding, and Minecraft Collaboration.

Changelog

• #60 Fixed critical error in hetero model loading, and allows more flexible critics and agents (see docs/model-loading).
  
• Change downstreaming repos' interfaces accordingly (align them into v1.3.6) and polish docs.

v.1.3.5

Changelog

• Add unit tests for hyperparameter constraints.
• Clean legacy interfaces.

v.1.3.4

Changelog

• Fix the bug of loading heterogeneous models and reform the loading logics.
• Enable MBGD in MAGRPO to align with MAAC and IAC.
• Remove redundant and legacy hyperparameters (e.g., model kwargs, patching hyperparameters).
• Clean multi-device legacy, like drop last and num_workers.
• Add unit tests for model loading and separate it from CI as a badge.
• Clean short functions.
• Reorganize the docs and align the parameters.

v.1.3.3

Changelog

• Compact MAREINFORCETrainer derivation, and move to the new folder.
• Unify the interface for different trainers.
• Remove redundant patches and wrappers.
• Reorganize the variables in the config yamls.

v.1.3.2

Warning

Deprecated: A new version supports more flexible interfaces for heterogeneous LLMs loading is provided in v1.3.6.

This release (v1.3.2) provides the code used in Learning Decentralized LLM Collaboration with Multi-Agent Actor Critic.
See CoMLRL GitHub and Docs for more details about the latest version.

Core Features

1. MAGRPO, IAC, MAAC Trainers (off-policy) for Decentralized LLM Collaboration
2. Environments including Writing, Coding, and Minecraft Collaboration.

Changelog

• Fixed wandb logging issue in MAGRPOTrainer.
• Align all environment repos with version 1.3.2.