Fed-GRPO; A Research-Stage Route to Collaborative Reasoning-Model Training
The proposed framework uses reward signals to coordinate federated GRPO training without sharing raw training data. Its reported evidence covers mathematical-reasoning experiments, not production deployment or formal compliance.
Fed-GRPO targets collaborative training where privacy or regulatory constraints make centralized data pooling unsuitable. It combines reward-standard-deviation-weighted client aggregation, local-global reward calibration for per-prompt objectives, and adaptive sparse communication based on client learning-signal strength.
In the reported mathematical-reasoning experiments, the framework outperformed the federated baselines, approached centralized GRPO performance, and reduced communication 32× without loss in the reported setting. Under tight bandwidth budgets, it supported compression of up to 621× with a gradual accuracy-bandwidth trade-off. The main reported table evaluates five mathematical-reasoning benchmarks using Qwen2.5-3B-Instruct as the base model.