ZorBA: Zeroth-order Federated Fine-tuning of LLMs with Heterogeneous Block Activation
arXiv:2603.04436v1 Announce Type: new Abstract: Federated fine-tuning of large language models (LLMs) enables collaborative tuning across distributed clients. However, due to the large size of LLMs, local updates in federated learning (FL) may incur substantial video random-access memory (VRAM) usage.…
