All Tags
Browse through all available tags to find articles on topics that interest you.
Browse through all available tags to find articles on topics that interest you.
Showing 1 results for this tag.
Mano: Restriking Manifold Optimization for LLM Training
This paper introduces Mano, a novel optimizer for training large language models that re-approaches manifold optimization. Mano addresses the limitations of existing optimizers like AdamW and Muon by projecting momentum onto the tangent space of model parameters and constraining it on a rotational Oblique manifold, demonstrating superior performance and efficiency.