当前标签：optimization

Communication Efficient Large-Scale Training with Adam's Convergence Speed

馒头and花卷 2025-02-13 21:28 阅读:5 评论:0 推荐:0 编辑

Convergence of Adam Under Relaxed Assumptions

馒头and花卷 2025-01-13 16:42 阅读:7 评论:0 推荐:0 编辑

Memory Efﬁcient Optimizers with 4-bit States

馒头and花卷 2024-12-07 13:58 阅读:18 评论:0 推荐:0 编辑

8-bit Optimizers via Block-wise Quantization

馒头and花卷 2024-12-06 11:21 阅读:67 评论:0 推荐:0 编辑

Symbolic Discovery of Optimization Algorithms

馒头and花卷 2024-11-28 14:53 阅读:10 评论:0 推荐:0 编辑

An Empirical Model of Large-Batch Training

馒头and花卷 2024-11-26 17:01 阅读:55 评论:0 推荐:0 编辑

Adafactor: Adaptive Learning Rates with Sublinear Memory Cost

馒头and花卷 2024-09-11 15:28 阅读:96 评论:0 推荐:0 编辑

Memory-Efficient Adaptive Optimization

馒头and花卷 2024-09-10 21:25 阅读:14 评论:0 推荐:0 编辑

Adam-mini Use Fewer Learning Rates To Gain More

馒头and花卷 2024-08-28 15:58 阅读:38 评论:0 推荐:0 编辑

GaLore Memory-Efficient LLM Training by Gradient Low-Rank Projection

馒头and花卷 2024-08-27 16:05 阅读:83 评论:0 推荐:0 编辑

BAdam: A Memory Efficient Full Parameter Optimization Method for Large Language Models

馒头and花卷 2024-08-27 10:12 阅读:120 评论:0 推荐:0 编辑

Why Transformers Need Adam: A Hessian Perspective

馒头and花卷 2024-08-26 17:13 阅读:65 评论:0 推荐:0 编辑

优化与收敛率小记

馒头and花卷 2024-07-18 20:19 阅读:114 评论:0 推荐:1 编辑

Regularized Stochastic Learning and Online Optimization

馒头and花卷 2024-07-16 09:27 阅读:53 评论:0 推荐:0 编辑

AdaBelief Optimizer: Adapting Stepsizes by the Belief in Observed Gradients

馒头and花卷 2024-07-10 17:05 阅读:36 评论:0 推荐:0 编辑

Double Descent in Adversarial Training: An Implicit Label Noise Perspective

馒头and花卷 2022-04-14 22:26 阅读:68 评论:0 推荐:0 编辑

Exploring Memorization in Adversarial Training

馒头and花卷 2022-04-12 21:32 阅读:92 评论:2 推荐:0 编辑

DropAttack: A Masked Weight Adversarial Training Method to Improve Generalization of Neural Networks

馒头and花卷 2022-04-02 14:41 阅读:87 评论:0 推荐:0 编辑

HAT

馒头and花卷 2021-11-10 16:12 阅读:519 评论:0 推荐:0 编辑

FAT

馒头and花卷 2021-11-09 19:01 阅读:361 评论:0 推荐:0 编辑