2024 年 5月 1 日随笔档案 - ForHHeart

2024年5月1日

摘要： Blog 1: Mixtral 8✖️7B=56B？错！一文带你看清Mixtral内部结构及参数计算 | Zhihu Blog 2: Mixtral 8x7B(Mistral MoE) 模型解析 | Zhihu Video 1: mixtral系列S1——MoE实现细节 | Bilibili Vid 阅读全文

posted @ 2024-05-01 03:08 ForHHeart 阅读(91) 评论(0) 推荐(0) 编辑

公告

昵称： ForHHeart
园龄： 2年10个月
粉丝： 0
关注： 0

+加关注

2025年3月

日

一

二

三

四

五

六

阅读排行榜

1. Distributed Training: DeepSpeed ZeRO 1/2/3 + Accelerate, Megatron-LM(1076)
2. Env Config(362)
3. Mamba(211)
4. Pre-trained Model Summary(194)
5. Quantization: fp16, bf16, int8, fp4, nf4(144)
6. Tokenizer: BPE, WordPiece, and SentencePiece(138)
7. Mixtral 8×7B (Mistral MoE)(91)
8. Navigation Page(53)
9. Python Basic Notes(53)
10. transformers(46)
11. DS/MLE Key Competency and Occupational Classification Concluded from Job Descriptions(46)
12. Build Lightweight AI SaaS: Next.js + Tailwind CSS(44)
13. DS / ML Basic Notes(44)
14. ML / AI Algorithms from Scratch(38)
15. Reinforcement Learning from Human Feedback(RLHF): TRPO, PPO, DPO(34)
16. Momentum Contrast (MoCo) for Unsupervised Visual Representation Learning(33)
17. LeetCode Notes in Python(32)
18. Transformer(29)
19. LeetCode Hot 100(27)
20. Retrieval-Augmented Generation(RAG)(22)
21. Multimodal Large Language Model(MLLM)(21)
22. LLaMA 2(19)
23. Text Representation: OneHot, BOW, N-grams, TF-IDF, Word2Vec, Glove, FastText, ELMO, BERT, SBERT(18)
24. RecSys & Ads Basic Notes(18)
25. SQL Basic Notes(18)
26. PyTorch Basic Notes(15)
27. BERT(15)
28. LLM local deployment: Xinference, Ollama(14)
29. Data Structures and Sort Algorithms Notes for Coding Interview(14)
30. Self-supervised Learning(12)
31. Python Web Dev(11)
32. RNNs & LSTMs & GRUs(11)
33. Loss Function(10)
34. Docker(10)
35. Interview Experience for RecSys & Ads(9)
36. Statistics Basic Notes(7)
37. Git(6)