06 2023 档案
摘要:注意力层: ``` 输入 -> LLQ -> @ -> /√ES -> softmax -> @ -> LLO -> Dropout -> 输出 | ↑ ↑ + > LLK + | | | + > LLV + ``` FFN 层: ``` 输入 -> LL1 -> GELU -> Dropout -
阅读全文
摘要:```py # Bert 编码器模块 # 由一个嵌入层和 NL 个 TF 层组成 class BERT(nn.Module): """ BERT model : Bidirectional Encoder Representations from Transformers. """ def __in
阅读全文
摘要:```py # PFF 层,基本相当于两个全连接 # 每个 TF 块中位于注意力层之后 class PositionwiseFeedForward(nn.Module): "Implements FFN equation." def __init__(self, d_model, d_ff, dro
阅读全文
摘要:```py # 注意力机制的具体模块 # 兼容单头和多头 class Attention(nn.Module): """ Compute 'Scaled Dot Product Attention """ # QKV 尺寸都是 BS * ML * ES # (或者多头情况下是 BS * HC * M
阅读全文
摘要:```py # 标记嵌入就是最普通的嵌入层 # 接受单词ID输出单词向量 # 直接转发给了`nn.Embedding` class TokenEmbedding(nn.Embedding): def __init__(self, vocab_size, embed_size=512): super(
阅读全文
摘要:```js import collapseWhitespace from './collapse-whitespace' import HTMLParser from './html-parser' import { isBlock, isVoid } from './utilities' // 单
阅读全文
摘要:```js import { repeat } from './utilities' var rules = {} // 段落 rules.paragraph = { filter: 'p', replacement: function (content) { // 前后加两个换行 return '
阅读全文
摘要:```js /** * Manages a collection of rules used to convert HTML to Markdown */ export default function Rules (options) { // 配置 this.options = options /
阅读全文
摘要:````js import COMMONMARK_RULES from './commonmark-rules' import Rules from './rules' import { extend, trimLeadingNewlines, trimTrailingNewlines } from
阅读全文