479 lines
22 KiB
Plaintext
479 lines
22 KiB
Plaintext
{
|
||
"cells": [
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"# 循环神经网络\n",
|
||
"\n",
|
||
"在之前的模块中，我们使用了丰富的文本语义表示，并在嵌入的基础上使用了一个简单的线性分类器。这种架构能够捕捉句子中词语的整体意义，但它没有考虑词语的**顺序**，因为嵌入上的聚合操作会将原始文本中的顺序信息移除。由于这些模型无法建模词语的顺序，它们无法解决更复杂或更模糊的任务，例如文本生成或问答。\n",
|
||
"\n",
|
||
"为了捕捉文本序列的意义，我们需要使用另一种神经网络架构，称为**循环神经网络**（Recurrent Neural Network，RNN）。在 RNN 中，我们将句子逐个符号传递给网络，网络会生成某种**状态**，然后我们将该状态与下一个符号一起再次传递给网络。\n",
|
||
"\n",
|
||
"给定输入序列的标记 $X_0,\\dots,X_n$，RNN 会创建一个神经网络块的序列，并通过反向传播对该序列进行端到端训练。每个网络块以 $(X_i,S_i)$ 作为输入，并生成 $S_{i+1}$ 作为结果。最终状态 $S_n$ 或输出 $X_n$ 会传递给线性分类器以生成结果。所有网络块共享相同的权重，并通过一次反向传播训练端到端。\n",
|
||
"\n",
|
||
"由于状态向量 $S_0,\\dots,S_n$ 会通过网络传递，RNN 能够学习词语之间的顺序依赖关系。例如，当单词 *not* 出现在序列中的某处时，它可以学习在状态向量中否定某些元素，从而实现否定。\n",
|
||
"\n",
|
||
"> 由于图片中所有 RNN 块的权重是共享的，因此同一张图片可以表示为一个带有循环反馈回路的单块（右侧），该回路将网络的输出状态传回输入。\n",
|
||
"\n",
|
||
"接下来我们看看循环神经网络如何帮助我们对新闻数据集进行分类。\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 1,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Loading dataset...\n",
|
||
"Building vocab...\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"import torch\n",
|
||
"import torchtext\n",
|
||
"from torchnlp import *\n",
|
||
"train_dataset, test_dataset, classes, vocab = load_dataset()\n",
|
||
"vocab_size = len(vocab)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"## 简单 RNN 分类器\n",
|
||
"\n",
|
||
"对于简单的 RNN，每个循环单元都是一个简单的线性网络，它接收拼接后的输入向量和状态向量，并生成一个新的状态向量。PyTorch 使用 `RNNCell` 类表示这个单元，并将由这些单元组成的网络表示为 `RNN` 层。\n",
|
||
"\n",
|
||
"为了定义一个 RNN 分类器，我们首先会应用一个嵌入层来降低输入词汇的维度，然后在其上添加一个 RNN 层：\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 2,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"class RNNClassifier(torch.nn.Module):\n",
|
||
" def __init__(self, vocab_size, embed_dim, hidden_dim, num_class):\n",
|
||
" super().__init__()\n",
|
||
" self.hidden_dim = hidden_dim\n",
|
||
" self.embedding = torch.nn.Embedding(vocab_size, embed_dim)\n",
|
||
" self.rnn = torch.nn.RNN(embed_dim,hidden_dim,batch_first=True)\n",
|
||
" self.fc = torch.nn.Linear(hidden_dim, num_class)\n",
|
||
"\n",
|
||
" def forward(self, x):\n",
|
||
" batch_size = x.size(0)\n",
|
||
" x = self.embedding(x)\n",
|
||
" x,h = self.rnn(x)\n",
|
||
" return self.fc(x.mean(dim=1))"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"> **注意：** 为了简化操作，我们这里使用了未训练的嵌入层，但如果想要更好的效果，可以使用预训练的嵌入层，例如 Word2Vec 或 GloVe 嵌入，这在前一单元中已有描述。为了更好地理解，您可能需要将此代码调整为使用预训练的嵌入。\n",
|
||
"\n",
|
||
"在我们的例子中，我们将使用填充数据加载器，因此每个批次将包含若干长度相同的填充序列。RNN 层将接收嵌入张量的序列，并生成两个输出：\n",
|
||
"* $x$ 是每一步 RNN 单元输出的序列\n",
|
||
"* $h$ 是序列最后一个元素的最终隐藏状态\n",
|
||
"\n",
|
||
"然后我们应用一个全连接的线性分类器来获得类别数量。\n",
|
||
"\n",
|
||
"> **注意：** RNN 的训练相当困难，因为一旦 RNN 单元沿着序列长度展开，反向传播中涉及的层数会非常多。因此，我们需要选择较小的学习率，并在更大的数据集上训练网络以获得良好的结果。这可能需要相当长的时间，因此建议使用 GPU。\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 3,
|
||
"metadata": {
|
||
"scrolled": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"3200: acc=0.3090625\n",
|
||
"6400: acc=0.38921875\n",
|
||
"9600: acc=0.4590625\n",
|
||
"12800: acc=0.511953125\n",
|
||
"16000: acc=0.5506875\n",
|
||
"19200: acc=0.57921875\n",
|
||
"22400: acc=0.6070089285714285\n",
|
||
"25600: acc=0.6304296875\n",
|
||
"28800: acc=0.6484027777777778\n",
|
||
"32000: acc=0.66509375\n",
|
||
"35200: acc=0.6790056818181818\n",
|
||
"38400: acc=0.6929166666666666\n",
|
||
"41600: acc=0.7035817307692308\n",
|
||
"44800: acc=0.7137276785714286\n",
|
||
"48000: acc=0.72225\n",
|
||
"51200: acc=0.73001953125\n",
|
||
"54400: acc=0.7372794117647059\n",
|
||
"57600: acc=0.7436631944444444\n",
|
||
"60800: acc=0.7503947368421052\n",
|
||
"64000: acc=0.75634375\n",
|
||
"67200: acc=0.7615773809523809\n",
|
||
"70400: acc=0.7662642045454545\n",
|
||
"73600: acc=0.7708423913043478\n",
|
||
"76800: acc=0.7751822916666666\n",
|
||
"80000: acc=0.7790625\n",
|
||
"83200: acc=0.7825\n",
|
||
"86400: acc=0.7858564814814815\n",
|
||
"89600: acc=0.7890513392857142\n",
|
||
"92800: acc=0.7920474137931034\n",
|
||
"96000: acc=0.7952708333333334\n",
|
||
"99200: acc=0.7982258064516129\n",
|
||
"102400: acc=0.80099609375\n",
|
||
"105600: acc=0.8037594696969697\n",
|
||
"108800: acc=0.8060569852941176\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"train_loader = torch.utils.data.DataLoader(train_dataset, batch_size=16, collate_fn=padify, shuffle=True)\n",
|
||
"net = RNNClassifier(vocab_size,64,32,len(classes)).to(device)\n",
|
||
"train_epoch(net,train_loader, lr=0.001)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"## 长短期记忆网络 (LSTM)\n",
|
||
"\n",
|
||
"经典 RNN 的主要问题之一是所谓的 **梯度消失** 问题。由于 RNN 是通过一次反向传播端到端训练的，因此在将误差传播到网络的第一层时会遇到困难，从而无法学习远距离的词元之间的关系。解决这一问题的一种方法是通过使用所谓的 **门控机制** 引入 **显式状态管理**。最著名的两种架构是：**长短期记忆网络** (LSTM) 和 **门控循环单元** (GRU)。\n",
|
||
"\n",
|
||
"\n",
|
||
"\n",
|
||
"LSTM 网络的组织方式与 RNN 类似，但有两个状态会从层传递到层：实际状态 $c$ 和隐藏向量 $h$。在每个单元中，隐藏向量 $h_i$ 与输入 $x_i$ 连接在一起，并通过 **门控机制** 控制状态 $c$ 的变化。每个门都是一个带有 sigmoid 激活函数的神经网络（输出范围为 $[0,1]$），可以通过与状态向量相乘来理解为逐位掩码。以下是这些门（如上图从左到右）：\n",
|
||
"* **遗忘门** 接收隐藏向量并决定状态向量 $c$ 的哪些部分需要遗忘，哪些需要保留。\n",
|
||
"* **输入门** 从输入和隐藏向量中提取一些信息，并将其插入状态中。\n",
|
||
"* **输出门** 通过某种带有 $\\tanh$ 激活的线性层转换状态，然后使用隐藏向量 $h_i$ 选择状态的某些部分以生成新的状态 $c_{i+1}$。\n",
|
||
"\n",
|
||
"状态 $c$ 的各个组成部分可以被视为一些可以打开或关闭的标志。例如，当我们在序列中遇到名字 *Alice* 时，我们可能会假设它指代女性角色，并在状态中设置一个标志，表示句子中有女性名词。当我们进一步遇到短语 *and Tom* 时，我们会设置一个标志，表示句子中有复数名词。因此，通过操控状态，我们可以追踪句子部分的语法属性。\n",
|
||
"\n",
|
||
"> **Note**: 理解 LSTM 内部结构的一个优秀资源是 Christopher Olah 的这篇文章 [Understanding LSTM Networks](https://colah.github.io/posts/2015-08-Understanding-LSTMs/)。\n",
|
||
"\n",
|
||
"虽然 LSTM 单元的内部结构看起来很复杂，但 PyTorch 将其实现隐藏在 `LSTMCell` 类中，并提供了 `LSTM` 对象来表示整个 LSTM 层。因此，LSTM 分类器的实现与我们之前看到的简单 RNN 非常相似：\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 4,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"class LSTMClassifier(torch.nn.Module):\n",
|
||
" def __init__(self, vocab_size, embed_dim, hidden_dim, num_class):\n",
|
||
" super().__init__()\n",
|
||
" self.hidden_dim = hidden_dim\n",
|
||
" self.embedding = torch.nn.Embedding(vocab_size, embed_dim)\n",
|
||
" self.embedding.weight.data = torch.randn_like(self.embedding.weight.data)-0.5\n",
|
||
" self.rnn = torch.nn.LSTM(embed_dim,hidden_dim,batch_first=True)\n",
|
||
" self.fc = torch.nn.Linear(hidden_dim, num_class)\n",
|
||
"\n",
|
||
" def forward(self, x):\n",
|
||
" batch_size = x.size(0)\n",
|
||
" x = self.embedding(x)\n",
|
||
" x,(h,c) = self.rnn(x)\n",
|
||
" return self.fc(h[-1])"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": []
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 5,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"3200: acc=0.259375\n",
|
||
"6400: acc=0.25859375\n",
|
||
"9600: acc=0.26177083333333334\n",
|
||
"12800: acc=0.2784375\n",
|
||
"16000: acc=0.313\n",
|
||
"19200: acc=0.3528645833333333\n",
|
||
"22400: acc=0.3965625\n",
|
||
"25600: acc=0.4385546875\n",
|
||
"28800: acc=0.4752777777777778\n",
|
||
"32000: acc=0.505375\n",
|
||
"35200: acc=0.5326704545454546\n",
|
||
"38400: acc=0.5557552083333334\n",
|
||
"41600: acc=0.5760817307692307\n",
|
||
"44800: acc=0.5954910714285714\n",
|
||
"48000: acc=0.6118333333333333\n",
|
||
"51200: acc=0.62681640625\n",
|
||
"54400: acc=0.6404779411764706\n",
|
||
"57600: acc=0.6520138888888889\n",
|
||
"60800: acc=0.662828947368421\n",
|
||
"64000: acc=0.673546875\n",
|
||
"67200: acc=0.6831547619047619\n",
|
||
"70400: acc=0.6917897727272727\n",
|
||
"73600: acc=0.6997146739130434\n",
|
||
"76800: acc=0.707109375\n",
|
||
"80000: acc=0.714075\n",
|
||
"83200: acc=0.7209134615384616\n",
|
||
"86400: acc=0.727037037037037\n",
|
||
"89600: acc=0.7326674107142858\n",
|
||
"92800: acc=0.7379633620689655\n",
|
||
"96000: acc=0.7433645833333333\n",
|
||
"99200: acc=0.7479032258064516\n",
|
||
"102400: acc=0.752119140625\n",
|
||
"105600: acc=0.7562405303030303\n",
|
||
"108800: acc=0.76015625\n",
|
||
"112000: acc=0.7641339285714286\n",
|
||
"115200: acc=0.7677777777777778\n",
|
||
"118400: acc=0.7711233108108108\n"
|
||
]
|
||
},
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"(0.03487814127604167, 0.7728)"
|
||
]
|
||
},
|
||
"execution_count": 5,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"net = LSTMClassifier(vocab_size,64,32,len(classes)).to(device)\n",
|
||
"train_epoch(net,train_loader, lr=0.001)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"## 填充序列\n",
|
||
"\n",
|
||
"在我们的示例中，我们需要用零向量填充小批量中的所有序列。虽然这会导致一些内存浪费，但对于 RNN 来说，更重要的是为填充的输入项创建额外的 RNN 单元，这些单元参与训练，但并不包含任何重要的输入信息。更好的方法是仅根据实际序列长度来训练 RNN。\n",
|
||
"\n",
|
||
"为此，PyTorch 引入了一种特殊的填充序列存储格式。假设我们有一个填充的小批量输入，如下所示：\n",
|
||
"```\n",
|
||
"[[1,2,3,4,5],\n",
|
||
" [6,7,8,0,0],\n",
|
||
" [9,0,0,0,0]]\n",
|
||
"```\n",
|
||
"这里的 0 表示填充值，输入序列的实际长度向量为 `[5,3,1]`。\n",
|
||
"\n",
|
||
"为了有效地用填充序列训练 RNN，我们希望首先用较大的小批量（`[1,6,9]`）开始训练第一组 RNN 单元，然后结束第三个序列的处理，并继续用较小的小批量（`[2,7]`，`[3,8]`）进行训练，依此类推。因此，填充序列被表示为一个向量——在我们的例子中是 `[1,6,9,2,7,3,8,4,5]`，以及长度向量（`[5,3,1]`），通过这些信息我们可以轻松地重建原始填充的小批量。\n",
|
||
"\n",
|
||
"为了生成填充序列，我们可以使用 `torch.nn.utils.rnn.pack_padded_sequence` 函数。所有循环层，包括 RNN、LSTM 和 GRU，都支持将填充序列作为输入，并生成填充序列的输出，这些输出可以通过 `torch.nn.utils.rnn.pad_packed_sequence` 解码。\n",
|
||
"\n",
|
||
"为了能够生成填充序列，我们需要将长度向量传递给网络，因此我们需要一个不同的函数来准备小批量：\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 6,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"def pad_length(b):\n",
|
||
" # build vectorized sequence\n",
|
||
" v = [encode(x[1]) for x in b]\n",
|
||
" # compute max length of a sequence in this minibatch and length sequence itself\n",
|
||
" len_seq = list(map(len,v))\n",
|
||
" l = max(len_seq)\n",
|
||
" return ( # tuple of three tensors - labels, padded features, length sequence\n",
|
||
" torch.LongTensor([t[0]-1 for t in b]),\n",
|
||
" torch.stack([torch.nn.functional.pad(torch.tensor(t),(0,l-len(t)),mode='constant',value=0) for t in v]),\n",
|
||
" torch.tensor(len_seq)\n",
|
||
" )\n",
|
||
"\n",
|
||
"train_loader_len = torch.utils.data.DataLoader(train_dataset, batch_size=16, collate_fn=pad_length, shuffle=True)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"实际的网络与上面的 `LSTMClassifier` 非常相似，但 `forward` 传递会接收填充后的小批量数据以及序列长度的向量。在计算嵌入后，我们计算打包序列，将其传递给 LSTM 层，然后将结果解包回来。\n",
|
||
"\n",
|
||
"> **注意**: 实际上我们并不使用解包后的结果 `x`，因为我们在后续计算中使用的是隐藏层的输出。因此，我们可以完全从代码中移除解包操作。之所以将其保留在这里，是为了方便您在需要使用网络输出进行进一步计算时能够轻松修改代码。\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 7,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"class LSTMPackClassifier(torch.nn.Module):\n",
|
||
" def __init__(self, vocab_size, embed_dim, hidden_dim, num_class):\n",
|
||
" super().__init__()\n",
|
||
" self.hidden_dim = hidden_dim\n",
|
||
" self.embedding = torch.nn.Embedding(vocab_size, embed_dim)\n",
|
||
" self.embedding.weight.data = torch.randn_like(self.embedding.weight.data)-0.5\n",
|
||
" self.rnn = torch.nn.LSTM(embed_dim,hidden_dim,batch_first=True)\n",
|
||
" self.fc = torch.nn.Linear(hidden_dim, num_class)\n",
|
||
"\n",
|
||
" def forward(self, x, lengths):\n",
|
||
" batch_size = x.size(0)\n",
|
||
" x = self.embedding(x)\n",
|
||
" pad_x = torch.nn.utils.rnn.pack_padded_sequence(x,lengths,batch_first=True,enforce_sorted=False)\n",
|
||
" pad_x,(h,c) = self.rnn(pad_x)\n",
|
||
" x, _ = torch.nn.utils.rnn.pad_packed_sequence(pad_x,batch_first=True)\n",
|
||
" return self.fc(h[-1])"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": []
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 8,
|
||
"metadata": {
|
||
"scrolled": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"3200: acc=0.285625\n",
|
||
"6400: acc=0.33359375\n",
|
||
"9600: acc=0.3876041666666667\n",
|
||
"12800: acc=0.44078125\n",
|
||
"16000: acc=0.4825\n",
|
||
"19200: acc=0.5235416666666667\n",
|
||
"22400: acc=0.5559821428571429\n",
|
||
"25600: acc=0.58609375\n",
|
||
"28800: acc=0.6116666666666667\n",
|
||
"32000: acc=0.63340625\n",
|
||
"35200: acc=0.6525284090909091\n",
|
||
"38400: acc=0.668515625\n",
|
||
"41600: acc=0.6822596153846154\n",
|
||
"44800: acc=0.6948214285714286\n",
|
||
"48000: acc=0.7052708333333333\n",
|
||
"51200: acc=0.71521484375\n",
|
||
"54400: acc=0.7239889705882353\n",
|
||
"57600: acc=0.7315277777777778\n",
|
||
"60800: acc=0.7388486842105263\n",
|
||
"64000: acc=0.74571875\n",
|
||
"67200: acc=0.7518303571428572\n",
|
||
"70400: acc=0.7576988636363636\n",
|
||
"73600: acc=0.7628940217391305\n",
|
||
"76800: acc=0.7681510416666667\n",
|
||
"80000: acc=0.7728125\n",
|
||
"83200: acc=0.7772235576923077\n",
|
||
"86400: acc=0.7815393518518519\n",
|
||
"89600: acc=0.7857700892857142\n",
|
||
"92800: acc=0.7895043103448276\n",
|
||
"96000: acc=0.7930520833333333\n",
|
||
"99200: acc=0.7959072580645161\n",
|
||
"102400: acc=0.798994140625\n",
|
||
"105600: acc=0.802064393939394\n",
|
||
"108800: acc=0.8051378676470589\n",
|
||
"112000: acc=0.8077857142857143\n",
|
||
"115200: acc=0.8104600694444445\n",
|
||
"118400: acc=0.8128293918918919\n"
|
||
]
|
||
},
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"(0.029785829671223958, 0.8138166666666666)"
|
||
]
|
||
},
|
||
"execution_count": 8,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"net = LSTMPackClassifier(vocab_size,64,32,len(classes)).to(device)\n",
|
||
"train_epoch_emb(net,train_loader_len, lr=0.001,use_pack_sequence=True)\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"> **注意：** 您可能已经注意到我们传递给训练函数的参数 `use_pack_sequence`。目前，`pack_padded_sequence` 函数要求长度序列张量位于 CPU 设备上，因此训练函数需要避免在训练时将长度序列数据移动到 GPU。您可以查看 [`torchnlp.py`](../../../../../lessons/5-NLP/16-RNN/torchnlp.py) 文件中 `train_emb` 函数的实现。\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"## 双向和多层 RNN\n",
|
||
"\n",
|
||
"在我们的示例中，所有的循环网络都是单向运行的，从序列的开始到结束。这看起来很自然，因为它类似于我们阅读和听语音的方式。然而，在许多实际情况下，我们可以随机访问输入序列，因此在两个方向上运行循环计算可能更有意义。这类网络称为**双向** RNN，可以通过向 RNN/LSTM/GRU 构造函数传递 `bidirectional=True` 参数来创建。\n",
|
||
"\n",
|
||
"在处理双向网络时，我们需要两个隐藏状态向量，每个方向一个。PyTorch 将这些向量编码为一个大小加倍的向量，这非常方便，因为通常会将生成的隐藏状态传递给全连接线性层，只需在创建该层时考虑到大小的增加即可。\n",
|
||
"\n",
|
||
"无论是单向还是双向的循环网络，都能捕捉序列中的某些模式，并将其存储到状态向量中或传递到输出中。与卷积网络类似，我们可以在第一层循环网络之上构建另一层循环网络，以捕捉更高层次的模式，这些模式是由第一层提取的低层次模式构建而成的。这引出了**多层 RNN**的概念，它由两个或更多的循环网络组成，其中上一层的输出作为下一层的输入。\n",
|
||
"\n",
|
||
"\n",
|
||
"\n",
|
||
"*图片来源于 [Fernando López 的这篇精彩文章](https://towardsdatascience.com/from-a-lstm-cell-to-a-multilayer-lstm-network-with-pytorch-2899eb5696f3)*\n",
|
||
"\n",
|
||
"PyTorch 使构建此类网络变得非常简单，因为只需向 RNN/LSTM/GRU 构造函数传递 `num_layers` 参数，就可以自动构建多层循环网络。这也意味着隐藏状态向量的大小会按比例增加，在处理循环层的输出时需要考虑这一点。\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"## RNN用于其他任务\n",
|
||
"\n",
|
||
"在本单元中，我们已经了解到RNN可以用于序列分类，但实际上，它还可以处理更多任务，例如文本生成、机器翻译等。我们将在下一单元中讨论这些任务。\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"\n---\n\n**免责声明**： \n本文档使用AI翻译服务[Co-op Translator](https://github.com/Azure/co-op-translator)进行翻译。虽然我们努力确保翻译的准确性，但请注意，自动翻译可能包含错误或不准确之处。应以原始语言的文档作为权威来源。对于重要信息，建议使用专业人工翻译。我们不对因使用此翻译而产生的任何误解或误读承担责任。\n"
|
||
]
|
||
}
|
||
],
|
||
"metadata": {
|
||
"interpreter": {
|
||
"hash": "16af2a8bbb083ea23e5e41c7f5787656b2ce26968575d8763f2c4b17f9cd711f"
|
||
},
|
||
"kernelspec": {
|
||
"display_name": "Python 3.8.12 ('py38')",
|
||
"language": "python",
|
||
"name": "python3"
|
||
},
|
||
"language_info": {
|
||
"codemirror_mode": {
|
||
"name": "ipython",
|
||
"version": 3
|
||
},
|
||
"file_extension": ".py",
|
||
"mimetype": "text/x-python",
|
||
"name": "python",
|
||
"nbconvert_exporter": "python",
|
||
"pygments_lexer": "ipython3",
|
||
"version": "3.8.12"
|
||
},
|
||
"coopTranslator": {
|
||
"original_hash": "522ee52ae3d5ae933e283286254e9a55",
|
||
"translation_date": "2025-08-31T10:49:43+00:00",
|
||
"source_file": "lessons/5-NLP/16-RNN/RNNPyTorch.ipynb",
|
||
"language_code": "zh"
|
||
}
|
||
},
|
||
"nbformat": 4,
|
||
"nbformat_minor": 2
|
||
} |