473 lines
22 KiB
Plaintext
473 lines
22 KiB
Plaintext
{
|
||
"cells": [
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"Given the input sequence of tokens $X_0,\\dots,X_n$, RNN creates a sequence of neural network blocks and trains this sequence end-to-end using backpropagation. Each network block takes a pair $(X_i,S_i)$ as input and produces $S_{i+1}$ as output. The final state $S_n$ or output $X_n$ is passed into a linear classifier to generate the result. All network blocks share the same weights and are trained end-to-end in a single backpropagation pass.\n",
|
||
"\n",
|
||
"Because the state vectors $S_0,\\dots,S_n$ are passed through the network, it can learn sequential dependencies between words. For instance, when the word *not* appears somewhere in the sequence, the network can learn to negate certain elements within the state vector, effectively capturing negation.\n",
|
||
"\n",
|
||
"> Since the weights of all RNN blocks in the diagram are shared, the same diagram can be simplified into a single block (on the right) with a recurrent feedback loop that feeds the network's output state back into its input.\n",
|
||
"\n",
|
||
"Let’s explore how recurrent neural networks can assist in classifying our news dataset.\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 1,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Loading dataset...\n",
|
||
"Building vocab...\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"import torch\n",
|
||
"import torchtext\n",
|
||
"from torchnlp import *\n",
|
||
"train_dataset, test_dataset, classes, vocab = load_dataset()\n",
|
||
"vocab_size = len(vocab)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"## Simple RNN classifier\n",
|
||
"\n",
|
||
"For a simple RNN, each recurrent unit is a straightforward linear network that takes a concatenated input vector and state vector to produce a new state vector. PyTorch represents this unit using the `RNNCell` class, while a network of such cells is represented as an `RNN` layer.\n",
|
||
"\n",
|
||
"To create an RNN classifier, we will first use an embedding layer to reduce the dimensionality of the input vocabulary, followed by an RNN layer on top:\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 2,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"class RNNClassifier(torch.nn.Module):\n",
|
||
" def __init__(self, vocab_size, embed_dim, hidden_dim, num_class):\n",
|
||
" super().__init__()\n",
|
||
" self.hidden_dim = hidden_dim\n",
|
||
" self.embedding = torch.nn.Embedding(vocab_size, embed_dim)\n",
|
||
" self.rnn = torch.nn.RNN(embed_dim,hidden_dim,batch_first=True)\n",
|
||
" self.fc = torch.nn.Linear(hidden_dim, num_class)\n",
|
||
"\n",
|
||
" def forward(self, x):\n",
|
||
" batch_size = x.size(0)\n",
|
||
" x = self.embedding(x)\n",
|
||
" x,h = self.rnn(x)\n",
|
||
" return self.fc(x.mean(dim=1))"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"> **Note:** Here, we use an untrained embedding layer for simplicity, but for better results, we can use a pre-trained embedding layer with Word2Vec or GloVe embeddings, as explained in the previous unit. To deepen your understanding, you might want to modify this code to work with pre-trained embeddings.\n",
|
||
"\n",
|
||
"In our case, we will use a padded data loader, so each batch will consist of padded sequences of the same length. The RNN layer will process the sequence of embedding tensors and produce two outputs: \n",
|
||
"* $x$ is a sequence of RNN cell outputs at each step \n",
|
||
"* $h$ is the final hidden state for the last element of the sequence \n",
|
||
"\n",
|
||
"We then apply a fully connected linear classifier to determine the number of classes.\n",
|
||
"\n",
|
||
"> **Note:** Training RNNs can be quite challenging because, once the RNN cells are unrolled along the sequence length, the number of layers involved in backpropagation becomes very large. Therefore, we need to choose a small learning rate and train the network on a larger dataset to achieve good results. This process can take a significant amount of time, so using a GPU is recommended.\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 3,
|
||
"metadata": {
|
||
"scrolled": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"3200: acc=0.3090625\n",
|
||
"6400: acc=0.38921875\n",
|
||
"9600: acc=0.4590625\n",
|
||
"12800: acc=0.511953125\n",
|
||
"16000: acc=0.5506875\n",
|
||
"19200: acc=0.57921875\n",
|
||
"22400: acc=0.6070089285714285\n",
|
||
"25600: acc=0.6304296875\n",
|
||
"28800: acc=0.6484027777777778\n",
|
||
"32000: acc=0.66509375\n",
|
||
"35200: acc=0.6790056818181818\n",
|
||
"38400: acc=0.6929166666666666\n",
|
||
"41600: acc=0.7035817307692308\n",
|
||
"44800: acc=0.7137276785714286\n",
|
||
"48000: acc=0.72225\n",
|
||
"51200: acc=0.73001953125\n",
|
||
"54400: acc=0.7372794117647059\n",
|
||
"57600: acc=0.7436631944444444\n",
|
||
"60800: acc=0.7503947368421052\n",
|
||
"64000: acc=0.75634375\n",
|
||
"67200: acc=0.7615773809523809\n",
|
||
"70400: acc=0.7662642045454545\n",
|
||
"73600: acc=0.7708423913043478\n",
|
||
"76800: acc=0.7751822916666666\n",
|
||
"80000: acc=0.7790625\n",
|
||
"83200: acc=0.7825\n",
|
||
"86400: acc=0.7858564814814815\n",
|
||
"89600: acc=0.7890513392857142\n",
|
||
"92800: acc=0.7920474137931034\n",
|
||
"96000: acc=0.7952708333333334\n",
|
||
"99200: acc=0.7982258064516129\n",
|
||
"102400: acc=0.80099609375\n",
|
||
"105600: acc=0.8037594696969697\n",
|
||
"108800: acc=0.8060569852941176\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"train_loader = torch.utils.data.DataLoader(train_dataset, batch_size=16, collate_fn=padify, shuffle=True)\n",
|
||
"net = RNNClassifier(vocab_size,64,32,len(classes)).to(device)\n",
|
||
"train_epoch(net,train_loader, lr=0.001)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"## Long Short Term Memory (LSTM)\n",
|
||
"\n",
|
||
"One of the main issues with classical RNNs is the so-called **vanishing gradients** problem. Since RNNs are trained end-to-end in a single backpropagation pass, it becomes difficult to propagate errors to the earlier layers of the network. As a result, the network struggles to learn relationships between distant tokens. One way to address this issue is by introducing **explicit state management** through the use of **gates**. Two of the most well-known architectures that use this approach are **Long Short Term Memory** (LSTM) and **Gated Relay Unit** (GRU).\n",
|
||
"\n",
|
||
"\n",
|
||
"\n",
|
||
"The LSTM network is structured similarly to an RNN, but it passes two states from layer to layer: the actual state $c$ and the hidden vector $h$. At each unit, the hidden vector $h_i$ is concatenated with the input $x_i$, and together they control what happens to the state $c$ through **gates**. Each gate is a neural network with a sigmoid activation function (output in the range $[0,1]$), which can be thought of as a bitwise mask when multiplied by the state vector. The following gates are present (from left to right in the image above):\n",
|
||
"* **Forget gate**: Takes the hidden vector and determines which components of the vector $c$ should be forgotten and which should be retained.\n",
|
||
"* **Input gate**: Extracts some information from the input and hidden vector and inserts it into the state.\n",
|
||
"* **Output gate**: Transforms the state using a linear layer with $\\tanh$ activation, then selects certain components using the hidden vector $h_i$ to produce the new state $c_{i+1}$.\n",
|
||
"\n",
|
||
"The components of the state $c$ can be thought of as flags that can be turned on or off. For example, when encountering the name *Alice* in a sequence, we might assume it refers to a female character and set a flag in the state to indicate the presence of a female noun in the sentence. Later, when encountering the phrase *and Tom*, we might set a flag to indicate the presence of a plural noun. By manipulating the state in this way, we can theoretically keep track of the grammatical properties of different parts of a sentence.\n",
|
||
"\n",
|
||
"> **Note**: A fantastic resource for understanding the inner workings of LSTMs is the article [Understanding LSTM Networks](https://colah.github.io/posts/2015-08-Understanding-LSTMs/) by Christopher Olah.\n",
|
||
"\n",
|
||
"Although the internal structure of an LSTM cell may seem complex, PyTorch abstracts this implementation within the `LSTMCell` class and provides the `LSTM` object to represent the entire LSTM layer. As a result, implementing an LSTM classifier is quite similar to the simple RNN we discussed earlier:\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 4,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"class LSTMClassifier(torch.nn.Module):\n",
|
||
" def __init__(self, vocab_size, embed_dim, hidden_dim, num_class):\n",
|
||
" super().__init__()\n",
|
||
" self.hidden_dim = hidden_dim\n",
|
||
" self.embedding = torch.nn.Embedding(vocab_size, embed_dim)\n",
|
||
" self.embedding.weight.data = torch.randn_like(self.embedding.weight.data)-0.5\n",
|
||
" self.rnn = torch.nn.LSTM(embed_dim,hidden_dim,batch_first=True)\n",
|
||
" self.fc = torch.nn.Linear(hidden_dim, num_class)\n",
|
||
"\n",
|
||
" def forward(self, x):\n",
|
||
" batch_size = x.size(0)\n",
|
||
" x = self.embedding(x)\n",
|
||
" x,(h,c) = self.rnn(x)\n",
|
||
" return self.fc(h[-1])"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": []
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 5,
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"3200: acc=0.259375\n",
|
||
"6400: acc=0.25859375\n",
|
||
"9600: acc=0.26177083333333334\n",
|
||
"12800: acc=0.2784375\n",
|
||
"16000: acc=0.313\n",
|
||
"19200: acc=0.3528645833333333\n",
|
||
"22400: acc=0.3965625\n",
|
||
"25600: acc=0.4385546875\n",
|
||
"28800: acc=0.4752777777777778\n",
|
||
"32000: acc=0.505375\n",
|
||
"35200: acc=0.5326704545454546\n",
|
||
"38400: acc=0.5557552083333334\n",
|
||
"41600: acc=0.5760817307692307\n",
|
||
"44800: acc=0.5954910714285714\n",
|
||
"48000: acc=0.6118333333333333\n",
|
||
"51200: acc=0.62681640625\n",
|
||
"54400: acc=0.6404779411764706\n",
|
||
"57600: acc=0.6520138888888889\n",
|
||
"60800: acc=0.662828947368421\n",
|
||
"64000: acc=0.673546875\n",
|
||
"67200: acc=0.6831547619047619\n",
|
||
"70400: acc=0.6917897727272727\n",
|
||
"73600: acc=0.6997146739130434\n",
|
||
"76800: acc=0.707109375\n",
|
||
"80000: acc=0.714075\n",
|
||
"83200: acc=0.7209134615384616\n",
|
||
"86400: acc=0.727037037037037\n",
|
||
"89600: acc=0.7326674107142858\n",
|
||
"92800: acc=0.7379633620689655\n",
|
||
"96000: acc=0.7433645833333333\n",
|
||
"99200: acc=0.7479032258064516\n",
|
||
"102400: acc=0.752119140625\n",
|
||
"105600: acc=0.7562405303030303\n",
|
||
"108800: acc=0.76015625\n",
|
||
"112000: acc=0.7641339285714286\n",
|
||
"115200: acc=0.7677777777777778\n",
|
||
"118400: acc=0.7711233108108108\n"
|
||
]
|
||
},
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"(0.03487814127604167, 0.7728)"
|
||
]
|
||
},
|
||
"execution_count": 5,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"net = LSTMClassifier(vocab_size,64,32,len(classes)).to(device)\n",
|
||
"train_epoch(net,train_loader, lr=0.001)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"## Packed sequences\n",
|
||
"\n",
|
||
"In our example, we had to pad all sequences in the minibatch with zero vectors. While this leads to some memory being wasted, the bigger issue with RNNs is that additional RNN cells are created for the padded input items. These cells participate in training but do not carry any meaningful input information. It would be much better to train the RNN only up to the actual sequence length.\n",
|
||
"\n",
|
||
"To address this, PyTorch introduces a special format for storing padded sequences. Suppose we have a padded input minibatch that looks like this:\n",
|
||
"```\n",
|
||
"[[1,2,3,4,5],\n",
|
||
" [6,7,8,0,0],\n",
|
||
" [9,0,0,0,0]]\n",
|
||
"```\n",
|
||
"Here, 0 represents the padded values, and the actual length vector of the input sequences is `[5,3,1]`.\n",
|
||
"\n",
|
||
"To train an RNN effectively with padded sequences, we want to start training the first group of RNN cells with the largest minibatch (`[1,6,9]`), then stop processing the third sequence, and continue training with smaller minibatches (`[2,7]`, `[3,8]`), and so on. A packed sequence is thus represented as a single vector—in this case, `[1,6,9,2,7,3,8,4,5]`—along with a length vector (`[5,3,1]`), from which the original padded minibatch can easily be reconstructed.\n",
|
||
"\n",
|
||
"To create a packed sequence, we can use the `torch.nn.utils.rnn.pack_padded_sequence` function. All recurrent layers, including RNN, LSTM, and GRU, support packed sequences as input and produce packed output, which can be converted back using `torch.nn.utils.rnn.pad_packed_sequence`.\n",
|
||
"\n",
|
||
"To generate a packed sequence, we need to pass the length vector to the network. Therefore, we require a different function to prepare minibatches:\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 6,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"def pad_length(b):\n",
|
||
" # build vectorized sequence\n",
|
||
" v = [encode(x[1]) for x in b]\n",
|
||
" # compute max length of a sequence in this minibatch and length sequence itself\n",
|
||
" len_seq = list(map(len,v))\n",
|
||
" l = max(len_seq)\n",
|
||
" return ( # tuple of three tensors - labels, padded features, length sequence\n",
|
||
" torch.LongTensor([t[0]-1 for t in b]),\n",
|
||
" torch.stack([torch.nn.functional.pad(torch.tensor(t),(0,l-len(t)),mode='constant',value=0) for t in v]),\n",
|
||
" torch.tensor(len_seq)\n",
|
||
" )\n",
|
||
"\n",
|
||
"train_loader_len = torch.utils.data.DataLoader(train_dataset, batch_size=16, collate_fn=pad_length, shuffle=True)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"The actual network will be very similar to the `LSTMClassifier` mentioned above, but the `forward` pass will take both the padded minibatch and the vector of sequence lengths as input. After calculating the embedding, we create a packed sequence, pass it through the LSTM layer, and then unpack the result.\n",
|
||
"\n",
|
||
"> **Note**: We don't actually use the unpacked result `x`, as the output from the hidden layers is used in subsequent computations. Therefore, the unpacking step can be removed entirely from this code. The reason it's included here is to make it easier for you to modify the code if you need to use the network's output in further computations.\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 7,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"class LSTMPackClassifier(torch.nn.Module):\n",
|
||
" def __init__(self, vocab_size, embed_dim, hidden_dim, num_class):\n",
|
||
" super().__init__()\n",
|
||
" self.hidden_dim = hidden_dim\n",
|
||
" self.embedding = torch.nn.Embedding(vocab_size, embed_dim)\n",
|
||
" self.embedding.weight.data = torch.randn_like(self.embedding.weight.data)-0.5\n",
|
||
" self.rnn = torch.nn.LSTM(embed_dim,hidden_dim,batch_first=True)\n",
|
||
" self.fc = torch.nn.Linear(hidden_dim, num_class)\n",
|
||
"\n",
|
||
" def forward(self, x, lengths):\n",
|
||
" batch_size = x.size(0)\n",
|
||
" x = self.embedding(x)\n",
|
||
" pad_x = torch.nn.utils.rnn.pack_padded_sequence(x,lengths,batch_first=True,enforce_sorted=False)\n",
|
||
" pad_x,(h,c) = self.rnn(pad_x)\n",
|
||
" x, _ = torch.nn.utils.rnn.pad_packed_sequence(pad_x,batch_first=True)\n",
|
||
" return self.fc(h[-1])"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": []
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 8,
|
||
"metadata": {
|
||
"scrolled": true
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"3200: acc=0.285625\n",
|
||
"6400: acc=0.33359375\n",
|
||
"9600: acc=0.3876041666666667\n",
|
||
"12800: acc=0.44078125\n",
|
||
"16000: acc=0.4825\n",
|
||
"19200: acc=0.5235416666666667\n",
|
||
"22400: acc=0.5559821428571429\n",
|
||
"25600: acc=0.58609375\n",
|
||
"28800: acc=0.6116666666666667\n",
|
||
"32000: acc=0.63340625\n",
|
||
"35200: acc=0.6525284090909091\n",
|
||
"38400: acc=0.668515625\n",
|
||
"41600: acc=0.6822596153846154\n",
|
||
"44800: acc=0.6948214285714286\n",
|
||
"48000: acc=0.7052708333333333\n",
|
||
"51200: acc=0.71521484375\n",
|
||
"54400: acc=0.7239889705882353\n",
|
||
"57600: acc=0.7315277777777778\n",
|
||
"60800: acc=0.7388486842105263\n",
|
||
"64000: acc=0.74571875\n",
|
||
"67200: acc=0.7518303571428572\n",
|
||
"70400: acc=0.7576988636363636\n",
|
||
"73600: acc=0.7628940217391305\n",
|
||
"76800: acc=0.7681510416666667\n",
|
||
"80000: acc=0.7728125\n",
|
||
"83200: acc=0.7772235576923077\n",
|
||
"86400: acc=0.7815393518518519\n",
|
||
"89600: acc=0.7857700892857142\n",
|
||
"92800: acc=0.7895043103448276\n",
|
||
"96000: acc=0.7930520833333333\n",
|
||
"99200: acc=0.7959072580645161\n",
|
||
"102400: acc=0.798994140625\n",
|
||
"105600: acc=0.802064393939394\n",
|
||
"108800: acc=0.8051378676470589\n",
|
||
"112000: acc=0.8077857142857143\n",
|
||
"115200: acc=0.8104600694444445\n",
|
||
"118400: acc=0.8128293918918919\n"
|
||
]
|
||
},
|
||
{
|
||
"data": {
|
||
"text/plain": [
|
||
"(0.029785829671223958, 0.8138166666666666)"
|
||
]
|
||
},
|
||
"execution_count": 8,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"net = LSTMPackClassifier(vocab_size,64,32,len(classes)).to(device)\n",
|
||
"train_epoch_emb(net,train_loader_len, lr=0.001,use_pack_sequence=True)\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"> **Note:** You may have noticed the parameter `use_pack_sequence` that we pass to the training function. Currently, the `pack_padded_sequence` function requires the length sequence tensor to be on the CPU device, and thus the training function needs to avoid moving the length sequence data to the GPU when training. You can look into the implementation of the `train_emb` function in the [`torchnlp.py`](../../../../../lessons/5-NLP/16-RNN/torchnlp.py) file.\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"## Bidirectional and Multilayer RNNs\n",
|
||
"\n",
|
||
"In our examples, all recurrent networks operated in a single direction, from the start of a sequence to its end. This seems natural because it mirrors how we read or listen to speech. However, in many practical scenarios, we have random access to the input sequence, so it might make sense to perform recurrent computation in both directions. Such networks are called **bidirectional** RNNs, and they can be created by passing the `bidirectional=True` parameter to the RNN/LSTM/GRU constructor.\n",
|
||
"\n",
|
||
"When working with a bidirectional network, we need two hidden state vectors—one for each direction. PyTorch encodes these vectors as a single vector with double the size, which is quite convenient. This is because you typically pass the resulting hidden state to a fully connected linear layer, and you only need to account for the increased size when creating the layer.\n",
|
||
"\n",
|
||
"A recurrent network, whether one-directional or bidirectional, captures certain patterns within a sequence and can store them in the state vector or pass them to the output. Similar to convolutional networks, we can stack another recurrent layer on top of the first one to capture higher-level patterns, built from the low-level patterns extracted by the first layer. This brings us to the concept of a **multi-layer RNN**, which consists of two or more recurrent networks, where the output of one layer is passed as input to the next layer.\n",
|
||
"\n",
|
||
"\n",
|
||
"\n",
|
||
"*Image from [this excellent post](https://towardsdatascience.com/from-a-lstm-cell-to-a-multilayer-lstm-network-with-pytorch-2899eb5696f3) by Fernando López*\n",
|
||
"\n",
|
||
"PyTorch makes constructing such networks straightforward. You simply need to pass the `num_layers` parameter to the RNN/LSTM/GRU constructor to automatically build multiple layers of recurrence. This also means that the size of the hidden/state vector will increase proportionally, and you’ll need to account for this when processing the output of the recurrent layers.\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"## RNNs for other tasks\n",
|
||
"\n",
|
||
"In this unit, we have seen that RNNs can be used for sequence classification, but they are capable of handling many other tasks, such as text generation, machine translation, and more. We will explore these tasks in the next unit.\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"\n---\n\n**Disclaimer**: \nThis document has been translated using the AI translation service [Co-op Translator](https://github.com/Azure/co-op-translator). While we strive for accuracy, please note that automated translations may contain errors or inaccuracies. The original document in its native language should be regarded as the authoritative source. For critical information, professional human translation is recommended. We are not responsible for any misunderstandings or misinterpretations resulting from the use of this translation.\n"
|
||
]
|
||
}
|
||
],
|
||
"metadata": {
|
||
"interpreter": {
|
||
"hash": "16af2a8bbb083ea23e5e41c7f5787656b2ce26968575d8763f2c4b17f9cd711f"
|
||
},
|
||
"kernelspec": {
|
||
"display_name": "Python 3.8.12 ('py38')",
|
||
"language": "python",
|
||
"name": "python3"
|
||
},
|
||
"language_info": {
|
||
"codemirror_mode": {
|
||
"name": "ipython",
|
||
"version": 3
|
||
},
|
||
"file_extension": ".py",
|
||
"mimetype": "text/x-python",
|
||
"name": "python",
|
||
"nbconvert_exporter": "python",
|
||
"pygments_lexer": "ipython3",
|
||
"version": "3.8.12"
|
||
},
|
||
"coopTranslator": {
|
||
"original_hash": "522ee52ae3d5ae933e283286254e9a55",
|
||
"translation_date": "2025-08-31T18:32:40+00:00",
|
||
"source_file": "lessons/5-NLP/16-RNN/RNNPyTorch.ipynb",
|
||
"language_code": "en"
|
||
}
|
||
},
|
||
"nbformat": 4,
|
||
"nbformat_minor": 2
|
||
} |