462 lines
23 KiB
Plaintext
462 lines
23 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Recurrent neural networks\n",
|
|
"\n",
|
|
"In the previous module, we explored rich semantic representations of text. The architecture we used captures the overall meaning of words in a sentence, but it doesn't account for the **order** of the words. This is because the aggregation operation applied after the embeddings removes the sequence information from the original text. Since these models cannot represent word order, they struggle with more complex or ambiguous tasks like text generation or question answering.\n",
|
|
"\n",
|
|
"To understand the meaning of a text sequence, we'll use a neural network architecture called **recurrent neural network**, or RNN. With an RNN, we process a sentence through the network one token at a time, and the network generates a **state**, which is then passed back into the network along with the next token.\n",
|
|
"\n",
|
|
"\n",
|
|
"\n",
|
|
"Given the input sequence of tokens $X_0,\\dots,X_n$, the RNN constructs a sequence of neural network blocks and trains this sequence end-to-end using backpropagation. Each network block takes a pair $(X_i,S_i)$ as input and produces $S_{i+1}$ as output. The final state $S_n$ or output $Y_n$ is passed into a linear classifier to generate the result. All network blocks share the same weights and are trained end-to-end in a single backpropagation pass.\n",
|
|
"\n",
|
|
"> The figure above illustrates a recurrent neural network in its unrolled form (on the left) and its more compact recurrent representation (on the right). It's important to note that all RNN cells share the same **weights**.\n",
|
|
"\n",
|
|
"Because the state vectors $S_0,\\dots,S_n$ are passed through the network, the RNN can learn sequential dependencies between words. For instance, if the word *not* appears somewhere in the sequence, the network can learn to negate certain elements within the state vector.\n",
|
|
"\n",
|
|
"Each RNN cell internally contains two weight matrices: $W_H$ and $W_I$, along with a bias $b$. At each RNN step, given the input $X_i$ and the input state $S_i$, the output state is calculated as $S_{i+1} = f(W_H\\times S_i + W_I\\times X_i+b)$, where $f$ is an activation function (commonly $\\tanh$).\n",
|
|
"\n",
|
|
"> For tasks like text generation (which we'll cover in the next unit) or machine translation, we also want to produce an output value at each RNN step. In such cases, an additional matrix $W_O$ is used, and the output is calculated as $Y_i=f(W_O\\times S_i+b_O)$.\n",
|
|
"\n",
|
|
"Now, let's explore how recurrent neural networks can help us classify our news dataset.\n",
|
|
"\n",
|
|
"> In the sandbox environment, we need to run the following cell to ensure the required library is installed and the data is preloaded. If you're working locally, you can skip this step.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 1,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"import sys\n",
|
|
"!{sys.executable} -m pip install --quiet tensorflow_datasets==4.4.0\n",
|
|
"!cd ~ && wget -q -O - https://mslearntensorflowlp.blob.core.windows.net/data/tfds-ag-news.tgz | tar xz"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 2,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"import tensorflow as tf\n",
|
|
"from tensorflow import keras\n",
|
|
"import tensorflow_datasets as tfds\n",
|
|
"import numpy as np\n",
|
|
"\n",
|
|
"# We are going to be training pretty large models. In order not to face errors, we need\n",
|
|
"# to set tensorflow option to grow GPU memory allocation when required\n",
|
|
"physical_devices = tf.config.list_physical_devices('GPU') \n",
|
|
"if len(physical_devices)>0:\n",
|
|
" tf.config.experimental.set_memory_growth(physical_devices[0], True)\n",
|
|
"\n",
|
|
"ds_train, ds_test = tfds.load('ag_news_subset').values()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"nteract": {
|
|
"transient": {
|
|
"deleting": false
|
|
}
|
|
}
|
|
},
|
|
"source": [
|
|
"When training large models, GPU memory allocation can become an issue. We might also need to experiment with different minibatch sizes to ensure the data fits into GPU memory while keeping the training process efficient. If you're running this code on your own GPU machine, you can try adjusting the minibatch size to accelerate training.\n",
|
|
"\n",
|
|
"> **Note**: Some versions of NVidia drivers are known to not release memory after training a model. Since we are running several examples in this notebook, this could lead to memory exhaustion in certain setups, especially if you're conducting your own experiments within the same notebook. If you encounter unusual errors when starting to train the model, consider restarting the notebook kernel.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 3,
|
|
"metadata": {
|
|
"collapsed": true,
|
|
"jupyter": {
|
|
"outputs_hidden": false,
|
|
"source_hidden": false
|
|
},
|
|
"nteract": {
|
|
"transient": {
|
|
"deleting": false
|
|
}
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"batch_size = 16\n",
|
|
"embed_size = 64"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Simple RNN classifier\n",
|
|
"\n",
|
|
"For a simple RNN, each recurrent unit is a straightforward linear network that takes an input vector and a state vector, then generates a new state vector. In Keras, this is implemented using the `SimpleRNN` layer.\n",
|
|
"\n",
|
|
"Although it's possible to feed one-hot encoded tokens directly into the RNN layer, this approach is not ideal due to their high dimensionality. Instead, we'll use an embedding layer to reduce the dimensionality of word vectors, followed by an RNN layer, and finally a `Dense` classifier.\n",
|
|
"\n",
|
|
"> **Note**: In scenarios where the dimensionality is not as high, such as when using character-level tokenization, it might be reasonable to input one-hot encoded tokens directly into the RNN cell.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 4,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Model: \"sequential\"\n",
|
|
"_________________________________________________________________\n",
|
|
"Layer (type) Output Shape Param # \n",
|
|
"=================================================================\n",
|
|
"text_vectorization (TextVect (None, None) 0 \n",
|
|
"_________________________________________________________________\n",
|
|
"embedding (Embedding) (None, None, 64) 1280000 \n",
|
|
"_________________________________________________________________\n",
|
|
"simple_rnn (SimpleRNN) (None, 16) 1296 \n",
|
|
"_________________________________________________________________\n",
|
|
"dense (Dense) (None, 4) 68 \n",
|
|
"=================================================================\n",
|
|
"Total params: 1,281,364\n",
|
|
"Trainable params: 1,281,364\n",
|
|
"Non-trainable params: 0\n",
|
|
"_________________________________________________________________\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"vocab_size = 20000\n",
|
|
"\n",
|
|
"vectorizer = keras.layers.experimental.preprocessing.TextVectorization(\n",
|
|
" max_tokens=vocab_size,\n",
|
|
" input_shape=(1,))\n",
|
|
"\n",
|
|
"model = keras.models.Sequential([\n",
|
|
" vectorizer,\n",
|
|
" keras.layers.Embedding(vocab_size, embed_size),\n",
|
|
" keras.layers.SimpleRNN(16),\n",
|
|
" keras.layers.Dense(4,activation='softmax')\n",
|
|
"])\n",
|
|
"\n",
|
|
"model.summary()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"> **Note:** We use an untrained embedding layer here for simplicity, but for better results, we can use a pretrained embedding layer using Word2Vec, as described in the previous unit. It would be a good exercise for you to modify this code to work with pretrained embeddings.\n",
|
|
"\n",
|
|
"Now let's train our RNN. RNNs are generally quite challenging to train because, once the RNN cells are unrolled along the sequence length, the number of layers involved in backpropagation becomes quite large. Therefore, we need to choose a smaller learning rate and train the network on a larger dataset to achieve good results. This process can take a significant amount of time, so using a GPU is recommended.\n",
|
|
"\n",
|
|
"To make the process faster, we will train the RNN model only on news titles, excluding the description. You can try training with the description included and see if you can get the model to train successfully.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 5,
|
|
"metadata": {
|
|
"scrolled": true
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Training vectorizer\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"def extract_title(x):\n",
|
|
" return x['title']\n",
|
|
"\n",
|
|
"def tupelize_title(x):\n",
|
|
" return (extract_title(x),x['label'])\n",
|
|
"\n",
|
|
"print('Training vectorizer')\n",
|
|
"vectorizer.adapt(ds_train.take(2000).map(extract_title))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 6,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"7500/7500 [==============================] - 82s 11ms/step - loss: 0.6629 - acc: 0.7623 - val_loss: 0.5559 - val_acc: 0.7995\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\n"
|
|
]
|
|
},
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"<tensorflow.python.keras.callbacks.History at 0x7f3e0030d350>"
|
|
]
|
|
},
|
|
"execution_count": 6,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"model.compile(loss='sparse_categorical_crossentropy',metrics=['acc'], optimizer='adam')\n",
|
|
"model.fit(ds_train.map(tupelize_title).batch(batch_size),validation_data=ds_test.map(tupelize_title).batch(batch_size))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"nteract": {
|
|
"transient": {
|
|
"deleting": false
|
|
}
|
|
}
|
|
},
|
|
"source": [
|
|
"> **Note** that accuracy is likely to be lower here, because we are training only on news titles.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Revisiting variable sequences \n",
|
|
"\n",
|
|
"Remember that the `TextVectorization` layer automatically pads sequences of varying lengths in a minibatch with padding tokens. However, these tokens also participate in training, which can make it harder for the model to converge.\n",
|
|
"\n",
|
|
"There are several strategies to reduce the amount of padding. One option is to reorder the dataset by sequence length, grouping all sequences of similar size together. This can be achieved using the `tf.data.experimental.bucket_by_sequence_length` function (see [documentation](https://www.tensorflow.org/api_docs/python/tf/data/experimental/bucket_by_sequence_length)).\n",
|
|
"\n",
|
|
"Another option is to use **masking**. In Keras, certain layers support additional input that indicates which tokens should be considered during training. To add masking to our model, we can either include a separate `Masking` layer ([docs](https://keras.io/api/layers/core_layers/masking/)) or set the `mask_zero=True` parameter in our `Embedding` layer.\n",
|
|
"\n",
|
|
"> **Note**: Training will take approximately 5 minutes per epoch for the entire dataset. Feel free to stop the training at any point if you lose patience. Alternatively, you can reduce the amount of data used for training by adding a `.take(...)` clause to the `ds_train` and `ds_test` datasets.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 7,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"7500/7500 [==============================] - 371s 49ms/step - loss: 0.5401 - acc: 0.8079 - val_loss: 0.3780 - val_acc: 0.8822\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\n"
|
|
]
|
|
},
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"<tensorflow.python.keras.callbacks.History at 0x7f3dec118850>"
|
|
]
|
|
},
|
|
"execution_count": 7,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"def extract_text(x):\n",
|
|
" return x['title']+' '+x['description']\n",
|
|
"\n",
|
|
"def tupelize(x):\n",
|
|
" return (extract_text(x),x['label'])\n",
|
|
"\n",
|
|
"model = keras.models.Sequential([\n",
|
|
" vectorizer,\n",
|
|
" keras.layers.Embedding(vocab_size,embed_size,mask_zero=True),\n",
|
|
" keras.layers.SimpleRNN(16),\n",
|
|
" keras.layers.Dense(4,activation='softmax')\n",
|
|
"])\n",
|
|
"\n",
|
|
"model.compile(loss='sparse_categorical_crossentropy',metrics=['acc'], optimizer='adam')\n",
|
|
"model.fit(ds_train.map(tupelize).batch(batch_size),validation_data=ds_test.map(tupelize).batch(batch_size))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Now that we're using masking, we can train the model on the entire dataset of titles and descriptions.\n",
|
|
"\n",
|
|
"> **Note**: Have you noticed that we have been using a vectorizer trained on the news titles, and not the full body of the article? This could potentially cause some tokens to be ignored, so it would be better to re-train the vectorizer. However, the impact might be minimal, so we will continue using the previously pre-trained vectorizer for the sake of simplicity.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## LSTM: Long short-term memory\n",
|
|
"\n",
|
|
"One of the main challenges with RNNs is **vanishing gradients**. RNNs can be quite deep, and during backpropagation, it may be difficult to propagate gradients all the way back to the first layer of the network. When this happens, the network struggles to learn relationships between distant tokens. A solution to this problem is to introduce **explicit state management** using **gates**. The two most common architectures that use gates are **long short-term memory** (LSTM) and **gated relay unit** (GRU). Here, we'll focus on LSTMs.\n",
|
|
"\n",
|
|
"\n",
|
|
"\n",
|
|
"An LSTM network is structured similarly to an RNN, but it passes two states from layer to layer: the actual state $c$, and the hidden vector $h$. At each unit, the hidden vector $h_{t-1}$ is combined with the input $x_t$, and together they control what happens to the state $c_t$ and the output $h_{t}$ through **gates**. Each gate uses sigmoid activation (output in the range $[0,1]$), which can be thought of as a bitwise mask when multiplied by the state vector. LSTMs include the following gates (from left to right in the image above):\n",
|
|
"* **Forget gate**: Determines which components of the vector $c_{t-1}$ should be discarded and which should be retained.\n",
|
|
"* **Input gate**: Decides how much information from the input vector and the previous hidden vector should be added to the state vector.\n",
|
|
"* **Output gate**: Takes the updated state vector and decides which of its components will be used to generate the new hidden vector $h_t$.\n",
|
|
"\n",
|
|
"The components of the state $c$ can be thought of as flags that can be toggled on or off. For instance, when we encounter the name *Alice* in a sequence, we might infer that it refers to a woman and activate a flag in the state indicating a female noun in the sentence. Later, when we come across the words *and Tom*, we might activate a flag indicating a plural noun. By manipulating the state, we can track grammatical properties of the sentence.\n",
|
|
"\n",
|
|
"> **Note**: Here's an excellent resource for understanding the inner workings of LSTMs: [Understanding LSTM Networks](https://colah.github.io/posts/2015-08-Understanding-LSTMs/) by Christopher Olah.\n",
|
|
"\n",
|
|
"Although the internal structure of an LSTM cell may seem complex, Keras abstracts this implementation within the `LSTM` layer. In the example above, all we need to do is replace the recurrent layer:\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 8,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"15000/15000 [==============================] - 188s 13ms/step - loss: 0.5692 - acc: 0.7916 - val_loss: 0.3441 - val_acc: 0.8870\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\n"
|
|
]
|
|
},
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"<tensorflow.python.keras.callbacks.History at 0x7f3d6af5c350>"
|
|
]
|
|
},
|
|
"execution_count": 8,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"model = keras.models.Sequential([\n",
|
|
" vectorizer,\n",
|
|
" keras.layers.Embedding(vocab_size, embed_size),\n",
|
|
" keras.layers.LSTM(8),\n",
|
|
" keras.layers.Dense(4,activation='softmax')\n",
|
|
"])\n",
|
|
"\n",
|
|
"model.compile(loss='sparse_categorical_crossentropy',metrics=['acc'], optimizer='adam')\n",
|
|
"model.fit(ds_train.map(tupelize).batch(8),validation_data=ds_test.map(tupelize).batch(8))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"> **Note** that training LSTMs is also quite slow, and you may not see much increase in accuracy in the beginning of training. You may need to continue training for some time to achieve good accuracy.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Bidirectional and multilayer RNNs\n",
|
|
"\n",
|
|
"In the examples we've covered so far, recurrent networks process sequences from start to finish. This approach feels intuitive because it mirrors the way we read or listen to speech. However, in situations where random access to the input sequence is required, it makes more sense to perform recurrent computations in both directions. RNNs that compute in both directions are called **bidirectional** RNNs, and they can be created by wrapping a recurrent layer with a special `Bidirectional` layer.\n",
|
|
"\n",
|
|
"> **Note**: The `Bidirectional` layer duplicates the layer it wraps and sets the `go_backwards` property of one copy to `True`, enabling it to process the sequence in reverse.\n",
|
|
"\n",
|
|
"Recurrent networks, whether unidirectional or bidirectional, identify patterns within a sequence and store them in state vectors or return them as output. Similar to convolutional networks, we can stack another recurrent layer on top of the first one to capture higher-level patterns derived from the lower-level patterns identified by the initial layer. This concept is known as a **multi-layer RNN**, which consists of two or more recurrent networks, where the output of one layer serves as the input for the next.\n",
|
|
"\n",
|
|
"\n",
|
|
"\n",
|
|
"*Image sourced from [this excellent post](https://towardsdatascience.com/from-a-lstm-cell-to-a-multilayer-lstm-network-with-pytorch-2899eb5696f3) by Fernando López.*\n",
|
|
"\n",
|
|
"Keras simplifies the process of building these networks. You just need to add additional recurrent layers to your model. For all layers except the final one, you must set the `return_sequences=True` parameter to ensure the layer outputs all intermediate states, rather than just the final state of the recurrent computation.\n",
|
|
"\n",
|
|
"Let's create a two-layer bidirectional LSTM for our classification task.\n",
|
|
"\n",
|
|
"> **Note**: This code takes a significant amount of time to execute, but it achieves the highest accuracy we've seen so far. It might be worth the wait to see the results.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 9,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"5044/7500 [===================>..........] - ETA: 2:33 - loss: 0.3709 - acc: 0.8706\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r5045/7500 [===================>..........] - ETA: 2:33 - loss: 0.3709 - acc: 0.8706"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"model = keras.models.Sequential([\n",
|
|
" vectorizer,\n",
|
|
" keras.layers.Embedding(vocab_size, 128, mask_zero=True),\n",
|
|
" keras.layers.Bidirectional(keras.layers.LSTM(64,return_sequences=True)),\n",
|
|
" keras.layers.Bidirectional(keras.layers.LSTM(64)), \n",
|
|
" keras.layers.Dense(4,activation='softmax')\n",
|
|
"])\n",
|
|
"\n",
|
|
"model.compile(loss='sparse_categorical_crossentropy',metrics=['acc'], optimizer='adam')\n",
|
|
"model.fit(ds_train.map(tupelize).batch(batch_size),\n",
|
|
" validation_data=ds_test.map(tupelize).batch(batch_size))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## RNNs for other tasks\n",
|
|
"\n",
|
|
"So far, we've concentrated on using RNNs to classify text sequences. However, they are capable of handling many other tasks, including text generation and machine translation — we'll explore these tasks in the next unit.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"\n---\n\n**Disclaimer**: \nThis document has been translated using the AI translation service [Co-op Translator](https://github.com/Azure/co-op-translator). While we strive for accuracy, please note that automated translations may contain errors or inaccuracies. The original document in its native language should be regarded as the authoritative source. For critical information, professional human translation is recommended. We are not responsible for any misunderstandings or misinterpretations resulting from the use of this translation.\n"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernel_info": {
|
|
"name": "conda-env-py37_tensorflow-py"
|
|
},
|
|
"kernelspec": {
|
|
"display_name": "py37_tensorflow",
|
|
"language": "python",
|
|
"name": "conda-env-py37_tensorflow-py"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.7.9"
|
|
},
|
|
"nteract": {
|
|
"version": "nteract-front-end@1.0.0"
|
|
},
|
|
"coopTranslator": {
|
|
"original_hash": "81351e61f619b432ff51010a4f993194",
|
|
"translation_date": "2025-08-31T18:31:59+00:00",
|
|
"source_file": "lessons/5-NLP/16-RNN/RNNTF.ipynb",
|
|
"language_code": "en"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 4
|
|
} |