462 lines
23 KiB
Plaintext
462 lines
23 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Recurrent neural networks\n",
|
|
"\n",
|
|
"For di module wey we do before, we talk about beta semantic representation of text. Di architecture wey we dey use dey capture di overall meaning of words for sentence, but e no dey consider di **order** of di words, because di aggregation operation wey dey follow di embeddings dey remove dis information from di original text. Since dis models no fit represent word ordering, dem no fit solve more complex or ambiguous tasks like text generation or question answering.\n",
|
|
"\n",
|
|
"To fit capture di meaning of text sequence, we go use one neural network architecture wey dem dey call **recurrent neural network**, or RNN. When we dey use RNN, we go pass our sentence through di network one token at a time, and di network go produce some **state**, wey we go pass to di network again with di next token.\n",
|
|
"\n",
|
|
"\n",
|
|
"\n",
|
|
"If we get di input sequence of tokens $X_0,\\dots,X_n$, di RNN go create one sequence of neural network blocks, and e go train dis sequence end-to-end using backpropagation. Each network block dey take one pair $(X_i,S_i)$ as input, and e dey produce $S_{i+1}$ as result. Di final state $S_n$ or output $Y_n$ go enter linear classifier to produce di result. All di network blocks dey share di same weights, and dem dey train am end-to-end using one back propagation pass.\n",
|
|
"\n",
|
|
"> Di figure wey dey up dey show recurrent neural network for di unrolled form (for di left), and di more compact recurrent representation (for di right). E dey important to understand say all RNN Cells dey get di same **shareable weights**.\n",
|
|
"\n",
|
|
"Because state vectors $S_0,\\dots,S_n$ dey pass through di network, di RNN fit learn sequential dependencies between words. For example, if di word *not* show somewhere for di sequence, e fit learn how to negate some elements inside di state vector.\n",
|
|
"\n",
|
|
"Inside, each RNN cell dey get two weight matrices: $W_H$ and $W_I$, and bias $b$. For each RNN step, if we get input $X_i$ and input state $S_i$, di output state go be $S_{i+1} = f(W_H\\times S_i + W_I\\times X_i+b)$, where $f$ na activation function (many times $\\tanh$).\n",
|
|
"\n",
|
|
"> For problems like text generation (wey we go talk about for di next unit) or machine translation, we go also want make we get some output value for each RNN step. For dis case, another matrix $W_O$ go dey, and di output go be $Y_i=f(W_O\\times S_i+b_O)$.\n",
|
|
"\n",
|
|
"Make we see how recurrent neural networks fit help us classify our news dataset.\n",
|
|
"\n",
|
|
"> For di sandbox environment, we need to run di cell wey dey below to make sure say di required library don install, and data don prefetch. If you dey run am locally, you fit skip di cell wey dey below.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 1,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"import sys\n",
|
|
"!{sys.executable} -m pip install --quiet tensorflow_datasets==4.4.0\n",
|
|
"!cd ~ && wget -q -O - https://mslearntensorflowlp.blob.core.windows.net/data/tfds-ag-news.tgz | tar xz"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 2,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"import tensorflow as tf\n",
|
|
"from tensorflow import keras\n",
|
|
"import tensorflow_datasets as tfds\n",
|
|
"import numpy as np\n",
|
|
"\n",
|
|
"# We are going to be training pretty large models. In order not to face errors, we need\n",
|
|
"# to set tensorflow option to grow GPU memory allocation when required\n",
|
|
"physical_devices = tf.config.list_physical_devices('GPU') \n",
|
|
"if len(physical_devices)>0:\n",
|
|
" tf.config.experimental.set_memory_growth(physical_devices[0], True)\n",
|
|
"\n",
|
|
"ds_train, ds_test = tfds.load('ag_news_subset').values()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"nteract": {
|
|
"transient": {
|
|
"deleting": false
|
|
}
|
|
}
|
|
},
|
|
"source": [
|
|
"Wen you dey train big models, GPU memory fit turn wahala. We fit also need try different minibatch sizes, so dat di data go fit enter our GPU memory, but di training go still dey fast enough. If you dey run dis code for your own GPU machine, you fit try adjust di minibatch size to make training quick.\n",
|
|
"\n",
|
|
"> **Note**: Some kain versions of NVidia drivers no dey release memory after you don train di model finish. We dey run plenty examples for dis notebook, and e fit make memory finish for some setups, especially if you dey do your own experiments join for di same notebook. If you see some kind funny errors wen you wan start to train di model, e go make sense to restart di notebook kernel.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 3,
|
|
"metadata": {
|
|
"collapsed": true,
|
|
"jupyter": {
|
|
"outputs_hidden": false,
|
|
"source_hidden": false
|
|
},
|
|
"nteract": {
|
|
"transient": {
|
|
"deleting": false
|
|
}
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"batch_size = 16\n",
|
|
"embed_size = 64"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Simple RNN classifier\n",
|
|
"\n",
|
|
"For simple RNN, each recurrent unit na simple linear network wey dey take input vector and state vector, then e go produce new state vector. For Keras, we fit represent am wit `SimpleRNN` layer.\n",
|
|
"\n",
|
|
"Even though we fit pass one-hot encoded tokens go the RNN layer directly, e no good idea because their dimensionality dey too high. So, we go use embedding layer to reduce the dimensionality of word vectors, then we go add RNN layer, and finally `Dense` classifier.\n",
|
|
"\n",
|
|
"> **Note**: For cases wey dimensionality no too high, like when we dey use character-level tokenization, e fit make sense to pass one-hot encoded tokens directly go the RNN cell.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 4,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Model: \"sequential\"\n",
|
|
"_________________________________________________________________\n",
|
|
"Layer (type) Output Shape Param # \n",
|
|
"=================================================================\n",
|
|
"text_vectorization (TextVect (None, None) 0 \n",
|
|
"_________________________________________________________________\n",
|
|
"embedding (Embedding) (None, None, 64) 1280000 \n",
|
|
"_________________________________________________________________\n",
|
|
"simple_rnn (SimpleRNN) (None, 16) 1296 \n",
|
|
"_________________________________________________________________\n",
|
|
"dense (Dense) (None, 4) 68 \n",
|
|
"=================================================================\n",
|
|
"Total params: 1,281,364\n",
|
|
"Trainable params: 1,281,364\n",
|
|
"Non-trainable params: 0\n",
|
|
"_________________________________________________________________\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"vocab_size = 20000\n",
|
|
"\n",
|
|
"vectorizer = keras.layers.experimental.preprocessing.TextVectorization(\n",
|
|
" max_tokens=vocab_size,\n",
|
|
" input_shape=(1,))\n",
|
|
"\n",
|
|
"model = keras.models.Sequential([\n",
|
|
" vectorizer,\n",
|
|
" keras.layers.Embedding(vocab_size, embed_size),\n",
|
|
" keras.layers.SimpleRNN(16),\n",
|
|
" keras.layers.Dense(4,activation='softmax')\n",
|
|
"])\n",
|
|
"\n",
|
|
"model.summary()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"> **Note:** We dey use untrained embedding layer here for simplicity, but if we wan get better result, we fit use pretrained embedding layer wey dey use Word2Vec, as dem describe for di previous unit. E go make sense if you try adapt dis code to work with pretrained embeddings.\n",
|
|
"\n",
|
|
"Make we train our RNN now. RNNs no too easy to train, because once di RNN cells don unroll along di sequence length, di number of layers wey go dey involved for backpropagation go plenty. So, we need to choose smaller learning rate, and train di network on bigger dataset to fit get better result. Dis one fit take plenty time, so e better make we use GPU.\n",
|
|
"\n",
|
|
"To make di process quick, we go only train di RNN model on news titles, we no go include di description. You fit try train am with description to see whether di model go fit train.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 5,
|
|
"metadata": {
|
|
"scrolled": true
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Training vectorizer\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"def extract_title(x):\n",
|
|
" return x['title']\n",
|
|
"\n",
|
|
"def tupelize_title(x):\n",
|
|
" return (extract_title(x),x['label'])\n",
|
|
"\n",
|
|
"print('Training vectorizer')\n",
|
|
"vectorizer.adapt(ds_train.take(2000).map(extract_title))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 6,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"7500/7500 [==============================] - 82s 11ms/step - loss: 0.6629 - acc: 0.7623 - val_loss: 0.5559 - val_acc: 0.7995\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\n"
|
|
]
|
|
},
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"<tensorflow.python.keras.callbacks.History at 0x7f3e0030d350>"
|
|
]
|
|
},
|
|
"execution_count": 6,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"model.compile(loss='sparse_categorical_crossentropy',metrics=['acc'], optimizer='adam')\n",
|
|
"model.fit(ds_train.map(tupelize_title).batch(batch_size),validation_data=ds_test.map(tupelize_title).batch(batch_size))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"nteract": {
|
|
"transient": {
|
|
"deleting": false
|
|
}
|
|
}
|
|
},
|
|
"source": [
|
|
"> **Note** say accuracy fit dey low for here, because we dey train only on news titles.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Look di variable sequences again\n",
|
|
"\n",
|
|
"Make sure say you sabi say di `TextVectorization` layer go dey add pad tokens to sequences wey get different length inside one minibatch. But di wahala be say, di pad tokens go still join for di training, and e fit make di model no quick learn well.\n",
|
|
"\n",
|
|
"We get some ways wey we fit use reduce di padding. One way na to arrange di dataset by di sequence length and group all di sequences wey get di same size together. You fit do dis one with di `tf.data.experimental.bucket_by_sequence_length` function (check [documentation](https://www.tensorflow.org/api_docs/python/tf/data/experimental/bucket_by_sequence_length)).\n",
|
|
"\n",
|
|
"Another way na to use **masking**. For Keras, some layers dey support extra input wey go show which tokens we suppose use for training. To add masking for our model, we fit either add one `Masking` layer ([docs](https://keras.io/api/layers/core_layers/masking/)), or we fit set di `mask_zero=True` parameter for our `Embedding` layer.\n",
|
|
"\n",
|
|
"> **Note**: Dis training go take like 5 minutes to finish one epoch for di whole dataset. If you no get patience, you fit stop di training anytime. Another thing wey you fit do na to reduce di amount of data wey you dey use for training, by adding `.take(...)` after `ds_train` and `ds_test` datasets.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 7,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"7500/7500 [==============================] - 371s 49ms/step - loss: 0.5401 - acc: 0.8079 - val_loss: 0.3780 - val_acc: 0.8822\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\n"
|
|
]
|
|
},
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"<tensorflow.python.keras.callbacks.History at 0x7f3dec118850>"
|
|
]
|
|
},
|
|
"execution_count": 7,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"def extract_text(x):\n",
|
|
" return x['title']+' '+x['description']\n",
|
|
"\n",
|
|
"def tupelize(x):\n",
|
|
" return (extract_text(x),x['label'])\n",
|
|
"\n",
|
|
"model = keras.models.Sequential([\n",
|
|
" vectorizer,\n",
|
|
" keras.layers.Embedding(vocab_size,embed_size,mask_zero=True),\n",
|
|
" keras.layers.SimpleRNN(16),\n",
|
|
" keras.layers.Dense(4,activation='softmax')\n",
|
|
"])\n",
|
|
"\n",
|
|
"model.compile(loss='sparse_categorical_crossentropy',metrics=['acc'], optimizer='adam')\n",
|
|
"model.fit(ds_train.map(tupelize).batch(batch_size),validation_data=ds_test.map(tupelize).batch(batch_size))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Now wey we dey use masking, we fit train di model wit di whole dataset of titles and descriptions.\n",
|
|
"\n",
|
|
"> **Note**: You don notice say we don dey use vectorizer wey dem train on di news titles, and no be di whole body of di article? E fit make some of di tokens no show, so e go better make we re-train di vectorizer. But, e fit no really change plenty tin, so we go just stick to di pre-trained vectorizer wey we don already get to make tins simple.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## LSTM: Long short-term memory\n",
|
|
"\n",
|
|
"One big wahala wey RNNs get na **vanishing gradients**. RNNs fit long wella, and e fit hard for dem to carry gradients go back reach di first layer of di network wen backpropagation dey happen. Wen dis kind tin happen, di network no fit sabi di connection wey dey between far tokens. One way wey we fit take dodge dis wahala na to use **explicit state management** wit **gates**. Di two common architectures wey dey use gates na **long short-term memory** (LSTM) and **gated relay unit** (GRU). We go talk about LSTMs for here.\n",
|
|
"\n",
|
|
"\n",
|
|
"\n",
|
|
"LSTM network dey arrange like RNN, but e get two states wey dey pass from layer to layer: di real state $c$, and di hidden vector $h$. For each unit, di hidden vector $h_{t-1}$ go join wit input $x_t$, and dem go control wetin go happen to di state $c_t$ and output $h_{t}$ through **gates**. Each gate get sigmoid activation (output dey between $[0,1]$), wey we fit think say e be like bitwise mask wen e multiply di state vector. LSTMs get di following gates (from left to right for di picture wey dey up):\n",
|
|
"* **forget gate** wey dey decide which part of di vector $c_{t-1}$ we go forget, and which one we go allow pass.\n",
|
|
"* **input gate** wey dey decide how much info from di input vector and di hidden vector wey dey before go enter di state vector.\n",
|
|
"* **output gate** wey dey carry di new state vector and decide which part of am go dey use to produce di new hidden vector $h_t$.\n",
|
|
"\n",
|
|
"Di parts of di state $c$ fit be like flags wey we fit switch on and off. For example, wen we see di name *Alice* for di sequence, we go guess say e dey talk about woman, and we go raise di flag for di state wey dey show say we get female noun for di sentence. Wen we later see di words *and Tom*, we go raise di flag wey dey show say we get plural noun. So, as we dey manipulate di state, we fit dey track di grammar properties of di sentence.\n",
|
|
"\n",
|
|
"> **Note**: Dis na one better resource wey fit help you understand di inside of LSTMs: [Understanding LSTM Networks](https://colah.github.io/posts/2015-08-Understanding-LSTMs/) by Christopher Olah.\n",
|
|
"\n",
|
|
"Even though di inside structure of LSTM cell fit look hard, Keras dey hide di implementation inside di `LSTM` layer, so di only tin we need do for di example wey dey up na to change di recurrent layer:\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 8,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"15000/15000 [==============================] - 188s 13ms/step - loss: 0.5692 - acc: 0.7916 - val_loss: 0.3441 - val_acc: 0.8870\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\n"
|
|
]
|
|
},
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"<tensorflow.python.keras.callbacks.History at 0x7f3d6af5c350>"
|
|
]
|
|
},
|
|
"execution_count": 8,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"model = keras.models.Sequential([\n",
|
|
" vectorizer,\n",
|
|
" keras.layers.Embedding(vocab_size, embed_size),\n",
|
|
" keras.layers.LSTM(8),\n",
|
|
" keras.layers.Dense(4,activation='softmax')\n",
|
|
"])\n",
|
|
"\n",
|
|
"model.compile(loss='sparse_categorical_crossentropy',metrics=['acc'], optimizer='adam')\n",
|
|
"model.fit(ds_train.map(tupelize).batch(8),validation_data=ds_test.map(tupelize).batch(8))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"> **Note** say to train LSTMs dey slow well, and you fit no see plenty increase for accuracy for di beginning of di training. You go need continue di training for some time to get better accuracy.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Bidirectional and multilayer RNNs\n",
|
|
"\n",
|
|
"For di examples wey we don show so far, di recurrent networks dey work from di beginning of di sequence go reach di end. E dey feel normal to us because na di same way we dey read or listen to talk. But for situations wey need random access to di input sequence, e go make sense to run di recurrent computation for both directions. RNNs wey fit do computation for both directions na wetin dem dey call **bidirectional** RNNs, and you fit create am by wrapping di recurrent layer with one special `Bidirectional` layer.\n",
|
|
"\n",
|
|
"> **Note**: Di `Bidirectional` layer dey make two copies of di layer wey dey inside am, and e dey set di `go_backwards` property for one of di copies to `True`, so e go fit run for di opposite direction along di sequence.\n",
|
|
"\n",
|
|
"Recurrent networks, whether na unidirectional or bidirectional, dey capture patterns inside sequence, and dem dey store am for state vectors or return am as output. Just like convolutional networks, we fit build another recurrent layer after di first one to capture higher level patterns, wey dem build from di lower level patterns wey di first layer extract. Dis one na wetin dem dey call **multi-layer RNN**, wey get two or more recurrent networks, where di output of di previous layer dey pass go di next layer as input.\n",
|
|
"\n",
|
|
"\n",
|
|
"\n",
|
|
"*Picture from [dis wonderful post](https://towardsdatascience.com/from-a-lstm-cell-to-a-multilayer-lstm-network-with-pytorch-2899eb5696f3) by Fernando López.*\n",
|
|
"\n",
|
|
"Keras dey make am easy to construct dis kind networks, because you just need to add more recurrent layers to di model. For all di layers except di last one, we need to set `return_sequences=True` parameter, because we need di layer to return all di intermediate states, and no be just di final state of di recurrent computation.\n",
|
|
"\n",
|
|
"Make we build one two-layer bidirectional LSTM for our classification problem.\n",
|
|
"\n",
|
|
"> **Note** dis code go still take plenty time to finish, but e dey give us di highest accuracy we don see so far. So e fit worth am to wait and see di result.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 9,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"5044/7500 [===================>..........] - ETA: 2:33 - loss: 0.3709 - acc: 0.8706\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r5045/7500 [===================>..........] - ETA: 2:33 - loss: 0.3709 - acc: 0.8706"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"model = keras.models.Sequential([\n",
|
|
" vectorizer,\n",
|
|
" keras.layers.Embedding(vocab_size, 128, mask_zero=True),\n",
|
|
" keras.layers.Bidirectional(keras.layers.LSTM(64,return_sequences=True)),\n",
|
|
" keras.layers.Bidirectional(keras.layers.LSTM(64)), \n",
|
|
" keras.layers.Dense(4,activation='softmax')\n",
|
|
"])\n",
|
|
"\n",
|
|
"model.compile(loss='sparse_categorical_crossentropy',metrics=['acc'], optimizer='adam')\n",
|
|
"model.fit(ds_train.map(tupelize).batch(batch_size),\n",
|
|
" validation_data=ds_test.map(tupelize).batch(batch_size))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## RNNs for oda tasks\n",
|
|
"\n",
|
|
"So far, we don dey use RNNs to classify text wey dey for sequence. But dem fit do plenty oda work, like text generation and machine translation — we go look those tasks for di next unit.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"---\n\n<!-- CO-OP TRANSLATOR DISCLAIMER START -->\n**Disclaimer**: \nDis dokyument don use AI translet service [Co-op Translator](https://github.com/Azure/co-op-translator) do di translet. Even as we dey try make am correct, abeg make you sabi say AI translet fit get mistake or no dey accurate well. Di original dokyument wey dey for im native language na di one wey you go take as di correct source. For important informate, e good make you use professional human translet. We no go fit take blame for any misunderstanding or wrong interpretation wey fit happen because you use dis translet.\n<!-- CO-OP TRANSLATOR DISCLAIMER END -->\n"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernel_info": {
|
|
"name": "conda-env-py37_tensorflow-py"
|
|
},
|
|
"kernelspec": {
|
|
"display_name": "py37_tensorflow",
|
|
"language": "python",
|
|
"name": "conda-env-py37_tensorflow-py"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.7.9"
|
|
},
|
|
"nteract": {
|
|
"version": "nteract-front-end@1.0.0"
|
|
},
|
|
"coopTranslator": {
|
|
"original_hash": "81351e61f619b432ff51010a4f993194",
|
|
"translation_date": "2025-11-18T19:34:13+00:00",
|
|
"source_file": "lessons/5-NLP/16-RNN/RNNTF.ipynb",
|
|
"language_code": "pcm"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 4
|
|
} |