576 lines
17 KiB
Plaintext
576 lines
17 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "NXTSugt6ieXh"
|
|
},
|
|
"source": [
|
|
"## Usposabljanje modela CBoW\n",
|
|
"\n",
|
|
"Ta zvezek je del [Učnega načrta za začetnike v AI](http://aka.ms/ai-beginners)\n",
|
|
"\n",
|
|
"V tem primeru bomo preučili usposabljanje jezikovnega modela CBoW za pridobitev lastnega vektorskega prostora Word2Vec. Kot vir besedila bomo uporabili podatkovno zbirko AG News.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"source": [
|
|
"import torch\n",
|
|
"import torchtext\n",
|
|
"import os\n",
|
|
"import collections\n",
|
|
"import builtins\n",
|
|
"import random\n",
|
|
"import numpy as np"
|
|
],
|
|
"metadata": {
|
|
"id": "q-UiiJUKaxHj"
|
|
},
|
|
"execution_count": null,
|
|
"outputs": []
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"source": [
|
|
"device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")"
|
|
],
|
|
"metadata": {
|
|
"id": "TFbR8CZaTZ1q"
|
|
},
|
|
"execution_count": null,
|
|
"outputs": []
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"source": [
|
|
"Najprej naložimo naš nabor podatkov in definiramo tokenizator ter besedišče. Nastavili bomo `vocab_size` na 5000, da nekoliko omejimo izračune.\n"
|
|
],
|
|
"metadata": {
|
|
"id": "HIwC7lI5T-ov"
|
|
}
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"source": [
|
|
"def load_dataset(ngrams = 1, min_freq = 1, vocab_size = 5000 , lines_cnt = 500):\n",
|
|
" tokenizer = torchtext.data.utils.get_tokenizer('basic_english')\n",
|
|
" print(\"Loading dataset...\")\n",
|
|
" test_dataset, train_dataset = torchtext.datasets.AG_NEWS(root='./data')\n",
|
|
" train_dataset = list(train_dataset)\n",
|
|
" test_dataset = list(test_dataset)\n",
|
|
" classes = ['World', 'Sports', 'Business', 'Sci/Tech']\n",
|
|
" print('Building vocab...')\n",
|
|
" counter = collections.Counter()\n",
|
|
" for i, (_, line) in enumerate(train_dataset):\n",
|
|
" counter.update(torchtext.data.utils.ngrams_iterator(tokenizer(line),ngrams=ngrams))\n",
|
|
" if i == lines_cnt:\n",
|
|
" break\n",
|
|
" vocab = torchtext.vocab.Vocab(collections.Counter(dict(counter.most_common(vocab_size))), min_freq=min_freq)\n",
|
|
" return train_dataset, test_dataset, classes, vocab, tokenizer"
|
|
],
|
|
"metadata": {
|
|
"id": "wdZuygtgiuLG"
|
|
},
|
|
"execution_count": null,
|
|
"outputs": []
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"source": [
|
|
"train_dataset, test_dataset, _, vocab, tokenizer = load_dataset()"
|
|
],
|
|
"metadata": {
|
|
"colab": {
|
|
"base_uri": "https://localhost:8080/"
|
|
},
|
|
"id": "4d1nU1gsivGu",
|
|
"outputId": "949fe272-ae0e-49f5-c373-6703458b3a74"
|
|
},
|
|
"execution_count": null,
|
|
"outputs": [
|
|
{
|
|
"output_type": "stream",
|
|
"name": "stdout",
|
|
"text": [
|
|
"Loading dataset...\n",
|
|
"Building vocab...\n"
|
|
]
|
|
}
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"source": [
|
|
"def encode(x, vocabulary, tokenizer = tokenizer):\n",
|
|
" return [vocabulary[s] for s in tokenizer(x)]"
|
|
],
|
|
"metadata": {
|
|
"id": "1XDYNhG8ToFV"
|
|
},
|
|
"execution_count": null,
|
|
"outputs": []
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "LIlQk6_PaHVY"
|
|
},
|
|
"source": [
|
|
"## Model CBoW\n",
|
|
"\n",
|
|
"CBoW se nauči napovedovati besedo na podlagi $2N$ sosednjih besed. Na primer, ko je $N=1$, dobimo naslednje pare iz stavka *I like to train networks*: (like,I), (I, like), (to, like), (like,to), (train,to), (to, train), (networks, train), (train,networks). Tukaj je prva beseda sosednja beseda, ki jo uporabimo kot vhod, druga beseda pa je tista, ki jo napovedujemo.\n",
|
|
"\n",
|
|
"Za izdelavo mreže, ki napoveduje naslednjo besedo, moramo sosednjo besedo podati kot vhod in dobiti številko besede kot izhod. Arhitektura mreže CBoW je naslednja:\n",
|
|
"\n",
|
|
"* Vhodna beseda se prenese skozi plast vgrajevanja. Ta plast vgrajevanja bo naša vgrajevanje Word2Vec, zato jo bomo definirali ločeno kot spremenljivko `embedder`. V tem primeru bomo uporabili velikost vgrajevanja = 30, čeprav bi morda želeli eksperimentirati z večjimi dimenzijami (pravi Word2Vec ima 300).\n",
|
|
"* Vektor vgrajevanja se nato prenese skozi linearno plast, ki bo napovedala izhodno besedo. Zato ima `vocab_size` nevronov.\n",
|
|
"\n",
|
|
"Za izhod, če uporabimo `CrossEntropyLoss` kot funkcijo izgube, moramo podati le številke besed kot pričakovane rezultate, brez enovektorskega kodiranja.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"source": [
|
|
"vocab_size = len(vocab)\n",
|
|
"\n",
|
|
"embedder = torch.nn.Embedding(num_embeddings = vocab_size, embedding_dim = 30)\n",
|
|
"model = torch.nn.Sequential(\n",
|
|
" embedder,\n",
|
|
" torch.nn.Linear(in_features = 30, out_features = vocab_size),\n",
|
|
")\n",
|
|
"\n",
|
|
"print(model)"
|
|
],
|
|
"metadata": {
|
|
"colab": {
|
|
"base_uri": "https://localhost:8080/"
|
|
},
|
|
"id": "akKTcKQKkfl2",
|
|
"outputId": "da687e3e-a8ec-4c1a-e456-ab8cd6ac7dad"
|
|
},
|
|
"execution_count": null,
|
|
"outputs": [
|
|
{
|
|
"output_type": "stream",
|
|
"name": "stdout",
|
|
"text": [
|
|
"Sequential(\n",
|
|
" (0): Embedding(5002, 30)\n",
|
|
" (1): Linear(in_features=30, out_features=5002, bias=True)\n",
|
|
")\n"
|
|
]
|
|
}
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "Nud6jgGPaHVa"
|
|
},
|
|
"source": [
|
|
"## Priprava podatkov za učenje\n",
|
|
"\n",
|
|
"Zdaj bomo napisali glavno funkcijo, ki bo iz besedila izračunala CBoW pare besed. Ta funkcija nam bo omogočila določiti velikost okna in bo vrnila niz parov - vhodno in izhodno besedo. Upoštevajte, da se ta funkcija lahko uporablja tako na besedah kot na vektorjih/tensorjih - kar nam bo omogočilo kodiranje besedila, preden ga posredujemo funkciji `to_cbow`.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"colab": {
|
|
"base_uri": "https://localhost:8080/"
|
|
},
|
|
"id": "x-dsXygOieXn",
|
|
"outputId": "c2218280-e540-40ba-9546-efe48d0d714f"
|
|
},
|
|
"outputs": [
|
|
{
|
|
"output_type": "stream",
|
|
"name": "stdout",
|
|
"text": [
|
|
"[['like', 'I'], ['to', 'I'], ['I', 'like'], ['to', 'like'], ['train', 'like'], ['I', 'to'], ['like', 'to'], ['train', 'to'], ['networks', 'to'], ['like', 'train'], ['to', 'train'], ['networks', 'train'], ['to', 'networks'], ['train', 'networks']]\n",
|
|
"[[232, 172], [5, 172], [172, 232], [5, 232], [0, 232], [172, 5], [232, 5], [0, 5], [1202, 5], [232, 0], [5, 0], [1202, 0], [5, 1202], [0, 1202]]\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"def to_cbow(sent,window_size=2):\n",
|
|
" res = []\n",
|
|
" for i,x in enumerate(sent):\n",
|
|
" for j in range(max(0,i-window_size),min(i+window_size+1,len(sent))):\n",
|
|
" if i!=j:\n",
|
|
" res.append([sent[j],x])\n",
|
|
" return res\n",
|
|
"\n",
|
|
"print(to_cbow(['I','like','to','train','networks']))\n",
|
|
"print(to_cbow(encode('I like to train networks', vocab)))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "XVaaDLjaaHVb"
|
|
},
|
|
"source": [
|
|
"Pripravimo učni nabor podatkov. Pregledali bomo vse novice, poklicali `to_cbow`, da dobimo seznam parov besed, in te pare dodali v `X` in `Y`. Zaradi prihranka časa bomo upoštevali le prvih 10 tisoč novic - omejitev lahko enostavno odstranite, če imate več časa za čakanje in želite dobiti boljše vdelave :)\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "54b-Gd9TieXo"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"X = []\n",
|
|
"Y = []\n",
|
|
"for i, x in zip(range(10000), train_dataset):\n",
|
|
" for w1, w2 in to_cbow(encode(x[1], vocab), window_size = 5):\n",
|
|
" X.append(w1)\n",
|
|
" Y.append(w2)\n",
|
|
"\n",
|
|
"X = torch.tensor(X)\n",
|
|
"Y = torch.tensor(Y)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"source": [
|
|
"Podatke bomo prav tako pretvorili v en nabor podatkov in ustvarili nalagalnik podatkov:\n"
|
|
],
|
|
"metadata": {
|
|
"id": "cwWy0PzXWhN5"
|
|
}
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"source": [
|
|
"class SimpleIterableDataset(torch.utils.data.IterableDataset):\n",
|
|
" def __init__(self, X, Y):\n",
|
|
" super(SimpleIterableDataset).__init__()\n",
|
|
" self.data = []\n",
|
|
" for i in range(len(X)):\n",
|
|
" self.data.append( (Y[i], X[i]) )\n",
|
|
" random.shuffle(self.data)\n",
|
|
"\n",
|
|
" def __iter__(self):\n",
|
|
" return iter(self.data)"
|
|
],
|
|
"metadata": {
|
|
"id": "mfoAcGPFZU8p"
|
|
},
|
|
"execution_count": null,
|
|
"outputs": []
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "e4NQ_-5waHVc"
|
|
},
|
|
"source": [
|
|
"Prav tako bomo te podatke pretvorili v en nabor podatkov in ustvarili nalagalnik podatkov:\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "AbLUcojlieXo"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"ds = SimpleIterableDataset(X, Y)\n",
|
|
"dl = torch.utils.data.DataLoader(ds, batch_size = 256)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "pKQr7sXeaHVc"
|
|
},
|
|
"source": [
|
|
"Zdaj začnimo s pravim treningom. Uporabili bomo optimizator `SGD` z dokaj visoko hitrostjo učenja. Lahko poskusite tudi z drugimi optimizatorji, kot je `Adam`. Za začetek bomo trenirali 10 epoh - to celico lahko ponovno zaženete, če želite še nižjo izgubo.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"source": [
|
|
"def train_epoch(net, dataloader, lr = 0.01, optimizer = None, loss_fn = torch.nn.CrossEntropyLoss(), epochs = None, report_freq = 1):\n",
|
|
" optimizer = optimizer or torch.optim.Adam(net.parameters(), lr = lr)\n",
|
|
" loss_fn = loss_fn.to(device)\n",
|
|
" net.train()\n",
|
|
"\n",
|
|
" for i in range(epochs):\n",
|
|
" total_loss, j = 0, 0, \n",
|
|
" for labels, features in dataloader:\n",
|
|
" optimizer.zero_grad()\n",
|
|
" features, labels = features.to(device), labels.to(device)\n",
|
|
" out = net(features)\n",
|
|
" loss = loss_fn(out, labels)\n",
|
|
" loss.backward()\n",
|
|
" optimizer.step()\n",
|
|
" total_loss += loss\n",
|
|
" j += 1\n",
|
|
" if i % report_freq == 0:\n",
|
|
" print(f\"Epoch: {i+1}: loss={total_loss.item()/j}\")\n",
|
|
"\n",
|
|
" return total_loss.item()/j"
|
|
],
|
|
"metadata": {
|
|
"id": "HeeCYKr_KF1w"
|
|
},
|
|
"execution_count": null,
|
|
"outputs": []
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"source": [
|
|
"train_epoch(net = model, dataloader = dl, optimizer = torch.optim.SGD(model.parameters(), lr = 0.1), loss_fn = torch.nn.CrossEntropyLoss(), epochs = 10)"
|
|
],
|
|
"metadata": {
|
|
"colab": {
|
|
"base_uri": "https://localhost:8080/"
|
|
},
|
|
"id": "KVgwGtDHgDlT",
|
|
"outputId": "2447833f-f0e3-4566-c33d-addbfe2f451d"
|
|
},
|
|
"execution_count": null,
|
|
"outputs": [
|
|
{
|
|
"output_type": "stream",
|
|
"name": "stdout",
|
|
"text": [
|
|
"Epoch: 1: loss=5.664632366860172\n",
|
|
"Epoch: 2: loss=5.632101973960962\n",
|
|
"Epoch: 3: loss=5.610399051405015\n",
|
|
"Epoch: 4: loss=5.594621561080262\n",
|
|
"Epoch: 5: loss=5.582538017415446\n",
|
|
"Epoch: 6: loss=5.572900234519603\n",
|
|
"Epoch: 7: loss=5.564951676341915\n",
|
|
"Epoch: 8: loss=5.558288112064614\n",
|
|
"Epoch: 9: loss=5.552576955031129\n",
|
|
"Epoch: 10: loss=5.547634165194347\n"
|
|
]
|
|
},
|
|
{
|
|
"output_type": "execute_result",
|
|
"data": {
|
|
"text/plain": [
|
|
"5.547634165194347"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"execution_count": 16
|
|
}
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "W8u2qXZmaHVd"
|
|
},
|
|
"source": [
|
|
"## Preizkušanje Word2Vec\n",
|
|
"\n",
|
|
"Za uporabo Word2Vec izvlecimo vektorje, ki ustrezajo vsem besedam v našem besedišču:\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "r8TatcXjkU_t"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"vectors = torch.stack([embedder(torch.tensor(vocab[s])) for s in vocab.itos], 0)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "3OcX21UOaHVd"
|
|
},
|
|
"source": [
|
|
"Poglejmo na primer, kako je beseda **Pariz** kodirana v vektor:\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"colab": {
|
|
"base_uri": "https://localhost:8080/"
|
|
},
|
|
"id": "bz6tAeLzieXp",
|
|
"outputId": "5b20850e-4342-45e9-f840-cfac2b4d61d8"
|
|
},
|
|
"outputs": [
|
|
{
|
|
"output_type": "stream",
|
|
"name": "stdout",
|
|
"text": [
|
|
"tensor([-0.0915, 2.1224, -0.0281, -0.6819, 1.1219, 0.6458, -1.3704, -1.3314,\n",
|
|
" -1.1437, 0.4496, 0.2301, -0.3515, -0.8485, 1.0481, 0.4386, -0.8949,\n",
|
|
" 0.5644, 1.0939, -2.5096, 3.2949, -0.2601, -0.8640, 0.1421, -0.0804,\n",
|
|
" -0.5083, -1.0560, 0.9753, -0.5949, -1.6046, 0.5774],\n",
|
|
" grad_fn=<EmbeddingBackward>)\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"paris_vec = embedder(torch.tensor(vocab['paris']))\n",
|
|
"print(paris_vec)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "pHTJlaeYaHVd"
|
|
},
|
|
"source": [
|
|
"Zanimivo je uporabiti Word2Vec za iskanje sinonimov. Naslednja funkcija bo vrnila `n` najbližjih besed glede na podan vhod. Da jih najdemo, izračunamo normo $|w_i - v|$, kjer je $v$ vektor, ki ustreza naši vhodni besedi, in $w_i$ je kodiranje $i$-te besede v besedišču. Nato razvrstimo tabelo in vrnemo ustrezne indekse z uporabo `argsort`, ter vzamemo prvih `n` elementov seznama, ki kodirajo položaje najbližjih besed v besedišču.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"colab": {
|
|
"base_uri": "https://localhost:8080/"
|
|
},
|
|
"id": "NlZyi-_olFar",
|
|
"outputId": "b5dbb163-88c4-4d5a-eaf2-6751f700e98c"
|
|
},
|
|
"outputs": [
|
|
{
|
|
"output_type": "execute_result",
|
|
"data": {
|
|
"text/plain": [
|
|
"['microsoft', 'quoted', 'lp', 'rate', 'top']"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"execution_count": 56
|
|
}
|
|
],
|
|
"source": [
|
|
"def close_words(x, n = 5):\n",
|
|
" vec = embedder(torch.tensor(vocab[x]))\n",
|
|
" top5 = np.linalg.norm(vectors.detach().numpy() - vec.detach().numpy(), axis = 1).argsort()[:n]\n",
|
|
" return [ vocab.itos[x] for x in top5 ]\n",
|
|
"\n",
|
|
"close_words('microsoft')"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"colab": {
|
|
"base_uri": "https://localhost:8080/"
|
|
},
|
|
"id": "-dQq7xeAln0U",
|
|
"outputId": "66f768c3-c248-4bfd-ce4f-c8ffc6d0dd0d"
|
|
},
|
|
"outputs": [
|
|
{
|
|
"output_type": "execute_result",
|
|
"data": {
|
|
"text/plain": [
|
|
"['basketball', 'lot', 'sinai', 'states', 'healthdaynews']"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"execution_count": 51
|
|
}
|
|
],
|
|
"source": [
|
|
"close_words('basketball')"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"colab": {
|
|
"base_uri": "https://localhost:8080/"
|
|
},
|
|
"id": "fJXqK26b29sa",
|
|
"outputId": "78f0baba-ffd0-485a-dd87-0a12bedfd7fa"
|
|
},
|
|
"outputs": [
|
|
{
|
|
"output_type": "execute_result",
|
|
"data": {
|
|
"text/plain": [
|
|
"['funds', 'travel', 'sydney', 'japan', 'business']"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"execution_count": 77
|
|
}
|
|
],
|
|
"source": [
|
|
"close_words('funds')"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "My0VeTDd3Ji8"
|
|
},
|
|
"source": [
|
|
"## Ključne točke\n",
|
|
"\n",
|
|
"Z uporabo pametnih tehnik, kot je CBoW, lahko treniramo model Word2Vec. Prav tako lahko poskusite trenirati model skip-gram, ki je zasnovan za napovedovanje sosednje besede glede na osrednjo, in preverite, kako dobro deluje.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"\n---\n\n**Omejitev odgovornosti**: \nTa dokument je bil preveden z uporabo storitve za strojno prevajanje [Co-op Translator](https://github.com/Azure/co-op-translator). Čeprav si prizadevamo za natančnost, vas opozarjamo, da lahko avtomatizirani prevodi vsebujejo napake ali netočnosti. Izvirni dokument v njegovem izvirnem jeziku je treba obravnavati kot avtoritativni vir. Za ključne informacije priporočamo strokovno človeško prevajanje. Ne prevzemamo odgovornosti za morebitna nesporazumevanja ali napačne razlage, ki bi izhajale iz uporabe tega prevoda.\n"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"colab": {
|
|
"collapsed_sections": [],
|
|
"name": "CBoW-PyTorch.ipynb",
|
|
"provenance": []
|
|
},
|
|
"interpreter": {
|
|
"hash": "16af2a8bbb083ea23e5e41c7f5787656b2ce26968575d8763f2c4b17f9cd711f"
|
|
},
|
|
"kernelspec": {
|
|
"display_name": "Python 3.8.12 ('py38')",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.8.12"
|
|
},
|
|
"orig_nbformat": 4,
|
|
"gpuClass": "standard",
|
|
"coopTranslator": {
|
|
"original_hash": "36df28efe3fe40b6fb0a7fa48fe3ea82",
|
|
"translation_date": "2025-08-30T08:02:02+00:00",
|
|
"source_file": "lessons/5-NLP/15-LanguageModeling/CBoW-PyTorch.ipynb",
|
|
"language_code": "sl"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 0
|
|
} |