414 lines
21 KiB
Plaintext
414 lines
21 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Mitandao ya Kizazi\n",
|
|
"\n",
|
|
"Mitandao ya Neural ya Kurudia (RNNs) na aina zake zenye seli zenye milango kama vile Long Short Term Memory Cells (LSTMs) na Gated Recurrent Units (GRUs) zilitoa njia ya kuunda mifano ya lugha, yaani, zinaweza kujifunza mpangilio wa maneno na kutoa utabiri wa neno linalofuata katika mfululizo. Hii inatuwezesha kutumia RNNs kwa **majukumu ya kizazi**, kama vile uzalishaji wa maandishi ya kawaida, tafsiri ya mashine, na hata maelezo ya picha.\n",
|
|
"\n",
|
|
"Katika usanifu wa RNN tuliojadili katika kitengo kilichopita, kila kitengo cha RNN kilizalisha hali ya siri inayofuata kama matokeo. Hata hivyo, tunaweza pia kuongeza matokeo mengine kwa kila kitengo cha kurudia, ambacho kingeturuhusu kutoa **mfululizo** (ambao ni sawa kwa urefu na mfululizo wa awali). Zaidi ya hayo, tunaweza kutumia vitengo vya RNN ambavyo havipokei ingizo katika kila hatua, na badala yake huchukua tu vekta ya hali ya awali, kisha huzalisha mfululizo wa matokeo.\n",
|
|
"\n",
|
|
"Katika daftari hili, tutazingatia mifano rahisi ya kizazi inayotusaidia kuzalisha maandishi. Kwa urahisi, hebu tujenge **mtandao wa kiwango cha herufi**, ambao huzalisha maandishi herufi moja moja. Wakati wa mafunzo, tunahitaji kuchukua maandishi fulani, na kuyagawanya katika mfululizo wa herufi.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 1,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Loading dataset...\n",
|
|
"Building vocab...\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"import torch\n",
|
|
"import torchtext\n",
|
|
"import numpy as np\n",
|
|
"from torchnlp import *\n",
|
|
"train_dataset,test_dataset,classes,vocab = load_dataset()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Kujenga Msamiati wa Herufi\n",
|
|
"\n",
|
|
"Ili kujenga mtandao wa kizazi cha kiwango cha herufi, tunahitaji kugawanya maandishi katika herufi moja moja badala ya maneno. Hili linaweza kufanyika kwa kufafanua njia tofauti ya kutenganisha:\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 2,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Vocabulary size = 82\n",
|
|
"Encoding of 'a' is 1\n",
|
|
"Character with code 13 is c\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"def char_tokenizer(words):\n",
|
|
" return list(words) #[word for word in words]\n",
|
|
"\n",
|
|
"counter = collections.Counter()\n",
|
|
"for (label, line) in train_dataset:\n",
|
|
" counter.update(char_tokenizer(line))\n",
|
|
"vocab = torchtext.vocab.vocab(counter)\n",
|
|
"\n",
|
|
"vocab_size = len(vocab)\n",
|
|
"print(f\"Vocabulary size = {vocab_size}\")\n",
|
|
"print(f\"Encoding of 'a' is {vocab.get_stoi()['a']}\")\n",
|
|
"print(f\"Character with code 13 is {vocab.get_itos()[13]}\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Hebu tuone mfano wa jinsi tunavyoweza kusimba maandishi kutoka kwenye seti yetu ya data:\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 3,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"tensor([ 0, 1, 2, 2, 3, 4, 5, 6, 3, 7, 8, 1, 9, 10, 3, 11, 2, 1,\n",
|
|
" 12, 3, 7, 1, 13, 14, 3, 15, 16, 5, 17, 3, 5, 18, 8, 3, 7, 2,\n",
|
|
" 1, 13, 14, 3, 19, 20, 8, 21, 5, 8, 9, 10, 22, 3, 20, 8, 21, 5,\n",
|
|
" 8, 9, 10, 3, 23, 3, 4, 18, 17, 9, 5, 23, 10, 8, 2, 2, 8, 9,\n",
|
|
" 10, 24, 3, 0, 1, 2, 2, 3, 4, 5, 9, 8, 8, 5, 25, 10, 3, 26,\n",
|
|
" 12, 27, 16, 26, 2, 27, 16, 28, 29, 30, 1, 16, 26, 3, 17, 31, 3, 21,\n",
|
|
" 2, 5, 9, 1, 23, 13, 32, 16, 27, 13, 10, 24, 3, 1, 9, 8, 3, 10,\n",
|
|
" 8, 8, 27, 16, 28, 3, 28, 9, 8, 8, 16, 3, 1, 28, 1, 27, 16, 6])"
|
|
]
|
|
},
|
|
"execution_count": 3,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"def enc(x):\n",
|
|
" return torch.LongTensor(encode(x,voc=vocab,tokenizer=char_tokenizer))\n",
|
|
"\n",
|
|
"enc(train_dataset[0][1])"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Kufundisha RNN ya kizazi\n",
|
|
"\n",
|
|
"Njia tutakayotumia kufundisha RNN ili kuzalisha maandishi ni kama ifuatavyo. Kwenye kila hatua, tutachukua mlolongo wa herufi zenye urefu wa `nchars`, na kuiomba mtandao uzalishe herufi inayofuata kwa kila herufi ya ingizo:\n",
|
|
"\n",
|
|
"\n",
|
|
"\n",
|
|
"Kulingana na hali halisi, tunaweza pia kutaka kujumuisha baadhi ya herufi maalum, kama vile *mwisho-wa-mlolongo* `<eos>`. Katika hali yetu, tunataka tu kufundisha mtandao kwa ajili ya uzalishaji wa maandishi usio na mwisho, kwa hivyo tutarekebisha ukubwa wa kila mlolongo kuwa sawa na tokeni `nchars`. Kwa hivyo, kila mfano wa mafunzo utajumuisha viingizo `nchars` na matokeo `nchars` (ambayo ni mlolongo wa ingizo uliosogezwa herufi moja kushoto). Minibatch itajumuisha mifululizo kadhaa kama hiyo.\n",
|
|
"\n",
|
|
"Njia tutakayotumia kuzalisha minibatches ni kuchukua kila maandishi ya habari yenye urefu wa `l`, na kuzalisha mchanganyiko wote wa ingizo-matokeo kutoka kwayo (kutakuwa na mchanganyiko `l-nchars` kama huo). Hii itaunda minibatch moja, na ukubwa wa minibatches utakuwa tofauti katika kila hatua ya mafunzo.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 4,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"(tensor([[ 0, 1, 2, ..., 28, 29, 30],\n",
|
|
" [ 1, 2, 2, ..., 29, 30, 1],\n",
|
|
" [ 2, 2, 3, ..., 30, 1, 16],\n",
|
|
" ...,\n",
|
|
" [20, 8, 21, ..., 1, 28, 1],\n",
|
|
" [ 8, 21, 5, ..., 28, 1, 27],\n",
|
|
" [21, 5, 8, ..., 1, 27, 16]]),\n",
|
|
" tensor([[ 1, 2, 2, ..., 29, 30, 1],\n",
|
|
" [ 2, 2, 3, ..., 30, 1, 16],\n",
|
|
" [ 2, 3, 4, ..., 1, 16, 26],\n",
|
|
" ...,\n",
|
|
" [ 8, 21, 5, ..., 28, 1, 27],\n",
|
|
" [21, 5, 8, ..., 1, 27, 16],\n",
|
|
" [ 5, 8, 9, ..., 27, 16, 6]]))"
|
|
]
|
|
},
|
|
"execution_count": 4,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"nchars = 100\n",
|
|
"\n",
|
|
"def get_batch(s,nchars=nchars):\n",
|
|
" ins = torch.zeros(len(s)-nchars,nchars,dtype=torch.long,device=device)\n",
|
|
" outs = torch.zeros(len(s)-nchars,nchars,dtype=torch.long,device=device)\n",
|
|
" for i in range(len(s)-nchars):\n",
|
|
" ins[i] = enc(s[i:i+nchars])\n",
|
|
" outs[i] = enc(s[i+1:i+nchars+1])\n",
|
|
" return ins,outs\n",
|
|
"\n",
|
|
"get_batch(train_dataset[0][1])"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Sasa hebu tueleze mtandao wa jenereta. Unaweza kutegemea seli yoyote ya kurudia ambayo tulijadili katika kitengo kilichopita (rahisi, LSTM au GRU). Katika mfano wetu tutatumia LSTM.\n",
|
|
"\n",
|
|
"Kwa sababu mtandao unachukua herufi kama pembejeo, na ukubwa wa msamiati ni mdogo sana, hatuhitaji safu ya kuweka alama (embedding layer), pembejeo iliyosimbwa kwa njia ya one-hot inaweza kwenda moja kwa moja kwenye seli ya LSTM. Hata hivyo, kwa sababu tunapitisha namba za herufi kama pembejeo, tunahitaji kuzisimba kwa njia ya one-hot kabla ya kuzipitisha kwenye LSTM. Hii inafanyika kwa kuita kazi ya `one_hot` wakati wa kupitisha mbele (`forward` pass). Kifaa cha kusimba matokeo kitakuwa safu ya mstari ambayo itabadilisha hali iliyofichwa kuwa matokeo yaliyosimbwa kwa njia ya one-hot.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 5,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"class LSTMGenerator(torch.nn.Module):\n",
|
|
" def __init__(self, vocab_size, hidden_dim):\n",
|
|
" super().__init__()\n",
|
|
" self.rnn = torch.nn.LSTM(vocab_size,hidden_dim,batch_first=True)\n",
|
|
" self.fc = torch.nn.Linear(hidden_dim, vocab_size)\n",
|
|
"\n",
|
|
" def forward(self, x, s=None):\n",
|
|
" x = torch.nn.functional.one_hot(x,vocab_size).to(torch.float32)\n",
|
|
" x,s = self.rnn(x,s)\n",
|
|
" return self.fc(x),s"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Wakati wa mafunzo, tunataka kuwa na uwezo wa kuchagua maandishi yanayozalishwa. Ili kufanya hivyo, tutaelezea kazi ya `generate` ambayo itazalisha mfululizo wa maandishi yenye urefu wa `size`, kuanzia na mfululizo wa awali `start`.\n",
|
|
"\n",
|
|
"Jinsi inavyofanya kazi ni kama ifuatavyo. Kwanza, tutapitisha mfululizo mzima wa `start` kupitia mtandao, na kuchukua hali ya matokeo `s` na herufi inayofuata iliyotabiriwa `out`. Kwa kuwa `out` imekodishwa kwa njia ya one-hot, tunachukua `argmax` ili kupata faharisi ya herufi `nc` katika msamiati, na kutumia `itos` kubaini herufi halisi na kuiongeza kwenye orodha ya herufi `chars`. Mchakato huu wa kuzalisha herufi moja unarudiwa mara `size` ili kuzalisha idadi inayohitajika ya herufi.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 8,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"def generate(net,size=100,start='today '):\n",
|
|
" chars = list(start)\n",
|
|
" out, s = net(enc(chars).view(1,-1).to(device))\n",
|
|
" for i in range(size):\n",
|
|
" nc = torch.argmax(out[0][-1])\n",
|
|
" chars.append(vocab.get_itos()[nc])\n",
|
|
" out, s = net(nc.view(1,-1),s)\n",
|
|
" return ''.join(chars)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Sasa tuanze mafunzo! Mzunguko wa mafunzo ni karibu sawa na katika mifano yetu ya awali, lakini badala ya usahihi tunachapisha maandishi yaliyotolewa kila mizunguko 1000.\n",
|
|
"\n",
|
|
"Umakini maalum unahitajika kwa njia tunavyohesabu hasara. Tunahitaji kuhesabu hasara tukizingatia matokeo yaliyosimbwa kwa njia ya one-hot `out`, na maandishi yanayotarajiwa `text_out`, ambayo ni orodha ya faharasa za herufi. Kwa bahati nzuri, kazi ya `cross_entropy` inatarajia matokeo ya mtandao yasiyosawazishwa kama hoja ya kwanza, na namba ya darasa kama ya pili, ambayo ni hasa tunacho nacho. Pia inafanya wastani wa kiotomatiki kulingana na ukubwa wa minibatch.\n",
|
|
"\n",
|
|
"Pia tunapunguza mafunzo kwa sampuli `samples_to_train`, ili tusisubiri kwa muda mrefu. Tunakuhimiza ujaribu na utafiti mafunzo marefu zaidi, labda kwa mizunguko kadhaa (ambapo ungehitaji kuunda mzunguko mwingine kuzunguka msimbo huu).\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 9,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Current loss = 4.398899078369141\n",
|
|
"today sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr s\n",
|
|
"Current loss = 2.161320447921753\n",
|
|
"today and to the tor to to the tor to to the tor to to the tor to to the tor to to the tor to to the tor t\n",
|
|
"Current loss = 1.6722588539123535\n",
|
|
"today and the court to the could to the could to the could to the could to the could to the could to the c\n",
|
|
"Current loss = 2.423795223236084\n",
|
|
"today and a second to the conternation of the conternation of the conternation of the conternation of the \n",
|
|
"Current loss = 1.702607274055481\n",
|
|
"today and the company to the company to the company to the company to the company to the company to the co\n",
|
|
"Current loss = 1.692358136177063\n",
|
|
"today and the company to the company to the company to the company to the company to the company to the co\n",
|
|
"Current loss = 1.9722288846969604\n",
|
|
"today and the control the control the control the control the control the control the control the control \n",
|
|
"Current loss = 1.8705692291259766\n",
|
|
"today and the second to the second to the second to the second to the second to the second to the second t\n",
|
|
"Current loss = 1.7626899480819702\n",
|
|
"today and a security and a security and a security and a security and a security and a security and a secu\n",
|
|
"Current loss = 1.5574463605880737\n",
|
|
"today and the company and the company and the company and the company and the company and the company and \n",
|
|
"Current loss = 1.5620026588439941\n",
|
|
"today and the be that the be the be that the be the be that the be the be that the be the be that the be t\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"net = LSTMGenerator(vocab_size,64).to(device)\n",
|
|
"\n",
|
|
"samples_to_train = 10000\n",
|
|
"optimizer = torch.optim.Adam(net.parameters(),0.01)\n",
|
|
"loss_fn = torch.nn.CrossEntropyLoss()\n",
|
|
"net.train()\n",
|
|
"for i,x in enumerate(train_dataset):\n",
|
|
" # x[0] is class label, x[1] is text\n",
|
|
" if len(x[1])-nchars<10:\n",
|
|
" continue\n",
|
|
" samples_to_train-=1\n",
|
|
" if not samples_to_train: break\n",
|
|
" text_in, text_out = get_batch(x[1])\n",
|
|
" optimizer.zero_grad()\n",
|
|
" out,s = net(text_in)\n",
|
|
" loss = torch.nn.functional.cross_entropy(out.view(-1,vocab_size),text_out.flatten()) #cross_entropy(out,labels)\n",
|
|
" loss.backward()\n",
|
|
" optimizer.step()\n",
|
|
" if i%1000==0:\n",
|
|
" print(f\"Current loss = {loss.item()}\")\n",
|
|
" print(generate(net))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Mfano huu tayari unazalisha maandishi mazuri, lakini unaweza kuboreshwa zaidi kwa njia kadhaa:\n",
|
|
"\n",
|
|
"* **Uboreshaji wa kizazi cha minibatch**. Njia tuliyotumia kuandaa data kwa mafunzo ilikuwa kuzalisha minibatch moja kutoka kwa sampuli moja. Hii si bora, kwa sababu minibatches zote zina ukubwa tofauti, na baadhi yao hata haziwezi kuzalishwa, kwa sababu maandishi ni madogo kuliko `nchars`. Pia, minibatches ndogo hazitumii GPU vya kutosha. Itakuwa busara kuchukua sehemu kubwa ya maandishi kutoka kwa sampuli zote, kisha kuzalisha jozi zote za pembejeo na matokeo, kuzichanganya, na kuzalisha minibatches za ukubwa sawa.\n",
|
|
"\n",
|
|
"* **LSTM yenye tabaka nyingi**. Inaleta maana kujaribu tabaka 2 au 3 za seli za LSTM. Kama tulivyotaja katika kipengele cha awali, kila tabaka ya LSTM huchota mifumo fulani kutoka kwa maandishi, na kwa kizazi cha kiwango cha herufi tunaweza kutarajia tabaka ya chini ya LSTM kuwa na jukumu la kuchota silabi, na tabaka za juu - maneno na mchanganyiko wa maneno. Hii inaweza kutekelezwa kwa urahisi kwa kupitisha parameter ya idadi ya tabaka kwa constructor ya LSTM.\n",
|
|
"\n",
|
|
"* Unaweza pia kutaka kujaribu **vitengo vya GRU** na kuona ni vipi vinafanya kazi vizuri zaidi, pamoja na **ukubwa tofauti wa tabaka fiche**. Tabaka fiche kubwa sana inaweza kusababisha overfitting (mfano mtandao utajifunza maandishi halisi), na ukubwa mdogo unaweza usizalishe matokeo mazuri.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Uzalishaji wa maandishi laini na joto\n",
|
|
"\n",
|
|
"Katika ufafanuzi wa awali wa `generate`, tulikuwa tunachukua herufi yenye uwezekano wa juu zaidi kama herufi inayofuata katika maandishi yanayozalishwa. Hii ilisababisha maandishi mara nyingi \"kurudia\" mfuatano wa herufi zilezile tena na tena, kama katika mfano huu:\n",
|
|
"```\n",
|
|
"today of the second the company and a second the company ...\n",
|
|
"```\n",
|
|
"\n",
|
|
"Hata hivyo, tukitazama usambazaji wa uwezekano kwa herufi inayofuata, inaweza kuwa tofauti kati ya uwezekano wa juu zaidi si kubwa sana, kwa mfano, herufi moja inaweza kuwa na uwezekano wa 0.2, nyingine - 0.19, n.k. Kwa mfano, tunapotafuta herufi inayofuata katika mfuatano '*play*', herufi inayofuata inaweza kuwa nafasi, au **e** (kama katika neno *player*).\n",
|
|
"\n",
|
|
"Hii inatufikisha kwenye hitimisho kwamba si kila wakati ni \"haki\" kuchagua herufi yenye uwezekano wa juu zaidi, kwa sababu kuchagua ya pili kwa juu bado kunaweza kutupeleka kwenye maandishi yenye maana. Ni busara zaidi **kuchagua kwa sampuli** herufi kutoka kwenye usambazaji wa uwezekano uliotolewa na matokeo ya mtandao.\n",
|
|
"\n",
|
|
"Uchaguzi huu wa sampuli unaweza kufanywa kwa kutumia kazi ya `multinomial` ambayo inatekeleza kile kinachoitwa **usambazaji wa multinomial**. Kazi inayotekeleza uzalishaji huu wa maandishi **laini** imefafanuliwa hapa chini:\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 10,
|
|
"metadata": {
|
|
"scrolled": true
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"--- Temperature = 0.3\n",
|
|
"Today and a company and complete an all the land the restrational the as a security and has provers the pay to and a report and the computer in the stand has filities and working the law the stations for a company and with the company and the final the first company and refight of the state and and workin\n",
|
|
"\n",
|
|
"--- Temperature = 0.8\n",
|
|
"Today he oniis its first to Aus bomblaties the marmation a to manan boogot that pirate assaid a relaid their that goverfin the the Cappets Ecrotional Assonia Cition targets it annight the w scyments Blamity #39;s TVeer Diercheg Reserals fran envyuil that of ster said access what succers of Dour-provelith\n",
|
|
"\n",
|
|
"--- Temperature = 1.0\n",
|
|
"Today holy they a 11 will meda a toket subsuaties, engins for Chanos, they's has stainger past to opening orital his thempting new Nattona was al innerforder advan-than #36;s night year his religuled talitatian what the but with Wednesday to Justment will wemen of Mark CCC Camp as Timed Nae wome a leaders\n",
|
|
"\n",
|
|
"--- Temperature = 1.3\n",
|
|
"Today gpone 2.5 fech atcusion poor cocles toparsdorM.cht Line Pamage put 43 his calt lowed to the book, that has authh-the silia rruch ailing to'ory andhes beutirsimi- Aefffive heading offil an auf eacklets is charged evis, Gunymy oy) Mony has it after-sloythyor loveId out filme, the Natabl -Najuntaxiggs \n",
|
|
"\n",
|
|
"--- Temperature = 1.8\n",
|
|
"Today plary, P.slan chly\\401 mardregationly #39;t 8.1Mide) closes ,filtcon alfly playin roven!\\grea.-QFBEP: Iss onfarchQ/itilia CCf Zivesigntwasta orce.-Peul-aw.uicrin of fuglinfsut aftaningwo, MIEX awayew Aice Woiduar Corvagiugge oppo esig ThusBratourid canthly-RyI.co lagitems\\eexciaishes.conBabntusmor I\n",
|
|
"\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"def generate_soft(net,size=100,start='today ',temperature=1.0):\n",
|
|
" chars = list(start)\n",
|
|
" out, s = net(enc(chars).view(1,-1).to(device))\n",
|
|
" for i in range(size):\n",
|
|
" #nc = torch.argmax(out[0][-1])\n",
|
|
" out_dist = out[0][-1].div(temperature).exp()\n",
|
|
" nc = torch.multinomial(out_dist,1)[0]\n",
|
|
" chars.append(vocab.get_itos()[nc])\n",
|
|
" out, s = net(nc.view(1,-1),s)\n",
|
|
" return ''.join(chars)\n",
|
|
" \n",
|
|
"for i in [0.3,0.8,1.0,1.3,1.8]:\n",
|
|
" print(f\"--- Temperature = {i}\\n{generate_soft(net,size=300,start='Today ',temperature=i)}\\n\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Tumetambulisha kipengele kingine kinachoitwa **joto**, ambacho kinatumika kuonyesha jinsi tunavyopaswa kushikilia kwa nguvu uwezekano wa juu zaidi. Ikiwa joto ni 1.0, tunafanya sampuli ya haki ya multinomial, na wakati joto linaenda hadi ukomo - uwezekano wote unakuwa sawa, na tunachagua herufi inayofuata kwa nasibu. Katika mfano hapa chini tunaweza kuona kwamba maandishi yanakuwa hayana maana tunapoongeza joto kupita kiasi, na yanakuwa kama maandishi magumu yaliyo \"zungushwa\" tunapokaribia 0.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"\n---\n\n**Kanusho**: \nHati hii imetafsiriwa kwa kutumia huduma ya tafsiri ya AI [Co-op Translator](https://github.com/Azure/co-op-translator). Ingawa tunajitahidi kuhakikisha usahihi, tafsiri za kiotomatiki zinaweza kuwa na makosa au kutokuwa sahihi. Hati ya asili katika lugha yake ya awali inapaswa kuchukuliwa kama chanzo cha mamlaka. Kwa taarifa muhimu, tafsiri ya kitaalamu ya binadamu inapendekezwa. Hatutawajibika kwa kutoelewana au tafsiri zisizo sahihi zinazotokana na matumizi ya tafsiri hii.\n"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"interpreter": {
|
|
"hash": "16af2a8bbb083ea23e5e41c7f5787656b2ce26968575d8763f2c4b17f9cd711f"
|
|
},
|
|
"kernelspec": {
|
|
"display_name": "Python 3.8.12 ('py38')",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.8.12"
|
|
},
|
|
"coopTranslator": {
|
|
"original_hash": "7673cd150d96c74c6d6011460094efb4",
|
|
"translation_date": "2025-08-29T15:46:10+00:00",
|
|
"source_file": "lessons/5-NLP/17-GenerativeNetworks/GenerativePyTorch.ipynb",
|
|
"language_code": "sw"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 4
|
|
} |