495 lines
27 KiB
Plaintext
495 lines
27 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Mitandao ya Kizazi\n",
|
|
"\n",
|
|
"Mitandao ya Neural ya Kurudiarudia (RNNs) na aina zake zenye seli zenye milango kama vile Long Short Term Memory Cells (LSTMs) na Gated Recurrent Units (GRUs) zilitoa njia ya kuunda mifano ya lugha, yaani, zinaweza kujifunza mpangilio wa maneno na kutoa utabiri wa neno linalofuata katika mfuatano. Hii inatuwezesha kutumia RNNs kwa **kazi za kizazi**, kama vile utengenezaji wa maandishi ya kawaida, tafsiri ya mashine, na hata uundaji wa maelezo ya picha.\n",
|
|
"\n",
|
|
"Katika usanifu wa RNN tuliojadili katika sehemu iliyopita, kila kitengo cha RNN kilizalisha hali iliyofichwa inayofuata kama matokeo. Hata hivyo, tunaweza pia kuongeza matokeo mengine kwa kila kitengo cha kurudiarudia, ambacho kingeturuhusu kutoa **mfuatano** (ambao ni sawa kwa urefu na mfuatano wa awali). Zaidi ya hayo, tunaweza kutumia vitengo vya RNN ambavyo havipokei ingizo katika kila hatua, na badala yake huchukua tu hali ya awali kama vekta, na kisha huzalisha mfuatano wa matokeo.\n",
|
|
"\n",
|
|
"Katika daftari hili, tutazingatia mifano rahisi ya kizazi inayotusaidia kuzalisha maandishi. Kwa urahisi, hebu tujenge **mtandao wa kiwango cha herufi**, ambao huzalisha maandishi herufi moja moja. Wakati wa mafunzo, tunahitaji kuchukua mkusanyiko wa maandishi, na kuugawanya katika mfuatano wa herufi.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 1,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"import tensorflow as tf\n",
|
|
"from tensorflow import keras\n",
|
|
"import tensorflow_datasets as tfds\n",
|
|
"import numpy as np\n",
|
|
"\n",
|
|
"ds_train, ds_test = tfds.load('ag_news_subset').values()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Kujenga Msamiati wa Herufi\n",
|
|
"\n",
|
|
"Ili kujenga mtandao wa kizazi wa kiwango cha herufi, tunahitaji kugawanya maandishi kuwa herufi moja-moja badala ya maneno. Safu ya `TextVectorization` ambayo tumekuwa tukitumia hapo awali haiwezi kufanya hivyo, kwa hivyo tuna chaguzi mbili:\n",
|
|
"\n",
|
|
"* Kupakia maandishi kwa mikono na kufanya tokenization 'kwa mkono', kama ilivyo katika [mfano huu rasmi wa Keras](https://keras.io/examples/generative/lstm_character_level_text_generation/)\n",
|
|
"* Kutumia darasa la `Tokenizer` kwa tokenization ya kiwango cha herufi.\n",
|
|
"\n",
|
|
"Tutachagua chaguo la pili. `Tokenizer` pia inaweza kutumika kugawanya katika maneno, kwa hivyo mtu anaweza kubadilisha kwa urahisi kutoka tokenization ya kiwango cha herufi hadi kiwango cha maneno.\n",
|
|
"\n",
|
|
"Ili kufanya tokenization ya kiwango cha herufi, tunahitaji kupitisha parameter `char_level=True`:\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 2,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"def extract_text(x):\n",
|
|
" return x['title']+' '+x['description']\n",
|
|
"\n",
|
|
"def tupelize(x):\n",
|
|
" return (extract_text(x),x['label'])\n",
|
|
"\n",
|
|
"tokenizer = keras.preprocessing.text.Tokenizer(char_level=True,lower=False)\n",
|
|
"tokenizer.fit_on_texts([x['title'].numpy().decode('utf-8') for x in ds_train])"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Tunataka pia kutumia token moja maalum kuashiria **mwisho wa mfuatano**, ambayo tutaiita `<eos>`. Wacha tuiongeze kwa mkono kwenye msamiati:\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 3,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"eos_token = len(tokenizer.word_index)+1\n",
|
|
"tokenizer.word_index['<eos>'] = eos_token\n",
|
|
"\n",
|
|
"vocab_size = eos_token + 1"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": []
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 4,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"[[48, 2, 10, 10, 5, 44, 1, 25, 5, 8, 10, 13, 78]]"
|
|
]
|
|
},
|
|
"execution_count": 4,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"tokenizer.texts_to_sequences(['Hello, world!'])"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Kufundisha RNN ya kizazi kutengeneza vichwa vya habari\n",
|
|
"\n",
|
|
"Njia tutakayotumia kufundisha RNN ili kutengeneza vichwa vya habari ni kama ifuatavyo. Kwenye kila hatua, tutachukua kichwa kimoja, ambacho kitaingizwa kwenye RNN, na kwa kila herufi ya ingizo tutaiomba mtandao kutengeneza herufi inayofuata ya matokeo:\n",
|
|
"\n",
|
|
"\n",
|
|
"\n",
|
|
"Kwa herufi ya mwisho ya mlolongo wetu, tutaiomba mtandao kutengeneza tokeni `<eos>`.\n",
|
|
"\n",
|
|
"Tofauti kuu ya RNN ya kizazi tunayotumia hapa ni kwamba tutachukua matokeo kutoka kila hatua ya RNN, na siyo tu kutoka kwenye seli ya mwisho. Hii inaweza kufanikishwa kwa kuweka kipengele `return_sequences` kwenye seli ya RNN.\n",
|
|
"\n",
|
|
"Kwa hivyo, wakati wa mafunzo, ingizo kwa mtandao litakuwa mlolongo wa herufi zilizofichwa wa urefu fulani, na matokeo yatakuwa mlolongo wa urefu huo huo, lakini uliosogezwa kwa kipengele kimoja na kumalizika na `<eos>`. Kundi dogo (minibatch) litajumuisha milolongo kadhaa kama hiyo, na tutahitaji kutumia **padding** ili kulinganisha milolongo yote.\n",
|
|
"\n",
|
|
"Hebu tuunde kazi zitakazobadilisha seti ya data kwa ajili yetu. Kwa sababu tunataka kuweka padding kwenye kiwango cha minibatch, tutaanza kwa kugawanya seti ya data kwa kupiga `.batch()`, na kisha kutumia `map` ili kufanya mabadiliko. Kwa hivyo, kazi ya mabadiliko itachukua minibatch nzima kama parameter:\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 5,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"def title_batch(x):\n",
|
|
" x = [t.numpy().decode('utf-8') for t in x]\n",
|
|
" z = tokenizer.texts_to_sequences(x)\n",
|
|
" z = tf.keras.preprocessing.sequence.pad_sequences(z)\n",
|
|
" return tf.one_hot(z,vocab_size), tf.one_hot(tf.concat([z[:,1:],tf.constant(eos_token,shape=(len(z),1))],axis=1),vocab_size)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Mambo muhimu tunayofanya hapa:\n",
|
|
"* Kwanza tunatoa maandishi halisi kutoka kwa tensor ya maandishi\n",
|
|
"* `text_to_sequences` hubadilisha orodha ya maandishi kuwa orodha ya tensors za nambari\n",
|
|
"* `pad_sequences` huongeza nafasi kwenye tensors hizo hadi urefu wake wa juu\n",
|
|
"* Hatimaye tunafanya one-hot encoding kwa herufi zote, na pia tunafanya mabadiliko na kuongeza `<eos>`. Tutakuja kuona hivi karibuni kwa nini tunahitaji herufi zilizo katika one-hot encoding.\n",
|
|
"\n",
|
|
"Hata hivyo, kazi hii ni **Pythonic**, yaani haiwezi kutafsiriwa moja kwa moja kuwa grafu ya kihesabu ya Tensorflow. Tutapata makosa tukijaribu kutumia kazi hii moja kwa moja katika kazi ya `Dataset.map`. Tunahitaji kufunika mwito huu wa Pythonic kwa kutumia kifuniko cha `py_function`:\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 6,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"def title_batch_fn(x):\n",
|
|
" x = x['title']\n",
|
|
" a,b = tf.py_function(title_batch,inp=[x],Tout=(tf.float32,tf.float32))\n",
|
|
" return a,b"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"> **Kumbuka**: Kutofautisha kati ya kazi za mabadiliko za Pythonic na Tensorflow kunaweza kuonekana kuwa ngumu kidogo, na unaweza kujiuliza kwa nini hatubadilishi dataset kwa kutumia kazi za kawaida za Python kabla ya kuipitisha kwa `fit`. Ingawa hili linaweza kufanyika, kutumia `Dataset.map` kuna faida kubwa, kwa sababu mchakato wa mabadiliko ya data unatekelezwa kwa kutumia grafu ya kihesabu ya Tensorflow, ambayo inachukua faida ya mahesabu ya GPU, na inapunguza hitaji la kupitisha data kati ya CPU/GPU.\n",
|
|
"\n",
|
|
"Sasa tunaweza kujenga mtandao wetu wa jenereta na kuanza mafunzo. Unaweza kutegemea seli yoyote ya kurudia tuliyojadili katika kitengo kilichopita (rahisi, LSTM au GRU). Katika mfano wetu tutatumia LSTM.\n",
|
|
"\n",
|
|
"Kwa sababu mtandao unachukua herufi kama pembejeo, na ukubwa wa msamiati ni mdogo sana, hatuhitaji safu ya embedding, pembejeo iliyowakilishwa kwa njia ya one-hot inaweza kwenda moja kwa moja kwenye seli ya LSTM. Safu ya pato itakuwa `Dense` classifier ambayo itabadilisha pato la LSTM kuwa namba za tokeni zilizowakilishwa kwa njia ya one-hot.\n",
|
|
"\n",
|
|
"Zaidi ya hayo, kwa kuwa tunashughulika na misururu ya urefu tofauti, tunaweza kutumia safu ya `Masking` kuunda maski ambayo itapuuza sehemu ya mfuatano iliyojazwa. Hii si lazima kabisa, kwa sababu hatuvutiwi sana na kila kitu kinachozidi tokeni `<eos>`, lakini tutaifanya kwa lengo la kupata uzoefu na aina hii ya safu. `input_shape` itakuwa `(None, vocab_size)`, ambapo `None` inaonyesha mfuatano wa urefu tofauti, na umbo la pato ni `(None, vocab_size)` pia, kama unavyoweza kuona kutoka kwa `summary`:\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 7,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Model: \"sequential\"\n",
|
|
"_________________________________________________________________\n",
|
|
"Layer (type) Output Shape Param # \n",
|
|
"=================================================================\n",
|
|
"masking (Masking) (None, None, 84) 0 \n",
|
|
"_________________________________________________________________\n",
|
|
"lstm (LSTM) (None, None, 128) 109056 \n",
|
|
"_________________________________________________________________\n",
|
|
"dense (Dense) (None, None, 84) 10836 \n",
|
|
"=================================================================\n",
|
|
"Total params: 119,892\n",
|
|
"Trainable params: 119,892\n",
|
|
"Non-trainable params: 0\n",
|
|
"_________________________________________________________________\n",
|
|
"15000/15000 [==============================] - 229s 15ms/step - loss: 1.5385\n"
|
|
]
|
|
},
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"<tensorflow.python.keras.callbacks.History at 0x7fa40c1245e0>"
|
|
]
|
|
},
|
|
"execution_count": 7,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"model = keras.models.Sequential([\n",
|
|
" keras.layers.Masking(input_shape=(None,vocab_size)),\n",
|
|
" keras.layers.LSTM(128,return_sequences=True),\n",
|
|
" keras.layers.Dense(vocab_size,activation='softmax')\n",
|
|
"])\n",
|
|
"\n",
|
|
"model.summary()\n",
|
|
"model.compile(loss='categorical_crossentropy')\n",
|
|
"\n",
|
|
"model.fit(ds_train.batch(8).map(title_batch_fn))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Kuzalisha matokeo\n",
|
|
"\n",
|
|
"Sasa kwamba tumefundisha modeli, tunataka kuitumia kuzalisha matokeo fulani. Kwanza kabisa, tunahitaji njia ya kutafsiri maandishi yanayowakilishwa na mfululizo wa namba za tokeni. Ili kufanya hivyo, tunaweza kutumia kazi ya `tokenizer.sequences_to_texts`; hata hivyo, haifanyi kazi vizuri na tokeni za kiwango cha herufi. Kwa hivyo tutachukua kamusi ya tokeni kutoka kwa tokenizer (inayoitwa `word_index`), kujenga ramani ya kurudi nyuma, na kuandika kazi yetu ya kutafsiri:\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 10,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"reverse_map = {val:key for key, val in tokenizer.word_index.items()}\n",
|
|
"\n",
|
|
"def decode(x):\n",
|
|
" return ''.join([reverse_map[t] for t in x])"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Sasa, hebu tufanye kizazi. Tutaanza na kamba fulani `start`, tuitafsiri kuwa mlolongo `inp`, na kisha katika kila hatua tutaita mtandao wetu ili kutabiri herufi inayofuata.\n",
|
|
"\n",
|
|
"Matokeo ya mtandao `out` ni vector yenye vipengele `vocab_size` vinavyowakilisha uwezekano wa kila tokeni, na tunaweza kupata namba ya tokeni yenye uwezekano mkubwa zaidi kwa kutumia `argmax`. Kisha tunaongeza herufi hii kwenye orodha ya tokeni zilizotengenezwa, na tunaendelea na kizazi. Mchakato huu wa kutengeneza herufi moja unarudiwa mara `size` ili kuzalisha idadi inayohitajika ya herufi, na tunakomesha mapema ikiwa `eos_token` itapatikana.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 12,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"'Today #39;s lead to strike for the strike for the strike for the strike (AFP)'"
|
|
]
|
|
},
|
|
"execution_count": 12,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"def generate(model,size=100,start='Today '):\n",
|
|
" inp = tokenizer.texts_to_sequences([start])[0]\n",
|
|
" chars = inp\n",
|
|
" for i in range(size):\n",
|
|
" out = model(tf.expand_dims(tf.one_hot(inp,vocab_size),0))[0][-1]\n",
|
|
" nc = tf.argmax(out)\n",
|
|
" if nc==eos_token:\n",
|
|
" break\n",
|
|
" chars.append(nc.numpy())\n",
|
|
" inp = inp+[nc]\n",
|
|
" return decode(chars)\n",
|
|
" \n",
|
|
"generate(model)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Kuchukua sampuli ya matokeo wakati wa mafunzo\n",
|
|
"\n",
|
|
"Kwa sababu hatuna vipimo vyovyote vya maana kama *usahihi*, njia pekee ya kuona kwamba modeli yetu inaboreka ni kwa **kuchukua sampuli** ya mfululizo wa maandishi yanayozalishwa wakati wa mafunzo. Ili kufanya hivyo, tutatumia **callbacks**, yaani, kazi ambazo tunaweza kupitisha kwa `fit` function, na ambazo zitaitwa mara kwa mara wakati wa mafunzo.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 13,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Epoch 1/3\n",
|
|
"15000/15000 [==============================] - 226s 15ms/step - loss: 1.2703\n",
|
|
"Today #39;s a lead in the company for the strike\n",
|
|
"Epoch 2/3\n",
|
|
"15000/15000 [==============================] - 227s 15ms/step - loss: 1.2057\n",
|
|
"Today #39;s the Market Service on Security Start (AP)\n",
|
|
"Epoch 3/3\n",
|
|
"15000/15000 [==============================] - 226s 15ms/step - loss: 1.1752\n",
|
|
"Today #39;s a line on the strike to start for the start\n"
|
|
]
|
|
},
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"<tensorflow.python.keras.callbacks.History at 0x7fa40c74e3d0>"
|
|
]
|
|
},
|
|
"execution_count": 13,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"sampling_callback = keras.callbacks.LambdaCallback(\n",
|
|
" on_epoch_end = lambda batch, logs: print(generate(model))\n",
|
|
")\n",
|
|
"\n",
|
|
"model.fit(ds_train.batch(8).map(title_batch_fn),callbacks=[sampling_callback],epochs=3)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Mfano huu tayari unazalisha maandishi mazuri, lakini unaweza kuboreshwa zaidi kwa njia kadhaa:\n",
|
|
"\n",
|
|
"* **Maandishi zaidi**. Tumetumia tu vichwa vya habari kwa kazi yetu, lakini unaweza kutaka kujaribu na maandishi kamili. Kumbuka kwamba RNN hazifanyi kazi vizuri sana na mfuatano mrefu, kwa hivyo inafaa kugawanya katika sentensi fupi, au kila wakati kufundisha kwa urefu wa mfuatano uliowekwa wa thamani fulani iliyotanguliwa `num_chars` (kwa mfano, 256). Unaweza kujaribu kubadilisha mfano hapo juu kuwa usanifu kama huo, ukitumia [mafunzo rasmi ya Keras](https://keras.io/examples/generative/lstm_character_level_text_generation/) kama msukumo.\n",
|
|
"\n",
|
|
"* **LSTM ya tabaka nyingi**. Inafaa kujaribu tabaka 2 au 3 za seli za LSTM. Kama tulivyotaja katika kitengo kilichopita, kila tabaka la LSTM huchota mifumo fulani kutoka kwa maandishi, na kwa kizazi cha kiwango cha herufi tunaweza kutarajia tabaka za chini za LSTM kushughulikia silabi, na tabaka za juu - maneno na mchanganyiko wa maneno. Hii inaweza kutekelezwa kwa urahisi kwa kupitisha kipengele cha idadi-ya-tabaka kwa mjenzi wa LSTM.\n",
|
|
"\n",
|
|
"* Unaweza pia kutaka kujaribu na **vitengo vya GRU** na kuona ni vipi vinafanya kazi bora, na pia na **saizi tofauti za tabaka zilizofichwa**. Tabaka kubwa sana iliyofichwa inaweza kusababisha kujifunza kupita kiasi (kwa mfano, mtandao utajifunza maandishi halisi), na saizi ndogo inaweza isizalishe matokeo mazuri.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Uzalishaji wa maandishi laini na joto\n",
|
|
"\n",
|
|
"Katika ufafanuzi wa awali wa `generate`, tulikuwa tunachukua herufi yenye uwezekano wa juu zaidi kama herufi inayofuata katika maandishi yanayozalishwa. Hii ilisababisha maandishi mara nyingi \"kurudia\" mfululizo wa herufi zile zile tena na tena, kama katika mfano huu:\n",
|
|
"```\n",
|
|
"today of the second the company and a second the company ...\n",
|
|
"```\n",
|
|
"\n",
|
|
"Hata hivyo, tukitazama usambazaji wa uwezekano kwa herufi inayofuata, inaweza kuwa tofauti kati ya uwezekano wa juu zaidi si kubwa sana, kwa mfano herufi moja inaweza kuwa na uwezekano wa 0.2, nyingine - 0.19, nk. Kwa mfano, tunapotafuta herufi inayofuata katika mfululizo '*play*', herufi inayofuata inaweza kuwa nafasi, au **e** (kama katika neno *player*).\n",
|
|
"\n",
|
|
"Hii inatupeleka kwenye hitimisho kwamba si kila wakati ni \"haki\" kuchagua herufi yenye uwezekano wa juu zaidi, kwa sababu kuchagua ya pili yenye uwezekano wa juu bado inaweza kutupeleka kwenye maandishi yenye maana. Ni busara zaidi **kuchagua kwa sampuli** herufi kutoka kwa usambazaji wa uwezekano uliotolewa na matokeo ya mtandao.\n",
|
|
"\n",
|
|
"Sampuli hii inaweza kufanywa kwa kutumia kazi ya `np.multinomial` ambayo inatekeleza kinachoitwa **usambazaji wa multinomial**. Kazi inayotekeleza uzalishaji huu wa maandishi **laini** imefafanuliwa hapa chini:\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 33,
|
|
"metadata": {
|
|
"scrolled": true
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"\n",
|
|
"--- Temperature = 0.3\n",
|
|
"Today #39;s strike #39; to start at the store return\n",
|
|
"On Sunday PO to Be Data Profit Up (Reuters)\n",
|
|
"Moscow, SP wins straight to the Microsoft #39;s control of the space start\n",
|
|
"President olding of the blast start for the strike to pay <b>...</b>\n",
|
|
"Little red riding hood ficed to the spam countered in European <b>...</b>\n",
|
|
"\n",
|
|
"--- Temperature = 0.8\n",
|
|
"Today countie strikes ryder missile faces food market blut\n",
|
|
"On Sunday collores lose-toppy of sale of Bullment in <b>...</b>\n",
|
|
"Moscow, IBM Diffeiting in Afghan Software Hotels (Reuters)\n",
|
|
"President Ol Luster for Profit Peaced Raised (AP)\n",
|
|
"Little red riding hood dace on depart talks #39; bank up\n",
|
|
"\n",
|
|
"--- Temperature = 1.0\n",
|
|
"Today wits House buiting debate fixes #39; supervice stake again\n",
|
|
"On Sunday arling digital poaching In for level\n",
|
|
"Moscow, DS Up 7, Top Proble Protest Caprey Mamarian Strike\n",
|
|
"President teps help of roubler stepted lessabul-Dhalitics (AFP)\n",
|
|
"Little red riding hood signs on cash in Carter-youb\n",
|
|
"\n",
|
|
"--- Temperature = 1.3\n",
|
|
"Today wits flawer ro, pSIA figat's co DroftwavesIs Talo up\n",
|
|
"On Sunday hround elitwing wint EU Powerburlinetien\n",
|
|
"Moscow, Bazz #39;s sentries olymen winnelds' next for Olympite Huc?\n",
|
|
"President lost securitys from power Elections in Smiltrials\n",
|
|
"Little red riding hood vides profit, exponituity, profitmainalist-at said listers\n",
|
|
"\n",
|
|
"--- Temperature = 1.8\n",
|
|
"Today #39;It: He deat: N.KA Asside\n",
|
|
"On Sunday i arry Par aldeup patient Wo stele1\n"
|
|
]
|
|
},
|
|
{
|
|
"ename": "KeyError",
|
|
"evalue": "0",
|
|
"output_type": "error",
|
|
"traceback": [
|
|
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
|
|
"\u001b[0;31mKeyError\u001b[0m Traceback (most recent call last)",
|
|
"\u001b[0;32m<ipython-input-33-db32367a0feb>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m 18\u001b[0m \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34mf\"\\n--- Temperature = {i}\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 19\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mj\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mrange\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m5\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 20\u001b[0;31m \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mgenerate_soft\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmodel\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0msize\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m300\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mstart\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mwords\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mj\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mtemperature\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mi\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
|
|
"\u001b[0;32m<ipython-input-33-db32367a0feb>\u001b[0m in \u001b[0;36mgenerate_soft\u001b[0;34m(model, size, start, temperature)\u001b[0m\n\u001b[1;32m 11\u001b[0m \u001b[0mchars\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnc\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 12\u001b[0m \u001b[0minp\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0minp\u001b[0m\u001b[0;34m+\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mnc\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 13\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0mdecode\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mchars\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 14\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 15\u001b[0m \u001b[0mwords\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;34m'Today '\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m'On Sunday '\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m'Moscow, '\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m'President '\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m'Little red riding hood '\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
|
|
"\u001b[0;32m<ipython-input-10-3f5fa6130b1d>\u001b[0m in \u001b[0;36mdecode\u001b[0;34m(x)\u001b[0m\n\u001b[1;32m 2\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 3\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mdecode\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 4\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0;34m''\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mjoin\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mreverse_map\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mt\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mt\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
|
|
"\u001b[0;32m<ipython-input-10-3f5fa6130b1d>\u001b[0m in \u001b[0;36m<listcomp>\u001b[0;34m(.0)\u001b[0m\n\u001b[1;32m 2\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 3\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mdecode\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 4\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0;34m''\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mjoin\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mreverse_map\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mt\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mt\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
|
|
"\u001b[0;31mKeyError\u001b[0m: 0"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"def generate_soft(model,size=100,start='Today ',temperature=1.0):\n",
|
|
" inp = tokenizer.texts_to_sequences([start])[0]\n",
|
|
" chars = inp\n",
|
|
" for i in range(size):\n",
|
|
" out = model(tf.expand_dims(tf.one_hot(inp,vocab_size),0))[0][-1]\n",
|
|
" probs = tf.exp(tf.math.log(out)/temperature).numpy().astype(np.float64)\n",
|
|
" probs = probs/np.sum(probs)\n",
|
|
" nc = np.argmax(np.random.multinomial(1,probs,1))\n",
|
|
" if nc==eos_token:\n",
|
|
" break\n",
|
|
" chars.append(nc)\n",
|
|
" inp = inp+[nc]\n",
|
|
" return decode(chars)\n",
|
|
"\n",
|
|
"words = ['Today ','On Sunday ','Moscow, ','President ','Little red riding hood ']\n",
|
|
" \n",
|
|
"for i in [0.3,0.8,1.0,1.3,1.8]:\n",
|
|
" print(f\"\\n--- Temperature = {i}\")\n",
|
|
" for j in range(5):\n",
|
|
" print(generate_soft(model,size=300,start=words[j],temperature=i))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Tumetambulisha kipengele kingine kinachoitwa **joto**, ambacho kinatumika kuonyesha jinsi tunavyopaswa kushikilia kwa nguvu uwezekano wa juu zaidi. Ikiwa joto ni 1.0, tunafanya sampuli ya haki ya multinomial, na wakati joto linaenda hadi ukomo - uwezekano wote unakuwa sawa, na tunachagua herufi inayofuata kwa nasibu. Katika mfano hapa chini tunaweza kuona kwamba maandishi yanakuwa hayana maana tunapoongeza joto kupita kiasi, na yanakuwa kama maandishi magumu yaliyo \"zungushwa\" tunapokaribia 0.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"\n---\n\n**Kanusho**: \nHati hii imetafsiriwa kwa kutumia huduma ya tafsiri ya AI [Co-op Translator](https://github.com/Azure/co-op-translator). Ingawa tunajitahidi kwa usahihi, tafadhali fahamu kuwa tafsiri za kiotomatiki zinaweza kuwa na makosa au kutokuwa sahihi. Hati ya asili katika lugha yake ya awali inapaswa kuzingatiwa kama chanzo cha mamlaka. Kwa taarifa muhimu, inashauriwa kutumia tafsiri ya kitaalamu ya binadamu. Hatutawajibika kwa maelewano mabaya au tafsiri zisizo sahihi zinazotokana na matumizi ya tafsiri hii.\n"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"interpreter": {
|
|
"hash": "16af2a8bbb083ea23e5e41c7f5787656b2ce26968575d8763f2c4b17f9cd711f"
|
|
},
|
|
"kernelspec": {
|
|
"display_name": "Python 3.8.12 ('py38')",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.8.12"
|
|
},
|
|
"coopTranslator": {
|
|
"original_hash": "9fbb7d5fda708537649f71f5f646fcde",
|
|
"translation_date": "2025-08-29T15:41:31+00:00",
|
|
"source_file": "lessons/5-NLP/17-GenerativeNetworks/GenerativeTF.ipynb",
|
|
"language_code": "sw"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 4
|
|
} |