RomanNum3ral
/
AI-For-Beginners
mirror of https://github.com/microsoft/AI-For-Beginners.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Minor content updates

This commit is contained in:
Dmitri Soshnikov 2022-05-25 11:58:12 +03:00
parent cbf95b16ac
commit fe479e5cb9
3 changed files with 72 additions and 59 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

95
lessons/5-NLP/13-TextRep/TextRepresentationPyTorch.ipynb
View File

@ -15,18 +15,9 @@
},
{
"cell_type": "code",
"execution_count": 2,
"execution_count": 1,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"d:\\WORK\\ai-for-beginners\\5-NLP\\13-TextRep\\data\\train.csv: 29.5MB [00:00, 48.1MB/s] \n",
"d:\\WORK\\ai-for-beginners\\5-NLP\\13-TextRep\\data\\test.csv: 1.86MB [00:00, 9.53MB/s] \n"
]
}
],
"outputs": [],
"source": [
"import torch\n",
"import torchtext\n",
@ -41,12 +32,12 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Here, `train_dataset` and `test_dataset` contain iterators that return pairs of label (number of class) and text respectively, for example:"
"Here, `train_dataset` and `test_dataset` contain collections that return pairs of label (number of class) and text respectively, for example:"
]
},
{
"cell_type": "code",
"execution_count": 3,
"execution_count": 2,
"metadata": {},
"outputs": [
{
@ -56,13 +47,13 @@
" \"Wall St. Bears Claw Back Into the Black (Reuters) Reuters - Short-sellers, Wall Street's dwindling\\\\band of ultra-cynics, are seeing green again.\")"
]
},
"execution_count": 3,
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"next(train_dataset)"
"list(train_dataset)[0]"
]
},
{
@ -74,18 +65,18 @@
},
{
"cell_type": "code",
"execution_count": 4,
"execution_count": 5,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"**Sci/Tech** -> Wall St. Bears Claw Back Into the Black (Reuters) Reuters - Short-sellers, Wall Street's dwindling\\band of ultra-cynics, are seeing green again.\n",
"**Sci/Tech** -> Carlyle Looks Toward Commercial Aerospace (Reuters) Reuters - Private investment firm Carlyle Group,\\which has a reputation for making well-timed and occasionally\\controversial plays in the defense industry, has quietly placed\\its bets on another part of the market.\n",
"**Sci/Tech** -> Oil and Economy Cloud Stocks' Outlook (Reuters) Reuters - Soaring crude prices plus worries\\about the economy and the outlook for earnings are expected to\\hang over the stock market next week during the depth of the\\summer doldrums.\n",
"**Sci/Tech** -> Iraq Halts Oil Exports from Main Southern Pipeline (Reuters) Reuters - Authorities have halted oil export\\flows from the main pipeline in southern Iraq after\\intelligence showed a rebel militia could strike\\infrastructure, an oil official said on Saturday.\n",
"**Sci/Tech** -> Oil prices soar to all-time record, posing new menace to US economy (AFP) AFP - Tearaway world oil prices, toppling records and straining wallets, present a new economic menace barely three months before the US presidential elections.\n",
"**Sci/Tech** -> Stocks End Up, But Near Year Lows (Reuters) Reuters - Stocks ended slightly higher on Friday\\but stayed near lows for the year as oil prices surged past #36;46\\a barrel, offsetting a positive outlook from computer maker\\Dell Inc. (DELL.O)\n"
"**Sci/Tech** -> Oil prices soar to all-time record, posing new menace to US economy (AFP) AFP - Tearaway world oil prices, toppling records and straining wallets, present a new economic menace barely three months before the US presidential elections.\n"
]
}
],
@ -103,7 +94,7 @@
},
{
"cell_type": "code",
"execution_count": 5,
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
@ -125,7 +116,7 @@
},
{
"cell_type": "code",
"execution_count": 9,
"execution_count": 4,
"metadata": {},
"outputs": [
{
@ -134,7 +125,7 @@
"['he', 'said', 'hello']"
]
},
"execution_count": 9,
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
@ -146,14 +137,14 @@
},
{
"cell_type": "code",
"execution_count": 10,
"execution_count": 5,
"metadata": {},
"outputs": [],
"source": [
"counter = collections.Counter()\n",
"for (label, line) in train_dataset:\n",
" counter.update(tokenizer(line))\n",
"vocab = torchtext.vocab.Vocab(counter, min_freq=1)"
"vocab = torchtext.vocab.vocab(counter, min_freq=1)"
]
},
{
@ -165,23 +156,23 @@
},
{
"cell_type": "code",
"execution_count": 11,
"execution_count": 19,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Vocab size if 95812\n"
"Vocab size if 95810\n"
]
},
{
"data": {
"text/plain": [
"[283, 2321, 5, 337, 19, 1301, 2357]"
"[599, 3279, 97, 1220, 329, 225, 7368]"
]
},
"execution_count": 11,
"execution_count": 19,
"metadata": {},
"output_type": "execute_result"
}
@ -190,8 +181,10 @@
"vocab_size = len(vocab)\n",
"print(f\"Vocab size if {vocab_size}\")\n",
"\n",
"stoi = vocab.get_stoi() # dict to convert tokens to indices\n",
"\n",
"def encode(x):\n",
" return [vocab.stoi[s] for s in tokenizer(x)]\n",
" return [stoi[s] for s in tokenizer(x)]\n",
"\n",
"encode('I love to play with my words')"
]
@ -215,7 +208,7 @@
},
{
"cell_type": "code",
"execution_count": 12,
"execution_count": 7,
"metadata": {},
"outputs": [
{
@ -224,7 +217,7 @@
"array([[1, 1, 0, 2, 0, 0, 0, 0, 0]], dtype=int64)"
]
},
"execution_count": 12,
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
@ -250,14 +243,14 @@
},
{
"cell_type": "code",
"execution_count": 13,
"execution_count": 20,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"tensor([0., 0., 2., ..., 0., 0., 0.])\n"
"tensor([2., 1., 2., ..., 0., 0., 0.])\n"
]
}
],
@ -292,7 +285,7 @@
},
{
"cell_type": "code",
"execution_count": 14,
"execution_count": 21,
"metadata": {},
"outputs": [],
"source": [
@ -320,7 +313,7 @@
},
{
"cell_type": "code",
"execution_count": 15,
"execution_count": 22,
"metadata": {},
"outputs": [],
"source": [
@ -336,7 +329,7 @@
},
{
"cell_type": "code",
"execution_count": 16,
"execution_count": 24,
"metadata": {},
"outputs": [],
"source": [
@ -364,26 +357,26 @@
},
{
"cell_type": "code",
"execution_count": 17,
"execution_count": 25,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"3200: acc=0.80625\n",
"6400: acc=0.841875\n",
"9600: acc=0.8564583333333333\n",
"12800: acc=0.86640625\n"
"3200: acc=0.8028125\n",
"6400: acc=0.8371875\n",
"9600: acc=0.8534375\n",
"12800: acc=0.85765625\n"
]
},
{
"data": {
"text/plain": [
"(0.025393278614036056, 0.8710021321961621)"
"(0.026090790722161722, 0.8620069296375267)"
]
},
"execution_count": 17,
"execution_count": 25,
"metadata": {},
"output_type": "execute_result"
}
@ -407,7 +400,7 @@
},
{
"cell_type": "code",
"execution_count": 18,
"execution_count": 26,
"metadata": {},
"outputs": [
{
@ -425,7 +418,7 @@
" dtype=int64)"
]
},
"execution_count": 18,
"execution_count": 26,
"metadata": {},
"output_type": "execute_result"
}
@ -453,14 +446,14 @@
},
{
"cell_type": "code",
"execution_count": 19,
"execution_count": 27,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Bigram vocabulary length = 1308844\n"
"Bigram vocabulary length = 1308842\n"
]
}
],
@ -470,7 +463,7 @@
" l = tokenizer(line)\n",
" counter.update(torchtext.data.utils.ngrams_iterator(l,ngrams=2))\n",
" \n",
"bi_vocab = torchtext.vocab.Vocab(counter, min_freq=1)\n",
"bi_vocab = torchtext.vocab.vocab(counter, min_freq=1)\n",
"\n",
"print(\"Bigram vocabulary length = \",len(bi_vocab))"
]
@ -510,7 +503,7 @@
},
{
"cell_type": "code",
"execution_count": 20,
"execution_count": 28,
"metadata": {},
"outputs": [
{
@ -522,7 +515,7 @@
" 0. ]])"
]
},
"execution_count": 20,
"execution_count": 28,
"metadata": {},
"output_type": "execute_result"
}
@ -546,10 +539,10 @@
],
"metadata": {
"interpreter": {
"hash": "0cb620c6d4b9f7a635928804c26cf22403d89d98d79684e4529119355ee6d5a5"
"hash": "16af2a8bbb083ea23e5e41c7f5787656b2ce26968575d8763f2c4b17f9cd711f"
},
"kernelspec": {
"display_name": "py37_pytorch",
"display_name": "Python 3.8.12 ('py38')",
"language": "python",
"name": "python3"
},

2
lessons/5-NLP/14-Embeddings/EmbeddingsPyTorch.ipynb
View File

@ -461,7 +461,7 @@
"\n",
"print('Populating matrix, this will take some time...',end='')\n",
"found, not_found = 0,0\n",
"for i,w in enumerate(vocab.itos):\n",
"for i,w in enumerate(vocab.get_itos()):\n",
" try:\n",
" net.embedding.weight[i].data = torch.tensor(w2v.get_vector(w))\n",
" found+=1\n",

34
lessons/5-NLP/17-GenerativeNetworks/GenerativePyTorch.ipynb
View File

@ -242,7 +242,27 @@
"output_type": "stream",
"text": [
"Current loss = 4.398899078369141\n",
"today sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr s\n"
"today sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr sr s\n",
"Current loss = 2.161320447921753\n",
"today and to the tor to to the tor to to the tor to to the tor to to the tor to to the tor to to the tor t\n",
"Current loss = 1.6722588539123535\n",
"today and the court to the could to the could to the could to the could to the could to the could to the c\n",
"Current loss = 2.423795223236084\n",
"today and a second to the conternation of the conternation of the conternation of the conternation of the \n",
"Current loss = 1.702607274055481\n",
"today and the company to the company to the company to the company to the company to the company to the co\n",
"Current loss = 1.692358136177063\n",
"today and the company to the company to the company to the company to the company to the company to the co\n",
"Current loss = 1.9722288846969604\n",
"today and the control the control the control the control the control the control the control the control \n",
"Current loss = 1.8705692291259766\n",
"today and the second to the second to the second to the second to the second to the second to the second t\n",
"Current loss = 1.7626899480819702\n",
"today and a security and a security and a security and a security and a security and a security and a secu\n",
"Current loss = 1.5574463605880737\n",
"today and the company and the company and the company and the company and the company and the company and \n",
"Current loss = 1.5620026588439941\n",
"today and the be that the be the be that the be the be that the be the be that the be the be that the be t\n"
]
}
],
@ -300,7 +320,7 @@
},
{
"cell_type": "code",
"execution_count": 15,
"execution_count": 10,
"metadata": {
"scrolled": true
},
@ -310,19 +330,19 @@
"output_type": "stream",
"text": [
"--- Temperature = 0.3\n",
"Today and to has a software to in the first the power the gold medal was of the first and succer to the company will a report the first the and the gain the company in the and a new a report a pack of the four the first the company of the such with the half to a security to the and a success the first she\n",
"Today and a company and complete an all the land the restrational the as a security and has provers the pay to and a report and the computer in the stand has filities and working the law the stations for a company and with the company and the final the first company and refight of the state and and workin\n",
"\n",
"--- Temperature = 0.8\n",
"Today drud out of the three-rent possiem that sales purssion has finminiaty women's from NAC Inc. (AP) -- Shimbon has weel with a may stelight first three flaw gold from their a scent, big study with a nighting sovicturner has slarh football at a hour of Angelage discression, into cubs, US year player sor\n",
"Today he oniis its first to Aus bomblaties the marmation a to manan boogot that pirate assaid a relaid their that goverfin the the Cappets Ecrotional Assonia Cition targets it annight the w scyments Blamity #39;s TVeer Diercheg Reserals fran envyuil that of ster said access what succers of Dour-provelith\n",
"\n",
"--- Temperature = 1.0\n",
"Today by compoy, said to hup the couns ay rrope iist\\fill sinie-5-1- than he of a fightier Corp. the Vew, Mkli Unite Hold Austria on Tuesday resfare rextarted in the new has buy thisnillials thrust first capuration of the it larget expected the ir edulagy Airin Penny after Emonet Cuc Washieve an are Gurry\n",
"Today holy they a 11 will meda a toket subsuaties, engins for Chanos, they's has stainger past to opening orital his thempting new Nattona was al innerforder advan-than #36;s night year his religuled talitatian what the but with Wednesday to Justment will wemen of Mark CCC Camp as Timed Nae wome a leaders\n",
"\n",
"--- Temperature = 1.3\n",
"Today cluscy,, wangled and-ox they, stee of as;\\seculity dillancrile inmution svanse gall ATHEYS today a first oresift 6-Jalf mangback explymate that wrook\" haffic illowbre overwage in Tecrian Hunrieleers to attowny service Adching, blanks governine? Aug. : : NE: Sir NFP (P2AAU) Bow SWDE: The ex2\"cut Pmoc\n",
"Today gpone 2.5 fech atcusion poor cocles toparsdorM.cht Line Pamage put 43 his calt lowed to the book, that has authh-the silia rruch ailing to'ory andhes beutirsimi- Aefffive heading offil an auf eacklets is charged evis, Gunymy oy) Mony has it after-sloythyor loveId out filme, the Natabl -Najuntaxiggs \n",
"\n",
"--- Temperature = 1.8\n",
"Today sas gom, twing hWe a Dajfcou hamb--5 to bemolecresem ig irkembets plentll repws, scatchey: Actuss.io Theffouge, cirw biggemed Goiga propperinut you racive #5-Aeia:riato..Lf. N7TNap:,ser,wploy a Fraull tbashonerdlantuanseve /bBT -$06 Wemob-e.P EvVlaicy), ZOf0 cUSeballd sturk out houselty, TARZM) AbAe\n",
"Today plary, P.slan chly\\401 mardregationly #39;t 8.1Mide) closes ,filtcon alfly playin roven!\\grea.-QFBEP: Iss onfarchQ/itilia CCf Zivesigntwasta orce.-Peul-aw.uicrin of fuglinfsut aftaningwo, MIEX awayew Aice Woiduar Corvagiugge oppo esig ThusBratourid canthly-RyI.co lagitems\\eexciaishes.conBabntusmor I\n",
"\n"
]
}